sentry-javascript: Dev console errors are thrown to sentry

The filter has been running for about 24h, and the issue count has dropped from 500+ to <20. The reported issues, for the most part, seem reasonable and related to my application. So for what its worth, it seems to work well.

Here is my modified version currently running, in case someone wants to use/modify it:

function filterConsoleErrors(event) {
  const originalException = event.exception?.values?.[0];

  // Console errors appear to always bubble up to `window.onerror` and to be unhandled.
  // So if, we don't have the original exception or the mechanism looks different,
  // we can early return the event.
  // (Note, this might change depending on the used framework, so feel free to remove this check.)
  if (
    !originalException ||
    !originalException.mechanism ||
    originalException.mechanism.type !== 'onerror' ||
    originalException.mechanism.handled
  ) {
    return event;
  }

  const stackFrames = originalException.stacktrace?.frames;
  const errorType = originalException.type?.toLowerCase();

  // If we don't have any information on error type or stacktrace, we have no information about the error
  // this is unlikely to happen but it doesn't appear to happen in console errors.
  // Hence, we can early return here as well.
  if (!stackFrames || !errorType) {
    return event;
  }

  // For simple console errors (e.g. users just typing a statement they want evaluated)
  // the stacktrace will only have one frame.
  // This condition will not catch errors that would be thrown if users type in multi-line
  // statements. For example, if they define a multi-line function.
  // You can try experimenting with this number but there's little guarantee that the other
  // conditions will work. Nevertheless, the checks below also work with multi-frame stacktraces.
  const hasShortStackTrace = stackFrames.length <= 2;

  if (hasShortStackTrace && isSuspiciousError(errorType) && hasSuspiciousFrames(stackFrames)) {
    console.warn('Dropping error due to suspicious stack frames.');

    return null;
  }

  return event;
}

function isSuspiciousError(errorType) {
  return ['syntaxerror', 'referenceerror', 'typeerror'].includes(errorType);
}

function hasSuspiciousFrames(stackFrames) {
  const allSuspicious = stackFrames.every(isSuspiciousFrame);

  // Certain type errors will include the thrown error message as the second stack frame,
  // but the first will still follow the suspicious pattern.

  const firstSuspicious = stackFrames.length === 2 && isSuspiciousFrame(stackFrames[0]);

  return allSuspicious || firstSuspicious;
}

function isSuspiciousFrame(frame) {
  const url = window.location.href;

  return frame.function === '?' && (frame.filename === '<anonymous>' || frame.filename === url);
}

export default filterConsoleErrors;

I have 99% errors in my sentry list from users trying things out in their console

This is a pretty good reason to reopen this issue 😉 Thanks for letting us know about it. Okay, so I think we need to think about strategies to filter out console errors thrown to window.onerror. Will reopen and backlog this again and bring your case up with our team. Let’s see what we can do here!

To everyone else reading this message: If you would like to see this implemented, please react to this answer. Also, PRs are more than welcome!

Hey @markivancho (and everyone else who’s getting these errors),

we’re still discussing how to best filter such browser-caused errors in the future automatically. In the meantime, you have a couple of options on how to filter these events out manually:

Using beforeSend

You can add a filter criterion for beforeSend in Sentry.init. Something along these lines:

1. Filter out all events from the buggy browser version:

import { Event as SentryEvent } from "@sentry/<yourSDK>";

beforeSend: (event: SentryEvent) => {
  const isBuggyBrowser = navigator.userAgent?.includes("Chrome/102");

  if (isBuggyBrowser) {
    return null;
  }
  return event;
}

1. Be more specific and filter for exception type and values:

import { Event as SentryEvent } from "@sentry/<yourSDK>";

beforeSend: (event: SentryEvent) => {
  const isBuggyBrowser = navigator.userAgent?.includes("Chrome/102");
  const error = event.exception?.values?.at(0);
  const isKnownBrowserBug =
    isBuggyBrowser &&
    (error?.type === "EvalError" &&
      error?.value?.includes("Possible side-effect in debug-evaluate")) ||
    (error?.type === "SyntaxError" &&
      (error?.value?.includes("Unexpected end of input") ||
        error?.value?.includes("Invalid or unexpected token") ||
        error.value?.includes("missing ) after argument list")));
  
  if (isKnownBrowserBug) {
    return null;
  }
 
  return event;
}

Using ignoreErrors

Alternatively, if you do not care about browser versions, you can go a simpler route and use the ignoreErrors property in Sentry.init. Note though, that this does not take the user agent into account at all and will thus ignore errors from all browsers:

ignoreErrors: [
  "Possible side-effect in debug-evaluate",
  "Unexpected end of input",
  "Invalid or unexpected token",
  "missing ) after argument list",
]

Using Inbound Filters in Sentry UI

We recognize that these approaches require a redeployment of your app with the changed SDK config. If this is not an option, you can do something similar to ignoreErrors above in the Sentry UI: In your project’s settings, go to “Inbound Filters” and at the bottom you can configure a filter for error messages.

Again, this option filters out errors of all browsers, so similariy to ignoreErrors it cannot be applied to a specific browser (version).

It looks like this is fixed in Chrome 106. I updated and now can’t recreate sending errors to Sentry from the dev tools console.

Awesome, closing this issue as a result, but please reach out if there is anything else.

Great work! I’ll publish a modified version of your code and see how it works out for me.

Thanks for the examples. I think most of them should be caught by the solution above. The EvalError: Possible side-effect in debug-evaluate could still be from the eager input bug that was fixed in Chromium 104 (workaround for this one is above). That leaves us with the errors having a higher line number than 1. You could experiment with loosening the filter criterion on the line number in the solution above but I can’t guarantee that this won’t cause false positives.

I would support raising an issue with the Chromium team.

Just opened up an issue: https://bugs.chromium.org/p/chromium/issues/detail?id=1350066

@Lms24 Thank you for the reply, you arrived at the same conclusion (and similar solution) as I.

I will integrate this into regex101 and deploy to see if it helps reduce the noiise.

Here’s a couple of typical errors I’m getting (raw stacktrace attached):

SyntaxError: Unexpected end of input
  at ? (https://regex101.com/:1:63)

EvalError: Possible side-effect in debug-evaluate
  at ? (https://regex101.com/:0:0)

SyntaxError: missing ) after argument list
  at ? (https://regex101.com/:1:41)

SyntaxError: Invalid regular expression: missing /
  at ? (https://regex101.com/:1:13)

SyntaxError: Unexpected identifier
  at ? (https://regex101.com/:1:3)

SyntaxError: Unexpected string
  at ? (https://regex101.com/:1:3)

SyntaxError: Unexpected token ':'
  at ? (https://regex101.com/:13:10)

SyntaxError: Unexpected token '/'
  at ? (https://regex101.com/:1:12)

SyntaxError: Unexpected token '}'
  at ? (https://regex101.com/:3:42)

SyntaxError: Unexpected token '.'
  at ? (https://regex101.com/:1:15)

ReferenceError: login is not defined
  at ? (<anonymous>:2:27)

I would support raising an issue with the Chromium team.

Hi @firasdib @Irevall and everyone else,

we investigated console errors a little more and we mostly came to the same conclusions. Unfortunately, there is no bullet-proof way of detecting if an error came from the dev console.

While toying around with the console, we detected some patterns that basic console errors have in common. For example, they mostly have a stacktrace with only one frame that is anonymous or whose file name is equal to the URL of the current page. It does however depend a lot on the type of error (syntax vs. type and reference errors). So with this information, we went ahead and hacked together a filter that is likely to filter out simple console errors:

Sentry.init({
  // ...
  beforeSend: (event: SentryEvent) => filterConsoleErrors(event),
});

function filterConsoleErrors(event: SentryEvent): SentryEvent | null {
  const originalException = event.exception?.values?.[0];

  // Console errors appear to always bubble up to `window.onerror` and to be unhandled.
  // So if, we don't have the original exception or the mechanism looks different,
  // we can early return the event.
  // (Note, this might change depending on the used framework, so feel free to remove this check.)
  if (
    !originalException ||
    !originalException.mechanism ||
    originalException.mechanism.type !== "onerror" ||
    originalException.mechanism.handled
  ) {
    return event;
  }

  const stackframes = originalException.stacktrace?.frames;
  const errortype = originalException.type?.toLowerCase();

  // If we don't have any information on error type or stacktrace, we have no information about the error
  // this is unlikely to happen but it doesn't appear to happen in console errors.
  // Hence, we can early return here as well.
  if (!stackframes || !errortype) {
    return event;
  }

  // For simple console errors (e.g. users just typing a statement they want evaluated)
  // the stacktrace will only have one frame.
  // This condition will not catch errors that would be thrown if users type in multi-line
  // statemets. For example, if they define a multi-line function.
  // You can try experimenting with this number but there's little guarantee that the other
  // conditions will work. Nevertheless, the checks below also work with multi-frame stacktraces.
  const hasShortStackTrace = stackframes.length <= 1;

  if (!hasShortStackTrace) { 
    return event;
  }

  switch (errortype) {
    case "typeerror":
    case "referenceerror":
      const hasOnlySuspiciousTypeRefFrames = stackframes.every(
        (frame) =>
          frame.function === "?" &&
          frame.lineno === 1 &&
          frame.filename === "<anonymous>"
      );
      if (hasOnlySuspiciousTypeRefFrames) {
        console.log("I found a console type or ref error. Dropping it!");
        return null;
      }
      break;

    case "syntaxerror":
      // For some reason, syntax errors have the current URL as a filename
      const url = window.location.href;
      const hasOnlySuspiciousSyntaxFrames = stackframes.every(
        (frame) =>
          frame.function === "?" && frame.lineno === 1 && frame.filename === url
      );
      if (hasOnlySuspiciousSyntaxFrames) {
        console.log("I found a console syntax error. Dropping it!");
        return null;
      }
      break;
  }

  return event;
}

Please note that this is not a perfect solution. There might be some weird edge cases in which we’d filter out actual non-console errors but given the conditions it’s rather unlikely I’d say. Also, these filters don’t work well for multi-line console statements (where the error might not be in the first line). However, our best guess is that users in the console usually just try out simple stuff but this is only an assumption ofc.

If you want, feel free to give this a try. If it turns out to work really well and you get rid of the noise, we might create an intergation from this code. However, let’s first see if it’s worth doing so. In case it doesn’t help with the noise, would you mind sharing a few errors so that we can get a sense of what else to filter on?

Regardless, the best way forward would be to get some sort of flag or indication from the browser that the error came from the console. We’ll therefore open a Chromium issue to ask for this because it is a real problem for our users. Let’s see if the team gets back to us.

Hope this helps a little!

Chatted with the team about this and we agreed it’s worth doing a little more research here (to see if we can hack a way to tell which errors are which), and if not, we’ll file an issue with the Chrome dev team and see what they say. We’ve pulled it out of the backlog and put it on our active to-do list.

If this can’t be resolved, I’ll just have to skip Sentry altogether, which would be a shame, since its been immensely useful.

As has regex101 for any number of us! We’ll see what we can do here.

Hi @markivancho, we found the reason why these errors are suddenly thrown. It’s Chromium related starting with 102 (as you reported) and was possibly introduced here. It was fixed a few days later but the fix isn’t yet in a stable chromium version which is why we’re gonna get quite a lot of events like these (there’s a couple more events we found related to Chromium eagerly parsing console input). If you install the latest chromium canary version and run your reproduction example, you shouldn’t get those errors anymore.

We’re currently investigating if there’s a way of automatically filtering out those errors.

Hi @markivancho and thanks for reporting! I was able to reproduce this in the SDK version 7.0.0 with Chrome 102, too. I’ll bring it up with the team and we’ll look into it.