sentry-javascript: Replays are randomly missing data (headers, API calls, console etc)

Is there an existing issue for this?

How do you use Sentry?

Self-hosted/on-premise (23.12.1)

Which SDK are you using?

@sentry/react

SDK Version

7.93.0

Framework Version

18.2.0

Link to Sentry event

No response

SDK Setup

const ALLOWED_HEADERS = ['content-type', 'x-custom-header',  'other-custom-header', '*'];

const isProductionEnvironment = (environment) => environment === 'prod';

const getDSN = (environment) =>
  isProductionEnvironment(environment)
    ? 'prod-dsn'
    : 'general-dsn';

const initSentry = ({ environment, brand, isTestBot }) => {
  init({
    environment,
    beforeSend: (event) => {
      const tool = getRoot();
      event.extra = { ...(event.extra || {}), tool, isTestBot };
      event.tags = { ...(event.tags || {}), module: 'app-shell', tool, isTestBot };
      return event;
    },
    dsn: getDSN(environment),
    normalizeDepth: 10,
    integrations: [
      new BrowserTracing({
        routingInstrumentation: reactRouterV6Instrumentation(useEffect, useLocation, useNavigationType, createRoutesFromChildren, matchRoutes),
        idleTimeout: 10000,
      }),
      new Replay({
        maskAllText: isProductionEnvironment(environment),
        useCompression: true,
        blockAllMedia: false,
        networkDetailAllowUrls: [/.*/g],
        networkRequestHeaders: clone(ALLOWED_HEADERS),
        networkResponseHeaders: clone(ALLOWED_HEADERS),
      }),
      isProductionEnvironment(environment) ? null : new BrowserProfilingIntegration(),
    ].filter(Boolean),
    tracesSampleRate: 1.0,
    tracePropagationTargets: ['localhost'],
    replaysSessionSampleRate: 0.0,
    replaysOnErrorSampleRate: 1.0,
    profilesSampleRate: isProductionEnvironment(environment) ? 0.0 : 1.0,
  });

  setTag('brand', brand);
};

Steps to Reproduce

  1. Configure the @sentry/react SDK to capture Replays as per the docs (see config)
  2. Deploy to live enviroment
  3. Capture actual replays
  4. Inspect them in Sentry

Expected Result

UPDATE: Initially this ticket was only about missing request/response headers, but upon further investigation we have found that other data goes randomly missing as well.

The Replay should consistently show expected header captures for all logged network calls.

Note that I have explicitly set networkDetailAllowUrls to /.*/g to capture any URL, just to make sure it was not some weirdness with certain subpaths of our API. Prior to this I just whitelisted that API domain itself - behaviour remains the same.

Note also that the response ALLOWED_HEADERS are ALWAYS sent by our API. There is no call where they will NOT be in the response, so the below behaviour cannot be explained by the headers simply missing.

Actual Result

Header captures - and just overall network behaviour overall - is extremely inconsistent.

For the same Replay, on calls to the same API (but different endpoints of said API), we see that:

  1. Sometimes, everything works as expected and headers are properly captured and displayed in Sentry:
Screenshot 2024-01-11 at 19 48 28
  1. Other times, it only captures only the default headers (as stated above - custom headers are ALWAYS sent, and calling the API manually or from the website confirms this to be the case):
Screenshot 2024-01-11 at 19 53 04
  1. Other times, it does not work at all - the response looks as if we had never setup capture in the first place:
Screenshot 2024-01-11 at 19 50 46

Again, keep in mind that ALL these happen in the SAME replay, for the SAME API domain being called. I don’t know what other differences we could rule out.

About this issue

  • Original URL
  • State: closed
  • Created 6 months ago
  • Comments: 19 (7 by maintainers)

Most upvoted comments

It may actually be related to this:

https://github.com/getsentry/self-hosted/issues/1929

We have quite a lot of traffic to our sentry instance and I noticed incidentally that we were getting the above error sometimes when debugging another issue. Raising it to 1 million requests and will see if it makes a difference.

So for the request headers, it can only capture headers that are user-provided in the code - sadly, we have no access to the headers the browser adds.

fetch(url, { 
  headers: {
    origin: 'xxx' // NOW we could access it, but you probably don't set this yourself
  }
});

So I think this is why origin and referer cannot be captured! Sadly, I don’t know of a way to access the request headers that the browser adds 🤔 I’ll make sure we update the docs to explain this better!

For the response headers, I would suspect that the reason is maybe CORS? If you say that some headers are captured (e.g. cache-control) but others aren’t, I think possibly on these headers are allowed/whitelisted by CORS, and thus exposed? Could you double check this?

I am having exactly the same issue with sentry/nextjs package, tried 7.93 (latest), 7.85, 7.50, it works inconsistently with any of them. I also have self-hosted Sentry, the version is 23.12.0.dev0.

This may have been caused by a mismatch in SDK versions. I will open again if the issue reappears.