nitro: Undefined cache headers

This causes some fairly painful cache issues, where browsers will keep using stale document HTML, and break across updates (because ETag and Last-Modified are always undefined so both users and intermediate caches just get 304s all the time until uncoordonated cache expiries… quite painful).

Here is a workaround that will do the job while waiting for a fix (with some drawbacks – see below):

server/plugins/cache-headers.ts for nuxt projects

// replace with a value that is specific to your application's
// specific build; in our case `git-$YYYMMddHHmm-$commitSha`
const buildVersion = "changeme"; // useRuntimeConfig().public.version for example
const etag = `W/"${buildVersion}"`;

/**
 * Ensure Nitro always thinks it's a new request; and give up on 304s from it
 * which is fine since Varnish can do those for us just fine
 * (Workaround for https://github.com/unjs/nitro/issues/1850)
 */
export default defineNitroPlugin((nitroApp) => {
  // avoid nitro considering cache headers coming to it
  nitroApp.hooks.hook("request", (event) => {
    delete event.node.req.headers["if-modified-since"];
    delete event.node.req.headers["if-match"];
    delete event.node.req.headers["if-none-match"];
  });

  // remove Last-Modified's undefined + add build-specific ETag
  nitroApp.hooks.hook("beforeResponse", (event, _) => {
    event.node.res.removeHeader("Last-Modified");
    event.node.res.setHeader("ETag", etag);
  });
});

This setup will make nitro always return 200s instead of 304s, essentially disabling a lot of caching measures. So if you don’t have something above Nitro doing caching + ETag handling (like CF, Varnish, …) then it will cause a lot more traffic for you and your users.

But if like us you do have some caching layer above it, it’s relatively harmless afaict.

@pi0 I had not deleted the node_modules folder (I was uninstalling and reinstalling Nuxt and Nitropack modules only). Now simply deleted entire folder of node_modules, and reinstalled everything and now I’m getting no undefined error in the main project’s APIs’ headers. 😁🙏🏻 so now no need for that plugin hack as well.

This issue is fixed in the nightly release channel ( via https://github.com/unjs/nitro/pull/1855) and a hotfix release soon.

I suggest switching to nightly in the meantime (https://nitro.unjs.io/guide/getting-started#nightly-release-channel)

In case anyone reads this, I fixed it by first removing the node_modules folder and package-lock.json, and then running npm install.

@pi0 I wanted to send you a demo project, and even created a new project, but fortunately or unfortunately 😁 in the new project, there’s no such error.

😳 but in my existing main project, even though I deleted the nuxt folder, and even reinstalled the Nuxt and Nitropack, I still get the undefined error in both dev and production. I thought it must be some misconfiguration in my project, but then even others are getting this error and also until last week, I didn’t have any cache issues. Also, one thing I am using is dayjs in every api route, and also I have named every cached route manually, i.e., using the name key, but I don’t know how it might be causing the issue. Because even in the brand new demo app, I had named the test api cached route, and there was no error.

While for the time being, I have used the plugin hack mentioned at the top of this thread (I even removed etag res header in the hook), and now the app is functioning, and even getting Cache hit in subsequent requests due to Firebase. So this issue is not breaking my project at the moment.

I’ll try to come up with a reproduction with a new project with trial and error though. Many appreciation and best wishes to you for your prompt responses and efforts 🙏🏻😊

for what it’s worth, I can still reproduce the issue on our end as well, with Nuxt 3.8.0 and Nitro 2.7.1

$ curl -I -XGET localhost:3000
HTTP/1.1 200 OK
Via: 1.1 mangadex-frontend
last-modified: undefined
etag: undefined
cache-control: s-maxage=30, stale-while-revalidate
content-type: text/html;charset=utf-8
x-powered-by: Nuxt
Date: Fri, 27 Oct 2023 06:16:13 GMT
Connection: keep-alive
Keep-Alive: timeout=5
Content-Length: 3523

That is on a fresh cache (deleted .nuxt then build+preview), with just about everything default. The relevant route rule here is "/": { swr: 30 },.

Can’t give a stackblitz example right now, but since we don’t configure Nitro explicitly in any particular way (we have 2 plugins, 1 to add logging, and 1 which appends a bit of HTML), it should be very reproducible.

import { IncomingMessage, ServerResponse } from "http";

export default defineEventHandler((event) => {
  const req: IncomingMessage = event.node.req;
  const reqHdr = req.headers;
  const reqRuid = reqHdr["x-request-id"] ?? "-";
  const reqSrc = reqHdr["x-forwarded-for"] ?? req.socket.remoteAddress;

  const res: ServerResponse = event.node.res;
  res.setHeader("Via", `${req.httpVersion} mangadex-frontend`);

  res.on("finish", () => {
    console.log(
      `${reqRuid} - ${reqSrc} - "${req.method} ${req.url}" ${res.statusCode}`
    );
  });
});

export default defineNitroPlugin((nitroApp) => {
  nitroApp.hooks.hook("render:html", (html, _: any) => {
    html.head.push(`<!--esi
<esi:comment text="ESI rendered" />
<esi:include src="ESI_OPENGRAPH" onerror="continue" />
-->`);
    html.body.push(`
<noscript>
    <div style="display: flex; justify-content: center; align-items: center; flex-direction: column; height: 100%; gap: 2rem;">
      <img
        src="/img/cat-books.svg"
        alt="Cat napping on a pile of books"
        style="max-width: 400px"
      />
      <p style="font-weight: 500; text-align: center; font-size: 1.5rem; line-height: 2rem;">Please enable JavaScript to view MangaDex.</p>
    </div>
</noscript>
    `);
  });
});

Also something of note, HTTP request headers are fully case-insensitive https://www.rfc-editor.org/rfc/rfc9110.html#section-5.1-3, so the checks performed are technically incorrect (ie any webserver should either fully support case-insensitive lookups in their headers structure, or normalize to a specific case and always use that one later on).