next.js: Custom server renderToHTML does not work with next 13.4

Link to the code that reproduces this issue or a replay of the bug

https://github.com/thilohaas/nextjs-custom-server/tree/main https://codesandbox.io/p/github/thilohaas/nextjs-custom-server/main

To Reproduce

  1. Start a new nextjs project with npx create-next-app@latest
  2. Add a server.js
const { createServer } = require('http');
const { parse } = require('url');
const next = require('next');

const dev = process.env.NODE_ENV !== "production";
const hostname = 'localhost';
const port = process.env.PORT || 3001;
// when using middleware `hostname` and `port` must be provided below
const app = next({ hostname, port, dev });
const handle = app.getRequestHandler();

app.prepare().then(() => {
  createServer(async (req, res) => {
    try {
      const parsedUrl = parse(req.url, true);
      const { pathname, query } = parsedUrl;

      if (pathname.startsWith('/_next') || pathname.startsWith('/__next')) {
        await handle(req, res, parsedUrl);
      } else {
        const html = await app.renderToHTML(req, res, pathname, query);
        res.end(html);
      }
    } catch (err) {
      console.error('Error occurred handling', req.url, err);
      res.statusCode = 500;
      res.end('internal server error');
    }
  }).listen(port, (err) => {
    if (err) throw err;
    console.log(`> Ready on http://${hostname}:${port}`);
  });
});
  1. Change the package.json dev script to: "dev": "node server.js",
  2. run npm run dev
  3. open http://localhost:3001

Current vs. Expected behavior

I expect it to display the welcome page but instead i get an error from the next server:

Error: Invariant ensurePage called outside render worker

Verify canary release

  • I verified that the issue exists in the latest Next.js canary release

Provide environment information

Operating System:
      Platform: linux
      Arch: x64
      Version: #1 SMP PREEMPT_DYNAMIC Sun Aug  6 20:05:33 UTC 2023
    Binaries:
      Node: 16.17.0
      npm: 8.15.0
      Yarn: 1.22.19
      pnpm: 7.1.0
    Relevant Packages:
      next: 13.4.20-canary.16
      eslint-config-next: N/A
      react: 18.2.0
      react-dom: 18.2.0
      typescript: 5.1.3
    Next.js Config:
      output: N/A

Which area(s) are affected? (Select all that apply)

Not sure

Additional context

No response

About this issue

  • Original URL
  • State: closed
  • Created 3 years ago
  • Reactions: 31
  • Comments: 32 (4 by maintainers)

Commits related to this issue

Most upvoted comments

We are having the same issue with a custom Express server

do not use custom next serve. manage routers via next.config.js

For me this problem is also related to this one: https://github.com/vercel/next.js/issues/53466 Obstacles have appeared in large projects with their own infrastructure (Kube). These problems are associated with the use of Custom-Server, which is a great need for additional collection of metrics, custom logging, control of response headers and status, etc. I also need the ability to run in one thread so that the AsyncLocalStorage context is saved and other global mechanics work.

On versions higher than 13.4.12 (I confirm, the problem appears with 13.4.13-canary.0) it became impossible to create a custom-server. Now when calling methods, and even render, I get errors when calling server methods: Invariant ... called outside render worker.

The Next server is initializing, but it seems to be isolated from my Fastify server, or the new code is broken. @ijjk It seems to me that you added these changes that affected an important part of the community.

Same error here (Next 14. while trying to work with Hydrate and Stencil (Web Components). TypeError: Cannot read properties of undefined (reading 'ensurePage')

Same problem here with custom server + renderToHTML + hydrate components 😞 any temporary solution?

image

I solved my problem by managing the routes in next.config.js

Hi, the renderToHtml API is not a public API/not documented nor used in any examples. So the response here https://github.com/vercel/next.js/issues/54977#issuecomment-2354102135 saying to use the proper public API render is correct.

If you want to work with the html generated, this is not an option

This is not supported with streaming as otherwise you are buffering the response to capture the HTML if you wanna do transforms on the stream which is highly risky that would be a use case for middleware or similar.

I’m gonna close this as it doesn’t seem to be a valid/supported use case.

@salman-mazhar We are waiting for @ijjk.

Any update or workaround on this ? We are planning to upgrade to next 14.2.3 from 12 and see this issue.

@duzgunemreozkan yeah, you mentioned.

Maybe you might elaborate on how you are able to replicate the functionality of Express in the next.config.js, that would be helpful

A workaround is to directly access the internal Dev/Node server inside the app object.

const appHtml = await (app as any).renderServer.server.renderToHTML(req, res, pathname!, query);

This works with 14.2.4.

Problem is the getServer() method. Which is called on the app ( CustomServer ) object, but not on the already created DevNextServer. Therefor another new server is created with an undefined bundlerService. app (which is a CustomServer), has no serverPromise.

app.renderServer or app.renderServer.server are the correct objects. image

We ran into this problem too, it’s with renderToHtml only in dev mode. We got around it by calling app.render in dev mode instead.

I have a same problem with next 14.1.0 with custom server. Any update for this issue ?

Same issue with Next 14.0.3 with custom server. We are having a hapi server

Fixed by using render instead of renderToHtml

render directly sends the payload to the response. If you want to work with the html generated, this is not an option.

Fixed by using render instead of renderToHtml

We are facing the same issue when using renderToHTML with a custom http server, anyone can help me ?

@jiebaibai please use English to report your issue, so others can understand, thanks

@ijjk 13.5.5 also has this problem, after customizing server.js, app.renderToHTML exception,catch error is: TypeError: Cannot read properties of undefined (reading ‘ensurePage’) image

We are facing the same issue when using renderToHTML with a custom http server, is there a temporary work around?