workers-sdk: 🐛 BUG: Your worker called response.clone(), but did not read the body of both clones

this is driving me nuts, trying to debug code in dev on wrangler is literally impossible with this junk filling up your console. Come on Cloudflare, this needs addressing ASAP … it’s been like this for months.

Hi folks, Carmen from Pages here. So very sorry about the delay on our side 🙏 . I totes understand your frustration. I’ll be taking this over and will look into what’s going on. I’ll update my findings here and let you know when I have a fix.

Pls hold on just a little while longer ❤️

Getting this nuisance message with this minimal example:

export const onRequestPost: PagesFunction = async (context) => { };

Every POST results in the message appearing twice.

For what it’s worth, adding some code that uses context.request makes the warning only appear a single time.

you are correct @zwily, request.body.cancel() does fix the logs issue! I’d need to test a little bit more to make sure there are no unintended side-effects, but it’d be awesome if this is all it took 😃

Here’s a simple fix: https://gist.github.com/russelldavis/6b1a1bf15898655ec18037bd737d075d

If you have middleware that reads the body of the request, you’ll need to update it to call request.clone().

It’d be great if cloudflare would fix this (it’s a breaking change, but that’s what major versions are for), but in the meanwhile, you can fix this in your own project by applying the patch above using patch-package or pnpm patch.

Looks like this error reappeared in version 3.22.3 I got the stream error message multiple times on a code than never calls response/request.clone()

this issue seems to have been fixed in wrangler version 3.15.0. Is anyone still experiencing issues?

Same. I can confirm that I’m not cloning anywhere, seems to reproduce with just adding a pages function. Seems like this might be a bug in the pages functions logic?

Edit: is it maybe the pages asset server? https://github.com/cloudflare/workers-sdk/blob/main/packages/pages-shared/asset-server/handler.ts#L340

I do not have any clone() neither on Request nor on Response objects anywhere in my code. This must be something else

Are you using any frameworks? I’m using Remix, and see the same thing on any POST requests with bodies, and having a hard time tracking down where the clone is happening…

Also, it looks like this will happen when cloning any ReadableStream (without reading both sides), I believe, based on where it originates in workerd: https://github.com/cloudflare/workerd/blob/ed4c67df1b231270410a72cb0161c203e55b0bb6/src/workerd/api/streams/internal.c%2B%2B#L291-L298

@CarmenPopoviciu My guess is after cloning the request, you want to call request.body.cancel() on the original request to signal that you’re not interested in that readable stream anymore. When you clone the request, it clones the stream, which is what is triggering the error. (I’d test this myself but I’m not sure how to wire everything up in development…)

Also, GETs don’t typically have bodies which would explain why there is not a body stream being cloned.

Hi there 👋 . A few updates from me, to keep everyone in the loop:

• I am definitely able to reproduce the issue with a basic Pages project ( see https://github.com/CarmenPopoviciu/pages-fns). Based on my tests I can confirm that the reported behaviour happens for POST/PUT/PATCH requests, both with and without existing _middleware. I could not reproduce the same for GET/DELETE requests, which is consistent with the reports as well
• the .clone() code that seems to be responsible for triggering the unwanted logs is https://github.com/cloudflare/workers-sdk/blob/main/packages/wrangler/templates/pages-template-worker.ts#L140.
• the log message itself however, comes from workerd: https://github.com/cloudflare/workerd/blob/e135660953231655dc068a9dce3fc1eca8f75da9/src/workerd/api/streams/internal.c%2B%2B#L393

There are reasons why we want to perform that request cloning on the Pages side, mainly so that each Pages Function handler can get a clean copy of the request. Removing that code would be a breaking change, so it requires a bit more thought on our side.

Alternatively, we could try to “turn off” these annoying warnings, but there is no way to my knowledge we could do that selectively. That means that were we to turn them off, we would turn them off for everything, including for .clone() instances that originate in user code.

I don’t have an answer yet as to why we’re seeing these logs for POST & co but not for GET requests. As far as my debugging goes, all these requests follow the same code path, so I am not yet sure where the difference is. Perhaps something further down the stack?..not sure, but I’ll keep investigating.

Not a fix just yet as I was hoping, but hopefully this provides a little bit more context for everyone. I will follow-up with updates as soon as I have them

❤️

I can confirm it’s still happening with Wrangler v3.23.0 and the “Hello world” example from the Pages Functions docs.

I can confirm the downgrading wrangler to 3.22.1 gets rid of the error. The change in https://github.com/cloudflare/workers-sdk/pull/4185 does silence the error as well, but my app does not work anymore. I really hope this error can be fixed in wrangler-side, it’s making local dev with CF Pages very cumbersome.

@aroman thanks for the ping 😃 i’ve added the regression label and re-opened the issue, and a fix is being worked on in https://github.com/cloudflare/workers-sdk/pull/4861 👍

@lrapoport-cf looks like this is a regression

per your comment here, you mentioned we should feel free to add the ‘regression’ label when appropriate ourselves, however, non-CF employees don’t seem to have those permissions (probably a wise decision). so, tagging you here for visibility/so the tag can get added

Can this issue still be re-opened or do we need to create a new issue? @CarmenPopoviciu

If I roll back to 3.22.1 I don’t see this error, but anything >=3.22.2 I’m getting it on every request to any Pages Function.

Utilization of function findings causes this to show up.

So, just using a provided feature can trigger this warning… which makes it no so much a warning. This warning is clearly coming from deeper in the stack and it should NOT be showing up.

What do we have to do to get rid of this? It hides everything every user cares to see.

Adding my voice to this one. I understand its some internal logging, but it throws folks in the team in for a loop since the tool will spam this warning virtually every request even with sample code that doesn’t clone anything (probably because its happening internally?).

I relied on piping the output to 2>/dev/null, but then real errors in my functions get swallowed. It’s an unfortunate situation 😦

We just tried wrangler 3.7.0 and can still observe this warning.

@GregBrimble @JacobMGEvans do you need anything from the community in order to be able to tackle this?

I’m seeing this as well with 3.1.1 and pages functions. Not cloning the response anywhere. The following function runs after a couple layers of _middleware.ts.

export const onRequestPost = async ({ data }) => {

  if (!data.userProfile.id) throw new Error('UserProfile ID is null!');

  const apiKey = await data.db
    .insert(userApiKeys)
    .values({
      ownerId: data.userProfile.id,
    })
    .returning()
    .get();

  return new Response(JSON.stringify(apiKey), {
    headers: { 'Content-Type': 'application/json' },
    status: 200,
  });
};

This error only seems to be thrown on a POST request, GET requests seem unaffected.

It also appears this message is generated at each level of _middleware that your request passes through. Simply reading the request body at each layer will squelch one instance of the message.

Something as simple as the following is enough:

const noopBody = request.json();

I was able to remove most of the recurrent message, but not all by using this method. Definitely seems like something is off here.

I think the lines below might be causing the problem, but I’m not sure.

https://github.com/cloudflare/workers-sdk/blob/a986f19f2d7989639524f9fd73761ea69aef4f6b/packages/wrangler/templates/pages-template-plugin.ts#L149

https://github.com/cloudflare/workers-sdk/blob/a986f19f2d7989639524f9fd73761ea69aef4f6b/packages/wrangler/templates/pages-template-worker.ts#L140

I’m using Astro though that should not matter when the request gets into the functions context.

Using the property bodyUsed I confirmed I have only read my request body one time. But I still get this warning about creating multiple streams.

It has to be a Pages functions bug of some sort.

I will probably test creating a Worker separate from Pages and use bindings to forward the request to the Worker and see what happens.

I’ll report back but it will have to be later today or tomorrow.

I’m getting five copies of this error on every call to a resource-only Remix action:

export const action = async () => { return new Response("Not implemented", { status: 501 }); };

Your worker created multiple branches of a single stream (for instance, by calling response.clone() or request.clone()) but did not read the body of both branches. This is wasteful, as it forces the system to buffer the entire stream of data in memory, rather than streaming it through. This may cause your worker to be unexpectedly terminated for going over the memory limit. If you only meant to copy the request or response headers and metadata (e.g. in order to be able to modify them), use the appropriate constructors instead (for instance, new Response(response.body, response), new Request(request), etc).

Edit: Found this Remix issue.

@EliBates you can upgrade the wrangler sdk to @3.1.0

Hi, I’m getting the same warning, I’m using wrangler@3.0.1 os: Ubuntu 23.04