koa: Can't pipe stream to ctx.res

I’m trying to pipe streams to my response but it seems didn’t work

Simple test case:

const Koa = require('koa');
const app = new Koa();
const Router = require('koa-better-router');
const router = Router().loadMethods();
const request = require('request');

app.use(async (ctx, next) => {
    const start = new Date();
    await next();
    const ms = new Date() - start;
    console.log(`${ctx.method} ${ctx.url} - ${ms}ms`);
});

router.get('/', async (ctx) => {
    ctx.req.pipe(request(`https://www.google.com/?q=${ctx.query.q}`)).pipe(ctx.res);
});

app.use(router.middleware());

app.listen(3000, () => {
    console.log('listening on port 3000');
});

An error occurred:

$ node index.js 
listening on port 3000
GET / - 35ms
internal/streams/legacy.js:59
      throw er; // Unhandled stream error in pipe.
      ^

Error: write after end
    at ServerResponse.write (_http_outgoing.js:450:15)
    at Request.ondata (internal/streams/legacy.js:16:26)
    at emitOne (events.js:96:13)
    at Request.emit (events.js:189:7)
    at IncomingMessage.<anonymous> (/Users/knowlet/test/node_modules/request/request.js:1088:12)
    at emitOne (events.js:96:13)
    at IncomingMessage.emit (events.js:189:7)
    at IncomingMessage.Readable.read (_stream_readable.js:381:10)
    at flow (_stream_readable.js:761:34)
    at resume_ (_stream_readable.js:743:3)

And req.pipe(request(...)).pipe(res) works in express.js, I’m wondering is there anything I do wrong?

$ node -v
v7.6.0

About this issue

Original URL
State: closed
Created 7 years ago
Comments: 18 (4 by maintainers)

Most upvoted comments

You’ll want to let Koa handle the stream:

ctx.body = ctx.req.pipe(request(`https://www.google.com/?q=${ctx.query.q}`));

+41

PlasmaPower on Mar 18, 2017

you can use ctx.respond = false

Oh, you have that. Ok, that will do.

However, that is not the recommended solution, and it may cause problems with other middleware.

How about this

.use(async (ctx) =>
{
	// https://medium.com/@aickin/whats-new-with-server-side-rendering-in-react-16-9b0d78585d67
	ctx.res.write(header)
	await pipe(stream, ctx.res, { end: false })
	ctx.res.write(footer)
	ctx.res.end()
})

// Pipes `from` stream to `to` stream.
// Returns a `Promise`.
function pipe(from, to, options)
{
	return new Promise((resolve, reject) =>
	{
		from.pipe(to, options)
		from.on('error', reject)
		from.on('end', resolve)
	})
}

+18

catamphetamine on Oct 7, 2017

ctx.body = request(`https://www.google.com/?q=${ctx.query.q}`); will also work.

+11

ChristianUlbrich on Mar 30, 2017

@catamphetamine Koa can be seen as a wrapper around the Node.JS HTTP APIs. Using those APIs directly bypasses Koa. Writing to Node.JS HTTP objects managed by Koa is likely to conflict with Koa, as it does in this case. If you don’t want Koa to touch the response, you can use ctx.respond = false, which also makes OP’s example work perfectly. However, that is not the recommended solution, and it may cause problems with other middleware.

PlasmaPower on Oct 7, 2017

That should work, as Koa checks ctx.res.writable. You could also use the combined-stream package. Create a combined stream, append the header and footer, and then set ctx.body to the combined stream.

PlasmaPower on Oct 7, 2017

You’ll want to let Koa handle the stream:
ctx.body = ctx.req.pipe(request(`https://www.google.com/?q=${ctx.query.q}`));
ctx.req is a readable stream, so ctx.req.pipe(request(https://www.google.com/?q=${ctx.query.q})) will return the outgoing req–“request(https://www.google.com/?q=${ctx.query.q})”.

Finally, koajs will pipe ctx.body to ctx.res, this will trigger error: OutgoingMessage should be write-only. Piping from it is disabled.

ERR_STREAM_CANNOT_PIPE

koajs@2.13.1 nodejs@12.18.3

  ctx.body = ctx.req.pipe(request({
          headers, host: `${codeServerProxifier.host}`, path: url, method: ctx.method, // data: ctx.request.body,
          agent: httpAgent, timeout: 6000,
        }));

alanhg on Dec 6, 2022

For anyone using node-fetch. This seems to work:

const response = await fetch(...)
ctx.body = response.body

zachsa on Dec 1, 2020

@saxofficial Try multistream

import string_stream from 'string-to-stream'
import multi_stream from 'multistream'

ctx.status = 200
ctx.body = multi_stream
([
	string_stream(before_content),
	typeof content === 'string' ? string_stream(content) : content,
	string_stream(after_content)
])

It works as part of react-isomorphic-render.

catamphetamine on Oct 11, 2017