koa: http2 throws ERR_HTTP2_INVALID_STREAM with minor amount of concurrency

Moving from https://github.com/nodejs/node/issues/22135; as per the comments, it was confirmed to be koa-specific.

Version: v10.8.0
Platform: Linux 86f664ae731c 4.9.93-linuxkit-aufs #1 SMP Wed Jun 6 16:55:56 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux (ubuntu:18.04 Docker image)
Subsystem: http2

After a small handful of concurrent requests, http2 fails with the following error:

internal/http2/core.js:1791
      throw new ERR_HTTP2_INVALID_STREAM();
      ^

Error [ERR_HTTP2_INVALID_STREAM]: The stream has been destroyed
    at ServerHttp2Stream.sendTrailers (internal/http2/core.js:1791:13)
    at ServerHttp2Stream.onStreamTrailersReady (internal/http2/compat.js:377:8)
    at ServerHttp2Stream.emit (events.js:182:13)
    at Http2Stream.onStreamTrailers [as ontrailers] (internal/http2/core.js:318:15)
    at ServerHttp2Stream.submitRstStream (internal/http2/core.js:328:19)
    at ServerHttp2Stream.finishCloseStream (internal/http2/core.js:1533:3)
    at closeStream (internal/http2/core.js:1517:7)
    at ServerHttp2Stream.close (internal/http2/core.js:1846:5)
    at state.streams.forEach (internal/http2/core.js:2691:46)
    at Map.forEach (<anonymous>)

Test case:

cat > server.js << EOM
	const app = new (require('koa'))();
	const crypto = require('crypto');
	const fs = require('fs');
	const http2 = require('http2');


	app.use(async ctx => {
		await new Promise(resolve => setTimeout(resolve, 1000));

		ctx.body = 'balls';
		ctx.status = 200;
	});

	http2.createSecureServer(
		{
			allowHTTP1: true,
			cert: fs.readFileSync('cert.pem'),
			key: fs.readFileSync('key.pem'),
			dhparam: fs.readFileSync('dhparams.pem'),
			secureOptions:
				crypto.constants.SSL_OP_NO_SSLv3 |
				crypto.constants.SSL_OP_NO_TLSv1
		},
		app.callback()
	).listen(
		31337
	);
EOM

killall node
node server.js &
echo -n > count
while ps ux | grep server.js | grep -v grep 2> /dev/null ; do
	echo -e '1\n2\n3\n4\n5\n6\n7\n8\n9\n10'
done | xargs -P10 bash -c '
	echo >> count
	curl -sk https://localhost:31337 > /dev/null
'

When the error occurs, you can manually kill the command with ctrl+C and wc -l count to view how many requests were initiated before the failure. For me it’s been anywhere between 10 and 70.

This doesn’t seem to be reproducible when using either https or spdy instead of http2.

About this issue

Original URL
State: closed
Created 6 years ago
Comments: 20 (11 by maintainers)

Most upvoted comments

I’ve fixed the issue in Node, and the PR has been merged. https://github.com/nodejs/node/pull/23146

If anyone wants to try they can build the latest master or wait for the next release.

edevil on Oct 3, 2018

It seems to be Koa-specific based on the issue linked at the top.

buu700 on Sep 26, 2018

@buu700 did you manage to solve this issue?

Here a small repro:

import pkg from './package.json';

import fs from 'fs';
import http2 from 'http2';
import koa from 'koa';
import axios from 'axios';

const app = new koa();

app.use(async ctx => {
    try {
        let { data: result } = await axios.get('xxx');
        ctx.body = result;
    }
    catch (error) {
        ctx.status = 500;
        console.error('Internal Server Error', error.message || error);
    }
});

const cert = fs.readFileSync(pkg.ssl.cert);
const key = fs.readFileSync(pkg.ssl.key);

/** KOA + HTTP2 **/
const server1 = http2.createSecureServer({ cert, key }, app.callback()).listen(8443, error => error ? console.error(error) : console.info(`koa+http2s serving on port ${8443}`));
server1.on('error', console.error);
/** KOA + HTTP2 end **/

/** HTTP2 plain **/
http2.createSecureServer({ cert, key }, async (req, res) => {
    try {
        let { data: result } = await axios.get('xxx');
        res.end(JSON.stringify(result, null, 4));
    }
    catch (error) {
        res.end('Internal Server Error');
    }
}).listen(8444, error => error ? console.error(error) : console.info(`http2s serving on port ${8444}`));
/** HTTP2 plain end **/

process.on('SIGINT', process.exit);

The plain http2 server works fine, but the koa+http2 server kills the process because of a SIGINT: it’s like koa is trying to send content over a closed stream because the process dies after koa processed the request (the “curl” actually receives data but in the meanwhile node dies!).

Error [ERR_HTTP2_INVALID_STREAM]: The stream has been destroyed
    at ServerHttp2Stream.sendTrailers (internal/http2/core.js:1786:13)
    at ServerHttp2Stream.onStreamTrailersReady (internal/http2/compat.js:382:8)
    at ServerHttp2Stream.emit (events.js:182:13)
    at ServerHttp2Stream.EventEmitter.emit (domain.js:442:20)
    at Http2Stream.onStreamTrailers [as ontrailers] (internal/http2/core.js:325:15)
    at ServerHttp2Stream.submitRstStream (internal/http2/core.js:335:19)
    at ServerHttp2Stream.finishCloseStream (internal/http2/core.js:1523:3)
    at closeStream (internal/http2/core.js:1507:7)
    at ServerHttp2Stream.close (internal/http2/core.js:1840:5)
    at state.streams.forEach (internal/http2/core.js:2687:46)

damianobarbati on Sep 26, 2018

Yea, I noticed you had Docker listed. I don’t have it installed and about to head to sleep for the night. If no one else stops by I’ll take another stab with Docker.

dougwilson on Aug 5, 2018