ssh2: Process hangs on node <= 14.1.0 and performance issue
I created a reproducible repo https://github.com/davidfirst/stream-issue-node-14-1-0 with instructions on how to reproduce.
In short, I run exec() and write data (string) to the ssh server through the stream.stdin. When the string is small (~1MB) it works with no issue. When the string is 10MB it works on node <=14.0.0 but it hangs on node >= 14.1.0.
Another issue I found, not related to any node version is the performance. The longer the string the lower the transfer rate. I have a feeling that it is related to the back-pressure, but I tried with piping as well, which suppose to handle the backpressure.
More details in the README of the repo above.
I didn’t create two issues because there is a good chance that fixing the performance issue will fix the first issue as well.
(to have some more context for this issue, see https://github.com/teambit/bit/issues/2844).
About this issue
- Original URL
- State: closed
- Created 4 years ago
- Reactions: 2
- Comments: 16 (7 by maintainers)
FWIW v1.0.0 is now released (and AFAIU should solve the problems detailed in here).
FYI, the underlying regression in Node appears to have been fixed in 15.1.0, and will likely be backported to the 14.x series.
@davidfirst The error with cmake should not prevent installation of
ssh2, as it comes from an optional dependency. Likewise with the compiler error,ssh2should still be usable and should fall back to using node core crypto instead. The line number in the error is really strange though, as that does not match the problematic line shown…Anyway, I’ve fixed the crypto binding compilation issue in 249a48c5e2263ac63fa59c53855fcfd080f88ab4 and fc689952cf1bbdc4de38c1c0ae866573bf383906.
Just to chime in with my experience. In case you need confirmation.
I had the exact same issue with file downloads, my implementation was slow and I rewrote it to make it faster. Once I had done that, it started to hang after a random interval, with the same debug output. I upgraded node to 15 and the problem went away, I now have fast downloads with no hangs.
@davidfirst No definite date. I think there’s only a few things left to do/fix at the moment unless someone reports a serious bug/issue in the meantime. After those things are resolved I will probably do the release.
Soon™
The issue in question just happens to be worked around (AFAICT) in the rewrite I’m working on behind the scenes as it avoids using streams for internals as much as possible.