aws-cdk: JsiiErrors / EAGAIN when using node>=13.2 with Python, Java & .NET

When deploying, some file is temporarily unavailable.

Reproduction Steps

from aws_cdk import (
    core
)

class Stack(core.Stack):
    def __init__(self, scope: core.Construct, id: str, **kwargs) -> None:
        super().__init__(scope, id, **kwargs)

app = core.App()
stack = Stack(app, "Test")

app.synth()

Then deploy

$ cdk deploy --app "python stack.py"

Error Log

$ cdk deploy --app "python stack.py"
internal/fs/utils.js:220
    throw err;
    ^

Error: EAGAIN: resource temporarily unavailable, read
    at Object.readSync (fs.js:516:3)
    at SyncStdio.readLine (/home/drluke/work/di-client-api-message-dispatcher/venv/lib/python3.7/site-packages/jsii/_embedded/jsii/jsii-runtime.js:13322:25)
    at InputOutput.read (/home/drluke/work/di-client-api-message-dispatcher/venv/lib/python3.7/site-packages/jsii/_embedded/jsii/jsii-runtime.js:13272:34)
    at KernelHost.run (/home/drluke/work/di-client-api-message-dispatcher/venv/lib/python3.7/site-packages/jsii/_embedded/jsii/jsii-runtime.js:7228:32)
    at Immediate.<anonymous> (/home/drluke/work/di-client-api-message-dispatcher/venv/lib/python3.7/site-packages/jsii/_embedded/jsii/jsii-runtime.js:7236:37)
    at processImmediate (internal/timers.js:439:21) {
  errno: -11,
  syscall: 'read',
  code: 'EAGAIN'
}
Traceback (most recent call last):
  File "stack_bugreport.py", line 1, in <module>
    from aws_cdk import (
  File "/home/drluke/work/di-client-api-message-dispatcher/venv/lib/python3.7/site-packages/aws_cdk/aws_apigateway/__init__.py", line 548, in <module>
    import aws_cdk.aws_certificatemanager
  File "/home/drluke/work/di-client-api-message-dispatcher/venv/lib/python3.7/site-packages/aws_cdk/aws_certificatemanager/__init__.py", line 104, in <module>
    import aws_cdk.aws_cloudformation
  File "/home/drluke/work/di-client-api-message-dispatcher/venv/lib/python3.7/site-packages/aws_cdk/aws_cloudformation/__init__.py", line 125, in <module>
    import aws_cdk.aws_iam
  File "/home/drluke/work/di-client-api-message-dispatcher/venv/lib/python3.7/site-packages/aws_cdk/aws_iam/__init__.py", line 169, in <module>
    import aws_cdk.region_info
  File "/home/drluke/work/di-client-api-message-dispatcher/venv/lib/python3.7/site-packages/aws_cdk/region_info/__init__.py", line 113, in <module>
    __jsii_assembly__ = jsii.JSIIAssembly.load("@aws-cdk/region-info", "1.18.0", __name__, "region-info@1.18.0.jsii.tgz")
  File "/home/drluke/work/di-client-api-message-dispatcher/venv/lib/python3.7/site-packages/jsii/_runtime.py", line 40, in load
    _kernel.load(assembly.name, assembly.version, os.fspath(assembly_path))
  File "/home/drluke/work/di-client-api-message-dispatcher/venv/lib/python3.7/site-packages/jsii/_kernel/__init__.py", line 209, in load
    self.provider.load(LoadRequest(name=name, version=version, tarball=tarball))
  File "/home/drluke/work/di-client-api-message-dispatcher/venv/lib/python3.7/site-packages/jsii/_kernel/providers/process.py", line 330, in load
    return self._process.send(request, LoadResponse)
  File "/home/drluke/work/di-client-api-message-dispatcher/venv/lib/python3.7/site-packages/jsii/_kernel/providers/process.py", line 310, in send
    self._next_message(), _ProcessResponse_R
  File "/home/drluke/work/di-client-api-message-dispatcher/venv/lib/python3.7/site-packages/jsii/_kernel/providers/process.py", line 261, in _next_message
    return json.loads(self._process.stdout.readline(), object_hook=ohook)
  File "/usr/lib/python3.7/json/__init__.py", line 361, in loads
    return cls(**kw).decode(s)
  File "/usr/lib/python3.7/json/decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/lib/python3.7/json/decoder.py", line 355, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
Subprocess exited with error 1

Workaround for the time being

Downgrade node to version 12.13.1

Environment

CLI Version : 1.18.0 (build bc924bc)
Framework Version: 1.18.0
OS : Linux
Language : Python

This is 🐛 Bug Report

About this issue

Original URL
State: closed
Created 5 years ago
Reactions: 16
Comments: 46 (22 by maintainers)

Commits related to this issue

fix(runtime): runtime crashes with EAGAIN trying to read from STDIN When using node >= 13.2, attempts to synchronously read from STDIN often result in `EAGAIN` being raised. This is due to the STDIN ... — committed to aws/jsii by RomainMuller 5 years ago
fix(runtime): runtime crashes with EAGAIN trying to read from STDIN When using node >= 13.2, attempts to synchronously read from STDIN often result in `EAGAIN` being raised. This is due to the STDIN ... — committed to aws/jsii by RomainMuller 5 years ago
fix(runtime): runtime crashes with EAGAIN trying to read from STDIN (#1143) When using node >= 13.2, attempts to synchronously read from STDIN often result in `EAGAIN` being raised. This is due to t... — committed to aws/jsii by RomainMuller 5 years ago
fix(@jsii/runtime): "maximum call stack size exceeded" in SyncStdio.readLine When using node `>= 12.17`, `EAGAIN` errors consistently occur when trying to read from `stdin` when there is no available... — committed to aws/jsii by RomainMuller 4 years ago
fix(@jsii/runtime): "maximum call stack size exceeded" in SyncStdio.readLine (#1717) When using node `>= 12.17`, `EAGAIN` errors consistently occur when trying to read from `stdin` when there is no ... — committed to aws/jsii by RomainMuller 4 years ago

Most upvoted comments

FYI, nodejs v13.1.0 works as well, but 13.2.0 fails.

+13

pd5rm on Nov 26, 2019

Okay - so we have a smoking gun - it appears that there is a behavior change in node 13.2 that causes the symptoms. We have to investigate deeper to understand whether node 13.2 reveals a latent bug in our code, or whether we should report a bug to them.

RomainMuller on Nov 27, 2019

I can confirm downgrading nodejs from 13.2.0 to 13.1.0 fixes this problem.

machielg on Dec 2, 2019

same here. Downgraded to stable node version: 12.3.1 fixed the issue

I was on 13.3

sebgoa on Dec 12, 2019

I am having the same issue. Tried downgrading CDK dependencies to 1.16.3, but it still failed with 1.17.1 cli and latest node. Having downgraded node to 12.13.1 helped me. Hope it helps someone.

pepastach on Nov 26, 2019

@brainstorm this is definitely under discussion. Keep an eye out for updates.

For others watching this issue: https://github.com/aws/jsii/pull/1143 implemented a short term solution to retry failed reads from stdin immediately after EAGAIN. This is not meant to be permanent and other issues have arisen that are related, specifically #5877. Downgrading to Node 12 is still the recommended workaround if you run into this problem on cdk >= v1.20.0.

https://github.com/aws/jsii/issues/1142 is the long term fix but it requires significant investigation.

MrArnoldPalmer on Jan 30, 2020

Indeed @chefren , that was the culprit, I fixed it like this:

$ pip freeze | grep aws | sed 's/==.*//' | xargs pip install -U

Since I have quite a few of aws-cdk modules on that virtualenv (miniconda, but same thing)… IMHO, there should be a toplevel aws-cdk metapackage that updates all submodules in lockstep by just running:

pip install -U aws-cdk

As I raised on https://github.com/aws/aws-cdk/issues/3406#issuecomment-525688693 and also mentioned on issue https://github.com/aws/aws-cdk/issues/972 … this issue will keep coming up folks.

The current xargs hack above is not pythonic and should be fixed if you would like to avoid more issues/requests/bugs from the python community 😉

Cheers!

brainstorm on Jan 29, 2020

What worked for me on nodejs v13.2.0 was to catch EAGAIN in SyncStdio#readLine, and return this.readLine(). This is a hacky workaround, as it simply ignores the EAGAIN.

This also made me wonder if SyncStdio can be completely replaced by nodejs’s readline module? SyncStdio seems to be implementing the same functionality that’s available natively in nodejs.

netroy on Nov 29, 2019

Using the Dockerimage I can confirm that downgrading node to version 12.13.1 is a workaround. I’ve added it to the first post in the issue.

DrLuke on Nov 26, 2019

This happened to me with a project that has already been deployed successfully multiple times before upgrading to 1.18.0, any cdk command gives me the same error message.

DrLuke on Nov 26, 2019