micronaut-core: Micronaut 2: Http client read timeouts errors

Not sure exactly what is going on but getting a lot of those when deployed. I saw there was some issue regarding HTTP2 but this the same code as in 1.3. I have tried different stuff but cannot reproduce locally.

i.m.h.c.e.ReadTimeoutException: Read Timeout
	at i.m.h.c.e.ReadTimeoutException.<clinit>(ReadTimeoutException.java:26)
	at i.m.h.c.n.DefaultHttpClient.lambda$null$35(DefaultHttpClient.java:1152)
	at i.r.i.o.f.FlowableOnErrorNext$OnErrorNextSubscriber.onError(FlowableOnErrorNext.java:103)
	at i.m.r.r.RxInstrumentedSubscriber.onError(RxInstrumentedSubscriber.java:66)
	at i.r.i.o.f.FlowableTimeoutTimed$TimeoutSubscriber.onTimeout(FlowableTimeoutTimed.java:139)
	at i.r.i.o.f.FlowableTimeoutTimed$TimeoutTask.run(FlowableTimeoutTimed.java:170)
	at i.r.i.s.ScheduledRunnable.run(ScheduledRunnable.java:66)
	at i.r.i.s.ScheduledRunnable.call(ScheduledRunnable.java:57)
	at j.u.c.FutureTask.run(FutureTask.java:264)
	at j.u.c.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
	at j.u.c.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at j.u.c.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at java.lang.Thread.run(Thread.java:834)

About this issue

Original URL
State: closed
Created 4 years ago
Comments: 48 (34 by maintainers)

Commits related to this issue

Don't fireRead if handler body publisher is gone. Issue #3651 — committed to micronaut-projects/micronaut-core by graemerocher 4 years ago
Don't fireRead if handler body publisher is gone. Issue #3651 (#3695) Co-authored-by: James Kleeh <james.kleeh@gmail.com> — committed to micronaut-projects/micronaut-core by graemerocher 4 years ago

Most upvoted comments

Switching thread selection to AUTO solves the problem, some kind of detection of blocking the event loop would be nice.

dstepanov on Jul 12, 2020

We have come up against this issue recently with micronautVersion=3.5.4

The issue appears only when running in ECS Fargate (docker). We experience the issue with a graphQL endpoint which configured in application.yml:

graphql:
  enabled: true

And it only happens when the graphQL operation calls a downstream service using the Micronaut http client. After some testing, we found that it usually happens on the 5th or 6th network call out of 10 requests. The request times out (~10 seconds) and Read Timeout exception occurs.

The fix for us was setting the event loops in config:

micronaut:
  application:
    name: xxx

  netty:
    event-loops:
      custom-event-loop:
        num-threads: 20

  http:
    client:
      event-loop-group: custom-event-loop
      read-timeout: 45s

Posting, in case anyone else comes across this issue in later versions.

dejanvasic85 on Sep 16, 2022

Hi @dynaxis we saw something similar in 2.5.1 which was caused by: https://github.com/micronaut-projects/micronaut-core/issues/5389. This caused the thread selection logic to be bypassed on controllers which specified a body.

It was resolved in 2.5.2 tho.

amckee23 on May 18, 2021

I am intermittently getting this issue when upgraded from1.3 to 2.2.2 tried with thread selection as AUTO/IO increased timeout of http client to 20s each controller is annotated with @ExecuteOn(TaskExecutors.IO) I have blocking rest api calls in my service but from controller returning Single<List<MyObject>>

Any suggestion or workaround is appreciated.

nirvana124 on Jan 11, 2021

See https://docs.micronaut.io/latest/guide/configurationreference.html#io.micronaut.http.server.HttpServerConfiguration

graemerocher on Oct 21, 2020

We are considering adding blocked thread detection for the event loop.

Users can still go back to 1.x behavior by setting the thread selection strategy

graemerocher on Jul 10, 2020

@forbi do you have steps to reproduce for that app?

Deploy the app to k8s and hit the endpoint between 5-10 times.

marianozunino on Jul 7, 2020