spring-cloud-netflix: Read Timeout on Request

I’m getting read timeout for a service that is taking a long time (error below). No problem with other service in the same machine, I can see result just the service that takes a long time. I set the ribbon readtimeout, no changes. It seems to be set at 15 seconds. Here’s my application.yaml

ribbon:
  ReadTimeout: 30000
  ConnectTimeout: 30000
zuul:
  routes:
    myservice:
      path: /service/**
      url: http://ip_here:8081/service

If I want it per service, do I put it below myservice? I tried that as well, no luck.

java.net.SocketTimeoutException: Read timed out
        at java.net.SocketInputStream.socketRead0(Native Method)
        at java.net.SocketInputStream.read(SocketInputStream.java:152)
        at java.net.SocketInputStream.read(SocketInputStream.java:122)
        at org.apache.http.impl.io.AbstractSessionInputBuffer.fillBuffer(AbstractSessionInputBuffer.java:166)
        at org.apache.http.impl.io.SocketInputBuffer.fillBuffer(SocketInputBuffer.java:90)
        at org.apache.http.impl.io.AbstractSessionInputBuffer.readLine(AbstractSessionInputBuffer.java:281)
        at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:92)
        at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:62)
        at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:254)
        at org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:289)
        at org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:252)
        at org.apache.http.impl.conn.AbstractClientConnAdapter.receiveResponseHeader(AbstractClientConnAdapter.java:219)
        at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:300)
        at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:127)
        at org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:712)
        at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:517)
        at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906)
        at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:827)
        at org.springframework.cloud.netflix.zuul.filters.route.SimpleHostRoutingFilter.forwardRequest(SimpleHostRoutingFilter.java:262)
        at org.springframework.cloud.netflix.zuul.filters.route.SitRoutingFilter.forward(SimpleHostRoutingFilter.java:225)
        at org.springframework.cloud.netflix.zuul.filters.route.SimpleHostRoutingFilter.run(SimpleHostRoutingFilter.java:177)
        at com.netflix.zuul.ZuulFilter.runFilter(ZuulFilter.java:112)
        at com.netflix.zuul.FilterProcessor.processZuulFilter(FilterProcessor.java:197)
        at com.netflix.zuul.FilterProcessor.runFilters(FilterProcessor.java:161)
        at com.netflix.zuul.FilterProcessor.route(FilterProcessor.java:120)
        at com.netflix.zuul.ZuulRunner.route(ZuulRunner.java:84)
        at com.netflix.zuul.http.ZuulServlet.route(ZuulServlet.java:111)
        at com.netflix.zuul.http.ZuulServlet.service(ZuulServlet.java:77)

Thanks in advance.

About this issue

  • Original URL
  • State: closed
  • Created 9 years ago
  • Comments: 32 (10 by maintainers)

Most upvoted comments

Don’t forget Zuul is using Hystric underneath - so you may well have a Hystrix timeout instead (1000ms by default). You can either disable the timeout feature of Hystrix or increase its value:

# Disable Hystrix timeout globally (for all services)
hystrix.command.default.execution.timeout.enabled: false

# Disable Hystrix timeout for a single service
hystrix.command.<serviceName>.execution.timeout.enabled: false

# Increase the Hystrix timeout to 60s (globally)
hystrix.command.default.execution.isolation.thread.timeoutInMilliseconds: 60000

# Increase the Hystrix timeout to 60s (per service)
hystrix.command.<serviceName>.execution.isolation.thread.timeoutInMilliseconds: 60000

Be careful however with the last two properties as they apply to the THREAD isolation strategy. Although the Netflix documentation says it is the default, I noticed SpringCloud version of Zuul defaults to SEMAPHORE. See https://github.com/Netflix/Hystrix/wiki/Configuration for more information.

The same error happened to me randomly. I had to wait some time and trigger the request again or restart the service completely to reproduce this behavior.

Changing the configuration to:

zuul:
    host:
        connect-timeout-millis: 10000
        socket-timeout-millis: 60000

hystrix:
    command:
        default:
            execution:
                isolation:
                    thread:
                        timeoutInMilliseconds: 60000

Ends up in some calls having a “delay” of about 2 seconds, but no forwarding error is popping up.

I know this is a closed issue, but I’m not sure where this comment should go. It would be nice to document the additional zuul properties such as the timeout settings used by SimpleHostRoutingFilter. It took me a while to figure out why ribbon.ReadTimeout and ribbon.ConnectTimeout had no effect on zuul request timeouts. Without documentation, it’s hard to know which native Netflix properties are used to adjust the default behavior of Spring Cloud Netflix components.

Only adding this configuration works for me:

ribbon:
  ReadTimeout: 20000
  ConnectTimeout: 20000

the only way I got it working is by disabling hystrix and setting those values, setting those properties from .propterites files did not work for me I had to access the com.netflix.config.ConfigurationManager

    @PostConstruct
    void disableHystrix() {


        ConfigurationManager.getConfigInstance().setProperty("hystrix.command.default.circuitBreaker.enabled", false);
        ConfigurationManager.getConfigInstance().setProperty("hystrix.command.default.execution.timeout.enabled", false);
        ConfigurationManager.getConfigInstance().setProperty("hystrix.command.default.execution.isolation.thread.interruptOnTimeout", false);
        ConfigurationManager.getConfigInstance().setProperty("hystrix.command.default.execution.isolation.thread.timeoutInMilliseconds", apiGatewayConfigs.getGlobalTimeout());
        ConfigurationManager.getConfigInstance().setProperty("ribbon.ReadTimeout", apiGatewayConfigs.getGlobalTimeout());

    }

I know this issue is closed, but for any people running into this problem for the first time the solutions above didn’t work for me, but setting zuul.host.connect-timeout-millis and zuul.host.socket-timeout-millis did the trick. See this issue discussion for details

I make it out. To make zuul & ribbon timeout to work, you shoule set hystrix.command.default.execution.timeout.enable to false. WTF!

hystrix:
  command:
    default:
      execution:
        timeout:
          enabled: false

zuul:
  host:
    socket-timeout-millis: 5000
    connect-timeout-millis: 5000

ribbon:
    ConnectTimeout: 5000
    ReadTimeout: 5000

The ReadTimeout is targeted at the the underlying RestClient that makes the actual call to the service. So, when does it take effect? Don’t forget Ribbon is there to perform routing, failover and load balancing - so a single client request may lead to multiple attempts to contact a service. Stated differently, if you have two instances of a service and one is not reachable, you end up with two RestClient calls within a single Hystrix command. A bit like multiple database queries (RestClient calls) within the same transaction (HystricCommand). The Hystrix timeout is like the transaction timeout, the RestClient ReadTimeout is like the query timeout… (hope it is clear)

As far as I can tell, you should be able to override it by adding the following to your Zuul configuration:

# Global - affects every Ribbon instances (not only your services but also Eureka client!)
ribbon.ReadTimeout = 6000

# Per service
<clientName>.ribbon.ReadTimeout = 60000

Thanks @ajanitin zuul.host.connect-timeout-millis: xxx zuul.host.socket-timeout-millis: xxx

This works for me.

Set appropriate timeout period of the property zuul.host.socket-timeout-millis = XXXXX

It would be hystrix.command.default.execution.isolation.thread.timeoutInMilliseconds or hystrix.command. myservice.execution.isolation.thread.timeoutInMilliseconds