kubernetes: Kubelet hardcoded 4h timeout is undocumented / doesn't align with other timeout

Kubelet (1.17+) has a hardcoded 4h timeout for all HTTP connections (https://github.com/kubernetes/kubernetes/commit/ff689ce3deff52148f09818fdbc66edea197270e). Context: #104580

While this might be working as intended, I think that:

• This behavior is currently undocumented / not easily discoverable by users
• It doesn’t logically align with other documented timeout related setting (streamingConnectionIdleTimeout)

What happened:

For long lived connection such as streaming logs (kubelet logs -f <POD>), it used to be that non-idle connection would never timeout prior to 1.17.

• non-idle is defined based on the streamingConnectionIdleTimeout config which default to 4h.
  • Meaning as far as a log is emitted at least every 4h it is considered non-idle

Starting 1.17+ (https://github.com/kubernetes/kubernetes/commit/ff689ce3deff52148f09818fdbc66edea197270e), kubelet has a hardcoded 4h timeout value:

• Nothing in the config page that mention the other timeout value point toward this. I don’t think it is documented
• To me this doesn’t align logically with the other timeout value:
  • default timeout for idle connection if 4h (streamingConnectionIdleTimeout)
    • As all connection will be timeout after 4h, increasing this idle Timeout would have no effect and leave the user confused

What you expected to happen:

• User should be able to understand connection timeout by looking at documentation and/or config
• Having setting for timeout / default config which are meaningful. Increasing streamingConnectionIdleTimeout higher than the default of 4h as no effect.

I am honestly not sure what should be changed here:

• If this 4h timeout should simply be documented and leave everything as is
• If this hardcoded timeout should be made configurable and added to config
• …

How to reproduce it:

Run time kubelet logs -f <POD> on a pod that generate logs at a fixed interval (ex: every minute).

Environment:

• Kubernetes version: Kubelet 1.17+ (includes (https://github.com/kubernetes/kubernetes/commit/ff689ce3deff52148f09818fdbc66edea197270e)