istio: "Envoy may have been out of memory killed. Check memory usage and limits."

Bug description I’m seeing a sidecar for one of our applications continually have its memory grow and eventually OOM:

2020-01-02T14:44:33.668963Z     info    Envoy proxy is ready
2020-01-04T15:26:26.987721Z     warn    Envoy may have been out of memory killed. Check memory usage and limits.
2020-01-04T15:26:26.987766Z     error   Epoch 0 exited with error: signal: killed
2020-01-04T15:26:26.987770Z     info    No more active epochs, terminating

You can see the memory growth in this graph: Screen Shot 2020-01-05 at 15 28 24

Typical sidecar usage is about 50mb.

It’s consistent, so i’m confident it’s something unique about this particular application that’s causing it - but can’t figure out what, I’m just not sure where to begin with debugging it - so looking for some advice please.

Expected behavior Envoy to not OOM

Steps to reproduce the bug Can’t give them yet

Version (include the output of istioctl version --remote and kubectl version and helm version if you used Helm) 1.4.2

How was Istio installed? Helm

Environment where bug was observed (cloud vendor, OS, etc) GKE 1.14

About this issue

Original URL
State: closed
Created 4 years ago
Reactions: 1
Comments: 60 (48 by maintainers)

Most upvoted comments

@15989206052 your version is about 2.5 years old. Please upgrade, no one will help you on this old of a version.

many thx, I forgot that I described it like this, i always use the almost latest version, by the way, i has resolved it by limiting MAX_ACTIVES_STREAMS.

Hi! you mean MAX_ACTIVE_STREAMS by max_concurrent_streams in http2_protocol_options? Which value have you set on that option?

Thx in advance!

lmgr10 on Nov 5, 2021