cilium: Agent can't start up: msg="Error while creating daemon" error="listen tcp :40457: bind: address already in use"

The agent is trying over and over before it fails:

test-output.log:2020-10-04T21:29:56.420565028Z level=warning msg="Attempt to bind DNS Proxy failed, retrying in 4s" error="listen tcp :40457: bind: address already in use" subsys=fqdn/dnsproxy
test-output.log:2020-10-04T21:30:00.420875556Z level=warning msg="Attempt to bind DNS Proxy failed, retrying in 4s" error="listen tcp :40457: bind: address already in use" subsys=fqdn/dnsproxy
test-output.log:2020-10-04T21:30:04.42116714Z level=warning msg="Attempt to bind DNS Proxy failed, retrying in 4s" error="listen tcp :40457: bind: address already in use" subsys=fqdn/dnsproxy
test-output.log:2020-10-04T21:30:08.421419711Z level=warning msg="Attempt to bind DNS Proxy failed, retrying in 4s" error="listen tcp :40457: bind: address already in use" subsys=fqdn/dnsproxy
test-output.log:2020-10-04T21:30:12.42518694Z level=warning msg="Attempt to bind DNS Proxy failed, retrying in 4s" error="listen tcp :40457: bind: address already in use" subsys=fqdn/dnsproxy
test-output.log:2020-10-04T21:30:16.42229825Z level=fatal msg="Error while creating daemon" error="listen tcp :40457: bind: address already in use" subsys=daemon

Example of failed CI run: https://jenkins.cilium.io/job/Cilium-PR-K8s-GKE/2645/

bf78dc42_K8sIdentity_Identity_expiration_Expiration_of_CiliumIdentity.zip

About this issue

Original URL
State: closed
Created 4 years ago
Comments: 16 (16 by maintainers)

Commits related to this issue

cilium, gops: remap to fixed port to avoid collision with nodeport range [ upstream commit 7757d31604740a663214678d7469d509c5508bcb ] Manually backported from #14329 to address #13400 for v1.8. Lee... — committed to tklauser/cilium by tklauser 3 years ago
cilium, gops: remap to fixed port to avoid collision with nodeport range [ upstream commit 7757d31604740a663214678d7469d509c5508bcb ] Manually backported from #14329 to address #13400 for v1.8. Lee... — committed to cilium/cilium by tklauser 3 years ago
cilium, gops: remap to fixed port to avoid collision with nodeport range [ upstream commit 7757d31604740a663214678d7469d509c5508bcb ] Manually backported from #14329 to address #13400 for v1.8. Lee... — committed to DataDog/cilium by tklauser 3 years ago

Most upvoted comments

In the last ~2 months the only re-occurrence of this flake was on v1.8 where #14329 wasn’t backported. So it indeed looks like this was caused by gops using the same port as the proxy. I’ll send a manual backport of #14329 targeting v1.8, so this issue can be closed.

tklauser on Apr 10, 2021