portainer: portainer-agent panic: runtime error: index out of range(main.retrieveAdvertiseAddress)

Bug description

when update agent from v1.0.0 to v1.1.1 (tried v1.1.0 also) today:

# docker service ls |grep portainer-agent
mphbzaqggbwx        portainer-agent           global              3/5                 portainer/agent:1.1.1

# docker service logs -f portainer-agent
portainer-agent.0.xygaf4h8klec@node-03    | panic: runtime error: index out of range
portainer-agent.0.xygaf4h8klec@node-03    | 
portainer-agent.0.xygaf4h8klec@node-03    | goroutine 1 [running]:
portainer-agent.0.xygaf4h8klec@node-03    | main.retrieveAdvertiseAddress(0xa49bc2, 0x1c, 0xc4201d7f10, 0x1)
portainer-agent.0.xygaf4h8klec@node-03    |      /src/cmd/agent/main.go:92 +0x314
portainer-agent.0.xygaf4h8klec@node-03    | main.main()
portainer-agent.0.xygaf4h8klec@node-03    |      /src/cmd/agent/main.go:133 +0x25b
portainer-agent.0.ssr43heib0hq@node-04    | panic: runtime error: index out of range
portainer-agent.0.ssr43heib0hq@node-04    | 
portainer-agent.0.ssr43heib0hq@node-04    | goroutine 1 [running]:
portainer-agent.0.ssr43heib0hq@node-04    | main.retrieveAdvertiseAddress(0xa49bc2, 0x1c, 0xc4201bdf10, 0x1)
portainer-agent.0.ssr43heib0hq@node-04    |      /src/cmd/agent/main.go:92 +0x314
portainer-agent.0.ssr43heib0hq@node-04    | main.main()
portainer-agent.0.ssr43heib0hq@node-04    |      /src/cmd/agent/main.go:133 +0x25b

downgrade to v1.0.0

# docker service ls |grep portainer-agent
p71id0brrvhn        portainer-agent           global              5/5                 portainer/agent:1.0.0  

Expected behavior all agent is running.

Steps to reproduce the issue:

Steps to reproduce the behavior:

docker service rm portainer_agent
docker service create \
    --name portainer-agent \
    --detach=true \
    --network net-portainer \
    -e AGENT_CLUSTER_ADDR=tasks.portainer-agent \
    --mode global \
    --mount type=bind,src=/var/run/docker.sock,dst=/var/run/docker.sock \
    --mount type=bind,src=/data/docker/volumes,dst=/var/lib/docker/volumes \
    portainer/agent:1.1.1

Technical details:

  • Portainer version: 1.17.1
  • Docker version (managed by Portainer): 18.03.1-ce
  • Platform (windows/linux): linux/amd64
  • Command used to start Portainer (docker run -p 9000:9000 portainer/portainer):
mkdir /data/server/portainer/data -p
docker network create --driver overlay net-portainer
docker service create \
    --name portainer \
    --detach=true \
    --network net-portainer \
    --publish 9000:9000 \
    --replicas=1 \
    --constraint 'node.role == manager' \
    --mount type=bind,src=/data/server/portainer/data,dst=/data \
    portainer/portainer -H "tcp://tasks.portainer-agent:9001" --tlsskipverify

  • Browser:

Additional context Add any other context about the problem here.

About this issue

  • Original URL
  • State: closed
  • Created 6 years ago
  • Comments: 26 (19 by maintainers)

Commits related to this issue

Most upvoted comments

We just released the version 1.1.2 of the agent that includes the fix.

Thanks again everyone for the work/feedback.

@dehy if that happens again try just stop agent container from non-working host and let swarm re-schedule it there.

And please create new issue so it easier to keep track of these.

OK perfect, thanks for your feedback @opera443399 and @mrmachine I’ll push the fix.

@deviantony @olljanat sorry for the late support with a jet-lag

thank you, it works.

~]# docker service ls |grep agent
p71id0brrvhn        portainer-agent           global              5/5                 portainer/agent:1.0.0                                                 
~]# docker service update --detach=false portainer-agent --image portainer/agent:2102
portainer-agent
overall progress: 5 out of 5 tasks 
votldk83rc32: running   [==================================================>] 
zhi9fzzjsj7n: running   [==================================================>] 
zfs2vs9hyds4: running   [==================================================>] 
6jdb33ukhj98: running   [==================================================>] 
tjhtfnfamcf0: running   [==================================================>] 
verify: Service converged 
~]# docker service ls |grep agent
p71id0brrvhn        portainer-agent           global              5/5                 portainer/agent:2102                                                  
~]# docker service ps portainer-agent
ID                  NAME                                        IMAGE                   NODE                     DESIRED STATE       CURRENT STATE             ERROR               PORTS
19i99zhhag68        portainer-agent.tjhtfnfamcf02edpp0dgxinot   portainer/agent:2102    node-07   Running             Running 28 seconds ago                        
abatavln2j6k        portainer-agent.6jdb33ukhj980jkswhu78d7dt   portainer/agent:2102    node-05   Running             Running 32 seconds ago                        
iodxmx4zdb6z        portainer-agent.zfs2vs9hyds4jdv5nq1vdgzrm   portainer/agent:2102    node-04   Running             Running 36 seconds ago                        
lygkuofatnki        portainer-agent.zhi9fzzjsj7n9fsbh4junbj8n   portainer/agent:2102    node-03   Running             Running 41 seconds ago                        
jq1pvd3bejhr        portainer-agent.votldk83rc32e7wppz5szfnwx   portainer/agent:2102    node-06   Running             Running 45 seconds ago                        
tnbqd0yly5q5        portainer-agent.zfs2vs9hyds4jdv5nq1vdgzrm   portainer/agent:1.0.0   node-04   Shutdown            Shutdown 37 seconds ago                       
qan27vfc285i        portainer-agent.zhi9fzzjsj7n9fsbh4junbj8n   portainer/agent:1.0.0   node-03   Shutdown            Shutdown 41 seconds ago                       
a7zbm8v5ts0u        portainer-agent.votldk83rc32e7wppz5szfnwx   portainer/agent:1.0.0   node-06   Shutdown            Shutdown 47 seconds ago                       
wpxpe3ro7vde        portainer-agent.tjhtfnfamcf02edpp0dgxinot   portainer/agent:1.0.0   node-07   Shutdown            Shutdown 28 seconds ago                       
8yoaig1cfkuy        portainer-agent.6jdb33ukhj980jkswhu78d7dt   portainer/agent:1.0.0   node-05   Shutdown            Shutdown 33 seconds ago                       
~]# docker service ps portainer-agent
ID                  NAME                                        IMAGE                   NODE                     DESIRED STATE       CURRENT STATE             ERROR               PORTS
19i99zhhag68        portainer-agent.tjhtfnfamcf02edpp0dgxinot   portainer/agent:2102    node-07   Running             Running 33 seconds ago                        
abatavln2j6k        portainer-agent.6jdb33ukhj980jkswhu78d7dt   portainer/agent:2102    node-05   Running             Running 37 seconds ago                        
iodxmx4zdb6z        portainer-agent.zfs2vs9hyds4jdv5nq1vdgzrm   portainer/agent:2102    node-04   Running             Running 41 seconds ago                        
lygkuofatnki        portainer-agent.zhi9fzzjsj7n9fsbh4junbj8n   portainer/agent:2102    node-03   Running             Running 46 seconds ago                        
jq1pvd3bejhr        portainer-agent.votldk83rc32e7wppz5szfnwx   portainer/agent:2102    node-06   Running             Running 50 seconds ago                        
tnbqd0yly5q5        portainer-agent.zfs2vs9hyds4jdv5nq1vdgzrm   portainer/agent:1.0.0   node-04   Shutdown            Shutdown 42 seconds ago                       
qan27vfc285i        portainer-agent.zhi9fzzjsj7n9fsbh4junbj8n   portainer/agent:1.0.0   node-03   Shutdown            Shutdown 47 seconds ago                       
a7zbm8v5ts0u        portainer-agent.votldk83rc32e7wppz5szfnwx   portainer/agent:1.0.0   node-06   Shutdown            Shutdown 52 seconds ago                       
wpxpe3ro7vde        portainer-agent.tjhtfnfamcf02edpp0dgxinot   portainer/agent:1.0.0   node-07   Shutdown            Shutdown 34 seconds ago                       
8yoaig1cfkuy        portainer-agent.6jdb33ukhj980jkswhu78d7dt   portainer/agent:1.0.0   node-05   Shutdown            Shutdown 38 seconds ago                       
~]# docker service ps portainer-agent
ID                  NAME                                        IMAGE                   NODE                     DESIRED STATE       CURRENT STATE            ERROR               PORTS
19i99zhhag68        portainer-agent.tjhtfnfamcf02edpp0dgxinot   portainer/agent:2102    node-07   Running             Running 3 minutes ago                        
abatavln2j6k        portainer-agent.6jdb33ukhj980jkswhu78d7dt   portainer/agent:2102    node-05   Running             Running 3 minutes ago                        
iodxmx4zdb6z        portainer-agent.zfs2vs9hyds4jdv5nq1vdgzrm   portainer/agent:2102    node-04   Running             Running 3 minutes ago                        
lygkuofatnki        portainer-agent.zhi9fzzjsj7n9fsbh4junbj8n   portainer/agent:2102    node-03   Running             Running 3 minutes ago                        
jq1pvd3bejhr        portainer-agent.votldk83rc32e7wppz5szfnwx   portainer/agent:2102    node-06   Running             Running 3 minutes ago                        
tnbqd0yly5q5        portainer-agent.zfs2vs9hyds4jdv5nq1vdgzrm   portainer/agent:1.0.0   node-04   Shutdown            Shutdown 3 minutes ago                       
qan27vfc285i        portainer-agent.zhi9fzzjsj7n9fsbh4junbj8n   portainer/agent:1.0.0   node-03   Shutdown            Shutdown 3 minutes ago                       
a7zbm8v5ts0u        portainer-agent.votldk83rc32e7wppz5szfnwx   portainer/agent:1.0.0   node-06   Shutdown            Shutdown 3 minutes ago                       
wpxpe3ro7vde        portainer-agent.tjhtfnfamcf02edpp0dgxinot   portainer/agent:1.0.0   node-07   Shutdown            Shutdown 3 minutes ago                       
8yoaig1cfkuy        portainer-agent.6jdb33ukhj980jkswhu78d7dt   portainer/agent:1.0.0   node-05   Shutdown            Shutdown 3 minutes ago 

@deviantony Looks like the index out of range error is no longer appearing in my logs with the latest image. Appears stable and working.

Hi ! I’m the another user 😃

My Technical details:

  • Agent version: 1.1.1
  • Portainer version: 1.19.1
  • Docker version: 18.06.0-ce
  • Platform (windows/linux): linux/amd64 (Ubuntu 16.04.5 LTS up-to-date)

2 network cards : eth0 (10.5.65.209/31) and eth1 (10.5.65.211/31) (+ lo and docker interfaces)

Thank @deviantony and @olljanat !

Hi @opera443399

Thanks for the report, this was also reported by another user.

@olljanat keen to have a look at this?