portainer: Portainer keeps crashing and restarting

Use Case (delete as appropriate): Using Portainer at Home

Have you reviewed our technical documentation and knowledge base? Yes

Question: Hi,

I installed Portainer CE :

docker run -d -p 8000:8000 -p 9443:9443 --name portainer \
--restart=always \
-v /var/run/docker.sock:/var/run/docker.sock \
-v portainer_data:/data \
portainer/portainer-ce:2.11.0

It worked. For some reasons, I restarted the container and apparently, Portainer keeps crashing and retarting.

Some diagnostics :

root@valinor:~# docker ps
CONTAINER ID   IMAGE                                          COMMAND        CREATED          STATUS                          PORTS     NAMES
da13b4e2d84d   portainer/portainer-ce:2.11.0                  "/portainer"   26 minutes ago   Restarting (1) 59 seconds ago             portainer```
root@valinor:~# tail -F /var/log/syslog
Jan 16 17:10:19 valinor systemd[1]: Started libcontainer container da13b4e2d84d9cb6f6b51271f11314b1f3e523974df3fde6663efb9a3583f9fa.
Jan 16 17:10:20 valinor kernel: [ 2775.604950] eth0: renamed from veth0e8fdee
Jan 16 17:10:20 valinor kernel: [ 2775.624457] IPv6: ADDRCONF(NETDEV_CHANGE): veth598adc9: link becomes ready
Jan 16 17:10:20 valinor kernel: [ 2775.628379] docker0: port 1(veth598adc9) entered blocking state
Jan 16 17:10:20 valinor kernel: [ 2775.628379] docker0: port 1(veth598adc9) entered forwarding state
Jan 16 17:10:20 valinor systemd-networkd[532]: veth598adc9: Gained carrier
Jan 16 17:10:20 valinor systemd-networkd[532]: docker0: Gained carrier
Jan 16 17:10:20 valinor systemd[1]: docker-da13b4e2d84d9cb6f6b51271f11314b1f3e523974df3fde6663efb9a3583f9fa.scope: Deactivated successfully.
Jan 16 17:10:20 valinor dockerd[645]: time="2022-01-16T17:10:20.597143792Z" level=info msg="ignoring event" container=da13b4e2d84d9cb6f6b51271f11314b1f3e523974df3fde6663efb9a3583f9fa module=libcontainerd namespace=moby topic=/tasks/delete type="*events.TaskDelete"
Jan 16 17:10:20 valinor containerd[571]: time="2022-01-16T17:10:20.601117753Z" level=info msg="shim disconnected" id=da13b4e2d84d9cb6f6b51271f11314b1f3e523974df3fde6663efb9a3583f9fa
Jan 16 17:10:20 valinor containerd[571]: time="2022-01-16T17:10:20.610786878Z" level=error msg="copy shim log" error="read /proc/self/fd/13: file already closed"
Jan 16 17:10:20 valinor kernel: [ 2776.250865] docker0: port 1(veth598adc9) entered disabled state
Jan 16 17:10:20 valinor kernel: [ 2776.250865] veth0e8fdee: renamed from eth0
Jan 16 17:10:20 valinor systemd-networkd[532]: veth598adc9: Lost carrier
Jan 16 17:10:20 valinor systemd-udevd[9607]: ethtool: autonegotiation is unset or enabled, the speed and duplex are not writable.
Jan 16 17:10:20 valinor networkd-dispatcher[557]: WARNING:Unknown index 116 seen, reloading interface list
Jan 16 17:10:20 valinor systemd-networkd[532]: veth598adc9: Link DOWN
Jan 16 17:10:20 valinor kernel: [ 2776.327528] docker0: port 1(veth598adc9) entered disabled state
Jan 16 17:10:20 valinor kernel: [ 2776.327528] device veth598adc9 left promiscuous mode
Jan 16 17:10:20 valinor kernel: [ 2776.327528] docker0: port 1(veth598adc9) entered disabled state
Jan 16 17:10:20 valinor systemd[1322]: run-docker-netns-dc359768e9f5.mount: Deactivated successfully.
Jan 16 17:10:20 valinor systemd[1322]: var-lib-docker-overlay2-cb862a618b1c4c43e8b3643aa50c8e0eabb34311f5e7f13621219b4d5b9ed02e-merged.mount: Deactivated successfully.
Jan 16 17:10:21 valinor systemd-networkd[532]: docker0: Lost carrier

I already destroyed the container, and recreated it with no success. Other docker instances run fine Thanks for any help Regards, Xavier

About this issue

  • Original URL
  • State: closed
  • Created 5 years ago
  • Reactions: 3
  • Comments: 22 (3 by maintainers)

Most upvoted comments

Hi, thanks @no1etal . Same problem and same solution…

docker rm portainer

docker rmi portainer/portainer-ce:latest

docker volume rm portainer_data

docker volume ls

docker volume create portainer_data

docker ps

docker run -d -p 8000:8000 -p 9443:9443 --name portainer --restart=always -v /var/run/docker.sock:/var/run/docker.sock -v portainer_data:/data portainer/portainer-ce:latest

Same issue here. Logs say level=info msg=“2022/01/18 01:57:26 failed initializing swarm stack manager: unexpected end of JSON input”

Had to do a “docker volume rm portainer_data” and recreate it with “docker volume create portainer_data” and relaunch docker run -d -p 8000:8000 -p 9443:9443 --name portainer
–restart=always
-v /var/run/docker.sock:/var/run/docker.sock
-v portainer_data:/data
cr.portainer.io/portainer/portainer-ce:2.11.0 Then just recreate the password for admin account

SOLVED: You need to remove old portainer_data volume

Hi, thanks @no1etal . Same problem and same solution…

docker rm portainer

docker rmi portainer/portainer-ce:latest

docker volume rm portainer_data

docker volume ls

docker volume create portainer_data

docker ps

docker run -d -p 8000:8000 -p 9443:9443 --name portainer --restart=always -v /var/run/docker.sock:/var/run/docker.sock -v portainer_data:/data portainer/portainer-ce:latest

Thanks that helped a lot!

I have new solution, just go inside the portainer_data volume and delete the malformed file(you’d better check if is really malformed before delete): /var/lib/docker/volumes/portainer_data/_data/docker_config/config.json, then the program will start properly.

Likewise, issue is repeating itself on 3 computers.

Same problem after examining the logs:

Find the Portainer container id docker ps -a

View the Portainer container logs docker container logs fa224a294958

failed initializing swarm stack manager | error="json: unexpected end of JSON input: "

I was able to resolve it by:

  1. Stop and remove portainer
  2. Create a dummy container with the Portainer volume attached
  3. Shell into the dummy container and snoop around, found an empty JSON file
  4. Deleted the file exited the dummy container and killed the dummy container
  5. Then started Portainer and everything seemed fine
docker stop portainer
docker rm portainer
docker run -d --rm --name dummy -v portainer_data:/data alpine tail -f /dev/null
docker exec -it dummy sh
rm /data/docker_config/config.json
exit
docker stop dummy
docker stop portainer
docker run -d -p 9000:9000 -p 8000:8000 --name=portainer --restart=always --privileged --env TZ=Africa/Johannesburg -v /etc/timezone:/etc/timezone:ro -v /etc/localtime:/etc/localtime:ro -v /var/run/docker.sock:/var/run/docker.sock -v portainer_data:/data portainer/portainer-ee

Afterward, I could log into Portainer and everything seemed to be fine. All my stacks were there and the rest of the config seemed fine.

my.portainer is keep crashing too .

This issue has been marked as stale as it has not had recent activity, it will be closed if no further activity occurs in the next 7 days. If you believe that it has been incorrectly labelled as stale, leave a comment and the label will be removed.