rancher: [rancher/server] Unable to start rancher/server:v0.30.0

Commands were:

docker pull rancher/server:v0.30.0
docker run -d --name=rancher-server-test --restart=always -p 8080:8080 rancher/server:v0.30.0

under docker ps it container stays there with STATUS: Restarting (0) 2 minutes ago etc. docker logs rancher-server-test give similar messages a few times:

150728 19:03:35 [Warning] Using unique option prefix key_buffer instead of key_buffer_size is deprecated and will be removed in a future release. Please use the full name instead.
150728 19:03:35 [Warning] option 'max_connections': unsigned value 10000000000000 adjusted to 100000
150728 19:03:35 [Warning] option 'max_binlog_size': unsigned value 10485760000000000000 adjusted to 1073741824
150728 19:03:35 [Note] /usr/sbin/mysqld (mysqld 5.5.43-0ubuntu0.14.04.1) starting as process 12 ...
s6-svscanctl: fatal: unable to control /service: supervisor not listening
150728 19:07:01 [Warning] Using unique option prefix key_buffer instead of key_buffer_size is deprecated and will be removed in a future release. Please use the full name instead.
150728 19:07:01 [Warning] option 'max_connections': unsigned value 100000000000000 adjusted to 100000
150728 19:07:01 [Warning] option 'max_binlog_size': unsigned value 12623879631452241920 adjusted to 1073741824
150728 19:07:01 [Note] /usr/sbin/mysqld (mysqld 5.5.43-0ubuntu0.14.04.1) starting as process 11 ...
s6-svscanctl: fatal: unable to control /service: supervisor not listening

Any advice?

About this issue

Original URL
State: closed
Created 9 years ago
Comments: 20 (6 by maintainers)

Most upvoted comments

The logs show that there was no space left on device. So checking the host machine, there was also no space left there, this was the issue.

Thanks for the help!

Mmdixon on Jul 28, 2015

@sebmoule if MySQL isn’t running because of out of disk space, is there anything you can free up on the host?

if its a docker host you could try the usual… docker rmi $(docker images -aq -f 'dangling=true') Sometimes docker logs get really large… if you go to /var/lib/docker/containers and do a du -sh * If you find a large one, you can cd to the directory and cat /dev/null > logfile (This will erase the log completely but frees up space.) Or, any usual tricks to clean stuff up on a linux host 😃

In v0.39.0 we set the log_expire_delay to 2 days and moved to file_per_table for innodb. So this shouldn’t happen any more.

Another approach would be to get a little bit of space freed up to start mysql, and manually clean up bin logs in mysql:

docker run -it --volumes-from=<oldrancher> rancher/server:<old rancher version> bash service mysql start mysql mysql> PURGE LOGS BEFORE 2015-09-30 00:00:00 service mysql stop exit

This approach might work a little better since you won’t have rancher trying to run the migrations to the new version.

cloudnautique on Sep 30, 2015

@Mmdixon can you start the container like so: docker run -it -p 8080:8080 rancher/server:v0.30.0 bash

That should place you on a command line. Then run /usr/bin/s6-svscan /service

In a separate terminal, get the ID of the container you just started… and run: docker exec -it <id> bash

From that shell, can you get the log output from /var/log/mysql/error.log?

cloudnautique on Jul 28, 2015