influxdb: Starting InfluxDB on CircleCI fails to start via a script even though it says it does

Hi,

I’m trying to install and run my own tests against InfluxDB on a CircleCI build instance. InfluxDB seems to be installed and start correctly with the following output but unfortunately my tests fail due to a ‘Connection refused’ error (which hints that InfluxDB may not be running properly):

$ wget http://s3.amazonaws.com/influxdb/influxdb_latest_amd64.deb
--2015-02-11 17:40:25--  http://s3.amazonaws.com/influxdb/influxdb_latest_amd64.deb
Resolving s3.amazonaws.com (s3.amazonaws.com)... 54.231.96.128
Connecting to s3.amazonaws.com (s3.amazonaws.com)|54.231.96.128|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 16572810 (16M) [binary/octet-stream]
Saving to: `influxdb_latest_amd64.deb'
100%[======================================>] 16,572,810  53.8M/s   in 0.3s    
2015-02-11 17:40:25 (53.8 MB/s) - `influxdb_latest_amd64.deb' saved [16572810/16572810]

$ sudo dpkg -i influxdb_latest_amd64.deb
Selecting previously unselected package influxdb.
(Reading database ... 525927 files and directories currently installed.)
Unpacking influxdb (from influxdb_latest_amd64.deb) ...
Setting up influxdb (0.8.8) ...
 Removing any system startup links for /etc/init.d/influxdb ...
 Adding system startup for /etc/init.d/influxdb ...
   /etc/rc0.d/K20influxdb -> ../init.d/influxdb
   /etc/rc1.d/K20influxdb -> ../init.d/influxdb
   /etc/rc6.d/K20influxdb -> ../init.d/influxdb
   /etc/rc2.d/S20influxdb -> ../init.d/influxdb
   /etc/rc3.d/S20influxdb -> ../init.d/influxdb
   /etc/rc4.d/S20influxdb -> ../init.d/influxdb
/etc/rc5.d/S20influxdb -> ../init.d/influxdb

$sudo /etc/init.d/influxdb start
Setting ulimit -n 65536
Starting the process influxdb [ OK ]
influxdb process was started [ OK ]

However when I ssh into the CircleCI build instance I run the following:

$ sudo /etc/init.d/influxdb stop
Setting ulimit -n 65536
influxdb process is not running [ FAILED ]

So this indicates InfluxDB was never running. I then start it with the following and it starts successfully and I am then then able to connect to InfluxDB and run the tests successfully:

$ sudo /etc/init.d/influxdb start
Setting ulimit -n 65536
Starting the process influxdb [ OK ]
influxdb process was started [ OK ]

The entire log file from both the start, stop and then start again is:

$ cat /opt/influxdb/shared/log.txt 
[2015/02/11 18:15:40 GMT] [INFO] (main.setupLogging:69) Redirectoring logging to /opt/influxdb/shared/log.txt
[2015/02/11 18:15:40 GMT] [INFO] (main.start:164) Starting Influx Server 0.8.8 bound to 0.0.0.0...
[2015/02/11 18:15:40 GMT] [INFO] (github.com/influxdb/influxdb/server.NewServer:43) Opening database at /opt/influxdb/shared/data/db
[2015/02/11 18:15:40 GMT] [INFO] (github.com/influxdb/influxdb/wal.NewWAL:40) Opening wal in /opt/influxdb/shared/data/wal
[2015/02/11 18:15:40 GMT] [INFO] (github.com/influxdb/influxdb/coordinator.NewRaftServer:79) Using /dev/urandom to initialize the raft server name
[2015/02/11 18:15:40 GMT] [INFO] (github.com/influxdb/influxdb/coordinator.NewRaftServer:104) Setting raft name to 3ccdfe35a0e7452e
[2015/02/11 18:15:40 GMT] [INFO] (github.com/influxdb/influxdb/api/http.(*HttpServer).EnableSsl:74) Ssl will be disabled since the ssl port or certificate path weren't set
[2015/02/11 18:15:40 GMT] [INFO] (github.com/influxdb/influxdb/coordinator.(*RaftServer).Serve:566) Initializing Raft HTTP server
[2015/02/11 18:15:40 GMT] [INFO] (github.com/influxdb/influxdb/coordinator.(*RaftServer).Serve:576) Raft Server Listening at 0.0.0.0:8090
[2015/02/11 18:15:40 GMT] [INFO] (github.com/influxdb/influxdb/coordinator.(*RaftServer).startRaft:384) Initializing Raft Server: http://box267:8090
[2015/02/11 18:15:40 GMT] [INFO] (github.com/influxdb/influxdb/coordinator.(*RaftServer).startRaft:416) Starting as new Raft leader...
[2015/02/11 18:15:40 GMT] [INFO] (github.com/influxdb/influxdb/coordinator.(*RaftServer).raftEventHandler:448) (raft:3ccdfe35a0e7452e) Selected as leader. Starting leader loop.
[2015/02/11 18:15:40 GMT] [INFO] (github.com/influxdb/influxdb/coordinator.(*InfluxJoinCommand).Apply:252) Adding new server to the cluster config 3ccdfe35a0e7452e
[2015/02/11 18:15:40 GMT] [INFO] (github.com/influxdb/influxdb/cluster.(*ClusterConfiguration).AddPotentialServer:291) Added server to cluster config: 1, http://box267:8090, box267:8099
[2015/02/11 18:15:40 GMT] [INFO] (github.com/influxdb/influxdb/cluster.(*ClusterConfiguration).AddPotentialServer:292) Checking whether this is the local server local: box267:8099, new: box267:8099
[2015/02/11 18:15:40 GMT] [INFO] (github.com/influxdb/influxdb/cluster.(*ClusterConfiguration).AddPotentialServer:301) Added the local server
[2015/02/11 18:15:40 GMT] [INFO] (github.com/influxdb/influxdb/server.(*Server).ListenAndServe:96) Waiting for local server to be added
[2015/02/11 18:15:40 GMT] [INFO] (github.com/influxdb/influxdb/wal.(*WAL).SetServerId:109) Setting server id to 1 and recovering
[2015/02/11 18:15:45 GMT] [INFO] (github.com/influxdb/influxdb/server.(*Server).ListenAndServe:128) Recovering from log...
[2015/02/11 18:15:45 GMT] [INFO] (github.com/influxdb/influxdb/cluster.NewWriteBuffer:30) local: Initializing write buffer with buffer size of 10000
[2015/02/11 18:15:45 GMT] [INFO] (github.com/influxdb/influxdb/cluster.(*ClusterConfiguration).RecoverFromWAL:1203) Waiting for servers to recover
[2015/02/11 18:15:45 GMT] [INFO] (github.com/influxdb/influxdb/coordinator.(*ProtobufServer).ListenAndServe:66) ProtobufServer listening on 0.0.0.0:8099
[2015/02/11 18:15:45 GMT] [INFO] (github.com/influxdb/influxdb/cluster.func·005:1185) Recovering local server
[2015/02/11 18:15:45 GMT] [INFO] (github.com/influxdb/influxdb/cluster.(*ClusterConfiguration).recover:1211) No shards to recover for 1
[2015/02/11 18:15:45 GMT] [INFO] (github.com/influxdb/influxdb/cluster.func·005:1187) Recovered local server
[2015/02/11 18:15:45 GMT] [INFO] (github.com/influxdb/influxdb/server.(*Server).ListenAndServe:133) recovered
[2015/02/11 18:15:45 GMT] [INFO] (github.com/influxdb/influxdb/coordinator.(*Coordinator).ConnectToProtobufServers:898) Connecting to other nodes in the cluster
[2015/02/11 18:15:45 GMT] [INFO] (github.com/influxdb/influxdb/server.(*Server).ListenAndServe:139) Starting admin interface on port 8083
[2015/02/11 18:15:45 GMT] [INFO] (github.com/influxdb/influxdb/server.(*Server).ListenAndServe:156) Graphite input plugins is disabled
[2015/02/11 18:15:45 GMT] [INFO] (github.com/influxdb/influxdb/server.(*Server).ListenAndServe:178) Collectd input plugins is disabled
[2015/02/11 18:15:45 GMT] [INFO] (github.com/influxdb/influxdb/server.(*Server).ListenAndServe:187) UDP server is disabled
[2015/02/11 18:15:45 GMT] [INFO] (github.com/influxdb/influxdb/server.(*Server).ListenAndServe:187) UDP server is disabled
[2015/02/11 18:15:45 GMT] [INFO] (github.com/influxdb/influxdb/server.(*Server).ListenAndServe:216) Starting Http Api server on port 8086
[2015/02/11 18:15:45 GMT] [INFO] (github.com/influxdb/influxdb/server.(*Server).reportStats:254) Reporting stats: &client.Series{Name:"reports", Columns:[]string{"os", "arch", "id", "version"}, Points:[][]interface {}{[]interface {}{"linux", "amd64", "3ccdfe35a0e7452e", "0.8.8"}}}

What’s interesting here is there is not a single log entry for the initial starting of InfluxDB and only the second start.

Any ideas why this is not starting the first time when invoked via a bash script automatically on a CircleCI build instance?

Thanks.