shuttle: [Bug]: database is locked

What happened?

Regularly reading this error while changing nothing and just rerunning the deployment:

error returned from database: (code: 5) database is locked

Version

v0.24.0

Which operating system(s) are you seeing the problem on?

In deployment

Which CPU architectures are you seeing the problem on?

In deployment

Relevant log output

cargo shuttle logs *****

 INFO Entering queued state
TRACE shuttle_deployer::deployment::queue: getting a build slot
TRACE {uri="http://gateway:8001/stats/load"} shuttle_deployer::deployment::gateway_client: calling gateway
DEBUG hyper::client::connect::http: connecting to 10.99.0.14:8001
DEBUG hyper::client::connect::http: connected to 10.99.0.14:8001
DEBUG hyper::client::pool: pooling idle connection for ("http", gateway:8001)
TRACE {response="Response { status: 200, version: HTTP/1.1, headers: {\"content-type\": \"application/json\", \"content-length\": \"38\", \"date\": \"Tue, 05 Sep 2023 21:59:58 GMT\"}, body: Body(Streaming) }"} shuttle_deployer::deployment::gateway_client: Load response

 INFO Entering building state
 INFO     Finished release [optimized] target(s) in 0.17s

 INFO Entering crashed state
ERROR {error="Build error: Database error: error returned from database: (code: 5) database is locked"} shuttle_deployer::deployment::queue: service build encountered an error

Duplicate declaration

I have searched the issues and there are none like this.

About this issue

Original URL
State: open
Created 10 months ago
Comments: 16 (9 by maintainers)

Most upvoted comments

Ah, okay, thanks for the info! It does seem like this error can occur when having an active deployment and one building too, which is unexpected. We’re about to make a change in the coming week that will stop writing logs to the deployer sqlite database, this should alleviate this issue since logs are the majority of writes to the DB. In the slightly longer term we aim to completely remove this DB from deployers (the container your service runs in), and of course support interuptible builds as stated in #1067.

oddgrd on Sep 8, 2023

We have continued investigating this, and we made some improvements in 0.39.0 that we hope help reduce the frequency of this bug.

jonaro00 on Feb 14, 2024

2024-01-03T03:55:09.866+05:30 [Deployer]  INFO Starting service
2024-01-03T03:55:14.870+05:30 [Deployer]  INFO Entering Crashed state
2024-01-03T03:55:14.870+05:30 [Deployer]  INFO Cleaning up crashed deployment
2024-01-03T03:55:14.870+05:30 [Deployer] ERROR shuttle_deployer::deployment::run: {error="Run error: errored while setting the new deployer address: Database error: error returned from database: (code: 5) database is locked"} Service encountered an error and crashed

Any idea how can I fix this ?

lazyfuhrer on Jan 2, 2024

Although this bug is more rare now, it can still happen.

jonaro00 on Nov 22, 2023