woodpecker: Server crashes and after a restart SQL lite db is locked
Component
server
Describe the bug
During the execution of a pipeline, often times the server crashes. After crash, when the server is restarted the server is in a very slow state, often times reporting that “db is locked”
When the server crashes the following info is outputted to the logs:
goroutine 33930 [running]: net/http.(http2responseWriter).write(0xc0017c6780?, 0xc00111fea8?, {0x0?, 0x12634a0?, 0x1362a80?}, {0x141f1bf?, 0xc000074d00?}) /usr/local/go/src/net/http/h2_bundle.go:6797 +0x13f net/http.(http2responseWriter).WriteString(0xc00111fee0?, {0x141f1bf?, 0xc000074d00?}) /usr/local/go/src/net/http/h2_bundle.go:6790 +0x28 io.WriteString({0x7f428c48e2b8, 0xc000074d00}, {0x141f1bf, 0x18}) /usr/local/go/src/io/io.go:316 +0x58 github.com/gin-gonic/gin.(*responseWriter).WriteString(0xc00154a000, {0x141f1bf, 0x18}) /woodpecker/src/github.com/woodpecker-ci/woodpecker/vendor/github.com/gin-gonic/gin/response_writer.go:90 +0x73 io.WriteString({0x7f428c48e278, 0xc00154a000}, {0x141f1bf, 0x18}) /usr/local/go/src/io/io.go:316 +0x58 go.woodpecker-ci.org/woodpecker/server/api.LogStreamSSE.func2() /woodpecker/src/github.com/woodpecker-ci/woodpecker/server/api/stream.go:224 +0x134 created by go.woodpecker-ci.org/woodpecker/server/api.LogStreamSSE in goroutine 33971 /woodpecker/src/github.com/woodpecker-ci/woodpecker/server/api/stream.go:208 +0x7a5
System Info
{"source":"https://github.com/woodpecker-ci/woodpecker","version":"2.0.0-rc.0"}
Additional context
No response
Validations
- Read the Contributing Guidelines.
- Read the docs.
- Check that there isn’t already an issue that reports the same bug to avoid creating a duplicate.
- Checked that the bug isn’t fixed in the
nextversion already [https://woodpecker-ci.org/faq#which-version-of-woodpecker-should-i-use] - Check that this is a concrete bug. For Q&A join our Discord Chat Server or the Matrix room.
About this issue
- Original URL
- State: closed
- Created 8 months ago
- Comments: 17 (8 by maintainers)
looks like an issue in cron handler
I’ll create a pull that add the option to disables cron, if it go away we know it’s this code part and can fix it
Just another update, I captured the logs when db locking presents itself. You can see that the db seems infinetly locked and so everything stops working
hmm the easyest way would be to use a framework like https://codeberg.org/6543/xorm_context but mentioned framework needs bit of testing …