certmagic: Empty lockfile, likely previous process crashed or storage medium failure; treating as stale

Alright, gotcha gotcha.

(I decided to change the count to 8; that’s two seconds total.)

according to the VPS provider, the host provides SSD: 20 GB RAID-10 to my guest, which has debian 12 installed. i think it shouldn’t be considered slow.

Wow, well there must be some bottleneck, I wonder if it’s the RAID, or maybe just shared hardware in general.

according to the code here, it updates the traffic record file on every request? i am not the developer but rather a user, and i don’t know a thing about Go, all are my guess based on my shell script experience. But i could notice sync.Mutex to the author with an issue.

I didn’t realize you aren’t the author, so now I really extra-appreciate your willingness to help investigate the code. 😃 Yes, I would highly recommend just using sync.Mutex unless shared synchronization across Caddy instances is necessary. The locking mechanism we use is only designed for longer-running operations like certificate management.

it is a single machine running one instance of caddy on port 80 and 443 providing a trojan-protocol proxy and file server.

For in-process locking, definitely just use sync.Mutex.

Either way, glad we are able to have figured out a workaround, even if it’s not absolutely great. I’m still bewildered that the FS is letting readers read while it is writing the file.

Lock files created by the filesystem storage driver https://github.com/caddyserver/certmagic/blob/8728b186fa6874e93bbc051dc73d1596dd63d390/filestorage.go#L350 contain JSON which encodes the created and updated times of the files. This is done because not all filesystems support keeping track of both those timestamps, so storing it in the file is a more reliable way to ensure we can track how long a lock has been open.

@mholt

thanks for all the help!

1. change the delay interval to 250ms and set the retry count to 6 , i bet this is even better
2. according to the VPS provider, the host provides SSD: 20 GB RAID-10 to my guest, which has debian 12 installed. i think it shouldn’t be considered slow.
3. according to the code here, it updates the traffic record file on every request? i am not the developer but rather a user, and i don’t know a thing about Go, all are my guess based on my shell script experience. But i could notice sync.Mutex to the author with an issue.
4. it is a single machine running one instance of caddy on port 80 and 443 providing a trojan-protocol proxy and file server.

let me do some experiment by changing if emptyCount < 2 to if emptyCount < 10, i’ll be right back.