caddy: acme_server: error on reload

I started with this config:

{
  "apps": {
    "http": {
      "servers": {
        "srv0": {
          "listen": [
            ":80"
          ],
          "routes": [
            {
              "match": [
                {
                  "host": [
                    "localhost"
                  ]
                }
              ],
              "handle": [
                {
                  "handler": "acme_server"
                }
              ]
            }
          ]
        }
      }
    }
  }
}

To which I only added logging:

{
  "logging": {
    "logs": {
      "log0": {
        "writer": {
          "output": "stderr"
        },
        "encoder": {
          "format": "json"
        }
      }
    }
  },
  "apps": {
    "http": {
      "servers": {
        "srv0": {
          "logs": {
            "logger_names": {
              "localhost": "log0"
            }
          },
          "listen": [
            ":80"
          ],
          "routes": [
            {
              "match": [
                {
                  "host": [
                    "localhost"
                  ]
                }
              ],
              "handle": [
                {
                  "handler": "acme_server"
                }
              ]
            }
          ]
        }
      }
    }
  }
}

Ran caddy reload --config caddy.json and got the error:

ERROR	admin.api	request error	{"error": "loading config: loading new config: loading http app module: provision http: server srv0: setting up route handlers: route 0: loading handler modules: position 0: loading module 'acme_server': provision http.handlers.acme_server: initializing certificate authority: Error opening database of Type badger with source /Users/mohammed/Library/Application Support/Caddy/acme_server/db: error opening Badger database: Cannot acquire directory lock on \"/Users/mohammed/Library/Application Support/Caddy/acme_server/db\".  Another process is using this Badger database.: resource temporarily unavailable", "status_code": 400}

reload: sending configuration to instance: caddy responded with error: HTTP 400: {"error":"loading config: loading new config: loading http app module: provision http: server srv0: setting up route handlers: route 0: loading handler modules: position 0: loading module 'acme_server': provision http.handlers.acme_server: initializing certificate authority: Error opening database of Type badger with source /Users/mohammed/Library/Application Support/Caddy/acme_server/db: error opening Badger database: Cannot acquire directory lock on \"/Users/mohammed/Library/Application Support/Caddy/acme_server/db\".  Another process is using this Badger database.: resource temporarily unavailable"}

2020/11/13 21:22:08.725	INFO	admin	stopped previous server

Note the stopped previous server. I haven’t checked whether the acme endpoint still works.

About this issue

  • Original URL
  • State: closed
  • Created 4 years ago
  • Comments: 16 (14 by maintainers)

Most upvoted comments

I think that rolling with BoltDB is totally fine as long as it’s working.

We’ve had to make a number of code changes to export various badger db configuration to allow compatibility for a wider array of operating systems / platforms. Hopefully using BoltDB helps to avoid that. We haven’t heard any issues with BoltDB (although that’s probably because users are defaulted to Badger 😉).

Thanks @dopey.

The fix to this bug in particular is not dependent on Badger vs BoltDB, but instead managing db instances properly.

Switching to BoltDB was because of this issue: https://github.com/caddyserver/caddy/issues/3847.

The issue with BadgerV1 was actually due to the in memory database. The issue was resolved by switching to either BadgerV2 or BoltDB. I actually had a branch for either and we decided on BoltDB as we were going to break backwards compatibility anyway. It would not be hard to switch this to a BadgerV2 database.

Sorry, late to the discussion. Boltdb is a good option if it fixes the issues folks were having.

We chose Badger as the default due to the larger and more recently engaged community.

The reason that we don’t default Badger to v2 is that it’s not backwards compatible. People who ran the old version would have to migrate their DB. We just opened an issue in our db integration layer (https://github.com/smallstep/nosql/issues/11) for defaulting to the right DB if one already exists, otherwise generating a new one. Sounds like this wouldn’t have fixed your issue on it’s own because you’d also have to set some DB configuration values.

Apparently Badger now has an in-memory mode, not sure how far along that is. @maraino did some quick sleuthing this morning.

Just pushed a branch that apparently resolves this issue.

Actually… I think I found there is an upstream for specifying an initialized database. I’ll start with that. Having multiple databases may still be necessary if running multiple authorities anyway.

Yea. That’s exactly what I was thinking. Although… is it possible to run multiple ACME servers with multiple CAs with Caddy? If so, it may actually be best to initialize the database based on the CA name provided anyway. That would resolve this issue from the Caddy side. If not, then I agree it makes sense to fix upstream.