jc: Data missing when parsing Let's encrypt certbot ini files

certbot generates INI files with paths like /etc/letsencrypt/renewal/example.org.conf and content like this:

# renew_before_expiry = 30 days
version = 1.21.0
archive_dir = /etc/letsencrypt/archive/example.org
cert = /etc/letsencrypt/live/example.org/cert.pem
privkey = /etc/letsencrypt/live/example.org/privkey.pem
chain = /etc/letsencrypt/live/example.org/chain.pem
fullchain = /etc/letsencrypt/live/example.org/fullchain.pem

# Options used in the renewal process
[renewalparams]
account = PkrrjRfryfuLvsg5oGiQ
authenticator = webroot
webroot_path = /var/www/html,
server = https://acme-v02.api.letsencrypt.org/directory
renew_hook = /etc/letsencrypt/renewal-hooks/deploy/certbot-restart-services
[[webroot_map]]
test.example.org = /var/www/html

As far as I can see these are valid INI files but JC only outputs the first section of the file:

cat /etc/letsencrypt/renewal/example.org.conf | jc --ini -py
---
version: 1.21.0
archive_dir: /etc/letsencrypt/archive/example.org
cert: /etc/letsencrypt/live/example.org/cert.pem
privkey: /etc/letsencrypt/live/example.org/privkey.pem
chain: /etc/letsencrypt/live/example.org/chain.pem
fullchain: /etc/letsencrypt/live/example.org/fullchain.pem

Am I missing something or is this a bug or a feature?

About this issue

  • Original URL
  • State: closed
  • Created a year ago
  • Comments: 26 (26 by maintainers)

Commits related to this issue

Most upvoted comments

Added in v1.22.5.

Yep, it makes sense to do that when there is a well-defined schema, like a systemd configuration file, but jc needs to be more generic since there is no schema for any type of INI file it has to process. By keeping the types the same, it reduces confusion because otherwise you will only know what the type is by experimentation.

With uniform types you should always be able to do any of the following and get a value without failure:

  • dict[key][0] (first or only item in many tools)
  • dict[key][-1] (last or only item in many tools)
  • dict.key[] (only or all items in jq syntax)
  • dict[key].items() (only or all items in ansible?)

If jc changes the type based on the number of items you will get failures for certain keys if you only use dict[key]. Also, dict[key][0] means “grab the first character” in a string vs. “grab the first item” in a list, so it can provide unexpected results.

I believe jc should be able to support the multi-line values as described in the systemd docs. The only limitation I’ve found so far is blank-line values are not supported e.g.:

This won’t work:

[section]
foo = this is a line

    this is a continuation of the first line

This should work:

[section]
foo = this is a line \
    \
    this is a continuation of the first line

Let me think about this. It just seems weird to change a value’s type instead of doing something like if len(dict[key]) > 1 so all values can be the same type. I could also change the behavior based on the raw option, but that is already doing quote removal so that would overload that option with multiple changes.

Ok, I just removed the special [DEFAULT] section handling behavior, so the INI parser should be pretty solid now.

Yeah it’s behavior inherited from the python configparser library. I think it’s a convenience feature for setting default values in subsequent sections but I don’t think it’s relevant to an INI parser. I’ll figure out how to disable it or use the top-level trick to get rid of it. Thanks for your help on this!

I’ve just tested it and it’s perfect!

Yep, I’ll look into that. Might have to separate the INI and KV parsers to make that happen, but that shouldn’t be an issue. I’ll take a look at how they did it in Ansible.

Not sure if this is a python-specific thing, but I guess there is no “real” INI standard. Python doesn’t think this is a valid INI file (more precisely a valid Config file) because it is missing the first section header. This is the exception I get when running it through the stock configparser library:

configparser.MissingSectionHeaderError: File contains no section headers.

The reason it takes the first values in jc is because I have modified the INI parser to create a fake section header if one doesn’t exist, which essentially turns it into the --kv parser. (the --kv parser just points to the --ini parser)

A workaround for this might be to just add a fake section header to the file before parsing with jc?