jc: Data missing when parsing Let's encrypt certbot ini files
certbot generates INI files with paths like /etc/letsencrypt/renewal/example.org.conf and content like this:
# renew_before_expiry = 30 days
version = 1.21.0
archive_dir = /etc/letsencrypt/archive/example.org
cert = /etc/letsencrypt/live/example.org/cert.pem
privkey = /etc/letsencrypt/live/example.org/privkey.pem
chain = /etc/letsencrypt/live/example.org/chain.pem
fullchain = /etc/letsencrypt/live/example.org/fullchain.pem
# Options used in the renewal process
[renewalparams]
account = PkrrjRfryfuLvsg5oGiQ
authenticator = webroot
webroot_path = /var/www/html,
server = https://acme-v02.api.letsencrypt.org/directory
renew_hook = /etc/letsencrypt/renewal-hooks/deploy/certbot-restart-services
[[webroot_map]]
test.example.org = /var/www/html
As far as I can see these are valid INI files but JC only outputs the first section of the file:
cat /etc/letsencrypt/renewal/example.org.conf | jc --ini -py
---
version: 1.21.0
archive_dir: /etc/letsencrypt/archive/example.org
cert: /etc/letsencrypt/live/example.org/cert.pem
privkey: /etc/letsencrypt/live/example.org/privkey.pem
chain: /etc/letsencrypt/live/example.org/chain.pem
fullchain: /etc/letsencrypt/live/example.org/fullchain.pem
Am I missing something or is this a bug or a feature?
About this issue
- Original URL
- State: closed
- Created a year ago
- Comments: 26 (26 by maintainers)
Commits related to this issue
- WIP https://github.com/kellyjonbrazil/jc/issues/344#issuecomment-1375897620 — committed to webarch-coop/ansible-role-systemd by chriscroome a year ago
Added in v1.22.5.
Yep, it makes sense to do that when there is a well-defined schema, like a
systemdconfiguration file, butjcneeds to be more generic since there is no schema for any type of INI file it has to process. By keeping the types the same, it reduces confusion because otherwise you will only know what the type is by experimentation.With uniform types you should always be able to do any of the following and get a value without failure:
dict[key][0](first or only item in many tools)dict[key][-1](last or only item in many tools)dict.key[](only or all items injqsyntax)dict[key].items()(only or all items in ansible?)If
jcchanges the type based on the number of items you will get failures for certain keys if you only usedict[key]. Also,dict[key][0]means “grab the first character” in a string vs. “grab the first item” in a list, so it can provide unexpected results.I believe
jcshould be able to support the multi-line values as described in thesystemddocs. The only limitation I’ve found so far is blank-line values are not supported e.g.:This won’t work:
This should work:
Let me think about this. It just seems weird to change a value’s type instead of doing something like
if len(dict[key]) > 1so all values can be the same type. I could also change the behavior based on therawoption, but that is already doing quote removal so that would overload that option with multiple changes.Ok, I just removed the special
[DEFAULT]section handling behavior, so the INI parser should be pretty solid now.Yeah it’s behavior inherited from the python configparser library. I think it’s a convenience feature for setting default values in subsequent sections but I don’t think it’s relevant to an INI parser. I’ll figure out how to disable it or use the top-level trick to get rid of it. Thanks for your help on this!
I’ve just tested it and it’s perfect!
Yep, I’ll look into that. Might have to separate the INI and KV parsers to make that happen, but that shouldn’t be an issue. I’ll take a look at how they did it in Ansible.
Not sure if this is a python-specific thing, but I guess there is no “real” INI standard. Python doesn’t think this is a valid INI file (more precisely a valid Config file) because it is missing the first section header. This is the exception I get when running it through the stock
configparserlibrary:The reason it takes the first values in
jcis because I have modified the INI parser to create a fake section header if one doesn’t exist, which essentially turns it into the--kvparser. (the--kvparser just points to the--iniparser)A workaround for this might be to just add a fake section header to the file before parsing with
jc?