cargo: Cargo doesn't uniformly read configuration from env vars

Most configuration read through cargo uses helpers like Config::get_string which automatically read env vars, but configuration reading portions of Cargo that go through Config::get_table do not read environment variables. The return value here is a HashMap which doesn’t proxy reads to an environment variable.

This affects, for example, source replacement. Source replacement is not configurable through environment variables (a bug) because of its use of get_table.

We should tackle this via one of two routes:

First, remove get_table entirely. Just don’t expose the ability to read a table and force all configuration reads to go through get_string and friends. This may or may not be uniformly possible as IIRC there’s portions of Cargo that rely on iter which is a bit of a bummer.
Second, change get_table to return a wrapper rather than a HashMap. Reads of the HashMap would “do the right thing” with environment variables. Again though it’s not clear how such a wrapper would implement iter as well…

In any case, some questions to still work through here!

About this issue

Original URL
State: open
Created 6 years ago
Reactions: 4
Comments: 15 (11 by maintainers)

Commits related to this issue

Auto merge of #7649 - ehuss:config2, r=alexcrichton Config enhancements. This is a collection of changes to config handling. I intended to split this into separate PRs, but they all built on one ano... — committed to rust-lang/cargo by bors 5 years ago

Most upvoted comments

I wonder if we need to provide a giant escape hatch like CARGO_CFG_TOML or CARGO_CFG_JSON, which just accepts a toml/json string with the contest of .cargo/config?

The benefit of the current setup is that a human can easily override config on the command line. However looks like to specify tables the syntax would need to be ugly and not human friendly anyway, so perhaps just providing a giant TOML escape hatch would work better for those who need it?

matklad on May 11, 2018

I have a prototype that auto-converts with serde here: https://github.com/ehuss/cargo/commit/63f76b0d24bc045809a7a0d4fceea6c9bd176390

Essentially you can do something like let p: TomlProfile = config.load_type("profile.dev") and it will auto-magically convert the config (including environment variables). It supports structs, arbitrary maps, optional values, etc.

If you’d like, I can continue working on this and submit a PR (just the serde part, no actual changes to Cargo, yet). I realize it is a lot of code, so maybe it’s not worth it (I’m no serde expert, so some parts can probably be simpler). However, I think it would be helpful for the Profile work I’m doing, and could be used for the source map, and anything else in the future. Overall I’m not sure how useful environment variables are.

Things left to do:

Clean up and finish unimplemented types. Possibly use macros to reduce the bloat.
Better error handling. Include the filename of any bad configs.
Write real tests.

Possible enhancements:

Support TOML in environment values. I think this would be possible to support for arbitrary keys (CARGO_PROFILE_DEV="..."). It could deduce that it needs a rich data type and parse as-needed.
Support lists. Not sure what the preferred way to handle env vars would be. Perhaps just TOML-style lists?
Underscores aren’t supported for environment variable map keys (but they are supported for structs like OPT_LEVEL). So you can’t specify a package with a name like ansi_term. Not sure if there is any way around that.

ehuss on May 15, 2018

@rtsuk sounds plausible to me! In general I’d be totally on board with basically reworking how [source] configuration is specified so it’s more amenable to reading it through environment variables.

Another possibility would be something like CARGO_SOURCES_ALL=comma,separated,list,of,names and that could inform Cargo which ones to read from env vars perhaps?

alexcrichton on Apr 30, 2018