process-compose: Weird interpolation with environment: script not working

I have the following yml file which I feed into process-compose up:

version: "0.5"

environment:
  - "I_AM_GLOBAL_EV=5"

processes:
  process1:
    description: This process will sleep for 2 seconds
    command: "echo $I_AM_GLOBAL_EV > /tmp/output"

This leads to a /tmp/output file being generated - but it’s empty. Why is that? If I execute env and redirect it, I see the variable.

About this issue

  • Original URL
  • State: closed
  • Created 3 years ago
  • Comments: 19 (15 by maintainers)

Commits related to this issue

Most upvoted comments

Fixed in v1.6.1

i mean fwiw, the process-compose.yaml file is versioned, so all that’s needed is a config file version bump. backwards compatibility is already solved here, the default behavior can be different in config file version 0.6

that said, my 2c is that templating (imho) is indeed unnecessary, and probably better served with an opt-in. ideally users handle env vars directly in their process-compose configs, where templating is already available to them in shell utilities

I’d also love an option to disable env substitution globally which could be done via cli flag to avoid double parsing.

If it was up to me, I’d disable substitution by default and have it enabled with cli flag. Given how many people get tripped up on this I’d say having it opt in would be better since it someone opting into it would first need to read about it and (hopefully) understand how it works.

This would require a major version bump, but so would adding the escaping mechanism so maybe it’s a good opportunity to do both at once.

Agreed! Hence my suggestion to add the option to disable interpolation via the yaml file first (opt out, no breakage, backward compatible). I would then update devenv process-compose YAML generation to include it.

This would immediately fix the surprise of the command: not behaving the same way as entering the same command in the given shell (bash or Powershell) for the majority of the users. In my own cases, this would eliminate the need to generate wrapper shell scripts in several processes.

Then after a while the default behavior can be changed in a major release. This is the slower/safer strategy, that allows people to migrate over. The primary concern is that many devenv users use the unstable channel of nixpkgs, so doing a breaking change will not allow dependents of process-compose to prepare for the new behavior.

I’d also love an option to disable env substitution globally which could be done via cli flag to avoid double parsing.

If it was up to me, I’d disable substitution by default and have it enabled with cli flag. Given how many people get tripped up on this I’d say having it opt in would be better since it someone opting into it would first need to read about it and (hopefully) understand how it works.

This would require a major version bump, but so would adding the escaping mechanism so maybe it’s a good opportunity to do both at once.

Will it help if I add the ability to escape env variables with $$?

foo:
  command: "echo log is $$ENV_TEST"
  environment:
    - 'ENV_TEST=ready'

Output: log is ready

That’s a very good point @cdmistman…

I wanted to avoid the double parsing, but I guess it’s a small price to pay for improved usability (sharing configuration). New plan:

  1. Global boolean flag in process-compose.yaml.
  2. Escaping with $$.
  3. Proper documentation.

Thoughts?

Will it help if I add the ability to escape env variables with $$?

that would help, yes - “add a $ to escape $ substitution” is pretty consistent though it does have the unfortunate side effect of $$$$

That’s what Google cloudbuild does for their yaml manifests

I’d also love an option to disable env substitution globally which could be done via cli flag to avoid double parsing.

Thanks for the tool, I find it really useful!

However it is not clear from the explanation (from @F1bonacc1 ) above what is the purpose of environment then. I expect example from the description to produce 5 in /tmp/output. Similar to tmuxp tool . This is really useful feature for many applications.

Thanks, Roman

Stuff you set in environment is present in env during runtime. You can test it by calling env in you script, or by putting your script into an executable file instead of inlining it. The confusing part is the PC does $ variable substitution within the config file at load time so the vars in your inline script are replaced with their values from env (or empty if they are not set) before the script is even ran. The worst part is that there is no way to escape it, which handicaps the usability of inline scripts in PC config files. Hence I never do that and always put scripts in their own files.