salt: file.managed with proxy broken in 2015.5

Hey all, I have a state that looks something like this:

unzip:
  pkg.installed

http_proxy:
  environ.setenv:
    - value: 'http://proxy-url'

https_proxy:
  environ.setenv:
    - value: 'https://proxy-url'

/usr/local/bin/consul_dir:
  archive.extracted:
    - source: https://dl.bintray.com/mitchellh/consul/0.5.0_linux_amd64.zip
    - archive_format: zip
    - source_hash: sha256=161f2a8803e31550bd92a00e95a3a517aa949714c19d3124c46e56cfdc97b088
    - require:
        - environ: http_proxy
        - environ: https_proxy
        - pkg: unzip

where proxy-url is a standin for the actual url of my http proxy. With a 2014.7 minion the archive extracted fine, with a 2015.5 minion, I get:

----------
          ID: /usr/local/bin/consul_dir
    Function: archive.extracted
      Result: False
     Comment: Unable to manage file: [Errno 111] Connection refused
     Started: 17:58:15.121916
    Duration: 420.282 ms
     Changes:
----------

Debugging with salt-call, this appears to be happening here, it looks like the new utils.http doesn’t use the http_proxy and https_proxy environment variables, whereas whatever was responsible for the http calls in 2014.7 did.

About this issue

Original URL
State: closed
Created 9 years ago
Reactions: 3
Comments: 21 (10 by maintainers)

Most upvoted comments

@jfindlay, @tingvold

Same issues here with managed.file & archive.extracted behind a proxy, it does indeed work if we set that proxy_host and proxy_port settings on the minion.

But actually we have lots of different environments, some of them have proxies, some not, some have different ones. We currently control these settings by salt formulas which are setting up the common http_proxy, https_proxy, no_proxy environment variables.

Having this additional/odd piece of configuration needed for the minions in environments with proxies really feels wrong. Especially as it is only possible to set them via file and the minion has to get restarted to respect them (so not even creating them on the fly on hosts with proxies through salt states is possible).

Why going into this completely strange direction of ignoring the existing env and widely spread standards in handling these settings?

peter-fe on Apr 18, 2020

It would be nice that salt honors properly all the protocol_proxy environment variables in salt-minion with all the backends it uses:

http_proxy
https_proxy
ftp_proxy and possibly other protocols supported by salt.

It seems that most of this can be fixed in salt.fileclient or salt.utils.{http,s3,…}

nlm on Dec 1, 2015

Sorry for reopening a closed thread, but the answer to this is not clear in the thread so I thought I’d provide it.

If you’re using the default tornado backend for http queries, then you need to configure the salt-minion to use the proxy.

It’s documented here: https://docs.saltproject.io/en/latest/topics/tutorials/http.html#proxy

on Linux create this config: /etc/salt/minion.d/tornado.conf

Then add your proxy details

proxy_host: xxx.xxx.xxx.xxx  
proxy_port: 3128 
proxy_username: proxyuser
proxy_password: P@ssword1!

Obviously change the values to what you need.

intentropy on May 19, 2022

No, the idea is you specify in the config a backend. Of that you have three choices:

requests; this backend should listen to the env vars we all know (ftp/http(s)_proxy, IIRC you need https_proxy set for HTTPS to be proxied)
urrlib2: No clue, haven’t used this one
The default, Tornado uses config options proxy_* to determine proxy usage, seemingly irrespective of proto.

Both backends need PyCuRL IIRC to do the proxying.

The-Loeki on Nov 4, 2016

See #29322; the new (default) Tornado backend should be able to use PycURL, which is proxy-capable.

The-Loeki on Oct 18, 2016