dask-gateway: "HTTPClientError: HTTP 403: Use Proxy Server" due to local HTTP_PROXY?

Dask-gateway feels like a great fit for a project I’m involved in, and I’m excited to try it. Thank you!

I’ve followed the Kubernetes + Helm instructions to install the server out in the wild, and I’m getting a HTTPClientError: HTTP 403: Use Proxy Server when I try to connect to the public gateway from my local machine. I can instantiate the gateway just fine, but I can’t .list_clusters() or .new_cluster() without the error above.

I’m behind a corporate web proxy that I usually handle with an HTTP_PROXY environmental variable. Could this be related to my error? If so, is there a way for me to handle a local web proxy so that I can connect my local gateway client to the external public gateway?

Thanks again.

About this issue

Original URL
State: closed
Created 4 years ago
Comments: 20 (10 by maintainers)

Most upvoted comments

That’s specific to the dask-kubernetes default images, and doesn’t work with dask-gateway. I recommend building your own docker images with whatever extra packages you may need, or create a new issue where we can discuss further options.

jcrist on Jan 9, 2020

Works! I was actually using a config file, but a had dummy instead of basic as the auth type. Must’ve copied it incorrectly from the server config file. Thank you again.

jkmacc-LANL on Jan 9, 2020

In the environment you’re running this in you do have HTTP_PROXY set?

Yes. os.environ['HTTP_PROXY'] shows my proxy.

Same script as above, but swap the .configure command for…

That did it:

Out[4]: b'{}'

jkmacc-LANL on Jan 8, 2020

This is likely related to #103, the http client we use in the Gateway client doesn’t respect HTTP_PROXY. You may be able to hack around this by configuring tornado to use CurlAsyncHTTPClient yourself (note that you’ll need pycurl installed).

import dask_gateway

# Use the curl client instead
from tornado.httpclient import AsyncHTTPClient
AsyncHTTPClient.configure("tornado.curl_httpclient.CurlAsyncHTTPClient")

gateway = dask_gateway.Gateway(...)
...

Let me know if this works.

jcrist on Jan 8, 2020