slim: Python Request Fails to download

I have created a testing repo to make it easy to find the reason - https://github.com/openedhardware/test_docker_slim

This repository is pretty simple… just calls an HTTP endpoint to download a static file in a thread! https://github.com/openedhardware/test_docker_slim/blob/master/main.py

But I get this error messages:

ubuntu@ubuntu-pc:~/test_docker_slim$ docker run --privileged -v /data:/data --rm --network host ts.slim
Starting Testing APP...
Downloading label file - https://raw.githubusercontent.com/pjreddie/darknet/master/data/coco.names
Exception in thread Thread-1:
Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/urllib3/connection.py", line 160, in _new_conn
    (self._dns_host, self.port), self.timeout, **extra_kw
  File "/usr/local/lib/python3.7/site-packages/urllib3/util/connection.py", line 61, in create_connection
    for res in socket.getaddrinfo(host, port, family, socket.SOCK_STREAM):
  File "/usr/local/lib/python3.7/socket.py", line 752, in getaddrinfo
    for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
OSError: [Errno 16] Device or resource busy

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/urllib3/connectionpool.py", line 677, in urlopen
    chunked=chunked,
  File "/usr/local/lib/python3.7/site-packages/urllib3/connectionpool.py", line 381, in _make_request
    self._validate_conn(conn)
  File "/usr/local/lib/python3.7/site-packages/urllib3/connectionpool.py", line 976, in _validate_conn
    conn.connect()
  File "/usr/local/lib/python3.7/site-packages/urllib3/connection.py", line 308, in connect
    conn = self._new_conn()
  File "/usr/local/lib/python3.7/site-packages/urllib3/connection.py", line 172, in _new_conn
    self, "Failed to establish a new connection: %s" % e
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPSConnection object at 0x7ff2865661d0>: Failed to establish a new connection: [Errno 16] Device or resource busy

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/requests/adapters.py", line 449, in send
    timeout=timeout
  File "/usr/local/lib/python3.7/site-packages/urllib3/connectionpool.py", line 725, in urlopen
    method, url, error=e, _pool=self, _stacktrace=sys.exc_info()[2]
  File "/usr/local/lib/python3.7/site-packages/urllib3/util/retry.py", line 439, in increment
    raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='raw.githubusercontent.com', port=443): Max retries exceeded with url: /pjreddie/darknet/master/data/coco.names (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7ff2865661d0>: Failed to establish a new connection: [Errno 16] Device or resource busy'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.7/threading.py", line 926, in _bootstrap_inner
    self.run()
  File "main.py", line 15, in run
    r = requests.get(COCO_LABELS_URL)
  File "/usr/local/lib/python3.7/site-packages/requests/api.py", line 76, in get
    return request('get', url, params=params, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/requests/api.py", line 61, in request
    return session.request(method=method, url=url, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/requests/sessions.py", line 530, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/local/lib/python3.7/site-packages/requests/sessions.py", line 643, in send
    r = adapter.send(request, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/requests/adapters.py", line 516, in send
    raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPSConnectionPool(host='raw.githubusercontent.com', port=443): Max retries exceeded with url: /pjreddie/darknet/master/data/coco.names (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7ff2865661d0>: Failed to establish a new connection: [Errno 16] Device or resource busy'))

Any idea what causes this weird issue?

Cheers.

About this issue

Original URL
State: closed
Created 4 years ago
Comments: 20 (8 by maintainers)

Most upvoted comments

Yes, it works! No error messages!

Just curious why $(pwd) is needed in the value of --mount flag?

Cheers!

$(pwd) is specific to my test environment. I had my data directory where I was running the commands (there’s no /data (no root data directory) on my test system). If you do have /data on your system then --mount /data:/data with docker-slim build might also work. It might still fail with Mac OS where Docker is doing extra magic to expose the local Mac directories to the Docker daemon. By default, Docker on Mac make /Users, /Volumes, /private and /tmp sharable/mountable (more about it here: https://docs.docker.com/docker-for-mac/#resources ).

Another option is to provide the local directory path explicitly with the --mount flag. I just used $(pwd) as a shortcut 😃

kcq on Jan 14, 2021

With this repro the docker-slim build command fails because the app in the container expects the /data directory to be there and it’s not because it’s not mounted in the temporary container.

Here’s the error snippet from the container logs:

Exception in thread Thread-1:
Traceback (most recent call last):
  File "/usr/local/lib/python3.7/threading.py", line 926, in _bootstrap_inner
    self.run()
  File "main.py", line 14, in run
    with open(os.path.join('/data/', 'labels.txt'), 'wb') as f:
FileNotFoundError: [Errno 2] No such file or directory: '/data/labels.txt'

Because the app in the container exits/terminates early a part part of the application doesn’t get to run, so the minified container image doesn’t include those parts and when you run docker run --privileged -v $(pwd)/data:/data --rm --network host ts.slim you end up with lots of failures.

When I changed the build command to this version (where I added the --mount flag) the application gets a chance to finish and the minified container images includes everything: docker-slim build --http-probe=false --mount $(pwd)/data:/data --include-path /usr/local/lib/python3.7/site-packages/certifi --show-clogs ts

Now running docker run --privileged -v /data:/data --rm --network host ts.slim doesn’t fail I get data/labels.txt that’s 625 bytes with the data the containerized app downloaded.

It’ll be good for docker-slim to detect the target application failures better to know if the minified container is any good.

kcq on Jan 13, 2021

I tried to repro it and I haven’t had much luck yet… My minified image works fine and it’s not failing. I’ll try adding code to it.

kcq on Aug 5, 2020