SWE-agent: "Runtime did not start within timeout" when running swe-agent within docker

Describe the bug

Using the 8000 container port in the docker run command seems to always trigger ERROR: [Errno 98] error while attempting to bind on address ('0.0.0.0', 8000): address already in use.

However, attempts to resolve this by specifying an alternate container port (which seems to work in a standalone Docker command) triggers the timeout error also encountered in #887. I’m working on a cluster that uses Podman under the hood for all Docker commands. I was wondering if there’s another way to fix the ‘address already in use’ error or a way to successfully use an alternate container port?

Steps/commands/code to Reproduce

My workflow is on a cluster environment with Podman emulating Docker. The Dockerfile I used to build the sweagent:tiny image used in the steps below is the one here (reproduced below)

FROM python:3.11.10-bullseye  

ARG DEBIAN_FRONTEND=noninteractive  
ENV TZ=Etc/UTC

WORKDIR /

RUN apt-get update && apt-get install -y dbus-user-session

# Install swe-rex for faster startup
RUN pip install pipx
RUN pipx install swe-rex
RUN pipx ensurepath
ENV PATH="$PATH:/root/.local/bin/"

SHELL ["/bin/bash", "-c"]

Standard Run Command with Port 8000

When I run the following command, based on the “solving coding challenges tutorial”:  sweagent run --config=config/coding_challenge.yaml --problem_statement.path=lc_test/prob.md --env.repo.path=lc_test/ --env.deployment.image=sweagent:tiny --agent.model.name=claude-3.5-sonnet --agent.model.per_instance_cost_limit=0.1

it results in the Docker run call below

🦖 DEBUG    Command: "docker run --rm -p 60769:8000 --name sweagenttiny-6xxx9 sweagent:tiny /bin/sh -c 'swerex-remote --auth-token XXX || (python3 -m pip install pipx && python3 -m pipx ensurepath && pipx run swe-rex --auth-token  XXX)'" 
🦖 INFO     Starting runtime at 60769   

and triggers the following error:

Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg.
INFO:     Started server process [3295635]
INFO:     Waiting for application startup.
INFO:     Application startup complete.
ERROR:    [Errno 98] error while attempting to bind on address ('0.0.0.0', 8000): address already in use
INFO:     Waiting for application shutdown.
INFO:     Application shutdown complete.

With a different container port

I tried variants of the Docker command alone, and noticed that changing the container port sometimes fixes the “Address already in Use” issue:

docker run --rm -p 50842:8999 sweagent:tiny /bin/sh -c 'swerex-remote --auth-token XXX --host 0.0.0.0 --port=8999'
Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg.
INFO:     Started server process [3296101]
INFO:     Waiting for application startup.
INFO:     Application startup complete.
INFO:     Uvicorn running on http://0.0.0.0:8999 (Press CTRL+C to quit)

Since the SWE-rex / SWE-Agent codebase seems to hardcode the container port as 8000, I tried modifying it so it can accept a container port that I specify (as in the Docker command above). Now, sweagent run --config=config/coding_challenge.yaml --problem_statement.path=lc_test/prob.md    --env.repo.path=lc_test/ --env.deployment.image=sweagent:tiny --env.deployment.container_port=8896 --agent.model.name=claude-3.5-sonnet --agent.model.per_instance_cost_limit=0.1

however, seems to trigger the issue in #887

🦖 DEBUG  Command: "docker run --rm -p 42347:8896 --name sweagenttiny-6XXXf sweagent:tiny  /bin/sh -c 'swerex-remote --auth-token  XXX --host 0.0.0.0 --port=8896'"
🦖 INFO   Starting runtime at 42347
🦖 ERROR    Runtime did not start within timeout. Here's the output from the container process.                                              

Error message/results

Added inline with the two commands above, for continuity.

System Information

Rocky Linux release 8.9, Python 3.11.10, SWE-agent version 1.0.0 (SWE-Rex 1.1.1). The system uses Podman to emulate Docker.

Checklist

  • I’m running with the latest docker container/on the latest development version (i.e., I ran git pull))
  • I have copied the full command/code that I ran (as text, not as screenshot!)
  • If applicable: I have copied the full log file/error message that was the result (as text, not as screenshot!)
  • I have enclosed code/log messages in triple backticks (docs) and clicked “Preview” to make sure it’s displayed correctly.

About this issue

  • Original URL
  • State: closed
  • Created 2 years ago
  • Comments: 15 (8 by maintainers)

Most upvoted comments

Hello @klieret I resolved this by changing the network mode of the task definition to ecs.NetworkMode.HOST, thank you for debugging the problem.

OK, at least in this setup, I think I understand what’s happening. Because it’s not a docker-in-docker scenario but rather swe-agent within the container calls the outer docker, in the end, it binds it’s environment/swe-rex runtime to port 58411 of the outer container. For example, in my example, I can see from outside of the container:

> docker ps
CONTAINER ID   IMAGE            COMMAND                  CREATED          STATUS          PORTS                     NAMES
f1bf81342a8d   57f62bd48dad     "/bin/sh -c '/root/p…"   46 seconds ago   Up 45 seconds   0.0.0.0:32797->8000/tcp   python3.11-e45e929e-d424-4d78-8179-2d39d15daa59

and

❯ lsof -i :32797
COMMAND     PID   USER   FD   TYPE             DEVICE SIZE/OFF NODE NAME
com.docke 11739 fuchur   97u  IPv6 0x24d19ca3ae777796      0t0  TCP *:32797 (LISTEN)

but

root@eaf1de41fc0e:/app# lsof -i :32797
root@eaf1de41fc0e:/app#

shows how we don’t have access to it from within the container.

OK, I can reproduce this. I take the dockerfile and do

docker build -t sweagentbundle .
docker run -v /var/run/docker.sock:/var/run/docker.sock -v /usr/bin/docker:/usr/bin/docker -it sweagentbundle /bin/bash

then from inside the container I run the hello world example:

sweagent run \
  --agent.model.name=claude-3-5-sonnet-20241022 \
  --agent.model.per_instance_cost_limit=2.00 \
  --env.repo.github_url=https://github.com/SWE-agent/test-repo \
  --problem_statement.github_url=https://github.com/SWE-agent/test-repo/issues/1

and I observe

👋 INFO     This is SWE-agent version 1.0.1 (hash='663803e21c1372c7853f7fdc8efda832fb5ba1c9') with SWE-ReX version 1.2.1
            (rex_hash='unavailable').
🔧 INFO     Loading default config from /app/config/anthropic_filemap.yaml, because no other config file is specified. Specify
            --no_config_file to disable this.
🔧 INFO     Setting problem statement based on github issue url
🔧 DEBUG    No .env file found
🏃 INFO     Starting environment
🦖 INFO     Building image python:3.11 to install a standalone python to /root. This might take a while (but you only have to do it
            once). To skip this step, set `python_standalone_dir` to None.
🦖 DEBUG    Found free port 32797
🦖 INFO     Starting container python3.11-e45e929e-d424-4d78-8179-2d39d15daa59 with image python:3.11 serving on port 32797
🦖 DEBUG    Command: "docker run --rm -p 32797:8000 --name python3.11-e45e929e-d424-4d78-8179-2d39d15daa59
            sha256:57f62bd48dad32ee1e9da784d11272adb9cdfd597c7bff04a624275da78e776e /bin/sh -c '/root/python3.11/bin/swerex-remote
            --auth-token e8b0e70c-8bbf-4c19-8f2e-3bbfc72e28d3'"
🦖 INFO     Starting runtime at 32797
🦖 ERROR    Runtime did not start within timeout. Here's the output from the container process.

<stuck>