compose: error on launching docker-compose by piping to sh ( echo 'docker-compose ... ' | sh )

I could get it to work by adding the -T parameter to not create a Pseudo-TTY.

docker-compose exec -T container_name ./build.sh for example.

for all those curious about getting around this while still getting information from docker-compose…

docker exec -i $(docker-compose ps -q cassandra) < someexample.cql

The -i keeps stdin open while not allotting a tty and the ps -q on the docker-compose provides us with the container ID we want.

I assume this will provide multiple IDs if you have many running, you’ll have to do some munging to use this method.

It would appear docker-compose isn’t implicitly using a -i when passing the exec along.

So basically, you need -i for docker exec when you are piping in data to the command.

And you should not use -t for docker exec when you are piping data out from the command.

And docker-compose exec only have -T, which means neither -i or -t. But you cannot represent -i without -t in docker-compose.

That’s the issue, right?

I’m having the same issue when running: docker-compose exec mycontainer cmd_that_takes_stdin < my/file/with/content

Everything seems to work after all (the command executes as expected), it’s just that nasty crash that docker-compose gives me afterwards:

$ docker-compose exec mycontainer cmd_that_takes_stdin < my/file/with/content
Traceback (most recent call last):
  File "<string>", line 3, in <module>
  File "compose/cli/main.py", line 61, in main
  File "compose/cli/main.py", line 113, in perform_command
  File "compose/cli/main.py", line 441, in exec_command
  File "site-packages/dockerpty/pty.py", line 338, in start
  File "site-packages/dockerpty/io.py", line 32, in set_blocking
ValueError: file descriptor cannot be a negative integer (-1)
docker-compose returned -1

$ echo $?
255

When I use the -T flag as recommended though, my command doesn’t work and I even get stuck on the docker-compose command (doesn’t return to my shell but just hangs instead).

I’m having the same problem, when piping docker-compose output via gzip to a file. When I run the command myself in Bash, it works just fine. However, as a cron job, I’m getting the same error as you — and the same stack trace file & line numbers & function names, as you, and return code -1.

Here’s the command:

/usr/local/bin/docker-compose exec postgres pg_dumpall --username=postgres \
   | gzip > $postgres_backup_path

(I have that line in a backup script that I run sometimes manually (works fine) and sometimes as a cron job (won’t work)).

Now I’ve worked around my problem by calling Docker directly like so:

/usr/bin/docker exec edm_postgres_1 pg_dumpall --username=postgres \
    | gzip > $postgres_backup_path

— works fine from both Cron and manually in Bash.

reproduced by gitlab-ci

Same here. docker-compose exec fails when running as part of a Jenkins job (works locally), but docker exec works. Thanks @paolomainardi !

I can reproduce this bug just when docker-compose gets called from jenkins or gitlab-ci

I’d say that the problem is that -T in docker-compose exec is mostly equivalent to not using any of -t or -i flags in docker run, so with -T there is not tty but also not attached stdin, and it seems that there is no way of attaching stdin without tty in docker-compose.

docker-compose exec should probably have different flags for tty and for attaching stdin as docker run has.

same problem on jenkins, solved by using -T: docker-compose exec -T container_name cmd

I have the same problem, but just by launching this from jenkins (it fails just there): docker-compose exec drupal bash -c "bin/behat --profile=ci", now changed to docker exec ${COMPOSE_PROJECT_NAME}_drupal_1 bash -c "bin/behat --profile=ci".

now I try in cron (again) crontab -e

*/10 * * * * time -p /usr/local/bin/docker-compose -f /path/to/docker-compose.yml exec php ./yii command

now I have same problem in cron (again)

[17362] Failed to execute script docker-compose
Traceback (most recent call last):
  File "site-packages/dockerpty/pty.py", line 334, in start
  File "site-packages/dockerpty/pty.py", line 367, in _hijack_tty
  File "site-packages/dockerpty/io.py", line 59, in select
  File "site-packages/dockerpty/io.py", line 351, in fileno
  File "site-packages/dockerpty/io.py", line 103, in fileno
  File "socket.py", line 635, in fileno
ValueError: I/O operation on closed file.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "bin/docker-compose", line 6, in <module>
  File "compose/cli/main.py", line 71, in main
  File "compose/cli/main.py", line 127, in perform_command
  File "compose/cli/main.py", line 519, in exec_command
  File "site-packages/dockerpty/pty.py", line 338, in start
  File "site-packages/dockerpty/io.py", line 32, in set_blocking
  File "site-packages/dockerpty/io.py", line 351, in fileno
  File "site-packages/dockerpty/io.py", line 103, in fileno
  File "socket.py", line 635, in fileno
ValueError: I/O operation on closed file.
Command exited with non-zero status 255
real 2.09
user 0.98

after add into crontab -e

COMPOSE_INTERACTIVE_NO_CLI=1
# Edit this file to introduce tasks to be run by cron.

https://stackoverflow.com/a/10657111/7670492

I try again with -T

/usr/local/bin/docker-compose -f /path/to/docker-compose.yml exec -T php ./yii command

success

So, as I understand, docker exec is the solution for now?

Just want to leave this minimal test-case for docker-compose:

echo 123 | docker-compose exec container_name tee

It triggers error:

Traceback (most recent call last):
  File "bin/docker-compose", line 3, in <module>
  File "compose/cli/main.py", line 88, in main
  File "compose/cli/main.py", line 140, in perform_command
  File "compose/cli/main.py", line 486, in exec_command
  File "site-packages/dockerpty/pty.py", line 338, in start
  File "site-packages/dockerpty/io.py", line 32, in set_blocking
ValueError: file descriptor cannot be a negative integer (-1)
Failed to execute script docker-compose

echo 123 | docker exec -i $(docker-compose ps -q service) cat - for now…

For people who are stumbling into this while trying to run docker-compose exec on Jenkins pipeline like myself:

The -T flag mentioned bypasses the error, but it does not actually execute the command for me. Jenkins just thinks the command executed successfully and moves on.

I had to fall back to docker exec, like so:

environment {
  PROJECT_NAME = "${env.BRANCH_NAME}.${env.BUILD_ID}"
  CONTAINER_PREFIX = "${PROJECT_NAME.replace('-', '').replace('.', '').toLowerCase()}"
}
stages {
  stage('run') {
    steps {
      sh 'docker-compose -p $PROJECT_NAME up -d --build'
      sh "docker exec ${CONTAINER_PREFIX}_app_1 bash -c \"hello world\""
    }
  }
}

The $CONTAINER_PREFIX is a workaround to get the docker compose prefix. It’s not pretty, and a little too verbose, but it’s the only way I could get it to work for me.

env: ansible playbook + ubuntu(vagrant)

error:

Traceback (most recent call last):
  File "/usr/local/bin/docker-compose", line 11, in <module>
    sys.exit(main())
  File "/usr/local/lib/python2.7/dist-packages/compose/cli/main.py", line 65, in main
    command()
  File "/usr/local/lib/python2.7/dist-packages/compose/cli/main.py", line 117, in perform_command
    handler(command, command_options)
  File "/usr/local/lib/python2.7/dist-packages/compose/cli/main.py", line 462, in exec_command
    pty.start()
  File "/usr/local/lib/python2.7/dist-packages/dockerpty/pty.py", line 338, in start
    io.set_blocking(pump, flag)
  File "/usr/local/lib/python2.7/dist-packages/dockerpty/io.py", line 32, in set_blocking
    old_flag = fcntl.fcntl(fd, fcntl.F_GETFL)
ValueError: file descriptor cannot be a negative integer (-1)

add -T fix the issue.

-T isn’t working for me. My only workaround is to use docker exec directly but means I have to parse the container ID from docker ps which isn’t ideal.

To test I have a basic python script that just prints stdin

$ cat myScript.py
#!/usr/bin/python
import sys
print "started"
for line in sys.stdin:
    print line

$ echo test | docker-compose exec -T myContainer /usr/src/myScript.py
.. nothing happens, just waits ..

$ echo test | docker-compose exec myContainer /usr/src/myScript.py
Traceback (most recent call last):
  File "<string>", line 3, in <module>
  File "compose/cli/main.py", line 65, in main
  File "compose/cli/main.py", line 117, in perform_command
  File "compose/cli/main.py", line 462, in exec_command
  File "site-packages/dockerpty/pty.py", line 338, in start
  File "site-packages/dockerpty/io.py", line 32, in set_blocking
ValueError: file descriptor cannot be a negative integer (-1)
docker-compose returned -1

$ echo test | docker exec -i d6380314bb89  /usr/src/myScript.py
started
test

docker-compose exec container_name cat logfile > logfile

also fails for me, but with -T

docker-compose exec -t container_name cat logfile > logfile

it just produce empty file, so no any output! Even there no error, it is unusable.

I think he means you can use docker-compose ps -q {app} to get the container ID. For example,

docker exec $(docker-compose ps -q {app}) bash -c …

I’m running into this exact issue as well. Would love to be able to pipe data into containers via docker-compose.

Have you tried adding the -T flag to docker-compose exec?