portainer: Unable to stop or remove stack (docker-standalone)

Bug description If I deploy a stack from a git repo (authorization needed) and add environment variables, I then can’t stop or delete the stack anymore. Updating is no problem.

Expected behavior Stack could be stopped or deleted with the set environment variables

Steps to reproduce the issue:

  1. create stack (from repository)
  2. compose-file should contain variables like ${VARIABLE_A}
  3. deploy stack with variables set via “environment variables” feature new in 2.9.0
  4. try to remove or stop the stack

Portainer Logs

level=info msg="2021/10/01 11:33:26 http error: failed to remove a stack: The VARIABLE_A variable is not set. Defaulting to a blank string."
level=info msg="The VARIABLE_B variable is not set. Defaulting to a blank string."
level=info msg="The VARIABLE_C variable is not set. Defaulting to a blank string."
level=info msg="The VARIABLE_D variable is not set. Defaulting to a blank string."
level=info msg="The VARIABLE_E variable is not set. Defaulting to a blank string."
level=info msg="The VARIABLE_F variable is not set. Defaulting to a blank string."
level=info msg="The VARIABLE_G variable is not set. Defaulting to a blank string."
level=info msg="The VARIABLE_H variable is not set. Defaulting to a blank string."
level=info msg="The VARIABLE_I variable is not set. Defaulting to a blank string."
level=info msg="The VARIABLE_J variable is not set. Defaulting to a blank string."
level=info msg="The VARIABLE_K variable is not set. Defaulting to a blank string."
level=info msg="The VARIABLE_L variable is not set. Defaulting to a blank string."
level=info msg="The VARIABLE_M variable is not set. Defaulting to a blank string."
level=info msg="The VARIABLE_N variable is not set. Defaulting to a blank string."
level=info msg="The VARIABLE_O variable is not set. Defaulting to a blank string."
level=info msg="The VARIABLE_P variable is not set. Defaulting to a blank string."
level=info msg="The VARIABLE_Q variable is not set. Defaulting to a blank string."
level=info msg="The VARIABLE_R variable is not set. Defaulting to a blank string."
level=info msg="The VARIABLE_S variable is not set. Defaulting to a blank string."
level=info msg="The VARIABLE_T variable is not set. Defaulting to a blank string."
level=info msg="The VARIABLE_U variable is not set. Defaulting to a blank string."
level=info msg="The Compose file './docker-compose.yaml' is invalid because:"
level=info msg="services.mssql.ports contains an invalid type, it should be a number, or an object"
level=info msg=": exit status 1) (code=500)"

img

Technical details:

  • Portainer version: 2.9.0
  • Platform (windows/linux): linux, amd64
  • Docker version: 20.10.8

About this issue

  • Original URL
  • State: closed
  • Created 3 years ago
  • Comments: 35 (9 by maintainers)

Most upvoted comments

Reproducing the issue

The smallest way to replicate it on docker standalone:

version: '3.6'
services:
  nginx-example:
    image: nginx:latest
networks:
  default:
    name: ${test}
    driver: bridge

Workaround

Use the variable with a default value

version: '3.6'
services:
  nginx-example:
    image: nginx:latest
networks:
  default:
    name: ${test:-tmp}
    driver: bridge

as per https://github.com/docker/compose/issues/6497#issuecomment-854485383

Technical details

The error in the Portainer logs will show

http error: Unable to stop stack (err=failed to remove a stack: The test variable is not set. Defaulting to a blank string.
[449] Failed to execute script docker-compose
Traceback (most recent call last):
  File \"docker-compose\", line 3, in <module>
  File \"compose/cli/main.py\", line 67, in main
  File \"compose/cli/main.py\", line 123, in perform_command
  File \"compose/cli/command.py\", line 69, in project_from_options
  File \"compose/cli/command.py\", line 141, in get_project
  File \"compose/project.py\", line 106, in from_config
  File \"compose/network.py\", line 318, in get_networks
  File \"compose/network.py\", line 123, in true_name
  File \"compose/network.py\", line 144, in _set_legacy_flag
  File \"compose/network.py\", line 104, in inspect
  File \"site-packages/docker/utils/decorators.py\", line 17, in wrapped
docker.errors.NullResource: Resource ID was not provided
: exit status 255) (code=500)

Portainer does https://github.com/portainer/portainer/blob/33001a8654bd391bd986352c010356a219205edb/api/exec/compose_stack.go#L77

which invokes https://github.com/portainer/docker-compose-wrapper/blob/10a04c9d4f19967d4eb471ccc64ca2adcc50b795/compose/internal/composeplugin/composeplugin.go#L76

So that’s using an empty string for the envFilePath.

Meaning that their api doesn’t allow us to use the mechanism: https://github.com/portainer/portainer/blob/33001a8654bd391bd986352c010356a219205edb/api/exec/compose_stack.go#L56

But compose requires it. The following will fail in the same way:

echo 'test=test' > .env
docker-compose -p "test-stack" up
rm .env
docker-compose -p "test-stack" down

So the alternative could be to invoke this one directly https://github.com/portainer/docker-compose-wrapper/blob/10a04c9d4f19967d4eb471ccc64ca2adcc50b795/compose/internal/composeplugin/composeplugin.go#L85

Any contributions would be most welcome.

@tiehfood I’ve tested with above and can reproduce the issue

version: '3'

services:

  wordpress:
    image: wordpress
    ports:
      - 8080:80
    environment:
      WORDPRESS_DB_HOST: ${DB_HOST}
      WORDPRESS_DB_NAME: ${DB_NAME}
      WORDPRESS_DB_USER: example
      WORDPRESS_DB_PASSWORD: examplePW
      NEWVAR: ${NEWVAR}
    volumes:
      - ./wordpress-data:/var/www/html

  mysql:
    image: mysql:5.6
    environment:
      MYSQL_ROOT_PASSWORD: examplerootPW
      MYSQL_DATABASE: wordpress
      MYSQL_USER: example
      MYSQL_PASSWORD: examplePW
    volumes:
      - mysql-data:/var/lib/mysql
networks:
  default:
    name: ${NET_NAME}
    driver: bridge
volumes:
  mysql-data:

image

When trying to stop the stack

image

Ok, the workaround with the default values is acceptable at the moment for me. Thanks 🙂

To give this fix a try, please use the image portainerci/portainer:develop.

I can confirm, that #6370 is fixed for me on develop.

Hi team. I would like to add that I’m facing the same problem as described on this ticket but I’ve figured out that, when my docker-compose have the dns directive set up, it fails with the same error. If I edit docker-compose and comment out the dns directive, I can delete the stack directly without any changing related to environment variables. An example of the dns configuration in place and commented out:

dns:

  • ${dns_ipv4}
  • ${dns_ipv6} dns_search: ${dns_search}

This is the only change that I need to do to correctly use the stack details commands.

Hope it helps to sort this out