tox: 2.8.0: {[testenv]setenv} substitution broken

While installing tempest as part of devstack install, below commnd blows up with an error - tox -r --notest -efull

Error -

2017-09-01 18:22:20.661 | ++ lib/tempest:install_tempest:612          :   pushd /opt/stack/new/tempest
2017-09-01 18:22:20.661 | ~/tempest ~/devstack
2017-09-01 18:22:20.662 | ++ lib/tempest:install_tempest:613          :   tox -r --notest -efull
2017-09-01 18:22:21.720 | Traceback (most recent call last):
2017-09-01 18:22:21.720 |   File "/usr/local/bin/tox", line 11, in <module>
2017-09-01 18:22:21.720 |     sys.exit(cmdline())
2017-09-01 18:22:21.720 |   File "/usr/local/lib/python2.7/dist-packages/tox/session.py", line 38, in main
2017-09-01 18:22:21.720 |     config = prepare(args)
2017-09-01 18:22:21.720 |   File "/usr/local/lib/python2.7/dist-packages/tox/session.py", line 26, in prepare
2017-09-01 18:22:21.720 |     config = parseconfig(args)
2017-09-01 18:22:21.720 |   File "/usr/local/lib/python2.7/dist-packages/tox/config.py", line 242, in parseconfig
2017-09-01 18:22:21.720 |     parseini(config, inipath)
2017-09-01 18:22:21.720 |   File "/usr/local/lib/python2.7/dist-packages/tox/config.py", line 796, in __init__
2017-09-01 18:22:21.721 |     replace=name in config.envlist)
2017-09-01 18:22:21.721 |   File "/usr/local/lib/python2.7/dist-packages/tox/config.py", line 827, in make_envconfig
2017-09-01 18:22:21.721 |     res = meth(env_attr.name, env_attr.default, replace=replace)
2017-09-01 18:22:21.721 |   File "/usr/local/lib/python2.7/dist-packages/tox/config.py", line 964, in getdict_setenv
2017-09-01 18:22:21.721 |     definitions = self._getdict(value, default=default, sep=sep)
2017-09-01 18:22:21.721 |   File "/usr/local/lib/python2.7/dist-packages/tox/config.py", line 975, in _getdict
2017-09-01 18:22:21.721 |     name, rest = line.split('=', 1)
2017-09-01 18:22:21.721 | ValueError: need more than 1 value to unpack
2017-09-01 18:22:21.733 | + lib/tempest:install_tempest:1            :   exit_trap

Logs link - http://logs.openstack.netapp.com/ci-logs/logs/65/499765/5/upstream-check/cinder-cDOT-FCP/3566c47/logs/devstacklog.txt.gz

About this issue

Original URL
State: closed
Created 7 years ago
Reactions: 2
Comments: 16 (10 by maintainers)

Commits related to this issue

Updated openstack/openstack Project: openstack-infra/project-config 0bcbb286b03007d87c217703605a77969a2d1cfb Pin tox to 2.7.0 There is a fun new bug [0] in tox 2.8.0 that affects tempest. To avoid ... — committed to openstack/openstack by cboylan 7 years ago
Pin tox to 2.7.0 There is a fun new bug [0] in tox 2.8.0 that affects tempest. To avoid this crashing everything to a hault pin to 2.7.0 for now. [0] https://github.com/tox-dev/tox/issues/595 Chang... — committed to openstack/project-config by cboylan 7 years ago
Merge pull request #596 from Avira/master changelog and reproducer for #595 — committed to tox-dev/tox by obestwalter 7 years ago
Don't attempt to parse dicts if not replacing When reading in the config and processing dict content we were splitting on = assuming we would always have key = value pairs. However in the case where ... — committed to cboylan/tox by cboylan 7 years ago
Don't attempt to parse dicts if not replacing When reading in the config and processing dict content we were splitting on = assuming we would always have key = value pairs. However in the case where ... — committed to cboylan/tox by cboylan 7 years ago
Updated openstack/openstack Project: openstack-infra/puppet-jenkins 5fc0bbbdb3d549bd0088e866180881b541da5646 Pin tox to 2.7.0 Tox 2.8.0 has broke us, clarkb has a patch upstream to fix. In the mean... — committed to openstack/openstack by pabelanger 7 years ago
Pin tox to 2.7.0 Tox 2.8.0 has broke us, clarkb has a patch upstream to fix. In the mean time, we can pin to 2.7.0. https://github.com/tox-dev/tox/issues/595 Change-Id: I1079fb1bc6606a33b5b2517cca0... — committed to openstack-infra/puppet-jenkins by pabelanger 7 years ago
Fix substitution problems introduced in 2.8 (#599) * #595 fix by updating implementation of #515 The way #515 was implemented did not play nicely with env var substitution. This might render the f... — committed to tox-dev/tox by obestwalter 7 years ago
fix oversight in fixing #595: inherit new exception from Error (#635) — committed to tox-dev/tox by obestwalter 7 years ago

Most upvoted comments

Thanks @obestwalter, I’ll try 2.8.1 out tomorrow and let you know how it goes.

adamrothman on Sep 4, 2017

https://github.com/tox-dev/tox/pull/521#discussion-diff-125055520R791 is the source of the bug. Basically if you run tox -efoo then config.envlist == [‘foo’] but tox will still attempt to parse envs bar and baz and foobar. But none of those will get replacements done beacuse those names are not in [‘foo’].

cboylan on Sep 1, 2017

This isn’t a complete fix. The original tempest tox.ini now fails on missing boolean values when run against the commit above. I figured I’d throw something up before my day ends though.

cboylan on Sep 1, 2017

{[testenv]setenv} breaks with the new changes looks like we don’t have a test for this substitution behaviour yet. I will create a reproducer and see if we can fix that with a quick bug fix soon. Please stick with 2.7 until then.

obestwalter on Sep 1, 2017

Here’s my tox.ini, if it helps:

[tox]
envlist = py36


[testenv]
deps =
    -rrequirements.txt
    flake8
    mypy
    pyflakes>=1.6.0
    pytest
    pytest-asyncio
passenv = ASYNC_TEST_TIMEOUT AWS_ACCESS_KEY_ID AWS_DEFAULT_REGION AWS_SECRET_ACCESS_KEY ORBIT_AWS__SERVICES__DYNAMODB__ENDPOINT_URL ORBIT_SERVICES__NEO4J__HOST ORBIT_SERVICES__REDIS__HOST
setenv =
    ORBIT_ENV = test
    ASYNC_TEST_TIMEOUT = 10
    AWS_DEFAULT_REGION = us-west-2
commands =
    flake8 orbit tests
    mypy --ignore-missing-imports orbit tests
    pytest {posargs} tests/unit
    pytest {posargs} tests/integration


[testenv:docker]
deps =
    flake8
    mypy
    pyflakes>=1.6.0
    pytest
    pytest-asyncio
sitepackages = True


[testenv:verbose]
setenv =
    {[testenv]setenv}
    ORBIT_AWS__SERVICES__DYNAMODB__ENDPOINT_URL = http://192.168.99.100:8000
    ORBIT_SERVICES__NEO4J__HOST = 192.168.99.100
    ORBIT_SERVICES__REDIS__HOST = 192.168.99.100
commands =
    pytest -s -vv {posargs}


[testenv:lint]
commands =
    flake8 orbit tests
    mypy --ignore-missing-imports orbit tests


[flake8]
ignore = E501

adamrothman on Sep 1, 2017