tox: Integration with `setup.py test` should not require `virtualenv` to be installed

• Bitbucket: https://bitbucket.org/hpk42/tox/issue/330
• Originally reported by: @bittner
• Originally created at: 2016-05-06T01:47:57.577

In my Python projects I want to separate the requirements only needed for testing cleanly from the actual project.

The beautiful thing of tox is I can specify all those additional requirements in the tox configuration (tox.ini), and with the integration in setup.py it’s possible to not even have tox installed to execute tests.

Actually, this is the theory, because unfortunately, running tests with python setup.py test still requires one dependency to be installed beforehand: virtualenv

Current Behavior

$ python setup.py test
running test
Searching for tox
Reading https://pypi.python.org/simple/tox/
Best match: tox 2.3.1
Downloading https://pypi.python.org/packages/46/39/e15a857fda1852da1485bc88ac4268dbcef037ab526e1ac21accf2a5c24c/tox-2.3.1.tar.gz#md5=9371b3d3e25c03751a0372e19602dfb9
Processing tox-2.3.1.tar.gz
Writing /tmp/easy_install-2zqkvsk9/tox-2.3.1/setup.cfg
Running tox-2.3.1/setup.py -q bdist_egg --dist-dir /tmp/easy_install-2zqkvsk9/tox-2.3.1/egg-dist-tmp-f202aq90
...
Installed /home/user/repos/my-project/.eggs/tox-2.3.1-py3.4.egg
Searching for pluggy<0.4.0,>=0.3.0
Reading https://pypi.python.org/simple/pluggy/
Best match: pluggy 0.3.1
Downloading https://pypi.python.org/packages/1b/a9/6f5f80b75a8d84d21a8a13486fe26a2da9f043f93b464b2e3928be256dc4/pluggy-0.3.1.tar.gz#md5=ecdd791e309f60668b66fec97c2ee7db
Processing pluggy-0.3.1.tar.gz
Writing /tmp/easy_install-p0d1rfyw/pluggy-0.3.1/setup.cfg
Running pluggy-0.3.1/setup.py -q bdist_egg --dist-dir /tmp/easy_install-p0d1rfyw/pluggy-0.3.1/egg-dist-tmp-gsv46avp
...
Installed /home/user/repos/my-project/.eggs/pluggy-0.3.1-py3.4.egg
Searching for py>=1.4.17
Reading https://pypi.python.org/simple/py/
Best match: py 1.4.31
Downloading https://pypi.python.org/packages/f4/9a/8dfda23f36600dd701c6722316ba8a3ab4b990261f83e7d3ffc6dfedf7ef/py-1.4.31.tar.gz#md5=5d2c63c56dc3f2115ec35c066ecd582b
Processing py-1.4.31.tar.gz
Writing /tmp/easy_install-ax6qj5s2/py-1.4.31/setup.cfg
...
Installed /home/user/repos/my-project/.eggs/py-1.4.31-py3.4.egg
Searching for virtualenv>=1.11.2
Reading https://pypi.python.org/simple/virtualenv/
Best match: virtualenv 15.0.1
Downloading https://pypi.python.org/packages/c8/82/7c1eb879dea5725fae239070b48187de74a8eb06b63d9087cd0a60436353/virtualenv-15.0.1.tar.gz#md5=28d76a0d9cbd5dc42046dd14e76a6ecc
Processing virtualenv-15.0.1.tar.gz
Writing /tmp/easy_install-qkl88xj8/virtualenv-15.0.1/setup.cfg
...
Installed /home/user/repos/my-project/.eggs/virtualenv-15.0.1-py3.4.egg
running egg_info
creating my_project.egg-info
writing my_project.egg-info/PKG-INFO
...
writing manifest file 'my_project.egg-info/SOURCES.txt'
running build_ext
GLOB sdist-make: /home/user/repos/my-project/setup.py
flake8 create: /home/user/repos/my-project/.tox/flake8
ERROR: invocation failed (exit code 1), logfile: /home/user/repos/my-project/.tox/flake8/log/flake8-0.log
ERROR: actionid: flake8
msg: getenv
cmdargs: ['/home/user/.virtualenvs/my-project/bin/python', '-m', 'virtualenv', '--python', '/home/user/.virtualenvs/my-project/bin/python3.4', 'flake8']
env: {...}

/home/user/.virtualenvs/my-project/bin/python: No module named virtualenv

ERROR: InvocationError: /home/user/.virtualenvs/my-project/bin/python -m virtualenv --python /home/user/.virtualenvs/my-project/bin/python3.4 flake8 (see /home/user/repos/my-project/.tox/flake8/log/flake8-0.log)
_____________________________________________________________________________________ summary ______________________________________________________________________________________
ERROR:   flake8: InvocationError: /home/user/.virtualenvs/my-project/bin/python -m virtualenv --python /home/user/.virtualenvs/my-project/bin/python3.4 flake8 (see /home/user/repos/my-project/.tox/flake8/log/flake8-0.log)

When I run pip install virtualenv the same testrun runs like a charm.

Is this a problem of tox, or an issue of setuptools?

Expected Behavior

After having set up tox with setuptools integration it should be possible to run python setup.py test to execute tests driven by tox without having had to install any requirements (such as virtualenv or tox) manually beforehand.

References

This problem is also documented in issue #245.