tox: [4.12.0][regression] The presence of `TOX_PARALLEL_NO_SPINNER=1` in CI suppresses output of single env run tox commands

Issue

I have a number for globally set environment variables, one of each is TOX_PARALLEL_NO_SPINNER: https://github.com/ansible/pylibssh/blob/ef77a85/.github/workflows/ci-cd.yml#L75.

In one of the steps, I run tox -e a-single-env -qq | tee a-file.txt. In tox 4.11.4, it produces output from the command specified in tox.ini. Starting with tox 4.12.0, the output is empty.

I’m pretty convinced this is a side effect of #3159.

Previously, setting this var allowed having it in one place, while allowing for parallel and non-parallel runs different job steps, not having to track whether to disable or enable the spinner in each. So I consider this to be a regression.

Environment

  • OS: ubuntu-latest @ GHA

tox == 4.12.0

Output of running tox

No output.

Minimal example

$ TOX_PARALLEL_NO_SPINNER=1 tox -e a-single-env --skip-pkg-install -qq

About this issue

  • Original URL
  • State: open
  • Created 3 years ago
  • Reactions: 3
  • Comments: 17 (16 by maintainers)

Commits related to this issue

Most upvoted comments

I think the key issue here is that parallel-no-spinner/ TOX_PARALLEL_NO_SPINNER is a bit of an ambiguous name.

Should it force parallel mode or just deactivate the spinner? We probably could have named it just no-spinner and then it would have been very clear, or no-spinner-in-parallel-mode, but this ship has sailed.

There are quite a couple of different ways forward now, but it looks like all have their pros and cons.

@sparkiegeek I am afraid, I definitely do not want to cancel our promise that configuration works the same for command line options and environment variables. Thinking of myself, I would never recall which one works the one or the other way.

From my past experience I know that tox users usually set up a pipeline, and leave it for years, and are highly reluctant to any change - if that wasn’t the case, I would deprecate the current naming and create new ones which are very clear.

I currently think that either reverting https://github.com/tox-dev/tox/issues/3158 or adding a warning if we have contrary settings is the way forward.

We could also create a new, dedicated “TOX_CI_xxx” option which would be a replacement for the old behavior.

And we definitely need to document the changes properly.

I have also been confused about this for some time on my local development machine, where I found the spinner obnoxious and had set TOX_PARALLEL_NO_SPINNER=1 in the env file of my shell.

One compromise between #3158 and wanting to set it globally is specifically giving a warning if passing --parallel-no-spinner on the command line without --parallel. That is technically a difference in behavior between options and env vars, but only in terms of command-line parsing.

Although, is there any effect (other than output-wise) between running a single environment in parallel vs not? That’s perhaps an even more disruptive change, but I could see specifying --parallel with only one tox env ignoring the flag, and/or raising a warning/error.

I could also see reverting #3158 and introducing a new convenience flag that is --parallel + --parallel-no-spinner. With docs being clearer, and the convenience flag being listed next to the old --parallel-no-spinner, I think anybody that was bothered by #3158 should be happy.

@tusharsadhwani maybe also update the docs/changelog to mention that setting no-spinner now forces parallel mode with a single env, suppressing the output? This is the most unobvious breaking change, it seems.

@gaborbernat I suppose my point is that this is one of the vars that was convenient to set globally to avoid repetition. I didn’t notice that the output changed (and was missing from the expected place) months after this was released. I’m pretty sure many people have it set similarly, because it’s pretty common to borrow ideas from other projects and copy a lot of env vars, especially the ones related prettifying the output since they tend to be useful globally.

This means that it’ll be annoying and unobvious for many to (1) notice something changed and (2) connect the dots with something they did 5 years ago. Obviously, a proper solution is dependency pinning but even with that, it’s not necessarily visible since this change doesn’t cause failures, just silently hides a part of the output. People don’t tend to check their successful CI runs and unfold green steps.

I suppose, I can accept the point that it’s useful for some to run in parallel mode. But then, why would tox even switch to parallel with a single parallel env requested? And if it does, shouldn’t it at least issue a warning?

@ssbarnea have you checked which env vars are set in both cases? Externally to tox and internally, on the testenv level. That’s probably the source of your differences. The version bump might be related to hiding the output, but unlikely to the underlying failure, I think.

@tusharsadhwani what’s your opinion on all this, having read my explanations above?

Yes, but I did not say that you should also set it for non-parallel targets 😆 Is a regression, yes, but I do not think this was intended to be used in this form. cc @tusharsadhwani

Well, it’s been recommended to have this env var since tox 3. I’m sure it’s present in many GHA workflows today and making workarounds will result in a lot of maintenance burden for the users.