pytest: Make --strict imply --strict-config / xfail_strict?

With pytest 4.5.0 (May 2019), we introduced --strict-markers as a replacement for --strict, and said:

The existing --strict option has the same behavior currently, but can be augmented in the future for additional checks.

With #7286, now we also have --strict-config. Yet, --strict is still an alias for --strict-markers.

Given that pytest 6 is allowed to do breaking changes, wouldn’t it make sense for --strict to imply both --strict-markers and --strict-config now?

(Though, on a second thought: Do we actually need --strict-markers in pytest 6 still, given the warning system? Probably still useful as a convenient way to turn those warnings into errors?)

This would also make pytest more in line with other tools with a --strict flag - mypy and tsc (the typescript compiler) come to mind, where --strict implies various different strict flags.

cc @gnikonorov (who added --strict-config), @webknjaz and @DahlitzFlorian (some existing discussion in #7233), and @nicoddemus (#5023) - what do you think?

About this issue

Original URL
State: open
Created 4 years ago
Comments: 36 (36 by maintainers)

Most upvoted comments

In that case I think we should just drop the --strict alias in v6.0 to make it semver-available as a comprehensive flag for all 3 modes in V6.1

Indeed I believe this is the safest option: the breaking change in 6.0 then becomes the fact that --strict gets removed, and then we can reintroduce it in a minor release (6.1 for example). This seems better to me than changing the behavior of an existing option, because a missing option is a hard error, the former is a change in behavior which might be harder to track down if one does not read the CHANGELOG. Of course it might happen that an user skips directly from 5.4 to 6.1 and gets the behavior change directly, but still seems like it is less likely to cause problems.

As for if it should also cover xfail_strict, I think for the sake of avoiding confusion, --strict should also cover xfail_strict, or for consistency we don’t reintroduce --strict as a command-line option but as a config option instead.

nicoddemus on Jul 22, 2020

Totally agree with @The-Compiler. In fact, --strict should be a shortcut for using all --strict* options.

DahlitzFlorian on Jul 15, 2020

I really like the idea of using --strict to activate all --strict* options. However, I would suggest adding a note to the documentation that --strict will have that behavior in version x.x and people should be aware of it. Of course, we already added a note that it might happen, but an explicit note that it is going to happen in version x.x is highly preferred.

IMHO we can discuss whether we need to wait until version 7 or can include it in some minor version of pytest 6.

DahlitzFlorian on Jul 15, 2020

@symonk We already have --strict-markers and --strict-config, I don’t think we should have a second way to do the same thing.

The-Compiler on Jul 15, 2020

Okay, so I suppose let’s only deprecate --strict for 6.0 then? I don’t see the point in removing it in a release just to reintroduce it again, that just makes things more confusing for everyone.

I think so yes. Created https://github.com/pytest-dev/pytest/issues/7530 for that.

Thanks everyone for participating in the discussion. I’ve moved this issue to the 7.0 milestone.

nicoddemus on Jul 22, 2020