pytest: Extend pytest to differentiate between assertion violations and other kind of failures
In pytest, I want to report all uncaught AssertionErrorexceptions as Failure and all other uncaught exceptions as Error (instead of the default behavior of reporting all uncaught exceptions in test cases and UUT as Failure and all uncaught exceptions raised elsewhere as Error). I thought it could be done with pytest hooks. However, passed, skipped, and failed seem to be the only valid outcome values in TestReport object. So,
- Is it possible to add “error” as a valid outcome and let the rest of pytest handle appropriate reporting, i.e., display E/ERROR instead of F/FAILURE on console output?
- If so, what would be the ideal part of the source to do this?
- If we cannot add “error” as a valid outcome, then what would be the best way to add this behavior to pytest?
About this issue
- Original URL
- State: closed
- Created 8 years ago
- Comments: 15 (8 by maintainers)
You can probably write a hookwrapper to not overwrite the original hook and then change the outcome as desired, similar to what I do with
pytest-vwhere. (Hah, never thought I could use that as an example!)I have created and released a plugin on pypi – https://pypi.python.org/pypi/pytest-finer-verdicts/1.0 – and the source repo is available on github – https://github.com/rvprasad/pytest-finer-verdicts.
BTW, I stumbled on this difference between nose and pytest while I was using PyTest and Hypothesis to introduce property-based testing in an upper-level undergraduate software testing course.