pytest: Allow pytest.raises cooperate with ExceptionGroups

What’s the problem this feature will solve?

Let pytest.raises() handle ExceptionGroup unwrapping, similar to what an “except*” clause would do. The following is valid python code that will catch the ValueError exception:

try:
  raise ExceptionGroup("", [ValueError])
except* ValueError:
  pass

However pytest.raises will not manage this, and will fail with catching the ExceptionGroup instead.

with pytest.raises(ValueError):
  raise ExceptionGroup("", [ValueError])

Describe the solution you’d like

Assert some code throws an exception or an exception as part of an exception group.

anyio/trio will now always wrap exception in ExceptionGroups when raised from taskgroups. This leads to silly checks like: https://github.com/agronholm/anyio/blob/3f1eca1addcd782e2347350a6ddb2ad2b80c6354/tests/test_taskgroups.py#L279C1-L287C31 to unwrap the exceptiongroup.

A basic solution to handle this (without the bells and whistles) would be:

@contextlib.contextmanager
def raises_nested(exc):
    try:
        yield
    except* exc:
        pass
    else:
        raise AssertionError("Did not raise expected exception")

Alternative Solutions

Additional context

About this issue

Original URL
State: open
Created 8 months ago
Reactions: 4
Comments: 27 (20 by maintainers)

Most upvoted comments

Seems pytest 8 solves this issue. Looks to have taken my preferred route of matching as old raises. https://docs.pytest.org/en/stable/how-to/assert.html#assert-matching-exception-groups

elupus on Jan 29, 2024

Related to #10441 which was marked as fixed in #11424 (not yet released). The main blocker here is someone designing an interface which:

is convenient to use, and simple in simple cases

doesn’t only handle simple cases

doesn’t encourage users to ignore e.g. the nesting depth of their exception groups or the possibility of multiple unrelated errors

We’re generally happy to review suggestions, if you’d like to propose something!

I’m not sure whether this should go in here, or in #10441 (which was maybe closed prematurely), or in a new issue - but when writing a helper for Trio I went through a couple ideas for ways to solve this - and I think my favorite one is introducing a class ExpectedExceptionGroup that pytest.raises can accept an instance of, in place of a type[Exception].

Example usage:

from pytest import ExpectedExceptionGroup

with pytest.raises(ExpectedExceptionGroup((ValueError,))):
  ...

supports nested structures:

with pytest.raises(ExpectedExceptionGroup(ExpectedExceptionGroup((KeyError,)), RuntimeError)):
  ...

and works with the current feature of passing a tuple to pytest.raises when you expect one of several errors - though maybe not in a super elegant way if you’re nested several layers deep:

with pytest.raises(
  (
    ExpectedExceptionGroup(KeyError),
    ExpectedExceptionGroup(ValueError),
    RuntimeError,
  )
):
  ...

Can additionally work in keyword args for matching message and/or note.

with pytest.raises(ExpectedExceptionGroup((KeyError, ValueError), message="foo", note="bar")):
  ...

and possibly also accept instances of exceptions, to be able to match the message in sub-exceptions

with pytest.raises(ExpectedExceptionGroup((KeyError(".* very bad"), RuntimeError)):
  ...

This would fulfill all of the requested bullet points, be a minimal change to the API, and shouldn’t be very complicated to implement. To experiment outside of pytest (as requested in https://github.com/pytest-dev/pytest/issues/10441#issuecomment-1293080834) should be somewhat doable - if a bit messy*

*can duplicate the functionality of pytest.raises, or monkey patch _pytest.python_api.RaisesContext.__exit__, or possibly with https://docs.python.org/3/library/abc.html#abc.ABCMeta.__subclasshook__ to trick https://github.com/pytest-dev/pytest/blob/fdb8bbf15499fc3dfe3547822df7a38fcf103538/src/_pytest/python_api.py#L1017C3-L1017C3 )

jakkdl on Nov 23, 2023