cython: [ENH] C functions should propagate exceptions by default

Is your feature request related to a problem? Please describe. C functions that do not return Python objects cannot currently propagate exceptions by default but require an explicit except clause. In Python code, declared return types default to safe exception propagation instead. Both should behave the same.

cdef int func():
    raise TypeError  # is not propagated

@cython.cfunc
def pyfunc() -> cython.int:
    raise TypeError  # is propagated

cdef int no_exc_func():
    return 0  # no exception raised

cdef extern from *:
    int no_exc_cfunc()  # no exception expected

Describe the solution you’d like C functions should also propagate their exceptions by default.

cdef int func():
    raise TypeError  # CHANGE: should get propagated as with `except? -1`, the default exception value.

@cython.exceptval(check=False)
cdef int func():
    raise TypeError  # is not propagated

cdef int no_exc_func():
    return 0  # CHANGE: no exception raised, but tested by caller

cdef int explicit_no_exc_func() noexcept:    # <- CHANGE
    return 0  # no exception raised, and not tested by caller

cdef extern from *:
    int no_exc_cfunc()  # unclear - should exceptions also be expected for `extern` functions?

Questions:

non-extern cdef functions declared in .pxd files should probably be affected, too, since they should match the implementation in the corresponding .pyx / .py file.
C functions defined in cdef extern blocks are unlikely to benefit from this change. Can we live with keeping them different?

Also see https://github.com/cython/cython/issues/3580 https://github.com/cython/cython/issues/933

About this issue

Original URL
State: closed
Created 3 years ago
Comments: 37 (32 by maintainers)

Most upvoted comments

This will rarely be used. We could add, say, except -, but you don’t need syntax as long as there is a way to do it.

I have to say I like the noexcept suggestion for this much better (and not just because it matches C++). I find except - quite cryptic and it relies on a single character that’s visually sparse and easy to overlook.

h-vetinari on Oct 10, 2021

Yes, I think the intended behaviour is pretty clear. Let’s get some code to discuss, also to see if the behaviour for extern declarations feels ok.

I changed the Cython syntax modifier to noexcept. It’s a context specific thing in the parser and thus won’t introduce new keywords.

scoder on Feb 15, 2022

If the pointer is not going to refer to a raising function, then calling through it would now be a little slower, which can be resolved by adding the noexcept marker (which, again, we should also allow in Cython 0.29.x). And that would be visible from cython -a.

Yes, although this should be combined with https://github.com/cython/cython/issues/4689 so that we can point to non-raising functions with a raising function pointer.

For pragmatic reasons, I’m leaning towards argument 1). Users seem more likely to implement a function and then need a pointer to it, than to point to extern functions. I think that should guide the semantics. The remaining cases can be handled with better error messages on assignments that point to the right fix.

I think what is quite common is to implement a Cython function (which’ll now raise) and pass it to a C function (which may not be expecting an exception). Therefore…

If there are no objections to this, then the decision is to make function pointers behave like Cython implemented functions.

No objections, but I think we should add a strongly worded warning for function pointers passed to extern functions that don’t have an exception specification. That’s potentially somewhere where it has the potential to go wrong and where users should be thinking “what happens to my exception”. (I don’t think we should warn about the a lack of exception specification anywhere else)

da-woods on Apr 16, 2022

I agree with these arguments. And we’re still in alpha, so let’s use that to gather feedback before the final release.

Decision:

We change to “always propagate with the default exception value” for cdef/cpdef functions implemented in Cython 3.0, unless there is an explicit except or noexcept clause.
We keep noexcept semantics for cdef extern blocks of function declarations (unless they return object types or have an explicit except clause).
We add noexcept as a no-op for Cython 0.29.x, so that users can start adding it to their function implementations, if they need it and want their code to be compatible with Cython 3.0.

We also need to update the documentation to reflect this, and to explain the difference for extern blocks. Note that extern functions that are declared as returning an object type always propagate exceptions anyway, so this needs to be part of the explanation as well. The semantic change only applies to functions returning C types, where it is unclear whether exceptions will be raised or not, but where not propagating them from a Cython implemented function is a common mistake.

scoder on Apr 16, 2022

So – are you voting for also changing extern functions?

I ~don’t think so.~ think @da-woods’ arguments are persuasive.

mattip on Apr 15, 2022

Since we can have cython3 specific pxd files (x.pxd for cython2 and x.cython-3.0.pxd) I think it is OK to make a breaking change.

mattip on Apr 14, 2022

FWIW I think I’m currently leaning towards “treat extern functions differently (and assume they don’t throw)”. […] I’m not sure that opinion brings us any closer to agreement though.

Happy to retract my preference if that furthers finding a solution; it could well be that I’m falling into the trap of “a foolish consistency”.

h-vetinari on Apr 15, 2022

Hi @shwina . I am not aware about anyone working on this. Feel free to pick it. PRs are welcome.

matusvalo on Feb 15, 2022

    raise TypeError  # CHANGE: should get propagated as with `except -1`, the default exception value.

Presumably except?-1 (i.e -1 may not be the exception value)

da-woods on Jul 13, 2021