cython: Combining @staticmethod with other decorators is broken

Consider

def debug(fun):
    print(fun)
    return fun

class A(object):
    @staticmethod
    @debug
    def foo(self):
        return self

In Python, this correctly prints <function foo at 0x7f4d305df410>. In Cython, this wrongly prints <staticmethod object at 0x7f4d30e5e718>.

When doing the same with a Cython cdef class, the decorators are somehow applied twice: the output becomes

<staticmethod object at 0x7f4d30e5eb40>
<staticmethod object at 0x7f4d305e52f0>

Migrated from http://trac.cython.org/ticket/880

About this issue

  • Original URL
  • State: open
  • Created 8 years ago
  • Reactions: 7
  • Comments: 15 (7 by maintainers)

Most upvoted comments

For peoplo who facing the same issue here, I got a method to work around this problem. convert a

class A:
    @staticmethod
    @decorator
    def test():
          pass

to

class A:
    def test():
          pass
    test = staticmethod(decorator(test))

A simple implementation of this kind of conversion using ast and astunparse is:

import ast, astunparse


class RewriteFuncdef(ast.NodeTransformer):
    def visit_ClassDef(self, node):
        for index, sub_node in enumerate(node.body):
            if not isinstance(sub_node, ast.FunctionDef):
                continue
                
            decorators = sub_node.decorator_list
            if self.need_rewrite(decorators):
                new_funcdef, assign = self.convert_func_def(sub_node)
                node.body.pop(index)
                node.body.insert(index ,assign)
                node.body.insert(index, new_funcdef)
        return node
                
    def need_rewrite(self, decorators) -> bool:
        for decorator in decorators:
            if isinstance(decorator, ast.Name) and decorator.id == "staticmethod":
                return True
        return False
    
    def convert_func_def(self, func_def):
        new_funcd = ast.FunctionDef(name=func_def.name, body=func_def.body, args=func_def.args, returns=func_def.returns, decorator_list=[])
        decorators = func_def.decorator_list
        a_name = ast.Name(id=func_def.name)
        a_value = self.gen_assign_value(a_name, decorators)
        assign = ast.Assign(targets=[a_name], value=a_value)
        return new_funcd, assign
    
    def gen_assign_value(self, func_name, decorators):
        args = [func_name]
        for decorator in reversed(decorators):
            args = [ast.Call(func=decorator, args=args, keywords=[])]
        return args[0]

module = ast.parse(open("static.py").read())
transformer = RewriteFuncdef()
transformer.visit(module)
generated_content = astunparse.unparse(module)

with open("new_file.py", "w") as f:
      f.write(generated_content)

Hope this can help somebody.

Looks like Cython handles the order of multiple decorators incorrectly.

class A(object):
    @decorate
    @staticmethod
    def a(self):
        pass

    @staticmethod
    @decorate
    def b():
        pass

Python can tell the difference between A.a and A.b but Cython can’t.

This would cause much trouble since we used to use functools.wraps in every decorator to preserve the original signature and docstring but wraps can only be applied on function object but not staticmethod object.

FWIW there is a PR that should fix this issue https://github.com/cython/cython/pull/3966 - testing that PR and letting me know whether it works or not for your would be useful. We don’t really need more comments saying “I hope this is fixed soon” though.

Using the deprecated abstractstaticmethod is a temp fix for now. Hoping the bug can be fixed soon.

Another quick and dirty workaround if you have access to the decorator implementation (and not to the code that uses the decorators):

I override staticmethod, store the information I require and am subsequently able to retrieve them if the decorator order is incorrect

# noinspection PyPep8Naming
class SM(staticmethod):
    def __init__(self, *args, **kwargs):
        super(staticmethod, self).__init__(*args, **kwargs)
        self.__inds_static_hack__ = args[0]

__static = globals()['__builtins__'].staticmethod = SM

# This has to be done for every decorator
def deco(_fun):
    fun = getattr(_fun, '__inds_static_hack__', _fun)
    def f(*args, **kwargs):
        return fun(*args, **kwargs)

    f.__name__ = fun.__name__ # etc...
    return f
# Showcase that it works:
class C:
    @staticmethod
    @deco
    def f():
        return 'f_return_value'

    @deco
    @staticmethod
    def g():
        return 'g_return_value'

print(f'both is fine : {C.f.__name__} returns {C.f()}, \n'
      f'               {C.g.__name__} returns {C.g()}. \n'
      f'This can also be used in a different file.')

I’m unsure if what I do is advisable, but it works for now 😃