scalene: Scalene: internal error: unable to find Python allocator functions

Describe the bug When starting scalene (with poetry run python -m scalene my_script.py), I get the error Scalene: internal error: unable to find Python allocator functions. This happens both with poetry run python -m scalene my_script.py or if started within the venv (python -m scalene my_script.py or just scalene my_script.py).

Some seconds after this output, i get a long traceback (reported afterwards).

I don’t really think this is something about my program, since it always worked without scalene, but I think it is something related to Trio.

To Reproduce Steps to reproduce the behavior:

Snippet that cause the error:

#asd.py

import time
import trio
time.sleep(3)

and then poetry run python -m scalene asd.py. This gives this traceback:

Error in program being profiled:
 string index out of range
Traceback (most recent call last):
  File "/home/didi/leaf/leaf-radio/.venv/lib/python3.7/site-packages/scalene/scalene_profiler.py", line 1326, in profile_code
    exec(code, the_globals, the_locals)
  File "asd.py", line 2, in <module>
    import trio
  File "/home/didi/leaf/leaf-radio/.venv/lib/python3.7/site-packages/trio/__init__.py", line 48, in <module>
    from ._sync import (
  File "/home/didi/leaf/leaf-radio/.venv/lib/python3.7/site-packages/trio/_sync.py", line 475, in <module>
    @attr.s(eq=False, hash=False, repr=False)
  File "/home/didi/leaf/leaf-radio/.venv/lib/python3.7/site-packages/attr/_make.py", line 1312, in wrap
    field_transformer,
  File "/home/didi/leaf/leaf-radio/.venv/lib/python3.7/site-packages/attr/_make.py", line 610, in __init__
    field_transformer,
  File "/home/didi/leaf/leaf-radio/.venv/lib/python3.7/site-packages/attr/_make.py", line 503, in _transform_attrs
    AttrsClass = _make_attr_tuple_class(cls.__name__, attr_names)
  File "/home/didi/leaf/leaf-radio/.venv/lib/python3.7/site-packages/attr/_make.py", line 299, in _make_attr_tuple_class
    eval(compile("\n".join(attr_class_template), "", "exec"), globs)
  File "", line 1, in <module>
  File "/home/didi/leaf/leaf-radio/.venv/lib/python3.7/site-packages/scalene/scalene_profiler.py", line 231, in invalidate_lines
    if f.f_code.co_filename[0] == "<" or "scalene" in f.f_code.co_filename:
IndexError: string index out of range

Commenting out the line import trio, makes everything works.

However, the line Scalene: internal error: unable to find Python allocator functions is always present.

Desktop (please complete the following information):

  • OS: WSL2
  • Python version: 3.7.10, installed with pyenv
  • Scalene version: 1.3.15

About this issue

  • Original URL
  • State: closed
  • Created 3 years ago
  • Comments: 19 (4 by maintainers)

Most upvoted comments

Update: we believe we have finally figured out what’s going on and hope to have a fix out soon!

This should be fixed now (in yesterday’s release). Please re-open if it’s not.

If anybody else has been following this issue and wants to know a bit more about the details of the fix, I believe these are the most important changes between v1.3.15 and v1.3.16

  • c9e2bf2 - handle empty filename edge case
  • #286 - only call PyMem_*() if all symbols are found
  • #287 - introduces check for Python

Running on the current HEAD of scalene’s master branch I am not seeing any issues, please try downloading Scalene’s source and adding it to pyproject.toml in the tool.poetry.dependencies section with scalene = { path = "/path/to/scalene/", develop = true } and post here if the error continues.