serenity: Ports/Python3: crash in child after fork

courage:~ $ python3
Python 3.13.0 (main, Nov  3 2024, 11:49:09) [GCC 13.2.0] on serenityos
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> os.getpid()
44
>>> os.fork()
45
>>> Fatal Python error: _PyRecursiveMutex_Unlock: unlocking a recursive mutex th
at is not owned by the current thread
Python runtime state: initialized

Current thread 0x000000000000002c (most recent call first):
  File "<stdin>-2", line 1 in <module>
  File "/usr/local/lib/python3.13/_pyrepl/main.py", line 30 in interactive_conso
le
  File "/usr/local/lib/python3.13/_pyrepl/__main__.py", line 6 in <module>
  File "<frozen runpy>", line 88 in _run_code
  File "<frozen runpy>", line 198 in _run_module_as_main

>>> os.getpid()
44
>>> 

Backtrace:

==== Thread #0 (TID 45) ====
0x00000019c923502b: [/usr/lib/libsystem.so] syscall2 +0xb (syscall.cpp:25 => syscall.cpp:24)
0x0000001ddcb6a45d: [/usr/lib/libc.so] abort +0x26 (stdlib.cpp:387)
0x0000000d3b3f5303: [/usr/local/lib/libpython3.13.so.1.0] fatal_error.cold +0x4 (pylifecycle.c:3032 => pylifecycle.c:3178)
0x0000000d3b623a33: [/usr/local/lib/libpython3.13.so.1.0] _Py_FatalErrorFunc +0x33 (pylifecycle.c:3263)
0x0000000d3b6133fe: [/usr/local/lib/libpython3.13.so.1.0] _PyRecursiveMutex_Unlock +0x7e (lock.c:385)
0x0000000d3b668679: [/usr/local/lib/libpython3.13.so.1.0] PyOS_AfterFork_Child.localalias +0x89 (posixmodule.c:680)
0x0000000d3b6689b6: [/usr/local/lib/libpython3.13.so.1.0] os_fork +0xc6 (posixmodule.c:8067)
0x0000000d3b4ca5e0: [/usr/local/lib/libpython3.13.so.1.0] cfunction_vectorcall_NOARGS +0x70 (methodobject.h:50)
0x0000000d3b463227: [/usr/local/lib/libpython3.13.so.1.0] PyObject_Vectorcall +0x57 (pycore_call.h:168)
0x0000000d3b3f74df: [/usr/local/lib/libpython3.13.so.1.0] _PyEval_EvalFrameDefault +0x1bff (generated_cases.c.h:813)
0x0000000d3b5bf8fc: [/usr/local/lib/libpython3.13.so.1.0] PyEval_EvalCode +0x24c (pycore_ceval.h:119 => ceval.c:1806)
0x0000000d3b62948f: [/usr/local/lib/libpython3.13.so.1.0] run_eval_code_obj +0x7f (pythonrun.c:1323)
0x0000000d3b629735: [/usr/local/lib/libpython3.13.so.1.0] run_mod +0x185 (pythonrun.c:1408)
0x0000000d3b629bc6: [/usr/local/lib/libpython3.13.so.1.0] PyRun_InteractiveOneObjectEx +0x196 (pythonrun.c:282)
0x0000000d3b62ba45: [/usr/local/lib/libpython3.13.so.1.0] _PyRun_InteractiveLoopObject +0xa5 (pythonrun.c:130)
0x0000000d3b62c175: [/usr/local/lib/libpython3.13.so.1.0] PyRun_AnyFileExFlags +0xb5 (pythonrun.c:71)
0x0000000d3b63a1be: [/usr/local/lib/libpython3.13.so.1.0] sys__baserepl +0x2e (sysmodule.c:2428)
0x0000000d3b4ca5e0: [/usr/local/lib/libpython3.13.so.1.0] cfunction_vectorcall_NOARGS +0x70 (methodobject.h:50)
0x0000000d3b463227: [/usr/local/lib/libpython3.13.so.1.0] PyObject_Vectorcall +0x57 (pycore_call.h:168)
0x0000000d3b3f74df: [/usr/local/lib/libpython3.13.so.1.0] _PyEval_EvalFrameDefault +0x1bff (generated_cases.c.h:813)
0x0000000d3b5bf8fc: [/usr/local/lib/libpython3.13.so.1.0] PyEval_EvalCode +0x24c (pycore_ceval.h:119 => ceval.c:1806)
0x0000000d3b5b9f0f: [/usr/local/lib/libpython3.13.so.1.0] builtin_exec +0x41f (bltinmodule.c:1145)
0x0000000d3b4ca217: [/usr/local/lib/libpython3.13.so.1.0] cfunction_vectorcall_FASTCALL_KEYWORDS +0x67 (methodobject.h:50)
0x0000000d3b463227: [/usr/local/lib/libpython3.13.so.1.0] PyObject_Vectorcall +0x57 (pycore_call.h:168)
0x0000000d3b3f74df: [/usr/local/lib/libpython3.13.so.1.0] _PyEval_EvalFrameDefault +0x1bff (generated_cases.c.h:813)
0x0000000d3b6531a1: [/usr/local/lib/libpython3.13.so.1.0] pymain_run_module +0xd1 (main.c:349)
0x0000000d3b654424: [/usr/local/lib/libpython3.13.so.1.0] pymain_run_python.constprop.0 +0xfa4 (main.c:574)
0x0000000d3b6547d6: [/usr/local/lib/libpython3.13.so.1.0] Py_BytesMain +0x56 (main.c:775)
0x00000005603c4654: [/usr/local/bin/python3.13] _entry +0x24 (crt0.cpp:47)

About this issue

  • Original URL
  • State: closed
  • Created 2 years ago
  • Comments: 16 (13 by maintainers)

Commits related to this issue

Most upvoted comments

Does Serenity preserve thread local storage after fork?

Yes, thread local storage is cloned. Instead of storing pthread data structures in %fs or other libc-allocated data, Serenity’s pthread APIs mostly just map to querying the kernel about its first-class Thread classes that each Process class owns, using the thread ID.

Thread-local storage is allocated and stored in %fs and other thread-pointer-registers. Just not the “info about the current thread” structure musl libc and glibc have.

fork in Python: confirmed working
threads in Python: No regression found with threads. A threading.Lock that is locked by the parent can still be unlocked by the child after fork even though threading.get_ident() returns different values.

thread_test.py.txt thread-test-Linux.txt (not 3.13) thread-test-SerenityOS.txt

It seems that at least Python 3.13 relies on pthread_self returning the same value in both the parent and the child process. PyThread_get_thread_ident_ex

This is the case on: Linux, Hurd, FreeBSD, OpenBSD, NetBSD, MidnightBSD, Cygwin, Haiku, newer versions of Solaris, and macOS But not on SerenityOS

#include <stdio.h>
#include <pthread.h>
#include <unistd.h>

int main(int argc, char **argv)
{
    pthread_t parent = pthread_self();
    if (fork() == 0) {
        pthread_t child = pthread_self();
        if (pthread_equal(parent, child))
            puts("parent == child");
        else
            puts("parent != child");
    }
    return 0;
}

Curiously, POSIX doesn’t seem to mention if the value returned by pthread_self should be inherited by the child process.
pthread_self fork

Yes, we can fix this upstream in CPython. Can you open an issue in https://github.com/python/cpython? You can tag me in it.

I did, and it is. Please read the code the traceback is pointing to: https://github.com/python/cpython/blob/c5b99f5c2c5347d66b9da362773969c531fb6c85/Lib/_pyrepl/main.py#L30