Repository navigation
Implement stop-the-world functionality (for --disable-gil builds) #111964
Description
Activity
Is someone already working on this? @colesbury
- added a commit that references this issue
on Nov 27, 2023 - added a commit that references this issue
on Dec 18, 2023 @markshannon: Carl mentioned you had a question about the stop-the-world implementation and reentrancy. The current implementation in the linked PR does not support reentrancy. Prior implementations in the nogil fork supported reentrancy and behaved like recursive mutexes: only the "outer most" calls stopped or resumed threads; "inner" calls only increment/decrement a counter.
The only place reentrancy was useful was during runtime finalization (i.e., interpreter shutdown). The stop-the-world mechanism is useful for forcing daemon threads 1 to a consistent state right before we delete their thread states, but we also trigger the GC later during shutdown. Making the stop-the-world calls reentrant was a convenient "hack", but I think there are other ways to achieve the same effect.
I don't think reentrant stop-the-world calls are generally useful because you are quite restricted on what's safe to call during a stop-the-world pause or you risk deadlock. Other paused threads may be holding the locks you need, so you can't call arbitrary python code or destructors. The GC is (or will be) careful to only call finalizers and weakref callbacks after resuming other threads.
Footnotes
-
and potentially non-daemon threads depending on when you press ctrl-c ↩
-
Feature or enhancement
The
--disable-gilbuilds occasionally need to pause all but one thread. Some examples include:fork(), to ensure a consistent state for internal data structuresIn the
nogil-3.12fork, a stop-the-world call paused all threads in all interpreters. In CPython 3.13, we probably want to provide two levels of "stop-the-world": per-interpreter and global. In general, per-interpreter pauses are preferable to global pauses, but there are some cases (like beforefork()) where want all threads to pause, so that we don'tfork()while a thread is modifying some global runtime structure.In the default build, stop-the-world calls should generally be no-ops, but global pauses may be useful in a few cases, such as before
fork()when using multiple interpreters each with their own "GIL".In the
nogil-3.12fork, this was implemented together with other GC modifications: colesbury/nogil-3.12@2864b6b36e. I'd like to implement it separately in 3.13 to keep the PRs smaller.Linked PRs