#155619

Reproduction

gh-155619 was reported from source inspection without a reproducer, so here is one first. The new test registers a watcher whose callback does nothing, then races context switches against add/clear. On unpatched main it aborts every time:

$ ./python.exe -m test test_free_threading.test_context_watcher
Assertion failed: (cb != NULL), function notify_context_watchers, file context.c
Fatal Python error: Aborted

$ PYTHON_GIL=1 ./python.exe -m test test_free_threading.test_context_watcher
== Tests result: SUCCESS ==   (2 tests, 33 ms)

Same binary, only the GIL toggled.

There are three races. PyContext_AddWatcher() claimed a slot with a plain test-then-store, so two threads could take the same slot and get the same watcher ID. It then stored the callback and set the active bit as ordinary writes, so a notifier on a weakly ordered machine could see the bit before the pointer. And notify_context_watchers() reads the bitmask and then loads the slot, which PyContext_ClearWatcher() can empty in between — that one is a plain interleaving, so no amount of memory ordering fixes it.

Fix

The fix claims slots with _Py_atomic_compare_exchange_ptr(), publishes the callback with a release store paired with an acquire load in notification, and skips a NULL callback instead of asserting on it, same as _PyDict_SendEvent(). No locks: notification runs on every context switch and does no atomic read-modify-write there. I kept the bitmask rather than scanning all eight slots because it makes the common zero-watcher case one load and a not-taken branch.

Docs

Doc/c-api/contextvars.rst had no threading guidance at all, unlike dict and type watchers, so I added some.

Testing

The new test passes 20/20 runs; test_free_threading, test_context, test_capi, test_contextlib and test_asyncio are clean on a free-threaded build (4735 tests), and the same binary with PYTHON_GIL=1 is clean too.

Benchmarks

pyperformance async_tree, async_tree_io, async_generators (asyncio runs each task step through ctx.run()), plus nbody as a control that never enters a context. Free-threaded build, baseline vs. this change:

The default build was the same story, except async_generators came out 1.08x slower there. I think that one is noise: its spread is about 12% of the mean, t=-2.54, and the same benchmark went faster on the free-threaded build. These were plain -O3 builds with --fast on a laptop, so they answer "does this cost anything" and not much else; happy to redo them properly if it matters.

No ThreadSanitizer run from me, though test_free_threading is in TSAN_TESTS, so CI should cover the new test.

Approach discussed with @corona10 at the PyCon KR sprint.