Bug report

I found a race condition happening in the rich comparison function for list where calling a rich comparison on items, which may release the GIL, may result in the list being mutated and the wrong items being compared later on in list's rich comparison.

The current code iterates through the list's ob_item to find the index of the first non-equal objects. It then calls PyObject_RichCompareBool on the two objects (which may call user-defined Python code, which may release the GIL), then checks whether the rich comparison said the items were different.
At this point, what the items exactly were is forgotten (vitem and witem go out of scope).

Later in the rich comparison function, if the desired comparison operation is not EQ/NEQ, a new call to PyObject_RichCompare is made to know which is larger/smaller. To do this, vitem and witem are re-read from the ob_item, but ob_item may actually have changed in the meantime, meaning the return value may be incorrect.

This is in a way similar to my other issue #148259 -- the root cause is mostly the same (re-reading a previously-read value that may have changed in the meantime).

I have a branch with a proposed fix, although there may be a better way as this one requires to add a lot of refcount changes which makes it all a bit hard to read / do bookkeeping for.

Note I don't currently have a reproducer, but I may try to make one if needed.

CPython versions tested on:

CPython main branch

Operating systems tested on:

macOS

Linked PRs

• gh-148442: eliminate race condition in list rich comparison #148531