I've been investigating why CPython test suite apparently hangs when run on a Linux server with 512 GB RAM and traced it to os.sync() call in test_posix.testNoArgFunctions(). If there are many dirty pages cached in RAM and the storage device is relatively slow (HDD in my case), sync() can take a lot of time. Worse, it puts the process into "uninterruptible sleep" state, so it's not possible to terminate the test worker process even with SIGKILL.
I think that grouping innocuous functions like os.getpid() and os.getuid() together with os.sync() as "not having side effects" is unwise, since os.sync() can have huge side effects (on a lower abstraction level). I propose to test os.sync() only if some resource is explicitly granted to the test system via --use option.
I don't see an exact match in the list of currently supported resources. The most appropriate one seems to be largefile: even though os.sync() doesn't create any files, it might cause heavy blocking I/O. I'm going to submit a PR making os.sync() test conditional on largefile.
Alternatively, we could add another resource (io?), but I'm not sure it's worth adding a new resource just for this case.
Linked PRs
I've been investigating why CPython test suite apparently hangs when run on a Linux server with 512 GB RAM and traced it to
os.sync()call in test_posix.testNoArgFunctions(). If there are many dirty pages cached in RAM and the storage device is relatively slow (HDD in my case),sync()can take a lot of time. Worse, it puts the process into "uninterruptible sleep" state, so it's not possible to terminate the test worker process even with SIGKILL.I think that grouping innocuous functions like
os.getpid()andos.getuid()together withos.sync()as "not having side effects" is unwise, sinceos.sync()can have huge side effects (on a lower abstraction level). I propose to testos.sync()only if some resource is explicitly granted to the test system via--useoption.I don't see an exact match in the list of currently supported resources. The most appropriate one seems to be
largefile: even thoughos.sync()doesn't create any files, it might cause heavy blocking I/O. I'm going to submit a PR makingos.sync()test conditional onlargefile.Alternatively, we could add another resource (
io?), but I'm not sure it's worth adding a new resource just for this case.Linked PRs