Skip to content

os.sync() in test_posix can uninterruptibly hang tests for minutes #102184

Description

@izbyshev

I've been investigating why CPython test suite apparently hangs when run on a Linux server with 512 GB RAM and traced it to os.sync() call in test_posix.testNoArgFunctions(). If there are many dirty pages cached in RAM and the storage device is relatively slow (HDD in my case), sync() can take a lot of time. Worse, it puts the process into "uninterruptible sleep" state, so it's not possible to terminate the test worker process even with SIGKILL.

I think that grouping innocuous functions like os.getpid() and os.getuid() together with os.sync() as "not having side effects" is unwise, since os.sync() can have huge side effects (on a lower abstraction level). I propose to test os.sync() only if some resource is explicitly granted to the test system via --use option.

I don't see an exact match in the list of currently supported resources. The most appropriate one seems to be largefile: even though os.sync() doesn't create any files, it might cause heavy blocking I/O. I'm going to submit a PR making os.sync() test conditional on largefile.

Alternatively, we could add another resource (io?), but I'm not sure it's worth adding a new resource just for this case.

Linked PRs

Metadata

Metadata

Assignees

No one assigned

    Labels

    performancePerformance or resource usagetestsTests in the Lib/test dirtype-bugAn unexpected behavior, bug, or error

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions