Skip to content

Commit 1d95a04

Browse files
youknowoneyouknowone
authored
Update test/libregrtest to CPython 3.13.10 (#6410)
* Update test/libregrtest to CPython 3.13.10 * Re-add try block around platform and encodings info * Disable WindowsLoadTracker on Windows * Disable struct._clearcache() * Disable saved_test_environment context manager * Disable support.record_original_stdout() * Update test/test_regrtest.py to CPython 3.13.11 * Add autotest shim from CPython 3.13.11 * Mark failing tests and skip flaky ones in test/test_regrtest.py * Mark failing test in test_fstring * Add _test_eintr.py from CPython 3.13.11 * Skip long-running test in _test_eintr * Mark failing/erroring tests in _test_eintr * Skip panicking tests in test_io * Unmark passing tests in test_io * Mark failing test in test_urllib2net * Unmark passing test in test_fstring * Time out if any one test takes longer than 10 minutes * Skip more flaky tests in test_threading * Skip tests that pollute the environment for some reason * Skip flaky tests in test.test_multiprocessing_fork.test_misc * Skip flaky tests in test.test_multiprocessing_fork.test_manager Had to go with the nuclear option. * Skip flaky tests in test_regrtest * Mark flaky tests in test_ftplib * Fix multiprocessing tests on Darwin * Remove test_sys from list of environment polluters * Remove test_ftplib from list of environment polluters * Add test.test_multiprocessing_fork.test_misc to list of environment polluters * Check 10 times if a test is still polluting the environment * Save more of the test environment before a test is run * Mark test_import as polluting the environment * Unmark passing test in test_regrtest * Mark flaky test in test_regrtest * Remove tests from list of environment polluters * Skip flaky test in test_logging * Remove tests from list of polluting tests * Skip certain problematic tests in test_threading * Skip test causing worker bug in test_builtin * Make env polluter skip in test_threading os-agnostic * Remove test_file_eintr from list of polluters * Remove test_subprocess from the list of polluters * Address coderabbit comments * fix threading._dangling * Skip environment polluter in test.test_concurrent_futures.test_thread_pool * Make skip in test_thread_pool os-agnostic * Skip problematic tests in test_wait * Skip flaky tests in test_process_pool * Remove test_import from list of polluters * Amend skips in test.test_multiprocessing_forkserver.test_processes * Remove test_raise from list of polluters * Remove test_ssl from list of polluters * Skip flaky test in test_ssl * Fix YAML formatting issue --------- Co-authored-by: Jeong, YunWon <jeong@youknowone.org>
2 parents 440b8de + 1dcded4 commit 1d95a04

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

47 files changed

+6947
-1854
lines changed

‎.github/workflows/ci.yaml‎

Lines changed: 124 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -105,6 +105,23 @@ env:
105105
test_unpack
106106
test_weakref
107107
test_yield_from
108+
ENV_POLLUTING_TESTS_COMMON: >-
109+
test_threading
110+
ENV_POLLUTING_TESTS_LINUX: >-
111+
test.test_concurrent_futures.test_thread_pool
112+
test.test_multiprocessing_fork.test_processes
113+
test.test_multiprocessing_fork.test_threads
114+
test.test_multiprocessing_forkserver.test_processes
115+
test.test_multiprocessing_forkserver.test_threads
116+
test.test_multiprocessing_spawn.test_processes
117+
test.test_multiprocessing_spawn.test_threads
118+
ENV_POLLUTING_TESTS_MACOS: >-
119+
test.test_concurrent_futures.test_thread_pool
120+
test.test_multiprocessing_forkserver.test_processes
121+
test.test_multiprocessing_forkserver.test_threads
122+
test.test_multiprocessing_spawn.test_processes
123+
test.test_multiprocessing_spawn.test_threads
124+
ENV_POLLUTING_TESTS_WINDOWS: >-
108125
# Python version targeted by the CI.
109126
PYTHON_VERSION: "3.13.1"
110127
X86_64_PC_WINDOWS_MSVC_OPENSSL_LIB_DIR: C:\Program Files\OpenSSL\lib\VC\x64\MD
@@ -116,7 +133,7 @@ jobs:
116133
env:
117134
RUST_BACKTRACE: full
118135
name: Run rust tests
119-
runs-on: ${{ matrix.os }}
136+
runs-on: ${{ matrix.os }}
120137
timeout-minutes: 45
121138
strategy:
122139
matrix:
@@ -147,8 +164,7 @@ jobs:
147164
run: cargo check ${{ env.CARGO_ARGS }}
148165

149166
- name: Test openssl build
150-
run:
151-
cargo build --no-default-features --features ssl-openssl
167+
run: cargo build --no-default-features --features ssl-openssl
152168
if: runner.os == 'Linux'
153169

154170
# - name: Install tk-dev for tkinter build
@@ -160,7 +176,7 @@ jobs:
160176
# if: runner.os == 'Linux'
161177

162178
- name: Test example projects
163-
run:
179+
run: |
164180
cargo run --manifest-path example_projects/barebone/Cargo.toml
165181
cargo run --manifest-path example_projects/frozen_stdlib/Cargo.toml
166182
if: runner.os == 'Linux'
@@ -248,7 +264,7 @@ jobs:
248264
env:
249265
RUST_BACKTRACE: full
250266
name: Run snippets and cpython tests
251-
runs-on: ${{ matrix.os }}
267+
runs-on: ${{ matrix.os }}
252268
strategy:
253269
matrix:
254270
os: [macos-latest, ubuntu-latest, windows-2025]
@@ -278,26 +294,116 @@ jobs:
278294

279295
- if: runner.os == 'Linux'
280296
name: run cpython platform-independent tests
281-
run:
282-
target/release/rustpython -m test -j 1 -u all --slowest --fail-env-changed -v ${{ env.PLATFORM_INDEPENDENT_TESTS }}
297+
env:
298+
RUSTPYTHON_SKIP_ENV_POLLUTERS: true
299+
run: target/release/rustpython -m test -j 1 -u all --slowest --fail-env-changed --timeout 600 -v ${{ env.PLATFORM_INDEPENDENT_TESTS }}
283300
timeout-minutes: 35
284301

285302
- if: runner.os == 'Linux'
286303
name: run cpython platform-dependent tests (Linux)
287-
run: target/release/rustpython -m test -j 1 -u all --slowest --fail-env-changed -v -x ${{ env.PLATFORM_INDEPENDENT_TESTS }}
304+
env:
305+
RUSTPYTHON_SKIP_ENV_POLLUTERS: true
306+
run: target/release/rustpython -m test -j 1 -u all --slowest --fail-env-changed --timeout 600 -v -x ${{ env.PLATFORM_INDEPENDENT_TESTS }}
288307
timeout-minutes: 35
289308

290309
- if: runner.os == 'macOS'
291310
name: run cpython platform-dependent tests (MacOS)
292-
run: target/release/rustpython -m test -j 1 --slowest --fail-env-changed -v -x ${{ env.PLATFORM_INDEPENDENT_TESTS }}
311+
env:
312+
RUSTPYTHON_SKIP_ENV_POLLUTERS: true
313+
run: target/release/rustpython -m test -j 1 --slowest --fail-env-changed --timeout 600 -v -x ${{ env.PLATFORM_INDEPENDENT_TESTS }}
293314
timeout-minutes: 35
294315

295316
- if: runner.os == 'Windows'
296317
name: run cpython platform-dependent tests (windows partial - fixme)
297-
run:
298-
target/release/rustpython -m test -j 1 --slowest --fail-env-changed -v -x ${{ env.PLATFORM_INDEPENDENT_TESTS }} ${{ env.WINDOWS_SKIPS }}
318+
env:
319+
RUSTPYTHON_SKIP_ENV_POLLUTERS: true
320+
run: target/release/rustpython -m test -j 1 --slowest --fail-env-changed --timeout 600 -v -x ${{ env.PLATFORM_INDEPENDENT_TESTS }} ${{ env.WINDOWS_SKIPS }}
299321
timeout-minutes: 45
300322

323+
- if: runner.os == 'Linux'
324+
name: run cpython tests to check if env polluters have stopped polluting (Common/Linux)
325+
shell: bash
326+
run: |
327+
for thing in ${{ env.ENV_POLLUTING_TESTS_COMMON }} ${{ env.ENV_POLLUTING_TESTS_LINUX }}; do
328+
for i in $(seq 1 10); do
329+
set +e
330+
target/release/rustpython -m test -j 1 --slowest --fail-env-changed --timeout 600 -v ${thing}
331+
exit_code=$?
332+
set -e
333+
if [ ${exit_code} -eq 3 ]; then
334+
echo "Test ${thing} polluted the environment on attempt ${i}."
335+
break
336+
fi
337+
done
338+
if [ ${exit_code} -ne 3 ]; then
339+
echo "Test ${thing} is no longer polluting the environment after ${i} attempts!"
340+
echo "Please remove ${thing} from either ENV_POLLUTING_TESTS_COMMON or ENV_POLLUTING_TESTS_LINUX in '.github/workflows/ci.yaml'."
341+
echo "Please also remove the skip decorators that include the word 'POLLUTERS' in ${thing}."
342+
if [ ${exit_code} -ne 0 ]; then
343+
echo "Test ${thing} failed with exit code ${exit_code}."
344+
echo "Please investigate which test item in ${thing} is failing and either mark it as an expected failure or a skip."
345+
fi
346+
exit 1
347+
fi
348+
done
349+
timeout-minutes: 15
350+
351+
- if: runner.os == 'macOS'
352+
name: run cpython tests to check if env polluters have stopped polluting (Common/macOS)
353+
shell: bash
354+
run: |
355+
for thing in ${{ env.ENV_POLLUTING_TESTS_COMMON }} ${{ env.ENV_POLLUTING_TESTS_MACOS }}; do
356+
for i in $(seq 1 10); do
357+
set +e
358+
target/release/rustpython -m test -j 1 --slowest --fail-env-changed --timeout 600 -v ${thing}
359+
exit_code=$?
360+
set -e
361+
if [ ${exit_code} -eq 3 ]; then
362+
echo "Test ${thing} polluted the environment on attempt ${i}."
363+
break
364+
fi
365+
done
366+
if [ ${exit_code} -ne 3 ]; then
367+
echo "Test ${thing} is no longer polluting the environment after ${i} attempts!"
368+
echo "Please remove ${thing} from either ENV_POLLUTING_TESTS_COMMON or ENV_POLLUTING_TESTS_MACOS in '.github/workflows/ci.yaml'."
369+
echo "Please also remove the skip decorators that include the word 'POLLUTERS' in ${thing}."
370+
if [ ${exit_code} -ne 0 ]; then
371+
echo "Test ${thing} failed with exit code ${exit_code}."
372+
echo "Please investigate which test item in ${thing} is failing and either mark it as an expected failure or a skip."
373+
fi
374+
exit 1
375+
fi
376+
done
377+
timeout-minutes: 15
378+
379+
- if: runner.os == 'Windows'
380+
name: run cpython tests to check if env polluters have stopped polluting (Common/windows)
381+
shell: bash
382+
run: |
383+
for thing in ${{ env.ENV_POLLUTING_TESTS_COMMON }} ${{ env.ENV_POLLUTING_TESTS_WINDOWS }}; do
384+
for i in $(seq 1 10); do
385+
set +e
386+
target/release/rustpython -m test -j 1 --slowest --fail-env-changed --timeout 600 -v ${thing}
387+
exit_code=$?
388+
set -e
389+
if [ ${exit_code} -eq 3 ]; then
390+
echo "Test ${thing} polluted the environment on attempt ${i}."
391+
break
392+
fi
393+
done
394+
if [ ${exit_code} -ne 3 ]; then
395+
echo "Test ${thing} is no longer polluting the environment after ${i} attempts!"
396+
echo "Please remove ${thing} from either ENV_POLLUTING_TESTS_COMMON or ENV_POLLUTING_TESTS_WINDOWS in '.github/workflows/ci.yaml'."
397+
echo "Please also remove the skip decorators that include the word 'POLLUTERS' in ${thing}."
398+
if [ ${exit_code} -ne 0 ]; then
399+
echo "Test ${thing} failed with exit code ${exit_code}."
400+
echo "Please investigate which test item in ${thing} is failing and either mark it as an expected failure or a skip."
401+
fi
402+
exit 1
403+
fi
404+
done
405+
timeout-minutes: 15
406+
301407
- if: runner.os != 'Windows'
302408
name: check that --install-pip succeeds
303409
run: |
@@ -323,7 +429,7 @@ jobs:
323429
- uses: actions/checkout@v6.0.1
324430
- uses: dtolnay/rust-toolchain@stable
325431
with:
326-
components: clippy
432+
components: clippy
327433

328434
- name: run clippy on wasm
329435
run: cargo clippy --manifest-path=crates/wasm/Cargo.toml -- -Dwarnings
@@ -360,7 +466,7 @@ jobs:
360466
- name: spell checker
361467
uses: streetsidesoftware/cspell-action@v8
362468
with:
363-
files: '**/*.rs'
469+
files: "**/*.rs"
364470
incremental_files_only: true
365471

366472
miri:
@@ -375,17 +481,17 @@ jobs:
375481

376482
- uses: dtolnay/rust-toolchain@master
377483
with:
378-
toolchain: ${{ env.NIGHTLY_CHANNEL }}
379-
components: miri
484+
toolchain: ${{ env.NIGHTLY_CHANNEL }}
485+
components: miri
380486

381487
- uses: Swatinem/rust-cache@v2
382488

383489
- name: Run tests under miri
384490
run: cargo +${{ env.NIGHTLY_CHANNEL }} miri test -p rustpython-vm -- miri_test
385491
env:
386-
# miri-ignore-leaks because the type-object circular reference means that there will always be
387-
# a memory leak, at least until we have proper cyclic gc
388-
MIRIFLAGS: '-Zmiri-ignore-leaks'
492+
# miri-ignore-leaks because the type-object circular reference means that there will always be
493+
# a memory leak, at least until we have proper cyclic gc
494+
MIRIFLAGS: "-Zmiri-ignore-leaks"
389495

390496
wasm:
391497
if: ${{ !contains(github.event.pull_request.labels.*.name, 'skip:ci') }}

‎Lib/test/__main__.py‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,2 +1,2 @@
1-
from test.libregrtest import main
2-
main()
1+
from test.libregrtest.main import main
2+
main(_add_python_opts=True)

0 commit comments

Comments
 (0)