Skip to content

Commit d7e41eb

Browse files
youknowoneyouknowone
authored
Split CI cargo run and test into build then execute (#8705)
* Run gc_collect ten times per benchmark iteration Assisted-by: Grok:4.6 * Reuse the prebuilt rustpython for CI whats_left whats_left was invoking cargo build --release without threading, so the snippets job rebuilt the interpreter after the earlier release build. Assisted-by: Grok:4.6 * Split CI cargo run and test into build then execute Assisted-by: Grok:4.6 * Compile doctests in the CI build step cargo test --no-run skips doctests, so rustdoc compilation was still happening in the run step. Assisted-by: Grok:4.6 * Drop cron-ci.yaml from this PR A pull_request path filter matches the whole PR, so every push was starting the periodic benchmark and coverage jobs. Assisted-by: Grok:4.6 * Run doctests in their own CI step cargo test --doc --no-run is rejected, so doctests cannot be compiled without also executing them. Assisted-by: Grok:4.6 * Do not run benchmark targets in rust tests --all-targets includes Criterion benches, which cargo test does not run by default. Assisted-by: Grok:4.6
1 parent a7217a0 commit d7e41eb

3 files changed

Lines changed: 84 additions & 27 deletions

File tree

‎.github/workflows/ci.yaml‎

Lines changed: 52 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -125,14 +125,34 @@ jobs:
125125
- name: Install macOS dependencies
126126
uses: ./.github/actions/install-macos-deps
127127

128+
- name: Build rust tests
129+
run: cargo test --no-run --locked --workspace --exclude rustpython-capi ${{ env.WORKSPACE_EXCLUDES }} --features threading ${{ env.CARGO_ARGS }}
130+
env:
131+
INSTA_WORKSPACE_ROOT: ${{ github.workspace }}
132+
128133
- name: run rust tests
129-
run: cargo test --locked --workspace --exclude rustpython-capi ${{ env.WORKSPACE_EXCLUDES }} --features threading ${{ env.CARGO_ARGS }}
134+
run: cargo test --locked --workspace --exclude rustpython-capi ${{ env.WORKSPACE_EXCLUDES }} --features threading ${{ env.CARGO_ARGS }} --lib --bins --tests
135+
env:
136+
INSTA_WORKSPACE_ROOT: ${{ github.workspace }}
137+
138+
- name: Run rust doctests
139+
run: cargo test --doc --locked --workspace --exclude rustpython-capi ${{ env.WORKSPACE_EXCLUDES }} --features threading ${{ env.CARGO_ARGS }}
130140
env:
131141
INSTA_WORKSPACE_ROOT: ${{ github.workspace }}
132142

143+
- name: Build c-api tests
144+
working-directory: crates/capi
145+
run: cargo test --no-run --locked
146+
if: runner.os != 'Windows' # Requires pyo3 0.29+ on Windows
147+
133148
- name: run c-api tests
134149
working-directory: crates/capi
135-
run: cargo test --locked
150+
run: cargo test --locked --lib --bins --tests
151+
if: runner.os != 'Windows' # Requires pyo3 0.29+ on Windows
152+
153+
- name: Run c-api doctests
154+
working-directory: crates/capi
155+
run: cargo test --doc --locked
136156
if: runner.os != 'Windows' # Requires pyo3 0.29+ on Windows
137157

138158
- name: check compilation without host_env (sandbox mode)
@@ -164,14 +184,24 @@ jobs:
164184
# run: cargo build --features tkinter
165185
# if: runner.os == 'Linux'
166186

167-
- name: Test example projects
187+
- name: Build example projects
188+
run: |
189+
cargo build --manifest-path example_projects/barebone/Cargo.toml
190+
cargo build --manifest-path example_projects/frozen_stdlib/Cargo.toml
191+
if: runner.os == 'Linux'
192+
193+
- name: Run example projects
168194
run: |
169-
cargo run --manifest-path example_projects/barebone/Cargo.toml
170-
cargo run --manifest-path example_projects/frozen_stdlib/Cargo.toml
195+
example_projects/barebone/target/debug/example-barebone
196+
example_projects/frozen_stdlib/target/debug/example_frozen_stdlib
197+
if: runner.os == 'Linux'
198+
199+
- name: Build rustpython for update_lib tests
200+
run: cargo build --locked
171201
if: runner.os == 'Linux'
172202

173203
- name: run update_lib tests
174-
run: cargo run --locked -- -m unittest discover -s scripts/update_lib/tests -v
204+
run: target/debug/rustpython -m unittest discover -s scripts/update_lib/tests -v
175205
env:
176206
PYTHONPATH: scripts
177207
if: runner.os == 'Linux'
@@ -487,7 +517,7 @@ jobs:
487517
488518
- name: Check whats_left is not broken
489519
shell: bash
490-
run: python -I scripts/whats_left.py ${{ env.CARGO_ARGS }} --features jit
520+
run: python -I scripts/whats_left.py --rustpython target/release/rustpython
491521

492522
clippy:
493523
# Named after the matrix entry rather than left to be named for it: a
@@ -681,6 +711,11 @@ jobs:
681711
restore-keys: |
682712
${{ runner.os }}-
683713
714+
- name: Build tests under miri
715+
run: cargo +${{ env.NIGHTLY_CHANNEL }} miri test --no-run --locked -p rustpython-vm -- miri_test
716+
env:
717+
MIRIFLAGS: "-Zmiri-ignore-leaks -Zmiri-permissive-provenance"
718+
684719
- name: Run tests under miri
685720
run: cargo +${{ env.NIGHTLY_CHANNEL }} miri test --locked -p rustpython-vm -- miri_test
686721
env:
@@ -769,15 +804,20 @@ jobs:
769804
with:
770805
wabt-version: "1.0.36"
771806

772-
- name: check wasm32-unknown without js
807+
- name: Build wasm32-unknown without js
808+
working-directory: example_projects/wasm32_without_js/rustpython-without-js
809+
run: cargo build
810+
811+
- name: Build wasm-runtime
812+
run: cargo build --release --manifest-path example_projects/wasm32_without_js/wasm-runtime/Cargo.toml
813+
814+
- name: Check wasm32-unknown without js
815+
working-directory: example_projects/wasm32_without_js
773816
run: |
774-
cd example_projects/wasm32_without_js/rustpython-without-js
775-
cargo build
776-
cd ..
777817
if wasm-objdump -xj Import rustpython-without-js/target/wasm32-unknown-unknown/debug/rustpython_without_js.wasm; then
778818
echo "ERROR: wasm32-unknown module expects imports from the host environment" >&2
779819
fi
780-
cargo run --release --manifest-path wasm-runtime/Cargo.toml rustpython-without-js/target/wasm32-unknown-unknown/debug/rustpython_without_js.wasm
820+
wasm-runtime/target/release/wasm-runtime rustpython-without-js/target/wasm32-unknown-unknown/debug/rustpython_without_js.wasm
781821
782822
- name: build notebook demo
783823
if: github.ref == 'refs/heads/release'

‎benches/benchmarks/gc_collect.py‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -52,4 +52,4 @@ def benchamark_collection(loops, cycles, links):
5252

5353
assert collected is None or collected >= cycles * (links + 1)
5454

55-
benchamark_collection(1, CYCLES, LINKS)
55+
benchamark_collection(10, CYCLES, LINKS)

‎scripts/whats_left.py‎

Lines changed: 31 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -71,6 +71,10 @@ def parse_args():
7171
help="which features to enable when building RustPython (default: [])",
7272
default=[],
7373
)
74+
parser.add_argument(
75+
"--rustpython",
76+
help="use this RustPython executable instead of building one with cargo",
77+
)
7478

7579
args = parser.parse_args()
7680
return args
@@ -446,27 +450,40 @@ def remove_one_indent(s):
446450
f.write(output + "\n")
447451

448452

449-
cargo_build_command = ["cargo", "build", "--release"]
450-
if args.no_default_features:
451-
cargo_build_command.append("--no-default-features")
453+
def resolve_rustpython(path):
454+
if os.path.isfile(path):
455+
return path
456+
if os.name == "nt" and not path.lower().endswith(".exe"):
457+
exe = path + ".exe"
458+
if os.path.isfile(exe):
459+
return exe
460+
sys.exit(f"RustPython executable not found: {path}")
461+
462+
463+
if args.rustpython:
464+
rustpython_run_command = [resolve_rustpython(args.rustpython), GENERATED_FILE]
465+
else:
466+
cargo_build_command = ["cargo", "build", "--release"]
467+
if args.no_default_features:
468+
cargo_build_command.append("--no-default-features")
452469

453-
joined_features = ",".join(args.features)
454-
if args.features:
455-
cargo_build_command.extend(["--features", joined_features])
470+
joined_features = ",".join(args.features)
471+
if args.features:
472+
cargo_build_command.extend(["--features", joined_features])
456473

457-
subprocess.run(cargo_build_command, check=True)
474+
subprocess.run(cargo_build_command, check=True)
458475

459-
cargo_run_command = ["cargo", "run", "--release"]
460-
if args.no_default_features:
461-
cargo_run_command.append("--no-default-features")
476+
rustpython_run_command = ["cargo", "run", "--release"]
477+
if args.no_default_features:
478+
rustpython_run_command.append("--no-default-features")
462479

463-
if args.features:
464-
cargo_run_command.extend(["--features", joined_features])
480+
if args.features:
481+
rustpython_run_command.extend(["--features", joined_features])
465482

466-
cargo_run_command.extend(["-q", "--", GENERATED_FILE])
483+
rustpython_run_command.extend(["-q", "--", GENERATED_FILE])
467484

468485
result = subprocess.run(
469-
cargo_run_command,
486+
rustpython_run_command,
470487
env={**os.environ.copy(), "RUSTPYTHONPATH": "Lib"},
471488
text=True,
472489
capture_output=True,

0 commit comments

Comments
 (0)