Skip to content

[MNT] Diagnose and address long test runtimes (#1633) #1692

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
Abhishek9639 wants to merge 3 commits into
base: main
Choose a base branch
from
Open

[MNT] Diagnose and address long test runtimes (#1633) #1692

Changes from 1 commit
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
149d23c
[MNT] Diagnose and address long test runtimes (#1633)
Feb 26, 2026
c8fd9a9
Address review feedback: revert CI/conftest changes, improve profile …
Mar 1, 2026
144cee9
Update profile_tests.sh: add -n workers, --dist=load, remove -q
Mar 1, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Update profile_tests.sh: add -n workers, --dist=load, remove -q 
- Add -n flag for parallel workers (default: 4)
- Add --dist=load to distribute tests across workers
- Remove -q flag for full pytest output
- Mimics exact pytest command used in CI
  • Loading branch information
@Abhishek9639
Abhishek authored and Abhishek9639 committed Apr 7, 2026
commit 144cee960de36430c6b4088f79f12a4854fff39e
13 changes: 9 additions & 4 deletions scripts/profile_tests.sh
View file
Open in desktop
Comment thread
Abhishek9639 marked this conversation as resolved.
Original file line number Diff line number Diff line change
Expand Up @@ -7,43 +7,48 @@
# -m MARKER Pytest marker filter (default: "not production_server and not test_server")
# -d DURATION Number of slowest durations to show, 0 for all (default: 20)
# -t TIMEOUT Per-test timeout in seconds (default: 300)
# -n WORKERS Number of parallel workers (default: 4)
# -o OUTPUT Output file path for the report (default: test_durations_report.txt)
#
# Examples:
# ./scripts/profile_tests.sh
# ./scripts/profile_tests.sh -m "production_server" -d 0 -t 600
# ./scripts/profile_tests.sh -m "sklearn" -o sklearn_report.txt
# ./scripts/profile_tests.sh -m "sklearn" -n 2 -o sklearn_report.txt

set -euo pipefail

# Default values
MARKER_FILTER="not production_server and not test_server"
DURATIONS=20
TIMEOUT=300
NUM_WORKERS=4
OUTPUT_FILE="test_durations_report.txt"

# Parse command line arguments
while getopts "m:d:t:o:" opt; do
while getopts "m:d:t:n:o:" opt; do
case $opt in
m) MARKER_FILTER="$OPTARG" ;;
d) DURATIONS="$OPTARG" ;;
t) TIMEOUT="$OPTARG" ;;
n) NUM_WORKERS="$OPTARG" ;;
o) OUTPUT_FILE="$OPTARG" ;;
*) echo "Usage: $0 [-m marker] [-d durations] [-t timeout] [-o output_file]" && exit 1 ;;
*) echo "Usage: $0 [-m marker] [-d durations] [-t timeout] [-n workers] [-o output_file]" && exit 1 ;;
esac
done

echo "=== OpenML Test Duration Profiler ==="
echo "Marker filter: $MARKER_FILTER"
echo "Durations to show: $DURATIONS"
echo "Timeout per test: ${TIMEOUT}s"
echo "Workers: $NUM_WORKERS"
echo "Output file: $OUTPUT_FILE"
echo ""

pytest \
--dist=load \
-n="$NUM_WORKERS" \
--durations="$DURATIONS" \
--timeout="$TIMEOUT" \
-q \
-m "$MARKER_FILTER" \
2>&1 | tee "$OUTPUT_FILE"
Comment thread
Abhishek9639 marked this conversation as resolved.

Expand Down
Loading