perf(ci): speed up unit tests with sharding + container scoping#3855
perf(ci): speed up unit tests with sharding + container scoping#3855nicktrn wants to merge 24 commits into
Conversation
…edis/postgres containers
…ickhouse isolation)
…inerTest for logical replication
…stgres (share redis+clickhouse)
…tests ~29s; keep each shard <2m)
|
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
WalkthroughThis PR adds duration-weighted test sharding (DurationShardingSequencer + test-timings.json), updates Vitest configs to use the sequencer, expands CI test shard matrices and adds Docker pull retry wrappers, refactors testcontainers to worker-scoped containers with per-test reset fixtures (Postgres template/clone, worker Redis/ClickHouse/MinIO), adds ClickHouse/MinIO reset helpers and timing tooling, and migrates/introduces many replication and repository integration tests to the new harnesses. 🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
…l-fast on bad timings/push
…n't stack on one shard
…cases shard apart
Speeds up and de-flakes the unit-test suite: testcontainers booted once per vitest worker (per-test isolation kept only where a test runs background redis work that outlives it), a duration-weighted shard sequencer so each shard does roughly equal work, the slowest suites split, two genuine flakes fixed (
streamBatchItemsshared-redis leak; run-engine waits that relied on fixed sleeps), and transient DockerHub pulls retried.Timings (CI, per-shard wall): worst unit-test shard ~771s → ~294s; packages/webapp shards ~250-270s, most internal ~190-240s. All 25 shards green.
A shard breaks down as ~70s fixed setup (install / image-pull / generate) + ~70s cold
^build+ the actual container tests. So the remaining cost is mostly the tests themselves plus that fixed setup.Next (separate, timings):
tsc, not the TS version (a TS6 branch is still ~6m17s). The real lever is tsgo (the Go compiler).cli-v3e2e andsdk-compatare path-gated (don't run on test-infra changes) and already comfortably fast.