Expected Behavior
The parallel unit suite should not fail because a shared CI runner briefly provides less CPU time to one pytest worker. Key encoding performance should be tracked by the benchmark suite.
Current Behavior
test_performance_bounds_single_entity measures 1,000 serialization and deserialization calls with time.perf_counter() and requires each loop to finish within 0.2 seconds. The unit suite runs with 8 pytest-xdist workers, so elapsed time includes time that the worker is descheduled.
The test failed on master in run 31768603920:
AssertionError: Serialization too slow: 0.2062s
assert 0.20623705900004552 < 0.2
This limit has already been raised twice to address CI failures. It started at 0.02 seconds in #5981, changed to 0.1 seconds in #6040, then changed to 0.2 seconds in #6043. Raising it again would postpone the next failure without making the measurement deterministic.
Steps to reproduce
- Run
make test-python-unit, which starts 8 pytest workers.
- Let
test_performance_bounds_single_entity execute while the runner is under contention.
- The wall-clock loop can exceed 0.2 seconds even though the implementation has not regressed.
The intermittent failure is captured in the linked master run. Local execution passes when the host is idle, which is consistent with runner contention.
Specifications
- Version: current
master at f771ea4
- Failing platform: Python 3.10 on
ubuntu-latest
- Subsystem: Python unit tests, entity key encoding
- File:
sdk/python/tests/unit/infra/test_key_encoding_utils.py
Possible Solution
Remove test_performance_bounds_single_entity. The same change that introduced it also added dedicated pytest-benchmark coverage in sdk/python/tests/benchmarks/test_key_encoding_benchmarks.py for single-entity string and integer serialization and deserialization. Those benchmarks run in the separate benchmark workflow and preserve the performance signal without turning shared-runner latency into a unit-test failure.
The existing test_single_entity_fast_path continues to cover correctness of the optimized path.
Expected Behavior
The parallel unit suite should not fail because a shared CI runner briefly provides less CPU time to one pytest worker. Key encoding performance should be tracked by the benchmark suite.
Current Behavior
test_performance_bounds_single_entitymeasures 1,000 serialization and deserialization calls withtime.perf_counter()and requires each loop to finish within 0.2 seconds. The unit suite runs with 8 pytest-xdist workers, so elapsed time includes time that the worker is descheduled.The test failed on
masterin run 31768603920:This limit has already been raised twice to address CI failures. It started at 0.02 seconds in #5981, changed to 0.1 seconds in #6040, then changed to 0.2 seconds in #6043. Raising it again would postpone the next failure without making the measurement deterministic.
Steps to reproduce
make test-python-unit, which starts 8 pytest workers.test_performance_bounds_single_entityexecute while the runner is under contention.The intermittent failure is captured in the linked
masterrun. Local execution passes when the host is idle, which is consistent with runner contention.Specifications
masterat f771ea4ubuntu-latestsdk/python/tests/unit/infra/test_key_encoding_utils.pyPossible Solution
Remove
test_performance_bounds_single_entity. The same change that introduced it also added dedicated pytest-benchmark coverage insdk/python/tests/benchmarks/test_key_encoding_benchmarks.pyfor single-entity string and integer serialization and deserialization. Those benchmarks run in the separate benchmark workflow and preserve the performance signal without turning shared-runner latency into a unit-test failure.The existing
test_single_entity_fast_pathcontinues to cover correctness of the optimized path.