Skip to content

test: widen safety margin in test_response_aggregation_timings_multiple#1737

Merged
bdraco merged 1 commit into
masterfrom
fix/flaky-response-aggregation-timings-multiple
May 19, 2026
Merged

test: widen safety margin in test_response_aggregation_timings_multiple#1737
bdraco merged 1 commit into
masterfrom
fix/flaky-response-aggregation-timings-multiple

Conversation

@bdraco
Copy link
Copy Markdown
Member

@bdraco bdraco commented May 19, 2026

Summary

test_response_aggregation_timings_multiple slept 900ms before asserting that no protected aggregation send had fired; the minimum protected send_after is 1020ms (1000ms RFC 6762 §14 protection plus a 20ms random floor), so the margin was only 100ms. On Windows runners with coarse timer resolution the sleep can overshoot that gap, the timer fires, and the assertion sees 1 call instead of 0.

Details

  • Drop the pre-assertion sleep to 500ms so the gap below the 1020ms minimum is ~520ms instead of ~100ms.
  • Extend the follow-up sleep to keep the total elapsed time identical (1340ms), so the final assert-1-call branch keeps the same headroom it had before.
  • Refresh the comments to reflect the new numbers; the test intent is unchanged.

Test plan

  • SKIP_CYTHON=1 poetry run pytest tests/test_handlers.py::test_response_aggregation_timings_multiple passes locally
  • Windows CI matrix entry that previously flaked stays green

@codspeed-hq
Copy link
Copy Markdown

codspeed-hq Bot commented May 19, 2026

Merging this PR will not alter performance

✅ 11 untouched benchmarks


Comparing fix/flaky-response-aggregation-timings-multiple (417f97e) with master (692a2c4)

Open in CodSpeed

@codecov
Copy link
Copy Markdown

codecov Bot commented May 19, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 99.76%. Comparing base (692a2c4) to head (417f97e).

Additional details and impacted files
@@           Coverage Diff           @@
##           master    #1737   +/-   ##
=======================================
  Coverage   99.76%   99.76%           
=======================================
  Files          33       33           
  Lines        3440     3440           
  Branches      473      473           
=======================================
  Hits         3432     3432           
  Misses          5        5           
  Partials        3        3           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@bdraco
Copy link
Copy Markdown
Member Author

bdraco commented May 19, 2026

@bluetoothbot review

@bluetoothbot
Copy link
Copy Markdown
Contributor

PR Review — test: widen safety margin in test_response_aggregation_timings_multiple

Targeted, well-reasoned fix for a Windows-CI flake. Math is internally consistent: the new pre-assertion gap below the 1020ms minimum widens from ~100ms to ~520ms (well clear of Windows' ~15.6ms default timer resolution), and the second sleep is rebalanced so the total elapsed time remains 1340ms — preserving the headroom of the assert-1-call branch exactly. The comment block is updated to match the new constants, the test still pins the same behaviour (RFC 6762 §14 protected aggregation: no send before 1020ms, exactly one send after the full window), and no production code is touched. Merge-ready.



Checklist

  • Math/timing is consistent before and after the change
  • Total elapsed time preserved (second-assert headroom unchanged)
  • Comments updated to match new constants
  • Test still asserts observable behaviour (send_mock call counts), not implementation
  • No production code touched; scope matches PR description

Summary

Targeted, well-reasoned fix for a Windows-CI flake. Math is internally consistent: the new pre-assertion gap below the 1020ms minimum widens from ~100ms to ~520ms (well clear of Windows' ~15.6ms default timer resolution), and the second sleep is rebalanced so the total elapsed time remains 1340ms — preserving the headroom of the assert-1-call branch exactly. The comment block is updated to match the new constants, the test still pins the same behaviour (RFC 6762 §14 protected aggregation: no send before 1020ms, exactly one send after the full window), and no production code is touched. Merge-ready.


Automated review by Kōan417f97e

@bdraco bdraco marked this pull request as ready for review May 19, 2026 18:58
@bdraco bdraco merged commit 228af17 into master May 19, 2026
37 checks passed
@bdraco bdraco deleted the fix/flaky-response-aggregation-timings-multiple branch May 19, 2026 18:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants