Tighten leak-test threshold to 10% to actually fail the bug

greateggsgreg · filmor · commit 7f8a2475f970 · 2026-05-11T13:25:56.000+02:00
The previous 90% threshold (0.9 MB/iter against a 1 MB allocation)
documented the issue but did not reproduce it: master leaks
~600-765 KB/iter, which the 0.9 MB threshold accepts as passing.

Drop the threshold to 10% (104 KB/iter). On the 2026-05-09 verification
run with Python 3.14 GIL on linux-aarch64:

  Without fix (master):   ~572-765 KB/iter (FAIL)
  With fix (this branch): ~-500 B/iter     (PASS)

Margin is roughly 6x in either direction across .NET 8 and .NET 10, so
the threshold cleanly separates buggy from fixed states without being
sensitive to GC noise.
diff --git a/tests/test_method.py b/tests/test_method.py
@@ -983,9 +983,10 @@ def test_getting_generic_method_binding_does_not_leak_memory(memory_usage_tracki
     bytesAllocatedPerIteration = pow(2, 20)  # 1MB
     bytesLeakedPerIteration = processBytesDelta / iterations
 
-    # Allow 90% threshold - this shows the original issue is fixed, which leaks the full allocated bytes per iteration
-    # Increased from 50% to ensure that it works on Windows with Python >3.13
-    failThresholdBytesLeakedPerIteration = bytesAllocatedPerIteration * 0.9
+    # Tight 10% threshold: with the fix the per-iteration leak is essentially
+    # zero, while the bug retains the bulk of the 1 MB payload (~600 KB/iter
+    # on 3.14 GIL). 100 KB/iter cleanly distinguishes the two states.
+    failThresholdBytesLeakedPerIteration = bytesAllocatedPerIteration * 0.1
 
     assert bytesLeakedPerIteration < failThresholdBytesLeakedPerIteration
 
@@ -1025,8 +1026,8 @@ def test_getting_overloaded_method_binding_does_not_leak_memory(memory_usage_tra
     bytesAllocatedPerIteration = pow(2, 20)  # 1MB
     bytesLeakedPerIteration = processBytesDelta / iterations
 
-    # Allow 90% threshold - this shows the original issue is fixed, which leaks the full allocated bytes per iteration
-    failThresholdBytesLeakedPerIteration = bytesAllocatedPerIteration * 0.9
+    # Tight 10% threshold; see test_getting_generic_method_binding_does_not_leak_memory.
+    failThresholdBytesLeakedPerIteration = bytesAllocatedPerIteration * 0.1
 
     assert bytesLeakedPerIteration < failThresholdBytesLeakedPerIteration
 
@@ -1068,8 +1069,8 @@ def test_getting_method_overloads_binding_does_not_leak_memory(memory_usage_trac
     bytesAllocatedPerIteration = pow(2, 20)  # 1MB
     bytesLeakedPerIteration = processBytesDelta / iterations
 
-    # Allow 90% threshold - this shows the original issue is fixed, which leaks the full allocated bytes per iteration
-    failThresholdBytesLeakedPerIteration = bytesAllocatedPerIteration * 0.9
+    # Tight 10% threshold; see test_getting_generic_method_binding_does_not_leak_memory.
+    failThresholdBytesLeakedPerIteration = bytesAllocatedPerIteration * 0.1
 
     assert bytesLeakedPerIteration < failThresholdBytesLeakedPerIteration
 
