Skip to content

refactor(array): use buffer allocator in execution context - #9669

Draft
gatesn wants to merge 3 commits into
ngates/buffer-allocator-corefrom
ngates/buffer-allocator-context
Draft

refactor(array): use buffer allocator in execution context#9669
gatesn wants to merge 3 commits into
ngates/buffer-allocator-corefrom
ngates/buffer-allocator-context

Conversation

@gatesn

@gatesn gatesn commented Aug 27, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Summary

Add a buffer allocator to execution and I/O contexts.

Changes

  • Store BufferAllocatorRef in ExecutionContext.
  • Use it for chunked canonicalization.
  • Allocate file and object-store reads with the caller allocator.
  • Remove the old array memory allocator wrapper.

@codspeed-hq

codspeed-hq Bot commented Aug 27, 2026 •
edited
Loading

Copy link
Copy Markdown

Merging this PR will degrade performance by 13.26%

⚠️ Unknown Walltime execution environment detected

Using the Walltime instrument on standard Hosted Runners will lead to inconsistent data.

For the most accurate results, we recommend using CodSpeed Macro Runners: bare-metal machines fine-tuned for performance measurement consistency.

❌ 33 regressed benchmarks
✅ 2136 untouched benchmarks
⏩ 106 skipped benchmarks1
🗄️ 4 archived benchmarks run2

Warning

Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

Mode Benchmark BASE HEAD Efficiency
❌ Simulation compact_sliced[(2048, 10)] 207.2 µs 271.9 µs -23.8%
❌ Simulation compact_sliced[(1024, 10)] 106.4 µs 138.4 µs -23.09%
❌ WallTime arrow_checked_add_u32_avx2[16384] 17.7 µs 21.3 µs -17.13%
❌ Simulation filter_powerlaw_by_mostly_true[250000] 179.8 µs 214.6 µs -16.2%
❌ Simulation filter_random_by_mostly_true[250000] 180 µs 214.6 µs -16.14%
❌ Simulation filter_random_by_random[250000] 164.2 µs 192.8 µs -14.82%
❌ Simulation filter_powerlaw_by_random[250000] 164.3 µs 192.7 µs -14.76%
❌ Simulation filter_random_by_power_law[250000] 154.1 µs 177.7 µs -13.31%
❌ Simulation filter_powerlaw_by_power_law[250000] 154.7 µs 178.4 µs -13.28%
❌ Simulation density_sweep_random[0.9] 89.3 µs 103 µs -13.24%
❌ Simulation density_sweep_dense_runs[0.9] 89.2 µs 102.7 µs -13.14%
❌ Simulation filter_powerlaw_by_mostly_true[100000] 90.1 µs 103.7 µs -13.07%
❌ Simulation density_sweep_random[0.95] 89.9 µs 103.3 µs -13.04%
❌ Simulation filter_random_by_mostly_true[100000] 90.1 µs 103.5 µs -13.01%
❌ Simulation density_sweep_dense_runs[0.95] 90 µs 103.4 µs -13%
❌ Simulation filter_powerlaw_by_mostly_false[250000] 150 µs 172.2 µs -12.88%
❌ Simulation filter_random_by_mostly_false[250000] 150.1 µs 172.3 µs -12.88%
❌ Simulation density_sweep_dense_runs[0.5] 83.9 µs 95.1 µs -11.85%
❌ Simulation filter_random_by_random[100000] 83.9 µs 95 µs -11.74%
❌ Simulation density_sweep_random[0.5] 83.3 µs 94.3 µs -11.69%
... ... ... ... ... ...

ℹ️ Only the first 20 benchmarks are displayed. Go to the app to view all benchmarks.

Tip

Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.

Comparing ngates/buffer-allocator-context (2e9ce47) with ngates/buffer-allocator-core (235c4c8)

Open in CodSpeed

Footnotes

  1. 106 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

  2. 4 benchmarks were run, but are now archived. If they were deleted in another branch, consider rebasing to remove them from the report. Instead if they were added back, click here to restore them. ↩

@gatesn gatesn mentioned this pull request Aug 27, 2026
Open
5 tasks
@gatesn
gatesn force-pushed the ngates/buffer-allocator-context branch from 318a807 to ea37874 Compare August 27, 2026 19:38
@gatesn
gatesn force-pushed the ngates/buffer-allocator-context branch 2 times, most recently from 16b6ae2 to 8b88427 Compare August 27, 2026 21:37
@gatesn
gatesn force-pushed the ngates/buffer-allocator-context branch from 8b88427 to 884d9d1 Compare August 27, 2026 21:38
@gatesn gatesn added the changelog/feature A new feature label Aug 27, 2026
@gatesn
gatesn force-pushed the ngates/buffer-allocator-context branch 2 times, most recently from 97a7955 to af8266f Compare August 28, 2026 14:56
@gatesn
gatesn force-pushed the ngates/buffer-allocator-context branch 3 times, most recently from 149f535 to 36fca40 Compare August 28, 2026 17:35
@gatesn
gatesn force-pushed the ngates/buffer-allocator-context branch from 36fca40 to f06d432 Compare August 28, 2026 18:02
@gatesn
gatesn force-pushed the ngates/buffer-allocator-context branch 3 times, most recently from 918a550 to ed3f7fc Compare August 28, 2026 20:41
@gatesn
gatesn force-pushed the ngates/buffer-allocator-context branch 2 times, most recently from 3e970c7 to dd87611 Compare August 29, 2026 01:01
@gatesn
gatesn force-pushed the ngates/buffer-allocator-context branch 2 times, most recently from fa9bf7f to 15d742c Compare August 29, 2026 02:00
gatesn added 2 commits August 28, 2026 22:45
@gatesn
refactor(array): use buffer allocator in execution context
8c6e97c 
Signed-off-by: Nicholas Gates <nick@nickgates.com>
@gatesn
fix(array): keep execution context sync
eb02767 
Signed-off-by: Nicholas Gates <nick@nickgates.com>
@gatesn
gatesn force-pushed the ngates/buffer-allocator-context branch from 15d742c to 8e1c1bd Compare August 29, 2026 02:45
@gatesn
fix(buffer): compare static allocator references
2e9ce47 
Signed-off-by: Nicholas Gates <nick@nickgates.com>
@gatesn
gatesn force-pushed the ngates/buffer-allocator-context branch from 8e1c1bd to 2e9ce47 Compare August 29, 2026 03:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

changelog/feature A new feature

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@gatesn