Conversation
Merging this PR will degrade performance by 15.34%
|
| Mode | Benchmark | BASE |
HEAD |
Efficiency | |
|---|---|---|---|---|---|
| ❌ | Simulation | decode_varbin[(1000, 2)] |
61 µs | 77.3 µs | -21.05% |
| ❌ | Simulation | decompress[u64, (10000, 4)] |
311.7 µs | 361.1 µs | -13.67% |
| ❌ | Simulation | compress_fsst[(1000, 64, 8)] |
1 ms | 1.2 ms | -10.96% |
Tip
Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.
Comparing mp/ssb (f540b24) with develop (a1057db)
Footnotes
-
51 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩
Adds SSB (O'Neil, O'Neil & Chen) to `vortex-bench`, wired end to end the same way Appian and SpatialBench are: a `Benchmark` impl, the 13 queries under `sql/ssb/`, and a `BenchmarkCase` in the orchestrator catalog at scale factor 10. SSB is TPC-H redesigned as a star schema — one wide `lineorder` fact table against four dimensions — so every query is a fact-table scan under dimension-derived filters of known, deliberately varied selectivity. That isolates filter pushdown, zone-map pruning, and dimension-join throughput rather than mixing them with TPC-H's subqueries. Signed-off-by: mprammer <martin@spiraldb.com> Co-Authored-By: Claude <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Rationale for this change
Vortex's SQL matrix has no star-schema workload. SSB (O'Neil, O'Neil & Chen) is TPC-H redesigned as one —
lineitemandordersdenormalized into a widelineorderfact table against four dimensions — so every query is a fact-table scan under dimension-derived filters of a known, deliberately varied selectivity. That isolates filter pushdown, zone-map pruning, and dimension-join throughput instead of mixing them with TPC-H's subqueries and correlated predicates. Unlike SpatialBench it runs on DataFusion and DuckDB unmodified, so it covers the full target grid.What changes are included in this PR?
The 13 queries under
vortex-bench/sql/ssb/, aBenchmarkimpl invortex-bench/src/ssb/, and a catalog entry at scale factor 10 with the same CI coverage as Appian —pr-fullanddevelop, not the quickprmatrix. Expected row counts are baked in for SF 1 and SF 10 and asserted on every run. The date dimension registers asdwdaterather thandate, which is a reserved word in both engines' parsers; the reference SSB load scripts rename it for the same reason.Data generation is the part worth reviewing. There is no Rust SSB generator, and SSB is not derivable from TPC-H output —
customeris SF × 30k rather than SF × 150k,supplierSF × 2k rather than SF × 10k,partis200000 · ⌊1 + log₂ SF⌋, and thedwdatecalendar has no TPC-H analogue. Sosrc/ssb/datagen.rsbuilds the reference Cdbgenfrom source and converts its.tbloutput to Parquet with theduckdbCLI, the same shell-out Appian uses. SSB has no official upstream, only a tree of unsynchronizeddbgenforks, and they are not interchangeable: this pinseyalroz/ssb-dbgen, which unifies them.What APIs are changed? Are there any user-facing changes?
No library API change. New
vx-bench run ssb --opt scale-factor=Nsubcommand. Generating SSB data needscmakeand a C compiler on top of theduckdbCLI the matrix already requires; the workflow installscmakeonly for the SSB job.🤖 Generated with Claude Code