Repository navigation
benchmark docs: retire the October chain narrative, condense built re-pin rows, fix stale facts - #230
Merged
Merged
Conversation
…-pin rows, fix stale facts (#227) Markdown only, under benchmarks/. No script, data file, Dockerfile, or workflow changes, so the stage scripts the 26.10.1 measurement pulls on mini are unchanged (make_2610_stages.py --after none --rep-pass 3 emits 34 byte-identical files before and after). CAMPAIGN.md: section 6 now describes the 26.10.1 measurement (generator, tiers, repetition passes) and keeps the October chain drawing only because structure_check.py parses it against make_october_stages.STAGES; the stage-by-stage October history became rules. Section 7 loses the skeleton and harness-fix history, condenses every built and verified row to what was built and where it lives (no row renumbered, reordered, or deleted; rows 70, 73, and 68 keep what their tests read), keeps open rows whole, adds the missing row 71 (Bolt, decided against, DECISIONS #162), and fixes wrong citations (DECISIONS #147 for rows 5 and 54, the deleted REPIN-REHEARSAL). Rows 45 and 53 describe the tiered, two-pass chain. Row 20 says the PostgreSQL fits are done, row 40 that the lifecycle expansion left the roster (DECISIONS #139), row 44 that DuckDB VSS and ArangoDB split ingest from index. Step 9 loses its dated done/next list. PROTOCOL, FAIRNESS, PAGE-SPEC, READING-RESULTS, PUBLISHING, RESULTS-MAP, COMPARATORS, QUANTIZATION, README: a paper is back (DECISIONS #137, #146); KNOWN_DISAGREEMENTS holds the three PostgreSQL + AGE entries; three engines have no durability knob (SurrealDB served has one, BUGS F165); INDEX_KINDS_GATE_FAILS is True; the graph tables ask undirected over one stored edge; the ArcadeDB pin row names the 26.10.1 release; FalkorDB moved to 6.0.1; MongoDB's graph arm chains $lookup; the cancelled September extension and qOD/qOL/qOM narrative is gone. benchmarks/python-bindings: delete docs/dev-log-2026-06.md (a planning log of a layout that was never built that way) and docs/ladybug-package.md (folded into the README), and cut the superseded wheel-size audit from jpype_overhead/REPORT.md. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Ubz3m6QXHESerCkehC1qgi
…only Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Ubz3m6QXHESerCkehC1qgi
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #227
Weekly stale-documentation audit of the benchmark docs, markdown only under
benchmarks/. No script, data file, Dockerfile, or workflow is touched, so the stage scripts mini pulls are unchanged:make_2610_stages.py --out DIR --after none --rep-pass 3emits the same 34 files before and after (sha256 of the concatenated checksums is identical), andqueue_lint.pyreports 0 problems on them.What changed
The October chain on miniheading and its arrow chain stay becausestructure_check.pyparses them againstmake_october_stages.STAGES. TheqOA6,qOA7,qOK,qOJ, repair, and preflight paragraphs became the rules they taughtqOJ,qOL, andqOMof the superseded chainrunner.PG_FIT_CMD); row 40 said the lifecycle expansion runs last (taken off the roster by DECISIONS #139); row 44 said DuckDB VSS and ArangoDB share one timer (they split); rows 45 and 53 described an 11-stage chain; row 18 had a missing cell separatorrepin-prepbranch exception, theBENCH_ALLOW_DEV=1andcampaign_env.shOctober-only remarks, and step 9's dated done, next, and not-yet-tried list (it also named Gremlin plugins)KNOWN_DISAGREEMENTS"empty now" (it holds three PostgreSQL + AGE entries); "four engines with no knob" (three; SurrealDB served hassync, BUGS F165); the dense-lane split list;paper.tex; the composed Qdrant row waiting forqOM; theSHARDSdisclosure; REPIN-REHEARSAL. Every symbol named after GATE, HAND-RUN, and REMEMBERED was checked against the code and existscheck_close_costprints F11; F14d said the unstamped-row gate flips later (INDEX_KINDS_GATE_FAILSis True); the intra-query parallelism bullet treated the October pin as current; the F6 table citedqOA5andqOB2qEA,qEB,qED), theqOD,qOL, andqOMe2 split paragraph, and the typedSTALE_UNTIL_RERUNlist (the registry has nine entries); the engine identity bullet; table rows now name the registered PostgreSQL + AGE, plain PostgreSQL, Memgraph, LadybugDB, and DuckDB arms. The generated "Published tables" block is untouchedKNOWN_DISAGREEMENTS; "unverified is SurrealDB served"; "four engines with no knob"; the dense split listlean_http.py,overrides.py)d36b4ca3a); "FalkorDB stays at v4.20.6" contradicted the row (6.0.1); the MongoDB graph row said$graphLookupwhere the arm chains$lookup;qOMandqEEnamed as pendingQUANTIZATION.mdwas missing from the document indexladybugpackage versus the frozenreal_ladybug) is now in the README rowKept, with reasons
PROTOCOL sections 1 to 6 and the section 7 override table (every tag verified,
test_overrides.pyreads it), FAIRNESS F1 to F15 (code-backed), PAGE-SPEC sections 0 to 4b and the generated block (refresh_web_page.pyrewrites it), CAMPAIGN sections 1 to 5a and 8 (methods that are still true), RESULTS-MAP's tracked-artifact and overlay tables, the COMPARATORS survey and dialect facts (they are the record of the pins in force), the SciPy artifact README, and theENV.md,evidence/, andresults/generated/files (evidence or generated).Checks
structure_check.pyok (gates, stage chain, ingest prose; the citation check skips without the notes, so I ran its logic against the notes index tables separately: no missing DECISIONS or BUGS entry);version_pin_check.py0 disagreements;version_consistency_check.pyok;queue_lint.pyon the regenerated stages 0 problems;pre-commit run --filespasses. Doc-reading tests (test_jpype_pin.pyrow 73,test_l4_analyze.pyrow 70,test_index_kinds.py,test_overrides.py,test_fairness_headers.py,test_arango_calibration.py) pass, and the full experiments suite is 443 passed, 9 failed, 11 skipped, the same 9 failures as onorigin/mainbefore the change:test_jpype_pin.pyhardcodes the stage idsqRAandqRJ, which the generator's default order no longer emits (not fixable in a markdown-only change).prose_check.pyandclaims_check.pyneed result files the clean checkout lacks and behave the same with and without this change.Not fixed here: the code comments in
l1_tpc.pyandtest_graph_undirected.pystill cite REPIN-REHEARSAL; the page generator does not yet disclose the two bind exceptions (row 2).🤖 Generated with Claude Code
https://claude.ai/code/session_01Ubz3m6QXHESerCkehC1qgi