Proposal for Improving simdjson on RISC-V: A First-Class RVV Backend (with measurable wins)

Disclaimer: This analysis has been made by AI, under my guidance.

0) Motivation and current state (from the codebase)

• The project already has dedicated CI workflows that build and run tests under QEMU with RVV enabled, including multiple VLEN configurations (e.g., VLEN=128, 256, 1024) and toolchains (clang/gcc).

• The preprocessor layer already detects RVV-capable builds via SIMDJSON_IS_RVV (and related macros).

• However, RVV is not currently treated as a first-class simdjson “implementation” in the same way as haswell, icelake, arm64, etc. Concretely:

  • CMake’s implementation list does not include rvv.
  • The builtin implementation include switch does not have an rvv case.
  • The runtime dispatch registry (get_available_implementation_pointers()) enumerates known implementations but has no RVV entry.
  • The SIMDJSON_SINGLE_IMPLEMENTATION macro sums the known implementations and would need RVV integration if we add it.

High-level goal: add a real RVV backend that is selectable via the existing runtime dispatch mechanism, passes the existing RVV CI, and shows clear speedups (first UTF-8 validation, then Stage 1 structural indexing).

1) Deliverables and acceptance criteria

Deliverable A — “Plumbing” RVV backend (compiles, selectable, no perf regression)

Acceptance:

• SIMDJSON_FORCE_IMPLEMENTATION=rvv selects RVV at runtime (same mechanism already used in dispatch).
• All existing RVV CI workflows pass under QEMU (VLEN=128/256/1024).

Deliverable B — RVV UTF-8 validation (first measurable win)

Acceptance:

• Correctness: existing UTF-8 related tests pass under RVV CI.
• Performance: a reproducible benchmark report showing RVV vs fallback on the same platform/emulation config.

Deliverable C — RVV Stage 1 structural indexing (main parsing throughput win)

Acceptance:

• Correctness: full test suite passes under RVV CI.
• Performance: benchmark results on structural indexing and end-to-end parse show clear improvement vs fallback.

2) Work plan (incremental PRs, each independently reviewable)

PR 1 — Make RVV a first-class “implementation” (no new SIMD yet)

Why: This reduces review risk: we land scaffolding first, then performance code.

Changes:

1. CMake integration

   • Add rvv to the CMake implementation list (currently: fallback westmere haswell icelake arm64 ppc64).
   • Ensure the new option can be included/excluded like the others.

2. Builtin integration

   • Add RVV cases to the builtin include/namespace switch (currently enumerates arm64/fallback/haswell/icelake/ppc64/westmere/lsx/lasx, no rvv).

3. Runtime dispatch / registry

   • Add get_rvv_singleton() and register it in get_available_implementation_pointers() (where all implementations are listed).

4. Single-implementation logic

   • Update SIMDJSON_SINGLE_IMPLEMENTATION accounting to include RVV (currently sums only existing impls).

Note: Platform detection macros already exist (SIMDJSON_IS_RVV).
So PR 1 is primarily wiring.

PR 2 — RVV implementation skeleton + “generic” functional parity

Goal: Provide a working RVV implementation class that is correct even before full vector optimizations.

Changes:

• Add include/simdjson/rvv/implementation.h and src/rvv.cpp (mirroring patterns used by other impls).

• Implement required virtuals:

  • create_dom_parser_implementation
  • minify
  • validate_utf8

• For the initial pass:

  • validate_utf8 can call the generic path to ensure correctness; then PR 3 replaces it with RVV-accelerated code.

Acceptance:

• RVV CI passes (the project already has RVV QEMU CI, so this is a reliable gate).

PR 3 — RVV UTF-8 validation (first targeted SIMD optimization)

Goal: Implement a true RVV fast path for validate_utf8().

Approach:

• Start with a pragmatic vector strategy that matches existing internal abstractions:

  • Implement “simd8” style operations on top of RVV intrinsics (VL-agnostic internally, but exposed as fixed-size chunks where needed to match existing generic logic).

• Use the existing SIMDJSON_IS_RVV gate for compilation.

• Add a benchmark mode that:

  • runs validate_utf8 repeatedly on representative data
  • prints throughput and compares against fallback

Measurement:

• Report benchmark results using the same QEMU CPU/VLEN settings used by CI (e.g., VLEN=128 and VLEN=1024).

PR 4 — RVV mask/bitset extraction primitives (enabler for Stage 1)

Problem: Stage 1 is dominated by producing bitmasks for structurals, quotes, backslashes, etc. RVV is predicate-driven and does not provide x86-like movemask.

Plan:

• Implement a stable, correct predicate→bitset lowering:

  1. produce a byte vector of 0/1 from the predicate
  2. store it
  3. pack scalarly into uint64_t (or a small struct of multiple uint64_t depending on chunk size)

• Add a microbenchmark for mask packing alone (so we can see if we need zvbb-assisted optimization).

Optional fast path:

• The code currently sets SIMDJSON_HAS_ZVBB_INTRINSICS to 0 with a comment about detection limitations.
  Yet CI builds with -march=rv64gcv_zvbb in at least one workflow.
  Proposal: add a build-time opt-in (CMake option or compile definition) that enables zvbb paths when the compiler supports them (without relying on fragile autodetection).

PR 5 — RVV Stage 1 structural indexing

Goal: Accelerate the core of parsing by porting Stage 1 to RVV.

Approach:

• Reuse the existing stage1 algorithm and swap in RVV SIMD primitives + bitmask extraction from PR 4.

• Validate with:

  • full CI test suite under RVV QEMU VLEN matrix
  • targeted JSON corpora with heavy quoting/escaping and dense punctuation

Acceptance:

• RVV CI passes and end-to-end parse throughput improves vs fallback.

PR 6 — RVV Stage 2 (strings, numbers) and “polish”

Goal: Make RVV a complete backend, not just stage1 + utf8.

Items:

• String parsing (quote/backslash search accelerations).
• Number parsing hotspots.
• Minify fast path.
• Documentation: “How to build RVV” section aligned with existing workflows (QEMU_CPU flags, VLEN notes).

3) Benchmarking/reporting plan (what I would submit with each perf PR)

• A small, reproducible benchmark script that:

  • builds fallback and RVV variants
  • runs the same datasets
  • prints throughput and CPU/VLEN configuration (matching CI’s QEMU setup).

• A short “before/after” table for:

  • UTF-8 validation
  • Stage 1 indexing
  • End-to-end parse

4) Risk management and review strategy

• Keep PRs small and orthogonal: wiring first, then correctness, then speed.
• Prioritize “correct & reviewable” over “maximal RVV cleverness” initially; follow with targeted optimization passes once correctness is locked.
• Use the existing RVV CI matrix as the main correctness gate.

5) Optional sidequests (high ROI, can be done in parallel)

1. Key comparison hotspot reduction (string compare acceleration) as an independent PR (useful on all architectures).
2. Improve zvbb enablement story (explicit opt-in / CMake flag), since CI already exercises rv64gcv_zvbb.
3. Microbench suite for SIMD primitives (mask packing, quote scan, structural scan).

If you want, I can rewrite the plan into a PR-by-PR checklist (exact files to add/modify, plus minimal API surface for the RVV simd layer) aligned with simdjson’s current dispatch and build wiring.