diff --git a/.claude/skills/improve-performance.md b/.claude/skills/improve-performance.md deleted file mode 100644 index 959a438..0000000 --- a/.claude/skills/improve-performance.md +++ /dev/null @@ -1,162 +0,0 @@ ---- -name: improve-performance -description: > - Guide iterative Java performance optimization using JMH benchmarks and JFR profiling. - Use this skill whenever the user wants to profile, benchmark, or optimize Java code performance, - mentions JMH, JFR, GC pauses, allocation hotspots, TLB misses, throughput regression, - or asks Claude Code to "make this faster", "find the bottleneck", "reduce allocations", - or "profile this code". Also triggers for HugeTLB / large pages JVM investigations, - or any request to run a benchmark and act on the results. ---- - -# JMH + JFR Performance Optimization Skill - -A skill for running JMH benchmarks, collecting JFR profiles, and driving iterative -performance improvements in Java/Maven projects. - -## Workflow Overview - -``` -setup → benchmark → profile → analyse → change → repeat -``` - ---- - -## Step 1 — Verify readiness - -Confirm the workload under test is already covered by a JMH benchmark in the `performance/` module -(e.g. `RustVsJavaReadBenchmark`). If the column / codec / path is not benched, add it before profiling. -Always pass a fully-qualified JMH filter (`ClassName.methodName`) when invoking the harness — never -class-only or empty. - ---- - -## Step 2 — Use the existing benchmark script - -`./benchmark.sh` in the repo root drives JMH + JFR. It runs: - -```bash -./mvnw package -DskipTests -q -pl performance --also-make -java -jar performance/target/benchmarks.jar \ - ${BENCHMARK_FILTER} \ - -f 1 -wi 3 -i 5 \ - -rf json -rff performance/target/jmh-results.json \ - -jvmArgs "-XX:StartFlightRecording=filename=performance/target/recording.jfr,settings=profile" -``` - -Never substitute `mvn` for `./mvnw` (project mandates the wrapper) and never run `mvn install`. - ---- - -## Step 3 — Run the baseline - -```bash -./benchmark.sh RustVsJavaReadBenchmark.javaReadVolume -``` - -Read both output files: - -- `performance/target/jmh-results.json` — throughput / latency per benchmark -- `performance/target/recording-filtered.json` — JFR events - -Store the baseline score. Compare every subsequent run against it. - ---- - -## Step 4 — Analyse results - -Key things to look for: - -**In jmh-results.json:** - -- `primaryMetric.score` — the main result (ops/s or ns/op depending on mode) -- `primaryMetric.scoreError` — high error = unstable benchmark, increase `-i` -- Compare `AverageTime` vs `Throughput` to understand the shape of the workload - -**In recording-filtered.json:** - -- `jdk.ObjectAllocationInNewTLAB` — allocation hotspots; look for large `allocationSize` - or high-frequency small allocations in hot methods -- `jdk.GarbageCollection` — GC pause duration and frequency; long pauses = allocation pressure -- `jdk.TLBMiss` — high counts suggest memory access patterns not benefiting from huge pages -- `jdk.GCHeapSummary` — heap growth between events reveals live-set size - -**Priority order for investigation:** - -1. If GC pauses are long → reduce allocations first -2. If allocations are high but pauses are short → object pooling or value types -3. If TLB misses are high → evaluate huge pages or access pattern changes -4. If CPU is saturated with no GC/alloc issues → algorithmic / cache-locality problem - ---- - -## Step 5 — Apply one change at a time - -**Critical rule: change one thing per iteration.** - -Do not batch multiple optimizations. Each iteration must produce a clear -before/after comparison so regressions are traceable. - -Common optimizations to consider (in order of typical impact): - -1. **Allocate decode output from `ctx.arena()`** — hard rule from `CLAUDE.md`. Never - `new byte[n]` + `MemorySegment.ofArray()` for codec output: heap allocation, GC pressure, - extra copy. Use `ctx.arena().allocate(n * elemBytes, alignment)` so the buffer lives on the - confined arena tied to the `VortexFile`. -2. **Reduce allocations** elsewhere — reuse objects, use primitives, avoid boxing. -3. **Hoist `ValueLayout` constants** — declare `static final ValueLayout.OfXxx L = ...` so JIT - constant-folds the stride / alignment / order. Inline `ValueLayout.JAVA_LONG_UNALIGNED` on each - call defeats this. -4. **Use `getAtIndex` / `setAtIndex`** in tight loops over a `MemorySegment` — stride is implicit, - bounds check hoists, and the auto-vectoriser reads the shape cleanly. -5. **Aligned arena allocation** — `arena.allocate(n, 64)` keeps SIMD-friendly addresses. -6. **Improve data locality** — colocate fields accessed together, prefer flat arrays / segments - over linked structures. -7. **Avoid synchronization on hot paths** — `VarHandle`, `AtomicLong`, lock-free structures. -8. **Reduce GC pressure** — pool expensive objects, avoid finalizers, off-heap where justified. -9. **Enable huge pages** — add `-XX:+UseTransparentHugePages` to `jvmArgs` for comparison. - -After each change, run `./benchmark.sh` and compare the new score against the stored baseline. - ---- - -## Step 6 — Report progress - -After each iteration, report: - -``` -Iteration N - Change : - Before : ± - After : ± - Delta : <+/- %> - JFR note : - Next : -``` - -If performance regressed, revert the change immediately and explain why it likely hurt. - ---- - -## Step 7 — Stop conditions - -Stop iterating when any of the following is true: - -- The goal stated by the user has been reached -- Three consecutive iterations produce < 2% improvement -- All obvious hotspots from JFR have been addressed -- The user says stop - -Produce a final summary table of all iterations. - ---- - -## Important constraints - -- **Never modify the benchmark harness** (`@Benchmark`, `@Setup`, `@TearDown`) unless - the user explicitly asks. Benchmark changes invalidate all prior comparisons. -- **Do not change JMH flags** (`-f`, `-wi`, `-i`) between iterations. Consistency is required - for valid comparisons. -- **Keep JFR event filter consistent** across all runs. -- If `jmh-results.json` shows `scoreError` > 10% of `score`, warn the user the benchmark - is noisy and suggest increasing `-i` before drawing conclusions. diff --git a/.claude/skills/review-performance.md b/.claude/skills/review-performance.md deleted file mode 100644 index 6625b11..0000000 --- a/.claude/skills/review-performance.md +++ /dev/null @@ -1,94 +0,0 @@ ---- -name: review-performance -description: Code review skill for high-throughput Java data processing, focusing on memory bounds (FFM API), zero-copy I/O, and hot-loop optimization. Tailored to the vortex-java columnar reader/writer (confined Arena, ctx.arena() for codec output). ---- - -## Overview - -This skill provides specialized instructions for reviewing high-performance Java code, particularly data processing -pipelines, columnar decoders, and hot-loop execution. When asked to review Java code for performance, apply these -advanced heuristics rather than generic advice. - ---- - -## Core Optimization Rules - -### 1. Hot-Loop Optimization - -- **Loop Unswitching:** Hoist polymorphic branches (e.g., `switch` on primitive types) *outside* tight loops to improve - branch predictability and enable vectorization. -- **Strength Reduction:** Replace repeated multiplications (`base + i * stride`) with incremental additions ( - `v += stride`). - - Warn about IEEE‑754 drift when accumulating `float`/`double` over large iteration counts. -- **Invariant Hoisting:** Move loop-invariant loads, bounds, and metadata outside the loop. -- **Predictability:** Flag unpredictable, data-dependent branches inside hot loops. -- **Bounds Check Elimination:** Encourage patterns that allow the JIT to remove array bounds checks (e.g., using indexed - loops with prevalidated ranges). -- **Mark opportunities for Java Vector API.** - ---- - -### 2. Memory Rules: Heap, Direct, and FFM - -- **2GB Limit:** Flag any `(int) (n * elemBytes)` or similar size computation that may overflow `Integer.MAX_VALUE` (~ - 2.14 GB). -- **Prefer FFM for Large Buffers:** Recommend migrating large `ByteBuffer` allocations to the Foreign Function & Memory - API (`MemorySegment`). - - **Codec output allocation rule (hard, from `CLAUDE.md`):** never allocate `byte[]` + wrap with - `MemorySegment.ofArray()` for decode output. Always allocate from the `DecodeContext` arena: - `ctx.arena().allocate(n * elemBytes, alignment)`. The buffer's lifetime is the confined `Arena` owned - by the `VortexFile` — `Arena.ofAuto()` / `Arena.ofShared()` violate that ownership model and leak / drop - the segment at the wrong time. - - If the allocation lives in a private static helper without `DecodeContext`, add an `Arena arena` - parameter and pass `ctx.arena()` at the call site. -- **Alignment & VarHandles:** - - Warn when using `ValueLayout.JAVA_INT` or similar on potentially unaligned addresses. - - Prefer `ValueLayout` constants over manual offset math. - - Flag aliasing between overlapping `MemorySegment` slices. -- **Mixed Heap/Off‑Heap Pipelines:** - - Flag transitions between heap arrays, direct buffers, and `MemorySegment` unless explicitly justified. - - Require clear ownership and lifetime rules when mixing arenas, slices, and native handles. - ---- - -### 3. Zero-Copy I/O and Protobuf - -- **Eliminate Intermediate `byte[]`:** Flag any heap array created solely to bridge between `ByteBuffer` and parsing - frameworks. -- **Direct Protobuf Parsing:** - - Recommend `CodedInputStream.newInstance(ByteBuffer)` for metadata parsing. - - Note: Protobuf does not support `MemorySegment`; `ByteBuffer` is optimal for small metadata payloads. - ---- - -## Profiling-Driven Overrides - -When the user provides profiling data (JFR, async-profiler, perf, flamegraphs): - -- Prioritize empirical evidence over static heuristics. -- If a rule contradicts profiling data, warn but defer to the measurements. -- Encourage validating changes with before/after profiles. - ---- - -## Conflict Resolution Rules - -- **Correctness > Performance:** If an optimization risks semantic drift, warn instead of rewriting. -- **FFM vs ByteBuffer:** Prefer FFM for large or long-lived buffers; prefer `ByteBuffer` for tiny metadata payloads or - Protobuf interop. -- **Precision vs Speed:** When strength reduction introduces floating-point drift, warn and let the user decide. -- **Safety First:** If memory aliasing, lifetime, or alignment is unclear, flag it even if performance would improve. - ---- - -## When to Apply - -Apply these guidelines automatically whenever the user asks to: - -- Review array, columnar, or sequence decoders. -- Audit large dataset memory allocations (I/O, buffers). -- Review code using FFM (`MemorySegment`), NIO (`ByteBuffer`), or JNI. -- Optimize hot loops, vectorizable loops, or tight decode/encode paths. -- Improve zero-copy I/O, Protobuf parsing, or off-heap memory usage. - - diff --git a/.github/workflows/benchmark.yml b/.github/workflows/benchmark.yml deleted file mode 100644 index 5fbfd97..0000000 --- a/.github/workflows/benchmark.yml +++ /dev/null @@ -1,54 +0,0 @@ -name: Benchmarks - -on: - schedule: - - cron: '0 20 * * *' # 22:00 CEST / 21:00 CET - workflow_dispatch: - -jobs: - benchmark: - runs-on: ubuntu-latest - permissions: - contents: write - - steps: - - uses: actions/checkout@v4 - - - name: Set up Azul Zulu JDK 25 - uses: actions/setup-java@v4 - with: - distribution: zulu - java-version: '25' - - - name: Cache Maven repository - uses: actions/cache@v4 - with: - path: ~/.m2/repository - key: ${{ runner.os }}-maven-${{ hashFiles('**/pom.xml') }} - restore-keys: | - ${{ runner.os }}-maven- - - - name: Install protobuf compiler - run: sudo apt-get install -y protobuf-compiler - - - name: Install flatc 25.12.19 - run: | - curl -fsSL -o flatc.zip https://github.com/google/flatbuffers/releases/download/v25.12.19/Linux.flatc.binary.g++-13.zip - unzip -j flatc.zip flatc - sudo install -m 755 flatc /usr/local/bin/flatc - - - name: Run benchmarks - run: ./bench "RustVsJava" -wi 1 -i 3 -f 1 - - - name: Store benchmark results - uses: benchmark-action/github-action-benchmark@v1 - with: - tool: jmh - output-file-path: performance/target/benchmark-result.json - github-token: ${{ secrets.GITHUB_TOKEN }} - auto-push: true - alert-threshold: '130%' - comment-on-alert: true - fail-on-alert: false - gh-pages-branch: gh-pages - benchmark-data-dir-path: dev/bench diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml deleted file mode 100644 index 621f739..0000000 --- a/.github/workflows/ci.yml +++ /dev/null @@ -1,35 +0,0 @@ -name: CI - -on: - push: - branches: [ main ] - pull_request: - branches: [ main ] - -jobs: - build: - runs-on: ubuntu-latest - - strategy: - matrix: - java-version: [ '25', '26' ] - - steps: - - uses: actions/checkout@v4 - - - name: Set up Azul Zulu JDK ${{ matrix.java-version }} - uses: actions/setup-java@v4 - with: - distribution: zulu - java-version: ${{ matrix.java-version }} - - - name: Cache Maven repository - uses: actions/cache@v4 - with: - path: ~/.m2/repository - key: ${{ runner.os }}-maven-${{ hashFiles('**/pom.xml') }} - restore-keys: | - ${{ runner.os }}-maven- - - - name: Build and test - run: ./mvnw verify diff --git a/.github/workflows/publish.yml b/.github/workflows/publish.yml deleted file mode 100644 index 045bbf4..0000000 --- a/.github/workflows/publish.yml +++ /dev/null @@ -1,35 +0,0 @@ -name: Publish to Maven Central - -on: - push: - tags: - - 'v*' - -permissions: - contents: read - -jobs: - publish: - runs-on: ubuntu-latest - environment: maven - steps: - - uses: actions/checkout@v4 - - - name: Set up JDK 25 with Maven Central publishing - uses: actions/setup-java@v4 - with: - java-version: '25' - distribution: 'zulu' - cache: 'maven' - server-id: central - server-username: CENTRAL_USERNAME - server-password: CENTRAL_PASSWORD - gpg-private-key: ${{ secrets.GPG_PRIVATE_KEY }} - gpg-passphrase: GPG_PASSPHRASE - - - name: Publish to Maven Central - env: - CENTRAL_USERNAME: ${{ secrets.CENTRAL_USERNAME }} - CENTRAL_PASSWORD: ${{ secrets.CENTRAL_PASSWORD }} - GPG_PASSPHRASE: ${{ secrets.GPG_PASSPHRASE }} - run: ./mvnw deploy -Prelease -DskipTests --batch-mode diff --git a/.gitignore b/.gitignore deleted file mode 100644 index 7d063e3..0000000 --- a/.gitignore +++ /dev/null @@ -1,7 +0,0 @@ -target/ -*.class -*.jar -.idea/ -*.iml -.DS_Store -dependency-reduced-pom.xml diff --git a/.mvn/wrapper/maven-wrapper.properties b/.mvn/wrapper/maven-wrapper.properties deleted file mode 100644 index 216df05..0000000 --- a/.mvn/wrapper/maven-wrapper.properties +++ /dev/null @@ -1,3 +0,0 @@ -wrapperVersion=3.3.4 -distributionType=only-script -distributionUrl=https://repo.maven.apache.org/maven2/org/apache/maven/apache-maven/3.9.16/apache-maven-3.9.16-bin.zip diff --git a/CLAUDE.md b/CLAUDE.md deleted file mode 100644 index b006fa6..0000000 --- a/CLAUDE.md +++ /dev/null @@ -1,236 +0,0 @@ -# CLAUDE.md - -This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository. - -## What it is - -Java 25 native implementation of the [Vortex](https://github.com/spiraldb/vortex) columnar file format. Uses FFM ( -`MemorySegment`/`Arena`) instead of JNI or `sun.misc.Unsafe`. - - -## Branching strategy - -Trunk-based development. PRs are fine but always squash or rebase — no merge commits. -Keep commits small and `main` always green. - -## Commands - -Never use `mvn install` or `./mvwn install`. - -Generated sources (`fbs`/`proto` → Java) are committed under `core/src/main/java`. -Normal builds need no external tools. -To regenerate after editing `.fbs` or `.proto` schemas: -```bash -brew install flatbuffers protobuf -./mvnw generate-sources -pl core -P regenerate-sources -# then commit the updated files -``` -Any `flatc` version works — the profile strips the version guard automatically. - -```bash -# Build all modules -./mvnw verify - -# Build without tests -./mvnw verify -DskipTests - -# Run all tests (unit only — excludes *IntegrationTest) -./mvnw test - -# Run tests in one module -./mvnw test -pl reader - -# Run a single test class -./mvnw test -pl reader -Dtest=MyTest - -# Run a single test method -./mvnw test -pl reader -Dtest=MyTest#myMethod - -# Run integration tests (use verify + failsafe, NOT test + surefire) -./mvnw verify -pl integration -am - -# Run a single integration test class -./mvnw verify -pl integration -am -Dit.test=RustWritesJavaReadsIntegrationTest - -# Run a single integration test method -./mvnw verify -pl integration -am -Dit.test="RustWritesJavaReadsIntegrationTest#s3_pcoVortex_javaDecodeMatchesJni" - -# Run all benchmarks -./bench - -# Run specific benchmark class or method (always use ClassName.methodName filter) -./bench RustVsJavaReadBenchmark -./bench RustVsJavaReadBenchmark.javaReadVolume -``` - -### Javadoc - -Every public method must have complete Javadoc. The build enforces this via -`failOnError=true` + `failOnWarnings=true` in the `maven-javadoc-plugin`. - -Rules: -- Every public method needs a main description, `@param` for each parameter, and `@return` (unless `void`). -- Every public record needs `@param` entries on the class-level doc (one per component). -- Cross-references use `[ClassName#method(ParamType)]` — verify the target exists before writing it. Wrong references are **errors**, not warnings. -- `@see`-only Javadoc counts as "no main description" — always add a prose sentence. - -**Check:** `./mvnw javadoc:javadoc -pl core` — must produce zero output. - -### Releasing - -```bash -./mvnw --batch-mode release:clean release:prepare \ - -DreleaseVersion= \ - -DdevelopmentVersion=-SNAPSHOT -git push && git push --tags -``` - -GitHub Actions picks up the tag and deploys to Maven Central. - -### File format - -8-byte trailer at EOF: `version(u16 LE) | postscriptLen(u16 LE) | magic(VTXF)`. The postscript is a FlatBuffer blob -immediately before the trailer; it points (offset+length) to the Footer (FlatBuffer), DType (Protobuf), and Layout ( -FlatBuffer) blobs elsewhere in the file. - -### Layout tree - -``` -Struct → Zoned(Stats) → Chunked → [Flat, Flat, ...] -``` - -- **Flat**: single encoded segment on disk -- **Chunked**: sequence of Flat children -- **Zoned** (`vortex.stats`): wraps a child with per-chunk min/max statistics used for zone-map pruning -- **Struct**: one child per column - -Encoding IDs are strings (e.g. `"vortex.flat"`, `"fastlanes.bitpacked"`). `DecoderRegistry` maps IDs → `Decoder` impls -via `ServiceLoader`; register custom decoders with `registry.register(decoder)`. - -**Adding a new encoding:** three touch-points, always all three: -1. `EncodingId.java` — add enum constant `VORTEX_FOO("vortex.foo")` -2. `AlpRdEncoding.java` (or `FooEncoding.java`) — implement `Encoding` -3. `core/src/main/resources/META-INF/services/io.github.dfa1.vortex.encoding.Encoding` — add the fully-qualified class name - -### Memory model - -`VortexFile` memory-maps the entire file into one `MemorySegment` (confined `Arena`). All `Array` buffers returned -during scan are zero-copy slices of that segment — their lifetime is tied to the `VortexFile`. Close the file to release -the mapped region. - -**Encoding output allocation rule:** never allocate `byte[]` + wrap with `MemorySegment.ofArray()` for decode output. -Always allocate from `ctx.arena()`: -```java -// WRONG — heap allocation, GC pressure, extra copy -byte[] outBytes = new byte[(int) (n * elemBytes)]; -MemorySegment out = MemorySegment.ofArray(outBytes); - -// CORRECT — off-heap, zero GC, same lifetime as the scan chunk -MemorySegment out = ctx.arena().allocate(n * elemBytes); -``` -If the allocation is in a private static helper that doesn't receive `DecodeContext`, add an `Arena arena` parameter -and pass `ctx.arena()` from the `decode()` call site. - -## Reference implementation - -When stuck on encoding/decoding behavior, consult the Rust reference implementation at -`https://github.com/spiraldb/vortex` (via `gh api repos/spiraldb/vortex/contents/`). - -Key paths: - -- `encodings/fastlanes/src/bitpacking/` — `fastlanes.bitpacked` wire format and algorithm -- `encodings/fastlanes/src/for/` — `fastlanes.for` (frame-of-reference) -- `encodings/sparse/src/` — `vortex.sparse` -- `encodings/alp/src/alp/` — `vortex.alp` -- `https://github.com/spiraldb/fastlanes-rs` — FastLanes bit-packing algorithm (`src/bitpacking.rs`, `src/macros.rs`) - -Never reverse-engineer wire formats by probing byte patterns. Read the vtable `serialize`/`deserialize` -methods in the Rust source to get the exact protobuf schema, then implement from spec. - -## Code style - -- Zero SonarQube bugs/smells policy. -- No `sun.misc.Unsafe` or internal JDK APIs. -- Prefer explicit over clever. Fail fast on unhandled cases. -- Always use braces for `if`/`else`/`for`/`while` bodies, even single-liners: - ```java - // WRONG - if (cond) return a; - - // CORRECT - if (cond) { - return a; - } - ``` - -## Encoding class structure - -Encoding classes with non-trivial encode **and** decode paths must separate them into -private static inner classes named `Encoder` and `Decoder`. Shared low-level helpers -(buffer math, proto serialization) live in the side that owns them or in a third inner -class if genuinely shared by both. - -```java -public final class FooEncoding implements Encoding { - - @Override public EncodeResult encode(DType dtype, Object data) { return Encoder.encode(dtype, data); } - @Override public Array decode(DecodeContext ctx) { return Decoder.decode(ctx); } - - private static final class Encoder { static EncodeResult encode(DType dtype, Object data) { ... } } - private static final class Decoder { static Array decode(DecodeContext ctx) { ... } } -} -``` - -Simple encodings (≤ ~80 lines total, e.g. `NullEncoding`, `BoolEncoding`) are exempt. - -### Metadata-only encodings - -Some encodings store all data in protobuf metadata — no buffers, no children (e.g. `SequenceEncoding`). -Their `EncodeResult` uses an `EncodeNode` with `metadata` set and an empty `bufferIndices` array: - -```java -ByteBuffer metaBuf = ByteBuffer.wrap(meta.toByteArray()); -EncodeNode node = new EncodeNode(encodingId, metaBuf, new EncodeNode[0], new int[]{}); -return new EncodeResult(node, List.of(), null, null); -``` - -The decoder reads back via `ctx.metadata()`, not `ctx.buffer(n)`. - -## Testing - -- Every feature needs unit tests covering: happy path, negative cases (invalid input, error conditions), and corner cases (empty, zero, max values, boundary conditions). -- Unit tests must be fast — no file I/O, no network, no sleep. Mock or use in-memory data. -- Integration tests are critical: there is no formal spec, so interoperability with the Rust reference implementation is the ground truth. Write integration tests for every encoding round-trip and file format boundary. -- JUnit 5 + Mockito (BDDMockito) + AssertJ. -- Every test has `// Given` / `// When` / `// Then` sections. -- Class under test is always named `sut`. -- Use `BDDMockito` exclusively: `given(mock.method()).willReturn(value)`. Never the reverse form. Only static-import - `given` and `then` — not `willReturn`/`willThrow`. -- Prefer `@ParameterizedTest` over copy-pasting tests. Use `@ValueSource` when possible; `@ArgumentsSource` when more - structure needed (test case must have a name). -- Use `@ParameterizedTest` with seeded random generators for encoding/decoding logic where input space is large — they find corner cases that example tests miss. -- Acceptance tests run the built jar end-to-end with hosh scripts. -- Use `@Nested` to group related tests by scenario or feature within a test class: - ```java - class FooEncodingTest { - @Nested class Encode { @Test void roundTrips() { ... } } - @Nested class Decode { @Test void handlesEmpty() { ... } } - } - ``` - `@BeforeEach` inside a `@Nested` class applies only to that group. Private helpers go - at the end of the class they serve, after all `@Test` methods. - -## Random-data parameterized tests - -Use `@ParameterizedTest` + `@MethodSource` for random-input coverage. Put generators in `RandomArrays` (integration module) -or a similar utility class. Static provider methods in the test class delegate to the generator: - -```java -static Stream i64ArrayProvider() { return RandomArrays.i64Arrays(30); } - -@ParameterizedTest -@MethodSource("i64ArrayProvider") -void roundTrips(long[] data) { ... } -``` - -Keep counts low (10–30) for integration tests that involve file I/O or JNI. diff --git a/LICENSE b/LICENSE deleted file mode 100644 index 3306d4f..0000000 --- a/LICENSE +++ /dev/null @@ -1,201 +0,0 @@ - Apache License - Version 2.0, January 2004 - http://www.apache.org/licenses/ - - TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION - - 1. Definitions. - - "License" shall mean the terms and conditions for use, reproduction, - and distribution as defined by Sections 1 through 9 of this document. - - "Licensor" shall mean the copyright owner or entity authorized by - the copyright owner that is granting the License. - - "Legal Entity" shall mean the union of the acting entity and all - other entities that control, are controlled by, or are under common - control with that entity. For the purposes of this definition, - "control" means (i) the power, direct or indirect, to cause the - direction or management of such entity, whether by contract or - otherwise, or (ii) ownership of fifty percent (50%) or more of the - outstanding shares, or (iii) beneficial ownership of such entity. - - "You" (or "Your") shall mean an individual or Legal Entity - exercising permissions granted by this License. - - "Source" form shall mean the preferred form for making modifications, - including but not limited to software source code, documentation - source, and configuration files. - - "Object" form shall mean any form resulting from mechanical - transformation or translation of a Source form, including but - not limited to compiled object code, generated documentation, - and conversions to other media types. - - "Work" shall mean the work of authorship, whether in Source or Object - form, made available under the License, as indicated by a copyright - notice that is included in or attached to the work (an example is - provided in the Appendix below). - - "Derivative Works" shall mean any work, whether in Source or Object - form, that is based on (or derived from) the Work and for which the - editorial revisions, annotations, elaborations, or other modifications - represent, as a whole, an original work of authorship. For the purposes - of this License, Derivative Works shall not include works that remain - separable from, or merely link (or bind by name) to the interfaces of, - the Work and Derivative Works thereof. - - "Contribution" shall mean any work of authorship, including - the original version of the Work and any modifications or additions - to that Work or Derivative Works thereof, that is intentionally - submitted to Licensor for inclusion in the Work by the copyright owner - or by an individual or Legal Entity authorized to submit on behalf of - the copyright owner. For the purposes of this definition, "submitted" - means any form of electronic, verbal, or written communication sent - to the Licensor or its representatives, including but not limited to - communication on electronic mailing lists, source code control systems, - and issue tracking systems that are managed by, or on behalf of, the - Licensor for the purpose of discussing and improving the Work, but - excluding communication that is conspicuously marked or otherwise - designated in writing by the copyright owner as "Not a Contribution." - - "Contributor" shall mean Licensor and any individual or Legal Entity - on behalf of whom a Contribution has been received by Licensor and - subsequently incorporated within the Work. - - 2. Grant of Copyright License. Subject to the terms and conditions of - this License, each Contributor hereby grants to You a perpetual, - worldwide, non-exclusive, no-charge, royalty-free, irrevocable - copyright license to reproduce, prepare Derivative Works of, - publicly display, publicly perform, sublicense, and distribute the - Work and such Derivative Works in Source or Object form. - - 3. Grant of Patent License. Subject to the terms and conditions of - this License, each Contributor hereby grants to You a perpetual, - worldwide, non-exclusive, no-charge, royalty-free, irrevocable - (except as stated in this section) patent license to make, have made, - use, offer to sell, sell, import, and otherwise transfer the Work, - where such license applies only to those patent claims licensable - by such Contributor that are necessarily infringed by their - Contribution(s) alone or by combination of their Contribution(s) - with the Work to which such Contribution(s) was submitted. If You - institute patent litigation against any entity (including a - cross-claim or counterclaim in a lawsuit) alleging that the Work - or a Contribution incorporated within the Work constitutes direct - or contributory patent infringement, then any patent licenses - granted to You under this License for that Work shall terminate - as of the date such litigation is filed. - - 4. Redistribution. You may reproduce and distribute copies of the - Work or Derivative Works thereof in any medium, with or without - modifications, and in Source or Object form, provided that You - meet the following conditions: - - (a) You must give any other recipients of the Work or - Derivative Works a copy of this License; and - - (b) You must cause any modified files to carry prominent notices - stating that You changed the files; and - - (c) You must retain, in the Source form of any Derivative Works - that You distribute, all copyright, patent, trademark, and - attribution notices from the Source form of the Work, - excluding those notices that do not pertain to any part of - the Derivative Works; and - - (d) If the Work includes a "NOTICE" text file as part of its - distribution, then any Derivative Works that You distribute must - include a readable copy of the attribution notices contained - within such NOTICE file, excluding those notices that do not - pertain to any part of the Derivative Works, in at least one - of the following places: within a NOTICE text file distributed - as part of the Derivative Works; within the Source form or - documentation, if provided along with the Derivative Works; or, - within a display generated by the Derivative Works, if and - wherever such third-party notices normally appear. The contents - of the NOTICE file are for informational purposes only and - do not modify the License. You may add Your own attribution - notices within Derivative Works that You distribute, alongside - or as an addendum to the NOTICE text from the Work, provided - that such additional attribution notices cannot be construed - as modifying the License. - - You may add Your own copyright statement to Your modifications and - may provide additional or different license terms and conditions - for use, reproduction, or distribution of Your modifications, or - for any such Derivative Works as a whole, provided Your use, - reproduction, and distribution of the Work otherwise complies with - the conditions stated in this License. - - 5. Submission of Contributions. Unless You explicitly state otherwise, - any Contribution intentionally submitted for inclusion in the Work - by You to the Licensor shall be under the terms and conditions of - this License, without any additional terms or conditions. - Notwithstanding the above, nothing herein shall supersede or modify - the terms of any separate license agreement you may have executed - with Licensor regarding such Contributions. - - 6. Trademarks. This License does not grant permission to use the trade - names, trademarks, service marks, or product names of the Licensor, - except as required for reasonable and customary use in describing the - origin of the Work and reproducing the content of the NOTICE file. - - 7. Disclaimer of Warranty. Unless required by applicable law or - agreed to in writing, Licensor provides the Work (and each - Contributor provides its Contributions) on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or - implied, including, without limitation, any warranties or conditions - of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A - PARTICULAR PURPOSE. You are solely responsible for determining the - appropriateness of using or redistributing the Work and assume any - risks associated with Your exercise of permissions under this License. - - 8. Limitation of Liability. In no event and under no legal theory, - whether in tort (including negligence), contract, or otherwise, - unless required by applicable law (such as deliberate and grossly - negligent acts) or agreed to in writing, shall any Contributor be - liable to You for damages, including any direct, indirect, special, - incidental, or consequential damages of any character arising as a - result of this License or out of the use or inability to use the - Work (including but not limited to damages for loss of goodwill, - work stoppage, computer failure or malfunction, or any and all - other commercial damages or losses), even if such Contributor - has been advised of the possibility of such damages. - - 9. Accepting Warranty or Additional Liability. While redistributing - the Work or Derivative Works thereof, You may choose to offer, - and charge a fee for, acceptance of support, warranty, indemnity, - or other liability obligations and/or rights consistent with this - License. However, in accepting such obligations, You may act only - on your own behalf and on your sole responsibility, not on behalf - of any other Contributor, and only if You agree to indemnify, - defend, and hold each Contributor harmless for any liability - incurred by, or claims asserted against, such Contributor by reason - of your accepting any such warranty or additional liability. - - END OF TERMS AND CONDITIONS - - APPENDIX: How to apply the Apache License to your work. - - To apply the Apache License to your work, attach the following - boilerplate notice, with the fields enclosed by brackets "[]" - replaced with your own identifying information. (Don't include - the brackets!) The text should be enclosed in the appropriate - comment syntax for the file format. We also recommend that a - file or class name and description of purpose be included on the - same "printed page" as the copyright notice for easier - identification within third-party archives. - - Copyright 2026 Davide Angelocola - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. diff --git a/README.md b/README.md deleted file mode 100644 index 8b3132a..0000000 --- a/README.md +++ /dev/null @@ -1,97 +0,0 @@ -# vortex-java - -[![CI](https://github.com/dfa1/vortex-java/actions/workflows/ci.yml/badge.svg)](https://github.com/dfa1/vortex-java/actions) -[![Maven Central](https://img.shields.io/maven-central/v/io.github.dfa1.vortex/reader.svg)](https://central.sonatype.com/artifact/io.github.dfa1.vortex/reader) -[![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](https://opensource.org/license/Apache-2.0) - -> **Alpha** — not production-ready. APIs will change without notice. - -Pure-Java reader/writer for the [Vortex](https://github.com/spiraldb/vortex) columnar file format. -100% Java, no JNI, no `sun.misc.Unsafe`. Uses the FFM API (`MemorySegment`/`Arena`, Java 25+) -for zero-copy memory-mapped reads. - -| Project | Language | Notes | -|---|---|---| -| [spiraldb/vortex](https://github.com/spiraldb/vortex) | Rust | Reference implementation + JNI bindings | -| [LaurieRhodes/vortex-go](https://github.com/LaurieRhodes/vortex-go) | Go | Pure-language port | -| **dfa1/vortex-java** | **Java** | **This library** | - -## Who is this for - -- JVM analytics engines and OLAP systems -- Anyone who wants mmap-backed, zero-copy columnar reads without native-library management - -## Quickstart - -```xml - - io.github.dfa1.vortex - reader - 0.2.0 - -``` - -### Minimal read example - -```java -try (VortexReader vf = VortexReader.open(Path.of("data/example.vortex")); - var iter = vf.scan(ScanOptions.all())) { - while (iter.hasNext()) { - var chunk = iter.next(); - LongArray ts = chunk.column("timestamp"); - for (long i = 0; i < ts.length(); i++) { - System.out.println(ts.getLong(i)); - } - } -} -``` - -> **Note:** `iter.hasNext()` closes the previous chunk's arena. Access all column data -> before calling `hasNext()` again. See [docs/explanation.md#memory-model](docs/explanation.md#memory-model). - -For more examples — writing, projection, filtering, custom encodings, and the CLI — -see the documentation below. - -## Documentation - -Docs follow the [Diátaxis](https://diataxis.fr/) framework. - -| Document | Mode | Contents | -|---|---|---| -| [docs/tutorial.md](docs/tutorial.md) | Tutorial | Step-by-step: write and read your first Vortex file | -| [docs/how-to.md](docs/how-to.md) | How-to | Recipes: count rows, convert Parquet, filter, project, custom encodings | -| [docs/reference.md](docs/reference.md) | Reference | API surface, CLI subcommands, operator tables, file-format trailer | -| [docs/compatibility.md](docs/compatibility.md) | Reference | Encoding support table, S3 fixture status | -| [docs/explanation.md](docs/explanation.md) | Explanation | Design rationale, memory model, testing strategy, benchmarks | - -## Development - -**Requirements:** Java 25+ - -Generated sources (`fbs`/`proto` → Java) are committed. Normal builds need no external tools. - -```bash -./mvnw verify # build + tests -./mvnw verify -DskipTests - -# run integration tests -./mvnw verify -pl integration -am - -# benchmarks (always pass ClassName.methodName filter) -./bench RustVsJavaReadBenchmark.javaReadVolume -``` - -To regenerate schemas after editing `.fbs`/`.proto`: - -```bash -brew install flatbuffers protobuf -./mvnw generate-sources -pl core -P regenerate-sources -``` - -## Contributing - -Forks and contributions welcome. Include tests and update documentation where applicable -(see CLAUDE.md for guidelines). - -This project uses [Claude Code](https://claude.ai/code) for implementation work. -Architecture, API design, and all decisions are human-driven. diff --git a/TODO.md b/TODO.md deleted file mode 100644 index aff6f89..0000000 --- a/TODO.md +++ /dev/null @@ -1,194 +0,0 @@ -# TODO - -## Project - -- [ ] Move project to a dedicated organization -- [ ] Create website -- [ ] Publish benchmarks — run `./bench` locally, push JMH JSON to gh-pages via `bench-publish` script, view at `https://jmh.morethan.io/?source=https://dfa1.github.io/vortex-java/benchmark-result.json`; dated files for history comparison via `?source=url1,url2`; then drop `.github/workflows/benchmark.yml` -- [ ] Build something like hardwood.dev but for vortex files - -## Compression ratio gaps vs Rust (NYC taxi 2024-01: Java 43.1 MB, Rust 42.8 MB, Parquet 47.6 MB) - -Progress: raw I64 → datetimeparts (76→53 MB), fix FOR/RLE/RunEnd accepts + sample size (53→51 MB), -fix ConstantEncoding encodeCascade + add ZstdEncoding to CASCADE_CODECS (51→43.1 MB). Java now beats -Parquet and is within 0.3 MB of Rust. - -Remaining 0.3 MB gap — biggest to smallest: - -- [ ] **Global dict encoding** — Rust applies `vortex.dict` across the ENTIRE column before chunking; produces one - tiny dict buffer + one globally-bitpacked codes array. Java applies dict per 131 072-row chunk. Low-cardinality - columns affected: `mta_tax` (8 unique F64), `Airport_fee` (4), `extra` (48), `PULocationID` (260), `DOLocationID` (261), - `payment_type` (5), `store_and_fwd_flag` (3), `congestion_surcharge` (few), `tolls_amount` (1127). - Requires a two-pass write pipeline: first pass collects all values and builds the global dict; second pass emits - coded chunks. Estimated remaining gain ~0.3 MB (most gains already captured by ZstdEncoding on per-chunk dicts). - -## Performance - -- [ ] Publish reproducible perf artifacts - - Capture JMH JSON + JFR profile alongside README table; cite hardware (CPU model), JDK build (`java -version`), - and benchmark commit SHA so numbers don't rot silently. -- [ ] performance tests must be peer reviewed -- [ ] run performance tests on other machines (I have access only to Apple M5) -- [ ] minimize `ctx.arena().allocate(...)` calls — prefer in-place decode when child buffer is writable (already done in ALP); audit all decoders for unnecessary off-heap allocs -- [ ] **Evaluate Vector API (JEP 469+) for hot decode loops** — candidates: FastLanes bitpacked unpack, - FrameOfReference add-base, ZigZag decode, ALP F64 reconstruction, future pco offset+base loop. Measure - vs scalar baseline with JMH; only adopt where speedup is material and code stays readable. Pin against - a specific JDK build since Vector API is incubating until Valhalla lands. - -## Testing - -- [ ] **Security review + adversarial tests** — the reader parses untrusted binary input (file - trailer, FlatBuffers, Protobuf, per-segment data). Attack surface: - - Malformed trailer: wrong magic, negative lengths, offsets past EOF - - FlatBuffer bombs: deeply nested layout trees, circular references, huge vectors - - Proto bombs: enormous `values_len`/`indices_len` in metadata triggering OOM allocations - - Integer overflows in offset arithmetic (`offset + length` wraps to negative) - - Out-of-bounds buffer reads via crafted `bufferIndices` arrays - - Zip-bomb style: tiny file that claims huge row counts - Add a fuzz corpus of malformed `.vortex` files; all must throw `VortexException`, never - `ArrayIndexOutOfBoundsException`, `NegativeArraySizeException`, or `OutOfMemoryError`. -- [ ] **Verify proto compatibility with upstream** — `dtype.proto` and `scalar.proto` exist in - `spiraldb/vortex/vortex-proto/proto/` and should be kept in sync with our copies. Encoding - metadata (e.g. `RLEMetadata`, `RunEndMetadata`) has no upstream `.proto`; tag mismatches - silently produce zero/default values in proto3. Add value-level assertions (not just - `rowCount > 0`) to integration tests to catch silent corruption. - -## Build - -- [ ] **Merge `core`/`reader`/`writer` into a single library jar** — the three modules are tightly - coupled (every `Encoding` class has both encode + decode; format constants are shared). A single - `vortex-java` artifact simplifies client dependency management and removes artificial module - boundaries. Keep `integration`, `performance`, and `cli` as separate modules. Package structure - (`encoding`, `io`, `writer`) already enforces internal boundaries without Maven. -- [ ] prefix all modules with "vortex-" -- [ ] switch back to module-path, but keep in mind these 2 blockers - -## Documentation - -- [ ] Format specification: byte-exact diagrams for file layout and each encoding, with annotated examples (Arrow spec style) - -## Tooling - - -- [ ] Optional `vortex-arrow` bridge module for Arrow ecosystem interop - - Primary API stays `ArrayLong`/`ArrayDouble` (zero-copy, no deps, no Unsafe) - - Bridge wraps typed views into Arrow `BigIntVector`, `Float8Vector`, etc. for users who need - Arrow Flight / DuckDB ADBC / pandas interop - - Conversion involves a copy (MemorySegment → Arrow off-heap buffer) — cost is explicit and opt-in - - Arrow JVM uses `sun.misc.Unsafe` / Netty internally; keeping it in a separate module means - the core library stays Unsafe-free - -## Large-file support - -- [ ] **Test read/write of files > 2 GB** - - [ ] Parquet baseline for comparison: same data should fail or require splitting when any - column chunk exceeds 2 GB. - -## API - -- [ ] Use domain primitives (`UInt32`, `UInt64`, etc.) as value classes via Project Valhalla instead of raw `long`/`int` - - See https://dfa1.github.io/articles/rethink-domain-primitives-with-valhalla - - Candidates: `PType` integer kinds, buffer offsets, row indices, byte lengths - - Goal: type-safety at zero cost (value class = no heap alloc, no boxing) - -- [ ] **Audit runtime pluggability vs Rust impl** — maintainer (2026-06-04) flagged that Rust supports - runtime registration for: Encodings, DTypes, Compute, Layouts. Java status: - - Encodings: ✅ `ServiceLoader` + `EncodingRegistry.register()`; ✅ `allowUnknown()` passthrough for unregistered encodings (mirrors `VortexSession::allow_unknown()`) - - DTypes: ❌ sealed hierarchy — no user-extensible type. If a downstream consumer needs a custom - logical type (e.g. UUID, IP address) they can't register one. Decide: keep sealed (simpler) or - open via SPI mirroring `EncodingRegistry`. - - Layouts: ❌ fixed set (Flat/Chunked/Zoned/Struct). Same trade-off as DTypes. - - Compute: ❌ no compute layer yet. Out of scope until reader feature-complete. - Action: short design note weighing sealed-vs-pluggable for DType + Layout; revisit when Java impl - has a real downstream consumer asking for it. Don't pre-open these without a use case. - - -## Encodings - -See [docs/compatibility.md](docs/compatibility.md) for the full encoding support table and S3 fixture status. - -### `vortex.zstd` known limitations - - -- [ ] **Multi-frame encode** — `ZstdEncoding.Encoder` always produces a single frame for the whole array. - Fix: accept a `valuesPerFrame` parameter (default: all values in one frame). Split the raw byte buffer at frame boundaries (`valuesPerFrame * byteWidth`), compress each slice independently, emit one `ZstdFrameMetadata` per frame. Enables partial decompression during slice scans. - -- [ ] **Nullable arrays (encode)** — `ZstdEncoding.Encoder` has no null handling. - Fix: accept nullable input (e.g. `Integer[]` or a validity mask alongside the data array). Strip null positions before compression. Encode the validity bitmap as a Bool child (child[0]) in the `EncodeNode`. Mirrors what Rust does: only valid values go into the compressed payload. - - -### `vortex.pco` encode plan - -Pure-Java encode. Only after decode is stable + a Java consumer asks for write. Not gated -on any S3 fixture (all fixtures are Rust-produced; decode unblocks them). - -**Refs**: [pco/src/wrapped/chunk_compressor.rs](https://github.com/pcodec/pcodec/blob/main/pco/src/wrapped/chunk_compressor.rs), -[pco/src/bin_optimization.rs](https://github.com/pcodec/pcodec/blob/main/pco/src/bin_optimization.rs), -[pco/src/histograms.rs](https://github.com/pcodec/pcodec/blob/main/pco/src/histograms.rs), -[pco/src/ans/](https://github.com/pcodec/pcodec/tree/main/pco/src/ans), -[pco/src/sampling.rs](https://github.com/pcodec/pcodec/blob/main/pco/src/sampling.rs). - -**Why harder than decode**: -- Encode chooses mode + bin layout + tANS weights; decode just executes a fixed program. -- Bin optimization is dynamic programming over partitions of a histogram (`bin_cost`). -- tANS encoding table differs from decode table (weight quantization → symbol table). -- Mode selection samples input, trial-compresses against candidates (Classic, FloatMult, - IntMult, FloatQuant), picks best ratio. See `sampling.rs`. -- No oracle: encode is non-deterministic. Validation = round-trip Java→Java decode AND - Java→Rust decode (existing `JavaWritesRustReadsIntegrationTest` harness). - -**Reuse from decode**: -- `LeBitReader` (decode) ↔ `LeBitWriter` (encode, new). Same bit layout, opposite direction. -- tANS table structure (decode-built) ↔ tANS encode table (`ans/encoding.rs`). -- Mode constants, delta constants, proto types — shared. -- Bit-exact wire format already validated by decode tests; encode just emits same bytes. - -**Phases**: -- [ ] **Phase E0 — gate**. Is there a consumer (CLI write path, vortex-arrow bridge)? If no, - stop. Decode is enough. -- [ ] **Phase E1 — bit writer**. `LeBitWriter` over `Arena`-backed `MemorySegment`. Mirrors - `pco/src/bit_writer.rs`. Property test: random bit sequences round-trip via `LeBitReader`. -- [ ] **Phase E2 — Classic mode, no delta, fixed bins, no optimization**. Hardcoded bin layout - + uniform tANS weights. Emits a valid (suboptimal) pco stream. Validates: header write, - chunk meta write, page write, byte alignment. Round-trip via Java decode. -- [ ] **Phase E3 — histogram + bin optimization**. Port `histograms.rs` (sort + bucket by - latent prefix) and `bin_optimization.rs` (DP partitioning, `bin_cost`, `log2_approx`). - Replace fixed bins with optimized layout. Compression ratio benchmark vs Rust on same - input — accept if Java within 5% of Rust ratio. -- [ ] **Phase E4 — tANS weight quantization + encoding table**. Port `ans/spec.rs` weight - quantizer and `ans/encoding.rs` symbol-table builder. Critical: ANS state values must - match what Rust decoder expects. -- [ ] **Phase E5 — delta Consecutive encoder**. Compute consecutive differences; store - initial state at page head. Mirrors Phase 4 of decode. -- [ ] **Phase E6 — mode selection**. Port `sampling.rs`: take stratified sample, trial-encode - with each candidate mode, pick lowest bit count. Add FloatMult, IntMult (likely), FloatQuant. - Skip Dict + Lookback + Conv1 unless requested. -- [ ] **Phase E7 — multi-chunk, multi-page, nullable**. Match decode: split into chunks of - `DEFAULT_MAX_PAGE_N`, pages per `ChunkConfig.paging_spec`. For nullable input, strip - nulls before encode, emit validity as child[0]. -- [ ] **Phase E8 — integration tests**. `JavaWritesRustReadsIntegrationTest`: produce a - `.vortex` with pco-encoded column, validate the Rust reference reader decodes it - byte-identical to input. Property test with `tries` low. -- [ ] **Phase E9 — `EncodeResult` glue**. `PcoEncoding.Encoder.encode(dtype, data)` returns - `EncodeNode(VORTEX_PCO, metadata, no children OR validity child, bufferIndices)` with - `chunk_metas` then `pages` as separate buffer indices. - -**Risks**: -- Bin optimization DP: bug → catastrophic ratio loss but still valid output. Symptom is - silent — only benchmarks catch it. Test ratio against Rust on known inputs. -- tANS weight quantization: bug → Rust decoder rejects with checksum mismatch. Caught fast - by Java→Rust integration test. -- Mode selection: wrong mode = valid output but poor ratio. Same silent failure as bin DP. -- `log2_approx` is a fast-math hack. Java port can use `Math.log` (slower but exact); - measure JMH cost before chasing parity. -- Encode unit test oracle problem: easier to assert bit-exact against a recorded Rust output - for fixed inputs than to assert "optimal encoding" — record golden encodings per ptype. - -**Estimate**: ~20 working days full encode. ~8 days for Classic+Consecutive+I64-only "valid -but suboptimal" encoder (Phase E1+E2+E5+E8 partial). Decode is the prerequisite — -don't start before decode lands. - -**Decision**: keep `Encoder` stub until a real write consumer materializes. Reassess -post-decode + post-`vortex-arrow` bridge. - - diff --git a/bench b/bench deleted file mode 100755 index 3a4f3d3..0000000 --- a/bench +++ /dev/null @@ -1,11 +0,0 @@ -#!/usr/bin/env bash -set -euo pipefail -./mvnw package -DskipTests -java \ - --add-opens java.base/java.nio=ALL-UNNAMED \ - --enable-native-access=ALL-UNNAMED \ - --sun-misc-unsafe-memory-access=allow \ - -jar performance/target/benchmarks.jar \ - "${@:-.+}" \ - -rf json \ - -rff performance/target/benchmark-result.json diff --git a/bom/pom.xml b/bom/pom.xml deleted file mode 100644 index 058f40f..0000000 --- a/bom/pom.xml +++ /dev/null @@ -1,50 +0,0 @@ - - - 4.0.0 - - io.github.dfa1.vortex - vortex-java - 0.3.0-SNAPSHOT - - - bom - pom - - vortex-java-bom - Bill of Materials for vortex-java. Import this POM in dependencyManagement to get consistent versions of all vortex-java modules. - - - - - io.github.dfa1.vortex - core - ${project.version} - - - io.github.dfa1.vortex - reader - ${project.version} - - - io.github.dfa1.vortex - writer - ${project.version} - - - io.github.dfa1.vortex - csv - ${project.version} - - - io.github.dfa1.vortex - parquet - ${project.version} - - - io.github.dfa1.vortex - jdbc - ${project.version} - - - - diff --git a/checkstyle.xml b/checkstyle.xml deleted file mode 100644 index d65f57c..0000000 --- a/checkstyle.xml +++ /dev/null @@ -1,62 +0,0 @@ - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - diff --git a/cli/pom.xml b/cli/pom.xml deleted file mode 100644 index 8009231..0000000 --- a/cli/pom.xml +++ /dev/null @@ -1,90 +0,0 @@ - - - 4.0.0 - - io.github.dfa1.vortex - vortex-java - 0.3.0-SNAPSHOT - - - cli - - - true - - - - - io.github.dfa1.vortex - csv - - - io.github.dfa1.vortex - parquet - - - io.github.dfa1.vortex - reader - - - - org.junit.jupiter - junit-jupiter - test - - - org.assertj - assertj-core - test - - - - - vortex - - - org.apache.maven.plugins - maven-surefire-plugin - - --sun-misc-unsafe-memory-access=allow - - **/*Test.java - **/*IT.java - - - - - org.apache.maven.plugins - maven-shade-plugin - - - package - - shade - - - false - - - io.github.dfa1.vortex.cli.VortexCli - - - - - - *:* - - module-info.class - META-INF/*.SF - META-INF/*.DSA - META-INF/*.RSA - - - - - - - - - - diff --git a/cli/src/main/java/io/github/dfa1/vortex/cli/CountCommand.java b/cli/src/main/java/io/github/dfa1/vortex/cli/CountCommand.java deleted file mode 100644 index 0629186..0000000 --- a/cli/src/main/java/io/github/dfa1/vortex/cli/CountCommand.java +++ /dev/null @@ -1,32 +0,0 @@ -package io.github.dfa1.vortex.cli; - -import io.github.dfa1.vortex.io.VortexReader; - -import java.io.IOException; -import java.nio.file.Files; -import java.nio.file.Path; - -final class CountCommand { - - private CountCommand() { - } - - static int run(String[] args) { - if (args.length != 2) { - System.err.println("usage: count "); - return ExitStatus.USAGE_ERROR; - } - Path path = Path.of(args[1]); - if (!Files.exists(path)) { - System.err.println("file not found: " + path); - return ExitStatus.FILE_NOT_FOUND; - } - try (VortexReader reader = VortexReader.open(path)) { - System.out.println(reader.layout().rowCount()); - return ExitStatus.OK; - } catch (IOException e) { - System.err.println("error: " + e.getMessage()); - return ExitStatus.ERROR; - } - } -} diff --git a/cli/src/main/java/io/github/dfa1/vortex/cli/ExitStatus.java b/cli/src/main/java/io/github/dfa1/vortex/cli/ExitStatus.java deleted file mode 100644 index 3682059..0000000 --- a/cli/src/main/java/io/github/dfa1/vortex/cli/ExitStatus.java +++ /dev/null @@ -1,12 +0,0 @@ -package io.github.dfa1.vortex.cli; - -public final class ExitStatus { - - public static final int OK = 0; - public static final int USAGE_ERROR = 1; - public static final int FILE_NOT_FOUND = 2; - public static final int ERROR = 3; - - private ExitStatus() { - } -} diff --git a/cli/src/main/java/io/github/dfa1/vortex/cli/ExportCommand.java b/cli/src/main/java/io/github/dfa1/vortex/cli/ExportCommand.java deleted file mode 100644 index b94324a..0000000 --- a/cli/src/main/java/io/github/dfa1/vortex/cli/ExportCommand.java +++ /dev/null @@ -1,38 +0,0 @@ -package io.github.dfa1.vortex.cli; - -import io.github.dfa1.vortex.csv.CsvExporter; -import io.github.dfa1.vortex.csv.ExportOptions; - -import java.io.IOException; -import java.io.OutputStreamWriter; -import java.io.Writer; -import java.nio.charset.StandardCharsets; -import java.nio.file.Files; -import java.nio.file.Path; - -final class ExportCommand { - - private ExportCommand() { - } - - static int run(String[] args) { - if (args.length != 2) { - System.err.println("usage: export "); - return ExitStatus.USAGE_ERROR; - } - Path path = Path.of(args[1]); - if (!Files.exists(path)) { - System.err.println("file not found: " + path); - return ExitStatus.FILE_NOT_FOUND; - } - try { - Writer stdout = new OutputStreamWriter(System.out, StandardCharsets.UTF_8); - CsvExporter.exportCsv(path, stdout, ExportOptions.defaults()); - stdout.flush(); - return ExitStatus.OK; - } catch (IOException e) { - System.err.println("error: " + e.getMessage()); - return ExitStatus.ERROR; - } - } -} diff --git a/cli/src/main/java/io/github/dfa1/vortex/cli/FilterCommand.java b/cli/src/main/java/io/github/dfa1/vortex/cli/FilterCommand.java deleted file mode 100644 index 2f4d213..0000000 --- a/cli/src/main/java/io/github/dfa1/vortex/cli/FilterCommand.java +++ /dev/null @@ -1,158 +0,0 @@ -package io.github.dfa1.vortex.cli; - -import io.github.dfa1.vortex.core.array.BoolArray; -import io.github.dfa1.vortex.core.array.ByteArray; -import io.github.dfa1.vortex.core.array.DoubleArray; -import io.github.dfa1.vortex.core.array.FloatArray; -import io.github.dfa1.vortex.core.array.IntArray; -import io.github.dfa1.vortex.core.array.LongArray; -import io.github.dfa1.vortex.core.array.ShortArray; -import io.github.dfa1.vortex.core.array.VarBinArray; -import io.github.dfa1.vortex.core.array.Array; -import io.github.dfa1.vortex.csv.CsvExporter; -import io.github.dfa1.vortex.csv.ExportOptions; -import io.github.dfa1.vortex.csv.RowPredicate; -import io.github.dfa1.vortex.scan.RowFilter; -import io.github.dfa1.vortex.scan.ScanOptions; - -import java.io.IOException; -import java.io.OutputStreamWriter; -import java.io.Writer; -import java.nio.charset.StandardCharsets; -import java.nio.file.Files; -import java.nio.file.Path; -import java.util.Arrays; -import java.util.List; -import java.util.regex.Matcher; -import java.util.regex.Pattern; - -final class FilterCommand { - - private FilterCommand() { - } - - private static final Pattern EXPR = Pattern.compile( - "^(\\w[\\w.]*?)\\s*(!=|>=|<=|==|>|<|=)\\s*(.+)$"); - - static int run(String[] args) { - if (args.length < 3) { - System.err.println("usage: filter (e.g. \"price >= 100\")"); - return ExitStatus.USAGE_ERROR; - } - Path path = Path.of(args[1]); - if (!Files.exists(path)) { - System.err.println("file not found: " + path); - return ExitStatus.FILE_NOT_FOUND; - } - String expr = String.join(" ", Arrays.asList(args).subList(2, args.length)); - RowFilter filter; - try { - filter = parseFilter(expr); - } catch (IllegalArgumentException e) { - System.err.println("error: " + e.getMessage()); - return ExitStatus.USAGE_ERROR; - } - ScanOptions scanOptions = new ScanOptions(List.of(), filter, ScanOptions.NO_LIMIT); - RowPredicate rowPred = toRowPredicate(filter); - try { - Writer stdout = new OutputStreamWriter(System.out, StandardCharsets.UTF_8); - CsvExporter.exportCsvFiltered(path, stdout, ExportOptions.defaults(), scanOptions, rowPred); - stdout.flush(); - return ExitStatus.OK; - } catch (IOException e) { - System.err.println("error: " + e.getMessage()); - return ExitStatus.ERROR; - } - } - - private static RowFilter parseFilter(String expr) { - Matcher m = EXPR.matcher(expr.trim()); - if (!m.matches()) { - throw new IllegalArgumentException("invalid filter expression: \"" + expr - + "\" expected: col op value (op: >, >=, <, <=, =, ==, !=)"); - } - String col = m.group(1); - Comparable value = parseValue(m.group(3).trim()); - return switch (m.group(2)) { - case ">" -> RowFilter.gt(col, value); - case ">=" -> RowFilter.gte(col, value); - case "<" -> RowFilter.lt(col, value); - case "<=" -> RowFilter.lte(col, value); - case "=", "==" -> RowFilter.eq(col, value); - case "!=" -> RowFilter.neq(col, value); - default -> throw new IllegalArgumentException("unknown operator: " + m.group(2)); - }; - } - - private static Comparable parseValue(String raw) { - try { - return Long.parseLong(raw); - } catch (NumberFormatException ignored) { - } - try { - return Double.parseDouble(raw); - } catch (NumberFormatException ignored) { - } - if ("true".equalsIgnoreCase(raw)) { - return Boolean.TRUE; - } - if ("false".equalsIgnoreCase(raw)) { - return Boolean.FALSE; - } - return raw; - } - - private static RowPredicate toRowPredicate(RowFilter filter) { - return switch (filter) { - case RowFilter.Gt(var col, var val) -> (chunk, rowIdx) -> compareValue(chunk.column(col), rowIdx, val) > 0; - case RowFilter.Gte(var col, var val) -> (chunk, rowIdx) -> compareValue(chunk.column(col), rowIdx, val) >= 0; - case RowFilter.Lt(var col, var val) -> (chunk, rowIdx) -> compareValue(chunk.column(col), rowIdx, val) < 0; - case RowFilter.Lte(var col, var val) -> (chunk, rowIdx) -> compareValue(chunk.column(col), rowIdx, val) <= 0; - case RowFilter.Eq(var col, var val) -> (chunk, rowIdx) -> compareValue(chunk.column(col), rowIdx, (Comparable) val) == 0; - case RowFilter.Neq(var col, var val) -> (chunk, rowIdx) -> compareValue(chunk.column(col), rowIdx, (Comparable) val) != 0; - case RowFilter.And(var filters) -> { - RowPredicate[] preds = filters.stream().map(FilterCommand::toRowPredicate).toArray(RowPredicate[]::new); - yield (chunk, rowIdx) -> { - for (RowPredicate p : preds) { - if (!p.test(chunk, rowIdx)) { - return false; - } - } - return true; - }; - } - }; - } - - private static int compareValue(Array arr, long rowIdx, Comparable value) { - return switch (arr) { - case LongArray la -> compareNumeric(la.getLong(rowIdx), value); - case IntArray ia -> compareNumeric(ia.getInt(rowIdx), value); - case ShortArray sa -> compareNumeric(sa.getShort(rowIdx), value); - case ByteArray ba -> compareNumeric(ba.getByte(rowIdx), value); - case DoubleArray da -> compareDouble(da.getDouble(rowIdx), value); - case FloatArray fa -> compareDouble(fa.getFloat(rowIdx), value); - case BoolArray ba -> Boolean.compare(ba.getBoolean(rowIdx), (Boolean) value); - case VarBinArray va -> { - String v = va.getString(rowIdx); - yield v.compareTo((String) value); - } - default -> throw new IllegalArgumentException( - "filter not supported for column type: " + arr.getClass().getSimpleName()); - }; - } - - private static int compareNumeric(long colVal, Comparable value) { - if (value instanceof Long l) { - return Long.compare(colVal, l); - } - return Double.compare(colVal, (Double) value); - } - - private static int compareDouble(double colVal, Comparable value) { - if (value instanceof Long l) { - return Double.compare(colVal, l); - } - return Double.compare(colVal, (Double) value); - } -} diff --git a/cli/src/main/java/io/github/dfa1/vortex/cli/ImportCommand.java b/cli/src/main/java/io/github/dfa1/vortex/cli/ImportCommand.java deleted file mode 100644 index d0819d6..0000000 --- a/cli/src/main/java/io/github/dfa1/vortex/cli/ImportCommand.java +++ /dev/null @@ -1,107 +0,0 @@ -package io.github.dfa1.vortex.cli; - -import io.github.dfa1.vortex.csv.CsvImporter; -import io.github.dfa1.vortex.csv.ImportOptions; -import io.github.dfa1.vortex.parquet.ParquetImporter; - -import java.io.IOException; -import java.nio.file.Files; -import java.nio.file.Path; - -final class ImportCommand { - - private ImportCommand() { - } - - static int run(String[] args) { - if (args.length < 2 || args.length > 3) { - System.err.println("usage: import [out.vortex]"); - return ExitStatus.USAGE_ERROR; - } - Path inputPath = Path.of(args[1]); - if (!Files.exists(inputPath)) { - System.err.println("file not found: " + inputPath); - return ExitStatus.FILE_NOT_FOUND; - } - Path vortexPath = args.length == 3 ? Path.of(args[2]) : deriveOutputPath(inputPath); - try { - String name = inputPath.getFileName().toString(); - if (name.endsWith(".parquet")) { - return runParquet(inputPath, vortexPath); - } else { - return runCsv(inputPath, vortexPath); - } - } catch (IOException e) { - clearProgress(); - System.err.println("error: " + e.getMessage()); - return ExitStatus.ERROR; - } - } - - private static int runCsv(Path csvPath, Path vortexPath) throws IOException { - ImportOptions options = ImportOptions.defaults() - .withProgressListener(ImportCommand::renderProgress); - CsvImporter.importCsv(csvPath, vortexPath, options); - clearProgress(); - printResult(csvPath, vortexPath, options.writeOptions().allowedCascading()); - return ExitStatus.OK; - } - - private static int runParquet(Path parquetPath, Path vortexPath) throws IOException { - io.github.dfa1.vortex.parquet.ImportOptions options = - io.github.dfa1.vortex.parquet.ImportOptions.defaults() - .withProgressListener(ImportCommand::renderProgress); - ParquetImporter.importParquet(parquetPath, vortexPath, options); - clearProgress(); - printResult(parquetPath, vortexPath, options.writeOptions().allowedCascading()); - return ExitStatus.OK; - } - - private static void printResult(Path inputPath, Path vortexPath, int cascadingDepth) throws IOException { - long inputBytes = Files.size(inputPath); - long vortexBytes = Files.size(vortexPath); - double ratio = (double) vortexBytes / inputBytes; - String sizeChange = ratio <= 1.0 - ? String.format("%.1f%% smaller", (1.0 - ratio) * 100) - : String.format("%.1f%% larger", (ratio - 1.0) * 100); - String cascadingInfo = cascadingDepth > 0 - ? String.format(", cascading depth %d", cascadingDepth) - : ""; - System.out.printf("written: %s (%s → %s, %s%s)%n", - vortexPath, formatBytes(inputBytes), formatBytes(vortexBytes), - sizeChange, cascadingInfo); - } - - private static void renderProgress(long done, long total) { - int pct = total > 0 ? (int) (done * 100L / total) : 100; - int filled = pct * 30 / 100; - String bar = "=".repeat(filled) + (filled < 30 ? ">" : "") + " ".repeat(Math.max(0, 29 - filled)); - System.err.printf("\r [%s] %3d%% %,d / %,d rows", bar, pct, done, total); - System.err.flush(); - } - - private static void clearProgress() { - System.err.printf("\r%-80s\r", ""); - System.err.flush(); - } - - private static String formatBytes(long bytes) { - if (bytes < 1024L) { - return bytes + " B"; - } - if (bytes < 1024L * 1024) { - return String.format("%.1f KB", bytes / 1024.0); - } - return String.format("%.1f MB", bytes / (1024.0 * 1024)); - } - - private static Path deriveOutputPath(Path inputPath) { - String name = inputPath.getFileName().toString(); - if (name.endsWith(".csv")) { - name = name.substring(0, name.length() - 4); - } else if (name.endsWith(".parquet")) { - name = name.substring(0, name.length() - 8); - } - return inputPath.resolveSibling(name + ".vortex"); - } -} diff --git a/cli/src/main/java/io/github/dfa1/vortex/cli/InspectCommand.java b/cli/src/main/java/io/github/dfa1/vortex/cli/InspectCommand.java deleted file mode 100644 index c79d130..0000000 --- a/cli/src/main/java/io/github/dfa1/vortex/cli/InspectCommand.java +++ /dev/null @@ -1,33 +0,0 @@ -package io.github.dfa1.vortex.cli; - -import io.github.dfa1.vortex.io.VortexInspector; -import io.github.dfa1.vortex.io.VortexReader; - -import java.io.IOException; -import java.nio.file.Files; -import java.nio.file.Path; - -final class InspectCommand { - - private InspectCommand() { - } - - static int run(String[] args) { - if (args.length != 2) { - System.err.println("usage: inspect "); - return ExitStatus.USAGE_ERROR; - } - Path path = Path.of(args[1]); - if (!Files.exists(path)) { - System.err.println("file not found: " + path); - return ExitStatus.FILE_NOT_FOUND; - } - try (VortexReader reader = VortexReader.open(path)) { - System.out.print(VortexInspector.inspect(reader)); - return ExitStatus.OK; - } catch (IOException e) { - System.err.println("error: " + e.getMessage()); - return ExitStatus.ERROR; - } - } -} diff --git a/cli/src/main/java/io/github/dfa1/vortex/cli/SchemaCommand.java b/cli/src/main/java/io/github/dfa1/vortex/cli/SchemaCommand.java deleted file mode 100644 index 079b0b5..0000000 --- a/cli/src/main/java/io/github/dfa1/vortex/cli/SchemaCommand.java +++ /dev/null @@ -1,60 +0,0 @@ -package io.github.dfa1.vortex.cli; - -import io.github.dfa1.vortex.core.DType; -import io.github.dfa1.vortex.io.VortexReader; - -import java.io.IOException; -import java.nio.file.Files; -import java.nio.file.Path; - -final class SchemaCommand { - - private SchemaCommand() { - } - - static int run(String[] args) { - if (args.length != 2) { - System.err.println("usage: schema "); - return ExitStatus.USAGE_ERROR; - } - Path path = Path.of(args[1]); - if (!Files.exists(path)) { - System.err.println("file not found: " + path); - return ExitStatus.FILE_NOT_FOUND; - } - try (VortexReader reader = VortexReader.open(path)) { - System.out.println(formatDType(reader.dtype())); - return ExitStatus.OK; - } catch (IOException e) { - System.err.println("error: " + e.getMessage()); - return ExitStatus.ERROR; - } - } - - private static String formatDType(DType dtype) { - return switch (dtype) { - case DType.Struct s -> { - var sb = new StringBuilder("struct<"); - for (int i = 0; i < s.fieldNames().size(); i++) { - if (i > 0) { - sb.append(", "); - } - sb.append(s.fieldNames().get(i)).append(": ").append(formatDType(s.fieldTypes().get(i))); - } - sb.append('>'); - yield sb.toString(); - } - case DType.Primitive(var pt, var nullable) -> pt.name() + (nullable ? "?" : ""); - case DType.Utf8(var nullable) -> "utf8" + (nullable ? "?" : ""); - case DType.Binary(var nullable) -> "binary" + (nullable ? "?" : ""); - case DType.Bool(var nullable) -> "bool" + (nullable ? "?" : ""); - case DType.Null ignored -> "null"; - case DType.Decimal(var p, var s, var nullable) -> "decimal(" + p + "," + s + ")" + (nullable ? "?" : ""); - case DType.List(var elem, var nullable) -> "list<" + formatDType(elem) + ">" + (nullable ? "?" : ""); - case DType.FixedSizeList(var elem, var size, var nullable) -> - "list<" + formatDType(elem) + ">[" + size + "]" + (nullable ? "?" : ""); - case DType.Extension(var id, var storage, var meta, var nullable) -> - "ext<" + id + ">" + (nullable ? "?" : ""); - }; - } -} diff --git a/cli/src/main/java/io/github/dfa1/vortex/cli/SelectCommand.java b/cli/src/main/java/io/github/dfa1/vortex/cli/SelectCommand.java deleted file mode 100644 index d31b448..0000000 --- a/cli/src/main/java/io/github/dfa1/vortex/cli/SelectCommand.java +++ /dev/null @@ -1,42 +0,0 @@ -package io.github.dfa1.vortex.cli; - -import io.github.dfa1.vortex.csv.CsvExporter; -import io.github.dfa1.vortex.csv.ExportOptions; - -import java.io.IOException; -import java.io.OutputStreamWriter; -import java.io.Writer; -import java.nio.charset.StandardCharsets; -import java.nio.file.Files; -import java.nio.file.Path; -import java.util.Arrays; -import java.util.List; - -final class SelectCommand { - - private SelectCommand() { - } - - static int run(String[] args) { - if (args.length < 3) { - System.err.println("usage: select [col2 ...]"); - return ExitStatus.USAGE_ERROR; - } - Path path = Path.of(args[1]); - if (!Files.exists(path)) { - System.err.println("file not found: " + path); - return ExitStatus.FILE_NOT_FOUND; - } - List columns = Arrays.asList(args).subList(2, args.length); - ExportOptions options = ExportOptions.defaults().withColumns(columns); - try { - Writer stdout = new OutputStreamWriter(System.out, StandardCharsets.UTF_8); - CsvExporter.exportCsv(path, stdout, options); - stdout.flush(); - return ExitStatus.OK; - } catch (IOException e) { - System.err.println("error: " + e.getMessage()); - return ExitStatus.ERROR; - } - } -} diff --git a/cli/src/main/java/io/github/dfa1/vortex/cli/StatsCommand.java b/cli/src/main/java/io/github/dfa1/vortex/cli/StatsCommand.java deleted file mode 100644 index 0f4782f..0000000 --- a/cli/src/main/java/io/github/dfa1/vortex/cli/StatsCommand.java +++ /dev/null @@ -1,66 +0,0 @@ -package io.github.dfa1.vortex.cli; - -import io.github.dfa1.vortex.core.ArrayStats; -import io.github.dfa1.vortex.core.DType; -import io.github.dfa1.vortex.io.VortexReader; - -import java.io.IOException; -import java.nio.file.Files; -import java.nio.file.Path; -import java.util.List; -import java.util.Map; - -final class StatsCommand { - - private StatsCommand() { - } - - static int run(String[] args) { - if (args.length != 2) { - System.err.println("usage: stats "); - return ExitStatus.USAGE_ERROR; - } - Path path = Path.of(args[1]); - if (!Files.exists(path)) { - System.err.println("file not found: " + path); - return ExitStatus.FILE_NOT_FOUND; - } - try (VortexReader reader = VortexReader.open(path)) { - if (!(reader.dtype() instanceof DType.Struct schema)) { - System.err.println("error: root dtype is not a struct"); - return ExitStatus.ERROR; - } - long totalRows = reader.layout().rowCount(); - Map stats = reader.columnStats(); - - System.out.printf("rows: %,d%n%n", totalRows); - System.out.printf("%-20s %-12s %15s %15s%n", "column", "type", "min", "max"); - System.out.println("-".repeat(67)); - - List names = schema.fieldNames(); - List types = schema.fieldTypes(); - for (int i = 0; i < names.size(); i++) { - String name = names.get(i); - String type = formatDType(types.get(i)); - ArrayStats s = stats.getOrDefault(name, ArrayStats.empty()); - String min = s.min() != null ? s.min().toString() : "n/a"; - String max = s.max() != null ? s.max().toString() : "n/a"; - System.out.printf("%-20s %-12s %15s %15s%n", name, type, min, max); - } - return ExitStatus.OK; - } catch (IOException e) { - System.err.println("error: " + e.getMessage()); - return ExitStatus.ERROR; - } - } - - private static String formatDType(DType dtype) { - return switch (dtype) { - case DType.Primitive p -> p.ptype().name().toLowerCase(); - case DType.Bool ignored -> "bool"; - case DType.Utf8 ignored -> "utf8"; - case DType.Struct ignored -> "struct"; - default -> dtype.getClass().getSimpleName().toLowerCase(); - }; - } -} diff --git a/cli/src/main/java/io/github/dfa1/vortex/cli/VortexCli.java b/cli/src/main/java/io/github/dfa1/vortex/cli/VortexCli.java deleted file mode 100644 index 394b10f..0000000 --- a/cli/src/main/java/io/github/dfa1/vortex/cli/VortexCli.java +++ /dev/null @@ -1,47 +0,0 @@ -package io.github.dfa1.vortex.cli; - -import java.io.PrintStream; - -/// Entry point for the Vortex command-line tool. -/// -/// Exit codes: see {@link ExitStatus}. -public final class VortexCli { - - private VortexCli() { - } - - public static void main(String[] args) { - if (args.length == 0) { - printUsage(System.err); - System.exit(ExitStatus.USAGE_ERROR); - } - int exit = switch (args[0]) { - case "inspect" -> InspectCommand.run(args); - case "export" -> ExportCommand.run(args); - case "import" -> ImportCommand.run(args); - case "schema" -> SchemaCommand.run(args); - case "count" -> CountCommand.run(args); - case "select" -> SelectCommand.run(args); - case "stats" -> StatsCommand.run(args); - case "filter" -> FilterCommand.run(args); - default -> { - System.err.println("unknown subcommand: " + args[0]); - printUsage(System.err); - yield ExitStatus.USAGE_ERROR; - } - }; - System.exit(exit); - } - - static void printUsage(PrintStream out) { - out.println("Usage: java -jar vortex.jar [args]"); - out.println(" inspect print file structure"); - out.println(" export write CSV to stdout"); - out.println(" import [out.vortex] convert CSV or Parquet to Vortex"); - out.println(" schema print dtype (machine-readable)"); - out.println(" count print row count"); - out.println(" select [...] project columns to CSV on stdout"); - out.println(" stats print per-column min/max statistics"); - out.println(" filter filter rows to CSV (e.g. \"price >= 100\")"); - } -} diff --git a/cli/src/test/java/io/github/dfa1/vortex/cli/CliIT.java b/cli/src/test/java/io/github/dfa1/vortex/cli/CliIT.java deleted file mode 100644 index 6612da3..0000000 --- a/cli/src/test/java/io/github/dfa1/vortex/cli/CliIT.java +++ /dev/null @@ -1,159 +0,0 @@ -package io.github.dfa1.vortex.cli; - -import org.junit.jupiter.api.Test; -import org.junit.jupiter.api.io.TempDir; - -import java.io.ByteArrayOutputStream; -import java.io.PrintStream; -import java.nio.charset.StandardCharsets; -import java.nio.file.Files; -import java.nio.file.Path; -import java.util.Objects; -import java.util.function.IntSupplier; - -import static org.assertj.core.api.Assertions.assertThat; - -class CliIT { - - @Test - void fullPipeline(@TempDir Path tmp) throws Exception { - // Given - Path csvIn = tmp.resolve("data.csv"); - Files.writeString(csvIn, "id,name\n1,Alice\n2,Bob\n"); - - // Step 1: import CSV → vortex - assertThat(ImportCommand.run(new String[]{"import", csvIn.toString()})).isZero(); - Path vortex = tmp.resolve("data.vortex"); - assertThat(vortex).exists(); - - // Step 2: inspect - String inspect = captureStdout( - () -> InspectCommand.run(new String[]{"inspect", vortex.toString()})); - assertThat(inspect).contains("Schema:").contains("id").contains("name"); - - // Step 3: schema - String schema = captureStdout( - () -> SchemaCommand.run(new String[]{"schema", vortex.toString()})); - assertThat(schema.strip()).isEqualTo("struct"); - - // Step 4: export vortex → CSV - String exported = captureStdout( - () -> ExportCommand.run(new String[]{"export", vortex.toString()})); - String[] exportedLines = exported.split("\r?\n"); - assertThat(exportedLines).hasSize(3); - assertThat(exportedLines[0]).isEqualTo("id,name"); - assertThat(exportedLines[1]).isEqualTo("1,Alice"); - assertThat(exportedLines[2]).isEqualTo("2,Bob"); - - // Step 5: import the exported CSV again and verify schema is preserved - Path csvIn2 = tmp.resolve("data2.csv"); - Files.writeString(csvIn2, exported); - assertThat(ImportCommand.run(new String[]{"import", csvIn2.toString()})).isZero(); - Path vortex2 = tmp.resolve("data2.vortex"); - String schema2 = captureStdout( - () -> SchemaCommand.run(new String[]{"schema", vortex2.toString()})); - assertThat(schema2.strip()).isEqualTo(schema.strip()); - } - - @Test - void importExportRoundTrip(@TempDir Path tmp) throws Exception { - // Given - Path csvIn = tmp.resolve("data.csv"); - Files.writeString(csvIn, "id,name\n1,Alice\n2,Bob\n3,Charlie\n"); - - // When - assertThat(ImportCommand.run(new String[]{"import", csvIn.toString()})).isZero(); - Path vortexPath = tmp.resolve("data.vortex"); - String exported = captureStdout( - () -> ExportCommand.run(new String[]{"export", vortexPath.toString()})); - - // Then - String[] lines = exported.split("\r?\n"); - assertThat(lines).hasSize(4); - assertThat(lines[0]).isEqualTo("id,name"); - assertThat(lines[1]).isEqualTo("1,Alice"); - assertThat(lines[2]).isEqualTo("2,Bob"); - assertThat(lines[3]).isEqualTo("3,Charlie"); - } - - @Test - void inspectPrintsSchema(@TempDir Path tmp) throws Exception { - // Given - Path csvIn = tmp.resolve("data.csv"); - Files.writeString(csvIn, "id,score\n1,9.5\n"); - ImportCommand.run(new String[]{"import", csvIn.toString()}); - Path vortexPath = tmp.resolve("data.vortex"); - - // When - String output = captureStdout( - () -> InspectCommand.run(new String[]{"inspect", vortexPath.toString()})); - - // Then - assertThat(output).contains("Schema:").contains("id").contains("score"); - } - - @Test - void schemaPrintsMachineReadableDType(@TempDir Path tmp) throws Exception { - // Given - Path csvIn = tmp.resolve("data.csv"); - Files.writeString(csvIn, "id,name\n1,Alice\n"); - ImportCommand.run(new String[]{"import", csvIn.toString()}); - Path vortexPath = tmp.resolve("data.vortex"); - - // When - String output = captureStdout( - () -> SchemaCommand.run(new String[]{"schema", vortexPath.toString()})); - - // Then - assertThat(output.strip()).isEqualTo("struct"); - } - - @Test - void importReportsLargerWhenVortexExceedsCsv(@TempDir Path tmp) throws Exception { - // Given — tiny CSV produces vortex overhead > input size - Path csvIn = tmp.resolve("data.csv"); - Files.writeString(csvIn, "id,name\n1,Alice\n2,Bob\n"); - - // When - String output = captureStdout( - () -> ImportCommand.run(new String[]{"import", csvIn.toString()})); - - // Then — must say "larger", never a negative "smaller" - assertThat(output).contains("larger").doesNotContain("smaller"); - } - - @Test - void importParquetPipeline(@TempDir Path tmp) throws Exception { - // Given — test.parquet: id (int64), name (string), 3 rows - Path parquetIn = tmp.resolve("test.parquet"); - Files.copy( - Objects.requireNonNull(CliIT.class.getResourceAsStream("/test.parquet")), - parquetIn); - - // When - assertThat(ImportCommand.run(new String[]{"import", parquetIn.toString()})).isZero(); - Path vortex = tmp.resolve("test.vortex"); - assertThat(vortex).exists(); - - // Then — schema and row count correct - String schema = captureStdout( - () -> SchemaCommand.run(new String[]{"schema", vortex.toString()})); - assertThat(schema.strip()).isEqualTo("struct"); - - String count = captureStdout( - () -> CountCommand.run(new String[]{"count", vortex.toString()})); - assertThat(count.strip()).isEqualTo("3"); - } - - private static String captureStdout(IntSupplier action) { - ByteArrayOutputStream baos = new ByteArrayOutputStream(); - PrintStream saved = System.out; - System.setOut(new PrintStream(baos, true, StandardCharsets.UTF_8)); - try { - action.getAsInt(); - } finally { - System.setOut(saved); - } - return baos.toString(StandardCharsets.UTF_8); - } -} diff --git a/cli/src/test/resources/test.parquet b/cli/src/test/resources/test.parquet deleted file mode 100644 index f451248..0000000 Binary files a/cli/src/test/resources/test.parquet and /dev/null differ diff --git a/core/pom.xml b/core/pom.xml deleted file mode 100644 index c1efc2b..0000000 --- a/core/pom.xml +++ /dev/null @@ -1,160 +0,0 @@ - - - 4.0.0 - - io.github.dfa1.vortex - vortex-java - 0.3.0-SNAPSHOT - - - core - - - - com.google.flatbuffers - flatbuffers-java - - - com.google.protobuf - protobuf-java - - - io.airlift - aircompressor-v3 - ${aircompressor.version} - - - com.github.luben - zstd-jni - - - - org.junit.jupiter - junit-jupiter - test - - - org.assertj - assertj-core - test - - - org.mockito - mockito-junit-jupiter - test - - - - - - - regenerate-sources - - - - - org.codehaus.mojo - exec-maven-plugin - - - flatc-generate - generate-sources - - exec - - - flatc - ${project.basedir}/src/main/flatbuffers - - --java - -I - . - -o - ${project.basedir}/src/main/java - vortex-file/footer.fbs - vortex-layout/layout.fbs - vortex-array/array.fbs - vortex-dtype/dtype.fbs - - - - - - protoc-generate - generate-sources - - exec - - - protoc - - --java_out=${project.basedir}/src/main/java - --proto_path=${project.basedir}/src/main/proto - ${project.basedir}/src/main/proto/dtype.proto - ${project.basedir}/src/main/proto/scalar.proto - ${project.basedir}/src/main/proto/encodings.proto - - - - - - - - - org.apache.maven.plugins - maven-antrun-plugin - - - flatc-strip-version-guard - generate-sources - - run - - - - - - - - - - - - - - - - - diff --git a/core/src/main/flatbuffers/vortex-array/array.fbs b/core/src/main/flatbuffers/vortex-array/array.fbs deleted file mode 100644 index e8db352..0000000 --- a/core/src/main/flatbuffers/vortex-array/array.fbs +++ /dev/null @@ -1,62 +0,0 @@ -// SPDX-License-Identifier: Apache-2.0 -// SPDX-FileCopyrightText: Copyright the Vortex contributors - -/// An Array describes the hierarchy of an array as well as the locations of the data buffers that appear -/// immediately after the message in the byte stream. -namespace io.github.dfa1.vortex.fbs; - -table Array { - /// The array's hierarchical definition. - root: ArrayNode; - /// The locations of the data buffers of the array - buffers: [Buffer]; -} - -/// The compression mechanism used to compress the buffer. -enum Compression: uint8 { - None = 0, - LZ4 = 1, -} - -/// A Buffer describes the location of a data buffer in the byte stream as a packed 64-bit struct. -struct Buffer { - /// The length of any padding bytes written immediately before the buffer. - padding: uint16; - /// The minimum alignment of the buffer, stored as an exponent of 2. - alignment_exponent: uint8; - /// The compression algorithm used to compress the buffer. - compression: Compression; - /// The length of the buffer in bytes. - length: uint32; -} - - -table ArrayNode { - encoding: uint16; - metadata: [ubyte]; - children: [ArrayNode]; - buffers: [uint16]; - stats: ArrayStats; -} - -enum Precision: uint8 { - Inexact = 0, - Exact = 1, -} - -table ArrayStats { - /// Protobuf serialized ScalarValue - min: [ubyte]; - min_precision: Precision; - max: [ubyte]; - max_precision: Precision; - sum: [ubyte]; - is_sorted: bool = null; - is_strict_sorted: bool = null; - is_constant: bool = null; - null_count: uint64 = null; - uncompressed_size_in_bytes: uint64 = null; - nan_count: uint64 = null; -} - -root_type Array; diff --git a/core/src/main/flatbuffers/vortex-dtype/dtype.fbs b/core/src/main/flatbuffers/vortex-dtype/dtype.fbs deleted file mode 100644 index 0759411..0000000 --- a/core/src/main/flatbuffers/vortex-dtype/dtype.fbs +++ /dev/null @@ -1,95 +0,0 @@ -// SPDX-License-Identifier: Apache-2.0 -// SPDX-FileCopyrightText: Copyright the Vortex contributors - -namespace io.github.dfa1.vortex.fbs; - -enum PType: uint8 { - U8, - U16, - U32, - U64, - I8, - I16, - I32, - I64, - F16, - F32, - F64, -} - -table Null {} - -table Bool { - nullable: bool; -} - -table Primitive { - ptype: PType; - nullable: bool; -} - -table Decimal { - precision: uint8; - scale: int8; - nullable: bool; -} - -table Utf8 { - nullable: bool; -} - -table Binary { - nullable: bool; -} - -table Struct_ { - names: [string]; - dtypes: [DType]; - nullable: bool; -} - -table List { - element_type: DType; - nullable: bool; -} - -table FixedSizeList { - element_type: DType; - size: uint32; - nullable: bool; -} - -table Extension { - id: string; - storage_dtype: DType; - metadata: [ubyte]; -} - -table Variant { - nullable: bool; -} - -table Union { - nullable: bool; -} - -union Type { - Null = 1, - Bool = 2, - Primitive = 3, - Decimal = 4, - Utf8 = 5, - Binary = 6, - Struct_ = 7, - List = 8, - Extension = 9, - FixedSizeList = 10, // This is after `Extension` for backwards compatibility. - Variant = 11, - Union = 12, -} - -table DType { - type: Type; -} - -root_type DType; diff --git a/core/src/main/flatbuffers/vortex-file/footer.fbs b/core/src/main/flatbuffers/vortex-file/footer.fbs deleted file mode 100644 index dad4de6..0000000 --- a/core/src/main/flatbuffers/vortex-file/footer.fbs +++ /dev/null @@ -1,119 +0,0 @@ -// SPDX-License-Identifier: Apache-2.0 -// SPDX-FileCopyrightText: Copyright the Vortex contributors - -include "vortex-array/array.fbs"; -include "vortex-layout/layout.fbs"; - -namespace io.github.dfa1.vortex.fbs; - -// [postscript] -/// The `Postscript` is guaranteed by the file format to never exceed -/// 65528 bytes (i.e., u16::MAX - 8 bytes) in length, and is immediately -/// followed by an 8-byte `EndOfFile` struct. -/// -/// An initial read of a Vortex file defaults to at least 64KB (u16::MAX bytes) and therefore -/// is guaranteed to cover at least the Postscript. -/// -/// The reason for a postscript at all is to ensure minimal but all necessary footer information -/// can be read in two round trips. Since the DType is optional and possibly large, it lives in -/// its own segment. If the footer were arbitrary size, with a pointer to the DType segment, then -/// in the worst case we would need one round trip to read the footer length, one to read the full -/// footer and parse the DType offset, and a third to fetch the DType segment. -/// -/// The segments pointed to by the postscript have inline compression and encryption specs to avoid -/// the need to fetch encryption schemes up-front. -table Postscript { - /// Segment containing the root `DType` flatbuffer. - dtype: PostscriptSegment; - /// Segment containing the root `Layout` flatbuffer (required). - layout: PostscriptSegment; - /// Segment containing the file-level `Statistics` flatbuffer. - statistics: PostscriptSegment; - /// Segment containing the 'Footer' flatbuffer (required) - footer: PostscriptSegment; -} - -/// A `PostscriptSegment` describes the location of a segment in the file without referencing any -/// specification objects. That is, encryption and compression are defined inline. -table PostscriptSegment { - offset: uint64; - length: uint32; - alignment_exponent: uint8; - _compression: CompressionSpec; - _encryption: EncryptionSpec; -} -// [postscript] - -// [footer] -/// The `FileStatistics` object contains file-level statistics for the Vortex file. -table FileStatistics { - /// Statistics for each field in the root schema. If the root schema is not a struct, there will - /// be a single entry in this array. - field_stats: [ArrayStats]; -} - -/// The `Registry` object stores dictionary-encoded configuration for segments, -/// compression schemes, encryption schemes, etc. -table Footer { - // Dictionary-encoded array specs, up to u16::MAX. - array_specs: [ArraySpec]; - // Dictionary-encoded layout specs, up to u16::MAX. - layout_specs: [LayoutSpec]; - // Dictionary-encoded segment specs, up to u32::MAX. - segment_specs: [SegmentSpec]; - // Dictionary-encoded compress specs, up to u3::MAX (8). - compression_specs: [CompressionSpec]; - // Dictionary-encoded encryption specs, up to u16::MAX. - encryption_specs: [EncryptionSpec]; -} - -/// An `ArraySpec` describes the type of a particular array. -/// -/// These are identified by a globally unique string identifier, and looked up in the Vortex registry -/// at read-time. -table ArraySpec { - id: string (required); -} - -/// A `LayoutSpec` describes the type of a particular layout. -/// -/// These are identified by a globally unique string identifier, and looked up in the Vortex registry -/// at read-time. -table LayoutSpec { - id: string (required); -} - -/// A `SegmentSpec` acts as the locator for a buffer within the file. -struct SegmentSpec { - /// Offset relative to the start of the file. - offset: uint64; - /// Length in bytes of the segment. - length: uint32; - /// Base-2 exponent of the alignment of the segment. - alignment_exponent: uint8; - // These two fields are reserved for future use and act as pointers - // into `FileLayout::compression_schemes` and `FileLayout::encryption_schemes` - // respectively. They are not used in the current version of the file format. - _compression: uint8; - _encryption: uint16; -} - -enum CompressionScheme: uint8 { - None = 0, - LZ4 = 1, - ZLib = 2, - ZStd = 3, -} - -/// Definition of a compression scheme. -table CompressionSpec { - scheme: CompressionScheme; -} - -table EncryptionSpec { -} -// [footer] - -root_type FileStatistics; -root_type Footer; -root_type Postscript; diff --git a/core/src/main/flatbuffers/vortex-layout/layout.fbs b/core/src/main/flatbuffers/vortex-layout/layout.fbs deleted file mode 100644 index ecc5925..0000000 --- a/core/src/main/flatbuffers/vortex-layout/layout.fbs +++ /dev/null @@ -1,34 +0,0 @@ -// SPDX-License-Identifier: Apache-2.0 -// SPDX-FileCopyrightText: Copyright the Vortex contributors - -/// A `Layout` is a recursive data structure describing the physical layout of Vortex arrays in random access storage. -/// As a starting, concrete example, the first three Layout encodings are defined as: -/// -/// 1. encoding == 1, `Flat` -> one buffer, zero child Layouts -/// 2. encoding == 2, `Chunked` -> zero buffers, one or more child Layouts (used for chunks of rows) -/// 3. encoding == 3, `Columnar` -> zero buffers, one or more child Layouts (used for columns of structs) -/// -/// The `row_count` represents the number of rows represented by this Layout. This is very useful for -/// pruning the Layout tree based on row filters. -/// -/// The `metadata` field is fully opaque at this layer, and allows the Layout implementation corresponding to -/// `encoding` to embed additional information that may be useful for the reader. For example, the `ChunkedLayout` -/// uses the first byte of the `metadata` array as a boolean to indicate whether the first child Layout represents -/// the statistics table for the other chunks. -namespace io.github.dfa1.vortex.fbs; - -table Layout { - /// The ID of the encoding used for this Layout. - encoding: uint16; - /// The number of rows of data represented by this Layout. - row_count: uint64; - /// Any additional metadata this layout needs to interpret its children. - /// This does not include data-specific metadata, which the layout should store in a segment. - metadata: [ubyte]; - /// The children of this Layout. - children: [Layout]; - /// Identifiers for each `SegmentSpec` of data required by this layout. - segments: [uint32]; -} - -root_type Layout; \ No newline at end of file diff --git a/core/src/main/java/io/github/dfa1/vortex/core/ArrayStats.java b/core/src/main/java/io/github/dfa1/vortex/core/ArrayStats.java deleted file mode 100644 index c50203a..0000000 --- a/core/src/main/java/io/github/dfa1/vortex/core/ArrayStats.java +++ /dev/null @@ -1,70 +0,0 @@ -package io.github.dfa1.vortex.core; - -import com.google.protobuf.InvalidProtocolBufferException; -import io.github.dfa1.vortex.proto.ScalarProtos; - -import java.nio.ByteBuffer; - -/// Per-array statistics embedded in the encoding tree. -/// -/// @param min minimum value in the array, or {@code null} if unknown -/// @param max maximum value in the array, or {@code null} if unknown -/// @param trueCount number of {@code true} values for bool columns, or {@code null} if unknown -/// @param nullCount number of null values, or {@code null} if unknown -/// @param isSorted {@code true} if the array is sorted in ascending order, or {@code null} if unknown -/// @param isStrictSorted {@code true} if the array is strictly sorted (no duplicates), or {@code null} if unknown -public record ArrayStats( - Object min, - Object max, - Long trueCount, - Long nullCount, - Boolean isSorted, - Boolean isStrictSorted -) { - private static final ArrayStats EMPTY = new ArrayStats(null, null, null, null, null, null); - - /// Returns an empty stats instance with all fields set to {@code null}. - /// - /// @return an empty {@link ArrayStats} with no known statistics - public static ArrayStats empty() { - return EMPTY; - } - - /// Parses stats from a FlatBuffers {@link io.github.dfa1.vortex.fbs.ArrayStats} table. - /// Returns an empty instance when {@code fbs} is {@code null} or carries no min/max. - /// - /// @param fbs the FlatBuffers stats table, or {@code null} - /// @return parsed stats, or an empty instance if no usable data is present - public static ArrayStats fromFbs(io.github.dfa1.vortex.fbs.ArrayStats fbs) { - if (fbs == null) { - return EMPTY; - } - Object min = decodeScalar(fbs.minAsByteBuffer()); - Object max = decodeScalar(fbs.maxAsByteBuffer()); - if (min == null && max == null) { - return EMPTY; - } - return new ArrayStats(min, max, null, null, null, null); - } - - private static Object decodeScalar(ByteBuffer bytes) { - if (bytes == null || !bytes.hasRemaining()) { - return null; - } - try { - ScalarProtos.ScalarValue sv = ScalarProtos.ScalarValue.parseFrom(bytes.duplicate()); - return switch (sv.getKindCase()) { - case INT64_VALUE -> sv.getInt64Value(); - case UINT64_VALUE -> sv.getUint64Value(); - case F32_VALUE -> sv.getF32Value(); - case F64_VALUE -> sv.getF64Value(); - case BOOL_VALUE -> sv.getBoolValue(); - case STRING_VALUE -> sv.getStringValue(); - case BYTES_VALUE -> sv.getBytesValue().toStringUtf8(); - default -> null; - }; - } catch (InvalidProtocolBufferException e) { - throw new VortexException("invalid scalar value in array stats", e); - } - } -} diff --git a/core/src/main/java/io/github/dfa1/vortex/core/CompressionScheme.java b/core/src/main/java/io/github/dfa1/vortex/core/CompressionScheme.java deleted file mode 100644 index c53d996..0000000 --- a/core/src/main/java/io/github/dfa1/vortex/core/CompressionScheme.java +++ /dev/null @@ -1,35 +0,0 @@ -package io.github.dfa1.vortex.core; - -/// Block-level compression scheme applied to flat segments in the Vortex file format. -public enum CompressionScheme { - /// No compression. - NONE(0), - /// LZ4 block compression. - LZ4(1), - /// Zlib deflate compression. - ZLIB(2), - /// Zstandard compression. - ZSTD(3); - - /// Numeric code as stored in the file postscript. - public final int code; - - CompressionScheme(int code) { - this.code = code; - } - - /// Returns the enum constant matching the given numeric code. - /// - /// @param code numeric compression code from the file postscript - /// @return the matching {@link CompressionScheme} - /// @throws IllegalArgumentException if {@code code} does not correspond to any known scheme - public static CompressionScheme of(int code) { - return switch (code) { - case 0 -> NONE; - case 1 -> LZ4; - case 2 -> ZLIB; - case 3 -> ZSTD; - default -> throw new IllegalArgumentException("unknown compression code: " + code); - }; - } -} diff --git a/core/src/main/java/io/github/dfa1/vortex/core/DType.java b/core/src/main/java/io/github/dfa1/vortex/core/DType.java deleted file mode 100644 index 71859a3..0000000 --- a/core/src/main/java/io/github/dfa1/vortex/core/DType.java +++ /dev/null @@ -1,135 +0,0 @@ -package io.github.dfa1.vortex.core; - -import java.nio.ByteBuffer; - -/// Vortex logical data type. Strictly logical — defines value domain, not physical storage. -/// -/// Usage with pattern matching: -/// ```java -/// switch (dtype) { -/// case DType.Primitive(var pt, var nullable) -> ... -/// case DType.Struct(var names, var types, var nullable) -> ... -/// default -> ... -/// } -/// ``` -public sealed interface DType - permits DType.Null, DType.Bool, DType.Primitive, DType.Decimal, - DType.Utf8, DType.Binary, DType.Struct, - DType.List, DType.FixedSizeList, DType.Extension { - - /// Returns whether this type allows null values. - /// - /// @return {@code true} if null values are permitted - boolean nullable(); - - /// The SQL {@code NULL} type — no values, always nullable. - /// - /// @param nullable whether null values are permitted - record Null(boolean nullable) implements DType { - } - - /// Boolean logical type. - /// - /// @param nullable whether null values are permitted - record Bool(boolean nullable) implements DType { - } - - /// Primitive numeric logical type backed by a physical {@link PType}. - /// - /// @param ptype physical primitive type (e.g. I32, F64) - /// @param nullable whether null values are permitted - record Primitive(PType ptype, boolean nullable) implements DType { - } - - /// Fixed-precision decimal logical type. - /// - /// @param precision total number of significant decimal digits - /// @param scale number of digits to the right of the decimal point - /// @param nullable whether null values are permitted - record Decimal(byte precision, byte scale, boolean nullable) implements DType { - } - - /// Variable-length UTF-8 string logical type. - /// - /// @param nullable whether null values are permitted - record Utf8(boolean nullable) implements DType { - } - - /// Variable-length binary (byte string) logical type. - /// - /// @param nullable whether null values are permitted - record Binary(boolean nullable) implements DType { - } - - /// Struct logical type with named, typed fields. - /// - /// @param fieldNames ordered list of field names - /// @param fieldTypes ordered list of field types, parallel to {@code fieldNames} - /// @param nullable whether null values are permitted - record Struct( - java.util.List fieldNames, - java.util.List fieldTypes, - boolean nullable - ) implements DType { - /// Returns the type of the field with the given name. - /// - /// @param name the field name to look up - /// @return the {@link DType} of the named field - /// @throws IllegalArgumentException if no field with that name exists - public DType field(String name) { - int i = fieldNames.indexOf(name); - if (i < 0) { - throw new IllegalArgumentException("unknown field: " + name); - } - return fieldTypes.get(i); - } - } - - /// Variable-length list logical type. - /// - /// @param elementType logical type of each list element - /// @param nullable whether null values are permitted - record List(DType elementType, boolean nullable) implements DType { - } - - /// Fixed-size list logical type where every list has the same length. - /// - /// @param elementType logical type of each list element - /// @param fixedSize number of elements in every list - /// @param nullable whether null values are permitted - record FixedSizeList(DType elementType, int fixedSize, boolean nullable) implements DType { - } - - /// Extension logical type with user-defined semantics layered over a storage type. - /// - /// @param extensionId unique string identifier for the extension type (e.g. {@code "vortex.timestamp"}) - /// @param storageDType underlying storage dtype used for physical encoding - /// @param metadata opaque extension-specific metadata bytes, or {@code null} - /// @param nullable whether null values are permitted - record Extension( - String extensionId, - DType storageDType, - ByteBuffer metadata, - boolean nullable - ) implements DType { - } - - /// Returns a copy of this type with the given nullability. - /// - /// @param nullable the desired nullability for the returned type - /// @return a new {@link DType} identical to this one except for its nullability - default DType withNullable(boolean nullable) { - return switch (this) { - case Null _ -> new Null(nullable); - case Bool _ -> new Bool(nullable); - case Primitive(var pt, _) -> new Primitive(pt, nullable); - case Decimal(var p, var s, _) -> new Decimal(p, s, nullable); - case Utf8 _ -> new Utf8(nullable); - case Binary _ -> new Binary(nullable); - case Struct(var names, var types, _) -> new Struct(names, types, nullable); - case List(var elem, _) -> new List(elem, nullable); - case FixedSizeList(var elem, var size, _) -> new FixedSizeList(elem, size, nullable); - case Extension(var id, var storage, var meta, _) -> new Extension(id, storage, meta, nullable); - }; - } -} diff --git a/core/src/main/java/io/github/dfa1/vortex/core/Footer.java b/core/src/main/java/io/github/dfa1/vortex/core/Footer.java deleted file mode 100644 index 3dd2368..0000000 --- a/core/src/main/java/io/github/dfa1/vortex/core/Footer.java +++ /dev/null @@ -1,20 +0,0 @@ -package io.github.dfa1.vortex.core; - -import java.util.List; - -/// Parsed file footer. Contains the dictionaries needed to resolve indices in the layout tree. -/// -/// All spec lists are index-stable: array/layout/segment indices in the tree refer -/// into these lists by position. -/// -/// @param arraySpecs encoding id strings indexed by array spec index -/// @param layoutSpecs layout type strings indexed by layout spec index -/// @param segmentSpecs segment byte ranges indexed by segment index -/// @param compressionSpecs compression schemes indexed by compression spec index -public record Footer( - List arraySpecs, - List layoutSpecs, - List segmentSpecs, - List compressionSpecs -) { -} diff --git a/core/src/main/java/io/github/dfa1/vortex/core/Layout.java b/core/src/main/java/io/github/dfa1/vortex/core/Layout.java deleted file mode 100644 index a9f2014..0000000 --- a/core/src/main/java/io/github/dfa1/vortex/core/Layout.java +++ /dev/null @@ -1,70 +0,0 @@ -package io.github.dfa1.vortex.core; - -import java.nio.ByteBuffer; -import java.util.List; - -/// Node in the Vortex layout tree. Describes physical arrangement of data in the file. -/// -/// Typical tree shape per column: -/// ``` -/// Struct → Zoned(Stats) → Chunked → [Flat, Flat, ...] -/// ``` -/// -/// @param encodingId encoding id string (e.g. {@code "vortex.flat"}) -/// @param rowCount number of logical rows covered by this node -/// @param metadata optional encoding-specific metadata bytes, or {@code null} -/// @param children child layout nodes (empty for leaf nodes) -/// @param segments indices into the file's segment table for buffers owned by this node -public record Layout( - String encodingId, - long rowCount, - ByteBuffer metadata, - List children, - List segments -) { - /// Encoding id for flat (leaf) layouts ({@code "vortex.flat"}). - public static final String FLAT = "vortex.flat"; - /// Encoding id for chunked layouts ({@code "vortex.chunked"}). - public static final String CHUNKED = "vortex.chunked"; - /// Encoding id for struct layouts ({@code "vortex.struct"}). - public static final String STRUCT = "vortex.struct"; - /// Encoding id for zone-map layouts ({@code "vortex.stats"}). - public static final String ZONED = "vortex.stats"; - /// Encoding id for dictionary layouts ({@code "vortex.dict"}). - public static final String DICT = "vortex.dict"; - - /// Returns {@code true} if this layout is a flat (leaf) layout. - /// - /// @return {@code true} when {@code encodingId} equals {@link #FLAT} - public boolean isFlat() { - return FLAT.equals(encodingId); - } - - /// Returns {@code true} if this layout is a chunked layout. - /// - /// @return {@code true} when {@code encodingId} equals {@link #CHUNKED} - public boolean isChunked() { - return CHUNKED.equals(encodingId); - } - - /// Returns {@code true} if this layout is a struct layout. - /// - /// @return {@code true} when {@code encodingId} equals {@link #STRUCT} - public boolean isStruct() { - return STRUCT.equals(encodingId); - } - - /// Returns {@code true} if this layout is a zone-map (stats) layout. - /// - /// @return {@code true} when {@code encodingId} equals {@link #ZONED} - public boolean isZoned() { - return ZONED.equals(encodingId); - } - - /// Returns {@code true} if this layout is a dictionary layout. - /// - /// @return {@code true} when {@code encodingId} equals {@link #DICT} - public boolean isDict() { - return DICT.equals(encodingId); - } -} diff --git a/core/src/main/java/io/github/dfa1/vortex/core/PType.java b/core/src/main/java/io/github/dfa1/vortex/core/PType.java deleted file mode 100644 index 6c1f68e..0000000 --- a/core/src/main/java/io/github/dfa1/vortex/core/PType.java +++ /dev/null @@ -1,58 +0,0 @@ -package io.github.dfa1.vortex.core; - -/// Physical primitive type — the wire-level numeric kind for a column. -/// -/// Unsigned integers ({@code U8}–{@code U64}) and signed integers ({@code I8}–{@code I64}) -/// share the same in-memory bit pattern; signedness only affects interpretation. -/// Floating-point types follow IEEE 754: {@code F16} (half), {@code F32} (single), {@code F64} (double). -public enum PType { - /// Unsigned 8-bit integer. - U8, - /// Unsigned 16-bit integer. - U16, - /// Unsigned 32-bit integer. - U32, - /// Unsigned 64-bit integer. - U64, - /// Signed 8-bit integer. - I8, - /// Signed 16-bit integer. - I16, - /// Signed 32-bit integer. - I32, - /// Signed 64-bit integer. - I64, - /// IEEE 754 half-precision float (16-bit). Decoding not yet supported. - F16, - /// IEEE 754 single-precision float (32-bit). - F32, - /// IEEE 754 double-precision float (64-bit). - F64; - - /// Number of bytes per element on the wire (1, 2, 4, or 8). - /// - /// @return the byte size of this physical type - public int byteSize() { - return switch (this) { - case U8, I8 -> 1; - case U16, I16, F16 -> 2; - case U32, I32, F32 -> 4; - case U64, I64, F64 -> 8; - }; - } - - /// Returns {@code true} for {@code F16}, {@code F32}, and {@code F64}. - /// - /// @return {@code true} if this ptype is a floating-point type - public boolean isFloating() { - return this == F16 || this == F32 || this == F64; - } - - /// Returns {@code true} for signed integers ({@code I8}–{@code I64}) and all floating-point types. - /// - /// @return {@code true} if this ptype is signed - public boolean isSigned() { - return this == I8 || this == I16 || this == I32 || this == I64 - || this == F16 || this == F32 || this == F64; - } -} diff --git a/core/src/main/java/io/github/dfa1/vortex/core/SegmentSpec.java b/core/src/main/java/io/github/dfa1/vortex/core/SegmentSpec.java deleted file mode 100644 index bd4bcb6..0000000 --- a/core/src/main/java/io/github/dfa1/vortex/core/SegmentSpec.java +++ /dev/null @@ -1,19 +0,0 @@ -package io.github.dfa1.vortex.core; - -/// Byte range and properties of one data segment in the file. -/// -/// {@code length} is {@code long} because the wire format encodes it as -/// {@code uint32} (up to 4 GB per segment). Storing it as a Java {@code int} -/// silently truncates segments larger than 2 GB into negative values. -/// -/// @param offset byte offset from the start of the file -/// @param length byte length of the segment (unsigned, stored as {@code long}) -/// @param alignmentExponent log2 of the required byte alignment; 0 means no alignment constraint -/// @param compression compression scheme applied to this segment's bytes -public record SegmentSpec( - long offset, - long length, - byte alignmentExponent, - CompressionScheme compression -) { -} diff --git a/core/src/main/java/io/github/dfa1/vortex/core/VortexException.java b/core/src/main/java/io/github/dfa1/vortex/core/VortexException.java deleted file mode 100644 index f7f1b3a..0000000 --- a/core/src/main/java/io/github/dfa1/vortex/core/VortexException.java +++ /dev/null @@ -1,67 +0,0 @@ -package io.github.dfa1.vortex.core; - -import io.github.dfa1.vortex.encoding.EncodingId; - -import java.util.Optional; - -/// Unrecoverable Vortex error: malformed file, unsupported feature, or encoding failure. -/// -///

Non-recoverable contract: once thrown, the underlying file or stream is -/// in an indeterminate state. Callers must propagate this exception — do not catch-and-swallow, -/// do not retry on the same input. The correct response is to abort the read, surface the error, -/// and close the {@code VortexFile}. -/// -///

Carries an optional {@link EncodingId} for diagnostic attribution; it is not intended -/// for recovery logic. -public final class VortexException extends RuntimeException { - - /// The encoding that raised this exception, or {@code null} if not attributed to a specific encoding. - private final EncodingId encodingId; - - /// Creates a {@code VortexException} with the given message and no associated encoding. - /// - /// @param message human-readable error description - public VortexException(String message) { - super(message); - this.encodingId = null; - } - - /// Creates a {@code VortexException} with the given message and cause, and no associated encoding. - /// - /// @param message human-readable error description - /// @param cause the underlying cause - public VortexException(String message, Throwable cause) { - super(message, cause); - this.encodingId = null; - } - - /// Creates a {@code VortexException} attributed to the given encoding. - /// - /// @param encodingId encoding responsible for this error - /// @param message human-readable error description (prefixed with the encoding id) - public VortexException(EncodingId encodingId, String message) { - super(prefix(encodingId) + message); - this.encodingId = encodingId; - } - - /// Creates a {@code VortexException} attributed to the given encoding, with a cause. - /// - /// @param encodingId encoding responsible for this error - /// @param message human-readable error description (prefixed with the encoding id) - /// @param cause the underlying cause - public VortexException(EncodingId encodingId, String message, Throwable cause) { - super(prefix(encodingId) + message, cause); - this.encodingId = encodingId; - } - - /// Returns the encoding that raised this exception, if known. - /// - /// @return an {@link Optional} containing the {@link EncodingId}, or empty if not attributed - public Optional encodingId() { - return Optional.ofNullable(encodingId); - } - - private static String prefix(EncodingId encodingId) { - return encodingId == null ? "" : encodingId.id() + ": "; - } -} diff --git a/core/src/main/java/io/github/dfa1/vortex/core/array/Array.java b/core/src/main/java/io/github/dfa1/vortex/core/array/Array.java deleted file mode 100644 index f8dce45..0000000 --- a/core/src/main/java/io/github/dfa1/vortex/core/array/Array.java +++ /dev/null @@ -1,44 +0,0 @@ -package io.github.dfa1.vortex.core.array; - -import io.github.dfa1.vortex.core.DType; -import io.github.dfa1.vortex.core.VortexException; - -import java.lang.foreign.MemorySegment; - -/// Decoded columnar data. Concrete subtypes specialise element access for the JIT; -/// each covers a specific dtype family. -/// -/// Buffers are `MemorySegment` slices backed by the memory-mapped file; lifetime -/// is tied to the `VortexFile`'s Arena. -public sealed interface Array - permits BoolArray, ByteArray, DoubleArray, EmptyArray, FixedSizeListArray, Float16Array, - FloatArray, GenericArray, IntArray, ListArray, ListViewArray, LongArray, - MaskedArray, NullArray, ShortArray, StructArray, UnknownArray, VarBinArray { - - /// Returns the number of elements in this array. - /// - /// @return element count - long length(); - - /// Returns the logical type of elements in this array. - /// - /// @return dtype - DType dtype(); - - /// Returns the raw buffer at position {@code i} (used by some {@link io.github.dfa1.vortex.encoding.Encoding}). - /// - /// @param i buffer index - /// @return memory segment for buffer {@code i} - default MemorySegment buffer(int i) { - throw new VortexException(getClass().getSimpleName() + " has no raw buffers"); - } - - /// Returns the child array at position {@code i} (used by some {@link io.github.dfa1.vortex.encoding.Encoding}). - /// - /// @param i child index - /// @return child array at index {@code i} - default Array child(int i) { - throw new VortexException(getClass().getSimpleName() + " has no children"); - } - -} diff --git a/core/src/main/java/io/github/dfa1/vortex/core/array/BoolArray.java b/core/src/main/java/io/github/dfa1/vortex/core/array/BoolArray.java deleted file mode 100644 index f2f3c73..0000000 --- a/core/src/main/java/io/github/dfa1/vortex/core/array/BoolArray.java +++ /dev/null @@ -1,63 +0,0 @@ -package io.github.dfa1.vortex.core.array; - -import io.github.dfa1.vortex.core.ArrayStats; -import io.github.dfa1.vortex.core.DType; - -import java.lang.foreign.MemorySegment; -import java.lang.foreign.ValueLayout; - -/// Concrete [Array] for bit-packed boolean columns (LSB-first, one byte per 8 elements). -public final class BoolArray implements Array { - - private final DType dtype; - private final long length; - private final MemorySegment buffer; - private final ArrayStats stats; - - /// Constructs a {@code BoolArray} backed by the given bit-packed buffer. - /// - /// @param dtype logical type, must be {@link io.github.dfa1.vortex.core.DType.Bool} - /// @param length number of logical boolean elements - /// @param buffer bit-packed boolean data (LSB-first, one byte per 8 elements) - /// @param stats per-array statistics, or {@link ArrayStats#empty()} if unknown - public BoolArray(DType dtype, long length, MemorySegment buffer, ArrayStats stats) { - this.dtype = dtype; - this.length = length; - this.buffer = buffer; - this.stats = stats; - } - - @Override - public DType dtype() { - return dtype; - } - - @Override - public long length() { - return length; - } - - /// Returns the per-array statistics for this array. - /// - /// @return the {@link ArrayStats} associated with this array - public ArrayStats stats() { - return stats; - } - - @Override - public MemorySegment buffer(int i) { - if (i != 0) { - throw new IndexOutOfBoundsException(i); - } - return buffer; - } - - /// Returns the boolean value at the given logical index. - /// - /// @param i zero-based logical index (must be in {@code [0, length)}) - /// @return the boolean value at position {@code i} - public boolean getBoolean(long i) { - byte b = buffer.get(ValueLayout.JAVA_BYTE, i >>> 3); - return (b & (1 << (i & 7))) != 0; - } -} diff --git a/core/src/main/java/io/github/dfa1/vortex/core/array/ByteArray.java b/core/src/main/java/io/github/dfa1/vortex/core/array/ByteArray.java deleted file mode 100644 index 8876052..0000000 --- a/core/src/main/java/io/github/dfa1/vortex/core/array/ByteArray.java +++ /dev/null @@ -1,89 +0,0 @@ -package io.github.dfa1.vortex.core.array; - -import io.github.dfa1.vortex.core.ArrayStats; -import io.github.dfa1.vortex.core.DType; -import io.github.dfa1.vortex.core.PType; - -import java.lang.foreign.MemorySegment; -import java.lang.foreign.ValueLayout; -import java.util.function.LongBinaryOperator; - -/// Concrete [Array] for I8/U8 primitive columns. -public final class ByteArray implements Array { - - private final DType dtype; - private final long length; - private final MemorySegment buffer; - private final ArrayStats stats; - - /// Constructs a {@code ByteArray} backed by the given buffer. - /// - /// @param dtype logical type, must be a {@link io.github.dfa1.vortex.core.DType.Primitive} with ptype I8 or U8 - /// @param length number of logical elements - /// @param buffer raw byte data (one byte per element, little-endian) - /// @param stats per-array statistics, or {@link ArrayStats#empty()} if unknown - public ByteArray(DType dtype, long length, MemorySegment buffer, ArrayStats stats) { - this.dtype = dtype; - this.length = length; - this.buffer = buffer; - this.stats = stats; - } - - @Override - public DType dtype() { - return dtype; - } - - @Override - public long length() { - return length; - } - - /// Returns the per-array statistics for this array. - /// - /// @return the {@link ArrayStats} associated with this array - public ArrayStats stats() { - return stats; - } - - @Override - public MemorySegment buffer(int i) { - if (i != 0) { - throw new IndexOutOfBoundsException(i); - } - return buffer; - } - - /// Returns the raw byte value at the given logical index. - /// - /// @param i zero-based logical index (must be in {@code [0, length)}) - /// @return the raw signed byte value at position {@code i} - public byte getByte(long i) { - return buffer.get(ValueLayout.JAVA_BYTE, i); - } - - /// Returns the int value at the given logical index, applying unsigned widening for U8 columns. - /// - /// @param i zero-based logical index (must be in {@code [0, length)}) - /// @return the value at position {@code i} as a signed int (U8 values are zero-extended) - public int getInt(long i) { - byte raw = buffer.get(ValueLayout.JAVA_BYTE, i); - boolean unsigned = dtype instanceof DType.Primitive p && p.ptype() == PType.U8; - return unsigned ? Byte.toUnsignedInt(raw) : raw; - } - - /// Reduces all elements to a single long using the supplied operator. - /// - /// @param identity initial accumulator value - /// @param op binary operator applied to accumulator and each element in order - /// @return the final accumulated value - public long fold(long identity, LongBinaryOperator op) { - MemorySegment buf = buffer; - long n = length; - long result = identity; - for (long i = 0; i < n; i++) { - result = op.applyAsLong(result, buf.get(ValueLayout.JAVA_BYTE, i)); - } - return result; - } -} diff --git a/core/src/main/java/io/github/dfa1/vortex/core/array/DoubleArray.java b/core/src/main/java/io/github/dfa1/vortex/core/array/DoubleArray.java deleted file mode 100644 index 507fdaf..0000000 --- a/core/src/main/java/io/github/dfa1/vortex/core/array/DoubleArray.java +++ /dev/null @@ -1,90 +0,0 @@ -package io.github.dfa1.vortex.core.array; - -import io.github.dfa1.vortex.core.ArrayStats; -import io.github.dfa1.vortex.core.DType; -import io.github.dfa1.vortex.encoding.PTypeIO; - -import java.lang.foreign.MemorySegment; -import java.util.function.DoubleBinaryOperator; -import java.util.function.DoubleConsumer; - -/// Concrete [Array] for F64 primitive columns. -public final class DoubleArray implements Array { - - private final DType dtype; - private final long length; - private final MemorySegment buffer; - private final ArrayStats stats; - - /// Constructs a {@code DoubleArray} backed by the given buffer. - /// - /// @param dtype logical type, must be a {@link io.github.dfa1.vortex.core.DType.Primitive} with ptype F64 - /// @param length number of logical elements - /// @param buffer raw double data (8 bytes per element, little-endian) - /// @param stats per-array statistics, or {@link ArrayStats#empty()} if unknown - public DoubleArray(DType dtype, long length, MemorySegment buffer, ArrayStats stats) { - this.dtype = dtype; - this.length = length; - this.buffer = buffer; - this.stats = stats; - } - - @Override - public DType dtype() { - return dtype; - } - - @Override - public long length() { - return length; - } - - /// Returns the per-array statistics for this array. - /// - /// @return the {@link ArrayStats} associated with this array - public ArrayStats stats() { - return stats; - } - - @Override - public MemorySegment buffer(int i) { - if (i != 0) { - throw new IndexOutOfBoundsException(i); - } - return buffer; - } - - /// Returns the double value at the given logical index. - /// - /// @param i zero-based logical index (must be in {@code [0, length)}) - /// @return the double value at position {@code i} - public double getDouble(long i) { - return buffer.getAtIndex(PTypeIO.LE_DOUBLE, i); - } - - /// Invokes the consumer for each element in order. - /// - /// @param c consumer called once per element with the double value at each index - public void forEachDouble(DoubleConsumer c) { - MemorySegment buf = buffer; - long n = length; - for (long i = 0; i < n; i++) { - c.accept(buf.getAtIndex(PTypeIO.LE_DOUBLE, i)); - } - } - - /// Reduces all elements to a single double using the supplied operator. - /// - /// @param identity initial accumulator value - /// @param op binary operator applied to accumulator and each element in order - /// @return the final accumulated value - public double fold(double identity, DoubleBinaryOperator op) { - MemorySegment buf = buffer; - long n = length; - double result = identity; - for (long i = 0; i < n; i++) { - result = op.applyAsDouble(result, buf.getAtIndex(PTypeIO.LE_DOUBLE, i)); - } - return result; - } -} diff --git a/core/src/main/java/io/github/dfa1/vortex/core/array/EmptyArray.java b/core/src/main/java/io/github/dfa1/vortex/core/array/EmptyArray.java deleted file mode 100644 index ba88297..0000000 --- a/core/src/main/java/io/github/dfa1/vortex/core/array/EmptyArray.java +++ /dev/null @@ -1,30 +0,0 @@ -package io.github.dfa1.vortex.core.array; - -import io.github.dfa1.vortex.core.ArrayStats; -import io.github.dfa1.vortex.core.DType; - -/// Zero-length [Array]. Has no buffers or children. -/// -/// @param dtype the logical type of this empty array -public record EmptyArray(DType dtype) implements Array { - - @Override - public long length() { - return 0; - } - - /// Returns empty statistics — an array with zero elements has no meaningful stats. - /// - /// @return an empty {@link ArrayStats} instance - public ArrayStats stats() { - return ArrayStats.empty(); - } - - /// Creates an empty array with the given logical type. - /// - /// @param dtype logical type for the returned empty array - /// @return a zero-length {@link Array} with the specified dtype - public static Array of(DType dtype) { - return new EmptyArray(dtype); - } -} diff --git a/core/src/main/java/io/github/dfa1/vortex/core/array/FixedSizeListArray.java b/core/src/main/java/io/github/dfa1/vortex/core/array/FixedSizeListArray.java deleted file mode 100644 index 5267f37..0000000 --- a/core/src/main/java/io/github/dfa1/vortex/core/array/FixedSizeListArray.java +++ /dev/null @@ -1,64 +0,0 @@ -package io.github.dfa1.vortex.core.array; - -import io.github.dfa1.vortex.core.ArrayStats; -import io.github.dfa1.vortex.core.DType; - -/// Decoded fixed-size list array: holds a flat elements [Array] of length {@code outerLen * fixedSize}. -/// -///

List {@code i} covers elements {@code [i*fixedSize, (i+1)*fixedSize)}. -public final class FixedSizeListArray implements Array { - - private final DType.FixedSizeList dtype; - private final long outerLen; - private final Array elements; - - /// Constructs a {@code FixedSizeListArray} from a flat elements array. - /// - /// @param dtype logical fixed-size list type (provides element type and fixed size) - /// @param outerLen number of outer list elements - /// @param elements flat array of {@code outerLen * fixedSize} element values - public FixedSizeListArray(DType.FixedSizeList dtype, long outerLen, Array elements) { - this.dtype = dtype; - this.outerLen = outerLen; - this.elements = elements; - } - - @Override - public long length() { - return outerLen; - } - - @Override - public DType dtype() { - return dtype; - } - - /// Returns per-array statistics (always empty for fixed-size list arrays). - /// - /// @return empty array statistics - public ArrayStats stats() { - return ArrayStats.empty(); - } - - /// Returns the flat elements array containing {@code outerLen * fixedSize} values. - /// - /// @return the flat elements array - public Array elements() { - return elements; - } - - /// Returns the fixed number of elements per outer list entry. - /// - /// @return the fixed size from the dtype - public int fixedSize() { - return dtype.fixedSize(); - } - - @Override - public Array child(int i) { - if (i != 0) { - throw new ArrayIndexOutOfBoundsException("FixedSizeListArray child index " + i); - } - return elements; - } -} diff --git a/core/src/main/java/io/github/dfa1/vortex/core/array/Float16Array.java b/core/src/main/java/io/github/dfa1/vortex/core/array/Float16Array.java deleted file mode 100644 index 7832f5c..0000000 --- a/core/src/main/java/io/github/dfa1/vortex/core/array/Float16Array.java +++ /dev/null @@ -1,64 +0,0 @@ -package io.github.dfa1.vortex.core.array; - -import io.github.dfa1.vortex.core.ArrayStats; -import io.github.dfa1.vortex.core.DType; -import io.github.dfa1.vortex.encoding.PTypeIO; - -import java.lang.foreign.MemorySegment; - -/// Concrete [Array] for F16 (IEEE 754 half-precision) columns. -/// Wire format: little-endian shorts (2 bytes/element). Element access -/// widens to `float` via [Float#float16ToFloat]. -public final class Float16Array implements Array { - - private final DType dtype; - private final long length; - private final MemorySegment buffer; - private final ArrayStats stats; - - /// Creates a new {@code Float16Array} backed by the given memory segment. - /// - /// @param dtype logical type, must be F16 - /// @param length number of elements - /// @param buffer little-endian half-precision float data (2 bytes per element) - /// @param stats per-array statistics, or {@link io.github.dfa1.vortex.core.ArrayStats#empty()} if unknown - public Float16Array(DType dtype, long length, MemorySegment buffer, ArrayStats stats) { - this.dtype = dtype; - this.length = length; - this.buffer = buffer; - this.stats = stats; - } - - @Override - public DType dtype() { - return dtype; - } - - @Override - public long length() { - return length; - } - - /// Returns per-array statistics. - /// - /// @return array statistics - public ArrayStats stats() { - return stats; - } - - @Override - public MemorySegment buffer(int i) { - if (i != 0) { - throw new IndexOutOfBoundsException(i); - } - return buffer; - } - - /// Returns the element at the given index widened to a single-precision float. - /// - /// @param i zero-based index (must be in {@code [0, length)}) - /// @return the half-precision value at position {@code i} converted to {@code float} - public float getFloat(long i) { - return Float.float16ToFloat(buffer.getAtIndex(PTypeIO.LE_SHORT, i)); - } -} diff --git a/core/src/main/java/io/github/dfa1/vortex/core/array/FloatArray.java b/core/src/main/java/io/github/dfa1/vortex/core/array/FloatArray.java deleted file mode 100644 index 0fe29c7..0000000 --- a/core/src/main/java/io/github/dfa1/vortex/core/array/FloatArray.java +++ /dev/null @@ -1,78 +0,0 @@ -package io.github.dfa1.vortex.core.array; - -import io.github.dfa1.vortex.core.ArrayStats; -import io.github.dfa1.vortex.core.DType; -import io.github.dfa1.vortex.encoding.PTypeIO; - -import java.lang.foreign.MemorySegment; -import java.util.function.DoubleBinaryOperator; - -/// Concrete [Array] for F32 primitive columns. -public final class FloatArray implements Array { - - private final DType dtype; - private final long length; - private final MemorySegment buffer; - private final ArrayStats stats; - - /// Creates a new {@code FloatArray} backed by the given memory segment. - /// - /// @param dtype logical type, must be F32 - /// @param length number of elements - /// @param buffer little-endian float data (4 bytes per element) - /// @param stats per-array statistics, or {@link io.github.dfa1.vortex.core.ArrayStats#empty()} if unknown - public FloatArray(DType dtype, long length, MemorySegment buffer, ArrayStats stats) { - this.dtype = dtype; - this.length = length; - this.buffer = buffer; - this.stats = stats; - } - - @Override - public DType dtype() { - return dtype; - } - - @Override - public long length() { - return length; - } - - /// Returns per-array statistics. - /// - /// @return array statistics - public ArrayStats stats() { - return stats; - } - - @Override - public MemorySegment buffer(int i) { - if (i != 0) { - throw new IndexOutOfBoundsException(i); - } - return buffer; - } - - /// Returns the float value at the given index. - /// - /// @param i zero-based index (must be in {@code [0, length)}) - /// @return the float value at position {@code i} - public float getFloat(long i) { - return buffer.getAtIndex(PTypeIO.LE_FLOAT, i); - } - - /// Folds all elements using the given binary operator and identity value. - /// - /// @param identity initial accumulator value - /// @param op binary operator applied to the accumulator and each float element (widened to double) - /// @return the final accumulated result - public double fold(double identity, DoubleBinaryOperator op) { - MemorySegment buf = buffer; - long n = length; - double result = identity; - for (long i = 0; i < n; i++) { - result = op.applyAsDouble(result, buf.getAtIndex(PTypeIO.LE_FLOAT, i)); - } - return result; - } -} diff --git a/core/src/main/java/io/github/dfa1/vortex/core/array/GenericArray.java b/core/src/main/java/io/github/dfa1/vortex/core/array/GenericArray.java deleted file mode 100644 index 4ccddcb..0000000 --- a/core/src/main/java/io/github/dfa1/vortex/core/array/GenericArray.java +++ /dev/null @@ -1,68 +0,0 @@ -package io.github.dfa1.vortex.core.array; - -import io.github.dfa1.vortex.core.ArrayStats; -import io.github.dfa1.vortex.core.DType; - -import java.lang.foreign.MemorySegment; - -/// Fallback [Array] for dtypes that lack a dedicated concrete subtype. -/// -/// Holds raw buffer segments and child arrays. Used by encodings during migration -/// and for less-common dtypes (e.g. Decimal, Ext) where typed accessors are -/// not yet implemented. -public final class GenericArray implements Array { - - private final DType dtype; - private final long length; - private final MemorySegment[] buffers; - private final Array[] children; - - /// Creates a new {@code GenericArray} with the given buffers and children. - /// - /// @param dtype logical type of this array - /// @param length number of logical elements - /// @param buffers raw memory segments backing this array's data - /// @param children child arrays (e.g. offsets, validity) - public GenericArray(DType dtype, long length, MemorySegment[] buffers, Array[] children) { - this.dtype = dtype; - this.length = length; - this.buffers = buffers; - this.children = children; - } - - /// Creates a new {@code GenericArray} with a single buffer and no children. - /// - /// @param dtype logical type of this array - /// @param length number of logical elements - /// @param buffer single raw memory segment backing this array's data - public GenericArray(DType dtype, long length, MemorySegment buffer) { - this(dtype, length, new MemorySegment[]{buffer}, new Array[0]); - } - - @Override - public DType dtype() { - return dtype; - } - - @Override - public long length() { - return length; - } - - /// Returns per-array statistics (always empty for generic arrays). - /// - /// @return empty array statistics - public ArrayStats stats() { - return ArrayStats.empty(); - } - - @Override - public MemorySegment buffer(int i) { - return buffers[i]; - } - - @Override - public Array child(int i) { - return children[i]; - } -} diff --git a/core/src/main/java/io/github/dfa1/vortex/core/array/IntArray.java b/core/src/main/java/io/github/dfa1/vortex/core/array/IntArray.java deleted file mode 100644 index c9ac1f0..0000000 --- a/core/src/main/java/io/github/dfa1/vortex/core/array/IntArray.java +++ /dev/null @@ -1,89 +0,0 @@ -package io.github.dfa1.vortex.core.array; - -import io.github.dfa1.vortex.core.ArrayStats; -import io.github.dfa1.vortex.core.DType; -import io.github.dfa1.vortex.encoding.PTypeIO; -import java.lang.foreign.MemorySegment; -import java.util.function.IntBinaryOperator; -import java.util.function.IntConsumer; - -/// Concrete [Array] for I32/U32 primitive columns. -public final class IntArray implements Array { - - private final DType dtype; - private final long length; - private final MemorySegment buffer; - private final ArrayStats stats; - - /// Creates a new {@code IntArray} backed by the given memory segment. - /// - /// @param dtype logical type, must be I32 or U32 - /// @param length number of elements - /// @param buffer little-endian int data (4 bytes per element) - /// @param stats per-array statistics, or {@link io.github.dfa1.vortex.core.ArrayStats#empty()} if unknown - public IntArray(DType dtype, long length, MemorySegment buffer, ArrayStats stats) { - this.dtype = dtype; - this.length = length; - this.buffer = buffer; - this.stats = stats; - } - - @Override - public DType dtype() { - return dtype; - } - - @Override - public long length() { - return length; - } - - /// Returns per-array statistics. - /// - /// @return array statistics - public ArrayStats stats() { - return stats; - } - - @Override - public MemorySegment buffer(int i) { - if (i != 0) { - throw new IndexOutOfBoundsException(i); - } - return buffer; - } - - /// Returns the int value at the given index. - /// - /// @param i zero-based index (must be in {@code [0, length)}) - /// @return the int value at position {@code i} - public int getInt(long i) { - return buffer.getAtIndex(PTypeIO.LE_INT, i); - } - - /// Passes each element to the given consumer in order. - /// - /// @param c consumer that receives each int element - public void forEachInt(IntConsumer c) { - MemorySegment buf = buffer; - long n = length; - for (long i = 0; i < n; i++) { - c.accept(buf.getAtIndex(PTypeIO.LE_INT, i)); - } - } - - /// Folds all elements using the given binary operator and identity value. - /// - /// @param identity initial accumulator value - /// @param op binary operator applied to the accumulator and each int element - /// @return the final accumulated result - public int fold(int identity, IntBinaryOperator op) { - MemorySegment buf = buffer; - long n = length; - int result = identity; - for (long i = 0; i < n; i++) { - result = op.applyAsInt(result, buf.getAtIndex(PTypeIO.LE_INT, i)); - } - return result; - } -} diff --git a/core/src/main/java/io/github/dfa1/vortex/core/array/ListArray.java b/core/src/main/java/io/github/dfa1/vortex/core/array/ListArray.java deleted file mode 100644 index d686df6..0000000 --- a/core/src/main/java/io/github/dfa1/vortex/core/array/ListArray.java +++ /dev/null @@ -1,62 +0,0 @@ -package io.github.dfa1.vortex.core.array; - -import io.github.dfa1.vortex.core.DType; -import io.github.dfa1.vortex.core.VortexException; - -/// Decoded variable-length list array. -/// -///

List {@code i} covers {@code elements[offsets[i]..offsets[i+1])}. -/// {@code offsets} has length {@code outerLen + 1}. -public final class ListArray implements Array { - - private final DType.List dtype; - private final long outerLen; - private final Array elements; - private final Array offsets; - - /// Creates a new {@code ListArray}. - /// - /// @param dtype logical list type - /// @param outerLen number of outer list elements - /// @param elements flat elements array (length = total element count) - /// @param offsets offsets array of length {@code outerLen + 1} - public ListArray(DType.List dtype, long outerLen, Array elements, Array offsets) { - this.dtype = dtype; - this.outerLen = outerLen; - this.elements = elements; - this.offsets = offsets; - } - - @Override - public long length() { - return outerLen; - } - - @Override - public DType dtype() { - return dtype; - } - - /// Returns the flat elements array. - /// - /// @return the elements array - public Array elements() { - return elements; - } - - /// Returns the offsets array (length {@code outerLen + 1}). - /// - /// @return the offsets array - public Array offsets() { - return offsets; - } - - @Override - public Array child(int i) { - return switch (i) { - case 0 -> elements; - case 1 -> offsets; - default -> throw new VortexException("ListArray child index out of bounds: " + i); - }; - } -} diff --git a/core/src/main/java/io/github/dfa1/vortex/core/array/ListViewArray.java b/core/src/main/java/io/github/dfa1/vortex/core/array/ListViewArray.java deleted file mode 100644 index 584a39b..0000000 --- a/core/src/main/java/io/github/dfa1/vortex/core/array/ListViewArray.java +++ /dev/null @@ -1,74 +0,0 @@ -package io.github.dfa1.vortex.core.array; - -import io.github.dfa1.vortex.core.DType; -import io.github.dfa1.vortex.core.VortexException; - -/// Decoded variable-length list-view array (Arrow ListView layout). -/// -///

Unlike {@link ListArray}, offsets and sizes are independent per row: -/// {@code list[i] = elements[offsets[i] .. offsets[i] + sizes[i]]}. -/// Both {@code offsets} and {@code sizes} have length {@code outerLen} (not outerLen+1). -public final class ListViewArray implements Array { - - private final DType.List dtype; - private final long outerLen; - private final Array elements; - private final Array offsets; - private final Array sizes; - - /// Creates a new {@code ListViewArray}. - /// - /// @param dtype logical list type - /// @param outerLen number of outer list elements - /// @param elements flat elements array - /// @param offsets per-row start offsets into {@code elements} (length {@code outerLen}) - /// @param sizes per-row element counts (length {@code outerLen}) - public ListViewArray(DType.List dtype, long outerLen, Array elements, Array offsets, Array sizes) { - this.dtype = dtype; - this.outerLen = outerLen; - this.elements = elements; - this.offsets = offsets; - this.sizes = sizes; - } - - @Override - public long length() { - return outerLen; - } - - @Override - public DType dtype() { - return dtype; - } - - /// Returns the flat elements array. - /// - /// @return the elements array - public Array elements() { - return elements; - } - - /// Returns the per-row start offsets array (length {@code outerLen}). - /// - /// @return the offsets array - public Array offsets() { - return offsets; - } - - /// Returns the per-row element count array (length {@code outerLen}). - /// - /// @return the sizes array - public Array sizes() { - return sizes; - } - - @Override - public Array child(int i) { - return switch (i) { - case 0 -> elements; - case 1 -> offsets; - case 2 -> sizes; - default -> throw new VortexException("ListViewArray child index out of bounds: " + i); - }; - } -} diff --git a/core/src/main/java/io/github/dfa1/vortex/core/array/LongArray.java b/core/src/main/java/io/github/dfa1/vortex/core/array/LongArray.java deleted file mode 100644 index 02eba4f..0000000 --- a/core/src/main/java/io/github/dfa1/vortex/core/array/LongArray.java +++ /dev/null @@ -1,90 +0,0 @@ -package io.github.dfa1.vortex.core.array; - -import io.github.dfa1.vortex.core.ArrayStats; -import io.github.dfa1.vortex.core.DType; -import io.github.dfa1.vortex.encoding.PTypeIO; - -import java.lang.foreign.MemorySegment; -import java.util.function.LongBinaryOperator; -import java.util.function.LongConsumer; - -/// Concrete [Array] for I64/U64 primitive columns. -public final class LongArray implements Array { - - private final DType dtype; - private final long length; - private final MemorySegment buffer; - private final ArrayStats stats; - - /// Creates a new {@code LongArray} backed by the given memory segment. - /// - /// @param dtype logical type, must be I64 or U64 - /// @param length number of elements - /// @param buffer little-endian long data (8 bytes per element) - /// @param stats per-array statistics, or {@link io.github.dfa1.vortex.core.ArrayStats#empty()} if unknown - public LongArray(DType dtype, long length, MemorySegment buffer, ArrayStats stats) { - this.dtype = dtype; - this.length = length; - this.buffer = buffer; - this.stats = stats; - } - - @Override - public DType dtype() { - return dtype; - } - - @Override - public long length() { - return length; - } - - /// Returns per-array statistics. - /// - /// @return array statistics - public ArrayStats stats() { - return stats; - } - - @Override - public MemorySegment buffer(int i) { - if (i != 0) { - throw new IndexOutOfBoundsException(i); - } - return buffer; - } - - /// Returns the long value at the given index. - /// - /// @param i zero-based index (must be in {@code [0, length)}) - /// @return the long value at position {@code i} - public long getLong(long i) { - return buffer.getAtIndex(PTypeIO.LE_LONG, i); - } - - /// Passes each element to the given consumer in order. - /// - /// @param c consumer that receives each long element - public void forEachLong(LongConsumer c) { - MemorySegment buf = buffer; - long n = length; - for (long i = 0; i < n; i++) { - c.accept(buf.getAtIndex(PTypeIO.LE_LONG, i)); - } - } - - /// Folds all elements using the given binary operator and identity value. - /// - /// @param identity initial accumulator value - /// @param op binary operator applied to the accumulator and each long element - /// @return the final accumulated result - public long fold(long identity, LongBinaryOperator op) { - MemorySegment buf = buffer; - long n = length; - long result = identity; - for (long i = 0; i < n; i++) { - result = op.applyAsLong(result, buf.getAtIndex(PTypeIO.LE_LONG, i)); - } - return result; - } -} diff --git a/core/src/main/java/io/github/dfa1/vortex/core/array/MaskedArray.java b/core/src/main/java/io/github/dfa1/vortex/core/array/MaskedArray.java deleted file mode 100644 index ace034d..0000000 --- a/core/src/main/java/io/github/dfa1/vortex/core/array/MaskedArray.java +++ /dev/null @@ -1,72 +0,0 @@ -package io.github.dfa1.vortex.core.array; - -import io.github.dfa1.vortex.core.DType; - -import java.lang.foreign.MemorySegment; - -/// Decoded {@code vortex.masked} array: a non-nullable child paired with an optional validity bitmap. -/// -///

Invariant: {@code child} has no actual nulls — nullability is expressed solely via -/// {@code validity}. When {@code validity} is {@code null} all positions are valid. -/// -///

{@link #buffer(int)} delegates to the child so that existing consumers that read raw -/// value buffers continue to work transparently. Use {@link #isValid(long)} to check -/// individual positions before trusting a value. -public final class MaskedArray implements Array { - - private final Array child; - private final BoolArray validity; - - /// Creates a new {@code MaskedArray} wrapping a non-nullable child with an optional validity bitmap. - /// - /// @param child non-nullable payload array - /// @param validity validity bitmap, or {@code null} if all positions are valid - public MaskedArray(Array child, BoolArray validity) { - this.child = child; - this.validity = validity; - } - - @Override - public DType dtype() { - return child.dtype().withNullable(true); - } - - @Override - public long length() { - return child.length(); - } - - @Override - public MemorySegment buffer(int i) { - return child.buffer(i); - } - - @Override - public Array child(int i) { - return switch (i) { - case 0 -> child; - case 1 -> { - if (validity == null) { - throw new IndexOutOfBoundsException("no validity child (AllValid)"); - } - yield validity; - } - default -> throw new IndexOutOfBoundsException(i); - }; - } - - /// Returns the validity bitmap, or {@code null} if all positions are valid. - /// - /// @return the validity bitmap, or {@code null} when all values are valid - public BoolArray validity() { - return validity; - } - - /// Returns {@code true} if the position at index {@code i} is valid (not null). - /// - /// @param i zero-based logical index - /// @return {@code true} if the value at {@code i} is valid - public boolean isValid(long i) { - return validity == null || validity.getBoolean(i); - } -} diff --git a/core/src/main/java/io/github/dfa1/vortex/core/array/NullArray.java b/core/src/main/java/io/github/dfa1/vortex/core/array/NullArray.java deleted file mode 100644 index c1df726..0000000 --- a/core/src/main/java/io/github/dfa1/vortex/core/array/NullArray.java +++ /dev/null @@ -1,19 +0,0 @@ -package io.github.dfa1.vortex.core.array; - -import io.github.dfa1.vortex.core.ArrayStats; -import io.github.dfa1.vortex.core.DType; - -/// Concrete [Array] for all-null columns ({@code DType.Null}). Holds only a row count. -/// -/// @param dtype the null dtype -/// @param length number of null elements -public record NullArray(DType dtype, long length) implements Array { - - - /// Returns per-array statistics (always empty for null arrays). - /// - /// @return empty array statistics - public ArrayStats stats() { - return ArrayStats.empty(); - } -} diff --git a/core/src/main/java/io/github/dfa1/vortex/core/array/ShortArray.java b/core/src/main/java/io/github/dfa1/vortex/core/array/ShortArray.java deleted file mode 100644 index 5c22991..0000000 --- a/core/src/main/java/io/github/dfa1/vortex/core/array/ShortArray.java +++ /dev/null @@ -1,89 +0,0 @@ -package io.github.dfa1.vortex.core.array; - -import io.github.dfa1.vortex.core.ArrayStats; -import io.github.dfa1.vortex.core.DType; -import io.github.dfa1.vortex.core.PType; -import io.github.dfa1.vortex.encoding.PTypeIO; - -import java.lang.foreign.MemorySegment; -import java.util.function.LongBinaryOperator; - -/// Concrete [Array] for I16/U16 primitive columns. -public final class ShortArray implements Array { - - private final DType dtype; - private final long length; - private final MemorySegment buffer; - private final ArrayStats stats; - - /// Creates a new {@code ShortArray} backed by the given memory segment. - /// - /// @param dtype logical type, must be I16 or U16 - /// @param length number of elements - /// @param buffer little-endian short data (2 bytes per element) - /// @param stats per-array statistics, or {@link io.github.dfa1.vortex.core.ArrayStats#empty()} if unknown - public ShortArray(DType dtype, long length, MemorySegment buffer, ArrayStats stats) { - this.dtype = dtype; - this.length = length; - this.buffer = buffer; - this.stats = stats; - } - - @Override - public DType dtype() { - return dtype; - } - - @Override - public long length() { - return length; - } - - /// Returns per-array statistics. - /// - /// @return array statistics - public ArrayStats stats() { - return stats; - } - - @Override - public MemorySegment buffer(int i) { - if (i != 0) { - throw new IndexOutOfBoundsException(i); - } - return buffer; - } - - /// Returns the raw signed short value at the given index. - /// - /// @param i zero-based index (must be in {@code [0, length)}) - /// @return the signed short value at position {@code i} - public short getShort(long i) { - return buffer.getAtIndex(PTypeIO.LE_SHORT, i); - } - - /// Returns the element at the given index as an {@code int}, widening to unsigned if the dtype is U16. - /// - /// @param i zero-based index (must be in {@code [0, length)}) - /// @return the element at position {@code i} as an int (unsigned-widened for U16) - public int getInt(long i) { - short raw = buffer.getAtIndex(PTypeIO.LE_SHORT, i); - boolean unsigned = dtype instanceof DType.Primitive p && p.ptype() == PType.U16; - return unsigned ? Short.toUnsignedInt(raw) : raw; - } - - /// Folds all elements using the given binary operator and identity value. - /// - /// @param identity initial accumulator value - /// @param op binary operator applied to the accumulator and each raw short element - /// @return the final accumulated result - public long fold(long identity, LongBinaryOperator op) { - MemorySegment buf = buffer; - long n = length; - long result = identity; - for (long i = 0; i < n; i++) { - result = op.applyAsLong(result, buf.getAtIndex(PTypeIO.LE_SHORT, i)); - } - return result; - } -} diff --git a/core/src/main/java/io/github/dfa1/vortex/core/array/StructArray.java b/core/src/main/java/io/github/dfa1/vortex/core/array/StructArray.java deleted file mode 100644 index 0066346..0000000 --- a/core/src/main/java/io/github/dfa1/vortex/core/array/StructArray.java +++ /dev/null @@ -1,75 +0,0 @@ -package io.github.dfa1.vortex.core.array; - -import io.github.dfa1.vortex.core.ArrayStats; -import io.github.dfa1.vortex.core.DType; -import io.github.dfa1.vortex.core.VortexException; - -import java.util.List; - -/// Decoded struct array: holds one [Array] per field, keyed by position. -/// -///

Returned by [io.github.dfa1.vortex.encoding.StructEncoding] when decoding a -/// multi-field {@code vortex.struct} segment (all columns packed into one flat). -public final class StructArray implements Array { - - private final DType.Struct dtype; - private final long length; - private final List fields; - - /// Creates a new {@code StructArray} from the given field arrays. - /// - /// @param dtype struct dtype providing field names and types - /// @param length number of rows (all field arrays must have the same length) - /// @param fields per-field arrays in the same order as {@code dtype.fieldTypes()} - public StructArray(DType.Struct dtype, long length, List fields) { - this.dtype = dtype; - this.length = length; - this.fields = List.copyOf(fields); - } - - @Override - public long length() { - return length; - } - - @Override - public DType dtype() { - return dtype; - } - - /// Returns per-array statistics (always empty for struct arrays). - /// - /// @return empty array statistics - public ArrayStats stats() { - return ArrayStats.empty(); - } - - /// Returns the number of fields in this struct. - /// - /// @return the field count - public int fieldCount() { - return fields.size(); - } - - /// Returns the field array at the given positional index. - /// - /// @param i zero-based field index - /// @return the field array at position {@code i} - public Array field(int i) { - return fields.get(i); - } - - /// Returns the field array with the given name. - /// - /// @param name the field name to look up - /// @return the field array with the given name - public Array field(String name) { - List names = dtype.fieldNames(); - for (int i = 0; i < names.size(); i++) { - if (names.get(i).equals(name)) { - return fields.get(i); - } - } - throw new VortexException("no field '" + name + "' in struct"); - } -} diff --git a/core/src/main/java/io/github/dfa1/vortex/core/array/UnknownArray.java b/core/src/main/java/io/github/dfa1/vortex/core/array/UnknownArray.java deleted file mode 100644 index 14a9853..0000000 --- a/core/src/main/java/io/github/dfa1/vortex/core/array/UnknownArray.java +++ /dev/null @@ -1,44 +0,0 @@ -package io.github.dfa1.vortex.core.array; - -import io.github.dfa1.vortex.core.ArrayStats; -import io.github.dfa1.vortex.core.DType; - -import java.lang.foreign.MemorySegment; -import java.nio.ByteBuffer; - -/// Opaque passthrough array for encodings unknown to this reader. -/// -/// Holds raw on-disk bytes: encoding id, metadata, buffers, child arrays (themselves -/// unknown), and stats. Children of an unknown node are always wrapped unknown — matches -/// Rust `decode_foreign` in `vortex-array/src/serde.rs`. -/// -/// Constructed by `EncodingRegistry` when `allowUnknown()` is set and an encoding id is not -/// in the registry. Data access beyond `buffer(i)` and `child(i)` is not supported. -/// -/// @param encodingId the unrecognised encoding id string -/// @param dtype logical type of the array -/// @param length number of logical rows -/// @param metadata raw encoding metadata bytes, or {@code null} -/// @param buffers raw data buffers owned by this node -/// @param children decoded child arrays (also wrapped as unknown) -/// @param stats per-array statistics -public record UnknownArray( - String encodingId, - DType dtype, - long length, - ByteBuffer metadata, - MemorySegment[] buffers, - Array[] children, - ArrayStats stats -) implements Array { - - @Override - public MemorySegment buffer(int i) { - return buffers[i]; - } - - @Override - public Array child(int i) { - return children[i]; - } -} diff --git a/core/src/main/java/io/github/dfa1/vortex/core/array/VarBinArray.java b/core/src/main/java/io/github/dfa1/vortex/core/array/VarBinArray.java deleted file mode 100644 index 462f8aa..0000000 --- a/core/src/main/java/io/github/dfa1/vortex/core/array/VarBinArray.java +++ /dev/null @@ -1,250 +0,0 @@ -package io.github.dfa1.vortex.core.array; - -import io.github.dfa1.vortex.core.ArrayStats; -import io.github.dfa1.vortex.core.DType; -import io.github.dfa1.vortex.core.PType; -import io.github.dfa1.vortex.core.VortexException; -import io.github.dfa1.vortex.encoding.PTypeIO; - -import java.lang.foreign.MemorySegment; -import java.lang.foreign.ValueLayout; -import java.nio.charset.StandardCharsets; -import java.util.function.IntConsumer; - -/// Concrete [Array] for variable-length binary / UTF-8 string columns. -/// -/// Holds a bytes segment and an offsets array; element {@code i} occupies -/// {@code bytes[offsets[i]..offsets[i+1]]}. Provides zero-allocation -/// {@link #getByteLength} and {@link #forEachByteLength} for callers that only -/// need lengths, and allocating {@link #getBytes} for full value retrieval. -/// -/// Stays backward-compatible via {@code buffer(0)} (→ bytes) and -/// {@code child(0)} (→ offsets Array) so encoding internals that haven't -/// been updated continue to work. -/// -/// Dict mode: created via {@link #ofDict}. Stores dict values + codes directly; -/// all accessors resolve through the dictionary without materializing strings. -public final class VarBinArray implements Array { - - private final DType dtype; - private final long length; - private final MemorySegment bytes; - private final Array offsetsArr; - private final MemorySegment offsetsSeg; - private final PType offsetsPtype; - private final ArrayStats stats; - - // dict mode: non-null when this array lazily resolves through a dictionary - private final MemorySegment dictValOffsets; - private final PType dictValOffPType; - private final MemorySegment dictCodesSegs; - private final PType dictCodesPType; - - /// Creates a new {@code VarBinArray} in offset mode. - /// - /// @param dtype logical type (Utf8 or Binary) - /// @param length number of variable-length elements - /// @param bytes concatenated raw byte data for all elements - /// @param offsetsArr offsets array of length {@code length + 1}; {@code offsets[i]..offsets[i+1]} is element {@code i} - /// @param offsetsPtype physical type of the offsets values (I32/U32 or I64/U64) - /// @param stats per-array statistics, or {@link io.github.dfa1.vortex.core.ArrayStats#empty()} if unknown - public VarBinArray(DType dtype, long length, MemorySegment bytes, - Array offsetsArr, PType offsetsPtype, ArrayStats stats) { - this.dtype = dtype; - this.length = length; - this.bytes = bytes; - this.offsetsArr = offsetsArr; - this.offsetsSeg = offsetsArr.buffer(0); - this.offsetsPtype = offsetsPtype; - this.stats = stats; - this.dictValOffsets = null; - this.dictValOffPType = null; - this.dictCodesSegs = null; - this.dictCodesPType = null; - } - - private VarBinArray(DType dtype, long length, - MemorySegment dictValBytes, MemorySegment dictValOffsets, PType dictValOffPType, - MemorySegment dictCodesSegs, PType dictCodesPType, ArrayStats stats) { - this.dtype = dtype; - this.length = length; - this.bytes = dictValBytes; - this.offsetsArr = null; - this.offsetsSeg = null; - this.offsetsPtype = null; - this.stats = stats; - this.dictValOffsets = dictValOffsets; - this.dictValOffPType = dictValOffPType; - this.dictCodesSegs = dictCodesSegs; - this.dictCodesPType = dictCodesPType; - } - - /// Creates a dict-mode VarBinArray. Lengths and bytes are resolved via the - /// dictionary on each access; no string materialization occurs at construction time. - /// - /// @param dtype logical type (Utf8 or Binary) - /// @param n number of logical elements (rows) - /// @param dictValBytes concatenated raw bytes for all dictionary values - /// @param dictValOffsets offsets into {@code dictValBytes} for each dictionary entry (length = dictSize + 1) - /// @param dictValOffPType physical type of the dictionary value offsets - /// @param dictCodesSegs per-row dictionary code indices (length = n) - /// @param dictCodesPType physical type of the dictionary codes - /// @param stats per-array statistics, or {@link io.github.dfa1.vortex.core.ArrayStats#empty()} if unknown - /// @return a new dict-mode {@code VarBinArray} - public static VarBinArray ofDict(DType dtype, long n, - MemorySegment dictValBytes, - MemorySegment dictValOffsets, PType dictValOffPType, - MemorySegment dictCodesSegs, PType dictCodesPType, - ArrayStats stats) { - return new VarBinArray(dtype, n, dictValBytes, dictValOffsets, dictValOffPType, - dictCodesSegs, dictCodesPType, stats); - } - - @Override - public DType dtype() { - return dtype; - } - - @Override - public long length() { - return length; - } - - /// Returns per-array statistics. - /// - /// @return array statistics - public ArrayStats stats() { - return stats; - } - - @Override - public MemorySegment buffer(int i) { - if (i != 0) { - throw new IndexOutOfBoundsException(i); - } - return bytes; - } - - @Override - public Array child(int i) { - if (i != 0) { - throw new IndexOutOfBoundsException(i); - } - if (offsetsArr == null) { - throw new IllegalStateException("child(0) not available in dict mode"); - } - return offsetsArr; - } - - /// Returns a copy of the raw bytes for element {@code i}. - /// - /// @param i zero-based logical index (must be in {@code [0, length)}) - /// @return a newly allocated byte array containing the raw bytes of element {@code i} - public byte[] getBytes(long i) { - if (dictCodesSegs != null) { - long code = dictReadCode(i); - long start = dictReadOff(code); - long end = dictReadOff(code + 1); - byte[] out = new byte[(int) (end - start)]; - MemorySegment.copy(bytes, start, MemorySegment.ofArray(out), 0, end - start); - return out; - } - long start = readOffset(i); - long end = readOffset(i + 1); - byte[] out = new byte[(int) (end - start)]; - MemorySegment.copy(bytes, start, MemorySegment.ofArray(out), 0, end - start); - return out; - } - - /// Returns the UTF-8 decoded string for element {@code i}. - /// - /// @param i zero-based logical index (must be in {@code [0, length)}) - /// @return the UTF-8 string at position {@code i} - public String getString(long i) { - return new String(getBytes(i), StandardCharsets.UTF_8); - } - - /// Returns the byte length of element {@code i} without copying the data. - /// - /// @param i zero-based logical index (must be in {@code [0, length)}) - /// @return the number of bytes in element {@code i} - public int getByteLength(long i) { - if (dictCodesSegs != null) { - long code = dictReadCode(i); - return (int) (dictReadOff(code + 1) - dictReadOff(code)); - } - return (int) (readOffset(i + 1) - readOffset(i)); - } - - /// Passes the byte length of each element to the given consumer in row order. - /// - /// @param c consumer that receives the byte length of each element - public void forEachByteLength(IntConsumer c) { - if (dictCodesSegs != null) { - for (long i = 0; i < length; i++) { - long code = dictReadCode(i); - c.accept((int) (dictReadOff(code + 1) - dictReadOff(code))); - } - return; - } - MemorySegment seg = offsetsSeg; - long n = length; - if (offsetsPtype == PType.I32 || offsetsPtype == PType.U32) { - for (long i = 0; i < n; i++) { - c.accept(seg.getAtIndex(PTypeIO.LE_INT, i + 1) - seg.getAtIndex(PTypeIO.LE_INT, i)); - } - } else { - for (long i = 0; i < n; i++) { - c.accept((int) (seg.getAtIndex(PTypeIO.LE_LONG, i + 1) - seg.getAtIndex(PTypeIO.LE_LONG, i))); - } - } - } - - private long readOffset(long i) { - if (offsetsPtype == PType.I32 || offsetsPtype == PType.U32) { - return offsetsSeg.getAtIndex(PTypeIO.LE_INT, i); - } - return offsetsSeg.getAtIndex(PTypeIO.LE_LONG, i); - } - - private long dictReadCode(long i) { - return switch (dictCodesPType) { - case U8 -> Byte.toUnsignedLong(dictCodesSegs.get(ValueLayout.JAVA_BYTE, i)); - case U16 -> Short.toUnsignedLong(dictCodesSegs.getAtIndex(PTypeIO.LE_SHORT, i)); - case U32 -> Integer.toUnsignedLong(dictCodesSegs.getAtIndex(PTypeIO.LE_INT, i)); - case I32 -> dictCodesSegs.getAtIndex(PTypeIO.LE_INT, i); - case I64, U64 -> dictCodesSegs.getAtIndex(PTypeIO.LE_LONG, i); - default -> throw new VortexException("unsupported codes ptype: " + dictCodesPType); - }; - } - - private long dictReadOff(long i) { - if (dictValOffPType == PType.I32 || dictValOffPType == PType.U32) { - return dictValOffsets.getAtIndex(PTypeIO.LE_INT, i); - } - return dictValOffsets.getAtIndex(PTypeIO.LE_LONG, i); - } - - /// Returns a new VarBinArray containing only the first {@code rows} elements. - /// - /// @param rows number of rows to retain; if {@code rows >= length} returns this array unchanged - /// @return a {@code VarBinArray} containing the first {@code rows} elements - public VarBinArray truncate(long rows) { - if (rows >= length) { - return this; - } - if (dictCodesSegs != null) { - int codeBytes = dictCodesPType.byteSize(); - return VarBinArray.ofDict(dtype, rows, bytes, dictValOffsets, dictValOffPType, - dictCodesSegs.asSlice(0, rows * codeBytes), dictCodesPType, ArrayStats.empty()); - } - long byteEnd = readOffset(rows); - int offBytes = (offsetsPtype == PType.I32 || offsetsPtype == PType.U32) ? Integer.BYTES : Long.BYTES; - MemorySegment newOffsetsSeg = offsetsSeg.asSlice(0, (rows + 1) * offBytes); - DType offDtype = new DType.Primitive(offsetsPtype, false); - Array newOffsetsArr = (offBytes == Integer.BYTES) - ? new IntArray(offDtype, rows + 1, newOffsetsSeg, ArrayStats.empty()) - : new LongArray(offDtype, rows + 1, newOffsetsSeg, ArrayStats.empty()); - return new VarBinArray(dtype, rows, bytes.asSlice(0, byteEnd > 0 ? byteEnd : 0), newOffsetsArr, offsetsPtype, ArrayStats.empty()); - } -} diff --git a/core/src/main/java/io/github/dfa1/vortex/encoding/AlpEncoding.java b/core/src/main/java/io/github/dfa1/vortex/encoding/AlpEncoding.java deleted file mode 100644 index f527467..0000000 --- a/core/src/main/java/io/github/dfa1/vortex/encoding/AlpEncoding.java +++ /dev/null @@ -1,519 +0,0 @@ -package io.github.dfa1.vortex.encoding; - -import com.google.protobuf.InvalidProtocolBufferException; -import io.github.dfa1.vortex.proto.DTypeProtos; -import io.github.dfa1.vortex.proto.EncodingProtos; -import io.github.dfa1.vortex.proto.ScalarProtos; -import io.github.dfa1.vortex.core.array.Array; -import io.github.dfa1.vortex.core.ArrayStats; -import io.github.dfa1.vortex.core.DType; -import io.github.dfa1.vortex.core.PType; -import io.github.dfa1.vortex.core.VortexException; -import io.github.dfa1.vortex.core.array.DoubleArray; -import io.github.dfa1.vortex.core.array.FloatArray; - -import java.lang.foreign.Arena; -import java.lang.foreign.MemorySegment; -import java.lang.foreign.ValueLayout; -import java.nio.ByteBuffer; -import java.util.ArrayList; -import java.util.List; - -/// Decoder for {@code vortex.alp} — Adaptive Lossless floating-Point compression. -/// -///

Metadata: protobuf {@code ALPMetadata} — {@code exp_e u32} (tag 1), -/// {@code exp_f u32} (tag 2), optional {@code patches PatchesMetadata} (tag 3). -/// -///

Child slot 0: encoded integers (I32 for F32, I64 for F64 parent), rowCount = array len. -/// Child slot 1: patch indices (if patches), rowCount = patches.len. -/// Child slot 2: patch values (if patches, same dtype as parent), rowCount = patches.len. -/// Child slot 3: chunk offsets (optional, ignored — patches applied by absolute index). -/// -///

Decode: {@code decoded[i] = (float/double) encoded[i] * F10[exp_f] * IF10[exp_e]}, -/// then overwrite {@code decoded[indices[j] - offset] = values[j]} for each patch. -public final class AlpEncoding implements Encoding { - - /// Creates a new {@code AlpEncoding} instance; use via {@link EncodingRegistry}. - public AlpEncoding() { - } - - // Powers of 10 for F64 — shared by encode (exponent search) and decode (reconstruction). - static final double[] F10_F64 = { - 1e0, 1e1, 1e2, 1e3, 1e4, 1e5, 1e6, 1e7, 1e8, 1e9, - 1e10, 1e11, 1e12, 1e13, 1e14, 1e15, 1e16, 1e17, 1e18, 1e19, - 1e20, 1e21, 1e22, 1e23 - }; - - static final double[] IF10_F64 = { - 1e-0, 1e-1, 1e-2, 1e-3, 1e-4, 1e-5, 1e-6, 1e-7, 1e-8, 1e-9, - 1e-10, 1e-11, 1e-12, 1e-13, 1e-14, 1e-15, 1e-16, 1e-17, 1e-18, 1e-19, - 1e-20, 1e-21, 1e-22, 1e-23 - }; - - // Powers of 10 for F32 — shared by encode (exponent search) and decode (reconstruction). - static final float[] F10_F32 = { - 1e0f, 1e1f, 1e2f, 1e3f, 1e4f, 1e5f, 1e6f, 1e7f, 1e8f, 1e9f, 1e10f - }; - - static final float[] IF10_F32 = { - 1e-0f, 1e-1f, 1e-2f, 1e-3f, 1e-4f, 1e-5f, 1e-6f, 1e-7f, 1e-8f, 1e-9f, 1e-10f - }; - - static final DType I64_DTYPE = new DType.Primitive(PType.I64, false); - static final DType I32_DTYPE = new DType.Primitive(PType.I32, false); - - @Override - public EncodingId encodingId() { - return EncodingId.VORTEX_ALP; - } - - @Override - public boolean accepts(DType dtype) { - if (!(dtype instanceof DType.Primitive p)) { - return false; - } - return p.ptype() == PType.F64 || p.ptype() == PType.F32; - } - - @Override - public EncodeResult encode(DType dtype, Object data) { - return Encoder.encode(dtype, data); - } - - @Override - public CascadeStep encodeCascade(DType dtype, Object data, CompressorContext ctx) { - return Encoder.encodeCascade(dtype, data); - } - - @Override - public Array decode(DecodeContext ctx) { - return Decoder.decode(ctx); - } - - private static final class Encoder { - - // Rust ALPFloat::MAX_EXPONENT bounds (inclusive): f64 → 18, f32 → 10. - private static final int MAX_EXPONENT_F64 = 18; - private static final int MAX_EXPONENT_F32 = 10; - private static final int SAMPLE_SIZE = 512; - - private record AlpF64Data(int expE, int expF, long[] encodedArr, - List patchIndices, List patchValues, - byte[] statsMin, byte[] statsMax) {} - - private static EncodeResult encode(DType dtype, Object data) { - PType ptype = ((DType.Primitive) dtype).ptype(); - return switch (ptype) { - case F64 -> encodeF64((double[]) data, dtype); - case F32 -> encodeF32((float[]) data, dtype); - default -> throw new UnsupportedOperationException("ALP encode not supported for " + ptype); - }; - } - - private static CascadeStep encodeCascade(DType dtype, Object data) { - PType ptype = ((DType.Primitive) dtype).ptype(); - if (ptype == PType.F64) { - return encodeCascadeF64((double[]) data); - } - return CascadeStep.terminal(encode(dtype, data)); - } - - private static int[] findExponentsF64(double[] values) { - int sampleLen = Math.min(SAMPLE_SIZE, values.length); - int bestExpE = 0, bestExpF = 0, bestExceptions = sampleLen + 1; - - outer: - for (int expE = 0; expE <= MAX_EXPONENT_F64; expE++) { - for (int expF = 0; expF <= MAX_EXPONENT_F64; expF++) { - double ef = F10_F64[expE]; - double iff = IF10_F64[expF]; - double df = F10_F64[expF]; - double de = IF10_F64[expE]; - int exceptions = 0; - for (int i = 0; i < sampleLen; i++) { - double enc = values[i] * ef * iff; - if (!Double.isFinite(enc) || (double) Math.round(enc) * df * de != values[i]) { - exceptions++; - } - } - if (exceptions < bestExceptions) { - bestExceptions = exceptions; - bestExpE = expE; - bestExpF = expF; - if (bestExceptions == 0) { - break outer; - } - } - } - } - return new int[]{bestExpE, bestExpF}; - } - - private static AlpF64Data computeF64(double[] values) { - int n = values.length; - int[] exps = findExponentsF64(values); - int expE = exps[0], expF = exps[1]; - double ef = F10_F64[expE]; - double iff = IF10_F64[expF]; - double df = F10_F64[expF]; - double de = IF10_F64[expE]; - - long[] encodedArr = new long[n]; - var patchIndices = new ArrayList(); - var patchValues = new ArrayList(); - - double min = Double.MAX_VALUE, max = -Double.MAX_VALUE; - for (int i = 0; i < n; i++) { - double v = values[i]; - double enc = v * ef * iff; - long encoded; - if (Double.isFinite(enc) && (double) (encoded = Math.round(enc)) * df * de == v) { - encodedArr[i] = encoded; - } else { - encodedArr[i] = 0L; - patchIndices.add(i); - patchValues.add(v); - } - if (v < min) { - min = v; - } - if (v > max) { - max = v; - } - } - - byte[] statsMin = n > 0 ? scalarF64(min) : null; - byte[] statsMax = n > 0 ? scalarF64(max) : null; - return new AlpF64Data(expE, expF, encodedArr, patchIndices, patchValues, statsMin, statsMax); - } - - private static EncodeResult encodeF64(double[] values, DType dtype) { - AlpF64Data d = computeF64(values); - int n = values.length; - - MemorySegment encodedBuf = Arena.ofAuto().allocate((long) n * 8, 8); - for (int i = 0; i < n; i++) { - encodedBuf.setAtIndex(PTypeIO.LE_LONG, i, d.encodedArr()[i]); - } - - EncodeNode encodedNode = EncodeNode.leaf(EncodingId.VORTEX_PRIMITIVE, 0); - - if (d.patchIndices().isEmpty()) { - byte[] metaBytes = EncodingProtos.ALPMetadata.newBuilder() - .setExpE(d.expE()).setExpF(d.expF()).build().toByteArray(); - EncodeNode root = new EncodeNode(EncodingId.VORTEX_ALP, - ByteBuffer.wrap(metaBytes), new EncodeNode[]{encodedNode}, new int[0]); - return new EncodeResult(root, List.of(encodedBuf), d.statsMin(), d.statsMax()); - } - - int numPatches = d.patchIndices().size(); - MemorySegment idxBuf = Arena.ofAuto().allocate((long) numPatches * 4, 4); - MemorySegment valBuf = Arena.ofAuto().allocate((long) numPatches * 8, 8); - for (int i = 0; i < numPatches; i++) { - idxBuf.setAtIndex(PTypeIO.LE_INT, i, d.patchIndices().get(i)); - valBuf.setAtIndex(PTypeIO.LE_DOUBLE, i, d.patchValues().get(i)); - } - - EncodingProtos.PatchesMetadata patches = buildPatchesMeta(numPatches); - byte[] metaBytes = EncodingProtos.ALPMetadata.newBuilder() - .setExpE(d.expE()).setExpF(d.expF()).setPatches(patches).build().toByteArray(); - - EncodeNode idxNode = EncodeNode.leaf(EncodingId.VORTEX_PRIMITIVE, 1); - EncodeNode valNode = EncodeNode.leaf(EncodingId.VORTEX_PRIMITIVE, 2); - EncodeNode root = new EncodeNode(EncodingId.VORTEX_ALP, - ByteBuffer.wrap(metaBytes), - new EncodeNode[]{encodedNode, idxNode, valNode}, - new int[0]); - return new EncodeResult(root, List.of(encodedBuf, idxBuf, valBuf), d.statsMin(), d.statsMax()); - } - - /// Cascade-aware F64 encode: I64 child is an open ChildSlot so the compressor - /// can bitpack it. Patch idx/val buffers (if any) stay in ownedBuffers. - private static CascadeStep encodeCascadeF64(double[] values) { - AlpF64Data d = computeF64(values); - int n = values.length; - - if (d.patchIndices().isEmpty()) { - byte[] metaBytes = EncodingProtos.ALPMetadata.newBuilder() - .setExpE(d.expE()).setExpF(d.expF()).build().toByteArray(); - // children[0] will be filled by the compressor after recursing on the ChildSlot - EncodeNode partialRoot = new EncodeNode(EncodingId.VORTEX_ALP, - ByteBuffer.wrap(metaBytes), new EncodeNode[1], new int[0]); - ChildSlot slot = new ChildSlot(I64_DTYPE, d.encodedArr(), 0); - return new CascadeStep(partialRoot, List.of(), List.of(slot), d.statsMin(), d.statsMax(), true); - } - - int numPatches = d.patchIndices().size(); - MemorySegment idxBuf = Arena.ofAuto().allocate((long) numPatches * 4, 4); - MemorySegment valBuf = Arena.ofAuto().allocate((long) numPatches * 8, 8); - for (int i = 0; i < numPatches; i++) { - idxBuf.setAtIndex(PTypeIO.LE_INT, i, d.patchIndices().get(i)); - valBuf.setAtIndex(PTypeIO.LE_DOUBLE, i, d.patchValues().get(i)); - } - - EncodingProtos.PatchesMetadata patches = buildPatchesMeta(numPatches); - byte[] metaBytes = EncodingProtos.ALPMetadata.newBuilder() - .setExpE(d.expE()).setExpF(d.expF()).setPatches(patches).build().toByteArray(); - - // ownedBuffers[0]=idxBuf, [1]=valBuf; child I64 will be appended after (starting at idx 2) - EncodeNode idxNode = EncodeNode.leaf(EncodingId.VORTEX_PRIMITIVE, 0); - EncodeNode valNode = EncodeNode.leaf(EncodingId.VORTEX_PRIMITIVE, 1); - EncodeNode partialRoot = new EncodeNode(EncodingId.VORTEX_ALP, - ByteBuffer.wrap(metaBytes), new EncodeNode[]{null, idxNode, valNode}, new int[0]); - ChildSlot slot = new ChildSlot(I64_DTYPE, d.encodedArr(), 0); - return new CascadeStep(partialRoot, List.of(idxBuf, valBuf), List.of(slot), d.statsMin(), d.statsMax(), true); - } - - private static int[] findExponentsF32(float[] values) { - int sampleLen = Math.min(SAMPLE_SIZE, values.length); - int bestExpE = 0, bestExpF = 0, bestExceptions = sampleLen + 1; - - outer: - for (int expE = 0; expE <= MAX_EXPONENT_F32; expE++) { - for (int expF = 0; expF <= MAX_EXPONENT_F32; expF++) { - float ef = F10_F32[expE]; - float iff = IF10_F32[expF]; - float df = F10_F32[expF]; - float de = IF10_F32[expE]; - int exceptions = 0; - for (int i = 0; i < sampleLen; i++) { - float enc = values[i] * ef * iff; - if (!Float.isFinite(enc) || (float) Math.round(enc) * df * de != values[i]) { - exceptions++; - } - } - if (exceptions < bestExceptions) { - bestExceptions = exceptions; - bestExpE = expE; - bestExpF = expF; - if (bestExceptions == 0) { - break outer; - } - } - } - } - return new int[]{bestExpE, bestExpF}; - } - - private static EncodeResult encodeF32(float[] values, DType dtype) { - int n = values.length; - int[] exps = findExponentsF32(values); - int expE = exps[0], expF = exps[1]; - float ef = F10_F32[expE]; - float iff = IF10_F32[expF]; - float df = F10_F32[expF]; - float de = IF10_F32[expE]; - - int[] encodedArr = new int[n]; - var patchIndices = new ArrayList(); - var patchValues = new ArrayList(); - - float min = Float.MAX_VALUE, max = -Float.MAX_VALUE; - for (int i = 0; i < n; i++) { - float v = values[i]; - float enc = v * ef * iff; - int encoded; - if (Float.isFinite(enc) && (float) (encoded = Math.round(enc)) * df * de == v) { - encodedArr[i] = encoded; - } else { - encodedArr[i] = 0; - patchIndices.add(i); - patchValues.add(v); - } - if (v < min) { - min = v; - } - if (v > max) { - max = v; - } - } - - byte[] statsMin = n > 0 ? scalarF32(min) : null; - byte[] statsMax = n > 0 ? scalarF32(max) : null; - - MemorySegment encodedBuf = Arena.ofAuto().allocate((long) n * 4, 4); - for (int i = 0; i < n; i++) { - encodedBuf.setAtIndex(PTypeIO.LE_INT, i, encodedArr[i]); - } - - EncodeNode encodedNode = EncodeNode.leaf(EncodingId.VORTEX_PRIMITIVE, 0); - - if (patchIndices.isEmpty()) { - byte[] metaBytes = EncodingProtos.ALPMetadata.newBuilder() - .setExpE(expE).setExpF(expF).build().toByteArray(); - EncodeNode root = new EncodeNode(EncodingId.VORTEX_ALP, - ByteBuffer.wrap(metaBytes), new EncodeNode[]{encodedNode}, new int[0]); - return new EncodeResult(root, List.of(encodedBuf), statsMin, statsMax); - } - - int numPatches = patchIndices.size(); - MemorySegment idxBuf = Arena.ofAuto().allocate((long) numPatches * 4, 4); - MemorySegment valBuf = Arena.ofAuto().allocate((long) numPatches * 4, 4); - for (int i = 0; i < numPatches; i++) { - idxBuf.setAtIndex(PTypeIO.LE_INT, i, patchIndices.get(i)); - valBuf.setAtIndex(PTypeIO.LE_FLOAT, i, patchValues.get(i)); - } - - EncodingProtos.PatchesMetadata patches = EncodingProtos.PatchesMetadata.newBuilder() - .setLen(numPatches) - .setOffset(0) - .setIndicesPtype(DTypeProtos.PType.forNumber(PType.U32.ordinal())) - .build(); - byte[] metaBytes = EncodingProtos.ALPMetadata.newBuilder() - .setExpE(expE).setExpF(expF).setPatches(patches).build().toByteArray(); - - EncodeNode idxNode = EncodeNode.leaf(EncodingId.VORTEX_PRIMITIVE, 1); - EncodeNode valNode = EncodeNode.leaf(EncodingId.VORTEX_PRIMITIVE, 2); - EncodeNode root = new EncodeNode(EncodingId.VORTEX_ALP, - ByteBuffer.wrap(metaBytes), - new EncodeNode[]{encodedNode, idxNode, valNode}, - new int[0]); - return new EncodeResult(root, List.of(encodedBuf, idxBuf, valBuf), statsMin, statsMax); - } - - private static EncodingProtos.PatchesMetadata buildPatchesMeta(int numPatches) { - return EncodingProtos.PatchesMetadata.newBuilder() - .setLen(numPatches) - .setOffset(0) - .setIndicesPtype(DTypeProtos.PType.forNumber(PType.U32.ordinal())) - .build(); - } - - private static byte[] scalarF64(double v) { - return ScalarProtos.ScalarValue.newBuilder().setF64Value(v).build().toByteArray(); - } - - private static byte[] scalarF32(float v) { - return ScalarProtos.ScalarValue.newBuilder().setF32Value(v).build().toByteArray(); - } - } - - private static final class Decoder { - - private static Array decode(DecodeContext ctx) { - ByteBuffer rawMeta = ctx.metadata(); - // Proto3 omits fields equal to their default value (0), so ALPMetadata{expE=0,expF=0} - // serialises to 0 bytes; treat null/empty metadata as the all-zero default instance. - EncodingProtos.ALPMetadata meta; - if (rawMeta == null || !rawMeta.hasRemaining()) { - meta = EncodingProtos.ALPMetadata.getDefaultInstance(); - } else { - try { - meta = EncodingProtos.ALPMetadata.parseFrom(rawMeta.duplicate()); - } catch (InvalidProtocolBufferException e) { - throw new VortexException(EncodingId.VORTEX_ALP, "invalid metadata", e); - } - } - - if (!(ctx.dtype() instanceof DType.Primitive p)) { - throw new VortexException(EncodingId.VORTEX_ALP, "expected primitive dtype, got " + ctx.dtype()); - } - - int expE = meta.getExpE(); - int expF = meta.getExpF(); - PType ptype = p.ptype(); - long n = ctx.rowCount(); - - return switch (ptype) { - case F64 -> decodeF64(ctx, meta, expE, expF, n); - case F32 -> decodeF32(ctx, meta, expE, expF, n); - default -> throw new VortexException(EncodingId.VORTEX_ALP, "unsupported dtype " + ptype); - }; - } - - private static Array decodeF64(DecodeContext ctx, EncodingProtos.ALPMetadata meta, int expE, int expF, long n) { - Array encoded = decodeChildAs(ctx, 0, I64_DTYPE, n); - - // Two separate lookups match Rust's left-to-right: (encoded * F10[f]) * IF10[e]. - // Precomputing a single factor = F10[f] * IF10[e] then encoded * factor uses different - // associativity and can produce different IEEE 754 results for some (expE, expF) pairs. - double df = F10_F64[expF]; - double de = IF10_F64[expE]; - - MemorySegment src = encoded.buffer(0); - // In-place when the child returned a writable arena buffer (e.g. BitpackedEncoding, DeltaEncoding). - // Fall back to a new allocation when the source is a read-only mmap slice (PrimitiveEncoding). - MemorySegment buf = src.isReadOnly() ? ctx.arena().allocate(n * 8, 8) : src; - if (src.isReadOnly()) { - for (long i = 0; i < n; i++) { - buf.setAtIndex(PTypeIO.LE_DOUBLE, i, (double) src.getAtIndex(PTypeIO.LE_LONG, i) * df * de); - } - } else { - for (long i = 0; i < n; i++) { - buf.setAtIndex(PTypeIO.LE_DOUBLE, i, (double) buf.getAtIndex(PTypeIO.LE_LONG, i) * df * de); - } - } - - if (meta.hasPatches()) { - applyPatches(ctx, meta.getPatches(), buf, PTypeIO.LE_LONG, 8); - } - - return new DoubleArray(ctx.dtype(), n, buf.asReadOnly(), ArrayStats.empty()); - } - - private static Array decodeF32(DecodeContext ctx, EncodingProtos.ALPMetadata meta, int expE, int expF, long n) { - Array encoded = decodeChildAs(ctx, 0, I32_DTYPE, n); - - // Two separate lookups match Rust's left-to-right: (encoded * F10[f]) * IF10[e]. - float df = F10_F32[expF]; - float de = IF10_F32[expE]; - - MemorySegment src32 = encoded.buffer(0); - MemorySegment buf32 = src32.isReadOnly() ? ctx.arena().allocate(n * 4, 4) : src32; - if (src32.isReadOnly()) { - for (long i = 0; i < n; i++) { - buf32.setAtIndex(PTypeIO.LE_FLOAT, i, (float) src32.getAtIndex(PTypeIO.LE_INT, i) * df * de); - } - } else { - for (long i = 0; i < n; i++) { - buf32.setAtIndex(PTypeIO.LE_FLOAT, i, (float) buf32.getAtIndex(PTypeIO.LE_INT, i) * df * de); - } - } - - if (meta.hasPatches()) { - applyPatches(ctx, meta.getPatches(), buf32, PTypeIO.LE_INT, 4); - } - - return new FloatArray(ctx.dtype(), n, buf32.asReadOnly(), ArrayStats.empty()); - } - - private static void applyPatches(DecodeContext ctx, EncodingProtos.PatchesMetadata pm, - MemorySegment out, ValueLayout elemLayout, int elemBytes) { - long numPatches = pm.getLen(); - long offset = pm.getOffset(); - PType idxPtype = ptypeFromProto(pm.getIndicesPtype()); - - Array idxArr = decodeChildAs(ctx, 1, new DType.Primitive(idxPtype, false), numPatches); - Array valArr = decodeChildAs(ctx, 2, ctx.dtype(), numPatches); - - MemorySegment idxSeg = idxArr.buffer(0); - MemorySegment valSeg = valArr.buffer(0); - - for (long i = 0; i < numPatches; i++) { - long absIdx = readUnsigned(idxSeg, i, idxPtype) - offset; - MemorySegment.copy(valSeg, i * elemBytes, out, absIdx * elemBytes, elemBytes); - } - } - - private static Array decodeChildAs(DecodeContext parent, int childIdx, DType dtype, long rowCount) { - ArrayNode childNode = parent.node().children()[childIdx]; - DecodeContext childCtx = new DecodeContext( - childNode, dtype, rowCount, parent.segmentBuffers(), parent.registry(), parent.arena()); - return parent.registry().decode(childCtx); - } - - private static long readUnsigned(MemorySegment seg, long i, PType ptype) { - return switch (ptype) { - case U8 -> Byte.toUnsignedLong(seg.get(ValueLayout.JAVA_BYTE, i)); - case U16 -> Short.toUnsignedLong(seg.get(PTypeIO.LE_SHORT, i * 2)); - case U32 -> Integer.toUnsignedLong(seg.get(PTypeIO.LE_INT, i * 4)); - case U64 -> seg.get(PTypeIO.LE_LONG, i * 8); - default -> throw new VortexException(EncodingId.VORTEX_ALP, "non-unsigned patch index ptype " + ptype); - }; - } - - private static PType ptypeFromProto(DTypeProtos.PType proto) { - return PType.values()[proto.getNumber()]; - } - } -} diff --git a/core/src/main/java/io/github/dfa1/vortex/encoding/AlpRdEncoding.java b/core/src/main/java/io/github/dfa1/vortex/encoding/AlpRdEncoding.java deleted file mode 100644 index 44eb80c..0000000 --- a/core/src/main/java/io/github/dfa1/vortex/encoding/AlpRdEncoding.java +++ /dev/null @@ -1,523 +0,0 @@ -package io.github.dfa1.vortex.encoding; - -import com.google.protobuf.InvalidProtocolBufferException; -import io.github.dfa1.vortex.core.DType; -import io.github.dfa1.vortex.core.PType; -import io.github.dfa1.vortex.core.VortexException; -import io.github.dfa1.vortex.core.ArrayStats; -import io.github.dfa1.vortex.core.array.Array; -import io.github.dfa1.vortex.core.array.DoubleArray; -import io.github.dfa1.vortex.core.array.FloatArray; -import io.github.dfa1.vortex.proto.DTypeProtos; -import io.github.dfa1.vortex.proto.EncodingProtos; - -import java.lang.foreign.MemorySegment; -import java.nio.ByteBuffer; -import java.util.ArrayList; -import java.util.HashMap; -import java.util.List; -import java.util.Map; - -/// Encoder/decoder for {@code vortex.alprd} — ALP Real Doubles. -/// -///

Splits each float's bit pattern at a cut point {@code p} (1..16 MSBs): -/// left {@code p} bits are dictionary-coded (≤8 entries, codes bitpacked to 1–3 bits); -/// right {@code BITS-p} bits are bitpacked directly. -/// Values whose left bits fall outside the dictionary are stored as exceptions (patches). -/// -///

Metadata: protobuf {@code ALPRDMetadata} — {@code right_bit_width u32} (tag 1), -/// {@code dict_len u32} (tag 2), {@code dict repeated u32} (tag 3), -/// {@code left_parts_ptype PType} (tag 4), optional {@code patches PatchesMetadata} (tag 5). -/// -///

Children: -///

    -///
  • 0: left_parts — bitpacked U16 dictionary codes
    • -///
  • 1: right_parts — bitpacked U32 (F32) or U64 (F64) right bit-patterns
    • -///
  • 2: patch_indices (optional) — bitpacked U64 exception positions
    • -///
  • 3: patch_values (optional) — U16 raw left bit-patterns for exceptions
    • -///
-public final class AlpRdEncoding implements Encoding { - - /// Creates a new {@code AlpRdEncoding} instance; use via {@link EncodingRegistry}. - public AlpRdEncoding() { - } - - private static final DType U16_DTYPE = new DType.Primitive(PType.U16, false); - private static final DType U32_DTYPE = new DType.Primitive(PType.U32, false); - private static final DType U64_DTYPE = new DType.Primitive(PType.U64, false); - - @Override - public EncodingId encodingId() { - return EncodingId.VORTEX_ALPRD; - } - - @Override - public boolean accepts(DType dtype) { - if (!(dtype instanceof DType.Primitive p)) { - return false; - } - return p.ptype() == PType.F32 || p.ptype() == PType.F64; - } - - @Override - public EncodeResult encode(DType dtype, Object data) { - return Encoder.encode(dtype, data); - } - - @Override - public Array decode(DecodeContext ctx) { - return Decoder.decode(ctx); - } - - // ------------------------------------------------------------------------- - // Encoder - // ------------------------------------------------------------------------- - - private static final class Encoder { - - private static final int SAMPLE_SIZE = 512; - private static final int MAX_CUT = 16; - private static final int MAX_DICT_SIZE = 8; - - static EncodeResult encode(DType dtype, Object data) { - PType ptype = ((DType.Primitive) dtype).ptype(); - return switch (ptype) { - case F64 -> encodeF64((double[]) data); - case F32 -> encodeF32((float[]) data); - default -> throw new UnsupportedOperationException("ALP-RD encode not supported for " + ptype); - }; - } - - // --- F64 --- - - private static EncodeResult encodeF64(double[] values) { - int n = values.length; - if (n == 0) { - return emptyResult(U64_DTYPE, U16_DTYPE); - } - - int sampleLen = Math.min(SAMPLE_SIZE, n); - Dictionary64 best = findBestDictionaryF64(values, sampleLen); - - Map lookup = buildLookup(best.dict); - long rightMask = -1L >>> (64 - best.rightBitWidth); - - short[] leftCodes = new short[n]; - long[] rightParts = new long[n]; - List excPos = new ArrayList<>(); - List excVals = new ArrayList<>(); - - for (int i = 0; i < n; i++) { - long bits = Double.doubleToRawLongBits(values[i]); - short leftU16 = (short) (bits >>> best.rightBitWidth); - rightParts[i] = bits & rightMask; - Short code = lookup.get(leftU16); - if (code != null) { - leftCodes[i] = code; - } else { - leftCodes[i] = 0; - excPos.add((long) i); - excVals.add(leftU16); - } - } - - return buildEncodeResult( - best.dict, best.rightBitWidth, leftCodes, rightParts, - U64_DTYPE, excPos, excVals); - } - - private record Dictionary64(short[] dict, int rightBitWidth) {} - - private static Dictionary64 findBestDictionaryF64(double[] values, int sampleLen) { - double bestEstSize = Double.MAX_VALUE; - int bestRightBw = 48; - short[] bestDict = new short[]{0}; - - for (int p = 1; p <= MAX_CUT; p++) { - int rightBw = 64 - p; - long mask = -1L >>> p; - Map counts = new HashMap<>(); - for (int i = 0; i < sampleLen; i++) { - long bits = Double.doubleToRawLongBits(values[i]); - short leftU16 = (short) (bits >>> rightBw); - counts.merge(leftU16, 1, Integer::sum); - } - short[] dict = topKByCount(counts, MAX_DICT_SIZE); - int excCount = countExceptions(values, sampleLen, dict, rightBw, mask); - int maxCode = dict.length - 1; - int leftBw = maxCode == 0 ? 1 : (Integer.SIZE - Integer.numberOfLeadingZeros(maxCode)); - double estSize = rightBw + leftBw + (double) (excCount * 32) / sampleLen; - if (estSize < bestEstSize) { - bestEstSize = estSize; - bestRightBw = rightBw; - bestDict = dict; - } - } - return new Dictionary64(bestDict, bestRightBw); - } - - private static int countExceptions(double[] values, int sampleLen, short[] dict, int rightBw, long mask) { - Map dictSet = new HashMap<>(); - for (short d : dict) { - dictSet.put(d, Boolean.TRUE); - } - int count = 0; - for (int i = 0; i < sampleLen; i++) { - long bits = Double.doubleToRawLongBits(values[i]); - short leftU16 = (short) (bits >>> rightBw); - if (!dictSet.containsKey(leftU16)) { - count++; - } - } - return count; - } - - // --- F32 --- - - private static EncodeResult encodeF32(float[] values) { - int n = values.length; - if (n == 0) { - return emptyResult(U32_DTYPE, U16_DTYPE); - } - - int sampleLen = Math.min(SAMPLE_SIZE, n); - Dictionary32 best = findBestDictionaryF32(values, sampleLen); - - Map lookup = buildLookup(best.dict); - int rightMask = -1 >>> (32 - best.rightBitWidth); - - short[] leftCodes = new short[n]; - int[] rightParts = new int[n]; - List excPos = new ArrayList<>(); - List excVals = new ArrayList<>(); - - for (int i = 0; i < n; i++) { - int bits = Float.floatToRawIntBits(values[i]); - short leftU16 = (short) (bits >>> best.rightBitWidth); - rightParts[i] = bits & rightMask; - Short code = lookup.get(leftU16); - if (code != null) { - leftCodes[i] = code; - } else { - leftCodes[i] = 0; - excPos.add((long) i); - excVals.add(leftU16); - } - } - - return buildEncodeResult( - best.dict, best.rightBitWidth, leftCodes, rightParts, - U32_DTYPE, excPos, excVals); - } - - private record Dictionary32(short[] dict, int rightBitWidth) {} - - private static Dictionary32 findBestDictionaryF32(float[] values, int sampleLen) { - double bestEstSize = Double.MAX_VALUE; - int bestRightBw = 16; - short[] bestDict = new short[]{0}; - - for (int p = 1; p <= MAX_CUT; p++) { - int rightBw = 32 - p; - int mask = -1 >>> p; - Map counts = new HashMap<>(); - for (int i = 0; i < sampleLen; i++) { - int bits = Float.floatToRawIntBits(values[i]); - short leftU16 = (short) (bits >>> rightBw); - counts.merge(leftU16, 1, Integer::sum); - } - short[] dict = topKByCount(counts, MAX_DICT_SIZE); - int excCount = countExceptionsF32(values, sampleLen, dict, rightBw, mask); - int maxCode = dict.length - 1; - int leftBw = maxCode == 0 ? 1 : (Integer.SIZE - Integer.numberOfLeadingZeros(maxCode)); - double estSize = rightBw + leftBw + (double) (excCount * 32) / sampleLen; - if (estSize < bestEstSize) { - bestEstSize = estSize; - bestRightBw = rightBw; - bestDict = dict; - } - } - return new Dictionary32(bestDict, bestRightBw); - } - - private static int countExceptionsF32(float[] values, int sampleLen, short[] dict, int rightBw, int mask) { - Map dictSet = new HashMap<>(); - for (short d : dict) { - dictSet.put(d, Boolean.TRUE); - } - int count = 0; - for (int i = 0; i < sampleLen; i++) { - int bits = Float.floatToRawIntBits(values[i]); - short leftU16 = (short) (bits >>> rightBw); - if (!dictSet.containsKey(leftU16)) { - count++; - } - } - return count; - } - - // --- shared helpers --- - - private static short[] topKByCount(Map counts, int k) { - List> sorted = new ArrayList<>(counts.entrySet()); - sorted.sort((a, b) -> b.getValue() - a.getValue()); - int dictSize = Math.min(sorted.size(), k); - short[] dict = new short[dictSize]; - for (int i = 0; i < dictSize; i++) { - dict[i] = sorted.get(i).getKey(); - } - return dict; - } - - private static Map buildLookup(short[] dict) { - Map lookup = new HashMap<>(); - for (short i = 0; i < dict.length; i++) { - lookup.put(dict[i], i); - } - return lookup; - } - - private static EncodeResult buildEncodeResult( - short[] dict, int rightBitWidth, - short[] leftCodes, Object rightPartsData, DType rightDtype, - List excPos, List excVals) { - - BitpackedEncoding bp = new BitpackedEncoding(); - EncodeResult leftResult = bp.encode(U16_DTYPE, leftCodes); - EncodeResult rightResult = bp.encode(rightDtype, rightPartsData); - - List allBuffers = new ArrayList<>(leftResult.buffers()); - int leftBufCount = allBuffers.size(); - allBuffers.addAll(rightResult.buffers()); - int rightBufCount = rightResult.buffers().size(); - - EncodeNode leftNode = EncodeNode.remapBufferIndices(leftResult.rootNode(), 0); - EncodeNode rightNode = EncodeNode.remapBufferIndices(rightResult.rootNode(), leftBufCount); - - EncodingProtos.ALPRDMetadata.Builder metaBuilder = EncodingProtos.ALPRDMetadata.newBuilder() - .setRightBitWidth(rightBitWidth) - .setDictLen(dict.length) - .setLeftPartsPtype(DTypeProtos.PType.forNumber(PType.U16.ordinal())); - for (short d : dict) { - metaBuilder.addDict(d & 0xFFFF); - } - - EncodeNode[] children; - if (excPos.isEmpty()) { - children = new EncodeNode[]{leftNode, rightNode}; - } else { - long[] excPosArr = excPos.stream().mapToLong(Long::longValue).toArray(); - short[] excValsArr = new short[excVals.size()]; - for (int i = 0; i < excVals.size(); i++) { - excValsArr[i] = excVals.get(i); - } - - EncodeResult idxResult = bp.encode(U64_DTYPE, excPosArr); - EncodeResult valResult = bp.encode(U16_DTYPE, excValsArr); - - int idxOffset = allBuffers.size(); - allBuffers.addAll(idxResult.buffers()); - int idxBufCount = idxResult.buffers().size(); - allBuffers.addAll(valResult.buffers()); - - EncodeNode idxNode = EncodeNode.remapBufferIndices(idxResult.rootNode(), idxOffset); - EncodeNode valNode = EncodeNode.remapBufferIndices(valResult.rootNode(), idxOffset + idxBufCount); - - EncodingProtos.PatchesMetadata patches = EncodingProtos.PatchesMetadata.newBuilder() - .setLen(excPos.size()) - .setOffset(0) - .setIndicesPtype(DTypeProtos.PType.forNumber(PType.U64.ordinal())) - .build(); - metaBuilder.setPatches(patches); - children = new EncodeNode[]{leftNode, rightNode, idxNode, valNode}; - } - - byte[] metaBytes = metaBuilder.build().toByteArray(); - EncodeNode root = new EncodeNode( - EncodingId.VORTEX_ALPRD, ByteBuffer.wrap(metaBytes), children, new int[]{}); - return new EncodeResult(root, List.copyOf(allBuffers), null, null); - } - - private static EncodeResult emptyResult(DType rightDtype, DType leftDtype) { - BitpackedEncoding bp = new BitpackedEncoding(); - EncodeResult leftResult = bp.encode(leftDtype, new short[0]); - EncodeResult rightResult = bp.encode(rightDtype, - rightDtype.equals(U32_DTYPE) ? new int[0] : new long[0]); - - List allBuffers = new ArrayList<>(leftResult.buffers()); - int leftBufCount = allBuffers.size(); - allBuffers.addAll(rightResult.buffers()); - - EncodeNode leftNode = EncodeNode.remapBufferIndices(leftResult.rootNode(), 0); - EncodeNode rightNode = EncodeNode.remapBufferIndices(rightResult.rootNode(), leftBufCount); - - byte[] metaBytes = EncodingProtos.ALPRDMetadata.newBuilder() - .setRightBitWidth(48) - .setDictLen(0) - .setLeftPartsPtype(DTypeProtos.PType.forNumber(PType.U16.ordinal())) - .build().toByteArray(); - - EncodeNode root = new EncodeNode( - EncodingId.VORTEX_ALPRD, ByteBuffer.wrap(metaBytes), - new EncodeNode[]{leftNode, rightNode}, new int[]{}); - return new EncodeResult(root, List.copyOf(allBuffers), null, null); - } - } - - // ------------------------------------------------------------------------- - // Decoder - // ------------------------------------------------------------------------- - - private static final class Decoder { - - static Array decode(DecodeContext ctx) { - EncodingProtos.ALPRDMetadata meta = parseMeta(ctx); - - if (!(ctx.dtype() instanceof DType.Primitive p)) { - throw new VortexException(EncodingId.VORTEX_ALPRD, - "expected primitive dtype, got " + ctx.dtype()); - } - - int rightBitWidth = meta.getRightBitWidth(); - int dictLen = meta.getDictLen(); - short[] dict = new short[dictLen]; - for (int i = 0; i < dictLen; i++) { - dict[i] = (short) (meta.getDict(i) & 0xFFFF); - } - - long n = ctx.rowCount(); - PType ptype = p.ptype(); - - return switch (ptype) { - case F64 -> decodeF64(ctx, meta, dict, rightBitWidth, n); - case F32 -> decodeF32(ctx, meta, dict, rightBitWidth, n); - default -> throw new VortexException(EncodingId.VORTEX_ALPRD, - "unsupported dtype " + ptype); - }; - } - - private static Array decodeF64(DecodeContext ctx, - EncodingProtos.ALPRDMetadata meta, - short[] dict, int rightBitWidth, long n) { - Array leftParts = decodeChildAs(ctx, 0, U16_DTYPE, n); - Array rightParts = decodeChildAs(ctx, 1, U64_DTYPE, n); - - MemorySegment leftSeg = leftParts.buffer(0); - MemorySegment rightSeg = rightParts.buffer(0); - MemorySegment out = ctx.arena().allocate(n * Long.BYTES, Long.BYTES); - - for (long i = 0; i < n; i++) { - int code = Short.toUnsignedInt(leftSeg.getAtIndex(PTypeIO.LE_SHORT, i)); - long leftBits = (long) (dict[code] & 0xFFFF) << rightBitWidth; - long rightBits = rightSeg.getAtIndex(PTypeIO.LE_LONG, i); - out.setAtIndex(PTypeIO.LE_LONG, i, leftBits | rightBits); - } - - if (meta.hasPatches()) { - applyPatchesF64(ctx, meta.getPatches(), out, rightSeg, dict, rightBitWidth); - } - - return new DoubleArray(ctx.dtype(), n, out.asReadOnly(), ArrayStats.empty()); - } - - private static Array decodeF32(DecodeContext ctx, - EncodingProtos.ALPRDMetadata meta, - short[] dict, int rightBitWidth, long n) { - Array leftParts = decodeChildAs(ctx, 0, U16_DTYPE, n); - Array rightParts = decodeChildAs(ctx, 1, U32_DTYPE, n); - - MemorySegment leftSeg = leftParts.buffer(0); - MemorySegment rightSeg = rightParts.buffer(0); - MemorySegment out = ctx.arena().allocate(n * Integer.BYTES, Integer.BYTES); - - for (long i = 0; i < n; i++) { - int code = Short.toUnsignedInt(leftSeg.getAtIndex(PTypeIO.LE_SHORT, i)); - int leftBits = (dict[code] & 0xFFFF) << rightBitWidth; - int rightBits = rightSeg.getAtIndex(PTypeIO.LE_INT, i); - out.setAtIndex(PTypeIO.LE_INT, i, leftBits | rightBits); - } - - if (meta.hasPatches()) { - applyPatchesF32(ctx, meta.getPatches(), out, rightSeg, dict, rightBitWidth); - } - - return new FloatArray(ctx.dtype(), n, out.asReadOnly(), ArrayStats.empty()); - } - - private static void applyPatchesF64(DecodeContext ctx, - EncodingProtos.PatchesMetadata pm, - MemorySegment out, MemorySegment rightSeg, - short[] dict, int rightBitWidth) { - long numPatches = pm.getLen(); - long offset = pm.getOffset(); - PType idxPtype = PType.values()[pm.getIndicesPtype().getNumber()]; - - Array idxArr = decodeChildAs(ctx, 2, new DType.Primitive(idxPtype, false), numPatches); - Array valArr = decodeChildAs(ctx, 3, U16_DTYPE, numPatches); - - MemorySegment idxSeg = idxArr.buffer(0); - MemorySegment valSeg = valArr.buffer(0); - - for (long j = 0; j < numPatches; j++) { - long absIdx = readUnsigned(idxSeg, j, idxPtype) - offset; - short actualLeftU16 = valSeg.getAtIndex(PTypeIO.LE_SHORT, j); - long leftBits = (long) (actualLeftU16 & 0xFFFF) << rightBitWidth; - long rightBits = rightSeg.getAtIndex(PTypeIO.LE_LONG, absIdx); - out.setAtIndex(PTypeIO.LE_LONG, absIdx, leftBits | rightBits); - } - } - - private static void applyPatchesF32(DecodeContext ctx, - EncodingProtos.PatchesMetadata pm, - MemorySegment out, MemorySegment rightSeg, - short[] dict, int rightBitWidth) { - long numPatches = pm.getLen(); - long offset = pm.getOffset(); - PType idxPtype = PType.values()[pm.getIndicesPtype().getNumber()]; - - Array idxArr = decodeChildAs(ctx, 2, new DType.Primitive(idxPtype, false), numPatches); - Array valArr = decodeChildAs(ctx, 3, U16_DTYPE, numPatches); - - MemorySegment idxSeg = idxArr.buffer(0); - MemorySegment valSeg = valArr.buffer(0); - - for (long j = 0; j < numPatches; j++) { - long absIdx = readUnsigned(idxSeg, j, idxPtype) - offset; - short actualLeftU16 = valSeg.getAtIndex(PTypeIO.LE_SHORT, j); - int leftBits = (actualLeftU16 & 0xFFFF) << rightBitWidth; - int rightBits = rightSeg.getAtIndex(PTypeIO.LE_INT, absIdx); - out.setAtIndex(PTypeIO.LE_INT, (int) absIdx, leftBits | rightBits); - } - } - - private static Array decodeChildAs(DecodeContext parent, int childIdx, DType dtype, long rowCount) { - ArrayNode childNode = parent.node().children()[childIdx]; - DecodeContext childCtx = new DecodeContext( - childNode, dtype, rowCount, parent.segmentBuffers(), parent.registry(), parent.arena()); - return parent.registry().decode(childCtx); - } - - private static long readUnsigned(MemorySegment seg, long i, PType ptype) { - return switch (ptype) { - case U8 -> Byte.toUnsignedLong(seg.get(java.lang.foreign.ValueLayout.JAVA_BYTE, i)); - case U16 -> Short.toUnsignedLong(seg.get(PTypeIO.LE_SHORT, i * 2)); - case U32 -> Integer.toUnsignedLong(seg.get(PTypeIO.LE_INT, i * 4)); - case U64 -> seg.get(PTypeIO.LE_LONG, i * 8); - default -> throw new VortexException(EncodingId.VORTEX_ALPRD, - "non-unsigned patch index ptype " + ptype); - }; - } - - private static EncodingProtos.ALPRDMetadata parseMeta(DecodeContext ctx) { - ByteBuffer rawMeta = ctx.metadata(); - if (rawMeta == null || !rawMeta.hasRemaining()) { - return EncodingProtos.ALPRDMetadata.getDefaultInstance(); - } - try { - return EncodingProtos.ALPRDMetadata.parseFrom(rawMeta.duplicate()); - } catch (InvalidProtocolBufferException e) { - throw new VortexException(EncodingId.VORTEX_ALPRD, "invalid metadata", e); - } - } - } -} diff --git a/core/src/main/java/io/github/dfa1/vortex/encoding/ArrayNode.java b/core/src/main/java/io/github/dfa1/vortex/encoding/ArrayNode.java deleted file mode 100644 index 7a3560f..0000000 --- a/core/src/main/java/io/github/dfa1/vortex/encoding/ArrayNode.java +++ /dev/null @@ -1,29 +0,0 @@ -package io.github.dfa1.vortex.encoding; - -import io.github.dfa1.vortex.core.ArrayStats; - -import java.nio.ByteBuffer; - -/// Encoded array node as stored in a Flat layout segment. -/// In-file representation before decoding; mirrors the Go ArrayNode struct. -/// -/// Sealed: a node is either [KnownArrayNode] (id resolves to an [EncodingId]) or -/// [UnknownArrayNode] (id is an arbitrary string only meaningful for -/// [EncodingRegistry#allowUnknown()] passthrough decode). -sealed interface ArrayNode permits KnownArrayNode, UnknownArrayNode { - ByteBuffer metadata(); - - ArrayNode[] children(); - - int[] bufferIndices(); - - ArrayStats stats(); - - /// Short factory for the common case: a node whose encoding id is well-known. - /// Mostly used by tests and helper code that converts an [EncodeNode] tree back into - /// an [ArrayNode] tree. - static ArrayNode of(EncodingId encodingId, ByteBuffer metadata, ArrayNode[] children, - int[] bufferIndices, ArrayStats stats) { - return new KnownArrayNode(encodingId, metadata, children, bufferIndices, stats); - } -} diff --git a/core/src/main/java/io/github/dfa1/vortex/encoding/BitpackedEncoding.java b/core/src/main/java/io/github/dfa1/vortex/encoding/BitpackedEncoding.java deleted file mode 100644 index 22b2dde..0000000 --- a/core/src/main/java/io/github/dfa1/vortex/encoding/BitpackedEncoding.java +++ /dev/null @@ -1,619 +0,0 @@ -package io.github.dfa1.vortex.encoding; - -import com.google.protobuf.InvalidProtocolBufferException; -import io.github.dfa1.vortex.proto.DTypeProtos; -import io.github.dfa1.vortex.proto.EncodingProtos; -import io.github.dfa1.vortex.proto.ScalarProtos; -import io.github.dfa1.vortex.core.array.Array; -import io.github.dfa1.vortex.core.ArrayStats; -import io.github.dfa1.vortex.core.DType; -import io.github.dfa1.vortex.core.PType; -import io.github.dfa1.vortex.core.array.ByteArray; -import io.github.dfa1.vortex.core.array.IntArray; -import io.github.dfa1.vortex.core.array.LongArray; -import io.github.dfa1.vortex.core.array.ShortArray; -import io.github.dfa1.vortex.core.VortexException; - -import java.lang.foreign.Arena; -import java.lang.foreign.MemorySegment; -import java.lang.foreign.ValueLayout; -import java.nio.ByteBuffer; -import java.util.List; - -/// Encoding for {@code fastlanes.bitpacked} — spec-compliant FastLanes bit-packing. -/// -///

Metadata: protobuf {@code BitPackedMetadata} — {@code bit_width u32} (tag 1), -/// {@code offset u32} (tag 2, element offset within the first 1024-element block). -/// -///

Buffer layout: {@code ceil((len + offset) / 1024)} blocks, each block {@code 128 * bit_width} -/// bytes. Within each block the values are transposed using the FastLanes FL_ORDER permutation so -/// that adjacent bit-planes are contiguous (enables SIMD-friendly decompression). -public final class BitpackedEncoding implements Encoding { - - // FL_ORDER permutation from the FastLanes paper / spiraldb/fastlanes-rs. - static final int[] FL_ORDER = {0, 4, 2, 6, 1, 5, 3, 7}; - - /// Creates a new {@code BitpackedEncoding} instance; use via {@link EncodingRegistry}. - public BitpackedEncoding() { - } - - @Override - public EncodingId encodingId() { - return EncodingId.FASTLANES_BITPACKED; - } - - @Override - public boolean accepts(DType dtype) { - if (!(dtype instanceof DType.Primitive p)) { - return false; - } - return switch (p.ptype()) { - case I8, I16, I32, I64, U8, U16, U32, U64 -> true; - default -> false; - }; - } - - @Override - public EncodeResult encode(DType dtype, Object data) { - return Encoder.encode(dtype, data); - } - - @Override - public Array decode(DecodeContext ctx) { - return Decoder.decode(ctx); - } - - private static final class Encoder { - - static EncodeResult encode(DType dtype, Object data) { - PType ptype = ((DType.Primitive) dtype).ptype(); - long[] longs = toLongs(data, ptype); - int n = longs.length; - int typeBits = ptype.byteSize() * 8; - long typeMask = typeMask(typeBits); - boolean unsign = isUnsigned(ptype); - - long signedMin = 0L; - long signedMax = 0L; - long maxUnsigned = 0L; - int bitWidth = 0; - - if (n > 0) { - signedMin = longs[0]; - signedMax = longs[0]; - for (long v : longs) { - if (unsign ? Long.compareUnsigned(v, signedMin) < 0 : v < signedMin) { - signedMin = v; - } - if (unsign ? Long.compareUnsigned(v, signedMax) > 0 : v > signedMax) { - signedMax = v; - } - long uv = v & typeMask; - if (Long.compareUnsigned(uv, maxUnsigned) > 0) { - maxUnsigned = uv; - } - } - bitWidth = maxUnsigned == 0L ? 0 : (Long.SIZE - Long.numberOfLeadingZeros(maxUnsigned)); - } - - MemorySegment packed = packFastLanes(longs, n, bitWidth, typeBits); - - byte[] metaBytes = EncodingProtos.BitPackedMetadata.newBuilder() - .setBitWidth(bitWidth) - .setOffset(0) - .build() - .toByteArray(); - - byte[] statsMin = n > 0 ? statsBytes(ptype, signedMin) : null; - byte[] statsMax = n > 0 ? statsBytes(ptype, signedMax) : null; - - EncodeNode root = new EncodeNode(EncodingId.FASTLANES_BITPACKED, ByteBuffer.wrap(metaBytes), - new EncodeNode[0], new int[]{0}); - return new EncodeResult(root, List.of(packed), statsMin, statsMax); - } - - private static MemorySegment packFastLanes(long[] values, int n, int bitWidth, int typeBits) { - if (bitWidth == 0 || n == 0) { - return MemorySegment.ofArray(new byte[0]); - } - int lanes = 1024 / typeBits; - int wordBytes = typeBits / 8; - int blockCount = (n + 1023) / 1024; - long typeMask = typeMask(typeBits); - MemorySegment seg = Arena.ofAuto().allocate((long) blockCount * 128 * bitWidth); - - for (int block = 0; block < blockCount; block++) { - int blockByteOff = block * 128 * bitWidth; - int blockStart = block * 1024; - - for (int row = 0; row < typeBits; row++) { - int currWord = (row * bitWidth) / typeBits; - int nextWord = ((row + 1) * bitWidth) / typeBits; - int shift = (row * bitWidth) % typeBits; - int remainingBits = (nextWord > currWord) ? ((row + 1) * bitWidth) % typeBits : 0; - int currentBits = bitWidth - remainingBits; - - for (int lane = 0; lane < lanes; lane++) { - int o = row / 8; - int s = row % 8; - int logicalIdx = blockStart + FL_ORDER[o] * 16 + s * 128 + lane; - long value = (logicalIdx < n) ? (values[logicalIdx] & typeMask) : 0L; - - int wordOff = blockByteOff + (lanes * currWord + lane) * wordBytes; - long existing = readWordFromSeg(seg, wordOff, typeBits); - existing |= (value << shift) & typeMask; - writeWordToSeg(seg, wordOff, existing, typeBits); - - if (remainingBits > 0) { - int hiWordOff = blockByteOff + (lanes * nextWord + lane) * wordBytes; - long existingHi = readWordFromSeg(seg, hiWordOff, typeBits); - existingHi |= (value >>> currentBits) & typeMask; - writeWordToSeg(seg, hiWordOff, existingHi, typeBits); - } - } - } - } - return seg; - } - - private static long[] toLongs(Object data, PType ptype) { - return switch (ptype) { - case I8 -> { - byte[] arr = (byte[]) data; - long[] r = new long[arr.length]; - for (int i = 0; i < arr.length; i++) { - r[i] = arr[i]; - } - yield r; - } - case U8 -> { - byte[] arr = (byte[]) data; - long[] r = new long[arr.length]; - for (int i = 0; i < arr.length; i++) { - r[i] = Byte.toUnsignedLong(arr[i]); - } - yield r; - } - case I16 -> { - short[] arr = (short[]) data; - long[] r = new long[arr.length]; - for (int i = 0; i < arr.length; i++) { - r[i] = arr[i]; - } - yield r; - } - case U16 -> { - short[] arr = (short[]) data; - long[] r = new long[arr.length]; - for (int i = 0; i < arr.length; i++) { - r[i] = Short.toUnsignedLong(arr[i]); - } - yield r; - } - case I32 -> { - int[] arr = (int[]) data; - long[] r = new long[arr.length]; - for (int i = 0; i < arr.length; i++) { - r[i] = arr[i]; - } - yield r; - } - case U32 -> { - int[] arr = (int[]) data; - long[] r = new long[arr.length]; - for (int i = 0; i < arr.length; i++) { - r[i] = Integer.toUnsignedLong(arr[i]); - } - yield r; - } - case I64, U64 -> (long[]) data; - default -> throw new VortexException(EncodingId.FASTLANES_BITPACKED, "unsupported ptype: " + ptype); - }; - } - - private static long typeMask(int typeBits) { - return typeBits == 64 ? -1L : (1L << typeBits) - 1L; - } - - private static boolean isUnsigned(PType ptype) { - return switch (ptype) { - case U8, U16, U32, U64 -> true; - default -> false; - }; - } - - private static byte[] statsBytes(PType ptype, long value) { - if (isUnsigned(ptype)) { - return ScalarProtos.ScalarValue.newBuilder().setUint64Value(value).build().toByteArray(); - } - return ScalarProtos.ScalarValue.newBuilder().setInt64Value(value).build().toByteArray(); - } - - private static long readWordFromSeg(MemorySegment seg, int off, int typeBits) { - return switch (typeBits) { - case 8 -> Byte.toUnsignedLong(seg.get(ValueLayout.JAVA_BYTE, off)); - case 16 -> Short.toUnsignedLong(seg.get(PTypeIO.LE_SHORT, off)); - case 32 -> Integer.toUnsignedLong(seg.get(PTypeIO.LE_INT, off)); - case 64 -> seg.get(PTypeIO.LE_LONG, off); - default -> throw new VortexException(EncodingId.FASTLANES_BITPACKED, "unsupported typeBits: " + typeBits); - }; - } - - private static void writeWordToSeg(MemorySegment seg, int off, long value, int typeBits) { - switch (typeBits) { - case 8 -> seg.set(ValueLayout.JAVA_BYTE, off, (byte) value); - case 16 -> seg.set(PTypeIO.LE_SHORT, off, (short) value); - case 32 -> seg.set(PTypeIO.LE_INT, off, (int) value); - case 64 -> seg.set(PTypeIO.LE_LONG, off, value); - default -> throw new VortexException(EncodingId.FASTLANES_BITPACKED, "unsupported typeBits: " + typeBits); - } - } - } - - private static final class Decoder { - - static Array decode(DecodeContext ctx) { - ByteBuffer rawMeta = ctx.metadata(); - if (rawMeta == null) { - throw new VortexException(EncodingId.FASTLANES_BITPACKED, "missing metadata"); - } - - EncodingProtos.BitPackedMetadata meta; - try { - meta = EncodingProtos.BitPackedMetadata.parseFrom(rawMeta.duplicate()); - } catch (InvalidProtocolBufferException e) { - throw new VortexException(EncodingId.FASTLANES_BITPACKED, "invalid metadata", e); - } - - int bitWidth = meta.getBitWidth(); - int offset = meta.getOffset(); - PType ptype = ((DType.Primitive) ctx.dtype()).ptype(); - int typeBits = ptype.byteSize() * 8; - long rowCount = ctx.rowCount(); - - MemorySegment packed = ctx.buffer(0); - MemorySegment output = ctx.arena().allocate(rowCount * ptype.byteSize()); - fastlanesUnpackToSeg(packed, bitWidth, offset, typeBits, rowCount, output); - - if (meta.hasPatches()) { - applyPatches(ctx, meta.getPatches(), output, ptype.byteSize()); - } - - return switch (ptype) { - case I64, U64 -> new LongArray(ctx.dtype(), rowCount, output, ArrayStats.empty()); - case I32, U32 -> new IntArray(ctx.dtype(), rowCount, output, ArrayStats.empty()); - case I16, U16 -> new ShortArray(ctx.dtype(), rowCount, output, ArrayStats.empty()); - case I8, U8 -> new ByteArray(ctx.dtype(), rowCount, output, ArrayStats.empty()); - default -> throw new VortexException(EncodingId.FASTLANES_BITPACKED, "unsupported ptype " + ptype); - }; - } - - private static void fastlanesUnpackToSeg( - MemorySegment buf, int bitWidth, int offset, int typeBits, long rowCount, - MemorySegment output) { - if (bitWidth == 0) { - return; - } - switch (typeBits) { - case 8 -> unpackLoop8(buf, bitWidth, offset, rowCount, output); - case 16 -> unpackLoop16(buf, bitWidth, offset, rowCount, output); - case 32 -> unpackLoop32(buf, bitWidth, offset, rowCount, output); - case 64 -> unpackLoop64(buf, bitWidth, offset, rowCount, output); - default -> throw new VortexException(EncodingId.FASTLANES_BITPACKED, "unsupported typeBits: " + typeBits); - } - } - - private static void unpackLoop8(MemorySegment buf, int bitWidth, int offset, long rowCount, MemorySegment out) { - final int lanes = 128; - long totalElems = rowCount + offset; - int blockCount = (int) ((totalElems + 1023) / 1024); - long bitMask = (1L << bitWidth) - 1L; - - long blockByteOff = 0L; - long blockByteStride = 128L * bitWidth; - for (int block = 0; block < blockCount; block++, blockByteOff += blockByteStride) { - int blockLogicStart = block * 1024 - offset; - boolean fullBlock = blockLogicStart >= 0 && (long) blockLogicStart + 1023L < rowCount; - - for (int row = 0; row < 8; row++) { - int currWord = (row * bitWidth) / 8; - int nextWord = ((row + 1) * bitWidth) / 8; - int shift = (row * bitWidth) % 8; - int remainingBits = (nextWord > currWord) ? ((row + 1) * bitWidth) % 8 : 0; - int currentBits = bitWidth - remainingBits; - int o = row / 8; - int s = row % 8; - int baseIdx = blockLogicStart + FL_ORDER[o] * 16 + s * 128; - long wordBase = blockByteOff + (long) lanes * currWord; - - if (fullBlock) { - if (remainingBits > 0) { - long hiBase = blockByteOff + (long) lanes * nextWord; - long loMask = (1L << currentBits) - 1L; - long hiMask = (1L << remainingBits) - 1L; - for (int lane = 0; lane < lanes; lane++) { - long lo = (Byte.toUnsignedLong(buf.get(ValueLayout.JAVA_BYTE, wordBase + lane)) >>> shift) & loMask; - long hi = Byte.toUnsignedLong(buf.get(ValueLayout.JAVA_BYTE, hiBase + lane)) & hiMask; - out.set(ValueLayout.JAVA_BYTE, baseIdx + lane, (byte) (lo | (hi << currentBits))); - } - } else { - for (int lane = 0; lane < lanes; lane++) { - out.set(ValueLayout.JAVA_BYTE, baseIdx + lane, - (byte) ((Byte.toUnsignedLong(buf.get(ValueLayout.JAVA_BYTE, wordBase + lane)) >>> shift) & bitMask)); - } - } - } else { - long hiBase = (remainingBits > 0) ? blockByteOff + (long) lanes * nextWord : 0L; - long loMask = (remainingBits > 0) ? (1L << currentBits) - 1L : 0L; - long hiMask = (remainingBits > 0) ? (1L << remainingBits) - 1L : 0L; - for (int lane = 0; lane < lanes; lane++) { - int logicalIdx = baseIdx + lane; - if (logicalIdx < 0 || logicalIdx >= rowCount) { - continue; - } - long src = Byte.toUnsignedLong(buf.get(ValueLayout.JAVA_BYTE, wordBase + lane)); - long value; - if (remainingBits > 0) { - long lo = (src >>> shift) & loMask; - long hi = Byte.toUnsignedLong(buf.get(ValueLayout.JAVA_BYTE, hiBase + lane)) & hiMask; - value = lo | (hi << currentBits); - } else { - value = (src >>> shift) & bitMask; - } - out.set(ValueLayout.JAVA_BYTE, logicalIdx, (byte) value); - } - } - } - } - } - - private static void unpackLoop16(MemorySegment buf, int bitWidth, int offset, long rowCount, MemorySegment out) { - final int lanes = 64; - long totalElems = rowCount + offset; - int blockCount = (int) ((totalElems + 1023) / 1024); - long bitMask = (1L << bitWidth) - 1L; - - long blockByteOff = 0L; - long blockByteStride = 128L * bitWidth; - for (int block = 0; block < blockCount; block++, blockByteOff += blockByteStride) { - int blockLogicStart = block * 1024 - offset; - boolean fullBlock = blockLogicStart >= 0 && (long) blockLogicStart + 1023L < rowCount; - - for (int row = 0; row < 16; row++) { - int currWord = (row * bitWidth) / 16; - int nextWord = ((row + 1) * bitWidth) / 16; - int shift = (row * bitWidth) % 16; - int remainingBits = (nextWord > currWord) ? ((row + 1) * bitWidth) % 16 : 0; - int currentBits = bitWidth - remainingBits; - int o = row / 8; - int s = row % 8; - int baseIdx = blockLogicStart + FL_ORDER[o] * 16 + s * 128; - long wordBase = blockByteOff + (long) lanes * currWord * 2; - - if (fullBlock) { - long outBase = (long) baseIdx * 2; - if (remainingBits > 0) { - long hiBase = blockByteOff + (long) lanes * nextWord * 2; - long loMask = (1L << currentBits) - 1L; - long hiMask = (1L << remainingBits) - 1L; - long laneOff = 0L; - for (int lane = 0; lane < lanes; lane++, laneOff += 2L) { - long lo = (Short.toUnsignedLong(buf.get(PTypeIO.LE_SHORT, wordBase + laneOff)) >>> shift) & loMask; - long hi = Short.toUnsignedLong(buf.get(PTypeIO.LE_SHORT, hiBase + laneOff)) & hiMask; - out.set(PTypeIO.LE_SHORT, outBase + laneOff, (short) (lo | (hi << currentBits))); - } - } else { - long laneOff = 0L; - for (int lane = 0; lane < lanes; lane++, laneOff += 2L) { - out.set(PTypeIO.LE_SHORT, outBase + laneOff, - (short) ((Short.toUnsignedLong(buf.get(PTypeIO.LE_SHORT, wordBase + laneOff)) >>> shift) & bitMask)); - } - } - } else { - long hiBase = (remainingBits > 0) ? blockByteOff + (long) lanes * nextWord * 2 : 0L; - long loMask = (remainingBits > 0) ? (1L << currentBits) - 1L : 0L; - long hiMask = (remainingBits > 0) ? (1L << remainingBits) - 1L : 0L; - for (int lane = 0; lane < lanes; lane++) { - int logicalIdx = baseIdx + lane; - if (logicalIdx < 0 || logicalIdx >= rowCount) { - continue; - } - long src = Short.toUnsignedLong(buf.get(PTypeIO.LE_SHORT, wordBase + (long) lane * 2)); - long value; - if (remainingBits > 0) { - long lo = (src >>> shift) & loMask; - long hi = Short.toUnsignedLong(buf.get(PTypeIO.LE_SHORT, hiBase + (long) lane * 2)) & hiMask; - value = lo | (hi << currentBits); - } else { - value = (src >>> shift) & bitMask; - } - out.set(PTypeIO.LE_SHORT, (long) logicalIdx * 2, (short) value); - } - } - } - } - } - - private static void unpackLoop32(MemorySegment buf, int bitWidth, int offset, long rowCount, MemorySegment out) { - final int lanes = 32; - long totalElems = rowCount + offset; - int blockCount = (int) ((totalElems + 1023) / 1024); - long bitMask = (1L << bitWidth) - 1L; - - long blockByteOff = 0L; - long blockByteStride = 128L * bitWidth; - for (int block = 0; block < blockCount; block++, blockByteOff += blockByteStride) { - int blockLogicStart = block * 1024 - offset; - boolean fullBlock = blockLogicStart >= 0 && (long) blockLogicStart + 1023L < rowCount; - - for (int row = 0; row < 32; row++) { - int currWord = (row * bitWidth) / 32; - int nextWord = ((row + 1) * bitWidth) / 32; - int shift = (row * bitWidth) % 32; - int remainingBits = (nextWord > currWord) ? ((row + 1) * bitWidth) % 32 : 0; - int currentBits = bitWidth - remainingBits; - int o = row / 8; - int s = row % 8; - int baseIdx = blockLogicStart + FL_ORDER[o] * 16 + s * 128; - long wordBase = blockByteOff + (long) lanes * currWord * 4; - - if (fullBlock) { - long outBase = (long) baseIdx * 4; - if (remainingBits > 0) { - long hiBase = blockByteOff + (long) lanes * nextWord * 4; - long loMask = (1L << currentBits) - 1L; - long hiMask = (1L << remainingBits) - 1L; - long laneOff = 0L; - for (int lane = 0; lane < lanes; lane++, laneOff += 4L) { - long lo = (Integer.toUnsignedLong(buf.get(PTypeIO.LE_INT, wordBase + laneOff)) >>> shift) & loMask; - long hi = Integer.toUnsignedLong(buf.get(PTypeIO.LE_INT, hiBase + laneOff)) & hiMask; - out.set(PTypeIO.LE_INT, outBase + laneOff, (int) (lo | (hi << currentBits))); - } - } else { - long laneOff = 0L; - for (int lane = 0; lane < lanes; lane++, laneOff += 4L) { - out.set(PTypeIO.LE_INT, outBase + laneOff, - (int) ((Integer.toUnsignedLong(buf.get(PTypeIO.LE_INT, wordBase + laneOff)) >>> shift) & bitMask)); - } - } - } else { - long hiBase = (remainingBits > 0) ? blockByteOff + (long) lanes * nextWord * 4 : 0L; - long loMask = (remainingBits > 0) ? (1L << currentBits) - 1L : 0L; - long hiMask = (remainingBits > 0) ? (1L << remainingBits) - 1L : 0L; - for (int lane = 0; lane < lanes; lane++) { - int logicalIdx = baseIdx + lane; - if (logicalIdx < 0 || logicalIdx >= rowCount) { - continue; - } - long src = Integer.toUnsignedLong(buf.get(PTypeIO.LE_INT, wordBase + (long) lane * 4)); - long value; - if (remainingBits > 0) { - long lo = (src >>> shift) & loMask; - long hi = Integer.toUnsignedLong(buf.get(PTypeIO.LE_INT, hiBase + (long) lane * 4)) & hiMask; - value = lo | (hi << currentBits); - } else { - value = (src >>> shift) & bitMask; - } - out.set(PTypeIO.LE_INT, (long) logicalIdx * 4, (int) value); - } - } - } - } - } - - private static void unpackLoop64(MemorySegment buf, int bitWidth, int offset, long rowCount, MemorySegment out) { - final int lanes = 16; - long totalElems = rowCount + offset; - int blockCount = (int) ((totalElems + 1023) / 1024); - long bitMask = bitWidth == 64 ? -1L : (1L << bitWidth) - 1L; - - long blockByteOff = 0L; - long blockByteStride = 128L * bitWidth; - for (int block = 0; block < blockCount; block++, blockByteOff += blockByteStride) { - int blockLogicStart = block * 1024 - offset; - boolean fullBlock = blockLogicStart >= 0 && (long) blockLogicStart + 1023L < rowCount; - - for (int row = 0; row < 64; row++) { - int currWord = (row * bitWidth) / 64; - int nextWord = ((row + 1) * bitWidth) / 64; - int shift = (row * bitWidth) % 64; - int remainingBits = (nextWord > currWord) ? ((row + 1) * bitWidth) % 64 : 0; - int currentBits = bitWidth - remainingBits; - int o = row / 8; - int s = row % 8; - int baseIdx = blockLogicStart + FL_ORDER[o] * 16 + s * 128; - long wordBase = blockByteOff + (long) lanes * currWord * 8; - - if (fullBlock) { - long outBase = (long) baseIdx * 8; - if (remainingBits > 0) { - long hiBase = blockByteOff + (long) lanes * nextWord * 8; - long loMask = (1L << currentBits) - 1L; - long hiMask = (1L << remainingBits) - 1L; - long laneOff = 0L; - for (int lane = 0; lane < lanes; lane++, laneOff += 8L) { - long lo = (buf.get(PTypeIO.LE_LONG, wordBase + laneOff) >>> shift) & loMask; - long hi = buf.get(PTypeIO.LE_LONG, hiBase + laneOff) & hiMask; - out.set(PTypeIO.LE_LONG, outBase + laneOff, lo | (hi << currentBits)); - } - } else { - long laneOff = 0L; - for (int lane = 0; lane < lanes; lane++, laneOff += 8L) { - out.set(PTypeIO.LE_LONG, outBase + laneOff, - (buf.get(PTypeIO.LE_LONG, wordBase + laneOff) >>> shift) & bitMask); - } - } - } else { - long hiBase = (remainingBits > 0) ? blockByteOff + (long) lanes * nextWord * 8 : 0L; - long loMask = (remainingBits > 0) ? (1L << currentBits) - 1L : 0L; - long hiMask = (remainingBits > 0) ? (1L << remainingBits) - 1L : 0L; - for (int lane = 0; lane < lanes; lane++) { - int logicalIdx = baseIdx + lane; - if (logicalIdx < 0 || logicalIdx >= rowCount) { - continue; - } - long src = buf.get(PTypeIO.LE_LONG, wordBase + (long) lane * 8); - long value; - if (remainingBits > 0) { - long lo = (src >>> shift) & loMask; - long hi = buf.get(PTypeIO.LE_LONG, hiBase + (long) lane * 8) & hiMask; - value = lo | (hi << currentBits); - } else { - value = (src >>> shift) & bitMask; - } - out.set(PTypeIO.LE_LONG, (long) logicalIdx * 8, value); - } - } - } - } - } - - private static void applyPatches(DecodeContext ctx, EncodingProtos.PatchesMetadata pm, - MemorySegment out, int elemBytes) { - long numPatches = pm.getLen(); - if (numPatches == 0) { - return; - } - long offset = pm.getOffset(); - PType idxPtype = ptypeFromProto(pm.getIndicesPtype()); - - Array idxArr = decodeChildAs(ctx, 0, new DType.Primitive(idxPtype, false), numPatches); - Array valArr = decodeChildAs(ctx, 1, ctx.dtype(), numPatches); - - MemorySegment idxSeg = idxArr.buffer(0); - MemorySegment valSeg = valArr.buffer(0); - - long n = ctx.rowCount(); - for (long i = 0; i < numPatches; i++) { - long absIdx = readUnsignedIdx(idxSeg, i, idxPtype) - offset; - if (absIdx < 0 || absIdx >= n) { - throw new VortexException(EncodingId.FASTLANES_BITPACKED, - "patch index " + absIdx + " out of range [0," + n + ")"); - } - MemorySegment.copy(valSeg, i * elemBytes, out, absIdx * elemBytes, elemBytes); - } - } - - private static Array decodeChildAs(DecodeContext parent, int childIdx, DType dtype, long rowCount) { - ArrayNode childNode = parent.node().children()[childIdx]; - DecodeContext childCtx = new DecodeContext( - childNode, dtype, rowCount, parent.segmentBuffers(), parent.registry(), parent.arena()); - return parent.registry().decode(childCtx); - } - - private static long readUnsignedIdx(MemorySegment seg, long i, PType ptype) { - return switch (ptype) { - case U8 -> Byte.toUnsignedLong(seg.get(ValueLayout.JAVA_BYTE, i)); - case U16 -> Short.toUnsignedLong(seg.get(PTypeIO.LE_SHORT, i * 2)); - case U32 -> Integer.toUnsignedLong(seg.get(PTypeIO.LE_INT, i * 4)); - case U64 -> seg.get(PTypeIO.LE_LONG, i * 8); - default -> throw new VortexException(EncodingId.FASTLANES_BITPACKED, - "non-unsigned patch index ptype " + ptype); - }; - } - - private static PType ptypeFromProto(DTypeProtos.PType proto) { - return PType.values()[proto.getNumber()]; - } - } -} diff --git a/core/src/main/java/io/github/dfa1/vortex/encoding/BoolEncoding.java b/core/src/main/java/io/github/dfa1/vortex/encoding/BoolEncoding.java deleted file mode 100644 index 6fe751b..0000000 --- a/core/src/main/java/io/github/dfa1/vortex/encoding/BoolEncoding.java +++ /dev/null @@ -1,74 +0,0 @@ -package io.github.dfa1.vortex.encoding; - -import io.github.dfa1.vortex.proto.ScalarProtos; -import io.github.dfa1.vortex.core.array.Array; -import io.github.dfa1.vortex.core.ArrayStats; -import io.github.dfa1.vortex.core.DType; -import io.github.dfa1.vortex.core.array.BoolArray; - -import java.lang.foreign.Arena; -import java.lang.foreign.MemorySegment; -import java.lang.foreign.ValueLayout; - -/// Encoding for `vortex.bool` — bit-packed boolean arrays (LSB first). -public final class BoolEncoding implements Encoding { - - /// Creates a new {@code BoolEncoding} instance. - public BoolEncoding() { - } - - private static MemorySegment encodeBool(boolean[] data) { - long packedBytes = (data.length + 7L) / 8; - if (packedBytes == 0) { - return MemorySegment.NULL; - } - MemorySegment seg = Arena.ofAuto().allocate(packedBytes); - for (int i = 0; i < data.length; i++) { - if (data[i]) { - long byteIdx = i / 8; - byte cur = seg.get(ValueLayout.JAVA_BYTE, byteIdx); - seg.set(ValueLayout.JAVA_BYTE, byteIdx, (byte) (cur | (1 << (i % 8)))); - } - } - return seg; - } - - @Override - public EncodingId encodingId() { - return EncodingId.VORTEX_BOOL; - } - - @Override - public boolean accepts(DType dtype) { - return dtype instanceof DType.Bool; - } - - @Override - public EncodeResult encode(DType dtype, Object data) { - boolean[] bools = (boolean[]) data; - boolean hasTrue = false; - boolean hasFalse = false; - for (boolean b : bools) { - if (b) { - hasTrue = true; - } else { - hasFalse = true; - } - if (hasTrue && hasFalse) { - break; - } - } - byte[] statsMin = bools.length > 0 - ? ScalarProtos.ScalarValue.newBuilder().setBoolValue(!hasFalse).build().toByteArray() - : null; - byte[] statsMax = bools.length > 0 - ? ScalarProtos.ScalarValue.newBuilder().setBoolValue(hasTrue).build().toByteArray() - : null; - return EncodeResult.simple(encodingId(), encodeBool(bools), statsMin, statsMax); - } - - @Override - public Array decode(DecodeContext ctx) { - return new BoolArray(ctx.dtype(), ctx.rowCount(), ctx.buffer(0), ArrayStats.empty()); - } -} diff --git a/core/src/main/java/io/github/dfa1/vortex/encoding/ByteBoolEncoding.java b/core/src/main/java/io/github/dfa1/vortex/encoding/ByteBoolEncoding.java deleted file mode 100644 index 82461d8..0000000 --- a/core/src/main/java/io/github/dfa1/vortex/encoding/ByteBoolEncoding.java +++ /dev/null @@ -1,60 +0,0 @@ -package io.github.dfa1.vortex.encoding; - -import io.github.dfa1.vortex.core.array.Array; -import io.github.dfa1.vortex.core.ArrayStats; -import io.github.dfa1.vortex.core.DType; -import io.github.dfa1.vortex.core.array.BoolArray; - -import java.lang.foreign.MemorySegment; -import java.lang.foreign.ValueLayout; - -/// Encoding for {@code vortex.bytebool} — one byte per boolean element (0 = false, non-zero = true). -/// -///

Buffer 0: raw byte values, length = rowCount. -/// Metadata: empty. -/// Child slot 0: validity array (optional, only when nullable with explicit validity bitmap). -/// -///

Decode: pack the byte buffer into a bit-packed {@link BoolArray} (LSB-first), -/// matching the layout of {@code vortex.bool}. -public final class ByteBoolEncoding implements Encoding { - - /// Creates a new {@code ByteBoolEncoding} instance. - public ByteBoolEncoding() { - } - - @Override - public EncodingId encodingId() { - return EncodingId.VORTEX_BYTEBOOL; - } - - @Override - public boolean accepts(DType dtype) { - return dtype instanceof DType.Bool; - } - - @Override - public EncodeResult encode(DType dtype, Object data) { - boolean[] bools = (boolean[]) data; - MemorySegment seg = java.lang.foreign.Arena.ofAuto().allocate(bools.length); - for (int i = 0; i < bools.length; i++) { - seg.set(ValueLayout.JAVA_BYTE, i, bools[i] ? (byte) 1 : (byte) 0); - } - return EncodeResult.simple(encodingId(), seg); - } - - @Override - public Array decode(DecodeContext ctx) { - long n = ctx.rowCount(); - MemorySegment bytes = ctx.buffer(0); - long packedBytes = (n + 7) >>> 3; - MemorySegment packed = ctx.arena().allocate(packedBytes > 0 ? packedBytes : 1); - for (long i = 0; i < n; i++) { - if (bytes.get(ValueLayout.JAVA_BYTE, i) != 0) { - long byteIdx = i >>> 3; - byte cur = packed.get(ValueLayout.JAVA_BYTE, byteIdx); - packed.set(ValueLayout.JAVA_BYTE, byteIdx, (byte) (cur | (1 << (i & 7)))); - } - } - return new BoolArray(ctx.dtype(), n, packed, ArrayStats.empty()); - } -} diff --git a/core/src/main/java/io/github/dfa1/vortex/encoding/CascadeStep.java b/core/src/main/java/io/github/dfa1/vortex/encoding/CascadeStep.java deleted file mode 100644 index cff7e78..0000000 --- a/core/src/main/java/io/github/dfa1/vortex/encoding/CascadeStep.java +++ /dev/null @@ -1,69 +0,0 @@ -package io.github.dfa1.vortex.encoding; - -import java.lang.foreign.MemorySegment; -import java.util.List; - -/// One step in cascade-aware encoding: a partially-assembled node tree plus open child slots. -/// -///

Terminal steps have no open children and carry a fully-resolved {@link EncodeResult}. -/// Intermediate steps have open children that the {@link CascadingCompressor} recursively fills. -/// -///

Buffer layout: {@code ownedBuffers} holds buffers belonging to the partial root -/// (e.g. patch index/value buffers for ALP). Child recursion results are appended after these; -/// child {@code bufferIndices} are remapped by {@code +ownedBuffers.size()}. -/// -///

When {@code applicable} is false the encoding cannot handle this data; {@link #ownedBytes()} -/// returns {@link Long#MAX_VALUE}/2 so the step never wins in size-based selection. -/// -/// @param partialRoot partially-assembled root encode node (may be {@code null} when not applicable) -/// @param ownedBuffers buffers owned directly by the root node, before child buffers are appended -/// @param openChildren child slots to be filled recursively by the cascading compressor -/// @param statsMin serialised minimum stat bytes, or {@code null} -/// @param statsMax serialised maximum stat bytes, or {@code null} -/// @param applicable {@code false} if this encoding cannot handle the input data -public record CascadeStep( - EncodeNode partialRoot, - List ownedBuffers, - List openChildren, - byte[] statsMin, - byte[] statsMax, - boolean applicable -) { - /// Convenience: terminal step — no open children, result is final. - /// - /// @param result the fully-resolved encode result to wrap - /// @return a terminal {@link CascadeStep} backed by {@code result} - public static CascadeStep terminal(EncodeResult result) { - return new CascadeStep(result.rootNode(), result.buffers(), List.of(), - result.statsMin(), result.statsMax(), true); - } - - /// Sentinel: encoding cannot handle this data — never wins in size selection. - /// - /// @return a non-applicable {@link CascadeStep} sentinel - public static CascadeStep notApplicable() { - return new CascadeStep(null, List.of(), List.of(), null, null, false); - } - - /// Returns {@code true} if this step has no open child slots. - /// - /// @return {@code true} if the step is terminal (no open children) - public boolean isTerminal() { - return openChildren.isEmpty(); - } - - /// Total byte size of owned buffers (used for size-based winner selection on samples). - /// Returns {@link Long#MAX_VALUE}/2 for non-applicable steps. - /// - /// @return total size in bytes of all owned buffers, or {@link Long#MAX_VALUE}/2 if not applicable - public long ownedBytes() { - if (!applicable) { - return Long.MAX_VALUE / 2; - } - long total = 0; - for (MemorySegment seg : ownedBuffers) { - total += seg.byteSize(); - } - return total; - } -} diff --git a/core/src/main/java/io/github/dfa1/vortex/encoding/CascadingCompressor.java b/core/src/main/java/io/github/dfa1/vortex/encoding/CascadingCompressor.java deleted file mode 100644 index e0c416c..0000000 --- a/core/src/main/java/io/github/dfa1/vortex/encoding/CascadingCompressor.java +++ /dev/null @@ -1,222 +0,0 @@ -package io.github.dfa1.vortex.encoding; - -import io.github.dfa1.vortex.core.DType; - -import java.lang.foreign.MemorySegment; -import java.util.ArrayList; -import java.util.Arrays; -import java.util.List; - -/// Cascading compressor: evaluates multiple encodings on a sample and picks the one -/// producing the smallest output. With {@code allowedCascading > 0}, also recurses -/// into open child slots (e.g. ALP → Bitpacked for F64 columns). -/// -///

Encodings that override {@link Encoding#encodeCascade} expose intermediate -/// representations as open children; encodings that use the default are terminal. -/// At depth 0 only terminal encodings are considered. -public final class CascadingCompressor { - - private final List encodings; - private final CompressorContext rootCtx; - - /// Constructs a {@code CascadingCompressor} with the given encodings and root context. - /// - /// @param encodings candidate encodings evaluated during compression - /// @param rootCtx root compressor context controlling cascade depth and sampling - public CascadingCompressor(List encodings, CompressorContext rootCtx) { - this.encodings = List.copyOf(encodings); - this.rootCtx = rootCtx; - } - - /// Entry point: encode {@code data} using the best cascading strategy. - /// - /// @param dtype logical type of the data to encode - /// @param data input data in the format expected by the candidate encodings - /// @return the {@link EncodeResult} produced by the winning encoding - public EncodeResult encode(DType dtype, Object data) { - return encodeWithCtx(dtype, data, rootCtx); - } - - private EncodeResult encodeWithCtx(DType dtype, Object data, CompressorContext ctx) { - if (dtype instanceof DType.Struct structDtype) { - return encodeStruct(structDtype, (StructData) data, ctx); - } - // Non-primitives (extension types): find the accepting encoding and splice - // through it so its cascaded children (e.g. datetimeparts → days/seconds/subseconds) - // are recursively compressed rather than stored as raw primitives. - if (!(dtype instanceof DType.Primitive)) { - return spliceResult(findPrimitiveEncoding(dtype), dtype, data, ctx); - } - int n = dataLength(data); - - // Build sample - int sampleSize = (int) Math.max(ctx.minSampleSize(), Math.ceil(n * ctx.sampleFraction())); - sampleSize = Math.min(sampleSize, n); - Object sample = (sampleSize < n) ? sliceSample(data, sampleSize) : data; - - long primitiveBaseline = primitiveBytes(dtype, sampleSize); - long bestSampleSize = primitiveBaseline; - Encoding winner = null; - - for (Encoding enc : encodings) { - if (!enc.accepts(dtype) || ctx.excluded().contains(enc.encodingId())) { - continue; - } - CascadeStep step = enc.encodeCascade(dtype, sample, ctx); - - // At depth 0, skip encodings that require cascade - if (!step.isTerminal() && ctx.allowedCascading() == 0) { - continue; - } - - long size = measureStep(enc, step, ctx); - if (size < bestSampleSize) { - bestSampleSize = size; - winner = enc; - } - } - - if (winner == null) { - // No encoding beats primitive — fall back - return findPrimitiveEncoding(dtype).encode(dtype, data); - } - - // Re-run winner on full data - return spliceResult(winner, dtype, data, ctx); - } - - // ── Size measurement (on sample) ────────────────────────────────────────── - - private long measureStep(Encoding enc, CascadeStep step, CompressorContext ctx) { - long total = step.ownedBytes(); - for (ChildSlot slot : step.openChildren()) { - CompressorContext childCtx = ctx.withDecrementedDepth().withExcluded(enc.encodingId()); - total += measureBestChild(slot.childDtype(), slot.childData(), childCtx); - } - return total; - } - - private long measureBestChild(DType dtype, Object data, CompressorContext ctx) { - int n = dataLength(data); - long best = primitiveBytes(dtype, n); - for (Encoding enc : encodings) { - if (!enc.accepts(dtype) || ctx.excluded().contains(enc.encodingId())) { - continue; - } - CascadeStep step = enc.encodeCascade(dtype, data, ctx); - if (!step.isTerminal() && ctx.allowedCascading() == 0) { - continue; - } - long size = measureStep(enc, step, ctx); - if (size < best) { - best = size; - } - } - return best; - } - - // ── Full-data run + buffer splicing ─────────────────────────────────────── - - private EncodeResult spliceResult(Encoding winner, DType dtype, Object data, CompressorContext ctx) { - CascadeStep step = winner.encodeCascade(dtype, data, ctx); - - if (!step.applicable()) { - // Winner was selected on a sample that looked applicable (e.g. all-constant prefix), - // but full data is not. Re-run without this encoding. - return encodeWithCtx(dtype, data, ctx.withExcluded(winner.encodingId())); - } - - if (step.isTerminal()) { - return new EncodeResult(step.partialRoot(), step.ownedBuffers(), step.statsMin(), step.statsMax()); - } - - List allBuffers = new ArrayList<>(step.ownedBuffers()); - EncodeNode[] children = step.partialRoot().children().clone(); - - for (ChildSlot slot : step.openChildren()) { - CompressorContext childCtx = ctx.withDecrementedDepth().withExcluded(winner.encodingId()); - EncodeResult childResult = encodeWithCtx(slot.childDtype(), slot.childData(), childCtx); - - int bufOffset = allBuffers.size(); - children[slot.parentChildIdx()] = EncodeNode.remapBufferIndices(childResult.rootNode(), bufOffset); - allBuffers.addAll(childResult.buffers()); - } - - EncodeNode root = new EncodeNode( - step.partialRoot().encodingId(), - step.partialRoot().metadata(), - children, - step.partialRoot().bufferIndices()); - return new EncodeResult(root, List.copyOf(allBuffers), step.statsMin(), step.statsMax()); - } - - // ── Struct encoding ─────────────────────────────────────────────────────── - - private EncodeResult encodeStruct(DType.Struct dtype, StructData data, CompressorContext ctx) { - List fields = data.fieldArrays(); - List fieldTypes = dtype.fieldTypes(); - List allBuffers = new ArrayList<>(); - EncodeNode[] children = new EncodeNode[fields.size()]; - for (int i = 0; i < fields.size(); i++) { - EncodeResult fieldResult = encodeWithCtx(fieldTypes.get(i), fields.get(i), ctx); - int bufOffset = allBuffers.size(); - children[i] = EncodeNode.remapBufferIndices(fieldResult.rootNode(), bufOffset); - allBuffers.addAll(fieldResult.buffers()); - } - EncodeNode root = new EncodeNode(EncodingId.VORTEX_STRUCT, null, children, new int[0]); - return new EncodeResult(root, List.copyOf(allBuffers), null, null); - } - - // ── Helpers ─────────────────────────────────────────────────────────────── - - private static int dataLength(Object data) { - return switch (data) { - case StructData sd -> sd.fieldArrays().isEmpty() ? 0 : dataLength(sd.fieldArrays().getFirst()); - case byte[] a -> a.length; - case short[] a -> a.length; - case int[] a -> a.length; - case long[] a -> a.length; - case float[] a -> a.length; - case double[] a -> a.length; - default -> throw new IllegalArgumentException("unsupported data type: " + data.getClass()); - }; - } - - private static Object sliceSample(Object data, int n) { - return switch (data) { - case StructData sd -> { - List sliced = sd.fieldArrays().stream().map(f -> sliceSample(f, n)).toList(); - yield new StructData(sliced); - } - case byte[] a -> Arrays.copyOf(a, n); - case short[] a -> Arrays.copyOf(a, n); - case int[] a -> Arrays.copyOf(a, n); - case long[] a -> Arrays.copyOf(a, n); - case float[] a -> Arrays.copyOf(a, n); - case double[] a -> Arrays.copyOf(a, n); - default -> throw new IllegalArgumentException("unsupported data type: " + data.getClass()); - }; - } - - private static long primitiveBytes(DType dtype, int n) { - if (!(dtype instanceof DType.Primitive p)) { - return (long) n * 8; - } - return (long) n * p.ptype().byteSize(); - } - - private Encoding findPrimitiveEncoding(DType dtype) { - for (Encoding enc : encodings) { - if (enc.encodingId().equals(EncodingId.VORTEX_PRIMITIVE) && enc.accepts(dtype)) { - return enc; - } - } - // Fall through to any accepting encoding - for (Encoding enc : encodings) { - if (enc.accepts(dtype)) { - return enc; - } - } - throw new UnsupportedOperationException("no encoding for dtype: " + dtype); - } -} diff --git a/core/src/main/java/io/github/dfa1/vortex/encoding/ChildSlot.java b/core/src/main/java/io/github/dfa1/vortex/encoding/ChildSlot.java deleted file mode 100644 index 4653e8f..0000000 --- a/core/src/main/java/io/github/dfa1/vortex/encoding/ChildSlot.java +++ /dev/null @@ -1,13 +0,0 @@ -package io.github.dfa1.vortex.encoding; - -import io.github.dfa1.vortex.core.DType; - -/// An open slot in a partially-assembled encoding tree. -/// The cascading compressor fills each slot recursively, then splices the result -/// into the parent node's children array at {@code parentChildIdx}. -/// -/// @param childDtype logical type of the child data -/// @param childData the raw child data to be encoded (type depends on the child encoding) -/// @param parentChildIdx index in the parent node's children array where the result will be placed -public record ChildSlot(DType childDtype, Object childData, int parentChildIdx) { -} diff --git a/core/src/main/java/io/github/dfa1/vortex/encoding/ChunkedData.java b/core/src/main/java/io/github/dfa1/vortex/encoding/ChunkedData.java deleted file mode 100644 index 665739f..0000000 --- a/core/src/main/java/io/github/dfa1/vortex/encoding/ChunkedData.java +++ /dev/null @@ -1,23 +0,0 @@ -package io.github.dfa1.vortex.encoding; - -import java.util.List; - -/// Input data for encoding a chunked array. -/// Each element of {@code chunks} is the raw data for one chunk, in the same format -/// the inner encoding expects (e.g. {@code long[]} for I64, {@link StructData} for Struct). -/// {@code chunkLengths[i]} is the row count of {@code chunks.get(i)}. -/// -/// @param chunks list of raw chunk data in the format the inner encoding expects -/// @param chunkLengths row count for each chunk; must have the same length as {@code chunks} -public record ChunkedData(List chunks, long[] chunkLengths) { - /// Validates that {@code chunks} and {@code chunkLengths} have the same size, - /// then makes defensive copies of both. - public ChunkedData { - if (chunks.size() != chunkLengths.length) { - throw new IllegalArgumentException( - "chunks.size() %d != chunkLengths.length %d".formatted(chunks.size(), chunkLengths.length)); - } - chunks = List.copyOf(chunks); - chunkLengths = chunkLengths.clone(); - } -} diff --git a/core/src/main/java/io/github/dfa1/vortex/encoding/ChunkedEncoding.java b/core/src/main/java/io/github/dfa1/vortex/encoding/ChunkedEncoding.java deleted file mode 100644 index 22cc993..0000000 --- a/core/src/main/java/io/github/dfa1/vortex/encoding/ChunkedEncoding.java +++ /dev/null @@ -1,207 +0,0 @@ -package io.github.dfa1.vortex.encoding; - -import io.github.dfa1.vortex.core.ArrayStats; -import io.github.dfa1.vortex.core.DType; -import io.github.dfa1.vortex.core.PType; -import io.github.dfa1.vortex.core.VortexException; -import io.github.dfa1.vortex.core.array.Array; -import io.github.dfa1.vortex.core.array.ByteArray; -import io.github.dfa1.vortex.core.array.DoubleArray; -import io.github.dfa1.vortex.core.array.FloatArray; -import io.github.dfa1.vortex.core.array.IntArray; -import io.github.dfa1.vortex.core.array.LongArray; -import io.github.dfa1.vortex.core.array.ShortArray; -import io.github.dfa1.vortex.core.array.StructArray; - -import java.lang.foreign.MemorySegment; -import java.lang.foreign.SegmentAllocator; -import java.lang.foreign.ValueLayout; -import java.nio.ByteBuffer; -import java.nio.ByteOrder; -import java.util.ArrayList; -import java.util.List; - -/// Encoding for {@code vortex.chunked} — a segment-level chunked array. -/// -///

Wire format (per Rust vtable): -///

    -///
  • Metadata: empty bytes -///
  • Buffers: 0 -///
  • Children: N+1 — {@code children[0]} is the chunk offsets (U64, non-nullable, -/// length = nchunks+1, cumulative from 0); {@code children[1..N]} are the chunk arrays. -///
-public final class ChunkedEncoding implements Encoding { - - /// Creates a new {@code ChunkedEncoding} instance. - public ChunkedEncoding() { - } - - @Override - public EncodingId encodingId() { - return EncodingId.VORTEX_CHUNKED; - } - - @Override - public boolean accepts(DType dtype) { - return dtype instanceof DType.Primitive || dtype instanceof DType.Struct; - } - - @Override - public EncodeResult encode(DType dtype, Object data) { - return Encoder.encode(dtype, (ChunkedData) data); - } - - @Override - public Array decode(DecodeContext ctx) { - return Decoder.decode(ctx); - } - - private static final class Encoder { - - private static final List FALLBACK = List.of( - new PrimitiveEncoding(), new VarBinEncoding(), new BoolEncoding(), - new NullEncoding(), new ByteBoolEncoding(), new StructEncoding()); - - static EncodeResult encode(DType dtype, ChunkedData data) { - List chunks = data.chunks(); - long[] chunkLengths = data.chunkLengths(); - int nchunks = chunks.size(); - if (nchunks == 0) { - throw new VortexException(EncodingId.VORTEX_CHUNKED, "at least one chunk required"); - } - - // Build cumulative offsets: [0, len0, len0+len1, ...] - long[] offsets = new long[nchunks + 1]; - offsets[0] = 0; - for (int i = 0; i < nchunks; i++) { - offsets[i + 1] = offsets[i] + chunkLengths[i]; - } - - // Encode offsets child (U64 primitive) - DType u64 = new DType.Primitive(PType.U64, false); - EncodeResult offsetsResult = new PrimitiveEncoding().encode(u64, offsets); - - List allBuffers = new ArrayList<>(offsetsResult.buffers()); - EncodeNode[] children = new EncodeNode[nchunks + 1]; - children[0] = offsetsResult.rootNode(); - - // Encode each chunk - Encoding inner = findEncoding(dtype); - for (int i = 0; i < nchunks; i++) { - EncodeResult chunkResult = inner.encode(dtype, chunks.get(i)); - int bufOffset = allBuffers.size(); - children[i + 1] = EncodeNode.remapBufferIndices(chunkResult.rootNode(), bufOffset); - allBuffers.addAll(chunkResult.buffers()); - } - - EncodeNode root = new EncodeNode( - EncodingId.VORTEX_CHUNKED, - ByteBuffer.wrap(new byte[0]), - children, - new int[]{}); - return new EncodeResult(root, List.copyOf(allBuffers), null, null); - } - - private static Encoding findEncoding(DType dtype) { - for (Encoding enc : FALLBACK) { - if (enc.accepts(dtype)) { - return enc; - } - } - throw new UnsupportedOperationException("no fallback encoding for dtype: " + dtype); - } - } - - private static final class Decoder { - - private static final ValueLayout.OfLong LE_LONG = - ValueLayout.JAVA_LONG_UNALIGNED.withOrder(ByteOrder.LITTLE_ENDIAN); - - static Array decode(DecodeContext ctx) { - int nchildren = ctx.node().children().length; - if (nchildren < 1) { - throw new VortexException(EncodingId.VORTEX_CHUNKED, - "needs at least one child (chunk offsets)"); - } - int nchunks = nchildren - 1; - long[] offsets = readOffsets(ctx, nchunks); - - DType dtype = ctx.dtype(); - List chunks = new ArrayList<>(nchunks); - for (int i = 0; i < nchunks; i++) { - long chunkLen = offsets[i + 1] - offsets[i]; - ArrayNode chunkNode = ctx.node().children()[i + 1]; - var chunkCtx = new DecodeContext( - chunkNode, dtype, chunkLen, ctx.segmentBuffers(), ctx.registry(), ctx.arena()); - chunks.add(ctx.registry().decode(chunkCtx)); - } - - return concat(chunks, dtype, ctx.rowCount(), ctx.arena()); - } - - private static long[] readOffsets(DecodeContext ctx, int nchunks) { - ArrayNode offsetsNode = ctx.node().children()[0]; - DType u64 = new DType.Primitive(PType.U64, false); - var offsetsCtx = new DecodeContext( - offsetsNode, u64, nchunks + 1L, ctx.segmentBuffers(), ctx.registry(), ctx.arena()); - Array offsetsArray = ctx.registry().decode(offsetsCtx); - MemorySegment offsetsBuf = offsetsArray.buffer(0); - long[] offsets = new long[nchunks + 1]; - for (int i = 0; i <= nchunks; i++) { - offsets[i] = offsetsBuf.get(LE_LONG, (long) i * 8); - } - return offsets; - } - - private static Array concat(List chunks, DType dtype, long totalRows, SegmentAllocator arena) { - if (dtype instanceof DType.Primitive pt) { - return concatPrimitive(chunks, pt, dtype, totalRows, arena); - } - if (dtype instanceof DType.Struct struct) { - return concatStruct(chunks, struct, totalRows, arena); - } - throw new VortexException(EncodingId.VORTEX_CHUNKED, - "concat not supported for dtype: " + dtype); - } - - private static Array concatPrimitive( - List chunks, DType.Primitive pt, DType dtype, long totalRows, SegmentAllocator arena - ) { - PType ptype = pt.ptype(); - MemorySegment combined = arena.allocate(totalRows * ptype.byteSize()); - long byteOffset = 0; - for (Array chunk : chunks) { - MemorySegment src = chunk.buffer(0); - MemorySegment.copy(src, 0, combined, byteOffset, src.byteSize()); - byteOffset += src.byteSize(); - } - MemorySegment ro = combined.asReadOnly(); - return switch (ptype) { - case I64, U64 -> new LongArray(dtype, totalRows, ro, ArrayStats.empty()); - case I32, U32 -> new IntArray(dtype, totalRows, ro, ArrayStats.empty()); - case F64 -> new DoubleArray(dtype, totalRows, ro, ArrayStats.empty()); - case F32 -> new FloatArray(dtype, totalRows, ro, ArrayStats.empty()); - case I16, U16 -> new ShortArray(dtype, totalRows, ro, ArrayStats.empty()); - case I8, U8 -> new ByteArray(dtype, totalRows, ro, ArrayStats.empty()); - default -> throw new VortexException(EncodingId.VORTEX_CHUNKED, - "unsupported ptype for concat: " + ptype); - }; - } - - private static StructArray concatStruct( - List chunks, DType.Struct struct, long totalRows, SegmentAllocator arena - ) { - int nfields = struct.fieldTypes().size(); - List concatFields = new ArrayList<>(nfields); - for (int f = 0; f < nfields; f++) { - DType fieldDtype = struct.fieldTypes().get(f); - List fieldChunks = new ArrayList<>(chunks.size()); - for (Array chunk : chunks) { - fieldChunks.add(((StructArray) chunk).field(f)); - } - concatFields.add(concat(fieldChunks, fieldDtype, totalRows, arena)); - } - return new StructArray(struct, totalRows, concatFields); - } - } -} diff --git a/core/src/main/java/io/github/dfa1/vortex/encoding/CompressorContext.java b/core/src/main/java/io/github/dfa1/vortex/encoding/CompressorContext.java deleted file mode 100644 index ef828e2..0000000 --- a/core/src/main/java/io/github/dfa1/vortex/encoding/CompressorContext.java +++ /dev/null @@ -1,48 +0,0 @@ -package io.github.dfa1.vortex.encoding; - -import java.util.Collections; -import java.util.HashSet; -import java.util.Set; - -/// Immutable context passed through cascading compression. -/// -///

{@code allowedCascading=0} means only terminal encodings (no open children). -/// Each recursive step decrements the depth and adds the current encoding to excluded. -/// -/// @param allowedCascading remaining cascade depth; 0 = only terminal encodings allowed -/// @param excluded encoding ids excluded from consideration at the current recursion level -/// @param sampleSeed random seed used for stratified sampling -/// @param minSampleSize minimum number of rows to include in a sample -/// @param sampleFraction fraction of rows to sample when the array is large -public record CompressorContext( - int allowedCascading, - Set excluded, - long sampleSeed, - int minSampleSize, - double sampleFraction -) { - /// Creates a default context with the given cascade depth and sensible sampling defaults. - /// - /// @param depth maximum allowed cascade depth - /// @return a new {@link CompressorContext} ready for top-level compression - public static CompressorContext ofDepth(int depth) { - return new CompressorContext(depth, Set.of(), 42L, 4096, 0.05); - } - - /// Returns a copy of this context with the cascade depth decremented by one. - /// - /// @return a new {@link CompressorContext} with {@code allowedCascading} reduced by 1 - public CompressorContext withDecrementedDepth() { - return new CompressorContext(allowedCascading - 1, excluded, sampleSeed, minSampleSize, sampleFraction); - } - - /// Returns a copy of this context with the given encoding id added to the excluded set. - /// - /// @param id the encoding id to exclude from consideration at this recursion level - /// @return a new {@link CompressorContext} with {@code id} added to the excluded set - public CompressorContext withExcluded(EncodingId id) { - Set next = new HashSet<>(excluded); - next.add(id); - return new CompressorContext(allowedCascading, Collections.unmodifiableSet(next), sampleSeed, minSampleSize, sampleFraction); - } -} diff --git a/core/src/main/java/io/github/dfa1/vortex/encoding/ConstantEncoding.java b/core/src/main/java/io/github/dfa1/vortex/encoding/ConstantEncoding.java deleted file mode 100644 index 2ac732c..0000000 --- a/core/src/main/java/io/github/dfa1/vortex/encoding/ConstantEncoding.java +++ /dev/null @@ -1,264 +0,0 @@ -package io.github.dfa1.vortex.encoding; - -import com.google.protobuf.InvalidProtocolBufferException; -import io.github.dfa1.vortex.proto.ScalarProtos; -import io.github.dfa1.vortex.core.array.Array; -import io.github.dfa1.vortex.core.ArrayStats; -import io.github.dfa1.vortex.core.DType; -import io.github.dfa1.vortex.core.PType; -import io.github.dfa1.vortex.core.array.BoolArray; -import io.github.dfa1.vortex.core.array.GenericArray; -import io.github.dfa1.vortex.core.array.NullArray; -import io.github.dfa1.vortex.core.array.ByteArray; -import io.github.dfa1.vortex.core.array.DoubleArray; -import io.github.dfa1.vortex.core.array.FloatArray; -import io.github.dfa1.vortex.core.array.IntArray; -import io.github.dfa1.vortex.core.array.LongArray; -import io.github.dfa1.vortex.core.array.ShortArray; -import io.github.dfa1.vortex.core.array.VarBinArray; -import io.github.dfa1.vortex.core.VortexException; - -import java.lang.foreign.MemorySegment; -import java.lang.foreign.ValueLayout; -import java.nio.ByteBuffer; -import java.nio.ByteOrder; -import java.nio.charset.StandardCharsets; - -/// Decoder for {@code vortex.constant} — all elements share the same value. -/// -///

No metadata (empty bytes). Buffer 0: the constant value as raw {@code ScalarValue} -/// proto bytes. No children. -/// -///

Decode: fill an output buffer of {@code rowCount} elements with the constant value. -public final class ConstantEncoding implements Encoding { - - /// Creates a new {@code ConstantEncoding} instance. - public ConstantEncoding() { - } - - @Override - public EncodingId encodingId() { - return EncodingId.VORTEX_CONSTANT; - } - - @Override - public boolean accepts(DType dtype) { - return dtype instanceof DType.Primitive; - } - - @Override - public EncodeResult encode(DType dtype, Object data) { - return Encoder.encode(dtype, data); - } - - @Override - public CascadeStep encodeCascade(DType dtype, Object data, CompressorContext ctx) { - if (!Encoder.isConstant(data, ((DType.Primitive) dtype).ptype())) { - return CascadeStep.notApplicable(); - } - return CascadeStep.terminal(Encoder.encode(dtype, data)); - } - - @Override - public Array decode(DecodeContext ctx) { - return Decoder.decode(ctx); - } - - private static final class Encoder { - - private static EncodeResult encode(DType dtype, Object data) { - if (!(dtype instanceof DType.Primitive p)) { - throw new VortexException(EncodingId.VORTEX_CONSTANT, "encode only supports Primitive dtype, got " + dtype); - } - PType ptype = p.ptype(); - if (!isConstant(data, ptype)) { - throw new VortexException(EncodingId.VORTEX_CONSTANT, "not a constant array"); - } - long firstRaw = readFirstRaw(data, ptype); - ScalarProtos.ScalarValue scalar = buildScalar(ptype, firstRaw); - return EncodeResult.simple(EncodingId.VORTEX_CONSTANT, MemorySegment.ofArray(scalar.toByteArray())); - } - - private static long readFirstRaw(Object data, PType ptype) { - return switch (ptype) { - case I8, U8 -> ((byte[]) data).length > 0 ? ((byte[]) data)[0] : 0L; - case I16, U16 -> ((short[]) data).length > 0 ? ((short[]) data)[0] : 0L; - case I32, U32 -> ((int[]) data).length > 0 ? ((int[]) data)[0] : 0L; - case I64, U64 -> ((long[]) data).length > 0 ? ((long[]) data)[0] : 0L; - case F32 -> ((float[]) data).length > 0 ? Float.floatToRawIntBits(((float[]) data)[0]) : 0L; - case F64 -> ((double[]) data).length > 0 ? Double.doubleToRawLongBits(((double[]) data)[0]) : 0L; - default -> throw new VortexException(EncodingId.VORTEX_CONSTANT, "unsupported ptype: " + ptype); - }; - } - - static boolean isConstant(Object data, PType ptype) { - long firstRaw = readFirstRaw(data, ptype); - int len = switch (ptype) { - case I8, U8 -> ((byte[]) data).length; - case I16, U16 -> ((short[]) data).length; - case I32, U32 -> ((int[]) data).length; - case I64, U64 -> ((long[]) data).length; - case F32 -> ((float[]) data).length; - case F64 -> ((double[]) data).length; - default -> throw new VortexException(EncodingId.VORTEX_CONSTANT, "unsupported ptype: " + ptype); - }; - for (int i = 1; i < len; i++) { - long raw = switch (ptype) { - case I8, U8 -> ((byte[]) data)[i]; - case I16, U16 -> ((short[]) data)[i]; - case I32, U32 -> ((int[]) data)[i]; - case I64, U64 -> ((long[]) data)[i]; - case F32 -> Float.floatToRawIntBits(((float[]) data)[i]); - case F64 -> Double.doubleToRawLongBits(((double[]) data)[i]); - default -> throw new VortexException(EncodingId.VORTEX_CONSTANT, "unsupported ptype: " + ptype); - }; - if (raw != firstRaw) { - return false; - } - } - return true; - } - - private static ScalarProtos.ScalarValue buildScalar(PType ptype, long rawBits) { - return switch (ptype) { - case U8, U16, U32, U64 -> ScalarProtos.ScalarValue.newBuilder().setUint64Value(rawBits).build(); - case I8, I16, I32, I64 -> ScalarProtos.ScalarValue.newBuilder().setInt64Value(rawBits).build(); - case F32 -> ScalarProtos.ScalarValue.newBuilder().setF32Value(Float.intBitsToFloat((int) rawBits)).build(); - case F64 -> ScalarProtos.ScalarValue.newBuilder().setF64Value(Double.longBitsToDouble(rawBits)).build(); - default -> throw new VortexException(EncodingId.VORTEX_CONSTANT, "unsupported ptype: " + ptype); - }; - } - } - - private static final class Decoder { - - private static Array decode(DecodeContext ctx) { - MemorySegment scalarBuf = ctx.buffer(0); - ScalarProtos.ScalarValue scalar; - try { - scalar = ScalarProtos.ScalarValue.parseFrom(scalarBuf.asByteBuffer()); - } catch (InvalidProtocolBufferException e) { - throw new VortexException(EncodingId.VORTEX_CONSTANT, "invalid scalar value", e); - } - - long n = ctx.rowCount(); - - if (ctx.dtype() instanceof DType.Null) { - return new NullArray(ctx.dtype(), n); - } - - if (ctx.dtype() instanceof DType.Utf8 || ctx.dtype() instanceof DType.Binary) { - return decodeString(ctx, scalar, n); - } - - if (ctx.dtype() instanceof DType.Bool) { - return decodeBool(ctx, scalar, n); - } - - if (ctx.dtype() instanceof DType.Decimal) { - return decodeDecimal(ctx, scalar, n); - } - - if (ctx.dtype() instanceof DType.Extension ext) { - // Decode using the storage dtype, re-wrap with the extension dtype - var storageCtx = new DecodeContext(ctx.node(), ext.storageDType(), ctx.rowCount(), - ctx.segmentBuffers(), ctx.registry(), ctx.arena()); - Array storage = decode(storageCtx); - return new GenericArray(ctx.dtype(), n, storage.buffer(0)); - } - - if (!(ctx.dtype() instanceof DType.Primitive p)) { - throw new VortexException(EncodingId.VORTEX_CONSTANT, "unsupported dtype " + ctx.dtype()); - } - - PType ptype = p.ptype(); - int elemBytes = ptype.byteSize(); - long rawBits = scalarToRawBits(scalar, ptype); - - MemorySegment outSeg = ctx.arena().allocate(n * elemBytes); - ByteBuffer out = outSeg.asByteBuffer().order(ByteOrder.LITTLE_ENDIAN); - for (long i = 0; i < n; i++) { - writeRaw(out, ptype, rawBits); - } - - MemorySegment ro = outSeg.asReadOnly(); - return switch (ptype) { - case I64, U64 -> new LongArray(ctx.dtype(), n, ro, ArrayStats.empty()); - case I32, U32 -> new IntArray(ctx.dtype(), n, ro, ArrayStats.empty()); - case F64 -> new DoubleArray(ctx.dtype(), n, ro, ArrayStats.empty()); - case F32 -> new FloatArray(ctx.dtype(), n, ro, ArrayStats.empty()); - case I16, U16 -> new ShortArray(ctx.dtype(), n, ro, ArrayStats.empty()); - case I8, U8 -> new ByteArray(ctx.dtype(), n, ro, ArrayStats.empty()); - default -> throw new VortexException(EncodingId.VORTEX_CONSTANT, "unsupported ptype " + ptype); - }; - } - - private static Array decodeDecimal(DecodeContext ctx, ScalarProtos.ScalarValue scalar, long n) { - // Decimal stored as i128 (16 bytes LE) in bytes_value - byte[] elemBytes = scalar.getBytesValue().toByteArray(); - int elemLen = elemBytes.length; - MemorySegment outSeg = ctx.arena().allocate(n * elemLen); - MemorySegment elemSeg = MemorySegment.ofArray(elemBytes); - for (long i = 0; i < n; i++) { - MemorySegment.copy(elemSeg, 0L, outSeg, i * elemLen, elemLen); - } - return new GenericArray(ctx.dtype(), n, outSeg.asReadOnly()); - } - - private static Array decodeBool(DecodeContext ctx, ScalarProtos.ScalarValue scalar, long n) { - boolean value = scalar.getBoolValue(); - long numBytes = (n + 7) >>> 3; - MemorySegment seg = ctx.arena().allocate(numBytes); - if (value) { - for (long i = 0; i < numBytes; i++) { - seg.set(ValueLayout.JAVA_BYTE, i, (byte) 0xFF); - } - } - return new BoolArray(ctx.dtype(), n, seg.asReadOnly(), ArrayStats.empty()); - } - - private static Array decodeString(DecodeContext ctx, ScalarProtos.ScalarValue scalar, long n) { - byte[] strBytes = scalar.hasStringValue() - ? scalar.getStringValue().getBytes(StandardCharsets.UTF_8) - : scalar.getBytesValue().toByteArray(); - - int strLen = strBytes.length; - - MemorySegment bytesSeg = ctx.arena().allocate((long) n * strLen); - for (long i = 0; i < n; i++) { - MemorySegment.copy(MemorySegment.ofArray(strBytes), 0L, bytesSeg, i * strLen, strLen); - } - - MemorySegment offsetsSeg = ctx.arena().allocate((n + 1) * 4L, 4); - for (long i = 0; i <= n; i++) { - offsetsSeg.setAtIndex(PTypeIO.LE_INT, i, (int) (i * strLen)); - } - - DType i32 = new DType.Primitive(PType.I32, false); - Array offsets = new IntArray(i32, n + 1, offsetsSeg.asReadOnly(), ArrayStats.empty()); - return new VarBinArray(ctx.dtype(), n, bytesSeg.asReadOnly(), offsets, PType.I32, ArrayStats.empty()); - } - - private static long scalarToRawBits(ScalarProtos.ScalarValue scalar, PType ptype) { - return switch (scalar.getKindCase()) { - case INT64_VALUE -> scalar.getInt64Value(); - case UINT64_VALUE -> scalar.getUint64Value(); - case F32_VALUE -> Float.floatToRawIntBits(scalar.getF32Value()); - case F64_VALUE -> Double.doubleToRawLongBits(scalar.getF64Value()); - case KIND_NOT_SET -> 0L; - default -> throw new VortexException(EncodingId.VORTEX_CONSTANT, - "unexpected scalar kind " + scalar.getKindCase()); - }; - } - - private static void writeRaw(ByteBuffer buf, PType ptype, long rawBits) { - switch (ptype.byteSize()) { - case 1 -> buf.put((byte) rawBits); - case 2 -> buf.putShort((short) rawBits); - case 4 -> buf.putInt((int) rawBits); - case 8 -> buf.putLong(rawBits); - default -> throw new VortexException(EncodingId.VORTEX_CONSTANT, "unsupported ptype " + ptype); - } - } - } -} diff --git a/core/src/main/java/io/github/dfa1/vortex/encoding/DateTimePartsData.java b/core/src/main/java/io/github/dfa1/vortex/encoding/DateTimePartsData.java deleted file mode 100644 index bc4164f..0000000 --- a/core/src/main/java/io/github/dfa1/vortex/encoding/DateTimePartsData.java +++ /dev/null @@ -1,8 +0,0 @@ -package io.github.dfa1.vortex.encoding; - -/// Input data for {@link DateTimePartsEncoding} encoder. -/// -/// @param timestamps raw i64 timestamps (number of time units since Unix epoch) -/// @param nullable whether the array has a validity (null) dimension -public record DateTimePartsData(long[] timestamps, boolean nullable) { -} diff --git a/core/src/main/java/io/github/dfa1/vortex/encoding/DateTimePartsEncoding.java b/core/src/main/java/io/github/dfa1/vortex/encoding/DateTimePartsEncoding.java deleted file mode 100644 index c75f291..0000000 --- a/core/src/main/java/io/github/dfa1/vortex/encoding/DateTimePartsEncoding.java +++ /dev/null @@ -1,223 +0,0 @@ -package io.github.dfa1.vortex.encoding; - -import com.google.protobuf.InvalidProtocolBufferException; -import io.github.dfa1.vortex.core.DType; -import io.github.dfa1.vortex.core.PType; -import io.github.dfa1.vortex.core.VortexException; -import io.github.dfa1.vortex.core.array.Array; -import io.github.dfa1.vortex.core.array.GenericArray; -import io.github.dfa1.vortex.proto.DTypeProtos; -import io.github.dfa1.vortex.proto.EncodingProtos; - -import java.lang.foreign.MemorySegment; -import java.nio.ByteBuffer; -import java.util.ArrayList; -import java.util.List; - -/// Encoder/decoder for {@code vortex.datetimeparts} — timestamp split into days/seconds/subseconds. -/// -///

Wire format (per Rust vtable): -///

    -///
  • Metadata: {@code DateTimePartsMetadata} — three PType fields (tag 1/2/3): -/// {@code days_ptype}, {@code seconds_ptype}, {@code subseconds_ptype} -///
  • Buffers: 0 -///
  • Children: 3 -///
      -///
    • Slot 0 — {@code days}: {@code Primitive(days_ptype, parentNullability)} -///
    • Slot 1 — {@code seconds}: {@code Primitive(seconds_ptype, false)} -///
    • Slot 2 — {@code subseconds}: {@code Primitive(subseconds_ptype, false)} -///
    -///
-/// -///

Extension metadata for {@code vortex.timestamp} dtype (hand-rolled, not protobuf): -/// {@code byte[0]=TimeUnit tag, bytes[1-2]=tz_len (u16 LE), bytes[3+]=tz UTF-8} -public final class DateTimePartsEncoding implements Encoding { - - /// Creates a new {@code DateTimePartsEncoding} instance. - public DateTimePartsEncoding() { - } - - @Override - public EncodingId encodingId() { - return EncodingId.VORTEX_DATETIMEPARTS; - } - - @Override - public boolean accepts(DType dtype) { - return dtype instanceof DType.Extension; - } - - @Override - public EncodeResult encode(DType dtype, Object data) { - return Encoder.encode((DType.Extension) dtype, (DateTimePartsData) data); - } - - @Override - public CascadeStep encodeCascade(DType dtype, Object data, CompressorContext ctx) { - return Encoder.encodeCascade((DType.Extension) dtype, (DateTimePartsData) data); - } - - @Override - public Array decode(DecodeContext ctx) { - return Decoder.decode(ctx); - } - - private static final class Encoder { - - private static final long SECONDS_PER_DAY = 86_400L; - private static final DType I64 = new DType.Primitive(PType.I64, false); - private static final DType I64_NULLABLE = new DType.Primitive(PType.I64, true); - private static final DTypeProtos.PType I64_PROTO = - DTypeProtos.PType.forNumber(PType.I64.ordinal()); - - static EncodeResult encode(DType.Extension dtype, DateTimePartsData data) { - ByteBuffer extMeta = dtype.metadata(); - if (extMeta == null || extMeta.remaining() < 3) { - throw new VortexException(EncodingId.VORTEX_DATETIMEPARTS, - "extension metadata missing or too short"); - } - byte[] extBytes = new byte[extMeta.remaining()]; - extMeta.duplicate().get(extBytes); - TimeUnit unit = TimeUnit.fromTag(extBytes[0]); - - long divisor = unit.divisor(); - long ticksPerDay = SECONDS_PER_DAY * divisor; - int n = data.timestamps().length; - - long[] days = new long[n]; - long[] seconds = new long[n]; - long[] subseconds = new long[n]; - - for (int i = 0; i < n; i++) { - long ts = data.timestamps()[i]; - long d = ts / ticksPerDay; - long rem = ts % ticksPerDay; - if (rem < 0) { - rem += ticksPerDay; - d--; - } - days[i] = d; - seconds[i] = rem / divisor; - subseconds[i] = rem % divisor; - } - - PrimitiveEncoding primEnc = new PrimitiveEncoding(); - DType daysDtype = data.nullable() ? I64_NULLABLE : I64; - - EncodeResult daysResult = primEnc.encode(daysDtype, days); - EncodeResult secondsResult = primEnc.encode(I64, seconds); - EncodeResult subsecondsResult = primEnc.encode(I64, subseconds); - - List allBuffers = new ArrayList<>(); - allBuffers.addAll(daysResult.buffers()); - allBuffers.addAll(secondsResult.buffers()); - allBuffers.addAll(subsecondsResult.buffers()); - - int off1 = daysResult.buffers().size(); - int off2 = off1 + secondsResult.buffers().size(); - - EncodeNode daysNode = EncodeNode.remapBufferIndices(daysResult.rootNode(), 0); - EncodeNode secondsNode = EncodeNode.remapBufferIndices(secondsResult.rootNode(), off1); - EncodeNode subsecondsNode = EncodeNode.remapBufferIndices(subsecondsResult.rootNode(), off2); - - byte[] metaBytes = EncodingProtos.DateTimePartsMetadata.newBuilder() - .setDaysPtype(I64_PROTO) - .setSecondsPtype(I64_PROTO) - .setSubsecondsPtype(I64_PROTO) - .build() - .toByteArray(); - - EncodeNode root = new EncodeNode( - EncodingId.VORTEX_DATETIMEPARTS, - ByteBuffer.wrap(metaBytes), - new EncodeNode[]{daysNode, secondsNode, subsecondsNode}, - new int[]{}); - return new EncodeResult(root, List.copyOf(allBuffers), null, null); - } - - static CascadeStep encodeCascade(DType.Extension dtype, DateTimePartsData data) { - ByteBuffer extMeta = dtype.metadata(); - byte[] extBytes = new byte[extMeta.remaining()]; - extMeta.duplicate().get(extBytes); - TimeUnit unit = TimeUnit.fromTag(extBytes[0]); - - long divisor = unit.divisor(); - long ticksPerDay = SECONDS_PER_DAY * divisor; - int n = data.timestamps().length; - - long[] days = new long[n]; - long[] seconds = new long[n]; - long[] subseconds = new long[n]; - - for (int i = 0; i < n; i++) { - long ts = data.timestamps()[i]; - long d = ts / ticksPerDay; - long rem = ts % ticksPerDay; - if (rem < 0) { - rem += ticksPerDay; - d--; - } - days[i] = d; - seconds[i] = rem / divisor; - subseconds[i] = rem % divisor; - } - - byte[] metaBytes = EncodingProtos.DateTimePartsMetadata.newBuilder() - .setDaysPtype(I64_PROTO).setSecondsPtype(I64_PROTO).setSubsecondsPtype(I64_PROTO) - .build().toByteArray(); - - // 3 null slots filled by the cascading compressor (days, seconds, subseconds) - EncodeNode partialRoot = new EncodeNode( - EncodingId.VORTEX_DATETIMEPARTS, - ByteBuffer.wrap(metaBytes), - new EncodeNode[3], - new int[0]); - - DType daysDtype = data.nullable() ? I64_NULLABLE : I64; - List children = List.of( - new ChildSlot(daysDtype, days, 0), - new ChildSlot(I64, seconds, 1), - new ChildSlot(I64, subseconds, 2)); - - return new CascadeStep(partialRoot, List.of(), children, null, null, true); - } - } - - private static final class Decoder { - - private static Array decode(DecodeContext ctx) { - ByteBuffer meta = ctx.metadata(); - if (meta == null || meta.remaining() == 0) { - throw new VortexException(EncodingId.VORTEX_DATETIMEPARTS, "missing metadata"); - } - EncodingProtos.DateTimePartsMetadata decoded; - try { - byte[] bytes = new byte[meta.remaining()]; - meta.duplicate().get(bytes); - decoded = EncodingProtos.DateTimePartsMetadata.parseFrom(bytes); - } catch (InvalidProtocolBufferException e) { - throw new VortexException(EncodingId.VORTEX_DATETIMEPARTS, "invalid metadata: " + e.getMessage()); - } - - PType daysPtype = PType.values()[decoded.getDaysPtypeValue()]; - PType secondsPtype = PType.values()[decoded.getSecondsPtypeValue()]; - PType subsecondsPtype = PType.values()[decoded.getSubsecondsPtypeValue()]; - boolean nullable = ctx.dtype().nullable(); - - Array days = decodeChild(ctx, 0, new DType.Primitive(daysPtype, nullable)); - Array seconds = decodeChild(ctx, 1, new DType.Primitive(secondsPtype, false)); - Array subseconds = decodeChild(ctx, 2, new DType.Primitive(subsecondsPtype, false)); - - return new GenericArray(ctx.dtype(), ctx.rowCount(), new MemorySegment[0], - new Array[]{days, seconds, subseconds}); - } - - private static Array decodeChild(DecodeContext ctx, int idx, DType childDtype) { - ArrayNode childNode = ctx.node().children()[idx]; - DecodeContext childCtx = new DecodeContext( - childNode, childDtype, ctx.rowCount(), - ctx.segmentBuffers(), ctx.registry(), ctx.arena()); - return ctx.registry().decode(childCtx); - } - } -} diff --git a/core/src/main/java/io/github/dfa1/vortex/encoding/DecimalBytePartsEncoding.java b/core/src/main/java/io/github/dfa1/vortex/encoding/DecimalBytePartsEncoding.java deleted file mode 100644 index 7024e37..0000000 --- a/core/src/main/java/io/github/dfa1/vortex/encoding/DecimalBytePartsEncoding.java +++ /dev/null @@ -1,105 +0,0 @@ -package io.github.dfa1.vortex.encoding; - -import com.google.protobuf.InvalidProtocolBufferException; -import io.github.dfa1.vortex.core.DType; -import io.github.dfa1.vortex.core.PType; -import io.github.dfa1.vortex.core.VortexException; -import io.github.dfa1.vortex.core.array.Array; -import io.github.dfa1.vortex.core.array.GenericArray; -import io.github.dfa1.vortex.proto.EncodingProtos; - -import java.lang.foreign.MemorySegment; -import java.nio.ByteBuffer; -import java.util.List; - -/// Decoder for {@code vortex.decimal_byte_parts} — decimal split into MSP + LSP children. -/// -///

Wire format (per Rust vtable): -///

    -///
  • Metadata: {@code DecimalBytePartsMetadata} — {@code zeroth_child_ptype PType} (tag 1), -/// {@code lower_part_count uint32} (tag 2, always 0 in current Rust impl) -///
  • Buffers: 0 -///
  • Children: 1 — {@code msp} (most-significant part), a signed primitive array carrying -/// the array's nullability; dtype is {@code Primitive(zeroth_child_ptype, parentNullability)} -///
-public final class DecimalBytePartsEncoding implements Encoding { - - /// Creates a new {@code DecimalBytePartsEncoding} instance. - public DecimalBytePartsEncoding() { - } - - @Override - public EncodingId encodingId() { - return EncodingId.VORTEX_DECIMAL_BYTE_PARTS; - } - - @Override - public boolean accepts(DType dtype) { - return dtype instanceof DType.Decimal; - } - - @Override - public EncodeResult encode(DType dtype, Object data) { - return Encoder.encode((DType.Decimal) dtype, (long[]) data); - } - - @Override - public Array decode(DecodeContext ctx) { - return Decoder.decode(ctx); - } - - private static final class Encoder { - - static EncodeResult encode(DType.Decimal dtype, long[] data) { - DType mspDtype = new DType.Primitive(PType.I64, dtype.nullable()); - EncodeResult mspResult = new PrimitiveEncoding().encode(mspDtype, data); - - EncodingProtos.DecimalBytePartsMetadata proto = EncodingProtos.DecimalBytePartsMetadata.newBuilder() - .setZerothChildPtype( - io.github.dfa1.vortex.proto.DTypeProtos.PType.forNumber(PType.I64.ordinal())) - .setLowerPartCount(0) - .build(); - ByteBuffer metaBuf = ByteBuffer.wrap(proto.toByteArray()); - - EncodeNode mspNode = EncodeNode.remapBufferIndices(mspResult.rootNode(), 0); - EncodeNode root = new EncodeNode( - EncodingId.VORTEX_DECIMAL_BYTE_PARTS, metaBuf, new EncodeNode[]{mspNode}, new int[]{}); - return new EncodeResult(root, List.copyOf(mspResult.buffers()), null, null); - } - } - - private static final class Decoder { - - private static Array decode(DecodeContext ctx) { - ByteBuffer meta = ctx.metadata(); - if (meta == null || meta.remaining() == 0) { - throw new VortexException(EncodingId.VORTEX_DECIMAL_BYTE_PARTS, "missing metadata"); - } - EncodingProtos.DecimalBytePartsMetadata decoded; - try { - byte[] bytes = new byte[meta.remaining()]; - meta.duplicate().get(bytes); - decoded = EncodingProtos.DecimalBytePartsMetadata.parseFrom(bytes); - } catch (InvalidProtocolBufferException e) { - throw new VortexException(EncodingId.VORTEX_DECIMAL_BYTE_PARTS, "invalid metadata: " + e.getMessage()); - } - - int lowerPartCount = decoded.getLowerPartCount(); - if (lowerPartCount != 0) { - throw new VortexException(EncodingId.VORTEX_DECIMAL_BYTE_PARTS, - "lower_part_count > 0 not supported, got " + lowerPartCount); - } - - PType mspPtype = PType.values()[decoded.getZerothChildPtypeValue()]; - boolean nullable = ctx.dtype().nullable(); - DType mspDtype = new DType.Primitive(mspPtype, nullable); - ArrayNode mspNode = ctx.node().children()[0]; - DecodeContext mspCtx = new DecodeContext( - mspNode, mspDtype, ctx.rowCount(), - ctx.segmentBuffers(), ctx.registry(), ctx.arena()); - Array mspArray = ctx.registry().decode(mspCtx); - return new GenericArray(ctx.dtype(), ctx.rowCount(), new MemorySegment[0], - new Array[]{mspArray}); - } - } -} diff --git a/core/src/main/java/io/github/dfa1/vortex/encoding/DecimalEncoding.java b/core/src/main/java/io/github/dfa1/vortex/encoding/DecimalEncoding.java deleted file mode 100644 index 228a549..0000000 --- a/core/src/main/java/io/github/dfa1/vortex/encoding/DecimalEncoding.java +++ /dev/null @@ -1,136 +0,0 @@ -package io.github.dfa1.vortex.encoding; - -import com.google.protobuf.InvalidProtocolBufferException; -import io.github.dfa1.vortex.core.DType; -import io.github.dfa1.vortex.core.VortexException; -import io.github.dfa1.vortex.core.array.Array; -import io.github.dfa1.vortex.core.array.GenericArray; -import io.github.dfa1.vortex.proto.EncodingProtos; - -import java.lang.foreign.MemorySegment; -import java.nio.ByteBuffer; -import java.util.List; - -/// Decoder for {@code vortex.decimal} — canonical flat decimal storage. -/// -///

Wire format (per Rust vtable): -///

    -///
  • Metadata: {@code DecimalMetadata} — {@code values_type int32} (tag 1): -/// DecimalType I8=0, I16=1, I32=2, I64=3, I128=4, I256=5 -///
  • Buffers: 1 — little-endian fixed-width integers sized per {@code values_type} -///
  • Children: 0 (non-nullable) or 1 (validity) -///
-public final class DecimalEncoding implements Encoding { - - /// Creates a new {@code DecimalEncoding} instance. - public DecimalEncoding() { - } - - @Override - public EncodingId encodingId() { - return EncodingId.VORTEX_DECIMAL; - } - - @Override - public boolean accepts(DType dtype) { - return dtype instanceof DType.Decimal; - } - - @Override - public EncodeResult encode(DType dtype, Object data) { - return Encoder.encode((DType.Decimal) dtype, (MemorySegment) data); - } - - @Override - public Array decode(DecodeContext ctx) { - return Decoder.decode(ctx); - } - - private static final class Encoder { - - static EncodeResult encode(DType.Decimal dtype, MemorySegment data) { - int valuesType = valuesType(dtype.precision()); - int bw = byteWidth(valuesType); - if (data.byteSize() % bw != 0) { - throw new VortexException(EncodingId.VORTEX_DECIMAL, - "buffer size %d not multiple of byteWidth %d".formatted(data.byteSize(), bw)); - } - ByteBuffer metaBuf = ByteBuffer.wrap( - EncodingProtos.DecimalMetadata.newBuilder().setValuesType(valuesType).build().toByteArray()); - EncodeNode node = new EncodeNode(EncodingId.VORTEX_DECIMAL, metaBuf, new EncodeNode[0], new int[]{0}); - return new EncodeResult(node, List.of(data), null, null); - } - - private static int valuesType(byte precision) { - if (precision <= 2) { - return 0; - } - if (precision <= 4) { - return 1; - } - if (precision <= 9) { - return 2; - } - if (precision <= 18) { - return 3; - } - if (precision <= 38) { - return 4; - } - return 5; - } - - private static int byteWidth(int valuesType) { - return switch (valuesType) { - case 0 -> 1; - case 1 -> 2; - case 2 -> 4; - case 3 -> 8; - case 4 -> 16; - case 5 -> 32; - default -> throw new VortexException(EncodingId.VORTEX_DECIMAL, - "unknown valuesType: " + valuesType); - }; - } - } - - private static final class Decoder { - - private static Array decode(DecodeContext ctx) { - ByteBuffer meta = ctx.metadata(); - if (meta == null || meta.remaining() == 0) { - throw new VortexException(EncodingId.VORTEX_DECIMAL, "missing metadata"); - } - EncodingProtos.DecimalMetadata decoded; - try { - byte[] bytes = new byte[meta.remaining()]; - meta.duplicate().get(bytes); - decoded = EncodingProtos.DecimalMetadata.parseFrom(bytes); - } catch (InvalidProtocolBufferException e) { - throw new VortexException(EncodingId.VORTEX_DECIMAL, "invalid metadata: " + e.getMessage()); - } - int valuesType = decoded.getValuesType(); - int byteWidth = decimalTypeByteWidth(valuesType); - MemorySegment buffer = ctx.buffer(0); - long expected = ctx.rowCount() * byteWidth; - if (buffer.byteSize() < expected) { - throw new VortexException(EncodingId.VORTEX_DECIMAL, - "buffer too small: expected %d bytes, got %d".formatted(expected, buffer.byteSize())); - } - return new GenericArray(ctx.dtype(), ctx.rowCount(), buffer); - } - - private static int decimalTypeByteWidth(int valuesType) { - return switch (valuesType) { - case 0 -> 1; // I8 - case 1 -> 2; // I16 - case 2 -> 4; // I32 - case 3 -> 8; // I64 - case 4 -> 16; // I128 - case 5 -> 32; // I256 - default -> throw new VortexException(EncodingId.VORTEX_DECIMAL, - "unknown DecimalType value: " + valuesType); - }; - } - } -} diff --git a/core/src/main/java/io/github/dfa1/vortex/encoding/DecodeContext.java b/core/src/main/java/io/github/dfa1/vortex/encoding/DecodeContext.java deleted file mode 100644 index ee5ed51..0000000 --- a/core/src/main/java/io/github/dfa1/vortex/encoding/DecodeContext.java +++ /dev/null @@ -1,55 +0,0 @@ -package io.github.dfa1.vortex.encoding; - -import io.github.dfa1.vortex.core.array.Array; -import io.github.dfa1.vortex.core.DType; - -import java.lang.foreign.MemorySegment; -import java.lang.foreign.SegmentAllocator; -import java.nio.ByteBuffer; - -/// Decoding context passed to each [Encoding]. -/// -/// Buffers are `MemorySegment` slices materialized from the file's segment table; -/// children are decoded recursively via [#decodeChild(int)]. -/// The arena is scoped to one chunk epoch — all decode output allocated from it is -/// valid until the next chunk is opened. -/// -/// @param node the array node describing this encoding's tree structure -/// @param dtype logical type expected for the decoded array -/// @param rowCount number of logical rows to decode -/// @param segmentBuffers all segment buffers for the current flat segment, indexed by segment position -/// @param registry encoding registry used for recursive child decoding -/// @param arena allocator for decode output; lifetime matches the current chunk epoch -public record DecodeContext( - ArrayNode node, - DType dtype, - long rowCount, - MemorySegment[] segmentBuffers, - EncodingRegistry registry, - SegmentAllocator arena -) { - /// Recursively decode child `i` using the same segment buffers, registry and arena. - /// - /// @param i zero-based child index within this node's children array - /// @return the decoded {@link Array} for child {@code i} - public Array decodeChild(int i) { - ArrayNode child = node.children()[i]; - var childCtx = new DecodeContext(child, dtype, rowCount, segmentBuffers, registry, arena); - return registry.decode(childCtx); - } - - /// Return the buffer at position `i` in this node's bufferIndices. - /// - /// @param i zero-based index into this node's {@code bufferIndices} array - /// @return the {@link MemorySegment} for the referenced segment buffer - public MemorySegment buffer(int i) { - return segmentBuffers[node.bufferIndices()[i]]; - } - - /// Returns the encoding-specific metadata bytes for this node, or {@code null} if absent. - /// - /// @return the metadata {@link java.nio.ByteBuffer}, or {@code null} - public ByteBuffer metadata() { - return node.metadata(); - } -} diff --git a/core/src/main/java/io/github/dfa1/vortex/encoding/DeltaEncoding.java b/core/src/main/java/io/github/dfa1/vortex/encoding/DeltaEncoding.java deleted file mode 100644 index e1e141a..0000000 --- a/core/src/main/java/io/github/dfa1/vortex/encoding/DeltaEncoding.java +++ /dev/null @@ -1,388 +0,0 @@ -package io.github.dfa1.vortex.encoding; - -import com.google.protobuf.InvalidProtocolBufferException; -import io.github.dfa1.vortex.proto.EncodingProtos; -import io.github.dfa1.vortex.proto.ScalarProtos; -import io.github.dfa1.vortex.core.ArrayStats; -import io.github.dfa1.vortex.core.DType; -import io.github.dfa1.vortex.core.PType; -import io.github.dfa1.vortex.core.VortexException; -import io.github.dfa1.vortex.core.array.Array; -import io.github.dfa1.vortex.core.array.ByteArray; -import io.github.dfa1.vortex.core.array.IntArray; -import io.github.dfa1.vortex.core.array.LongArray; -import io.github.dfa1.vortex.core.array.ShortArray; - -import java.lang.foreign.Arena; -import java.lang.foreign.MemorySegment; -import java.lang.foreign.SegmentAllocator; -import java.lang.foreign.ValueLayout; -import java.nio.ByteBuffer; -import java.util.List; - -/// Encoding for {@code fastlanes.delta} — FastLanes delta encoding for integer columns. -/// -/// Wire format matches the Rust reference implementation: -///
    -///
  • 0 buffers, 2 child slots: {@code bases} (child 0) and {@code deltas} (child 1).
    • -///
  • Metadata: protobuf {@code DeltaMetadata{deltas_len, offset}}.
    • -///
  • {@code bases} holds {@code (paddedLen / 1024) * LANES} elements of the column dtype.
    • -///
  • {@code deltas} holds {@code paddedLen} elements (padded to a multiple of 1024).
    • -///
-/// -/// Algorithm: FastLanes transpose → per-lane delta with wrapping arithmetic. -/// {@code LANES = 1024 / typeBits}, where {@code typeBits = byteSize * 8}. -public final class DeltaEncoding implements Encoding { - - /// Creates a new {@code DeltaEncoding} instance. - public DeltaEncoding() { - } - - static final int FL_CHUNK_SIZE = 1024; - static final int[] FL_ORDER = {0, 4, 2, 6, 1, 5, 3, 7}; - - @Override - public EncodingId encodingId() { - return EncodingId.FASTLANES_DELTA; - } - - @Override - public boolean accepts(DType dtype) { - if (!(dtype instanceof DType.Primitive p)) { - return false; - } - return switch (p.ptype()) { - case I8, I16, I32, I64, U8, U16, U32, U64 -> true; - default -> false; - }; - } - - @Override - public EncodeResult encode(DType dtype, Object data) { - return Encoder.encode(dtype, data); - } - - @Override - public Array decode(DecodeContext ctx) { - return Decoder.decode(ctx); - } - - // ── Shared helpers (FastLanes index + type math + buffer I/O) ──────────── - - static int transposeIndex(int idx) { - int lane = idx % 16; - int order = (idx / 16) % 8; - int row = idx / 128; - return lane * 64 + FL_ORDER[order] * 8 + row; - } - - static int iterateIndex(int row, int lane) { - int o = row / 8; - int s = row % 8; - return FL_ORDER[o] * 16 + s * 128 + lane; - } - - static int lanes(PType ptype) { - return FL_CHUNK_SIZE / (ptype.byteSize() * 8); - } - - static int typeBits(PType ptype) { - return ptype.byteSize() * 8; - } - - static long typeMask(PType ptype) { - int bits = ptype.byteSize() * 8; - return bits == 64 ? -1L : (1L << bits) - 1; - } - - static MemorySegment fromLongs(long[] longs, PType ptype, SegmentAllocator arena) { - if (ptype == PType.I64 || ptype == PType.U64) { - MemorySegment dst = arena.allocate((long) longs.length * 8); - MemorySegment.copy(MemorySegment.ofArray(longs), ValueLayout.JAVA_LONG, 0L, - dst, PTypeIO.LE_LONG, 0L, longs.length); - return dst; - } - int n = longs.length; - long elemSize = ptype.byteSize(); - MemorySegment seg = arena.allocate(n * elemSize); - for (int i = 0; i < n; i++) { - PTypeIO.set(seg, i * elemSize, ptype, longs[i]); - } - return seg; - } - - private static final class Encoder { - - static EncodeResult encode(DType dtype, Object data) { - PType ptype = ((DType.Primitive) dtype).ptype(); - long[] longs = toLongs(data, ptype); - int n = longs.length; - int typeBits = typeBits(ptype); - int lanes = lanes(ptype); - long mask = typeMask(ptype); - boolean unsign = isUnsigned(ptype); - - long minVal = 0L, maxVal = 0L; - if (n > 0) { - minVal = longs[0]; - maxVal = longs[0]; - for (int i = 1; i < n; i++) { - long v = longs[i]; - if (unsign ? Long.compareUnsigned(v, minVal) < 0 : v < minVal) { - minVal = v; - } - if (unsign ? Long.compareUnsigned(v, maxVal) > 0 : v > maxVal) { - maxVal = v; - } - } - } - - int numChunks = n == 0 ? 0 : (n + FL_CHUNK_SIZE - 1) / FL_CHUNK_SIZE; - long paddedLen = (long) numChunks * FL_CHUNK_SIZE; - int basesLen = numChunks * lanes; - - long[] basesAll = new long[basesLen]; - long[] deltasAll = new long[(int) paddedLen]; - long[] chunkBuf = new long[FL_CHUNK_SIZE]; - long[] transposed = new long[FL_CHUNK_SIZE]; - long[] chunkBases = new long[lanes]; - long[] chunkDelta = new long[FL_CHUNK_SIZE]; - - for (int chunk = 0; chunk < numChunks; chunk++) { - int start = chunk * FL_CHUNK_SIZE; - int end = Math.min(start + FL_CHUNK_SIZE, n); - for (int i = start; i < end; i++) { - chunkBuf[i - start] = longs[i] & mask; - } - for (int i = end - start; i < FL_CHUNK_SIZE; i++) { - chunkBuf[i] = 0L; - } - for (int i = 0; i < FL_CHUNK_SIZE; i++) { - transposed[i] = chunkBuf[transposeIndex(i)]; - } - int basesOff = chunk * lanes; - System.arraycopy(transposed, 0, basesAll, basesOff, lanes); - System.arraycopy(basesAll, basesOff, chunkBases, 0, lanes); - deltaChunk(transposed, chunkBases, lanes, typeBits, mask, chunkDelta); - System.arraycopy(chunkDelta, 0, deltasAll, chunk * FL_CHUNK_SIZE, FL_CHUNK_SIZE); - } - - MemorySegment basesSeg = fromLongs(basesAll, ptype, Arena.ofAuto()); - MemorySegment deltasSeg = fromLongs(deltasAll, ptype, Arena.ofAuto()); - - byte[] metaBytes = EncodingProtos.DeltaMetadata.newBuilder() - .setDeltasLen(paddedLen) - .setOffset(0) - .build() - .toByteArray(); - - byte[] statsMin = n > 0 ? statsBytes(ptype, minVal) : null; - byte[] statsMax = n > 0 ? statsBytes(ptype, maxVal) : null; - - EncodeNode basesNode = EncodeNode.leaf(EncodingId.VORTEX_PRIMITIVE, 0); - EncodeNode deltasNode = EncodeNode.leaf(EncodingId.VORTEX_PRIMITIVE, 1); - EncodeNode root = new EncodeNode(EncodingId.FASTLANES_DELTA, ByteBuffer.wrap(metaBytes), - new EncodeNode[]{basesNode, deltasNode}, new int[0]); - return new EncodeResult(root, List.of(basesSeg, deltasSeg), statsMin, statsMax); - } - - private static void deltaChunk(long[] transposed, long[] bases, int lanes, int typeBits, long mask, long[] out) { - for (int lane = 0; lane < lanes; lane++) { - long prev = bases[lane] & mask; - for (int row = 0; row < typeBits; row++) { - int idx = iterateIndex(row, lane); - long next = transposed[idx] & mask; - out[idx] = (next - prev) & mask; - prev = next; - } - } - } - - private static long[] toLongs(Object data, PType ptype) { - return switch (ptype) { - case I8 -> { - byte[] arr = (byte[]) data; - long[] r = new long[arr.length]; - for (int i = 0; i < arr.length; i++) { - r[i] = arr[i]; - } - yield r; - } - case U8 -> { - byte[] arr = (byte[]) data; - long[] r = new long[arr.length]; - for (int i = 0; i < arr.length; i++) { - r[i] = Byte.toUnsignedLong(arr[i]); - } - yield r; - } - case I16 -> { - short[] arr = (short[]) data; - long[] r = new long[arr.length]; - for (int i = 0; i < arr.length; i++) { - r[i] = arr[i]; - } - yield r; - } - case U16 -> { - short[] arr = (short[]) data; - long[] r = new long[arr.length]; - for (int i = 0; i < arr.length; i++) { - r[i] = Short.toUnsignedLong(arr[i]); - } - yield r; - } - case I32 -> { - int[] arr = (int[]) data; - long[] r = new long[arr.length]; - for (int i = 0; i < arr.length; i++) { - r[i] = arr[i]; - } - yield r; - } - case U32 -> { - int[] arr = (int[]) data; - long[] r = new long[arr.length]; - for (int i = 0; i < arr.length; i++) { - r[i] = Integer.toUnsignedLong(arr[i]); - } - yield r; - } - case I64, U64 -> (long[]) data; - default -> throw new VortexException(EncodingId.FASTLANES_DELTA, "unsupported ptype: " + ptype); - }; - } - - private static boolean isUnsigned(PType ptype) { - return switch (ptype) { - case U8, U16, U32, U64 -> true; - default -> false; - }; - } - - private static byte[] statsBytes(PType ptype, long value) { - if (isUnsigned(ptype)) { - return ScalarProtos.ScalarValue.newBuilder().setUint64Value(value).build().toByteArray(); - } - return ScalarProtos.ScalarValue.newBuilder().setInt64Value(value).build().toByteArray(); - } - } - - private static final class Decoder { - - static Array decode(DecodeContext ctx) { - ByteBuffer rawMeta = ctx.metadata(); - EncodingProtos.DeltaMetadata meta; - if (rawMeta == null || !rawMeta.hasRemaining()) { - meta = EncodingProtos.DeltaMetadata.getDefaultInstance(); - } else { - try { - meta = EncodingProtos.DeltaMetadata.parseFrom(rawMeta.duplicate()); - } catch (InvalidProtocolBufferException e) { - throw new VortexException(EncodingId.FASTLANES_DELTA, "invalid metadata", e); - } - } - - PType ptype = ((DType.Primitive) ctx.dtype()).ptype(); - long rowCount = ctx.rowCount(); - int typeBits = typeBits(ptype); - int lanes = lanes(ptype); - long mask = typeMask(ptype); - - long deltasLen = meta.getDeltasLen(); - int offset = (int) meta.getOffset(); - - if (deltasLen == 0L) { - MemorySegment empty = ctx.arena().allocate(0); - return switch (ptype) { - case I64, U64 -> new LongArray(ctx.dtype(), 0L, empty, ArrayStats.empty()); - case I32, U32 -> new IntArray(ctx.dtype(), 0L, empty, ArrayStats.empty()); - case I16, U16 -> new ShortArray(ctx.dtype(), 0L, empty, ArrayStats.empty()); - case I8, U8 -> new ByteArray(ctx.dtype(), 0L, empty, ArrayStats.empty()); - default -> throw new VortexException(EncodingId.FASTLANES_DELTA, "unsupported ptype: " + ptype); - }; - } - - long basesLen = (deltasLen / FL_CHUNK_SIZE) * lanes; - DType dtype = ctx.dtype(); - - Array basesArr = decodeChildAs(ctx, 0, dtype, basesLen); - Array deltasArr = decodeChildAs(ctx, 1, dtype, deltasLen); - - long[] basesAll = readLongs(basesArr.buffer(0), (int) basesLen, ptype); - long[] deltasAll = readLongs(deltasArr.buffer(0), (int) deltasLen, ptype); - - int numChunks = (int) (deltasLen / FL_CHUNK_SIZE); - long[] decoded = new long[(int) deltasLen]; - long[] untransposedChunk = new long[FL_CHUNK_SIZE]; - long[] chunkBases = new long[lanes]; - long[] chunkDeltas = new long[FL_CHUNK_SIZE]; - long[] chunkUndelta = new long[FL_CHUNK_SIZE]; - - for (int chunk = 0; chunk < numChunks; chunk++) { - int basesOff = chunk * lanes; - int deltaOff = chunk * FL_CHUNK_SIZE; - - System.arraycopy(basesAll, basesOff, chunkBases, 0, lanes); - System.arraycopy(deltasAll, deltaOff, chunkDeltas, 0, FL_CHUNK_SIZE); - - undeltaChunk(chunkDeltas, chunkBases, lanes, typeBits, mask, chunkUndelta); - - for (int i = 0; i < FL_CHUNK_SIZE; i++) { - untransposedChunk[transposeIndex(i)] = chunkUndelta[i]; - } - System.arraycopy(untransposedChunk, 0, decoded, deltaOff, FL_CHUNK_SIZE); - } - - long[] result = new long[(int) rowCount]; - System.arraycopy(decoded, offset, result, 0, (int) rowCount); - - MemorySegment seg = fromLongs(result, ptype, ctx.arena()); - return switch (ptype) { - case I64, U64 -> new LongArray(ctx.dtype(), rowCount, seg, ArrayStats.empty()); - case I32, U32 -> new IntArray(ctx.dtype(), rowCount, seg, ArrayStats.empty()); - case I16, U16 -> new ShortArray(ctx.dtype(), rowCount, seg, ArrayStats.empty()); - case I8, U8 -> new ByteArray(ctx.dtype(), rowCount, seg, ArrayStats.empty()); - default -> throw new VortexException(EncodingId.FASTLANES_DELTA, "unsupported ptype: " + ptype); - }; - } - - private static void undeltaChunk(long[] deltas, long[] bases, int lanes, int typeBits, long mask, long[] out) { - for (int lane = 0; lane < lanes; lane++) { - long prev = bases[lane] & mask; - for (int row = 0; row < typeBits; row++) { - int idx = iterateIndex(row, lane); - long next = ((deltas[idx] & mask) + prev) & mask; - out[idx] = next; - prev = next; - } - } - } - - private static long[] readLongs(MemorySegment buf, int count, PType ptype) { - long[] out = new long[count]; - int elemSize = ptype.byteSize(); - for (int i = 0; i < count; i++) { - long off = (long) i * elemSize; - out[i] = switch (ptype) { - case I8 -> buf.get(ValueLayout.JAVA_BYTE, off); - case U8 -> Byte.toUnsignedLong(buf.get(ValueLayout.JAVA_BYTE, off)); - case I16 -> buf.get(PTypeIO.LE_SHORT, off); - case U16 -> Short.toUnsignedLong(buf.get(PTypeIO.LE_SHORT, off)); - case I32 -> buf.get(PTypeIO.LE_INT, off); - case U32 -> Integer.toUnsignedLong(buf.get(PTypeIO.LE_INT, off)); - case I64, U64 -> buf.get(PTypeIO.LE_LONG, off); - default -> throw new VortexException(EncodingId.FASTLANES_DELTA, "unsupported ptype: " + ptype); - }; - } - return out; - } - - private static Array decodeChildAs(DecodeContext parent, int childIdx, DType dtype, long rowCount) { - ArrayNode childNode = parent.node().children()[childIdx]; - DecodeContext childCtx = new DecodeContext( - childNode, dtype, rowCount, parent.segmentBuffers(), parent.registry(), parent.arena()); - return parent.registry().decode(childCtx); - } - } -} diff --git a/core/src/main/java/io/github/dfa1/vortex/encoding/DictEncoding.java b/core/src/main/java/io/github/dfa1/vortex/encoding/DictEncoding.java deleted file mode 100644 index 4fa402a..0000000 --- a/core/src/main/java/io/github/dfa1/vortex/encoding/DictEncoding.java +++ /dev/null @@ -1,620 +0,0 @@ -package io.github.dfa1.vortex.encoding; - -import com.google.protobuf.InvalidProtocolBufferException; -import io.github.dfa1.vortex.proto.EncodingProtos; -import io.github.dfa1.vortex.proto.ScalarProtos; -import io.github.dfa1.vortex.core.array.Array; -import io.github.dfa1.vortex.core.ArrayStats; -import io.github.dfa1.vortex.core.DType; -import io.github.dfa1.vortex.core.PType; -import io.github.dfa1.vortex.core.array.ByteArray; -import io.github.dfa1.vortex.core.array.DoubleArray; -import io.github.dfa1.vortex.core.array.FloatArray; -import io.github.dfa1.vortex.core.array.IntArray; -import io.github.dfa1.vortex.core.array.LongArray; -import io.github.dfa1.vortex.core.array.ShortArray; -import io.github.dfa1.vortex.core.array.VarBinArray; -import io.github.dfa1.vortex.core.VortexException; - -import java.lang.foreign.Arena; -import java.lang.foreign.MemorySegment; -import java.lang.foreign.ValueLayout; -import java.nio.ByteBuffer; -import java.nio.charset.StandardCharsets; -import java.util.LinkedHashMap; -import java.util.List; - -/// Encoding for `vortex.dict` — dictionary encoding for low-cardinality columns. -/// -/// Primitive: `values_buffer(unique_values)` + `codes_buffer`. Metadata (1 byte): code PType ordinal. -/// Node tree: DictNode{ children=[ValuesNode{buf=0},CodesNode{buf=1}] }. -/// -/// Utf8: buf 0 = dict bytes, buf 1 = dict offsets (I64 LE, n_unique+1), buf 2 = codes. -/// Metadata (2 bytes): codePType ordinal, I64 ordinal. Flat node (no children), bufferIndices=[0,1,2]. -/// Decoded as a {@link io.github.dfa1.vortex.core.array.VarBinArray} in dict mode — no string materialization at decode time. -public final class DictEncoding implements Encoding { - - /// Creates a new {@code DictEncoding} instance. - public DictEncoding() { - } - - @Override - public EncodingId encodingId() { - return EncodingId.VORTEX_DICT; - } - - @Override - public boolean accepts(DType dtype) { - return dtype instanceof DType.Primitive || dtype instanceof DType.Utf8; - } - - @Override - public EncodeResult encode(DType dtype, Object data) { - return Encoder.encode(dtype, data); - } - - @Override - public CascadeStep encodeCascade(DType dtype, Object data, CompressorContext ctx) { - return Encoder.encodeCascade(dtype, data); - } - - @Override - public Array decode(DecodeContext ctx) { - return Decoder.decode(ctx); - } - - private static final class Encoder { - - private record DictData(MemorySegment valuesBuf, Object codesArr, PType codePType, int len) {} - - private static EncodeResult encode(DType dtype, Object data) { - if (dtype instanceof DType.Utf8) { - return encodeUtf8((String[]) data, dtype); - } - DictData d = buildDictData(dtype, data); - PType codePType = d.codePType(); - int codeBytes = codePType.byteSize(); - - MemorySegment codesBuf = Arena.ofAuto().allocate((long) d.len() * codeBytes); - for (int i = 0; i < d.len(); i++) { - writeCodeToSeg(codesBuf, codePType, i, readCodeFromArr(d.codesArr(), codePType, i)); - } - - ByteBuffer meta = ByteBuffer.allocate(1).put(0, (byte) codePType.ordinal()); - EncodeNode valuesNode = EncodeNode.leaf(EncodingId.VORTEX_PRIMITIVE, 0); - EncodeNode codesNode = EncodeNode.leaf(EncodingId.VORTEX_PRIMITIVE, 1); - EncodeNode rootNode = new EncodeNode( - EncodingId.VORTEX_DICT, meta, - new EncodeNode[]{valuesNode, codesNode}, - new int[0]); - - return new EncodeResult(rootNode, List.of(d.valuesBuf(), codesBuf), null, null); - } - - private static CascadeStep encodeCascade(DType dtype, Object data) { - if (dtype instanceof DType.Utf8) { - return CascadeStep.terminal(encodeUtf8((String[]) data, dtype)); - } - DictData d = buildDictData(dtype, data); - PType codePType = d.codePType(); - - ByteBuffer meta = ByteBuffer.allocate(1).put(0, (byte) codePType.ordinal()); - EncodeNode valuesNode = EncodeNode.leaf(EncodingId.VORTEX_PRIMITIVE, 0); - EncodeNode partialRoot = new EncodeNode( - EncodingId.VORTEX_DICT, meta, - new EncodeNode[]{valuesNode, null}, - new int[0]); - - DType codesDtype = new DType.Primitive(codePType, false); - ChildSlot slot = new ChildSlot(codesDtype, d.codesArr(), 1); - return new CascadeStep(partialRoot, List.of(d.valuesBuf()), List.of(slot), null, null, true); - } - - private static EncodeResult encodeUtf8(String[] strings, DType dtype) { - int n = strings.length; - - var valueMap = new LinkedHashMap(); - for (String s : strings) { - valueMap.computeIfAbsent(s, k -> valueMap.size()); - } - - int dictSize = valueMap.size(); - PType codePType = codePType(dictSize); - int codeBytes = codePType.byteSize(); - - byte[][] dictByteArrays = new byte[dictSize][]; - int j = 0; - long totalDictBytes = 0; - for (String s : valueMap.keySet()) { - dictByteArrays[j] = s.getBytes(StandardCharsets.UTF_8); - totalDictBytes += dictByteArrays[j].length; - j++; - } - - Arena arena = Arena.ofAuto(); - MemorySegment dictBytesBuf = arena.allocate(totalDictBytes > 0 ? totalDictBytes : 1); - MemorySegment dictOffsetsBuf = arena.allocate((long) (dictSize + 1) * Long.BYTES, Long.BYTES); - - long pos = 0; - dictOffsetsBuf.setAtIndex(PTypeIO.LE_LONG, 0, 0L); - for (int i = 0; i < dictSize; i++) { - MemorySegment.copy(MemorySegment.ofArray(dictByteArrays[i]), 0, dictBytesBuf, pos, dictByteArrays[i].length); - pos += dictByteArrays[i].length; - dictOffsetsBuf.setAtIndex(PTypeIO.LE_LONG, i + 1, pos); - } - - MemorySegment codesBuf = Arena.ofAuto().allocate((long) n * codeBytes); - for (int i = 0; i < n; i++) { - writeCodeToSeg(codesBuf, codePType, i, valueMap.get(strings[i])); - } - - byte[] metaBytes = EncodingProtos.DictMetadata.newBuilder() - .setValuesLen(dictSize) - .setCodesPtype(io.github.dfa1.vortex.proto.DTypeProtos.PType.forNumber(codePType.ordinal())) - .build() - .toByteArray(); - - byte[] varBinMetaBytes = EncodingProtos.VarBinMetadata.newBuilder() - .setOffsetsPtype(io.github.dfa1.vortex.proto.DTypeProtos.PType.forNumber(PType.I64.ordinal())) - .build() - .toByteArray(); - - // Rust layout: children[0]=codes, children[1]=values(VarBin) - EncodeNode offsetsNode = EncodeNode.leaf(EncodingId.VORTEX_PRIMITIVE, 1); - EncodeNode valuesNode = new EncodeNode(EncodingId.VORTEX_VARBIN, - ByteBuffer.wrap(varBinMetaBytes), - new EncodeNode[]{offsetsNode}, - new int[]{0}); - EncodeNode codesNode = EncodeNode.leaf(EncodingId.VORTEX_PRIMITIVE, 2); - EncodeNode root = new EncodeNode( - EncodingId.VORTEX_DICT, ByteBuffer.wrap(metaBytes), - new EncodeNode[]{codesNode, valuesNode}, - new int[0]); - - String minStr = valueMap.keySet().stream().min(String::compareTo).orElse(null); - String maxStr = valueMap.keySet().stream().max(String::compareTo).orElse(null); - byte[] statsMin = minStr != null - ? ScalarProtos.ScalarValue.newBuilder().setStringValue(minStr).build().toByteArray() : null; - byte[] statsMax = maxStr != null - ? ScalarProtos.ScalarValue.newBuilder().setStringValue(maxStr).build().toByteArray() : null; - return new EncodeResult(root, List.of(dictBytesBuf, dictOffsetsBuf, codesBuf), statsMin, statsMax); - } - - private static DictData buildDictData(DType dtype, Object data) { - PType ptype = ((DType.Primitive) dtype).ptype(); - var valueMap = new LinkedHashMap(); - int len = arrayLength(data, ptype); - for (int i = 0; i < len; i++) { - Object v = readElement(data, ptype, i); - valueMap.computeIfAbsent(v, k -> valueMap.size()); - } - - int dictSize = valueMap.size(); - PType codePType = codePType(dictSize); - int codeBytes = codePType.byteSize(); - - Object uniqueArray = buildUniqueArray(ptype, valueMap.keySet(), dictSize); - MemorySegment valuesBuf = PTypeIO.copyArray(ptype, uniqueArray, dictSize); - - MemorySegment codesBuf = Arena.ofAuto().allocate((long) len * codeBytes); - for (int i = 0; i < len; i++) { - Object v = readElement(data, ptype, i); - int code = valueMap.get(v); - writeCodeToSeg(codesBuf, codePType, i, code); - } - - // Native array form needed for cascading (downstream encoder receives typed array) - Object codesArr = switch (codePType) { - case U8 -> { - byte[] a = new byte[len]; - for (int i = 0; i < len; i++) { - a[i] = codesBuf.get(ValueLayout.JAVA_BYTE, i); - } - yield a; - } - case U16 -> { - short[] a = new short[len]; - for (int i = 0; i < len; i++) { - a[i] = codesBuf.get(PTypeIO.LE_SHORT, (long) i * 2); - } - yield a; - } - default -> { - int[] a = new int[len]; - for (int i = 0; i < len; i++) { - a[i] = codesBuf.get(PTypeIO.LE_INT, (long) i * 4); - } - yield a; - } - }; - return new DictData(valuesBuf, codesArr, codePType, len); - } - - private static PType codePType(int dictSize) { - if (dictSize <= 256) { - return PType.U8; - } - if (dictSize <= 65536) { - return PType.U16; - } - return PType.U32; - } - - private static int arrayLength(Object data, PType ptype) { - return switch (ptype) { - case I8, U8 -> ((byte[]) data).length; - case I16, U16 -> ((short[]) data).length; - case I32, U32 -> ((int[]) data).length; - case I64, U64 -> ((long[]) data).length; - case F32 -> ((float[]) data).length; - case F64 -> ((double[]) data).length; - case F16 -> ((short[]) data).length; - }; - } - - private static Object readElement(Object data, PType ptype, int i) { - return switch (ptype) { - case I8, U8 -> ((byte[]) data)[i]; - case I16, U16, F16 -> ((short[]) data)[i]; - case I32, U32 -> ((int[]) data)[i]; - case I64, U64 -> ((long[]) data)[i]; - case F32 -> ((float[]) data)[i]; - case F64 -> ((double[]) data)[i]; - }; - } - - private static Object buildUniqueArray(PType ptype, Iterable uniques, int dictSize) { - return switch (ptype) { - case I8, U8 -> { - byte[] a = new byte[dictSize]; - int i = 0; - for (Object v : uniques) { - a[i++] = (Byte) v; - } - yield a; - } - case I16, U16 -> { - short[] a = new short[dictSize]; - int i = 0; - for (Object v : uniques) { - a[i++] = (Short) v; - } - yield a; - } - case I32, U32 -> { - int[] a = new int[dictSize]; - int i = 0; - for (Object v : uniques) { - a[i++] = (Integer) v; - } - yield a; - } - case I64, U64 -> { - long[] a = new long[dictSize]; - int i = 0; - for (Object v : uniques) { - a[i++] = (Long) v; - } - yield a; - } - case F32 -> { - float[] a = new float[dictSize]; - int i = 0; - for (Object v : uniques) { - a[i++] = (Float) v; - } - yield a; - } - case F64 -> { - double[] a = new double[dictSize]; - int i = 0; - for (Object v : uniques) { - a[i++] = (Double) v; - } - yield a; - } - case F16 -> { - short[] a = new short[dictSize]; - int i = 0; - for (Object v : uniques) { - a[i++] = (Short) v; - } - yield a; - } - }; - } - - private static void writeCodeToSeg(MemorySegment seg, PType codePType, int idx, int code) { - switch (codePType) { - case U8 -> seg.set(ValueLayout.JAVA_BYTE, idx, (byte) code); - case U16 -> seg.set(PTypeIO.LE_SHORT, (long) idx * 2, (short) code); - case U32 -> seg.set(PTypeIO.LE_INT, (long) idx * 4, code); - default -> throw new VortexException(EncodingId.VORTEX_DICT, "unexpected code type: " + codePType); - } - } - - private static int readCodeFromArr(Object arr, PType codePType, int i) { - return switch (codePType) { - case U8 -> Byte.toUnsignedInt(((byte[]) arr)[i]); - case U16 -> Short.toUnsignedInt(((short[]) arr)[i]); - default -> ((int[]) arr)[i]; - }; - } - } - - private static final class Decoder { - - private static Array decode(DecodeContext ctx) { - ByteBuffer meta = ctx.metadata(); - - if (ctx.dtype() instanceof DType.Utf8) { - if (ctx.node().children().length == 0) { - if (meta == null || !meta.hasRemaining()) { - throw new VortexException(EncodingId.VORTEX_DICT, "missing metadata for legacy utf8 dict"); - } - return decodeUtf8DictLegacy(ctx, meta); - } - if (meta == null || !meta.hasRemaining()) { - throw new VortexException(EncodingId.VORTEX_DICT, "missing metadata for utf8 dict"); - } - return decodeUtf8DictProto(ctx, meta.duplicate()); - } - - if (meta == null || !meta.hasRemaining()) { - throw new VortexException(EncodingId.VORTEX_DICT, "missing metadata"); - } - - // 1-byte = legacy Java format; multi-byte = Rust proto format - if (meta.remaining() == 1) { - return decodeLegacyJava(ctx, meta.get(0)); - } - return decodeRustProto(ctx, meta.duplicate()); - } - - private static Array decodeLegacyJava(DecodeContext ctx, byte codeTypeByte) { - PType codePType = PType.values()[Byte.toUnsignedInt(codeTypeByte)]; - PType valPType = ((DType.Primitive) ctx.dtype()).ptype(); - int elemSize = valPType.byteSize(); - long rowCount = ctx.rowCount(); - - // Values: always VORTEX_PRIMITIVE leaf, read direct - MemorySegment valuesBuf = ctx.segmentBuffers()[ctx.node().children()[0].bufferIndices()[0]]; - - // Codes: decode through registry — supports both raw (VORTEX_PRIMITIVE) and cascade (FASTLANES_BITPACKED) children - DType codesDtype = new DType.Primitive(codePType, false); - Array codesArr = decodeChildAs(ctx, 1, codesDtype, rowCount); - MemorySegment codesBuf = codesArr.buffer(0); - - MemorySegment out = ctx.arena().allocate(rowCount * (long) elemSize); - switch (codePType) { - case U8 -> expandU8(codesBuf, valuesBuf, out, rowCount, elemSize); - case U16 -> expandU16(codesBuf, valuesBuf, out, rowCount, elemSize); - case U32 -> expandU32(codesBuf, valuesBuf, out, rowCount, elemSize); - default -> { - for (long i = 0; i < rowCount; i++) { - long code = readCode(codesBuf, codePType, i); - MemorySegment.copy(valuesBuf, code * elemSize, out, i * elemSize, elemSize); - } - } - } - return typedArray(ctx.dtype(), valPType, rowCount, out.asReadOnly()); - } - - private static Array decodeRustProto(DecodeContext ctx, ByteBuffer metaBuf) { - EncodingProtos.DictMetadata meta; - try { - meta = EncodingProtos.DictMetadata.parseFrom(metaBuf); - } catch (InvalidProtocolBufferException e) { - throw new VortexException(EncodingId.VORTEX_DICT, "invalid proto metadata", e); - } - - PType codePType = PType.values()[meta.getCodesPtype().getNumber()]; - long valuesLen = meta.getValuesLen(); - long rowCount = ctx.rowCount(); - PType valPType = ((DType.Primitive) ctx.dtype()).ptype(); - int elemSize = valPType.byteSize(); - - // Rust layout: children[0]=codes, children[1]=values - DType codesDtype = new DType.Primitive(codePType, false); - Array codesArr = decodeChildAs(ctx, 0, codesDtype, rowCount); - Array valuesArr = decodeChildAs(ctx, 1, ctx.dtype(), valuesLen); - - MemorySegment codesBuf = codesArr.buffer(0); - MemorySegment valuesBuf = valuesArr.buffer(0); - - MemorySegment out = ctx.arena().allocate(rowCount * (long) elemSize); - // Loop-unswitch: pull the codePType switch outside the hot loop so the JIT - // sees a tight, type-specific loop with predictable branches. - switch (codePType) { - case U8 -> expandU8(codesBuf, valuesBuf, out, rowCount, elemSize); - case U16 -> expandU16(codesBuf, valuesBuf, out, rowCount, elemSize); - case U32 -> expandU32(codesBuf, valuesBuf, out, rowCount, elemSize); - default -> throw new VortexException(EncodingId.VORTEX_DICT, "unexpected code type: " + codePType); - } - return typedArray(ctx.dtype(), valPType, rowCount, out.asReadOnly()); - } - - private static Array decodeUtf8DictLegacy(DecodeContext ctx, ByteBuffer meta) { - PType codePType = PType.values()[Byte.toUnsignedInt(meta.get(0))]; - long n = ctx.rowCount(); - - MemorySegment dictBytes = ctx.buffer(0); - MemorySegment dictOffsets = ctx.buffer(1); - MemorySegment codes = ctx.buffer(2); - - return VarBinArray.ofDict(ctx.dtype(), n, - dictBytes, dictOffsets, PType.I64, - codes, codePType, - ArrayStats.empty()); - } - - private static Array decodeUtf8DictProto(DecodeContext ctx, ByteBuffer metaBuf) { - // DictMetadata proto (field 1=values_len, field 2=codes_ptype) — same message Rust uses. - EncodingProtos.DictMetadata meta; - try { - meta = EncodingProtos.DictMetadata.parseFrom(metaBuf); - } catch (com.google.protobuf.InvalidProtocolBufferException e) { - throw new VortexException(EncodingId.VORTEX_DICT, "invalid utf8 dict proto metadata", e); - } - PType codePType = PType.values()[meta.getCodesPtype().getNumber()]; - long dictSize = meta.getValuesLen(); - long n = ctx.rowCount(); - - // Rust layout: children[0]=codes, children[1]=values(VarBin) - DType codesDtype = new DType.Primitive(codePType, false); - Array codesArr = decodeChildAs(ctx, 0, codesDtype, n); - MemorySegment codesBuf = codesArr.buffer(0); - - Array valuesArr = decodeChildAs(ctx, 1, ctx.dtype(), dictSize); - MemorySegment dictBytes = valuesArr.buffer(0); - MemorySegment dictOffsets = valuesArr.child(0).buffer(0); - - return VarBinArray.ofDict(ctx.dtype(), n, - dictBytes, dictOffsets, PType.I64, - codesBuf, codePType, - ArrayStats.empty()); - } - - private static Array decodeChildAs(DecodeContext parent, int childIdx, DType dtype, long rowCount) { - ArrayNode childNode = parent.node().children()[childIdx]; - DecodeContext childCtx = new DecodeContext( - childNode, dtype, rowCount, parent.segmentBuffers(), parent.registry(), parent.arena()); - return parent.registry().decode(childCtx); - } - - private static long readCode(MemorySegment buf, PType codePType, long i) { - return switch (codePType) { - case U8 -> Byte.toUnsignedLong(buf.get(ValueLayout.JAVA_BYTE, i)); - case U16 -> Short.toUnsignedLong(buf.get(PTypeIO.LE_SHORT, i * 2)); - case U32 -> Integer.toUnsignedLong(buf.get(PTypeIO.LE_INT, i * 4)); - default -> throw new VortexException(EncodingId.VORTEX_DICT, "unexpected code type: " + codePType); - }; - } - - private static void expandU8( - MemorySegment codes, MemorySegment values, MemorySegment out, - long rowCount, int elemSize - ) { - switch (elemSize) { - case 8 -> { - for (long i = 0; i < rowCount; i++) { - long code = Byte.toUnsignedLong(codes.get(ValueLayout.JAVA_BYTE, i)); - out.setAtIndex(PTypeIO.LE_LONG, i, values.getAtIndex(PTypeIO.LE_LONG, code)); - } - } - case 4 -> { - for (long i = 0; i < rowCount; i++) { - long code = Byte.toUnsignedLong(codes.get(ValueLayout.JAVA_BYTE, i)); - out.setAtIndex(PTypeIO.LE_INT, i, values.getAtIndex(PTypeIO.LE_INT, code)); - } - } - case 2 -> { - for (long i = 0; i < rowCount; i++) { - long code = Byte.toUnsignedLong(codes.get(ValueLayout.JAVA_BYTE, i)); - out.setAtIndex(PTypeIO.LE_SHORT, i, values.getAtIndex(PTypeIO.LE_SHORT, code)); - } - } - case 1 -> { - for (long i = 0; i < rowCount; i++) { - long code = Byte.toUnsignedLong(codes.get(ValueLayout.JAVA_BYTE, i)); - out.set(ValueLayout.JAVA_BYTE, i, values.get(ValueLayout.JAVA_BYTE, code)); - } - } - default -> { - for (long i = 0, outOff = 0; i < rowCount; i++, outOff += elemSize) { - long code = Byte.toUnsignedLong(codes.get(ValueLayout.JAVA_BYTE, i)); - MemorySegment.copy(values, code * elemSize, out, outOff, elemSize); - } - } - } - } - - private static void expandU16( - MemorySegment codes, MemorySegment values, MemorySegment out, - long rowCount, int elemSize - ) { - switch (elemSize) { - case 8 -> { - for (long i = 0; i < rowCount; i++) { - long code = Short.toUnsignedLong(codes.get(PTypeIO.LE_SHORT, i * 2)); - out.setAtIndex(PTypeIO.LE_LONG, i, values.getAtIndex(PTypeIO.LE_LONG, code)); - } - } - case 4 -> { - for (long i = 0; i < rowCount; i++) { - long code = Short.toUnsignedLong(codes.get(PTypeIO.LE_SHORT, i * 2)); - out.setAtIndex(PTypeIO.LE_INT, i, values.getAtIndex(PTypeIO.LE_INT, code)); - } - } - case 2 -> { - for (long i = 0; i < rowCount; i++) { - long code = Short.toUnsignedLong(codes.get(PTypeIO.LE_SHORT, i * 2)); - out.setAtIndex(PTypeIO.LE_SHORT, i, values.getAtIndex(PTypeIO.LE_SHORT, code)); - } - } - case 1 -> { - for (long i = 0; i < rowCount; i++) { - long code = Short.toUnsignedLong(codes.get(PTypeIO.LE_SHORT, i * 2)); - out.set(ValueLayout.JAVA_BYTE, i, values.get(ValueLayout.JAVA_BYTE, code)); - } - } - default -> { - for (long i = 0, outOff = 0; i < rowCount; i++, outOff += elemSize) { - long code = Short.toUnsignedLong(codes.get(PTypeIO.LE_SHORT, i * 2)); - MemorySegment.copy(values, code * elemSize, out, outOff, elemSize); - } - } - } - } - - private static void expandU32( - MemorySegment codes, MemorySegment values, MemorySegment out, - long rowCount, int elemSize - ) { - switch (elemSize) { - case 8 -> { - for (long i = 0; i < rowCount; i++) { - long code = Integer.toUnsignedLong(codes.get(PTypeIO.LE_INT, i * 4)); - out.setAtIndex(PTypeIO.LE_LONG, i, values.getAtIndex(PTypeIO.LE_LONG, code)); - } - } - case 4 -> { - for (long i = 0; i < rowCount; i++) { - long code = Integer.toUnsignedLong(codes.get(PTypeIO.LE_INT, i * 4)); - out.setAtIndex(PTypeIO.LE_INT, i, values.getAtIndex(PTypeIO.LE_INT, code)); - } - } - case 2 -> { - for (long i = 0; i < rowCount; i++) { - long code = Integer.toUnsignedLong(codes.get(PTypeIO.LE_INT, i * 4)); - out.setAtIndex(PTypeIO.LE_SHORT, i, values.getAtIndex(PTypeIO.LE_SHORT, code)); - } - } - case 1 -> { - for (long i = 0; i < rowCount; i++) { - long code = Integer.toUnsignedLong(codes.get(PTypeIO.LE_INT, i * 4)); - out.set(ValueLayout.JAVA_BYTE, i, values.get(ValueLayout.JAVA_BYTE, code)); - } - } - default -> { - for (long i = 0, outOff = 0; i < rowCount; i++, outOff += elemSize) { - long code = Integer.toUnsignedLong(codes.get(PTypeIO.LE_INT, i * 4)); - MemorySegment.copy(values, code * elemSize, out, outOff, elemSize); - } - } - } - } - - private static Array typedArray(DType dtype, PType ptype, long n, MemorySegment seg) { - return switch (ptype) { - case I64, U64 -> new LongArray(dtype, n, seg, ArrayStats.empty()); - case I32, U32 -> new IntArray(dtype, n, seg, ArrayStats.empty()); - case F64 -> new DoubleArray(dtype, n, seg, ArrayStats.empty()); - case F32 -> new FloatArray(dtype, n, seg, ArrayStats.empty()); - case I16, U16 -> new ShortArray(dtype, n, seg, ArrayStats.empty()); - case I8, U8 -> new ByteArray(dtype, n, seg, ArrayStats.empty()); - default -> throw new VortexException(EncodingId.VORTEX_DICT, "unsupported ptype " + ptype); - }; - } - } -} diff --git a/core/src/main/java/io/github/dfa1/vortex/encoding/EncodeNode.java b/core/src/main/java/io/github/dfa1/vortex/encoding/EncodeNode.java deleted file mode 100644 index 4c0b80b..0000000 --- a/core/src/main/java/io/github/dfa1/vortex/encoding/EncodeNode.java +++ /dev/null @@ -1,48 +0,0 @@ -package io.github.dfa1.vortex.encoding; - -import java.nio.ByteBuffer; - -/// Describes the ArrayNode tree written into a flat segment's FlatBuffer. -/// Mirrors [ArrayNode] for the encode path. -/// -/// @param encodingId encoding id for this node -/// @param metadata optional encoding-specific metadata bytes, or {@code null} -/// @param children child encode nodes (empty for leaf nodes) -/// @param bufferIndices indices into the flat buffer list for data owned by this node -public record EncodeNode( - EncodingId encodingId, - ByteBuffer metadata, - EncodeNode[] children, - int[] bufferIndices -) { - /// Creates a leaf node with a single buffer and no children or metadata. - /// - /// @param encodingId the encoding identifier for this leaf node - /// @param bufferIndex index into the flat buffer list for the data buffer - /// @return a leaf {@link EncodeNode} referencing the given buffer - public static EncodeNode leaf(EncodingId encodingId, int bufferIndex) { - return new EncodeNode(encodingId, null, new EncodeNode[0], new int[]{bufferIndex}); - } - - /// Shift all buffer indices in this node and its descendants by {@code offset}. - /// - /// @param node the root node whose buffer indices to remap - /// @param offset the value to add to every buffer index in the subtree - /// @return a new {@link EncodeNode} tree with all buffer indices shifted by {@code offset} - public static EncodeNode remapBufferIndices(EncodeNode node, int offset) { - if (offset == 0) { - return node; - } - int[] oldIdx = node.bufferIndices(); - int[] newIdx = new int[oldIdx.length]; - for (int i = 0; i < oldIdx.length; i++) { - newIdx[i] = oldIdx[i] + offset; - } - EncodeNode[] oldChildren = node.children(); - EncodeNode[] newChildren = new EncodeNode[oldChildren.length]; - for (int i = 0; i < oldChildren.length; i++) { - newChildren[i] = remapBufferIndices(oldChildren[i], offset); - } - return new EncodeNode(node.encodingId(), node.metadata(), newChildren, newIdx); - } -} diff --git a/core/src/main/java/io/github/dfa1/vortex/encoding/EncodeResult.java b/core/src/main/java/io/github/dfa1/vortex/encoding/EncodeResult.java deleted file mode 100644 index a8c4a1d..0000000 --- a/core/src/main/java/io/github/dfa1/vortex/encoding/EncodeResult.java +++ /dev/null @@ -1,44 +0,0 @@ -package io.github.dfa1.vortex.encoding; - -import java.lang.foreign.MemorySegment; -import java.util.List; - -/// Output of encoding an array to bytes for one flat segment. -/// -/// @param rootNode the root encode node describing the encoding tree structure -/// @param buffers flat list of data buffers in the order referenced by {@code rootNode} -/// @param statsMin serialised minimum value bytes for zone-map pruning, or {@code null} -/// @param statsMax serialised maximum value bytes for zone-map pruning, or {@code null} -public record EncodeResult( - EncodeNode rootNode, - List buffers, - byte[] statsMin, - byte[] statsMax -) { - /// Convenience factory for single-buffer leaf encodings with stats. - /// - /// @param encodingId the encoding identifier for the leaf node - /// @param data the single data buffer - /// @param min serialised minimum stat bytes, or {@code null} - /// @param max serialised maximum stat bytes, or {@code null} - /// @return an {@link EncodeResult} backed by a single-buffer leaf node - public static EncodeResult simple(EncodingId encodingId, MemorySegment data, byte[] min, byte[] max) { - return new EncodeResult(EncodeNode.leaf(encodingId, 0), List.of(data), min, max); - } - - /// Convenience factory for single-buffer leaf encodings without stats. - /// - /// @param encodingId the encoding identifier for the leaf node - /// @param data the single data buffer - /// @return an {@link EncodeResult} backed by a single-buffer leaf node with no stats - public static EncodeResult simple(EncodingId encodingId, MemorySegment data) { - return simple(encodingId, data, null, null); - } - - /// Returns {@code true} if both {@code statsMin} and {@code statsMax} are present. - /// - /// @return {@code true} if zone-map statistics are available for this result - public boolean hasStats() { - return statsMin != null && statsMax != null; - } -} diff --git a/core/src/main/java/io/github/dfa1/vortex/encoding/Encoding.java b/core/src/main/java/io/github/dfa1/vortex/encoding/Encoding.java deleted file mode 100644 index 79832f6..0000000 --- a/core/src/main/java/io/github/dfa1/vortex/encoding/Encoding.java +++ /dev/null @@ -1,45 +0,0 @@ -package io.github.dfa1.vortex.encoding; - -import io.github.dfa1.vortex.core.array.Array; -import io.github.dfa1.vortex.core.DType; - -/// Combines encode and decode for one encoding type. -/// Register via [EncodingRegistry] — implementations are discoverable via ServiceLoader. -public interface Encoding { - /// Returns the encoding id for this encoding. - /// - /// @return encoding id - EncodingId encodingId(); - - /// Decodes an array node from the file using the provided context. - /// - /// @param ctx decoding context containing buffers, dtype, row count, and child registry - /// @return decoded array - Array decode(DecodeContext ctx); - - /// Returns {@code true} if this encoding can encode the given dtype. - /// - /// @param dtype the dtype to test - /// @return {@code true} if this encoding accepts {@code dtype} - default boolean accepts(DType dtype) { - return false; - } - - /// Encodes {@code data} to bytes, including per-chunk min/max stats when available. - /// - /// @param dtype logical type of the data - /// @param data the data to encode (type depends on encoding; typically a primitive array) - /// @return encode result containing the root node, buffers, and optional stats - EncodeResult encode(DType dtype, Object data); - - /// Cascade-aware encode: returns a partial step with open child slots. - /// Default wraps the terminal {@link #encode} result; override to expose children. - /// - /// @param dtype the logical type of the data - /// @param data the data to encode - /// @param ctx cascade compressor context controlling recursion depth and exclusions - /// @return cascade step with optional open child slots - default CascadeStep encodeCascade(DType dtype, Object data, CompressorContext ctx) { - return CascadeStep.terminal(encode(dtype, data)); - } -} diff --git a/core/src/main/java/io/github/dfa1/vortex/encoding/EncodingId.java b/core/src/main/java/io/github/dfa1/vortex/encoding/EncodingId.java deleted file mode 100644 index 467d5df..0000000 --- a/core/src/main/java/io/github/dfa1/vortex/encoding/EncodingId.java +++ /dev/null @@ -1,127 +0,0 @@ -package io.github.dfa1.vortex.encoding; - -import io.github.dfa1.vortex.core.VortexException; - -import java.util.Map; -import java.util.function.Function; -import java.util.stream.Collectors; -import java.util.stream.Stream; - -/// Strongly-typed encoding identifier used in place of raw strings. -public enum EncodingId { - /// Canonical flat primitive encoding ({@code vortex.primitive}). - VORTEX_PRIMITIVE("vortex.primitive"), - /// Bit-packed boolean encoding ({@code vortex.bool}). - VORTEX_BOOL("vortex.bool"), - /// Dictionary encoding for low-cardinality columns ({@code vortex.dict}). - VORTEX_DICT("vortex.dict"), - /// Sparse encoding for columns with many nulls or zeros ({@code vortex.sparse}). - VORTEX_SPARSE("vortex.sparse"), - /// Sequence encoding ({@code vortex.sequence}). - VORTEX_SEQUENCE("vortex.sequence"), - /// Run-end encoding for sorted/repetitive columns ({@code vortex.runend}). - VORTEX_RUNEND("vortex.runend"), - /// Constant encoding — all elements share one value ({@code vortex.constant}). - VORTEX_CONSTANT("vortex.constant"), - /// ALP (Adaptive Lossless floating-Point) encoding for F32/F64 ({@code vortex.alp}). - VORTEX_ALP("vortex.alp"), - /// Variable-length binary encoding ({@code vortex.varbin}). - VORTEX_VARBIN("vortex.varbin"), - /// FSST compressed string encoding ({@code vortex.fsst}). - VORTEX_FSST("vortex.fsst"), - /// All-null encoding ({@code vortex.null}). - VORTEX_NULL("vortex.null"), - /// One-byte-per-boolean encoding ({@code vortex.bytebool}). - VORTEX_BYTEBOOL("vortex.bytebool"), - /// Zig-zag encoding for signed integers ({@code vortex.zigzag}). - VORTEX_ZIGZAG("vortex.zigzag"), - /// Extension type wrapper encoding ({@code vortex.ext}). - VORTEX_EXT("vortex.ext"), - /// Variable-length binary view encoding ({@code vortex.varbinview}). - VORTEX_VARBINVIEW("vortex.varbinview"), - /// pcodec (Pco) floating-point/integer encoding ({@code vortex.pco}). - VORTEX_PCO("vortex.pco"), - /// Canonical flat decimal storage ({@code vortex.decimal}). - VORTEX_DECIMAL("vortex.decimal"), - /// Decimal split into MSP + LSP children ({@code vortex.decimal_byte_parts}). - VORTEX_DECIMAL_BYTE_PARTS("vortex.decimal_byte_parts"), - /// Timestamp split into days/seconds/subseconds ({@code vortex.datetimeparts}). - VORTEX_DATETIMEPARTS("vortex.datetimeparts"), - /// Zstandard compressed encoding ({@code vortex.zstd}). - VORTEX_ZSTD("vortex.zstd"), - /// Fixed-size list encoding ({@code vortex.fixed_size_list}). - VORTEX_FIXED_SIZE_LIST("vortex.fixed_size_list"), - /// Variable-length list encoding ({@code vortex.list}). - VORTEX_LIST("vortex.list"), - /// List-view encoding ({@code vortex.listview}). - VORTEX_LISTVIEW("vortex.listview"), - /// ALP-RD (ALP with remainder dictionary) encoding ({@code vortex.alprd}). - VORTEX_ALPRD("vortex.alprd"), - - // Layout encoding IDs included so parser/registry can represent them safely - /// Chunked layout encoding ({@code vortex.chunked}). - VORTEX_CHUNKED("vortex.chunked"), - /// Struct layout encoding ({@code vortex.struct}). - VORTEX_STRUCT("vortex.struct"), - - /// FastLanes bit-packed encoding ({@code fastlanes.bitpacked}). - FASTLANES_BITPACKED("fastlanes.bitpacked"), - /// FastLanes frame-of-reference encoding ({@code fastlanes.for}). - FASTLANES_FOR("fastlanes.for"), - /// FastLanes delta encoding ({@code fastlanes.delta}). - FASTLANES_DELTA("fastlanes.delta"), - /// FastLanes run-length encoding ({@code fastlanes.rle}). - FASTLANES_RLE("fastlanes.rle"), - - // Known in Rust but not yet implemented; registered so EncodingId.from() doesn't throw - /// Masked encoding (not yet implemented; registered to prevent parse errors). - VORTEX_MASKED("vortex.masked"), - /// Patched encoding (not yet implemented; registered to prevent parse errors). - VORTEX_PATCHED("vortex.patched"), - /// Variant encoding (not yet implemented; registered to prevent parse errors). - VORTEX_VARIANT("vortex.variant"), -; - - // O(1) access to EncodingId by its string representation - private static final Map LOOKUP = Stream.of(EncodingId.values()) - .collect(Collectors.toUnmodifiableMap(EncodingId::id, Function.identity())); - private final String id; - - EncodingId(String id) { - this.id = id; - } - - /// Returns the enum constant for the given raw encoding id string. - /// - /// @param id raw encoding id string (e.g. {@code "vortex.primitive"}) - /// @return the matching {@link EncodingId} - /// @throws io.github.dfa1.vortex.core.VortexException if the id is not recognised - public static EncodingId from(String id) { - EncodingId result = LOOKUP.get(id); - if (result == null) { - throw new VortexException("unknown encoding id: " + id); - } - return result; - } - - /// Non-throwing lookup: returns the matching constant or `null` for ids not in this enum. - /// Used by [EncodingRegistry] to discriminate [KnownArrayNode] from [UnknownArrayNode]. - /// - /// @param id raw encoding id string to look up - /// @return the matching {@link EncodingId}, or {@code null} if not recognised - public static EncodingId tryFrom(String id) { - return LOOKUP.get(id); - } - - /// Returns the raw encoding id string for this constant (e.g. {@code "vortex.primitive"}). - /// - /// @return the raw string encoding id - public String id() { - return id; - } - - @Override - public String toString() { - return id; - } -} diff --git a/core/src/main/java/io/github/dfa1/vortex/encoding/EncodingRegistry.java b/core/src/main/java/io/github/dfa1/vortex/encoding/EncodingRegistry.java deleted file mode 100644 index f1f69b0..0000000 --- a/core/src/main/java/io/github/dfa1/vortex/encoding/EncodingRegistry.java +++ /dev/null @@ -1,189 +0,0 @@ -package io.github.dfa1.vortex.encoding; - -import io.github.dfa1.vortex.core.array.Array; -import io.github.dfa1.vortex.core.array.UnknownArray; -import io.github.dfa1.vortex.core.ArrayStats; -import io.github.dfa1.vortex.core.DType; -import io.github.dfa1.vortex.core.VortexException; -import io.github.dfa1.vortex.fbs.Buffer; - -import java.lang.foreign.MemorySegment; -import java.lang.foreign.SegmentAllocator; -import java.nio.ByteBuffer; -import java.nio.ByteOrder; -import java.util.HashMap; -import java.util.List; -import java.util.Map; -import java.util.ServiceLoader; - -/// Registry mapping encoding IDs to [Encoding] implementations. -public final class EncodingRegistry { - - private final Map encodings = new HashMap<>(); - private boolean allowUnknown; - - private EncodingRegistry() { - } - - /// Enable passthrough decode for unknown encoding ids. - /// - /// Default is strict: unknown ids throw [VortexException]. When enabled, unknown nodes - /// (and all their children, recursively) are wrapped as [UnknownArray] preserving raw - /// metadata + buffers + stats. Mirrors Rust `VortexSession::allow_unknown()`. - /// - /// @return this registry, for chaining - public EncodingRegistry allowUnknown() { - this.allowUnknown = true; - return this; - } - - /// Returns whether passthrough decode for unknown encoding ids is enabled. - /// - /// @return {@code true} if unknown encodings are silently wrapped as {@link io.github.dfa1.vortex.core.array.UnknownArray} - public boolean isAllowUnknown() { - return allowUnknown; - } - - /// Load all [Encoding]s registered via `ServiceLoader`. - /// - /// @return a new {@link EncodingRegistry} populated with all service-loaded encodings - public static EncodingRegistry loadAll() { - var registry = new EncodingRegistry(); - for (Encoding encoding : ServiceLoader.load(Encoding.class)) { - registry.register(encoding); - } - return registry; - } - - /// Creates an empty registry with no encodings registered. - /// - /// @return a new empty {@link EncodingRegistry} - public static EncodingRegistry empty() { - return new EncodingRegistry(); - } - - /// Returns {@code true} if an encoding is registered for the given id. - /// - /// @param encodingId the encoding id to query - /// @return {@code true} if an {@link Encoding} is registered for {@code encodingId} - public boolean hasEncoding(EncodingId encodingId) { - return encodings.containsKey(encodingId); - } - - private static ArrayNode convertArrayNode( - io.github.dfa1.vortex.fbs.ArrayNode fbs, - List encodingSpecs - ) { - String rawEncodingId = encodingSpecs.get(fbs.encoding()); - - ArrayNode[] children = new ArrayNode[fbs.childrenLength()]; - for (int i = 0; i < children.length; i++) { - children[i] = convertArrayNode(fbs.children(i), encodingSpecs); - } - - int[] bufferIndices = new int[fbs.buffersLength()]; - for (int i = 0; i < bufferIndices.length; i++) { - bufferIndices[i] = fbs.buffers(i); - } - - // metadataAsByteBuffer() returns duplicate with position=vectorStart; slice to normalize to 0 - ByteBuffer rawMeta = fbs.metadataAsByteBuffer(); - ByteBuffer meta = (rawMeta != null) ? rawMeta.slice() : null; - ArrayStats stats = ArrayStats.fromFbs(fbs.stats()); - EncodingId known = EncodingId.tryFrom(rawEncodingId); - if (known != null) { - return new KnownArrayNode(known, meta, children, bufferIndices, stats); - } - return new UnknownArrayNode(rawEncodingId, meta, children, bufferIndices, stats); - } - - /// Registers an encoding implementation in this registry. - /// - /// @param encoding the {@link Encoding} to register - /// @throws io.github.dfa1.vortex.core.VortexException if an encoding with the same id is already registered - public void register(Encoding encoding) { - Encoding old = encodings.put(encoding.encodingId(), encoding); - if (old != null) { - throw new VortexException("encoding %s already registered".formatted(encoding.encodingId())); - } - } - - /// Decode a flat segment from the file's memory-mapped region. - /// - /// Segment format: {@code buffer_data... | FlatBuffer(Array) | u32 LE = FlatBuffer size}. - /// - /// @param seg memory-mapped region for the flat segment - /// @param encodingSpecs ordered list of encoding id strings from the file's encoding table - /// @param dtype expected logical type for the decoded array - /// @param rowCount number of logical rows in this segment - /// @param arena allocator for decode output; lifetime matches the current chunk epoch - /// @return the decoded {@link Array} for this segment - public Array decodeSegment(MemorySegment seg, List encodingSpecs, - DType dtype, long rowCount, SegmentAllocator arena) { - int segLen = (int) seg.byteSize(); - ByteBuffer bb = seg.asByteBuffer().order(ByteOrder.LITTLE_ENDIAN); - - int fbLen = bb.getInt(segLen - 4); - int fbStart = segLen - 4 - fbLen; - ByteBuffer fbBuf = bb.slice(fbStart, fbLen).order(ByteOrder.LITTLE_ENDIAN); - var fbArray = io.github.dfa1.vortex.fbs.Array.getRootAsArray(fbBuf); - - int numBuffers = fbArray.buffersLength(); - MemorySegment[] bufs = new MemorySegment[numBuffers]; - long dataOffset = 0; - for (int i = 0; i < numBuffers; i++) { - Buffer bufDesc = fbArray.buffers(i); - dataOffset += bufDesc.padding(); - bufs[i] = seg.asSlice(dataOffset, bufDesc.length()); - dataOffset += bufDesc.length(); - } - - ArrayNode rootNode = convertArrayNode(fbArray.root(), encodingSpecs); - var ctx = new DecodeContext(rootNode, dtype, rowCount, bufs, this, arena); - return decode(ctx); - } - - Array decode(DecodeContext ctx) { - ArrayNode node = ctx.node(); - Encoding encoding = switch (node) { - case KnownArrayNode k -> encodings.get(k.encodingId()); - case UnknownArrayNode u -> null; - }; - if (encoding != null) { - return encoding.decode(ctx); - } - if (allowUnknown) { - return decodeUnknown(ctx, node); - } - String id = switch (node) { - case KnownArrayNode k -> k.encodingId().id(); - case UnknownArrayNode u -> u.rawEncodingId(); - }; - throw new VortexException("no encoding registered for " + id); - } - - /// Recursively wrap a node and its children as [UnknownArray]. Children of an unknown - /// parent are always wrapped unknown regardless of whether their own id is recognised — - /// matches Rust `decode_foreign` in `vortex-array/src/serde.rs`. - private static UnknownArray decodeUnknown(DecodeContext ctx, ArrayNode node) { - String rawId = switch (node) { - case KnownArrayNode k -> k.encodingId().id(); - case UnknownArrayNode u -> u.rawEncodingId(); - }; - MemorySegment[] bufs = new MemorySegment[node.bufferIndices().length]; - for (int i = 0; i < bufs.length; i++) { - bufs[i] = ctx.buffer(i); - } - Array[] children = new Array[node.children().length]; - for (int i = 0; i < children.length; i++) { - ArrayNode childNode = node.children()[i]; - DecodeContext childCtx = new DecodeContext( - childNode, ctx.dtype(), ctx.rowCount(), - ctx.segmentBuffers(), ctx.registry(), ctx.arena()); - children[i] = decodeUnknown(childCtx, childNode); - } - return new UnknownArray( - rawId, ctx.dtype(), ctx.rowCount(), - node.metadata(), bufs, children, node.stats()); - } -} diff --git a/core/src/main/java/io/github/dfa1/vortex/encoding/ExtEncoding.java b/core/src/main/java/io/github/dfa1/vortex/encoding/ExtEncoding.java deleted file mode 100644 index 520844c..0000000 --- a/core/src/main/java/io/github/dfa1/vortex/encoding/ExtEncoding.java +++ /dev/null @@ -1,52 +0,0 @@ -package io.github.dfa1.vortex.encoding; - -import io.github.dfa1.vortex.core.DType; -import io.github.dfa1.vortex.core.VortexException; -import io.github.dfa1.vortex.core.array.Array; - -/// Encoder/decoder for {@code vortex.ext} — extension type transparent storage wrapper. -/// -///

No buffers, empty metadata. Child slot 0: the storage array encoded/decoded using -/// the extension dtype's storage dtype and same row count. -/// -///

Encode: delegates to {@link PrimitiveEncoding} for the storage child, wraps in ext node. -/// Decode: unwraps the single child and returns it directly. -public final class ExtEncoding implements Encoding { - - /// Creates a new {@code ExtEncoding} instance. - public ExtEncoding() { - } - - @Override - public EncodingId encodingId() { - return EncodingId.VORTEX_EXT; - } - - @Override - public boolean accepts(DType dtype) { - return dtype instanceof DType.Extension; - } - - @Override - public EncodeResult encode(DType dtype, Object data) { - if (!(dtype instanceof DType.Extension ext)) { - throw new VortexException(EncodingId.VORTEX_EXT, "expected extension dtype, got " + dtype); - } - EncodeResult childResult = new PrimitiveEncoding().encode(ext.storageDType(), data); - EncodeNode root = new EncodeNode(EncodingId.VORTEX_EXT, null, new EncodeNode[]{childResult.rootNode()}, new int[0]); - return new EncodeResult(root, childResult.buffers(), childResult.statsMin(), childResult.statsMax()); - } - - @Override - public Array decode(DecodeContext ctx) { - if (!(ctx.dtype() instanceof DType.Extension ext)) { - throw new VortexException(EncodingId.VORTEX_EXT, "expected extension dtype, got " + ctx.dtype()); - } - long n = ctx.rowCount(); - ArrayNode childNode = ctx.node().children()[0]; - DecodeContext childCtx = new DecodeContext( - childNode, ext.storageDType(), n, - ctx.segmentBuffers(), ctx.registry(), ctx.arena()); - return ctx.registry().decode(childCtx); - } -} diff --git a/core/src/main/java/io/github/dfa1/vortex/encoding/FixedSizeListData.java b/core/src/main/java/io/github/dfa1/vortex/encoding/FixedSizeListData.java deleted file mode 100644 index 30ee7a1..0000000 --- a/core/src/main/java/io/github/dfa1/vortex/encoding/FixedSizeListData.java +++ /dev/null @@ -1,12 +0,0 @@ -package io.github.dfa1.vortex.encoding; - -/// Input data for encoding a {@code vortex.fixed_size_list} column. -/// -///

{@code elements} is the flat array of inner values; its length must equal -/// {@code outerLen * fixedSize}. The element type is determined by the DType passed to -/// {@link FixedSizeListEncoding#encode}. -/// -/// @param elements flat array of all inner values; length must equal {@code outerLen * fixedSize} -/// @param outerLen number of outer list elements -public record FixedSizeListData(Object elements, long outerLen) { -} diff --git a/core/src/main/java/io/github/dfa1/vortex/encoding/FixedSizeListEncoding.java b/core/src/main/java/io/github/dfa1/vortex/encoding/FixedSizeListEncoding.java deleted file mode 100644 index 913b76b..0000000 --- a/core/src/main/java/io/github/dfa1/vortex/encoding/FixedSizeListEncoding.java +++ /dev/null @@ -1,112 +0,0 @@ -package io.github.dfa1.vortex.encoding; - -import io.github.dfa1.vortex.core.DType; -import io.github.dfa1.vortex.core.VortexException; -import io.github.dfa1.vortex.core.array.Array; -import io.github.dfa1.vortex.core.array.FixedSizeListArray; - -import java.lang.foreign.MemorySegment; -import java.nio.ByteBuffer; -import java.util.ArrayList; -import java.util.List; - -/// Encoder/decoder for {@code vortex.fixed_size_list}. -/// -///

Wire format (per Rust vtable): -///

    -///
  • Buffers: 0 -///
  • Metadata: empty byte array -///
  • Children: 1 or 2. -/// {@code children[0]} is the flat elements array (len = outerLen * fixedSize, dtype = elementType). -/// {@code children[1]}, when present, is the validity (bool) array (len = outerLen). -///
-/// -///

The {@code fixedSize} is read directly from the {@link DType.FixedSizeList} dtype — -/// it is not stored in metadata. -public final class FixedSizeListEncoding implements Encoding { - - /// Creates a new {@code FixedSizeListEncoding} instance; use via {@link EncodingRegistry}. - public FixedSizeListEncoding() { - } - - @Override - public EncodingId encodingId() { - return EncodingId.VORTEX_FIXED_SIZE_LIST; - } - - @Override - public boolean accepts(DType dtype) { - return dtype instanceof DType.FixedSizeList; - } - - @Override - public EncodeResult encode(DType dtype, Object data) { - return Encoder.encode((DType.FixedSizeList) dtype, (FixedSizeListData) data); - } - - @Override - public Array decode(DecodeContext ctx) { - return Decoder.decode(ctx); - } - - private static final class Encoder { - - private static final List FALLBACK = List.of( - new PrimitiveEncoding(), new VarBinEncoding(), new BoolEncoding(), - new NullEncoding(), new ByteBoolEncoding()); - - static EncodeResult encode(DType.FixedSizeList dtype, FixedSizeListData data) { - DType elementType = dtype.elementType(); - Encoding inner = findEncoding(elementType); - - EncodeResult elemResult = inner.encode(elementType, data.elements()); - - List allBuffers = new ArrayList<>(elemResult.buffers()); - EncodeNode elemNode = EncodeNode.remapBufferIndices(elemResult.rootNode(), 0); - - EncodeNode root = new EncodeNode( - EncodingId.VORTEX_FIXED_SIZE_LIST, - ByteBuffer.wrap(new byte[0]), - new EncodeNode[]{elemNode}, - new int[]{}); - return new EncodeResult(root, List.copyOf(allBuffers), null, null); - } - - private static Encoding findEncoding(DType dtype) { - for (Encoding enc : FALLBACK) { - if (enc.accepts(dtype)) { - return enc; - } - } - throw new UnsupportedOperationException("no fallback encoding for dtype: " + dtype); - } - } - - private static final class Decoder { - - static Array decode(DecodeContext ctx) { - if (!(ctx.dtype() instanceof DType.FixedSizeList fsl)) { - throw new VortexException(EncodingId.VORTEX_FIXED_SIZE_LIST, - "expected DType.FixedSizeList, got " + ctx.dtype()); - } - - int nchildren = ctx.node().children().length; - if (nchildren < 1 || nchildren > 2) { - throw new VortexException(EncodingId.VORTEX_FIXED_SIZE_LIST, - "expected 1 or 2 children, got " + nchildren); - } - - long outerLen = ctx.rowCount(); - long elemLen = outerLen * fsl.fixedSize(); - DType elementType = fsl.elementType(); - - ArrayNode elemNode = ctx.node().children()[0]; - var elemCtx = new DecodeContext( - elemNode, elementType, elemLen, - ctx.segmentBuffers(), ctx.registry(), ctx.arena()); - Array elements = ctx.registry().decode(elemCtx); - - return new FixedSizeListArray(fsl, outerLen, elements); - } - } -} diff --git a/core/src/main/java/io/github/dfa1/vortex/encoding/FrameOfReferenceEncoding.java b/core/src/main/java/io/github/dfa1/vortex/encoding/FrameOfReferenceEncoding.java deleted file mode 100644 index b80cba8..0000000 --- a/core/src/main/java/io/github/dfa1/vortex/encoding/FrameOfReferenceEncoding.java +++ /dev/null @@ -1,287 +0,0 @@ -package io.github.dfa1.vortex.encoding; - -import com.google.protobuf.InvalidProtocolBufferException; -import io.github.dfa1.vortex.proto.ScalarProtos; -import io.github.dfa1.vortex.core.array.Array; -import io.github.dfa1.vortex.core.ArrayStats; -import io.github.dfa1.vortex.core.DType; -import io.github.dfa1.vortex.core.PType; -import io.github.dfa1.vortex.core.array.BoolArray; -import io.github.dfa1.vortex.core.array.ByteArray; -import io.github.dfa1.vortex.core.array.DoubleArray; -import io.github.dfa1.vortex.core.array.IntArray; -import io.github.dfa1.vortex.core.array.LongArray; -import io.github.dfa1.vortex.core.array.MaskedArray; -import io.github.dfa1.vortex.core.array.ShortArray; -import io.github.dfa1.vortex.core.VortexException; - -import java.lang.foreign.Arena; -import java.lang.foreign.MemorySegment; -import java.lang.foreign.SegmentAllocator; -import java.lang.foreign.ValueLayout; -import java.nio.ByteBuffer; -import java.util.List; - -/// Decoder for {@code fastlanes.for} (Frame of Reference). -/// -///

Metadata: raw {@code ScalarValue} protobuf bytes — the reference (minimum) value. -/// Child slot 0: encoded residuals array (same dtype as parent, typically bitpacked). -/// Decode: {@code output[i] = encoded[i] + reference} (wrapping arithmetic). -public final class FrameOfReferenceEncoding implements Encoding { - - /// Creates a new {@code FrameOfReferenceEncoding} instance; use via {@link EncodingRegistry}. - public FrameOfReferenceEncoding() { - } - - @Override - public EncodingId encodingId() { - return EncodingId.FASTLANES_FOR; - } - - @Override - public boolean accepts(DType dtype) { - return dtype instanceof DType.Primitive p && !p.ptype().isFloating(); - } - - @Override - public EncodeResult encode(DType dtype, Object data) { - return Encoder.encode(dtype, data); - } - - @Override - public CascadeStep encodeCascade(DType dtype, Object data, CompressorContext ctx) { - return Encoder.encodeCascade(dtype, data); - } - - @Override - public Array decode(DecodeContext ctx) { - return Decoder.decode(ctx); - } - - private static final class Encoder { - - private static EncodeResult encode(DType dtype, Object data) { - if (!(dtype instanceof DType.Primitive p)) { - throw new VortexException(EncodingId.FASTLANES_FOR, "expected primitive dtype, got " + dtype); - } - PType ptype = p.ptype(); - long[] longs = toLongs(data, ptype); - int n = longs.length; - - long ref = computeRef(longs, n); - MemorySegment residuals = toResidualBuffer(longs, ref, ptype); - ByteBuffer meta = buildForMeta(ref, ptype); - - EncodeNode child = EncodeNode.leaf(EncodingId.VORTEX_PRIMITIVE, 0); - EncodeNode root = new EncodeNode(EncodingId.FASTLANES_FOR, meta, new EncodeNode[]{child}, new int[0]); - return new EncodeResult(root, List.of(residuals), null, null); - } - - private static CascadeStep encodeCascade(DType dtype, Object data) { - if (!(dtype instanceof DType.Primitive p)) { - throw new VortexException(EncodingId.FASTLANES_FOR, "expected primitive dtype, got " + dtype); - } - PType ptype = p.ptype(); - long[] longs = toLongs(data, ptype); - int n = longs.length; - - long ref = computeRef(longs, n); - ByteBuffer meta = buildForMeta(ref, ptype); - - // residuals are the open child slot — compressor will bitpack them - EncodeNode partialRoot = new EncodeNode(EncodingId.FASTLANES_FOR, meta, new EncodeNode[1], new int[0]); - ChildSlot slot = new ChildSlot(dtype, residualsAsNativeArray(longs, ref, ptype), 0); - return new CascadeStep(partialRoot, List.of(), List.of(slot), null, null, true); - } - - private static long computeRef(long[] longs, int n) { - long ref = n > 0 ? longs[0] : 0L; - for (long v : longs) { - if (v < ref) { - ref = v; - } - } - return ref; - } - - private static ByteBuffer buildForMeta(long ref, PType ptype) { - boolean unsigned = switch (ptype) { - case U8, U16, U32, U64 -> true; - default -> false; - }; - ScalarProtos.ScalarValue scalar = unsigned - ? ScalarProtos.ScalarValue.newBuilder().setUint64Value(ref).build() - : ScalarProtos.ScalarValue.newBuilder().setInt64Value(ref).build(); - return ByteBuffer.wrap(scalar.toByteArray()); - } - - /// Returns residuals as a Java primitive array of the correct type (for passing as ChildSlot data). - private static Object residualsAsNativeArray(long[] longs, long ref, PType ptype) { - int n = longs.length; - return switch (ptype) { - case I8, U8 -> { - byte[] r = new byte[n]; - for (int i = 0; i < n; i++) { - r[i] = (byte) (longs[i] - ref); - } - yield r; - } - case I16, U16 -> { - short[] r = new short[n]; - for (int i = 0; i < n; i++) { - r[i] = (short) (longs[i] - ref); - } - yield r; - } - case I32, U32 -> { - int[] r = new int[n]; - for (int i = 0; i < n; i++) { - r[i] = (int) (longs[i] - ref); - } - yield r; - } - case I64, U64 -> { - long[] r = new long[n]; - for (int i = 0; i < n; i++) { - r[i] = longs[i] - ref; - } - yield r; - } - default -> throw new VortexException(EncodingId.FASTLANES_FOR, "unsupported ptype: " + ptype); - }; - } - - private static long[] toLongs(Object data, PType ptype) { - return switch (ptype) { - case I8, U8 -> { - byte[] arr = (byte[]) data; - long[] r = new long[arr.length]; - for (int i = 0; i < arr.length; i++) { - r[i] = ptype == PType.U8 ? Byte.toUnsignedLong(arr[i]) : arr[i]; - } - yield r; - } - case I16, U16 -> { - short[] arr = (short[]) data; - long[] r = new long[arr.length]; - for (int i = 0; i < arr.length; i++) { - r[i] = ptype == PType.U16 ? Short.toUnsignedLong(arr[i]) : arr[i]; - } - yield r; - } - case I32, U32 -> { - int[] arr = (int[]) data; - long[] r = new long[arr.length]; - for (int i = 0; i < arr.length; i++) { - r[i] = ptype == PType.U32 ? Integer.toUnsignedLong(arr[i]) : arr[i]; - } - yield r; - } - case I64, U64 -> (long[]) data; - default -> throw new VortexException(EncodingId.FASTLANES_FOR, "unsupported ptype: " + ptype); - }; - } - - private static MemorySegment toResidualBuffer(long[] longs, long ref, PType ptype) { - int n = longs.length; - int elemBytes = ptype.byteSize(); - MemorySegment seg = Arena.ofAuto().allocate((long) n * elemBytes, elemBytes); - for (int i = 0; i < n; i++) { - long r = longs[i] - ref; - PTypeIO.set(seg, (long) i * elemBytes, ptype, r); - } - return seg; - } - } - - private static final class Decoder { - - private static Array decode(DecodeContext ctx) { - ByteBuffer rawMeta = ctx.metadata(); - if (rawMeta == null || !rawMeta.hasRemaining()) { - throw new VortexException(EncodingId.FASTLANES_FOR, "missing metadata"); - } - ScalarProtos.ScalarValue scalar; - try { - scalar = ScalarProtos.ScalarValue.parseFrom(rawMeta.duplicate()); - } catch (InvalidProtocolBufferException e) { - throw new VortexException(EncodingId.FASTLANES_FOR, "invalid metadata", e); - } - - Array encoded = ctx.decodeChild(0); - - // Nullable primitive: child decodes as MaskedArray; extract values and propagate validity. - BoolArray validity = null; - Array rawEncoded = encoded; - if (encoded instanceof MaskedArray masked) { - rawEncoded = masked.child(0); - validity = (BoolArray) masked.child(1); - } - - if (!(ctx.dtype() instanceof DType.Primitive p)) { - throw new VortexException(EncodingId.FASTLANES_FOR, "expected primitive dtype, got " + ctx.dtype()); - } - - long ref = referenceValue(scalar); - if (ref == 0L) { - return validity != null ? new MaskedArray(rawEncoded, validity) : rawEncoded; - } - - MemorySegment src = rawEncoded.buffer(0); - long n = ctx.rowCount(); - MemorySegment dst = applyReference(src, n, p.ptype(), ref, ctx.arena()); - Array result = switch (p.ptype()) { - case I64, U64 -> new LongArray(ctx.dtype(), n, dst, ArrayStats.empty()); - case I32, U32 -> new IntArray(ctx.dtype(), n, dst, ArrayStats.empty()); - case F64 -> new DoubleArray(ctx.dtype(), n, dst, ArrayStats.empty()); - case I16, U16 -> new ShortArray(ctx.dtype(), n, dst, ArrayStats.empty()); - case I8, U8 -> new ByteArray(ctx.dtype(), n, dst, ArrayStats.empty()); - default -> throw new VortexException(EncodingId.FASTLANES_FOR, "unsupported ptype " + p.ptype()); - }; - return validity != null ? new MaskedArray(result, validity) : result; - } - - private static long referenceValue(ScalarProtos.ScalarValue scalar) { - return switch (scalar.getKindCase()) { - case INT64_VALUE -> scalar.getInt64Value(); - case UINT64_VALUE -> scalar.getUint64Value(); - case KIND_NOT_SET -> 0L; - default -> throw new VortexException(EncodingId.FASTLANES_FOR, - "unexpected scalar kind " + scalar.getKindCase()); - }; - } - - private static MemorySegment applyReference(MemorySegment src, long n, PType ptype, long ref, SegmentAllocator arena) { - int wordBytes = ptype.byteSize(); - MemorySegment dst = arena.allocate(n * wordBytes); - switch (ptype) { - case I8, U8 -> { - for (long off = 0, end = n; off < end; off++) { - byte v = src.get(ValueLayout.JAVA_BYTE, off); - dst.set(ValueLayout.JAVA_BYTE, off, (byte) (v + (byte) ref)); - } - } - case I16, U16 -> { - for (long off = 0, end = n * 2; off < end; off += 2) { - short v = src.get(PTypeIO.LE_SHORT, off); - dst.set(PTypeIO.LE_SHORT, off, (short) (v + (short) ref)); - } - } - case I32, U32 -> { - for (long off = 0, end = n * 4; off < end; off += 4) { - int v = src.get(PTypeIO.LE_INT, off); - dst.set(PTypeIO.LE_INT, off, v + (int) ref); - } - } - case I64, U64 -> { - for (long off = 0, end = n * 8; off < end; off += 8) { - long v = src.get(PTypeIO.LE_LONG, off); - dst.set(PTypeIO.LE_LONG, off, v + ref); - } - } - default -> throw new VortexException(EncodingId.FASTLANES_FOR, "unsupported ptype " + ptype); - } - return dst; - } - } -} diff --git a/core/src/main/java/io/github/dfa1/vortex/encoding/FsstEncoding.java b/core/src/main/java/io/github/dfa1/vortex/encoding/FsstEncoding.java deleted file mode 100644 index 6f36c64..0000000 --- a/core/src/main/java/io/github/dfa1/vortex/encoding/FsstEncoding.java +++ /dev/null @@ -1,286 +0,0 @@ -package io.github.dfa1.vortex.encoding; - -import com.google.protobuf.InvalidProtocolBufferException; -import io.github.dfa1.vortex.proto.EncodingProtos; -import io.github.dfa1.vortex.core.array.Array; -import io.github.dfa1.vortex.core.ArrayStats; -import io.github.dfa1.vortex.core.DType; -import io.github.dfa1.vortex.core.PType; -import io.github.dfa1.vortex.core.array.IntArray; -import io.github.dfa1.vortex.core.array.VarBinArray; -import io.github.dfa1.vortex.core.VortexException; - -import java.lang.foreign.Arena; -import java.lang.foreign.MemorySegment; -import java.lang.foreign.ValueLayout; -import java.nio.ByteBuffer; -import java.nio.charset.StandardCharsets; -import java.util.Arrays; -import java.util.List; - -/// Decoder for {@code vortex.fsst} — Fast Static Symbol Table string compression. -/// -///

Wire format (3-buffer current format): -///

    -///
  • Buffer 0: symbol table — up to 256 entries, each 8 bytes LE (Symbol = u64)
    • -///
  • Buffer 1: symbol_lengths — 1 byte per symbol, actual byte count in the symbol
    • -///
  • Buffer 2: compressed code bytes for all strings concatenated
    • -///
  • Child 0: uncompressed_lengths — primitive array (I32/I64) of original string lengths
    • -///
  • Child 1: codes_offsets — primitive array (I32/I64) of offsets into buf[2], length = n+1
    • -///
-/// -///

Output: varbin-compatible {@code Array} (buffer[0] = uncompressed bytes, child[0] = I32 offsets). -public final class FsstEncoding implements Encoding { - - /// Creates a new {@code FsstEncoding} instance; use via {@link EncodingRegistry}. - public FsstEncoding() { - } - - @Override - public EncodingId encodingId() { - return EncodingId.VORTEX_FSST; - } - - @Override - public boolean accepts(DType dtype) { - return dtype instanceof DType.Utf8 || dtype instanceof DType.Binary; - } - - @Override - public EncodeResult encode(DType dtype, Object data) { - return Encoder.encode((String[]) data); - } - - @Override - public Array decode(DecodeContext ctx) { - return Decoder.decode(ctx); - } - - private static final class Encoder { - - private static final int MAX_SYMBOLS = 255; - private static final int BIGRAM_COUNT = 65536; - - static EncodeResult encode(String[] strings) { - int n = strings.length; - - byte[][] byteArrays = new byte[n][]; - for (int i = 0; i < n; i++) { - byteArrays[i] = strings[i].getBytes(StandardCharsets.UTF_8); - } - - // Count bigram frequencies; pack freq+index into long for sort-free top-K - int[] freq = new int[BIGRAM_COUNT]; - for (byte[] b : byteArrays) { - for (int i = 0; i + 1 < b.length; i++) { - freq[(Byte.toUnsignedInt(b[i]) << 8) | Byte.toUnsignedInt(b[i + 1])]++; - } - } - long[] ranked = new long[BIGRAM_COUNT]; - for (int i = 0; i < BIGRAM_COUNT; i++) { - ranked[i] = ((long) freq[i] << 16) | i; - } - Arrays.sort(ranked); - - // Build symbol table (top-255 bigrams, descending freq from end of sorted array) - int numSymbols = 0; - int[] codeForBigram = new int[BIGRAM_COUNT]; - Arrays.fill(codeForBigram, -1); - long[] symbolValues = new long[MAX_SYMBOLS]; - for (int rank = BIGRAM_COUNT - 1; rank >= 0 && numSymbols < MAX_SYMBOLS; rank--) { - int bg = (int) (ranked[rank] & 0xFFFF); - if (freq[bg] == 0) { - break; - } - codeForBigram[bg] = numSymbols; - int hi = bg >>> 8; - int lo = bg & 0xFF; - symbolValues[numSymbols] = hi | ((long) lo << 8); - numSymbols++; - } - - // Compress each string - byte[][] compressed = new byte[n][]; - for (int i = 0; i < n; i++) { - compressed[i] = compressString(byteArrays[i], codeForBigram); - } - - Arena arena = Arena.ofAuto(); - - // Buffer 0: symbol table (8 bytes per symbol, LE u64) - MemorySegment symBuf = arena.allocate(Math.max(numSymbols * 8L, 1), 8); - for (int i = 0; i < numSymbols; i++) { - symBuf.setAtIndex(PTypeIO.LE_LONG, i, symbolValues[i]); - } - - // Buffer 1: symbol lengths (all 2 = bigram) - MemorySegment symLenBuf = arena.allocate(Math.max(numSymbols, 1)); - for (int i = 0; i < numSymbols; i++) { - symLenBuf.set(ValueLayout.JAVA_BYTE, i, (byte) 2); - } - - // Buffer 2: compressed bytes (all strings concatenated) - int totalCompressed = 0; - for (byte[] c : compressed) { - totalCompressed += c.length; - } - MemorySegment compBuf = arena.allocate(Math.max(totalCompressed, 1)); - long pos = 0; - for (byte[] c : compressed) { - MemorySegment.copy(MemorySegment.ofArray(c), 0, compBuf, pos, c.length); - pos += c.length; - } - - // Buffer 3: uncompressed lengths (I32, n elements) - MemorySegment uncompLenBuf = arena.allocate(Math.max(n * 4L, 1), 4); - for (int i = 0; i < n; i++) { - uncompLenBuf.setAtIndex(PTypeIO.LE_INT, i, byteArrays[i].length); - } - - // Buffer 4: codes offsets (I32, n+1 elements) - MemorySegment codesOffBuf = arena.allocate((long) (n + 1) * 4, 4); - long off = 0; - codesOffBuf.setAtIndex(PTypeIO.LE_INT, 0, 0); - for (int i = 0; i < n; i++) { - off += compressed[i].length; - codesOffBuf.setAtIndex(PTypeIO.LE_INT, i + 1, (int) off); - } - - byte[] metaBytes = EncodingProtos.FSSTMetadata.newBuilder() - .setUncompressedLengthsPtype( - io.github.dfa1.vortex.proto.DTypeProtos.PType.forNumber(PType.I32.ordinal())) - .setCodesOffsetsPtype( - io.github.dfa1.vortex.proto.DTypeProtos.PType.forNumber(PType.I32.ordinal())) - .build() - .toByteArray(); - - EncodeNode uncompLensNode = EncodeNode.leaf(EncodingId.VORTEX_PRIMITIVE, 3); - EncodeNode codesOffNode = EncodeNode.leaf(EncodingId.VORTEX_PRIMITIVE, 4); - EncodeNode root = new EncodeNode( - EncodingId.VORTEX_FSST, - ByteBuffer.wrap(metaBytes), - new EncodeNode[]{uncompLensNode, codesOffNode}, - new int[]{0, 1, 2}); - - return new EncodeResult(root, - List.of(symBuf, symLenBuf, compBuf, uncompLenBuf, codesOffBuf), - null, null); - } - - private static byte[] compressString(byte[] input, int[] codeForBigram) { - byte[] out = new byte[input.length * 2]; - int outLen = 0; - int i = 0; - while (i < input.length) { - if (i + 1 < input.length) { - int bg = (Byte.toUnsignedInt(input[i]) << 8) | Byte.toUnsignedInt(input[i + 1]); - int code = codeForBigram[bg]; - if (code >= 0) { - out[outLen++] = (byte) code; - i += 2; - continue; - } - } - out[outLen++] = (byte) 0xFF; - out[outLen++] = input[i]; - i++; - } - return Arrays.copyOf(out, outLen); - } - } - - private static final class Decoder { - - private static final int ESCAPE = 0xFF; - - private static Array decode(DecodeContext ctx) { - ByteBuffer rawMeta = ctx.metadata(); - if (rawMeta == null) { - throw new VortexException(EncodingId.VORTEX_FSST, "missing metadata"); - } - EncodingProtos.FSSTMetadata meta; - try { - meta = EncodingProtos.FSSTMetadata.parseFrom(rawMeta.duplicate()); - } catch (InvalidProtocolBufferException e) { - throw new VortexException(EncodingId.VORTEX_FSST, "invalid metadata", e); - } - - PType uncompLenPType = PType.values()[meta.getUncompressedLengthsPtype().getNumber()]; - PType codesOffPType = PType.values()[meta.getCodesOffsetsPtype().getNumber()]; - - long n = ctx.rowCount(); - - MemorySegment symbolsBuf = ctx.buffer(0); // 8 bytes per symbol (LE u64) - MemorySegment symbolLensBuf = ctx.buffer(1); // 1 byte per symbol - MemorySegment compressedBytes = ctx.buffer(2); // FSST-compressed heap - - Array uncompLens = decodeChild(ctx, 0, uncompLenPType, n); - Array codesOffsets = decodeChild(ctx, 1, codesOffPType, n + 1); - MemorySegment uncompLensSeg = uncompLens.buffer(0); - MemorySegment codesOffsetsSeg = codesOffsets.buffer(0); - - long totalUncompressed = 0L; - for (long i = 0; i < n; i++) { - totalUncompressed += readUnsigned(uncompLensSeg, i, uncompLenPType); - } - - MemorySegment outBytes = ctx.arena().allocate(totalUncompressed); - MemorySegment outOffsets = ctx.arena().allocate((n + 1) * 4L, 4); - outOffsets.setAtIndex(PTypeIO.LE_INT, 0, 0); - - long outPos = 0L; - for (long i = 0; i < n; i++) { - long cStart = readUnsigned(codesOffsetsSeg, i, codesOffPType); - long cEnd = readUnsigned(codesOffsetsSeg, i + 1, codesOffPType); - outPos = decompressString(compressedBytes, symbolsBuf, symbolLensBuf, - cStart, cEnd, outBytes, outPos); - outOffsets.setAtIndex(PTypeIO.LE_INT, i + 1, (int) outPos); - } - - DType i32 = new DType.Primitive(PType.I32, false); - Array offsets = new IntArray(i32, n + 1, outOffsets.asReadOnly(), ArrayStats.empty()); - return new VarBinArray(ctx.dtype(), n, outBytes.asReadOnly(), offsets, PType.I32, ArrayStats.empty()); - } - - private static Array decodeChild(DecodeContext parent, int idx, PType ptype, long rowCount) { - ArrayNode childNode = parent.node().children()[idx]; - DType dtype = new DType.Primitive(ptype, false); - DecodeContext childCtx = new DecodeContext( - childNode, dtype, rowCount, - parent.segmentBuffers(), parent.registry(), parent.arena()); - return parent.registry().decode(childCtx); - } - - private static long decompressString( - MemorySegment compressed, MemorySegment symbols, MemorySegment symLens, - long start, long end, MemorySegment out, long outPos - ) { - for (long j = start; j < end; j++) { - int b = Byte.toUnsignedInt(compressed.get(ValueLayout.JAVA_BYTE, j)); - if (b == ESCAPE) { - // next byte is literal - out.set(ValueLayout.JAVA_BYTE, outPos++, compressed.get(ValueLayout.JAVA_BYTE, ++j)); - } else { - int symLen = Byte.toUnsignedInt(symLens.get(ValueLayout.JAVA_BYTE, b)); - // symbols[b] is 8 bytes LE; emit first symLen bytes - long sym = symbols.getAtIndex(PTypeIO.LE_LONG, b); - for (int k = 0; k < symLen; k++) { - out.set(ValueLayout.JAVA_BYTE, outPos++, (byte) (sym >>> (k * 8))); - } - } - } - return outPos; - } - - private static long readUnsigned(MemorySegment seg, long idx, PType ptype) { - return switch (ptype) { - case U8 -> Byte.toUnsignedLong(seg.get(ValueLayout.JAVA_BYTE, idx)); - case U16 -> Short.toUnsignedLong(seg.get(PTypeIO.LE_SHORT, idx * 2)); - case U32 -> Integer.toUnsignedLong(seg.getAtIndex(PTypeIO.LE_INT, idx)); - case I32 -> seg.getAtIndex(PTypeIO.LE_INT, idx); - case I64, U64 -> seg.getAtIndex(PTypeIO.LE_LONG, idx); - default -> throw new VortexException(EncodingId.VORTEX_FSST, "unsupported ptype " + ptype); - }; - } - } -} diff --git a/core/src/main/java/io/github/dfa1/vortex/encoding/KnownArrayNode.java b/core/src/main/java/io/github/dfa1/vortex/encoding/KnownArrayNode.java deleted file mode 100644 index 39698e5..0000000 --- a/core/src/main/java/io/github/dfa1/vortex/encoding/KnownArrayNode.java +++ /dev/null @@ -1,15 +0,0 @@ -package io.github.dfa1.vortex.encoding; - -import io.github.dfa1.vortex.core.ArrayStats; - -import java.nio.ByteBuffer; - -/// Array node whose encoding id is well-known to this build (an [EncodingId] enum constant). -record KnownArrayNode( - EncodingId encodingId, - ByteBuffer metadata, - ArrayNode[] children, - int[] bufferIndices, - ArrayStats stats -) implements ArrayNode { -} diff --git a/core/src/main/java/io/github/dfa1/vortex/encoding/LeBitReader.java b/core/src/main/java/io/github/dfa1/vortex/encoding/LeBitReader.java deleted file mode 100644 index 3f2c91c..0000000 --- a/core/src/main/java/io/github/dfa1/vortex/encoding/LeBitReader.java +++ /dev/null @@ -1,67 +0,0 @@ -package io.github.dfa1.vortex.encoding; - -import io.github.dfa1.vortex.core.VortexException; - -import java.lang.foreign.MemorySegment; -import java.lang.foreign.ValueLayout; - -/// Little-endian bit reader over a {@link MemorySegment}. -/// -/// Bits are packed LSB-first within each byte (pcodec wire format convention). -/// Bit 0 of the stream is the LSB of byte 0; bit 8 is the LSB of byte 1. -final class LeBitReader { - - private static final ValueLayout.OfByte BYTE = ValueLayout.JAVA_BYTE; - - private final MemorySegment data; - private long bitPos; - - LeBitReader(MemorySegment data) { - this.data = data; - this.bitPos = 0; - } - - /// Read {@code n} bits (0 ≤ n ≤ 64) from the stream, LSB-first. - long readBits(int n) { - if (n == 0) { - return 0L; - } - long result = 0L; - int remaining = n; - int shift = 0; - while (remaining > 0) { - int byteIndex = (int) (bitPos >>> 3); - int bitInByte = (int) (bitPos & 7); - int available = 8 - bitInByte; - int take = Math.min(remaining, available); - int b; - try { - b = data.get(BYTE, byteIndex) & 0xFF; - } catch (IndexOutOfBoundsException e) { - throw new VortexException("pco: truncated data at bit " + bitPos, e); - } - result |= (long) ((b >>> bitInByte) & ((1 << take) - 1)) << shift; - shift += take; - remaining -= take; - bitPos += take; - } - return result; - } - - /// Discard bits to align the stream to the next byte boundary. - void alignToByte() { - int bitsInCurrentByte = (int) (bitPos & 7); - if (bitsInCurrentByte != 0) { - bitPos += 8 - bitsInCurrentByte; - } - } - - /// Current byte offset (only meaningful after {@link #alignToByte()}). - long byteOffset() { - return bitPos >>> 3; - } - - long bitsConsumed() { - return bitPos; - } -} diff --git a/core/src/main/java/io/github/dfa1/vortex/encoding/ListData.java b/core/src/main/java/io/github/dfa1/vortex/encoding/ListData.java deleted file mode 100644 index a60e181..0000000 --- a/core/src/main/java/io/github/dfa1/vortex/encoding/ListData.java +++ /dev/null @@ -1,13 +0,0 @@ -package io.github.dfa1.vortex.encoding; - -/// Input data for encoding a {@code vortex.list} column. -/// -///

{@code elements} is the flat array of all inner values concatenated. -/// {@code offsets} has length {@code outerLen + 1}; {@code offsets[i]..offsets[i+1]} -/// is the range of elements for list {@code i}. {@code offsets[0]} must be 0. -/// -/// @param elements flat array of all concatenated inner values -/// @param offsets offset array of length {@code outerLen + 1}; {@code offsets[0]} must be 0 -/// @param outerLen number of outer list elements -public record ListData(Object elements, long[] offsets, long outerLen) { -} diff --git a/core/src/main/java/io/github/dfa1/vortex/encoding/ListEncoding.java b/core/src/main/java/io/github/dfa1/vortex/encoding/ListEncoding.java deleted file mode 100644 index 7240d67..0000000 --- a/core/src/main/java/io/github/dfa1/vortex/encoding/ListEncoding.java +++ /dev/null @@ -1,145 +0,0 @@ -package io.github.dfa1.vortex.encoding; - -import com.google.protobuf.InvalidProtocolBufferException; -import io.github.dfa1.vortex.core.DType; -import io.github.dfa1.vortex.core.PType; -import io.github.dfa1.vortex.core.VortexException; -import io.github.dfa1.vortex.core.array.Array; -import io.github.dfa1.vortex.core.array.ListArray; -import io.github.dfa1.vortex.proto.EncodingProtos; - -import java.lang.foreign.Arena; -import java.lang.foreign.MemorySegment; -import java.nio.ByteBuffer; -import java.util.ArrayList; -import java.util.List; - -/// Encoder/decoder for {@code vortex.list}. -/// -///

Wire format (per Rust vtable): -///

    -///
  • Buffers: 0 -///
  • Metadata: protobuf {@code ListMetadata} — {@code elements_len} (u64) + {@code offset_ptype} (PType). -///
  • Children: 2 or 3. -/// {@code children[0]} = elements array (len = elements_len, dtype = elementType). -/// {@code children[1]} = offsets array (len = outerLen + 1, dtype = offset_ptype, non-nullable). -/// {@code children[2]} = validity (optional, Bool, len = outerLen). -///
-public final class ListEncoding implements Encoding { - - /// Creates a new {@code ListEncoding} instance; use via {@link EncodingRegistry}. - public ListEncoding() { - } - - @Override - public EncodingId encodingId() { - return EncodingId.VORTEX_LIST; - } - - @Override - public boolean accepts(DType dtype) { - return dtype instanceof DType.List; - } - - @Override - public EncodeResult encode(DType dtype, Object data) { - return Encoder.encode((DType.List) dtype, (ListData) data); - } - - @Override - public Array decode(DecodeContext ctx) { - return Decoder.decode(ctx); - } - - private static final class Encoder { - - private static final List FALLBACK = List.of( - new PrimitiveEncoding(), new VarBinEncoding(), new BoolEncoding(), - new NullEncoding(), new ByteBoolEncoding()); - - static EncodeResult encode(DType.List dtype, ListData data) { - DType elementType = dtype.elementType(); - Encoding elemEncoding = findEncoding(elementType); - EncodeResult elemResult = elemEncoding.encode(elementType, data.elements()); - - List allBuffers = new ArrayList<>(elemResult.buffers()); - int elemBufCount = allBuffers.size(); - EncodeNode elemNode = EncodeNode.remapBufferIndices(elemResult.rootNode(), 0); - - long nOffsets = data.outerLen() + 1; - Arena arena = Arena.ofAuto(); - MemorySegment offsetsBuf = arena.allocate(nOffsets * Long.BYTES, Long.BYTES); - for (int i = 0; i < nOffsets; i++) { - offsetsBuf.setAtIndex(PTypeIO.LE_LONG, i, data.offsets()[i]); - } - allBuffers.add(offsetsBuf); - EncodeNode offsetsNode = EncodeNode.leaf(EncodingId.VORTEX_PRIMITIVE, elemBufCount); - - long elementsLen = data.offsets()[(int) data.outerLen()]; - byte[] metaBytes = EncodingProtos.ListMetadata.newBuilder() - .setElementsLen(elementsLen) - .setOffsetPtype(io.github.dfa1.vortex.proto.DTypeProtos.PType.forNumber(PType.I64.ordinal())) - .build() - .toByteArray(); - - EncodeNode root = new EncodeNode( - EncodingId.VORTEX_LIST, - ByteBuffer.wrap(metaBytes), - new EncodeNode[]{elemNode, offsetsNode}, - new int[]{}); - return new EncodeResult(root, List.copyOf(allBuffers), null, null); - } - - private static Encoding findEncoding(DType dtype) { - for (Encoding enc : FALLBACK) { - if (enc.accepts(dtype)) { - return enc; - } - } - throw new UnsupportedOperationException("no fallback encoding for dtype: " + dtype); - } - } - - private static final class Decoder { - - static Array decode(DecodeContext ctx) { - if (!(ctx.dtype() instanceof DType.List listDtype)) { - throw new VortexException(EncodingId.VORTEX_LIST, - "expected DType.List, got " + ctx.dtype()); - } - - int nchildren = ctx.node().children().length; - if (nchildren < 2 || nchildren > 3) { - throw new VortexException(EncodingId.VORTEX_LIST, - "expected 2 or 3 children, got " + nchildren); - } - - EncodingProtos.ListMetadata meta; - try { - meta = EncodingProtos.ListMetadata.parseFrom(ctx.metadata().duplicate()); - } catch (InvalidProtocolBufferException e) { - throw new VortexException(EncodingId.VORTEX_LIST, "invalid metadata", e); - } - - long elementsLen = meta.getElementsLen(); - PType offsetPtype = PType.values()[meta.getOffsetPtype().getNumber()]; - long outerLen = ctx.rowCount(); - - DType elementDtype = listDtype.elementType(); - DType offsetsDtype = new DType.Primitive(offsetPtype, false); - - Array elements = decodeChildAs(ctx, 0, elementDtype, elementsLen); - Array offsets = decodeChildAs(ctx, 1, offsetsDtype, outerLen + 1); - - return new ListArray(listDtype, outerLen, elements, offsets); - } - - private static Array decodeChildAs(DecodeContext parent, int childIdx, DType dtype, long rowCount) { - ArrayNode childNode = parent.node().children()[childIdx]; - DecodeContext childCtx = new DecodeContext( - childNode, dtype, rowCount, - parent.segmentBuffers(), parent.registry(), parent.arena()); - return parent.registry().decode(childCtx); - } - } -} diff --git a/core/src/main/java/io/github/dfa1/vortex/encoding/ListViewData.java b/core/src/main/java/io/github/dfa1/vortex/encoding/ListViewData.java deleted file mode 100644 index 945b7e7..0000000 --- a/core/src/main/java/io/github/dfa1/vortex/encoding/ListViewData.java +++ /dev/null @@ -1,15 +0,0 @@ -package io.github.dfa1.vortex.encoding; - -/// Input data for encoding a {@code vortex.listview} column. -/// -///

{@code elements} is the flat array of all inner values (may be non-contiguous on disk). -/// {@code offsets[i]} is the start index of list {@code i} in elements. -/// {@code sizes[i]} is the number of elements in list {@code i}. -/// Both {@code offsets} and {@code sizes} have length {@code outerLen}. -/// -/// @param elements flat array of all inner values (may be non-contiguous) -/// @param offsets start index of each list in {@code elements}; length must equal {@code outerLen} -/// @param sizes element count for each list; length must equal {@code outerLen} -/// @param outerLen number of outer list elements -public record ListViewData(Object elements, int[] offsets, int[] sizes, long outerLen) { -} diff --git a/core/src/main/java/io/github/dfa1/vortex/encoding/ListViewEncoding.java b/core/src/main/java/io/github/dfa1/vortex/encoding/ListViewEncoding.java deleted file mode 100644 index 9e45470..0000000 --- a/core/src/main/java/io/github/dfa1/vortex/encoding/ListViewEncoding.java +++ /dev/null @@ -1,162 +0,0 @@ -package io.github.dfa1.vortex.encoding; - -import com.google.protobuf.InvalidProtocolBufferException; -import io.github.dfa1.vortex.core.DType; -import io.github.dfa1.vortex.core.PType; -import io.github.dfa1.vortex.core.VortexException; -import io.github.dfa1.vortex.core.array.Array; -import io.github.dfa1.vortex.core.array.ListViewArray; -import io.github.dfa1.vortex.proto.EncodingProtos; - -import java.lang.foreign.Arena; -import java.lang.foreign.MemorySegment; -import java.nio.ByteBuffer; -import java.util.ArrayList; -import java.util.List; - -/// Encoder/decoder for {@code vortex.listview}. -/// -///

Wire format (per Rust vtable): -///

    -///
  • Buffers: 0 -///
  • Metadata: protobuf {@code ListViewMetadata} — {@code elements_len} (u64), -/// {@code offset_ptype} (PType), {@code size_ptype} (PType). -///
  • Children: 3 or 4. -/// {@code children[0]} = elements (len = elements_len, dtype = elementType). -/// {@code children[1]} = offsets (len = outerLen, dtype = offset_ptype, non-nullable). -/// {@code children[2]} = sizes (len = outerLen, dtype = size_ptype, non-nullable). -/// {@code children[3]} = validity (optional, Bool, len = outerLen). -///
-/// -///

Unlike {@code vortex.list}, offsets and sizes have length {@code outerLen} (not outerLen+1); -/// list {@code i} covers {@code elements[offsets[i]..offsets[i]+sizes[i]]}. -public final class ListViewEncoding implements Encoding { - - /// Creates a new {@code ListViewEncoding} instance; use via {@link EncodingRegistry}. - public ListViewEncoding() { - } - - @Override - public EncodingId encodingId() { - return EncodingId.VORTEX_LISTVIEW; - } - - @Override - public boolean accepts(DType dtype) { - return dtype instanceof DType.List; - } - - @Override - public EncodeResult encode(DType dtype, Object data) { - return Encoder.encode((DType.List) dtype, (ListViewData) data); - } - - @Override - public Array decode(DecodeContext ctx) { - return Decoder.decode(ctx); - } - - private static final class Encoder { - - private static final List FALLBACK = List.of( - new PrimitiveEncoding(), new VarBinEncoding(), new BoolEncoding(), - new NullEncoding(), new ByteBoolEncoding()); - - static EncodeResult encode(DType.List dtype, ListViewData data) { - DType elementType = dtype.elementType(); - Encoding elemEncoding = findEncoding(elementType); - EncodeResult elemResult = elemEncoding.encode(elementType, data.elements()); - - List allBuffers = new ArrayList<>(elemResult.buffers()); - int elemBufCount = allBuffers.size(); - EncodeNode elemNode = EncodeNode.remapBufferIndices(elemResult.rootNode(), 0); - - Arena arena = Arena.ofAuto(); - long n = data.outerLen(); - - MemorySegment offsetsBuf = arena.allocate(n * Integer.BYTES, Integer.BYTES); - for (int i = 0; i < n; i++) { - offsetsBuf.setAtIndex(PTypeIO.LE_INT, i, data.offsets()[i]); - } - allBuffers.add(offsetsBuf); - EncodeNode offsetsNode = EncodeNode.leaf(EncodingId.VORTEX_PRIMITIVE, elemBufCount); - - MemorySegment sizesBuf = arena.allocate(n * Integer.BYTES, Integer.BYTES); - for (int i = 0; i < n; i++) { - sizesBuf.setAtIndex(PTypeIO.LE_INT, i, data.sizes()[i]); - } - allBuffers.add(sizesBuf); - EncodeNode sizesNode = EncodeNode.leaf(EncodingId.VORTEX_PRIMITIVE, elemBufCount + 1); - - long elementsLen = java.lang.reflect.Array.getLength(data.elements()); - byte[] metaBytes = EncodingProtos.ListViewMetadata.newBuilder() - .setElementsLen(elementsLen) - .setOffsetPtype(io.github.dfa1.vortex.proto.DTypeProtos.PType.forNumber(PType.I32.ordinal())) - .setSizePtype(io.github.dfa1.vortex.proto.DTypeProtos.PType.forNumber(PType.I32.ordinal())) - .build() - .toByteArray(); - - EncodeNode root = new EncodeNode( - EncodingId.VORTEX_LISTVIEW, - ByteBuffer.wrap(metaBytes), - new EncodeNode[]{elemNode, offsetsNode, sizesNode}, - new int[]{}); - return new EncodeResult(root, List.copyOf(allBuffers), null, null); - } - - private static Encoding findEncoding(DType dtype) { - for (Encoding enc : FALLBACK) { - if (enc.accepts(dtype)) { - return enc; - } - } - throw new UnsupportedOperationException("no fallback encoding for dtype: " + dtype); - } - } - - private static final class Decoder { - - static Array decode(DecodeContext ctx) { - if (!(ctx.dtype() instanceof DType.List listDtype)) { - throw new VortexException(EncodingId.VORTEX_LISTVIEW, - "expected DType.List, got " + ctx.dtype()); - } - - int nchildren = ctx.node().children().length; - if (nchildren < 3 || nchildren > 4) { - throw new VortexException(EncodingId.VORTEX_LISTVIEW, - "expected 3 or 4 children, got " + nchildren); - } - - EncodingProtos.ListViewMetadata meta; - try { - meta = EncodingProtos.ListViewMetadata.parseFrom(ctx.metadata().duplicate()); - } catch (InvalidProtocolBufferException e) { - throw new VortexException(EncodingId.VORTEX_LISTVIEW, "invalid metadata", e); - } - - long elementsLen = meta.getElementsLen(); - PType offsetPtype = PType.values()[meta.getOffsetPtype().getNumber()]; - PType sizePtype = PType.values()[meta.getSizePtype().getNumber()]; - long outerLen = ctx.rowCount(); - - DType elementDtype = listDtype.elementType(); - DType offsetsDtype = new DType.Primitive(offsetPtype, false); - DType sizesDtype = new DType.Primitive(sizePtype, false); - - Array elements = decodeChildAs(ctx, 0, elementDtype, elementsLen); - Array offsets = decodeChildAs(ctx, 1, offsetsDtype, outerLen); - Array sizes = decodeChildAs(ctx, 2, sizesDtype, outerLen); - - return new ListViewArray(listDtype, outerLen, elements, offsets, sizes); - } - - private static Array decodeChildAs(DecodeContext parent, int childIdx, DType dtype, long rowCount) { - ArrayNode childNode = parent.node().children()[childIdx]; - DecodeContext childCtx = new DecodeContext( - childNode, dtype, rowCount, - parent.segmentBuffers(), parent.registry(), parent.arena()); - return parent.registry().decode(childCtx); - } - } -} diff --git a/core/src/main/java/io/github/dfa1/vortex/encoding/MaskedEncoding.java b/core/src/main/java/io/github/dfa1/vortex/encoding/MaskedEncoding.java deleted file mode 100644 index abab0db..0000000 --- a/core/src/main/java/io/github/dfa1/vortex/encoding/MaskedEncoding.java +++ /dev/null @@ -1,75 +0,0 @@ -package io.github.dfa1.vortex.encoding; - -import io.github.dfa1.vortex.core.DType; -import io.github.dfa1.vortex.core.VortexException; -import io.github.dfa1.vortex.core.array.Array; -import io.github.dfa1.vortex.core.array.BoolArray; -import io.github.dfa1.vortex.core.array.MaskedArray; - -/// Decoder for {@code vortex.masked}. -/// -///

Wire format: -///

    -///
  • Metadata: empty.
    • -///
  • Buffers: none.
    • -///
  • Child 0: payload array encoded with non-nullable dtype (no actual nulls by invariant).
    • -///
  • Child 1 (optional): validity bitmap, dtype {@code Bool(false)}. Absent means AllValid.
    • -///
-public final class MaskedEncoding implements Encoding { - - /// Creates a new {@code MaskedEncoding} instance; use via {@link EncodingRegistry}. - public MaskedEncoding() { - } - - @Override - public EncodingId encodingId() { - return EncodingId.VORTEX_MASKED; - } - - @Override - public EncodeResult encode(DType dtype, Object data) { - throw new VortexException(EncodingId.VORTEX_MASKED, "encode not yet implemented"); - } - - @Override - public Array decode(DecodeContext ctx) { - return Decoder.decode(ctx); - } - - private static final class Decoder { - - static Array decode(DecodeContext ctx) { - if (ctx.node().bufferIndices().length != 0) { - throw new VortexException(EncodingId.VORTEX_MASKED, - "expected 0 buffers, got " + ctx.node().bufferIndices().length); - } - int numChildren = ctx.node().children().length; - if (numChildren < 1 || numChildren > 2) { - throw new VortexException(EncodingId.VORTEX_MASKED, - "expected 1 or 2 children, got " + numChildren); - } - - Array child = decodeChild(ctx, 0, ctx.dtype().withNullable(false), ctx.rowCount()); - - BoolArray validity = null; - if (numChildren == 2) { - Array validityArray = decodeChild(ctx, 1, new DType.Bool(false), ctx.rowCount()); - if (!(validityArray instanceof BoolArray ba)) { - throw new VortexException(EncodingId.VORTEX_MASKED, - "validity child decoded to unexpected type: " + validityArray.getClass().getSimpleName()); - } - validity = ba; - } - - return new MaskedArray(child, validity); - } - - private static Array decodeChild(DecodeContext parent, int idx, DType dtype, long rowCount) { - ArrayNode childNode = parent.node().children()[idx]; - DecodeContext childCtx = new DecodeContext( - childNode, dtype, rowCount, - parent.segmentBuffers(), parent.registry(), parent.arena()); - return parent.registry().decode(childCtx); - } - } -} diff --git a/core/src/main/java/io/github/dfa1/vortex/encoding/NullEncoding.java b/core/src/main/java/io/github/dfa1/vortex/encoding/NullEncoding.java deleted file mode 100644 index e566176..0000000 --- a/core/src/main/java/io/github/dfa1/vortex/encoding/NullEncoding.java +++ /dev/null @@ -1,38 +0,0 @@ -package io.github.dfa1.vortex.encoding; - -import io.github.dfa1.vortex.core.array.Array; -import io.github.dfa1.vortex.core.array.NullArray; -import io.github.dfa1.vortex.core.DType; - -import java.util.List; - -/// Encoding for {@code vortex.null} — all-null arrays. -/// -/// No buffers, no children, empty metadata. Decode returns a [NullArray] of the requested length. -public final class NullEncoding implements Encoding { - - /// Creates a new {@code NullEncoding} instance; use via {@link EncodingRegistry}. - public NullEncoding() { - } - - @Override - public EncodingId encodingId() { - return EncodingId.VORTEX_NULL; - } - - @Override - public boolean accepts(DType dtype) { - return dtype instanceof DType.Null; - } - - @Override - public EncodeResult encode(DType dtype, Object data) { - EncodeNode root = new EncodeNode(EncodingId.VORTEX_NULL, null, new EncodeNode[0], new int[0]); - return new EncodeResult(root, List.of(), null, null); - } - - @Override - public Array decode(DecodeContext ctx) { - return new NullArray(ctx.dtype(), ctx.rowCount()); - } -} diff --git a/core/src/main/java/io/github/dfa1/vortex/encoding/PTypeIO.java b/core/src/main/java/io/github/dfa1/vortex/encoding/PTypeIO.java deleted file mode 100644 index 5b46fcc..0000000 --- a/core/src/main/java/io/github/dfa1/vortex/encoding/PTypeIO.java +++ /dev/null @@ -1,140 +0,0 @@ -package io.github.dfa1.vortex.encoding; - -import io.github.dfa1.vortex.core.PType; - -import java.lang.foreign.Arena; -import java.lang.foreign.MemorySegment; -import java.lang.foreign.ValueLayout; -import java.nio.ByteOrder; -import java.lang.invoke.MethodHandle; -import java.lang.invoke.MethodHandles; -import java.lang.invoke.MethodType; -import java.lang.invoke.VarHandle; - -/// Bulk I/O helpers for primitive ptypes, backed by `MemorySegment`/`ValueLayout`/`MethodHandle`. -/// -/// `SETTERS[ordinal]` is a MethodHandle `(MemorySegment, long offset, long bits) -> void` -/// that narrows / bit-reinterprets `bits` to the carrier type of the ptype and writes via -/// the unaligned little-endian VarHandle. This lets hot loops avoid per-element `switch` -/// dispatch on `PType`. -/// -/// The LE_* layout constants are public so callers outside this package can share them -/// without duplicating the `withOrder(LITTLE_ENDIAN)` boilerplate. -public final class PTypeIO { - - /// Unaligned little-endian layout for 16-bit shorts. - public static final ValueLayout.OfShort LE_SHORT = ValueLayout.JAVA_SHORT_UNALIGNED.withOrder(ByteOrder.LITTLE_ENDIAN); - /// Unaligned little-endian layout for 32-bit ints. - public static final ValueLayout.OfInt LE_INT = ValueLayout.JAVA_INT_UNALIGNED.withOrder(ByteOrder.LITTLE_ENDIAN); - /// Unaligned little-endian layout for 64-bit longs. - public static final ValueLayout.OfLong LE_LONG = ValueLayout.JAVA_LONG_UNALIGNED.withOrder(ByteOrder.LITTLE_ENDIAN); - /// Unaligned little-endian layout for 32-bit floats. - public static final ValueLayout.OfFloat LE_FLOAT = ValueLayout.JAVA_FLOAT_UNALIGNED.withOrder(ByteOrder.LITTLE_ENDIAN); - /// Unaligned little-endian layout for 64-bit doubles. - public static final ValueLayout.OfDouble LE_DOUBLE = ValueLayout.JAVA_DOUBLE_UNALIGNED.withOrder(ByteOrder.LITTLE_ENDIAN); - - private static final MethodHandle[] SETTERS = buildSetters(); - - private PTypeIO() { - } - - /// Little-endian, unaligned `ValueLayout` for the given ptype. - static ValueLayout valueLayout(PType ptype) { - return switch (ptype) { - case I8, U8 -> ValueLayout.JAVA_BYTE; - case I16, U16 -> LE_SHORT; - case I32, U32 -> LE_INT; - case I64, U64 -> LE_LONG; - case F32 -> LE_FLOAT; - case F64 -> LE_DOUBLE; - case F16 -> LE_SHORT; - }; - } - - private static MethodHandle[] buildSetters() { - MethodHandle[] mhs = new MethodHandle[PType.values().length]; - MethodHandles.Lookup lookup = MethodHandles.lookup(); - - // (long) -> int narrowing — used for raw-bits to int for F32. - MethodHandle longToInt = MethodHandles.explicitCastArguments( - MethodHandles.identity(int.class), - MethodType.methodType(int.class, long.class)); - - MethodHandle intBitsToFloat; - MethodHandle longBitsToDouble; - try { - intBitsToFloat = lookup.findStatic(Float.class, "intBitsToFloat", - MethodType.methodType(float.class, int.class)); - longBitsToDouble = lookup.findStatic(Double.class, "longBitsToDouble", - MethodType.methodType(double.class, long.class)); - } catch (ReflectiveOperationException e) { - throw new AssertionError(e); - } - // (long) -> float (raw-bits reinterpret) for F32. - MethodHandle longBitsToFloat = MethodHandles.filterArguments(intBitsToFloat, 0, longToInt); - - for (PType p : PType.values()) { - VarHandle vh = valueLayout(p).varHandle(); - // VarHandle SET access: (MemorySegment, long offset, T) -> void - MethodHandle setter = vh.toMethodHandle(VarHandle.AccessMode.SET); - - MethodHandle narrowed; - switch (p) { - case F32 -> narrowed = MethodHandles.filterArguments(setter, 2, longBitsToFloat); - case F64 -> narrowed = MethodHandles.filterArguments(setter, 2, longBitsToDouble); - default -> narrowed = MethodHandles.explicitCastArguments(setter, - MethodType.methodType(void.class, MemorySegment.class, long.class, long.class)); - } - mhs[p.ordinal()] = narrowed; - } - return mhs; - } - - /// Write `bits` at `offset` in `seg`, narrowed or bit-reinterpreted to the ptype's carrier. - /// Float bits use `Float.intBitsToFloat` / `Double.longBitsToDouble` semantics. - static void set(MemorySegment seg, long offset, PType ptype, long bits) { - try { - SETTERS[ptype.ordinal()].invokeExact(seg, offset, bits); - } catch (Throwable t) { - throw new AssertionError(t); - } - } - - /// Bulk-copy a primitive Java array into a freshly allocated little-endian segment. - /// Element layout conversion (host order → LE) is delegated to `MemorySegment.copy`. - static MemorySegment copyArray(PType ptype, Object typedArray, int count) { - MemorySegment src; - ValueLayout srcLayout; - switch (typedArray) { - case byte[] a -> { - src = MemorySegment.ofArray(a); - srcLayout = ValueLayout.JAVA_BYTE; - } - case short[] a -> { - src = MemorySegment.ofArray(a); - srcLayout = ValueLayout.JAVA_SHORT; - } - case int[] a -> { - src = MemorySegment.ofArray(a); - srcLayout = ValueLayout.JAVA_INT; - } - case long[] a -> { - src = MemorySegment.ofArray(a); - srcLayout = ValueLayout.JAVA_LONG; - } - case float[] a -> { - src = MemorySegment.ofArray(a); - srcLayout = ValueLayout.JAVA_FLOAT; - } - case double[] a -> { - src = MemorySegment.ofArray(a); - srcLayout = ValueLayout.JAVA_DOUBLE; - } - default -> throw new UnsupportedOperationException( - "unsupported array type: " + typedArray.getClass()); - } - MemorySegment dst = Arena.ofAuto().allocate((long) count * ptype.byteSize()); - MemorySegment.copy(src, srcLayout, 0L, dst, valueLayout(ptype), 0L, count); - return dst; - } -} diff --git a/core/src/main/java/io/github/dfa1/vortex/encoding/PatchedEncoding.java b/core/src/main/java/io/github/dfa1/vortex/encoding/PatchedEncoding.java deleted file mode 100644 index 9cc35a9..0000000 --- a/core/src/main/java/io/github/dfa1/vortex/encoding/PatchedEncoding.java +++ /dev/null @@ -1,28 +0,0 @@ -package io.github.dfa1.vortex.encoding; - -import io.github.dfa1.vortex.core.DType; -import io.github.dfa1.vortex.core.VortexException; -import io.github.dfa1.vortex.core.array.Array; - -/// Stub for {@code vortex.patched} — not yet implemented. -public final class PatchedEncoding implements Encoding { - - /// Creates a new {@code PatchedEncoding} instance; use via {@link EncodingRegistry}. - public PatchedEncoding() { - } - - @Override - public EncodingId encodingId() { - return EncodingId.VORTEX_PATCHED; - } - - @Override - public EncodeResult encode(DType dtype, Object data) { - throw new VortexException(EncodingId.VORTEX_PATCHED, "not yet implemented"); - } - - @Override - public Array decode(DecodeContext ctx) { - throw new VortexException(EncodingId.VORTEX_PATCHED, "not yet implemented"); - } -} diff --git a/core/src/main/java/io/github/dfa1/vortex/encoding/PcoBin.java b/core/src/main/java/io/github/dfa1/vortex/encoding/PcoBin.java deleted file mode 100644 index 0a6519a..0000000 --- a/core/src/main/java/io/github/dfa1/vortex/encoding/PcoBin.java +++ /dev/null @@ -1,9 +0,0 @@ -package io.github.dfa1.vortex.encoding; - -/// One bin in a pco latent variable: a numerical range [lower, lower + 2^offsetBits). -/// -/// {@code weight} is the bin's count in the tANS table (sum of weights == table size). -/// {@code lower} is the raw unsigned lower bound (U64 for 64-bit latents). -/// {@code offsetBits} is the log2 of the range size (0 = single value). -record PcoBin(int weight, long lower, int offsetBits) { -} diff --git a/core/src/main/java/io/github/dfa1/vortex/encoding/PcoEncoding.java b/core/src/main/java/io/github/dfa1/vortex/encoding/PcoEncoding.java deleted file mode 100644 index a565ee0..0000000 --- a/core/src/main/java/io/github/dfa1/vortex/encoding/PcoEncoding.java +++ /dev/null @@ -1,895 +0,0 @@ -package io.github.dfa1.vortex.encoding; - -import com.google.protobuf.InvalidProtocolBufferException; -import io.github.dfa1.vortex.core.ArrayStats; -import io.github.dfa1.vortex.core.DType; -import io.github.dfa1.vortex.core.PType; -import io.github.dfa1.vortex.core.VortexException; -import io.github.dfa1.vortex.core.array.Array; -import io.github.dfa1.vortex.core.array.BoolArray; -import io.github.dfa1.vortex.core.array.DoubleArray; -import io.github.dfa1.vortex.core.array.FloatArray; -import io.github.dfa1.vortex.core.array.IntArray; -import io.github.dfa1.vortex.core.array.LongArray; -import io.github.dfa1.vortex.core.array.MaskedArray; -import io.github.dfa1.vortex.core.array.ShortArray; -import io.github.dfa1.vortex.proto.EncodingProtos; - -import java.lang.foreign.MemorySegment; -import java.lang.foreign.SegmentAllocator; -import java.lang.foreign.ValueLayout; -import java.nio.ByteBuffer; -import java.nio.ByteOrder; - -/// Decoder for {@code vortex.pco} (pcodec numerical compression). -/// -///

Wire format (Vortex layer): -///

    -///
  • Metadata: {@code PcoMetadata} — 2-byte header (major.minor) + repeated {@code PcoChunkInfo}
    • -///
  • Buffers: {@code chunk_metas[0..N-1]} then {@code pages[0..M-1]}
    • -///
  • Optional child[0]: validity bitmap (Bool array); pco stores only valid values
    • -///
-/// -///

Wire format (pcodec layer, per chunk/page): -///

    -///
  • Chunk meta: [4b mode][extra mode bits][4b delta][extra delta bits] -/// [optional delta latent var (U32) for Lookback] -/// [per-latent: 4b ans_size_log, 15b n_bins, per-bin {weight-1, lower, offset_bits}] -/// [0–7b alignment]
    • -///
  • Classic/Dict page: [deltaOrder×dtypeSize b moments, 4×ansSizeLog b ANS states] -/// [0–7b alignment] [per 256-batch: ANS bits, offset bits]
    • -///
  • IntMult/FloatMult/FloatQuant page: [primary header][secondary header][0–7b alignment] -/// [per 256-batch: primary ANS+offsets, secondary ANS+offsets]
    • -///
  • Lookback page: [delta ANS states][stateN×dtypeSize moments][primary ANS states] -/// [0–7b alignment] [per 256-batch: delta ANS+offsets, primary ANS+offsets]
    • -///
  • All bit packing little-endian (LSB first)
    • -///
-/// -///

Supported: Classic, IntMult, FloatMult, FloatQuant, Dict modes; -/// None+Consecutive+Lookback+Conv1 delta; nullable (validity child[0]); -/// all integer/float ptypes except F16 (Conv1 additionally excludes 64-bit dtypes). -public final class PcoEncoding implements Encoding { - - /// Creates a new {@code PcoEncoding} instance; use via {@link EncodingRegistry}. - public PcoEncoding() { - } - - static final byte PCO_FORMAT_MAJOR = 0x04; - static final byte PCO_FORMAT_MINOR = 0x01; - - // bits needed to encode offset_bits field per latent type - static final int BITS_TO_ENCODE_OFFSET_BITS_64 = 7; // log2(64) + 1 - static final int BITS_TO_ENCODE_OFFSET_BITS_32 = 6; // log2(32) + 1 - static final int BITS_TO_ENCODE_OFFSET_BITS_16 = 5; // log2(16) + 1 - - @Override - public EncodingId encodingId() { - return EncodingId.VORTEX_PCO; - } - - @Override - public EncodeResult encode(DType dtype, Object data) { - return Encoder.encode(dtype, data); - } - - @Override - public Array decode(DecodeContext ctx) { - return Decoder.decode(ctx); - } - - private static final class Encoder { - - static EncodeResult encode(DType dtype, Object data) { - throw new VortexException(EncodingId.VORTEX_PCO, - "encode not implemented — pco encode port pending"); - } - } - - static final class Decoder { - - private static final ValueLayout.OfLong LE_LONG = - ValueLayout.JAVA_LONG_UNALIGNED.withOrder(ByteOrder.LITTLE_ENDIAN); - - static Array decode(DecodeContext ctx) { - EncodingProtos.PcoMetadata meta = parseMeta(ctx); - validateHeader(meta); - - DType dtype = ctx.dtype(); - if (!(dtype instanceof DType.Primitive dt)) { - throw new VortexException(EncodingId.VORTEX_PCO, - "pco decode requires Primitive dtype, got: " + dtype); - } - PType ptype = dt.ptype(); - int dtypeSize = dtypeSize(ptype); - - long n = ctx.rowCount(); - - // Nullable: child[0] is a validity bitmap; pco encodes only valid values. - BoolArray validity = null; - long validCount = n; - if (ctx.node().children().length > 0) { - Array validityArr = decodeChild(ctx, 0, new DType.Bool(false), n); - if (!(validityArr instanceof BoolArray ba)) { - throw new VortexException(EncodingId.VORTEX_PCO, - "pco validity child must be Bool, got: " + validityArr.getClass().getSimpleName()); - } - validity = ba; - validCount = 0; - for (long i = 0; i < n; i++) { - if (validity.getBoolean(i)) { - validCount++; - } - } - } - - // decodePage always writes U64 latents (8 bytes per element). - MemorySegment rawLatents = ctx.arena().allocate(validCount * Long.BYTES); - - int nChunks = meta.getChunksCount(); - int bufIdx = 0; - long rawByteOffset = 0L; - - long[] batchLowers1 = new long[PcoTansDecoder.BATCH_N]; - int[] batchOffsetBits1 = new int[PcoTansDecoder.BATCH_N]; - long[] batchLowers2 = new long[PcoTansDecoder.BATCH_N]; - int[] batchOffsetBits2 = new int[PcoTansDecoder.BATCH_N]; - - for (int c = 0; c < nChunks; c++) { - EncodingProtos.PcoChunkInfo chunkInfo = meta.getChunks(c); - MemorySegment chunkMetaBuf = ctx.buffer(bufIdx++); - PcoChunkMeta chunkMeta = readChunkMeta(chunkMetaBuf, dtypeSize); - - int mode = chunkMeta.mode(); - int deltaVariant = chunkMeta.deltaVariant(); - long chunkStartOffset = rawByteOffset; - - // Compute total values in this chunk. - int chunkN = 0; - for (int p = 0; p < chunkInfo.getPagesCount(); p++) { - chunkN += chunkInfo.getPages(p).getNValues(); - } - - if (deltaVariant == 3) { - // Conv1 delta: 64-bit check is in readChunkMeta; 64-bit case never reaches here. - PcoTansDecoder primaryTans = PcoTansDecoder.build( - chunkMeta.ansSizeLog(), chunkMeta.bins()); - for (int p = 0; p < chunkInfo.getPagesCount(); p++) { - int pageN = chunkInfo.getPages(p).getNValues(); - MemorySegment pageBuf = ctx.buffer(bufIdx++); - rawByteOffset = decodeConv1Page( - primaryTans, chunkMeta.ansSizeLog(), - chunkMeta.conv1Weights().length, - chunkMeta.conv1Quantization(), chunkMeta.conv1Bias(), - chunkMeta.conv1Weights(), - dtypeSize, pageBuf, pageN, - rawLatents, rawByteOffset, - batchLowers1, batchOffsetBits1); - } - } else if (deltaVariant == 2) { - // Lookback delta: currently only Classic mode supported. - if (mode != 0) { - throw new VortexException(EncodingId.VORTEX_PCO, - "pco Lookback delta with non-Classic mode " + mode + " not yet implemented"); - } - PcoTansDecoder deltaTans = PcoTansDecoder.build( - chunkMeta.deltaAnsSizeLog(), chunkMeta.deltaBins()); - PcoTansDecoder primaryTans = PcoTansDecoder.build( - chunkMeta.ansSizeLog(), chunkMeta.bins()); - int stateN = 1 << chunkMeta.stateNLog(); - int windowN = 1 << chunkMeta.windowNLog(); - long mid = typeMid(dtypeSize); - long mask = typeMask(dtypeSize); - for (int p = 0; p < chunkInfo.getPagesCount(); p++) { - int pageN = chunkInfo.getPages(p).getNValues(); - MemorySegment pageBuf = ctx.buffer(bufIdx++); - rawByteOffset = decodeLookbackPage( - deltaTans, chunkMeta.deltaAnsSizeLog(), - primaryTans, chunkMeta.ansSizeLog(), - stateN, windowN, mid, mask, - dtypeSize, pageBuf, pageN, - rawLatents, rawByteOffset, ctx.arena(), - batchLowers1, batchOffsetBits1, - batchLowers2, batchOffsetBits2); - } - } else if (mode == 0 || mode == 4) { - // Single-latent var: Classic or Dict. - int primaryDtypeSize = (mode == 4) ? 32 : dtypeSize; - PcoTansDecoder tans = PcoTansDecoder.build(chunkMeta.ansSizeLog(), chunkMeta.bins()); - for (int p = 0; p < chunkInfo.getPagesCount(); p++) { - int pageN = chunkInfo.getPages(p).getNValues(); - MemorySegment pageBuf = ctx.buffer(bufIdx++); - rawByteOffset = decodeClassicPage(tans, chunkMeta.ansSizeLog(), - chunkMeta.deltaOrder(), primaryDtypeSize, - pageBuf, pageN, rawLatents, rawByteOffset, - batchLowers1, batchOffsetBits1); - } - if (mode == 4) { - combineDict(chunkMeta.dict(), chunkN, rawLatents, chunkStartOffset); - } - } else { - // Two-latent var: IntMult (1), FloatMult (2), FloatQuant (3). - long base = chunkMeta.base(); - int primaryAnsSizeLog = chunkMeta.ansSizeLog(); - int secondaryAnsSizeLog = chunkMeta.secondaryAnsSizeLog(); - PcoTansDecoder primaryTans = PcoTansDecoder.build(primaryAnsSizeLog, chunkMeta.bins()); - PcoTansDecoder secondaryTans = PcoTansDecoder.build(secondaryAnsSizeLog, chunkMeta.secondaryBins()); - int deltaOrder = chunkMeta.deltaOrder(); - int secondaryDeltaOrder = chunkMeta.secondaryUsesDelta() ? deltaOrder : 0; - - MemorySegment rawAdjs = ctx.arena().allocate((long) chunkN * Long.BYTES); - long adjByteOffset = 0L; - for (int p = 0; p < chunkInfo.getPagesCount(); p++) { - int pageN = chunkInfo.getPages(p).getNValues(); - MemorySegment pageBuf = ctx.buffer(bufIdx++); - decodeIntMultPage(primaryTans, primaryAnsSizeLog, deltaOrder, - secondaryTans, secondaryAnsSizeLog, secondaryDeltaOrder, - dtypeSize, pageBuf, pageN, - rawLatents, rawByteOffset, - rawAdjs, adjByteOffset, - batchLowers1, batchOffsetBits1, - batchLowers2, batchOffsetBits2); - rawByteOffset += (long) pageN * Long.BYTES; - adjByteOffset += (long) pageN * Long.BYTES; - } - - if (mode == 1) { - long mask = typeMask(dtypeSize); - for (int i = 0; i < chunkN; i++) { - long off = chunkStartOffset + (long) i * Long.BYTES; - long mult = rawLatents.get(LE_LONG, off); - long adj = rawAdjs.get(LE_LONG, (long) i * Long.BYTES); - rawLatents.set(LE_LONG, off, (mult * base + adj) & mask); - } - } else if (mode == 2) { - combineFloatMult(ptype, base, chunkN, rawLatents, chunkStartOffset, rawAdjs); - } else { - combineFloatQuant(ptype, chunkMeta.quantizeK(), chunkN, rawLatents, chunkStartOffset, rawAdjs); - } - } - } - - // Convert raw U-latents → typed values; resize from 8-byte slots to elemBytes. - int elemBytes = ptype.byteSize(); - MemorySegment compactOut = ctx.arena().allocate(validCount * elemBytes); - for (long i = 0; i < validCount; i++) { - long latent = rawLatents.get(LE_LONG, i * Long.BYTES); - PTypeIO.set(compactOut, i * elemBytes, ptype, fromLatentOrdered(latent, ptype)); - } - - if (validity == null) { - return toArray(dtype, n, compactOut); - } - - // Scatter validCount compact values into full-length output; null slots stay zeroed. - MemorySegment fullOut = ctx.arena().allocate(n * elemBytes); - long srcOff = 0; - for (long i = 0; i < n; i++) { - if (validity.getBoolean(i)) { - MemorySegment.copy(compactOut, srcOff, fullOut, i * elemBytes, elemBytes); - srcOff += elemBytes; - } - } - DType nonNullDtype = new DType.Primitive(ptype, false); - return new MaskedArray(toArray(nonNullDtype, n, fullOut), validity); - } - - private static Array decodeChild(DecodeContext parent, int idx, DType dtype, long rowCount) { - ArrayNode childNode = parent.node().children()[idx]; - DecodeContext childCtx = new DecodeContext( - childNode, dtype, rowCount, parent.segmentBuffers(), parent.registry(), parent.arena()); - return parent.registry().decode(childCtx); - } - - /// Decode one Classic-mode page into rawLatents and return the updated byte offset. - private static long decodeClassicPage(PcoTansDecoder tans, int ansSizeLog, int deltaOrder, - int primaryDtypeSize, MemorySegment pageBuf, int pageN, - MemorySegment rawLatents, long rawByteOffset, - long[] batchLowers, int[] batchOffsetBits) { - LeBitReader pageReader = new LeBitReader(pageBuf); - - long[] moments = new long[deltaOrder]; - for (int m = 0; m < deltaOrder; m++) { - moments[m] = pageReader.readBits(primaryDtypeSize); - } - - int[] stateIdxs = new int[PcoTansDecoder.ANS_INTERLEAVING]; - for (int i = 0; i < PcoTansDecoder.ANS_INTERLEAVING; i++) { - stateIdxs[i] = (int) pageReader.readBits(ansSizeLog); - } - pageReader.alignToByte(); - - int decodedN = pageN - deltaOrder; - tans.decodePage(pageReader, stateIdxs, decodedN, rawLatents, rawByteOffset, - batchLowers, batchOffsetBits); - - if (deltaOrder > 0) { - applyConsecutiveDelta(rawLatents, rawByteOffset, pageN, moments, primaryDtypeSize); - } - - return rawByteOffset + (long) pageN * Long.BYTES; - } - - /// Decode one IntMult/FloatMult/FloatQuant-mode page: primary latents into rawMults, - /// secondary into rawAdjs. Page body interleaved per 256-value batch. - private static void decodeIntMultPage( - PcoTansDecoder primaryTans, int primaryAnsSizeLog, int deltaOrder, - PcoTansDecoder secondaryTans, int secondaryAnsSizeLog, int secondaryDeltaOrder, - int dtypeSize, MemorySegment pageBuf, int pageN, - MemorySegment rawMults, long multsOffset, - MemorySegment rawAdjs, long adjsOffset, - long[] batchLowersP, int[] batchOffsetBitsP, - long[] batchLowersS, int[] batchOffsetBitsS) { - LeBitReader pageReader = new LeBitReader(pageBuf); - - long[] primaryMoments = new long[deltaOrder]; - for (int m = 0; m < deltaOrder; m++) { - primaryMoments[m] = pageReader.readBits(dtypeSize); - } - int[] primaryStateIdxs = new int[PcoTansDecoder.ANS_INTERLEAVING]; - for (int i = 0; i < PcoTansDecoder.ANS_INTERLEAVING; i++) { - primaryStateIdxs[i] = (int) pageReader.readBits(primaryAnsSizeLog); - } - - long[] secondaryMoments = new long[secondaryDeltaOrder]; - for (int m = 0; m < secondaryDeltaOrder; m++) { - secondaryMoments[m] = pageReader.readBits(dtypeSize); - } - int[] secondaryStateIdxs = new int[PcoTansDecoder.ANS_INTERLEAVING]; - for (int i = 0; i < PcoTansDecoder.ANS_INTERLEAVING; i++) { - secondaryStateIdxs[i] = (int) pageReader.readBits(secondaryAnsSizeLog); - } - - pageReader.alignToByte(); - - int nRemaining = pageN; - long primaryPos = multsOffset; - long secondaryPos = adjsOffset; - - while (nRemaining > 0) { - int batchN = Math.min(nRemaining, PcoTansDecoder.BATCH_N); - int primaryPreDeltaN = Math.min(batchN, Math.max(0, nRemaining - deltaOrder)); - int secondaryPreDeltaN = Math.min(batchN, Math.max(0, nRemaining - secondaryDeltaOrder)); - - primaryTans.decodeBatch(pageReader, primaryStateIdxs, primaryPreDeltaN, - batchLowersP, batchOffsetBitsP, rawMults, primaryPos); - secondaryTans.decodeBatch(pageReader, secondaryStateIdxs, secondaryPreDeltaN, - batchLowersS, batchOffsetBitsS, rawAdjs, secondaryPos); - - primaryPos += (long) batchN * Long.BYTES; - secondaryPos += (long) batchN * Long.BYTES; - nRemaining -= batchN; - } - - if (deltaOrder > 0) { - applyConsecutiveDelta(rawMults, multsOffset, pageN, primaryMoments, dtypeSize); - } - if (secondaryDeltaOrder > 0) { - applyConsecutiveDelta(rawAdjs, adjsOffset, pageN, secondaryMoments, dtypeSize); - } - } - - /// Decode one Lookback-delta page into rawLatents. Per-page window seeded from page header. - /// - /// Page layout: [delta ANS states (U32)][stateN×dtypeSize initial values][primary ANS states] - /// [0–7b align] [per 256-batch: delta ANS+offsets (lookback idx), primary ANS+offsets (residuals)] - private static long decodeLookbackPage( - PcoTansDecoder deltaTans, int deltaAnsSizeLog, - PcoTansDecoder primaryTans, int primaryAnsSizeLog, - int stateN, int windowN, long mid, long mask, - int dtypeSize, MemorySegment pageBuf, int pageN, - MemorySegment rawLatents, long latentsOffset, - SegmentAllocator arena, - long[] batchLowersD, int[] batchOffsetBitsD, - long[] batchLowersP, int[] batchOffsetBitsP) { - if (pageN < stateN) { - throw new VortexException(EncodingId.VORTEX_PCO, - "pco corrupt lookback page: stateN " + stateN + " exceeds pageN " + pageN); - } - LeBitReader pageReader = new LeBitReader(pageBuf); - - // Delta page header: 0 moments (NoOp delta-of-delta) + 4 × deltaAnsSizeLog ANS states. - int[] deltaStateIdxs = new int[PcoTansDecoder.ANS_INTERLEAVING]; - for (int i = 0; i < PcoTansDecoder.ANS_INTERLEAVING; i++) { - deltaStateIdxs[i] = (int) pageReader.readBits(deltaAnsSizeLog); - } - - // Primary page header: stateN × dtypeSize initial values + 4 × primaryAnsSizeLog ANS states. - long[] initialState = new long[stateN]; - for (int m = 0; m < stateN; m++) { - initialState[m] = pageReader.readBits(dtypeSize); - } - int[] primaryStateIdxs = new int[PcoTansDecoder.ANS_INTERLEAVING]; - for (int i = 0; i < PcoTansDecoder.ANS_INTERLEAVING; i++) { - primaryStateIdxs[i] = (int) pageReader.readBits(primaryAnsSizeLog); - } - pageReader.alignToByte(); - - int decodeN = pageN - stateN; - if (decodeN > 1 << 23) { - throw new VortexException(EncodingId.VORTEX_PCO, - "pco corrupt lookback page: decodeN " + decodeN + " exceeds max 8388608"); - } - MemorySegment rawLookbacks = arena.allocate((long) decodeN * Long.BYTES); - MemorySegment rawResiduals = arena.allocate((long) decodeN * Long.BYTES); - - int remaining = decodeN; - long dPos = 0L; - long pPos = 0L; - while (remaining > 0) { - int batchN = Math.min(remaining, PcoTansDecoder.BATCH_N); - deltaTans.decodeBatch(pageReader, deltaStateIdxs, batchN, - batchLowersD, batchOffsetBitsD, rawLookbacks, dPos); - primaryTans.decodeBatch(pageReader, primaryStateIdxs, batchN, - batchLowersP, batchOffsetBitsP, rawResiduals, pPos); - dPos += (long) batchN * Long.BYTES; - pPos += (long) batchN * Long.BYTES; - remaining -= batchN; - } - - // Toggle-center undo (XOR mid) on residuals. - for (int i = 0; i < decodeN; i++) { - long off = (long) i * Long.BYTES; - rawResiduals.set(LE_LONG, off, (rawResiduals.get(LE_LONG, off) ^ mid) & mask); - } - - // Write initial state to output. - for (int i = 0; i < stateN; i++) { - rawLatents.set(LE_LONG, latentsOffset + (long) i * Long.BYTES, initialState[i] & mask); - } - - if (stateN > windowN) { - throw new VortexException(EncodingId.VORTEX_PCO, - "pco corrupt lookback: stateN " + stateN + " exceeds windowN " + windowN); - } - // Lookback reconstruction: window[0..windowN] seeded from initialState. - long[] window = new long[windowN + decodeN]; - for (int i = 0; i < stateN; i++) { - window[windowN - stateN + i] = initialState[i] & mask; - } - for (int i = 0; i < decodeN; i++) { - int lb = (int) rawLookbacks.get(LE_LONG, (long) i * Long.BYTES); - if (lb < 1 || lb > windowN) { - throw new VortexException(EncodingId.VORTEX_PCO, - "pco corrupt lookback index " + lb + " not in [1, " + windowN + "]"); - } - long decoded = (rawResiduals.get(LE_LONG, (long) i * Long.BYTES) + window[windowN + i - lb]) & mask; - window[windowN + i] = decoded; - rawLatents.set(LE_LONG, latentsOffset + (long) (stateN + i) * Long.BYTES, decoded); - } - - return latentsOffset + (long) pageN * Long.BYTES; - } - - /// Decode one Conv1-delta page into rawLatents and return the updated byte offset. - /// - /// Page layout: [order×dtypeSize initial state][4×ansSizeLog ANS states][0–7b align] - /// [per 256-batch: ANS bits + offset bits for (pageN-order) residuals] - /// - /// Port of {@code conv1::decode_in_place} from pcodec. - private static long decodeConv1Page( - PcoTansDecoder tans, int ansSizeLog, - int order, int quantization, long bias, long[] weights, - int dtypeSize, MemorySegment pageBuf, int pageN, - MemorySegment rawLatents, long latentsOffset, - long[] batchLowers, int[] batchOffsetBits) { - LeBitReader pageReader = new LeBitReader(pageBuf); - - long[] state = new long[order]; - for (int i = 0; i < order; i++) { - state[i] = pageReader.readBits(dtypeSize); - } - int[] stateIdxs = new int[PcoTansDecoder.ANS_INTERLEAVING]; - for (int i = 0; i < PcoTansDecoder.ANS_INTERLEAVING; i++) { - stateIdxs[i] = (int) pageReader.readBits(ansSizeLog); - } - pageReader.alignToByte(); - - int decodeN = pageN - order; - long mid = typeMid(dtypeSize); - long mask = typeMask(dtypeSize); - - // Write initial state directly into rawLatents. - for (int i = 0; i < order; i++) { - rawLatents.set(LE_LONG, latentsOffset + (long) i * Long.BYTES, state[i]); - } - - // Decode residuals directly into rawLatents[latentsOffset + order*8..]. - tans.decodePage(pageReader, stateIdxs, decodeN, rawLatents, - latentsOffset + (long) order * Long.BYTES, - batchLowers, batchOffsetBits); - - // Toggle-center decoded residuals in-place. - for (int i = order; i < pageN; i++) { - long off = latentsOffset + (long) i * Long.BYTES; - rawLatents.set(LE_LONG, off, (rawLatents.get(LE_LONG, off) ^ mid) & mask); - } - - // Reconstruct in-place: rawLatents[i] += predict(rawLatents[i-order..i]). - for (int i = order; i < pageN; i++) { - long pred = predictConv1(rawLatents, latentsOffset, i, order, - weights, bias, quantization, mask, dtypeSize); - long off = latentsOffset + (long) i * Long.BYTES; - rawLatents.set(LE_LONG, off, (rawLatents.get(LE_LONG, off) + pred) & mask); - } - - return latentsOffset + (long) pageN * Long.BYTES; - } - - /// Compute Conv1 prediction for position {@code pos} in {@code seg[baseOff..]}. - /// - /// Accumulator is i64 for dtypeSize=32, or i32 (bias/weights truncated) for dtypeSize=16. - /// {@code mask} is pre-computed by the caller to avoid a switch per call. - private static long predictConv1(MemorySegment seg, long baseOff, int pos, int order, - long[] weights, long bias, int quantization, long mask, int dtypeSize) { - // For dtypeSize=16, accumulator is i32 (truncate bias/weights from i64). - long s = (dtypeSize == 16) ? (int) bias : bias; - for (int k = 0; k < order; k++) { - long w = (dtypeSize == 16) ? (int) weights[k] : weights[k]; - long l = seg.get(LE_LONG, baseOff + (long) (pos - order + k) * Long.BYTES); - s += w * l; - } - if (s < 0) { - s = 0; - } - return (s >> quantization) & mask; - } - - /// Inverse of pcodec {@code to_latent_ordered}: maps raw U-latent back to typed bits. - private static long fromLatentOrdered(long latent, PType ptype) { - return switch (ptype) { - case I16 -> latent ^ 0x8000L; - case I32 -> latent ^ 0x80000000L; - case I64 -> latent ^ Long.MIN_VALUE; - case F32 -> { - long l32 = latent & 0xFFFFFFFFL; - yield (l32 & 0x80000000L) != 0 ? l32 ^ 0x80000000L : l32 ^ 0xFFFFFFFFL; - } - case F64 -> (latent & Long.MIN_VALUE) != 0 ? latent ^ Long.MIN_VALUE : ~latent; - default -> latent; // U16, U32, U64: identity - }; - } - - /// Undo pcodec consecutive delta encoding for one page. - /// - /// toggle_center (XOR dtype_mid) + cumulative sum restores original U-latents. - private static void applyConsecutiveDelta(MemorySegment rawLatents, long offset, - int pageN, long[] moments, int dtypeSize) { - long mid = typeMid(dtypeSize); - long mask = typeMask(dtypeSize); - - for (int i = 0; i < pageN; i++) { - long byteOff = offset + (long) i * Long.BYTES; - rawLatents.set(LE_LONG, byteOff, (rawLatents.get(LE_LONG, byteOff) ^ mid) & mask); - } - - for (int m = moments.length - 1; m >= 0; m--) { - long moment = moments[m] & mask; - for (int i = 0; i < pageN; i++) { - long byteOff = offset + (long) i * Long.BYTES; - long tmp = rawLatents.get(LE_LONG, byteOff); - rawLatents.set(LE_LONG, byteOff, moment); - moment = (moment + tmp) & mask; - } - } - } - - private static long typeMid(int dtypeSize) { - return switch (dtypeSize) { - case 64 -> Long.MIN_VALUE; - case 32 -> 0x80000000L; - case 16 -> 0x8000L; - default -> throw new VortexException(EncodingId.VORTEX_PCO, - "pco: invalid dtypeSize " + dtypeSize); - }; - } - - private static long typeMask(int dtypeSize) { - return switch (dtypeSize) { - case 64 -> -1L; - case 32 -> 0xFFFFFFFFL; - case 16 -> 0xFFFFL; - default -> throw new VortexException(EncodingId.VORTEX_PCO, - "pco: invalid dtypeSize " + dtypeSize); - }; - } - - private static int dtypeSize(PType ptype) { - return switch (ptype) { - case I16, U16 -> 16; - case I32, U32, F32 -> 32; - case I64, U64, F64 -> 64; - default -> throw new VortexException(EncodingId.VORTEX_PCO, - "pco: unsupported ptype " + ptype); - }; - } - - private static int bitsToEncodeOffsetBits(int dtypeSize) { - return switch (dtypeSize) { - case 64 -> BITS_TO_ENCODE_OFFSET_BITS_64; - case 32 -> BITS_TO_ENCODE_OFFSET_BITS_32; - case 16 -> BITS_TO_ENCODE_OFFSET_BITS_16; - default -> throw new VortexException(EncodingId.VORTEX_PCO, - "pco: invalid dtypeSize " + dtypeSize); - }; - } - - private static Array toArray(DType dtype, long n, MemorySegment out) { - PType ptype = ((DType.Primitive) dtype).ptype(); - return switch (ptype) { - case I16, U16 -> new ShortArray(dtype, n, out, ArrayStats.empty()); - case I32, U32 -> new IntArray(dtype, n, out, ArrayStats.empty()); - case F32 -> new FloatArray(dtype, n, out, ArrayStats.empty()); - case I64, U64 -> new LongArray(dtype, n, out, ArrayStats.empty()); - case F64 -> new DoubleArray(dtype, n, out, ArrayStats.empty()); - default -> throw new VortexException(EncodingId.VORTEX_PCO, - "pco: unsupported ptype " + ptype); - }; - } - - /// Int-float latent → integer-valued float for F32. - /// - /// mid = 2^31; negative when l < mid; GPI = 2^24 (f32 mantissa bits). - private static float intFloatFromLatentF32(long l) { - long mid = 0x80000000L; - boolean negative = (l < mid); - long absInt = negative ? (0x7FFFFFFFL - l) : (l ^ 0x80000000L); - long gpi = 1L << 24; - float absFloat = (absInt < gpi) ? (float) absInt - : Float.intBitsToFloat(0x4B800000 + (int) (absInt - gpi)); - return negative ? -absFloat : absFloat; - } - - /// Int-float latent → integer-valued float for F64. - /// - /// mid = 2^63; l < 2^63 unsigned (l ≥ 0 signed) → negative float; GPI = 2^53. - private static double intFloatFromLatentF64(long l) { - boolean negative = (l >= 0); - long absInt = negative ? (Long.MAX_VALUE - l) : (l ^ Long.MIN_VALUE); - long gpi = 1L << 53; - double absFloat = (absInt < gpi) ? (double) absInt - : Double.longBitsToDouble(0x4340000000000000L + (absInt - gpi)); - return negative ? -absFloat : absFloat; - } - - /// Inverse of {@link #fromLatentOrdered} for F32: float → ordered latent. - private static long toLatentOrderedF32(float f) { - int bits = Float.floatToRawIntBits(f); - if ((bits & 0x80000000) != 0) { - return (~bits) & 0xFFFFFFFFL; - } else { - return (bits ^ 0x80000000) & 0xFFFFFFFFL; - } - } - - /// Inverse of {@link #fromLatentOrdered} for F64: float → ordered latent. - private static long toLatentOrderedF64(double d) { - long bits = Double.doubleToRawLongBits(d); - if ((bits & Long.MIN_VALUE) != 0) { - return ~bits; - } else { - return bits ^ Long.MIN_VALUE; - } - } - - /// FloatMult combine: rawLatents[i] = toLatentOrdered(intFloatFromLatent(mult) * baseFloat) + adj. - private static void combineFloatMult(PType ptype, long baseLatent, int chunkN, - MemorySegment rawLatents, long multsOffset, MemorySegment rawAdjs) { - if (ptype == PType.F32) { - float baseFloat = Float.intBitsToFloat((int) fromLatentOrdered(baseLatent, PType.F32)); - for (int i = 0; i < chunkN; i++) { - long off = multsOffset + (long) i * Long.BYTES; - long mult = rawLatents.get(LE_LONG, off); - long adj = rawAdjs.get(LE_LONG, (long) i * Long.BYTES); - long unadjusted = toLatentOrderedF32(intFloatFromLatentF32(mult) * baseFloat); - rawLatents.set(LE_LONG, off, (unadjusted + adj) & 0xFFFFFFFFL); - } - } else { - double baseDouble = Double.longBitsToDouble(fromLatentOrdered(baseLatent, PType.F64)); - for (int i = 0; i < chunkN; i++) { - long off = multsOffset + (long) i * Long.BYTES; - long mult = rawLatents.get(LE_LONG, off); - long adj = rawAdjs.get(LE_LONG, (long) i * Long.BYTES); - long unadjusted = toLatentOrderedF64(intFloatFromLatentF64(mult) * baseDouble); - rawLatents.set(LE_LONG, off, unadjusted + adj); - } - } - } - - /// FloatQuant combine: num_latent = (quantum << k) + lowest_k_bits. - /// - /// sign_cutoff = MID >> k; is_positive = quantum >= sign_cutoff (unsigned). - /// lowest_k_bits = adj if positive, else (lowestKBitsMax - adj). - /// Port of pcodec float_quant.rs join_latents. - private static void combineFloatQuant(PType ptype, int k, int chunkN, - MemorySegment rawLatents, long multsOffset, MemorySegment rawAdjs) { - if (ptype == PType.F32) { - long signCutoff = 0x80000000L >>> k; - long lowestKBitsMax = (1L << k) - 1L; - for (int i = 0; i < chunkN; i++) { - long off = multsOffset + (long) i * Long.BYTES; - long quantum = rawLatents.get(LE_LONG, off); - long adj = rawAdjs.get(LE_LONG, (long) i * Long.BYTES); - long lowestKBits = (quantum >= signCutoff) ? adj : (lowestKBitsMax - adj); - rawLatents.set(LE_LONG, off, (quantum << k) + lowestKBits); - } - } else { - // F64: unsigned comparison via Long.compareUnsigned. - long signCutoff = Long.MIN_VALUE >>> k; - long lowestKBitsMax = (1L << k) - 1L; - for (int i = 0; i < chunkN; i++) { - long off = multsOffset + (long) i * Long.BYTES; - long quantum = rawLatents.get(LE_LONG, off); - long adj = rawAdjs.get(LE_LONG, (long) i * Long.BYTES); - boolean isPos = Long.compareUnsigned(quantum, signCutoff) >= 0; - long lowestKBits = isPos ? adj : (lowestKBitsMax - adj); - rawLatents.set(LE_LONG, off, (quantum << k) + lowestKBits); - } - } - } - - /// Dict combine: rawLatents[i] = dict[index], where dict entries are ordered latents. - private static void combineDict(long[] dict, int chunkN, - MemorySegment rawLatents, long offset) { - for (int i = 0; i < chunkN; i++) { - long off = offset + (long) i * Long.BYTES; - int idx = (int) rawLatents.get(LE_LONG, off); - if (idx < 0 || idx >= dict.length) { - throw new VortexException(EncodingId.VORTEX_PCO, - "pco dict index " + idx + " out of range [0, " + dict.length + ")"); - } - rawLatents.set(LE_LONG, off, dict[idx]); - } - } - - private static PcoChunkMeta readChunkMeta(MemorySegment buf, int dtypeSize) { - LeBitReader r = new LeBitReader(buf); - - // Mode nibble + mode-specific extra bits. - int modeNibble = (int) r.readBits(4); - long base = 0L; - int quantizeK = 0; - long[] dict = null; - if (modeNibble == 1 || modeNibble == 2) { - base = r.readBits(dtypeSize); - } else if (modeNibble == 3) { - quantizeK = (int) r.readBits(8); // BITS_TO_ENCODE_QUANTIZE_K - } else if (modeNibble == 4) { - int nUnique = (int) r.readBits(25); // BITS_TO_ENCODE_DICT_LEN - if (nUnique > 1 << 16) { - throw new VortexException(EncodingId.VORTEX_PCO, - "pco dict nUnique " + nUnique + " exceeds max 65536"); - } - r.alignToByte(); - dict = new long[nUnique]; - for (int i = 0; i < nUnique; i++) { - dict[i] = r.readBits(dtypeSize); - } - } else if (modeNibble != 0) { - throw new VortexException(EncodingId.VORTEX_PCO, - "pco mode " + modeNibble + " not yet implemented " - + "(Classic=0, IntMult=1, FloatMult=2, FloatQuant=3, Dict=4 supported)"); - } - - // Delta encoding variant + extra bits. - int deltaVariant = (int) r.readBits(4); - int deltaOrder = 0; - boolean secondaryUsesDelta = false; - int windowNLog = 0; - int stateNLog = 0; - int conv1Quantization = 0; - long conv1Bias = 0L; - long[] conv1Weights = new long[0]; - if (deltaVariant == 0) { - // NoOp - } else if (deltaVariant == 1) { - deltaOrder = (int) r.readBits(3); // BITS_TO_ENCODE_DELTA_ENCODING_ORDER - secondaryUsesDelta = r.readBits(1) != 0; - } else if (deltaVariant == 2) { - windowNLog = 1 + (int) r.readBits(5); // BITS_TO_ENCODE_DELTA_LOOKBACK_WINDOW_N_LOG - if (windowNLog > 24) { - throw new VortexException(EncodingId.VORTEX_PCO, - "pco lookback windowNLog " + windowNLog + " exceeds max 24"); - } - stateNLog = (int) r.readBits(4); // BITS_TO_ENCODE_DELTA_LOOKBACK_STATE_N_LOG - secondaryUsesDelta = r.readBits(1) != 0; - } else if (deltaVariant == 3) { - // Conv1: 64-bit dtypes unsupported — fail before reading Conv1 fields. - if (dtypeSize == 64) { - throw new VortexException(EncodingId.VORTEX_PCO, - "pco Conv1 delta not supported for 64-bit dtypes (I64/U64/F64)"); - } - // Conv1: quantization(5b) + bias(64b latent-ordered i64) + (order-1)(5b) + weights(order×32b) - conv1Quantization = (int) r.readBits(5); - conv1Bias = r.readBits(64) ^ Long.MIN_VALUE; // from_latent_ordered for i64 - int conv1Order = 1 + (int) r.readBits(5); - conv1Weights = new long[conv1Order]; - for (int i = 0; i < conv1Order; i++) { - // from_latent_ordered for i32: XOR with 0x80000000, then sign-extend to i64 - conv1Weights[i] = (int) (r.readBits(32) ^ 0x80000000L); - } - } else { - throw new VortexException(EncodingId.VORTEX_PCO, - "pco delta variant " + deltaVariant + " not yet implemented " - + "(NoOp=0, Consecutive=1, Lookback=2, Conv1=3 supported)"); - } - - // Delta latent var (U32 lookback indices) — only present for Lookback delta. - int deltaAnsSizeLog = 0; - PcoBin[] deltaBins = new PcoBin[0]; - if (deltaVariant == 2) { - deltaAnsSizeLog = (int) r.readBits(4); - int nDeltaBins = (int) r.readBits(15); - deltaBins = readBins(r, nDeltaBins, deltaAnsSizeLog, 32); // U32 indices - } - - // Primary latent var. Dict uses U32 indices (32-bit); all others use dtypeSize. - int primaryDtypeSize = (modeNibble == 4) ? 32 : dtypeSize; - int ansSizeLog = (int) r.readBits(4); - int nBins = (int) r.readBits(15); - PcoBin[] bins = readBins(r, nBins, ansSizeLog, primaryDtypeSize); - - // Secondary latent var — IntMult (1), FloatMult (2), FloatQuant (3) only. - int secondaryAnsSizeLog = 0; - PcoBin[] secondaryBins = new PcoBin[0]; - if (modeNibble == 1 || modeNibble == 2 || modeNibble == 3) { - secondaryAnsSizeLog = (int) r.readBits(4); - int nSecondaryBins = (int) r.readBits(15); - secondaryBins = readBins(r, nSecondaryBins, secondaryAnsSizeLog, dtypeSize); - } - r.alignToByte(); - - return new PcoChunkMeta(modeNibble, base, quantizeK, dict, - deltaVariant, deltaOrder, secondaryUsesDelta, - windowNLog, stateNLog, deltaAnsSizeLog, deltaBins, - conv1Quantization, conv1Bias, conv1Weights, - ansSizeLog, bins, secondaryAnsSizeLog, secondaryBins); - } - - private static PcoBin[] readBins(LeBitReader r, int nBins, int ansSizeLog, int dtypeSize) { - PcoBin[] bins = new PcoBin[nBins]; - int offsetBitsWidth = bitsToEncodeOffsetBits(dtypeSize); - for (int b = 0; b < nBins; b++) { - int weight = (int) r.readBits(ansSizeLog) + 1; - long lower = r.readBits(dtypeSize); - int offsetBits = (int) r.readBits(offsetBitsWidth); - bins[b] = new PcoBin(weight, lower, offsetBits); - } - return bins; - } - - private static EncodingProtos.PcoMetadata parseMeta(DecodeContext ctx) { - ByteBuffer raw = ctx.metadata(); - if (raw == null) { - throw new VortexException(EncodingId.VORTEX_PCO, "missing PcoMetadata"); - } - try { - return EncodingProtos.PcoMetadata.parseFrom(raw.duplicate()); - } catch (InvalidProtocolBufferException e) { - throw new VortexException(EncodingId.VORTEX_PCO, - "invalid PcoMetadata: " + e.getMessage()); - } - } - - private static void validateHeader(EncodingProtos.PcoMetadata meta) { - byte[] header = meta.getHeader().toByteArray(); - if (header.length < 2) { - throw new VortexException(EncodingId.VORTEX_PCO, - "pco header too short: " + header.length + " bytes"); - } - if (header[0] != PCO_FORMAT_MAJOR || header[1] != PCO_FORMAT_MINOR) { - throw new VortexException(EncodingId.VORTEX_PCO, - String.format("unsupported pco format version %02x.%02x (expected %02x.%02x)", - header[0] & 0xFF, header[1] & 0xFF, - PCO_FORMAT_MAJOR & 0xFF, PCO_FORMAT_MINOR & 0xFF)); - } - } - } - - private record PcoChunkMeta(int mode, long base, int quantizeK, long[] dict, - int deltaVariant, int deltaOrder, boolean secondaryUsesDelta, - int windowNLog, int stateNLog, int deltaAnsSizeLog, PcoBin[] deltaBins, - int conv1Quantization, long conv1Bias, long[] conv1Weights, - int ansSizeLog, PcoBin[] bins, - int secondaryAnsSizeLog, PcoBin[] secondaryBins) { - } -} diff --git a/core/src/main/java/io/github/dfa1/vortex/encoding/PcoTansDecoder.java b/core/src/main/java/io/github/dfa1/vortex/encoding/PcoTansDecoder.java deleted file mode 100644 index b9f9a60..0000000 --- a/core/src/main/java/io/github/dfa1/vortex/encoding/PcoTansDecoder.java +++ /dev/null @@ -1,141 +0,0 @@ -package io.github.dfa1.vortex.encoding; - -import io.github.dfa1.vortex.core.VortexException; - -import java.lang.foreign.MemorySegment; -import java.lang.foreign.ValueLayout; -import java.nio.ByteOrder; - -/// 4-way interleaved tANS decoder for one pco latent variable. -/// -/// Build via {@link #build(int, PcoBin[])}; then call {@link #decodePage} once per page. -/// Port of {@code pco/src/ans/spec.rs} (spread) and {@code pco/src/ans/decoding.rs} (nodes). -final class PcoTansDecoder { - - private static final ValueLayout.OfLong LE_LONG = - ValueLayout.JAVA_LONG_UNALIGNED.withOrder(ByteOrder.LITTLE_ENDIAN); - - static final int BATCH_N = 256; - static final int ANS_INTERLEAVING = 4; - - // All arrays indexed by state index in [0, tableSize). - private final int[] nextStateIdxBase; // = (symbolXs[sym] << bitsToRead) - tableSize - private final int[] bitsToRead; // bits consumed from bit stream per ANS step - private final int[] nodeOffsetBits; // offset bits for this bin (stored per-state for cache locality) - private final long[] stateLowers; // bin.lower for each state - - private PcoTansDecoder(int[] nextStateIdxBase, int[] bitsToRead, - int[] nodeOffsetBits, long[] stateLowers) { - this.nextStateIdxBase = nextStateIdxBase; - this.bitsToRead = bitsToRead; - this.nodeOffsetBits = nodeOffsetBits; - this.stateLowers = stateLowers; - } - - /// Build the decode table from chunk latent-var metadata. - /// - /// Port of {@code Spec::from_weights} + {@code Decoder::new} from pcodec. - static PcoTansDecoder build(int ansSizeLog, PcoBin[] bins) { - if (bins.length == 0) { - // Degenerate: no bins → 1-state table, all offsets zero. - return new PcoTansDecoder(new int[]{0}, new int[]{0}, new int[]{0}, new long[]{0L}); - } - - int tableSize = 1 << ansSizeLog; - int[] weights = new int[bins.length]; - for (int i = 0; i < bins.length; i++) { - weights[i] = bins[i].weight(); - } - - int[] stateSymbols = spreadStateSymbols(ansSizeLog, weights, tableSize); - - int[] symbolXs = weights.clone(); - int[] nextStateIdxBase = new int[tableSize]; - int[] bitsToRead = new int[tableSize]; - int[] nodeOffsetBits = new int[tableSize]; - long[] stateLowers = new long[tableSize]; - - for (int s = 0; s < tableSize; s++) { - int sym = stateSymbols[s]; - int xs = symbolXs[sym]; - int btr = Integer.numberOfLeadingZeros(xs) - Integer.numberOfLeadingZeros(tableSize); - int nextBase = xs << btr; - nextStateIdxBase[s] = nextBase - tableSize; - bitsToRead[s] = btr; - nodeOffsetBits[s] = sym < bins.length ? bins[sym].offsetBits() : 0; - stateLowers[s] = sym < bins.length ? bins[sym].lower() : 0L; - symbolXs[sym]++; - } - - return new PcoTansDecoder(nextStateIdxBase, bitsToRead, nodeOffsetBits, stateLowers); - } - - /// Port of {@code Spec::spread_state_symbols} from pcodec. - /// - /// Spreads symbols across the table with a stride of ~3/5 * tableSize (odd). - static int[] spreadStateSymbols(int ansSizeLog, int[] weights, int tableSize) { - int[] stateSymbols = new int[tableSize]; - int stride = (3 * tableSize) / 5; - if (stride % 2 == 0) { - stride++; - } - int modMask = tableSize - 1; - int step = 0; - for (int sym = 0; sym < weights.length; sym++) { - for (int k = 0; k < weights[sym]; k++) { - stateSymbols[(stride * step) & modMask] = sym; - step++; - } - } - return stateSymbols; - } - - /// Decode {@code n} raw latent values (U64) from {@code reader} into {@code out}. - /// - /// Caller must have already read 4 initial ANS state indices and called - /// {@link LeBitReader#alignToByte()} before this call. - /// {@code ansStateIdxs} is modified in place and not valid after return. - /// {@code batchLowers} and {@code batchOffsetBits} are caller-provided scratch arrays of - /// length ≥ {@link #BATCH_N}; they are fully overwritten before use. - void decodePage(LeBitReader reader, int[] ansStateIdxs, int n, - MemorySegment out, long outByteOffset, - long[] batchLowers, int[] batchOffsetBits) { - int remaining = n; - long pos = outByteOffset; - while (remaining > 0) { - int batchN = Math.min(remaining, BATCH_N); - decodeBatch(reader, ansStateIdxs, batchN, batchLowers, batchOffsetBits, out, pos); - pos += (long) batchN * Long.BYTES; - remaining -= batchN; - } - } - - /// Decode exactly {@code batchN} latent values into {@code out[outByteOffset..]} and advance - /// the ANS states. - /// - /// {@code batchLowers} and {@code batchOffsetBits} are caller-provided scratch arrays of - /// length ≥ {@code batchN}; they are fully overwritten before use. - void decodeBatch(LeBitReader reader, int[] ansStateIdxs, int batchN, - long[] batchLowers, int[] batchOffsetBits, - MemorySegment out, long outByteOffset) { - int tableSize = nextStateIdxBase.length; - for (int i = 0; i < batchN; i++) { - int si = ansStateIdxs[i % ANS_INTERLEAVING]; - batchLowers[i] = stateLowers[si]; - batchOffsetBits[i] = nodeOffsetBits[si]; - long ansVal = reader.readBits(bitsToRead[si]); - int nextSi = nextStateIdxBase[si] + (int) ansVal; - if (nextSi < 0 || nextSi >= tableSize) { - throw new VortexException(EncodingId.VORTEX_PCO, - "corrupt pco ANS state " + nextSi + " out of range [0, " + tableSize + ")"); - } - ansStateIdxs[i % ANS_INTERLEAVING] = nextSi; - } - long pos = outByteOffset; - for (int i = 0; i < batchN; i++) { - long offset = reader.readBits(batchOffsetBits[i]); - out.set(LE_LONG, pos, batchLowers[i] + offset); - pos += Long.BYTES; - } - } -} diff --git a/core/src/main/java/io/github/dfa1/vortex/encoding/PrimitiveEncoding.java b/core/src/main/java/io/github/dfa1/vortex/encoding/PrimitiveEncoding.java deleted file mode 100644 index e32291e..0000000 --- a/core/src/main/java/io/github/dfa1/vortex/encoding/PrimitiveEncoding.java +++ /dev/null @@ -1,343 +0,0 @@ -package io.github.dfa1.vortex.encoding; - -import io.github.dfa1.vortex.proto.ScalarProtos; -import io.github.dfa1.vortex.core.array.Array; -import io.github.dfa1.vortex.core.DType; -import io.github.dfa1.vortex.core.PType; -import io.github.dfa1.vortex.core.VortexException; -import io.github.dfa1.vortex.core.array.BoolArray; -import io.github.dfa1.vortex.core.array.ByteArray; -import io.github.dfa1.vortex.core.array.DoubleArray; -import io.github.dfa1.vortex.core.array.Float16Array; -import io.github.dfa1.vortex.core.array.FloatArray; -import io.github.dfa1.vortex.core.array.IntArray; -import io.github.dfa1.vortex.core.array.LongArray; -import io.github.dfa1.vortex.core.array.MaskedArray; -import io.github.dfa1.vortex.core.array.ShortArray; -import java.lang.foreign.Arena; -import java.lang.foreign.MemorySegment; - -/// Encoding for `vortex.primitive` — raw little-endian primitive arrays. -/// Encodes all [DType.Primitive] types; embeds min/max stats as Protobuf ScalarValue bytes. -public final class PrimitiveEncoding implements Encoding { - - /// Creates a new {@code PrimitiveEncoding} instance; use via {@link EncodingRegistry}. - public PrimitiveEncoding() { - } - - @Override - public EncodingId encodingId() { - return EncodingId.VORTEX_PRIMITIVE; - } - - @Override - public boolean accepts(DType dtype) { - return dtype instanceof DType.Primitive; - } - - @Override - public EncodeResult encode(DType dtype, Object data) { - return Encoder.encode(dtype, data); - } - - @Override - public Array decode(DecodeContext ctx) { - return Decoder.decode(ctx); - } - - private static final class Encoder { - - private static EncodeResult encode(DType dtype, Object data) { - PType ptype = ((DType.Primitive) dtype).ptype(); - MemorySegment seg = encodePrimitive(ptype, data); - byte[] min = null; - byte[] max = null; - byte[][] stats = computeStats(ptype, data); - if (stats != null) { - min = stats[0]; - max = stats[1]; - } - return EncodeResult.simple(EncodingId.VORTEX_PRIMITIVE, seg, min, max); - } - - private static MemorySegment encodePrimitive(PType ptype, Object data) { - return switch (ptype) { - case I8, U8 -> MemorySegment.ofArray((byte[]) data); - case I16, U16, F16 -> { - short[] arr = (short[]) data; - MemorySegment seg = Arena.ofAuto().allocate((long) arr.length * 2, 2); - for (int i = 0; i < arr.length; i++) { - seg.setAtIndex(PTypeIO.LE_SHORT, i, arr[i]); - } - yield seg; - } - case I32, U32 -> { - int[] arr = (int[]) data; - MemorySegment seg = Arena.ofAuto().allocate((long) arr.length * 4, 4); - for (int i = 0; i < arr.length; i++) { - seg.setAtIndex(PTypeIO.LE_INT, i, arr[i]); - } - yield seg; - } - case I64, U64 -> { - long[] arr = (long[]) data; - MemorySegment seg = Arena.ofAuto().allocate((long) arr.length * 8, 8); - for (int i = 0; i < arr.length; i++) { - seg.setAtIndex(PTypeIO.LE_LONG, i, arr[i]); - } - yield seg; - } - case F32 -> { - float[] arr = (float[]) data; - MemorySegment seg = Arena.ofAuto().allocate((long) arr.length * 4, 4); - for (int i = 0; i < arr.length; i++) { - seg.setAtIndex(PTypeIO.LE_FLOAT, i, arr[i]); - } - yield seg; - } - case F64 -> { - double[] arr = (double[]) data; - MemorySegment seg = Arena.ofAuto().allocate((long) arr.length * 8, 8); - for (int i = 0; i < arr.length; i++) { - seg.setAtIndex(PTypeIO.LE_DOUBLE, i, arr[i]); - } - yield seg; - } - }; - } - - private static byte[][] computeStats(PType ptype, Object data) { - return switch (ptype) { - case I8 -> { - byte[] arr = (byte[]) data; - if (arr.length == 0) { - yield null; - } - long min = arr[0], max = arr[0]; - for (byte v : arr) { - if (v < min) { - min = v; - } - if (v > max) { - max = v; - } - } - yield new byte[][]{scalarI64(min), scalarI64(max)}; - } - case I16 -> { - short[] arr = (short[]) data; - if (arr.length == 0) { - yield null; - } - long min = arr[0], max = arr[0]; - for (short v : arr) { - if (v < min) { - min = v; - } - if (v > max) { - max = v; - } - } - yield new byte[][]{scalarI64(min), scalarI64(max)}; - } - case I32 -> { - int[] arr = (int[]) data; - if (arr.length == 0) { - yield null; - } - long min = arr[0], max = arr[0]; - for (int v : arr) { - if (v < min) { - min = v; - } - if (v > max) { - max = v; - } - } - yield new byte[][]{scalarI64(min), scalarI64(max)}; - } - case I64 -> { - long[] arr = (long[]) data; - if (arr.length == 0) { - yield null; - } - long min = arr[0], max = arr[0]; - for (long v : arr) { - if (v < min) { - min = v; - } - if (v > max) { - max = v; - } - } - yield new byte[][]{scalarI64(min), scalarI64(max)}; - } - case U8 -> { - byte[] arr = (byte[]) data; - if (arr.length == 0) { - yield null; - } - long min = Byte.toUnsignedInt(arr[0]), max = Byte.toUnsignedInt(arr[0]); - for (byte v : arr) { - long uv = Byte.toUnsignedInt(v); - if (uv < min) { - min = uv; - } - if (uv > max) { - max = uv; - } - } - yield new byte[][]{scalarU64(min), scalarU64(max)}; - } - case U16 -> { - short[] arr = (short[]) data; - if (arr.length == 0) { - yield null; - } - long min = Short.toUnsignedInt(arr[0]), max = Short.toUnsignedInt(arr[0]); - for (short v : arr) { - long uv = Short.toUnsignedInt(v); - if (uv < min) { - min = uv; - } - if (uv > max) { - max = uv; - } - } - yield new byte[][]{scalarU64(min), scalarU64(max)}; - } - case U32 -> { - int[] arr = (int[]) data; - if (arr.length == 0) { - yield null; - } - long min = Integer.toUnsignedLong(arr[0]), max = Integer.toUnsignedLong(arr[0]); - for (int v : arr) { - long uv = Integer.toUnsignedLong(v); - if (uv < min) { - min = uv; - } - if (uv > max) { - max = uv; - } - } - yield new byte[][]{scalarU64(min), scalarU64(max)}; - } - case U64 -> { - long[] arr = (long[]) data; - if (arr.length == 0) { - yield null; - } - long min = arr[0], max = arr[0]; - for (long v : arr) { - if (Long.compareUnsigned(v, min) < 0) { - min = v; - } - if (Long.compareUnsigned(v, max) > 0) { - max = v; - } - } - yield new byte[][]{scalarU64(min), scalarU64(max)}; - } - case F32 -> { - float[] arr = (float[]) data; - if (arr.length == 0) { - yield null; - } - float min = arr[0], max = arr[0]; - for (float v : arr) { - if (v < min) { - min = v; - } - if (v > max) { - max = v; - } - } - yield new byte[][]{scalarF32(min), scalarF32(max)}; - } - case F64 -> { - double[] arr = (double[]) data; - if (arr.length == 0) { - yield null; - } - double min = arr[0], max = arr[0]; - for (double v : arr) { - if (v < min) { - min = v; - } - if (v > max) { - max = v; - } - } - yield new byte[][]{scalarF64(min), scalarF64(max)}; - } - case F16 -> { - short[] arr = (short[]) data; - if (arr.length == 0) { - yield null; - } - float min = Float.float16ToFloat(arr[0]), max = Float.float16ToFloat(arr[0]); - for (short v : arr) { - float fv = Float.float16ToFloat(v); - if (fv < min) { - min = fv; - } - if (fv > max) { - max = fv; - } - } - yield new byte[][]{scalarF32(min), scalarF32(max)}; - } - }; - } - - private static byte[] scalarI64(long v) { - return ScalarProtos.ScalarValue.newBuilder().setInt64Value(v).build().toByteArray(); - } - - private static byte[] scalarU64(long v) { - return ScalarProtos.ScalarValue.newBuilder().setUint64Value(v).build().toByteArray(); - } - - private static byte[] scalarF32(float v) { - return ScalarProtos.ScalarValue.newBuilder().setF32Value(v).build().toByteArray(); - } - - private static byte[] scalarF64(double v) { - return ScalarProtos.ScalarValue.newBuilder().setF64Value(v).build().toByteArray(); - } - } - - private static final class Decoder { - - private static Array decode(DecodeContext ctx) { - MemorySegment buf = ctx.buffer(0); - long n = ctx.rowCount(); - DType dt = ctx.dtype(); - PType ptype = ((DType.Primitive) dt).ptype(); - var stats = ctx.node().stats(); - Array values = switch (ptype) { - case I64, U64 -> new LongArray(dt, n, buf, stats); - case I32, U32 -> new IntArray(dt, n, buf, stats); - case F64 -> new DoubleArray(dt, n, buf, stats); - case F32 -> new FloatArray(dt, n, buf, stats); - case I16, U16 -> new ShortArray(dt, n, buf, stats); - case I8, U8 -> new ByteArray(dt, n, buf, stats); - case F16 -> new Float16Array(dt, n, buf, stats); - }; - if (ctx.node().children().length == 1) { - ArrayNode validityNode = ctx.node().children()[0]; - var validityCtx = new DecodeContext( - validityNode, new DType.Bool(false), n, - ctx.segmentBuffers(), ctx.registry(), ctx.arena()); - Array va = ctx.registry().decode(validityCtx); - if (!(va instanceof BoolArray validity)) { - throw new VortexException(EncodingId.VORTEX_PRIMITIVE, - "validity child decoded to unexpected type: " + va.getClass().getSimpleName()); - } - return new MaskedArray(values, validity); - } - return values; - } - } -} diff --git a/core/src/main/java/io/github/dfa1/vortex/encoding/RleEncoding.java b/core/src/main/java/io/github/dfa1/vortex/encoding/RleEncoding.java deleted file mode 100644 index 5f50928..0000000 --- a/core/src/main/java/io/github/dfa1/vortex/encoding/RleEncoding.java +++ /dev/null @@ -1,498 +0,0 @@ -package io.github.dfa1.vortex.encoding; - -import com.google.protobuf.InvalidProtocolBufferException; -import io.github.dfa1.vortex.core.ArrayStats; -import io.github.dfa1.vortex.core.DType; -import io.github.dfa1.vortex.core.PType; -import io.github.dfa1.vortex.core.VortexException; -import io.github.dfa1.vortex.core.array.Array; -import io.github.dfa1.vortex.core.array.BoolArray; -import io.github.dfa1.vortex.core.array.ByteArray; -import io.github.dfa1.vortex.core.array.DoubleArray; -import io.github.dfa1.vortex.core.array.Float16Array; -import io.github.dfa1.vortex.core.array.FloatArray; -import io.github.dfa1.vortex.core.array.IntArray; -import io.github.dfa1.vortex.core.array.LongArray; -import io.github.dfa1.vortex.core.array.MaskedArray; -import io.github.dfa1.vortex.core.array.ShortArray; -import io.github.dfa1.vortex.proto.DTypeProtos; -import io.github.dfa1.vortex.proto.EncodingProtos; - -import java.lang.foreign.Arena; -import java.lang.foreign.MemorySegment; -import java.lang.foreign.SegmentAllocator; -import java.lang.foreign.ValueLayout; -import java.nio.ByteBuffer; -import java.util.List; - -/// Encoding for {@code fastlanes.rle} — FastLanes chunk-based run-length encoding. -/// -///

Wire format (0 buffers, 3 child slots): -///

    -///
  • child 0: {@code values} — non-nullable primitive, same ptype as output; global dictionary -/// across all chunks (concatenation of per-chunk dictionaries).
    • -///
  • child 1: {@code indices} — u8 or u16, same nullability as output; chunk-local dictionary -/// indices, padded to a multiple of 1024.
    • -///
  • child 2: {@code values_idx_offsets} — non-nullable unsigned int (typically u64); one -/// entry per chunk recording the absolute start position in the global {@code values} -/// array.
    • -///
-/// -///

Metadata: protobuf {@code RLEMetadata}: -/// {@code values_len u64} (tag 1), {@code indices_len u64} (tag 2), -/// {@code indices_ptype PType} (tag 3), {@code values_idx_offsets_len u64} (tag 4), -/// {@code values_idx_offsets_ptype PType} (tag 5), {@code offset u64} (tag 6, always < 1024). -/// -///

Encode: split input into 1024-element chunks; within each chunk build a run-length -/// dictionary (new entry per run boundary, NOT per unique value) and a per-element index array. -/// -///

Decode: for each needed chunk, look up {@code values[values_idx_offsets[chunk] + indices[i]]} -/// to reconstruct each element, then slice the padded result to {@code [offset, offset+len)}. -public final class RleEncoding implements Encoding { - - static final int FL_CHUNK_SIZE = 1024; - - /// Creates a new {@code RleEncoding} instance; use via {@link EncodingRegistry}. - public RleEncoding() { - } - - @Override - public EncodingId encodingId() { - return EncodingId.FASTLANES_RLE; - } - - @Override - public boolean accepts(DType dtype) { - return dtype instanceof DType.Primitive p && !p.ptype().isFloating(); - } - - @Override - public EncodeResult encode(DType dtype, Object data) { - return Encoder.encode(dtype, data); - } - - @Override - public Array decode(DecodeContext ctx) { - return Decoder.decode(ctx); - } - - private static final class Encoder { - - static EncodeResult encode(DType dtype, Object data) { - if (!(dtype instanceof DType.Primitive p)) { - throw new VortexException(EncodingId.FASTLANES_RLE, "encode only supports Primitive dtype, got " + dtype); - } - PType ptype = p.ptype(); - long[] longs = toLongs(data, ptype); - int n = longs.length; - - if (n == 0) { - return encodeEmpty(ptype, dtype); - } - - int numChunks = (n + FL_CHUNK_SIZE - 1) / FL_CHUNK_SIZE; - int paddedLen = numChunks * FL_CHUNK_SIZE; - - long[] globalValues = new long[paddedLen]; - short[] globalIndices = new short[paddedLen]; - long[] valuesIdxOffsets = new long[numChunks]; - - long[] chunkInput = new long[FL_CHUNK_SIZE]; - long[] chunkValues = new long[FL_CHUNK_SIZE]; - short[] chunkIndices = new short[FL_CHUNK_SIZE]; - - int globalValuesCount = 0; - - for (int chunk = 0; chunk < numChunks; chunk++) { - int chunkStart = chunk * FL_CHUNK_SIZE; - int chunkEnd = Math.min(chunkStart + FL_CHUNK_SIZE, n); - int chunkLen = chunkEnd - chunkStart; - - System.arraycopy(longs, chunkStart, chunkInput, 0, chunkLen); - // Pad remainder with last value to avoid spurious run boundaries. - long lastVal = longs[chunkEnd - 1]; - for (int i = chunkLen; i < FL_CHUNK_SIZE; i++) { - chunkInput[i] = lastVal; - } - - int numChunkValues = rleEncode(chunkInput, chunkValues, chunkIndices); - - valuesIdxOffsets[chunk] = globalValuesCount; - System.arraycopy(chunkValues, 0, globalValues, globalValuesCount, numChunkValues); - globalValuesCount += numChunkValues; - - System.arraycopy(chunkIndices, 0, globalIndices, chunkStart, FL_CHUNK_SIZE); - } - - MemorySegment valuesSeg = fromLongs(globalValues, 0, globalValuesCount, ptype, Arena.ofAuto()); - MemorySegment indicesSeg = toIndicesSeg(globalIndices, paddedLen, Arena.ofAuto()); - MemorySegment offsetsSeg = fromLongsU64(valuesIdxOffsets, numChunks, Arena.ofAuto()); - - PType indicesPtype = PType.U16; - PType offsetsPtype = PType.U64; - - byte[] metaBytes = EncodingProtos.RLEMetadata.newBuilder() - .setValuesLen(globalValuesCount) - .setIndicesLen(paddedLen) - .setIndicesPtype(DTypeProtos.PType.forNumber(indicesPtype.ordinal())) - .setValuesIdxOffsetsLen(numChunks) - .setValuesIdxOffsetsPtype(DTypeProtos.PType.forNumber(offsetsPtype.ordinal())) - .setOffset(0) - .build() - .toByteArray(); - - EncodeNode valuesNode = EncodeNode.leaf(EncodingId.VORTEX_PRIMITIVE, 0); - EncodeNode indicesNode = EncodeNode.leaf(EncodingId.VORTEX_PRIMITIVE, 1); - EncodeNode offsetsNode = EncodeNode.leaf(EncodingId.VORTEX_PRIMITIVE, 2); - EncodeNode root = new EncodeNode( - EncodingId.FASTLANES_RLE, - ByteBuffer.wrap(metaBytes), - new EncodeNode[]{valuesNode, indicesNode, offsetsNode}, - new int[0]); - return new EncodeResult(root, List.of(valuesSeg, indicesSeg, offsetsSeg), null, null); - } - - // FastLanes RLE encode for one 1024-element chunk. - // Returns the number of unique run entries written to chunkValues. - private static int rleEncode(long[] input, long[] chunkValues, short[] chunkIndices) { - short posVal = 0; - int valIdx = 1; - long prev = input[0]; - chunkValues[0] = prev; - chunkIndices[0] = 0; - - for (int i = 1; i < FL_CHUNK_SIZE; i++) { - long cur = input[i]; - if (cur != prev) { - chunkValues[valIdx] = cur; - valIdx++; - posVal++; - prev = cur; - } - chunkIndices[i] = posVal; - } - return valIdx; - } - - private static EncodeResult encodeEmpty(PType ptype, DType dtype) { - MemorySegment empty = Arena.ofAuto().allocate(0); - PType indicesPtype = PType.U16; - PType offsetsPtype = PType.U64; - byte[] metaBytes = EncodingProtos.RLEMetadata.newBuilder() - .setValuesLen(0) - .setIndicesLen(0) - .setIndicesPtype(DTypeProtos.PType.forNumber(indicesPtype.ordinal())) - .setValuesIdxOffsetsLen(0) - .setValuesIdxOffsetsPtype(DTypeProtos.PType.forNumber(offsetsPtype.ordinal())) - .setOffset(0) - .build() - .toByteArray(); - EncodeNode valuesNode = EncodeNode.leaf(EncodingId.VORTEX_PRIMITIVE, 0); - EncodeNode indicesNode = EncodeNode.leaf(EncodingId.VORTEX_PRIMITIVE, 1); - EncodeNode offsetsNode = EncodeNode.leaf(EncodingId.VORTEX_PRIMITIVE, 2); - EncodeNode root = new EncodeNode( - EncodingId.FASTLANES_RLE, - ByteBuffer.wrap(metaBytes), - new EncodeNode[]{valuesNode, indicesNode, offsetsNode}, - new int[0]); - return new EncodeResult(root, List.of(empty, empty, empty), null, null); - } - - private static long[] toLongs(Object data, PType ptype) { - return switch (ptype) { - case I8 -> { - byte[] arr = (byte[]) data; - long[] r = new long[arr.length]; - for (int i = 0; i < arr.length; i++) { - r[i] = arr[i]; - } - yield r; - } - case U8 -> { - byte[] arr = (byte[]) data; - long[] r = new long[arr.length]; - for (int i = 0; i < arr.length; i++) { - r[i] = Byte.toUnsignedLong(arr[i]); - } - yield r; - } - case I16 -> { - short[] arr = (short[]) data; - long[] r = new long[arr.length]; - for (int i = 0; i < arr.length; i++) { - r[i] = arr[i]; - } - yield r; - } - case U16 -> { - short[] arr = (short[]) data; - long[] r = new long[arr.length]; - for (int i = 0; i < arr.length; i++) { - r[i] = Short.toUnsignedLong(arr[i]); - } - yield r; - } - case I32 -> { - int[] arr = (int[]) data; - long[] r = new long[arr.length]; - for (int i = 0; i < arr.length; i++) { - r[i] = arr[i]; - } - yield r; - } - case U32 -> { - int[] arr = (int[]) data; - long[] r = new long[arr.length]; - for (int i = 0; i < arr.length; i++) { - r[i] = Integer.toUnsignedLong(arr[i]); - } - yield r; - } - case I64, U64 -> (long[]) data; - case F32 -> { - float[] arr = (float[]) data; - long[] r = new long[arr.length]; - for (int i = 0; i < arr.length; i++) { - r[i] = Float.floatToRawIntBits(arr[i]); - } - yield r; - } - case F64 -> { - double[] arr = (double[]) data; - long[] r = new long[arr.length]; - for (int i = 0; i < arr.length; i++) { - r[i] = Double.doubleToRawLongBits(arr[i]); - } - yield r; - } - case F16 -> { - short[] arr = (short[]) data; - long[] r = new long[arr.length]; - for (int i = 0; i < arr.length; i++) { - r[i] = Short.toUnsignedLong(arr[i]); - } - yield r; - } - }; - } - - private static MemorySegment fromLongs(long[] values, int start, int count, PType ptype, SegmentAllocator arena) { - int elemSize = ptype.byteSize(); - MemorySegment seg = arena.allocate((long) count * elemSize); - for (int i = 0; i < count; i++) { - PTypeIO.set(seg, (long) i * elemSize, ptype, values[start + i]); - } - return seg; - } - - private static MemorySegment fromLongsU64(long[] values, int count, SegmentAllocator arena) { - MemorySegment seg = arena.allocate((long) count * 8); - for (int i = 0; i < count; i++) { - seg.setAtIndex(PTypeIO.LE_LONG, i, values[i]); - } - return seg; - } - - private static MemorySegment toIndicesSeg(short[] indices, int count, SegmentAllocator arena) { - MemorySegment seg = arena.allocate((long) count * 2); - for (int i = 0; i < count; i++) { - seg.setAtIndex(PTypeIO.LE_SHORT, i, indices[i]); - } - return seg; - } - } - - private static final class Decoder { - - static Array decode(DecodeContext ctx) { - ByteBuffer rawMeta = ctx.metadata(); - EncodingProtos.RLEMetadata meta; - try { - meta = EncodingProtos.RLEMetadata.parseFrom(rawMeta.duplicate()); - } catch (InvalidProtocolBufferException e) { - throw new VortexException(EncodingId.FASTLANES_RLE, "invalid metadata", e); - } - - long valuesLen = meta.getValuesLen(); - long indicesLen = meta.getIndicesLen(); - PType indicesPtype = ptypeFromProto(meta.getIndicesPtype()); - long offsetsLen = meta.getValuesIdxOffsetsLen(); - PType offsetsPtype = ptypeFromProto(meta.getValuesIdxOffsetsPtype()); - int offset = (int) meta.getOffset(); - - long rowCount = ctx.rowCount(); - if (rowCount == 0 || indicesLen == 0) { - return emptyArray(ctx); - } - - if (!(ctx.dtype() instanceof DType.Primitive p)) { - throw new VortexException(EncodingId.FASTLANES_RLE, "expected Primitive dtype, got " + ctx.dtype()); - } - PType ptype = p.ptype(); - - DType valuesDtype = new DType.Primitive(ptype, false); - DType indicesDtype = new DType.Primitive(indicesPtype, false); - DType offsetsDtype = new DType.Primitive(offsetsPtype, false); - - Array valuesArr = decodeChildAs(ctx, 0, valuesDtype, valuesLen); - Array indicesRaw = decodeChildAs(ctx, 1, indicesDtype, indicesLen); - Array offsetsArr = decodeChildAs(ctx, 2, offsetsDtype, offsetsLen); - - // Indices may carry a validity bitmap when the output column is nullable. - BoolArray indicesValidity = null; - Array indicesArr = indicesRaw; - if (indicesRaw instanceof MaskedArray masked) { - indicesArr = masked.child(0); - indicesValidity = (BoolArray) masked.child(1); - } - - long[] values = readLongs(valuesArr.buffer(0), (int) valuesLen, ptype); - int[] indices = readIndices(indicesArr.buffer(0), (int) indicesLen, indicesPtype); - long[] valuesIdxOffsets = readUnsignedLongs(offsetsArr.buffer(0), (int) offsetsLen, offsetsPtype); - - // offset < FL_CHUNK_SIZE so chunk_start = 0 always. - int numChunks = (int) (indicesLen / FL_CHUNK_SIZE); - int chunkEnd = (int) ((offset + rowCount + FL_CHUNK_SIZE - 1) / FL_CHUNK_SIZE); - chunkEnd = Math.min(chunkEnd, numChunks); - - long[] decoded = new long[chunkEnd * FL_CHUNK_SIZE]; - long firstOffset = valuesLen > 0 ? valuesIdxOffsets[0] : 0L; - - for (int chunkIdx = 0; chunkIdx < chunkEnd; chunkIdx++) { - long valueIdxOffset = valuesIdxOffsets[chunkIdx] - firstOffset; - long nextValueIdxOffset = (chunkIdx + 1 < numChunks) - ? (valuesIdxOffsets[chunkIdx + 1] - firstOffset) - : valuesLen; - int numChunkValues = (int) (nextValueIdxOffset - valueIdxOffset); - - int chunkBase = chunkIdx * FL_CHUNK_SIZE; - if (numChunkValues <= 1) { - long fillVal = numChunkValues == 1 ? values[(int) valueIdxOffset] : 0L; - for (int i = 0; i < FL_CHUNK_SIZE; i++) { - decoded[chunkBase + i] = fillVal; - } - } else { - for (int i = 0; i < FL_CHUNK_SIZE; i++) { - int idx = indices[chunkBase + i]; - // Clamp index to handle garbage values at null positions. - if (idx >= numChunkValues) { - idx = numChunkValues - 1; - } - decoded[chunkBase + i] = values[(int) valueIdxOffset + idx]; - } - } - } - - MemorySegment seg = fromLongs(decoded, offset, (int) rowCount, ptype, ctx.arena()); - Array result = toArray(ctx.dtype(), rowCount, seg, ptype); - if (indicesValidity == null) { - return result; - } - // Propagate indices validity to output: output[j] is null iff indices[offset+j] is null. - int validityBytes = (int) ((rowCount + 7) / 8); - MemorySegment validityBuf = ctx.arena().allocate(validityBytes); - for (long j = 0; j < rowCount; j++) { - if (indicesValidity.getBoolean(offset + j)) { - int byteIdx = (int) (j >>> 3); - byte current = validityBuf.get(ValueLayout.JAVA_BYTE, (long) byteIdx); - validityBuf.set(ValueLayout.JAVA_BYTE, (long) byteIdx, (byte) (current | (1 << (j & 7)))); - } - } - BoolArray outputValidity = new BoolArray(new DType.Bool(false), rowCount, validityBuf, ArrayStats.empty()); - return new MaskedArray(result, outputValidity); - } - - private static Array emptyArray(DecodeContext ctx) { - MemorySegment empty = ctx.arena().allocate(0); - DType dt = ctx.dtype(); - PType ptype = ((DType.Primitive) dt).ptype(); - return toArray(dt, 0L, empty, ptype); - } - - private static Array toArray(DType dtype, long n, MemorySegment seg, PType ptype) { - return switch (ptype) { - case I64, U64 -> new LongArray(dtype, n, seg, ArrayStats.empty()); - case I32, U32 -> new IntArray(dtype, n, seg, ArrayStats.empty()); - case I16, U16 -> new ShortArray(dtype, n, seg, ArrayStats.empty()); - case I8, U8 -> new ByteArray(dtype, n, seg, ArrayStats.empty()); - case F64 -> new DoubleArray(dtype, n, seg, ArrayStats.empty()); - case F32 -> new FloatArray(dtype, n, seg, ArrayStats.empty()); - case F16 -> new Float16Array(dtype, n, seg, ArrayStats.empty()); - }; - } - - private static long[] readLongs(MemorySegment buf, int count, PType ptype) { - long[] out = new long[count]; - int elemSize = ptype.byteSize(); - for (int i = 0; i < count; i++) { - long off = (long) i * elemSize; - out[i] = switch (ptype) { - case I8 -> buf.get(ValueLayout.JAVA_BYTE, off); - case U8 -> Byte.toUnsignedLong(buf.get(ValueLayout.JAVA_BYTE, off)); - case I16 -> buf.get(PTypeIO.LE_SHORT, off); - case U16, F16 -> Short.toUnsignedLong(buf.get(PTypeIO.LE_SHORT, off)); - case I32 -> buf.get(PTypeIO.LE_INT, off); - case U32 -> Integer.toUnsignedLong(buf.get(PTypeIO.LE_INT, off)); - case I64, U64 -> buf.get(PTypeIO.LE_LONG, off); - case F32 -> Integer.toUnsignedLong(buf.get(PTypeIO.LE_INT, off)); - case F64 -> buf.get(PTypeIO.LE_LONG, off); - }; - } - return out; - } - - private static int[] readIndices(MemorySegment buf, int count, PType indicesPtype) { - int[] out = new int[count]; - switch (indicesPtype) { - case U8 -> { - for (int i = 0; i < count; i++) { - out[i] = Byte.toUnsignedInt(buf.get(ValueLayout.JAVA_BYTE, i)); - } - } - case U16 -> { - for (int i = 0; i < count; i++) { - out[i] = Short.toUnsignedInt(buf.get(PTypeIO.LE_SHORT, (long) i * 2)); - } - } - default -> throw new VortexException(EncodingId.FASTLANES_RLE, "unsupported indices ptype: " + indicesPtype); - } - return out; - } - - private static long[] readUnsignedLongs(MemorySegment buf, int count, PType ptype) { - long[] out = new long[count]; - int elemSize = ptype.byteSize(); - for (int i = 0; i < count; i++) { - long off = (long) i * elemSize; - out[i] = switch (ptype) { - case U8 -> Byte.toUnsignedLong(buf.get(ValueLayout.JAVA_BYTE, off)); - case U16 -> Short.toUnsignedLong(buf.get(PTypeIO.LE_SHORT, off)); - case U32 -> Integer.toUnsignedLong(buf.get(PTypeIO.LE_INT, off)); - case U64 -> buf.get(PTypeIO.LE_LONG, off); - default -> throw new VortexException(EncodingId.FASTLANES_RLE, "unsupported offsets ptype: " + ptype); - }; - } - return out; - } - - private static MemorySegment fromLongs(long[] decoded, int offset, int count, PType ptype, SegmentAllocator arena) { - int elemSize = ptype.byteSize(); - MemorySegment seg = arena.allocate((long) count * elemSize); - for (int i = 0; i < count; i++) { - PTypeIO.set(seg, (long) i * elemSize, ptype, decoded[offset + i]); - } - return seg; - } - - private static Array decodeChildAs(DecodeContext parent, int childIdx, DType dtype, long rowCount) { - ArrayNode childNode = parent.node().children()[childIdx]; - DecodeContext childCtx = new DecodeContext( - childNode, dtype, rowCount, parent.segmentBuffers(), parent.registry(), parent.arena()); - return parent.registry().decode(childCtx); - } - - private static PType ptypeFromProto(DTypeProtos.PType proto) { - return PType.values()[proto.getNumber()]; - } - } -} diff --git a/core/src/main/java/io/github/dfa1/vortex/encoding/RunEndEncoding.java b/core/src/main/java/io/github/dfa1/vortex/encoding/RunEndEncoding.java deleted file mode 100644 index 9f803ca..0000000 --- a/core/src/main/java/io/github/dfa1/vortex/encoding/RunEndEncoding.java +++ /dev/null @@ -1,372 +0,0 @@ -package io.github.dfa1.vortex.encoding; - -import com.google.protobuf.InvalidProtocolBufferException; -import io.github.dfa1.vortex.proto.DTypeProtos; -import io.github.dfa1.vortex.proto.EncodingProtos; -import io.github.dfa1.vortex.core.array.Array; -import io.github.dfa1.vortex.core.ArrayStats; -import io.github.dfa1.vortex.core.DType; -import io.github.dfa1.vortex.core.PType; -import io.github.dfa1.vortex.core.array.ByteArray; -import io.github.dfa1.vortex.core.array.IntArray; -import io.github.dfa1.vortex.core.array.LongArray; -import io.github.dfa1.vortex.core.array.ShortArray; -import io.github.dfa1.vortex.core.array.BoolArray; -import io.github.dfa1.vortex.core.array.VarBinArray; -import io.github.dfa1.vortex.core.VortexException; - -import java.lang.foreign.Arena; -import java.lang.foreign.SegmentAllocator; -import java.lang.foreign.MemorySegment; -import java.lang.foreign.ValueLayout; -import java.nio.ByteBuffer; -import java.util.ArrayList; -import java.util.List; - -/// Decoder for {@code vortex.runend} — run-end (RLE) encoding. -/// -///

Metadata: protobuf {@code RunEndMetadata} — {@code ends_ptype PType} (tag 1), -/// {@code num_runs u64} (tag 2), {@code offset u64} (tag 3). -/// -///

Child slot 0: run-end positions (unsigned, cumulative exclusive ends). -/// Child slot 1: run values (same dtype as parent, one per run). -/// -///

Decode: for each run i, repeat {@code values[i]} for positions -/// {@code [ends[i-1], ends[i])} in the output, skipping the first {@code offset} logical elements. -public final class RunEndEncoding implements Encoding { - - /// Creates a new {@code RunEndEncoding} instance; use via {@link EncodingRegistry}. - public RunEndEncoding() { - } - - @Override - public EncodingId encodingId() { - return EncodingId.VORTEX_RUNEND; - } - - @Override - public boolean accepts(DType dtype) { - return dtype instanceof DType.Primitive p && !p.ptype().isFloating(); - } - - @Override - public EncodeResult encode(DType dtype, Object data) { - return Encoder.encode(dtype, data); - } - - @Override - public Array decode(DecodeContext ctx) { - return Decoder.decode(ctx); - } - - private static final class Encoder { - - private static EncodeResult encode(DType dtype, Object data) { - if (!(dtype instanceof DType.Primitive p)) { - throw new VortexException(EncodingId.VORTEX_RUNEND, "encode only supports Primitive dtype, got " + dtype); - } - PType ptype = p.ptype(); - int n = arrayLength(data, ptype); - - List ends = new ArrayList<>(); - List values = new ArrayList<>(); - if (n > 0) { - long runVal = readLong(data, ptype, 0); - for (int i = 1; i < n; i++) { - long cur = readLong(data, ptype, i); - if (cur != runVal) { - ends.add(i); - values.add(runVal); - runVal = cur; - } - } - ends.add(n); - values.add(runVal); - } - - int numRuns = ends.size(); - - MemorySegment endsBuf = Arena.ofAuto().allocate((long) numRuns * 4, 4); - for (int i = 0; i < numRuns; i++) { - endsBuf.setAtIndex(PTypeIO.LE_INT, i, ends.get(i)); - } - - int elemBytes = ptype.byteSize(); - MemorySegment valuesBuf = Arena.ofAuto().allocate((long) numRuns * elemBytes, elemBytes); - for (int i = 0; i < numRuns; i++) { - PTypeIO.set(valuesBuf, (long) i * elemBytes, ptype, values.get(i)); - } - - byte[] metaBytes = EncodingProtos.RunEndMetadata.newBuilder() - .setEndsPtype(io.github.dfa1.vortex.proto.DTypeProtos.PType.forNumber(PType.U32.ordinal())) - .setNumRuns(numRuns) - .setOffset(0) - .build() - .toByteArray(); - - EncodeNode endsNode = EncodeNode.leaf(EncodingId.VORTEX_PRIMITIVE, 0); - EncodeNode valuesNode = EncodeNode.leaf(EncodingId.VORTEX_PRIMITIVE, 1); - EncodeNode root = new EncodeNode(EncodingId.VORTEX_RUNEND, ByteBuffer.wrap(metaBytes), - new EncodeNode[]{endsNode, valuesNode}, new int[0]); - return new EncodeResult(root, List.of(endsBuf, valuesBuf), null, null); - } - - private static int arrayLength(Object data, PType ptype) { - return switch (ptype) { - case I8, U8 -> ((byte[]) data).length; - case I16, U16 -> ((short[]) data).length; - case I32, U32 -> ((int[]) data).length; - case I64, U64 -> ((long[]) data).length; - default -> throw new VortexException(EncodingId.VORTEX_RUNEND, "unsupported ptype: " + ptype); - }; - } - - private static long readLong(Object data, PType ptype, int i) { - return switch (ptype) { - case I8 -> ((byte[]) data)[i]; - case U8 -> Byte.toUnsignedLong(((byte[]) data)[i]); - case I16 -> ((short[]) data)[i]; - case U16 -> Short.toUnsignedLong(((short[]) data)[i]); - case I32 -> ((int[]) data)[i]; - case U32 -> Integer.toUnsignedLong(((int[]) data)[i]); - case I64, U64 -> ((long[]) data)[i]; - default -> throw new VortexException(EncodingId.VORTEX_RUNEND, "unsupported ptype: " + ptype); - }; - } - } - - private static final class Decoder { - - private static Array decode(DecodeContext ctx) { - ByteBuffer rawMeta = ctx.metadata(); - if (rawMeta == null) { - throw new VortexException(EncodingId.VORTEX_RUNEND, "missing metadata"); - } - - EncodingProtos.RunEndMetadata meta; - try { - meta = EncodingProtos.RunEndMetadata.parseFrom(rawMeta.duplicate()); - } catch (InvalidProtocolBufferException e) { - throw new VortexException(EncodingId.VORTEX_RUNEND, "invalid metadata", e); - } - - PType endsPtype = ptypeFromProto(meta.getEndsPtype()); - long numRuns = meta.getNumRuns(); - long offset = meta.getOffset(); - - long n = ctx.rowCount(); - DType endsDtype = new DType.Primitive(endsPtype, false); - Array endsArr = decodeChildAs(ctx, 0, endsDtype, numRuns); - - if (ctx.dtype() instanceof DType.Utf8 || ctx.dtype() instanceof DType.Binary) { - Array valuesArr = decodeChildAs(ctx, 1, ctx.dtype(), numRuns); - return expandStrings(endsArr, (VarBinArray) valuesArr, endsPtype, numRuns, offset, n, ctx.dtype(), ctx.arena()); - } - - if (ctx.dtype() instanceof DType.Bool) { - Array valuesArr = decodeChildAs(ctx, 1, ctx.dtype(), numRuns); - return expandBool(endsArr, (BoolArray) valuesArr, endsPtype, numRuns, offset, n, ctx.dtype(), ctx.arena()); - } - - if (!(ctx.dtype() instanceof DType.Primitive p)) { - throw new VortexException(EncodingId.VORTEX_RUNEND, "expected primitive dtype, got " + ctx.dtype()); - } - PType valuePtype = p.ptype(); - Array valuesArr = decodeChildAs(ctx, 1, ctx.dtype(), numRuns); - - return expand(endsArr.buffer(0), valuesArr.buffer(0), - endsPtype, valuePtype, numRuns, offset, n, ctx.dtype(), ctx.arena()); - } - - private static Array expand( - MemorySegment endsSeg, MemorySegment valuesSeg, - PType endsPtype, PType valuePtype, - long numRuns, long offset, long n, - DType dtype, SegmentAllocator arena - ) { - MemorySegment out = arena.allocate(n * valuePtype.byteSize()); - switch (valuePtype) { - case I8, U8 -> expandByte(endsSeg, valuesSeg, endsPtype, numRuns, offset, n, out); - case I16, U16 -> expandShort(endsSeg, valuesSeg, endsPtype, numRuns, offset, n, out); - case I32, U32 -> expandInt(endsSeg, valuesSeg, endsPtype, numRuns, offset, n, out); - case I64, U64 -> expandLong(endsSeg, valuesSeg, endsPtype, numRuns, offset, n, out); - default -> throw new VortexException(EncodingId.VORTEX_RUNEND, "unsupported ptype " + valuePtype); - } - MemorySegment ro = out.asReadOnly(); - return switch (valuePtype) { - case I64, U64 -> new LongArray(dtype, n, ro, ArrayStats.empty()); - case I32, U32 -> new IntArray(dtype, n, ro, ArrayStats.empty()); - case I16, U16 -> new ShortArray(dtype, n, ro, ArrayStats.empty()); - case I8, U8 -> new ByteArray(dtype, n, ro, ArrayStats.empty()); - default -> throw new VortexException(EncodingId.VORTEX_RUNEND, "unsupported ptype " + valuePtype); - }; - } - - private static void expandByte(MemorySegment endsSeg, MemorySegment valuesSeg, - PType endsPtype, long numRuns, long offset, long n, MemorySegment out) { - long logicalPos = 0L, outPos = 0L; - for (long run = 0; run < numRuns && outPos < n; run++) { - long runEnd = readUnsigned(endsSeg, run, endsPtype); - byte rawValue = valuesSeg.get(ValueLayout.JAVA_BYTE, run); - long writeEnd = Math.min(runEnd, offset + n); - for (long lp = Math.max(logicalPos, offset); lp < writeEnd; lp++, outPos++) { - out.set(ValueLayout.JAVA_BYTE, outPos, rawValue); - } - logicalPos = runEnd; - } - } - - private static void expandShort(MemorySegment endsSeg, MemorySegment valuesSeg, - PType endsPtype, long numRuns, long offset, long n, MemorySegment out) { - long logicalPos = 0L, outPos = 0L; - for (long run = 0; run < numRuns && outPos < n; run++) { - long runEnd = readUnsigned(endsSeg, run, endsPtype); - short rawValue = valuesSeg.get(PTypeIO.LE_SHORT, run * 2); - long writeEnd = Math.min(runEnd, offset + n); - for (long lp = Math.max(logicalPos, offset); lp < writeEnd; lp++, outPos++) { - out.set(PTypeIO.LE_SHORT, outPos * 2, rawValue); - } - logicalPos = runEnd; - } - } - - private static void expandInt(MemorySegment endsSeg, MemorySegment valuesSeg, - PType endsPtype, long numRuns, long offset, long n, MemorySegment out) { - long logicalPos = 0L, outPos = 0L; - for (long run = 0; run < numRuns && outPos < n; run++) { - long runEnd = readUnsigned(endsSeg, run, endsPtype); - int rawValue = valuesSeg.get(PTypeIO.LE_INT, run * 4); - long writeEnd = Math.min(runEnd, offset + n); - for (long lp = Math.max(logicalPos, offset); lp < writeEnd; lp++, outPos++) { - out.set(PTypeIO.LE_INT, outPos * 4, rawValue); - } - logicalPos = runEnd; - } - } - - private static void expandLong(MemorySegment endsSeg, MemorySegment valuesSeg, - PType endsPtype, long numRuns, long offset, long n, MemorySegment out) { - long logicalPos = 0L, outPos = 0L; - for (long run = 0; run < numRuns && outPos < n; run++) { - long runEnd = readUnsigned(endsSeg, run, endsPtype); - long rawValue = valuesSeg.get(PTypeIO.LE_LONG, run * 8); - long writeEnd = Math.min(runEnd, offset + n); - for (long lp = Math.max(logicalPos, offset); lp < writeEnd; lp++, outPos++) { - out.set(PTypeIO.LE_LONG, outPos * 8, rawValue); - } - logicalPos = runEnd; - } - } - - private static Array expandBool( - Array endsArr, BoolArray valuesArr, - PType endsPtype, long numRuns, long offset, long n, - DType dtype, SegmentAllocator arena - ) { - MemorySegment endsSeg = endsArr.buffer(0); - long numBytes = (n + 7) >>> 3; - MemorySegment out = arena.allocate(numBytes); - - long outIdx = 0; - long logicalPos = 0; - for (long run = 0; run < numRuns && outIdx < n; run++) { - long runEnd = readUnsigned(endsSeg, run, endsPtype); - boolean val = valuesArr.getBoolean(run); - long lo = Math.max(logicalPos, offset); - long hi = Math.min(runEnd, offset + n); - for (long lp = lo; lp < hi; lp++, outIdx++) { - if (val) { - long byteIdx = outIdx >>> 3; - byte cur = out.get(ValueLayout.JAVA_BYTE, byteIdx); - out.set(ValueLayout.JAVA_BYTE, byteIdx, (byte) (cur | (1 << (outIdx & 7)))); - } - } - logicalPos = runEnd; - } - return new BoolArray(dtype, n, out.asReadOnly(), ArrayStats.empty()); - } - - private static Array expandStrings( - Array endsArr, VarBinArray valuesArr, - PType endsPtype, long numRuns, long offset, long n, - DType dtype, SegmentAllocator arena - ) { - MemorySegment endsSeg = endsArr.buffer(0); - MemorySegment valBytes = valuesArr.buffer(0); - MemorySegment valOffsets = valuesArr.child(0).buffer(0); - PType valOffPtype = ((DType.Primitive) valuesArr.child(0).dtype()).ptype(); - - long totalBytes = 0; - long logicalPos = 0; - for (long run = 0; run < numRuns; run++) { - long runEnd = readUnsigned(endsSeg, run, endsPtype); - long lo = Math.max(logicalPos, offset); - long hi = Math.min(runEnd, offset + n); - long count = Math.max(0, hi - lo); - long strLen = readVarBinOffset(valOffsets, run + 1, valOffPtype) - - readVarBinOffset(valOffsets, run, valOffPtype); - totalBytes += count * strLen; - logicalPos = runEnd; - } - - MemorySegment outBytes = arena.allocate(totalBytes > 0 ? totalBytes : 1); - MemorySegment outOffsets = arena.allocate((n + 1) * 4L, 4); - outOffsets.setAtIndex(PTypeIO.LE_INT, 0, 0); - - long bytePos = 0; - long outIdx = 0; - logicalPos = 0; - for (long run = 0; run < numRuns && outIdx < n; run++) { - long runEnd = readUnsigned(endsSeg, run, endsPtype); - long lo = Math.max(logicalPos, offset); - long hi = Math.min(runEnd, offset + n); - if (hi > lo) { - long strStart = readVarBinOffset(valOffsets, run, valOffPtype); - long strEnd = readVarBinOffset(valOffsets, run + 1, valOffPtype); - long strLen = strEnd - strStart; - for (long lp = lo; lp < hi; lp++, outIdx++) { - if (strLen > 0) { - MemorySegment.copy(valBytes, strStart, outBytes, bytePos, strLen); - bytePos += strLen; - } - outOffsets.setAtIndex(PTypeIO.LE_INT, outIdx + 1, (int) bytePos); - } - } - logicalPos = runEnd; - } - - DType i32 = new DType.Primitive(PType.I32, false); - Array offsetArr = new IntArray(i32, n + 1, outOffsets.asReadOnly(), ArrayStats.empty()); - return new VarBinArray(dtype, n, outBytes.asReadOnly(), offsetArr, PType.I32, ArrayStats.empty()); - } - - private static Array decodeChildAs(DecodeContext parent, int childIdx, DType dtype, long rowCount) { - ArrayNode childNode = parent.node().children()[childIdx]; - DecodeContext childCtx = new DecodeContext( - childNode, dtype, rowCount, parent.segmentBuffers(), parent.registry(), parent.arena()); - return parent.registry().decode(childCtx); - } - - private static long readUnsigned(MemorySegment seg, long i, PType ptype) { - return switch (ptype) { - case U8 -> Byte.toUnsignedLong(seg.get(ValueLayout.JAVA_BYTE, i)); - case U16 -> Short.toUnsignedLong(seg.get(PTypeIO.LE_SHORT, i * 2)); - case U32 -> Integer.toUnsignedLong(seg.get(PTypeIO.LE_INT, i * 4)); - case U64 -> seg.get(PTypeIO.LE_LONG, i * 8); - default -> throw new VortexException(EncodingId.VORTEX_RUNEND, "non-unsigned ends ptype " + ptype); - }; - } - - private static long readVarBinOffset(MemorySegment seg, long i, PType ptype) { - return switch (ptype) { - case I32, U32 -> Integer.toUnsignedLong(seg.getAtIndex(PTypeIO.LE_INT, i)); - case I64, U64 -> seg.getAtIndex(PTypeIO.LE_LONG, i); - default -> throw new VortexException(EncodingId.VORTEX_RUNEND, "unsupported offset ptype " + ptype); - }; - } - - private static PType ptypeFromProto(DTypeProtos.PType proto) { - return PType.values()[proto.getNumber()]; - } - } -} diff --git a/core/src/main/java/io/github/dfa1/vortex/encoding/SequenceEncoding.java b/core/src/main/java/io/github/dfa1/vortex/encoding/SequenceEncoding.java deleted file mode 100644 index 7f7e69f..0000000 --- a/core/src/main/java/io/github/dfa1/vortex/encoding/SequenceEncoding.java +++ /dev/null @@ -1,231 +0,0 @@ -package io.github.dfa1.vortex.encoding; - -import com.google.protobuf.InvalidProtocolBufferException; -import io.github.dfa1.vortex.proto.EncodingProtos; -import io.github.dfa1.vortex.proto.ScalarProtos; -import io.github.dfa1.vortex.core.array.Array; -import io.github.dfa1.vortex.core.ArrayStats; -import io.github.dfa1.vortex.core.DType; -import io.github.dfa1.vortex.core.PType; -import io.github.dfa1.vortex.core.array.ByteArray; -import io.github.dfa1.vortex.core.array.DoubleArray; -import io.github.dfa1.vortex.core.array.FloatArray; -import io.github.dfa1.vortex.core.array.IntArray; -import io.github.dfa1.vortex.core.array.LongArray; -import io.github.dfa1.vortex.core.array.ShortArray; -import io.github.dfa1.vortex.core.VortexException; - -import java.lang.foreign.SegmentAllocator; -import java.lang.foreign.MemorySegment; -import java.lang.foreign.ValueLayout; -import java.nio.ByteBuffer; -import java.util.List; - -/// Decoder for {@code vortex.sequence}: {@code A[i] = base + i * multiplier}. -/// -/// No buffers, no children. Metadata is a protobuf {@code SequenceMetadata} -/// with {@code base} (tag 1) and {@code multiplier} (tag 2) as {@code ScalarValue}. -/// Output is allocated on the heap; not backed by the file's mapped region. -public final class SequenceEncoding implements Encoding { - - /// Creates a new {@code SequenceEncoding} instance; use via {@link EncodingRegistry}. - public SequenceEncoding() { - } - - @Override - public EncodingId encodingId() { - return EncodingId.VORTEX_SEQUENCE; - } - - @Override - public EncodeResult encode(DType dtype, Object data) { - return Encoder.encode(dtype, data); - } - - @Override - public Array decode(DecodeContext ctx) { - return Decoder.decode(ctx); - } - - private static final class Encoder { - - private static EncodeResult encode(DType dtype, Object data) { - if (!(dtype instanceof DType.Primitive p)) { - throw new VortexException(EncodingId.VORTEX_SEQUENCE, "encode only supports Primitive dtype, got " + dtype); - } - PType pt = p.ptype(); - return switch (pt) { - case I8, I16, I32, I64, U8, U16, U32, U64 -> encodeInteger(pt, data); - case F32 -> encodeF32((float[]) data); - case F64 -> encodeF64((double[]) data); - case F16 -> throw new VortexException(EncodingId.VORTEX_SEQUENCE, "F16 not supported"); - }; - } - - private static EncodeResult encodeInteger(PType pt, Object data) { - int n = intArrayLength(pt, data); - long base = 0; - long multiplier = 0; - if (n > 0) { - base = readLong(pt, data, 0); - multiplier = n > 1 ? readLong(pt, data, 1) - base : 0; - for (int i = 2; i < n; i++) { - long expected = base + (long) i * multiplier; - if (readLong(pt, data, i) != expected) { - throw new VortexException(EncodingId.VORTEX_SEQUENCE, "not an arithmetic sequence at index " + i); - } - } - } - ScalarProtos.ScalarValue baseScalar = buildIntScalar(pt, base); - ScalarProtos.ScalarValue mulScalar = buildIntScalar(pt, multiplier); - return buildResult(baseScalar, mulScalar); - } - - private static EncodeResult encodeF32(float[] data) { - float base = data.length > 0 ? data[0] : 0f; - float mul = data.length > 1 ? data[1] - base : 0f; - for (int i = 2; i < data.length; i++) { - if (data[i] != base + (float) i * mul) { - throw new VortexException(EncodingId.VORTEX_SEQUENCE, "not an arithmetic sequence at index " + i); - } - } - ScalarProtos.ScalarValue baseScalar = ScalarProtos.ScalarValue.newBuilder().setF32Value(base).build(); - ScalarProtos.ScalarValue mulScalar = ScalarProtos.ScalarValue.newBuilder().setF32Value(mul).build(); - return buildResult(baseScalar, mulScalar); - } - - private static EncodeResult encodeF64(double[] data) { - double base = data.length > 0 ? data[0] : 0.0; - double mul = data.length > 1 ? data[1] - base : 0.0; - for (int i = 2; i < data.length; i++) { - if (data[i] != base + (double) i * mul) { - throw new VortexException(EncodingId.VORTEX_SEQUENCE, "not an arithmetic sequence at index " + i); - } - } - ScalarProtos.ScalarValue baseScalar = ScalarProtos.ScalarValue.newBuilder().setF64Value(base).build(); - ScalarProtos.ScalarValue mulScalar = ScalarProtos.ScalarValue.newBuilder().setF64Value(mul).build(); - return buildResult(baseScalar, mulScalar); - } - - private static EncodeResult buildResult(ScalarProtos.ScalarValue base, ScalarProtos.ScalarValue mul) { - EncodingProtos.SequenceMetadata meta = EncodingProtos.SequenceMetadata.newBuilder() - .setBase(base) - .setMultiplier(mul) - .build(); - ByteBuffer metaBuf = ByteBuffer.wrap(meta.toByteArray()); - EncodeNode node = new EncodeNode(EncodingId.VORTEX_SEQUENCE, metaBuf, new EncodeNode[0], new int[]{}); - return new EncodeResult(node, List.of(), null, null); - } - - private static ScalarProtos.ScalarValue buildIntScalar(PType pt, long value) { - return switch (pt) { - case U8, U16, U32, U64 -> ScalarProtos.ScalarValue.newBuilder().setUint64Value(value).build(); - default -> ScalarProtos.ScalarValue.newBuilder().setInt64Value(value).build(); - }; - } - - private static int intArrayLength(PType pt, Object data) { - return switch (pt) { - case I8, U8 -> ((byte[]) data).length; - case I16, U16 -> ((short[]) data).length; - case I32, U32 -> ((int[]) data).length; - case I64, U64 -> ((long[]) data).length; - default -> throw new VortexException(EncodingId.VORTEX_SEQUENCE, "unsupported ptype: " + pt); - }; - } - - private static long readLong(PType pt, Object data, int i) { - return switch (pt) { - case I8, U8 -> ((byte[]) data)[i]; - case I16, U16 -> ((short[]) data)[i]; - case I32, U32 -> ((int[]) data)[i]; - case I64, U64 -> ((long[]) data)[i]; - default -> throw new VortexException(EncodingId.VORTEX_SEQUENCE, "unsupported ptype: " + pt); - }; - } - } - - private static final class Decoder { - - private static Array decode(DecodeContext ctx) { - ByteBuffer metaBuf = ctx.metadata(); - if (metaBuf == null || !metaBuf.hasRemaining()) { - throw new VortexException(EncodingId.VORTEX_SEQUENCE, "missing metadata"); - } - EncodingProtos.SequenceMetadata meta; - try { - meta = EncodingProtos.SequenceMetadata.parseFrom(metaBuf.duplicate()); - } catch (InvalidProtocolBufferException e) { - throw new VortexException(EncodingId.VORTEX_SEQUENCE, "invalid metadata", e); - } - - if (!(ctx.dtype() instanceof DType.Primitive p)) { - throw new VortexException(EncodingId.VORTEX_SEQUENCE, "expected primitive dtype, got " + ctx.dtype()); - } - - long n = ctx.rowCount(); - PType pt = p.ptype(); - return switch (pt) { - case I8, I16, I32, I64, U8, U16, U32, U64 -> decodeInteger(meta, pt, n, ctx.dtype(), ctx.arena()); - case F32 -> decodeF32(meta, n, ctx.dtype(), ctx.arena()); - case F64 -> decodeF64(meta, n, ctx.dtype(), ctx.arena()); - case F16 -> throw new VortexException(EncodingId.VORTEX_SEQUENCE, "F16 not supported"); - }; - } - - private static Array decodeInteger( - EncodingProtos.SequenceMetadata meta, PType pt, long n, DType dtype, SegmentAllocator arena - ) { - long base = signedValue(meta.getBase()); - long mul = signedValue(meta.getMultiplier()); - int elemBytes = pt.byteSize(); - MemorySegment seg = arena.allocate(n * elemBytes); - for (long i = 0; i < n; i++) { - long v = base + i * mul; - switch (pt) { - case I8, U8 -> seg.set(ValueLayout.JAVA_BYTE, i, (byte) v); - case I16, U16 -> seg.setAtIndex(PTypeIO.LE_SHORT, i, (short) v); - case I32, U32 -> seg.setAtIndex(PTypeIO.LE_INT, i, (int) v); - case I64, U64 -> seg.setAtIndex(PTypeIO.LE_LONG, i, v); - default -> throw new IllegalStateException("unreachable"); - } - } - return switch (pt) { - case I64, U64 -> new LongArray(dtype, n, seg, ArrayStats.empty()); - case I32, U32 -> new IntArray(dtype, n, seg, ArrayStats.empty()); - case I16, U16 -> new ShortArray(dtype, n, seg, ArrayStats.empty()); - case I8, U8 -> new ByteArray(dtype, n, seg, ArrayStats.empty()); - default -> throw new VortexException(EncodingId.VORTEX_SEQUENCE, "unsupported ptype " + pt); - }; - } - - private static Array decodeF32(EncodingProtos.SequenceMetadata meta, long n, DType dtype, SegmentAllocator arena) { - float base = meta.getBase().getF32Value(); - float mul = meta.getMultiplier().getF32Value(); - MemorySegment seg = arena.allocate(n * 4L); - for (long i = 0; i < n; i++) { - seg.setAtIndex(PTypeIO.LE_FLOAT, i, base + i * mul); - } - return new FloatArray(dtype, n, seg, ArrayStats.empty()); - } - - private static Array decodeF64(EncodingProtos.SequenceMetadata meta, long n, DType dtype, SegmentAllocator arena) { - double base = meta.getBase().getF64Value(); - double mul = meta.getMultiplier().getF64Value(); - MemorySegment seg = arena.allocate(n * 8L); - for (long i = 0; i < n; i++) { - seg.setAtIndex(PTypeIO.LE_DOUBLE, i, base + i * mul); - } - return new DoubleArray(dtype, n, seg, ArrayStats.empty()); - } - - private static long signedValue(io.github.dfa1.vortex.proto.ScalarProtos.ScalarValue sv) { - return switch (sv.getKindCase()) { - case INT64_VALUE -> sv.getInt64Value(); - case UINT64_VALUE -> sv.getUint64Value(); - case KIND_NOT_SET -> 0L; - default -> throw new VortexException(EncodingId.VORTEX_SEQUENCE, "unexpected scalar kind " + sv.getKindCase()); - }; - } - } -} diff --git a/core/src/main/java/io/github/dfa1/vortex/encoding/SparseEncoding.java b/core/src/main/java/io/github/dfa1/vortex/encoding/SparseEncoding.java deleted file mode 100644 index 0f888d3..0000000 --- a/core/src/main/java/io/github/dfa1/vortex/encoding/SparseEncoding.java +++ /dev/null @@ -1,412 +0,0 @@ -package io.github.dfa1.vortex.encoding; - -import com.google.protobuf.InvalidProtocolBufferException; -import io.github.dfa1.vortex.proto.DTypeProtos; -import io.github.dfa1.vortex.proto.EncodingProtos; -import io.github.dfa1.vortex.proto.ScalarProtos; -import io.github.dfa1.vortex.core.array.Array; -import io.github.dfa1.vortex.core.ArrayStats; -import io.github.dfa1.vortex.core.DType; -import io.github.dfa1.vortex.core.PType; -import io.github.dfa1.vortex.core.array.BoolArray; -import io.github.dfa1.vortex.core.array.ByteArray; -import io.github.dfa1.vortex.core.array.DoubleArray; -import io.github.dfa1.vortex.core.array.FloatArray; -import io.github.dfa1.vortex.core.array.IntArray; -import io.github.dfa1.vortex.core.array.LongArray; -import io.github.dfa1.vortex.core.array.ShortArray; -import io.github.dfa1.vortex.core.array.VarBinArray; -import io.github.dfa1.vortex.core.VortexException; - -import java.lang.foreign.Arena; -import java.lang.foreign.MemorySegment; -import java.lang.foreign.ValueLayout; -import java.nio.ByteBuffer; -import java.nio.ByteOrder; -import java.util.ArrayList; -import java.util.List; - -/// Decoder for {@code vortex.sparse}. -/// -///

Metadata: protobuf {@code SparseMetadata} wrapping a {@code PatchesMetadata} -/// (patch count, index offset, index ptype). -/// Buffer 0: fill value as raw {@code ScalarValue} proto bytes. -/// Child slot 0: patch indices (unsigned int, dtype from PatchesMetadata.indices_ptype). -/// Child slot 1: patch values (same dtype as parent). -/// -///

Decode: allocate output filled with fill_value, then overwrite -/// {@code output[indices[i] - offset] = values[i]} for each patch. -public final class SparseEncoding implements Encoding { - - /// Creates a new {@code SparseEncoding} instance; use via {@link EncodingRegistry}. - public SparseEncoding() { - } - - @Override - public EncodingId encodingId() { - return EncodingId.VORTEX_SPARSE; - } - - @Override - public boolean accepts(DType dtype) { - return dtype instanceof DType.Primitive; - } - - @Override - public EncodeResult encode(DType dtype, Object data) { - return Encoder.encode(dtype, data); - } - - @Override - public Array decode(DecodeContext ctx) { - return Decoder.decode(ctx); - } - - private static final class Encoder { - - private static EncodeResult encode(DType dtype, Object data) { - if (!(dtype instanceof DType.Primitive p)) { - throw new VortexException(EncodingId.VORTEX_SPARSE, - "encode only supports Primitive dtype, got " + dtype); - } - PType ptype = p.ptype(); - int n = arrayLength(data, ptype); - - List patchIdx = new ArrayList<>(); - List patchBits = new ArrayList<>(); - for (int i = 0; i < n; i++) { - long bits = readBits(data, ptype, i); - if (bits != 0L) { - patchIdx.add(i); - patchBits.add(bits); - } - } - - int numPatches = patchIdx.size(); - PType idxPtype = chooseIdxPtype(n); - - ScalarProtos.ScalarValue fillScalar = zeroScalar(ptype); - byte[] fillBytes = fillScalar.toByteArray(); - MemorySegment fillBuf = Arena.ofAuto().allocate(fillBytes.length); - MemorySegment.copy(MemorySegment.ofArray(fillBytes), 0, fillBuf, 0, fillBytes.length); - - MemorySegment idxBuf = buildIdxBuf(patchIdx, idxPtype, numPatches); - MemorySegment valBuf = buildValBuf(patchBits, ptype, numPatches); - - byte[] metaBytes = EncodingProtos.SparseMetadata.newBuilder() - .setPatches(EncodingProtos.PatchesMetadata.newBuilder() - .setLen(numPatches) - .setOffset(0) - .setIndicesPtype(DTypeProtos.PType.forNumber(idxPtype.ordinal())) - .build()) - .build() - .toByteArray(); - - EncodeNode idxNode = EncodeNode.leaf(EncodingId.VORTEX_PRIMITIVE, 1); - EncodeNode valNode = EncodeNode.leaf(EncodingId.VORTEX_PRIMITIVE, 2); - EncodeNode root = new EncodeNode(EncodingId.VORTEX_SPARSE, ByteBuffer.wrap(metaBytes), - new EncodeNode[]{idxNode, valNode}, new int[]{0}); - return new EncodeResult(root, List.of(fillBuf, idxBuf, valBuf), null, null); - } - - private static int arrayLength(Object data, PType ptype) { - return switch (ptype) { - case I8, U8 -> ((byte[]) data).length; - case I16, U16 -> ((short[]) data).length; - case I32, U32 -> ((int[]) data).length; - case I64, U64 -> ((long[]) data).length; - case F32 -> ((float[]) data).length; - case F64 -> ((double[]) data).length; - default -> throw new VortexException(EncodingId.VORTEX_SPARSE, "unsupported ptype: " + ptype); - }; - } - - private static long readBits(Object data, PType ptype, int i) { - return switch (ptype) { - case I8 -> ((byte[]) data)[i]; - case U8 -> Byte.toUnsignedLong(((byte[]) data)[i]); - case I16 -> ((short[]) data)[i]; - case U16 -> Short.toUnsignedLong(((short[]) data)[i]); - case I32 -> ((int[]) data)[i]; - case U32 -> Integer.toUnsignedLong(((int[]) data)[i]); - case I64, U64 -> ((long[]) data)[i]; - case F32 -> Float.floatToRawIntBits(((float[]) data)[i]); - case F64 -> Double.doubleToRawLongBits(((double[]) data)[i]); - default -> throw new VortexException(EncodingId.VORTEX_SPARSE, "unsupported ptype: " + ptype); - }; - } - - private static PType chooseIdxPtype(int n) { - if (n <= 0xFF) { - return PType.U8; - } else if (n <= 0xFFFF) { - return PType.U16; - } else { - return PType.U32; - } - } - - private static ScalarProtos.ScalarValue zeroScalar(PType ptype) { - ScalarProtos.ScalarValue.Builder b = ScalarProtos.ScalarValue.newBuilder(); - return switch (ptype) { - case I8, I16, I32, I64 -> b.setInt64Value(0L).build(); - case U8, U16, U32, U64 -> b.setUint64Value(0L).build(); - case F32 -> b.setF32Value(0.0f).build(); - case F64 -> b.setF64Value(0.0).build(); - default -> throw new VortexException(EncodingId.VORTEX_SPARSE, "unsupported ptype: " + ptype); - }; - } - - private static MemorySegment buildIdxBuf(List patchIdx, PType idxPtype, int numPatches) { - int elemBytes = idxPtype.byteSize(); - MemorySegment seg = Arena.ofAuto().allocate(Math.max(1L, (long) numPatches * elemBytes), elemBytes); - for (int i = 0; i < numPatches; i++) { - PTypeIO.set(seg, (long) i * elemBytes, idxPtype, patchIdx.get(i)); - } - return seg; - } - - private static MemorySegment buildValBuf(List patchBits, PType ptype, int numPatches) { - int elemBytes = ptype.byteSize(); - MemorySegment seg = Arena.ofAuto().allocate(Math.max(1L, (long) numPatches * elemBytes), elemBytes); - for (int i = 0; i < numPatches; i++) { - PTypeIO.set(seg, (long) i * elemBytes, ptype, patchBits.get(i)); - } - return seg; - } - } - - private static final class Decoder { - - private static Array decode(DecodeContext ctx) { - ByteBuffer rawMeta = ctx.metadata(); - if (rawMeta == null || !rawMeta.hasRemaining()) { - throw new VortexException(EncodingId.VORTEX_SPARSE, "missing metadata"); - } - EncodingProtos.SparseMetadata sparseMeta; - try { - sparseMeta = EncodingProtos.SparseMetadata.parseFrom(rawMeta.duplicate()); - } catch (InvalidProtocolBufferException e) { - throw new VortexException(EncodingId.VORTEX_SPARSE, "invalid metadata", e); - } - - EncodingProtos.PatchesMetadata patches = sparseMeta.getPatches(); - long numPatches = patches.getLen(); - long offset = patches.getOffset(); - PType indicesPtype = ptypeFromProto(patches.getIndicesPtype()); - - long n = ctx.rowCount(); - - if (ctx.dtype() instanceof DType.Utf8 || ctx.dtype() instanceof DType.Binary) { - return decodeVarBin(ctx, n, numPatches, offset, indicesPtype); - } - - if (ctx.dtype() instanceof DType.Bool) { - return decodeBool(ctx, n, numPatches, offset, indicesPtype); - } - - if (!(ctx.dtype() instanceof DType.Primitive)) { - throw new VortexException(EncodingId.VORTEX_SPARSE, "expected primitive dtype, got " + ctx.dtype()); - } - PType valuePtype = ((DType.Primitive) ctx.dtype()).ptype(); - - MemorySegment fillBuf = ctx.buffer(0); - ScalarProtos.ScalarValue fillScalar; - try { - fillScalar = ScalarProtos.ScalarValue.parseFrom(fillBuf.asByteBuffer()); - } catch (InvalidProtocolBufferException e) { - throw new VortexException(EncodingId.VORTEX_SPARSE, "invalid fill value", e); - } - - int elemBytes = valuePtype.byteSize(); - MemorySegment out = ctx.arena().allocate(n * elemBytes); - fillSegment(out, n, valuePtype, fillScalar); - - if (numPatches > 0) { - DType indicesDtype = new DType.Primitive(indicesPtype, false); - Array indicesArray = decodeChild(ctx, 0, indicesDtype, numPatches); - Array valuesArray = decodeChild(ctx, 1, ctx.dtype(), numPatches); - applyPatches(out, n, valuePtype, - indicesArray.buffer(0), valuesArray.buffer(0), indicesPtype, numPatches, offset); - } - - return switch (valuePtype) { - case I64, U64 -> new LongArray(ctx.dtype(), n, out, ArrayStats.empty()); - case I32, U32 -> new IntArray(ctx.dtype(), n, out, ArrayStats.empty()); - case F64 -> new DoubleArray(ctx.dtype(), n, out, ArrayStats.empty()); - case F32 -> new FloatArray(ctx.dtype(), n, out, ArrayStats.empty()); - case I16, U16 -> new ShortArray(ctx.dtype(), n, out, ArrayStats.empty()); - case I8, U8 -> new ByteArray(ctx.dtype(), n, out, ArrayStats.empty()); - default -> throw new VortexException(EncodingId.VORTEX_SPARSE, "unsupported ptype " + valuePtype); - }; - } - - private static Array decodeBool( - DecodeContext ctx, long n, long numPatches, long offset, PType indicesPtype - ) { - long numBytes = (n + 7) >>> 3; - MemorySegment out = ctx.arena().allocate(numBytes); - if (numPatches > 0) { - DType indicesDtype = new DType.Primitive(indicesPtype, false); - Array indicesArray = decodeChild(ctx, 0, indicesDtype, numPatches); - Array valuesArray = decodeChild(ctx, 1, ctx.dtype(), numPatches); - MemorySegment idxSeg = indicesArray.buffer(0); - BoolArray bools = (BoolArray) valuesArray; - for (long i = 0; i < numPatches; i++) { - if (bools.getBoolean(i)) { - long pos = readUnsignedIdx(idxSeg, i, indicesPtype) - offset; - long byteIdx = pos >>> 3; - byte cur = out.get(ValueLayout.JAVA_BYTE, byteIdx); - out.set(ValueLayout.JAVA_BYTE, byteIdx, (byte) (cur | (1 << (pos & 7)))); - } - } - } - return new BoolArray(ctx.dtype(), n, out, ArrayStats.empty()); - } - - private static Array decodeVarBin( - DecodeContext ctx, long n, long numPatches, long offset, PType indicesPtype - ) { - MemorySegment outOffsets = ctx.arena().allocate((n + 1) * 4L, 4); - if (numPatches == 0) { - MemorySegment outBytes = ctx.arena().allocate(1); - DType i32dtype = new DType.Primitive(PType.I32, false); - Array offsetArr = new IntArray(i32dtype, n + 1, outOffsets, ArrayStats.empty()); - return new VarBinArray(ctx.dtype(), n, outBytes, offsetArr, PType.I32, ArrayStats.empty()); - } - - DType indicesDtype = new DType.Primitive(indicesPtype, false); - Array indicesArray = decodeChild(ctx, 0, indicesDtype, numPatches); - Array valuesArray = decodeChild(ctx, 1, ctx.dtype(), numPatches); - - MemorySegment idxSeg = indicesArray.buffer(0); - VarBinArray varBin = (VarBinArray) valuesArray; - MemorySegment valBytes = varBin.buffer(0); - MemorySegment valOffsets = varBin.child(0).buffer(0); - PType valOffPtype = ((DType.Primitive) varBin.child(0).dtype()).ptype(); - - long totalBytes = 0; - for (long i = 0; i < numPatches; i++) { - totalBytes += readVarBinOffset(valOffsets, i + 1, valOffPtype) - - readVarBinOffset(valOffsets, i, valOffPtype); - } - - MemorySegment outBytes = ctx.arena().allocate(Math.max(1, totalBytes)); - long patchCursor = 0; - long bytePos = 0; - for (long pos = 0; pos < n; pos++) { - if (patchCursor < numPatches) { - long patchPos = readUnsignedIdx(idxSeg, patchCursor, indicesPtype) - offset; - if (patchPos == pos) { - long strStart = readVarBinOffset(valOffsets, patchCursor, valOffPtype); - long strEnd = readVarBinOffset(valOffsets, patchCursor + 1, valOffPtype); - long strLen = strEnd - strStart; - if (strLen > 0) { - MemorySegment.copy(valBytes, strStart, outBytes, bytePos, strLen); - bytePos += strLen; - } - patchCursor++; - } - } - outOffsets.setAtIndex(PTypeIO.LE_INT, pos + 1, (int) bytePos); - } - - DType i32dtype = new DType.Primitive(PType.I32, false); - Array offsetArr = new IntArray(i32dtype, n + 1, outOffsets, ArrayStats.empty()); - return new VarBinArray(ctx.dtype(), n, outBytes, offsetArr, PType.I32, ArrayStats.empty()); - } - - private static long readVarBinOffset(MemorySegment seg, long i, PType ptype) { - return switch (ptype) { - case I32, U32 -> Integer.toUnsignedLong(seg.getAtIndex(PTypeIO.LE_INT, i)); - case I64, U64 -> seg.getAtIndex(PTypeIO.LE_LONG, i); - default -> throw new VortexException(EncodingId.VORTEX_SPARSE, "unsupported offset ptype " + ptype); - }; - } - - private static Array decodeChild(DecodeContext parent, int childIdx, DType dtype, long rowCount) { - ArrayNode childNode = parent.node().children()[childIdx]; - DecodeContext childCtx = new DecodeContext( - childNode, dtype, rowCount, parent.segmentBuffers(), parent.registry(), parent.arena()); - try { - return parent.registry().decode(childCtx); - } catch (Exception e) { - throw new VortexException(EncodingId.VORTEX_SPARSE, "failed to decode child " + childIdx, e); - } - } - - private static void fillSegment(MemorySegment out, long n, PType ptype, ScalarProtos.ScalarValue scalar) { - long fillLong = scalarToLong(scalar, ptype); - ByteBuffer bb = out.asByteBuffer().order(ByteOrder.LITTLE_ENDIAN); - for (long i = 0; i < n; i++) { - writeElem(bb, ptype, fillLong); - } - } - - private static void applyPatches( - MemorySegment out, long n, PType valuePtype, - MemorySegment idxSeg, MemorySegment valSeg, - PType idxPtype, long numPatches, long offset - ) { - int elemBytes = valuePtype.byteSize(); - ByteBuffer outBuf = out.asByteBuffer().order(ByteOrder.LITTLE_ENDIAN); - for (long i = 0; i < numPatches; i++) { - long idx = readUnsignedIdx(idxSeg, i, idxPtype) - offset; - if (idx < 0 || idx >= n) { - throw new VortexException(EncodingId.VORTEX_SPARSE, - "patch index " + idx + " out of range [0," + n + ")"); - } - long val = readElem(valSeg, i, valuePtype); - outBuf.position((int) (idx * elemBytes)); - writeElem(outBuf, valuePtype, val); - } - } - - private static long readUnsignedIdx(MemorySegment seg, long i, PType ptype) { - return switch (ptype) { - case U8 -> Byte.toUnsignedLong(seg.get(ValueLayout.JAVA_BYTE, i)); - case U16 -> Short.toUnsignedLong(seg.get(PTypeIO.LE_SHORT, i * 2)); - case U32 -> Integer.toUnsignedLong(seg.get(PTypeIO.LE_INT, i * 4)); - case U64 -> seg.get(PTypeIO.LE_LONG, i * 8); - default -> throw new VortexException(EncodingId.VORTEX_SPARSE, "non-unsigned index ptype " + ptype); - }; - } - - private static long readElem(MemorySegment seg, long i, PType ptype) { - return switch (ptype) { - case I8, U8 -> Byte.toUnsignedLong(seg.get(ValueLayout.JAVA_BYTE, i)); - case I16, U16 -> Short.toUnsignedLong(seg.get(PTypeIO.LE_SHORT, i * 2)); - case I32, U32 -> Integer.toUnsignedLong(seg.get(PTypeIO.LE_INT, i * 4)); - case I64, U64, F32, F64 -> seg.get(PTypeIO.LE_LONG, i * 8); - default -> throw new UnsupportedOperationException("vortex.sparse: unsupported ptype " + ptype); - }; - } - - private static void writeElem(ByteBuffer bb, PType ptype, long bits) { - switch (ptype) { - case I8, U8 -> bb.put((byte) bits); - case I16, U16 -> bb.putShort((short) bits); - case I32, U32 -> bb.putInt((int) bits); - case I64, U64, F32, F64 -> bb.putLong(bits); - default -> throw new UnsupportedOperationException("vortex.sparse: unsupported ptype " + ptype); - } - } - - private static long scalarToLong(ScalarProtos.ScalarValue scalar, PType ptype) { - return switch (scalar.getKindCase()) { - case INT64_VALUE -> scalar.getInt64Value(); - case UINT64_VALUE -> scalar.getUint64Value(); - case F32_VALUE -> Float.floatToRawIntBits(scalar.getF32Value()); - case F64_VALUE -> Double.doubleToRawLongBits(scalar.getF64Value()); - case NULL_VALUE, KIND_NOT_SET -> 0L; - default -> throw new VortexException(EncodingId.VORTEX_SPARSE, - "unexpected scalar kind " + scalar.getKindCase()); - }; - } - - // PType proto enum ordinals match Java PType ordinals (U8=0..F64=10) - private static PType ptypeFromProto(DTypeProtos.PType proto) { - return PType.values()[proto.getNumber()]; - } - } -} diff --git a/core/src/main/java/io/github/dfa1/vortex/encoding/StructData.java b/core/src/main/java/io/github/dfa1/vortex/encoding/StructData.java deleted file mode 100644 index 58389dc..0000000 --- a/core/src/main/java/io/github/dfa1/vortex/encoding/StructData.java +++ /dev/null @@ -1,14 +0,0 @@ -package io.github.dfa1.vortex.encoding; - -import java.util.List; - -/// Input data for encoding a struct-typed column. -/// {@code fieldArrays} is parallel to {@link io.github.dfa1.vortex.core.DType.Struct#fieldTypes()}. -/// -/// @param fieldArrays per-field data arrays in the same order as the struct's field types -public record StructData(List fieldArrays) { - /// Validates and defensively copies the field arrays list. - public StructData { - fieldArrays = List.copyOf(fieldArrays); - } -} diff --git a/core/src/main/java/io/github/dfa1/vortex/encoding/StructEncoding.java b/core/src/main/java/io/github/dfa1/vortex/encoding/StructEncoding.java deleted file mode 100644 index 2ebb639..0000000 --- a/core/src/main/java/io/github/dfa1/vortex/encoding/StructEncoding.java +++ /dev/null @@ -1,169 +0,0 @@ -package io.github.dfa1.vortex.encoding; - -import io.github.dfa1.vortex.core.DType; -import io.github.dfa1.vortex.core.VortexException; -import io.github.dfa1.vortex.core.array.Array; -import io.github.dfa1.vortex.core.array.BoolArray; -import io.github.dfa1.vortex.core.array.MaskedArray; -import io.github.dfa1.vortex.core.array.StructArray; - -import java.lang.foreign.MemorySegment; -import java.util.ArrayList; -import java.util.List; - -/// Decoder for {@code vortex.struct}. -/// -///

Wire format (per Rust vtable): -///

    -///
  • Buffers: 0 -///
  • Metadata: empty byte array -///
  • Children: {@code nfields} or {@code nfields + 1}. When {@code nfields + 1}, -/// {@code children[0]} is the validity (bool) array and {@code children[1..]} are fields. -/// When {@code nfields}, there is no validity child. -///
-/// -///

For multi-field structs the outer dtype is {@link DType.Struct}. For single-field -/// nullable scalar wrappers the outer dtype is the scalar type (e.g. {@link DType.Primitive}) -/// and {@code nfields == 1}. -public final class StructEncoding implements Encoding { - - /// Creates a new {@code StructEncoding} instance; use via {@link EncodingRegistry}. - public StructEncoding() { - } - - @Override - public EncodingId encodingId() { - return EncodingId.VORTEX_STRUCT; - } - - @Override - public boolean accepts(DType dtype) { - return dtype instanceof DType.Struct; - } - - @Override - public EncodeResult encode(DType dtype, Object data) { - return Encoder.encode((DType.Struct) dtype, (StructData) data); - } - - @Override - public Array decode(DecodeContext ctx) { - return Decoder.decode(ctx); - } - - private static final class Encoder { - - private static final List FALLBACK = List.of( - new PrimitiveEncoding(), new VarBinEncoding(), new BoolEncoding(), - new NullEncoding(), new ByteBoolEncoding()); - - static EncodeResult encode(DType.Struct dtype, StructData data) { - List fields = data.fieldArrays(); - List fieldTypes = dtype.fieldTypes(); - if (fields.size() != fieldTypes.size()) { - throw new VortexException(EncodingId.VORTEX_STRUCT, - "fieldArrays length %d != fieldTypes length %d".formatted(fields.size(), fieldTypes.size())); - } - List allBuffers = new ArrayList<>(); - EncodeNode[] children = new EncodeNode[fields.size()]; - for (int i = 0; i < fields.size(); i++) { - DType fieldDtype = fieldTypes.get(i); - EncodeResult fieldResult = findEncoding(fieldDtype).encode(fieldDtype, fields.get(i)); - int bufOffset = allBuffers.size(); - children[i] = EncodeNode.remapBufferIndices(fieldResult.rootNode(), bufOffset); - allBuffers.addAll(fieldResult.buffers()); - } - EncodeNode root = new EncodeNode(EncodingId.VORTEX_STRUCT, null, children, new int[0]); - return new EncodeResult(root, List.copyOf(allBuffers), null, null); - } - - private static Encoding findEncoding(DType dtype) { - for (Encoding enc : FALLBACK) { - if (enc.accepts(dtype)) { - return enc; - } - } - throw new UnsupportedOperationException("no fallback encoding for dtype: " + dtype); - } - } - - private static final class Decoder { - - private static Array decode(DecodeContext ctx) { - int numChildren = ctx.node().children().length; - - if (ctx.dtype() instanceof DType.Struct structDtype) { - // Struct array: children = [validity?, field_0, ..., field_n-1] - int nfields = structDtype.fieldTypes().size(); - if (numChildren != nfields && numChildren != nfields + 1) { - throw new VortexException(EncodingId.VORTEX_STRUCT, - "expected %d or %d children for struct dtype, got %d" - .formatted(nfields, nfields + 1, numChildren)); - } - boolean hasValidity = (numChildren == nfields + 1); - int fieldOffset = hasValidity ? 1 : 0; - - BoolArray structValidity = null; - if (hasValidity) { - ArrayNode validityNode = ctx.node().children()[0]; - var validityCtx = new DecodeContext(validityNode, new DType.Bool(false), - ctx.rowCount(), ctx.segmentBuffers(), ctx.registry(), ctx.arena()); - Array va = ctx.registry().decode(validityCtx); - if (!(va instanceof BoolArray ba)) { - throw new VortexException(EncodingId.VORTEX_STRUCT, - "struct validity decoded to unexpected type: " + va.getClass().getSimpleName()); - } - structValidity = ba; - } - - if (nfields == 1) { - DType fieldDtype = structDtype.fieldTypes().getFirst(); - ArrayNode fieldNode = ctx.node().children()[fieldOffset]; - var fieldCtx = new DecodeContext(fieldNode, fieldDtype.withNullable(false), - ctx.rowCount(), ctx.segmentBuffers(), ctx.registry(), ctx.arena()); - Array field = ctx.registry().decode(fieldCtx); - return structValidity != null ? new MaskedArray(field, structValidity) : field; - } - - List fieldArrays = new ArrayList<>(nfields); - for (int i = 0; i < nfields; i++) { - ArrayNode fieldNode = ctx.node().children()[fieldOffset + i]; - DType fieldDtype = structDtype.fieldTypes().get(i); - var fieldCtx = new DecodeContext(fieldNode, fieldDtype.withNullable(false), - ctx.rowCount(), ctx.segmentBuffers(), ctx.registry(), ctx.arena()); - Array field = ctx.registry().decode(fieldCtx); - fieldArrays.add(structValidity != null ? new MaskedArray(field, structValidity) : field); - } - return new StructArray(structDtype, ctx.rowCount(), fieldArrays); - } - - // Scalar nullable wrapper: nfields == 1 - // children = [values] (non-nullable) or [validity, values] (nullable) - if (numChildren == 1) { - ArrayNode valuesNode = ctx.node().children()[0]; - var valuesCtx = new DecodeContext( - valuesNode, ctx.dtype(), ctx.rowCount(), - ctx.segmentBuffers(), ctx.registry(), ctx.arena()); - return ctx.registry().decode(valuesCtx); - } else if (numChildren == 2) { - ArrayNode validityNode = ctx.node().children()[0]; - var validityCtx = new DecodeContext(validityNode, new DType.Bool(false), - ctx.rowCount(), ctx.segmentBuffers(), ctx.registry(), ctx.arena()); - Array va = ctx.registry().decode(validityCtx); - if (!(va instanceof BoolArray validity)) { - throw new VortexException(EncodingId.VORTEX_STRUCT, - "scalar wrapper validity decoded to unexpected type: " + va.getClass().getSimpleName()); - } - ArrayNode valuesNode = ctx.node().children()[1]; - var valuesCtx = new DecodeContext( - valuesNode, ctx.dtype().withNullable(false), ctx.rowCount(), - ctx.segmentBuffers(), ctx.registry(), ctx.arena()); - Array values = ctx.registry().decode(valuesCtx); - return new MaskedArray(values, validity); - } else { - throw new VortexException(EncodingId.VORTEX_STRUCT, - "unexpected child count " + numChildren + " for scalar wrapper"); - } - } - } -} diff --git a/core/src/main/java/io/github/dfa1/vortex/encoding/TimeUnit.java b/core/src/main/java/io/github/dfa1/vortex/encoding/TimeUnit.java deleted file mode 100644 index ec5a7e4..0000000 --- a/core/src/main/java/io/github/dfa1/vortex/encoding/TimeUnit.java +++ /dev/null @@ -1,42 +0,0 @@ -package io.github.dfa1.vortex.encoding; - -/// Time unit for timestamp values. Ordinals match Rust's {@code TimeUnit} enum. -public enum TimeUnit { - /// Nanoseconds — ordinal 0. - Nanoseconds, - /// Microseconds — ordinal 1. - Microseconds, - /// Milliseconds — ordinal 2. - Milliseconds, - /// Seconds — ordinal 3. - Seconds, - /// Days — ordinal 4. Not sub-dividable; {@link #divisor()} throws for this value. - Days; - - /// Returns the {@code TimeUnit} for the given wire tag byte. - /// - /// @param tag wire tag byte (unsigned ordinal) - /// @return the corresponding {@code TimeUnit} - public static TimeUnit fromTag(byte tag) { - int i = Byte.toUnsignedInt(tag); - TimeUnit[] values = values(); - if (i >= values.length) { - throw new IllegalArgumentException("unknown TimeUnit tag: " + i); - } - return values[i]; - } - - /// Returns the number of this time unit's divisions in one second (e.g. 1_000_000_000 for nanoseconds). - /// Throws {@link IllegalArgumentException} for {@link #Days}. - /// - /// @return divisor for converting between this unit and seconds - public long divisor() { - return switch (this) { - case Nanoseconds -> 1_000_000_000L; - case Microseconds -> 1_000_000L; - case Milliseconds -> 1_000L; - case Seconds -> 1L; - case Days -> throw new IllegalArgumentException("Days cannot be split"); - }; - } -} diff --git a/core/src/main/java/io/github/dfa1/vortex/encoding/UnknownArrayNode.java b/core/src/main/java/io/github/dfa1/vortex/encoding/UnknownArrayNode.java deleted file mode 100644 index 729911f..0000000 --- a/core/src/main/java/io/github/dfa1/vortex/encoding/UnknownArrayNode.java +++ /dev/null @@ -1,18 +0,0 @@ -package io.github.dfa1.vortex.encoding; - -import io.github.dfa1.vortex.core.ArrayStats; - -import java.nio.ByteBuffer; - -/// Array node whose encoding id is not a recognised [EncodingId]. Produced when a file uses -/// an encoding this build does not know about. Decoded as -/// [io.github.dfa1.vortex.core.array.UnknownArray] when -/// [EncodingRegistry#allowUnknown()] is set; otherwise the decode call throws. -record UnknownArrayNode( - String rawEncodingId, - ByteBuffer metadata, - ArrayNode[] children, - int[] bufferIndices, - ArrayStats stats -) implements ArrayNode { -} diff --git a/core/src/main/java/io/github/dfa1/vortex/encoding/VarBinEncoding.java b/core/src/main/java/io/github/dfa1/vortex/encoding/VarBinEncoding.java deleted file mode 100644 index 02aa4b1..0000000 --- a/core/src/main/java/io/github/dfa1/vortex/encoding/VarBinEncoding.java +++ /dev/null @@ -1,144 +0,0 @@ -package io.github.dfa1.vortex.encoding; - -import com.google.protobuf.InvalidProtocolBufferException; -import io.github.dfa1.vortex.proto.EncodingProtos; -import io.github.dfa1.vortex.proto.ScalarProtos; -import io.github.dfa1.vortex.core.array.Array; -import io.github.dfa1.vortex.core.ArrayStats; -import io.github.dfa1.vortex.core.DType; -import io.github.dfa1.vortex.core.PType; -import io.github.dfa1.vortex.core.array.VarBinArray; -import io.github.dfa1.vortex.core.VortexException; - -import java.lang.foreign.Arena; -import java.lang.foreign.MemorySegment; -import java.nio.ByteBuffer; -import java.nio.charset.StandardCharsets; -import java.util.List; - -/// Decoder for {@code vortex.varbin} — variable-length binary / UTF-8 string arrays. -/// -///

Metadata: protobuf {@code VarBinMetadata} — {@code offsets_ptype PType} (tag 1). -/// -///

Buffer 0: concatenated raw bytes of all strings. -/// Child slot 0: offsets array (length = rowCount + 1, dtype = offsets_ptype). -/// {@code offsets[i]..offsets[i+1]} is the byte range of element {@code i} in the bytes buffer. -/// Child slot 1 (optional): validity array — ignored for non-nullable columns. -/// -///

The returned {@code Array} exposes: -///

    -///
  • {@code buffer(0)} — the raw bytes segment
    • -///
  • {@code child(0)} — the offsets array (I32 or I64 primitives)
    • -///
-public final class VarBinEncoding implements Encoding { - - /// Creates a new {@code VarBinEncoding} instance; use via {@link EncodingRegistry}. - public VarBinEncoding() { - } - - @Override - public EncodingId encodingId() { - return EncodingId.VORTEX_VARBIN; - } - - @Override - public boolean accepts(DType dtype) { - return dtype instanceof DType.Utf8 || dtype instanceof DType.Binary; - } - - @Override - public EncodeResult encode(DType dtype, Object data) { - return Encoder.encode(dtype, data); - } - - @Override - public Array decode(DecodeContext ctx) { - return Decoder.decode(ctx); - } - - private static final class Encoder { - - private static EncodeResult encode(DType dtype, Object data) { - String[] strings = (String[]) data; - int n = strings.length; - - byte[][] byteArrays = new byte[n][]; - int totalBytes = 0; - for (int i = 0; i < n; i++) { - byteArrays[i] = strings[i].getBytes(StandardCharsets.UTF_8); - totalBytes += byteArrays[i].length; - } - - Arena arena = Arena.ofAuto(); - MemorySegment bytesBuf = arena.allocate(totalBytes > 0 ? totalBytes : 1); - MemorySegment offsetsBuf = arena.allocate((long) (n + 1) * Long.BYTES, Long.BYTES); - - long pos = 0; - offsetsBuf.setAtIndex(PTypeIO.LE_LONG, 0, 0L); - for (int i = 0; i < n; i++) { - MemorySegment.copy(MemorySegment.ofArray(byteArrays[i]), 0, bytesBuf, pos, byteArrays[i].length); - pos += byteArrays[i].length; - offsetsBuf.setAtIndex(PTypeIO.LE_LONG, i + 1, pos); - } - - byte[] metaBytes = EncodingProtos.VarBinMetadata.newBuilder() - .setOffsetsPtype(io.github.dfa1.vortex.proto.DTypeProtos.PType.forNumber(PType.I64.ordinal())) - .build() - .toByteArray(); - - String minStr = null; - String maxStr = null; - for (String s : strings) { - if (s == null) { - continue; - } - if (minStr == null || s.compareTo(minStr) < 0) { - minStr = s; - } - if (maxStr == null || s.compareTo(maxStr) > 0) { - maxStr = s; - } - } - byte[] statsMin = minStr != null - ? ScalarProtos.ScalarValue.newBuilder().setStringValue(minStr).build().toByteArray() : null; - byte[] statsMax = maxStr != null - ? ScalarProtos.ScalarValue.newBuilder().setStringValue(maxStr).build().toByteArray() : null; - - EncodeNode offsetsNode = EncodeNode.leaf(EncodingId.VORTEX_PRIMITIVE, 1); - EncodeNode root = new EncodeNode(EncodingId.VORTEX_VARBIN, ByteBuffer.wrap(metaBytes), - new EncodeNode[]{offsetsNode}, new int[]{0}); - return new EncodeResult(root, List.of(bytesBuf, offsetsBuf), statsMin, statsMax); - } - } - - private static final class Decoder { - - private static Array decode(DecodeContext ctx) { - ByteBuffer rawMeta = ctx.metadata(); - if (rawMeta == null) { - throw new VortexException(EncodingId.VORTEX_VARBIN, "missing metadata"); - } - EncodingProtos.VarBinMetadata meta; - try { - meta = EncodingProtos.VarBinMetadata.parseFrom(rawMeta.duplicate()); - } catch (InvalidProtocolBufferException e) { - throw new VortexException(EncodingId.VORTEX_VARBIN, "invalid metadata", e); - } - - PType offsetsPtype = PType.values()[meta.getOffsetsPtype().getNumber()]; - DType offsetsDtype = new DType.Primitive(offsetsPtype, false); - long n = ctx.rowCount(); - - // Offsets: n+1 elements; bytes: raw string data. - ArrayNode offsetsNode = ctx.node().children()[0]; - DecodeContext offsetsCtx = new DecodeContext( - offsetsNode, offsetsDtype, n + 1, - ctx.segmentBuffers(), ctx.registry(), ctx.arena()); - Array offsets = ctx.registry().decode(offsetsCtx); - - MemorySegment bytes = ctx.buffer(0); - - return new VarBinArray(ctx.dtype(), n, bytes, offsets, offsetsPtype, ArrayStats.empty()); - } - } -} diff --git a/core/src/main/java/io/github/dfa1/vortex/encoding/VarBinViewEncoding.java b/core/src/main/java/io/github/dfa1/vortex/encoding/VarBinViewEncoding.java deleted file mode 100644 index 7283032..0000000 --- a/core/src/main/java/io/github/dfa1/vortex/encoding/VarBinViewEncoding.java +++ /dev/null @@ -1,164 +0,0 @@ -package io.github.dfa1.vortex.encoding; - -import io.github.dfa1.vortex.core.ArrayStats; -import io.github.dfa1.vortex.core.DType; -import io.github.dfa1.vortex.core.PType; -import io.github.dfa1.vortex.core.VortexException; -import io.github.dfa1.vortex.core.array.Array; -import io.github.dfa1.vortex.core.array.LongArray; -import io.github.dfa1.vortex.core.array.VarBinArray; - -import java.lang.foreign.Arena; -import java.lang.foreign.MemorySegment; -import java.nio.charset.StandardCharsets; -import java.util.List; - -/// Decoder for {@code vortex.varbinview} — Apache Arrow StringView/BinaryView format. -/// -///

Each element is a 16-byte view (LE): -///

    -///
  • Inlined (size ≤ 12): {@code [size:u32 | data:12bytes]}
    • -///
  • Reference (size > 12): {@code [size:u32 | prefix:4bytes | buffer_index:u32 | offset:u32]}
    • -///
-/// -///

Buffer layout: {@code [data_buf_0, ..., data_buf_k, views_buf]} — views is always last. -/// Metadata: empty. Children: 0 (non-nullable) or 1 (validity bitmap, ignored). -/// -///

Decode materialises all values into a flat {@link VarBinArray} (bytes + I64 offsets). -public final class VarBinViewEncoding implements Encoding { - - /// Creates a new {@code VarBinViewEncoding} instance; use via {@link EncodingRegistry}. - public VarBinViewEncoding() { - } - - @Override - public EncodingId encodingId() { - return EncodingId.VORTEX_VARBINVIEW; - } - - @Override - public boolean accepts(DType dtype) { - return dtype instanceof DType.Utf8 || dtype instanceof DType.Binary; - } - - @Override - public EncodeResult encode(DType dtype, Object data) { - return Encoder.encode((String[]) data); - } - - @Override - public Array decode(DecodeContext ctx) { - return Decoder.decode(ctx); - } - - private static final class Encoder { - - private static final int MAX_INLINED_SIZE = 12; - private static final int VIEW_SIZE = 16; - - static EncodeResult encode(String[] strings) { - int n = strings.length; - - byte[][] bytes = new byte[n][]; - int totalDataBytes = 0; - for (int i = 0; i < n; i++) { - bytes[i] = strings[i].getBytes(StandardCharsets.UTF_8); - if (bytes[i].length > MAX_INLINED_SIZE) { - totalDataBytes += bytes[i].length; - } - } - - Arena arena = Arena.ofAuto(); - boolean hasDataBuf = totalDataBytes > 0; - MemorySegment dataBuf = arena.allocate(hasDataBuf ? totalDataBytes : 1); - MemorySegment viewsBuf = arena.allocate(n > 0 ? (long) n * VIEW_SIZE : 1); - - int dataOffset = 0; - for (int i = 0; i < n; i++) { - byte[] b = bytes[i]; - long viewOff = (long) i * VIEW_SIZE; - viewsBuf.set(PTypeIO.LE_INT, viewOff, b.length); - if (b.length <= MAX_INLINED_SIZE) { - MemorySegment.copy(MemorySegment.ofArray(b), 0, viewsBuf, viewOff + 4, b.length); - } else { - MemorySegment.copy(MemorySegment.ofArray(b), 0, viewsBuf, viewOff + 4, 4); - viewsBuf.set(PTypeIO.LE_INT, viewOff + 8, 0); - viewsBuf.set(PTypeIO.LE_INT, viewOff + 12, dataOffset); - MemorySegment.copy(MemorySegment.ofArray(b), 0, dataBuf, dataOffset, b.length); - dataOffset += b.length; - } - } - - int[] bufIndices; - List buffers; - if (hasDataBuf) { - bufIndices = new int[]{0, 1}; - buffers = List.of(dataBuf, viewsBuf); - } else { - bufIndices = new int[]{0}; - buffers = List.of(viewsBuf); - } - - EncodeNode root = new EncodeNode(EncodingId.VORTEX_VARBINVIEW, null, new EncodeNode[0], bufIndices); - return new EncodeResult(root, buffers, null, null); - } - } - - private static final class Decoder { - - private static final int MAX_INLINED_SIZE = 12; - private static final int VIEW_SIZE = 16; - - private static Array decode(DecodeContext ctx) { - if (!(ctx.dtype() instanceof DType.Utf8 || ctx.dtype() instanceof DType.Binary)) { - throw new VortexException(EncodingId.VORTEX_VARBINVIEW, - "expected Utf8/Binary dtype, got " + ctx.dtype()); - } - - int numBufs = ctx.node().bufferIndices().length; - if (numBufs < 1) { - throw new VortexException(EncodingId.VORTEX_VARBINVIEW, - "expected at least 1 buffer (views), got 0"); - } - - // Views buffer is the last; data buffers are 0..numBufs-2 - MemorySegment viewsBuf = ctx.buffer(numBufs - 1); - MemorySegment[] dataBufs = new MemorySegment[numBufs - 1]; - for (int i = 0; i < dataBufs.length; i++) { - dataBufs[i] = ctx.buffer(i); - } - - long n = ctx.rowCount(); - - long totalBytes = 0; - for (long i = 0; i < n; i++) { - long size = Integer.toUnsignedLong(viewsBuf.get(PTypeIO.LE_INT, i * VIEW_SIZE)); - totalBytes += size; - } - - MemorySegment outBytes = ctx.arena().allocate(totalBytes > 0 ? totalBytes : 1); - MemorySegment outOffsets = ctx.arena().allocate((n + 1) * Long.BYTES, Long.BYTES); - - long bytePos = 0; - outOffsets.setAtIndex(PTypeIO.LE_LONG, 0, 0L); - for (long i = 0; i < n; i++) { - long viewOff = i * VIEW_SIZE; - long size = Integer.toUnsignedLong(viewsBuf.get(PTypeIO.LE_INT, viewOff)); - if (size <= MAX_INLINED_SIZE) { - MemorySegment.copy(viewsBuf, viewOff + 4, outBytes, bytePos, size); - } else { - int bufferIndex = viewsBuf.get(PTypeIO.LE_INT, viewOff + 8); - long srcOffset = Integer.toUnsignedLong(viewsBuf.get(PTypeIO.LE_INT, viewOff + 12)); - MemorySegment.copy(dataBufs[bufferIndex], srcOffset, outBytes, bytePos, size); - } - bytePos += size; - outOffsets.setAtIndex(PTypeIO.LE_LONG, i + 1, bytePos); - } - - Array offsetsArr = new LongArray(new DType.Primitive(PType.I64, false), n + 1, - outOffsets, ArrayStats.empty()); - return new VarBinArray(ctx.dtype(), n, outBytes.asReadOnly(), offsetsArr, PType.I64, - ArrayStats.empty()); - } - } -} diff --git a/core/src/main/java/io/github/dfa1/vortex/encoding/VariantEncoding.java b/core/src/main/java/io/github/dfa1/vortex/encoding/VariantEncoding.java deleted file mode 100644 index 5f7ae3a..0000000 --- a/core/src/main/java/io/github/dfa1/vortex/encoding/VariantEncoding.java +++ /dev/null @@ -1,28 +0,0 @@ -package io.github.dfa1.vortex.encoding; - -import io.github.dfa1.vortex.core.DType; -import io.github.dfa1.vortex.core.VortexException; -import io.github.dfa1.vortex.core.array.Array; - -/// Stub for {@code vortex.variant} — not yet implemented. -public final class VariantEncoding implements Encoding { - - /// Creates a new {@code VariantEncoding} instance; use via {@link EncodingRegistry}. - public VariantEncoding() { - } - - @Override - public EncodingId encodingId() { - return EncodingId.VORTEX_VARIANT; - } - - @Override - public EncodeResult encode(DType dtype, Object data) { - throw new VortexException(EncodingId.VORTEX_VARIANT, "not yet implemented"); - } - - @Override - public Array decode(DecodeContext ctx) { - throw new VortexException(EncodingId.VORTEX_VARIANT, "not yet implemented"); - } -} diff --git a/core/src/main/java/io/github/dfa1/vortex/encoding/ZigZagEncoding.java b/core/src/main/java/io/github/dfa1/vortex/encoding/ZigZagEncoding.java deleted file mode 100644 index 51ed559..0000000 --- a/core/src/main/java/io/github/dfa1/vortex/encoding/ZigZagEncoding.java +++ /dev/null @@ -1,166 +0,0 @@ -package io.github.dfa1.vortex.encoding; - -import io.github.dfa1.vortex.core.array.Array; -import io.github.dfa1.vortex.core.ArrayStats; -import io.github.dfa1.vortex.core.DType; -import io.github.dfa1.vortex.core.PType; -import io.github.dfa1.vortex.core.VortexException; -import io.github.dfa1.vortex.core.array.ByteArray; -import io.github.dfa1.vortex.core.array.IntArray; -import io.github.dfa1.vortex.core.array.LongArray; -import io.github.dfa1.vortex.core.array.ShortArray; - -import java.lang.foreign.Arena; -import java.lang.foreign.MemorySegment; -import java.lang.foreign.ValueLayout; -import java.util.List; - -/// Encoding for {@code vortex.zigzag} — signed integers stored as zigzag-encoded unsigned values. -/// -///

No buffers, empty metadata. -/// Child slot 0: the encoded unsigned integer array (U8/U16/U32/U64, same length as parent). -/// Parent dtype is the signed counterpart (I8/I16/I32/I64). -/// -///

Decode: {@code signed = (unsigned >>> 1) ^ -(unsigned & 1)} applied element-wise. -public final class ZigZagEncoding implements Encoding { - - /// Creates a new {@code ZigZagEncoding} instance; use via {@link EncodingRegistry}. - public ZigZagEncoding() { - } - - @Override - public EncodingId encodingId() { - return EncodingId.VORTEX_ZIGZAG; - } - - @Override - public boolean accepts(DType dtype) { - if (!(dtype instanceof DType.Primitive p)) { - return false; - } - PType pt = p.ptype(); - return pt == PType.I8 || pt == PType.I16 || pt == PType.I32 || pt == PType.I64; - } - - @Override - public EncodeResult encode(DType dtype, Object data) { - return Encoder.encode(dtype, data); - } - - @Override - public Array decode(DecodeContext ctx) { - return Decoder.decode(ctx); - } - - private static final class Encoder { - - private static EncodeResult encode(DType dtype, Object data) { - PType signed = ((DType.Primitive) dtype).ptype(); - MemorySegment seg = switch (signed) { - case I8 -> { - byte[] arr = (byte[]) data; - MemorySegment s = Arena.ofAuto().allocate(arr.length); - for (int i = 0; i < arr.length; i++) { - byte v = arr[i]; - s.set(ValueLayout.JAVA_BYTE, i, (byte) ((v << 1) ^ (v >> 7))); - } - yield s; - } - case I16 -> { - short[] arr = (short[]) data; - MemorySegment s = Arena.ofAuto().allocate((long) arr.length * 2, 2); - for (int i = 0; i < arr.length; i++) { - short v = arr[i]; - s.setAtIndex(PTypeIO.LE_SHORT, i, (short) ((v << 1) ^ (v >> 15))); - } - yield s; - } - case I32 -> { - int[] arr = (int[]) data; - MemorySegment s = Arena.ofAuto().allocate((long) arr.length * 4, 4); - for (int i = 0; i < arr.length; i++) { - int v = arr[i]; - s.setAtIndex(PTypeIO.LE_INT, i, (v << 1) ^ (v >> 31)); - } - yield s; - } - case I64 -> { - long[] arr = (long[]) data; - MemorySegment s = Arena.ofAuto().allocate((long) arr.length * 8, 8); - for (int i = 0; i < arr.length; i++) { - long v = arr[i]; - s.setAtIndex(PTypeIO.LE_LONG, i, (v << 1) ^ (v >> 63)); - } - yield s; - } - default -> throw new VortexException(EncodingId.VORTEX_ZIGZAG, "unsupported ptype: " + signed); - }; - EncodeNode child = EncodeNode.leaf(EncodingId.VORTEX_PRIMITIVE, 0); - EncodeNode root = new EncodeNode(EncodingId.VORTEX_ZIGZAG, null, new EncodeNode[]{child}, new int[0]); - return new EncodeResult(root, List.of(seg), null, null); - } - } - - private static final class Decoder { - - private static Array decode(DecodeContext ctx) { - if (!(ctx.dtype() instanceof DType.Primitive p)) { - throw new VortexException(EncodingId.VORTEX_ZIGZAG, "expected primitive dtype, got " + ctx.dtype()); - } - PType signed = p.ptype(); - PType unsigned = toUnsigned(signed); - long n = ctx.rowCount(); - - ArrayNode childNode = ctx.node().children()[0]; - DecodeContext childCtx = new DecodeContext( - childNode, new DType.Primitive(unsigned, false), n, - ctx.segmentBuffers(), ctx.registry(), ctx.arena()); - Array encoded = ctx.registry().decode(childCtx); - - MemorySegment src = encoded.buffer(0); - MemorySegment dst = ctx.arena().allocate(n * signed.byteSize()); - - return switch (signed) { - case I8 -> { - for (long i = 0; i < n; i++) { - int u = Byte.toUnsignedInt(src.get(ValueLayout.JAVA_BYTE, i)); - dst.set(ValueLayout.JAVA_BYTE, i, (byte) ((u >>> 1) ^ -(u & 1))); - } - yield new ByteArray(ctx.dtype(), n, dst, ArrayStats.empty()); - } - case I16 -> { - for (long i = 0; i < n; i++) { - int u = Short.toUnsignedInt(src.get(PTypeIO.LE_SHORT, i * 2)); - dst.set(PTypeIO.LE_SHORT, i * 2, (short) ((u >>> 1) ^ -(u & 1))); - } - yield new ShortArray(ctx.dtype(), n, dst, ArrayStats.empty()); - } - case I32 -> { - for (long i = 0; i < n; i++) { - int u = src.get(PTypeIO.LE_INT, i * 4); - dst.set(PTypeIO.LE_INT, i * 4, (u >>> 1) ^ -(u & 1)); - } - yield new IntArray(ctx.dtype(), n, dst, ArrayStats.empty()); - } - case I64 -> { - for (long i = 0; i < n; i++) { - long u = src.get(PTypeIO.LE_LONG, i * 8); - dst.set(PTypeIO.LE_LONG, i * 8, (u >>> 1) ^ -(u & 1)); - } - yield new LongArray(ctx.dtype(), n, dst, ArrayStats.empty()); - } - default -> throw new VortexException(EncodingId.VORTEX_ZIGZAG, "unreachable"); - }; - } - - private static PType toUnsigned(PType signed) { - return switch (signed) { - case I8 -> PType.U8; - case I16 -> PType.U16; - case I32 -> PType.U32; - case I64 -> PType.U64; - default -> throw new VortexException(EncodingId.VORTEX_ZIGZAG, "not a signed integer: " + signed); - }; - } - } -} diff --git a/core/src/main/java/io/github/dfa1/vortex/encoding/ZstdEncoding.java b/core/src/main/java/io/github/dfa1/vortex/encoding/ZstdEncoding.java deleted file mode 100644 index 29947fc..0000000 --- a/core/src/main/java/io/github/dfa1/vortex/encoding/ZstdEncoding.java +++ /dev/null @@ -1,424 +0,0 @@ -package io.github.dfa1.vortex.encoding; - -import com.google.protobuf.InvalidProtocolBufferException; -import com.github.luben.zstd.ZstdDecompressCtx; -import io.airlift.compress.v3.zstd.ZstdCompressor; -import io.airlift.compress.v3.zstd.ZstdDecompressor; -import io.airlift.compress.v3.zstd.ZstdJavaCompressor; -import io.airlift.compress.v3.zstd.ZstdJavaDecompressor; -import io.github.dfa1.vortex.core.ArrayStats; -import io.github.dfa1.vortex.core.DType; -import io.github.dfa1.vortex.core.PType; -import io.github.dfa1.vortex.core.VortexException; -import io.github.dfa1.vortex.core.array.Array; -import io.github.dfa1.vortex.core.array.BoolArray; -import io.github.dfa1.vortex.core.array.ByteArray; -import io.github.dfa1.vortex.core.array.DoubleArray; -import io.github.dfa1.vortex.core.array.Float16Array; -import io.github.dfa1.vortex.core.array.FloatArray; -import io.github.dfa1.vortex.core.array.IntArray; -import io.github.dfa1.vortex.core.array.LongArray; -import io.github.dfa1.vortex.core.array.MaskedArray; -import io.github.dfa1.vortex.core.array.ShortArray; -import io.github.dfa1.vortex.core.array.VarBinArray; -import io.github.dfa1.vortex.proto.EncodingProtos; - -import java.lang.foreign.Arena; -import java.lang.foreign.MemorySegment; -import java.lang.foreign.ValueLayout; -import java.nio.ByteBuffer; -import java.nio.charset.StandardCharsets; -import java.util.Arrays; -import java.util.List; - -/// Encoder/decoder for {@code vortex.zstd} — Zstandard-compressed columnar array. -/// -///

When to use in the cascade: good for low-entropy strings and binary columns. -/// Do NOT add to the numeric cascade alongside ALP/bitpack — on the NYC Taxi dataset, -/// Zstd wins compression for F64 columns (50 MB → 43 MB) but decode throughput collapses -/// 6× (240 → 40 ops/s single-column), because ZSTD decompression is much slower than -/// ALP reconstruction or bitpack unpack. Use it only for {@link io.github.dfa1.vortex.core.DType.Utf8} -/// and {@link io.github.dfa1.vortex.core.DType.Binary} where there is no faster structural alternative. -/// -///

Wire format: -///

    -///
  • Metadata: {@code ZstdMetadata} — {@code dictionary_size} (0 = no dict) + repeated {@code ZstdFrameMetadata}
    • -///
  • Buffers: one compressed frame per metadata entry (no dictionary buffer when {@code dictionary_size == 0})
    • -///
  • Child 0: validity bitmap (optional; this decoder rejects nullable arrays)
    • -///
-/// -///

Primitive dtype: raw LE values compressed into a single frame. -///

Utf8/Binary dtype: {@code [u32-LE length][data]} per string, compressed into a single frame. -/// -///

Dictionary support: when {@code dictionary_size > 0} the first buffer is the dictionary and -/// frames start at buffer index 1. Decompression uses {@link ZstdDecompressCtx} from {@code zstd-jni}. -/// Nullable arrays (validity child present) are decoded by scattering valid values back into a -/// full-length array wrapped in {@link MaskedArray}. -public final class ZstdEncoding implements Encoding { - - /// Creates a new {@code ZstdEncoding} instance; use via {@link EncodingRegistry}. - public ZstdEncoding() { - } - - @Override - public EncodingId encodingId() { - return EncodingId.VORTEX_ZSTD; - } - - @Override - public boolean accepts(DType dtype) { - return dtype instanceof DType.Primitive || dtype instanceof DType.Utf8 || dtype instanceof DType.Binary; - } - - @Override - public EncodeResult encode(DType dtype, Object data) { - return Encoder.encode(dtype, data); - } - - @Override - public Array decode(DecodeContext ctx) { - return Decoder.decode(ctx); - } - - private static final class Encoder { - - private static EncodeResult encode(DType dtype, Object data) { - if (dtype instanceof DType.Primitive dt) { - return encodePrimitive(dt, data); - } - if (dtype instanceof DType.Utf8 || dtype instanceof DType.Binary) { - return encodeVarBin(dtype, (String[]) data); - } - throw new VortexException(EncodingId.VORTEX_ZSTD, "unsupported dtype: " + dtype); - } - - private static EncodeResult encodePrimitive(DType.Primitive dt, Object data) { - MemorySegment raw = primitiveToLeBytes(dt.ptype(), data); - long n = primitiveLength(dt.ptype(), data); - byte[] rawBytes = raw.toArray(ValueLayout.JAVA_BYTE); - return buildResult(rawBytes, n); - } - - private static EncodeResult encodeVarBin(DType dtype, String[] strings) { - byte[] raw = buildLengthPrefixed(strings); - return buildResult(raw, strings.length); - } - - private static EncodeResult buildResult(byte[] raw, long n) { - byte[] compressed = compress(raw); - byte[] meta = EncodingProtos.ZstdMetadata.newBuilder() - .setDictionarySize(0) - .addFrames(EncodingProtos.ZstdFrameMetadata.newBuilder() - .setUncompressedSize(raw.length) - .setNValues(n)) - .build().toByteArray(); - EncodeNode root = new EncodeNode(EncodingId.VORTEX_ZSTD, ByteBuffer.wrap(meta), - new EncodeNode[0], new int[]{0}); - return new EncodeResult(root, List.of(MemorySegment.ofArray(compressed)), null, null); - } - - private static byte[] compress(byte[] input) { - ZstdCompressor compressor = new ZstdJavaCompressor(); - byte[] out = new byte[compressor.maxCompressedLength(input.length)]; - int len = compressor.compress(input, 0, input.length, out, 0, out.length); - return Arrays.copyOf(out, len); - } - - private static MemorySegment primitiveToLeBytes(PType ptype, Object data) { - return switch (ptype) { - case I8, U8 -> MemorySegment.ofArray((byte[]) data); - case I16, U16, F16 -> { - short[] arr = (short[]) data; - MemorySegment seg = Arena.ofAuto().allocate((long) arr.length * 2, 2); - for (int i = 0; i < arr.length; i++) { - seg.setAtIndex(PTypeIO.LE_SHORT, i, arr[i]); - } - yield seg; - } - case I32, U32 -> { - int[] arr = (int[]) data; - MemorySegment seg = Arena.ofAuto().allocate((long) arr.length * 4, 4); - for (int i = 0; i < arr.length; i++) { - seg.setAtIndex(PTypeIO.LE_INT, i, arr[i]); - } - yield seg; - } - case I64, U64 -> { - long[] arr = (long[]) data; - MemorySegment seg = Arena.ofAuto().allocate((long) arr.length * 8, 8); - for (int i = 0; i < arr.length; i++) { - seg.setAtIndex(PTypeIO.LE_LONG, i, arr[i]); - } - yield seg; - } - case F32 -> { - float[] arr = (float[]) data; - MemorySegment seg = Arena.ofAuto().allocate((long) arr.length * 4, 4); - for (int i = 0; i < arr.length; i++) { - seg.setAtIndex(PTypeIO.LE_FLOAT, i, arr[i]); - } - yield seg; - } - case F64 -> { - double[] arr = (double[]) data; - MemorySegment seg = Arena.ofAuto().allocate((long) arr.length * 8, 8); - for (int i = 0; i < arr.length; i++) { - seg.setAtIndex(PTypeIO.LE_DOUBLE, i, arr[i]); - } - yield seg; - } - }; - } - - private static long primitiveLength(PType ptype, Object data) { - return switch (ptype) { - case I8, U8 -> ((byte[]) data).length; - case I16, U16, F16 -> ((short[]) data).length; - case I32, U32 -> ((int[]) data).length; - case F32 -> ((float[]) data).length; - case I64, U64 -> ((long[]) data).length; - case F64 -> ((double[]) data).length; - }; - } - - private static byte[] buildLengthPrefixed(String[] strings) { - int total = 0; - byte[][] encoded = new byte[strings.length][]; - for (int i = 0; i < strings.length; i++) { - encoded[i] = strings[i].getBytes(StandardCharsets.UTF_8); - total += 4 + encoded[i].length; - } - MemorySegment seg = Arena.ofAuto().allocate(total > 0 ? total : 1); - long pos = 0; - for (byte[] bytes : encoded) { - seg.set(PTypeIO.LE_INT, pos, bytes.length); - pos += 4; - MemorySegment.copy(MemorySegment.ofArray(bytes), 0, seg, pos, bytes.length); - pos += bytes.length; - } - return seg.asSlice(0, total).toArray(ValueLayout.JAVA_BYTE); - } - } - - private static final class Decoder { - - private static Array decode(DecodeContext ctx) { - ByteBuffer rawMeta = ctx.metadata(); - if (rawMeta == null) { - throw new VortexException(EncodingId.VORTEX_ZSTD, "missing metadata"); - } - EncodingProtos.ZstdMetadata meta; - try { - meta = EncodingProtos.ZstdMetadata.parseFrom(rawMeta.duplicate()); - } catch (InvalidProtocolBufferException e) { - throw new VortexException(EncodingId.VORTEX_ZSTD, "invalid metadata", e); - } - boolean hasDictionary = meta.getDictionarySize() != 0; - - BoolArray validity = null; - if (ctx.node().children().length > 0) { - ArrayNode childNode = ctx.node().children()[0]; - DecodeContext childCtx = new DecodeContext( - childNode, new DType.Bool(false), ctx.rowCount(), - ctx.segmentBuffers(), ctx.registry(), ctx.arena()); - Array validityArray = ctx.registry().decode(childCtx); - if (!(validityArray instanceof BoolArray ba)) { - throw new VortexException(EncodingId.VORTEX_ZSTD, - "validity child decoded to unexpected type: " + validityArray.getClass().getSimpleName()); - } - validity = ba; - } - - int frameCount = meta.getFramesCount(); - long totalUncompressed = 0; - for (int i = 0; i < frameCount; i++) { - totalUncompressed += meta.getFrames(i).getUncompressedSize(); - } - - MemorySegment decompressed = hasDictionary - ? decompressFramesWithDict(ctx, meta, frameCount, totalUncompressed) - : decompressFrames(ctx, meta, frameCount, totalUncompressed); - - if (validity == null) { - return buildArray(ctx.dtype(), ctx.rowCount(), decompressed, ctx); - } else { - return buildNullableArray(ctx.dtype(), ctx.rowCount(), decompressed, validity, ctx); - } - } - - private static Array buildNullableArray( - DType dtype, long rowCount, MemorySegment validValues, BoolArray validity, DecodeContext ctx - ) { - Array child; - if (dtype instanceof DType.Primitive dt) { - child = buildScatteredPrimitive(dt, rowCount, validValues, validity, ctx); - } else if (dtype instanceof DType.Utf8 || dtype instanceof DType.Binary) { - child = buildScatteredVarBin(dtype, rowCount, validValues, validity, ctx); - } else { - throw new VortexException(EncodingId.VORTEX_ZSTD, "unsupported nullable dtype: " + dtype); - } - return new MaskedArray(child, validity); - } - - private static Array buildScatteredPrimitive( - DType.Primitive dt, long rowCount, MemorySegment validValues, BoolArray validity, DecodeContext ctx - ) { - int byteSize = dt.ptype().byteSize(); - MemorySegment out = ctx.arena().allocate(rowCount * byteSize); - long readPos = 0; - for (long i = 0; i < rowCount; i++) { - if (validity.getBoolean(i)) { - MemorySegment.copy(validValues, readPos, out, i * byteSize, byteSize); - readPos += byteSize; - } - } - DType.Primitive nonNull = new DType.Primitive(dt.ptype(), false); - return buildPrimitive(nonNull, rowCount, out); - } - - private static VarBinArray buildScatteredVarBin( - DType dtype, long rowCount, MemorySegment validValues, BoolArray validity, DecodeContext ctx - ) { - // First pass: total data bytes across valid positions only - long totalDataBytes = 0; - long scanPos = 0; - for (long i = 0; i < rowCount; i++) { - if (validity.getBoolean(i)) { - int len = validValues.get(PTypeIO.LE_INT, scanPos); - scanPos += 4L + len; - totalDataBytes += len; - } - } - - MemorySegment values = ctx.arena().allocate(totalDataBytes > 0 ? totalDataBytes : 1); - MemorySegment offsets = ctx.arena().allocate((rowCount + 1) * 4L, 4); - offsets.setAtIndex(PTypeIO.LE_INT, 0, 0); - - long readPos = 0; - long dataPos = 0; - for (long i = 0; i < rowCount; i++) { - if (validity.getBoolean(i)) { - int len = validValues.get(PTypeIO.LE_INT, readPos); - readPos += 4; - MemorySegment.copy(validValues, readPos, values, dataPos, len); - readPos += len; - dataPos += len; - } - offsets.setAtIndex(PTypeIO.LE_INT, i + 1, (int) dataPos); - } - - DType i32 = new DType.Primitive(PType.I32, false); - IntArray offsetsArr = new IntArray(i32, rowCount + 1, offsets, ArrayStats.empty()); - return new VarBinArray(dtype.withNullable(false), rowCount, values, offsetsArr, PType.I32, ArrayStats.empty()); - } - - private static MemorySegment decompressFramesWithDict( - DecodeContext ctx, - EncodingProtos.ZstdMetadata meta, - int frameCount, - long totalUncompressed - ) { - MemorySegment out = ctx.arena().allocate(totalUncompressed); - byte[] dictBytes = ctx.buffer(0).toArray(ValueLayout.JAVA_BYTE); - try (ZstdDecompressCtx zctx = new ZstdDecompressCtx()) { - zctx.loadDict(dictBytes); - long outOffset = 0; - for (int i = 0; i < frameCount; i++) { - byte[] compressed = ctx.buffer(i + 1).toArray(ValueLayout.JAVA_BYTE); - int uncompSize = (int) meta.getFrames(i).getUncompressedSize(); - byte[] temp = new byte[uncompSize]; - int written = zctx.decompressByteArray(temp, 0, uncompSize, compressed, 0, compressed.length); - if (written != uncompSize) { - throw new VortexException(EncodingId.VORTEX_ZSTD, - "frame " + i + ": expected " + uncompSize + " bytes, got " + written); - } - MemorySegment.copy(MemorySegment.ofArray(temp), 0, out, outOffset, uncompSize); - outOffset += uncompSize; - } - } catch (VortexException e) { - throw e; - } catch (Exception e) { - throw new VortexException(EncodingId.VORTEX_ZSTD, "dict decompression failed", e); - } - return out; - } - - private static MemorySegment decompressFrames( - DecodeContext ctx, - EncodingProtos.ZstdMetadata meta, - int frameCount, - long totalUncompressed - ) { - MemorySegment out = ctx.arena().allocate(totalUncompressed); - ZstdDecompressor decompressor = new ZstdJavaDecompressor(); - long outOffset = 0; - for (int i = 0; i < frameCount; i++) { - MemorySegment frameSeg = ctx.buffer(i); - byte[] compressed = frameSeg.toArray(ValueLayout.JAVA_BYTE); - int uncompSize = (int) meta.getFrames(i).getUncompressedSize(); - byte[] temp = new byte[uncompSize]; - int written = decompressor.decompress(compressed, 0, compressed.length, temp, 0, uncompSize); - if (written != uncompSize) { - throw new VortexException(EncodingId.VORTEX_ZSTD, - "frame " + i + ": expected " + uncompSize + " bytes, got " + written); - } - MemorySegment.copy(MemorySegment.ofArray(temp), 0, out, outOffset, uncompSize); - outOffset += uncompSize; - } - return out; - } - - private static Array buildArray(DType dtype, long n, MemorySegment decompressed, DecodeContext ctx) { - if (dtype instanceof DType.Primitive dt) { - return buildPrimitive(dt, n, decompressed); - } - if (dtype instanceof DType.Utf8 || dtype instanceof DType.Binary) { - return buildVarBin(dtype, n, decompressed, ctx); - } - throw new VortexException(EncodingId.VORTEX_ZSTD, "unsupported dtype: " + dtype); - } - - private static Array buildPrimitive(DType.Primitive dt, long n, MemorySegment decompressed) { - PType ptype = dt.ptype(); - return switch (ptype) { - case I64, U64 -> new LongArray(dt, n, decompressed, ArrayStats.empty()); - case I32, U32 -> new IntArray(dt, n, decompressed, ArrayStats.empty()); - case F64 -> new DoubleArray(dt, n, decompressed, ArrayStats.empty()); - case F32 -> new FloatArray(dt, n, decompressed, ArrayStats.empty()); - case I16, U16 -> new ShortArray(dt, n, decompressed, ArrayStats.empty()); - case I8, U8 -> new ByteArray(dt, n, decompressed, ArrayStats.empty()); - case F16 -> new Float16Array(dt, n, decompressed, ArrayStats.empty()); - }; - } - - private static VarBinArray buildVarBin(DType dtype, long n, MemorySegment decompressed, DecodeContext ctx) { - // scan [u32-LE length][data] pairs to compute total data bytes - long totalDataBytes = 0; - long pos = 0; - for (long i = 0; i < n; i++) { - int len = decompressed.get(PTypeIO.LE_INT, pos); - pos += 4 + len; - totalDataBytes += len; - } - - MemorySegment values = ctx.arena().allocate(totalDataBytes); - MemorySegment offsets = ctx.arena().allocate((n + 1) * 4L, 4); - offsets.setAtIndex(PTypeIO.LE_INT, 0, 0); - - pos = 0; - long dataPos = 0; - for (long i = 0; i < n; i++) { - int len = decompressed.get(PTypeIO.LE_INT, pos); - pos += 4; - MemorySegment.copy(decompressed, pos, values, dataPos, len); - pos += len; - dataPos += len; - offsets.setAtIndex(PTypeIO.LE_INT, i + 1, (int) dataPos); - } - - DType i32 = new DType.Primitive(PType.I32, false); - IntArray offsetsArr = new IntArray(i32, n + 1, offsets, ArrayStats.empty()); - return new VarBinArray(dtype, n, values, offsetsArr, PType.I32, ArrayStats.empty()); - } - } -} diff --git a/core/src/main/java/io/github/dfa1/vortex/fbs/Array.java b/core/src/main/java/io/github/dfa1/vortex/fbs/Array.java deleted file mode 100644 index b250f91..0000000 --- a/core/src/main/java/io/github/dfa1/vortex/fbs/Array.java +++ /dev/null @@ -1,69 +0,0 @@ -// automatically generated by the FlatBuffers compiler, do not modify - -package io.github.dfa1.vortex.fbs; - -import com.google.flatbuffers.BaseVector; -import com.google.flatbuffers.BooleanVector; -import com.google.flatbuffers.ByteVector; -import com.google.flatbuffers.Constants; -import com.google.flatbuffers.DoubleVector; -import com.google.flatbuffers.FlatBufferBuilder; -import com.google.flatbuffers.FloatVector; -import com.google.flatbuffers.IntVector; -import com.google.flatbuffers.LongVector; -import com.google.flatbuffers.ShortVector; -import com.google.flatbuffers.StringVector; -import com.google.flatbuffers.Struct; -import com.google.flatbuffers.UnionVector; -import java.nio.ByteBuffer; -import java.nio.ByteOrder; - -@SuppressWarnings("unused") -public final class Array extends com.google.flatbuffers.Table { - public static Array getRootAsArray(ByteBuffer _bb) { return getRootAsArray(_bb, new Array()); } - public static Array getRootAsArray(ByteBuffer _bb, Array obj) { _bb.order(ByteOrder.LITTLE_ENDIAN); return (obj.__assign(_bb.getInt(_bb.position()) + _bb.position(), _bb)); } - public void __init(int _i, ByteBuffer _bb) { __reset(_i, _bb); } - public Array __assign(int _i, ByteBuffer _bb) { __init(_i, _bb); return this; } - - /** - * The array's hierarchical definition. - */ - public io.github.dfa1.vortex.fbs.ArrayNode root() { return root(new io.github.dfa1.vortex.fbs.ArrayNode()); } - public io.github.dfa1.vortex.fbs.ArrayNode root(io.github.dfa1.vortex.fbs.ArrayNode obj) { int o = __offset(4); return o != 0 ? obj.__assign(__indirect(o + bb_pos), bb) : null; } - /** - * The locations of the data buffers of the array - */ - public io.github.dfa1.vortex.fbs.Buffer buffers(int j) { return buffers(new io.github.dfa1.vortex.fbs.Buffer(), j); } - public io.github.dfa1.vortex.fbs.Buffer buffers(io.github.dfa1.vortex.fbs.Buffer obj, int j) { int o = __offset(6); return o != 0 ? obj.__assign(__vector(o) + j * 8, bb) : null; } - public int buffersLength() { int o = __offset(6); return o != 0 ? __vector_len(o) : 0; } - public io.github.dfa1.vortex.fbs.Buffer.Vector buffersVector() { return buffersVector(new io.github.dfa1.vortex.fbs.Buffer.Vector()); } - public io.github.dfa1.vortex.fbs.Buffer.Vector buffersVector(io.github.dfa1.vortex.fbs.Buffer.Vector obj) { int o = __offset(6); return o != 0 ? obj.__assign(__vector(o), 8, bb) : null; } - - public static int createArray(FlatBufferBuilder builder, - int rootOffset, - int buffersOffset) { - builder.startTable(2); - Array.addBuffers(builder, buffersOffset); - Array.addRoot(builder, rootOffset); - return Array.endArray(builder); - } - - public static void startArray(FlatBufferBuilder builder) { builder.startTable(2); } - public static void addRoot(FlatBufferBuilder builder, int rootOffset) { builder.addOffset(0, rootOffset, 0); } - public static void addBuffers(FlatBufferBuilder builder, int buffersOffset) { builder.addOffset(1, buffersOffset, 0); } - public static void startBuffersVector(FlatBufferBuilder builder, int numElems) { builder.startVector(8, numElems, 4); } - public static int endArray(FlatBufferBuilder builder) { - int o = builder.endTable(); - return o; - } - public static void finishArrayBuffer(FlatBufferBuilder builder, int offset) { builder.finish(offset); } - public static void finishSizePrefixedArrayBuffer(FlatBufferBuilder builder, int offset) { builder.finishSizePrefixed(offset); } - - public static final class Vector extends BaseVector { - public Vector __assign(int _vector, int _element_size, ByteBuffer _bb) { __reset(_vector, _element_size, _bb); return this; } - - public Array get(int j) { return get(new Array(), j); } - public Array get(Array obj, int j) { return obj.__assign(__indirect(__element(j), bb), bb); } - } -} - diff --git a/core/src/main/java/io/github/dfa1/vortex/fbs/ArrayNode.java b/core/src/main/java/io/github/dfa1/vortex/fbs/ArrayNode.java deleted file mode 100644 index 9061987..0000000 --- a/core/src/main/java/io/github/dfa1/vortex/fbs/ArrayNode.java +++ /dev/null @@ -1,89 +0,0 @@ -// automatically generated by the FlatBuffers compiler, do not modify - -package io.github.dfa1.vortex.fbs; - -import com.google.flatbuffers.BaseVector; -import com.google.flatbuffers.BooleanVector; -import com.google.flatbuffers.ByteVector; -import com.google.flatbuffers.Constants; -import com.google.flatbuffers.DoubleVector; -import com.google.flatbuffers.FlatBufferBuilder; -import com.google.flatbuffers.FloatVector; -import com.google.flatbuffers.IntVector; -import com.google.flatbuffers.LongVector; -import com.google.flatbuffers.ShortVector; -import com.google.flatbuffers.StringVector; -import com.google.flatbuffers.Struct; -import com.google.flatbuffers.UnionVector; -import java.nio.ByteBuffer; -import java.nio.ByteOrder; - -@SuppressWarnings("unused") -public final class ArrayNode extends com.google.flatbuffers.Table { - public static ArrayNode getRootAsArrayNode(ByteBuffer _bb) { return getRootAsArrayNode(_bb, new ArrayNode()); } - public static ArrayNode getRootAsArrayNode(ByteBuffer _bb, ArrayNode obj) { _bb.order(ByteOrder.LITTLE_ENDIAN); return (obj.__assign(_bb.getInt(_bb.position()) + _bb.position(), _bb)); } - public void __init(int _i, ByteBuffer _bb) { __reset(_i, _bb); } - public ArrayNode __assign(int _i, ByteBuffer _bb) { __init(_i, _bb); return this; } - - public int encoding() { int o = __offset(4); return o != 0 ? bb.getShort(o + bb_pos) & 0xFFFF : 0; } - public int metadata(int j) { int o = __offset(6); return o != 0 ? bb.get(__vector(o) + j * 1) & 0xFF : 0; } - public int metadataLength() { int o = __offset(6); return o != 0 ? __vector_len(o) : 0; } - public ByteVector metadataVector() { return metadataVector(new ByteVector()); } - public ByteVector metadataVector(ByteVector obj) { int o = __offset(6); return o != 0 ? obj.__assign(__vector(o), bb) : null; } - public ByteBuffer metadataAsByteBuffer() { return __vector_as_bytebuffer(6, 1); } - public ByteBuffer metadataInByteBuffer(ByteBuffer _bb) { return __vector_in_bytebuffer(_bb, 6, 1); } - public io.github.dfa1.vortex.fbs.ArrayNode children(int j) { return children(new io.github.dfa1.vortex.fbs.ArrayNode(), j); } - public io.github.dfa1.vortex.fbs.ArrayNode children(io.github.dfa1.vortex.fbs.ArrayNode obj, int j) { int o = __offset(8); return o != 0 ? obj.__assign(__indirect(__vector(o) + j * 4), bb) : null; } - public int childrenLength() { int o = __offset(8); return o != 0 ? __vector_len(o) : 0; } - public io.github.dfa1.vortex.fbs.ArrayNode.Vector childrenVector() { return childrenVector(new io.github.dfa1.vortex.fbs.ArrayNode.Vector()); } - public io.github.dfa1.vortex.fbs.ArrayNode.Vector childrenVector(io.github.dfa1.vortex.fbs.ArrayNode.Vector obj) { int o = __offset(8); return o != 0 ? obj.__assign(__vector(o), 4, bb) : null; } - public int buffers(int j) { int o = __offset(10); return o != 0 ? bb.getShort(__vector(o) + j * 2) & 0xFFFF : 0; } - public int buffersLength() { int o = __offset(10); return o != 0 ? __vector_len(o) : 0; } - public ShortVector buffersVector() { return buffersVector(new ShortVector()); } - public ShortVector buffersVector(ShortVector obj) { int o = __offset(10); return o != 0 ? obj.__assign(__vector(o), bb) : null; } - public ByteBuffer buffersAsByteBuffer() { return __vector_as_bytebuffer(10, 2); } - public ByteBuffer buffersInByteBuffer(ByteBuffer _bb) { return __vector_in_bytebuffer(_bb, 10, 2); } - public io.github.dfa1.vortex.fbs.ArrayStats stats() { return stats(new io.github.dfa1.vortex.fbs.ArrayStats()); } - public io.github.dfa1.vortex.fbs.ArrayStats stats(io.github.dfa1.vortex.fbs.ArrayStats obj) { int o = __offset(12); return o != 0 ? obj.__assign(__indirect(o + bb_pos), bb) : null; } - - public static int createArrayNode(FlatBufferBuilder builder, - int encoding, - int metadataOffset, - int childrenOffset, - int buffersOffset, - int statsOffset) { - builder.startTable(5); - ArrayNode.addStats(builder, statsOffset); - ArrayNode.addBuffers(builder, buffersOffset); - ArrayNode.addChildren(builder, childrenOffset); - ArrayNode.addMetadata(builder, metadataOffset); - ArrayNode.addEncoding(builder, encoding); - return ArrayNode.endArrayNode(builder); - } - - public static void startArrayNode(FlatBufferBuilder builder) { builder.startTable(5); } - public static void addEncoding(FlatBufferBuilder builder, int encoding) { builder.addShort(0, (short) encoding, (short) 0); } - public static void addMetadata(FlatBufferBuilder builder, int metadataOffset) { builder.addOffset(1, metadataOffset, 0); } - public static int createMetadataVector(FlatBufferBuilder builder, byte[] data) { return builder.createByteVector(data); } - public static int createMetadataVector(FlatBufferBuilder builder, ByteBuffer data) { return builder.createByteVector(data); } - public static void startMetadataVector(FlatBufferBuilder builder, int numElems) { builder.startVector(1, numElems, 1); } - public static void addChildren(FlatBufferBuilder builder, int childrenOffset) { builder.addOffset(2, childrenOffset, 0); } - public static int createChildrenVector(FlatBufferBuilder builder, int[] data) { builder.startVector(4, data.length, 4); for (int i = data.length - 1; i >= 0; i--) builder.addOffset(data[i]); return builder.endVector(); } - public static void startChildrenVector(FlatBufferBuilder builder, int numElems) { builder.startVector(4, numElems, 4); } - public static void addBuffers(FlatBufferBuilder builder, int buffersOffset) { builder.addOffset(3, buffersOffset, 0); } - public static int createBuffersVector(FlatBufferBuilder builder, int[] data) { builder.startVector(2, data.length, 2); for (int i = data.length - 1; i >= 0; i--) builder.addShort((short) data[i]); return builder.endVector(); } - public static void startBuffersVector(FlatBufferBuilder builder, int numElems) { builder.startVector(2, numElems, 2); } - public static void addStats(FlatBufferBuilder builder, int statsOffset) { builder.addOffset(4, statsOffset, 0); } - public static int endArrayNode(FlatBufferBuilder builder) { - int o = builder.endTable(); - return o; - } - - public static final class Vector extends BaseVector { - public Vector __assign(int _vector, int _element_size, ByteBuffer _bb) { __reset(_vector, _element_size, _bb); return this; } - - public ArrayNode get(int j) { return get(new ArrayNode(), j); } - public ArrayNode get(ArrayNode obj, int j) { return obj.__assign(__indirect(__element(j), bb), bb); } - } -} - diff --git a/core/src/main/java/io/github/dfa1/vortex/fbs/ArraySpec.java b/core/src/main/java/io/github/dfa1/vortex/fbs/ArraySpec.java deleted file mode 100644 index 9b4a800..0000000 --- a/core/src/main/java/io/github/dfa1/vortex/fbs/ArraySpec.java +++ /dev/null @@ -1,60 +0,0 @@ -// automatically generated by the FlatBuffers compiler, do not modify - -package io.github.dfa1.vortex.fbs; - -import com.google.flatbuffers.BaseVector; -import com.google.flatbuffers.BooleanVector; -import com.google.flatbuffers.ByteVector; -import com.google.flatbuffers.Constants; -import com.google.flatbuffers.DoubleVector; -import com.google.flatbuffers.FlatBufferBuilder; -import com.google.flatbuffers.FloatVector; -import com.google.flatbuffers.IntVector; -import com.google.flatbuffers.LongVector; -import com.google.flatbuffers.ShortVector; -import com.google.flatbuffers.StringVector; -import com.google.flatbuffers.Struct; -import com.google.flatbuffers.UnionVector; -import java.nio.ByteBuffer; -import java.nio.ByteOrder; - -/** - * An `ArraySpec` describes the type of a particular array. - * - * These are identified by a globally unique string identifier, and looked up in the Vortex registry - * at read-time. - */ -@SuppressWarnings("unused") -public final class ArraySpec extends com.google.flatbuffers.Table { - public static ArraySpec getRootAsArraySpec(ByteBuffer _bb) { return getRootAsArraySpec(_bb, new ArraySpec()); } - public static ArraySpec getRootAsArraySpec(ByteBuffer _bb, ArraySpec obj) { _bb.order(ByteOrder.LITTLE_ENDIAN); return (obj.__assign(_bb.getInt(_bb.position()) + _bb.position(), _bb)); } - public void __init(int _i, ByteBuffer _bb) { __reset(_i, _bb); } - public ArraySpec __assign(int _i, ByteBuffer _bb) { __init(_i, _bb); return this; } - - public String id() { int o = __offset(4); return o != 0 ? __string(o + bb_pos) : null; } - public ByteBuffer idAsByteBuffer() { return __vector_as_bytebuffer(4, 1); } - public ByteBuffer idInByteBuffer(ByteBuffer _bb) { return __vector_in_bytebuffer(_bb, 4, 1); } - - public static int createArraySpec(FlatBufferBuilder builder, - int idOffset) { - builder.startTable(1); - ArraySpec.addId(builder, idOffset); - return ArraySpec.endArraySpec(builder); - } - - public static void startArraySpec(FlatBufferBuilder builder) { builder.startTable(1); } - public static void addId(FlatBufferBuilder builder, int idOffset) { builder.addOffset(0, idOffset, 0); } - public static int endArraySpec(FlatBufferBuilder builder) { - int o = builder.endTable(); - builder.required(o, 4); // id - return o; - } - - public static final class Vector extends BaseVector { - public Vector __assign(int _vector, int _element_size, ByteBuffer _bb) { __reset(_vector, _element_size, _bb); return this; } - - public ArraySpec get(int j) { return get(new ArraySpec(), j); } - public ArraySpec get(ArraySpec obj, int j) { return obj.__assign(__indirect(__element(j), bb), bb); } - } -} - diff --git a/core/src/main/java/io/github/dfa1/vortex/fbs/ArrayStats.java b/core/src/main/java/io/github/dfa1/vortex/fbs/ArrayStats.java deleted file mode 100644 index b26e449..0000000 --- a/core/src/main/java/io/github/dfa1/vortex/fbs/ArrayStats.java +++ /dev/null @@ -1,124 +0,0 @@ -// automatically generated by the FlatBuffers compiler, do not modify - -package io.github.dfa1.vortex.fbs; - -import com.google.flatbuffers.BaseVector; -import com.google.flatbuffers.BooleanVector; -import com.google.flatbuffers.ByteVector; -import com.google.flatbuffers.Constants; -import com.google.flatbuffers.DoubleVector; -import com.google.flatbuffers.FlatBufferBuilder; -import com.google.flatbuffers.FloatVector; -import com.google.flatbuffers.IntVector; -import com.google.flatbuffers.LongVector; -import com.google.flatbuffers.ShortVector; -import com.google.flatbuffers.StringVector; -import com.google.flatbuffers.Struct; -import com.google.flatbuffers.UnionVector; -import java.nio.ByteBuffer; -import java.nio.ByteOrder; - -@SuppressWarnings("unused") -public final class ArrayStats extends com.google.flatbuffers.Table { - public static ArrayStats getRootAsArrayStats(ByteBuffer _bb) { return getRootAsArrayStats(_bb, new ArrayStats()); } - public static ArrayStats getRootAsArrayStats(ByteBuffer _bb, ArrayStats obj) { _bb.order(ByteOrder.LITTLE_ENDIAN); return (obj.__assign(_bb.getInt(_bb.position()) + _bb.position(), _bb)); } - public void __init(int _i, ByteBuffer _bb) { __reset(_i, _bb); } - public ArrayStats __assign(int _i, ByteBuffer _bb) { __init(_i, _bb); return this; } - - /** - * Protobuf serialized ScalarValue - */ - public int min(int j) { int o = __offset(4); return o != 0 ? bb.get(__vector(o) + j * 1) & 0xFF : 0; } - public int minLength() { int o = __offset(4); return o != 0 ? __vector_len(o) : 0; } - public ByteVector minVector() { return minVector(new ByteVector()); } - public ByteVector minVector(ByteVector obj) { int o = __offset(4); return o != 0 ? obj.__assign(__vector(o), bb) : null; } - public ByteBuffer minAsByteBuffer() { return __vector_as_bytebuffer(4, 1); } - public ByteBuffer minInByteBuffer(ByteBuffer _bb) { return __vector_in_bytebuffer(_bb, 4, 1); } - public int minPrecision() { int o = __offset(6); return o != 0 ? bb.get(o + bb_pos) & 0xFF : 0; } - public int max(int j) { int o = __offset(8); return o != 0 ? bb.get(__vector(o) + j * 1) & 0xFF : 0; } - public int maxLength() { int o = __offset(8); return o != 0 ? __vector_len(o) : 0; } - public ByteVector maxVector() { return maxVector(new ByteVector()); } - public ByteVector maxVector(ByteVector obj) { int o = __offset(8); return o != 0 ? obj.__assign(__vector(o), bb) : null; } - public ByteBuffer maxAsByteBuffer() { return __vector_as_bytebuffer(8, 1); } - public ByteBuffer maxInByteBuffer(ByteBuffer _bb) { return __vector_in_bytebuffer(_bb, 8, 1); } - public int maxPrecision() { int o = __offset(10); return o != 0 ? bb.get(o + bb_pos) & 0xFF : 0; } - public int sum(int j) { int o = __offset(12); return o != 0 ? bb.get(__vector(o) + j * 1) & 0xFF : 0; } - public int sumLength() { int o = __offset(12); return o != 0 ? __vector_len(o) : 0; } - public ByteVector sumVector() { return sumVector(new ByteVector()); } - public ByteVector sumVector(ByteVector obj) { int o = __offset(12); return o != 0 ? obj.__assign(__vector(o), bb) : null; } - public ByteBuffer sumAsByteBuffer() { return __vector_as_bytebuffer(12, 1); } - public ByteBuffer sumInByteBuffer(ByteBuffer _bb) { return __vector_in_bytebuffer(_bb, 12, 1); } - public boolean hasIsSorted() { return 0 != __offset(14); } - public boolean isSorted() { int o = __offset(14); return o != 0 ? 0!=bb.get(o + bb_pos) : false; } - public boolean hasIsStrictSorted() { return 0 != __offset(16); } - public boolean isStrictSorted() { int o = __offset(16); return o != 0 ? 0!=bb.get(o + bb_pos) : false; } - public boolean hasIsConstant() { return 0 != __offset(18); } - public boolean isConstant() { int o = __offset(18); return o != 0 ? 0!=bb.get(o + bb_pos) : false; } - public boolean hasNullCount() { return 0 != __offset(20); } - public long nullCount() { int o = __offset(20); return o != 0 ? bb.getLong(o + bb_pos) : 0L; } - public boolean hasUncompressedSizeInBytes() { return 0 != __offset(22); } - public long uncompressedSizeInBytes() { int o = __offset(22); return o != 0 ? bb.getLong(o + bb_pos) : 0L; } - public boolean hasNanCount() { return 0 != __offset(24); } - public long nanCount() { int o = __offset(24); return o != 0 ? bb.getLong(o + bb_pos) : 0L; } - - public static int createArrayStats(FlatBufferBuilder builder, - int minOffset, - int minPrecision, - int maxOffset, - int maxPrecision, - int sumOffset, - boolean isSorted, - boolean isStrictSorted, - boolean isConstant, - long nullCount, - long uncompressedSizeInBytes, - long nanCount) { - builder.startTable(11); - ArrayStats.addNanCount(builder, nanCount); - ArrayStats.addUncompressedSizeInBytes(builder, uncompressedSizeInBytes); - ArrayStats.addNullCount(builder, nullCount); - ArrayStats.addSum(builder, sumOffset); - ArrayStats.addMax(builder, maxOffset); - ArrayStats.addMin(builder, minOffset); - ArrayStats.addIsConstant(builder, isConstant); - ArrayStats.addIsStrictSorted(builder, isStrictSorted); - ArrayStats.addIsSorted(builder, isSorted); - ArrayStats.addMaxPrecision(builder, maxPrecision); - ArrayStats.addMinPrecision(builder, minPrecision); - return ArrayStats.endArrayStats(builder); - } - - public static void startArrayStats(FlatBufferBuilder builder) { builder.startTable(11); } - public static void addMin(FlatBufferBuilder builder, int minOffset) { builder.addOffset(0, minOffset, 0); } - public static int createMinVector(FlatBufferBuilder builder, byte[] data) { return builder.createByteVector(data); } - public static int createMinVector(FlatBufferBuilder builder, ByteBuffer data) { return builder.createByteVector(data); } - public static void startMinVector(FlatBufferBuilder builder, int numElems) { builder.startVector(1, numElems, 1); } - public static void addMinPrecision(FlatBufferBuilder builder, int minPrecision) { builder.addByte(1, (byte) minPrecision, (byte) 0); } - public static void addMax(FlatBufferBuilder builder, int maxOffset) { builder.addOffset(2, maxOffset, 0); } - public static int createMaxVector(FlatBufferBuilder builder, byte[] data) { return builder.createByteVector(data); } - public static int createMaxVector(FlatBufferBuilder builder, ByteBuffer data) { return builder.createByteVector(data); } - public static void startMaxVector(FlatBufferBuilder builder, int numElems) { builder.startVector(1, numElems, 1); } - public static void addMaxPrecision(FlatBufferBuilder builder, int maxPrecision) { builder.addByte(3, (byte) maxPrecision, (byte) 0); } - public static void addSum(FlatBufferBuilder builder, int sumOffset) { builder.addOffset(4, sumOffset, 0); } - public static int createSumVector(FlatBufferBuilder builder, byte[] data) { return builder.createByteVector(data); } - public static int createSumVector(FlatBufferBuilder builder, ByteBuffer data) { return builder.createByteVector(data); } - public static void startSumVector(FlatBufferBuilder builder, int numElems) { builder.startVector(1, numElems, 1); } - public static void addIsSorted(FlatBufferBuilder builder, boolean isSorted) { builder.addBoolean(5, isSorted, false); } - public static void addIsStrictSorted(FlatBufferBuilder builder, boolean isStrictSorted) { builder.addBoolean(6, isStrictSorted, false); } - public static void addIsConstant(FlatBufferBuilder builder, boolean isConstant) { builder.addBoolean(7, isConstant, false); } - public static void addNullCount(FlatBufferBuilder builder, long nullCount) { builder.addLong(8, nullCount, 0L); } - public static void addUncompressedSizeInBytes(FlatBufferBuilder builder, long uncompressedSizeInBytes) { builder.addLong(9, uncompressedSizeInBytes, 0L); } - public static void addNanCount(FlatBufferBuilder builder, long nanCount) { builder.addLong(10, nanCount, 0L); } - public static int endArrayStats(FlatBufferBuilder builder) { - int o = builder.endTable(); - return o; - } - - public static final class Vector extends BaseVector { - public Vector __assign(int _vector, int _element_size, ByteBuffer _bb) { __reset(_vector, _element_size, _bb); return this; } - - public ArrayStats get(int j) { return get(new ArrayStats(), j); } - public ArrayStats get(ArrayStats obj, int j) { return obj.__assign(__indirect(__element(j), bb), bb); } - } -} - diff --git a/core/src/main/java/io/github/dfa1/vortex/fbs/Binary.java b/core/src/main/java/io/github/dfa1/vortex/fbs/Binary.java deleted file mode 100644 index 174cd75..0000000 --- a/core/src/main/java/io/github/dfa1/vortex/fbs/Binary.java +++ /dev/null @@ -1,51 +0,0 @@ -// automatically generated by the FlatBuffers compiler, do not modify - -package io.github.dfa1.vortex.fbs; - -import com.google.flatbuffers.BaseVector; -import com.google.flatbuffers.BooleanVector; -import com.google.flatbuffers.ByteVector; -import com.google.flatbuffers.Constants; -import com.google.flatbuffers.DoubleVector; -import com.google.flatbuffers.FlatBufferBuilder; -import com.google.flatbuffers.FloatVector; -import com.google.flatbuffers.IntVector; -import com.google.flatbuffers.LongVector; -import com.google.flatbuffers.ShortVector; -import com.google.flatbuffers.StringVector; -import com.google.flatbuffers.Struct; -import com.google.flatbuffers.UnionVector; -import java.nio.ByteBuffer; -import java.nio.ByteOrder; - -@SuppressWarnings("unused") -public final class Binary extends com.google.flatbuffers.Table { - public static Binary getRootAsBinary(ByteBuffer _bb) { return getRootAsBinary(_bb, new Binary()); } - public static Binary getRootAsBinary(ByteBuffer _bb, Binary obj) { _bb.order(ByteOrder.LITTLE_ENDIAN); return (obj.__assign(_bb.getInt(_bb.position()) + _bb.position(), _bb)); } - public void __init(int _i, ByteBuffer _bb) { __reset(_i, _bb); } - public Binary __assign(int _i, ByteBuffer _bb) { __init(_i, _bb); return this; } - - public boolean nullable() { int o = __offset(4); return o != 0 ? 0!=bb.get(o + bb_pos) : false; } - - public static int createBinary(FlatBufferBuilder builder, - boolean nullable) { - builder.startTable(1); - Binary.addNullable(builder, nullable); - return Binary.endBinary(builder); - } - - public static void startBinary(FlatBufferBuilder builder) { builder.startTable(1); } - public static void addNullable(FlatBufferBuilder builder, boolean nullable) { builder.addBoolean(0, nullable, false); } - public static int endBinary(FlatBufferBuilder builder) { - int o = builder.endTable(); - return o; - } - - public static final class Vector extends BaseVector { - public Vector __assign(int _vector, int _element_size, ByteBuffer _bb) { __reset(_vector, _element_size, _bb); return this; } - - public Binary get(int j) { return get(new Binary(), j); } - public Binary get(Binary obj, int j) { return obj.__assign(__indirect(__element(j), bb), bb); } - } -} - diff --git a/core/src/main/java/io/github/dfa1/vortex/fbs/Bool.java b/core/src/main/java/io/github/dfa1/vortex/fbs/Bool.java deleted file mode 100644 index fa81d35..0000000 --- a/core/src/main/java/io/github/dfa1/vortex/fbs/Bool.java +++ /dev/null @@ -1,51 +0,0 @@ -// automatically generated by the FlatBuffers compiler, do not modify - -package io.github.dfa1.vortex.fbs; - -import com.google.flatbuffers.BaseVector; -import com.google.flatbuffers.BooleanVector; -import com.google.flatbuffers.ByteVector; -import com.google.flatbuffers.Constants; -import com.google.flatbuffers.DoubleVector; -import com.google.flatbuffers.FlatBufferBuilder; -import com.google.flatbuffers.FloatVector; -import com.google.flatbuffers.IntVector; -import com.google.flatbuffers.LongVector; -import com.google.flatbuffers.ShortVector; -import com.google.flatbuffers.StringVector; -import com.google.flatbuffers.Struct; -import com.google.flatbuffers.UnionVector; -import java.nio.ByteBuffer; -import java.nio.ByteOrder; - -@SuppressWarnings("unused") -public final class Bool extends com.google.flatbuffers.Table { - public static Bool getRootAsBool(ByteBuffer _bb) { return getRootAsBool(_bb, new Bool()); } - public static Bool getRootAsBool(ByteBuffer _bb, Bool obj) { _bb.order(ByteOrder.LITTLE_ENDIAN); return (obj.__assign(_bb.getInt(_bb.position()) + _bb.position(), _bb)); } - public void __init(int _i, ByteBuffer _bb) { __reset(_i, _bb); } - public Bool __assign(int _i, ByteBuffer _bb) { __init(_i, _bb); return this; } - - public boolean nullable() { int o = __offset(4); return o != 0 ? 0!=bb.get(o + bb_pos) : false; } - - public static int createBool(FlatBufferBuilder builder, - boolean nullable) { - builder.startTable(1); - Bool.addNullable(builder, nullable); - return Bool.endBool(builder); - } - - public static void startBool(FlatBufferBuilder builder) { builder.startTable(1); } - public static void addNullable(FlatBufferBuilder builder, boolean nullable) { builder.addBoolean(0, nullable, false); } - public static int endBool(FlatBufferBuilder builder) { - int o = builder.endTable(); - return o; - } - - public static final class Vector extends BaseVector { - public Vector __assign(int _vector, int _element_size, ByteBuffer _bb) { __reset(_vector, _element_size, _bb); return this; } - - public Bool get(int j) { return get(new Bool(), j); } - public Bool get(Bool obj, int j) { return obj.__assign(__indirect(__element(j), bb), bb); } - } -} - diff --git a/core/src/main/java/io/github/dfa1/vortex/fbs/Buffer.java b/core/src/main/java/io/github/dfa1/vortex/fbs/Buffer.java deleted file mode 100644 index 258ba09..0000000 --- a/core/src/main/java/io/github/dfa1/vortex/fbs/Buffer.java +++ /dev/null @@ -1,62 +0,0 @@ -// automatically generated by the FlatBuffers compiler, do not modify - -package io.github.dfa1.vortex.fbs; - -import com.google.flatbuffers.BaseVector; -import com.google.flatbuffers.BooleanVector; -import com.google.flatbuffers.ByteVector; -import com.google.flatbuffers.Constants; -import com.google.flatbuffers.DoubleVector; -import com.google.flatbuffers.FlatBufferBuilder; -import com.google.flatbuffers.FloatVector; -import com.google.flatbuffers.IntVector; -import com.google.flatbuffers.LongVector; -import com.google.flatbuffers.ShortVector; -import com.google.flatbuffers.StringVector; -import com.google.flatbuffers.Struct; -import com.google.flatbuffers.UnionVector; -import java.nio.ByteBuffer; -import java.nio.ByteOrder; - -/** - * A Buffer describes the location of a data buffer in the byte stream as a packed 64-bit struct. - */ -@SuppressWarnings("unused") -public final class Buffer extends Struct { - public void __init(int _i, ByteBuffer _bb) { __reset(_i, _bb); } - public Buffer __assign(int _i, ByteBuffer _bb) { __init(_i, _bb); return this; } - - /** - * The length of any padding bytes written immediately before the buffer. - */ - public int padding() { return bb.getShort(bb_pos + 0) & 0xFFFF; } - /** - * The minimum alignment of the buffer, stored as an exponent of 2. - */ - public int alignmentExponent() { return bb.get(bb_pos + 2) & 0xFF; } - /** - * The compression algorithm used to compress the buffer. - */ - public int compression() { return bb.get(bb_pos + 3) & 0xFF; } - /** - * The length of the buffer in bytes. - */ - public long length() { return (long)bb.getInt(bb_pos + 4) & 0xFFFFFFFFL; } - - public static int createBuffer(FlatBufferBuilder builder, int padding, int alignmentExponent, int compression, long length) { - builder.prep(4, 8); - builder.putInt((int) length); - builder.putByte((byte) compression); - builder.putByte((byte) alignmentExponent); - builder.putShort((short) padding); - return builder.offset(); - } - - public static final class Vector extends BaseVector { - public Vector __assign(int _vector, int _element_size, ByteBuffer _bb) { __reset(_vector, _element_size, _bb); return this; } - - public Buffer get(int j) { return get(new Buffer(), j); } - public Buffer get(Buffer obj, int j) { return obj.__assign(__element(j), bb); } - } -} - diff --git a/core/src/main/java/io/github/dfa1/vortex/fbs/Compression.java b/core/src/main/java/io/github/dfa1/vortex/fbs/Compression.java deleted file mode 100644 index 5cb999e..0000000 --- a/core/src/main/java/io/github/dfa1/vortex/fbs/Compression.java +++ /dev/null @@ -1,18 +0,0 @@ -// automatically generated by the FlatBuffers compiler, do not modify - -package io.github.dfa1.vortex.fbs; - -/** - * The compression mechanism used to compress the buffer. - */ -@SuppressWarnings("unused") -public final class Compression { - private Compression() { } - public static final int None = 0; - public static final int LZ4 = 1; - - public static final String[] names = { "None", "LZ4", }; - - public static String name(int e) { return names[e]; } -} - diff --git a/core/src/main/java/io/github/dfa1/vortex/fbs/CompressionScheme.java b/core/src/main/java/io/github/dfa1/vortex/fbs/CompressionScheme.java deleted file mode 100644 index 5ea1ce9..0000000 --- a/core/src/main/java/io/github/dfa1/vortex/fbs/CompressionScheme.java +++ /dev/null @@ -1,17 +0,0 @@ -// automatically generated by the FlatBuffers compiler, do not modify - -package io.github.dfa1.vortex.fbs; - -@SuppressWarnings("unused") -public final class CompressionScheme { - private CompressionScheme() { } - public static final int None = 0; - public static final int LZ4 = 1; - public static final int ZLib = 2; - public static final int ZStd = 3; - - public static final String[] names = { "None", "LZ4", "ZLib", "ZStd", }; - - public static String name(int e) { return names[e]; } -} - diff --git a/core/src/main/java/io/github/dfa1/vortex/fbs/CompressionSpec.java b/core/src/main/java/io/github/dfa1/vortex/fbs/CompressionSpec.java deleted file mode 100644 index 684a685..0000000 --- a/core/src/main/java/io/github/dfa1/vortex/fbs/CompressionSpec.java +++ /dev/null @@ -1,54 +0,0 @@ -// automatically generated by the FlatBuffers compiler, do not modify - -package io.github.dfa1.vortex.fbs; - -import com.google.flatbuffers.BaseVector; -import com.google.flatbuffers.BooleanVector; -import com.google.flatbuffers.ByteVector; -import com.google.flatbuffers.Constants; -import com.google.flatbuffers.DoubleVector; -import com.google.flatbuffers.FlatBufferBuilder; -import com.google.flatbuffers.FloatVector; -import com.google.flatbuffers.IntVector; -import com.google.flatbuffers.LongVector; -import com.google.flatbuffers.ShortVector; -import com.google.flatbuffers.StringVector; -import com.google.flatbuffers.Struct; -import com.google.flatbuffers.UnionVector; -import java.nio.ByteBuffer; -import java.nio.ByteOrder; - -/** - * Definition of a compression scheme. - */ -@SuppressWarnings("unused") -public final class CompressionSpec extends com.google.flatbuffers.Table { - public static CompressionSpec getRootAsCompressionSpec(ByteBuffer _bb) { return getRootAsCompressionSpec(_bb, new CompressionSpec()); } - public static CompressionSpec getRootAsCompressionSpec(ByteBuffer _bb, CompressionSpec obj) { _bb.order(ByteOrder.LITTLE_ENDIAN); return (obj.__assign(_bb.getInt(_bb.position()) + _bb.position(), _bb)); } - public void __init(int _i, ByteBuffer _bb) { __reset(_i, _bb); } - public CompressionSpec __assign(int _i, ByteBuffer _bb) { __init(_i, _bb); return this; } - - public int scheme() { int o = __offset(4); return o != 0 ? bb.get(o + bb_pos) & 0xFF : 0; } - - public static int createCompressionSpec(FlatBufferBuilder builder, - int scheme) { - builder.startTable(1); - CompressionSpec.addScheme(builder, scheme); - return CompressionSpec.endCompressionSpec(builder); - } - - public static void startCompressionSpec(FlatBufferBuilder builder) { builder.startTable(1); } - public static void addScheme(FlatBufferBuilder builder, int scheme) { builder.addByte(0, (byte) scheme, (byte) 0); } - public static int endCompressionSpec(FlatBufferBuilder builder) { - int o = builder.endTable(); - return o; - } - - public static final class Vector extends BaseVector { - public Vector __assign(int _vector, int _element_size, ByteBuffer _bb) { __reset(_vector, _element_size, _bb); return this; } - - public CompressionSpec get(int j) { return get(new CompressionSpec(), j); } - public CompressionSpec get(CompressionSpec obj, int j) { return obj.__assign(__indirect(__element(j), bb), bb); } - } -} - diff --git a/core/src/main/java/io/github/dfa1/vortex/fbs/DType.java b/core/src/main/java/io/github/dfa1/vortex/fbs/DType.java deleted file mode 100644 index 96465fd..0000000 --- a/core/src/main/java/io/github/dfa1/vortex/fbs/DType.java +++ /dev/null @@ -1,57 +0,0 @@ -// automatically generated by the FlatBuffers compiler, do not modify - -package io.github.dfa1.vortex.fbs; - -import com.google.flatbuffers.BaseVector; -import com.google.flatbuffers.BooleanVector; -import com.google.flatbuffers.ByteVector; -import com.google.flatbuffers.Constants; -import com.google.flatbuffers.DoubleVector; -import com.google.flatbuffers.FlatBufferBuilder; -import com.google.flatbuffers.FloatVector; -import com.google.flatbuffers.IntVector; -import com.google.flatbuffers.LongVector; -import com.google.flatbuffers.ShortVector; -import com.google.flatbuffers.StringVector; -import com.google.flatbuffers.Struct; -import com.google.flatbuffers.UnionVector; -import java.nio.ByteBuffer; -import java.nio.ByteOrder; - -@SuppressWarnings("unused") -public final class DType extends com.google.flatbuffers.Table { - public static DType getRootAsDType(ByteBuffer _bb) { return getRootAsDType(_bb, new DType()); } - public static DType getRootAsDType(ByteBuffer _bb, DType obj) { _bb.order(ByteOrder.LITTLE_ENDIAN); return (obj.__assign(_bb.getInt(_bb.position()) + _bb.position(), _bb)); } - public void __init(int _i, ByteBuffer _bb) { __reset(_i, _bb); } - public DType __assign(int _i, ByteBuffer _bb) { __init(_i, _bb); return this; } - - public byte typeType() { int o = __offset(4); return o != 0 ? bb.get(o + bb_pos) : 0; } - public com.google.flatbuffers.Table type(com.google.flatbuffers.Table obj) { int o = __offset(6); return o != 0 ? __union(obj, o + bb_pos) : null; } - - public static int createDType(FlatBufferBuilder builder, - byte typeType, - int typeOffset) { - builder.startTable(2); - DType.addType(builder, typeOffset); - DType.addTypeType(builder, typeType); - return DType.endDType(builder); - } - - public static void startDType(FlatBufferBuilder builder) { builder.startTable(2); } - public static void addTypeType(FlatBufferBuilder builder, byte typeType) { builder.addByte(0, typeType, 0); } - public static void addType(FlatBufferBuilder builder, int typeOffset) { builder.addOffset(1, typeOffset, 0); } - public static int endDType(FlatBufferBuilder builder) { - int o = builder.endTable(); - return o; - } - public static void finishDTypeBuffer(FlatBufferBuilder builder, int offset) { builder.finish(offset); } - public static void finishSizePrefixedDTypeBuffer(FlatBufferBuilder builder, int offset) { builder.finishSizePrefixed(offset); } - - public static final class Vector extends BaseVector { - public Vector __assign(int _vector, int _element_size, ByteBuffer _bb) { __reset(_vector, _element_size, _bb); return this; } - - public DType get(int j) { return get(new DType(), j); } - public DType get(DType obj, int j) { return obj.__assign(__indirect(__element(j), bb), bb); } - } -} - diff --git a/core/src/main/java/io/github/dfa1/vortex/fbs/Decimal.java b/core/src/main/java/io/github/dfa1/vortex/fbs/Decimal.java deleted file mode 100644 index 15a3828..0000000 --- a/core/src/main/java/io/github/dfa1/vortex/fbs/Decimal.java +++ /dev/null @@ -1,59 +0,0 @@ -// automatically generated by the FlatBuffers compiler, do not modify - -package io.github.dfa1.vortex.fbs; - -import com.google.flatbuffers.BaseVector; -import com.google.flatbuffers.BooleanVector; -import com.google.flatbuffers.ByteVector; -import com.google.flatbuffers.Constants; -import com.google.flatbuffers.DoubleVector; -import com.google.flatbuffers.FlatBufferBuilder; -import com.google.flatbuffers.FloatVector; -import com.google.flatbuffers.IntVector; -import com.google.flatbuffers.LongVector; -import com.google.flatbuffers.ShortVector; -import com.google.flatbuffers.StringVector; -import com.google.flatbuffers.Struct; -import com.google.flatbuffers.UnionVector; -import java.nio.ByteBuffer; -import java.nio.ByteOrder; - -@SuppressWarnings("unused") -public final class Decimal extends com.google.flatbuffers.Table { - public static Decimal getRootAsDecimal(ByteBuffer _bb) { return getRootAsDecimal(_bb, new Decimal()); } - public static Decimal getRootAsDecimal(ByteBuffer _bb, Decimal obj) { _bb.order(ByteOrder.LITTLE_ENDIAN); return (obj.__assign(_bb.getInt(_bb.position()) + _bb.position(), _bb)); } - public void __init(int _i, ByteBuffer _bb) { __reset(_i, _bb); } - public Decimal __assign(int _i, ByteBuffer _bb) { __init(_i, _bb); return this; } - - public int precision() { int o = __offset(4); return o != 0 ? bb.get(o + bb_pos) & 0xFF : 0; } - public byte scale() { int o = __offset(6); return o != 0 ? bb.get(o + bb_pos) : 0; } - public boolean nullable() { int o = __offset(8); return o != 0 ? 0!=bb.get(o + bb_pos) : false; } - - public static int createDecimal(FlatBufferBuilder builder, - int precision, - byte scale, - boolean nullable) { - builder.startTable(3); - Decimal.addNullable(builder, nullable); - Decimal.addScale(builder, scale); - Decimal.addPrecision(builder, precision); - return Decimal.endDecimal(builder); - } - - public static void startDecimal(FlatBufferBuilder builder) { builder.startTable(3); } - public static void addPrecision(FlatBufferBuilder builder, int precision) { builder.addByte(0, (byte) precision, (byte) 0); } - public static void addScale(FlatBufferBuilder builder, byte scale) { builder.addByte(1, scale, 0); } - public static void addNullable(FlatBufferBuilder builder, boolean nullable) { builder.addBoolean(2, nullable, false); } - public static int endDecimal(FlatBufferBuilder builder) { - int o = builder.endTable(); - return o; - } - - public static final class Vector extends BaseVector { - public Vector __assign(int _vector, int _element_size, ByteBuffer _bb) { __reset(_vector, _element_size, _bb); return this; } - - public Decimal get(int j) { return get(new Decimal(), j); } - public Decimal get(Decimal obj, int j) { return obj.__assign(__indirect(__element(j), bb), bb); } - } -} - diff --git a/core/src/main/java/io/github/dfa1/vortex/fbs/EncryptionSpec.java b/core/src/main/java/io/github/dfa1/vortex/fbs/EncryptionSpec.java deleted file mode 100644 index 1f55fe0..0000000 --- a/core/src/main/java/io/github/dfa1/vortex/fbs/EncryptionSpec.java +++ /dev/null @@ -1,42 +0,0 @@ -// automatically generated by the FlatBuffers compiler, do not modify - -package io.github.dfa1.vortex.fbs; - -import com.google.flatbuffers.BaseVector; -import com.google.flatbuffers.BooleanVector; -import com.google.flatbuffers.ByteVector; -import com.google.flatbuffers.Constants; -import com.google.flatbuffers.DoubleVector; -import com.google.flatbuffers.FlatBufferBuilder; -import com.google.flatbuffers.FloatVector; -import com.google.flatbuffers.IntVector; -import com.google.flatbuffers.LongVector; -import com.google.flatbuffers.ShortVector; -import com.google.flatbuffers.StringVector; -import com.google.flatbuffers.Struct; -import com.google.flatbuffers.UnionVector; -import java.nio.ByteBuffer; -import java.nio.ByteOrder; - -@SuppressWarnings("unused") -public final class EncryptionSpec extends com.google.flatbuffers.Table { - public static EncryptionSpec getRootAsEncryptionSpec(ByteBuffer _bb) { return getRootAsEncryptionSpec(_bb, new EncryptionSpec()); } - public static EncryptionSpec getRootAsEncryptionSpec(ByteBuffer _bb, EncryptionSpec obj) { _bb.order(ByteOrder.LITTLE_ENDIAN); return (obj.__assign(_bb.getInt(_bb.position()) + _bb.position(), _bb)); } - public void __init(int _i, ByteBuffer _bb) { __reset(_i, _bb); } - public EncryptionSpec __assign(int _i, ByteBuffer _bb) { __init(_i, _bb); return this; } - - - public static void startEncryptionSpec(FlatBufferBuilder builder) { builder.startTable(0); } - public static int endEncryptionSpec(FlatBufferBuilder builder) { - int o = builder.endTable(); - return o; - } - - public static final class Vector extends BaseVector { - public Vector __assign(int _vector, int _element_size, ByteBuffer _bb) { __reset(_vector, _element_size, _bb); return this; } - - public EncryptionSpec get(int j) { return get(new EncryptionSpec(), j); } - public EncryptionSpec get(EncryptionSpec obj, int j) { return obj.__assign(__indirect(__element(j), bb), bb); } - } -} - diff --git a/core/src/main/java/io/github/dfa1/vortex/fbs/Extension.java b/core/src/main/java/io/github/dfa1/vortex/fbs/Extension.java deleted file mode 100644 index c05f1e7..0000000 --- a/core/src/main/java/io/github/dfa1/vortex/fbs/Extension.java +++ /dev/null @@ -1,70 +0,0 @@ -// automatically generated by the FlatBuffers compiler, do not modify - -package io.github.dfa1.vortex.fbs; - -import com.google.flatbuffers.BaseVector; -import com.google.flatbuffers.BooleanVector; -import com.google.flatbuffers.ByteVector; -import com.google.flatbuffers.Constants; -import com.google.flatbuffers.DoubleVector; -import com.google.flatbuffers.FlatBufferBuilder; -import com.google.flatbuffers.FloatVector; -import com.google.flatbuffers.IntVector; -import com.google.flatbuffers.LongVector; -import com.google.flatbuffers.ShortVector; -import com.google.flatbuffers.StringVector; -import com.google.flatbuffers.Struct; -import com.google.flatbuffers.UnionVector; -import java.nio.ByteBuffer; -import java.nio.ByteOrder; - -@SuppressWarnings("unused") -public final class Extension extends com.google.flatbuffers.Table { - public static Extension getRootAsExtension(ByteBuffer _bb) { return getRootAsExtension(_bb, new Extension()); } - public static Extension getRootAsExtension(ByteBuffer _bb, Extension obj) { _bb.order(ByteOrder.LITTLE_ENDIAN); return (obj.__assign(_bb.getInt(_bb.position()) + _bb.position(), _bb)); } - public void __init(int _i, ByteBuffer _bb) { __reset(_i, _bb); } - public Extension __assign(int _i, ByteBuffer _bb) { __init(_i, _bb); return this; } - - public String id() { int o = __offset(4); return o != 0 ? __string(o + bb_pos) : null; } - public ByteBuffer idAsByteBuffer() { return __vector_as_bytebuffer(4, 1); } - public ByteBuffer idInByteBuffer(ByteBuffer _bb) { return __vector_in_bytebuffer(_bb, 4, 1); } - public io.github.dfa1.vortex.fbs.DType storageDtype() { return storageDtype(new io.github.dfa1.vortex.fbs.DType()); } - public io.github.dfa1.vortex.fbs.DType storageDtype(io.github.dfa1.vortex.fbs.DType obj) { int o = __offset(6); return o != 0 ? obj.__assign(__indirect(o + bb_pos), bb) : null; } - public int metadata(int j) { int o = __offset(8); return o != 0 ? bb.get(__vector(o) + j * 1) & 0xFF : 0; } - public int metadataLength() { int o = __offset(8); return o != 0 ? __vector_len(o) : 0; } - public ByteVector metadataVector() { return metadataVector(new ByteVector()); } - public ByteVector metadataVector(ByteVector obj) { int o = __offset(8); return o != 0 ? obj.__assign(__vector(o), bb) : null; } - public ByteBuffer metadataAsByteBuffer() { return __vector_as_bytebuffer(8, 1); } - public ByteBuffer metadataInByteBuffer(ByteBuffer _bb) { return __vector_in_bytebuffer(_bb, 8, 1); } - - public static int createExtension(FlatBufferBuilder builder, - int idOffset, - int storageDtypeOffset, - int metadataOffset) { - builder.startTable(3); - Extension.addMetadata(builder, metadataOffset); - Extension.addStorageDtype(builder, storageDtypeOffset); - Extension.addId(builder, idOffset); - return Extension.endExtension(builder); - } - - public static void startExtension(FlatBufferBuilder builder) { builder.startTable(3); } - public static void addId(FlatBufferBuilder builder, int idOffset) { builder.addOffset(0, idOffset, 0); } - public static void addStorageDtype(FlatBufferBuilder builder, int storageDtypeOffset) { builder.addOffset(1, storageDtypeOffset, 0); } - public static void addMetadata(FlatBufferBuilder builder, int metadataOffset) { builder.addOffset(2, metadataOffset, 0); } - public static int createMetadataVector(FlatBufferBuilder builder, byte[] data) { return builder.createByteVector(data); } - public static int createMetadataVector(FlatBufferBuilder builder, ByteBuffer data) { return builder.createByteVector(data); } - public static void startMetadataVector(FlatBufferBuilder builder, int numElems) { builder.startVector(1, numElems, 1); } - public static int endExtension(FlatBufferBuilder builder) { - int o = builder.endTable(); - return o; - } - - public static final class Vector extends BaseVector { - public Vector __assign(int _vector, int _element_size, ByteBuffer _bb) { __reset(_vector, _element_size, _bb); return this; } - - public Extension get(int j) { return get(new Extension(), j); } - public Extension get(Extension obj, int j) { return obj.__assign(__indirect(__element(j), bb), bb); } - } -} - diff --git a/core/src/main/java/io/github/dfa1/vortex/fbs/FileStatistics.java b/core/src/main/java/io/github/dfa1/vortex/fbs/FileStatistics.java deleted file mode 100644 index 6c0bd6b..0000000 --- a/core/src/main/java/io/github/dfa1/vortex/fbs/FileStatistics.java +++ /dev/null @@ -1,64 +0,0 @@ -// automatically generated by the FlatBuffers compiler, do not modify - -package io.github.dfa1.vortex.fbs; - -import com.google.flatbuffers.BaseVector; -import com.google.flatbuffers.BooleanVector; -import com.google.flatbuffers.ByteVector; -import com.google.flatbuffers.Constants; -import com.google.flatbuffers.DoubleVector; -import com.google.flatbuffers.FlatBufferBuilder; -import com.google.flatbuffers.FloatVector; -import com.google.flatbuffers.IntVector; -import com.google.flatbuffers.LongVector; -import com.google.flatbuffers.ShortVector; -import com.google.flatbuffers.StringVector; -import com.google.flatbuffers.Struct; -import com.google.flatbuffers.UnionVector; -import java.nio.ByteBuffer; -import java.nio.ByteOrder; - -/** - * The `FileStatistics` object contains file-level statistics for the Vortex file. - */ -@SuppressWarnings("unused") -public final class FileStatistics extends com.google.flatbuffers.Table { - public static FileStatistics getRootAsFileStatistics(ByteBuffer _bb) { return getRootAsFileStatistics(_bb, new FileStatistics()); } - public static FileStatistics getRootAsFileStatistics(ByteBuffer _bb, FileStatistics obj) { _bb.order(ByteOrder.LITTLE_ENDIAN); return (obj.__assign(_bb.getInt(_bb.position()) + _bb.position(), _bb)); } - public void __init(int _i, ByteBuffer _bb) { __reset(_i, _bb); } - public FileStatistics __assign(int _i, ByteBuffer _bb) { __init(_i, _bb); return this; } - - /** - * Statistics for each field in the root schema. If the root schema is not a struct, there will - * be a single entry in this array. - */ - public io.github.dfa1.vortex.fbs.ArrayStats fieldStats(int j) { return fieldStats(new io.github.dfa1.vortex.fbs.ArrayStats(), j); } - public io.github.dfa1.vortex.fbs.ArrayStats fieldStats(io.github.dfa1.vortex.fbs.ArrayStats obj, int j) { int o = __offset(4); return o != 0 ? obj.__assign(__indirect(__vector(o) + j * 4), bb) : null; } - public int fieldStatsLength() { int o = __offset(4); return o != 0 ? __vector_len(o) : 0; } - public io.github.dfa1.vortex.fbs.ArrayStats.Vector fieldStatsVector() { return fieldStatsVector(new io.github.dfa1.vortex.fbs.ArrayStats.Vector()); } - public io.github.dfa1.vortex.fbs.ArrayStats.Vector fieldStatsVector(io.github.dfa1.vortex.fbs.ArrayStats.Vector obj) { int o = __offset(4); return o != 0 ? obj.__assign(__vector(o), 4, bb) : null; } - - public static int createFileStatistics(FlatBufferBuilder builder, - int fieldStatsOffset) { - builder.startTable(1); - FileStatistics.addFieldStats(builder, fieldStatsOffset); - return FileStatistics.endFileStatistics(builder); - } - - public static void startFileStatistics(FlatBufferBuilder builder) { builder.startTable(1); } - public static void addFieldStats(FlatBufferBuilder builder, int fieldStatsOffset) { builder.addOffset(0, fieldStatsOffset, 0); } - public static int createFieldStatsVector(FlatBufferBuilder builder, int[] data) { builder.startVector(4, data.length, 4); for (int i = data.length - 1; i >= 0; i--) builder.addOffset(data[i]); return builder.endVector(); } - public static void startFieldStatsVector(FlatBufferBuilder builder, int numElems) { builder.startVector(4, numElems, 4); } - public static int endFileStatistics(FlatBufferBuilder builder) { - int o = builder.endTable(); - return o; - } - - public static final class Vector extends BaseVector { - public Vector __assign(int _vector, int _element_size, ByteBuffer _bb) { __reset(_vector, _element_size, _bb); return this; } - - public FileStatistics get(int j) { return get(new FileStatistics(), j); } - public FileStatistics get(FileStatistics obj, int j) { return obj.__assign(__indirect(__element(j), bb), bb); } - } -} - diff --git a/core/src/main/java/io/github/dfa1/vortex/fbs/FixedSizeList.java b/core/src/main/java/io/github/dfa1/vortex/fbs/FixedSizeList.java deleted file mode 100644 index 42d145d..0000000 --- a/core/src/main/java/io/github/dfa1/vortex/fbs/FixedSizeList.java +++ /dev/null @@ -1,60 +0,0 @@ -// automatically generated by the FlatBuffers compiler, do not modify - -package io.github.dfa1.vortex.fbs; - -import com.google.flatbuffers.BaseVector; -import com.google.flatbuffers.BooleanVector; -import com.google.flatbuffers.ByteVector; -import com.google.flatbuffers.Constants; -import com.google.flatbuffers.DoubleVector; -import com.google.flatbuffers.FlatBufferBuilder; -import com.google.flatbuffers.FloatVector; -import com.google.flatbuffers.IntVector; -import com.google.flatbuffers.LongVector; -import com.google.flatbuffers.ShortVector; -import com.google.flatbuffers.StringVector; -import com.google.flatbuffers.Struct; -import com.google.flatbuffers.UnionVector; -import java.nio.ByteBuffer; -import java.nio.ByteOrder; - -@SuppressWarnings("unused") -public final class FixedSizeList extends com.google.flatbuffers.Table { - public static FixedSizeList getRootAsFixedSizeList(ByteBuffer _bb) { return getRootAsFixedSizeList(_bb, new FixedSizeList()); } - public static FixedSizeList getRootAsFixedSizeList(ByteBuffer _bb, FixedSizeList obj) { _bb.order(ByteOrder.LITTLE_ENDIAN); return (obj.__assign(_bb.getInt(_bb.position()) + _bb.position(), _bb)); } - public void __init(int _i, ByteBuffer _bb) { __reset(_i, _bb); } - public FixedSizeList __assign(int _i, ByteBuffer _bb) { __init(_i, _bb); return this; } - - public io.github.dfa1.vortex.fbs.DType elementType() { return elementType(new io.github.dfa1.vortex.fbs.DType()); } - public io.github.dfa1.vortex.fbs.DType elementType(io.github.dfa1.vortex.fbs.DType obj) { int o = __offset(4); return o != 0 ? obj.__assign(__indirect(o + bb_pos), bb) : null; } - public long size() { int o = __offset(6); return o != 0 ? (long)bb.getInt(o + bb_pos) & 0xFFFFFFFFL : 0L; } - public boolean nullable() { int o = __offset(8); return o != 0 ? 0!=bb.get(o + bb_pos) : false; } - - public static int createFixedSizeList(FlatBufferBuilder builder, - int elementTypeOffset, - long size, - boolean nullable) { - builder.startTable(3); - FixedSizeList.addSize(builder, size); - FixedSizeList.addElementType(builder, elementTypeOffset); - FixedSizeList.addNullable(builder, nullable); - return FixedSizeList.endFixedSizeList(builder); - } - - public static void startFixedSizeList(FlatBufferBuilder builder) { builder.startTable(3); } - public static void addElementType(FlatBufferBuilder builder, int elementTypeOffset) { builder.addOffset(0, elementTypeOffset, 0); } - public static void addSize(FlatBufferBuilder builder, long size) { builder.addInt(1, (int) size, (int) 0L); } - public static void addNullable(FlatBufferBuilder builder, boolean nullable) { builder.addBoolean(2, nullable, false); } - public static int endFixedSizeList(FlatBufferBuilder builder) { - int o = builder.endTable(); - return o; - } - - public static final class Vector extends BaseVector { - public Vector __assign(int _vector, int _element_size, ByteBuffer _bb) { __reset(_vector, _element_size, _bb); return this; } - - public FixedSizeList get(int j) { return get(new FixedSizeList(), j); } - public FixedSizeList get(FixedSizeList obj, int j) { return obj.__assign(__indirect(__element(j), bb), bb); } - } -} - diff --git a/core/src/main/java/io/github/dfa1/vortex/fbs/Footer.java b/core/src/main/java/io/github/dfa1/vortex/fbs/Footer.java deleted file mode 100644 index d664e86..0000000 --- a/core/src/main/java/io/github/dfa1/vortex/fbs/Footer.java +++ /dev/null @@ -1,100 +0,0 @@ -// automatically generated by the FlatBuffers compiler, do not modify - -package io.github.dfa1.vortex.fbs; - -import com.google.flatbuffers.BaseVector; -import com.google.flatbuffers.BooleanVector; -import com.google.flatbuffers.ByteVector; -import com.google.flatbuffers.Constants; -import com.google.flatbuffers.DoubleVector; -import com.google.flatbuffers.FlatBufferBuilder; -import com.google.flatbuffers.FloatVector; -import com.google.flatbuffers.IntVector; -import com.google.flatbuffers.LongVector; -import com.google.flatbuffers.ShortVector; -import com.google.flatbuffers.StringVector; -import com.google.flatbuffers.Struct; -import com.google.flatbuffers.UnionVector; -import java.nio.ByteBuffer; -import java.nio.ByteOrder; - -/** - * The `Registry` object stores dictionary-encoded configuration for segments, - * compression schemes, encryption schemes, etc. - */ -@SuppressWarnings("unused") -public final class Footer extends com.google.flatbuffers.Table { - public static Footer getRootAsFooter(ByteBuffer _bb) { return getRootAsFooter(_bb, new Footer()); } - public static Footer getRootAsFooter(ByteBuffer _bb, Footer obj) { _bb.order(ByteOrder.LITTLE_ENDIAN); return (obj.__assign(_bb.getInt(_bb.position()) + _bb.position(), _bb)); } - public void __init(int _i, ByteBuffer _bb) { __reset(_i, _bb); } - public Footer __assign(int _i, ByteBuffer _bb) { __init(_i, _bb); return this; } - - public io.github.dfa1.vortex.fbs.ArraySpec arraySpecs(int j) { return arraySpecs(new io.github.dfa1.vortex.fbs.ArraySpec(), j); } - public io.github.dfa1.vortex.fbs.ArraySpec arraySpecs(io.github.dfa1.vortex.fbs.ArraySpec obj, int j) { int o = __offset(4); return o != 0 ? obj.__assign(__indirect(__vector(o) + j * 4), bb) : null; } - public int arraySpecsLength() { int o = __offset(4); return o != 0 ? __vector_len(o) : 0; } - public io.github.dfa1.vortex.fbs.ArraySpec.Vector arraySpecsVector() { return arraySpecsVector(new io.github.dfa1.vortex.fbs.ArraySpec.Vector()); } - public io.github.dfa1.vortex.fbs.ArraySpec.Vector arraySpecsVector(io.github.dfa1.vortex.fbs.ArraySpec.Vector obj) { int o = __offset(4); return o != 0 ? obj.__assign(__vector(o), 4, bb) : null; } - public io.github.dfa1.vortex.fbs.LayoutSpec layoutSpecs(int j) { return layoutSpecs(new io.github.dfa1.vortex.fbs.LayoutSpec(), j); } - public io.github.dfa1.vortex.fbs.LayoutSpec layoutSpecs(io.github.dfa1.vortex.fbs.LayoutSpec obj, int j) { int o = __offset(6); return o != 0 ? obj.__assign(__indirect(__vector(o) + j * 4), bb) : null; } - public int layoutSpecsLength() { int o = __offset(6); return o != 0 ? __vector_len(o) : 0; } - public io.github.dfa1.vortex.fbs.LayoutSpec.Vector layoutSpecsVector() { return layoutSpecsVector(new io.github.dfa1.vortex.fbs.LayoutSpec.Vector()); } - public io.github.dfa1.vortex.fbs.LayoutSpec.Vector layoutSpecsVector(io.github.dfa1.vortex.fbs.LayoutSpec.Vector obj) { int o = __offset(6); return o != 0 ? obj.__assign(__vector(o), 4, bb) : null; } - public io.github.dfa1.vortex.fbs.SegmentSpec segmentSpecs(int j) { return segmentSpecs(new io.github.dfa1.vortex.fbs.SegmentSpec(), j); } - public io.github.dfa1.vortex.fbs.SegmentSpec segmentSpecs(io.github.dfa1.vortex.fbs.SegmentSpec obj, int j) { int o = __offset(8); return o != 0 ? obj.__assign(__vector(o) + j * 16, bb) : null; } - public int segmentSpecsLength() { int o = __offset(8); return o != 0 ? __vector_len(o) : 0; } - public io.github.dfa1.vortex.fbs.SegmentSpec.Vector segmentSpecsVector() { return segmentSpecsVector(new io.github.dfa1.vortex.fbs.SegmentSpec.Vector()); } - public io.github.dfa1.vortex.fbs.SegmentSpec.Vector segmentSpecsVector(io.github.dfa1.vortex.fbs.SegmentSpec.Vector obj) { int o = __offset(8); return o != 0 ? obj.__assign(__vector(o), 16, bb) : null; } - public io.github.dfa1.vortex.fbs.CompressionSpec compressionSpecs(int j) { return compressionSpecs(new io.github.dfa1.vortex.fbs.CompressionSpec(), j); } - public io.github.dfa1.vortex.fbs.CompressionSpec compressionSpecs(io.github.dfa1.vortex.fbs.CompressionSpec obj, int j) { int o = __offset(10); return o != 0 ? obj.__assign(__indirect(__vector(o) + j * 4), bb) : null; } - public int compressionSpecsLength() { int o = __offset(10); return o != 0 ? __vector_len(o) : 0; } - public io.github.dfa1.vortex.fbs.CompressionSpec.Vector compressionSpecsVector() { return compressionSpecsVector(new io.github.dfa1.vortex.fbs.CompressionSpec.Vector()); } - public io.github.dfa1.vortex.fbs.CompressionSpec.Vector compressionSpecsVector(io.github.dfa1.vortex.fbs.CompressionSpec.Vector obj) { int o = __offset(10); return o != 0 ? obj.__assign(__vector(o), 4, bb) : null; } - public io.github.dfa1.vortex.fbs.EncryptionSpec encryptionSpecs(int j) { return encryptionSpecs(new io.github.dfa1.vortex.fbs.EncryptionSpec(), j); } - public io.github.dfa1.vortex.fbs.EncryptionSpec encryptionSpecs(io.github.dfa1.vortex.fbs.EncryptionSpec obj, int j) { int o = __offset(12); return o != 0 ? obj.__assign(__indirect(__vector(o) + j * 4), bb) : null; } - public int encryptionSpecsLength() { int o = __offset(12); return o != 0 ? __vector_len(o) : 0; } - public io.github.dfa1.vortex.fbs.EncryptionSpec.Vector encryptionSpecsVector() { return encryptionSpecsVector(new io.github.dfa1.vortex.fbs.EncryptionSpec.Vector()); } - public io.github.dfa1.vortex.fbs.EncryptionSpec.Vector encryptionSpecsVector(io.github.dfa1.vortex.fbs.EncryptionSpec.Vector obj) { int o = __offset(12); return o != 0 ? obj.__assign(__vector(o), 4, bb) : null; } - - public static int createFooter(FlatBufferBuilder builder, - int arraySpecsOffset, - int layoutSpecsOffset, - int segmentSpecsOffset, - int compressionSpecsOffset, - int encryptionSpecsOffset) { - builder.startTable(5); - Footer.addEncryptionSpecs(builder, encryptionSpecsOffset); - Footer.addCompressionSpecs(builder, compressionSpecsOffset); - Footer.addSegmentSpecs(builder, segmentSpecsOffset); - Footer.addLayoutSpecs(builder, layoutSpecsOffset); - Footer.addArraySpecs(builder, arraySpecsOffset); - return Footer.endFooter(builder); - } - - public static void startFooter(FlatBufferBuilder builder) { builder.startTable(5); } - public static void addArraySpecs(FlatBufferBuilder builder, int arraySpecsOffset) { builder.addOffset(0, arraySpecsOffset, 0); } - public static int createArraySpecsVector(FlatBufferBuilder builder, int[] data) { builder.startVector(4, data.length, 4); for (int i = data.length - 1; i >= 0; i--) builder.addOffset(data[i]); return builder.endVector(); } - public static void startArraySpecsVector(FlatBufferBuilder builder, int numElems) { builder.startVector(4, numElems, 4); } - public static void addLayoutSpecs(FlatBufferBuilder builder, int layoutSpecsOffset) { builder.addOffset(1, layoutSpecsOffset, 0); } - public static int createLayoutSpecsVector(FlatBufferBuilder builder, int[] data) { builder.startVector(4, data.length, 4); for (int i = data.length - 1; i >= 0; i--) builder.addOffset(data[i]); return builder.endVector(); } - public static void startLayoutSpecsVector(FlatBufferBuilder builder, int numElems) { builder.startVector(4, numElems, 4); } - public static void addSegmentSpecs(FlatBufferBuilder builder, int segmentSpecsOffset) { builder.addOffset(2, segmentSpecsOffset, 0); } - public static void startSegmentSpecsVector(FlatBufferBuilder builder, int numElems) { builder.startVector(16, numElems, 8); } - public static void addCompressionSpecs(FlatBufferBuilder builder, int compressionSpecsOffset) { builder.addOffset(3, compressionSpecsOffset, 0); } - public static int createCompressionSpecsVector(FlatBufferBuilder builder, int[] data) { builder.startVector(4, data.length, 4); for (int i = data.length - 1; i >= 0; i--) builder.addOffset(data[i]); return builder.endVector(); } - public static void startCompressionSpecsVector(FlatBufferBuilder builder, int numElems) { builder.startVector(4, numElems, 4); } - public static void addEncryptionSpecs(FlatBufferBuilder builder, int encryptionSpecsOffset) { builder.addOffset(4, encryptionSpecsOffset, 0); } - public static int createEncryptionSpecsVector(FlatBufferBuilder builder, int[] data) { builder.startVector(4, data.length, 4); for (int i = data.length - 1; i >= 0; i--) builder.addOffset(data[i]); return builder.endVector(); } - public static void startEncryptionSpecsVector(FlatBufferBuilder builder, int numElems) { builder.startVector(4, numElems, 4); } - public static int endFooter(FlatBufferBuilder builder) { - int o = builder.endTable(); - return o; - } - - public static final class Vector extends BaseVector { - public Vector __assign(int _vector, int _element_size, ByteBuffer _bb) { __reset(_vector, _element_size, _bb); return this; } - - public Footer get(int j) { return get(new Footer(), j); } - public Footer get(Footer obj, int j) { return obj.__assign(__indirect(__element(j), bb), bb); } - } -} - diff --git a/core/src/main/java/io/github/dfa1/vortex/fbs/Layout.java b/core/src/main/java/io/github/dfa1/vortex/fbs/Layout.java deleted file mode 100644 index 9fab117..0000000 --- a/core/src/main/java/io/github/dfa1/vortex/fbs/Layout.java +++ /dev/null @@ -1,106 +0,0 @@ -// automatically generated by the FlatBuffers compiler, do not modify - -package io.github.dfa1.vortex.fbs; - -import com.google.flatbuffers.BaseVector; -import com.google.flatbuffers.BooleanVector; -import com.google.flatbuffers.ByteVector; -import com.google.flatbuffers.Constants; -import com.google.flatbuffers.DoubleVector; -import com.google.flatbuffers.FlatBufferBuilder; -import com.google.flatbuffers.FloatVector; -import com.google.flatbuffers.IntVector; -import com.google.flatbuffers.LongVector; -import com.google.flatbuffers.ShortVector; -import com.google.flatbuffers.StringVector; -import com.google.flatbuffers.Struct; -import com.google.flatbuffers.UnionVector; -import java.nio.ByteBuffer; -import java.nio.ByteOrder; - -@SuppressWarnings("unused") -public final class Layout extends com.google.flatbuffers.Table { - public static Layout getRootAsLayout(ByteBuffer _bb) { return getRootAsLayout(_bb, new Layout()); } - public static Layout getRootAsLayout(ByteBuffer _bb, Layout obj) { _bb.order(ByteOrder.LITTLE_ENDIAN); return (obj.__assign(_bb.getInt(_bb.position()) + _bb.position(), _bb)); } - public void __init(int _i, ByteBuffer _bb) { __reset(_i, _bb); } - public Layout __assign(int _i, ByteBuffer _bb) { __init(_i, _bb); return this; } - - /** - * The ID of the encoding used for this Layout. - */ - public int encoding() { int o = __offset(4); return o != 0 ? bb.getShort(o + bb_pos) & 0xFFFF : 0; } - /** - * The number of rows of data represented by this Layout. - */ - public long rowCount() { int o = __offset(6); return o != 0 ? bb.getLong(o + bb_pos) : 0L; } - /** - * Any additional metadata this layout needs to interpret its children. - * This does not include data-specific metadata, which the layout should store in a segment. - */ - public int metadata(int j) { int o = __offset(8); return o != 0 ? bb.get(__vector(o) + j * 1) & 0xFF : 0; } - public int metadataLength() { int o = __offset(8); return o != 0 ? __vector_len(o) : 0; } - public ByteVector metadataVector() { return metadataVector(new ByteVector()); } - public ByteVector metadataVector(ByteVector obj) { int o = __offset(8); return o != 0 ? obj.__assign(__vector(o), bb) : null; } - public ByteBuffer metadataAsByteBuffer() { return __vector_as_bytebuffer(8, 1); } - public ByteBuffer metadataInByteBuffer(ByteBuffer _bb) { return __vector_in_bytebuffer(_bb, 8, 1); } - /** - * The children of this Layout. - */ - public io.github.dfa1.vortex.fbs.Layout children(int j) { return children(new io.github.dfa1.vortex.fbs.Layout(), j); } - public io.github.dfa1.vortex.fbs.Layout children(io.github.dfa1.vortex.fbs.Layout obj, int j) { int o = __offset(10); return o != 0 ? obj.__assign(__indirect(__vector(o) + j * 4), bb) : null; } - public int childrenLength() { int o = __offset(10); return o != 0 ? __vector_len(o) : 0; } - public io.github.dfa1.vortex.fbs.Layout.Vector childrenVector() { return childrenVector(new io.github.dfa1.vortex.fbs.Layout.Vector()); } - public io.github.dfa1.vortex.fbs.Layout.Vector childrenVector(io.github.dfa1.vortex.fbs.Layout.Vector obj) { int o = __offset(10); return o != 0 ? obj.__assign(__vector(o), 4, bb) : null; } - /** - * Identifiers for each `SegmentSpec` of data required by this layout. - */ - public long segments(int j) { int o = __offset(12); return o != 0 ? (long)bb.getInt(__vector(o) + j * 4) & 0xFFFFFFFFL : 0; } - public int segmentsLength() { int o = __offset(12); return o != 0 ? __vector_len(o) : 0; } - public IntVector segmentsVector() { return segmentsVector(new IntVector()); } - public IntVector segmentsVector(IntVector obj) { int o = __offset(12); return o != 0 ? obj.__assign(__vector(o), bb) : null; } - public ByteBuffer segmentsAsByteBuffer() { return __vector_as_bytebuffer(12, 4); } - public ByteBuffer segmentsInByteBuffer(ByteBuffer _bb) { return __vector_in_bytebuffer(_bb, 12, 4); } - - public static int createLayout(FlatBufferBuilder builder, - int encoding, - long rowCount, - int metadataOffset, - int childrenOffset, - int segmentsOffset) { - builder.startTable(5); - Layout.addRowCount(builder, rowCount); - Layout.addSegments(builder, segmentsOffset); - Layout.addChildren(builder, childrenOffset); - Layout.addMetadata(builder, metadataOffset); - Layout.addEncoding(builder, encoding); - return Layout.endLayout(builder); - } - - public static void startLayout(FlatBufferBuilder builder) { builder.startTable(5); } - public static void addEncoding(FlatBufferBuilder builder, int encoding) { builder.addShort(0, (short) encoding, (short) 0); } - public static void addRowCount(FlatBufferBuilder builder, long rowCount) { builder.addLong(1, rowCount, 0L); } - public static void addMetadata(FlatBufferBuilder builder, int metadataOffset) { builder.addOffset(2, metadataOffset, 0); } - public static int createMetadataVector(FlatBufferBuilder builder, byte[] data) { return builder.createByteVector(data); } - public static int createMetadataVector(FlatBufferBuilder builder, ByteBuffer data) { return builder.createByteVector(data); } - public static void startMetadataVector(FlatBufferBuilder builder, int numElems) { builder.startVector(1, numElems, 1); } - public static void addChildren(FlatBufferBuilder builder, int childrenOffset) { builder.addOffset(3, childrenOffset, 0); } - public static int createChildrenVector(FlatBufferBuilder builder, int[] data) { builder.startVector(4, data.length, 4); for (int i = data.length - 1; i >= 0; i--) builder.addOffset(data[i]); return builder.endVector(); } - public static void startChildrenVector(FlatBufferBuilder builder, int numElems) { builder.startVector(4, numElems, 4); } - public static void addSegments(FlatBufferBuilder builder, int segmentsOffset) { builder.addOffset(4, segmentsOffset, 0); } - public static int createSegmentsVector(FlatBufferBuilder builder, long[] data) { builder.startVector(4, data.length, 4); for (int i = data.length - 1; i >= 0; i--) builder.addInt((int) data[i]); return builder.endVector(); } - public static void startSegmentsVector(FlatBufferBuilder builder, int numElems) { builder.startVector(4, numElems, 4); } - public static int endLayout(FlatBufferBuilder builder) { - int o = builder.endTable(); - return o; - } - public static void finishLayoutBuffer(FlatBufferBuilder builder, int offset) { builder.finish(offset); } - public static void finishSizePrefixedLayoutBuffer(FlatBufferBuilder builder, int offset) { builder.finishSizePrefixed(offset); } - - public static final class Vector extends BaseVector { - public Vector __assign(int _vector, int _element_size, ByteBuffer _bb) { __reset(_vector, _element_size, _bb); return this; } - - public Layout get(int j) { return get(new Layout(), j); } - public Layout get(Layout obj, int j) { return obj.__assign(__indirect(__element(j), bb), bb); } - } -} - diff --git a/core/src/main/java/io/github/dfa1/vortex/fbs/LayoutSpec.java b/core/src/main/java/io/github/dfa1/vortex/fbs/LayoutSpec.java deleted file mode 100644 index 1ee8c31..0000000 --- a/core/src/main/java/io/github/dfa1/vortex/fbs/LayoutSpec.java +++ /dev/null @@ -1,60 +0,0 @@ -// automatically generated by the FlatBuffers compiler, do not modify - -package io.github.dfa1.vortex.fbs; - -import com.google.flatbuffers.BaseVector; -import com.google.flatbuffers.BooleanVector; -import com.google.flatbuffers.ByteVector; -import com.google.flatbuffers.Constants; -import com.google.flatbuffers.DoubleVector; -import com.google.flatbuffers.FlatBufferBuilder; -import com.google.flatbuffers.FloatVector; -import com.google.flatbuffers.IntVector; -import com.google.flatbuffers.LongVector; -import com.google.flatbuffers.ShortVector; -import com.google.flatbuffers.StringVector; -import com.google.flatbuffers.Struct; -import com.google.flatbuffers.UnionVector; -import java.nio.ByteBuffer; -import java.nio.ByteOrder; - -/** - * A `LayoutSpec` describes the type of a particular layout. - * - * These are identified by a globally unique string identifier, and looked up in the Vortex registry - * at read-time. - */ -@SuppressWarnings("unused") -public final class LayoutSpec extends com.google.flatbuffers.Table { - public static LayoutSpec getRootAsLayoutSpec(ByteBuffer _bb) { return getRootAsLayoutSpec(_bb, new LayoutSpec()); } - public static LayoutSpec getRootAsLayoutSpec(ByteBuffer _bb, LayoutSpec obj) { _bb.order(ByteOrder.LITTLE_ENDIAN); return (obj.__assign(_bb.getInt(_bb.position()) + _bb.position(), _bb)); } - public void __init(int _i, ByteBuffer _bb) { __reset(_i, _bb); } - public LayoutSpec __assign(int _i, ByteBuffer _bb) { __init(_i, _bb); return this; } - - public String id() { int o = __offset(4); return o != 0 ? __string(o + bb_pos) : null; } - public ByteBuffer idAsByteBuffer() { return __vector_as_bytebuffer(4, 1); } - public ByteBuffer idInByteBuffer(ByteBuffer _bb) { return __vector_in_bytebuffer(_bb, 4, 1); } - - public static int createLayoutSpec(FlatBufferBuilder builder, - int idOffset) { - builder.startTable(1); - LayoutSpec.addId(builder, idOffset); - return LayoutSpec.endLayoutSpec(builder); - } - - public static void startLayoutSpec(FlatBufferBuilder builder) { builder.startTable(1); } - public static void addId(FlatBufferBuilder builder, int idOffset) { builder.addOffset(0, idOffset, 0); } - public static int endLayoutSpec(FlatBufferBuilder builder) { - int o = builder.endTable(); - builder.required(o, 4); // id - return o; - } - - public static final class Vector extends BaseVector { - public Vector __assign(int _vector, int _element_size, ByteBuffer _bb) { __reset(_vector, _element_size, _bb); return this; } - - public LayoutSpec get(int j) { return get(new LayoutSpec(), j); } - public LayoutSpec get(LayoutSpec obj, int j) { return obj.__assign(__indirect(__element(j), bb), bb); } - } -} - diff --git a/core/src/main/java/io/github/dfa1/vortex/fbs/List.java b/core/src/main/java/io/github/dfa1/vortex/fbs/List.java deleted file mode 100644 index f0e792c..0000000 --- a/core/src/main/java/io/github/dfa1/vortex/fbs/List.java +++ /dev/null @@ -1,56 +0,0 @@ -// automatically generated by the FlatBuffers compiler, do not modify - -package io.github.dfa1.vortex.fbs; - -import com.google.flatbuffers.BaseVector; -import com.google.flatbuffers.BooleanVector; -import com.google.flatbuffers.ByteVector; -import com.google.flatbuffers.Constants; -import com.google.flatbuffers.DoubleVector; -import com.google.flatbuffers.FlatBufferBuilder; -import com.google.flatbuffers.FloatVector; -import com.google.flatbuffers.IntVector; -import com.google.flatbuffers.LongVector; -import com.google.flatbuffers.ShortVector; -import com.google.flatbuffers.StringVector; -import com.google.flatbuffers.Struct; -import com.google.flatbuffers.UnionVector; -import java.nio.ByteBuffer; -import java.nio.ByteOrder; - -@SuppressWarnings("unused") -public final class List extends com.google.flatbuffers.Table { - public static List getRootAsList(ByteBuffer _bb) { return getRootAsList(_bb, new List()); } - public static List getRootAsList(ByteBuffer _bb, List obj) { _bb.order(ByteOrder.LITTLE_ENDIAN); return (obj.__assign(_bb.getInt(_bb.position()) + _bb.position(), _bb)); } - public void __init(int _i, ByteBuffer _bb) { __reset(_i, _bb); } - public List __assign(int _i, ByteBuffer _bb) { __init(_i, _bb); return this; } - - public io.github.dfa1.vortex.fbs.DType elementType() { return elementType(new io.github.dfa1.vortex.fbs.DType()); } - public io.github.dfa1.vortex.fbs.DType elementType(io.github.dfa1.vortex.fbs.DType obj) { int o = __offset(4); return o != 0 ? obj.__assign(__indirect(o + bb_pos), bb) : null; } - public boolean nullable() { int o = __offset(6); return o != 0 ? 0!=bb.get(o + bb_pos) : false; } - - public static int createList(FlatBufferBuilder builder, - int elementTypeOffset, - boolean nullable) { - builder.startTable(2); - List.addElementType(builder, elementTypeOffset); - List.addNullable(builder, nullable); - return List.endList(builder); - } - - public static void startList(FlatBufferBuilder builder) { builder.startTable(2); } - public static void addElementType(FlatBufferBuilder builder, int elementTypeOffset) { builder.addOffset(0, elementTypeOffset, 0); } - public static void addNullable(FlatBufferBuilder builder, boolean nullable) { builder.addBoolean(1, nullable, false); } - public static int endList(FlatBufferBuilder builder) { - int o = builder.endTable(); - return o; - } - - public static final class Vector extends BaseVector { - public Vector __assign(int _vector, int _element_size, ByteBuffer _bb) { __reset(_vector, _element_size, _bb); return this; } - - public List get(int j) { return get(new List(), j); } - public List get(List obj, int j) { return obj.__assign(__indirect(__element(j), bb), bb); } - } -} - diff --git a/core/src/main/java/io/github/dfa1/vortex/fbs/Null.java b/core/src/main/java/io/github/dfa1/vortex/fbs/Null.java deleted file mode 100644 index fb3f8dd..0000000 --- a/core/src/main/java/io/github/dfa1/vortex/fbs/Null.java +++ /dev/null @@ -1,42 +0,0 @@ -// automatically generated by the FlatBuffers compiler, do not modify - -package io.github.dfa1.vortex.fbs; - -import com.google.flatbuffers.BaseVector; -import com.google.flatbuffers.BooleanVector; -import com.google.flatbuffers.ByteVector; -import com.google.flatbuffers.Constants; -import com.google.flatbuffers.DoubleVector; -import com.google.flatbuffers.FlatBufferBuilder; -import com.google.flatbuffers.FloatVector; -import com.google.flatbuffers.IntVector; -import com.google.flatbuffers.LongVector; -import com.google.flatbuffers.ShortVector; -import com.google.flatbuffers.StringVector; -import com.google.flatbuffers.Struct; -import com.google.flatbuffers.UnionVector; -import java.nio.ByteBuffer; -import java.nio.ByteOrder; - -@SuppressWarnings("unused") -public final class Null extends com.google.flatbuffers.Table { - public static Null getRootAsNull(ByteBuffer _bb) { return getRootAsNull(_bb, new Null()); } - public static Null getRootAsNull(ByteBuffer _bb, Null obj) { _bb.order(ByteOrder.LITTLE_ENDIAN); return (obj.__assign(_bb.getInt(_bb.position()) + _bb.position(), _bb)); } - public void __init(int _i, ByteBuffer _bb) { __reset(_i, _bb); } - public Null __assign(int _i, ByteBuffer _bb) { __init(_i, _bb); return this; } - - - public static void startNull(FlatBufferBuilder builder) { builder.startTable(0); } - public static int endNull(FlatBufferBuilder builder) { - int o = builder.endTable(); - return o; - } - - public static final class Vector extends BaseVector { - public Vector __assign(int _vector, int _element_size, ByteBuffer _bb) { __reset(_vector, _element_size, _bb); return this; } - - public Null get(int j) { return get(new Null(), j); } - public Null get(Null obj, int j) { return obj.__assign(__indirect(__element(j), bb), bb); } - } -} - diff --git a/core/src/main/java/io/github/dfa1/vortex/fbs/PType.java b/core/src/main/java/io/github/dfa1/vortex/fbs/PType.java deleted file mode 100644 index b062d91..0000000 --- a/core/src/main/java/io/github/dfa1/vortex/fbs/PType.java +++ /dev/null @@ -1,24 +0,0 @@ -// automatically generated by the FlatBuffers compiler, do not modify - -package io.github.dfa1.vortex.fbs; - -@SuppressWarnings("unused") -public final class PType { - private PType() { } - public static final int U8 = 0; - public static final int U16 = 1; - public static final int U32 = 2; - public static final int U64 = 3; - public static final int I8 = 4; - public static final int I16 = 5; - public static final int I32 = 6; - public static final int I64 = 7; - public static final int F16 = 8; - public static final int F32 = 9; - public static final int F64 = 10; - - public static final String[] names = { "U8", "U16", "U32", "U64", "I8", "I16", "I32", "I64", "F16", "F32", "F64", }; - - public static String name(int e) { return names[e]; } -} - diff --git a/core/src/main/java/io/github/dfa1/vortex/fbs/Postscript.java b/core/src/main/java/io/github/dfa1/vortex/fbs/Postscript.java deleted file mode 100644 index bab660d..0000000 --- a/core/src/main/java/io/github/dfa1/vortex/fbs/Postscript.java +++ /dev/null @@ -1,98 +0,0 @@ -// automatically generated by the FlatBuffers compiler, do not modify - -package io.github.dfa1.vortex.fbs; - -import com.google.flatbuffers.BaseVector; -import com.google.flatbuffers.BooleanVector; -import com.google.flatbuffers.ByteVector; -import com.google.flatbuffers.Constants; -import com.google.flatbuffers.DoubleVector; -import com.google.flatbuffers.FlatBufferBuilder; -import com.google.flatbuffers.FloatVector; -import com.google.flatbuffers.IntVector; -import com.google.flatbuffers.LongVector; -import com.google.flatbuffers.ShortVector; -import com.google.flatbuffers.StringVector; -import com.google.flatbuffers.Struct; -import com.google.flatbuffers.UnionVector; -import java.nio.ByteBuffer; -import java.nio.ByteOrder; - -/** - * The `Postscript` is guaranteed by the file format to never exceed - * 65528 bytes (i.e., u16::MAX - 8 bytes) in length, and is immediately - * followed by an 8-byte `EndOfFile` struct. - * - * An initial read of a Vortex file defaults to at least 64KB (u16::MAX bytes) and therefore - * is guaranteed to cover at least the Postscript. - * - * The reason for a postscript at all is to ensure minimal but all necessary footer information - * can be read in two round trips. Since the DType is optional and possibly large, it lives in - * its own segment. If the footer were arbitrary size, with a pointer to the DType segment, then - * in the worst case we would need one round trip to read the footer length, one to read the full - * footer and parse the DType offset, and a third to fetch the DType segment. - * - * The segments pointed to by the postscript have inline compression and encryption specs to avoid - * the need to fetch encryption schemes up-front. - */ -@SuppressWarnings("unused") -public final class Postscript extends com.google.flatbuffers.Table { - public static Postscript getRootAsPostscript(ByteBuffer _bb) { return getRootAsPostscript(_bb, new Postscript()); } - public static Postscript getRootAsPostscript(ByteBuffer _bb, Postscript obj) { _bb.order(ByteOrder.LITTLE_ENDIAN); return (obj.__assign(_bb.getInt(_bb.position()) + _bb.position(), _bb)); } - public void __init(int _i, ByteBuffer _bb) { __reset(_i, _bb); } - public Postscript __assign(int _i, ByteBuffer _bb) { __init(_i, _bb); return this; } - - /** - * Segment containing the root `DType` flatbuffer. - */ - public io.github.dfa1.vortex.fbs.PostscriptSegment dtype() { return dtype(new io.github.dfa1.vortex.fbs.PostscriptSegment()); } - public io.github.dfa1.vortex.fbs.PostscriptSegment dtype(io.github.dfa1.vortex.fbs.PostscriptSegment obj) { int o = __offset(4); return o != 0 ? obj.__assign(__indirect(o + bb_pos), bb) : null; } - /** - * Segment containing the root `Layout` flatbuffer (required). - */ - public io.github.dfa1.vortex.fbs.PostscriptSegment layout() { return layout(new io.github.dfa1.vortex.fbs.PostscriptSegment()); } - public io.github.dfa1.vortex.fbs.PostscriptSegment layout(io.github.dfa1.vortex.fbs.PostscriptSegment obj) { int o = __offset(6); return o != 0 ? obj.__assign(__indirect(o + bb_pos), bb) : null; } - /** - * Segment containing the file-level `Statistics` flatbuffer. - */ - public io.github.dfa1.vortex.fbs.PostscriptSegment statistics() { return statistics(new io.github.dfa1.vortex.fbs.PostscriptSegment()); } - public io.github.dfa1.vortex.fbs.PostscriptSegment statistics(io.github.dfa1.vortex.fbs.PostscriptSegment obj) { int o = __offset(8); return o != 0 ? obj.__assign(__indirect(o + bb_pos), bb) : null; } - /** - * Segment containing the 'Footer' flatbuffer (required) - */ - public io.github.dfa1.vortex.fbs.PostscriptSegment footer() { return footer(new io.github.dfa1.vortex.fbs.PostscriptSegment()); } - public io.github.dfa1.vortex.fbs.PostscriptSegment footer(io.github.dfa1.vortex.fbs.PostscriptSegment obj) { int o = __offset(10); return o != 0 ? obj.__assign(__indirect(o + bb_pos), bb) : null; } - - public static int createPostscript(FlatBufferBuilder builder, - int dtypeOffset, - int layoutOffset, - int statisticsOffset, - int footerOffset) { - builder.startTable(4); - Postscript.addFooter(builder, footerOffset); - Postscript.addStatistics(builder, statisticsOffset); - Postscript.addLayout(builder, layoutOffset); - Postscript.addDtype(builder, dtypeOffset); - return Postscript.endPostscript(builder); - } - - public static void startPostscript(FlatBufferBuilder builder) { builder.startTable(4); } - public static void addDtype(FlatBufferBuilder builder, int dtypeOffset) { builder.addOffset(0, dtypeOffset, 0); } - public static void addLayout(FlatBufferBuilder builder, int layoutOffset) { builder.addOffset(1, layoutOffset, 0); } - public static void addStatistics(FlatBufferBuilder builder, int statisticsOffset) { builder.addOffset(2, statisticsOffset, 0); } - public static void addFooter(FlatBufferBuilder builder, int footerOffset) { builder.addOffset(3, footerOffset, 0); } - public static int endPostscript(FlatBufferBuilder builder) { - int o = builder.endTable(); - return o; - } - public static void finishPostscriptBuffer(FlatBufferBuilder builder, int offset) { builder.finish(offset); } - public static void finishSizePrefixedPostscriptBuffer(FlatBufferBuilder builder, int offset) { builder.finishSizePrefixed(offset); } - - public static final class Vector extends BaseVector { - public Vector __assign(int _vector, int _element_size, ByteBuffer _bb) { __reset(_vector, _element_size, _bb); return this; } - - public Postscript get(int j) { return get(new Postscript(), j); } - public Postscript get(Postscript obj, int j) { return obj.__assign(__indirect(__element(j), bb), bb); } - } -} - diff --git a/core/src/main/java/io/github/dfa1/vortex/fbs/PostscriptSegment.java b/core/src/main/java/io/github/dfa1/vortex/fbs/PostscriptSegment.java deleted file mode 100644 index ba515ce..0000000 --- a/core/src/main/java/io/github/dfa1/vortex/fbs/PostscriptSegment.java +++ /dev/null @@ -1,73 +0,0 @@ -// automatically generated by the FlatBuffers compiler, do not modify - -package io.github.dfa1.vortex.fbs; - -import com.google.flatbuffers.BaseVector; -import com.google.flatbuffers.BooleanVector; -import com.google.flatbuffers.ByteVector; -import com.google.flatbuffers.Constants; -import com.google.flatbuffers.DoubleVector; -import com.google.flatbuffers.FlatBufferBuilder; -import com.google.flatbuffers.FloatVector; -import com.google.flatbuffers.IntVector; -import com.google.flatbuffers.LongVector; -import com.google.flatbuffers.ShortVector; -import com.google.flatbuffers.StringVector; -import com.google.flatbuffers.Struct; -import com.google.flatbuffers.UnionVector; -import java.nio.ByteBuffer; -import java.nio.ByteOrder; - -/** - * A `PostscriptSegment` describes the location of a segment in the file without referencing any - * specification objects. That is, encryption and compression are defined inline. - */ -@SuppressWarnings("unused") -public final class PostscriptSegment extends com.google.flatbuffers.Table { - public static PostscriptSegment getRootAsPostscriptSegment(ByteBuffer _bb) { return getRootAsPostscriptSegment(_bb, new PostscriptSegment()); } - public static PostscriptSegment getRootAsPostscriptSegment(ByteBuffer _bb, PostscriptSegment obj) { _bb.order(ByteOrder.LITTLE_ENDIAN); return (obj.__assign(_bb.getInt(_bb.position()) + _bb.position(), _bb)); } - public void __init(int _i, ByteBuffer _bb) { __reset(_i, _bb); } - public PostscriptSegment __assign(int _i, ByteBuffer _bb) { __init(_i, _bb); return this; } - - public long offset() { int o = __offset(4); return o != 0 ? bb.getLong(o + bb_pos) : 0L; } - public long length() { int o = __offset(6); return o != 0 ? (long)bb.getInt(o + bb_pos) & 0xFFFFFFFFL : 0L; } - public int alignmentExponent() { int o = __offset(8); return o != 0 ? bb.get(o + bb_pos) & 0xFF : 0; } - public io.github.dfa1.vortex.fbs.CompressionSpec _Compression() { return _Compression(new io.github.dfa1.vortex.fbs.CompressionSpec()); } - public io.github.dfa1.vortex.fbs.CompressionSpec _Compression(io.github.dfa1.vortex.fbs.CompressionSpec obj) { int o = __offset(10); return o != 0 ? obj.__assign(__indirect(o + bb_pos), bb) : null; } - public io.github.dfa1.vortex.fbs.EncryptionSpec _Encryption() { return _Encryption(new io.github.dfa1.vortex.fbs.EncryptionSpec()); } - public io.github.dfa1.vortex.fbs.EncryptionSpec _Encryption(io.github.dfa1.vortex.fbs.EncryptionSpec obj) { int o = __offset(12); return o != 0 ? obj.__assign(__indirect(o + bb_pos), bb) : null; } - - public static int createPostscriptSegment(FlatBufferBuilder builder, - long offset, - long length, - int alignmentExponent, - int _CompressionOffset, - int _EncryptionOffset) { - builder.startTable(5); - PostscriptSegment.addOffset(builder, offset); - PostscriptSegment.add_encryption(builder, _EncryptionOffset); - PostscriptSegment.add_compression(builder, _CompressionOffset); - PostscriptSegment.addLength(builder, length); - PostscriptSegment.addAlignmentExponent(builder, alignmentExponent); - return PostscriptSegment.endPostscriptSegment(builder); - } - - public static void startPostscriptSegment(FlatBufferBuilder builder) { builder.startTable(5); } - public static void addOffset(FlatBufferBuilder builder, long offset) { builder.addLong(0, offset, 0L); } - public static void addLength(FlatBufferBuilder builder, long length) { builder.addInt(1, (int) length, (int) 0L); } - public static void addAlignmentExponent(FlatBufferBuilder builder, int alignmentExponent) { builder.addByte(2, (byte) alignmentExponent, (byte) 0); } - public static void add_compression(FlatBufferBuilder builder, int _CompressionOffset) { builder.addOffset(3, _CompressionOffset, 0); } - public static void add_encryption(FlatBufferBuilder builder, int _EncryptionOffset) { builder.addOffset(4, _EncryptionOffset, 0); } - public static int endPostscriptSegment(FlatBufferBuilder builder) { - int o = builder.endTable(); - return o; - } - - public static final class Vector extends BaseVector { - public Vector __assign(int _vector, int _element_size, ByteBuffer _bb) { __reset(_vector, _element_size, _bb); return this; } - - public PostscriptSegment get(int j) { return get(new PostscriptSegment(), j); } - public PostscriptSegment get(PostscriptSegment obj, int j) { return obj.__assign(__indirect(__element(j), bb), bb); } - } -} - diff --git a/core/src/main/java/io/github/dfa1/vortex/fbs/Precision.java b/core/src/main/java/io/github/dfa1/vortex/fbs/Precision.java deleted file mode 100644 index 759e586..0000000 --- a/core/src/main/java/io/github/dfa1/vortex/fbs/Precision.java +++ /dev/null @@ -1,15 +0,0 @@ -// automatically generated by the FlatBuffers compiler, do not modify - -package io.github.dfa1.vortex.fbs; - -@SuppressWarnings("unused") -public final class Precision { - private Precision() { } - public static final int Inexact = 0; - public static final int Exact = 1; - - public static final String[] names = { "Inexact", "Exact", }; - - public static String name(int e) { return names[e]; } -} - diff --git a/core/src/main/java/io/github/dfa1/vortex/fbs/Primitive.java b/core/src/main/java/io/github/dfa1/vortex/fbs/Primitive.java deleted file mode 100644 index 3008f55..0000000 --- a/core/src/main/java/io/github/dfa1/vortex/fbs/Primitive.java +++ /dev/null @@ -1,55 +0,0 @@ -// automatically generated by the FlatBuffers compiler, do not modify - -package io.github.dfa1.vortex.fbs; - -import com.google.flatbuffers.BaseVector; -import com.google.flatbuffers.BooleanVector; -import com.google.flatbuffers.ByteVector; -import com.google.flatbuffers.Constants; -import com.google.flatbuffers.DoubleVector; -import com.google.flatbuffers.FlatBufferBuilder; -import com.google.flatbuffers.FloatVector; -import com.google.flatbuffers.IntVector; -import com.google.flatbuffers.LongVector; -import com.google.flatbuffers.ShortVector; -import com.google.flatbuffers.StringVector; -import com.google.flatbuffers.Struct; -import com.google.flatbuffers.UnionVector; -import java.nio.ByteBuffer; -import java.nio.ByteOrder; - -@SuppressWarnings("unused") -public final class Primitive extends com.google.flatbuffers.Table { - public static Primitive getRootAsPrimitive(ByteBuffer _bb) { return getRootAsPrimitive(_bb, new Primitive()); } - public static Primitive getRootAsPrimitive(ByteBuffer _bb, Primitive obj) { _bb.order(ByteOrder.LITTLE_ENDIAN); return (obj.__assign(_bb.getInt(_bb.position()) + _bb.position(), _bb)); } - public void __init(int _i, ByteBuffer _bb) { __reset(_i, _bb); } - public Primitive __assign(int _i, ByteBuffer _bb) { __init(_i, _bb); return this; } - - public int ptype() { int o = __offset(4); return o != 0 ? bb.get(o + bb_pos) & 0xFF : 0; } - public boolean nullable() { int o = __offset(6); return o != 0 ? 0!=bb.get(o + bb_pos) : false; } - - public static int createPrimitive(FlatBufferBuilder builder, - int ptype, - boolean nullable) { - builder.startTable(2); - Primitive.addNullable(builder, nullable); - Primitive.addPtype(builder, ptype); - return Primitive.endPrimitive(builder); - } - - public static void startPrimitive(FlatBufferBuilder builder) { builder.startTable(2); } - public static void addPtype(FlatBufferBuilder builder, int ptype) { builder.addByte(0, (byte) ptype, (byte) 0); } - public static void addNullable(FlatBufferBuilder builder, boolean nullable) { builder.addBoolean(1, nullable, false); } - public static int endPrimitive(FlatBufferBuilder builder) { - int o = builder.endTable(); - return o; - } - - public static final class Vector extends BaseVector { - public Vector __assign(int _vector, int _element_size, ByteBuffer _bb) { __reset(_vector, _element_size, _bb); return this; } - - public Primitive get(int j) { return get(new Primitive(), j); } - public Primitive get(Primitive obj, int j) { return obj.__assign(__indirect(__element(j), bb), bb); } - } -} - diff --git a/core/src/main/java/io/github/dfa1/vortex/fbs/SegmentSpec.java b/core/src/main/java/io/github/dfa1/vortex/fbs/SegmentSpec.java deleted file mode 100644 index 0ba4e7f..0000000 --- a/core/src/main/java/io/github/dfa1/vortex/fbs/SegmentSpec.java +++ /dev/null @@ -1,61 +0,0 @@ -// automatically generated by the FlatBuffers compiler, do not modify - -package io.github.dfa1.vortex.fbs; - -import com.google.flatbuffers.BaseVector; -import com.google.flatbuffers.BooleanVector; -import com.google.flatbuffers.ByteVector; -import com.google.flatbuffers.Constants; -import com.google.flatbuffers.DoubleVector; -import com.google.flatbuffers.FlatBufferBuilder; -import com.google.flatbuffers.FloatVector; -import com.google.flatbuffers.IntVector; -import com.google.flatbuffers.LongVector; -import com.google.flatbuffers.ShortVector; -import com.google.flatbuffers.StringVector; -import com.google.flatbuffers.Struct; -import com.google.flatbuffers.UnionVector; -import java.nio.ByteBuffer; -import java.nio.ByteOrder; - -/** - * A `SegmentSpec` acts as the locator for a buffer within the file. - */ -@SuppressWarnings("unused") -public final class SegmentSpec extends Struct { - public void __init(int _i, ByteBuffer _bb) { __reset(_i, _bb); } - public SegmentSpec __assign(int _i, ByteBuffer _bb) { __init(_i, _bb); return this; } - - /** - * Offset relative to the start of the file. - */ - public long offset() { return bb.getLong(bb_pos + 0); } - /** - * Length in bytes of the segment. - */ - public long length() { return (long)bb.getInt(bb_pos + 8) & 0xFFFFFFFFL; } - /** - * Base-2 exponent of the alignment of the segment. - */ - public int alignmentExponent() { return bb.get(bb_pos + 12) & 0xFF; } - public int _Compression() { return bb.get(bb_pos + 13) & 0xFF; } - public int _Encryption() { return bb.getShort(bb_pos + 14) & 0xFFFF; } - - public static int createSegmentSpec(FlatBufferBuilder builder, long offset, long length, int alignmentExponent, int _Compression, int _Encryption) { - builder.prep(8, 16); - builder.putShort((short) _Encryption); - builder.putByte((byte) _Compression); - builder.putByte((byte) alignmentExponent); - builder.putInt((int) length); - builder.putLong(offset); - return builder.offset(); - } - - public static final class Vector extends BaseVector { - public Vector __assign(int _vector, int _element_size, ByteBuffer _bb) { __reset(_vector, _element_size, _bb); return this; } - - public SegmentSpec get(int j) { return get(new SegmentSpec(), j); } - public SegmentSpec get(SegmentSpec obj, int j) { return obj.__assign(__element(j), bb); } - } -} - diff --git a/core/src/main/java/io/github/dfa1/vortex/fbs/Struct_.java b/core/src/main/java/io/github/dfa1/vortex/fbs/Struct_.java deleted file mode 100644 index 3a51f59..0000000 --- a/core/src/main/java/io/github/dfa1/vortex/fbs/Struct_.java +++ /dev/null @@ -1,70 +0,0 @@ -// automatically generated by the FlatBuffers compiler, do not modify - -package io.github.dfa1.vortex.fbs; - -import com.google.flatbuffers.BaseVector; -import com.google.flatbuffers.BooleanVector; -import com.google.flatbuffers.ByteVector; -import com.google.flatbuffers.Constants; -import com.google.flatbuffers.DoubleVector; -import com.google.flatbuffers.FlatBufferBuilder; -import com.google.flatbuffers.FloatVector; -import com.google.flatbuffers.IntVector; -import com.google.flatbuffers.LongVector; -import com.google.flatbuffers.ShortVector; -import com.google.flatbuffers.StringVector; -import com.google.flatbuffers.Struct; -import com.google.flatbuffers.UnionVector; -import java.nio.ByteBuffer; -import java.nio.ByteOrder; - -@SuppressWarnings("unused") -public final class Struct_ extends com.google.flatbuffers.Table { - public static Struct_ getRootAsStruct_(ByteBuffer _bb) { return getRootAsStruct_(_bb, new Struct_()); } - public static Struct_ getRootAsStruct_(ByteBuffer _bb, Struct_ obj) { _bb.order(ByteOrder.LITTLE_ENDIAN); return (obj.__assign(_bb.getInt(_bb.position()) + _bb.position(), _bb)); } - public void __init(int _i, ByteBuffer _bb) { __reset(_i, _bb); } - public Struct_ __assign(int _i, ByteBuffer _bb) { __init(_i, _bb); return this; } - - public String names(int j) { int o = __offset(4); return o != 0 ? __string(__vector(o) + j * 4) : null; } - public int namesLength() { int o = __offset(4); return o != 0 ? __vector_len(o) : 0; } - public StringVector namesVector() { return namesVector(new StringVector()); } - public StringVector namesVector(StringVector obj) { int o = __offset(4); return o != 0 ? obj.__assign(__vector(o), 4, bb) : null; } - public io.github.dfa1.vortex.fbs.DType dtypes(int j) { return dtypes(new io.github.dfa1.vortex.fbs.DType(), j); } - public io.github.dfa1.vortex.fbs.DType dtypes(io.github.dfa1.vortex.fbs.DType obj, int j) { int o = __offset(6); return o != 0 ? obj.__assign(__indirect(__vector(o) + j * 4), bb) : null; } - public int dtypesLength() { int o = __offset(6); return o != 0 ? __vector_len(o) : 0; } - public io.github.dfa1.vortex.fbs.DType.Vector dtypesVector() { return dtypesVector(new io.github.dfa1.vortex.fbs.DType.Vector()); } - public io.github.dfa1.vortex.fbs.DType.Vector dtypesVector(io.github.dfa1.vortex.fbs.DType.Vector obj) { int o = __offset(6); return o != 0 ? obj.__assign(__vector(o), 4, bb) : null; } - public boolean nullable() { int o = __offset(8); return o != 0 ? 0!=bb.get(o + bb_pos) : false; } - - public static int createStruct_(FlatBufferBuilder builder, - int namesOffset, - int dtypesOffset, - boolean nullable) { - builder.startTable(3); - Struct_.addDtypes(builder, dtypesOffset); - Struct_.addNames(builder, namesOffset); - Struct_.addNullable(builder, nullable); - return Struct_.endStruct_(builder); - } - - public static void startStruct_(FlatBufferBuilder builder) { builder.startTable(3); } - public static void addNames(FlatBufferBuilder builder, int namesOffset) { builder.addOffset(0, namesOffset, 0); } - public static int createNamesVector(FlatBufferBuilder builder, int[] data) { builder.startVector(4, data.length, 4); for (int i = data.length - 1; i >= 0; i--) builder.addOffset(data[i]); return builder.endVector(); } - public static void startNamesVector(FlatBufferBuilder builder, int numElems) { builder.startVector(4, numElems, 4); } - public static void addDtypes(FlatBufferBuilder builder, int dtypesOffset) { builder.addOffset(1, dtypesOffset, 0); } - public static int createDtypesVector(FlatBufferBuilder builder, int[] data) { builder.startVector(4, data.length, 4); for (int i = data.length - 1; i >= 0; i--) builder.addOffset(data[i]); return builder.endVector(); } - public static void startDtypesVector(FlatBufferBuilder builder, int numElems) { builder.startVector(4, numElems, 4); } - public static void addNullable(FlatBufferBuilder builder, boolean nullable) { builder.addBoolean(2, nullable, false); } - public static int endStruct_(FlatBufferBuilder builder) { - int o = builder.endTable(); - return o; - } - - public static final class Vector extends BaseVector { - public Vector __assign(int _vector, int _element_size, ByteBuffer _bb) { __reset(_vector, _element_size, _bb); return this; } - - public Struct_ get(int j) { return get(new Struct_(), j); } - public Struct_ get(Struct_ obj, int j) { return obj.__assign(__indirect(__element(j), bb), bb); } - } -} - diff --git a/core/src/main/java/io/github/dfa1/vortex/fbs/Type.java b/core/src/main/java/io/github/dfa1/vortex/fbs/Type.java deleted file mode 100644 index 39c1e77..0000000 --- a/core/src/main/java/io/github/dfa1/vortex/fbs/Type.java +++ /dev/null @@ -1,26 +0,0 @@ -// automatically generated by the FlatBuffers compiler, do not modify - -package io.github.dfa1.vortex.fbs; - -@SuppressWarnings("unused") -public final class Type { - private Type() { } - public static final byte NONE = 0; - public static final byte Null = 1; - public static final byte Bool = 2; - public static final byte Primitive = 3; - public static final byte Decimal = 4; - public static final byte Utf8 = 5; - public static final byte Binary = 6; - public static final byte Struct_ = 7; - public static final byte List = 8; - public static final byte Extension = 9; - public static final byte FixedSizeList = 10; - public static final byte Variant = 11; - public static final byte Union = 12; - - public static final String[] names = { "NONE", "Null", "Bool", "Primitive", "Decimal", "Utf8", "Binary", "Struct_", "List", "Extension", "FixedSizeList", "Variant", "Union", }; - - public static String name(int e) { return names[e]; } -} - diff --git a/core/src/main/java/io/github/dfa1/vortex/fbs/Union.java b/core/src/main/java/io/github/dfa1/vortex/fbs/Union.java deleted file mode 100644 index 96a67f8..0000000 --- a/core/src/main/java/io/github/dfa1/vortex/fbs/Union.java +++ /dev/null @@ -1,51 +0,0 @@ -// automatically generated by the FlatBuffers compiler, do not modify - -package io.github.dfa1.vortex.fbs; - -import com.google.flatbuffers.BaseVector; -import com.google.flatbuffers.BooleanVector; -import com.google.flatbuffers.ByteVector; -import com.google.flatbuffers.Constants; -import com.google.flatbuffers.DoubleVector; -import com.google.flatbuffers.FlatBufferBuilder; -import com.google.flatbuffers.FloatVector; -import com.google.flatbuffers.IntVector; -import com.google.flatbuffers.LongVector; -import com.google.flatbuffers.ShortVector; -import com.google.flatbuffers.StringVector; -import com.google.flatbuffers.Struct; -import com.google.flatbuffers.UnionVector; -import java.nio.ByteBuffer; -import java.nio.ByteOrder; - -@SuppressWarnings("unused") -public final class Union extends com.google.flatbuffers.Table { - public static Union getRootAsUnion(ByteBuffer _bb) { return getRootAsUnion(_bb, new Union()); } - public static Union getRootAsUnion(ByteBuffer _bb, Union obj) { _bb.order(ByteOrder.LITTLE_ENDIAN); return (obj.__assign(_bb.getInt(_bb.position()) + _bb.position(), _bb)); } - public void __init(int _i, ByteBuffer _bb) { __reset(_i, _bb); } - public Union __assign(int _i, ByteBuffer _bb) { __init(_i, _bb); return this; } - - public boolean nullable() { int o = __offset(4); return o != 0 ? 0!=bb.get(o + bb_pos) : false; } - - public static int createUnion(FlatBufferBuilder builder, - boolean nullable) { - builder.startTable(1); - Union.addNullable(builder, nullable); - return Union.endUnion(builder); - } - - public static void startUnion(FlatBufferBuilder builder) { builder.startTable(1); } - public static void addNullable(FlatBufferBuilder builder, boolean nullable) { builder.addBoolean(0, nullable, false); } - public static int endUnion(FlatBufferBuilder builder) { - int o = builder.endTable(); - return o; - } - - public static final class Vector extends BaseVector { - public Vector __assign(int _vector, int _element_size, ByteBuffer _bb) { __reset(_vector, _element_size, _bb); return this; } - - public Union get(int j) { return get(new Union(), j); } - public Union get(Union obj, int j) { return obj.__assign(__indirect(__element(j), bb), bb); } - } -} - diff --git a/core/src/main/java/io/github/dfa1/vortex/fbs/Utf8.java b/core/src/main/java/io/github/dfa1/vortex/fbs/Utf8.java deleted file mode 100644 index 5049b86..0000000 --- a/core/src/main/java/io/github/dfa1/vortex/fbs/Utf8.java +++ /dev/null @@ -1,51 +0,0 @@ -// automatically generated by the FlatBuffers compiler, do not modify - -package io.github.dfa1.vortex.fbs; - -import com.google.flatbuffers.BaseVector; -import com.google.flatbuffers.BooleanVector; -import com.google.flatbuffers.ByteVector; -import com.google.flatbuffers.Constants; -import com.google.flatbuffers.DoubleVector; -import com.google.flatbuffers.FlatBufferBuilder; -import com.google.flatbuffers.FloatVector; -import com.google.flatbuffers.IntVector; -import com.google.flatbuffers.LongVector; -import com.google.flatbuffers.ShortVector; -import com.google.flatbuffers.StringVector; -import com.google.flatbuffers.Struct; -import com.google.flatbuffers.UnionVector; -import java.nio.ByteBuffer; -import java.nio.ByteOrder; - -@SuppressWarnings("unused") -public final class Utf8 extends com.google.flatbuffers.Table { - public static Utf8 getRootAsUtf8(ByteBuffer _bb) { return getRootAsUtf8(_bb, new Utf8()); } - public static Utf8 getRootAsUtf8(ByteBuffer _bb, Utf8 obj) { _bb.order(ByteOrder.LITTLE_ENDIAN); return (obj.__assign(_bb.getInt(_bb.position()) + _bb.position(), _bb)); } - public void __init(int _i, ByteBuffer _bb) { __reset(_i, _bb); } - public Utf8 __assign(int _i, ByteBuffer _bb) { __init(_i, _bb); return this; } - - public boolean nullable() { int o = __offset(4); return o != 0 ? 0!=bb.get(o + bb_pos) : false; } - - public static int createUtf8(FlatBufferBuilder builder, - boolean nullable) { - builder.startTable(1); - Utf8.addNullable(builder, nullable); - return Utf8.endUtf8(builder); - } - - public static void startUtf8(FlatBufferBuilder builder) { builder.startTable(1); } - public static void addNullable(FlatBufferBuilder builder, boolean nullable) { builder.addBoolean(0, nullable, false); } - public static int endUtf8(FlatBufferBuilder builder) { - int o = builder.endTable(); - return o; - } - - public static final class Vector extends BaseVector { - public Vector __assign(int _vector, int _element_size, ByteBuffer _bb) { __reset(_vector, _element_size, _bb); return this; } - - public Utf8 get(int j) { return get(new Utf8(), j); } - public Utf8 get(Utf8 obj, int j) { return obj.__assign(__indirect(__element(j), bb), bb); } - } -} - diff --git a/core/src/main/java/io/github/dfa1/vortex/fbs/Variant.java b/core/src/main/java/io/github/dfa1/vortex/fbs/Variant.java deleted file mode 100644 index 6b4e6bf..0000000 --- a/core/src/main/java/io/github/dfa1/vortex/fbs/Variant.java +++ /dev/null @@ -1,51 +0,0 @@ -// automatically generated by the FlatBuffers compiler, do not modify - -package io.github.dfa1.vortex.fbs; - -import com.google.flatbuffers.BaseVector; -import com.google.flatbuffers.BooleanVector; -import com.google.flatbuffers.ByteVector; -import com.google.flatbuffers.Constants; -import com.google.flatbuffers.DoubleVector; -import com.google.flatbuffers.FlatBufferBuilder; -import com.google.flatbuffers.FloatVector; -import com.google.flatbuffers.IntVector; -import com.google.flatbuffers.LongVector; -import com.google.flatbuffers.ShortVector; -import com.google.flatbuffers.StringVector; -import com.google.flatbuffers.Struct; -import com.google.flatbuffers.UnionVector; -import java.nio.ByteBuffer; -import java.nio.ByteOrder; - -@SuppressWarnings("unused") -public final class Variant extends com.google.flatbuffers.Table { - public static Variant getRootAsVariant(ByteBuffer _bb) { return getRootAsVariant(_bb, new Variant()); } - public static Variant getRootAsVariant(ByteBuffer _bb, Variant obj) { _bb.order(ByteOrder.LITTLE_ENDIAN); return (obj.__assign(_bb.getInt(_bb.position()) + _bb.position(), _bb)); } - public void __init(int _i, ByteBuffer _bb) { __reset(_i, _bb); } - public Variant __assign(int _i, ByteBuffer _bb) { __init(_i, _bb); return this; } - - public boolean nullable() { int o = __offset(4); return o != 0 ? 0!=bb.get(o + bb_pos) : false; } - - public static int createVariant(FlatBufferBuilder builder, - boolean nullable) { - builder.startTable(1); - Variant.addNullable(builder, nullable); - return Variant.endVariant(builder); - } - - public static void startVariant(FlatBufferBuilder builder) { builder.startTable(1); } - public static void addNullable(FlatBufferBuilder builder, boolean nullable) { builder.addBoolean(0, nullable, false); } - public static int endVariant(FlatBufferBuilder builder) { - int o = builder.endTable(); - return o; - } - - public static final class Vector extends BaseVector { - public Vector __assign(int _vector, int _element_size, ByteBuffer _bb) { __reset(_vector, _element_size, _bb); return this; } - - public Variant get(int j) { return get(new Variant(), j); } - public Variant get(Variant obj, int j) { return obj.__assign(__indirect(__element(j), bb), bb); } - } -} - diff --git a/core/src/main/java/io/github/dfa1/vortex/proto/DTypeProtos.java b/core/src/main/java/io/github/dfa1/vortex/proto/DTypeProtos.java deleted file mode 100644 index 58dce6c..0000000 --- a/core/src/main/java/io/github/dfa1/vortex/proto/DTypeProtos.java +++ /dev/null @@ -1,11971 +0,0 @@ -// Generated by the protocol buffer compiler. DO NOT EDIT! -// NO CHECKED-IN PROTOBUF GENCODE -// source: dtype.proto -// Protobuf Java Version: 4.35.0 - -package io.github.dfa1.vortex.proto; - -@com.google.protobuf.Generated -public final class DTypeProtos extends com.google.protobuf.GeneratedFile { - private DTypeProtos() {} - static { - com.google.protobuf.RuntimeVersion.validateProtobufGencodeVersion( - com.google.protobuf.RuntimeVersion.RuntimeDomain.PUBLIC, - /* major= */ 4, - /* minor= */ 35, - /* patch= */ 0, - /* suffix= */ "", - "DTypeProtos"); - } - public static void registerAllExtensions( - com.google.protobuf.ExtensionRegistryLite registry) { - } - - public static void registerAllExtensions( - com.google.protobuf.ExtensionRegistry registry) { - registerAllExtensions( - (com.google.protobuf.ExtensionRegistryLite) registry); - } - /** - * Protobuf enum {@code vortex.dtype.PType} - */ - public enum PType - implements com.google.protobuf.ProtocolMessageEnum { - /** - * U8 = 0; - */ - U8(0), - /** - * U16 = 1; - */ - U16(1), - /** - * U32 = 2; - */ - U32(2), - /** - * U64 = 3; - */ - U64(3), - /** - * I8 = 4; - */ - I8(4), - /** - * I16 = 5; - */ - I16(5), - /** - * I32 = 6; - */ - I32(6), - /** - * I64 = 7; - */ - I64(7), - /** - * F16 = 8; - */ - F16(8), - /** - * F32 = 9; - */ - F32(9), - /** - * F64 = 10; - */ - F64(10), - UNRECOGNIZED(-1), - ; - - static { - com.google.protobuf.RuntimeVersion.validateProtobufGencodeVersion( - com.google.protobuf.RuntimeVersion.RuntimeDomain.PUBLIC, - /* major= */ 4, - /* minor= */ 35, - /* patch= */ 0, - /* suffix= */ "", - "PType"); - } - /** - * U8 = 0; - */ - public static final int U8_VALUE = 0; - /** - * U16 = 1; - */ - public static final int U16_VALUE = 1; - /** - * U32 = 2; - */ - public static final int U32_VALUE = 2; - /** - * U64 = 3; - */ - public static final int U64_VALUE = 3; - /** - * I8 = 4; - */ - public static final int I8_VALUE = 4; - /** - * I16 = 5; - */ - public static final int I16_VALUE = 5; - /** - * I32 = 6; - */ - public static final int I32_VALUE = 6; - /** - * I64 = 7; - */ - public static final int I64_VALUE = 7; - /** - * F16 = 8; - */ - public static final int F16_VALUE = 8; - /** - * F32 = 9; - */ - public static final int F32_VALUE = 9; - /** - * F64 = 10; - */ - public static final int F64_VALUE = 10; - - - public final int getNumber() { - if (this == UNRECOGNIZED) { - throw new java.lang.IllegalArgumentException( - "Can't get the number of an unknown enum value."); - } - return value; - } - - /** - * @param value The numeric wire value of the corresponding enum entry. - * @return The enum associated with the given numeric wire value. - * @deprecated Use {@link #forNumber(int)} instead. - */ - @java.lang.Deprecated - public static PType valueOf(int value) { - return forNumber(value); - } - - /** - * @param value The numeric wire value of the corresponding enum entry. - * @return The enum associated with the given numeric wire value. - */ - public static PType forNumber(int value) { - switch (value) { - case 0: return U8; - case 1: return U16; - case 2: return U32; - case 3: return U64; - case 4: return I8; - case 5: return I16; - case 6: return I32; - case 7: return I64; - case 8: return F16; - case 9: return F32; - case 10: return F64; - default: return null; - } - } - - public static com.google.protobuf.Internal.EnumLiteMap - internalGetValueMap() { - return internalValueMap; - } - private static final com.google.protobuf.Internal.EnumLiteMap< - PType> internalValueMap = - new com.google.protobuf.Internal.EnumLiteMap() { - public PType findValueByNumber(int number) { - return PType.forNumber(number); - } - }; - - public final com.google.protobuf.Descriptors.EnumValueDescriptor - getValueDescriptor() { - if (this == UNRECOGNIZED) { - throw new java.lang.IllegalStateException( - "Can't get the descriptor of an unrecognized enum value."); - } - return getDescriptor().getValue(ordinal()); - } - public final com.google.protobuf.Descriptors.EnumDescriptor - getDescriptorForType() { - return getDescriptor(); - } - public static com.google.protobuf.Descriptors.EnumDescriptor - getDescriptor() { - return io.github.dfa1.vortex.proto.DTypeProtos.getDescriptor().getEnumType(0); - } - - private static final PType[] VALUES = values(); - - public static PType valueOf( - com.google.protobuf.Descriptors.EnumValueDescriptor desc) { - if (desc.getType() != getDescriptor()) { - throw new java.lang.IllegalArgumentException( - "EnumValueDescriptor is not for this type."); - } - if (desc.getIndex() == -1) { - return UNRECOGNIZED; - } - return VALUES[desc.getIndex()]; - } - - private final int value; - - private PType(int value) { - this.value = value; - } - - // @@protoc_insertion_point(enum_scope:vortex.dtype.PType) - } - - public interface NullOrBuilder extends - // @@protoc_insertion_point(interface_extends:vortex.dtype.Null) - com.google.protobuf.MessageOrBuilder { - } - /** - * Protobuf type {@code vortex.dtype.Null} - */ - public static final class Null extends - com.google.protobuf.GeneratedMessage implements - // @@protoc_insertion_point(message_implements:vortex.dtype.Null) - NullOrBuilder { - private static final long serialVersionUID = 0L; - static { - com.google.protobuf.RuntimeVersion.validateProtobufGencodeVersion( - com.google.protobuf.RuntimeVersion.RuntimeDomain.PUBLIC, - /* major= */ 4, - /* minor= */ 35, - /* patch= */ 0, - /* suffix= */ "", - "Null"); - } - // Use Null.newBuilder() to construct. - private Null(com.google.protobuf.GeneratedMessage.Builder builder) { - super(builder); - } - private Null() { - } - - public static final com.google.protobuf.Descriptors.Descriptor - getDescriptor() { - return io.github.dfa1.vortex.proto.DTypeProtos.internal_static_vortex_dtype_Null_descriptor; - } - - @java.lang.Override - public com.google.protobuf.Descriptors.Descriptor getDescriptorForType() { - return io.github.dfa1.vortex.proto.DTypeProtos.internal_static_vortex_dtype_Null_descriptor; - } - - @java.lang.Override - protected com.google.protobuf.GeneratedMessage.FieldAccessorTable - internalGetFieldAccessorTable() { - return io.github.dfa1.vortex.proto.DTypeProtos.internal_static_vortex_dtype_Null_fieldAccessorTable - .ensureFieldAccessorsInitialized( - io.github.dfa1.vortex.proto.DTypeProtos.Null.class, io.github.dfa1.vortex.proto.DTypeProtos.Null.Builder.class); - } - - private byte memoizedIsInitialized = -1; - @java.lang.Override - public final boolean isInitialized() { - byte isInitialized = memoizedIsInitialized; - if (isInitialized == 1) return true; - if (isInitialized == 0) return false; - - memoizedIsInitialized = 1; - return true; - } - - @java.lang.Override - public void writeTo(com.google.protobuf.CodedOutputStream output) - throws java.io.IOException { - getUnknownFields().writeTo(output); - } - @java.lang.Override - public int getSerializedSize() { - int size = memoizedSize; - if (size != -1) return size; - - size = 0; - size += getUnknownFields().getSerializedSize(); - memoizedSize = size; - return size; - } - - @java.lang.Override - public boolean equals(final java.lang.Object obj) { - if (obj == this) { - return true; - } - if (!(obj instanceof io.github.dfa1.vortex.proto.DTypeProtos.Null)) { - return super.equals(obj); - } - io.github.dfa1.vortex.proto.DTypeProtos.Null other = (io.github.dfa1.vortex.proto.DTypeProtos.Null) obj; - - if (!getUnknownFields().equals(other.getUnknownFields())) return false; - return true; - } - - @java.lang.Override - public int hashCode() { - if (memoizedHashCode != 0) { - return memoizedHashCode; - } - int hash = 41; - hash = (19 * hash) + getDescriptor().hashCode(); - hash = (29 * hash) + getUnknownFields().hashCode(); - memoizedHashCode = hash; - return hash; - } - - public static io.github.dfa1.vortex.proto.DTypeProtos.Null parseFrom( - java.nio.ByteBuffer data) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data); - } - public static io.github.dfa1.vortex.proto.DTypeProtos.Null parseFrom( - java.nio.ByteBuffer data, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data, extensionRegistry); - } - public static io.github.dfa1.vortex.proto.DTypeProtos.Null parseFrom( - com.google.protobuf.ByteString data) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data); - } - public static io.github.dfa1.vortex.proto.DTypeProtos.Null parseFrom( - com.google.protobuf.ByteString data, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data, extensionRegistry); - } - public static io.github.dfa1.vortex.proto.DTypeProtos.Null parseFrom(byte[] data) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data); - } - public static io.github.dfa1.vortex.proto.DTypeProtos.Null parseFrom( - byte[] data, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data, extensionRegistry); - } - public static io.github.dfa1.vortex.proto.DTypeProtos.Null parseFrom(java.io.InputStream input) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseWithIOException(PARSER, input); - } - public static io.github.dfa1.vortex.proto.DTypeProtos.Null parseFrom( - java.io.InputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseWithIOException(PARSER, input, extensionRegistry); - } - - public static io.github.dfa1.vortex.proto.DTypeProtos.Null parseDelimitedFrom(java.io.InputStream input) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseDelimitedWithIOException(PARSER, input); - } - - public static io.github.dfa1.vortex.proto.DTypeProtos.Null parseDelimitedFrom( - java.io.InputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseDelimitedWithIOException(PARSER, input, extensionRegistry); - } - public static io.github.dfa1.vortex.proto.DTypeProtos.Null parseFrom( - com.google.protobuf.CodedInputStream input) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseWithIOException(PARSER, input); - } - public static io.github.dfa1.vortex.proto.DTypeProtos.Null parseFrom( - com.google.protobuf.CodedInputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseWithIOException(PARSER, input, extensionRegistry); - } - - @java.lang.Override - public Builder newBuilderForType() { return newBuilder(); } - public static Builder newBuilder() { - return DEFAULT_INSTANCE.toBuilder(); - } - public static Builder newBuilder(io.github.dfa1.vortex.proto.DTypeProtos.Null prototype) { - return DEFAULT_INSTANCE.toBuilder().mergeFrom(prototype); - } - @java.lang.Override - public Builder toBuilder() { - return this == DEFAULT_INSTANCE - ? new Builder() : new Builder().mergeFrom(this); - } - - @java.lang.Override - protected Builder newBuilderForType( - com.google.protobuf.GeneratedMessage.BuilderParent parent) { - Builder builder = new Builder(parent); - return builder; - } - /** - * Protobuf type {@code vortex.dtype.Null} - */ - public static final class Builder extends - com.google.protobuf.GeneratedMessage.Builder implements - // @@protoc_insertion_point(builder_implements:vortex.dtype.Null) - io.github.dfa1.vortex.proto.DTypeProtos.NullOrBuilder { - public static final com.google.protobuf.Descriptors.Descriptor - getDescriptor() { - return io.github.dfa1.vortex.proto.DTypeProtos.internal_static_vortex_dtype_Null_descriptor; - } - - @java.lang.Override - protected com.google.protobuf.GeneratedMessage.FieldAccessorTable - internalGetFieldAccessorTable() { - return io.github.dfa1.vortex.proto.DTypeProtos.internal_static_vortex_dtype_Null_fieldAccessorTable - .ensureFieldAccessorsInitialized( - io.github.dfa1.vortex.proto.DTypeProtos.Null.class, io.github.dfa1.vortex.proto.DTypeProtos.Null.Builder.class); - } - - // Construct using io.github.dfa1.vortex.proto.DTypeProtos.Null.newBuilder() - private Builder() { - - } - - private Builder( - com.google.protobuf.GeneratedMessage.BuilderParent parent) { - super(parent); - - } - @java.lang.Override - public Builder clear() { - super.clear(); - return this; - } - - @java.lang.Override - public com.google.protobuf.Descriptors.Descriptor - getDescriptorForType() { - return io.github.dfa1.vortex.proto.DTypeProtos.internal_static_vortex_dtype_Null_descriptor; - } - - @java.lang.Override - public io.github.dfa1.vortex.proto.DTypeProtos.Null getDefaultInstanceForType() { - return io.github.dfa1.vortex.proto.DTypeProtos.Null.getDefaultInstance(); - } - - @java.lang.Override - public io.github.dfa1.vortex.proto.DTypeProtos.Null build() { - io.github.dfa1.vortex.proto.DTypeProtos.Null result = buildPartial(); - if (!result.isInitialized()) { - throw newUninitializedMessageException(result); - } - return result; - } - - @java.lang.Override - public io.github.dfa1.vortex.proto.DTypeProtos.Null buildPartial() { - io.github.dfa1.vortex.proto.DTypeProtos.Null result = new io.github.dfa1.vortex.proto.DTypeProtos.Null(this); - onBuilt(); - return result; - } - - @java.lang.Override - public Builder mergeFrom(com.google.protobuf.Message other) { - if (other instanceof io.github.dfa1.vortex.proto.DTypeProtos.Null) { - return mergeFrom((io.github.dfa1.vortex.proto.DTypeProtos.Null)other); - } else { - super.mergeFrom(other); - return this; - } - } - - public Builder mergeFrom(io.github.dfa1.vortex.proto.DTypeProtos.Null other) { - if (other == io.github.dfa1.vortex.proto.DTypeProtos.Null.getDefaultInstance()) return this; - this.mergeUnknownFields(other.getUnknownFields()); - onChanged(); - return this; - } - - @java.lang.Override - public final boolean isInitialized() { - return true; - } - - @java.lang.Override - public Builder mergeFrom( - com.google.protobuf.CodedInputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws java.io.IOException { - if (extensionRegistry == null) { - throw new java.lang.NullPointerException(); - } - try { - boolean done = false; - while (!done) { - int tag = input.readTag(); - switch (tag) { - case 0: - done = true; - break; - default: { - if (!super.parseUnknownField(input, extensionRegistry, tag)) { - done = true; // was an endgroup tag - } - break; - } // default: - } // switch (tag) - } // while (!done) - } catch (com.google.protobuf.InvalidProtocolBufferException e) { - throw e.unwrapIOException(); - } finally { - onChanged(); - } // finally - return this; - } - - // @@protoc_insertion_point(builder_scope:vortex.dtype.Null) - } - - // @@protoc_insertion_point(class_scope:vortex.dtype.Null) - private static final io.github.dfa1.vortex.proto.DTypeProtos.Null DEFAULT_INSTANCE; - static { - DEFAULT_INSTANCE = new io.github.dfa1.vortex.proto.DTypeProtos.Null(); - } - - public static io.github.dfa1.vortex.proto.DTypeProtos.Null getDefaultInstance() { - return DEFAULT_INSTANCE; - } - - private static final com.google.protobuf.Parser - PARSER = new com.google.protobuf.AbstractParser() { - @java.lang.Override - public Null parsePartialFrom( - com.google.protobuf.CodedInputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws com.google.protobuf.InvalidProtocolBufferException { - Builder builder = newBuilder(); - try { - builder.mergeFrom(input, extensionRegistry); - } catch (com.google.protobuf.InvalidProtocolBufferException e) { - throw e.setUnfinishedMessage(builder.buildPartial()); - } catch (com.google.protobuf.UninitializedMessageException e) { - throw e.asInvalidProtocolBufferException().setUnfinishedMessage(builder.buildPartial()); - } catch (java.io.IOException e) { - throw new com.google.protobuf.InvalidProtocolBufferException(e) - .setUnfinishedMessage(builder.buildPartial()); - } - return builder.buildPartial(); - } - }; - - public static com.google.protobuf.Parser parser() { - return PARSER; - } - - @java.lang.Override - public com.google.protobuf.Parser getParserForType() { - return PARSER; - } - - @java.lang.Override - public io.github.dfa1.vortex.proto.DTypeProtos.Null getDefaultInstanceForType() { - return DEFAULT_INSTANCE; - } - - } - - public interface BoolOrBuilder extends - // @@protoc_insertion_point(interface_extends:vortex.dtype.Bool) - com.google.protobuf.MessageOrBuilder { - - /** - * bool nullable = 1; - * @return The nullable. - */ - boolean getNullable(); - } - /** - * Protobuf type {@code vortex.dtype.Bool} - */ - public static final class Bool extends - com.google.protobuf.GeneratedMessage implements - // @@protoc_insertion_point(message_implements:vortex.dtype.Bool) - BoolOrBuilder { - private static final long serialVersionUID = 0L; - static { - com.google.protobuf.RuntimeVersion.validateProtobufGencodeVersion( - com.google.protobuf.RuntimeVersion.RuntimeDomain.PUBLIC, - /* major= */ 4, - /* minor= */ 35, - /* patch= */ 0, - /* suffix= */ "", - "Bool"); - } - // Use Bool.newBuilder() to construct. - private Bool(com.google.protobuf.GeneratedMessage.Builder builder) { - super(builder); - } - private Bool() { - } - - public static final com.google.protobuf.Descriptors.Descriptor - getDescriptor() { - return io.github.dfa1.vortex.proto.DTypeProtos.internal_static_vortex_dtype_Bool_descriptor; - } - - @java.lang.Override - public com.google.protobuf.Descriptors.Descriptor getDescriptorForType() { - return io.github.dfa1.vortex.proto.DTypeProtos.internal_static_vortex_dtype_Bool_descriptor; - } - - @java.lang.Override - protected com.google.protobuf.GeneratedMessage.FieldAccessorTable - internalGetFieldAccessorTable() { - return io.github.dfa1.vortex.proto.DTypeProtos.internal_static_vortex_dtype_Bool_fieldAccessorTable - .ensureFieldAccessorsInitialized( - io.github.dfa1.vortex.proto.DTypeProtos.Bool.class, io.github.dfa1.vortex.proto.DTypeProtos.Bool.Builder.class); - } - - public static final int NULLABLE_FIELD_NUMBER = 1; - private boolean nullable_ = false; - /** - * bool nullable = 1; - * @return The nullable. - */ - @java.lang.Override - public boolean getNullable() { - return nullable_; - } - - private byte memoizedIsInitialized = -1; - @java.lang.Override - public final boolean isInitialized() { - byte isInitialized = memoizedIsInitialized; - if (isInitialized == 1) return true; - if (isInitialized == 0) return false; - - memoizedIsInitialized = 1; - return true; - } - - @java.lang.Override - public void writeTo(com.google.protobuf.CodedOutputStream output) - throws java.io.IOException { - if (nullable_ != false) { - output.writeBool(1, nullable_); - } - getUnknownFields().writeTo(output); - } - private int computeSerializedSize_0() { - int size = 0; - if (nullable_ != false) { - size += com.google.protobuf.CodedOutputStream - .computeBoolSize(1, nullable_); - } - return size; - } - @java.lang.Override - public int getSerializedSize() { - int size = memoizedSize; - if (size != -1) return size; - - size = 0; - size += computeSerializedSize_0(); - size += getUnknownFields().getSerializedSize(); - memoizedSize = size; - return size; - } - - @java.lang.Override - public boolean equals(final java.lang.Object obj) { - if (obj == this) { - return true; - } - if (!(obj instanceof io.github.dfa1.vortex.proto.DTypeProtos.Bool)) { - return super.equals(obj); - } - io.github.dfa1.vortex.proto.DTypeProtos.Bool other = (io.github.dfa1.vortex.proto.DTypeProtos.Bool) obj; - - if (getNullable() - != other.getNullable()) return false; - if (!getUnknownFields().equals(other.getUnknownFields())) return false; - return true; - } - - @java.lang.Override - public int hashCode() { - if (memoizedHashCode != 0) { - return memoizedHashCode; - } - int hash = 41; - hash = (19 * hash) + getDescriptor().hashCode(); - hash = (37 * hash) + NULLABLE_FIELD_NUMBER; - hash = (53 * hash) + com.google.protobuf.Internal.hashBoolean( - getNullable()); - hash = (29 * hash) + getUnknownFields().hashCode(); - memoizedHashCode = hash; - return hash; - } - - public static io.github.dfa1.vortex.proto.DTypeProtos.Bool parseFrom( - java.nio.ByteBuffer data) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data); - } - public static io.github.dfa1.vortex.proto.DTypeProtos.Bool parseFrom( - java.nio.ByteBuffer data, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data, extensionRegistry); - } - public static io.github.dfa1.vortex.proto.DTypeProtos.Bool parseFrom( - com.google.protobuf.ByteString data) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data); - } - public static io.github.dfa1.vortex.proto.DTypeProtos.Bool parseFrom( - com.google.protobuf.ByteString data, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data, extensionRegistry); - } - public static io.github.dfa1.vortex.proto.DTypeProtos.Bool parseFrom(byte[] data) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data); - } - public static io.github.dfa1.vortex.proto.DTypeProtos.Bool parseFrom( - byte[] data, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data, extensionRegistry); - } - public static io.github.dfa1.vortex.proto.DTypeProtos.Bool parseFrom(java.io.InputStream input) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseWithIOException(PARSER, input); - } - public static io.github.dfa1.vortex.proto.DTypeProtos.Bool parseFrom( - java.io.InputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseWithIOException(PARSER, input, extensionRegistry); - } - - public static io.github.dfa1.vortex.proto.DTypeProtos.Bool parseDelimitedFrom(java.io.InputStream input) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseDelimitedWithIOException(PARSER, input); - } - - public static io.github.dfa1.vortex.proto.DTypeProtos.Bool parseDelimitedFrom( - java.io.InputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseDelimitedWithIOException(PARSER, input, extensionRegistry); - } - public static io.github.dfa1.vortex.proto.DTypeProtos.Bool parseFrom( - com.google.protobuf.CodedInputStream input) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseWithIOException(PARSER, input); - } - public static io.github.dfa1.vortex.proto.DTypeProtos.Bool parseFrom( - com.google.protobuf.CodedInputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseWithIOException(PARSER, input, extensionRegistry); - } - - @java.lang.Override - public Builder newBuilderForType() { return newBuilder(); } - public static Builder newBuilder() { - return DEFAULT_INSTANCE.toBuilder(); - } - public static Builder newBuilder(io.github.dfa1.vortex.proto.DTypeProtos.Bool prototype) { - return DEFAULT_INSTANCE.toBuilder().mergeFrom(prototype); - } - @java.lang.Override - public Builder toBuilder() { - return this == DEFAULT_INSTANCE - ? new Builder() : new Builder().mergeFrom(this); - } - - @java.lang.Override - protected Builder newBuilderForType( - com.google.protobuf.GeneratedMessage.BuilderParent parent) { - Builder builder = new Builder(parent); - return builder; - } - /** - * Protobuf type {@code vortex.dtype.Bool} - */ - public static final class Builder extends - com.google.protobuf.GeneratedMessage.Builder implements - // @@protoc_insertion_point(builder_implements:vortex.dtype.Bool) - io.github.dfa1.vortex.proto.DTypeProtos.BoolOrBuilder { - public static final com.google.protobuf.Descriptors.Descriptor - getDescriptor() { - return io.github.dfa1.vortex.proto.DTypeProtos.internal_static_vortex_dtype_Bool_descriptor; - } - - @java.lang.Override - protected com.google.protobuf.GeneratedMessage.FieldAccessorTable - internalGetFieldAccessorTable() { - return io.github.dfa1.vortex.proto.DTypeProtos.internal_static_vortex_dtype_Bool_fieldAccessorTable - .ensureFieldAccessorsInitialized( - io.github.dfa1.vortex.proto.DTypeProtos.Bool.class, io.github.dfa1.vortex.proto.DTypeProtos.Bool.Builder.class); - } - - // Construct using io.github.dfa1.vortex.proto.DTypeProtos.Bool.newBuilder() - private Builder() { - - } - - private Builder( - com.google.protobuf.GeneratedMessage.BuilderParent parent) { - super(parent); - - } - @java.lang.Override - public Builder clear() { - super.clear(); - bitField0_ = 0; - nullable_ = false; - return this; - } - - @java.lang.Override - public com.google.protobuf.Descriptors.Descriptor - getDescriptorForType() { - return io.github.dfa1.vortex.proto.DTypeProtos.internal_static_vortex_dtype_Bool_descriptor; - } - - @java.lang.Override - public io.github.dfa1.vortex.proto.DTypeProtos.Bool getDefaultInstanceForType() { - return io.github.dfa1.vortex.proto.DTypeProtos.Bool.getDefaultInstance(); - } - - @java.lang.Override - public io.github.dfa1.vortex.proto.DTypeProtos.Bool build() { - io.github.dfa1.vortex.proto.DTypeProtos.Bool result = buildPartial(); - if (!result.isInitialized()) { - throw newUninitializedMessageException(result); - } - return result; - } - - @java.lang.Override - public io.github.dfa1.vortex.proto.DTypeProtos.Bool buildPartial() { - io.github.dfa1.vortex.proto.DTypeProtos.Bool result = new io.github.dfa1.vortex.proto.DTypeProtos.Bool(this); - if (bitField0_ != 0) { buildPartial0(result); } - onBuilt(); - return result; - } - - private void buildPartial0(io.github.dfa1.vortex.proto.DTypeProtos.Bool result) { - int from_bitField0_ = bitField0_; - if (((from_bitField0_ & 0x00000001) != 0)) { - result.nullable_ = nullable_; - } - } - - @java.lang.Override - public Builder mergeFrom(com.google.protobuf.Message other) { - if (other instanceof io.github.dfa1.vortex.proto.DTypeProtos.Bool) { - return mergeFrom((io.github.dfa1.vortex.proto.DTypeProtos.Bool)other); - } else { - super.mergeFrom(other); - return this; - } - } - - public Builder mergeFrom(io.github.dfa1.vortex.proto.DTypeProtos.Bool other) { - if (other == io.github.dfa1.vortex.proto.DTypeProtos.Bool.getDefaultInstance()) return this; - if (other.getNullable() != false) { - setNullable(other.getNullable()); - } - this.mergeUnknownFields(other.getUnknownFields()); - onChanged(); - return this; - } - - @java.lang.Override - public final boolean isInitialized() { - return true; - } - - @java.lang.Override - public Builder mergeFrom( - com.google.protobuf.CodedInputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws java.io.IOException { - if (extensionRegistry == null) { - throw new java.lang.NullPointerException(); - } - try { - boolean done = false; - while (!done) { - int tag = input.readTag(); - switch (tag) { - case 0: - done = true; - break; - case 8: { - nullable_ = input.readBool(); - bitField0_ |= 0x00000001; - break; - } // case 8 - default: { - if (!super.parseUnknownField(input, extensionRegistry, tag)) { - done = true; // was an endgroup tag - } - break; - } // default: - } // switch (tag) - } // while (!done) - } catch (com.google.protobuf.InvalidProtocolBufferException e) { - throw e.unwrapIOException(); - } finally { - onChanged(); - } // finally - return this; - } - private int bitField0_; - - private boolean nullable_ ; - /** - * bool nullable = 1; - * @return The nullable. - */ - @java.lang.Override - public boolean getNullable() { - return nullable_; - } - /** - * bool nullable = 1; - * @param value The nullable to set. - * @return This builder for chaining. - */ - public Builder setNullable(boolean value) { - - nullable_ = value; - bitField0_ |= 0x00000001; - onChanged(); - return this; - } - /** - * bool nullable = 1; - * @return This builder for chaining. - */ - public Builder clearNullable() { - bitField0_ = (bitField0_ & ~0x00000001); - nullable_ = false; - onChanged(); - return this; - } - - // @@protoc_insertion_point(builder_scope:vortex.dtype.Bool) - } - - // @@protoc_insertion_point(class_scope:vortex.dtype.Bool) - private static final io.github.dfa1.vortex.proto.DTypeProtos.Bool DEFAULT_INSTANCE; - static { - DEFAULT_INSTANCE = new io.github.dfa1.vortex.proto.DTypeProtos.Bool(); - } - - public static io.github.dfa1.vortex.proto.DTypeProtos.Bool getDefaultInstance() { - return DEFAULT_INSTANCE; - } - - private static final com.google.protobuf.Parser - PARSER = new com.google.protobuf.AbstractParser() { - @java.lang.Override - public Bool parsePartialFrom( - com.google.protobuf.CodedInputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws com.google.protobuf.InvalidProtocolBufferException { - Builder builder = newBuilder(); - try { - builder.mergeFrom(input, extensionRegistry); - } catch (com.google.protobuf.InvalidProtocolBufferException e) { - throw e.setUnfinishedMessage(builder.buildPartial()); - } catch (com.google.protobuf.UninitializedMessageException e) { - throw e.asInvalidProtocolBufferException().setUnfinishedMessage(builder.buildPartial()); - } catch (java.io.IOException e) { - throw new com.google.protobuf.InvalidProtocolBufferException(e) - .setUnfinishedMessage(builder.buildPartial()); - } - return builder.buildPartial(); - } - }; - - public static com.google.protobuf.Parser parser() { - return PARSER; - } - - @java.lang.Override - public com.google.protobuf.Parser getParserForType() { - return PARSER; - } - - @java.lang.Override - public io.github.dfa1.vortex.proto.DTypeProtos.Bool getDefaultInstanceForType() { - return DEFAULT_INSTANCE; - } - - } - - public interface PrimitiveOrBuilder extends - // @@protoc_insertion_point(interface_extends:vortex.dtype.Primitive) - com.google.protobuf.MessageOrBuilder { - - /** - * .vortex.dtype.PType type = 1; - * @return The enum numeric value on the wire for type. - */ - int getTypeValue(); - /** - * .vortex.dtype.PType type = 1; - * @return The type. - */ - io.github.dfa1.vortex.proto.DTypeProtos.PType getType(); - - /** - * bool nullable = 2; - * @return The nullable. - */ - boolean getNullable(); - } - /** - * Protobuf type {@code vortex.dtype.Primitive} - */ - public static final class Primitive extends - com.google.protobuf.GeneratedMessage implements - // @@protoc_insertion_point(message_implements:vortex.dtype.Primitive) - PrimitiveOrBuilder { - private static final long serialVersionUID = 0L; - static { - com.google.protobuf.RuntimeVersion.validateProtobufGencodeVersion( - com.google.protobuf.RuntimeVersion.RuntimeDomain.PUBLIC, - /* major= */ 4, - /* minor= */ 35, - /* patch= */ 0, - /* suffix= */ "", - "Primitive"); - } - // Use Primitive.newBuilder() to construct. - private Primitive(com.google.protobuf.GeneratedMessage.Builder builder) { - super(builder); - } - private Primitive() { - type_ = 0; - } - - public static final com.google.protobuf.Descriptors.Descriptor - getDescriptor() { - return io.github.dfa1.vortex.proto.DTypeProtos.internal_static_vortex_dtype_Primitive_descriptor; - } - - @java.lang.Override - public com.google.protobuf.Descriptors.Descriptor getDescriptorForType() { - return io.github.dfa1.vortex.proto.DTypeProtos.internal_static_vortex_dtype_Primitive_descriptor; - } - - @java.lang.Override - protected com.google.protobuf.GeneratedMessage.FieldAccessorTable - internalGetFieldAccessorTable() { - return io.github.dfa1.vortex.proto.DTypeProtos.internal_static_vortex_dtype_Primitive_fieldAccessorTable - .ensureFieldAccessorsInitialized( - io.github.dfa1.vortex.proto.DTypeProtos.Primitive.class, io.github.dfa1.vortex.proto.DTypeProtos.Primitive.Builder.class); - } - - public static final int TYPE_FIELD_NUMBER = 1; - private int type_ = 0; - /** - * .vortex.dtype.PType type = 1; - * @return The enum numeric value on the wire for type. - */ - @java.lang.Override public int getTypeValue() { - return type_; - } - /** - * .vortex.dtype.PType type = 1; - * @return The type. - */ - @java.lang.Override public io.github.dfa1.vortex.proto.DTypeProtos.PType getType() { - io.github.dfa1.vortex.proto.DTypeProtos.PType result = io.github.dfa1.vortex.proto.DTypeProtos.PType.forNumber(type_); - return result == null ? io.github.dfa1.vortex.proto.DTypeProtos.PType.UNRECOGNIZED : result; - } - - public static final int NULLABLE_FIELD_NUMBER = 2; - private boolean nullable_ = false; - /** - * bool nullable = 2; - * @return The nullable. - */ - @java.lang.Override - public boolean getNullable() { - return nullable_; - } - - private byte memoizedIsInitialized = -1; - @java.lang.Override - public final boolean isInitialized() { - byte isInitialized = memoizedIsInitialized; - if (isInitialized == 1) return true; - if (isInitialized == 0) return false; - - memoizedIsInitialized = 1; - return true; - } - - @java.lang.Override - public void writeTo(com.google.protobuf.CodedOutputStream output) - throws java.io.IOException { - if (type_ != io.github.dfa1.vortex.proto.DTypeProtos.PType.U8.getNumber()) { - output.writeEnum(1, type_); - } - if (nullable_ != false) { - output.writeBool(2, nullable_); - } - getUnknownFields().writeTo(output); - } - private int computeSerializedSize_0() { - int size = 0; - if (type_ != io.github.dfa1.vortex.proto.DTypeProtos.PType.U8.getNumber()) { - size += com.google.protobuf.CodedOutputStream - .computeEnumSize(1, type_); - } - if (nullable_ != false) { - size += com.google.protobuf.CodedOutputStream - .computeBoolSize(2, nullable_); - } - return size; - } - @java.lang.Override - public int getSerializedSize() { - int size = memoizedSize; - if (size != -1) return size; - - size = 0; - size += computeSerializedSize_0(); - size += getUnknownFields().getSerializedSize(); - memoizedSize = size; - return size; - } - - @java.lang.Override - public boolean equals(final java.lang.Object obj) { - if (obj == this) { - return true; - } - if (!(obj instanceof io.github.dfa1.vortex.proto.DTypeProtos.Primitive)) { - return super.equals(obj); - } - io.github.dfa1.vortex.proto.DTypeProtos.Primitive other = (io.github.dfa1.vortex.proto.DTypeProtos.Primitive) obj; - - if (type_ != other.type_) return false; - if (getNullable() - != other.getNullable()) return false; - if (!getUnknownFields().equals(other.getUnknownFields())) return false; - return true; - } - - @java.lang.Override - public int hashCode() { - if (memoizedHashCode != 0) { - return memoizedHashCode; - } - int hash = 41; - hash = (19 * hash) + getDescriptor().hashCode(); - hash = (37 * hash) + TYPE_FIELD_NUMBER; - hash = (53 * hash) + type_; - hash = (37 * hash) + NULLABLE_FIELD_NUMBER; - hash = (53 * hash) + com.google.protobuf.Internal.hashBoolean( - getNullable()); - hash = (29 * hash) + getUnknownFields().hashCode(); - memoizedHashCode = hash; - return hash; - } - - public static io.github.dfa1.vortex.proto.DTypeProtos.Primitive parseFrom( - java.nio.ByteBuffer data) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data); - } - public static io.github.dfa1.vortex.proto.DTypeProtos.Primitive parseFrom( - java.nio.ByteBuffer data, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data, extensionRegistry); - } - public static io.github.dfa1.vortex.proto.DTypeProtos.Primitive parseFrom( - com.google.protobuf.ByteString data) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data); - } - public static io.github.dfa1.vortex.proto.DTypeProtos.Primitive parseFrom( - com.google.protobuf.ByteString data, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data, extensionRegistry); - } - public static io.github.dfa1.vortex.proto.DTypeProtos.Primitive parseFrom(byte[] data) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data); - } - public static io.github.dfa1.vortex.proto.DTypeProtos.Primitive parseFrom( - byte[] data, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data, extensionRegistry); - } - public static io.github.dfa1.vortex.proto.DTypeProtos.Primitive parseFrom(java.io.InputStream input) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseWithIOException(PARSER, input); - } - public static io.github.dfa1.vortex.proto.DTypeProtos.Primitive parseFrom( - java.io.InputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseWithIOException(PARSER, input, extensionRegistry); - } - - public static io.github.dfa1.vortex.proto.DTypeProtos.Primitive parseDelimitedFrom(java.io.InputStream input) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseDelimitedWithIOException(PARSER, input); - } - - public static io.github.dfa1.vortex.proto.DTypeProtos.Primitive parseDelimitedFrom( - java.io.InputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseDelimitedWithIOException(PARSER, input, extensionRegistry); - } - public static io.github.dfa1.vortex.proto.DTypeProtos.Primitive parseFrom( - com.google.protobuf.CodedInputStream input) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseWithIOException(PARSER, input); - } - public static io.github.dfa1.vortex.proto.DTypeProtos.Primitive parseFrom( - com.google.protobuf.CodedInputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseWithIOException(PARSER, input, extensionRegistry); - } - - @java.lang.Override - public Builder newBuilderForType() { return newBuilder(); } - public static Builder newBuilder() { - return DEFAULT_INSTANCE.toBuilder(); - } - public static Builder newBuilder(io.github.dfa1.vortex.proto.DTypeProtos.Primitive prototype) { - return DEFAULT_INSTANCE.toBuilder().mergeFrom(prototype); - } - @java.lang.Override - public Builder toBuilder() { - return this == DEFAULT_INSTANCE - ? new Builder() : new Builder().mergeFrom(this); - } - - @java.lang.Override - protected Builder newBuilderForType( - com.google.protobuf.GeneratedMessage.BuilderParent parent) { - Builder builder = new Builder(parent); - return builder; - } - /** - * Protobuf type {@code vortex.dtype.Primitive} - */ - public static final class Builder extends - com.google.protobuf.GeneratedMessage.Builder implements - // @@protoc_insertion_point(builder_implements:vortex.dtype.Primitive) - io.github.dfa1.vortex.proto.DTypeProtos.PrimitiveOrBuilder { - public static final com.google.protobuf.Descriptors.Descriptor - getDescriptor() { - return io.github.dfa1.vortex.proto.DTypeProtos.internal_static_vortex_dtype_Primitive_descriptor; - } - - @java.lang.Override - protected com.google.protobuf.GeneratedMessage.FieldAccessorTable - internalGetFieldAccessorTable() { - return io.github.dfa1.vortex.proto.DTypeProtos.internal_static_vortex_dtype_Primitive_fieldAccessorTable - .ensureFieldAccessorsInitialized( - io.github.dfa1.vortex.proto.DTypeProtos.Primitive.class, io.github.dfa1.vortex.proto.DTypeProtos.Primitive.Builder.class); - } - - // Construct using io.github.dfa1.vortex.proto.DTypeProtos.Primitive.newBuilder() - private Builder() { - - } - - private Builder( - com.google.protobuf.GeneratedMessage.BuilderParent parent) { - super(parent); - - } - @java.lang.Override - public Builder clear() { - super.clear(); - bitField0_ = 0; - type_ = 0; - nullable_ = false; - return this; - } - - @java.lang.Override - public com.google.protobuf.Descriptors.Descriptor - getDescriptorForType() { - return io.github.dfa1.vortex.proto.DTypeProtos.internal_static_vortex_dtype_Primitive_descriptor; - } - - @java.lang.Override - public io.github.dfa1.vortex.proto.DTypeProtos.Primitive getDefaultInstanceForType() { - return io.github.dfa1.vortex.proto.DTypeProtos.Primitive.getDefaultInstance(); - } - - @java.lang.Override - public io.github.dfa1.vortex.proto.DTypeProtos.Primitive build() { - io.github.dfa1.vortex.proto.DTypeProtos.Primitive result = buildPartial(); - if (!result.isInitialized()) { - throw newUninitializedMessageException(result); - } - return result; - } - - @java.lang.Override - public io.github.dfa1.vortex.proto.DTypeProtos.Primitive buildPartial() { - io.github.dfa1.vortex.proto.DTypeProtos.Primitive result = new io.github.dfa1.vortex.proto.DTypeProtos.Primitive(this); - if (bitField0_ != 0) { buildPartial0(result); } - onBuilt(); - return result; - } - - private void buildPartial0(io.github.dfa1.vortex.proto.DTypeProtos.Primitive result) { - int from_bitField0_ = bitField0_; - if (((from_bitField0_ & 0x00000001) != 0)) { - result.type_ = type_; - } - if (((from_bitField0_ & 0x00000002) != 0)) { - result.nullable_ = nullable_; - } - } - - @java.lang.Override - public Builder mergeFrom(com.google.protobuf.Message other) { - if (other instanceof io.github.dfa1.vortex.proto.DTypeProtos.Primitive) { - return mergeFrom((io.github.dfa1.vortex.proto.DTypeProtos.Primitive)other); - } else { - super.mergeFrom(other); - return this; - } - } - - public Builder mergeFrom(io.github.dfa1.vortex.proto.DTypeProtos.Primitive other) { - if (other == io.github.dfa1.vortex.proto.DTypeProtos.Primitive.getDefaultInstance()) return this; - if (other.type_ != 0) { - setTypeValue(other.getTypeValue()); - } - if (other.getNullable() != false) { - setNullable(other.getNullable()); - } - this.mergeUnknownFields(other.getUnknownFields()); - onChanged(); - return this; - } - - @java.lang.Override - public final boolean isInitialized() { - return true; - } - - @java.lang.Override - public Builder mergeFrom( - com.google.protobuf.CodedInputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws java.io.IOException { - if (extensionRegistry == null) { - throw new java.lang.NullPointerException(); - } - try { - boolean done = false; - while (!done) { - int tag = input.readTag(); - switch (tag) { - case 0: - done = true; - break; - case 8: { - type_ = input.readEnum(); - bitField0_ |= 0x00000001; - break; - } // case 8 - case 16: { - nullable_ = input.readBool(); - bitField0_ |= 0x00000002; - break; - } // case 16 - default: { - if (!super.parseUnknownField(input, extensionRegistry, tag)) { - done = true; // was an endgroup tag - } - break; - } // default: - } // switch (tag) - } // while (!done) - } catch (com.google.protobuf.InvalidProtocolBufferException e) { - throw e.unwrapIOException(); - } finally { - onChanged(); - } // finally - return this; - } - private int bitField0_; - - private int type_ = 0; - /** - * .vortex.dtype.PType type = 1; - * @return The enum numeric value on the wire for type. - */ - @java.lang.Override public int getTypeValue() { - return type_; - } - /** - * .vortex.dtype.PType type = 1; - * @param value The enum numeric value on the wire for type to set. - * @return This builder for chaining. - */ - public Builder setTypeValue(int value) { - type_ = value; - bitField0_ |= 0x00000001; - onChanged(); - return this; - } - /** - * .vortex.dtype.PType type = 1; - * @return The type. - */ - @java.lang.Override - public io.github.dfa1.vortex.proto.DTypeProtos.PType getType() { - io.github.dfa1.vortex.proto.DTypeProtos.PType result = io.github.dfa1.vortex.proto.DTypeProtos.PType.forNumber(type_); - return result == null ? io.github.dfa1.vortex.proto.DTypeProtos.PType.UNRECOGNIZED : result; - } - /** - * .vortex.dtype.PType type = 1; - * @param value The type to set. - * @throws IllegalArgumentException if UNRECOGNIZED is provided. - * @return This builder for chaining. - */ - public Builder setType(io.github.dfa1.vortex.proto.DTypeProtos.PType value) { - if (value == null) { throw new NullPointerException(); } - bitField0_ |= 0x00000001; - type_ = value.getNumber(); - onChanged(); - return this; - } - /** - * .vortex.dtype.PType type = 1; - * @return This builder for chaining. - */ - public Builder clearType() { - bitField0_ = (bitField0_ & ~0x00000001); - type_ = 0; - onChanged(); - return this; - } - - private boolean nullable_ ; - /** - * bool nullable = 2; - * @return The nullable. - */ - @java.lang.Override - public boolean getNullable() { - return nullable_; - } - /** - * bool nullable = 2; - * @param value The nullable to set. - * @return This builder for chaining. - */ - public Builder setNullable(boolean value) { - - nullable_ = value; - bitField0_ |= 0x00000002; - onChanged(); - return this; - } - /** - * bool nullable = 2; - * @return This builder for chaining. - */ - public Builder clearNullable() { - bitField0_ = (bitField0_ & ~0x00000002); - nullable_ = false; - onChanged(); - return this; - } - - // @@protoc_insertion_point(builder_scope:vortex.dtype.Primitive) - } - - // @@protoc_insertion_point(class_scope:vortex.dtype.Primitive) - private static final io.github.dfa1.vortex.proto.DTypeProtos.Primitive DEFAULT_INSTANCE; - static { - DEFAULT_INSTANCE = new io.github.dfa1.vortex.proto.DTypeProtos.Primitive(); - } - - public static io.github.dfa1.vortex.proto.DTypeProtos.Primitive getDefaultInstance() { - return DEFAULT_INSTANCE; - } - - private static final com.google.protobuf.Parser - PARSER = new com.google.protobuf.AbstractParser() { - @java.lang.Override - public Primitive parsePartialFrom( - com.google.protobuf.CodedInputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws com.google.protobuf.InvalidProtocolBufferException { - Builder builder = newBuilder(); - try { - builder.mergeFrom(input, extensionRegistry); - } catch (com.google.protobuf.InvalidProtocolBufferException e) { - throw e.setUnfinishedMessage(builder.buildPartial()); - } catch (com.google.protobuf.UninitializedMessageException e) { - throw e.asInvalidProtocolBufferException().setUnfinishedMessage(builder.buildPartial()); - } catch (java.io.IOException e) { - throw new com.google.protobuf.InvalidProtocolBufferException(e) - .setUnfinishedMessage(builder.buildPartial()); - } - return builder.buildPartial(); - } - }; - - public static com.google.protobuf.Parser parser() { - return PARSER; - } - - @java.lang.Override - public com.google.protobuf.Parser getParserForType() { - return PARSER; - } - - @java.lang.Override - public io.github.dfa1.vortex.proto.DTypeProtos.Primitive getDefaultInstanceForType() { - return DEFAULT_INSTANCE; - } - - } - - public interface DecimalOrBuilder extends - // @@protoc_insertion_point(interface_extends:vortex.dtype.Decimal) - com.google.protobuf.MessageOrBuilder { - - /** - * uint32 precision = 1; - * @return The precision. - */ - int getPrecision(); - - /** - * int32 scale = 2; - * @return The scale. - */ - int getScale(); - - /** - * bool nullable = 3; - * @return The nullable. - */ - boolean getNullable(); - } - /** - * Protobuf type {@code vortex.dtype.Decimal} - */ - public static final class Decimal extends - com.google.protobuf.GeneratedMessage implements - // @@protoc_insertion_point(message_implements:vortex.dtype.Decimal) - DecimalOrBuilder { - private static final long serialVersionUID = 0L; - static { - com.google.protobuf.RuntimeVersion.validateProtobufGencodeVersion( - com.google.protobuf.RuntimeVersion.RuntimeDomain.PUBLIC, - /* major= */ 4, - /* minor= */ 35, - /* patch= */ 0, - /* suffix= */ "", - "Decimal"); - } - // Use Decimal.newBuilder() to construct. - private Decimal(com.google.protobuf.GeneratedMessage.Builder builder) { - super(builder); - } - private Decimal() { - } - - public static final com.google.protobuf.Descriptors.Descriptor - getDescriptor() { - return io.github.dfa1.vortex.proto.DTypeProtos.internal_static_vortex_dtype_Decimal_descriptor; - } - - @java.lang.Override - public com.google.protobuf.Descriptors.Descriptor getDescriptorForType() { - return io.github.dfa1.vortex.proto.DTypeProtos.internal_static_vortex_dtype_Decimal_descriptor; - } - - @java.lang.Override - protected com.google.protobuf.GeneratedMessage.FieldAccessorTable - internalGetFieldAccessorTable() { - return io.github.dfa1.vortex.proto.DTypeProtos.internal_static_vortex_dtype_Decimal_fieldAccessorTable - .ensureFieldAccessorsInitialized( - io.github.dfa1.vortex.proto.DTypeProtos.Decimal.class, io.github.dfa1.vortex.proto.DTypeProtos.Decimal.Builder.class); - } - - public static final int PRECISION_FIELD_NUMBER = 1; - private int precision_ = 0; - /** - * uint32 precision = 1; - * @return The precision. - */ - @java.lang.Override - public int getPrecision() { - return precision_; - } - - public static final int SCALE_FIELD_NUMBER = 2; - private int scale_ = 0; - /** - * int32 scale = 2; - * @return The scale. - */ - @java.lang.Override - public int getScale() { - return scale_; - } - - public static final int NULLABLE_FIELD_NUMBER = 3; - private boolean nullable_ = false; - /** - * bool nullable = 3; - * @return The nullable. - */ - @java.lang.Override - public boolean getNullable() { - return nullable_; - } - - private byte memoizedIsInitialized = -1; - @java.lang.Override - public final boolean isInitialized() { - byte isInitialized = memoizedIsInitialized; - if (isInitialized == 1) return true; - if (isInitialized == 0) return false; - - memoizedIsInitialized = 1; - return true; - } - - @java.lang.Override - public void writeTo(com.google.protobuf.CodedOutputStream output) - throws java.io.IOException { - if (precision_ != 0) { - output.writeUInt32(1, precision_); - } - if (scale_ != 0) { - output.writeInt32(2, scale_); - } - if (nullable_ != false) { - output.writeBool(3, nullable_); - } - getUnknownFields().writeTo(output); - } - private int computeSerializedSize_0() { - int size = 0; - if (precision_ != 0) { - size += com.google.protobuf.CodedOutputStream - .computeUInt32Size(1, precision_); - } - if (scale_ != 0) { - size += com.google.protobuf.CodedOutputStream - .computeInt32Size(2, scale_); - } - if (nullable_ != false) { - size += com.google.protobuf.CodedOutputStream - .computeBoolSize(3, nullable_); - } - return size; - } - @java.lang.Override - public int getSerializedSize() { - int size = memoizedSize; - if (size != -1) return size; - - size = 0; - size += computeSerializedSize_0(); - size += getUnknownFields().getSerializedSize(); - memoizedSize = size; - return size; - } - - @java.lang.Override - public boolean equals(final java.lang.Object obj) { - if (obj == this) { - return true; - } - if (!(obj instanceof io.github.dfa1.vortex.proto.DTypeProtos.Decimal)) { - return super.equals(obj); - } - io.github.dfa1.vortex.proto.DTypeProtos.Decimal other = (io.github.dfa1.vortex.proto.DTypeProtos.Decimal) obj; - - if (getPrecision() - != other.getPrecision()) return false; - if (getScale() - != other.getScale()) return false; - if (getNullable() - != other.getNullable()) return false; - if (!getUnknownFields().equals(other.getUnknownFields())) return false; - return true; - } - - @java.lang.Override - public int hashCode() { - if (memoizedHashCode != 0) { - return memoizedHashCode; - } - int hash = 41; - hash = (19 * hash) + getDescriptor().hashCode(); - hash = (37 * hash) + PRECISION_FIELD_NUMBER; - hash = (53 * hash) + getPrecision(); - hash = (37 * hash) + SCALE_FIELD_NUMBER; - hash = (53 * hash) + getScale(); - hash = (37 * hash) + NULLABLE_FIELD_NUMBER; - hash = (53 * hash) + com.google.protobuf.Internal.hashBoolean( - getNullable()); - hash = (29 * hash) + getUnknownFields().hashCode(); - memoizedHashCode = hash; - return hash; - } - - public static io.github.dfa1.vortex.proto.DTypeProtos.Decimal parseFrom( - java.nio.ByteBuffer data) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data); - } - public static io.github.dfa1.vortex.proto.DTypeProtos.Decimal parseFrom( - java.nio.ByteBuffer data, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data, extensionRegistry); - } - public static io.github.dfa1.vortex.proto.DTypeProtos.Decimal parseFrom( - com.google.protobuf.ByteString data) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data); - } - public static io.github.dfa1.vortex.proto.DTypeProtos.Decimal parseFrom( - com.google.protobuf.ByteString data, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data, extensionRegistry); - } - public static io.github.dfa1.vortex.proto.DTypeProtos.Decimal parseFrom(byte[] data) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data); - } - public static io.github.dfa1.vortex.proto.DTypeProtos.Decimal parseFrom( - byte[] data, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data, extensionRegistry); - } - public static io.github.dfa1.vortex.proto.DTypeProtos.Decimal parseFrom(java.io.InputStream input) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseWithIOException(PARSER, input); - } - public static io.github.dfa1.vortex.proto.DTypeProtos.Decimal parseFrom( - java.io.InputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseWithIOException(PARSER, input, extensionRegistry); - } - - public static io.github.dfa1.vortex.proto.DTypeProtos.Decimal parseDelimitedFrom(java.io.InputStream input) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseDelimitedWithIOException(PARSER, input); - } - - public static io.github.dfa1.vortex.proto.DTypeProtos.Decimal parseDelimitedFrom( - java.io.InputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseDelimitedWithIOException(PARSER, input, extensionRegistry); - } - public static io.github.dfa1.vortex.proto.DTypeProtos.Decimal parseFrom( - com.google.protobuf.CodedInputStream input) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseWithIOException(PARSER, input); - } - public static io.github.dfa1.vortex.proto.DTypeProtos.Decimal parseFrom( - com.google.protobuf.CodedInputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseWithIOException(PARSER, input, extensionRegistry); - } - - @java.lang.Override - public Builder newBuilderForType() { return newBuilder(); } - public static Builder newBuilder() { - return DEFAULT_INSTANCE.toBuilder(); - } - public static Builder newBuilder(io.github.dfa1.vortex.proto.DTypeProtos.Decimal prototype) { - return DEFAULT_INSTANCE.toBuilder().mergeFrom(prototype); - } - @java.lang.Override - public Builder toBuilder() { - return this == DEFAULT_INSTANCE - ? new Builder() : new Builder().mergeFrom(this); - } - - @java.lang.Override - protected Builder newBuilderForType( - com.google.protobuf.GeneratedMessage.BuilderParent parent) { - Builder builder = new Builder(parent); - return builder; - } - /** - * Protobuf type {@code vortex.dtype.Decimal} - */ - public static final class Builder extends - com.google.protobuf.GeneratedMessage.Builder implements - // @@protoc_insertion_point(builder_implements:vortex.dtype.Decimal) - io.github.dfa1.vortex.proto.DTypeProtos.DecimalOrBuilder { - public static final com.google.protobuf.Descriptors.Descriptor - getDescriptor() { - return io.github.dfa1.vortex.proto.DTypeProtos.internal_static_vortex_dtype_Decimal_descriptor; - } - - @java.lang.Override - protected com.google.protobuf.GeneratedMessage.FieldAccessorTable - internalGetFieldAccessorTable() { - return io.github.dfa1.vortex.proto.DTypeProtos.internal_static_vortex_dtype_Decimal_fieldAccessorTable - .ensureFieldAccessorsInitialized( - io.github.dfa1.vortex.proto.DTypeProtos.Decimal.class, io.github.dfa1.vortex.proto.DTypeProtos.Decimal.Builder.class); - } - - // Construct using io.github.dfa1.vortex.proto.DTypeProtos.Decimal.newBuilder() - private Builder() { - - } - - private Builder( - com.google.protobuf.GeneratedMessage.BuilderParent parent) { - super(parent); - - } - @java.lang.Override - public Builder clear() { - super.clear(); - bitField0_ = 0; - precision_ = 0; - scale_ = 0; - nullable_ = false; - return this; - } - - @java.lang.Override - public com.google.protobuf.Descriptors.Descriptor - getDescriptorForType() { - return io.github.dfa1.vortex.proto.DTypeProtos.internal_static_vortex_dtype_Decimal_descriptor; - } - - @java.lang.Override - public io.github.dfa1.vortex.proto.DTypeProtos.Decimal getDefaultInstanceForType() { - return io.github.dfa1.vortex.proto.DTypeProtos.Decimal.getDefaultInstance(); - } - - @java.lang.Override - public io.github.dfa1.vortex.proto.DTypeProtos.Decimal build() { - io.github.dfa1.vortex.proto.DTypeProtos.Decimal result = buildPartial(); - if (!result.isInitialized()) { - throw newUninitializedMessageException(result); - } - return result; - } - - @java.lang.Override - public io.github.dfa1.vortex.proto.DTypeProtos.Decimal buildPartial() { - io.github.dfa1.vortex.proto.DTypeProtos.Decimal result = new io.github.dfa1.vortex.proto.DTypeProtos.Decimal(this); - if (bitField0_ != 0) { buildPartial0(result); } - onBuilt(); - return result; - } - - private void buildPartial0(io.github.dfa1.vortex.proto.DTypeProtos.Decimal result) { - int from_bitField0_ = bitField0_; - if (((from_bitField0_ & 0x00000001) != 0)) { - result.precision_ = precision_; - } - if (((from_bitField0_ & 0x00000002) != 0)) { - result.scale_ = scale_; - } - if (((from_bitField0_ & 0x00000004) != 0)) { - result.nullable_ = nullable_; - } - } - - @java.lang.Override - public Builder mergeFrom(com.google.protobuf.Message other) { - if (other instanceof io.github.dfa1.vortex.proto.DTypeProtos.Decimal) { - return mergeFrom((io.github.dfa1.vortex.proto.DTypeProtos.Decimal)other); - } else { - super.mergeFrom(other); - return this; - } - } - - public Builder mergeFrom(io.github.dfa1.vortex.proto.DTypeProtos.Decimal other) { - if (other == io.github.dfa1.vortex.proto.DTypeProtos.Decimal.getDefaultInstance()) return this; - if (other.getPrecision() != 0) { - setPrecision(other.getPrecision()); - } - if (other.getScale() != 0) { - setScale(other.getScale()); - } - if (other.getNullable() != false) { - setNullable(other.getNullable()); - } - this.mergeUnknownFields(other.getUnknownFields()); - onChanged(); - return this; - } - - @java.lang.Override - public final boolean isInitialized() { - return true; - } - - @java.lang.Override - public Builder mergeFrom( - com.google.protobuf.CodedInputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws java.io.IOException { - if (extensionRegistry == null) { - throw new java.lang.NullPointerException(); - } - try { - boolean done = false; - while (!done) { - int tag = input.readTag(); - switch (tag) { - case 0: - done = true; - break; - case 8: { - precision_ = input.readUInt32(); - bitField0_ |= 0x00000001; - break; - } // case 8 - case 16: { - scale_ = input.readInt32(); - bitField0_ |= 0x00000002; - break; - } // case 16 - case 24: { - nullable_ = input.readBool(); - bitField0_ |= 0x00000004; - break; - } // case 24 - default: { - if (!super.parseUnknownField(input, extensionRegistry, tag)) { - done = true; // was an endgroup tag - } - break; - } // default: - } // switch (tag) - } // while (!done) - } catch (com.google.protobuf.InvalidProtocolBufferException e) { - throw e.unwrapIOException(); - } finally { - onChanged(); - } // finally - return this; - } - private int bitField0_; - - private int precision_ ; - /** - * uint32 precision = 1; - * @return The precision. - */ - @java.lang.Override - public int getPrecision() { - return precision_; - } - /** - * uint32 precision = 1; - * @param value The precision to set. - * @return This builder for chaining. - */ - public Builder setPrecision(int value) { - - precision_ = value; - bitField0_ |= 0x00000001; - onChanged(); - return this; - } - /** - * uint32 precision = 1; - * @return This builder for chaining. - */ - public Builder clearPrecision() { - bitField0_ = (bitField0_ & ~0x00000001); - precision_ = 0; - onChanged(); - return this; - } - - private int scale_ ; - /** - * int32 scale = 2; - * @return The scale. - */ - @java.lang.Override - public int getScale() { - return scale_; - } - /** - * int32 scale = 2; - * @param value The scale to set. - * @return This builder for chaining. - */ - public Builder setScale(int value) { - - scale_ = value; - bitField0_ |= 0x00000002; - onChanged(); - return this; - } - /** - * int32 scale = 2; - * @return This builder for chaining. - */ - public Builder clearScale() { - bitField0_ = (bitField0_ & ~0x00000002); - scale_ = 0; - onChanged(); - return this; - } - - private boolean nullable_ ; - /** - * bool nullable = 3; - * @return The nullable. - */ - @java.lang.Override - public boolean getNullable() { - return nullable_; - } - /** - * bool nullable = 3; - * @param value The nullable to set. - * @return This builder for chaining. - */ - public Builder setNullable(boolean value) { - - nullable_ = value; - bitField0_ |= 0x00000004; - onChanged(); - return this; - } - /** - * bool nullable = 3; - * @return This builder for chaining. - */ - public Builder clearNullable() { - bitField0_ = (bitField0_ & ~0x00000004); - nullable_ = false; - onChanged(); - return this; - } - - // @@protoc_insertion_point(builder_scope:vortex.dtype.Decimal) - } - - // @@protoc_insertion_point(class_scope:vortex.dtype.Decimal) - private static final io.github.dfa1.vortex.proto.DTypeProtos.Decimal DEFAULT_INSTANCE; - static { - DEFAULT_INSTANCE = new io.github.dfa1.vortex.proto.DTypeProtos.Decimal(); - } - - public static io.github.dfa1.vortex.proto.DTypeProtos.Decimal getDefaultInstance() { - return DEFAULT_INSTANCE; - } - - private static final com.google.protobuf.Parser - PARSER = new com.google.protobuf.AbstractParser() { - @java.lang.Override - public Decimal parsePartialFrom( - com.google.protobuf.CodedInputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws com.google.protobuf.InvalidProtocolBufferException { - Builder builder = newBuilder(); - try { - builder.mergeFrom(input, extensionRegistry); - } catch (com.google.protobuf.InvalidProtocolBufferException e) { - throw e.setUnfinishedMessage(builder.buildPartial()); - } catch (com.google.protobuf.UninitializedMessageException e) { - throw e.asInvalidProtocolBufferException().setUnfinishedMessage(builder.buildPartial()); - } catch (java.io.IOException e) { - throw new com.google.protobuf.InvalidProtocolBufferException(e) - .setUnfinishedMessage(builder.buildPartial()); - } - return builder.buildPartial(); - } - }; - - public static com.google.protobuf.Parser parser() { - return PARSER; - } - - @java.lang.Override - public com.google.protobuf.Parser getParserForType() { - return PARSER; - } - - @java.lang.Override - public io.github.dfa1.vortex.proto.DTypeProtos.Decimal getDefaultInstanceForType() { - return DEFAULT_INSTANCE; - } - - } - - public interface Utf8OrBuilder extends - // @@protoc_insertion_point(interface_extends:vortex.dtype.Utf8) - com.google.protobuf.MessageOrBuilder { - - /** - * bool nullable = 1; - * @return The nullable. - */ - boolean getNullable(); - } - /** - * Protobuf type {@code vortex.dtype.Utf8} - */ - public static final class Utf8 extends - com.google.protobuf.GeneratedMessage implements - // @@protoc_insertion_point(message_implements:vortex.dtype.Utf8) - Utf8OrBuilder { - private static final long serialVersionUID = 0L; - static { - com.google.protobuf.RuntimeVersion.validateProtobufGencodeVersion( - com.google.protobuf.RuntimeVersion.RuntimeDomain.PUBLIC, - /* major= */ 4, - /* minor= */ 35, - /* patch= */ 0, - /* suffix= */ "", - "Utf8"); - } - // Use Utf8.newBuilder() to construct. - private Utf8(com.google.protobuf.GeneratedMessage.Builder builder) { - super(builder); - } - private Utf8() { - } - - public static final com.google.protobuf.Descriptors.Descriptor - getDescriptor() { - return io.github.dfa1.vortex.proto.DTypeProtos.internal_static_vortex_dtype_Utf8_descriptor; - } - - @java.lang.Override - public com.google.protobuf.Descriptors.Descriptor getDescriptorForType() { - return io.github.dfa1.vortex.proto.DTypeProtos.internal_static_vortex_dtype_Utf8_descriptor; - } - - @java.lang.Override - protected com.google.protobuf.GeneratedMessage.FieldAccessorTable - internalGetFieldAccessorTable() { - return io.github.dfa1.vortex.proto.DTypeProtos.internal_static_vortex_dtype_Utf8_fieldAccessorTable - .ensureFieldAccessorsInitialized( - io.github.dfa1.vortex.proto.DTypeProtos.Utf8.class, io.github.dfa1.vortex.proto.DTypeProtos.Utf8.Builder.class); - } - - public static final int NULLABLE_FIELD_NUMBER = 1; - private boolean nullable_ = false; - /** - * bool nullable = 1; - * @return The nullable. - */ - @java.lang.Override - public boolean getNullable() { - return nullable_; - } - - private byte memoizedIsInitialized = -1; - @java.lang.Override - public final boolean isInitialized() { - byte isInitialized = memoizedIsInitialized; - if (isInitialized == 1) return true; - if (isInitialized == 0) return false; - - memoizedIsInitialized = 1; - return true; - } - - @java.lang.Override - public void writeTo(com.google.protobuf.CodedOutputStream output) - throws java.io.IOException { - if (nullable_ != false) { - output.writeBool(1, nullable_); - } - getUnknownFields().writeTo(output); - } - private int computeSerializedSize_0() { - int size = 0; - if (nullable_ != false) { - size += com.google.protobuf.CodedOutputStream - .computeBoolSize(1, nullable_); - } - return size; - } - @java.lang.Override - public int getSerializedSize() { - int size = memoizedSize; - if (size != -1) return size; - - size = 0; - size += computeSerializedSize_0(); - size += getUnknownFields().getSerializedSize(); - memoizedSize = size; - return size; - } - - @java.lang.Override - public boolean equals(final java.lang.Object obj) { - if (obj == this) { - return true; - } - if (!(obj instanceof io.github.dfa1.vortex.proto.DTypeProtos.Utf8)) { - return super.equals(obj); - } - io.github.dfa1.vortex.proto.DTypeProtos.Utf8 other = (io.github.dfa1.vortex.proto.DTypeProtos.Utf8) obj; - - if (getNullable() - != other.getNullable()) return false; - if (!getUnknownFields().equals(other.getUnknownFields())) return false; - return true; - } - - @java.lang.Override - public int hashCode() { - if (memoizedHashCode != 0) { - return memoizedHashCode; - } - int hash = 41; - hash = (19 * hash) + getDescriptor().hashCode(); - hash = (37 * hash) + NULLABLE_FIELD_NUMBER; - hash = (53 * hash) + com.google.protobuf.Internal.hashBoolean( - getNullable()); - hash = (29 * hash) + getUnknownFields().hashCode(); - memoizedHashCode = hash; - return hash; - } - - public static io.github.dfa1.vortex.proto.DTypeProtos.Utf8 parseFrom( - java.nio.ByteBuffer data) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data); - } - public static io.github.dfa1.vortex.proto.DTypeProtos.Utf8 parseFrom( - java.nio.ByteBuffer data, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data, extensionRegistry); - } - public static io.github.dfa1.vortex.proto.DTypeProtos.Utf8 parseFrom( - com.google.protobuf.ByteString data) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data); - } - public static io.github.dfa1.vortex.proto.DTypeProtos.Utf8 parseFrom( - com.google.protobuf.ByteString data, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data, extensionRegistry); - } - public static io.github.dfa1.vortex.proto.DTypeProtos.Utf8 parseFrom(byte[] data) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data); - } - public static io.github.dfa1.vortex.proto.DTypeProtos.Utf8 parseFrom( - byte[] data, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data, extensionRegistry); - } - public static io.github.dfa1.vortex.proto.DTypeProtos.Utf8 parseFrom(java.io.InputStream input) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseWithIOException(PARSER, input); - } - public static io.github.dfa1.vortex.proto.DTypeProtos.Utf8 parseFrom( - java.io.InputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseWithIOException(PARSER, input, extensionRegistry); - } - - public static io.github.dfa1.vortex.proto.DTypeProtos.Utf8 parseDelimitedFrom(java.io.InputStream input) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseDelimitedWithIOException(PARSER, input); - } - - public static io.github.dfa1.vortex.proto.DTypeProtos.Utf8 parseDelimitedFrom( - java.io.InputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseDelimitedWithIOException(PARSER, input, extensionRegistry); - } - public static io.github.dfa1.vortex.proto.DTypeProtos.Utf8 parseFrom( - com.google.protobuf.CodedInputStream input) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseWithIOException(PARSER, input); - } - public static io.github.dfa1.vortex.proto.DTypeProtos.Utf8 parseFrom( - com.google.protobuf.CodedInputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseWithIOException(PARSER, input, extensionRegistry); - } - - @java.lang.Override - public Builder newBuilderForType() { return newBuilder(); } - public static Builder newBuilder() { - return DEFAULT_INSTANCE.toBuilder(); - } - public static Builder newBuilder(io.github.dfa1.vortex.proto.DTypeProtos.Utf8 prototype) { - return DEFAULT_INSTANCE.toBuilder().mergeFrom(prototype); - } - @java.lang.Override - public Builder toBuilder() { - return this == DEFAULT_INSTANCE - ? new Builder() : new Builder().mergeFrom(this); - } - - @java.lang.Override - protected Builder newBuilderForType( - com.google.protobuf.GeneratedMessage.BuilderParent parent) { - Builder builder = new Builder(parent); - return builder; - } - /** - * Protobuf type {@code vortex.dtype.Utf8} - */ - public static final class Builder extends - com.google.protobuf.GeneratedMessage.Builder implements - // @@protoc_insertion_point(builder_implements:vortex.dtype.Utf8) - io.github.dfa1.vortex.proto.DTypeProtos.Utf8OrBuilder { - public static final com.google.protobuf.Descriptors.Descriptor - getDescriptor() { - return io.github.dfa1.vortex.proto.DTypeProtos.internal_static_vortex_dtype_Utf8_descriptor; - } - - @java.lang.Override - protected com.google.protobuf.GeneratedMessage.FieldAccessorTable - internalGetFieldAccessorTable() { - return io.github.dfa1.vortex.proto.DTypeProtos.internal_static_vortex_dtype_Utf8_fieldAccessorTable - .ensureFieldAccessorsInitialized( - io.github.dfa1.vortex.proto.DTypeProtos.Utf8.class, io.github.dfa1.vortex.proto.DTypeProtos.Utf8.Builder.class); - } - - // Construct using io.github.dfa1.vortex.proto.DTypeProtos.Utf8.newBuilder() - private Builder() { - - } - - private Builder( - com.google.protobuf.GeneratedMessage.BuilderParent parent) { - super(parent); - - } - @java.lang.Override - public Builder clear() { - super.clear(); - bitField0_ = 0; - nullable_ = false; - return this; - } - - @java.lang.Override - public com.google.protobuf.Descriptors.Descriptor - getDescriptorForType() { - return io.github.dfa1.vortex.proto.DTypeProtos.internal_static_vortex_dtype_Utf8_descriptor; - } - - @java.lang.Override - public io.github.dfa1.vortex.proto.DTypeProtos.Utf8 getDefaultInstanceForType() { - return io.github.dfa1.vortex.proto.DTypeProtos.Utf8.getDefaultInstance(); - } - - @java.lang.Override - public io.github.dfa1.vortex.proto.DTypeProtos.Utf8 build() { - io.github.dfa1.vortex.proto.DTypeProtos.Utf8 result = buildPartial(); - if (!result.isInitialized()) { - throw newUninitializedMessageException(result); - } - return result; - } - - @java.lang.Override - public io.github.dfa1.vortex.proto.DTypeProtos.Utf8 buildPartial() { - io.github.dfa1.vortex.proto.DTypeProtos.Utf8 result = new io.github.dfa1.vortex.proto.DTypeProtos.Utf8(this); - if (bitField0_ != 0) { buildPartial0(result); } - onBuilt(); - return result; - } - - private void buildPartial0(io.github.dfa1.vortex.proto.DTypeProtos.Utf8 result) { - int from_bitField0_ = bitField0_; - if (((from_bitField0_ & 0x00000001) != 0)) { - result.nullable_ = nullable_; - } - } - - @java.lang.Override - public Builder mergeFrom(com.google.protobuf.Message other) { - if (other instanceof io.github.dfa1.vortex.proto.DTypeProtos.Utf8) { - return mergeFrom((io.github.dfa1.vortex.proto.DTypeProtos.Utf8)other); - } else { - super.mergeFrom(other); - return this; - } - } - - public Builder mergeFrom(io.github.dfa1.vortex.proto.DTypeProtos.Utf8 other) { - if (other == io.github.dfa1.vortex.proto.DTypeProtos.Utf8.getDefaultInstance()) return this; - if (other.getNullable() != false) { - setNullable(other.getNullable()); - } - this.mergeUnknownFields(other.getUnknownFields()); - onChanged(); - return this; - } - - @java.lang.Override - public final boolean isInitialized() { - return true; - } - - @java.lang.Override - public Builder mergeFrom( - com.google.protobuf.CodedInputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws java.io.IOException { - if (extensionRegistry == null) { - throw new java.lang.NullPointerException(); - } - try { - boolean done = false; - while (!done) { - int tag = input.readTag(); - switch (tag) { - case 0: - done = true; - break; - case 8: { - nullable_ = input.readBool(); - bitField0_ |= 0x00000001; - break; - } // case 8 - default: { - if (!super.parseUnknownField(input, extensionRegistry, tag)) { - done = true; // was an endgroup tag - } - break; - } // default: - } // switch (tag) - } // while (!done) - } catch (com.google.protobuf.InvalidProtocolBufferException e) { - throw e.unwrapIOException(); - } finally { - onChanged(); - } // finally - return this; - } - private int bitField0_; - - private boolean nullable_ ; - /** - * bool nullable = 1; - * @return The nullable. - */ - @java.lang.Override - public boolean getNullable() { - return nullable_; - } - /** - * bool nullable = 1; - * @param value The nullable to set. - * @return This builder for chaining. - */ - public Builder setNullable(boolean value) { - - nullable_ = value; - bitField0_ |= 0x00000001; - onChanged(); - return this; - } - /** - * bool nullable = 1; - * @return This builder for chaining. - */ - public Builder clearNullable() { - bitField0_ = (bitField0_ & ~0x00000001); - nullable_ = false; - onChanged(); - return this; - } - - // @@protoc_insertion_point(builder_scope:vortex.dtype.Utf8) - } - - // @@protoc_insertion_point(class_scope:vortex.dtype.Utf8) - private static final io.github.dfa1.vortex.proto.DTypeProtos.Utf8 DEFAULT_INSTANCE; - static { - DEFAULT_INSTANCE = new io.github.dfa1.vortex.proto.DTypeProtos.Utf8(); - } - - public static io.github.dfa1.vortex.proto.DTypeProtos.Utf8 getDefaultInstance() { - return DEFAULT_INSTANCE; - } - - private static final com.google.protobuf.Parser - PARSER = new com.google.protobuf.AbstractParser() { - @java.lang.Override - public Utf8 parsePartialFrom( - com.google.protobuf.CodedInputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws com.google.protobuf.InvalidProtocolBufferException { - Builder builder = newBuilder(); - try { - builder.mergeFrom(input, extensionRegistry); - } catch (com.google.protobuf.InvalidProtocolBufferException e) { - throw e.setUnfinishedMessage(builder.buildPartial()); - } catch (com.google.protobuf.UninitializedMessageException e) { - throw e.asInvalidProtocolBufferException().setUnfinishedMessage(builder.buildPartial()); - } catch (java.io.IOException e) { - throw new com.google.protobuf.InvalidProtocolBufferException(e) - .setUnfinishedMessage(builder.buildPartial()); - } - return builder.buildPartial(); - } - }; - - public static com.google.protobuf.Parser parser() { - return PARSER; - } - - @java.lang.Override - public com.google.protobuf.Parser getParserForType() { - return PARSER; - } - - @java.lang.Override - public io.github.dfa1.vortex.proto.DTypeProtos.Utf8 getDefaultInstanceForType() { - return DEFAULT_INSTANCE; - } - - } - - public interface BinaryOrBuilder extends - // @@protoc_insertion_point(interface_extends:vortex.dtype.Binary) - com.google.protobuf.MessageOrBuilder { - - /** - * bool nullable = 1; - * @return The nullable. - */ - boolean getNullable(); - } - /** - * Protobuf type {@code vortex.dtype.Binary} - */ - public static final class Binary extends - com.google.protobuf.GeneratedMessage implements - // @@protoc_insertion_point(message_implements:vortex.dtype.Binary) - BinaryOrBuilder { - private static final long serialVersionUID = 0L; - static { - com.google.protobuf.RuntimeVersion.validateProtobufGencodeVersion( - com.google.protobuf.RuntimeVersion.RuntimeDomain.PUBLIC, - /* major= */ 4, - /* minor= */ 35, - /* patch= */ 0, - /* suffix= */ "", - "Binary"); - } - // Use Binary.newBuilder() to construct. - private Binary(com.google.protobuf.GeneratedMessage.Builder builder) { - super(builder); - } - private Binary() { - } - - public static final com.google.protobuf.Descriptors.Descriptor - getDescriptor() { - return io.github.dfa1.vortex.proto.DTypeProtos.internal_static_vortex_dtype_Binary_descriptor; - } - - @java.lang.Override - public com.google.protobuf.Descriptors.Descriptor getDescriptorForType() { - return io.github.dfa1.vortex.proto.DTypeProtos.internal_static_vortex_dtype_Binary_descriptor; - } - - @java.lang.Override - protected com.google.protobuf.GeneratedMessage.FieldAccessorTable - internalGetFieldAccessorTable() { - return io.github.dfa1.vortex.proto.DTypeProtos.internal_static_vortex_dtype_Binary_fieldAccessorTable - .ensureFieldAccessorsInitialized( - io.github.dfa1.vortex.proto.DTypeProtos.Binary.class, io.github.dfa1.vortex.proto.DTypeProtos.Binary.Builder.class); - } - - public static final int NULLABLE_FIELD_NUMBER = 1; - private boolean nullable_ = false; - /** - * bool nullable = 1; - * @return The nullable. - */ - @java.lang.Override - public boolean getNullable() { - return nullable_; - } - - private byte memoizedIsInitialized = -1; - @java.lang.Override - public final boolean isInitialized() { - byte isInitialized = memoizedIsInitialized; - if (isInitialized == 1) return true; - if (isInitialized == 0) return false; - - memoizedIsInitialized = 1; - return true; - } - - @java.lang.Override - public void writeTo(com.google.protobuf.CodedOutputStream output) - throws java.io.IOException { - if (nullable_ != false) { - output.writeBool(1, nullable_); - } - getUnknownFields().writeTo(output); - } - private int computeSerializedSize_0() { - int size = 0; - if (nullable_ != false) { - size += com.google.protobuf.CodedOutputStream - .computeBoolSize(1, nullable_); - } - return size; - } - @java.lang.Override - public int getSerializedSize() { - int size = memoizedSize; - if (size != -1) return size; - - size = 0; - size += computeSerializedSize_0(); - size += getUnknownFields().getSerializedSize(); - memoizedSize = size; - return size; - } - - @java.lang.Override - public boolean equals(final java.lang.Object obj) { - if (obj == this) { - return true; - } - if (!(obj instanceof io.github.dfa1.vortex.proto.DTypeProtos.Binary)) { - return super.equals(obj); - } - io.github.dfa1.vortex.proto.DTypeProtos.Binary other = (io.github.dfa1.vortex.proto.DTypeProtos.Binary) obj; - - if (getNullable() - != other.getNullable()) return false; - if (!getUnknownFields().equals(other.getUnknownFields())) return false; - return true; - } - - @java.lang.Override - public int hashCode() { - if (memoizedHashCode != 0) { - return memoizedHashCode; - } - int hash = 41; - hash = (19 * hash) + getDescriptor().hashCode(); - hash = (37 * hash) + NULLABLE_FIELD_NUMBER; - hash = (53 * hash) + com.google.protobuf.Internal.hashBoolean( - getNullable()); - hash = (29 * hash) + getUnknownFields().hashCode(); - memoizedHashCode = hash; - return hash; - } - - public static io.github.dfa1.vortex.proto.DTypeProtos.Binary parseFrom( - java.nio.ByteBuffer data) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data); - } - public static io.github.dfa1.vortex.proto.DTypeProtos.Binary parseFrom( - java.nio.ByteBuffer data, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data, extensionRegistry); - } - public static io.github.dfa1.vortex.proto.DTypeProtos.Binary parseFrom( - com.google.protobuf.ByteString data) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data); - } - public static io.github.dfa1.vortex.proto.DTypeProtos.Binary parseFrom( - com.google.protobuf.ByteString data, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data, extensionRegistry); - } - public static io.github.dfa1.vortex.proto.DTypeProtos.Binary parseFrom(byte[] data) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data); - } - public static io.github.dfa1.vortex.proto.DTypeProtos.Binary parseFrom( - byte[] data, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data, extensionRegistry); - } - public static io.github.dfa1.vortex.proto.DTypeProtos.Binary parseFrom(java.io.InputStream input) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseWithIOException(PARSER, input); - } - public static io.github.dfa1.vortex.proto.DTypeProtos.Binary parseFrom( - java.io.InputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseWithIOException(PARSER, input, extensionRegistry); - } - - public static io.github.dfa1.vortex.proto.DTypeProtos.Binary parseDelimitedFrom(java.io.InputStream input) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseDelimitedWithIOException(PARSER, input); - } - - public static io.github.dfa1.vortex.proto.DTypeProtos.Binary parseDelimitedFrom( - java.io.InputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseDelimitedWithIOException(PARSER, input, extensionRegistry); - } - public static io.github.dfa1.vortex.proto.DTypeProtos.Binary parseFrom( - com.google.protobuf.CodedInputStream input) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseWithIOException(PARSER, input); - } - public static io.github.dfa1.vortex.proto.DTypeProtos.Binary parseFrom( - com.google.protobuf.CodedInputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseWithIOException(PARSER, input, extensionRegistry); - } - - @java.lang.Override - public Builder newBuilderForType() { return newBuilder(); } - public static Builder newBuilder() { - return DEFAULT_INSTANCE.toBuilder(); - } - public static Builder newBuilder(io.github.dfa1.vortex.proto.DTypeProtos.Binary prototype) { - return DEFAULT_INSTANCE.toBuilder().mergeFrom(prototype); - } - @java.lang.Override - public Builder toBuilder() { - return this == DEFAULT_INSTANCE - ? new Builder() : new Builder().mergeFrom(this); - } - - @java.lang.Override - protected Builder newBuilderForType( - com.google.protobuf.GeneratedMessage.BuilderParent parent) { - Builder builder = new Builder(parent); - return builder; - } - /** - * Protobuf type {@code vortex.dtype.Binary} - */ - public static final class Builder extends - com.google.protobuf.GeneratedMessage.Builder implements - // @@protoc_insertion_point(builder_implements:vortex.dtype.Binary) - io.github.dfa1.vortex.proto.DTypeProtos.BinaryOrBuilder { - public static final com.google.protobuf.Descriptors.Descriptor - getDescriptor() { - return io.github.dfa1.vortex.proto.DTypeProtos.internal_static_vortex_dtype_Binary_descriptor; - } - - @java.lang.Override - protected com.google.protobuf.GeneratedMessage.FieldAccessorTable - internalGetFieldAccessorTable() { - return io.github.dfa1.vortex.proto.DTypeProtos.internal_static_vortex_dtype_Binary_fieldAccessorTable - .ensureFieldAccessorsInitialized( - io.github.dfa1.vortex.proto.DTypeProtos.Binary.class, io.github.dfa1.vortex.proto.DTypeProtos.Binary.Builder.class); - } - - // Construct using io.github.dfa1.vortex.proto.DTypeProtos.Binary.newBuilder() - private Builder() { - - } - - private Builder( - com.google.protobuf.GeneratedMessage.BuilderParent parent) { - super(parent); - - } - @java.lang.Override - public Builder clear() { - super.clear(); - bitField0_ = 0; - nullable_ = false; - return this; - } - - @java.lang.Override - public com.google.protobuf.Descriptors.Descriptor - getDescriptorForType() { - return io.github.dfa1.vortex.proto.DTypeProtos.internal_static_vortex_dtype_Binary_descriptor; - } - - @java.lang.Override - public io.github.dfa1.vortex.proto.DTypeProtos.Binary getDefaultInstanceForType() { - return io.github.dfa1.vortex.proto.DTypeProtos.Binary.getDefaultInstance(); - } - - @java.lang.Override - public io.github.dfa1.vortex.proto.DTypeProtos.Binary build() { - io.github.dfa1.vortex.proto.DTypeProtos.Binary result = buildPartial(); - if (!result.isInitialized()) { - throw newUninitializedMessageException(result); - } - return result; - } - - @java.lang.Override - public io.github.dfa1.vortex.proto.DTypeProtos.Binary buildPartial() { - io.github.dfa1.vortex.proto.DTypeProtos.Binary result = new io.github.dfa1.vortex.proto.DTypeProtos.Binary(this); - if (bitField0_ != 0) { buildPartial0(result); } - onBuilt(); - return result; - } - - private void buildPartial0(io.github.dfa1.vortex.proto.DTypeProtos.Binary result) { - int from_bitField0_ = bitField0_; - if (((from_bitField0_ & 0x00000001) != 0)) { - result.nullable_ = nullable_; - } - } - - @java.lang.Override - public Builder mergeFrom(com.google.protobuf.Message other) { - if (other instanceof io.github.dfa1.vortex.proto.DTypeProtos.Binary) { - return mergeFrom((io.github.dfa1.vortex.proto.DTypeProtos.Binary)other); - } else { - super.mergeFrom(other); - return this; - } - } - - public Builder mergeFrom(io.github.dfa1.vortex.proto.DTypeProtos.Binary other) { - if (other == io.github.dfa1.vortex.proto.DTypeProtos.Binary.getDefaultInstance()) return this; - if (other.getNullable() != false) { - setNullable(other.getNullable()); - } - this.mergeUnknownFields(other.getUnknownFields()); - onChanged(); - return this; - } - - @java.lang.Override - public final boolean isInitialized() { - return true; - } - - @java.lang.Override - public Builder mergeFrom( - com.google.protobuf.CodedInputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws java.io.IOException { - if (extensionRegistry == null) { - throw new java.lang.NullPointerException(); - } - try { - boolean done = false; - while (!done) { - int tag = input.readTag(); - switch (tag) { - case 0: - done = true; - break; - case 8: { - nullable_ = input.readBool(); - bitField0_ |= 0x00000001; - break; - } // case 8 - default: { - if (!super.parseUnknownField(input, extensionRegistry, tag)) { - done = true; // was an endgroup tag - } - break; - } // default: - } // switch (tag) - } // while (!done) - } catch (com.google.protobuf.InvalidProtocolBufferException e) { - throw e.unwrapIOException(); - } finally { - onChanged(); - } // finally - return this; - } - private int bitField0_; - - private boolean nullable_ ; - /** - * bool nullable = 1; - * @return The nullable. - */ - @java.lang.Override - public boolean getNullable() { - return nullable_; - } - /** - * bool nullable = 1; - * @param value The nullable to set. - * @return This builder for chaining. - */ - public Builder setNullable(boolean value) { - - nullable_ = value; - bitField0_ |= 0x00000001; - onChanged(); - return this; - } - /** - * bool nullable = 1; - * @return This builder for chaining. - */ - public Builder clearNullable() { - bitField0_ = (bitField0_ & ~0x00000001); - nullable_ = false; - onChanged(); - return this; - } - - // @@protoc_insertion_point(builder_scope:vortex.dtype.Binary) - } - - // @@protoc_insertion_point(class_scope:vortex.dtype.Binary) - private static final io.github.dfa1.vortex.proto.DTypeProtos.Binary DEFAULT_INSTANCE; - static { - DEFAULT_INSTANCE = new io.github.dfa1.vortex.proto.DTypeProtos.Binary(); - } - - public static io.github.dfa1.vortex.proto.DTypeProtos.Binary getDefaultInstance() { - return DEFAULT_INSTANCE; - } - - private static final com.google.protobuf.Parser - PARSER = new com.google.protobuf.AbstractParser() { - @java.lang.Override - public Binary parsePartialFrom( - com.google.protobuf.CodedInputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws com.google.protobuf.InvalidProtocolBufferException { - Builder builder = newBuilder(); - try { - builder.mergeFrom(input, extensionRegistry); - } catch (com.google.protobuf.InvalidProtocolBufferException e) { - throw e.setUnfinishedMessage(builder.buildPartial()); - } catch (com.google.protobuf.UninitializedMessageException e) { - throw e.asInvalidProtocolBufferException().setUnfinishedMessage(builder.buildPartial()); - } catch (java.io.IOException e) { - throw new com.google.protobuf.InvalidProtocolBufferException(e) - .setUnfinishedMessage(builder.buildPartial()); - } - return builder.buildPartial(); - } - }; - - public static com.google.protobuf.Parser parser() { - return PARSER; - } - - @java.lang.Override - public com.google.protobuf.Parser getParserForType() { - return PARSER; - } - - @java.lang.Override - public io.github.dfa1.vortex.proto.DTypeProtos.Binary getDefaultInstanceForType() { - return DEFAULT_INSTANCE; - } - - } - - public interface StructOrBuilder extends - // @@protoc_insertion_point(interface_extends:vortex.dtype.Struct) - com.google.protobuf.MessageOrBuilder { - - /** - * repeated string names = 1; - * @return A list containing the names. - */ - java.util.List - getNamesList(); - /** - * repeated string names = 1; - * @return The count of names. - */ - int getNamesCount(); - /** - * repeated string names = 1; - * @param index The index of the element to return. - * @return The names at the given index. - */ - java.lang.String getNames(int index); - /** - * repeated string names = 1; - * @param index The index of the value to return. - * @return The bytes of the names at the given index. - */ - com.google.protobuf.ByteString - getNamesBytes(int index); - - /** - * repeated .vortex.dtype.DType dtypes = 2; - */ - java.util.List - getDtypesList(); - /** - * repeated .vortex.dtype.DType dtypes = 2; - */ - io.github.dfa1.vortex.proto.DTypeProtos.DType getDtypes(int index); - /** - * repeated .vortex.dtype.DType dtypes = 2; - */ - int getDtypesCount(); - /** - * repeated .vortex.dtype.DType dtypes = 2; - */ - java.util.List - getDtypesOrBuilderList(); - /** - * repeated .vortex.dtype.DType dtypes = 2; - */ - io.github.dfa1.vortex.proto.DTypeProtos.DTypeOrBuilder getDtypesOrBuilder( - int index); - - /** - * bool nullable = 3; - * @return The nullable. - */ - boolean getNullable(); - } - /** - * Protobuf type {@code vortex.dtype.Struct} - */ - public static final class Struct extends - com.google.protobuf.GeneratedMessage implements - // @@protoc_insertion_point(message_implements:vortex.dtype.Struct) - StructOrBuilder { - private static final long serialVersionUID = 0L; - static { - com.google.protobuf.RuntimeVersion.validateProtobufGencodeVersion( - com.google.protobuf.RuntimeVersion.RuntimeDomain.PUBLIC, - /* major= */ 4, - /* minor= */ 35, - /* patch= */ 0, - /* suffix= */ "", - "Struct"); - } - // Use Struct.newBuilder() to construct. - private Struct(com.google.protobuf.GeneratedMessage.Builder builder) { - super(builder); - } - private Struct() { - names_ = - com.google.protobuf.LazyStringArrayList.emptyList(); - dtypes_ = java.util.Collections.emptyList(); - } - - public static final com.google.protobuf.Descriptors.Descriptor - getDescriptor() { - return io.github.dfa1.vortex.proto.DTypeProtos.internal_static_vortex_dtype_Struct_descriptor; - } - - @java.lang.Override - public com.google.protobuf.Descriptors.Descriptor getDescriptorForType() { - return io.github.dfa1.vortex.proto.DTypeProtos.internal_static_vortex_dtype_Struct_descriptor; - } - - @java.lang.Override - protected com.google.protobuf.GeneratedMessage.FieldAccessorTable - internalGetFieldAccessorTable() { - return io.github.dfa1.vortex.proto.DTypeProtos.internal_static_vortex_dtype_Struct_fieldAccessorTable - .ensureFieldAccessorsInitialized( - io.github.dfa1.vortex.proto.DTypeProtos.Struct.class, io.github.dfa1.vortex.proto.DTypeProtos.Struct.Builder.class); - } - - public static final int NAMES_FIELD_NUMBER = 1; - @SuppressWarnings("serial") - private com.google.protobuf.LazyStringArrayList names_ = - com.google.protobuf.LazyStringArrayList.emptyList(); - /** - * repeated string names = 1; - * @return A list containing the names. - */ - public com.google.protobuf.ProtocolStringList - getNamesList() { - return names_; - } - /** - * repeated string names = 1; - * @return The count of names. - */ - public int getNamesCount() { - return names_.size(); - } - /** - * repeated string names = 1; - * @param index The index of the element to return. - * @return The names at the given index. - */ - public java.lang.String getNames(int index) { - return names_.get(index); - } - /** - * repeated string names = 1; - * @param index The index of the value to return. - * @return The bytes of the names at the given index. - */ - public com.google.protobuf.ByteString - getNamesBytes(int index) { - return names_.getByteString(index); - } - - public static final int DTYPES_FIELD_NUMBER = 2; - @SuppressWarnings("serial") - private java.util.List dtypes_; - /** - * repeated .vortex.dtype.DType dtypes = 2; - */ - @java.lang.Override - public java.util.List getDtypesList() { - return dtypes_; - } - /** - * repeated .vortex.dtype.DType dtypes = 2; - */ - @java.lang.Override - public java.util.List - getDtypesOrBuilderList() { - return dtypes_; - } - /** - * repeated .vortex.dtype.DType dtypes = 2; - */ - @java.lang.Override - public int getDtypesCount() { - return dtypes_.size(); - } - /** - * repeated .vortex.dtype.DType dtypes = 2; - */ - @java.lang.Override - public io.github.dfa1.vortex.proto.DTypeProtos.DType getDtypes(int index) { - return dtypes_.get(index); - } - /** - * repeated .vortex.dtype.DType dtypes = 2; - */ - @java.lang.Override - public io.github.dfa1.vortex.proto.DTypeProtos.DTypeOrBuilder getDtypesOrBuilder( - int index) { - return dtypes_.get(index); - } - - public static final int NULLABLE_FIELD_NUMBER = 3; - private boolean nullable_ = false; - /** - * bool nullable = 3; - * @return The nullable. - */ - @java.lang.Override - public boolean getNullable() { - return nullable_; - } - - private byte memoizedIsInitialized = -1; - @java.lang.Override - public final boolean isInitialized() { - byte isInitialized = memoizedIsInitialized; - if (isInitialized == 1) return true; - if (isInitialized == 0) return false; - - memoizedIsInitialized = 1; - return true; - } - - @java.lang.Override - public void writeTo(com.google.protobuf.CodedOutputStream output) - throws java.io.IOException { - for (int i = 0; i < names_.size(); i++) { - com.google.protobuf.GeneratedMessage.writeString(output, 1, names_.getRaw(i)); - } - for (int i = 0; i < dtypes_.size(); i++) { - output.writeMessage(2, dtypes_.get(i)); - } - if (nullable_ != false) { - output.writeBool(3, nullable_); - } - getUnknownFields().writeTo(output); - } - private int computeSerializedSize_0() { - int size = 0; - { - int dataSize = 0; - for (int i = 0; i < names_.size(); i++) { - dataSize += computeStringSizeNoTag(names_.getRaw(i)); - } - size += dataSize; - size += 1 * getNamesList().size(); - } - - { - final int count = dtypes_.size(); - for (int i = 0; i < count; i++) { - size += com.google.protobuf.CodedOutputStream - .computeMessageSizeNoTag(dtypes_.get(i)); - } - size += 1 * count; - } - if (nullable_ != false) { - size += com.google.protobuf.CodedOutputStream - .computeBoolSize(3, nullable_); - } - return size; - } - @java.lang.Override - public int getSerializedSize() { - int size = memoizedSize; - if (size != -1) return size; - - size = 0; - size += computeSerializedSize_0(); - size += getUnknownFields().getSerializedSize(); - memoizedSize = size; - return size; - } - - @java.lang.Override - public boolean equals(final java.lang.Object obj) { - if (obj == this) { - return true; - } - if (!(obj instanceof io.github.dfa1.vortex.proto.DTypeProtos.Struct)) { - return super.equals(obj); - } - io.github.dfa1.vortex.proto.DTypeProtos.Struct other = (io.github.dfa1.vortex.proto.DTypeProtos.Struct) obj; - - if (!getNamesList() - .equals(other.getNamesList())) return false; - if (!getDtypesList() - .equals(other.getDtypesList())) return false; - if (getNullable() - != other.getNullable()) return false; - if (!getUnknownFields().equals(other.getUnknownFields())) return false; - return true; - } - - @java.lang.Override - public int hashCode() { - if (memoizedHashCode != 0) { - return memoizedHashCode; - } - int hash = 41; - hash = (19 * hash) + getDescriptor().hashCode(); - if (getNamesCount() > 0) { - hash = (37 * hash) + NAMES_FIELD_NUMBER; - hash = (53 * hash) + getNamesList().hashCode(); - } - if (getDtypesCount() > 0) { - hash = (37 * hash) + DTYPES_FIELD_NUMBER; - hash = (53 * hash) + getDtypesList().hashCode(); - } - hash = (37 * hash) + NULLABLE_FIELD_NUMBER; - hash = (53 * hash) + com.google.protobuf.Internal.hashBoolean( - getNullable()); - hash = (29 * hash) + getUnknownFields().hashCode(); - memoizedHashCode = hash; - return hash; - } - - public static io.github.dfa1.vortex.proto.DTypeProtos.Struct parseFrom( - java.nio.ByteBuffer data) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data); - } - public static io.github.dfa1.vortex.proto.DTypeProtos.Struct parseFrom( - java.nio.ByteBuffer data, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data, extensionRegistry); - } - public static io.github.dfa1.vortex.proto.DTypeProtos.Struct parseFrom( - com.google.protobuf.ByteString data) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data); - } - public static io.github.dfa1.vortex.proto.DTypeProtos.Struct parseFrom( - com.google.protobuf.ByteString data, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data, extensionRegistry); - } - public static io.github.dfa1.vortex.proto.DTypeProtos.Struct parseFrom(byte[] data) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data); - } - public static io.github.dfa1.vortex.proto.DTypeProtos.Struct parseFrom( - byte[] data, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data, extensionRegistry); - } - public static io.github.dfa1.vortex.proto.DTypeProtos.Struct parseFrom(java.io.InputStream input) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseWithIOException(PARSER, input); - } - public static io.github.dfa1.vortex.proto.DTypeProtos.Struct parseFrom( - java.io.InputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseWithIOException(PARSER, input, extensionRegistry); - } - - public static io.github.dfa1.vortex.proto.DTypeProtos.Struct parseDelimitedFrom(java.io.InputStream input) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseDelimitedWithIOException(PARSER, input); - } - - public static io.github.dfa1.vortex.proto.DTypeProtos.Struct parseDelimitedFrom( - java.io.InputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseDelimitedWithIOException(PARSER, input, extensionRegistry); - } - public static io.github.dfa1.vortex.proto.DTypeProtos.Struct parseFrom( - com.google.protobuf.CodedInputStream input) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseWithIOException(PARSER, input); - } - public static io.github.dfa1.vortex.proto.DTypeProtos.Struct parseFrom( - com.google.protobuf.CodedInputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseWithIOException(PARSER, input, extensionRegistry); - } - - @java.lang.Override - public Builder newBuilderForType() { return newBuilder(); } - public static Builder newBuilder() { - return DEFAULT_INSTANCE.toBuilder(); - } - public static Builder newBuilder(io.github.dfa1.vortex.proto.DTypeProtos.Struct prototype) { - return DEFAULT_INSTANCE.toBuilder().mergeFrom(prototype); - } - @java.lang.Override - public Builder toBuilder() { - return this == DEFAULT_INSTANCE - ? new Builder() : new Builder().mergeFrom(this); - } - - @java.lang.Override - protected Builder newBuilderForType( - com.google.protobuf.GeneratedMessage.BuilderParent parent) { - Builder builder = new Builder(parent); - return builder; - } - /** - * Protobuf type {@code vortex.dtype.Struct} - */ - public static final class Builder extends - com.google.protobuf.GeneratedMessage.Builder implements - // @@protoc_insertion_point(builder_implements:vortex.dtype.Struct) - io.github.dfa1.vortex.proto.DTypeProtos.StructOrBuilder { - public static final com.google.protobuf.Descriptors.Descriptor - getDescriptor() { - return io.github.dfa1.vortex.proto.DTypeProtos.internal_static_vortex_dtype_Struct_descriptor; - } - - @java.lang.Override - protected com.google.protobuf.GeneratedMessage.FieldAccessorTable - internalGetFieldAccessorTable() { - return io.github.dfa1.vortex.proto.DTypeProtos.internal_static_vortex_dtype_Struct_fieldAccessorTable - .ensureFieldAccessorsInitialized( - io.github.dfa1.vortex.proto.DTypeProtos.Struct.class, io.github.dfa1.vortex.proto.DTypeProtos.Struct.Builder.class); - } - - // Construct using io.github.dfa1.vortex.proto.DTypeProtos.Struct.newBuilder() - private Builder() { - - } - - private Builder( - com.google.protobuf.GeneratedMessage.BuilderParent parent) { - super(parent); - - } - @java.lang.Override - public Builder clear() { - super.clear(); - bitField0_ = 0; - names_ = - com.google.protobuf.LazyStringArrayList.emptyList(); - if (dtypesBuilder_ == null) { - dtypes_ = java.util.Collections.emptyList(); - } else { - dtypes_ = null; - dtypesBuilder_.clear(); - } - bitField0_ = (bitField0_ & ~0x00000002); - nullable_ = false; - return this; - } - - @java.lang.Override - public com.google.protobuf.Descriptors.Descriptor - getDescriptorForType() { - return io.github.dfa1.vortex.proto.DTypeProtos.internal_static_vortex_dtype_Struct_descriptor; - } - - @java.lang.Override - public io.github.dfa1.vortex.proto.DTypeProtos.Struct getDefaultInstanceForType() { - return io.github.dfa1.vortex.proto.DTypeProtos.Struct.getDefaultInstance(); - } - - @java.lang.Override - public io.github.dfa1.vortex.proto.DTypeProtos.Struct build() { - io.github.dfa1.vortex.proto.DTypeProtos.Struct result = buildPartial(); - if (!result.isInitialized()) { - throw newUninitializedMessageException(result); - } - return result; - } - - @java.lang.Override - public io.github.dfa1.vortex.proto.DTypeProtos.Struct buildPartial() { - io.github.dfa1.vortex.proto.DTypeProtos.Struct result = new io.github.dfa1.vortex.proto.DTypeProtos.Struct(this); - buildPartialRepeatedFields(result); - if (bitField0_ != 0) { buildPartial0(result); } - onBuilt(); - return result; - } - - private void buildPartialRepeatedFields(io.github.dfa1.vortex.proto.DTypeProtos.Struct result) { - if (dtypesBuilder_ == null) { - if (((bitField0_ & 0x00000002) != 0)) { - dtypes_ = java.util.Collections.unmodifiableList(dtypes_); - bitField0_ = (bitField0_ & ~0x00000002); - } - result.dtypes_ = dtypes_; - } else { - result.dtypes_ = dtypesBuilder_.build(); - } - } - - private void buildPartial0(io.github.dfa1.vortex.proto.DTypeProtos.Struct result) { - int from_bitField0_ = bitField0_; - if (((from_bitField0_ & 0x00000001) != 0)) { - names_.makeImmutable(); - result.names_ = names_; - } - if (((from_bitField0_ & 0x00000004) != 0)) { - result.nullable_ = nullable_; - } - } - - @java.lang.Override - public Builder mergeFrom(com.google.protobuf.Message other) { - if (other instanceof io.github.dfa1.vortex.proto.DTypeProtos.Struct) { - return mergeFrom((io.github.dfa1.vortex.proto.DTypeProtos.Struct)other); - } else { - super.mergeFrom(other); - return this; - } - } - - public Builder mergeFrom(io.github.dfa1.vortex.proto.DTypeProtos.Struct other) { - if (other == io.github.dfa1.vortex.proto.DTypeProtos.Struct.getDefaultInstance()) return this; - if (!other.names_.isEmpty()) { - if (names_.isEmpty()) { - names_ = other.names_; - bitField0_ |= 0x00000001; - } else { - ensureNamesIsMutable(); - names_.addAll(other.names_); - } - onChanged(); - } - if (dtypesBuilder_ == null) { - if (!other.dtypes_.isEmpty()) { - if (dtypes_.isEmpty()) { - dtypes_ = other.dtypes_; - bitField0_ = (bitField0_ & ~0x00000002); - } else { - ensureDtypesIsMutable(); - dtypes_.addAll(other.dtypes_); - } - onChanged(); - } - } else { - if (!other.dtypes_.isEmpty()) { - if (dtypesBuilder_.isEmpty()) { - dtypesBuilder_.dispose(); - dtypesBuilder_ = null; - dtypes_ = other.dtypes_; - bitField0_ = (bitField0_ & ~0x00000002); - dtypesBuilder_ = - com.google.protobuf.GeneratedMessage.alwaysUseFieldBuilders ? - internalGetDtypesFieldBuilder() : null; - } else { - dtypesBuilder_.addAllMessages(other.dtypes_); - } - } - } - if (other.getNullable() != false) { - setNullable(other.getNullable()); - } - this.mergeUnknownFields(other.getUnknownFields()); - onChanged(); - return this; - } - - @java.lang.Override - public final boolean isInitialized() { - return true; - } - - @java.lang.Override - public Builder mergeFrom( - com.google.protobuf.CodedInputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws java.io.IOException { - if (extensionRegistry == null) { - throw new java.lang.NullPointerException(); - } - try { - boolean done = false; - while (!done) { - int tag = input.readTag(); - switch (tag) { - case 0: - done = true; - break; - case 10: { - ensureNamesIsMutable(); - names_.add(input.readStringRequireUtf8()); - break; - } // case 10 - case 18: { - io.github.dfa1.vortex.proto.DTypeProtos.DType m = - input.readMessage( - io.github.dfa1.vortex.proto.DTypeProtos.DType.parser(), - extensionRegistry); - if (dtypesBuilder_ == null) { - ensureDtypesIsMutable(); - dtypes_.add(m); - } else { - dtypesBuilder_.addMessage(m); - } - break; - } // case 18 - case 24: { - nullable_ = input.readBool(); - bitField0_ |= 0x00000004; - break; - } // case 24 - default: { - if (!super.parseUnknownField(input, extensionRegistry, tag)) { - done = true; // was an endgroup tag - } - break; - } // default: - } // switch (tag) - } // while (!done) - } catch (com.google.protobuf.InvalidProtocolBufferException e) { - throw e.unwrapIOException(); - } finally { - onChanged(); - } // finally - return this; - } - private int bitField0_; - - private com.google.protobuf.LazyStringArrayList names_ = - com.google.protobuf.LazyStringArrayList.emptyList(); - private void ensureNamesIsMutable() { - if (!names_.isModifiable()) { - names_ = new com.google.protobuf.LazyStringArrayList(names_); - } - bitField0_ |= 0x00000001; - } - /** - * repeated string names = 1; - * @return A list containing the names. - */ - public com.google.protobuf.ProtocolStringList - getNamesList() { - names_.makeImmutable(); - return names_; - } - /** - * repeated string names = 1; - * @return The count of names. - */ - public int getNamesCount() { - return names_.size(); - } - /** - * repeated string names = 1; - * @param index The index of the element to return. - * @return The names at the given index. - */ - public java.lang.String getNames(int index) { - return names_.get(index); - } - /** - * repeated string names = 1; - * @param index The index of the value to return. - * @return The bytes of the names at the given index. - */ - public com.google.protobuf.ByteString - getNamesBytes(int index) { - return names_.getByteString(index); - } - /** - * repeated string names = 1; - * @param index The index to set the value at. - * @param value The names to set. - * @return This builder for chaining. - */ - public Builder setNames( - int index, java.lang.String value) { - if (value == null) { throw new NullPointerException(); } - ensureNamesIsMutable(); - names_.set(index, value); - bitField0_ |= 0x00000001; - onChanged(); - return this; - } - /** - * repeated string names = 1; - * @param value The names to add. - * @return This builder for chaining. - */ - public Builder addNames( - java.lang.String value) { - if (value == null) { throw new NullPointerException(); } - ensureNamesIsMutable(); - names_.add(value); - bitField0_ |= 0x00000001; - onChanged(); - return this; - } - /** - * repeated string names = 1; - * @param values The names to add. - * @return This builder for chaining. - */ - public Builder addAllNames( - java.lang.Iterable values) { - ensureNamesIsMutable(); - com.google.protobuf.AbstractMessageLite.Builder.addAll( - values, names_); - bitField0_ |= 0x00000001; - onChanged(); - return this; - } - /** - * repeated string names = 1; - * @return This builder for chaining. - */ - public Builder clearNames() { - names_ = - com.google.protobuf.LazyStringArrayList.emptyList(); - bitField0_ = (bitField0_ & ~0x00000001);; - onChanged(); - return this; - } - /** - * repeated string names = 1; - * @param value The bytes of the names to add. - * @return This builder for chaining. - */ - public Builder addNamesBytes( - com.google.protobuf.ByteString value) { - if (value == null) { throw new NullPointerException(); } - checkByteStringIsUtf8(value); - ensureNamesIsMutable(); - names_.add(value); - bitField0_ |= 0x00000001; - onChanged(); - return this; - } - - private java.util.List dtypes_ = - java.util.Collections.emptyList(); - private void ensureDtypesIsMutable() { - if (!((bitField0_ & 0x00000002) != 0)) { - dtypes_ = new java.util.ArrayList(dtypes_); - bitField0_ |= 0x00000002; - } - } - - private com.google.protobuf.RepeatedFieldBuilder< - io.github.dfa1.vortex.proto.DTypeProtos.DType, io.github.dfa1.vortex.proto.DTypeProtos.DType.Builder, io.github.dfa1.vortex.proto.DTypeProtos.DTypeOrBuilder> dtypesBuilder_; - - /** - * repeated .vortex.dtype.DType dtypes = 2; - */ - public java.util.List getDtypesList() { - if (dtypesBuilder_ == null) { - return java.util.Collections.unmodifiableList(dtypes_); - } else { - return dtypesBuilder_.getMessageList(); - } - } - /** - * repeated .vortex.dtype.DType dtypes = 2; - */ - public int getDtypesCount() { - if (dtypesBuilder_ == null) { - return dtypes_.size(); - } else { - return dtypesBuilder_.getCount(); - } - } - /** - * repeated .vortex.dtype.DType dtypes = 2; - */ - public io.github.dfa1.vortex.proto.DTypeProtos.DType getDtypes(int index) { - if (dtypesBuilder_ == null) { - return dtypes_.get(index); - } else { - return dtypesBuilder_.getMessage(index); - } - } - /** - * repeated .vortex.dtype.DType dtypes = 2; - */ - public Builder setDtypes( - int index, io.github.dfa1.vortex.proto.DTypeProtos.DType value) { - if (dtypesBuilder_ == null) { - if (value == null) { - throw new NullPointerException(); - } - ensureDtypesIsMutable(); - dtypes_.set(index, value); - onChanged(); - } else { - dtypesBuilder_.setMessage(index, value); - } - return this; - } - /** - * repeated .vortex.dtype.DType dtypes = 2; - */ - public Builder setDtypes( - int index, io.github.dfa1.vortex.proto.DTypeProtos.DType.Builder builderForValue) { - if (dtypesBuilder_ == null) { - ensureDtypesIsMutable(); - dtypes_.set(index, builderForValue.build()); - onChanged(); - } else { - dtypesBuilder_.setMessage(index, builderForValue.build()); - } - return this; - } - /** - * repeated .vortex.dtype.DType dtypes = 2; - */ - public Builder addDtypes(io.github.dfa1.vortex.proto.DTypeProtos.DType value) { - if (dtypesBuilder_ == null) { - if (value == null) { - throw new NullPointerException(); - } - ensureDtypesIsMutable(); - dtypes_.add(value); - onChanged(); - } else { - dtypesBuilder_.addMessage(value); - } - return this; - } - /** - * repeated .vortex.dtype.DType dtypes = 2; - */ - public Builder addDtypes( - int index, io.github.dfa1.vortex.proto.DTypeProtos.DType value) { - if (dtypesBuilder_ == null) { - if (value == null) { - throw new NullPointerException(); - } - ensureDtypesIsMutable(); - dtypes_.add(index, value); - onChanged(); - } else { - dtypesBuilder_.addMessage(index, value); - } - return this; - } - /** - * repeated .vortex.dtype.DType dtypes = 2; - */ - public Builder addDtypes( - io.github.dfa1.vortex.proto.DTypeProtos.DType.Builder builderForValue) { - if (dtypesBuilder_ == null) { - ensureDtypesIsMutable(); - dtypes_.add(builderForValue.build()); - onChanged(); - } else { - dtypesBuilder_.addMessage(builderForValue.build()); - } - return this; - } - /** - * repeated .vortex.dtype.DType dtypes = 2; - */ - public Builder addDtypes( - int index, io.github.dfa1.vortex.proto.DTypeProtos.DType.Builder builderForValue) { - if (dtypesBuilder_ == null) { - ensureDtypesIsMutable(); - dtypes_.add(index, builderForValue.build()); - onChanged(); - } else { - dtypesBuilder_.addMessage(index, builderForValue.build()); - } - return this; - } - /** - * repeated .vortex.dtype.DType dtypes = 2; - */ - public Builder addAllDtypes( - java.lang.Iterable values) { - if (dtypesBuilder_ == null) { - ensureDtypesIsMutable(); - com.google.protobuf.AbstractMessageLite.Builder.addAll( - values, dtypes_); - onChanged(); - } else { - dtypesBuilder_.addAllMessages(values); - } - return this; - } - /** - * repeated .vortex.dtype.DType dtypes = 2; - */ - public Builder clearDtypes() { - if (dtypesBuilder_ == null) { - dtypes_ = java.util.Collections.emptyList(); - bitField0_ = (bitField0_ & ~0x00000002); - onChanged(); - } else { - dtypesBuilder_.clear(); - } - return this; - } - /** - * repeated .vortex.dtype.DType dtypes = 2; - */ - public Builder removeDtypes(int index) { - if (dtypesBuilder_ == null) { - ensureDtypesIsMutable(); - dtypes_.remove(index); - onChanged(); - } else { - dtypesBuilder_.remove(index); - } - return this; - } - /** - * repeated .vortex.dtype.DType dtypes = 2; - */ - public io.github.dfa1.vortex.proto.DTypeProtos.DType.Builder getDtypesBuilder( - int index) { - return internalGetDtypesFieldBuilder().getBuilder(index); - } - /** - * repeated .vortex.dtype.DType dtypes = 2; - */ - public io.github.dfa1.vortex.proto.DTypeProtos.DTypeOrBuilder getDtypesOrBuilder( - int index) { - if (dtypesBuilder_ == null) { - return dtypes_.get(index); } else { - return dtypesBuilder_.getMessageOrBuilder(index); - } - } - /** - * repeated .vortex.dtype.DType dtypes = 2; - */ - public java.util.List - getDtypesOrBuilderList() { - if (dtypesBuilder_ != null) { - return dtypesBuilder_.getMessageOrBuilderList(); - } else { - return java.util.Collections.unmodifiableList(dtypes_); - } - } - /** - * repeated .vortex.dtype.DType dtypes = 2; - */ - public io.github.dfa1.vortex.proto.DTypeProtos.DType.Builder addDtypesBuilder() { - return internalGetDtypesFieldBuilder().addBuilder( - io.github.dfa1.vortex.proto.DTypeProtos.DType.getDefaultInstance()); - } - /** - * repeated .vortex.dtype.DType dtypes = 2; - */ - public io.github.dfa1.vortex.proto.DTypeProtos.DType.Builder addDtypesBuilder( - int index) { - return internalGetDtypesFieldBuilder().addBuilder( - index, io.github.dfa1.vortex.proto.DTypeProtos.DType.getDefaultInstance()); - } - /** - * repeated .vortex.dtype.DType dtypes = 2; - */ - public java.util.List - getDtypesBuilderList() { - return internalGetDtypesFieldBuilder().getBuilderList(); - } - private com.google.protobuf.RepeatedFieldBuilder< - io.github.dfa1.vortex.proto.DTypeProtos.DType, io.github.dfa1.vortex.proto.DTypeProtos.DType.Builder, io.github.dfa1.vortex.proto.DTypeProtos.DTypeOrBuilder> - internalGetDtypesFieldBuilder() { - if (dtypesBuilder_ == null) { - dtypesBuilder_ = new com.google.protobuf.RepeatedFieldBuilder< - io.github.dfa1.vortex.proto.DTypeProtos.DType, io.github.dfa1.vortex.proto.DTypeProtos.DType.Builder, io.github.dfa1.vortex.proto.DTypeProtos.DTypeOrBuilder>( - dtypes_, - ((bitField0_ & 0x00000002) != 0), - getParentForChildren(), - isClean()); - dtypes_ = null; - } - return dtypesBuilder_; - } - - private boolean nullable_ ; - /** - * bool nullable = 3; - * @return The nullable. - */ - @java.lang.Override - public boolean getNullable() { - return nullable_; - } - /** - * bool nullable = 3; - * @param value The nullable to set. - * @return This builder for chaining. - */ - public Builder setNullable(boolean value) { - - nullable_ = value; - bitField0_ |= 0x00000004; - onChanged(); - return this; - } - /** - * bool nullable = 3; - * @return This builder for chaining. - */ - public Builder clearNullable() { - bitField0_ = (bitField0_ & ~0x00000004); - nullable_ = false; - onChanged(); - return this; - } - - // @@protoc_insertion_point(builder_scope:vortex.dtype.Struct) - } - - // @@protoc_insertion_point(class_scope:vortex.dtype.Struct) - private static final io.github.dfa1.vortex.proto.DTypeProtos.Struct DEFAULT_INSTANCE; - static { - DEFAULT_INSTANCE = new io.github.dfa1.vortex.proto.DTypeProtos.Struct(); - } - - public static io.github.dfa1.vortex.proto.DTypeProtos.Struct getDefaultInstance() { - return DEFAULT_INSTANCE; - } - - private static final com.google.protobuf.Parser - PARSER = new com.google.protobuf.AbstractParser() { - @java.lang.Override - public Struct parsePartialFrom( - com.google.protobuf.CodedInputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws com.google.protobuf.InvalidProtocolBufferException { - Builder builder = newBuilder(); - try { - builder.mergeFrom(input, extensionRegistry); - } catch (com.google.protobuf.InvalidProtocolBufferException e) { - throw e.setUnfinishedMessage(builder.buildPartial()); - } catch (com.google.protobuf.UninitializedMessageException e) { - throw e.asInvalidProtocolBufferException().setUnfinishedMessage(builder.buildPartial()); - } catch (java.io.IOException e) { - throw new com.google.protobuf.InvalidProtocolBufferException(e) - .setUnfinishedMessage(builder.buildPartial()); - } - return builder.buildPartial(); - } - }; - - public static com.google.protobuf.Parser parser() { - return PARSER; - } - - @java.lang.Override - public com.google.protobuf.Parser getParserForType() { - return PARSER; - } - - @java.lang.Override - public io.github.dfa1.vortex.proto.DTypeProtos.Struct getDefaultInstanceForType() { - return DEFAULT_INSTANCE; - } - - } - - public interface ListOrBuilder extends - // @@protoc_insertion_point(interface_extends:vortex.dtype.List) - com.google.protobuf.MessageOrBuilder { - - /** - * .vortex.dtype.DType element_type = 1; - * @return Whether the elementType field is set. - */ - boolean hasElementType(); - /** - * .vortex.dtype.DType element_type = 1; - * @return The elementType. - */ - io.github.dfa1.vortex.proto.DTypeProtos.DType getElementType(); - /** - * .vortex.dtype.DType element_type = 1; - */ - io.github.dfa1.vortex.proto.DTypeProtos.DTypeOrBuilder getElementTypeOrBuilder(); - - /** - * bool nullable = 2; - * @return The nullable. - */ - boolean getNullable(); - } - /** - * Protobuf type {@code vortex.dtype.List} - */ - public static final class List extends - com.google.protobuf.GeneratedMessage implements - // @@protoc_insertion_point(message_implements:vortex.dtype.List) - ListOrBuilder { - private static final long serialVersionUID = 0L; - static { - com.google.protobuf.RuntimeVersion.validateProtobufGencodeVersion( - com.google.protobuf.RuntimeVersion.RuntimeDomain.PUBLIC, - /* major= */ 4, - /* minor= */ 35, - /* patch= */ 0, - /* suffix= */ "", - "List"); - } - // Use List.newBuilder() to construct. - private List(com.google.protobuf.GeneratedMessage.Builder builder) { - super(builder); - } - private List() { - } - - public static final com.google.protobuf.Descriptors.Descriptor - getDescriptor() { - return io.github.dfa1.vortex.proto.DTypeProtos.internal_static_vortex_dtype_List_descriptor; - } - - @java.lang.Override - public com.google.protobuf.Descriptors.Descriptor getDescriptorForType() { - return io.github.dfa1.vortex.proto.DTypeProtos.internal_static_vortex_dtype_List_descriptor; - } - - @java.lang.Override - protected com.google.protobuf.GeneratedMessage.FieldAccessorTable - internalGetFieldAccessorTable() { - return io.github.dfa1.vortex.proto.DTypeProtos.internal_static_vortex_dtype_List_fieldAccessorTable - .ensureFieldAccessorsInitialized( - io.github.dfa1.vortex.proto.DTypeProtos.List.class, io.github.dfa1.vortex.proto.DTypeProtos.List.Builder.class); - } - - private int bitField0_; - public static final int ELEMENT_TYPE_FIELD_NUMBER = 1; - private io.github.dfa1.vortex.proto.DTypeProtos.DType elementType_; - /** - * .vortex.dtype.DType element_type = 1; - * @return Whether the elementType field is set. - */ - @java.lang.Override - public boolean hasElementType() { - return ((bitField0_ & 0x00000001) != 0); - } - /** - * .vortex.dtype.DType element_type = 1; - * @return The elementType. - */ - @java.lang.Override - public io.github.dfa1.vortex.proto.DTypeProtos.DType getElementType() { - return elementType_ == null ? io.github.dfa1.vortex.proto.DTypeProtos.DType.getDefaultInstance() : elementType_; - } - /** - * .vortex.dtype.DType element_type = 1; - */ - @java.lang.Override - public io.github.dfa1.vortex.proto.DTypeProtos.DTypeOrBuilder getElementTypeOrBuilder() { - return elementType_ == null ? io.github.dfa1.vortex.proto.DTypeProtos.DType.getDefaultInstance() : elementType_; - } - - public static final int NULLABLE_FIELD_NUMBER = 2; - private boolean nullable_ = false; - /** - * bool nullable = 2; - * @return The nullable. - */ - @java.lang.Override - public boolean getNullable() { - return nullable_; - } - - private byte memoizedIsInitialized = -1; - @java.lang.Override - public final boolean isInitialized() { - byte isInitialized = memoizedIsInitialized; - if (isInitialized == 1) return true; - if (isInitialized == 0) return false; - - memoizedIsInitialized = 1; - return true; - } - - @java.lang.Override - public void writeTo(com.google.protobuf.CodedOutputStream output) - throws java.io.IOException { - if (((bitField0_ & 0x00000001) != 0)) { - output.writeMessage(1, getElementType()); - } - if (nullable_ != false) { - output.writeBool(2, nullable_); - } - getUnknownFields().writeTo(output); - } - private int computeSerializedSize_0() { - int size = 0; - if (((bitField0_ & 0x00000001) != 0)) { - size += com.google.protobuf.CodedOutputStream - .computeMessageSize(1, getElementType()); - } - if (nullable_ != false) { - size += com.google.protobuf.CodedOutputStream - .computeBoolSize(2, nullable_); - } - return size; - } - @java.lang.Override - public int getSerializedSize() { - int size = memoizedSize; - if (size != -1) return size; - - size = 0; - size += computeSerializedSize_0(); - size += getUnknownFields().getSerializedSize(); - memoizedSize = size; - return size; - } - - @java.lang.Override - public boolean equals(final java.lang.Object obj) { - if (obj == this) { - return true; - } - if (!(obj instanceof io.github.dfa1.vortex.proto.DTypeProtos.List)) { - return super.equals(obj); - } - io.github.dfa1.vortex.proto.DTypeProtos.List other = (io.github.dfa1.vortex.proto.DTypeProtos.List) obj; - - if (hasElementType() != other.hasElementType()) return false; - if (hasElementType()) { - if (!getElementType() - .equals(other.getElementType())) return false; - } - if (getNullable() - != other.getNullable()) return false; - if (!getUnknownFields().equals(other.getUnknownFields())) return false; - return true; - } - - @java.lang.Override - public int hashCode() { - if (memoizedHashCode != 0) { - return memoizedHashCode; - } - int hash = 41; - hash = (19 * hash) + getDescriptor().hashCode(); - if (hasElementType()) { - hash = (37 * hash) + ELEMENT_TYPE_FIELD_NUMBER; - hash = (53 * hash) + getElementType().hashCode(); - } - hash = (37 * hash) + NULLABLE_FIELD_NUMBER; - hash = (53 * hash) + com.google.protobuf.Internal.hashBoolean( - getNullable()); - hash = (29 * hash) + getUnknownFields().hashCode(); - memoizedHashCode = hash; - return hash; - } - - public static io.github.dfa1.vortex.proto.DTypeProtos.List parseFrom( - java.nio.ByteBuffer data) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data); - } - public static io.github.dfa1.vortex.proto.DTypeProtos.List parseFrom( - java.nio.ByteBuffer data, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data, extensionRegistry); - } - public static io.github.dfa1.vortex.proto.DTypeProtos.List parseFrom( - com.google.protobuf.ByteString data) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data); - } - public static io.github.dfa1.vortex.proto.DTypeProtos.List parseFrom( - com.google.protobuf.ByteString data, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data, extensionRegistry); - } - public static io.github.dfa1.vortex.proto.DTypeProtos.List parseFrom(byte[] data) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data); - } - public static io.github.dfa1.vortex.proto.DTypeProtos.List parseFrom( - byte[] data, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data, extensionRegistry); - } - public static io.github.dfa1.vortex.proto.DTypeProtos.List parseFrom(java.io.InputStream input) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseWithIOException(PARSER, input); - } - public static io.github.dfa1.vortex.proto.DTypeProtos.List parseFrom( - java.io.InputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseWithIOException(PARSER, input, extensionRegistry); - } - - public static io.github.dfa1.vortex.proto.DTypeProtos.List parseDelimitedFrom(java.io.InputStream input) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseDelimitedWithIOException(PARSER, input); - } - - public static io.github.dfa1.vortex.proto.DTypeProtos.List parseDelimitedFrom( - java.io.InputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseDelimitedWithIOException(PARSER, input, extensionRegistry); - } - public static io.github.dfa1.vortex.proto.DTypeProtos.List parseFrom( - com.google.protobuf.CodedInputStream input) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseWithIOException(PARSER, input); - } - public static io.github.dfa1.vortex.proto.DTypeProtos.List parseFrom( - com.google.protobuf.CodedInputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseWithIOException(PARSER, input, extensionRegistry); - } - - @java.lang.Override - public Builder newBuilderForType() { return newBuilder(); } - public static Builder newBuilder() { - return DEFAULT_INSTANCE.toBuilder(); - } - public static Builder newBuilder(io.github.dfa1.vortex.proto.DTypeProtos.List prototype) { - return DEFAULT_INSTANCE.toBuilder().mergeFrom(prototype); - } - @java.lang.Override - public Builder toBuilder() { - return this == DEFAULT_INSTANCE - ? new Builder() : new Builder().mergeFrom(this); - } - - @java.lang.Override - protected Builder newBuilderForType( - com.google.protobuf.GeneratedMessage.BuilderParent parent) { - Builder builder = new Builder(parent); - return builder; - } - /** - * Protobuf type {@code vortex.dtype.List} - */ - public static final class Builder extends - com.google.protobuf.GeneratedMessage.Builder implements - // @@protoc_insertion_point(builder_implements:vortex.dtype.List) - io.github.dfa1.vortex.proto.DTypeProtos.ListOrBuilder { - public static final com.google.protobuf.Descriptors.Descriptor - getDescriptor() { - return io.github.dfa1.vortex.proto.DTypeProtos.internal_static_vortex_dtype_List_descriptor; - } - - @java.lang.Override - protected com.google.protobuf.GeneratedMessage.FieldAccessorTable - internalGetFieldAccessorTable() { - return io.github.dfa1.vortex.proto.DTypeProtos.internal_static_vortex_dtype_List_fieldAccessorTable - .ensureFieldAccessorsInitialized( - io.github.dfa1.vortex.proto.DTypeProtos.List.class, io.github.dfa1.vortex.proto.DTypeProtos.List.Builder.class); - } - - // Construct using io.github.dfa1.vortex.proto.DTypeProtos.List.newBuilder() - private Builder() { - maybeForceBuilderInitialization(); - } - - private Builder( - com.google.protobuf.GeneratedMessage.BuilderParent parent) { - super(parent); - maybeForceBuilderInitialization(); - } - private void maybeForceBuilderInitialization() { - if (com.google.protobuf.GeneratedMessage - .alwaysUseFieldBuilders) { - internalGetElementTypeFieldBuilder(); - } - } - @java.lang.Override - public Builder clear() { - super.clear(); - bitField0_ = 0; - elementType_ = null; - if (elementTypeBuilder_ != null) { - elementTypeBuilder_.dispose(); - elementTypeBuilder_ = null; - } - nullable_ = false; - return this; - } - - @java.lang.Override - public com.google.protobuf.Descriptors.Descriptor - getDescriptorForType() { - return io.github.dfa1.vortex.proto.DTypeProtos.internal_static_vortex_dtype_List_descriptor; - } - - @java.lang.Override - public io.github.dfa1.vortex.proto.DTypeProtos.List getDefaultInstanceForType() { - return io.github.dfa1.vortex.proto.DTypeProtos.List.getDefaultInstance(); - } - - @java.lang.Override - public io.github.dfa1.vortex.proto.DTypeProtos.List build() { - io.github.dfa1.vortex.proto.DTypeProtos.List result = buildPartial(); - if (!result.isInitialized()) { - throw newUninitializedMessageException(result); - } - return result; - } - - @java.lang.Override - public io.github.dfa1.vortex.proto.DTypeProtos.List buildPartial() { - io.github.dfa1.vortex.proto.DTypeProtos.List result = new io.github.dfa1.vortex.proto.DTypeProtos.List(this); - if (bitField0_ != 0) { buildPartial0(result); } - onBuilt(); - return result; - } - - private void buildPartial0(io.github.dfa1.vortex.proto.DTypeProtos.List result) { - int from_bitField0_ = bitField0_; - int to_bitField0_ = 0; - if (((from_bitField0_ & 0x00000001) != 0)) { - result.elementType_ = elementTypeBuilder_ == null - ? elementType_ - : elementTypeBuilder_.build(); - to_bitField0_ |= 0x00000001; - } - if (((from_bitField0_ & 0x00000002) != 0)) { - result.nullable_ = nullable_; - } - result.bitField0_ |= to_bitField0_; - } - - @java.lang.Override - public Builder mergeFrom(com.google.protobuf.Message other) { - if (other instanceof io.github.dfa1.vortex.proto.DTypeProtos.List) { - return mergeFrom((io.github.dfa1.vortex.proto.DTypeProtos.List)other); - } else { - super.mergeFrom(other); - return this; - } - } - - public Builder mergeFrom(io.github.dfa1.vortex.proto.DTypeProtos.List other) { - if (other == io.github.dfa1.vortex.proto.DTypeProtos.List.getDefaultInstance()) return this; - if (other.hasElementType()) { - mergeElementType(other.getElementType()); - } - if (other.getNullable() != false) { - setNullable(other.getNullable()); - } - this.mergeUnknownFields(other.getUnknownFields()); - onChanged(); - return this; - } - - @java.lang.Override - public final boolean isInitialized() { - return true; - } - - @java.lang.Override - public Builder mergeFrom( - com.google.protobuf.CodedInputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws java.io.IOException { - if (extensionRegistry == null) { - throw new java.lang.NullPointerException(); - } - try { - boolean done = false; - while (!done) { - int tag = input.readTag(); - switch (tag) { - case 0: - done = true; - break; - case 10: { - input.readMessage( - internalGetElementTypeFieldBuilder().getBuilder(), - extensionRegistry); - bitField0_ |= 0x00000001; - break; - } // case 10 - case 16: { - nullable_ = input.readBool(); - bitField0_ |= 0x00000002; - break; - } // case 16 - default: { - if (!super.parseUnknownField(input, extensionRegistry, tag)) { - done = true; // was an endgroup tag - } - break; - } // default: - } // switch (tag) - } // while (!done) - } catch (com.google.protobuf.InvalidProtocolBufferException e) { - throw e.unwrapIOException(); - } finally { - onChanged(); - } // finally - return this; - } - private int bitField0_; - - private io.github.dfa1.vortex.proto.DTypeProtos.DType elementType_; - private com.google.protobuf.SingleFieldBuilder< - io.github.dfa1.vortex.proto.DTypeProtos.DType, io.github.dfa1.vortex.proto.DTypeProtos.DType.Builder, io.github.dfa1.vortex.proto.DTypeProtos.DTypeOrBuilder> elementTypeBuilder_; - /** - * .vortex.dtype.DType element_type = 1; - * @return Whether the elementType field is set. - */ - public boolean hasElementType() { - return ((bitField0_ & 0x00000001) != 0); - } - /** - * .vortex.dtype.DType element_type = 1; - * @return The elementType. - */ - public io.github.dfa1.vortex.proto.DTypeProtos.DType getElementType() { - if (elementTypeBuilder_ == null) { - return elementType_ == null ? io.github.dfa1.vortex.proto.DTypeProtos.DType.getDefaultInstance() : elementType_; - } else { - return elementTypeBuilder_.getMessage(); - } - } - /** - * .vortex.dtype.DType element_type = 1; - */ - public Builder setElementType(io.github.dfa1.vortex.proto.DTypeProtos.DType value) { - if (elementTypeBuilder_ == null) { - if (value == null) { - throw new NullPointerException(); - } - elementType_ = value; - } else { - elementTypeBuilder_.setMessage(value); - } - bitField0_ |= 0x00000001; - onChanged(); - return this; - } - /** - * .vortex.dtype.DType element_type = 1; - */ - public Builder setElementType( - io.github.dfa1.vortex.proto.DTypeProtos.DType.Builder builderForValue) { - if (elementTypeBuilder_ == null) { - elementType_ = builderForValue.build(); - } else { - elementTypeBuilder_.setMessage(builderForValue.build()); - } - bitField0_ |= 0x00000001; - onChanged(); - return this; - } - /** - * .vortex.dtype.DType element_type = 1; - */ - public Builder mergeElementType(io.github.dfa1.vortex.proto.DTypeProtos.DType value) { - if (elementTypeBuilder_ == null) { - if (((bitField0_ & 0x00000001) != 0) && - elementType_ != null && - elementType_ != io.github.dfa1.vortex.proto.DTypeProtos.DType.getDefaultInstance()) { - getElementTypeBuilder().mergeFrom(value); - } else { - elementType_ = value; - } - } else { - elementTypeBuilder_.mergeFrom(value); - } - if (elementType_ != null) { - bitField0_ |= 0x00000001; - onChanged(); - } - return this; - } - /** - * .vortex.dtype.DType element_type = 1; - */ - public Builder clearElementType() { - bitField0_ = (bitField0_ & ~0x00000001); - elementType_ = null; - if (elementTypeBuilder_ != null) { - elementTypeBuilder_.dispose(); - elementTypeBuilder_ = null; - } - onChanged(); - return this; - } - /** - * .vortex.dtype.DType element_type = 1; - */ - public io.github.dfa1.vortex.proto.DTypeProtos.DType.Builder getElementTypeBuilder() { - bitField0_ |= 0x00000001; - onChanged(); - return internalGetElementTypeFieldBuilder().getBuilder(); - } - /** - * .vortex.dtype.DType element_type = 1; - */ - public io.github.dfa1.vortex.proto.DTypeProtos.DTypeOrBuilder getElementTypeOrBuilder() { - if (elementTypeBuilder_ != null) { - return elementTypeBuilder_.getMessageOrBuilder(); - } else { - return elementType_ == null ? - io.github.dfa1.vortex.proto.DTypeProtos.DType.getDefaultInstance() : elementType_; - } - } - /** - * .vortex.dtype.DType element_type = 1; - */ - private com.google.protobuf.SingleFieldBuilder< - io.github.dfa1.vortex.proto.DTypeProtos.DType, io.github.dfa1.vortex.proto.DTypeProtos.DType.Builder, io.github.dfa1.vortex.proto.DTypeProtos.DTypeOrBuilder> - internalGetElementTypeFieldBuilder() { - if (elementTypeBuilder_ == null) { - elementTypeBuilder_ = new com.google.protobuf.SingleFieldBuilder< - io.github.dfa1.vortex.proto.DTypeProtos.DType, io.github.dfa1.vortex.proto.DTypeProtos.DType.Builder, io.github.dfa1.vortex.proto.DTypeProtos.DTypeOrBuilder>( - getElementType(), - getParentForChildren(), - isClean()); - elementType_ = null; - } - return elementTypeBuilder_; - } - - private boolean nullable_ ; - /** - * bool nullable = 2; - * @return The nullable. - */ - @java.lang.Override - public boolean getNullable() { - return nullable_; - } - /** - * bool nullable = 2; - * @param value The nullable to set. - * @return This builder for chaining. - */ - public Builder setNullable(boolean value) { - - nullable_ = value; - bitField0_ |= 0x00000002; - onChanged(); - return this; - } - /** - * bool nullable = 2; - * @return This builder for chaining. - */ - public Builder clearNullable() { - bitField0_ = (bitField0_ & ~0x00000002); - nullable_ = false; - onChanged(); - return this; - } - - // @@protoc_insertion_point(builder_scope:vortex.dtype.List) - } - - // @@protoc_insertion_point(class_scope:vortex.dtype.List) - private static final io.github.dfa1.vortex.proto.DTypeProtos.List DEFAULT_INSTANCE; - static { - DEFAULT_INSTANCE = new io.github.dfa1.vortex.proto.DTypeProtos.List(); - } - - public static io.github.dfa1.vortex.proto.DTypeProtos.List getDefaultInstance() { - return DEFAULT_INSTANCE; - } - - private static final com.google.protobuf.Parser - PARSER = new com.google.protobuf.AbstractParser() { - @java.lang.Override - public List parsePartialFrom( - com.google.protobuf.CodedInputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws com.google.protobuf.InvalidProtocolBufferException { - Builder builder = newBuilder(); - try { - builder.mergeFrom(input, extensionRegistry); - } catch (com.google.protobuf.InvalidProtocolBufferException e) { - throw e.setUnfinishedMessage(builder.buildPartial()); - } catch (com.google.protobuf.UninitializedMessageException e) { - throw e.asInvalidProtocolBufferException().setUnfinishedMessage(builder.buildPartial()); - } catch (java.io.IOException e) { - throw new com.google.protobuf.InvalidProtocolBufferException(e) - .setUnfinishedMessage(builder.buildPartial()); - } - return builder.buildPartial(); - } - }; - - public static com.google.protobuf.Parser parser() { - return PARSER; - } - - @java.lang.Override - public com.google.protobuf.Parser getParserForType() { - return PARSER; - } - - @java.lang.Override - public io.github.dfa1.vortex.proto.DTypeProtos.List getDefaultInstanceForType() { - return DEFAULT_INSTANCE; - } - - } - - public interface FixedSizeListOrBuilder extends - // @@protoc_insertion_point(interface_extends:vortex.dtype.FixedSizeList) - com.google.protobuf.MessageOrBuilder { - - /** - * .vortex.dtype.DType element_type = 1; - * @return Whether the elementType field is set. - */ - boolean hasElementType(); - /** - * .vortex.dtype.DType element_type = 1; - * @return The elementType. - */ - io.github.dfa1.vortex.proto.DTypeProtos.DType getElementType(); - /** - * .vortex.dtype.DType element_type = 1; - */ - io.github.dfa1.vortex.proto.DTypeProtos.DTypeOrBuilder getElementTypeOrBuilder(); - - /** - * uint32 size = 2; - * @return The size. - */ - int getSize(); - - /** - * bool nullable = 3; - * @return The nullable. - */ - boolean getNullable(); - } - /** - * Protobuf type {@code vortex.dtype.FixedSizeList} - */ - public static final class FixedSizeList extends - com.google.protobuf.GeneratedMessage implements - // @@protoc_insertion_point(message_implements:vortex.dtype.FixedSizeList) - FixedSizeListOrBuilder { - private static final long serialVersionUID = 0L; - static { - com.google.protobuf.RuntimeVersion.validateProtobufGencodeVersion( - com.google.protobuf.RuntimeVersion.RuntimeDomain.PUBLIC, - /* major= */ 4, - /* minor= */ 35, - /* patch= */ 0, - /* suffix= */ "", - "FixedSizeList"); - } - // Use FixedSizeList.newBuilder() to construct. - private FixedSizeList(com.google.protobuf.GeneratedMessage.Builder builder) { - super(builder); - } - private FixedSizeList() { - } - - public static final com.google.protobuf.Descriptors.Descriptor - getDescriptor() { - return io.github.dfa1.vortex.proto.DTypeProtos.internal_static_vortex_dtype_FixedSizeList_descriptor; - } - - @java.lang.Override - public com.google.protobuf.Descriptors.Descriptor getDescriptorForType() { - return io.github.dfa1.vortex.proto.DTypeProtos.internal_static_vortex_dtype_FixedSizeList_descriptor; - } - - @java.lang.Override - protected com.google.protobuf.GeneratedMessage.FieldAccessorTable - internalGetFieldAccessorTable() { - return io.github.dfa1.vortex.proto.DTypeProtos.internal_static_vortex_dtype_FixedSizeList_fieldAccessorTable - .ensureFieldAccessorsInitialized( - io.github.dfa1.vortex.proto.DTypeProtos.FixedSizeList.class, io.github.dfa1.vortex.proto.DTypeProtos.FixedSizeList.Builder.class); - } - - private int bitField0_; - public static final int ELEMENT_TYPE_FIELD_NUMBER = 1; - private io.github.dfa1.vortex.proto.DTypeProtos.DType elementType_; - /** - * .vortex.dtype.DType element_type = 1; - * @return Whether the elementType field is set. - */ - @java.lang.Override - public boolean hasElementType() { - return ((bitField0_ & 0x00000001) != 0); - } - /** - * .vortex.dtype.DType element_type = 1; - * @return The elementType. - */ - @java.lang.Override - public io.github.dfa1.vortex.proto.DTypeProtos.DType getElementType() { - return elementType_ == null ? io.github.dfa1.vortex.proto.DTypeProtos.DType.getDefaultInstance() : elementType_; - } - /** - * .vortex.dtype.DType element_type = 1; - */ - @java.lang.Override - public io.github.dfa1.vortex.proto.DTypeProtos.DTypeOrBuilder getElementTypeOrBuilder() { - return elementType_ == null ? io.github.dfa1.vortex.proto.DTypeProtos.DType.getDefaultInstance() : elementType_; - } - - public static final int SIZE_FIELD_NUMBER = 2; - private int size_ = 0; - /** - * uint32 size = 2; - * @return The size. - */ - @java.lang.Override - public int getSize() { - return size_; - } - - public static final int NULLABLE_FIELD_NUMBER = 3; - private boolean nullable_ = false; - /** - * bool nullable = 3; - * @return The nullable. - */ - @java.lang.Override - public boolean getNullable() { - return nullable_; - } - - private byte memoizedIsInitialized = -1; - @java.lang.Override - public final boolean isInitialized() { - byte isInitialized = memoizedIsInitialized; - if (isInitialized == 1) return true; - if (isInitialized == 0) return false; - - memoizedIsInitialized = 1; - return true; - } - - @java.lang.Override - public void writeTo(com.google.protobuf.CodedOutputStream output) - throws java.io.IOException { - if (((bitField0_ & 0x00000001) != 0)) { - output.writeMessage(1, getElementType()); - } - if (size_ != 0) { - output.writeUInt32(2, size_); - } - if (nullable_ != false) { - output.writeBool(3, nullable_); - } - getUnknownFields().writeTo(output); - } - private int computeSerializedSize_0() { - int size = 0; - if (((bitField0_ & 0x00000001) != 0)) { - size += com.google.protobuf.CodedOutputStream - .computeMessageSize(1, getElementType()); - } - if (size_ != 0) { - size += com.google.protobuf.CodedOutputStream - .computeUInt32Size(2, size_); - } - if (nullable_ != false) { - size += com.google.protobuf.CodedOutputStream - .computeBoolSize(3, nullable_); - } - return size; - } - @java.lang.Override - public int getSerializedSize() { - int size = memoizedSize; - if (size != -1) return size; - - size = 0; - size += computeSerializedSize_0(); - size += getUnknownFields().getSerializedSize(); - memoizedSize = size; - return size; - } - - @java.lang.Override - public boolean equals(final java.lang.Object obj) { - if (obj == this) { - return true; - } - if (!(obj instanceof io.github.dfa1.vortex.proto.DTypeProtos.FixedSizeList)) { - return super.equals(obj); - } - io.github.dfa1.vortex.proto.DTypeProtos.FixedSizeList other = (io.github.dfa1.vortex.proto.DTypeProtos.FixedSizeList) obj; - - if (hasElementType() != other.hasElementType()) return false; - if (hasElementType()) { - if (!getElementType() - .equals(other.getElementType())) return false; - } - if (getSize() - != other.getSize()) return false; - if (getNullable() - != other.getNullable()) return false; - if (!getUnknownFields().equals(other.getUnknownFields())) return false; - return true; - } - - @java.lang.Override - public int hashCode() { - if (memoizedHashCode != 0) { - return memoizedHashCode; - } - int hash = 41; - hash = (19 * hash) + getDescriptor().hashCode(); - if (hasElementType()) { - hash = (37 * hash) + ELEMENT_TYPE_FIELD_NUMBER; - hash = (53 * hash) + getElementType().hashCode(); - } - hash = (37 * hash) + SIZE_FIELD_NUMBER; - hash = (53 * hash) + getSize(); - hash = (37 * hash) + NULLABLE_FIELD_NUMBER; - hash = (53 * hash) + com.google.protobuf.Internal.hashBoolean( - getNullable()); - hash = (29 * hash) + getUnknownFields().hashCode(); - memoizedHashCode = hash; - return hash; - } - - public static io.github.dfa1.vortex.proto.DTypeProtos.FixedSizeList parseFrom( - java.nio.ByteBuffer data) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data); - } - public static io.github.dfa1.vortex.proto.DTypeProtos.FixedSizeList parseFrom( - java.nio.ByteBuffer data, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data, extensionRegistry); - } - public static io.github.dfa1.vortex.proto.DTypeProtos.FixedSizeList parseFrom( - com.google.protobuf.ByteString data) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data); - } - public static io.github.dfa1.vortex.proto.DTypeProtos.FixedSizeList parseFrom( - com.google.protobuf.ByteString data, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data, extensionRegistry); - } - public static io.github.dfa1.vortex.proto.DTypeProtos.FixedSizeList parseFrom(byte[] data) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data); - } - public static io.github.dfa1.vortex.proto.DTypeProtos.FixedSizeList parseFrom( - byte[] data, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data, extensionRegistry); - } - public static io.github.dfa1.vortex.proto.DTypeProtos.FixedSizeList parseFrom(java.io.InputStream input) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseWithIOException(PARSER, input); - } - public static io.github.dfa1.vortex.proto.DTypeProtos.FixedSizeList parseFrom( - java.io.InputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseWithIOException(PARSER, input, extensionRegistry); - } - - public static io.github.dfa1.vortex.proto.DTypeProtos.FixedSizeList parseDelimitedFrom(java.io.InputStream input) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseDelimitedWithIOException(PARSER, input); - } - - public static io.github.dfa1.vortex.proto.DTypeProtos.FixedSizeList parseDelimitedFrom( - java.io.InputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseDelimitedWithIOException(PARSER, input, extensionRegistry); - } - public static io.github.dfa1.vortex.proto.DTypeProtos.FixedSizeList parseFrom( - com.google.protobuf.CodedInputStream input) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseWithIOException(PARSER, input); - } - public static io.github.dfa1.vortex.proto.DTypeProtos.FixedSizeList parseFrom( - com.google.protobuf.CodedInputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseWithIOException(PARSER, input, extensionRegistry); - } - - @java.lang.Override - public Builder newBuilderForType() { return newBuilder(); } - public static Builder newBuilder() { - return DEFAULT_INSTANCE.toBuilder(); - } - public static Builder newBuilder(io.github.dfa1.vortex.proto.DTypeProtos.FixedSizeList prototype) { - return DEFAULT_INSTANCE.toBuilder().mergeFrom(prototype); - } - @java.lang.Override - public Builder toBuilder() { - return this == DEFAULT_INSTANCE - ? new Builder() : new Builder().mergeFrom(this); - } - - @java.lang.Override - protected Builder newBuilderForType( - com.google.protobuf.GeneratedMessage.BuilderParent parent) { - Builder builder = new Builder(parent); - return builder; - } - /** - * Protobuf type {@code vortex.dtype.FixedSizeList} - */ - public static final class Builder extends - com.google.protobuf.GeneratedMessage.Builder implements - // @@protoc_insertion_point(builder_implements:vortex.dtype.FixedSizeList) - io.github.dfa1.vortex.proto.DTypeProtos.FixedSizeListOrBuilder { - public static final com.google.protobuf.Descriptors.Descriptor - getDescriptor() { - return io.github.dfa1.vortex.proto.DTypeProtos.internal_static_vortex_dtype_FixedSizeList_descriptor; - } - - @java.lang.Override - protected com.google.protobuf.GeneratedMessage.FieldAccessorTable - internalGetFieldAccessorTable() { - return io.github.dfa1.vortex.proto.DTypeProtos.internal_static_vortex_dtype_FixedSizeList_fieldAccessorTable - .ensureFieldAccessorsInitialized( - io.github.dfa1.vortex.proto.DTypeProtos.FixedSizeList.class, io.github.dfa1.vortex.proto.DTypeProtos.FixedSizeList.Builder.class); - } - - // Construct using io.github.dfa1.vortex.proto.DTypeProtos.FixedSizeList.newBuilder() - private Builder() { - maybeForceBuilderInitialization(); - } - - private Builder( - com.google.protobuf.GeneratedMessage.BuilderParent parent) { - super(parent); - maybeForceBuilderInitialization(); - } - private void maybeForceBuilderInitialization() { - if (com.google.protobuf.GeneratedMessage - .alwaysUseFieldBuilders) { - internalGetElementTypeFieldBuilder(); - } - } - @java.lang.Override - public Builder clear() { - super.clear(); - bitField0_ = 0; - elementType_ = null; - if (elementTypeBuilder_ != null) { - elementTypeBuilder_.dispose(); - elementTypeBuilder_ = null; - } - size_ = 0; - nullable_ = false; - return this; - } - - @java.lang.Override - public com.google.protobuf.Descriptors.Descriptor - getDescriptorForType() { - return io.github.dfa1.vortex.proto.DTypeProtos.internal_static_vortex_dtype_FixedSizeList_descriptor; - } - - @java.lang.Override - public io.github.dfa1.vortex.proto.DTypeProtos.FixedSizeList getDefaultInstanceForType() { - return io.github.dfa1.vortex.proto.DTypeProtos.FixedSizeList.getDefaultInstance(); - } - - @java.lang.Override - public io.github.dfa1.vortex.proto.DTypeProtos.FixedSizeList build() { - io.github.dfa1.vortex.proto.DTypeProtos.FixedSizeList result = buildPartial(); - if (!result.isInitialized()) { - throw newUninitializedMessageException(result); - } - return result; - } - - @java.lang.Override - public io.github.dfa1.vortex.proto.DTypeProtos.FixedSizeList buildPartial() { - io.github.dfa1.vortex.proto.DTypeProtos.FixedSizeList result = new io.github.dfa1.vortex.proto.DTypeProtos.FixedSizeList(this); - if (bitField0_ != 0) { buildPartial0(result); } - onBuilt(); - return result; - } - - private void buildPartial0(io.github.dfa1.vortex.proto.DTypeProtos.FixedSizeList result) { - int from_bitField0_ = bitField0_; - int to_bitField0_ = 0; - if (((from_bitField0_ & 0x00000001) != 0)) { - result.elementType_ = elementTypeBuilder_ == null - ? elementType_ - : elementTypeBuilder_.build(); - to_bitField0_ |= 0x00000001; - } - if (((from_bitField0_ & 0x00000002) != 0)) { - result.size_ = size_; - } - if (((from_bitField0_ & 0x00000004) != 0)) { - result.nullable_ = nullable_; - } - result.bitField0_ |= to_bitField0_; - } - - @java.lang.Override - public Builder mergeFrom(com.google.protobuf.Message other) { - if (other instanceof io.github.dfa1.vortex.proto.DTypeProtos.FixedSizeList) { - return mergeFrom((io.github.dfa1.vortex.proto.DTypeProtos.FixedSizeList)other); - } else { - super.mergeFrom(other); - return this; - } - } - - public Builder mergeFrom(io.github.dfa1.vortex.proto.DTypeProtos.FixedSizeList other) { - if (other == io.github.dfa1.vortex.proto.DTypeProtos.FixedSizeList.getDefaultInstance()) return this; - if (other.hasElementType()) { - mergeElementType(other.getElementType()); - } - if (other.getSize() != 0) { - setSize(other.getSize()); - } - if (other.getNullable() != false) { - setNullable(other.getNullable()); - } - this.mergeUnknownFields(other.getUnknownFields()); - onChanged(); - return this; - } - - @java.lang.Override - public final boolean isInitialized() { - return true; - } - - @java.lang.Override - public Builder mergeFrom( - com.google.protobuf.CodedInputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws java.io.IOException { - if (extensionRegistry == null) { - throw new java.lang.NullPointerException(); - } - try { - boolean done = false; - while (!done) { - int tag = input.readTag(); - switch (tag) { - case 0: - done = true; - break; - case 10: { - input.readMessage( - internalGetElementTypeFieldBuilder().getBuilder(), - extensionRegistry); - bitField0_ |= 0x00000001; - break; - } // case 10 - case 16: { - size_ = input.readUInt32(); - bitField0_ |= 0x00000002; - break; - } // case 16 - case 24: { - nullable_ = input.readBool(); - bitField0_ |= 0x00000004; - break; - } // case 24 - default: { - if (!super.parseUnknownField(input, extensionRegistry, tag)) { - done = true; // was an endgroup tag - } - break; - } // default: - } // switch (tag) - } // while (!done) - } catch (com.google.protobuf.InvalidProtocolBufferException e) { - throw e.unwrapIOException(); - } finally { - onChanged(); - } // finally - return this; - } - private int bitField0_; - - private io.github.dfa1.vortex.proto.DTypeProtos.DType elementType_; - private com.google.protobuf.SingleFieldBuilder< - io.github.dfa1.vortex.proto.DTypeProtos.DType, io.github.dfa1.vortex.proto.DTypeProtos.DType.Builder, io.github.dfa1.vortex.proto.DTypeProtos.DTypeOrBuilder> elementTypeBuilder_; - /** - * .vortex.dtype.DType element_type = 1; - * @return Whether the elementType field is set. - */ - public boolean hasElementType() { - return ((bitField0_ & 0x00000001) != 0); - } - /** - * .vortex.dtype.DType element_type = 1; - * @return The elementType. - */ - public io.github.dfa1.vortex.proto.DTypeProtos.DType getElementType() { - if (elementTypeBuilder_ == null) { - return elementType_ == null ? io.github.dfa1.vortex.proto.DTypeProtos.DType.getDefaultInstance() : elementType_; - } else { - return elementTypeBuilder_.getMessage(); - } - } - /** - * .vortex.dtype.DType element_type = 1; - */ - public Builder setElementType(io.github.dfa1.vortex.proto.DTypeProtos.DType value) { - if (elementTypeBuilder_ == null) { - if (value == null) { - throw new NullPointerException(); - } - elementType_ = value; - } else { - elementTypeBuilder_.setMessage(value); - } - bitField0_ |= 0x00000001; - onChanged(); - return this; - } - /** - * .vortex.dtype.DType element_type = 1; - */ - public Builder setElementType( - io.github.dfa1.vortex.proto.DTypeProtos.DType.Builder builderForValue) { - if (elementTypeBuilder_ == null) { - elementType_ = builderForValue.build(); - } else { - elementTypeBuilder_.setMessage(builderForValue.build()); - } - bitField0_ |= 0x00000001; - onChanged(); - return this; - } - /** - * .vortex.dtype.DType element_type = 1; - */ - public Builder mergeElementType(io.github.dfa1.vortex.proto.DTypeProtos.DType value) { - if (elementTypeBuilder_ == null) { - if (((bitField0_ & 0x00000001) != 0) && - elementType_ != null && - elementType_ != io.github.dfa1.vortex.proto.DTypeProtos.DType.getDefaultInstance()) { - getElementTypeBuilder().mergeFrom(value); - } else { - elementType_ = value; - } - } else { - elementTypeBuilder_.mergeFrom(value); - } - if (elementType_ != null) { - bitField0_ |= 0x00000001; - onChanged(); - } - return this; - } - /** - * .vortex.dtype.DType element_type = 1; - */ - public Builder clearElementType() { - bitField0_ = (bitField0_ & ~0x00000001); - elementType_ = null; - if (elementTypeBuilder_ != null) { - elementTypeBuilder_.dispose(); - elementTypeBuilder_ = null; - } - onChanged(); - return this; - } - /** - * .vortex.dtype.DType element_type = 1; - */ - public io.github.dfa1.vortex.proto.DTypeProtos.DType.Builder getElementTypeBuilder() { - bitField0_ |= 0x00000001; - onChanged(); - return internalGetElementTypeFieldBuilder().getBuilder(); - } - /** - * .vortex.dtype.DType element_type = 1; - */ - public io.github.dfa1.vortex.proto.DTypeProtos.DTypeOrBuilder getElementTypeOrBuilder() { - if (elementTypeBuilder_ != null) { - return elementTypeBuilder_.getMessageOrBuilder(); - } else { - return elementType_ == null ? - io.github.dfa1.vortex.proto.DTypeProtos.DType.getDefaultInstance() : elementType_; - } - } - /** - * .vortex.dtype.DType element_type = 1; - */ - private com.google.protobuf.SingleFieldBuilder< - io.github.dfa1.vortex.proto.DTypeProtos.DType, io.github.dfa1.vortex.proto.DTypeProtos.DType.Builder, io.github.dfa1.vortex.proto.DTypeProtos.DTypeOrBuilder> - internalGetElementTypeFieldBuilder() { - if (elementTypeBuilder_ == null) { - elementTypeBuilder_ = new com.google.protobuf.SingleFieldBuilder< - io.github.dfa1.vortex.proto.DTypeProtos.DType, io.github.dfa1.vortex.proto.DTypeProtos.DType.Builder, io.github.dfa1.vortex.proto.DTypeProtos.DTypeOrBuilder>( - getElementType(), - getParentForChildren(), - isClean()); - elementType_ = null; - } - return elementTypeBuilder_; - } - - private int size_ ; - /** - * uint32 size = 2; - * @return The size. - */ - @java.lang.Override - public int getSize() { - return size_; - } - /** - * uint32 size = 2; - * @param value The size to set. - * @return This builder for chaining. - */ - public Builder setSize(int value) { - - size_ = value; - bitField0_ |= 0x00000002; - onChanged(); - return this; - } - /** - * uint32 size = 2; - * @return This builder for chaining. - */ - public Builder clearSize() { - bitField0_ = (bitField0_ & ~0x00000002); - size_ = 0; - onChanged(); - return this; - } - - private boolean nullable_ ; - /** - * bool nullable = 3; - * @return The nullable. - */ - @java.lang.Override - public boolean getNullable() { - return nullable_; - } - /** - * bool nullable = 3; - * @param value The nullable to set. - * @return This builder for chaining. - */ - public Builder setNullable(boolean value) { - - nullable_ = value; - bitField0_ |= 0x00000004; - onChanged(); - return this; - } - /** - * bool nullable = 3; - * @return This builder for chaining. - */ - public Builder clearNullable() { - bitField0_ = (bitField0_ & ~0x00000004); - nullable_ = false; - onChanged(); - return this; - } - - // @@protoc_insertion_point(builder_scope:vortex.dtype.FixedSizeList) - } - - // @@protoc_insertion_point(class_scope:vortex.dtype.FixedSizeList) - private static final io.github.dfa1.vortex.proto.DTypeProtos.FixedSizeList DEFAULT_INSTANCE; - static { - DEFAULT_INSTANCE = new io.github.dfa1.vortex.proto.DTypeProtos.FixedSizeList(); - } - - public static io.github.dfa1.vortex.proto.DTypeProtos.FixedSizeList getDefaultInstance() { - return DEFAULT_INSTANCE; - } - - private static final com.google.protobuf.Parser - PARSER = new com.google.protobuf.AbstractParser() { - @java.lang.Override - public FixedSizeList parsePartialFrom( - com.google.protobuf.CodedInputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws com.google.protobuf.InvalidProtocolBufferException { - Builder builder = newBuilder(); - try { - builder.mergeFrom(input, extensionRegistry); - } catch (com.google.protobuf.InvalidProtocolBufferException e) { - throw e.setUnfinishedMessage(builder.buildPartial()); - } catch (com.google.protobuf.UninitializedMessageException e) { - throw e.asInvalidProtocolBufferException().setUnfinishedMessage(builder.buildPartial()); - } catch (java.io.IOException e) { - throw new com.google.protobuf.InvalidProtocolBufferException(e) - .setUnfinishedMessage(builder.buildPartial()); - } - return builder.buildPartial(); - } - }; - - public static com.google.protobuf.Parser parser() { - return PARSER; - } - - @java.lang.Override - public com.google.protobuf.Parser getParserForType() { - return PARSER; - } - - @java.lang.Override - public io.github.dfa1.vortex.proto.DTypeProtos.FixedSizeList getDefaultInstanceForType() { - return DEFAULT_INSTANCE; - } - - } - - public interface ExtensionOrBuilder extends - // @@protoc_insertion_point(interface_extends:vortex.dtype.Extension) - com.google.protobuf.MessageOrBuilder { - - /** - * string id = 1; - * @return The id. - */ - java.lang.String getId(); - /** - * string id = 1; - * @return The bytes for id. - */ - com.google.protobuf.ByteString - getIdBytes(); - - /** - * .vortex.dtype.DType storage_dtype = 2; - * @return Whether the storageDtype field is set. - */ - boolean hasStorageDtype(); - /** - * .vortex.dtype.DType storage_dtype = 2; - * @return The storageDtype. - */ - io.github.dfa1.vortex.proto.DTypeProtos.DType getStorageDtype(); - /** - * .vortex.dtype.DType storage_dtype = 2; - */ - io.github.dfa1.vortex.proto.DTypeProtos.DTypeOrBuilder getStorageDtypeOrBuilder(); - - /** - * optional bytes metadata = 3; - * @return Whether the metadata field is set. - */ - boolean hasMetadata(); - /** - * optional bytes metadata = 3; - * @return The metadata. - */ - com.google.protobuf.ByteString getMetadata(); - } - /** - * Protobuf type {@code vortex.dtype.Extension} - */ - public static final class Extension extends - com.google.protobuf.GeneratedMessage implements - // @@protoc_insertion_point(message_implements:vortex.dtype.Extension) - ExtensionOrBuilder { - private static final long serialVersionUID = 0L; - static { - com.google.protobuf.RuntimeVersion.validateProtobufGencodeVersion( - com.google.protobuf.RuntimeVersion.RuntimeDomain.PUBLIC, - /* major= */ 4, - /* minor= */ 35, - /* patch= */ 0, - /* suffix= */ "", - "Extension"); - } - // Use Extension.newBuilder() to construct. - private Extension(com.google.protobuf.GeneratedMessage.Builder builder) { - super(builder); - } - private Extension() { - id_ = ""; - metadata_ = com.google.protobuf.ByteString.EMPTY; - } - - public static final com.google.protobuf.Descriptors.Descriptor - getDescriptor() { - return io.github.dfa1.vortex.proto.DTypeProtos.internal_static_vortex_dtype_Extension_descriptor; - } - - @java.lang.Override - public com.google.protobuf.Descriptors.Descriptor getDescriptorForType() { - return io.github.dfa1.vortex.proto.DTypeProtos.internal_static_vortex_dtype_Extension_descriptor; - } - - @java.lang.Override - protected com.google.protobuf.GeneratedMessage.FieldAccessorTable - internalGetFieldAccessorTable() { - return io.github.dfa1.vortex.proto.DTypeProtos.internal_static_vortex_dtype_Extension_fieldAccessorTable - .ensureFieldAccessorsInitialized( - io.github.dfa1.vortex.proto.DTypeProtos.Extension.class, io.github.dfa1.vortex.proto.DTypeProtos.Extension.Builder.class); - } - - private int bitField0_; - public static final int ID_FIELD_NUMBER = 1; - @SuppressWarnings("serial") - private volatile java.lang.Object id_ = ""; - /** - * string id = 1; - * @return The id. - */ - @java.lang.Override - public java.lang.String getId() { - java.lang.Object ref = id_; - if (ref instanceof java.lang.String) { - return (java.lang.String) ref; - } else { - com.google.protobuf.ByteString bs = - (com.google.protobuf.ByteString) ref; - java.lang.String s = bs.toStringUtf8(); - id_ = s; - return s; - } - } - /** - * string id = 1; - * @return The bytes for id. - */ - @java.lang.Override - public com.google.protobuf.ByteString - getIdBytes() { - java.lang.Object ref = id_; - if (ref instanceof java.lang.String) { - com.google.protobuf.ByteString b = - com.google.protobuf.ByteString.copyFromUtf8( - (java.lang.String) ref); - id_ = b; - return b; - } else { - return (com.google.protobuf.ByteString) ref; - } - } - - public static final int STORAGE_DTYPE_FIELD_NUMBER = 2; - private io.github.dfa1.vortex.proto.DTypeProtos.DType storageDtype_; - /** - * .vortex.dtype.DType storage_dtype = 2; - * @return Whether the storageDtype field is set. - */ - @java.lang.Override - public boolean hasStorageDtype() { - return ((bitField0_ & 0x00000001) != 0); - } - /** - * .vortex.dtype.DType storage_dtype = 2; - * @return The storageDtype. - */ - @java.lang.Override - public io.github.dfa1.vortex.proto.DTypeProtos.DType getStorageDtype() { - return storageDtype_ == null ? io.github.dfa1.vortex.proto.DTypeProtos.DType.getDefaultInstance() : storageDtype_; - } - /** - * .vortex.dtype.DType storage_dtype = 2; - */ - @java.lang.Override - public io.github.dfa1.vortex.proto.DTypeProtos.DTypeOrBuilder getStorageDtypeOrBuilder() { - return storageDtype_ == null ? io.github.dfa1.vortex.proto.DTypeProtos.DType.getDefaultInstance() : storageDtype_; - } - - public static final int METADATA_FIELD_NUMBER = 3; - private com.google.protobuf.ByteString metadata_ = com.google.protobuf.ByteString.EMPTY; - /** - * optional bytes metadata = 3; - * @return Whether the metadata field is set. - */ - @java.lang.Override - public boolean hasMetadata() { - return ((bitField0_ & 0x00000002) != 0); - } - /** - * optional bytes metadata = 3; - * @return The metadata. - */ - @java.lang.Override - public com.google.protobuf.ByteString getMetadata() { - return metadata_; - } - - private byte memoizedIsInitialized = -1; - @java.lang.Override - public final boolean isInitialized() { - byte isInitialized = memoizedIsInitialized; - if (isInitialized == 1) return true; - if (isInitialized == 0) return false; - - memoizedIsInitialized = 1; - return true; - } - - @java.lang.Override - public void writeTo(com.google.protobuf.CodedOutputStream output) - throws java.io.IOException { - if (!com.google.protobuf.GeneratedMessage.isStringEmpty(id_)) { - com.google.protobuf.GeneratedMessage.writeString(output, 1, id_); - } - if (((bitField0_ & 0x00000001) != 0)) { - output.writeMessage(2, getStorageDtype()); - } - if (((bitField0_ & 0x00000002) != 0)) { - output.writeBytes(3, metadata_); - } - getUnknownFields().writeTo(output); - } - private int computeSerializedSize_0() { - int size = 0; - if (!com.google.protobuf.GeneratedMessage.isStringEmpty(id_)) { - size += com.google.protobuf.GeneratedMessage.computeStringSize(1, id_); - } - if (((bitField0_ & 0x00000001) != 0)) { - size += com.google.protobuf.CodedOutputStream - .computeMessageSize(2, getStorageDtype()); - } - if (((bitField0_ & 0x00000002) != 0)) { - size += com.google.protobuf.CodedOutputStream - .computeBytesSize(3, metadata_); - } - return size; - } - @java.lang.Override - public int getSerializedSize() { - int size = memoizedSize; - if (size != -1) return size; - - size = 0; - size += computeSerializedSize_0(); - size += getUnknownFields().getSerializedSize(); - memoizedSize = size; - return size; - } - - @java.lang.Override - public boolean equals(final java.lang.Object obj) { - if (obj == this) { - return true; - } - if (!(obj instanceof io.github.dfa1.vortex.proto.DTypeProtos.Extension)) { - return super.equals(obj); - } - io.github.dfa1.vortex.proto.DTypeProtos.Extension other = (io.github.dfa1.vortex.proto.DTypeProtos.Extension) obj; - - if (!getId() - .equals(other.getId())) return false; - if (hasStorageDtype() != other.hasStorageDtype()) return false; - if (hasStorageDtype()) { - if (!getStorageDtype() - .equals(other.getStorageDtype())) return false; - } - if (hasMetadata() != other.hasMetadata()) return false; - if (hasMetadata()) { - if (!getMetadata() - .equals(other.getMetadata())) return false; - } - if (!getUnknownFields().equals(other.getUnknownFields())) return false; - return true; - } - - @java.lang.Override - public int hashCode() { - if (memoizedHashCode != 0) { - return memoizedHashCode; - } - int hash = 41; - hash = (19 * hash) + getDescriptor().hashCode(); - hash = (37 * hash) + ID_FIELD_NUMBER; - hash = (53 * hash) + getId().hashCode(); - if (hasStorageDtype()) { - hash = (37 * hash) + STORAGE_DTYPE_FIELD_NUMBER; - hash = (53 * hash) + getStorageDtype().hashCode(); - } - if (hasMetadata()) { - hash = (37 * hash) + METADATA_FIELD_NUMBER; - hash = (53 * hash) + getMetadata().hashCode(); - } - hash = (29 * hash) + getUnknownFields().hashCode(); - memoizedHashCode = hash; - return hash; - } - - public static io.github.dfa1.vortex.proto.DTypeProtos.Extension parseFrom( - java.nio.ByteBuffer data) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data); - } - public static io.github.dfa1.vortex.proto.DTypeProtos.Extension parseFrom( - java.nio.ByteBuffer data, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data, extensionRegistry); - } - public static io.github.dfa1.vortex.proto.DTypeProtos.Extension parseFrom( - com.google.protobuf.ByteString data) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data); - } - public static io.github.dfa1.vortex.proto.DTypeProtos.Extension parseFrom( - com.google.protobuf.ByteString data, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data, extensionRegistry); - } - public static io.github.dfa1.vortex.proto.DTypeProtos.Extension parseFrom(byte[] data) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data); - } - public static io.github.dfa1.vortex.proto.DTypeProtos.Extension parseFrom( - byte[] data, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data, extensionRegistry); - } - public static io.github.dfa1.vortex.proto.DTypeProtos.Extension parseFrom(java.io.InputStream input) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseWithIOException(PARSER, input); - } - public static io.github.dfa1.vortex.proto.DTypeProtos.Extension parseFrom( - java.io.InputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseWithIOException(PARSER, input, extensionRegistry); - } - - public static io.github.dfa1.vortex.proto.DTypeProtos.Extension parseDelimitedFrom(java.io.InputStream input) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseDelimitedWithIOException(PARSER, input); - } - - public static io.github.dfa1.vortex.proto.DTypeProtos.Extension parseDelimitedFrom( - java.io.InputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseDelimitedWithIOException(PARSER, input, extensionRegistry); - } - public static io.github.dfa1.vortex.proto.DTypeProtos.Extension parseFrom( - com.google.protobuf.CodedInputStream input) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseWithIOException(PARSER, input); - } - public static io.github.dfa1.vortex.proto.DTypeProtos.Extension parseFrom( - com.google.protobuf.CodedInputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseWithIOException(PARSER, input, extensionRegistry); - } - - @java.lang.Override - public Builder newBuilderForType() { return newBuilder(); } - public static Builder newBuilder() { - return DEFAULT_INSTANCE.toBuilder(); - } - public static Builder newBuilder(io.github.dfa1.vortex.proto.DTypeProtos.Extension prototype) { - return DEFAULT_INSTANCE.toBuilder().mergeFrom(prototype); - } - @java.lang.Override - public Builder toBuilder() { - return this == DEFAULT_INSTANCE - ? new Builder() : new Builder().mergeFrom(this); - } - - @java.lang.Override - protected Builder newBuilderForType( - com.google.protobuf.GeneratedMessage.BuilderParent parent) { - Builder builder = new Builder(parent); - return builder; - } - /** - * Protobuf type {@code vortex.dtype.Extension} - */ - public static final class Builder extends - com.google.protobuf.GeneratedMessage.Builder implements - // @@protoc_insertion_point(builder_implements:vortex.dtype.Extension) - io.github.dfa1.vortex.proto.DTypeProtos.ExtensionOrBuilder { - public static final com.google.protobuf.Descriptors.Descriptor - getDescriptor() { - return io.github.dfa1.vortex.proto.DTypeProtos.internal_static_vortex_dtype_Extension_descriptor; - } - - @java.lang.Override - protected com.google.protobuf.GeneratedMessage.FieldAccessorTable - internalGetFieldAccessorTable() { - return io.github.dfa1.vortex.proto.DTypeProtos.internal_static_vortex_dtype_Extension_fieldAccessorTable - .ensureFieldAccessorsInitialized( - io.github.dfa1.vortex.proto.DTypeProtos.Extension.class, io.github.dfa1.vortex.proto.DTypeProtos.Extension.Builder.class); - } - - // Construct using io.github.dfa1.vortex.proto.DTypeProtos.Extension.newBuilder() - private Builder() { - maybeForceBuilderInitialization(); - } - - private Builder( - com.google.protobuf.GeneratedMessage.BuilderParent parent) { - super(parent); - maybeForceBuilderInitialization(); - } - private void maybeForceBuilderInitialization() { - if (com.google.protobuf.GeneratedMessage - .alwaysUseFieldBuilders) { - internalGetStorageDtypeFieldBuilder(); - } - } - @java.lang.Override - public Builder clear() { - super.clear(); - bitField0_ = 0; - id_ = ""; - storageDtype_ = null; - if (storageDtypeBuilder_ != null) { - storageDtypeBuilder_.dispose(); - storageDtypeBuilder_ = null; - } - metadata_ = com.google.protobuf.ByteString.EMPTY; - return this; - } - - @java.lang.Override - public com.google.protobuf.Descriptors.Descriptor - getDescriptorForType() { - return io.github.dfa1.vortex.proto.DTypeProtos.internal_static_vortex_dtype_Extension_descriptor; - } - - @java.lang.Override - public io.github.dfa1.vortex.proto.DTypeProtos.Extension getDefaultInstanceForType() { - return io.github.dfa1.vortex.proto.DTypeProtos.Extension.getDefaultInstance(); - } - - @java.lang.Override - public io.github.dfa1.vortex.proto.DTypeProtos.Extension build() { - io.github.dfa1.vortex.proto.DTypeProtos.Extension result = buildPartial(); - if (!result.isInitialized()) { - throw newUninitializedMessageException(result); - } - return result; - } - - @java.lang.Override - public io.github.dfa1.vortex.proto.DTypeProtos.Extension buildPartial() { - io.github.dfa1.vortex.proto.DTypeProtos.Extension result = new io.github.dfa1.vortex.proto.DTypeProtos.Extension(this); - if (bitField0_ != 0) { buildPartial0(result); } - onBuilt(); - return result; - } - - private void buildPartial0(io.github.dfa1.vortex.proto.DTypeProtos.Extension result) { - int from_bitField0_ = bitField0_; - if (((from_bitField0_ & 0x00000001) != 0)) { - result.id_ = id_; - } - int to_bitField0_ = 0; - if (((from_bitField0_ & 0x00000002) != 0)) { - result.storageDtype_ = storageDtypeBuilder_ == null - ? storageDtype_ - : storageDtypeBuilder_.build(); - to_bitField0_ |= 0x00000001; - } - if (((from_bitField0_ & 0x00000004) != 0)) { - result.metadata_ = metadata_; - to_bitField0_ |= 0x00000002; - } - result.bitField0_ |= to_bitField0_; - } - - @java.lang.Override - public Builder mergeFrom(com.google.protobuf.Message other) { - if (other instanceof io.github.dfa1.vortex.proto.DTypeProtos.Extension) { - return mergeFrom((io.github.dfa1.vortex.proto.DTypeProtos.Extension)other); - } else { - super.mergeFrom(other); - return this; - } - } - - public Builder mergeFrom(io.github.dfa1.vortex.proto.DTypeProtos.Extension other) { - if (other == io.github.dfa1.vortex.proto.DTypeProtos.Extension.getDefaultInstance()) return this; - if (!other.getId().isEmpty()) { - id_ = other.id_; - bitField0_ |= 0x00000001; - onChanged(); - } - if (other.hasStorageDtype()) { - mergeStorageDtype(other.getStorageDtype()); - } - if (other.hasMetadata()) { - setMetadata(other.getMetadata()); - } - this.mergeUnknownFields(other.getUnknownFields()); - onChanged(); - return this; - } - - @java.lang.Override - public final boolean isInitialized() { - return true; - } - - @java.lang.Override - public Builder mergeFrom( - com.google.protobuf.CodedInputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws java.io.IOException { - if (extensionRegistry == null) { - throw new java.lang.NullPointerException(); - } - try { - boolean done = false; - while (!done) { - int tag = input.readTag(); - switch (tag) { - case 0: - done = true; - break; - case 10: { - id_ = input.readStringRequireUtf8(); - bitField0_ |= 0x00000001; - break; - } // case 10 - case 18: { - input.readMessage( - internalGetStorageDtypeFieldBuilder().getBuilder(), - extensionRegistry); - bitField0_ |= 0x00000002; - break; - } // case 18 - case 26: { - metadata_ = input.readBytes(); - bitField0_ |= 0x00000004; - break; - } // case 26 - default: { - if (!super.parseUnknownField(input, extensionRegistry, tag)) { - done = true; // was an endgroup tag - } - break; - } // default: - } // switch (tag) - } // while (!done) - } catch (com.google.protobuf.InvalidProtocolBufferException e) { - throw e.unwrapIOException(); - } finally { - onChanged(); - } // finally - return this; - } - private int bitField0_; - - private java.lang.Object id_ = ""; - /** - * string id = 1; - * @return The id. - */ - public java.lang.String getId() { - java.lang.Object ref = id_; - if (!(ref instanceof java.lang.String)) { - com.google.protobuf.ByteString bs = - (com.google.protobuf.ByteString) ref; - java.lang.String s = bs.toStringUtf8(); - id_ = s; - return s; - } else { - return (java.lang.String) ref; - } - } - /** - * string id = 1; - * @return The bytes for id. - */ - public com.google.protobuf.ByteString - getIdBytes() { - java.lang.Object ref = id_; - if (ref instanceof String) { - com.google.protobuf.ByteString b = - com.google.protobuf.ByteString.copyFromUtf8( - (java.lang.String) ref); - id_ = b; - return b; - } else { - return (com.google.protobuf.ByteString) ref; - } - } - /** - * string id = 1; - * @param value The id to set. - * @return This builder for chaining. - */ - public Builder setId( - java.lang.String value) { - if (value == null) { throw new NullPointerException(); } - id_ = value; - bitField0_ |= 0x00000001; - onChanged(); - return this; - } - /** - * string id = 1; - * @return This builder for chaining. - */ - public Builder clearId() { - id_ = getDefaultInstance().getId(); - bitField0_ = (bitField0_ & ~0x00000001); - onChanged(); - return this; - } - /** - * string id = 1; - * @param value The bytes for id to set. - * @return This builder for chaining. - */ - public Builder setIdBytes( - com.google.protobuf.ByteString value) { - if (value == null) { throw new NullPointerException(); } - checkByteStringIsUtf8(value); - id_ = value; - bitField0_ |= 0x00000001; - onChanged(); - return this; - } - - private io.github.dfa1.vortex.proto.DTypeProtos.DType storageDtype_; - private com.google.protobuf.SingleFieldBuilder< - io.github.dfa1.vortex.proto.DTypeProtos.DType, io.github.dfa1.vortex.proto.DTypeProtos.DType.Builder, io.github.dfa1.vortex.proto.DTypeProtos.DTypeOrBuilder> storageDtypeBuilder_; - /** - * .vortex.dtype.DType storage_dtype = 2; - * @return Whether the storageDtype field is set. - */ - public boolean hasStorageDtype() { - return ((bitField0_ & 0x00000002) != 0); - } - /** - * .vortex.dtype.DType storage_dtype = 2; - * @return The storageDtype. - */ - public io.github.dfa1.vortex.proto.DTypeProtos.DType getStorageDtype() { - if (storageDtypeBuilder_ == null) { - return storageDtype_ == null ? io.github.dfa1.vortex.proto.DTypeProtos.DType.getDefaultInstance() : storageDtype_; - } else { - return storageDtypeBuilder_.getMessage(); - } - } - /** - * .vortex.dtype.DType storage_dtype = 2; - */ - public Builder setStorageDtype(io.github.dfa1.vortex.proto.DTypeProtos.DType value) { - if (storageDtypeBuilder_ == null) { - if (value == null) { - throw new NullPointerException(); - } - storageDtype_ = value; - } else { - storageDtypeBuilder_.setMessage(value); - } - bitField0_ |= 0x00000002; - onChanged(); - return this; - } - /** - * .vortex.dtype.DType storage_dtype = 2; - */ - public Builder setStorageDtype( - io.github.dfa1.vortex.proto.DTypeProtos.DType.Builder builderForValue) { - if (storageDtypeBuilder_ == null) { - storageDtype_ = builderForValue.build(); - } else { - storageDtypeBuilder_.setMessage(builderForValue.build()); - } - bitField0_ |= 0x00000002; - onChanged(); - return this; - } - /** - * .vortex.dtype.DType storage_dtype = 2; - */ - public Builder mergeStorageDtype(io.github.dfa1.vortex.proto.DTypeProtos.DType value) { - if (storageDtypeBuilder_ == null) { - if (((bitField0_ & 0x00000002) != 0) && - storageDtype_ != null && - storageDtype_ != io.github.dfa1.vortex.proto.DTypeProtos.DType.getDefaultInstance()) { - getStorageDtypeBuilder().mergeFrom(value); - } else { - storageDtype_ = value; - } - } else { - storageDtypeBuilder_.mergeFrom(value); - } - if (storageDtype_ != null) { - bitField0_ |= 0x00000002; - onChanged(); - } - return this; - } - /** - * .vortex.dtype.DType storage_dtype = 2; - */ - public Builder clearStorageDtype() { - bitField0_ = (bitField0_ & ~0x00000002); - storageDtype_ = null; - if (storageDtypeBuilder_ != null) { - storageDtypeBuilder_.dispose(); - storageDtypeBuilder_ = null; - } - onChanged(); - return this; - } - /** - * .vortex.dtype.DType storage_dtype = 2; - */ - public io.github.dfa1.vortex.proto.DTypeProtos.DType.Builder getStorageDtypeBuilder() { - bitField0_ |= 0x00000002; - onChanged(); - return internalGetStorageDtypeFieldBuilder().getBuilder(); - } - /** - * .vortex.dtype.DType storage_dtype = 2; - */ - public io.github.dfa1.vortex.proto.DTypeProtos.DTypeOrBuilder getStorageDtypeOrBuilder() { - if (storageDtypeBuilder_ != null) { - return storageDtypeBuilder_.getMessageOrBuilder(); - } else { - return storageDtype_ == null ? - io.github.dfa1.vortex.proto.DTypeProtos.DType.getDefaultInstance() : storageDtype_; - } - } - /** - * .vortex.dtype.DType storage_dtype = 2; - */ - private com.google.protobuf.SingleFieldBuilder< - io.github.dfa1.vortex.proto.DTypeProtos.DType, io.github.dfa1.vortex.proto.DTypeProtos.DType.Builder, io.github.dfa1.vortex.proto.DTypeProtos.DTypeOrBuilder> - internalGetStorageDtypeFieldBuilder() { - if (storageDtypeBuilder_ == null) { - storageDtypeBuilder_ = new com.google.protobuf.SingleFieldBuilder< - io.github.dfa1.vortex.proto.DTypeProtos.DType, io.github.dfa1.vortex.proto.DTypeProtos.DType.Builder, io.github.dfa1.vortex.proto.DTypeProtos.DTypeOrBuilder>( - getStorageDtype(), - getParentForChildren(), - isClean()); - storageDtype_ = null; - } - return storageDtypeBuilder_; - } - - private com.google.protobuf.ByteString metadata_ = com.google.protobuf.ByteString.EMPTY; - /** - * optional bytes metadata = 3; - * @return Whether the metadata field is set. - */ - @java.lang.Override - public boolean hasMetadata() { - return ((bitField0_ & 0x00000004) != 0); - } - /** - * optional bytes metadata = 3; - * @return The metadata. - */ - @java.lang.Override - public com.google.protobuf.ByteString getMetadata() { - return metadata_; - } - /** - * optional bytes metadata = 3; - * @param value The metadata to set. - * @return This builder for chaining. - */ - public Builder setMetadata(com.google.protobuf.ByteString value) { - if (value == null) { throw new NullPointerException(); } - metadata_ = value; - bitField0_ |= 0x00000004; - onChanged(); - return this; - } - /** - * optional bytes metadata = 3; - * @return This builder for chaining. - */ - public Builder clearMetadata() { - bitField0_ = (bitField0_ & ~0x00000004); - metadata_ = getDefaultInstance().getMetadata(); - onChanged(); - return this; - } - - // @@protoc_insertion_point(builder_scope:vortex.dtype.Extension) - } - - // @@protoc_insertion_point(class_scope:vortex.dtype.Extension) - private static final io.github.dfa1.vortex.proto.DTypeProtos.Extension DEFAULT_INSTANCE; - static { - DEFAULT_INSTANCE = new io.github.dfa1.vortex.proto.DTypeProtos.Extension(); - } - - public static io.github.dfa1.vortex.proto.DTypeProtos.Extension getDefaultInstance() { - return DEFAULT_INSTANCE; - } - - private static final com.google.protobuf.Parser - PARSER = new com.google.protobuf.AbstractParser() { - @java.lang.Override - public Extension parsePartialFrom( - com.google.protobuf.CodedInputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws com.google.protobuf.InvalidProtocolBufferException { - Builder builder = newBuilder(); - try { - builder.mergeFrom(input, extensionRegistry); - } catch (com.google.protobuf.InvalidProtocolBufferException e) { - throw e.setUnfinishedMessage(builder.buildPartial()); - } catch (com.google.protobuf.UninitializedMessageException e) { - throw e.asInvalidProtocolBufferException().setUnfinishedMessage(builder.buildPartial()); - } catch (java.io.IOException e) { - throw new com.google.protobuf.InvalidProtocolBufferException(e) - .setUnfinishedMessage(builder.buildPartial()); - } - return builder.buildPartial(); - } - }; - - public static com.google.protobuf.Parser parser() { - return PARSER; - } - - @java.lang.Override - public com.google.protobuf.Parser getParserForType() { - return PARSER; - } - - @java.lang.Override - public io.github.dfa1.vortex.proto.DTypeProtos.Extension getDefaultInstanceForType() { - return DEFAULT_INSTANCE; - } - - } - - public interface VariantOrBuilder extends - // @@protoc_insertion_point(interface_extends:vortex.dtype.Variant) - com.google.protobuf.MessageOrBuilder { - - /** - * bool nullable = 1; - * @return The nullable. - */ - boolean getNullable(); - } - /** - * Protobuf type {@code vortex.dtype.Variant} - */ - public static final class Variant extends - com.google.protobuf.GeneratedMessage implements - // @@protoc_insertion_point(message_implements:vortex.dtype.Variant) - VariantOrBuilder { - private static final long serialVersionUID = 0L; - static { - com.google.protobuf.RuntimeVersion.validateProtobufGencodeVersion( - com.google.protobuf.RuntimeVersion.RuntimeDomain.PUBLIC, - /* major= */ 4, - /* minor= */ 35, - /* patch= */ 0, - /* suffix= */ "", - "Variant"); - } - // Use Variant.newBuilder() to construct. - private Variant(com.google.protobuf.GeneratedMessage.Builder builder) { - super(builder); - } - private Variant() { - } - - public static final com.google.protobuf.Descriptors.Descriptor - getDescriptor() { - return io.github.dfa1.vortex.proto.DTypeProtos.internal_static_vortex_dtype_Variant_descriptor; - } - - @java.lang.Override - public com.google.protobuf.Descriptors.Descriptor getDescriptorForType() { - return io.github.dfa1.vortex.proto.DTypeProtos.internal_static_vortex_dtype_Variant_descriptor; - } - - @java.lang.Override - protected com.google.protobuf.GeneratedMessage.FieldAccessorTable - internalGetFieldAccessorTable() { - return io.github.dfa1.vortex.proto.DTypeProtos.internal_static_vortex_dtype_Variant_fieldAccessorTable - .ensureFieldAccessorsInitialized( - io.github.dfa1.vortex.proto.DTypeProtos.Variant.class, io.github.dfa1.vortex.proto.DTypeProtos.Variant.Builder.class); - } - - public static final int NULLABLE_FIELD_NUMBER = 1; - private boolean nullable_ = false; - /** - * bool nullable = 1; - * @return The nullable. - */ - @java.lang.Override - public boolean getNullable() { - return nullable_; - } - - private byte memoizedIsInitialized = -1; - @java.lang.Override - public final boolean isInitialized() { - byte isInitialized = memoizedIsInitialized; - if (isInitialized == 1) return true; - if (isInitialized == 0) return false; - - memoizedIsInitialized = 1; - return true; - } - - @java.lang.Override - public void writeTo(com.google.protobuf.CodedOutputStream output) - throws java.io.IOException { - if (nullable_ != false) { - output.writeBool(1, nullable_); - } - getUnknownFields().writeTo(output); - } - private int computeSerializedSize_0() { - int size = 0; - if (nullable_ != false) { - size += com.google.protobuf.CodedOutputStream - .computeBoolSize(1, nullable_); - } - return size; - } - @java.lang.Override - public int getSerializedSize() { - int size = memoizedSize; - if (size != -1) return size; - - size = 0; - size += computeSerializedSize_0(); - size += getUnknownFields().getSerializedSize(); - memoizedSize = size; - return size; - } - - @java.lang.Override - public boolean equals(final java.lang.Object obj) { - if (obj == this) { - return true; - } - if (!(obj instanceof io.github.dfa1.vortex.proto.DTypeProtos.Variant)) { - return super.equals(obj); - } - io.github.dfa1.vortex.proto.DTypeProtos.Variant other = (io.github.dfa1.vortex.proto.DTypeProtos.Variant) obj; - - if (getNullable() - != other.getNullable()) return false; - if (!getUnknownFields().equals(other.getUnknownFields())) return false; - return true; - } - - @java.lang.Override - public int hashCode() { - if (memoizedHashCode != 0) { - return memoizedHashCode; - } - int hash = 41; - hash = (19 * hash) + getDescriptor().hashCode(); - hash = (37 * hash) + NULLABLE_FIELD_NUMBER; - hash = (53 * hash) + com.google.protobuf.Internal.hashBoolean( - getNullable()); - hash = (29 * hash) + getUnknownFields().hashCode(); - memoizedHashCode = hash; - return hash; - } - - public static io.github.dfa1.vortex.proto.DTypeProtos.Variant parseFrom( - java.nio.ByteBuffer data) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data); - } - public static io.github.dfa1.vortex.proto.DTypeProtos.Variant parseFrom( - java.nio.ByteBuffer data, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data, extensionRegistry); - } - public static io.github.dfa1.vortex.proto.DTypeProtos.Variant parseFrom( - com.google.protobuf.ByteString data) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data); - } - public static io.github.dfa1.vortex.proto.DTypeProtos.Variant parseFrom( - com.google.protobuf.ByteString data, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data, extensionRegistry); - } - public static io.github.dfa1.vortex.proto.DTypeProtos.Variant parseFrom(byte[] data) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data); - } - public static io.github.dfa1.vortex.proto.DTypeProtos.Variant parseFrom( - byte[] data, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data, extensionRegistry); - } - public static io.github.dfa1.vortex.proto.DTypeProtos.Variant parseFrom(java.io.InputStream input) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseWithIOException(PARSER, input); - } - public static io.github.dfa1.vortex.proto.DTypeProtos.Variant parseFrom( - java.io.InputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseWithIOException(PARSER, input, extensionRegistry); - } - - public static io.github.dfa1.vortex.proto.DTypeProtos.Variant parseDelimitedFrom(java.io.InputStream input) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseDelimitedWithIOException(PARSER, input); - } - - public static io.github.dfa1.vortex.proto.DTypeProtos.Variant parseDelimitedFrom( - java.io.InputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseDelimitedWithIOException(PARSER, input, extensionRegistry); - } - public static io.github.dfa1.vortex.proto.DTypeProtos.Variant parseFrom( - com.google.protobuf.CodedInputStream input) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseWithIOException(PARSER, input); - } - public static io.github.dfa1.vortex.proto.DTypeProtos.Variant parseFrom( - com.google.protobuf.CodedInputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseWithIOException(PARSER, input, extensionRegistry); - } - - @java.lang.Override - public Builder newBuilderForType() { return newBuilder(); } - public static Builder newBuilder() { - return DEFAULT_INSTANCE.toBuilder(); - } - public static Builder newBuilder(io.github.dfa1.vortex.proto.DTypeProtos.Variant prototype) { - return DEFAULT_INSTANCE.toBuilder().mergeFrom(prototype); - } - @java.lang.Override - public Builder toBuilder() { - return this == DEFAULT_INSTANCE - ? new Builder() : new Builder().mergeFrom(this); - } - - @java.lang.Override - protected Builder newBuilderForType( - com.google.protobuf.GeneratedMessage.BuilderParent parent) { - Builder builder = new Builder(parent); - return builder; - } - /** - * Protobuf type {@code vortex.dtype.Variant} - */ - public static final class Builder extends - com.google.protobuf.GeneratedMessage.Builder implements - // @@protoc_insertion_point(builder_implements:vortex.dtype.Variant) - io.github.dfa1.vortex.proto.DTypeProtos.VariantOrBuilder { - public static final com.google.protobuf.Descriptors.Descriptor - getDescriptor() { - return io.github.dfa1.vortex.proto.DTypeProtos.internal_static_vortex_dtype_Variant_descriptor; - } - - @java.lang.Override - protected com.google.protobuf.GeneratedMessage.FieldAccessorTable - internalGetFieldAccessorTable() { - return io.github.dfa1.vortex.proto.DTypeProtos.internal_static_vortex_dtype_Variant_fieldAccessorTable - .ensureFieldAccessorsInitialized( - io.github.dfa1.vortex.proto.DTypeProtos.Variant.class, io.github.dfa1.vortex.proto.DTypeProtos.Variant.Builder.class); - } - - // Construct using io.github.dfa1.vortex.proto.DTypeProtos.Variant.newBuilder() - private Builder() { - - } - - private Builder( - com.google.protobuf.GeneratedMessage.BuilderParent parent) { - super(parent); - - } - @java.lang.Override - public Builder clear() { - super.clear(); - bitField0_ = 0; - nullable_ = false; - return this; - } - - @java.lang.Override - public com.google.protobuf.Descriptors.Descriptor - getDescriptorForType() { - return io.github.dfa1.vortex.proto.DTypeProtos.internal_static_vortex_dtype_Variant_descriptor; - } - - @java.lang.Override - public io.github.dfa1.vortex.proto.DTypeProtos.Variant getDefaultInstanceForType() { - return io.github.dfa1.vortex.proto.DTypeProtos.Variant.getDefaultInstance(); - } - - @java.lang.Override - public io.github.dfa1.vortex.proto.DTypeProtos.Variant build() { - io.github.dfa1.vortex.proto.DTypeProtos.Variant result = buildPartial(); - if (!result.isInitialized()) { - throw newUninitializedMessageException(result); - } - return result; - } - - @java.lang.Override - public io.github.dfa1.vortex.proto.DTypeProtos.Variant buildPartial() { - io.github.dfa1.vortex.proto.DTypeProtos.Variant result = new io.github.dfa1.vortex.proto.DTypeProtos.Variant(this); - if (bitField0_ != 0) { buildPartial0(result); } - onBuilt(); - return result; - } - - private void buildPartial0(io.github.dfa1.vortex.proto.DTypeProtos.Variant result) { - int from_bitField0_ = bitField0_; - if (((from_bitField0_ & 0x00000001) != 0)) { - result.nullable_ = nullable_; - } - } - - @java.lang.Override - public Builder mergeFrom(com.google.protobuf.Message other) { - if (other instanceof io.github.dfa1.vortex.proto.DTypeProtos.Variant) { - return mergeFrom((io.github.dfa1.vortex.proto.DTypeProtos.Variant)other); - } else { - super.mergeFrom(other); - return this; - } - } - - public Builder mergeFrom(io.github.dfa1.vortex.proto.DTypeProtos.Variant other) { - if (other == io.github.dfa1.vortex.proto.DTypeProtos.Variant.getDefaultInstance()) return this; - if (other.getNullable() != false) { - setNullable(other.getNullable()); - } - this.mergeUnknownFields(other.getUnknownFields()); - onChanged(); - return this; - } - - @java.lang.Override - public final boolean isInitialized() { - return true; - } - - @java.lang.Override - public Builder mergeFrom( - com.google.protobuf.CodedInputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws java.io.IOException { - if (extensionRegistry == null) { - throw new java.lang.NullPointerException(); - } - try { - boolean done = false; - while (!done) { - int tag = input.readTag(); - switch (tag) { - case 0: - done = true; - break; - case 8: { - nullable_ = input.readBool(); - bitField0_ |= 0x00000001; - break; - } // case 8 - default: { - if (!super.parseUnknownField(input, extensionRegistry, tag)) { - done = true; // was an endgroup tag - } - break; - } // default: - } // switch (tag) - } // while (!done) - } catch (com.google.protobuf.InvalidProtocolBufferException e) { - throw e.unwrapIOException(); - } finally { - onChanged(); - } // finally - return this; - } - private int bitField0_; - - private boolean nullable_ ; - /** - * bool nullable = 1; - * @return The nullable. - */ - @java.lang.Override - public boolean getNullable() { - return nullable_; - } - /** - * bool nullable = 1; - * @param value The nullable to set. - * @return This builder for chaining. - */ - public Builder setNullable(boolean value) { - - nullable_ = value; - bitField0_ |= 0x00000001; - onChanged(); - return this; - } - /** - * bool nullable = 1; - * @return This builder for chaining. - */ - public Builder clearNullable() { - bitField0_ = (bitField0_ & ~0x00000001); - nullable_ = false; - onChanged(); - return this; - } - - // @@protoc_insertion_point(builder_scope:vortex.dtype.Variant) - } - - // @@protoc_insertion_point(class_scope:vortex.dtype.Variant) - private static final io.github.dfa1.vortex.proto.DTypeProtos.Variant DEFAULT_INSTANCE; - static { - DEFAULT_INSTANCE = new io.github.dfa1.vortex.proto.DTypeProtos.Variant(); - } - - public static io.github.dfa1.vortex.proto.DTypeProtos.Variant getDefaultInstance() { - return DEFAULT_INSTANCE; - } - - private static final com.google.protobuf.Parser - PARSER = new com.google.protobuf.AbstractParser() { - @java.lang.Override - public Variant parsePartialFrom( - com.google.protobuf.CodedInputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws com.google.protobuf.InvalidProtocolBufferException { - Builder builder = newBuilder(); - try { - builder.mergeFrom(input, extensionRegistry); - } catch (com.google.protobuf.InvalidProtocolBufferException e) { - throw e.setUnfinishedMessage(builder.buildPartial()); - } catch (com.google.protobuf.UninitializedMessageException e) { - throw e.asInvalidProtocolBufferException().setUnfinishedMessage(builder.buildPartial()); - } catch (java.io.IOException e) { - throw new com.google.protobuf.InvalidProtocolBufferException(e) - .setUnfinishedMessage(builder.buildPartial()); - } - return builder.buildPartial(); - } - }; - - public static com.google.protobuf.Parser parser() { - return PARSER; - } - - @java.lang.Override - public com.google.protobuf.Parser getParserForType() { - return PARSER; - } - - @java.lang.Override - public io.github.dfa1.vortex.proto.DTypeProtos.Variant getDefaultInstanceForType() { - return DEFAULT_INSTANCE; - } - - } - - public interface UnionOrBuilder extends - // @@protoc_insertion_point(interface_extends:vortex.dtype.Union) - com.google.protobuf.MessageOrBuilder { - - /** - * bool nullable = 4; - * @return The nullable. - */ - boolean getNullable(); - } - /** - * Protobuf type {@code vortex.dtype.Union} - */ - public static final class Union extends - com.google.protobuf.GeneratedMessage implements - // @@protoc_insertion_point(message_implements:vortex.dtype.Union) - UnionOrBuilder { - private static final long serialVersionUID = 0L; - static { - com.google.protobuf.RuntimeVersion.validateProtobufGencodeVersion( - com.google.protobuf.RuntimeVersion.RuntimeDomain.PUBLIC, - /* major= */ 4, - /* minor= */ 35, - /* patch= */ 0, - /* suffix= */ "", - "Union"); - } - // Use Union.newBuilder() to construct. - private Union(com.google.protobuf.GeneratedMessage.Builder builder) { - super(builder); - } - private Union() { - } - - public static final com.google.protobuf.Descriptors.Descriptor - getDescriptor() { - return io.github.dfa1.vortex.proto.DTypeProtos.internal_static_vortex_dtype_Union_descriptor; - } - - @java.lang.Override - public com.google.protobuf.Descriptors.Descriptor getDescriptorForType() { - return io.github.dfa1.vortex.proto.DTypeProtos.internal_static_vortex_dtype_Union_descriptor; - } - - @java.lang.Override - protected com.google.protobuf.GeneratedMessage.FieldAccessorTable - internalGetFieldAccessorTable() { - return io.github.dfa1.vortex.proto.DTypeProtos.internal_static_vortex_dtype_Union_fieldAccessorTable - .ensureFieldAccessorsInitialized( - io.github.dfa1.vortex.proto.DTypeProtos.Union.class, io.github.dfa1.vortex.proto.DTypeProtos.Union.Builder.class); - } - - public static final int NULLABLE_FIELD_NUMBER = 4; - private boolean nullable_ = false; - /** - * bool nullable = 4; - * @return The nullable. - */ - @java.lang.Override - public boolean getNullable() { - return nullable_; - } - - private byte memoizedIsInitialized = -1; - @java.lang.Override - public final boolean isInitialized() { - byte isInitialized = memoizedIsInitialized; - if (isInitialized == 1) return true; - if (isInitialized == 0) return false; - - memoizedIsInitialized = 1; - return true; - } - - @java.lang.Override - public void writeTo(com.google.protobuf.CodedOutputStream output) - throws java.io.IOException { - if (nullable_ != false) { - output.writeBool(4, nullable_); - } - getUnknownFields().writeTo(output); - } - private int computeSerializedSize_0() { - int size = 0; - if (nullable_ != false) { - size += com.google.protobuf.CodedOutputStream - .computeBoolSize(4, nullable_); - } - return size; - } - @java.lang.Override - public int getSerializedSize() { - int size = memoizedSize; - if (size != -1) return size; - - size = 0; - size += computeSerializedSize_0(); - size += getUnknownFields().getSerializedSize(); - memoizedSize = size; - return size; - } - - @java.lang.Override - public boolean equals(final java.lang.Object obj) { - if (obj == this) { - return true; - } - if (!(obj instanceof io.github.dfa1.vortex.proto.DTypeProtos.Union)) { - return super.equals(obj); - } - io.github.dfa1.vortex.proto.DTypeProtos.Union other = (io.github.dfa1.vortex.proto.DTypeProtos.Union) obj; - - if (getNullable() - != other.getNullable()) return false; - if (!getUnknownFields().equals(other.getUnknownFields())) return false; - return true; - } - - @java.lang.Override - public int hashCode() { - if (memoizedHashCode != 0) { - return memoizedHashCode; - } - int hash = 41; - hash = (19 * hash) + getDescriptor().hashCode(); - hash = (37 * hash) + NULLABLE_FIELD_NUMBER; - hash = (53 * hash) + com.google.protobuf.Internal.hashBoolean( - getNullable()); - hash = (29 * hash) + getUnknownFields().hashCode(); - memoizedHashCode = hash; - return hash; - } - - public static io.github.dfa1.vortex.proto.DTypeProtos.Union parseFrom( - java.nio.ByteBuffer data) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data); - } - public static io.github.dfa1.vortex.proto.DTypeProtos.Union parseFrom( - java.nio.ByteBuffer data, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data, extensionRegistry); - } - public static io.github.dfa1.vortex.proto.DTypeProtos.Union parseFrom( - com.google.protobuf.ByteString data) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data); - } - public static io.github.dfa1.vortex.proto.DTypeProtos.Union parseFrom( - com.google.protobuf.ByteString data, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data, extensionRegistry); - } - public static io.github.dfa1.vortex.proto.DTypeProtos.Union parseFrom(byte[] data) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data); - } - public static io.github.dfa1.vortex.proto.DTypeProtos.Union parseFrom( - byte[] data, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data, extensionRegistry); - } - public static io.github.dfa1.vortex.proto.DTypeProtos.Union parseFrom(java.io.InputStream input) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseWithIOException(PARSER, input); - } - public static io.github.dfa1.vortex.proto.DTypeProtos.Union parseFrom( - java.io.InputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseWithIOException(PARSER, input, extensionRegistry); - } - - public static io.github.dfa1.vortex.proto.DTypeProtos.Union parseDelimitedFrom(java.io.InputStream input) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseDelimitedWithIOException(PARSER, input); - } - - public static io.github.dfa1.vortex.proto.DTypeProtos.Union parseDelimitedFrom( - java.io.InputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseDelimitedWithIOException(PARSER, input, extensionRegistry); - } - public static io.github.dfa1.vortex.proto.DTypeProtos.Union parseFrom( - com.google.protobuf.CodedInputStream input) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseWithIOException(PARSER, input); - } - public static io.github.dfa1.vortex.proto.DTypeProtos.Union parseFrom( - com.google.protobuf.CodedInputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseWithIOException(PARSER, input, extensionRegistry); - } - - @java.lang.Override - public Builder newBuilderForType() { return newBuilder(); } - public static Builder newBuilder() { - return DEFAULT_INSTANCE.toBuilder(); - } - public static Builder newBuilder(io.github.dfa1.vortex.proto.DTypeProtos.Union prototype) { - return DEFAULT_INSTANCE.toBuilder().mergeFrom(prototype); - } - @java.lang.Override - public Builder toBuilder() { - return this == DEFAULT_INSTANCE - ? new Builder() : new Builder().mergeFrom(this); - } - - @java.lang.Override - protected Builder newBuilderForType( - com.google.protobuf.GeneratedMessage.BuilderParent parent) { - Builder builder = new Builder(parent); - return builder; - } - /** - * Protobuf type {@code vortex.dtype.Union} - */ - public static final class Builder extends - com.google.protobuf.GeneratedMessage.Builder implements - // @@protoc_insertion_point(builder_implements:vortex.dtype.Union) - io.github.dfa1.vortex.proto.DTypeProtos.UnionOrBuilder { - public static final com.google.protobuf.Descriptors.Descriptor - getDescriptor() { - return io.github.dfa1.vortex.proto.DTypeProtos.internal_static_vortex_dtype_Union_descriptor; - } - - @java.lang.Override - protected com.google.protobuf.GeneratedMessage.FieldAccessorTable - internalGetFieldAccessorTable() { - return io.github.dfa1.vortex.proto.DTypeProtos.internal_static_vortex_dtype_Union_fieldAccessorTable - .ensureFieldAccessorsInitialized( - io.github.dfa1.vortex.proto.DTypeProtos.Union.class, io.github.dfa1.vortex.proto.DTypeProtos.Union.Builder.class); - } - - // Construct using io.github.dfa1.vortex.proto.DTypeProtos.Union.newBuilder() - private Builder() { - - } - - private Builder( - com.google.protobuf.GeneratedMessage.BuilderParent parent) { - super(parent); - - } - @java.lang.Override - public Builder clear() { - super.clear(); - bitField0_ = 0; - nullable_ = false; - return this; - } - - @java.lang.Override - public com.google.protobuf.Descriptors.Descriptor - getDescriptorForType() { - return io.github.dfa1.vortex.proto.DTypeProtos.internal_static_vortex_dtype_Union_descriptor; - } - - @java.lang.Override - public io.github.dfa1.vortex.proto.DTypeProtos.Union getDefaultInstanceForType() { - return io.github.dfa1.vortex.proto.DTypeProtos.Union.getDefaultInstance(); - } - - @java.lang.Override - public io.github.dfa1.vortex.proto.DTypeProtos.Union build() { - io.github.dfa1.vortex.proto.DTypeProtos.Union result = buildPartial(); - if (!result.isInitialized()) { - throw newUninitializedMessageException(result); - } - return result; - } - - @java.lang.Override - public io.github.dfa1.vortex.proto.DTypeProtos.Union buildPartial() { - io.github.dfa1.vortex.proto.DTypeProtos.Union result = new io.github.dfa1.vortex.proto.DTypeProtos.Union(this); - if (bitField0_ != 0) { buildPartial0(result); } - onBuilt(); - return result; - } - - private void buildPartial0(io.github.dfa1.vortex.proto.DTypeProtos.Union result) { - int from_bitField0_ = bitField0_; - if (((from_bitField0_ & 0x00000001) != 0)) { - result.nullable_ = nullable_; - } - } - - @java.lang.Override - public Builder mergeFrom(com.google.protobuf.Message other) { - if (other instanceof io.github.dfa1.vortex.proto.DTypeProtos.Union) { - return mergeFrom((io.github.dfa1.vortex.proto.DTypeProtos.Union)other); - } else { - super.mergeFrom(other); - return this; - } - } - - public Builder mergeFrom(io.github.dfa1.vortex.proto.DTypeProtos.Union other) { - if (other == io.github.dfa1.vortex.proto.DTypeProtos.Union.getDefaultInstance()) return this; - if (other.getNullable() != false) { - setNullable(other.getNullable()); - } - this.mergeUnknownFields(other.getUnknownFields()); - onChanged(); - return this; - } - - @java.lang.Override - public final boolean isInitialized() { - return true; - } - - @java.lang.Override - public Builder mergeFrom( - com.google.protobuf.CodedInputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws java.io.IOException { - if (extensionRegistry == null) { - throw new java.lang.NullPointerException(); - } - try { - boolean done = false; - while (!done) { - int tag = input.readTag(); - switch (tag) { - case 0: - done = true; - break; - case 32: { - nullable_ = input.readBool(); - bitField0_ |= 0x00000001; - break; - } // case 32 - default: { - if (!super.parseUnknownField(input, extensionRegistry, tag)) { - done = true; // was an endgroup tag - } - break; - } // default: - } // switch (tag) - } // while (!done) - } catch (com.google.protobuf.InvalidProtocolBufferException e) { - throw e.unwrapIOException(); - } finally { - onChanged(); - } // finally - return this; - } - private int bitField0_; - - private boolean nullable_ ; - /** - * bool nullable = 4; - * @return The nullable. - */ - @java.lang.Override - public boolean getNullable() { - return nullable_; - } - /** - * bool nullable = 4; - * @param value The nullable to set. - * @return This builder for chaining. - */ - public Builder setNullable(boolean value) { - - nullable_ = value; - bitField0_ |= 0x00000001; - onChanged(); - return this; - } - /** - * bool nullable = 4; - * @return This builder for chaining. - */ - public Builder clearNullable() { - bitField0_ = (bitField0_ & ~0x00000001); - nullable_ = false; - onChanged(); - return this; - } - - // @@protoc_insertion_point(builder_scope:vortex.dtype.Union) - } - - // @@protoc_insertion_point(class_scope:vortex.dtype.Union) - private static final io.github.dfa1.vortex.proto.DTypeProtos.Union DEFAULT_INSTANCE; - static { - DEFAULT_INSTANCE = new io.github.dfa1.vortex.proto.DTypeProtos.Union(); - } - - public static io.github.dfa1.vortex.proto.DTypeProtos.Union getDefaultInstance() { - return DEFAULT_INSTANCE; - } - - private static final com.google.protobuf.Parser - PARSER = new com.google.protobuf.AbstractParser() { - @java.lang.Override - public Union parsePartialFrom( - com.google.protobuf.CodedInputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws com.google.protobuf.InvalidProtocolBufferException { - Builder builder = newBuilder(); - try { - builder.mergeFrom(input, extensionRegistry); - } catch (com.google.protobuf.InvalidProtocolBufferException e) { - throw e.setUnfinishedMessage(builder.buildPartial()); - } catch (com.google.protobuf.UninitializedMessageException e) { - throw e.asInvalidProtocolBufferException().setUnfinishedMessage(builder.buildPartial()); - } catch (java.io.IOException e) { - throw new com.google.protobuf.InvalidProtocolBufferException(e) - .setUnfinishedMessage(builder.buildPartial()); - } - return builder.buildPartial(); - } - }; - - public static com.google.protobuf.Parser parser() { - return PARSER; - } - - @java.lang.Override - public com.google.protobuf.Parser getParserForType() { - return PARSER; - } - - @java.lang.Override - public io.github.dfa1.vortex.proto.DTypeProtos.Union getDefaultInstanceForType() { - return DEFAULT_INSTANCE; - } - - } - - public interface DTypeOrBuilder extends - // @@protoc_insertion_point(interface_extends:vortex.dtype.DType) - com.google.protobuf.MessageOrBuilder { - - /** - * .vortex.dtype.Null null = 1; - * @return Whether the null field is set. - */ - boolean hasNull(); - /** - * .vortex.dtype.Null null = 1; - * @return The null. - */ - io.github.dfa1.vortex.proto.DTypeProtos.Null getNull(); - /** - * .vortex.dtype.Null null = 1; - */ - io.github.dfa1.vortex.proto.DTypeProtos.NullOrBuilder getNullOrBuilder(); - - /** - * .vortex.dtype.Bool bool = 2; - * @return Whether the bool field is set. - */ - boolean hasBool(); - /** - * .vortex.dtype.Bool bool = 2; - * @return The bool. - */ - io.github.dfa1.vortex.proto.DTypeProtos.Bool getBool(); - /** - * .vortex.dtype.Bool bool = 2; - */ - io.github.dfa1.vortex.proto.DTypeProtos.BoolOrBuilder getBoolOrBuilder(); - - /** - * .vortex.dtype.Primitive primitive = 3; - * @return Whether the primitive field is set. - */ - boolean hasPrimitive(); - /** - * .vortex.dtype.Primitive primitive = 3; - * @return The primitive. - */ - io.github.dfa1.vortex.proto.DTypeProtos.Primitive getPrimitive(); - /** - * .vortex.dtype.Primitive primitive = 3; - */ - io.github.dfa1.vortex.proto.DTypeProtos.PrimitiveOrBuilder getPrimitiveOrBuilder(); - - /** - * .vortex.dtype.Decimal decimal = 4; - * @return Whether the decimal field is set. - */ - boolean hasDecimal(); - /** - * .vortex.dtype.Decimal decimal = 4; - * @return The decimal. - */ - io.github.dfa1.vortex.proto.DTypeProtos.Decimal getDecimal(); - /** - * .vortex.dtype.Decimal decimal = 4; - */ - io.github.dfa1.vortex.proto.DTypeProtos.DecimalOrBuilder getDecimalOrBuilder(); - - /** - * .vortex.dtype.Utf8 utf8 = 5; - * @return Whether the utf8 field is set. - */ - boolean hasUtf8(); - /** - * .vortex.dtype.Utf8 utf8 = 5; - * @return The utf8. - */ - io.github.dfa1.vortex.proto.DTypeProtos.Utf8 getUtf8(); - /** - * .vortex.dtype.Utf8 utf8 = 5; - */ - io.github.dfa1.vortex.proto.DTypeProtos.Utf8OrBuilder getUtf8OrBuilder(); - - /** - * .vortex.dtype.Binary binary = 6; - * @return Whether the binary field is set. - */ - boolean hasBinary(); - /** - * .vortex.dtype.Binary binary = 6; - * @return The binary. - */ - io.github.dfa1.vortex.proto.DTypeProtos.Binary getBinary(); - /** - * .vortex.dtype.Binary binary = 6; - */ - io.github.dfa1.vortex.proto.DTypeProtos.BinaryOrBuilder getBinaryOrBuilder(); - - /** - * .vortex.dtype.Struct struct = 7; - * @return Whether the struct field is set. - */ - boolean hasStruct(); - /** - * .vortex.dtype.Struct struct = 7; - * @return The struct. - */ - io.github.dfa1.vortex.proto.DTypeProtos.Struct getStruct(); - /** - * .vortex.dtype.Struct struct = 7; - */ - io.github.dfa1.vortex.proto.DTypeProtos.StructOrBuilder getStructOrBuilder(); - - /** - * .vortex.dtype.List list = 8; - * @return Whether the list field is set. - */ - boolean hasList(); - /** - * .vortex.dtype.List list = 8; - * @return The list. - */ - io.github.dfa1.vortex.proto.DTypeProtos.List getList(); - /** - * .vortex.dtype.List list = 8; - */ - io.github.dfa1.vortex.proto.DTypeProtos.ListOrBuilder getListOrBuilder(); - - /** - * .vortex.dtype.Extension extension = 9; - * @return Whether the extension field is set. - */ - boolean hasExtension(); - /** - * .vortex.dtype.Extension extension = 9; - * @return The extension. - */ - io.github.dfa1.vortex.proto.DTypeProtos.Extension getExtension(); - /** - * .vortex.dtype.Extension extension = 9; - */ - io.github.dfa1.vortex.proto.DTypeProtos.ExtensionOrBuilder getExtensionOrBuilder(); - - /** - * 

-     * This is after `Extension` for backwards compatibility.
-     *
- * - * .vortex.dtype.FixedSizeList fixed_size_list = 10; - * @return Whether the fixedSizeList field is set. - */ - boolean hasFixedSizeList(); - /** - * 
-     * This is after `Extension` for backwards compatibility.
-     *
- * - * .vortex.dtype.FixedSizeList fixed_size_list = 10; - * @return The fixedSizeList. - */ - io.github.dfa1.vortex.proto.DTypeProtos.FixedSizeList getFixedSizeList(); - /** - * 
-     * This is after `Extension` for backwards compatibility.
-     *
- * - * .vortex.dtype.FixedSizeList fixed_size_list = 10; - */ - io.github.dfa1.vortex.proto.DTypeProtos.FixedSizeListOrBuilder getFixedSizeListOrBuilder(); - - /** - * .vortex.dtype.Variant variant = 11; - * @return Whether the variant field is set. - */ - boolean hasVariant(); - /** - * .vortex.dtype.Variant variant = 11; - * @return The variant. - */ - io.github.dfa1.vortex.proto.DTypeProtos.Variant getVariant(); - /** - * .vortex.dtype.Variant variant = 11; - */ - io.github.dfa1.vortex.proto.DTypeProtos.VariantOrBuilder getVariantOrBuilder(); - - /** - * .vortex.dtype.Union union = 12; - * @return Whether the union field is set. - */ - boolean hasUnion(); - /** - * .vortex.dtype.Union union = 12; - * @return The union. - */ - io.github.dfa1.vortex.proto.DTypeProtos.Union getUnion(); - /** - * .vortex.dtype.Union union = 12; - */ - io.github.dfa1.vortex.proto.DTypeProtos.UnionOrBuilder getUnionOrBuilder(); - - io.github.dfa1.vortex.proto.DTypeProtos.DType.DtypeTypeCase getDtypeTypeCase(); - } - /** - * Protobuf type {@code vortex.dtype.DType} - */ - public static final class DType extends - com.google.protobuf.GeneratedMessage implements - // @@protoc_insertion_point(message_implements:vortex.dtype.DType) - DTypeOrBuilder { - private static final long serialVersionUID = 0L; - static { - com.google.protobuf.RuntimeVersion.validateProtobufGencodeVersion( - com.google.protobuf.RuntimeVersion.RuntimeDomain.PUBLIC, - /* major= */ 4, - /* minor= */ 35, - /* patch= */ 0, - /* suffix= */ "", - "DType"); - } - // Use DType.newBuilder() to construct. - private DType(com.google.protobuf.GeneratedMessage.Builder builder) { - super(builder); - } - private DType() { - } - - public static final com.google.protobuf.Descriptors.Descriptor - getDescriptor() { - return io.github.dfa1.vortex.proto.DTypeProtos.internal_static_vortex_dtype_DType_descriptor; - } - - @java.lang.Override - public com.google.protobuf.Descriptors.Descriptor getDescriptorForType() { - return io.github.dfa1.vortex.proto.DTypeProtos.internal_static_vortex_dtype_DType_descriptor; - } - - @java.lang.Override - protected com.google.protobuf.GeneratedMessage.FieldAccessorTable - internalGetFieldAccessorTable() { - return io.github.dfa1.vortex.proto.DTypeProtos.internal_static_vortex_dtype_DType_fieldAccessorTable - .ensureFieldAccessorsInitialized( - io.github.dfa1.vortex.proto.DTypeProtos.DType.class, io.github.dfa1.vortex.proto.DTypeProtos.DType.Builder.class); - } - - private int dtypeTypeCase_ = 0; - @SuppressWarnings("serial") - private java.lang.Object dtypeType_; - public enum DtypeTypeCase - implements com.google.protobuf.Internal.EnumLite, - com.google.protobuf.AbstractMessage.InternalOneOfEnum { - NULL(1), - BOOL(2), - PRIMITIVE(3), - DECIMAL(4), - UTF8(5), - BINARY(6), - STRUCT(7), - LIST(8), - EXTENSION(9), - FIXED_SIZE_LIST(10), - VARIANT(11), - UNION(12), - DTYPETYPE_NOT_SET(0); - private final int value; - private DtypeTypeCase(int value) { - this.value = value; - } - /** - * @param value The number of the enum to look for. - * @return The enum associated with the given number. - * @deprecated Use {@link #forNumber(int)} instead. - */ - @java.lang.Deprecated - public static DtypeTypeCase valueOf(int value) { - return forNumber(value); - } - - public static DtypeTypeCase forNumber(int value) { - switch (value) { - case 1: return NULL; - case 2: return BOOL; - case 3: return PRIMITIVE; - case 4: return DECIMAL; - case 5: return UTF8; - case 6: return BINARY; - case 7: return STRUCT; - case 8: return LIST; - case 9: return EXTENSION; - case 10: return FIXED_SIZE_LIST; - case 11: return VARIANT; - case 12: return UNION; - case 0: return DTYPETYPE_NOT_SET; - default: return null; - } - } - public int getNumber() { - return this.value; - } - }; - - public DtypeTypeCase - getDtypeTypeCase() { - return DtypeTypeCase.forNumber( - dtypeTypeCase_); - } - - public static final int NULL_FIELD_NUMBER = 1; - /** - * .vortex.dtype.Null null = 1; - * @return Whether the null field is set. - */ - @java.lang.Override - public boolean hasNull() { - return dtypeTypeCase_ == 1; - } - /** - * .vortex.dtype.Null null = 1; - * @return The null. - */ - @java.lang.Override - public io.github.dfa1.vortex.proto.DTypeProtos.Null getNull() { - if (dtypeTypeCase_ == 1) { - return (io.github.dfa1.vortex.proto.DTypeProtos.Null) dtypeType_; - } - return io.github.dfa1.vortex.proto.DTypeProtos.Null.getDefaultInstance(); - } - /** - * .vortex.dtype.Null null = 1; - */ - @java.lang.Override - public io.github.dfa1.vortex.proto.DTypeProtos.NullOrBuilder getNullOrBuilder() { - if (dtypeTypeCase_ == 1) { - return (io.github.dfa1.vortex.proto.DTypeProtos.Null) dtypeType_; - } - return io.github.dfa1.vortex.proto.DTypeProtos.Null.getDefaultInstance(); - } - - public static final int BOOL_FIELD_NUMBER = 2; - /** - * .vortex.dtype.Bool bool = 2; - * @return Whether the bool field is set. - */ - @java.lang.Override - public boolean hasBool() { - return dtypeTypeCase_ == 2; - } - /** - * .vortex.dtype.Bool bool = 2; - * @return The bool. - */ - @java.lang.Override - public io.github.dfa1.vortex.proto.DTypeProtos.Bool getBool() { - if (dtypeTypeCase_ == 2) { - return (io.github.dfa1.vortex.proto.DTypeProtos.Bool) dtypeType_; - } - return io.github.dfa1.vortex.proto.DTypeProtos.Bool.getDefaultInstance(); - } - /** - * .vortex.dtype.Bool bool = 2; - */ - @java.lang.Override - public io.github.dfa1.vortex.proto.DTypeProtos.BoolOrBuilder getBoolOrBuilder() { - if (dtypeTypeCase_ == 2) { - return (io.github.dfa1.vortex.proto.DTypeProtos.Bool) dtypeType_; - } - return io.github.dfa1.vortex.proto.DTypeProtos.Bool.getDefaultInstance(); - } - - public static final int PRIMITIVE_FIELD_NUMBER = 3; - /** - * .vortex.dtype.Primitive primitive = 3; - * @return Whether the primitive field is set. - */ - @java.lang.Override - public boolean hasPrimitive() { - return dtypeTypeCase_ == 3; - } - /** - * .vortex.dtype.Primitive primitive = 3; - * @return The primitive. - */ - @java.lang.Override - public io.github.dfa1.vortex.proto.DTypeProtos.Primitive getPrimitive() { - if (dtypeTypeCase_ == 3) { - return (io.github.dfa1.vortex.proto.DTypeProtos.Primitive) dtypeType_; - } - return io.github.dfa1.vortex.proto.DTypeProtos.Primitive.getDefaultInstance(); - } - /** - * .vortex.dtype.Primitive primitive = 3; - */ - @java.lang.Override - public io.github.dfa1.vortex.proto.DTypeProtos.PrimitiveOrBuilder getPrimitiveOrBuilder() { - if (dtypeTypeCase_ == 3) { - return (io.github.dfa1.vortex.proto.DTypeProtos.Primitive) dtypeType_; - } - return io.github.dfa1.vortex.proto.DTypeProtos.Primitive.getDefaultInstance(); - } - - public static final int DECIMAL_FIELD_NUMBER = 4; - /** - * .vortex.dtype.Decimal decimal = 4; - * @return Whether the decimal field is set. - */ - @java.lang.Override - public boolean hasDecimal() { - return dtypeTypeCase_ == 4; - } - /** - * .vortex.dtype.Decimal decimal = 4; - * @return The decimal. - */ - @java.lang.Override - public io.github.dfa1.vortex.proto.DTypeProtos.Decimal getDecimal() { - if (dtypeTypeCase_ == 4) { - return (io.github.dfa1.vortex.proto.DTypeProtos.Decimal) dtypeType_; - } - return io.github.dfa1.vortex.proto.DTypeProtos.Decimal.getDefaultInstance(); - } - /** - * .vortex.dtype.Decimal decimal = 4; - */ - @java.lang.Override - public io.github.dfa1.vortex.proto.DTypeProtos.DecimalOrBuilder getDecimalOrBuilder() { - if (dtypeTypeCase_ == 4) { - return (io.github.dfa1.vortex.proto.DTypeProtos.Decimal) dtypeType_; - } - return io.github.dfa1.vortex.proto.DTypeProtos.Decimal.getDefaultInstance(); - } - - public static final int UTF8_FIELD_NUMBER = 5; - /** - * .vortex.dtype.Utf8 utf8 = 5; - * @return Whether the utf8 field is set. - */ - @java.lang.Override - public boolean hasUtf8() { - return dtypeTypeCase_ == 5; - } - /** - * .vortex.dtype.Utf8 utf8 = 5; - * @return The utf8. - */ - @java.lang.Override - public io.github.dfa1.vortex.proto.DTypeProtos.Utf8 getUtf8() { - if (dtypeTypeCase_ == 5) { - return (io.github.dfa1.vortex.proto.DTypeProtos.Utf8) dtypeType_; - } - return io.github.dfa1.vortex.proto.DTypeProtos.Utf8.getDefaultInstance(); - } - /** - * .vortex.dtype.Utf8 utf8 = 5; - */ - @java.lang.Override - public io.github.dfa1.vortex.proto.DTypeProtos.Utf8OrBuilder getUtf8OrBuilder() { - if (dtypeTypeCase_ == 5) { - return (io.github.dfa1.vortex.proto.DTypeProtos.Utf8) dtypeType_; - } - return io.github.dfa1.vortex.proto.DTypeProtos.Utf8.getDefaultInstance(); - } - - public static final int BINARY_FIELD_NUMBER = 6; - /** - * .vortex.dtype.Binary binary = 6; - * @return Whether the binary field is set. - */ - @java.lang.Override - public boolean hasBinary() { - return dtypeTypeCase_ == 6; - } - /** - * .vortex.dtype.Binary binary = 6; - * @return The binary. - */ - @java.lang.Override - public io.github.dfa1.vortex.proto.DTypeProtos.Binary getBinary() { - if (dtypeTypeCase_ == 6) { - return (io.github.dfa1.vortex.proto.DTypeProtos.Binary) dtypeType_; - } - return io.github.dfa1.vortex.proto.DTypeProtos.Binary.getDefaultInstance(); - } - /** - * .vortex.dtype.Binary binary = 6; - */ - @java.lang.Override - public io.github.dfa1.vortex.proto.DTypeProtos.BinaryOrBuilder getBinaryOrBuilder() { - if (dtypeTypeCase_ == 6) { - return (io.github.dfa1.vortex.proto.DTypeProtos.Binary) dtypeType_; - } - return io.github.dfa1.vortex.proto.DTypeProtos.Binary.getDefaultInstance(); - } - - public static final int STRUCT_FIELD_NUMBER = 7; - /** - * .vortex.dtype.Struct struct = 7; - * @return Whether the struct field is set. - */ - @java.lang.Override - public boolean hasStruct() { - return dtypeTypeCase_ == 7; - } - /** - * .vortex.dtype.Struct struct = 7; - * @return The struct. - */ - @java.lang.Override - public io.github.dfa1.vortex.proto.DTypeProtos.Struct getStruct() { - if (dtypeTypeCase_ == 7) { - return (io.github.dfa1.vortex.proto.DTypeProtos.Struct) dtypeType_; - } - return io.github.dfa1.vortex.proto.DTypeProtos.Struct.getDefaultInstance(); - } - /** - * .vortex.dtype.Struct struct = 7; - */ - @java.lang.Override - public io.github.dfa1.vortex.proto.DTypeProtos.StructOrBuilder getStructOrBuilder() { - if (dtypeTypeCase_ == 7) { - return (io.github.dfa1.vortex.proto.DTypeProtos.Struct) dtypeType_; - } - return io.github.dfa1.vortex.proto.DTypeProtos.Struct.getDefaultInstance(); - } - - public static final int LIST_FIELD_NUMBER = 8; - /** - * .vortex.dtype.List list = 8; - * @return Whether the list field is set. - */ - @java.lang.Override - public boolean hasList() { - return dtypeTypeCase_ == 8; - } - /** - * .vortex.dtype.List list = 8; - * @return The list. - */ - @java.lang.Override - public io.github.dfa1.vortex.proto.DTypeProtos.List getList() { - if (dtypeTypeCase_ == 8) { - return (io.github.dfa1.vortex.proto.DTypeProtos.List) dtypeType_; - } - return io.github.dfa1.vortex.proto.DTypeProtos.List.getDefaultInstance(); - } - /** - * .vortex.dtype.List list = 8; - */ - @java.lang.Override - public io.github.dfa1.vortex.proto.DTypeProtos.ListOrBuilder getListOrBuilder() { - if (dtypeTypeCase_ == 8) { - return (io.github.dfa1.vortex.proto.DTypeProtos.List) dtypeType_; - } - return io.github.dfa1.vortex.proto.DTypeProtos.List.getDefaultInstance(); - } - - public static final int EXTENSION_FIELD_NUMBER = 9; - /** - * .vortex.dtype.Extension extension = 9; - * @return Whether the extension field is set. - */ - @java.lang.Override - public boolean hasExtension() { - return dtypeTypeCase_ == 9; - } - /** - * .vortex.dtype.Extension extension = 9; - * @return The extension. - */ - @java.lang.Override - public io.github.dfa1.vortex.proto.DTypeProtos.Extension getExtension() { - if (dtypeTypeCase_ == 9) { - return (io.github.dfa1.vortex.proto.DTypeProtos.Extension) dtypeType_; - } - return io.github.dfa1.vortex.proto.DTypeProtos.Extension.getDefaultInstance(); - } - /** - * .vortex.dtype.Extension extension = 9; - */ - @java.lang.Override - public io.github.dfa1.vortex.proto.DTypeProtos.ExtensionOrBuilder getExtensionOrBuilder() { - if (dtypeTypeCase_ == 9) { - return (io.github.dfa1.vortex.proto.DTypeProtos.Extension) dtypeType_; - } - return io.github.dfa1.vortex.proto.DTypeProtos.Extension.getDefaultInstance(); - } - - public static final int FIXED_SIZE_LIST_FIELD_NUMBER = 10; - /** - * 
-     * This is after `Extension` for backwards compatibility.
-     *
- * - * .vortex.dtype.FixedSizeList fixed_size_list = 10; - * @return Whether the fixedSizeList field is set. - */ - @java.lang.Override - public boolean hasFixedSizeList() { - return dtypeTypeCase_ == 10; - } - /** - * 
-     * This is after `Extension` for backwards compatibility.
-     *
- * - * .vortex.dtype.FixedSizeList fixed_size_list = 10; - * @return The fixedSizeList. - */ - @java.lang.Override - public io.github.dfa1.vortex.proto.DTypeProtos.FixedSizeList getFixedSizeList() { - if (dtypeTypeCase_ == 10) { - return (io.github.dfa1.vortex.proto.DTypeProtos.FixedSizeList) dtypeType_; - } - return io.github.dfa1.vortex.proto.DTypeProtos.FixedSizeList.getDefaultInstance(); - } - /** - * 
-     * This is after `Extension` for backwards compatibility.
-     *
- * - * .vortex.dtype.FixedSizeList fixed_size_list = 10; - */ - @java.lang.Override - public io.github.dfa1.vortex.proto.DTypeProtos.FixedSizeListOrBuilder getFixedSizeListOrBuilder() { - if (dtypeTypeCase_ == 10) { - return (io.github.dfa1.vortex.proto.DTypeProtos.FixedSizeList) dtypeType_; - } - return io.github.dfa1.vortex.proto.DTypeProtos.FixedSizeList.getDefaultInstance(); - } - - public static final int VARIANT_FIELD_NUMBER = 11; - /** - * .vortex.dtype.Variant variant = 11; - * @return Whether the variant field is set. - */ - @java.lang.Override - public boolean hasVariant() { - return dtypeTypeCase_ == 11; - } - /** - * .vortex.dtype.Variant variant = 11; - * @return The variant. - */ - @java.lang.Override - public io.github.dfa1.vortex.proto.DTypeProtos.Variant getVariant() { - if (dtypeTypeCase_ == 11) { - return (io.github.dfa1.vortex.proto.DTypeProtos.Variant) dtypeType_; - } - return io.github.dfa1.vortex.proto.DTypeProtos.Variant.getDefaultInstance(); - } - /** - * .vortex.dtype.Variant variant = 11; - */ - @java.lang.Override - public io.github.dfa1.vortex.proto.DTypeProtos.VariantOrBuilder getVariantOrBuilder() { - if (dtypeTypeCase_ == 11) { - return (io.github.dfa1.vortex.proto.DTypeProtos.Variant) dtypeType_; - } - return io.github.dfa1.vortex.proto.DTypeProtos.Variant.getDefaultInstance(); - } - - public static final int UNION_FIELD_NUMBER = 12; - /** - * .vortex.dtype.Union union = 12; - * @return Whether the union field is set. - */ - @java.lang.Override - public boolean hasUnion() { - return dtypeTypeCase_ == 12; - } - /** - * .vortex.dtype.Union union = 12; - * @return The union. - */ - @java.lang.Override - public io.github.dfa1.vortex.proto.DTypeProtos.Union getUnion() { - if (dtypeTypeCase_ == 12) { - return (io.github.dfa1.vortex.proto.DTypeProtos.Union) dtypeType_; - } - return io.github.dfa1.vortex.proto.DTypeProtos.Union.getDefaultInstance(); - } - /** - * .vortex.dtype.Union union = 12; - */ - @java.lang.Override - public io.github.dfa1.vortex.proto.DTypeProtos.UnionOrBuilder getUnionOrBuilder() { - if (dtypeTypeCase_ == 12) { - return (io.github.dfa1.vortex.proto.DTypeProtos.Union) dtypeType_; - } - return io.github.dfa1.vortex.proto.DTypeProtos.Union.getDefaultInstance(); - } - - private byte memoizedIsInitialized = -1; - @java.lang.Override - public final boolean isInitialized() { - byte isInitialized = memoizedIsInitialized; - if (isInitialized == 1) return true; - if (isInitialized == 0) return false; - - memoizedIsInitialized = 1; - return true; - } - - @java.lang.Override - public void writeTo(com.google.protobuf.CodedOutputStream output) - throws java.io.IOException { - if (dtypeTypeCase_ == 1) { - output.writeMessage(1, (io.github.dfa1.vortex.proto.DTypeProtos.Null) dtypeType_); - } - if (dtypeTypeCase_ == 2) { - output.writeMessage(2, (io.github.dfa1.vortex.proto.DTypeProtos.Bool) dtypeType_); - } - if (dtypeTypeCase_ == 3) { - output.writeMessage(3, (io.github.dfa1.vortex.proto.DTypeProtos.Primitive) dtypeType_); - } - if (dtypeTypeCase_ == 4) { - output.writeMessage(4, (io.github.dfa1.vortex.proto.DTypeProtos.Decimal) dtypeType_); - } - if (dtypeTypeCase_ == 5) { - output.writeMessage(5, (io.github.dfa1.vortex.proto.DTypeProtos.Utf8) dtypeType_); - } - if (dtypeTypeCase_ == 6) { - output.writeMessage(6, (io.github.dfa1.vortex.proto.DTypeProtos.Binary) dtypeType_); - } - if (dtypeTypeCase_ == 7) { - output.writeMessage(7, (io.github.dfa1.vortex.proto.DTypeProtos.Struct) dtypeType_); - } - if (dtypeTypeCase_ == 8) { - output.writeMessage(8, (io.github.dfa1.vortex.proto.DTypeProtos.List) dtypeType_); - } - if (dtypeTypeCase_ == 9) { - output.writeMessage(9, (io.github.dfa1.vortex.proto.DTypeProtos.Extension) dtypeType_); - } - if (dtypeTypeCase_ == 10) { - output.writeMessage(10, (io.github.dfa1.vortex.proto.DTypeProtos.FixedSizeList) dtypeType_); - } - if (dtypeTypeCase_ == 11) { - output.writeMessage(11, (io.github.dfa1.vortex.proto.DTypeProtos.Variant) dtypeType_); - } - if (dtypeTypeCase_ == 12) { - output.writeMessage(12, (io.github.dfa1.vortex.proto.DTypeProtos.Union) dtypeType_); - } - getUnknownFields().writeTo(output); - } - private int computeSerializedSize_0() { - int size = 0; - if (dtypeTypeCase_ == 1) { - size += com.google.protobuf.CodedOutputStream - .computeMessageSize(1, (io.github.dfa1.vortex.proto.DTypeProtos.Null) dtypeType_); - } - if (dtypeTypeCase_ == 2) { - size += com.google.protobuf.CodedOutputStream - .computeMessageSize(2, (io.github.dfa1.vortex.proto.DTypeProtos.Bool) dtypeType_); - } - if (dtypeTypeCase_ == 3) { - size += com.google.protobuf.CodedOutputStream - .computeMessageSize(3, (io.github.dfa1.vortex.proto.DTypeProtos.Primitive) dtypeType_); - } - if (dtypeTypeCase_ == 4) { - size += com.google.protobuf.CodedOutputStream - .computeMessageSize(4, (io.github.dfa1.vortex.proto.DTypeProtos.Decimal) dtypeType_); - } - if (dtypeTypeCase_ == 5) { - size += com.google.protobuf.CodedOutputStream - .computeMessageSize(5, (io.github.dfa1.vortex.proto.DTypeProtos.Utf8) dtypeType_); - } - if (dtypeTypeCase_ == 6) { - size += com.google.protobuf.CodedOutputStream - .computeMessageSize(6, (io.github.dfa1.vortex.proto.DTypeProtos.Binary) dtypeType_); - } - if (dtypeTypeCase_ == 7) { - size += com.google.protobuf.CodedOutputStream - .computeMessageSize(7, (io.github.dfa1.vortex.proto.DTypeProtos.Struct) dtypeType_); - } - if (dtypeTypeCase_ == 8) { - size += com.google.protobuf.CodedOutputStream - .computeMessageSize(8, (io.github.dfa1.vortex.proto.DTypeProtos.List) dtypeType_); - } - if (dtypeTypeCase_ == 9) { - size += com.google.protobuf.CodedOutputStream - .computeMessageSize(9, (io.github.dfa1.vortex.proto.DTypeProtos.Extension) dtypeType_); - } - if (dtypeTypeCase_ == 10) { - size += com.google.protobuf.CodedOutputStream - .computeMessageSize(10, (io.github.dfa1.vortex.proto.DTypeProtos.FixedSizeList) dtypeType_); - } - if (dtypeTypeCase_ == 11) { - size += com.google.protobuf.CodedOutputStream - .computeMessageSize(11, (io.github.dfa1.vortex.proto.DTypeProtos.Variant) dtypeType_); - } - if (dtypeTypeCase_ == 12) { - size += com.google.protobuf.CodedOutputStream - .computeMessageSize(12, (io.github.dfa1.vortex.proto.DTypeProtos.Union) dtypeType_); - } - return size; - } - @java.lang.Override - public int getSerializedSize() { - int size = memoizedSize; - if (size != -1) return size; - - size = 0; - size += computeSerializedSize_0(); - size += getUnknownFields().getSerializedSize(); - memoizedSize = size; - return size; - } - - @java.lang.Override - public boolean equals(final java.lang.Object obj) { - if (obj == this) { - return true; - } - if (!(obj instanceof io.github.dfa1.vortex.proto.DTypeProtos.DType)) { - return super.equals(obj); - } - io.github.dfa1.vortex.proto.DTypeProtos.DType other = (io.github.dfa1.vortex.proto.DTypeProtos.DType) obj; - - if (!getDtypeTypeCase().equals(other.getDtypeTypeCase())) return false; - switch (dtypeTypeCase_) { - case 1: - if (!getNull() - .equals(other.getNull())) return false; - break; - case 2: - if (!getBool() - .equals(other.getBool())) return false; - break; - case 3: - if (!getPrimitive() - .equals(other.getPrimitive())) return false; - break; - case 4: - if (!getDecimal() - .equals(other.getDecimal())) return false; - break; - case 5: - if (!getUtf8() - .equals(other.getUtf8())) return false; - break; - case 6: - if (!getBinary() - .equals(other.getBinary())) return false; - break; - case 7: - if (!getStruct() - .equals(other.getStruct())) return false; - break; - case 8: - if (!getList() - .equals(other.getList())) return false; - break; - case 9: - if (!getExtension() - .equals(other.getExtension())) return false; - break; - case 10: - if (!getFixedSizeList() - .equals(other.getFixedSizeList())) return false; - break; - case 11: - if (!getVariant() - .equals(other.getVariant())) return false; - break; - case 12: - if (!getUnion() - .equals(other.getUnion())) return false; - break; - case 0: - default: - } - if (!getUnknownFields().equals(other.getUnknownFields())) return false; - return true; - } - - @java.lang.Override - public int hashCode() { - if (memoizedHashCode != 0) { - return memoizedHashCode; - } - int hash = 41; - hash = (19 * hash) + getDescriptor().hashCode(); - switch (dtypeTypeCase_) { - case 1: - hash = (37 * hash) + NULL_FIELD_NUMBER; - hash = (53 * hash) + getNull().hashCode(); - break; - case 2: - hash = (37 * hash) + BOOL_FIELD_NUMBER; - hash = (53 * hash) + getBool().hashCode(); - break; - case 3: - hash = (37 * hash) + PRIMITIVE_FIELD_NUMBER; - hash = (53 * hash) + getPrimitive().hashCode(); - break; - case 4: - hash = (37 * hash) + DECIMAL_FIELD_NUMBER; - hash = (53 * hash) + getDecimal().hashCode(); - break; - case 5: - hash = (37 * hash) + UTF8_FIELD_NUMBER; - hash = (53 * hash) + getUtf8().hashCode(); - break; - case 6: - hash = (37 * hash) + BINARY_FIELD_NUMBER; - hash = (53 * hash) + getBinary().hashCode(); - break; - case 7: - hash = (37 * hash) + STRUCT_FIELD_NUMBER; - hash = (53 * hash) + getStruct().hashCode(); - break; - case 8: - hash = (37 * hash) + LIST_FIELD_NUMBER; - hash = (53 * hash) + getList().hashCode(); - break; - case 9: - hash = (37 * hash) + EXTENSION_FIELD_NUMBER; - hash = (53 * hash) + getExtension().hashCode(); - break; - case 10: - hash = (37 * hash) + FIXED_SIZE_LIST_FIELD_NUMBER; - hash = (53 * hash) + getFixedSizeList().hashCode(); - break; - case 11: - hash = (37 * hash) + VARIANT_FIELD_NUMBER; - hash = (53 * hash) + getVariant().hashCode(); - break; - case 12: - hash = (37 * hash) + UNION_FIELD_NUMBER; - hash = (53 * hash) + getUnion().hashCode(); - break; - case 0: - default: - } - hash = (29 * hash) + getUnknownFields().hashCode(); - memoizedHashCode = hash; - return hash; - } - - public static io.github.dfa1.vortex.proto.DTypeProtos.DType parseFrom( - java.nio.ByteBuffer data) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data); - } - public static io.github.dfa1.vortex.proto.DTypeProtos.DType parseFrom( - java.nio.ByteBuffer data, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data, extensionRegistry); - } - public static io.github.dfa1.vortex.proto.DTypeProtos.DType parseFrom( - com.google.protobuf.ByteString data) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data); - } - public static io.github.dfa1.vortex.proto.DTypeProtos.DType parseFrom( - com.google.protobuf.ByteString data, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data, extensionRegistry); - } - public static io.github.dfa1.vortex.proto.DTypeProtos.DType parseFrom(byte[] data) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data); - } - public static io.github.dfa1.vortex.proto.DTypeProtos.DType parseFrom( - byte[] data, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data, extensionRegistry); - } - public static io.github.dfa1.vortex.proto.DTypeProtos.DType parseFrom(java.io.InputStream input) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseWithIOException(PARSER, input); - } - public static io.github.dfa1.vortex.proto.DTypeProtos.DType parseFrom( - java.io.InputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseWithIOException(PARSER, input, extensionRegistry); - } - - public static io.github.dfa1.vortex.proto.DTypeProtos.DType parseDelimitedFrom(java.io.InputStream input) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseDelimitedWithIOException(PARSER, input); - } - - public static io.github.dfa1.vortex.proto.DTypeProtos.DType parseDelimitedFrom( - java.io.InputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseDelimitedWithIOException(PARSER, input, extensionRegistry); - } - public static io.github.dfa1.vortex.proto.DTypeProtos.DType parseFrom( - com.google.protobuf.CodedInputStream input) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseWithIOException(PARSER, input); - } - public static io.github.dfa1.vortex.proto.DTypeProtos.DType parseFrom( - com.google.protobuf.CodedInputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseWithIOException(PARSER, input, extensionRegistry); - } - - @java.lang.Override - public Builder newBuilderForType() { return newBuilder(); } - public static Builder newBuilder() { - return DEFAULT_INSTANCE.toBuilder(); - } - public static Builder newBuilder(io.github.dfa1.vortex.proto.DTypeProtos.DType prototype) { - return DEFAULT_INSTANCE.toBuilder().mergeFrom(prototype); - } - @java.lang.Override - public Builder toBuilder() { - return this == DEFAULT_INSTANCE - ? new Builder() : new Builder().mergeFrom(this); - } - - @java.lang.Override - protected Builder newBuilderForType( - com.google.protobuf.GeneratedMessage.BuilderParent parent) { - Builder builder = new Builder(parent); - return builder; - } - /** - * Protobuf type {@code vortex.dtype.DType} - */ - public static final class Builder extends - com.google.protobuf.GeneratedMessage.Builder implements - // @@protoc_insertion_point(builder_implements:vortex.dtype.DType) - io.github.dfa1.vortex.proto.DTypeProtos.DTypeOrBuilder { - public static final com.google.protobuf.Descriptors.Descriptor - getDescriptor() { - return io.github.dfa1.vortex.proto.DTypeProtos.internal_static_vortex_dtype_DType_descriptor; - } - - @java.lang.Override - protected com.google.protobuf.GeneratedMessage.FieldAccessorTable - internalGetFieldAccessorTable() { - return io.github.dfa1.vortex.proto.DTypeProtos.internal_static_vortex_dtype_DType_fieldAccessorTable - .ensureFieldAccessorsInitialized( - io.github.dfa1.vortex.proto.DTypeProtos.DType.class, io.github.dfa1.vortex.proto.DTypeProtos.DType.Builder.class); - } - - // Construct using io.github.dfa1.vortex.proto.DTypeProtos.DType.newBuilder() - private Builder() { - - } - - private Builder( - com.google.protobuf.GeneratedMessage.BuilderParent parent) { - super(parent); - - } - @java.lang.Override - public Builder clear() { - super.clear(); - bitField0_ = 0; - if (nullBuilder_ != null) { - nullBuilder_.clear(); - } - if (boolBuilder_ != null) { - boolBuilder_.clear(); - } - if (primitiveBuilder_ != null) { - primitiveBuilder_.clear(); - } - if (decimalBuilder_ != null) { - decimalBuilder_.clear(); - } - if (utf8Builder_ != null) { - utf8Builder_.clear(); - } - if (binaryBuilder_ != null) { - binaryBuilder_.clear(); - } - if (structBuilder_ != null) { - structBuilder_.clear(); - } - if (listBuilder_ != null) { - listBuilder_.clear(); - } - if (extensionBuilder_ != null) { - extensionBuilder_.clear(); - } - if (fixedSizeListBuilder_ != null) { - fixedSizeListBuilder_.clear(); - } - if (variantBuilder_ != null) { - variantBuilder_.clear(); - } - if (unionBuilder_ != null) { - unionBuilder_.clear(); - } - dtypeTypeCase_ = 0; - dtypeType_ = null; - return this; - } - - @java.lang.Override - public com.google.protobuf.Descriptors.Descriptor - getDescriptorForType() { - return io.github.dfa1.vortex.proto.DTypeProtos.internal_static_vortex_dtype_DType_descriptor; - } - - @java.lang.Override - public io.github.dfa1.vortex.proto.DTypeProtos.DType getDefaultInstanceForType() { - return io.github.dfa1.vortex.proto.DTypeProtos.DType.getDefaultInstance(); - } - - @java.lang.Override - public io.github.dfa1.vortex.proto.DTypeProtos.DType build() { - io.github.dfa1.vortex.proto.DTypeProtos.DType result = buildPartial(); - if (!result.isInitialized()) { - throw newUninitializedMessageException(result); - } - return result; - } - - @java.lang.Override - public io.github.dfa1.vortex.proto.DTypeProtos.DType buildPartial() { - io.github.dfa1.vortex.proto.DTypeProtos.DType result = new io.github.dfa1.vortex.proto.DTypeProtos.DType(this); - if (bitField0_ != 0) { buildPartial0(result); } - buildPartialOneofs(result); - onBuilt(); - return result; - } - - private void buildPartial0(io.github.dfa1.vortex.proto.DTypeProtos.DType result) { - int from_bitField0_ = bitField0_; - } - - private void buildPartialOneofs(io.github.dfa1.vortex.proto.DTypeProtos.DType result) { - result.dtypeTypeCase_ = dtypeTypeCase_; - result.dtypeType_ = this.dtypeType_; - if (dtypeTypeCase_ == 1 && - nullBuilder_ != null) { - result.dtypeType_ = nullBuilder_.build(); - } - if (dtypeTypeCase_ == 2 && - boolBuilder_ != null) { - result.dtypeType_ = boolBuilder_.build(); - } - if (dtypeTypeCase_ == 3 && - primitiveBuilder_ != null) { - result.dtypeType_ = primitiveBuilder_.build(); - } - if (dtypeTypeCase_ == 4 && - decimalBuilder_ != null) { - result.dtypeType_ = decimalBuilder_.build(); - } - if (dtypeTypeCase_ == 5 && - utf8Builder_ != null) { - result.dtypeType_ = utf8Builder_.build(); - } - if (dtypeTypeCase_ == 6 && - binaryBuilder_ != null) { - result.dtypeType_ = binaryBuilder_.build(); - } - if (dtypeTypeCase_ == 7 && - structBuilder_ != null) { - result.dtypeType_ = structBuilder_.build(); - } - if (dtypeTypeCase_ == 8 && - listBuilder_ != null) { - result.dtypeType_ = listBuilder_.build(); - } - if (dtypeTypeCase_ == 9 && - extensionBuilder_ != null) { - result.dtypeType_ = extensionBuilder_.build(); - } - if (dtypeTypeCase_ == 10 && - fixedSizeListBuilder_ != null) { - result.dtypeType_ = fixedSizeListBuilder_.build(); - } - if (dtypeTypeCase_ == 11 && - variantBuilder_ != null) { - result.dtypeType_ = variantBuilder_.build(); - } - if (dtypeTypeCase_ == 12 && - unionBuilder_ != null) { - result.dtypeType_ = unionBuilder_.build(); - } - } - - @java.lang.Override - public Builder mergeFrom(com.google.protobuf.Message other) { - if (other instanceof io.github.dfa1.vortex.proto.DTypeProtos.DType) { - return mergeFrom((io.github.dfa1.vortex.proto.DTypeProtos.DType)other); - } else { - super.mergeFrom(other); - return this; - } - } - - public Builder mergeFrom(io.github.dfa1.vortex.proto.DTypeProtos.DType other) { - if (other == io.github.dfa1.vortex.proto.DTypeProtos.DType.getDefaultInstance()) return this; - switch (other.getDtypeTypeCase()) { - case NULL: { - mergeNull(other.getNull()); - break; - } - case BOOL: { - mergeBool(other.getBool()); - break; - } - case PRIMITIVE: { - mergePrimitive(other.getPrimitive()); - break; - } - case DECIMAL: { - mergeDecimal(other.getDecimal()); - break; - } - case UTF8: { - mergeUtf8(other.getUtf8()); - break; - } - case BINARY: { - mergeBinary(other.getBinary()); - break; - } - case STRUCT: { - mergeStruct(other.getStruct()); - break; - } - case LIST: { - mergeList(other.getList()); - break; - } - case EXTENSION: { - mergeExtension(other.getExtension()); - break; - } - case FIXED_SIZE_LIST: { - mergeFixedSizeList(other.getFixedSizeList()); - break; - } - case VARIANT: { - mergeVariant(other.getVariant()); - break; - } - case UNION: { - mergeUnion(other.getUnion()); - break; - } - case DTYPETYPE_NOT_SET: { - break; - } - } - this.mergeUnknownFields(other.getUnknownFields()); - onChanged(); - return this; - } - - @java.lang.Override - public final boolean isInitialized() { - return true; - } - - @java.lang.Override - public Builder mergeFrom( - com.google.protobuf.CodedInputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws java.io.IOException { - if (extensionRegistry == null) { - throw new java.lang.NullPointerException(); - } - try { - boolean done = false; - while (!done) { - int tag = input.readTag(); - switch (tag) { - case 0: - done = true; - break; - case 10: { - input.readMessage( - internalGetNullFieldBuilder().getBuilder(), - extensionRegistry); - dtypeTypeCase_ = 1; - break; - } // case 10 - case 18: { - input.readMessage( - internalGetBoolFieldBuilder().getBuilder(), - extensionRegistry); - dtypeTypeCase_ = 2; - break; - } // case 18 - case 26: { - input.readMessage( - internalGetPrimitiveFieldBuilder().getBuilder(), - extensionRegistry); - dtypeTypeCase_ = 3; - break; - } // case 26 - case 34: { - input.readMessage( - internalGetDecimalFieldBuilder().getBuilder(), - extensionRegistry); - dtypeTypeCase_ = 4; - break; - } // case 34 - case 42: { - input.readMessage( - internalGetUtf8FieldBuilder().getBuilder(), - extensionRegistry); - dtypeTypeCase_ = 5; - break; - } // case 42 - case 50: { - input.readMessage( - internalGetBinaryFieldBuilder().getBuilder(), - extensionRegistry); - dtypeTypeCase_ = 6; - break; - } // case 50 - case 58: { - input.readMessage( - internalGetStructFieldBuilder().getBuilder(), - extensionRegistry); - dtypeTypeCase_ = 7; - break; - } // case 58 - case 66: { - input.readMessage( - internalGetListFieldBuilder().getBuilder(), - extensionRegistry); - dtypeTypeCase_ = 8; - break; - } // case 66 - case 74: { - input.readMessage( - internalGetExtensionFieldBuilder().getBuilder(), - extensionRegistry); - dtypeTypeCase_ = 9; - break; - } // case 74 - case 82: { - input.readMessage( - internalGetFixedSizeListFieldBuilder().getBuilder(), - extensionRegistry); - dtypeTypeCase_ = 10; - break; - } // case 82 - case 90: { - input.readMessage( - internalGetVariantFieldBuilder().getBuilder(), - extensionRegistry); - dtypeTypeCase_ = 11; - break; - } // case 90 - case 98: { - input.readMessage( - internalGetUnionFieldBuilder().getBuilder(), - extensionRegistry); - dtypeTypeCase_ = 12; - break; - } // case 98 - default: { - if (!super.parseUnknownField(input, extensionRegistry, tag)) { - done = true; // was an endgroup tag - } - break; - } // default: - } // switch (tag) - } // while (!done) - } catch (com.google.protobuf.InvalidProtocolBufferException e) { - throw e.unwrapIOException(); - } finally { - onChanged(); - } // finally - return this; - } - private int dtypeTypeCase_ = 0; - private java.lang.Object dtypeType_; - public DtypeTypeCase - getDtypeTypeCase() { - return DtypeTypeCase.forNumber( - dtypeTypeCase_); - } - - public Builder clearDtypeType() { - dtypeTypeCase_ = 0; - dtypeType_ = null; - onChanged(); - return this; - } - - private int bitField0_; - - private com.google.protobuf.SingleFieldBuilder< - io.github.dfa1.vortex.proto.DTypeProtos.Null, io.github.dfa1.vortex.proto.DTypeProtos.Null.Builder, io.github.dfa1.vortex.proto.DTypeProtos.NullOrBuilder> nullBuilder_; - /** - * .vortex.dtype.Null null = 1; - * @return Whether the null field is set. - */ - @java.lang.Override - public boolean hasNull() { - return dtypeTypeCase_ == 1; - } - /** - * .vortex.dtype.Null null = 1; - * @return The null. - */ - @java.lang.Override - public io.github.dfa1.vortex.proto.DTypeProtos.Null getNull() { - if (nullBuilder_ == null) { - if (dtypeTypeCase_ == 1) { - return (io.github.dfa1.vortex.proto.DTypeProtos.Null) dtypeType_; - } - return io.github.dfa1.vortex.proto.DTypeProtos.Null.getDefaultInstance(); - } else { - if (dtypeTypeCase_ == 1) { - return nullBuilder_.getMessage(); - } - return io.github.dfa1.vortex.proto.DTypeProtos.Null.getDefaultInstance(); - } - } - /** - * .vortex.dtype.Null null = 1; - */ - public Builder setNull(io.github.dfa1.vortex.proto.DTypeProtos.Null value) { - if (nullBuilder_ == null) { - if (value == null) { - throw new NullPointerException(); - } - dtypeType_ = value; - onChanged(); - } else { - nullBuilder_.setMessage(value); - } - dtypeTypeCase_ = 1; - return this; - } - /** - * .vortex.dtype.Null null = 1; - */ - public Builder setNull( - io.github.dfa1.vortex.proto.DTypeProtos.Null.Builder builderForValue) { - if (nullBuilder_ == null) { - dtypeType_ = builderForValue.build(); - onChanged(); - } else { - nullBuilder_.setMessage(builderForValue.build()); - } - dtypeTypeCase_ = 1; - return this; - } - /** - * .vortex.dtype.Null null = 1; - */ - public Builder mergeNull(io.github.dfa1.vortex.proto.DTypeProtos.Null value) { - if (nullBuilder_ == null) { - if (dtypeTypeCase_ == 1 && - dtypeType_ != io.github.dfa1.vortex.proto.DTypeProtos.Null.getDefaultInstance()) { - dtypeType_ = io.github.dfa1.vortex.proto.DTypeProtos.Null.newBuilder((io.github.dfa1.vortex.proto.DTypeProtos.Null) dtypeType_) - .mergeFrom(value).buildPartial(); - } else { - dtypeType_ = value; - } - onChanged(); - } else { - if (dtypeTypeCase_ == 1) { - nullBuilder_.mergeFrom(value); - } else { - nullBuilder_.setMessage(value); - } - } - dtypeTypeCase_ = 1; - return this; - } - /** - * .vortex.dtype.Null null = 1; - */ - public Builder clearNull() { - if (nullBuilder_ == null) { - if (dtypeTypeCase_ == 1) { - dtypeTypeCase_ = 0; - dtypeType_ = null; - onChanged(); - } - } else { - if (dtypeTypeCase_ == 1) { - dtypeTypeCase_ = 0; - dtypeType_ = null; - } - nullBuilder_.clear(); - } - return this; - } - /** - * .vortex.dtype.Null null = 1; - */ - public io.github.dfa1.vortex.proto.DTypeProtos.Null.Builder getNullBuilder() { - return internalGetNullFieldBuilder().getBuilder(); - } - /** - * .vortex.dtype.Null null = 1; - */ - @java.lang.Override - public io.github.dfa1.vortex.proto.DTypeProtos.NullOrBuilder getNullOrBuilder() { - if ((dtypeTypeCase_ == 1) && (nullBuilder_ != null)) { - return nullBuilder_.getMessageOrBuilder(); - } else { - if (dtypeTypeCase_ == 1) { - return (io.github.dfa1.vortex.proto.DTypeProtos.Null) dtypeType_; - } - return io.github.dfa1.vortex.proto.DTypeProtos.Null.getDefaultInstance(); - } - } - /** - * .vortex.dtype.Null null = 1; - */ - private com.google.protobuf.SingleFieldBuilder< - io.github.dfa1.vortex.proto.DTypeProtos.Null, io.github.dfa1.vortex.proto.DTypeProtos.Null.Builder, io.github.dfa1.vortex.proto.DTypeProtos.NullOrBuilder> - internalGetNullFieldBuilder() { - if (nullBuilder_ == null) { - if (!(dtypeTypeCase_ == 1)) { - dtypeType_ = io.github.dfa1.vortex.proto.DTypeProtos.Null.getDefaultInstance(); - } - nullBuilder_ = new com.google.protobuf.SingleFieldBuilder< - io.github.dfa1.vortex.proto.DTypeProtos.Null, io.github.dfa1.vortex.proto.DTypeProtos.Null.Builder, io.github.dfa1.vortex.proto.DTypeProtos.NullOrBuilder>( - (io.github.dfa1.vortex.proto.DTypeProtos.Null) dtypeType_, - getParentForChildren(), - isClean()); - dtypeType_ = null; - } - dtypeTypeCase_ = 1; - onChanged(); - return nullBuilder_; - } - - private com.google.protobuf.SingleFieldBuilder< - io.github.dfa1.vortex.proto.DTypeProtos.Bool, io.github.dfa1.vortex.proto.DTypeProtos.Bool.Builder, io.github.dfa1.vortex.proto.DTypeProtos.BoolOrBuilder> boolBuilder_; - /** - * .vortex.dtype.Bool bool = 2; - * @return Whether the bool field is set. - */ - @java.lang.Override - public boolean hasBool() { - return dtypeTypeCase_ == 2; - } - /** - * .vortex.dtype.Bool bool = 2; - * @return The bool. - */ - @java.lang.Override - public io.github.dfa1.vortex.proto.DTypeProtos.Bool getBool() { - if (boolBuilder_ == null) { - if (dtypeTypeCase_ == 2) { - return (io.github.dfa1.vortex.proto.DTypeProtos.Bool) dtypeType_; - } - return io.github.dfa1.vortex.proto.DTypeProtos.Bool.getDefaultInstance(); - } else { - if (dtypeTypeCase_ == 2) { - return boolBuilder_.getMessage(); - } - return io.github.dfa1.vortex.proto.DTypeProtos.Bool.getDefaultInstance(); - } - } - /** - * .vortex.dtype.Bool bool = 2; - */ - public Builder setBool(io.github.dfa1.vortex.proto.DTypeProtos.Bool value) { - if (boolBuilder_ == null) { - if (value == null) { - throw new NullPointerException(); - } - dtypeType_ = value; - onChanged(); - } else { - boolBuilder_.setMessage(value); - } - dtypeTypeCase_ = 2; - return this; - } - /** - * .vortex.dtype.Bool bool = 2; - */ - public Builder setBool( - io.github.dfa1.vortex.proto.DTypeProtos.Bool.Builder builderForValue) { - if (boolBuilder_ == null) { - dtypeType_ = builderForValue.build(); - onChanged(); - } else { - boolBuilder_.setMessage(builderForValue.build()); - } - dtypeTypeCase_ = 2; - return this; - } - /** - * .vortex.dtype.Bool bool = 2; - */ - public Builder mergeBool(io.github.dfa1.vortex.proto.DTypeProtos.Bool value) { - if (boolBuilder_ == null) { - if (dtypeTypeCase_ == 2 && - dtypeType_ != io.github.dfa1.vortex.proto.DTypeProtos.Bool.getDefaultInstance()) { - dtypeType_ = io.github.dfa1.vortex.proto.DTypeProtos.Bool.newBuilder((io.github.dfa1.vortex.proto.DTypeProtos.Bool) dtypeType_) - .mergeFrom(value).buildPartial(); - } else { - dtypeType_ = value; - } - onChanged(); - } else { - if (dtypeTypeCase_ == 2) { - boolBuilder_.mergeFrom(value); - } else { - boolBuilder_.setMessage(value); - } - } - dtypeTypeCase_ = 2; - return this; - } - /** - * .vortex.dtype.Bool bool = 2; - */ - public Builder clearBool() { - if (boolBuilder_ == null) { - if (dtypeTypeCase_ == 2) { - dtypeTypeCase_ = 0; - dtypeType_ = null; - onChanged(); - } - } else { - if (dtypeTypeCase_ == 2) { - dtypeTypeCase_ = 0; - dtypeType_ = null; - } - boolBuilder_.clear(); - } - return this; - } - /** - * .vortex.dtype.Bool bool = 2; - */ - public io.github.dfa1.vortex.proto.DTypeProtos.Bool.Builder getBoolBuilder() { - return internalGetBoolFieldBuilder().getBuilder(); - } - /** - * .vortex.dtype.Bool bool = 2; - */ - @java.lang.Override - public io.github.dfa1.vortex.proto.DTypeProtos.BoolOrBuilder getBoolOrBuilder() { - if ((dtypeTypeCase_ == 2) && (boolBuilder_ != null)) { - return boolBuilder_.getMessageOrBuilder(); - } else { - if (dtypeTypeCase_ == 2) { - return (io.github.dfa1.vortex.proto.DTypeProtos.Bool) dtypeType_; - } - return io.github.dfa1.vortex.proto.DTypeProtos.Bool.getDefaultInstance(); - } - } - /** - * .vortex.dtype.Bool bool = 2; - */ - private com.google.protobuf.SingleFieldBuilder< - io.github.dfa1.vortex.proto.DTypeProtos.Bool, io.github.dfa1.vortex.proto.DTypeProtos.Bool.Builder, io.github.dfa1.vortex.proto.DTypeProtos.BoolOrBuilder> - internalGetBoolFieldBuilder() { - if (boolBuilder_ == null) { - if (!(dtypeTypeCase_ == 2)) { - dtypeType_ = io.github.dfa1.vortex.proto.DTypeProtos.Bool.getDefaultInstance(); - } - boolBuilder_ = new com.google.protobuf.SingleFieldBuilder< - io.github.dfa1.vortex.proto.DTypeProtos.Bool, io.github.dfa1.vortex.proto.DTypeProtos.Bool.Builder, io.github.dfa1.vortex.proto.DTypeProtos.BoolOrBuilder>( - (io.github.dfa1.vortex.proto.DTypeProtos.Bool) dtypeType_, - getParentForChildren(), - isClean()); - dtypeType_ = null; - } - dtypeTypeCase_ = 2; - onChanged(); - return boolBuilder_; - } - - private com.google.protobuf.SingleFieldBuilder< - io.github.dfa1.vortex.proto.DTypeProtos.Primitive, io.github.dfa1.vortex.proto.DTypeProtos.Primitive.Builder, io.github.dfa1.vortex.proto.DTypeProtos.PrimitiveOrBuilder> primitiveBuilder_; - /** - * .vortex.dtype.Primitive primitive = 3; - * @return Whether the primitive field is set. - */ - @java.lang.Override - public boolean hasPrimitive() { - return dtypeTypeCase_ == 3; - } - /** - * .vortex.dtype.Primitive primitive = 3; - * @return The primitive. - */ - @java.lang.Override - public io.github.dfa1.vortex.proto.DTypeProtos.Primitive getPrimitive() { - if (primitiveBuilder_ == null) { - if (dtypeTypeCase_ == 3) { - return (io.github.dfa1.vortex.proto.DTypeProtos.Primitive) dtypeType_; - } - return io.github.dfa1.vortex.proto.DTypeProtos.Primitive.getDefaultInstance(); - } else { - if (dtypeTypeCase_ == 3) { - return primitiveBuilder_.getMessage(); - } - return io.github.dfa1.vortex.proto.DTypeProtos.Primitive.getDefaultInstance(); - } - } - /** - * .vortex.dtype.Primitive primitive = 3; - */ - public Builder setPrimitive(io.github.dfa1.vortex.proto.DTypeProtos.Primitive value) { - if (primitiveBuilder_ == null) { - if (value == null) { - throw new NullPointerException(); - } - dtypeType_ = value; - onChanged(); - } else { - primitiveBuilder_.setMessage(value); - } - dtypeTypeCase_ = 3; - return this; - } - /** - * .vortex.dtype.Primitive primitive = 3; - */ - public Builder setPrimitive( - io.github.dfa1.vortex.proto.DTypeProtos.Primitive.Builder builderForValue) { - if (primitiveBuilder_ == null) { - dtypeType_ = builderForValue.build(); - onChanged(); - } else { - primitiveBuilder_.setMessage(builderForValue.build()); - } - dtypeTypeCase_ = 3; - return this; - } - /** - * .vortex.dtype.Primitive primitive = 3; - */ - public Builder mergePrimitive(io.github.dfa1.vortex.proto.DTypeProtos.Primitive value) { - if (primitiveBuilder_ == null) { - if (dtypeTypeCase_ == 3 && - dtypeType_ != io.github.dfa1.vortex.proto.DTypeProtos.Primitive.getDefaultInstance()) { - dtypeType_ = io.github.dfa1.vortex.proto.DTypeProtos.Primitive.newBuilder((io.github.dfa1.vortex.proto.DTypeProtos.Primitive) dtypeType_) - .mergeFrom(value).buildPartial(); - } else { - dtypeType_ = value; - } - onChanged(); - } else { - if (dtypeTypeCase_ == 3) { - primitiveBuilder_.mergeFrom(value); - } else { - primitiveBuilder_.setMessage(value); - } - } - dtypeTypeCase_ = 3; - return this; - } - /** - * .vortex.dtype.Primitive primitive = 3; - */ - public Builder clearPrimitive() { - if (primitiveBuilder_ == null) { - if (dtypeTypeCase_ == 3) { - dtypeTypeCase_ = 0; - dtypeType_ = null; - onChanged(); - } - } else { - if (dtypeTypeCase_ == 3) { - dtypeTypeCase_ = 0; - dtypeType_ = null; - } - primitiveBuilder_.clear(); - } - return this; - } - /** - * .vortex.dtype.Primitive primitive = 3; - */ - public io.github.dfa1.vortex.proto.DTypeProtos.Primitive.Builder getPrimitiveBuilder() { - return internalGetPrimitiveFieldBuilder().getBuilder(); - } - /** - * .vortex.dtype.Primitive primitive = 3; - */ - @java.lang.Override - public io.github.dfa1.vortex.proto.DTypeProtos.PrimitiveOrBuilder getPrimitiveOrBuilder() { - if ((dtypeTypeCase_ == 3) && (primitiveBuilder_ != null)) { - return primitiveBuilder_.getMessageOrBuilder(); - } else { - if (dtypeTypeCase_ == 3) { - return (io.github.dfa1.vortex.proto.DTypeProtos.Primitive) dtypeType_; - } - return io.github.dfa1.vortex.proto.DTypeProtos.Primitive.getDefaultInstance(); - } - } - /** - * .vortex.dtype.Primitive primitive = 3; - */ - private com.google.protobuf.SingleFieldBuilder< - io.github.dfa1.vortex.proto.DTypeProtos.Primitive, io.github.dfa1.vortex.proto.DTypeProtos.Primitive.Builder, io.github.dfa1.vortex.proto.DTypeProtos.PrimitiveOrBuilder> - internalGetPrimitiveFieldBuilder() { - if (primitiveBuilder_ == null) { - if (!(dtypeTypeCase_ == 3)) { - dtypeType_ = io.github.dfa1.vortex.proto.DTypeProtos.Primitive.getDefaultInstance(); - } - primitiveBuilder_ = new com.google.protobuf.SingleFieldBuilder< - io.github.dfa1.vortex.proto.DTypeProtos.Primitive, io.github.dfa1.vortex.proto.DTypeProtos.Primitive.Builder, io.github.dfa1.vortex.proto.DTypeProtos.PrimitiveOrBuilder>( - (io.github.dfa1.vortex.proto.DTypeProtos.Primitive) dtypeType_, - getParentForChildren(), - isClean()); - dtypeType_ = null; - } - dtypeTypeCase_ = 3; - onChanged(); - return primitiveBuilder_; - } - - private com.google.protobuf.SingleFieldBuilder< - io.github.dfa1.vortex.proto.DTypeProtos.Decimal, io.github.dfa1.vortex.proto.DTypeProtos.Decimal.Builder, io.github.dfa1.vortex.proto.DTypeProtos.DecimalOrBuilder> decimalBuilder_; - /** - * .vortex.dtype.Decimal decimal = 4; - * @return Whether the decimal field is set. - */ - @java.lang.Override - public boolean hasDecimal() { - return dtypeTypeCase_ == 4; - } - /** - * .vortex.dtype.Decimal decimal = 4; - * @return The decimal. - */ - @java.lang.Override - public io.github.dfa1.vortex.proto.DTypeProtos.Decimal getDecimal() { - if (decimalBuilder_ == null) { - if (dtypeTypeCase_ == 4) { - return (io.github.dfa1.vortex.proto.DTypeProtos.Decimal) dtypeType_; - } - return io.github.dfa1.vortex.proto.DTypeProtos.Decimal.getDefaultInstance(); - } else { - if (dtypeTypeCase_ == 4) { - return decimalBuilder_.getMessage(); - } - return io.github.dfa1.vortex.proto.DTypeProtos.Decimal.getDefaultInstance(); - } - } - /** - * .vortex.dtype.Decimal decimal = 4; - */ - public Builder setDecimal(io.github.dfa1.vortex.proto.DTypeProtos.Decimal value) { - if (decimalBuilder_ == null) { - if (value == null) { - throw new NullPointerException(); - } - dtypeType_ = value; - onChanged(); - } else { - decimalBuilder_.setMessage(value); - } - dtypeTypeCase_ = 4; - return this; - } - /** - * .vortex.dtype.Decimal decimal = 4; - */ - public Builder setDecimal( - io.github.dfa1.vortex.proto.DTypeProtos.Decimal.Builder builderForValue) { - if (decimalBuilder_ == null) { - dtypeType_ = builderForValue.build(); - onChanged(); - } else { - decimalBuilder_.setMessage(builderForValue.build()); - } - dtypeTypeCase_ = 4; - return this; - } - /** - * .vortex.dtype.Decimal decimal = 4; - */ - public Builder mergeDecimal(io.github.dfa1.vortex.proto.DTypeProtos.Decimal value) { - if (decimalBuilder_ == null) { - if (dtypeTypeCase_ == 4 && - dtypeType_ != io.github.dfa1.vortex.proto.DTypeProtos.Decimal.getDefaultInstance()) { - dtypeType_ = io.github.dfa1.vortex.proto.DTypeProtos.Decimal.newBuilder((io.github.dfa1.vortex.proto.DTypeProtos.Decimal) dtypeType_) - .mergeFrom(value).buildPartial(); - } else { - dtypeType_ = value; - } - onChanged(); - } else { - if (dtypeTypeCase_ == 4) { - decimalBuilder_.mergeFrom(value); - } else { - decimalBuilder_.setMessage(value); - } - } - dtypeTypeCase_ = 4; - return this; - } - /** - * .vortex.dtype.Decimal decimal = 4; - */ - public Builder clearDecimal() { - if (decimalBuilder_ == null) { - if (dtypeTypeCase_ == 4) { - dtypeTypeCase_ = 0; - dtypeType_ = null; - onChanged(); - } - } else { - if (dtypeTypeCase_ == 4) { - dtypeTypeCase_ = 0; - dtypeType_ = null; - } - decimalBuilder_.clear(); - } - return this; - } - /** - * .vortex.dtype.Decimal decimal = 4; - */ - public io.github.dfa1.vortex.proto.DTypeProtos.Decimal.Builder getDecimalBuilder() { - return internalGetDecimalFieldBuilder().getBuilder(); - } - /** - * .vortex.dtype.Decimal decimal = 4; - */ - @java.lang.Override - public io.github.dfa1.vortex.proto.DTypeProtos.DecimalOrBuilder getDecimalOrBuilder() { - if ((dtypeTypeCase_ == 4) && (decimalBuilder_ != null)) { - return decimalBuilder_.getMessageOrBuilder(); - } else { - if (dtypeTypeCase_ == 4) { - return (io.github.dfa1.vortex.proto.DTypeProtos.Decimal) dtypeType_; - } - return io.github.dfa1.vortex.proto.DTypeProtos.Decimal.getDefaultInstance(); - } - } - /** - * .vortex.dtype.Decimal decimal = 4; - */ - private com.google.protobuf.SingleFieldBuilder< - io.github.dfa1.vortex.proto.DTypeProtos.Decimal, io.github.dfa1.vortex.proto.DTypeProtos.Decimal.Builder, io.github.dfa1.vortex.proto.DTypeProtos.DecimalOrBuilder> - internalGetDecimalFieldBuilder() { - if (decimalBuilder_ == null) { - if (!(dtypeTypeCase_ == 4)) { - dtypeType_ = io.github.dfa1.vortex.proto.DTypeProtos.Decimal.getDefaultInstance(); - } - decimalBuilder_ = new com.google.protobuf.SingleFieldBuilder< - io.github.dfa1.vortex.proto.DTypeProtos.Decimal, io.github.dfa1.vortex.proto.DTypeProtos.Decimal.Builder, io.github.dfa1.vortex.proto.DTypeProtos.DecimalOrBuilder>( - (io.github.dfa1.vortex.proto.DTypeProtos.Decimal) dtypeType_, - getParentForChildren(), - isClean()); - dtypeType_ = null; - } - dtypeTypeCase_ = 4; - onChanged(); - return decimalBuilder_; - } - - private com.google.protobuf.SingleFieldBuilder< - io.github.dfa1.vortex.proto.DTypeProtos.Utf8, io.github.dfa1.vortex.proto.DTypeProtos.Utf8.Builder, io.github.dfa1.vortex.proto.DTypeProtos.Utf8OrBuilder> utf8Builder_; - /** - * .vortex.dtype.Utf8 utf8 = 5; - * @return Whether the utf8 field is set. - */ - @java.lang.Override - public boolean hasUtf8() { - return dtypeTypeCase_ == 5; - } - /** - * .vortex.dtype.Utf8 utf8 = 5; - * @return The utf8. - */ - @java.lang.Override - public io.github.dfa1.vortex.proto.DTypeProtos.Utf8 getUtf8() { - if (utf8Builder_ == null) { - if (dtypeTypeCase_ == 5) { - return (io.github.dfa1.vortex.proto.DTypeProtos.Utf8) dtypeType_; - } - return io.github.dfa1.vortex.proto.DTypeProtos.Utf8.getDefaultInstance(); - } else { - if (dtypeTypeCase_ == 5) { - return utf8Builder_.getMessage(); - } - return io.github.dfa1.vortex.proto.DTypeProtos.Utf8.getDefaultInstance(); - } - } - /** - * .vortex.dtype.Utf8 utf8 = 5; - */ - public Builder setUtf8(io.github.dfa1.vortex.proto.DTypeProtos.Utf8 value) { - if (utf8Builder_ == null) { - if (value == null) { - throw new NullPointerException(); - } - dtypeType_ = value; - onChanged(); - } else { - utf8Builder_.setMessage(value); - } - dtypeTypeCase_ = 5; - return this; - } - /** - * .vortex.dtype.Utf8 utf8 = 5; - */ - public Builder setUtf8( - io.github.dfa1.vortex.proto.DTypeProtos.Utf8.Builder builderForValue) { - if (utf8Builder_ == null) { - dtypeType_ = builderForValue.build(); - onChanged(); - } else { - utf8Builder_.setMessage(builderForValue.build()); - } - dtypeTypeCase_ = 5; - return this; - } - /** - * .vortex.dtype.Utf8 utf8 = 5; - */ - public Builder mergeUtf8(io.github.dfa1.vortex.proto.DTypeProtos.Utf8 value) { - if (utf8Builder_ == null) { - if (dtypeTypeCase_ == 5 && - dtypeType_ != io.github.dfa1.vortex.proto.DTypeProtos.Utf8.getDefaultInstance()) { - dtypeType_ = io.github.dfa1.vortex.proto.DTypeProtos.Utf8.newBuilder((io.github.dfa1.vortex.proto.DTypeProtos.Utf8) dtypeType_) - .mergeFrom(value).buildPartial(); - } else { - dtypeType_ = value; - } - onChanged(); - } else { - if (dtypeTypeCase_ == 5) { - utf8Builder_.mergeFrom(value); - } else { - utf8Builder_.setMessage(value); - } - } - dtypeTypeCase_ = 5; - return this; - } - /** - * .vortex.dtype.Utf8 utf8 = 5; - */ - public Builder clearUtf8() { - if (utf8Builder_ == null) { - if (dtypeTypeCase_ == 5) { - dtypeTypeCase_ = 0; - dtypeType_ = null; - onChanged(); - } - } else { - if (dtypeTypeCase_ == 5) { - dtypeTypeCase_ = 0; - dtypeType_ = null; - } - utf8Builder_.clear(); - } - return this; - } - /** - * .vortex.dtype.Utf8 utf8 = 5; - */ - public io.github.dfa1.vortex.proto.DTypeProtos.Utf8.Builder getUtf8Builder() { - return internalGetUtf8FieldBuilder().getBuilder(); - } - /** - * .vortex.dtype.Utf8 utf8 = 5; - */ - @java.lang.Override - public io.github.dfa1.vortex.proto.DTypeProtos.Utf8OrBuilder getUtf8OrBuilder() { - if ((dtypeTypeCase_ == 5) && (utf8Builder_ != null)) { - return utf8Builder_.getMessageOrBuilder(); - } else { - if (dtypeTypeCase_ == 5) { - return (io.github.dfa1.vortex.proto.DTypeProtos.Utf8) dtypeType_; - } - return io.github.dfa1.vortex.proto.DTypeProtos.Utf8.getDefaultInstance(); - } - } - /** - * .vortex.dtype.Utf8 utf8 = 5; - */ - private com.google.protobuf.SingleFieldBuilder< - io.github.dfa1.vortex.proto.DTypeProtos.Utf8, io.github.dfa1.vortex.proto.DTypeProtos.Utf8.Builder, io.github.dfa1.vortex.proto.DTypeProtos.Utf8OrBuilder> - internalGetUtf8FieldBuilder() { - if (utf8Builder_ == null) { - if (!(dtypeTypeCase_ == 5)) { - dtypeType_ = io.github.dfa1.vortex.proto.DTypeProtos.Utf8.getDefaultInstance(); - } - utf8Builder_ = new com.google.protobuf.SingleFieldBuilder< - io.github.dfa1.vortex.proto.DTypeProtos.Utf8, io.github.dfa1.vortex.proto.DTypeProtos.Utf8.Builder, io.github.dfa1.vortex.proto.DTypeProtos.Utf8OrBuilder>( - (io.github.dfa1.vortex.proto.DTypeProtos.Utf8) dtypeType_, - getParentForChildren(), - isClean()); - dtypeType_ = null; - } - dtypeTypeCase_ = 5; - onChanged(); - return utf8Builder_; - } - - private com.google.protobuf.SingleFieldBuilder< - io.github.dfa1.vortex.proto.DTypeProtos.Binary, io.github.dfa1.vortex.proto.DTypeProtos.Binary.Builder, io.github.dfa1.vortex.proto.DTypeProtos.BinaryOrBuilder> binaryBuilder_; - /** - * .vortex.dtype.Binary binary = 6; - * @return Whether the binary field is set. - */ - @java.lang.Override - public boolean hasBinary() { - return dtypeTypeCase_ == 6; - } - /** - * .vortex.dtype.Binary binary = 6; - * @return The binary. - */ - @java.lang.Override - public io.github.dfa1.vortex.proto.DTypeProtos.Binary getBinary() { - if (binaryBuilder_ == null) { - if (dtypeTypeCase_ == 6) { - return (io.github.dfa1.vortex.proto.DTypeProtos.Binary) dtypeType_; - } - return io.github.dfa1.vortex.proto.DTypeProtos.Binary.getDefaultInstance(); - } else { - if (dtypeTypeCase_ == 6) { - return binaryBuilder_.getMessage(); - } - return io.github.dfa1.vortex.proto.DTypeProtos.Binary.getDefaultInstance(); - } - } - /** - * .vortex.dtype.Binary binary = 6; - */ - public Builder setBinary(io.github.dfa1.vortex.proto.DTypeProtos.Binary value) { - if (binaryBuilder_ == null) { - if (value == null) { - throw new NullPointerException(); - } - dtypeType_ = value; - onChanged(); - } else { - binaryBuilder_.setMessage(value); - } - dtypeTypeCase_ = 6; - return this; - } - /** - * .vortex.dtype.Binary binary = 6; - */ - public Builder setBinary( - io.github.dfa1.vortex.proto.DTypeProtos.Binary.Builder builderForValue) { - if (binaryBuilder_ == null) { - dtypeType_ = builderForValue.build(); - onChanged(); - } else { - binaryBuilder_.setMessage(builderForValue.build()); - } - dtypeTypeCase_ = 6; - return this; - } - /** - * .vortex.dtype.Binary binary = 6; - */ - public Builder mergeBinary(io.github.dfa1.vortex.proto.DTypeProtos.Binary value) { - if (binaryBuilder_ == null) { - if (dtypeTypeCase_ == 6 && - dtypeType_ != io.github.dfa1.vortex.proto.DTypeProtos.Binary.getDefaultInstance()) { - dtypeType_ = io.github.dfa1.vortex.proto.DTypeProtos.Binary.newBuilder((io.github.dfa1.vortex.proto.DTypeProtos.Binary) dtypeType_) - .mergeFrom(value).buildPartial(); - } else { - dtypeType_ = value; - } - onChanged(); - } else { - if (dtypeTypeCase_ == 6) { - binaryBuilder_.mergeFrom(value); - } else { - binaryBuilder_.setMessage(value); - } - } - dtypeTypeCase_ = 6; - return this; - } - /** - * .vortex.dtype.Binary binary = 6; - */ - public Builder clearBinary() { - if (binaryBuilder_ == null) { - if (dtypeTypeCase_ == 6) { - dtypeTypeCase_ = 0; - dtypeType_ = null; - onChanged(); - } - } else { - if (dtypeTypeCase_ == 6) { - dtypeTypeCase_ = 0; - dtypeType_ = null; - } - binaryBuilder_.clear(); - } - return this; - } - /** - * .vortex.dtype.Binary binary = 6; - */ - public io.github.dfa1.vortex.proto.DTypeProtos.Binary.Builder getBinaryBuilder() { - return internalGetBinaryFieldBuilder().getBuilder(); - } - /** - * .vortex.dtype.Binary binary = 6; - */ - @java.lang.Override - public io.github.dfa1.vortex.proto.DTypeProtos.BinaryOrBuilder getBinaryOrBuilder() { - if ((dtypeTypeCase_ == 6) && (binaryBuilder_ != null)) { - return binaryBuilder_.getMessageOrBuilder(); - } else { - if (dtypeTypeCase_ == 6) { - return (io.github.dfa1.vortex.proto.DTypeProtos.Binary) dtypeType_; - } - return io.github.dfa1.vortex.proto.DTypeProtos.Binary.getDefaultInstance(); - } - } - /** - * .vortex.dtype.Binary binary = 6; - */ - private com.google.protobuf.SingleFieldBuilder< - io.github.dfa1.vortex.proto.DTypeProtos.Binary, io.github.dfa1.vortex.proto.DTypeProtos.Binary.Builder, io.github.dfa1.vortex.proto.DTypeProtos.BinaryOrBuilder> - internalGetBinaryFieldBuilder() { - if (binaryBuilder_ == null) { - if (!(dtypeTypeCase_ == 6)) { - dtypeType_ = io.github.dfa1.vortex.proto.DTypeProtos.Binary.getDefaultInstance(); - } - binaryBuilder_ = new com.google.protobuf.SingleFieldBuilder< - io.github.dfa1.vortex.proto.DTypeProtos.Binary, io.github.dfa1.vortex.proto.DTypeProtos.Binary.Builder, io.github.dfa1.vortex.proto.DTypeProtos.BinaryOrBuilder>( - (io.github.dfa1.vortex.proto.DTypeProtos.Binary) dtypeType_, - getParentForChildren(), - isClean()); - dtypeType_ = null; - } - dtypeTypeCase_ = 6; - onChanged(); - return binaryBuilder_; - } - - private com.google.protobuf.SingleFieldBuilder< - io.github.dfa1.vortex.proto.DTypeProtos.Struct, io.github.dfa1.vortex.proto.DTypeProtos.Struct.Builder, io.github.dfa1.vortex.proto.DTypeProtos.StructOrBuilder> structBuilder_; - /** - * .vortex.dtype.Struct struct = 7; - * @return Whether the struct field is set. - */ - @java.lang.Override - public boolean hasStruct() { - return dtypeTypeCase_ == 7; - } - /** - * .vortex.dtype.Struct struct = 7; - * @return The struct. - */ - @java.lang.Override - public io.github.dfa1.vortex.proto.DTypeProtos.Struct getStruct() { - if (structBuilder_ == null) { - if (dtypeTypeCase_ == 7) { - return (io.github.dfa1.vortex.proto.DTypeProtos.Struct) dtypeType_; - } - return io.github.dfa1.vortex.proto.DTypeProtos.Struct.getDefaultInstance(); - } else { - if (dtypeTypeCase_ == 7) { - return structBuilder_.getMessage(); - } - return io.github.dfa1.vortex.proto.DTypeProtos.Struct.getDefaultInstance(); - } - } - /** - * .vortex.dtype.Struct struct = 7; - */ - public Builder setStruct(io.github.dfa1.vortex.proto.DTypeProtos.Struct value) { - if (structBuilder_ == null) { - if (value == null) { - throw new NullPointerException(); - } - dtypeType_ = value; - onChanged(); - } else { - structBuilder_.setMessage(value); - } - dtypeTypeCase_ = 7; - return this; - } - /** - * .vortex.dtype.Struct struct = 7; - */ - public Builder setStruct( - io.github.dfa1.vortex.proto.DTypeProtos.Struct.Builder builderForValue) { - if (structBuilder_ == null) { - dtypeType_ = builderForValue.build(); - onChanged(); - } else { - structBuilder_.setMessage(builderForValue.build()); - } - dtypeTypeCase_ = 7; - return this; - } - /** - * .vortex.dtype.Struct struct = 7; - */ - public Builder mergeStruct(io.github.dfa1.vortex.proto.DTypeProtos.Struct value) { - if (structBuilder_ == null) { - if (dtypeTypeCase_ == 7 && - dtypeType_ != io.github.dfa1.vortex.proto.DTypeProtos.Struct.getDefaultInstance()) { - dtypeType_ = io.github.dfa1.vortex.proto.DTypeProtos.Struct.newBuilder((io.github.dfa1.vortex.proto.DTypeProtos.Struct) dtypeType_) - .mergeFrom(value).buildPartial(); - } else { - dtypeType_ = value; - } - onChanged(); - } else { - if (dtypeTypeCase_ == 7) { - structBuilder_.mergeFrom(value); - } else { - structBuilder_.setMessage(value); - } - } - dtypeTypeCase_ = 7; - return this; - } - /** - * .vortex.dtype.Struct struct = 7; - */ - public Builder clearStruct() { - if (structBuilder_ == null) { - if (dtypeTypeCase_ == 7) { - dtypeTypeCase_ = 0; - dtypeType_ = null; - onChanged(); - } - } else { - if (dtypeTypeCase_ == 7) { - dtypeTypeCase_ = 0; - dtypeType_ = null; - } - structBuilder_.clear(); - } - return this; - } - /** - * .vortex.dtype.Struct struct = 7; - */ - public io.github.dfa1.vortex.proto.DTypeProtos.Struct.Builder getStructBuilder() { - return internalGetStructFieldBuilder().getBuilder(); - } - /** - * .vortex.dtype.Struct struct = 7; - */ - @java.lang.Override - public io.github.dfa1.vortex.proto.DTypeProtos.StructOrBuilder getStructOrBuilder() { - if ((dtypeTypeCase_ == 7) && (structBuilder_ != null)) { - return structBuilder_.getMessageOrBuilder(); - } else { - if (dtypeTypeCase_ == 7) { - return (io.github.dfa1.vortex.proto.DTypeProtos.Struct) dtypeType_; - } - return io.github.dfa1.vortex.proto.DTypeProtos.Struct.getDefaultInstance(); - } - } - /** - * .vortex.dtype.Struct struct = 7; - */ - private com.google.protobuf.SingleFieldBuilder< - io.github.dfa1.vortex.proto.DTypeProtos.Struct, io.github.dfa1.vortex.proto.DTypeProtos.Struct.Builder, io.github.dfa1.vortex.proto.DTypeProtos.StructOrBuilder> - internalGetStructFieldBuilder() { - if (structBuilder_ == null) { - if (!(dtypeTypeCase_ == 7)) { - dtypeType_ = io.github.dfa1.vortex.proto.DTypeProtos.Struct.getDefaultInstance(); - } - structBuilder_ = new com.google.protobuf.SingleFieldBuilder< - io.github.dfa1.vortex.proto.DTypeProtos.Struct, io.github.dfa1.vortex.proto.DTypeProtos.Struct.Builder, io.github.dfa1.vortex.proto.DTypeProtos.StructOrBuilder>( - (io.github.dfa1.vortex.proto.DTypeProtos.Struct) dtypeType_, - getParentForChildren(), - isClean()); - dtypeType_ = null; - } - dtypeTypeCase_ = 7; - onChanged(); - return structBuilder_; - } - - private com.google.protobuf.SingleFieldBuilder< - io.github.dfa1.vortex.proto.DTypeProtos.List, io.github.dfa1.vortex.proto.DTypeProtos.List.Builder, io.github.dfa1.vortex.proto.DTypeProtos.ListOrBuilder> listBuilder_; - /** - * .vortex.dtype.List list = 8; - * @return Whether the list field is set. - */ - @java.lang.Override - public boolean hasList() { - return dtypeTypeCase_ == 8; - } - /** - * .vortex.dtype.List list = 8; - * @return The list. - */ - @java.lang.Override - public io.github.dfa1.vortex.proto.DTypeProtos.List getList() { - if (listBuilder_ == null) { - if (dtypeTypeCase_ == 8) { - return (io.github.dfa1.vortex.proto.DTypeProtos.List) dtypeType_; - } - return io.github.dfa1.vortex.proto.DTypeProtos.List.getDefaultInstance(); - } else { - if (dtypeTypeCase_ == 8) { - return listBuilder_.getMessage(); - } - return io.github.dfa1.vortex.proto.DTypeProtos.List.getDefaultInstance(); - } - } - /** - * .vortex.dtype.List list = 8; - */ - public Builder setList(io.github.dfa1.vortex.proto.DTypeProtos.List value) { - if (listBuilder_ == null) { - if (value == null) { - throw new NullPointerException(); - } - dtypeType_ = value; - onChanged(); - } else { - listBuilder_.setMessage(value); - } - dtypeTypeCase_ = 8; - return this; - } - /** - * .vortex.dtype.List list = 8; - */ - public Builder setList( - io.github.dfa1.vortex.proto.DTypeProtos.List.Builder builderForValue) { - if (listBuilder_ == null) { - dtypeType_ = builderForValue.build(); - onChanged(); - } else { - listBuilder_.setMessage(builderForValue.build()); - } - dtypeTypeCase_ = 8; - return this; - } - /** - * .vortex.dtype.List list = 8; - */ - public Builder mergeList(io.github.dfa1.vortex.proto.DTypeProtos.List value) { - if (listBuilder_ == null) { - if (dtypeTypeCase_ == 8 && - dtypeType_ != io.github.dfa1.vortex.proto.DTypeProtos.List.getDefaultInstance()) { - dtypeType_ = io.github.dfa1.vortex.proto.DTypeProtos.List.newBuilder((io.github.dfa1.vortex.proto.DTypeProtos.List) dtypeType_) - .mergeFrom(value).buildPartial(); - } else { - dtypeType_ = value; - } - onChanged(); - } else { - if (dtypeTypeCase_ == 8) { - listBuilder_.mergeFrom(value); - } else { - listBuilder_.setMessage(value); - } - } - dtypeTypeCase_ = 8; - return this; - } - /** - * .vortex.dtype.List list = 8; - */ - public Builder clearList() { - if (listBuilder_ == null) { - if (dtypeTypeCase_ == 8) { - dtypeTypeCase_ = 0; - dtypeType_ = null; - onChanged(); - } - } else { - if (dtypeTypeCase_ == 8) { - dtypeTypeCase_ = 0; - dtypeType_ = null; - } - listBuilder_.clear(); - } - return this; - } - /** - * .vortex.dtype.List list = 8; - */ - public io.github.dfa1.vortex.proto.DTypeProtos.List.Builder getListBuilder() { - return internalGetListFieldBuilder().getBuilder(); - } - /** - * .vortex.dtype.List list = 8; - */ - @java.lang.Override - public io.github.dfa1.vortex.proto.DTypeProtos.ListOrBuilder getListOrBuilder() { - if ((dtypeTypeCase_ == 8) && (listBuilder_ != null)) { - return listBuilder_.getMessageOrBuilder(); - } else { - if (dtypeTypeCase_ == 8) { - return (io.github.dfa1.vortex.proto.DTypeProtos.List) dtypeType_; - } - return io.github.dfa1.vortex.proto.DTypeProtos.List.getDefaultInstance(); - } - } - /** - * .vortex.dtype.List list = 8; - */ - private com.google.protobuf.SingleFieldBuilder< - io.github.dfa1.vortex.proto.DTypeProtos.List, io.github.dfa1.vortex.proto.DTypeProtos.List.Builder, io.github.dfa1.vortex.proto.DTypeProtos.ListOrBuilder> - internalGetListFieldBuilder() { - if (listBuilder_ == null) { - if (!(dtypeTypeCase_ == 8)) { - dtypeType_ = io.github.dfa1.vortex.proto.DTypeProtos.List.getDefaultInstance(); - } - listBuilder_ = new com.google.protobuf.SingleFieldBuilder< - io.github.dfa1.vortex.proto.DTypeProtos.List, io.github.dfa1.vortex.proto.DTypeProtos.List.Builder, io.github.dfa1.vortex.proto.DTypeProtos.ListOrBuilder>( - (io.github.dfa1.vortex.proto.DTypeProtos.List) dtypeType_, - getParentForChildren(), - isClean()); - dtypeType_ = null; - } - dtypeTypeCase_ = 8; - onChanged(); - return listBuilder_; - } - - private com.google.protobuf.SingleFieldBuilder< - io.github.dfa1.vortex.proto.DTypeProtos.Extension, io.github.dfa1.vortex.proto.DTypeProtos.Extension.Builder, io.github.dfa1.vortex.proto.DTypeProtos.ExtensionOrBuilder> extensionBuilder_; - /** - * .vortex.dtype.Extension extension = 9; - * @return Whether the extension field is set. - */ - @java.lang.Override - public boolean hasExtension() { - return dtypeTypeCase_ == 9; - } - /** - * .vortex.dtype.Extension extension = 9; - * @return The extension. - */ - @java.lang.Override - public io.github.dfa1.vortex.proto.DTypeProtos.Extension getExtension() { - if (extensionBuilder_ == null) { - if (dtypeTypeCase_ == 9) { - return (io.github.dfa1.vortex.proto.DTypeProtos.Extension) dtypeType_; - } - return io.github.dfa1.vortex.proto.DTypeProtos.Extension.getDefaultInstance(); - } else { - if (dtypeTypeCase_ == 9) { - return extensionBuilder_.getMessage(); - } - return io.github.dfa1.vortex.proto.DTypeProtos.Extension.getDefaultInstance(); - } - } - /** - * .vortex.dtype.Extension extension = 9; - */ - public Builder setExtension(io.github.dfa1.vortex.proto.DTypeProtos.Extension value) { - if (extensionBuilder_ == null) { - if (value == null) { - throw new NullPointerException(); - } - dtypeType_ = value; - onChanged(); - } else { - extensionBuilder_.setMessage(value); - } - dtypeTypeCase_ = 9; - return this; - } - /** - * .vortex.dtype.Extension extension = 9; - */ - public Builder setExtension( - io.github.dfa1.vortex.proto.DTypeProtos.Extension.Builder builderForValue) { - if (extensionBuilder_ == null) { - dtypeType_ = builderForValue.build(); - onChanged(); - } else { - extensionBuilder_.setMessage(builderForValue.build()); - } - dtypeTypeCase_ = 9; - return this; - } - /** - * .vortex.dtype.Extension extension = 9; - */ - public Builder mergeExtension(io.github.dfa1.vortex.proto.DTypeProtos.Extension value) { - if (extensionBuilder_ == null) { - if (dtypeTypeCase_ == 9 && - dtypeType_ != io.github.dfa1.vortex.proto.DTypeProtos.Extension.getDefaultInstance()) { - dtypeType_ = io.github.dfa1.vortex.proto.DTypeProtos.Extension.newBuilder((io.github.dfa1.vortex.proto.DTypeProtos.Extension) dtypeType_) - .mergeFrom(value).buildPartial(); - } else { - dtypeType_ = value; - } - onChanged(); - } else { - if (dtypeTypeCase_ == 9) { - extensionBuilder_.mergeFrom(value); - } else { - extensionBuilder_.setMessage(value); - } - } - dtypeTypeCase_ = 9; - return this; - } - /** - * .vortex.dtype.Extension extension = 9; - */ - public Builder clearExtension() { - if (extensionBuilder_ == null) { - if (dtypeTypeCase_ == 9) { - dtypeTypeCase_ = 0; - dtypeType_ = null; - onChanged(); - } - } else { - if (dtypeTypeCase_ == 9) { - dtypeTypeCase_ = 0; - dtypeType_ = null; - } - extensionBuilder_.clear(); - } - return this; - } - /** - * .vortex.dtype.Extension extension = 9; - */ - public io.github.dfa1.vortex.proto.DTypeProtos.Extension.Builder getExtensionBuilder() { - return internalGetExtensionFieldBuilder().getBuilder(); - } - /** - * .vortex.dtype.Extension extension = 9; - */ - @java.lang.Override - public io.github.dfa1.vortex.proto.DTypeProtos.ExtensionOrBuilder getExtensionOrBuilder() { - if ((dtypeTypeCase_ == 9) && (extensionBuilder_ != null)) { - return extensionBuilder_.getMessageOrBuilder(); - } else { - if (dtypeTypeCase_ == 9) { - return (io.github.dfa1.vortex.proto.DTypeProtos.Extension) dtypeType_; - } - return io.github.dfa1.vortex.proto.DTypeProtos.Extension.getDefaultInstance(); - } - } - /** - * .vortex.dtype.Extension extension = 9; - */ - private com.google.protobuf.SingleFieldBuilder< - io.github.dfa1.vortex.proto.DTypeProtos.Extension, io.github.dfa1.vortex.proto.DTypeProtos.Extension.Builder, io.github.dfa1.vortex.proto.DTypeProtos.ExtensionOrBuilder> - internalGetExtensionFieldBuilder() { - if (extensionBuilder_ == null) { - if (!(dtypeTypeCase_ == 9)) { - dtypeType_ = io.github.dfa1.vortex.proto.DTypeProtos.Extension.getDefaultInstance(); - } - extensionBuilder_ = new com.google.protobuf.SingleFieldBuilder< - io.github.dfa1.vortex.proto.DTypeProtos.Extension, io.github.dfa1.vortex.proto.DTypeProtos.Extension.Builder, io.github.dfa1.vortex.proto.DTypeProtos.ExtensionOrBuilder>( - (io.github.dfa1.vortex.proto.DTypeProtos.Extension) dtypeType_, - getParentForChildren(), - isClean()); - dtypeType_ = null; - } - dtypeTypeCase_ = 9; - onChanged(); - return extensionBuilder_; - } - - private com.google.protobuf.SingleFieldBuilder< - io.github.dfa1.vortex.proto.DTypeProtos.FixedSizeList, io.github.dfa1.vortex.proto.DTypeProtos.FixedSizeList.Builder, io.github.dfa1.vortex.proto.DTypeProtos.FixedSizeListOrBuilder> fixedSizeListBuilder_; - /** - * 
-       * This is after `Extension` for backwards compatibility.
-       *
- * - * .vortex.dtype.FixedSizeList fixed_size_list = 10; - * @return Whether the fixedSizeList field is set. - */ - @java.lang.Override - public boolean hasFixedSizeList() { - return dtypeTypeCase_ == 10; - } - /** - * 
-       * This is after `Extension` for backwards compatibility.
-       *
- * - * .vortex.dtype.FixedSizeList fixed_size_list = 10; - * @return The fixedSizeList. - */ - @java.lang.Override - public io.github.dfa1.vortex.proto.DTypeProtos.FixedSizeList getFixedSizeList() { - if (fixedSizeListBuilder_ == null) { - if (dtypeTypeCase_ == 10) { - return (io.github.dfa1.vortex.proto.DTypeProtos.FixedSizeList) dtypeType_; - } - return io.github.dfa1.vortex.proto.DTypeProtos.FixedSizeList.getDefaultInstance(); - } else { - if (dtypeTypeCase_ == 10) { - return fixedSizeListBuilder_.getMessage(); - } - return io.github.dfa1.vortex.proto.DTypeProtos.FixedSizeList.getDefaultInstance(); - } - } - /** - * 
-       * This is after `Extension` for backwards compatibility.
-       *
- * - * .vortex.dtype.FixedSizeList fixed_size_list = 10; - */ - public Builder setFixedSizeList(io.github.dfa1.vortex.proto.DTypeProtos.FixedSizeList value) { - if (fixedSizeListBuilder_ == null) { - if (value == null) { - throw new NullPointerException(); - } - dtypeType_ = value; - onChanged(); - } else { - fixedSizeListBuilder_.setMessage(value); - } - dtypeTypeCase_ = 10; - return this; - } - /** - * 
-       * This is after `Extension` for backwards compatibility.
-       *
- * - * .vortex.dtype.FixedSizeList fixed_size_list = 10; - */ - public Builder setFixedSizeList( - io.github.dfa1.vortex.proto.DTypeProtos.FixedSizeList.Builder builderForValue) { - if (fixedSizeListBuilder_ == null) { - dtypeType_ = builderForValue.build(); - onChanged(); - } else { - fixedSizeListBuilder_.setMessage(builderForValue.build()); - } - dtypeTypeCase_ = 10; - return this; - } - /** - * 
-       * This is after `Extension` for backwards compatibility.
-       *
- * - * .vortex.dtype.FixedSizeList fixed_size_list = 10; - */ - public Builder mergeFixedSizeList(io.github.dfa1.vortex.proto.DTypeProtos.FixedSizeList value) { - if (fixedSizeListBuilder_ == null) { - if (dtypeTypeCase_ == 10 && - dtypeType_ != io.github.dfa1.vortex.proto.DTypeProtos.FixedSizeList.getDefaultInstance()) { - dtypeType_ = io.github.dfa1.vortex.proto.DTypeProtos.FixedSizeList.newBuilder((io.github.dfa1.vortex.proto.DTypeProtos.FixedSizeList) dtypeType_) - .mergeFrom(value).buildPartial(); - } else { - dtypeType_ = value; - } - onChanged(); - } else { - if (dtypeTypeCase_ == 10) { - fixedSizeListBuilder_.mergeFrom(value); - } else { - fixedSizeListBuilder_.setMessage(value); - } - } - dtypeTypeCase_ = 10; - return this; - } - /** - * 
-       * This is after `Extension` for backwards compatibility.
-       *
- * - * .vortex.dtype.FixedSizeList fixed_size_list = 10; - */ - public Builder clearFixedSizeList() { - if (fixedSizeListBuilder_ == null) { - if (dtypeTypeCase_ == 10) { - dtypeTypeCase_ = 0; - dtypeType_ = null; - onChanged(); - } - } else { - if (dtypeTypeCase_ == 10) { - dtypeTypeCase_ = 0; - dtypeType_ = null; - } - fixedSizeListBuilder_.clear(); - } - return this; - } - /** - * 
-       * This is after `Extension` for backwards compatibility.
-       *
- * - * .vortex.dtype.FixedSizeList fixed_size_list = 10; - */ - public io.github.dfa1.vortex.proto.DTypeProtos.FixedSizeList.Builder getFixedSizeListBuilder() { - return internalGetFixedSizeListFieldBuilder().getBuilder(); - } - /** - * 
-       * This is after `Extension` for backwards compatibility.
-       *
- * - * .vortex.dtype.FixedSizeList fixed_size_list = 10; - */ - @java.lang.Override - public io.github.dfa1.vortex.proto.DTypeProtos.FixedSizeListOrBuilder getFixedSizeListOrBuilder() { - if ((dtypeTypeCase_ == 10) && (fixedSizeListBuilder_ != null)) { - return fixedSizeListBuilder_.getMessageOrBuilder(); - } else { - if (dtypeTypeCase_ == 10) { - return (io.github.dfa1.vortex.proto.DTypeProtos.FixedSizeList) dtypeType_; - } - return io.github.dfa1.vortex.proto.DTypeProtos.FixedSizeList.getDefaultInstance(); - } - } - /** - * 
-       * This is after `Extension` for backwards compatibility.
-       *
- * - * .vortex.dtype.FixedSizeList fixed_size_list = 10; - */ - private com.google.protobuf.SingleFieldBuilder< - io.github.dfa1.vortex.proto.DTypeProtos.FixedSizeList, io.github.dfa1.vortex.proto.DTypeProtos.FixedSizeList.Builder, io.github.dfa1.vortex.proto.DTypeProtos.FixedSizeListOrBuilder> - internalGetFixedSizeListFieldBuilder() { - if (fixedSizeListBuilder_ == null) { - if (!(dtypeTypeCase_ == 10)) { - dtypeType_ = io.github.dfa1.vortex.proto.DTypeProtos.FixedSizeList.getDefaultInstance(); - } - fixedSizeListBuilder_ = new com.google.protobuf.SingleFieldBuilder< - io.github.dfa1.vortex.proto.DTypeProtos.FixedSizeList, io.github.dfa1.vortex.proto.DTypeProtos.FixedSizeList.Builder, io.github.dfa1.vortex.proto.DTypeProtos.FixedSizeListOrBuilder>( - (io.github.dfa1.vortex.proto.DTypeProtos.FixedSizeList) dtypeType_, - getParentForChildren(), - isClean()); - dtypeType_ = null; - } - dtypeTypeCase_ = 10; - onChanged(); - return fixedSizeListBuilder_; - } - - private com.google.protobuf.SingleFieldBuilder< - io.github.dfa1.vortex.proto.DTypeProtos.Variant, io.github.dfa1.vortex.proto.DTypeProtos.Variant.Builder, io.github.dfa1.vortex.proto.DTypeProtos.VariantOrBuilder> variantBuilder_; - /** - * .vortex.dtype.Variant variant = 11; - * @return Whether the variant field is set. - */ - @java.lang.Override - public boolean hasVariant() { - return dtypeTypeCase_ == 11; - } - /** - * .vortex.dtype.Variant variant = 11; - * @return The variant. - */ - @java.lang.Override - public io.github.dfa1.vortex.proto.DTypeProtos.Variant getVariant() { - if (variantBuilder_ == null) { - if (dtypeTypeCase_ == 11) { - return (io.github.dfa1.vortex.proto.DTypeProtos.Variant) dtypeType_; - } - return io.github.dfa1.vortex.proto.DTypeProtos.Variant.getDefaultInstance(); - } else { - if (dtypeTypeCase_ == 11) { - return variantBuilder_.getMessage(); - } - return io.github.dfa1.vortex.proto.DTypeProtos.Variant.getDefaultInstance(); - } - } - /** - * .vortex.dtype.Variant variant = 11; - */ - public Builder setVariant(io.github.dfa1.vortex.proto.DTypeProtos.Variant value) { - if (variantBuilder_ == null) { - if (value == null) { - throw new NullPointerException(); - } - dtypeType_ = value; - onChanged(); - } else { - variantBuilder_.setMessage(value); - } - dtypeTypeCase_ = 11; - return this; - } - /** - * .vortex.dtype.Variant variant = 11; - */ - public Builder setVariant( - io.github.dfa1.vortex.proto.DTypeProtos.Variant.Builder builderForValue) { - if (variantBuilder_ == null) { - dtypeType_ = builderForValue.build(); - onChanged(); - } else { - variantBuilder_.setMessage(builderForValue.build()); - } - dtypeTypeCase_ = 11; - return this; - } - /** - * .vortex.dtype.Variant variant = 11; - */ - public Builder mergeVariant(io.github.dfa1.vortex.proto.DTypeProtos.Variant value) { - if (variantBuilder_ == null) { - if (dtypeTypeCase_ == 11 && - dtypeType_ != io.github.dfa1.vortex.proto.DTypeProtos.Variant.getDefaultInstance()) { - dtypeType_ = io.github.dfa1.vortex.proto.DTypeProtos.Variant.newBuilder((io.github.dfa1.vortex.proto.DTypeProtos.Variant) dtypeType_) - .mergeFrom(value).buildPartial(); - } else { - dtypeType_ = value; - } - onChanged(); - } else { - if (dtypeTypeCase_ == 11) { - variantBuilder_.mergeFrom(value); - } else { - variantBuilder_.setMessage(value); - } - } - dtypeTypeCase_ = 11; - return this; - } - /** - * .vortex.dtype.Variant variant = 11; - */ - public Builder clearVariant() { - if (variantBuilder_ == null) { - if (dtypeTypeCase_ == 11) { - dtypeTypeCase_ = 0; - dtypeType_ = null; - onChanged(); - } - } else { - if (dtypeTypeCase_ == 11) { - dtypeTypeCase_ = 0; - dtypeType_ = null; - } - variantBuilder_.clear(); - } - return this; - } - /** - * .vortex.dtype.Variant variant = 11; - */ - public io.github.dfa1.vortex.proto.DTypeProtos.Variant.Builder getVariantBuilder() { - return internalGetVariantFieldBuilder().getBuilder(); - } - /** - * .vortex.dtype.Variant variant = 11; - */ - @java.lang.Override - public io.github.dfa1.vortex.proto.DTypeProtos.VariantOrBuilder getVariantOrBuilder() { - if ((dtypeTypeCase_ == 11) && (variantBuilder_ != null)) { - return variantBuilder_.getMessageOrBuilder(); - } else { - if (dtypeTypeCase_ == 11) { - return (io.github.dfa1.vortex.proto.DTypeProtos.Variant) dtypeType_; - } - return io.github.dfa1.vortex.proto.DTypeProtos.Variant.getDefaultInstance(); - } - } - /** - * .vortex.dtype.Variant variant = 11; - */ - private com.google.protobuf.SingleFieldBuilder< - io.github.dfa1.vortex.proto.DTypeProtos.Variant, io.github.dfa1.vortex.proto.DTypeProtos.Variant.Builder, io.github.dfa1.vortex.proto.DTypeProtos.VariantOrBuilder> - internalGetVariantFieldBuilder() { - if (variantBuilder_ == null) { - if (!(dtypeTypeCase_ == 11)) { - dtypeType_ = io.github.dfa1.vortex.proto.DTypeProtos.Variant.getDefaultInstance(); - } - variantBuilder_ = new com.google.protobuf.SingleFieldBuilder< - io.github.dfa1.vortex.proto.DTypeProtos.Variant, io.github.dfa1.vortex.proto.DTypeProtos.Variant.Builder, io.github.dfa1.vortex.proto.DTypeProtos.VariantOrBuilder>( - (io.github.dfa1.vortex.proto.DTypeProtos.Variant) dtypeType_, - getParentForChildren(), - isClean()); - dtypeType_ = null; - } - dtypeTypeCase_ = 11; - onChanged(); - return variantBuilder_; - } - - private com.google.protobuf.SingleFieldBuilder< - io.github.dfa1.vortex.proto.DTypeProtos.Union, io.github.dfa1.vortex.proto.DTypeProtos.Union.Builder, io.github.dfa1.vortex.proto.DTypeProtos.UnionOrBuilder> unionBuilder_; - /** - * .vortex.dtype.Union union = 12; - * @return Whether the union field is set. - */ - @java.lang.Override - public boolean hasUnion() { - return dtypeTypeCase_ == 12; - } - /** - * .vortex.dtype.Union union = 12; - * @return The union. - */ - @java.lang.Override - public io.github.dfa1.vortex.proto.DTypeProtos.Union getUnion() { - if (unionBuilder_ == null) { - if (dtypeTypeCase_ == 12) { - return (io.github.dfa1.vortex.proto.DTypeProtos.Union) dtypeType_; - } - return io.github.dfa1.vortex.proto.DTypeProtos.Union.getDefaultInstance(); - } else { - if (dtypeTypeCase_ == 12) { - return unionBuilder_.getMessage(); - } - return io.github.dfa1.vortex.proto.DTypeProtos.Union.getDefaultInstance(); - } - } - /** - * .vortex.dtype.Union union = 12; - */ - public Builder setUnion(io.github.dfa1.vortex.proto.DTypeProtos.Union value) { - if (unionBuilder_ == null) { - if (value == null) { - throw new NullPointerException(); - } - dtypeType_ = value; - onChanged(); - } else { - unionBuilder_.setMessage(value); - } - dtypeTypeCase_ = 12; - return this; - } - /** - * .vortex.dtype.Union union = 12; - */ - public Builder setUnion( - io.github.dfa1.vortex.proto.DTypeProtos.Union.Builder builderForValue) { - if (unionBuilder_ == null) { - dtypeType_ = builderForValue.build(); - onChanged(); - } else { - unionBuilder_.setMessage(builderForValue.build()); - } - dtypeTypeCase_ = 12; - return this; - } - /** - * .vortex.dtype.Union union = 12; - */ - public Builder mergeUnion(io.github.dfa1.vortex.proto.DTypeProtos.Union value) { - if (unionBuilder_ == null) { - if (dtypeTypeCase_ == 12 && - dtypeType_ != io.github.dfa1.vortex.proto.DTypeProtos.Union.getDefaultInstance()) { - dtypeType_ = io.github.dfa1.vortex.proto.DTypeProtos.Union.newBuilder((io.github.dfa1.vortex.proto.DTypeProtos.Union) dtypeType_) - .mergeFrom(value).buildPartial(); - } else { - dtypeType_ = value; - } - onChanged(); - } else { - if (dtypeTypeCase_ == 12) { - unionBuilder_.mergeFrom(value); - } else { - unionBuilder_.setMessage(value); - } - } - dtypeTypeCase_ = 12; - return this; - } - /** - * .vortex.dtype.Union union = 12; - */ - public Builder clearUnion() { - if (unionBuilder_ == null) { - if (dtypeTypeCase_ == 12) { - dtypeTypeCase_ = 0; - dtypeType_ = null; - onChanged(); - } - } else { - if (dtypeTypeCase_ == 12) { - dtypeTypeCase_ = 0; - dtypeType_ = null; - } - unionBuilder_.clear(); - } - return this; - } - /** - * .vortex.dtype.Union union = 12; - */ - public io.github.dfa1.vortex.proto.DTypeProtos.Union.Builder getUnionBuilder() { - return internalGetUnionFieldBuilder().getBuilder(); - } - /** - * .vortex.dtype.Union union = 12; - */ - @java.lang.Override - public io.github.dfa1.vortex.proto.DTypeProtos.UnionOrBuilder getUnionOrBuilder() { - if ((dtypeTypeCase_ == 12) && (unionBuilder_ != null)) { - return unionBuilder_.getMessageOrBuilder(); - } else { - if (dtypeTypeCase_ == 12) { - return (io.github.dfa1.vortex.proto.DTypeProtos.Union) dtypeType_; - } - return io.github.dfa1.vortex.proto.DTypeProtos.Union.getDefaultInstance(); - } - } - /** - * .vortex.dtype.Union union = 12; - */ - private com.google.protobuf.SingleFieldBuilder< - io.github.dfa1.vortex.proto.DTypeProtos.Union, io.github.dfa1.vortex.proto.DTypeProtos.Union.Builder, io.github.dfa1.vortex.proto.DTypeProtos.UnionOrBuilder> - internalGetUnionFieldBuilder() { - if (unionBuilder_ == null) { - if (!(dtypeTypeCase_ == 12)) { - dtypeType_ = io.github.dfa1.vortex.proto.DTypeProtos.Union.getDefaultInstance(); - } - unionBuilder_ = new com.google.protobuf.SingleFieldBuilder< - io.github.dfa1.vortex.proto.DTypeProtos.Union, io.github.dfa1.vortex.proto.DTypeProtos.Union.Builder, io.github.dfa1.vortex.proto.DTypeProtos.UnionOrBuilder>( - (io.github.dfa1.vortex.proto.DTypeProtos.Union) dtypeType_, - getParentForChildren(), - isClean()); - dtypeType_ = null; - } - dtypeTypeCase_ = 12; - onChanged(); - return unionBuilder_; - } - - // @@protoc_insertion_point(builder_scope:vortex.dtype.DType) - } - - // @@protoc_insertion_point(class_scope:vortex.dtype.DType) - private static final io.github.dfa1.vortex.proto.DTypeProtos.DType DEFAULT_INSTANCE; - static { - DEFAULT_INSTANCE = new io.github.dfa1.vortex.proto.DTypeProtos.DType(); - } - - public static io.github.dfa1.vortex.proto.DTypeProtos.DType getDefaultInstance() { - return DEFAULT_INSTANCE; - } - - private static final com.google.protobuf.Parser - PARSER = new com.google.protobuf.AbstractParser() { - @java.lang.Override - public DType parsePartialFrom( - com.google.protobuf.CodedInputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws com.google.protobuf.InvalidProtocolBufferException { - Builder builder = newBuilder(); - try { - builder.mergeFrom(input, extensionRegistry); - } catch (com.google.protobuf.InvalidProtocolBufferException e) { - throw e.setUnfinishedMessage(builder.buildPartial()); - } catch (com.google.protobuf.UninitializedMessageException e) { - throw e.asInvalidProtocolBufferException().setUnfinishedMessage(builder.buildPartial()); - } catch (java.io.IOException e) { - throw new com.google.protobuf.InvalidProtocolBufferException(e) - .setUnfinishedMessage(builder.buildPartial()); - } - return builder.buildPartial(); - } - }; - - public static com.google.protobuf.Parser parser() { - return PARSER; - } - - @java.lang.Override - public com.google.protobuf.Parser getParserForType() { - return PARSER; - } - - @java.lang.Override - public io.github.dfa1.vortex.proto.DTypeProtos.DType getDefaultInstanceForType() { - return DEFAULT_INSTANCE; - } - - } - - public interface FieldOrBuilder extends - // @@protoc_insertion_point(interface_extends:vortex.dtype.Field) - com.google.protobuf.MessageOrBuilder { - - /** - * string name = 1; - * @return Whether the name field is set. - */ - boolean hasName(); - /** - * string name = 1; - * @return The name. - */ - java.lang.String getName(); - /** - * string name = 1; - * @return The bytes for name. - */ - com.google.protobuf.ByteString - getNameBytes(); - - io.github.dfa1.vortex.proto.DTypeProtos.Field.FieldTypeCase getFieldTypeCase(); - } - /** - * Protobuf type {@code vortex.dtype.Field} - */ - public static final class Field extends - com.google.protobuf.GeneratedMessage implements - // @@protoc_insertion_point(message_implements:vortex.dtype.Field) - FieldOrBuilder { - private static final long serialVersionUID = 0L; - static { - com.google.protobuf.RuntimeVersion.validateProtobufGencodeVersion( - com.google.protobuf.RuntimeVersion.RuntimeDomain.PUBLIC, - /* major= */ 4, - /* minor= */ 35, - /* patch= */ 0, - /* suffix= */ "", - "Field"); - } - // Use Field.newBuilder() to construct. - private Field(com.google.protobuf.GeneratedMessage.Builder builder) { - super(builder); - } - private Field() { - } - - public static final com.google.protobuf.Descriptors.Descriptor - getDescriptor() { - return io.github.dfa1.vortex.proto.DTypeProtos.internal_static_vortex_dtype_Field_descriptor; - } - - @java.lang.Override - public com.google.protobuf.Descriptors.Descriptor getDescriptorForType() { - return io.github.dfa1.vortex.proto.DTypeProtos.internal_static_vortex_dtype_Field_descriptor; - } - - @java.lang.Override - protected com.google.protobuf.GeneratedMessage.FieldAccessorTable - internalGetFieldAccessorTable() { - return io.github.dfa1.vortex.proto.DTypeProtos.internal_static_vortex_dtype_Field_fieldAccessorTable - .ensureFieldAccessorsInitialized( - io.github.dfa1.vortex.proto.DTypeProtos.Field.class, io.github.dfa1.vortex.proto.DTypeProtos.Field.Builder.class); - } - - private int fieldTypeCase_ = 0; - @SuppressWarnings("serial") - private java.lang.Object fieldType_; - public enum FieldTypeCase - implements com.google.protobuf.Internal.EnumLite, - com.google.protobuf.AbstractMessage.InternalOneOfEnum { - NAME(1), - FIELDTYPE_NOT_SET(0); - private final int value; - private FieldTypeCase(int value) { - this.value = value; - } - /** - * @param value The number of the enum to look for. - * @return The enum associated with the given number. - * @deprecated Use {@link #forNumber(int)} instead. - */ - @java.lang.Deprecated - public static FieldTypeCase valueOf(int value) { - return forNumber(value); - } - - public static FieldTypeCase forNumber(int value) { - switch (value) { - case 1: return NAME; - case 0: return FIELDTYPE_NOT_SET; - default: return null; - } - } - public int getNumber() { - return this.value; - } - }; - - public FieldTypeCase - getFieldTypeCase() { - return FieldTypeCase.forNumber( - fieldTypeCase_); - } - - public static final int NAME_FIELD_NUMBER = 1; - /** - * string name = 1; - * @return Whether the name field is set. - */ - public boolean hasName() { - return fieldTypeCase_ == 1; - } - /** - * string name = 1; - * @return The name. - */ - public java.lang.String getName() { - if (fieldTypeCase_ != 1) { - return ""; - } - java.lang.Object ref = fieldType_; - if (ref instanceof java.lang.String) { - return (java.lang.String) ref; - } else { - com.google.protobuf.ByteString bs = - (com.google.protobuf.ByteString) ref; - java.lang.String s = bs.toStringUtf8(); - fieldType_ = s; - return s; - } - } - /** - * string name = 1; - * @return The bytes for name. - */ - public com.google.protobuf.ByteString - getNameBytes() { - if (fieldTypeCase_ != 1) { - return com.google.protobuf.ByteString.copyFromUtf8(""); - } - java.lang.Object ref = fieldType_; - if (ref instanceof java.lang.String) { - com.google.protobuf.ByteString b = - com.google.protobuf.ByteString.copyFromUtf8( - (java.lang.String) ref); - fieldType_ = b; - return b; - } else { - return (com.google.protobuf.ByteString) ref; - } - } - - private byte memoizedIsInitialized = -1; - @java.lang.Override - public final boolean isInitialized() { - byte isInitialized = memoizedIsInitialized; - if (isInitialized == 1) return true; - if (isInitialized == 0) return false; - - memoizedIsInitialized = 1; - return true; - } - - @java.lang.Override - public void writeTo(com.google.protobuf.CodedOutputStream output) - throws java.io.IOException { - if (fieldTypeCase_ == 1) { - com.google.protobuf.GeneratedMessage.writeString(output, 1, fieldType_); - } - getUnknownFields().writeTo(output); - } - private int computeSerializedSize_0() { - int size = 0; - if (fieldTypeCase_ == 1) { - size += com.google.protobuf.GeneratedMessage.computeStringSize(1, fieldType_); - } - return size; - } - @java.lang.Override - public int getSerializedSize() { - int size = memoizedSize; - if (size != -1) return size; - - size = 0; - size += computeSerializedSize_0(); - size += getUnknownFields().getSerializedSize(); - memoizedSize = size; - return size; - } - - @java.lang.Override - public boolean equals(final java.lang.Object obj) { - if (obj == this) { - return true; - } - if (!(obj instanceof io.github.dfa1.vortex.proto.DTypeProtos.Field)) { - return super.equals(obj); - } - io.github.dfa1.vortex.proto.DTypeProtos.Field other = (io.github.dfa1.vortex.proto.DTypeProtos.Field) obj; - - if (!getFieldTypeCase().equals(other.getFieldTypeCase())) return false; - switch (fieldTypeCase_) { - case 1: - if (!getName() - .equals(other.getName())) return false; - break; - case 0: - default: - } - if (!getUnknownFields().equals(other.getUnknownFields())) return false; - return true; - } - - @java.lang.Override - public int hashCode() { - if (memoizedHashCode != 0) { - return memoizedHashCode; - } - int hash = 41; - hash = (19 * hash) + getDescriptor().hashCode(); - switch (fieldTypeCase_) { - case 1: - hash = (37 * hash) + NAME_FIELD_NUMBER; - hash = (53 * hash) + getName().hashCode(); - break; - case 0: - default: - } - hash = (29 * hash) + getUnknownFields().hashCode(); - memoizedHashCode = hash; - return hash; - } - - public static io.github.dfa1.vortex.proto.DTypeProtos.Field parseFrom( - java.nio.ByteBuffer data) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data); - } - public static io.github.dfa1.vortex.proto.DTypeProtos.Field parseFrom( - java.nio.ByteBuffer data, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data, extensionRegistry); - } - public static io.github.dfa1.vortex.proto.DTypeProtos.Field parseFrom( - com.google.protobuf.ByteString data) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data); - } - public static io.github.dfa1.vortex.proto.DTypeProtos.Field parseFrom( - com.google.protobuf.ByteString data, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data, extensionRegistry); - } - public static io.github.dfa1.vortex.proto.DTypeProtos.Field parseFrom(byte[] data) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data); - } - public static io.github.dfa1.vortex.proto.DTypeProtos.Field parseFrom( - byte[] data, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data, extensionRegistry); - } - public static io.github.dfa1.vortex.proto.DTypeProtos.Field parseFrom(java.io.InputStream input) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseWithIOException(PARSER, input); - } - public static io.github.dfa1.vortex.proto.DTypeProtos.Field parseFrom( - java.io.InputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseWithIOException(PARSER, input, extensionRegistry); - } - - public static io.github.dfa1.vortex.proto.DTypeProtos.Field parseDelimitedFrom(java.io.InputStream input) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseDelimitedWithIOException(PARSER, input); - } - - public static io.github.dfa1.vortex.proto.DTypeProtos.Field parseDelimitedFrom( - java.io.InputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseDelimitedWithIOException(PARSER, input, extensionRegistry); - } - public static io.github.dfa1.vortex.proto.DTypeProtos.Field parseFrom( - com.google.protobuf.CodedInputStream input) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseWithIOException(PARSER, input); - } - public static io.github.dfa1.vortex.proto.DTypeProtos.Field parseFrom( - com.google.protobuf.CodedInputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseWithIOException(PARSER, input, extensionRegistry); - } - - @java.lang.Override - public Builder newBuilderForType() { return newBuilder(); } - public static Builder newBuilder() { - return DEFAULT_INSTANCE.toBuilder(); - } - public static Builder newBuilder(io.github.dfa1.vortex.proto.DTypeProtos.Field prototype) { - return DEFAULT_INSTANCE.toBuilder().mergeFrom(prototype); - } - @java.lang.Override - public Builder toBuilder() { - return this == DEFAULT_INSTANCE - ? new Builder() : new Builder().mergeFrom(this); - } - - @java.lang.Override - protected Builder newBuilderForType( - com.google.protobuf.GeneratedMessage.BuilderParent parent) { - Builder builder = new Builder(parent); - return builder; - } - /** - * Protobuf type {@code vortex.dtype.Field} - */ - public static final class Builder extends - com.google.protobuf.GeneratedMessage.Builder implements - // @@protoc_insertion_point(builder_implements:vortex.dtype.Field) - io.github.dfa1.vortex.proto.DTypeProtos.FieldOrBuilder { - public static final com.google.protobuf.Descriptors.Descriptor - getDescriptor() { - return io.github.dfa1.vortex.proto.DTypeProtos.internal_static_vortex_dtype_Field_descriptor; - } - - @java.lang.Override - protected com.google.protobuf.GeneratedMessage.FieldAccessorTable - internalGetFieldAccessorTable() { - return io.github.dfa1.vortex.proto.DTypeProtos.internal_static_vortex_dtype_Field_fieldAccessorTable - .ensureFieldAccessorsInitialized( - io.github.dfa1.vortex.proto.DTypeProtos.Field.class, io.github.dfa1.vortex.proto.DTypeProtos.Field.Builder.class); - } - - // Construct using io.github.dfa1.vortex.proto.DTypeProtos.Field.newBuilder() - private Builder() { - - } - - private Builder( - com.google.protobuf.GeneratedMessage.BuilderParent parent) { - super(parent); - - } - @java.lang.Override - public Builder clear() { - super.clear(); - bitField0_ = 0; - fieldTypeCase_ = 0; - fieldType_ = null; - return this; - } - - @java.lang.Override - public com.google.protobuf.Descriptors.Descriptor - getDescriptorForType() { - return io.github.dfa1.vortex.proto.DTypeProtos.internal_static_vortex_dtype_Field_descriptor; - } - - @java.lang.Override - public io.github.dfa1.vortex.proto.DTypeProtos.Field getDefaultInstanceForType() { - return io.github.dfa1.vortex.proto.DTypeProtos.Field.getDefaultInstance(); - } - - @java.lang.Override - public io.github.dfa1.vortex.proto.DTypeProtos.Field build() { - io.github.dfa1.vortex.proto.DTypeProtos.Field result = buildPartial(); - if (!result.isInitialized()) { - throw newUninitializedMessageException(result); - } - return result; - } - - @java.lang.Override - public io.github.dfa1.vortex.proto.DTypeProtos.Field buildPartial() { - io.github.dfa1.vortex.proto.DTypeProtos.Field result = new io.github.dfa1.vortex.proto.DTypeProtos.Field(this); - if (bitField0_ != 0) { buildPartial0(result); } - buildPartialOneofs(result); - onBuilt(); - return result; - } - - private void buildPartial0(io.github.dfa1.vortex.proto.DTypeProtos.Field result) { - int from_bitField0_ = bitField0_; - } - - private void buildPartialOneofs(io.github.dfa1.vortex.proto.DTypeProtos.Field result) { - result.fieldTypeCase_ = fieldTypeCase_; - result.fieldType_ = this.fieldType_; - } - - @java.lang.Override - public Builder mergeFrom(com.google.protobuf.Message other) { - if (other instanceof io.github.dfa1.vortex.proto.DTypeProtos.Field) { - return mergeFrom((io.github.dfa1.vortex.proto.DTypeProtos.Field)other); - } else { - super.mergeFrom(other); - return this; - } - } - - public Builder mergeFrom(io.github.dfa1.vortex.proto.DTypeProtos.Field other) { - if (other == io.github.dfa1.vortex.proto.DTypeProtos.Field.getDefaultInstance()) return this; - switch (other.getFieldTypeCase()) { - case NAME: { - fieldTypeCase_ = 1; - fieldType_ = other.fieldType_; - onChanged(); - break; - } - case FIELDTYPE_NOT_SET: { - break; - } - } - this.mergeUnknownFields(other.getUnknownFields()); - onChanged(); - return this; - } - - @java.lang.Override - public final boolean isInitialized() { - return true; - } - - @java.lang.Override - public Builder mergeFrom( - com.google.protobuf.CodedInputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws java.io.IOException { - if (extensionRegistry == null) { - throw new java.lang.NullPointerException(); - } - try { - boolean done = false; - while (!done) { - int tag = input.readTag(); - switch (tag) { - case 0: - done = true; - break; - case 10: { - fieldTypeCase_ = 1; - fieldType_ = input.readStringRequireUtf8(); - break; - } // case 10 - default: { - if (!super.parseUnknownField(input, extensionRegistry, tag)) { - done = true; // was an endgroup tag - } - break; - } // default: - } // switch (tag) - } // while (!done) - } catch (com.google.protobuf.InvalidProtocolBufferException e) { - throw e.unwrapIOException(); - } finally { - onChanged(); - } // finally - return this; - } - private int fieldTypeCase_ = 0; - private java.lang.Object fieldType_; - public FieldTypeCase - getFieldTypeCase() { - return FieldTypeCase.forNumber( - fieldTypeCase_); - } - - public Builder clearFieldType() { - fieldTypeCase_ = 0; - fieldType_ = null; - onChanged(); - return this; - } - - private int bitField0_; - - /** - * string name = 1; - * @return Whether the name field is set. - */ - @java.lang.Override - public boolean hasName() { - return fieldTypeCase_ == 1; - } - /** - * string name = 1; - * @return The name. - */ - @java.lang.Override - public java.lang.String getName() { - if (fieldTypeCase_ != 1) { - return ""; - } - java.lang.Object ref = fieldType_; - if (!(ref instanceof java.lang.String)) { - com.google.protobuf.ByteString bs = - (com.google.protobuf.ByteString) ref; - java.lang.String s = bs.toStringUtf8(); - fieldType_ = s; - return s; - } else { - return (java.lang.String) ref; - } - } - /** - * string name = 1; - * @return The bytes for name. - */ - @java.lang.Override - public com.google.protobuf.ByteString - getNameBytes() { - if (fieldTypeCase_ != 1) { - return com.google.protobuf.ByteString.copyFromUtf8( ""); - } - java.lang.Object ref = fieldType_; - if (ref instanceof String) { - com.google.protobuf.ByteString b = - com.google.protobuf.ByteString.copyFromUtf8( - (java.lang.String) ref); - fieldType_ = b; - return b; - } else { - return (com.google.protobuf.ByteString) ref; - } - } - /** - * string name = 1; - * @param value The name to set. - * @return This builder for chaining. - */ - public Builder setName( - java.lang.String value) { - if (value == null) { throw new NullPointerException(); } - fieldTypeCase_ = 1; - fieldType_ = value; - onChanged(); - return this; - } - /** - * string name = 1; - * @return This builder for chaining. - */ - public Builder clearName() { - if (fieldTypeCase_ == 1) { - fieldTypeCase_ = 0; - fieldType_ = null; - onChanged(); - } - return this; - } - /** - * string name = 1; - * @param value The bytes for name to set. - * @return This builder for chaining. - */ - public Builder setNameBytes( - com.google.protobuf.ByteString value) { - if (value == null) { throw new NullPointerException(); } - checkByteStringIsUtf8(value); - fieldTypeCase_ = 1; - fieldType_ = value; - onChanged(); - return this; - } - - // @@protoc_insertion_point(builder_scope:vortex.dtype.Field) - } - - // @@protoc_insertion_point(class_scope:vortex.dtype.Field) - private static final io.github.dfa1.vortex.proto.DTypeProtos.Field DEFAULT_INSTANCE; - static { - DEFAULT_INSTANCE = new io.github.dfa1.vortex.proto.DTypeProtos.Field(); - } - - public static io.github.dfa1.vortex.proto.DTypeProtos.Field getDefaultInstance() { - return DEFAULT_INSTANCE; - } - - private static final com.google.protobuf.Parser - PARSER = new com.google.protobuf.AbstractParser() { - @java.lang.Override - public Field parsePartialFrom( - com.google.protobuf.CodedInputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws com.google.protobuf.InvalidProtocolBufferException { - Builder builder = newBuilder(); - try { - builder.mergeFrom(input, extensionRegistry); - } catch (com.google.protobuf.InvalidProtocolBufferException e) { - throw e.setUnfinishedMessage(builder.buildPartial()); - } catch (com.google.protobuf.UninitializedMessageException e) { - throw e.asInvalidProtocolBufferException().setUnfinishedMessage(builder.buildPartial()); - } catch (java.io.IOException e) { - throw new com.google.protobuf.InvalidProtocolBufferException(e) - .setUnfinishedMessage(builder.buildPartial()); - } - return builder.buildPartial(); - } - }; - - public static com.google.protobuf.Parser parser() { - return PARSER; - } - - @java.lang.Override - public com.google.protobuf.Parser getParserForType() { - return PARSER; - } - - @java.lang.Override - public io.github.dfa1.vortex.proto.DTypeProtos.Field getDefaultInstanceForType() { - return DEFAULT_INSTANCE; - } - - } - - public interface FieldPathOrBuilder extends - // @@protoc_insertion_point(interface_extends:vortex.dtype.FieldPath) - com.google.protobuf.MessageOrBuilder { - - /** - * repeated .vortex.dtype.Field path = 1; - */ - java.util.List - getPathList(); - /** - * repeated .vortex.dtype.Field path = 1; - */ - io.github.dfa1.vortex.proto.DTypeProtos.Field getPath(int index); - /** - * repeated .vortex.dtype.Field path = 1; - */ - int getPathCount(); - /** - * repeated .vortex.dtype.Field path = 1; - */ - java.util.List - getPathOrBuilderList(); - /** - * repeated .vortex.dtype.Field path = 1; - */ - io.github.dfa1.vortex.proto.DTypeProtos.FieldOrBuilder getPathOrBuilder( - int index); - } - /** - * Protobuf type {@code vortex.dtype.FieldPath} - */ - public static final class FieldPath extends - com.google.protobuf.GeneratedMessage implements - // @@protoc_insertion_point(message_implements:vortex.dtype.FieldPath) - FieldPathOrBuilder { - private static final long serialVersionUID = 0L; - static { - com.google.protobuf.RuntimeVersion.validateProtobufGencodeVersion( - com.google.protobuf.RuntimeVersion.RuntimeDomain.PUBLIC, - /* major= */ 4, - /* minor= */ 35, - /* patch= */ 0, - /* suffix= */ "", - "FieldPath"); - } - // Use FieldPath.newBuilder() to construct. - private FieldPath(com.google.protobuf.GeneratedMessage.Builder builder) { - super(builder); - } - private FieldPath() { - path_ = java.util.Collections.emptyList(); - } - - public static final com.google.protobuf.Descriptors.Descriptor - getDescriptor() { - return io.github.dfa1.vortex.proto.DTypeProtos.internal_static_vortex_dtype_FieldPath_descriptor; - } - - @java.lang.Override - public com.google.protobuf.Descriptors.Descriptor getDescriptorForType() { - return io.github.dfa1.vortex.proto.DTypeProtos.internal_static_vortex_dtype_FieldPath_descriptor; - } - - @java.lang.Override - protected com.google.protobuf.GeneratedMessage.FieldAccessorTable - internalGetFieldAccessorTable() { - return io.github.dfa1.vortex.proto.DTypeProtos.internal_static_vortex_dtype_FieldPath_fieldAccessorTable - .ensureFieldAccessorsInitialized( - io.github.dfa1.vortex.proto.DTypeProtos.FieldPath.class, io.github.dfa1.vortex.proto.DTypeProtos.FieldPath.Builder.class); - } - - public static final int PATH_FIELD_NUMBER = 1; - @SuppressWarnings("serial") - private java.util.List path_; - /** - * repeated .vortex.dtype.Field path = 1; - */ - @java.lang.Override - public java.util.List getPathList() { - return path_; - } - /** - * repeated .vortex.dtype.Field path = 1; - */ - @java.lang.Override - public java.util.List - getPathOrBuilderList() { - return path_; - } - /** - * repeated .vortex.dtype.Field path = 1; - */ - @java.lang.Override - public int getPathCount() { - return path_.size(); - } - /** - * repeated .vortex.dtype.Field path = 1; - */ - @java.lang.Override - public io.github.dfa1.vortex.proto.DTypeProtos.Field getPath(int index) { - return path_.get(index); - } - /** - * repeated .vortex.dtype.Field path = 1; - */ - @java.lang.Override - public io.github.dfa1.vortex.proto.DTypeProtos.FieldOrBuilder getPathOrBuilder( - int index) { - return path_.get(index); - } - - private byte memoizedIsInitialized = -1; - @java.lang.Override - public final boolean isInitialized() { - byte isInitialized = memoizedIsInitialized; - if (isInitialized == 1) return true; - if (isInitialized == 0) return false; - - memoizedIsInitialized = 1; - return true; - } - - @java.lang.Override - public void writeTo(com.google.protobuf.CodedOutputStream output) - throws java.io.IOException { - for (int i = 0; i < path_.size(); i++) { - output.writeMessage(1, path_.get(i)); - } - getUnknownFields().writeTo(output); - } - private int computeSerializedSize_0() { - int size = 0; - - { - final int count = path_.size(); - for (int i = 0; i < count; i++) { - size += com.google.protobuf.CodedOutputStream - .computeMessageSizeNoTag(path_.get(i)); - } - size += 1 * count; - } - return size; - } - @java.lang.Override - public int getSerializedSize() { - int size = memoizedSize; - if (size != -1) return size; - - size = 0; - size += computeSerializedSize_0(); - size += getUnknownFields().getSerializedSize(); - memoizedSize = size; - return size; - } - - @java.lang.Override - public boolean equals(final java.lang.Object obj) { - if (obj == this) { - return true; - } - if (!(obj instanceof io.github.dfa1.vortex.proto.DTypeProtos.FieldPath)) { - return super.equals(obj); - } - io.github.dfa1.vortex.proto.DTypeProtos.FieldPath other = (io.github.dfa1.vortex.proto.DTypeProtos.FieldPath) obj; - - if (!getPathList() - .equals(other.getPathList())) return false; - if (!getUnknownFields().equals(other.getUnknownFields())) return false; - return true; - } - - @java.lang.Override - public int hashCode() { - if (memoizedHashCode != 0) { - return memoizedHashCode; - } - int hash = 41; - hash = (19 * hash) + getDescriptor().hashCode(); - if (getPathCount() > 0) { - hash = (37 * hash) + PATH_FIELD_NUMBER; - hash = (53 * hash) + getPathList().hashCode(); - } - hash = (29 * hash) + getUnknownFields().hashCode(); - memoizedHashCode = hash; - return hash; - } - - public static io.github.dfa1.vortex.proto.DTypeProtos.FieldPath parseFrom( - java.nio.ByteBuffer data) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data); - } - public static io.github.dfa1.vortex.proto.DTypeProtos.FieldPath parseFrom( - java.nio.ByteBuffer data, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data, extensionRegistry); - } - public static io.github.dfa1.vortex.proto.DTypeProtos.FieldPath parseFrom( - com.google.protobuf.ByteString data) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data); - } - public static io.github.dfa1.vortex.proto.DTypeProtos.FieldPath parseFrom( - com.google.protobuf.ByteString data, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data, extensionRegistry); - } - public static io.github.dfa1.vortex.proto.DTypeProtos.FieldPath parseFrom(byte[] data) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data); - } - public static io.github.dfa1.vortex.proto.DTypeProtos.FieldPath parseFrom( - byte[] data, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data, extensionRegistry); - } - public static io.github.dfa1.vortex.proto.DTypeProtos.FieldPath parseFrom(java.io.InputStream input) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseWithIOException(PARSER, input); - } - public static io.github.dfa1.vortex.proto.DTypeProtos.FieldPath parseFrom( - java.io.InputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseWithIOException(PARSER, input, extensionRegistry); - } - - public static io.github.dfa1.vortex.proto.DTypeProtos.FieldPath parseDelimitedFrom(java.io.InputStream input) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseDelimitedWithIOException(PARSER, input); - } - - public static io.github.dfa1.vortex.proto.DTypeProtos.FieldPath parseDelimitedFrom( - java.io.InputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseDelimitedWithIOException(PARSER, input, extensionRegistry); - } - public static io.github.dfa1.vortex.proto.DTypeProtos.FieldPath parseFrom( - com.google.protobuf.CodedInputStream input) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseWithIOException(PARSER, input); - } - public static io.github.dfa1.vortex.proto.DTypeProtos.FieldPath parseFrom( - com.google.protobuf.CodedInputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseWithIOException(PARSER, input, extensionRegistry); - } - - @java.lang.Override - public Builder newBuilderForType() { return newBuilder(); } - public static Builder newBuilder() { - return DEFAULT_INSTANCE.toBuilder(); - } - public static Builder newBuilder(io.github.dfa1.vortex.proto.DTypeProtos.FieldPath prototype) { - return DEFAULT_INSTANCE.toBuilder().mergeFrom(prototype); - } - @java.lang.Override - public Builder toBuilder() { - return this == DEFAULT_INSTANCE - ? new Builder() : new Builder().mergeFrom(this); - } - - @java.lang.Override - protected Builder newBuilderForType( - com.google.protobuf.GeneratedMessage.BuilderParent parent) { - Builder builder = new Builder(parent); - return builder; - } - /** - * Protobuf type {@code vortex.dtype.FieldPath} - */ - public static final class Builder extends - com.google.protobuf.GeneratedMessage.Builder implements - // @@protoc_insertion_point(builder_implements:vortex.dtype.FieldPath) - io.github.dfa1.vortex.proto.DTypeProtos.FieldPathOrBuilder { - public static final com.google.protobuf.Descriptors.Descriptor - getDescriptor() { - return io.github.dfa1.vortex.proto.DTypeProtos.internal_static_vortex_dtype_FieldPath_descriptor; - } - - @java.lang.Override - protected com.google.protobuf.GeneratedMessage.FieldAccessorTable - internalGetFieldAccessorTable() { - return io.github.dfa1.vortex.proto.DTypeProtos.internal_static_vortex_dtype_FieldPath_fieldAccessorTable - .ensureFieldAccessorsInitialized( - io.github.dfa1.vortex.proto.DTypeProtos.FieldPath.class, io.github.dfa1.vortex.proto.DTypeProtos.FieldPath.Builder.class); - } - - // Construct using io.github.dfa1.vortex.proto.DTypeProtos.FieldPath.newBuilder() - private Builder() { - - } - - private Builder( - com.google.protobuf.GeneratedMessage.BuilderParent parent) { - super(parent); - - } - @java.lang.Override - public Builder clear() { - super.clear(); - bitField0_ = 0; - if (pathBuilder_ == null) { - path_ = java.util.Collections.emptyList(); - } else { - path_ = null; - pathBuilder_.clear(); - } - bitField0_ = (bitField0_ & ~0x00000001); - return this; - } - - @java.lang.Override - public com.google.protobuf.Descriptors.Descriptor - getDescriptorForType() { - return io.github.dfa1.vortex.proto.DTypeProtos.internal_static_vortex_dtype_FieldPath_descriptor; - } - - @java.lang.Override - public io.github.dfa1.vortex.proto.DTypeProtos.FieldPath getDefaultInstanceForType() { - return io.github.dfa1.vortex.proto.DTypeProtos.FieldPath.getDefaultInstance(); - } - - @java.lang.Override - public io.github.dfa1.vortex.proto.DTypeProtos.FieldPath build() { - io.github.dfa1.vortex.proto.DTypeProtos.FieldPath result = buildPartial(); - if (!result.isInitialized()) { - throw newUninitializedMessageException(result); - } - return result; - } - - @java.lang.Override - public io.github.dfa1.vortex.proto.DTypeProtos.FieldPath buildPartial() { - io.github.dfa1.vortex.proto.DTypeProtos.FieldPath result = new io.github.dfa1.vortex.proto.DTypeProtos.FieldPath(this); - buildPartialRepeatedFields(result); - if (bitField0_ != 0) { buildPartial0(result); } - onBuilt(); - return result; - } - - private void buildPartialRepeatedFields(io.github.dfa1.vortex.proto.DTypeProtos.FieldPath result) { - if (pathBuilder_ == null) { - if (((bitField0_ & 0x00000001) != 0)) { - path_ = java.util.Collections.unmodifiableList(path_); - bitField0_ = (bitField0_ & ~0x00000001); - } - result.path_ = path_; - } else { - result.path_ = pathBuilder_.build(); - } - } - - private void buildPartial0(io.github.dfa1.vortex.proto.DTypeProtos.FieldPath result) { - int from_bitField0_ = bitField0_; - } - - @java.lang.Override - public Builder mergeFrom(com.google.protobuf.Message other) { - if (other instanceof io.github.dfa1.vortex.proto.DTypeProtos.FieldPath) { - return mergeFrom((io.github.dfa1.vortex.proto.DTypeProtos.FieldPath)other); - } else { - super.mergeFrom(other); - return this; - } - } - - public Builder mergeFrom(io.github.dfa1.vortex.proto.DTypeProtos.FieldPath other) { - if (other == io.github.dfa1.vortex.proto.DTypeProtos.FieldPath.getDefaultInstance()) return this; - if (pathBuilder_ == null) { - if (!other.path_.isEmpty()) { - if (path_.isEmpty()) { - path_ = other.path_; - bitField0_ = (bitField0_ & ~0x00000001); - } else { - ensurePathIsMutable(); - path_.addAll(other.path_); - } - onChanged(); - } - } else { - if (!other.path_.isEmpty()) { - if (pathBuilder_.isEmpty()) { - pathBuilder_.dispose(); - pathBuilder_ = null; - path_ = other.path_; - bitField0_ = (bitField0_ & ~0x00000001); - pathBuilder_ = - com.google.protobuf.GeneratedMessage.alwaysUseFieldBuilders ? - internalGetPathFieldBuilder() : null; - } else { - pathBuilder_.addAllMessages(other.path_); - } - } - } - this.mergeUnknownFields(other.getUnknownFields()); - onChanged(); - return this; - } - - @java.lang.Override - public final boolean isInitialized() { - return true; - } - - @java.lang.Override - public Builder mergeFrom( - com.google.protobuf.CodedInputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws java.io.IOException { - if (extensionRegistry == null) { - throw new java.lang.NullPointerException(); - } - try { - boolean done = false; - while (!done) { - int tag = input.readTag(); - switch (tag) { - case 0: - done = true; - break; - case 10: { - io.github.dfa1.vortex.proto.DTypeProtos.Field m = - input.readMessage( - io.github.dfa1.vortex.proto.DTypeProtos.Field.parser(), - extensionRegistry); - if (pathBuilder_ == null) { - ensurePathIsMutable(); - path_.add(m); - } else { - pathBuilder_.addMessage(m); - } - break; - } // case 10 - default: { - if (!super.parseUnknownField(input, extensionRegistry, tag)) { - done = true; // was an endgroup tag - } - break; - } // default: - } // switch (tag) - } // while (!done) - } catch (com.google.protobuf.InvalidProtocolBufferException e) { - throw e.unwrapIOException(); - } finally { - onChanged(); - } // finally - return this; - } - private int bitField0_; - - private java.util.List path_ = - java.util.Collections.emptyList(); - private void ensurePathIsMutable() { - if (!((bitField0_ & 0x00000001) != 0)) { - path_ = new java.util.ArrayList(path_); - bitField0_ |= 0x00000001; - } - } - - private com.google.protobuf.RepeatedFieldBuilder< - io.github.dfa1.vortex.proto.DTypeProtos.Field, io.github.dfa1.vortex.proto.DTypeProtos.Field.Builder, io.github.dfa1.vortex.proto.DTypeProtos.FieldOrBuilder> pathBuilder_; - - /** - * repeated .vortex.dtype.Field path = 1; - */ - public java.util.List getPathList() { - if (pathBuilder_ == null) { - return java.util.Collections.unmodifiableList(path_); - } else { - return pathBuilder_.getMessageList(); - } - } - /** - * repeated .vortex.dtype.Field path = 1; - */ - public int getPathCount() { - if (pathBuilder_ == null) { - return path_.size(); - } else { - return pathBuilder_.getCount(); - } - } - /** - * repeated .vortex.dtype.Field path = 1; - */ - public io.github.dfa1.vortex.proto.DTypeProtos.Field getPath(int index) { - if (pathBuilder_ == null) { - return path_.get(index); - } else { - return pathBuilder_.getMessage(index); - } - } - /** - * repeated .vortex.dtype.Field path = 1; - */ - public Builder setPath( - int index, io.github.dfa1.vortex.proto.DTypeProtos.Field value) { - if (pathBuilder_ == null) { - if (value == null) { - throw new NullPointerException(); - } - ensurePathIsMutable(); - path_.set(index, value); - onChanged(); - } else { - pathBuilder_.setMessage(index, value); - } - return this; - } - /** - * repeated .vortex.dtype.Field path = 1; - */ - public Builder setPath( - int index, io.github.dfa1.vortex.proto.DTypeProtos.Field.Builder builderForValue) { - if (pathBuilder_ == null) { - ensurePathIsMutable(); - path_.set(index, builderForValue.build()); - onChanged(); - } else { - pathBuilder_.setMessage(index, builderForValue.build()); - } - return this; - } - /** - * repeated .vortex.dtype.Field path = 1; - */ - public Builder addPath(io.github.dfa1.vortex.proto.DTypeProtos.Field value) { - if (pathBuilder_ == null) { - if (value == null) { - throw new NullPointerException(); - } - ensurePathIsMutable(); - path_.add(value); - onChanged(); - } else { - pathBuilder_.addMessage(value); - } - return this; - } - /** - * repeated .vortex.dtype.Field path = 1; - */ - public Builder addPath( - int index, io.github.dfa1.vortex.proto.DTypeProtos.Field value) { - if (pathBuilder_ == null) { - if (value == null) { - throw new NullPointerException(); - } - ensurePathIsMutable(); - path_.add(index, value); - onChanged(); - } else { - pathBuilder_.addMessage(index, value); - } - return this; - } - /** - * repeated .vortex.dtype.Field path = 1; - */ - public Builder addPath( - io.github.dfa1.vortex.proto.DTypeProtos.Field.Builder builderForValue) { - if (pathBuilder_ == null) { - ensurePathIsMutable(); - path_.add(builderForValue.build()); - onChanged(); - } else { - pathBuilder_.addMessage(builderForValue.build()); - } - return this; - } - /** - * repeated .vortex.dtype.Field path = 1; - */ - public Builder addPath( - int index, io.github.dfa1.vortex.proto.DTypeProtos.Field.Builder builderForValue) { - if (pathBuilder_ == null) { - ensurePathIsMutable(); - path_.add(index, builderForValue.build()); - onChanged(); - } else { - pathBuilder_.addMessage(index, builderForValue.build()); - } - return this; - } - /** - * repeated .vortex.dtype.Field path = 1; - */ - public Builder addAllPath( - java.lang.Iterable values) { - if (pathBuilder_ == null) { - ensurePathIsMutable(); - com.google.protobuf.AbstractMessageLite.Builder.addAll( - values, path_); - onChanged(); - } else { - pathBuilder_.addAllMessages(values); - } - return this; - } - /** - * repeated .vortex.dtype.Field path = 1; - */ - public Builder clearPath() { - if (pathBuilder_ == null) { - path_ = java.util.Collections.emptyList(); - bitField0_ = (bitField0_ & ~0x00000001); - onChanged(); - } else { - pathBuilder_.clear(); - } - return this; - } - /** - * repeated .vortex.dtype.Field path = 1; - */ - public Builder removePath(int index) { - if (pathBuilder_ == null) { - ensurePathIsMutable(); - path_.remove(index); - onChanged(); - } else { - pathBuilder_.remove(index); - } - return this; - } - /** - * repeated .vortex.dtype.Field path = 1; - */ - public io.github.dfa1.vortex.proto.DTypeProtos.Field.Builder getPathBuilder( - int index) { - return internalGetPathFieldBuilder().getBuilder(index); - } - /** - * repeated .vortex.dtype.Field path = 1; - */ - public io.github.dfa1.vortex.proto.DTypeProtos.FieldOrBuilder getPathOrBuilder( - int index) { - if (pathBuilder_ == null) { - return path_.get(index); } else { - return pathBuilder_.getMessageOrBuilder(index); - } - } - /** - * repeated .vortex.dtype.Field path = 1; - */ - public java.util.List - getPathOrBuilderList() { - if (pathBuilder_ != null) { - return pathBuilder_.getMessageOrBuilderList(); - } else { - return java.util.Collections.unmodifiableList(path_); - } - } - /** - * repeated .vortex.dtype.Field path = 1; - */ - public io.github.dfa1.vortex.proto.DTypeProtos.Field.Builder addPathBuilder() { - return internalGetPathFieldBuilder().addBuilder( - io.github.dfa1.vortex.proto.DTypeProtos.Field.getDefaultInstance()); - } - /** - * repeated .vortex.dtype.Field path = 1; - */ - public io.github.dfa1.vortex.proto.DTypeProtos.Field.Builder addPathBuilder( - int index) { - return internalGetPathFieldBuilder().addBuilder( - index, io.github.dfa1.vortex.proto.DTypeProtos.Field.getDefaultInstance()); - } - /** - * repeated .vortex.dtype.Field path = 1; - */ - public java.util.List - getPathBuilderList() { - return internalGetPathFieldBuilder().getBuilderList(); - } - private com.google.protobuf.RepeatedFieldBuilder< - io.github.dfa1.vortex.proto.DTypeProtos.Field, io.github.dfa1.vortex.proto.DTypeProtos.Field.Builder, io.github.dfa1.vortex.proto.DTypeProtos.FieldOrBuilder> - internalGetPathFieldBuilder() { - if (pathBuilder_ == null) { - pathBuilder_ = new com.google.protobuf.RepeatedFieldBuilder< - io.github.dfa1.vortex.proto.DTypeProtos.Field, io.github.dfa1.vortex.proto.DTypeProtos.Field.Builder, io.github.dfa1.vortex.proto.DTypeProtos.FieldOrBuilder>( - path_, - ((bitField0_ & 0x00000001) != 0), - getParentForChildren(), - isClean()); - path_ = null; - } - return pathBuilder_; - } - - // @@protoc_insertion_point(builder_scope:vortex.dtype.FieldPath) - } - - // @@protoc_insertion_point(class_scope:vortex.dtype.FieldPath) - private static final io.github.dfa1.vortex.proto.DTypeProtos.FieldPath DEFAULT_INSTANCE; - static { - DEFAULT_INSTANCE = new io.github.dfa1.vortex.proto.DTypeProtos.FieldPath(); - } - - public static io.github.dfa1.vortex.proto.DTypeProtos.FieldPath getDefaultInstance() { - return DEFAULT_INSTANCE; - } - - private static final com.google.protobuf.Parser - PARSER = new com.google.protobuf.AbstractParser() { - @java.lang.Override - public FieldPath parsePartialFrom( - com.google.protobuf.CodedInputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws com.google.protobuf.InvalidProtocolBufferException { - Builder builder = newBuilder(); - try { - builder.mergeFrom(input, extensionRegistry); - } catch (com.google.protobuf.InvalidProtocolBufferException e) { - throw e.setUnfinishedMessage(builder.buildPartial()); - } catch (com.google.protobuf.UninitializedMessageException e) { - throw e.asInvalidProtocolBufferException().setUnfinishedMessage(builder.buildPartial()); - } catch (java.io.IOException e) { - throw new com.google.protobuf.InvalidProtocolBufferException(e) - .setUnfinishedMessage(builder.buildPartial()); - } - return builder.buildPartial(); - } - }; - - public static com.google.protobuf.Parser parser() { - return PARSER; - } - - @java.lang.Override - public com.google.protobuf.Parser getParserForType() { - return PARSER; - } - - @java.lang.Override - public io.github.dfa1.vortex.proto.DTypeProtos.FieldPath getDefaultInstanceForType() { - return DEFAULT_INSTANCE; - } - - } - - private static final com.google.protobuf.Descriptors.Descriptor - internal_static_vortex_dtype_Null_descriptor; - private static final - com.google.protobuf.GeneratedMessage.FieldAccessorTable - internal_static_vortex_dtype_Null_fieldAccessorTable; - private static final com.google.protobuf.Descriptors.Descriptor - internal_static_vortex_dtype_Bool_descriptor; - private static final - com.google.protobuf.GeneratedMessage.FieldAccessorTable - internal_static_vortex_dtype_Bool_fieldAccessorTable; - private static final com.google.protobuf.Descriptors.Descriptor - internal_static_vortex_dtype_Primitive_descriptor; - private static final - com.google.protobuf.GeneratedMessage.FieldAccessorTable - internal_static_vortex_dtype_Primitive_fieldAccessorTable; - private static final com.google.protobuf.Descriptors.Descriptor - internal_static_vortex_dtype_Decimal_descriptor; - private static final - com.google.protobuf.GeneratedMessage.FieldAccessorTable - internal_static_vortex_dtype_Decimal_fieldAccessorTable; - private static final com.google.protobuf.Descriptors.Descriptor - internal_static_vortex_dtype_Utf8_descriptor; - private static final - com.google.protobuf.GeneratedMessage.FieldAccessorTable - internal_static_vortex_dtype_Utf8_fieldAccessorTable; - private static final com.google.protobuf.Descriptors.Descriptor - internal_static_vortex_dtype_Binary_descriptor; - private static final - com.google.protobuf.GeneratedMessage.FieldAccessorTable - internal_static_vortex_dtype_Binary_fieldAccessorTable; - private static final com.google.protobuf.Descriptors.Descriptor - internal_static_vortex_dtype_Struct_descriptor; - private static final - com.google.protobuf.GeneratedMessage.FieldAccessorTable - internal_static_vortex_dtype_Struct_fieldAccessorTable; - private static final com.google.protobuf.Descriptors.Descriptor - internal_static_vortex_dtype_List_descriptor; - private static final - com.google.protobuf.GeneratedMessage.FieldAccessorTable - internal_static_vortex_dtype_List_fieldAccessorTable; - private static final com.google.protobuf.Descriptors.Descriptor - internal_static_vortex_dtype_FixedSizeList_descriptor; - private static final - com.google.protobuf.GeneratedMessage.FieldAccessorTable - internal_static_vortex_dtype_FixedSizeList_fieldAccessorTable; - private static final com.google.protobuf.Descriptors.Descriptor - internal_static_vortex_dtype_Extension_descriptor; - private static final - com.google.protobuf.GeneratedMessage.FieldAccessorTable - internal_static_vortex_dtype_Extension_fieldAccessorTable; - private static final com.google.protobuf.Descriptors.Descriptor - internal_static_vortex_dtype_Variant_descriptor; - private static final - com.google.protobuf.GeneratedMessage.FieldAccessorTable - internal_static_vortex_dtype_Variant_fieldAccessorTable; - private static final com.google.protobuf.Descriptors.Descriptor - internal_static_vortex_dtype_Union_descriptor; - private static final - com.google.protobuf.GeneratedMessage.FieldAccessorTable - internal_static_vortex_dtype_Union_fieldAccessorTable; - private static final com.google.protobuf.Descriptors.Descriptor - internal_static_vortex_dtype_DType_descriptor; - private static final - com.google.protobuf.GeneratedMessage.FieldAccessorTable - internal_static_vortex_dtype_DType_fieldAccessorTable; - private static final com.google.protobuf.Descriptors.Descriptor - internal_static_vortex_dtype_Field_descriptor; - private static final - com.google.protobuf.GeneratedMessage.FieldAccessorTable - internal_static_vortex_dtype_Field_fieldAccessorTable; - private static final com.google.protobuf.Descriptors.Descriptor - internal_static_vortex_dtype_FieldPath_descriptor; - private static final - com.google.protobuf.GeneratedMessage.FieldAccessorTable - internal_static_vortex_dtype_FieldPath_fieldAccessorTable; - - public static com.google.protobuf.Descriptors.FileDescriptor - getDescriptor() { - return descriptor; - } - private static final com.google.protobuf.Descriptors.FileDescriptor - descriptor; - static { - java.lang.String[] descriptorData = { - "\n\013dtype.proto\022\014vortex.dtype\"\006\n\004Null\"\030\n\004B" + - "ool\022\020\n\010nullable\030\001 \001(\010\"@\n\tPrimitive\022!\n\004ty" + - "pe\030\001 \001(\0162\023.vortex.dtype.PType\022\020\n\010nullabl" + - "e\030\002 \001(\010\"=\n\007Decimal\022\021\n\tprecision\030\001 \001(\r\022\r\n" + - "\005scale\030\002 \001(\005\022\020\n\010nullable\030\003 \001(\010\"\030\n\004Utf8\022\020" + - "\n\010nullable\030\001 \001(\010\"\032\n\006Binary\022\020\n\010nullable\030\001" + - " \001(\010\"N\n\006Struct\022\r\n\005names\030\001 \003(\t\022#\n\006dtypes\030" + - "\002 \003(\0132\023.vortex.dtype.DType\022\020\n\010nullable\030\003" + - " \001(\010\"C\n\004List\022)\n\014element_type\030\001 \001(\0132\023.vor" + - "tex.dtype.DType\022\020\n\010nullable\030\002 \001(\010\"Z\n\rFix" + - "edSizeList\022)\n\014element_type\030\001 \001(\0132\023.vorte" + - "x.dtype.DType\022\014\n\004size\030\002 \001(\r\022\020\n\010nullable\030" + - "\003 \001(\010\"g\n\tExtension\022\n\n\002id\030\001 \001(\t\022*\n\rstorag" + - "e_dtype\030\002 \001(\0132\023.vortex.dtype.DType\022\025\n\010me" + - "tadata\030\003 \001(\014H\000\210\001\001B\013\n\t_metadata\"\033\n\007Varian" + - "t\022\020\n\010nullable\030\001 \001(\010\"\031\n\005Union\022\020\n\010nullable" + - "\030\004 \001(\010\"\203\004\n\005DType\022\"\n\004null\030\001 \001(\0132\022.vortex." + - "dtype.NullH\000\022\"\n\004bool\030\002 \001(\0132\022.vortex.dtyp" + - "e.BoolH\000\022,\n\tprimitive\030\003 \001(\0132\027.vortex.dty" + - "pe.PrimitiveH\000\022(\n\007decimal\030\004 \001(\0132\025.vortex" + - ".dtype.DecimalH\000\022\"\n\004utf8\030\005 \001(\0132\022.vortex." + - "dtype.Utf8H\000\022&\n\006binary\030\006 \001(\0132\024.vortex.dt" + - "ype.BinaryH\000\022&\n\006struct\030\007 \001(\0132\024.vortex.dt" + - "ype.StructH\000\022\"\n\004list\030\010 \001(\0132\022.vortex.dtyp" + - "e.ListH\000\022,\n\textension\030\t \001(\0132\027.vortex.dty" + - "pe.ExtensionH\000\0226\n\017fixed_size_list\030\n \001(\0132" + - "\033.vortex.dtype.FixedSizeListH\000\022(\n\007varian" + - "t\030\013 \001(\0132\025.vortex.dtype.VariantH\000\022$\n\005unio" + - "n\030\014 \001(\0132\023.vortex.dtype.UnionH\000B\014\n\ndtype_" + - "type\"%\n\005Field\022\016\n\004name\030\001 \001(\tH\000B\014\n\nfield_t" + - "ype\".\n\tFieldPath\022!\n\004path\030\001 \003(\0132\023.vortex." + - "dtype.Field*h\n\005PType\022\006\n\002U8\020\000\022\007\n\003U16\020\001\022\007\n" + - "\003U32\020\002\022\007\n\003U64\020\003\022\006\n\002I8\020\004\022\007\n\003I16\020\005\022\007\n\003I32\020" + - "\006\022\007\n\003I64\020\007\022\007\n\003F16\020\010\022\007\n\003F32\020\t\022\007\n\003F64\020\nB*\n" + - "\033io.github.dfa1.vortex.protoB\013DTypeProto" + - "sb\006proto3" - }; - descriptor = com.google.protobuf.Descriptors.FileDescriptor - .internalBuildGeneratedFileFrom(descriptorData, - new com.google.protobuf.Descriptors.FileDescriptor[] { - }); - internal_static_vortex_dtype_Null_descriptor = - getDescriptor().getMessageType(0); - internal_static_vortex_dtype_Null_fieldAccessorTable = new - com.google.protobuf.GeneratedMessage.FieldAccessorTable( - internal_static_vortex_dtype_Null_descriptor, - new java.lang.String[] { }); - internal_static_vortex_dtype_Bool_descriptor = - getDescriptor().getMessageType(1); - internal_static_vortex_dtype_Bool_fieldAccessorTable = new - com.google.protobuf.GeneratedMessage.FieldAccessorTable( - internal_static_vortex_dtype_Bool_descriptor, - new java.lang.String[] { "Nullable", }); - internal_static_vortex_dtype_Primitive_descriptor = - getDescriptor().getMessageType(2); - internal_static_vortex_dtype_Primitive_fieldAccessorTable = new - com.google.protobuf.GeneratedMessage.FieldAccessorTable( - internal_static_vortex_dtype_Primitive_descriptor, - new java.lang.String[] { "Type", "Nullable", }); - internal_static_vortex_dtype_Decimal_descriptor = - getDescriptor().getMessageType(3); - internal_static_vortex_dtype_Decimal_fieldAccessorTable = new - com.google.protobuf.GeneratedMessage.FieldAccessorTable( - internal_static_vortex_dtype_Decimal_descriptor, - new java.lang.String[] { "Precision", "Scale", "Nullable", }); - internal_static_vortex_dtype_Utf8_descriptor = - getDescriptor().getMessageType(4); - internal_static_vortex_dtype_Utf8_fieldAccessorTable = new - com.google.protobuf.GeneratedMessage.FieldAccessorTable( - internal_static_vortex_dtype_Utf8_descriptor, - new java.lang.String[] { "Nullable", }); - internal_static_vortex_dtype_Binary_descriptor = - getDescriptor().getMessageType(5); - internal_static_vortex_dtype_Binary_fieldAccessorTable = new - com.google.protobuf.GeneratedMessage.FieldAccessorTable( - internal_static_vortex_dtype_Binary_descriptor, - new java.lang.String[] { "Nullable", }); - internal_static_vortex_dtype_Struct_descriptor = - getDescriptor().getMessageType(6); - internal_static_vortex_dtype_Struct_fieldAccessorTable = new - com.google.protobuf.GeneratedMessage.FieldAccessorTable( - internal_static_vortex_dtype_Struct_descriptor, - new java.lang.String[] { "Names", "Dtypes", "Nullable", }); - internal_static_vortex_dtype_List_descriptor = - getDescriptor().getMessageType(7); - internal_static_vortex_dtype_List_fieldAccessorTable = new - com.google.protobuf.GeneratedMessage.FieldAccessorTable( - internal_static_vortex_dtype_List_descriptor, - new java.lang.String[] { "ElementType", "Nullable", }); - internal_static_vortex_dtype_FixedSizeList_descriptor = - getDescriptor().getMessageType(8); - internal_static_vortex_dtype_FixedSizeList_fieldAccessorTable = new - com.google.protobuf.GeneratedMessage.FieldAccessorTable( - internal_static_vortex_dtype_FixedSizeList_descriptor, - new java.lang.String[] { "ElementType", "Size", "Nullable", }); - internal_static_vortex_dtype_Extension_descriptor = - getDescriptor().getMessageType(9); - internal_static_vortex_dtype_Extension_fieldAccessorTable = new - com.google.protobuf.GeneratedMessage.FieldAccessorTable( - internal_static_vortex_dtype_Extension_descriptor, - new java.lang.String[] { "Id", "StorageDtype", "Metadata", }); - internal_static_vortex_dtype_Variant_descriptor = - getDescriptor().getMessageType(10); - internal_static_vortex_dtype_Variant_fieldAccessorTable = new - com.google.protobuf.GeneratedMessage.FieldAccessorTable( - internal_static_vortex_dtype_Variant_descriptor, - new java.lang.String[] { "Nullable", }); - internal_static_vortex_dtype_Union_descriptor = - getDescriptor().getMessageType(11); - internal_static_vortex_dtype_Union_fieldAccessorTable = new - com.google.protobuf.GeneratedMessage.FieldAccessorTable( - internal_static_vortex_dtype_Union_descriptor, - new java.lang.String[] { "Nullable", }); - internal_static_vortex_dtype_DType_descriptor = - getDescriptor().getMessageType(12); - internal_static_vortex_dtype_DType_fieldAccessorTable = new - com.google.protobuf.GeneratedMessage.FieldAccessorTable( - internal_static_vortex_dtype_DType_descriptor, - new java.lang.String[] { "Null", "Bool", "Primitive", "Decimal", "Utf8", "Binary", "Struct", "List", "Extension", "FixedSizeList", "Variant", "Union", "DtypeType", }); - internal_static_vortex_dtype_Field_descriptor = - getDescriptor().getMessageType(13); - internal_static_vortex_dtype_Field_fieldAccessorTable = new - com.google.protobuf.GeneratedMessage.FieldAccessorTable( - internal_static_vortex_dtype_Field_descriptor, - new java.lang.String[] { "Name", "FieldType", }); - internal_static_vortex_dtype_FieldPath_descriptor = - getDescriptor().getMessageType(14); - internal_static_vortex_dtype_FieldPath_fieldAccessorTable = new - com.google.protobuf.GeneratedMessage.FieldAccessorTable( - internal_static_vortex_dtype_FieldPath_descriptor, - new java.lang.String[] { "Path", }); - descriptor.resolveAllFeaturesImmutable(); - } - - // @@protoc_insertion_point(outer_class_scope) -} diff --git a/core/src/main/java/io/github/dfa1/vortex/proto/EncodingProtos.java b/core/src/main/java/io/github/dfa1/vortex/proto/EncodingProtos.java deleted file mode 100644 index 1c92f32..0000000 --- a/core/src/main/java/io/github/dfa1/vortex/proto/EncodingProtos.java +++ /dev/null @@ -1,15206 +0,0 @@ -// Generated by the protocol buffer compiler. DO NOT EDIT! -// NO CHECKED-IN PROTOBUF GENCODE -// source: encodings.proto -// Protobuf Java Version: 4.35.0 - -package io.github.dfa1.vortex.proto; - -@com.google.protobuf.Generated -public final class EncodingProtos extends com.google.protobuf.GeneratedFile { - private EncodingProtos() {} - static { - com.google.protobuf.RuntimeVersion.validateProtobufGencodeVersion( - com.google.protobuf.RuntimeVersion.RuntimeDomain.PUBLIC, - /* major= */ 4, - /* minor= */ 35, - /* patch= */ 0, - /* suffix= */ "", - "EncodingProtos"); - } - public static void registerAllExtensions( - com.google.protobuf.ExtensionRegistryLite registry) { - } - - public static void registerAllExtensions( - com.google.protobuf.ExtensionRegistry registry) { - registerAllExtensions( - (com.google.protobuf.ExtensionRegistryLite) registry); - } - public interface PatchesMetadataOrBuilder extends - // @@protoc_insertion_point(interface_extends:vortex.encodings.PatchesMetadata) - com.google.protobuf.MessageOrBuilder { - - /** - * uint64 len = 1; - * @return The len. - */ - long getLen(); - - /** - * uint64 offset = 2; - * @return The offset. - */ - long getOffset(); - - /** - * .vortex.dtype.PType indices_ptype = 3; - * @return The enum numeric value on the wire for indicesPtype. - */ - int getIndicesPtypeValue(); - /** - * .vortex.dtype.PType indices_ptype = 3; - * @return The indicesPtype. - */ - io.github.dfa1.vortex.proto.DTypeProtos.PType getIndicesPtype(); - - /** - * optional uint64 chunk_offsets_len = 4; - * @return Whether the chunkOffsetsLen field is set. - */ - boolean hasChunkOffsetsLen(); - /** - * optional uint64 chunk_offsets_len = 4; - * @return The chunkOffsetsLen. - */ - long getChunkOffsetsLen(); - - /** - * optional .vortex.dtype.PType chunk_offsets_ptype = 5; - * @return Whether the chunkOffsetsPtype field is set. - */ - boolean hasChunkOffsetsPtype(); - /** - * optional .vortex.dtype.PType chunk_offsets_ptype = 5; - * @return The enum numeric value on the wire for chunkOffsetsPtype. - */ - int getChunkOffsetsPtypeValue(); - /** - * optional .vortex.dtype.PType chunk_offsets_ptype = 5; - * @return The chunkOffsetsPtype. - */ - io.github.dfa1.vortex.proto.DTypeProtos.PType getChunkOffsetsPtype(); - - /** - * optional uint64 offset_within_chunk = 6; - * @return Whether the offsetWithinChunk field is set. - */ - boolean hasOffsetWithinChunk(); - /** - * optional uint64 offset_within_chunk = 6; - * @return The offsetWithinChunk. - */ - long getOffsetWithinChunk(); - } - /** - * Protobuf type {@code vortex.encodings.PatchesMetadata} - */ - public static final class PatchesMetadata extends - com.google.protobuf.GeneratedMessage implements - // @@protoc_insertion_point(message_implements:vortex.encodings.PatchesMetadata) - PatchesMetadataOrBuilder { - private static final long serialVersionUID = 0L; - static { - com.google.protobuf.RuntimeVersion.validateProtobufGencodeVersion( - com.google.protobuf.RuntimeVersion.RuntimeDomain.PUBLIC, - /* major= */ 4, - /* minor= */ 35, - /* patch= */ 0, - /* suffix= */ "", - "PatchesMetadata"); - } - // Use PatchesMetadata.newBuilder() to construct. - private PatchesMetadata(com.google.protobuf.GeneratedMessage.Builder builder) { - super(builder); - } - private PatchesMetadata() { - indicesPtype_ = 0; - chunkOffsetsPtype_ = 0; - } - - public static final com.google.protobuf.Descriptors.Descriptor - getDescriptor() { - return io.github.dfa1.vortex.proto.EncodingProtos.internal_static_vortex_encodings_PatchesMetadata_descriptor; - } - - @java.lang.Override - public com.google.protobuf.Descriptors.Descriptor getDescriptorForType() { - return io.github.dfa1.vortex.proto.EncodingProtos.internal_static_vortex_encodings_PatchesMetadata_descriptor; - } - - @java.lang.Override - protected com.google.protobuf.GeneratedMessage.FieldAccessorTable - internalGetFieldAccessorTable() { - return io.github.dfa1.vortex.proto.EncodingProtos.internal_static_vortex_encodings_PatchesMetadata_fieldAccessorTable - .ensureFieldAccessorsInitialized( - io.github.dfa1.vortex.proto.EncodingProtos.PatchesMetadata.class, io.github.dfa1.vortex.proto.EncodingProtos.PatchesMetadata.Builder.class); - } - - private int bitField0_; - public static final int LEN_FIELD_NUMBER = 1; - private long len_ = 0L; - /** - * uint64 len = 1; - * @return The len. - */ - @java.lang.Override - public long getLen() { - return len_; - } - - public static final int OFFSET_FIELD_NUMBER = 2; - private long offset_ = 0L; - /** - * uint64 offset = 2; - * @return The offset. - */ - @java.lang.Override - public long getOffset() { - return offset_; - } - - public static final int INDICES_PTYPE_FIELD_NUMBER = 3; - private int indicesPtype_ = 0; - /** - * .vortex.dtype.PType indices_ptype = 3; - * @return The enum numeric value on the wire for indicesPtype. - */ - @java.lang.Override public int getIndicesPtypeValue() { - return indicesPtype_; - } - /** - * .vortex.dtype.PType indices_ptype = 3; - * @return The indicesPtype. - */ - @java.lang.Override public io.github.dfa1.vortex.proto.DTypeProtos.PType getIndicesPtype() { - io.github.dfa1.vortex.proto.DTypeProtos.PType result = io.github.dfa1.vortex.proto.DTypeProtos.PType.forNumber(indicesPtype_); - return result == null ? io.github.dfa1.vortex.proto.DTypeProtos.PType.UNRECOGNIZED : result; - } - - public static final int CHUNK_OFFSETS_LEN_FIELD_NUMBER = 4; - private long chunkOffsetsLen_ = 0L; - /** - * optional uint64 chunk_offsets_len = 4; - * @return Whether the chunkOffsetsLen field is set. - */ - @java.lang.Override - public boolean hasChunkOffsetsLen() { - return ((bitField0_ & 0x00000001) != 0); - } - /** - * optional uint64 chunk_offsets_len = 4; - * @return The chunkOffsetsLen. - */ - @java.lang.Override - public long getChunkOffsetsLen() { - return chunkOffsetsLen_; - } - - public static final int CHUNK_OFFSETS_PTYPE_FIELD_NUMBER = 5; - private int chunkOffsetsPtype_ = 0; - /** - * optional .vortex.dtype.PType chunk_offsets_ptype = 5; - * @return Whether the chunkOffsetsPtype field is set. - */ - @java.lang.Override public boolean hasChunkOffsetsPtype() { - return ((bitField0_ & 0x00000002) != 0); - } - /** - * optional .vortex.dtype.PType chunk_offsets_ptype = 5; - * @return The enum numeric value on the wire for chunkOffsetsPtype. - */ - @java.lang.Override public int getChunkOffsetsPtypeValue() { - return chunkOffsetsPtype_; - } - /** - * optional .vortex.dtype.PType chunk_offsets_ptype = 5; - * @return The chunkOffsetsPtype. - */ - @java.lang.Override public io.github.dfa1.vortex.proto.DTypeProtos.PType getChunkOffsetsPtype() { - io.github.dfa1.vortex.proto.DTypeProtos.PType result = io.github.dfa1.vortex.proto.DTypeProtos.PType.forNumber(chunkOffsetsPtype_); - return result == null ? io.github.dfa1.vortex.proto.DTypeProtos.PType.UNRECOGNIZED : result; - } - - public static final int OFFSET_WITHIN_CHUNK_FIELD_NUMBER = 6; - private long offsetWithinChunk_ = 0L; - /** - * optional uint64 offset_within_chunk = 6; - * @return Whether the offsetWithinChunk field is set. - */ - @java.lang.Override - public boolean hasOffsetWithinChunk() { - return ((bitField0_ & 0x00000004) != 0); - } - /** - * optional uint64 offset_within_chunk = 6; - * @return The offsetWithinChunk. - */ - @java.lang.Override - public long getOffsetWithinChunk() { - return offsetWithinChunk_; - } - - private byte memoizedIsInitialized = -1; - @java.lang.Override - public final boolean isInitialized() { - byte isInitialized = memoizedIsInitialized; - if (isInitialized == 1) return true; - if (isInitialized == 0) return false; - - memoizedIsInitialized = 1; - return true; - } - - @java.lang.Override - public void writeTo(com.google.protobuf.CodedOutputStream output) - throws java.io.IOException { - if (len_ != 0L) { - output.writeUInt64(1, len_); - } - if (offset_ != 0L) { - output.writeUInt64(2, offset_); - } - if (indicesPtype_ != io.github.dfa1.vortex.proto.DTypeProtos.PType.U8.getNumber()) { - output.writeEnum(3, indicesPtype_); - } - if (((bitField0_ & 0x00000001) != 0)) { - output.writeUInt64(4, chunkOffsetsLen_); - } - if (((bitField0_ & 0x00000002) != 0)) { - output.writeEnum(5, chunkOffsetsPtype_); - } - if (((bitField0_ & 0x00000004) != 0)) { - output.writeUInt64(6, offsetWithinChunk_); - } - getUnknownFields().writeTo(output); - } - private int computeSerializedSize_0() { - int size = 0; - if (len_ != 0L) { - size += com.google.protobuf.CodedOutputStream - .computeUInt64Size(1, len_); - } - if (offset_ != 0L) { - size += com.google.protobuf.CodedOutputStream - .computeUInt64Size(2, offset_); - } - if (indicesPtype_ != io.github.dfa1.vortex.proto.DTypeProtos.PType.U8.getNumber()) { - size += com.google.protobuf.CodedOutputStream - .computeEnumSize(3, indicesPtype_); - } - if (((bitField0_ & 0x00000001) != 0)) { - size += com.google.protobuf.CodedOutputStream - .computeUInt64Size(4, chunkOffsetsLen_); - } - if (((bitField0_ & 0x00000002) != 0)) { - size += com.google.protobuf.CodedOutputStream - .computeEnumSize(5, chunkOffsetsPtype_); - } - if (((bitField0_ & 0x00000004) != 0)) { - size += com.google.protobuf.CodedOutputStream - .computeUInt64Size(6, offsetWithinChunk_); - } - return size; - } - @java.lang.Override - public int getSerializedSize() { - int size = memoizedSize; - if (size != -1) return size; - - size = 0; - size += computeSerializedSize_0(); - size += getUnknownFields().getSerializedSize(); - memoizedSize = size; - return size; - } - - @java.lang.Override - public boolean equals(final java.lang.Object obj) { - if (obj == this) { - return true; - } - if (!(obj instanceof io.github.dfa1.vortex.proto.EncodingProtos.PatchesMetadata)) { - return super.equals(obj); - } - io.github.dfa1.vortex.proto.EncodingProtos.PatchesMetadata other = (io.github.dfa1.vortex.proto.EncodingProtos.PatchesMetadata) obj; - - if (getLen() - != other.getLen()) return false; - if (getOffset() - != other.getOffset()) return false; - if (indicesPtype_ != other.indicesPtype_) return false; - if (hasChunkOffsetsLen() != other.hasChunkOffsetsLen()) return false; - if (hasChunkOffsetsLen()) { - if (getChunkOffsetsLen() - != other.getChunkOffsetsLen()) return false; - } - if (hasChunkOffsetsPtype() != other.hasChunkOffsetsPtype()) return false; - if (hasChunkOffsetsPtype()) { - if (chunkOffsetsPtype_ != other.chunkOffsetsPtype_) return false; - } - if (hasOffsetWithinChunk() != other.hasOffsetWithinChunk()) return false; - if (hasOffsetWithinChunk()) { - if (getOffsetWithinChunk() - != other.getOffsetWithinChunk()) return false; - } - if (!getUnknownFields().equals(other.getUnknownFields())) return false; - return true; - } - - @java.lang.Override - public int hashCode() { - if (memoizedHashCode != 0) { - return memoizedHashCode; - } - int hash = 41; - hash = (19 * hash) + getDescriptor().hashCode(); - hash = (37 * hash) + LEN_FIELD_NUMBER; - hash = (53 * hash) + com.google.protobuf.Internal.hashLong( - getLen()); - hash = (37 * hash) + OFFSET_FIELD_NUMBER; - hash = (53 * hash) + com.google.protobuf.Internal.hashLong( - getOffset()); - hash = (37 * hash) + INDICES_PTYPE_FIELD_NUMBER; - hash = (53 * hash) + indicesPtype_; - if (hasChunkOffsetsLen()) { - hash = (37 * hash) + CHUNK_OFFSETS_LEN_FIELD_NUMBER; - hash = (53 * hash) + com.google.protobuf.Internal.hashLong( - getChunkOffsetsLen()); - } - if (hasChunkOffsetsPtype()) { - hash = (37 * hash) + CHUNK_OFFSETS_PTYPE_FIELD_NUMBER; - hash = (53 * hash) + chunkOffsetsPtype_; - } - if (hasOffsetWithinChunk()) { - hash = (37 * hash) + OFFSET_WITHIN_CHUNK_FIELD_NUMBER; - hash = (53 * hash) + com.google.protobuf.Internal.hashLong( - getOffsetWithinChunk()); - } - hash = (29 * hash) + getUnknownFields().hashCode(); - memoizedHashCode = hash; - return hash; - } - - public static io.github.dfa1.vortex.proto.EncodingProtos.PatchesMetadata parseFrom( - java.nio.ByteBuffer data) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.PatchesMetadata parseFrom( - java.nio.ByteBuffer data, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data, extensionRegistry); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.PatchesMetadata parseFrom( - com.google.protobuf.ByteString data) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.PatchesMetadata parseFrom( - com.google.protobuf.ByteString data, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data, extensionRegistry); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.PatchesMetadata parseFrom(byte[] data) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.PatchesMetadata parseFrom( - byte[] data, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data, extensionRegistry); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.PatchesMetadata parseFrom(java.io.InputStream input) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseWithIOException(PARSER, input); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.PatchesMetadata parseFrom( - java.io.InputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseWithIOException(PARSER, input, extensionRegistry); - } - - public static io.github.dfa1.vortex.proto.EncodingProtos.PatchesMetadata parseDelimitedFrom(java.io.InputStream input) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseDelimitedWithIOException(PARSER, input); - } - - public static io.github.dfa1.vortex.proto.EncodingProtos.PatchesMetadata parseDelimitedFrom( - java.io.InputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseDelimitedWithIOException(PARSER, input, extensionRegistry); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.PatchesMetadata parseFrom( - com.google.protobuf.CodedInputStream input) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseWithIOException(PARSER, input); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.PatchesMetadata parseFrom( - com.google.protobuf.CodedInputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseWithIOException(PARSER, input, extensionRegistry); - } - - @java.lang.Override - public Builder newBuilderForType() { return newBuilder(); } - public static Builder newBuilder() { - return DEFAULT_INSTANCE.toBuilder(); - } - public static Builder newBuilder(io.github.dfa1.vortex.proto.EncodingProtos.PatchesMetadata prototype) { - return DEFAULT_INSTANCE.toBuilder().mergeFrom(prototype); - } - @java.lang.Override - public Builder toBuilder() { - return this == DEFAULT_INSTANCE - ? new Builder() : new Builder().mergeFrom(this); - } - - @java.lang.Override - protected Builder newBuilderForType( - com.google.protobuf.GeneratedMessage.BuilderParent parent) { - Builder builder = new Builder(parent); - return builder; - } - /** - * Protobuf type {@code vortex.encodings.PatchesMetadata} - */ - public static final class Builder extends - com.google.protobuf.GeneratedMessage.Builder implements - // @@protoc_insertion_point(builder_implements:vortex.encodings.PatchesMetadata) - io.github.dfa1.vortex.proto.EncodingProtos.PatchesMetadataOrBuilder { - public static final com.google.protobuf.Descriptors.Descriptor - getDescriptor() { - return io.github.dfa1.vortex.proto.EncodingProtos.internal_static_vortex_encodings_PatchesMetadata_descriptor; - } - - @java.lang.Override - protected com.google.protobuf.GeneratedMessage.FieldAccessorTable - internalGetFieldAccessorTable() { - return io.github.dfa1.vortex.proto.EncodingProtos.internal_static_vortex_encodings_PatchesMetadata_fieldAccessorTable - .ensureFieldAccessorsInitialized( - io.github.dfa1.vortex.proto.EncodingProtos.PatchesMetadata.class, io.github.dfa1.vortex.proto.EncodingProtos.PatchesMetadata.Builder.class); - } - - // Construct using io.github.dfa1.vortex.proto.EncodingProtos.PatchesMetadata.newBuilder() - private Builder() { - - } - - private Builder( - com.google.protobuf.GeneratedMessage.BuilderParent parent) { - super(parent); - - } - @java.lang.Override - public Builder clear() { - super.clear(); - bitField0_ = 0; - len_ = 0L; - offset_ = 0L; - indicesPtype_ = 0; - chunkOffsetsLen_ = 0L; - chunkOffsetsPtype_ = 0; - offsetWithinChunk_ = 0L; - return this; - } - - @java.lang.Override - public com.google.protobuf.Descriptors.Descriptor - getDescriptorForType() { - return io.github.dfa1.vortex.proto.EncodingProtos.internal_static_vortex_encodings_PatchesMetadata_descriptor; - } - - @java.lang.Override - public io.github.dfa1.vortex.proto.EncodingProtos.PatchesMetadata getDefaultInstanceForType() { - return io.github.dfa1.vortex.proto.EncodingProtos.PatchesMetadata.getDefaultInstance(); - } - - @java.lang.Override - public io.github.dfa1.vortex.proto.EncodingProtos.PatchesMetadata build() { - io.github.dfa1.vortex.proto.EncodingProtos.PatchesMetadata result = buildPartial(); - if (!result.isInitialized()) { - throw newUninitializedMessageException(result); - } - return result; - } - - @java.lang.Override - public io.github.dfa1.vortex.proto.EncodingProtos.PatchesMetadata buildPartial() { - io.github.dfa1.vortex.proto.EncodingProtos.PatchesMetadata result = new io.github.dfa1.vortex.proto.EncodingProtos.PatchesMetadata(this); - if (bitField0_ != 0) { buildPartial0(result); } - onBuilt(); - return result; - } - - private void buildPartial0(io.github.dfa1.vortex.proto.EncodingProtos.PatchesMetadata result) { - int from_bitField0_ = bitField0_; - if (((from_bitField0_ & 0x00000001) != 0)) { - result.len_ = len_; - } - if (((from_bitField0_ & 0x00000002) != 0)) { - result.offset_ = offset_; - } - if (((from_bitField0_ & 0x00000004) != 0)) { - result.indicesPtype_ = indicesPtype_; - } - int to_bitField0_ = 0; - if (((from_bitField0_ & 0x00000008) != 0)) { - result.chunkOffsetsLen_ = chunkOffsetsLen_; - to_bitField0_ |= 0x00000001; - } - if (((from_bitField0_ & 0x00000010) != 0)) { - result.chunkOffsetsPtype_ = chunkOffsetsPtype_; - to_bitField0_ |= 0x00000002; - } - if (((from_bitField0_ & 0x00000020) != 0)) { - result.offsetWithinChunk_ = offsetWithinChunk_; - to_bitField0_ |= 0x00000004; - } - result.bitField0_ |= to_bitField0_; - } - - @java.lang.Override - public Builder mergeFrom(com.google.protobuf.Message other) { - if (other instanceof io.github.dfa1.vortex.proto.EncodingProtos.PatchesMetadata) { - return mergeFrom((io.github.dfa1.vortex.proto.EncodingProtos.PatchesMetadata)other); - } else { - super.mergeFrom(other); - return this; - } - } - - public Builder mergeFrom(io.github.dfa1.vortex.proto.EncodingProtos.PatchesMetadata other) { - if (other == io.github.dfa1.vortex.proto.EncodingProtos.PatchesMetadata.getDefaultInstance()) return this; - if (other.getLen() != 0L) { - setLen(other.getLen()); - } - if (other.getOffset() != 0L) { - setOffset(other.getOffset()); - } - if (other.indicesPtype_ != 0) { - setIndicesPtypeValue(other.getIndicesPtypeValue()); - } - if (other.hasChunkOffsetsLen()) { - setChunkOffsetsLen(other.getChunkOffsetsLen()); - } - if (other.hasChunkOffsetsPtype()) { - setChunkOffsetsPtypeValue(other.getChunkOffsetsPtypeValue()); - } - if (other.hasOffsetWithinChunk()) { - setOffsetWithinChunk(other.getOffsetWithinChunk()); - } - this.mergeUnknownFields(other.getUnknownFields()); - onChanged(); - return this; - } - - @java.lang.Override - public final boolean isInitialized() { - return true; - } - - @java.lang.Override - public Builder mergeFrom( - com.google.protobuf.CodedInputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws java.io.IOException { - if (extensionRegistry == null) { - throw new java.lang.NullPointerException(); - } - try { - boolean done = false; - while (!done) { - int tag = input.readTag(); - switch (tag) { - case 0: - done = true; - break; - case 8: { - len_ = input.readUInt64(); - bitField0_ |= 0x00000001; - break; - } // case 8 - case 16: { - offset_ = input.readUInt64(); - bitField0_ |= 0x00000002; - break; - } // case 16 - case 24: { - indicesPtype_ = input.readEnum(); - bitField0_ |= 0x00000004; - break; - } // case 24 - case 32: { - chunkOffsetsLen_ = input.readUInt64(); - bitField0_ |= 0x00000008; - break; - } // case 32 - case 40: { - chunkOffsetsPtype_ = input.readEnum(); - bitField0_ |= 0x00000010; - break; - } // case 40 - case 48: { - offsetWithinChunk_ = input.readUInt64(); - bitField0_ |= 0x00000020; - break; - } // case 48 - default: { - if (!super.parseUnknownField(input, extensionRegistry, tag)) { - done = true; // was an endgroup tag - } - break; - } // default: - } // switch (tag) - } // while (!done) - } catch (com.google.protobuf.InvalidProtocolBufferException e) { - throw e.unwrapIOException(); - } finally { - onChanged(); - } // finally - return this; - } - private int bitField0_; - - private long len_ ; - /** - * uint64 len = 1; - * @return The len. - */ - @java.lang.Override - public long getLen() { - return len_; - } - /** - * uint64 len = 1; - * @param value The len to set. - * @return This builder for chaining. - */ - public Builder setLen(long value) { - - len_ = value; - bitField0_ |= 0x00000001; - onChanged(); - return this; - } - /** - * uint64 len = 1; - * @return This builder for chaining. - */ - public Builder clearLen() { - bitField0_ = (bitField0_ & ~0x00000001); - len_ = 0L; - onChanged(); - return this; - } - - private long offset_ ; - /** - * uint64 offset = 2; - * @return The offset. - */ - @java.lang.Override - public long getOffset() { - return offset_; - } - /** - * uint64 offset = 2; - * @param value The offset to set. - * @return This builder for chaining. - */ - public Builder setOffset(long value) { - - offset_ = value; - bitField0_ |= 0x00000002; - onChanged(); - return this; - } - /** - * uint64 offset = 2; - * @return This builder for chaining. - */ - public Builder clearOffset() { - bitField0_ = (bitField0_ & ~0x00000002); - offset_ = 0L; - onChanged(); - return this; - } - - private int indicesPtype_ = 0; - /** - * .vortex.dtype.PType indices_ptype = 3; - * @return The enum numeric value on the wire for indicesPtype. - */ - @java.lang.Override public int getIndicesPtypeValue() { - return indicesPtype_; - } - /** - * .vortex.dtype.PType indices_ptype = 3; - * @param value The enum numeric value on the wire for indicesPtype to set. - * @return This builder for chaining. - */ - public Builder setIndicesPtypeValue(int value) { - indicesPtype_ = value; - bitField0_ |= 0x00000004; - onChanged(); - return this; - } - /** - * .vortex.dtype.PType indices_ptype = 3; - * @return The indicesPtype. - */ - @java.lang.Override - public io.github.dfa1.vortex.proto.DTypeProtos.PType getIndicesPtype() { - io.github.dfa1.vortex.proto.DTypeProtos.PType result = io.github.dfa1.vortex.proto.DTypeProtos.PType.forNumber(indicesPtype_); - return result == null ? io.github.dfa1.vortex.proto.DTypeProtos.PType.UNRECOGNIZED : result; - } - /** - * .vortex.dtype.PType indices_ptype = 3; - * @param value The indicesPtype to set. - * @throws IllegalArgumentException if UNRECOGNIZED is provided. - * @return This builder for chaining. - */ - public Builder setIndicesPtype(io.github.dfa1.vortex.proto.DTypeProtos.PType value) { - if (value == null) { throw new NullPointerException(); } - bitField0_ |= 0x00000004; - indicesPtype_ = value.getNumber(); - onChanged(); - return this; - } - /** - * .vortex.dtype.PType indices_ptype = 3; - * @return This builder for chaining. - */ - public Builder clearIndicesPtype() { - bitField0_ = (bitField0_ & ~0x00000004); - indicesPtype_ = 0; - onChanged(); - return this; - } - - private long chunkOffsetsLen_ ; - /** - * optional uint64 chunk_offsets_len = 4; - * @return Whether the chunkOffsetsLen field is set. - */ - @java.lang.Override - public boolean hasChunkOffsetsLen() { - return ((bitField0_ & 0x00000008) != 0); - } - /** - * optional uint64 chunk_offsets_len = 4; - * @return The chunkOffsetsLen. - */ - @java.lang.Override - public long getChunkOffsetsLen() { - return chunkOffsetsLen_; - } - /** - * optional uint64 chunk_offsets_len = 4; - * @param value The chunkOffsetsLen to set. - * @return This builder for chaining. - */ - public Builder setChunkOffsetsLen(long value) { - - chunkOffsetsLen_ = value; - bitField0_ |= 0x00000008; - onChanged(); - return this; - } - /** - * optional uint64 chunk_offsets_len = 4; - * @return This builder for chaining. - */ - public Builder clearChunkOffsetsLen() { - bitField0_ = (bitField0_ & ~0x00000008); - chunkOffsetsLen_ = 0L; - onChanged(); - return this; - } - - private int chunkOffsetsPtype_ = 0; - /** - * optional .vortex.dtype.PType chunk_offsets_ptype = 5; - * @return Whether the chunkOffsetsPtype field is set. - */ - @java.lang.Override public boolean hasChunkOffsetsPtype() { - return ((bitField0_ & 0x00000010) != 0); - } - /** - * optional .vortex.dtype.PType chunk_offsets_ptype = 5; - * @return The enum numeric value on the wire for chunkOffsetsPtype. - */ - @java.lang.Override public int getChunkOffsetsPtypeValue() { - return chunkOffsetsPtype_; - } - /** - * optional .vortex.dtype.PType chunk_offsets_ptype = 5; - * @param value The enum numeric value on the wire for chunkOffsetsPtype to set. - * @return This builder for chaining. - */ - public Builder setChunkOffsetsPtypeValue(int value) { - chunkOffsetsPtype_ = value; - bitField0_ |= 0x00000010; - onChanged(); - return this; - } - /** - * optional .vortex.dtype.PType chunk_offsets_ptype = 5; - * @return The chunkOffsetsPtype. - */ - @java.lang.Override - public io.github.dfa1.vortex.proto.DTypeProtos.PType getChunkOffsetsPtype() { - io.github.dfa1.vortex.proto.DTypeProtos.PType result = io.github.dfa1.vortex.proto.DTypeProtos.PType.forNumber(chunkOffsetsPtype_); - return result == null ? io.github.dfa1.vortex.proto.DTypeProtos.PType.UNRECOGNIZED : result; - } - /** - * optional .vortex.dtype.PType chunk_offsets_ptype = 5; - * @param value The chunkOffsetsPtype to set. - * @throws IllegalArgumentException if UNRECOGNIZED is provided. - * @return This builder for chaining. - */ - public Builder setChunkOffsetsPtype(io.github.dfa1.vortex.proto.DTypeProtos.PType value) { - if (value == null) { throw new NullPointerException(); } - bitField0_ |= 0x00000010; - chunkOffsetsPtype_ = value.getNumber(); - onChanged(); - return this; - } - /** - * optional .vortex.dtype.PType chunk_offsets_ptype = 5; - * @return This builder for chaining. - */ - public Builder clearChunkOffsetsPtype() { - bitField0_ = (bitField0_ & ~0x00000010); - chunkOffsetsPtype_ = 0; - onChanged(); - return this; - } - - private long offsetWithinChunk_ ; - /** - * optional uint64 offset_within_chunk = 6; - * @return Whether the offsetWithinChunk field is set. - */ - @java.lang.Override - public boolean hasOffsetWithinChunk() { - return ((bitField0_ & 0x00000020) != 0); - } - /** - * optional uint64 offset_within_chunk = 6; - * @return The offsetWithinChunk. - */ - @java.lang.Override - public long getOffsetWithinChunk() { - return offsetWithinChunk_; - } - /** - * optional uint64 offset_within_chunk = 6; - * @param value The offsetWithinChunk to set. - * @return This builder for chaining. - */ - public Builder setOffsetWithinChunk(long value) { - - offsetWithinChunk_ = value; - bitField0_ |= 0x00000020; - onChanged(); - return this; - } - /** - * optional uint64 offset_within_chunk = 6; - * @return This builder for chaining. - */ - public Builder clearOffsetWithinChunk() { - bitField0_ = (bitField0_ & ~0x00000020); - offsetWithinChunk_ = 0L; - onChanged(); - return this; - } - - // @@protoc_insertion_point(builder_scope:vortex.encodings.PatchesMetadata) - } - - // @@protoc_insertion_point(class_scope:vortex.encodings.PatchesMetadata) - private static final io.github.dfa1.vortex.proto.EncodingProtos.PatchesMetadata DEFAULT_INSTANCE; - static { - DEFAULT_INSTANCE = new io.github.dfa1.vortex.proto.EncodingProtos.PatchesMetadata(); - } - - public static io.github.dfa1.vortex.proto.EncodingProtos.PatchesMetadata getDefaultInstance() { - return DEFAULT_INSTANCE; - } - - private static final com.google.protobuf.Parser - PARSER = new com.google.protobuf.AbstractParser() { - @java.lang.Override - public PatchesMetadata parsePartialFrom( - com.google.protobuf.CodedInputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws com.google.protobuf.InvalidProtocolBufferException { - Builder builder = newBuilder(); - try { - builder.mergeFrom(input, extensionRegistry); - } catch (com.google.protobuf.InvalidProtocolBufferException e) { - throw e.setUnfinishedMessage(builder.buildPartial()); - } catch (com.google.protobuf.UninitializedMessageException e) { - throw e.asInvalidProtocolBufferException().setUnfinishedMessage(builder.buildPartial()); - } catch (java.io.IOException e) { - throw new com.google.protobuf.InvalidProtocolBufferException(e) - .setUnfinishedMessage(builder.buildPartial()); - } - return builder.buildPartial(); - } - }; - - public static com.google.protobuf.Parser parser() { - return PARSER; - } - - @java.lang.Override - public com.google.protobuf.Parser getParserForType() { - return PARSER; - } - - @java.lang.Override - public io.github.dfa1.vortex.proto.EncodingProtos.PatchesMetadata getDefaultInstanceForType() { - return DEFAULT_INSTANCE; - } - - } - - public interface SparseMetadataOrBuilder extends - // @@protoc_insertion_point(interface_extends:vortex.encodings.SparseMetadata) - com.google.protobuf.MessageOrBuilder { - - /** - * .vortex.encodings.PatchesMetadata patches = 1; - * @return Whether the patches field is set. - */ - boolean hasPatches(); - /** - * .vortex.encodings.PatchesMetadata patches = 1; - * @return The patches. - */ - io.github.dfa1.vortex.proto.EncodingProtos.PatchesMetadata getPatches(); - /** - * .vortex.encodings.PatchesMetadata patches = 1; - */ - io.github.dfa1.vortex.proto.EncodingProtos.PatchesMetadataOrBuilder getPatchesOrBuilder(); - } - /** - * Protobuf type {@code vortex.encodings.SparseMetadata} - */ - public static final class SparseMetadata extends - com.google.protobuf.GeneratedMessage implements - // @@protoc_insertion_point(message_implements:vortex.encodings.SparseMetadata) - SparseMetadataOrBuilder { - private static final long serialVersionUID = 0L; - static { - com.google.protobuf.RuntimeVersion.validateProtobufGencodeVersion( - com.google.protobuf.RuntimeVersion.RuntimeDomain.PUBLIC, - /* major= */ 4, - /* minor= */ 35, - /* patch= */ 0, - /* suffix= */ "", - "SparseMetadata"); - } - // Use SparseMetadata.newBuilder() to construct. - private SparseMetadata(com.google.protobuf.GeneratedMessage.Builder builder) { - super(builder); - } - private SparseMetadata() { - } - - public static final com.google.protobuf.Descriptors.Descriptor - getDescriptor() { - return io.github.dfa1.vortex.proto.EncodingProtos.internal_static_vortex_encodings_SparseMetadata_descriptor; - } - - @java.lang.Override - public com.google.protobuf.Descriptors.Descriptor getDescriptorForType() { - return io.github.dfa1.vortex.proto.EncodingProtos.internal_static_vortex_encodings_SparseMetadata_descriptor; - } - - @java.lang.Override - protected com.google.protobuf.GeneratedMessage.FieldAccessorTable - internalGetFieldAccessorTable() { - return io.github.dfa1.vortex.proto.EncodingProtos.internal_static_vortex_encodings_SparseMetadata_fieldAccessorTable - .ensureFieldAccessorsInitialized( - io.github.dfa1.vortex.proto.EncodingProtos.SparseMetadata.class, io.github.dfa1.vortex.proto.EncodingProtos.SparseMetadata.Builder.class); - } - - private int bitField0_; - public static final int PATCHES_FIELD_NUMBER = 1; - private io.github.dfa1.vortex.proto.EncodingProtos.PatchesMetadata patches_; - /** - * .vortex.encodings.PatchesMetadata patches = 1; - * @return Whether the patches field is set. - */ - @java.lang.Override - public boolean hasPatches() { - return ((bitField0_ & 0x00000001) != 0); - } - /** - * .vortex.encodings.PatchesMetadata patches = 1; - * @return The patches. - */ - @java.lang.Override - public io.github.dfa1.vortex.proto.EncodingProtos.PatchesMetadata getPatches() { - return patches_ == null ? io.github.dfa1.vortex.proto.EncodingProtos.PatchesMetadata.getDefaultInstance() : patches_; - } - /** - * .vortex.encodings.PatchesMetadata patches = 1; - */ - @java.lang.Override - public io.github.dfa1.vortex.proto.EncodingProtos.PatchesMetadataOrBuilder getPatchesOrBuilder() { - return patches_ == null ? io.github.dfa1.vortex.proto.EncodingProtos.PatchesMetadata.getDefaultInstance() : patches_; - } - - private byte memoizedIsInitialized = -1; - @java.lang.Override - public final boolean isInitialized() { - byte isInitialized = memoizedIsInitialized; - if (isInitialized == 1) return true; - if (isInitialized == 0) return false; - - memoizedIsInitialized = 1; - return true; - } - - @java.lang.Override - public void writeTo(com.google.protobuf.CodedOutputStream output) - throws java.io.IOException { - if (((bitField0_ & 0x00000001) != 0)) { - output.writeMessage(1, getPatches()); - } - getUnknownFields().writeTo(output); - } - private int computeSerializedSize_0() { - int size = 0; - if (((bitField0_ & 0x00000001) != 0)) { - size += com.google.protobuf.CodedOutputStream - .computeMessageSize(1, getPatches()); - } - return size; - } - @java.lang.Override - public int getSerializedSize() { - int size = memoizedSize; - if (size != -1) return size; - - size = 0; - size += computeSerializedSize_0(); - size += getUnknownFields().getSerializedSize(); - memoizedSize = size; - return size; - } - - @java.lang.Override - public boolean equals(final java.lang.Object obj) { - if (obj == this) { - return true; - } - if (!(obj instanceof io.github.dfa1.vortex.proto.EncodingProtos.SparseMetadata)) { - return super.equals(obj); - } - io.github.dfa1.vortex.proto.EncodingProtos.SparseMetadata other = (io.github.dfa1.vortex.proto.EncodingProtos.SparseMetadata) obj; - - if (hasPatches() != other.hasPatches()) return false; - if (hasPatches()) { - if (!getPatches() - .equals(other.getPatches())) return false; - } - if (!getUnknownFields().equals(other.getUnknownFields())) return false; - return true; - } - - @java.lang.Override - public int hashCode() { - if (memoizedHashCode != 0) { - return memoizedHashCode; - } - int hash = 41; - hash = (19 * hash) + getDescriptor().hashCode(); - if (hasPatches()) { - hash = (37 * hash) + PATCHES_FIELD_NUMBER; - hash = (53 * hash) + getPatches().hashCode(); - } - hash = (29 * hash) + getUnknownFields().hashCode(); - memoizedHashCode = hash; - return hash; - } - - public static io.github.dfa1.vortex.proto.EncodingProtos.SparseMetadata parseFrom( - java.nio.ByteBuffer data) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.SparseMetadata parseFrom( - java.nio.ByteBuffer data, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data, extensionRegistry); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.SparseMetadata parseFrom( - com.google.protobuf.ByteString data) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.SparseMetadata parseFrom( - com.google.protobuf.ByteString data, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data, extensionRegistry); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.SparseMetadata parseFrom(byte[] data) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.SparseMetadata parseFrom( - byte[] data, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data, extensionRegistry); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.SparseMetadata parseFrom(java.io.InputStream input) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseWithIOException(PARSER, input); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.SparseMetadata parseFrom( - java.io.InputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseWithIOException(PARSER, input, extensionRegistry); - } - - public static io.github.dfa1.vortex.proto.EncodingProtos.SparseMetadata parseDelimitedFrom(java.io.InputStream input) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseDelimitedWithIOException(PARSER, input); - } - - public static io.github.dfa1.vortex.proto.EncodingProtos.SparseMetadata parseDelimitedFrom( - java.io.InputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseDelimitedWithIOException(PARSER, input, extensionRegistry); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.SparseMetadata parseFrom( - com.google.protobuf.CodedInputStream input) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseWithIOException(PARSER, input); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.SparseMetadata parseFrom( - com.google.protobuf.CodedInputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseWithIOException(PARSER, input, extensionRegistry); - } - - @java.lang.Override - public Builder newBuilderForType() { return newBuilder(); } - public static Builder newBuilder() { - return DEFAULT_INSTANCE.toBuilder(); - } - public static Builder newBuilder(io.github.dfa1.vortex.proto.EncodingProtos.SparseMetadata prototype) { - return DEFAULT_INSTANCE.toBuilder().mergeFrom(prototype); - } - @java.lang.Override - public Builder toBuilder() { - return this == DEFAULT_INSTANCE - ? new Builder() : new Builder().mergeFrom(this); - } - - @java.lang.Override - protected Builder newBuilderForType( - com.google.protobuf.GeneratedMessage.BuilderParent parent) { - Builder builder = new Builder(parent); - return builder; - } - /** - * Protobuf type {@code vortex.encodings.SparseMetadata} - */ - public static final class Builder extends - com.google.protobuf.GeneratedMessage.Builder implements - // @@protoc_insertion_point(builder_implements:vortex.encodings.SparseMetadata) - io.github.dfa1.vortex.proto.EncodingProtos.SparseMetadataOrBuilder { - public static final com.google.protobuf.Descriptors.Descriptor - getDescriptor() { - return io.github.dfa1.vortex.proto.EncodingProtos.internal_static_vortex_encodings_SparseMetadata_descriptor; - } - - @java.lang.Override - protected com.google.protobuf.GeneratedMessage.FieldAccessorTable - internalGetFieldAccessorTable() { - return io.github.dfa1.vortex.proto.EncodingProtos.internal_static_vortex_encodings_SparseMetadata_fieldAccessorTable - .ensureFieldAccessorsInitialized( - io.github.dfa1.vortex.proto.EncodingProtos.SparseMetadata.class, io.github.dfa1.vortex.proto.EncodingProtos.SparseMetadata.Builder.class); - } - - // Construct using io.github.dfa1.vortex.proto.EncodingProtos.SparseMetadata.newBuilder() - private Builder() { - maybeForceBuilderInitialization(); - } - - private Builder( - com.google.protobuf.GeneratedMessage.BuilderParent parent) { - super(parent); - maybeForceBuilderInitialization(); - } - private void maybeForceBuilderInitialization() { - if (com.google.protobuf.GeneratedMessage - .alwaysUseFieldBuilders) { - internalGetPatchesFieldBuilder(); - } - } - @java.lang.Override - public Builder clear() { - super.clear(); - bitField0_ = 0; - patches_ = null; - if (patchesBuilder_ != null) { - patchesBuilder_.dispose(); - patchesBuilder_ = null; - } - return this; - } - - @java.lang.Override - public com.google.protobuf.Descriptors.Descriptor - getDescriptorForType() { - return io.github.dfa1.vortex.proto.EncodingProtos.internal_static_vortex_encodings_SparseMetadata_descriptor; - } - - @java.lang.Override - public io.github.dfa1.vortex.proto.EncodingProtos.SparseMetadata getDefaultInstanceForType() { - return io.github.dfa1.vortex.proto.EncodingProtos.SparseMetadata.getDefaultInstance(); - } - - @java.lang.Override - public io.github.dfa1.vortex.proto.EncodingProtos.SparseMetadata build() { - io.github.dfa1.vortex.proto.EncodingProtos.SparseMetadata result = buildPartial(); - if (!result.isInitialized()) { - throw newUninitializedMessageException(result); - } - return result; - } - - @java.lang.Override - public io.github.dfa1.vortex.proto.EncodingProtos.SparseMetadata buildPartial() { - io.github.dfa1.vortex.proto.EncodingProtos.SparseMetadata result = new io.github.dfa1.vortex.proto.EncodingProtos.SparseMetadata(this); - if (bitField0_ != 0) { buildPartial0(result); } - onBuilt(); - return result; - } - - private void buildPartial0(io.github.dfa1.vortex.proto.EncodingProtos.SparseMetadata result) { - int from_bitField0_ = bitField0_; - int to_bitField0_ = 0; - if (((from_bitField0_ & 0x00000001) != 0)) { - result.patches_ = patchesBuilder_ == null - ? patches_ - : patchesBuilder_.build(); - to_bitField0_ |= 0x00000001; - } - result.bitField0_ |= to_bitField0_; - } - - @java.lang.Override - public Builder mergeFrom(com.google.protobuf.Message other) { - if (other instanceof io.github.dfa1.vortex.proto.EncodingProtos.SparseMetadata) { - return mergeFrom((io.github.dfa1.vortex.proto.EncodingProtos.SparseMetadata)other); - } else { - super.mergeFrom(other); - return this; - } - } - - public Builder mergeFrom(io.github.dfa1.vortex.proto.EncodingProtos.SparseMetadata other) { - if (other == io.github.dfa1.vortex.proto.EncodingProtos.SparseMetadata.getDefaultInstance()) return this; - if (other.hasPatches()) { - mergePatches(other.getPatches()); - } - this.mergeUnknownFields(other.getUnknownFields()); - onChanged(); - return this; - } - - @java.lang.Override - public final boolean isInitialized() { - return true; - } - - @java.lang.Override - public Builder mergeFrom( - com.google.protobuf.CodedInputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws java.io.IOException { - if (extensionRegistry == null) { - throw new java.lang.NullPointerException(); - } - try { - boolean done = false; - while (!done) { - int tag = input.readTag(); - switch (tag) { - case 0: - done = true; - break; - case 10: { - input.readMessage( - internalGetPatchesFieldBuilder().getBuilder(), - extensionRegistry); - bitField0_ |= 0x00000001; - break; - } // case 10 - default: { - if (!super.parseUnknownField(input, extensionRegistry, tag)) { - done = true; // was an endgroup tag - } - break; - } // default: - } // switch (tag) - } // while (!done) - } catch (com.google.protobuf.InvalidProtocolBufferException e) { - throw e.unwrapIOException(); - } finally { - onChanged(); - } // finally - return this; - } - private int bitField0_; - - private io.github.dfa1.vortex.proto.EncodingProtos.PatchesMetadata patches_; - private com.google.protobuf.SingleFieldBuilder< - io.github.dfa1.vortex.proto.EncodingProtos.PatchesMetadata, io.github.dfa1.vortex.proto.EncodingProtos.PatchesMetadata.Builder, io.github.dfa1.vortex.proto.EncodingProtos.PatchesMetadataOrBuilder> patchesBuilder_; - /** - * .vortex.encodings.PatchesMetadata patches = 1; - * @return Whether the patches field is set. - */ - public boolean hasPatches() { - return ((bitField0_ & 0x00000001) != 0); - } - /** - * .vortex.encodings.PatchesMetadata patches = 1; - * @return The patches. - */ - public io.github.dfa1.vortex.proto.EncodingProtos.PatchesMetadata getPatches() { - if (patchesBuilder_ == null) { - return patches_ == null ? io.github.dfa1.vortex.proto.EncodingProtos.PatchesMetadata.getDefaultInstance() : patches_; - } else { - return patchesBuilder_.getMessage(); - } - } - /** - * .vortex.encodings.PatchesMetadata patches = 1; - */ - public Builder setPatches(io.github.dfa1.vortex.proto.EncodingProtos.PatchesMetadata value) { - if (patchesBuilder_ == null) { - if (value == null) { - throw new NullPointerException(); - } - patches_ = value; - } else { - patchesBuilder_.setMessage(value); - } - bitField0_ |= 0x00000001; - onChanged(); - return this; - } - /** - * .vortex.encodings.PatchesMetadata patches = 1; - */ - public Builder setPatches( - io.github.dfa1.vortex.proto.EncodingProtos.PatchesMetadata.Builder builderForValue) { - if (patchesBuilder_ == null) { - patches_ = builderForValue.build(); - } else { - patchesBuilder_.setMessage(builderForValue.build()); - } - bitField0_ |= 0x00000001; - onChanged(); - return this; - } - /** - * .vortex.encodings.PatchesMetadata patches = 1; - */ - public Builder mergePatches(io.github.dfa1.vortex.proto.EncodingProtos.PatchesMetadata value) { - if (patchesBuilder_ == null) { - if (((bitField0_ & 0x00000001) != 0) && - patches_ != null && - patches_ != io.github.dfa1.vortex.proto.EncodingProtos.PatchesMetadata.getDefaultInstance()) { - getPatchesBuilder().mergeFrom(value); - } else { - patches_ = value; - } - } else { - patchesBuilder_.mergeFrom(value); - } - if (patches_ != null) { - bitField0_ |= 0x00000001; - onChanged(); - } - return this; - } - /** - * .vortex.encodings.PatchesMetadata patches = 1; - */ - public Builder clearPatches() { - bitField0_ = (bitField0_ & ~0x00000001); - patches_ = null; - if (patchesBuilder_ != null) { - patchesBuilder_.dispose(); - patchesBuilder_ = null; - } - onChanged(); - return this; - } - /** - * .vortex.encodings.PatchesMetadata patches = 1; - */ - public io.github.dfa1.vortex.proto.EncodingProtos.PatchesMetadata.Builder getPatchesBuilder() { - bitField0_ |= 0x00000001; - onChanged(); - return internalGetPatchesFieldBuilder().getBuilder(); - } - /** - * .vortex.encodings.PatchesMetadata patches = 1; - */ - public io.github.dfa1.vortex.proto.EncodingProtos.PatchesMetadataOrBuilder getPatchesOrBuilder() { - if (patchesBuilder_ != null) { - return patchesBuilder_.getMessageOrBuilder(); - } else { - return patches_ == null ? - io.github.dfa1.vortex.proto.EncodingProtos.PatchesMetadata.getDefaultInstance() : patches_; - } - } - /** - * .vortex.encodings.PatchesMetadata patches = 1; - */ - private com.google.protobuf.SingleFieldBuilder< - io.github.dfa1.vortex.proto.EncodingProtos.PatchesMetadata, io.github.dfa1.vortex.proto.EncodingProtos.PatchesMetadata.Builder, io.github.dfa1.vortex.proto.EncodingProtos.PatchesMetadataOrBuilder> - internalGetPatchesFieldBuilder() { - if (patchesBuilder_ == null) { - patchesBuilder_ = new com.google.protobuf.SingleFieldBuilder< - io.github.dfa1.vortex.proto.EncodingProtos.PatchesMetadata, io.github.dfa1.vortex.proto.EncodingProtos.PatchesMetadata.Builder, io.github.dfa1.vortex.proto.EncodingProtos.PatchesMetadataOrBuilder>( - getPatches(), - getParentForChildren(), - isClean()); - patches_ = null; - } - return patchesBuilder_; - } - - // @@protoc_insertion_point(builder_scope:vortex.encodings.SparseMetadata) - } - - // @@protoc_insertion_point(class_scope:vortex.encodings.SparseMetadata) - private static final io.github.dfa1.vortex.proto.EncodingProtos.SparseMetadata DEFAULT_INSTANCE; - static { - DEFAULT_INSTANCE = new io.github.dfa1.vortex.proto.EncodingProtos.SparseMetadata(); - } - - public static io.github.dfa1.vortex.proto.EncodingProtos.SparseMetadata getDefaultInstance() { - return DEFAULT_INSTANCE; - } - - private static final com.google.protobuf.Parser - PARSER = new com.google.protobuf.AbstractParser() { - @java.lang.Override - public SparseMetadata parsePartialFrom( - com.google.protobuf.CodedInputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws com.google.protobuf.InvalidProtocolBufferException { - Builder builder = newBuilder(); - try { - builder.mergeFrom(input, extensionRegistry); - } catch (com.google.protobuf.InvalidProtocolBufferException e) { - throw e.setUnfinishedMessage(builder.buildPartial()); - } catch (com.google.protobuf.UninitializedMessageException e) { - throw e.asInvalidProtocolBufferException().setUnfinishedMessage(builder.buildPartial()); - } catch (java.io.IOException e) { - throw new com.google.protobuf.InvalidProtocolBufferException(e) - .setUnfinishedMessage(builder.buildPartial()); - } - return builder.buildPartial(); - } - }; - - public static com.google.protobuf.Parser parser() { - return PARSER; - } - - @java.lang.Override - public com.google.protobuf.Parser getParserForType() { - return PARSER; - } - - @java.lang.Override - public io.github.dfa1.vortex.proto.EncodingProtos.SparseMetadata getDefaultInstanceForType() { - return DEFAULT_INSTANCE; - } - - } - - public interface DictMetadataOrBuilder extends - // @@protoc_insertion_point(interface_extends:vortex.encodings.DictMetadata) - com.google.protobuf.MessageOrBuilder { - - /** - * uint32 values_len = 1; - * @return The valuesLen. - */ - int getValuesLen(); - - /** - * .vortex.dtype.PType codes_ptype = 2; - * @return The enum numeric value on the wire for codesPtype. - */ - int getCodesPtypeValue(); - /** - * .vortex.dtype.PType codes_ptype = 2; - * @return The codesPtype. - */ - io.github.dfa1.vortex.proto.DTypeProtos.PType getCodesPtype(); - - /** - * optional bool is_nullable_codes = 3; - * @return Whether the isNullableCodes field is set. - */ - boolean hasIsNullableCodes(); - /** - * optional bool is_nullable_codes = 3; - * @return The isNullableCodes. - */ - boolean getIsNullableCodes(); - - /** - * optional bool all_values_referenced = 4; - * @return Whether the allValuesReferenced field is set. - */ - boolean hasAllValuesReferenced(); - /** - * optional bool all_values_referenced = 4; - * @return The allValuesReferenced. - */ - boolean getAllValuesReferenced(); - } - /** - * Protobuf type {@code vortex.encodings.DictMetadata} - */ - public static final class DictMetadata extends - com.google.protobuf.GeneratedMessage implements - // @@protoc_insertion_point(message_implements:vortex.encodings.DictMetadata) - DictMetadataOrBuilder { - private static final long serialVersionUID = 0L; - static { - com.google.protobuf.RuntimeVersion.validateProtobufGencodeVersion( - com.google.protobuf.RuntimeVersion.RuntimeDomain.PUBLIC, - /* major= */ 4, - /* minor= */ 35, - /* patch= */ 0, - /* suffix= */ "", - "DictMetadata"); - } - // Use DictMetadata.newBuilder() to construct. - private DictMetadata(com.google.protobuf.GeneratedMessage.Builder builder) { - super(builder); - } - private DictMetadata() { - codesPtype_ = 0; - } - - public static final com.google.protobuf.Descriptors.Descriptor - getDescriptor() { - return io.github.dfa1.vortex.proto.EncodingProtos.internal_static_vortex_encodings_DictMetadata_descriptor; - } - - @java.lang.Override - public com.google.protobuf.Descriptors.Descriptor getDescriptorForType() { - return io.github.dfa1.vortex.proto.EncodingProtos.internal_static_vortex_encodings_DictMetadata_descriptor; - } - - @java.lang.Override - protected com.google.protobuf.GeneratedMessage.FieldAccessorTable - internalGetFieldAccessorTable() { - return io.github.dfa1.vortex.proto.EncodingProtos.internal_static_vortex_encodings_DictMetadata_fieldAccessorTable - .ensureFieldAccessorsInitialized( - io.github.dfa1.vortex.proto.EncodingProtos.DictMetadata.class, io.github.dfa1.vortex.proto.EncodingProtos.DictMetadata.Builder.class); - } - - private int bitField0_; - public static final int VALUES_LEN_FIELD_NUMBER = 1; - private int valuesLen_ = 0; - /** - * uint32 values_len = 1; - * @return The valuesLen. - */ - @java.lang.Override - public int getValuesLen() { - return valuesLen_; - } - - public static final int CODES_PTYPE_FIELD_NUMBER = 2; - private int codesPtype_ = 0; - /** - * .vortex.dtype.PType codes_ptype = 2; - * @return The enum numeric value on the wire for codesPtype. - */ - @java.lang.Override public int getCodesPtypeValue() { - return codesPtype_; - } - /** - * .vortex.dtype.PType codes_ptype = 2; - * @return The codesPtype. - */ - @java.lang.Override public io.github.dfa1.vortex.proto.DTypeProtos.PType getCodesPtype() { - io.github.dfa1.vortex.proto.DTypeProtos.PType result = io.github.dfa1.vortex.proto.DTypeProtos.PType.forNumber(codesPtype_); - return result == null ? io.github.dfa1.vortex.proto.DTypeProtos.PType.UNRECOGNIZED : result; - } - - public static final int IS_NULLABLE_CODES_FIELD_NUMBER = 3; - private boolean isNullableCodes_ = false; - /** - * optional bool is_nullable_codes = 3; - * @return Whether the isNullableCodes field is set. - */ - @java.lang.Override - public boolean hasIsNullableCodes() { - return ((bitField0_ & 0x00000001) != 0); - } - /** - * optional bool is_nullable_codes = 3; - * @return The isNullableCodes. - */ - @java.lang.Override - public boolean getIsNullableCodes() { - return isNullableCodes_; - } - - public static final int ALL_VALUES_REFERENCED_FIELD_NUMBER = 4; - private boolean allValuesReferenced_ = false; - /** - * optional bool all_values_referenced = 4; - * @return Whether the allValuesReferenced field is set. - */ - @java.lang.Override - public boolean hasAllValuesReferenced() { - return ((bitField0_ & 0x00000002) != 0); - } - /** - * optional bool all_values_referenced = 4; - * @return The allValuesReferenced. - */ - @java.lang.Override - public boolean getAllValuesReferenced() { - return allValuesReferenced_; - } - - private byte memoizedIsInitialized = -1; - @java.lang.Override - public final boolean isInitialized() { - byte isInitialized = memoizedIsInitialized; - if (isInitialized == 1) return true; - if (isInitialized == 0) return false; - - memoizedIsInitialized = 1; - return true; - } - - @java.lang.Override - public void writeTo(com.google.protobuf.CodedOutputStream output) - throws java.io.IOException { - if (valuesLen_ != 0) { - output.writeUInt32(1, valuesLen_); - } - if (codesPtype_ != io.github.dfa1.vortex.proto.DTypeProtos.PType.U8.getNumber()) { - output.writeEnum(2, codesPtype_); - } - if (((bitField0_ & 0x00000001) != 0)) { - output.writeBool(3, isNullableCodes_); - } - if (((bitField0_ & 0x00000002) != 0)) { - output.writeBool(4, allValuesReferenced_); - } - getUnknownFields().writeTo(output); - } - private int computeSerializedSize_0() { - int size = 0; - if (valuesLen_ != 0) { - size += com.google.protobuf.CodedOutputStream - .computeUInt32Size(1, valuesLen_); - } - if (codesPtype_ != io.github.dfa1.vortex.proto.DTypeProtos.PType.U8.getNumber()) { - size += com.google.protobuf.CodedOutputStream - .computeEnumSize(2, codesPtype_); - } - if (((bitField0_ & 0x00000001) != 0)) { - size += com.google.protobuf.CodedOutputStream - .computeBoolSize(3, isNullableCodes_); - } - if (((bitField0_ & 0x00000002) != 0)) { - size += com.google.protobuf.CodedOutputStream - .computeBoolSize(4, allValuesReferenced_); - } - return size; - } - @java.lang.Override - public int getSerializedSize() { - int size = memoizedSize; - if (size != -1) return size; - - size = 0; - size += computeSerializedSize_0(); - size += getUnknownFields().getSerializedSize(); - memoizedSize = size; - return size; - } - - @java.lang.Override - public boolean equals(final java.lang.Object obj) { - if (obj == this) { - return true; - } - if (!(obj instanceof io.github.dfa1.vortex.proto.EncodingProtos.DictMetadata)) { - return super.equals(obj); - } - io.github.dfa1.vortex.proto.EncodingProtos.DictMetadata other = (io.github.dfa1.vortex.proto.EncodingProtos.DictMetadata) obj; - - if (getValuesLen() - != other.getValuesLen()) return false; - if (codesPtype_ != other.codesPtype_) return false; - if (hasIsNullableCodes() != other.hasIsNullableCodes()) return false; - if (hasIsNullableCodes()) { - if (getIsNullableCodes() - != other.getIsNullableCodes()) return false; - } - if (hasAllValuesReferenced() != other.hasAllValuesReferenced()) return false; - if (hasAllValuesReferenced()) { - if (getAllValuesReferenced() - != other.getAllValuesReferenced()) return false; - } - if (!getUnknownFields().equals(other.getUnknownFields())) return false; - return true; - } - - @java.lang.Override - public int hashCode() { - if (memoizedHashCode != 0) { - return memoizedHashCode; - } - int hash = 41; - hash = (19 * hash) + getDescriptor().hashCode(); - hash = (37 * hash) + VALUES_LEN_FIELD_NUMBER; - hash = (53 * hash) + getValuesLen(); - hash = (37 * hash) + CODES_PTYPE_FIELD_NUMBER; - hash = (53 * hash) + codesPtype_; - if (hasIsNullableCodes()) { - hash = (37 * hash) + IS_NULLABLE_CODES_FIELD_NUMBER; - hash = (53 * hash) + com.google.protobuf.Internal.hashBoolean( - getIsNullableCodes()); - } - if (hasAllValuesReferenced()) { - hash = (37 * hash) + ALL_VALUES_REFERENCED_FIELD_NUMBER; - hash = (53 * hash) + com.google.protobuf.Internal.hashBoolean( - getAllValuesReferenced()); - } - hash = (29 * hash) + getUnknownFields().hashCode(); - memoizedHashCode = hash; - return hash; - } - - public static io.github.dfa1.vortex.proto.EncodingProtos.DictMetadata parseFrom( - java.nio.ByteBuffer data) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.DictMetadata parseFrom( - java.nio.ByteBuffer data, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data, extensionRegistry); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.DictMetadata parseFrom( - com.google.protobuf.ByteString data) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.DictMetadata parseFrom( - com.google.protobuf.ByteString data, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data, extensionRegistry); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.DictMetadata parseFrom(byte[] data) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.DictMetadata parseFrom( - byte[] data, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data, extensionRegistry); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.DictMetadata parseFrom(java.io.InputStream input) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseWithIOException(PARSER, input); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.DictMetadata parseFrom( - java.io.InputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseWithIOException(PARSER, input, extensionRegistry); - } - - public static io.github.dfa1.vortex.proto.EncodingProtos.DictMetadata parseDelimitedFrom(java.io.InputStream input) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseDelimitedWithIOException(PARSER, input); - } - - public static io.github.dfa1.vortex.proto.EncodingProtos.DictMetadata parseDelimitedFrom( - java.io.InputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseDelimitedWithIOException(PARSER, input, extensionRegistry); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.DictMetadata parseFrom( - com.google.protobuf.CodedInputStream input) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseWithIOException(PARSER, input); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.DictMetadata parseFrom( - com.google.protobuf.CodedInputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseWithIOException(PARSER, input, extensionRegistry); - } - - @java.lang.Override - public Builder newBuilderForType() { return newBuilder(); } - public static Builder newBuilder() { - return DEFAULT_INSTANCE.toBuilder(); - } - public static Builder newBuilder(io.github.dfa1.vortex.proto.EncodingProtos.DictMetadata prototype) { - return DEFAULT_INSTANCE.toBuilder().mergeFrom(prototype); - } - @java.lang.Override - public Builder toBuilder() { - return this == DEFAULT_INSTANCE - ? new Builder() : new Builder().mergeFrom(this); - } - - @java.lang.Override - protected Builder newBuilderForType( - com.google.protobuf.GeneratedMessage.BuilderParent parent) { - Builder builder = new Builder(parent); - return builder; - } - /** - * Protobuf type {@code vortex.encodings.DictMetadata} - */ - public static final class Builder extends - com.google.protobuf.GeneratedMessage.Builder implements - // @@protoc_insertion_point(builder_implements:vortex.encodings.DictMetadata) - io.github.dfa1.vortex.proto.EncodingProtos.DictMetadataOrBuilder { - public static final com.google.protobuf.Descriptors.Descriptor - getDescriptor() { - return io.github.dfa1.vortex.proto.EncodingProtos.internal_static_vortex_encodings_DictMetadata_descriptor; - } - - @java.lang.Override - protected com.google.protobuf.GeneratedMessage.FieldAccessorTable - internalGetFieldAccessorTable() { - return io.github.dfa1.vortex.proto.EncodingProtos.internal_static_vortex_encodings_DictMetadata_fieldAccessorTable - .ensureFieldAccessorsInitialized( - io.github.dfa1.vortex.proto.EncodingProtos.DictMetadata.class, io.github.dfa1.vortex.proto.EncodingProtos.DictMetadata.Builder.class); - } - - // Construct using io.github.dfa1.vortex.proto.EncodingProtos.DictMetadata.newBuilder() - private Builder() { - - } - - private Builder( - com.google.protobuf.GeneratedMessage.BuilderParent parent) { - super(parent); - - } - @java.lang.Override - public Builder clear() { - super.clear(); - bitField0_ = 0; - valuesLen_ = 0; - codesPtype_ = 0; - isNullableCodes_ = false; - allValuesReferenced_ = false; - return this; - } - - @java.lang.Override - public com.google.protobuf.Descriptors.Descriptor - getDescriptorForType() { - return io.github.dfa1.vortex.proto.EncodingProtos.internal_static_vortex_encodings_DictMetadata_descriptor; - } - - @java.lang.Override - public io.github.dfa1.vortex.proto.EncodingProtos.DictMetadata getDefaultInstanceForType() { - return io.github.dfa1.vortex.proto.EncodingProtos.DictMetadata.getDefaultInstance(); - } - - @java.lang.Override - public io.github.dfa1.vortex.proto.EncodingProtos.DictMetadata build() { - io.github.dfa1.vortex.proto.EncodingProtos.DictMetadata result = buildPartial(); - if (!result.isInitialized()) { - throw newUninitializedMessageException(result); - } - return result; - } - - @java.lang.Override - public io.github.dfa1.vortex.proto.EncodingProtos.DictMetadata buildPartial() { - io.github.dfa1.vortex.proto.EncodingProtos.DictMetadata result = new io.github.dfa1.vortex.proto.EncodingProtos.DictMetadata(this); - if (bitField0_ != 0) { buildPartial0(result); } - onBuilt(); - return result; - } - - private void buildPartial0(io.github.dfa1.vortex.proto.EncodingProtos.DictMetadata result) { - int from_bitField0_ = bitField0_; - if (((from_bitField0_ & 0x00000001) != 0)) { - result.valuesLen_ = valuesLen_; - } - if (((from_bitField0_ & 0x00000002) != 0)) { - result.codesPtype_ = codesPtype_; - } - int to_bitField0_ = 0; - if (((from_bitField0_ & 0x00000004) != 0)) { - result.isNullableCodes_ = isNullableCodes_; - to_bitField0_ |= 0x00000001; - } - if (((from_bitField0_ & 0x00000008) != 0)) { - result.allValuesReferenced_ = allValuesReferenced_; - to_bitField0_ |= 0x00000002; - } - result.bitField0_ |= to_bitField0_; - } - - @java.lang.Override - public Builder mergeFrom(com.google.protobuf.Message other) { - if (other instanceof io.github.dfa1.vortex.proto.EncodingProtos.DictMetadata) { - return mergeFrom((io.github.dfa1.vortex.proto.EncodingProtos.DictMetadata)other); - } else { - super.mergeFrom(other); - return this; - } - } - - public Builder mergeFrom(io.github.dfa1.vortex.proto.EncodingProtos.DictMetadata other) { - if (other == io.github.dfa1.vortex.proto.EncodingProtos.DictMetadata.getDefaultInstance()) return this; - if (other.getValuesLen() != 0) { - setValuesLen(other.getValuesLen()); - } - if (other.codesPtype_ != 0) { - setCodesPtypeValue(other.getCodesPtypeValue()); - } - if (other.hasIsNullableCodes()) { - setIsNullableCodes(other.getIsNullableCodes()); - } - if (other.hasAllValuesReferenced()) { - setAllValuesReferenced(other.getAllValuesReferenced()); - } - this.mergeUnknownFields(other.getUnknownFields()); - onChanged(); - return this; - } - - @java.lang.Override - public final boolean isInitialized() { - return true; - } - - @java.lang.Override - public Builder mergeFrom( - com.google.protobuf.CodedInputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws java.io.IOException { - if (extensionRegistry == null) { - throw new java.lang.NullPointerException(); - } - try { - boolean done = false; - while (!done) { - int tag = input.readTag(); - switch (tag) { - case 0: - done = true; - break; - case 8: { - valuesLen_ = input.readUInt32(); - bitField0_ |= 0x00000001; - break; - } // case 8 - case 16: { - codesPtype_ = input.readEnum(); - bitField0_ |= 0x00000002; - break; - } // case 16 - case 24: { - isNullableCodes_ = input.readBool(); - bitField0_ |= 0x00000004; - break; - } // case 24 - case 32: { - allValuesReferenced_ = input.readBool(); - bitField0_ |= 0x00000008; - break; - } // case 32 - default: { - if (!super.parseUnknownField(input, extensionRegistry, tag)) { - done = true; // was an endgroup tag - } - break; - } // default: - } // switch (tag) - } // while (!done) - } catch (com.google.protobuf.InvalidProtocolBufferException e) { - throw e.unwrapIOException(); - } finally { - onChanged(); - } // finally - return this; - } - private int bitField0_; - - private int valuesLen_ ; - /** - * uint32 values_len = 1; - * @return The valuesLen. - */ - @java.lang.Override - public int getValuesLen() { - return valuesLen_; - } - /** - * uint32 values_len = 1; - * @param value The valuesLen to set. - * @return This builder for chaining. - */ - public Builder setValuesLen(int value) { - - valuesLen_ = value; - bitField0_ |= 0x00000001; - onChanged(); - return this; - } - /** - * uint32 values_len = 1; - * @return This builder for chaining. - */ - public Builder clearValuesLen() { - bitField0_ = (bitField0_ & ~0x00000001); - valuesLen_ = 0; - onChanged(); - return this; - } - - private int codesPtype_ = 0; - /** - * .vortex.dtype.PType codes_ptype = 2; - * @return The enum numeric value on the wire for codesPtype. - */ - @java.lang.Override public int getCodesPtypeValue() { - return codesPtype_; - } - /** - * .vortex.dtype.PType codes_ptype = 2; - * @param value The enum numeric value on the wire for codesPtype to set. - * @return This builder for chaining. - */ - public Builder setCodesPtypeValue(int value) { - codesPtype_ = value; - bitField0_ |= 0x00000002; - onChanged(); - return this; - } - /** - * .vortex.dtype.PType codes_ptype = 2; - * @return The codesPtype. - */ - @java.lang.Override - public io.github.dfa1.vortex.proto.DTypeProtos.PType getCodesPtype() { - io.github.dfa1.vortex.proto.DTypeProtos.PType result = io.github.dfa1.vortex.proto.DTypeProtos.PType.forNumber(codesPtype_); - return result == null ? io.github.dfa1.vortex.proto.DTypeProtos.PType.UNRECOGNIZED : result; - } - /** - * .vortex.dtype.PType codes_ptype = 2; - * @param value The codesPtype to set. - * @throws IllegalArgumentException if UNRECOGNIZED is provided. - * @return This builder for chaining. - */ - public Builder setCodesPtype(io.github.dfa1.vortex.proto.DTypeProtos.PType value) { - if (value == null) { throw new NullPointerException(); } - bitField0_ |= 0x00000002; - codesPtype_ = value.getNumber(); - onChanged(); - return this; - } - /** - * .vortex.dtype.PType codes_ptype = 2; - * @return This builder for chaining. - */ - public Builder clearCodesPtype() { - bitField0_ = (bitField0_ & ~0x00000002); - codesPtype_ = 0; - onChanged(); - return this; - } - - private boolean isNullableCodes_ ; - /** - * optional bool is_nullable_codes = 3; - * @return Whether the isNullableCodes field is set. - */ - @java.lang.Override - public boolean hasIsNullableCodes() { - return ((bitField0_ & 0x00000004) != 0); - } - /** - * optional bool is_nullable_codes = 3; - * @return The isNullableCodes. - */ - @java.lang.Override - public boolean getIsNullableCodes() { - return isNullableCodes_; - } - /** - * optional bool is_nullable_codes = 3; - * @param value The isNullableCodes to set. - * @return This builder for chaining. - */ - public Builder setIsNullableCodes(boolean value) { - - isNullableCodes_ = value; - bitField0_ |= 0x00000004; - onChanged(); - return this; - } - /** - * optional bool is_nullable_codes = 3; - * @return This builder for chaining. - */ - public Builder clearIsNullableCodes() { - bitField0_ = (bitField0_ & ~0x00000004); - isNullableCodes_ = false; - onChanged(); - return this; - } - - private boolean allValuesReferenced_ ; - /** - * optional bool all_values_referenced = 4; - * @return Whether the allValuesReferenced field is set. - */ - @java.lang.Override - public boolean hasAllValuesReferenced() { - return ((bitField0_ & 0x00000008) != 0); - } - /** - * optional bool all_values_referenced = 4; - * @return The allValuesReferenced. - */ - @java.lang.Override - public boolean getAllValuesReferenced() { - return allValuesReferenced_; - } - /** - * optional bool all_values_referenced = 4; - * @param value The allValuesReferenced to set. - * @return This builder for chaining. - */ - public Builder setAllValuesReferenced(boolean value) { - - allValuesReferenced_ = value; - bitField0_ |= 0x00000008; - onChanged(); - return this; - } - /** - * optional bool all_values_referenced = 4; - * @return This builder for chaining. - */ - public Builder clearAllValuesReferenced() { - bitField0_ = (bitField0_ & ~0x00000008); - allValuesReferenced_ = false; - onChanged(); - return this; - } - - // @@protoc_insertion_point(builder_scope:vortex.encodings.DictMetadata) - } - - // @@protoc_insertion_point(class_scope:vortex.encodings.DictMetadata) - private static final io.github.dfa1.vortex.proto.EncodingProtos.DictMetadata DEFAULT_INSTANCE; - static { - DEFAULT_INSTANCE = new io.github.dfa1.vortex.proto.EncodingProtos.DictMetadata(); - } - - public static io.github.dfa1.vortex.proto.EncodingProtos.DictMetadata getDefaultInstance() { - return DEFAULT_INSTANCE; - } - - private static final com.google.protobuf.Parser - PARSER = new com.google.protobuf.AbstractParser() { - @java.lang.Override - public DictMetadata parsePartialFrom( - com.google.protobuf.CodedInputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws com.google.protobuf.InvalidProtocolBufferException { - Builder builder = newBuilder(); - try { - builder.mergeFrom(input, extensionRegistry); - } catch (com.google.protobuf.InvalidProtocolBufferException e) { - throw e.setUnfinishedMessage(builder.buildPartial()); - } catch (com.google.protobuf.UninitializedMessageException e) { - throw e.asInvalidProtocolBufferException().setUnfinishedMessage(builder.buildPartial()); - } catch (java.io.IOException e) { - throw new com.google.protobuf.InvalidProtocolBufferException(e) - .setUnfinishedMessage(builder.buildPartial()); - } - return builder.buildPartial(); - } - }; - - public static com.google.protobuf.Parser parser() { - return PARSER; - } - - @java.lang.Override - public com.google.protobuf.Parser getParserForType() { - return PARSER; - } - - @java.lang.Override - public io.github.dfa1.vortex.proto.EncodingProtos.DictMetadata getDefaultInstanceForType() { - return DEFAULT_INSTANCE; - } - - } - - public interface RunEndMetadataOrBuilder extends - // @@protoc_insertion_point(interface_extends:vortex.encodings.RunEndMetadata) - com.google.protobuf.MessageOrBuilder { - - /** - * .vortex.dtype.PType ends_ptype = 1; - * @return The enum numeric value on the wire for endsPtype. - */ - int getEndsPtypeValue(); - /** - * .vortex.dtype.PType ends_ptype = 1; - * @return The endsPtype. - */ - io.github.dfa1.vortex.proto.DTypeProtos.PType getEndsPtype(); - - /** - * uint64 num_runs = 2; - * @return The numRuns. - */ - long getNumRuns(); - - /** - * uint64 offset = 3; - * @return The offset. - */ - long getOffset(); - } - /** - * Protobuf type {@code vortex.encodings.RunEndMetadata} - */ - public static final class RunEndMetadata extends - com.google.protobuf.GeneratedMessage implements - // @@protoc_insertion_point(message_implements:vortex.encodings.RunEndMetadata) - RunEndMetadataOrBuilder { - private static final long serialVersionUID = 0L; - static { - com.google.protobuf.RuntimeVersion.validateProtobufGencodeVersion( - com.google.protobuf.RuntimeVersion.RuntimeDomain.PUBLIC, - /* major= */ 4, - /* minor= */ 35, - /* patch= */ 0, - /* suffix= */ "", - "RunEndMetadata"); - } - // Use RunEndMetadata.newBuilder() to construct. - private RunEndMetadata(com.google.protobuf.GeneratedMessage.Builder builder) { - super(builder); - } - private RunEndMetadata() { - endsPtype_ = 0; - } - - public static final com.google.protobuf.Descriptors.Descriptor - getDescriptor() { - return io.github.dfa1.vortex.proto.EncodingProtos.internal_static_vortex_encodings_RunEndMetadata_descriptor; - } - - @java.lang.Override - public com.google.protobuf.Descriptors.Descriptor getDescriptorForType() { - return io.github.dfa1.vortex.proto.EncodingProtos.internal_static_vortex_encodings_RunEndMetadata_descriptor; - } - - @java.lang.Override - protected com.google.protobuf.GeneratedMessage.FieldAccessorTable - internalGetFieldAccessorTable() { - return io.github.dfa1.vortex.proto.EncodingProtos.internal_static_vortex_encodings_RunEndMetadata_fieldAccessorTable - .ensureFieldAccessorsInitialized( - io.github.dfa1.vortex.proto.EncodingProtos.RunEndMetadata.class, io.github.dfa1.vortex.proto.EncodingProtos.RunEndMetadata.Builder.class); - } - - public static final int ENDS_PTYPE_FIELD_NUMBER = 1; - private int endsPtype_ = 0; - /** - * .vortex.dtype.PType ends_ptype = 1; - * @return The enum numeric value on the wire for endsPtype. - */ - @java.lang.Override public int getEndsPtypeValue() { - return endsPtype_; - } - /** - * .vortex.dtype.PType ends_ptype = 1; - * @return The endsPtype. - */ - @java.lang.Override public io.github.dfa1.vortex.proto.DTypeProtos.PType getEndsPtype() { - io.github.dfa1.vortex.proto.DTypeProtos.PType result = io.github.dfa1.vortex.proto.DTypeProtos.PType.forNumber(endsPtype_); - return result == null ? io.github.dfa1.vortex.proto.DTypeProtos.PType.UNRECOGNIZED : result; - } - - public static final int NUM_RUNS_FIELD_NUMBER = 2; - private long numRuns_ = 0L; - /** - * uint64 num_runs = 2; - * @return The numRuns. - */ - @java.lang.Override - public long getNumRuns() { - return numRuns_; - } - - public static final int OFFSET_FIELD_NUMBER = 3; - private long offset_ = 0L; - /** - * uint64 offset = 3; - * @return The offset. - */ - @java.lang.Override - public long getOffset() { - return offset_; - } - - private byte memoizedIsInitialized = -1; - @java.lang.Override - public final boolean isInitialized() { - byte isInitialized = memoizedIsInitialized; - if (isInitialized == 1) return true; - if (isInitialized == 0) return false; - - memoizedIsInitialized = 1; - return true; - } - - @java.lang.Override - public void writeTo(com.google.protobuf.CodedOutputStream output) - throws java.io.IOException { - if (endsPtype_ != io.github.dfa1.vortex.proto.DTypeProtos.PType.U8.getNumber()) { - output.writeEnum(1, endsPtype_); - } - if (numRuns_ != 0L) { - output.writeUInt64(2, numRuns_); - } - if (offset_ != 0L) { - output.writeUInt64(3, offset_); - } - getUnknownFields().writeTo(output); - } - private int computeSerializedSize_0() { - int size = 0; - if (endsPtype_ != io.github.dfa1.vortex.proto.DTypeProtos.PType.U8.getNumber()) { - size += com.google.protobuf.CodedOutputStream - .computeEnumSize(1, endsPtype_); - } - if (numRuns_ != 0L) { - size += com.google.protobuf.CodedOutputStream - .computeUInt64Size(2, numRuns_); - } - if (offset_ != 0L) { - size += com.google.protobuf.CodedOutputStream - .computeUInt64Size(3, offset_); - } - return size; - } - @java.lang.Override - public int getSerializedSize() { - int size = memoizedSize; - if (size != -1) return size; - - size = 0; - size += computeSerializedSize_0(); - size += getUnknownFields().getSerializedSize(); - memoizedSize = size; - return size; - } - - @java.lang.Override - public boolean equals(final java.lang.Object obj) { - if (obj == this) { - return true; - } - if (!(obj instanceof io.github.dfa1.vortex.proto.EncodingProtos.RunEndMetadata)) { - return super.equals(obj); - } - io.github.dfa1.vortex.proto.EncodingProtos.RunEndMetadata other = (io.github.dfa1.vortex.proto.EncodingProtos.RunEndMetadata) obj; - - if (endsPtype_ != other.endsPtype_) return false; - if (getNumRuns() - != other.getNumRuns()) return false; - if (getOffset() - != other.getOffset()) return false; - if (!getUnknownFields().equals(other.getUnknownFields())) return false; - return true; - } - - @java.lang.Override - public int hashCode() { - if (memoizedHashCode != 0) { - return memoizedHashCode; - } - int hash = 41; - hash = (19 * hash) + getDescriptor().hashCode(); - hash = (37 * hash) + ENDS_PTYPE_FIELD_NUMBER; - hash = (53 * hash) + endsPtype_; - hash = (37 * hash) + NUM_RUNS_FIELD_NUMBER; - hash = (53 * hash) + com.google.protobuf.Internal.hashLong( - getNumRuns()); - hash = (37 * hash) + OFFSET_FIELD_NUMBER; - hash = (53 * hash) + com.google.protobuf.Internal.hashLong( - getOffset()); - hash = (29 * hash) + getUnknownFields().hashCode(); - memoizedHashCode = hash; - return hash; - } - - public static io.github.dfa1.vortex.proto.EncodingProtos.RunEndMetadata parseFrom( - java.nio.ByteBuffer data) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.RunEndMetadata parseFrom( - java.nio.ByteBuffer data, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data, extensionRegistry); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.RunEndMetadata parseFrom( - com.google.protobuf.ByteString data) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.RunEndMetadata parseFrom( - com.google.protobuf.ByteString data, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data, extensionRegistry); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.RunEndMetadata parseFrom(byte[] data) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.RunEndMetadata parseFrom( - byte[] data, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data, extensionRegistry); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.RunEndMetadata parseFrom(java.io.InputStream input) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseWithIOException(PARSER, input); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.RunEndMetadata parseFrom( - java.io.InputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseWithIOException(PARSER, input, extensionRegistry); - } - - public static io.github.dfa1.vortex.proto.EncodingProtos.RunEndMetadata parseDelimitedFrom(java.io.InputStream input) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseDelimitedWithIOException(PARSER, input); - } - - public static io.github.dfa1.vortex.proto.EncodingProtos.RunEndMetadata parseDelimitedFrom( - java.io.InputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseDelimitedWithIOException(PARSER, input, extensionRegistry); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.RunEndMetadata parseFrom( - com.google.protobuf.CodedInputStream input) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseWithIOException(PARSER, input); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.RunEndMetadata parseFrom( - com.google.protobuf.CodedInputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseWithIOException(PARSER, input, extensionRegistry); - } - - @java.lang.Override - public Builder newBuilderForType() { return newBuilder(); } - public static Builder newBuilder() { - return DEFAULT_INSTANCE.toBuilder(); - } - public static Builder newBuilder(io.github.dfa1.vortex.proto.EncodingProtos.RunEndMetadata prototype) { - return DEFAULT_INSTANCE.toBuilder().mergeFrom(prototype); - } - @java.lang.Override - public Builder toBuilder() { - return this == DEFAULT_INSTANCE - ? new Builder() : new Builder().mergeFrom(this); - } - - @java.lang.Override - protected Builder newBuilderForType( - com.google.protobuf.GeneratedMessage.BuilderParent parent) { - Builder builder = new Builder(parent); - return builder; - } - /** - * Protobuf type {@code vortex.encodings.RunEndMetadata} - */ - public static final class Builder extends - com.google.protobuf.GeneratedMessage.Builder implements - // @@protoc_insertion_point(builder_implements:vortex.encodings.RunEndMetadata) - io.github.dfa1.vortex.proto.EncodingProtos.RunEndMetadataOrBuilder { - public static final com.google.protobuf.Descriptors.Descriptor - getDescriptor() { - return io.github.dfa1.vortex.proto.EncodingProtos.internal_static_vortex_encodings_RunEndMetadata_descriptor; - } - - @java.lang.Override - protected com.google.protobuf.GeneratedMessage.FieldAccessorTable - internalGetFieldAccessorTable() { - return io.github.dfa1.vortex.proto.EncodingProtos.internal_static_vortex_encodings_RunEndMetadata_fieldAccessorTable - .ensureFieldAccessorsInitialized( - io.github.dfa1.vortex.proto.EncodingProtos.RunEndMetadata.class, io.github.dfa1.vortex.proto.EncodingProtos.RunEndMetadata.Builder.class); - } - - // Construct using io.github.dfa1.vortex.proto.EncodingProtos.RunEndMetadata.newBuilder() - private Builder() { - - } - - private Builder( - com.google.protobuf.GeneratedMessage.BuilderParent parent) { - super(parent); - - } - @java.lang.Override - public Builder clear() { - super.clear(); - bitField0_ = 0; - endsPtype_ = 0; - numRuns_ = 0L; - offset_ = 0L; - return this; - } - - @java.lang.Override - public com.google.protobuf.Descriptors.Descriptor - getDescriptorForType() { - return io.github.dfa1.vortex.proto.EncodingProtos.internal_static_vortex_encodings_RunEndMetadata_descriptor; - } - - @java.lang.Override - public io.github.dfa1.vortex.proto.EncodingProtos.RunEndMetadata getDefaultInstanceForType() { - return io.github.dfa1.vortex.proto.EncodingProtos.RunEndMetadata.getDefaultInstance(); - } - - @java.lang.Override - public io.github.dfa1.vortex.proto.EncodingProtos.RunEndMetadata build() { - io.github.dfa1.vortex.proto.EncodingProtos.RunEndMetadata result = buildPartial(); - if (!result.isInitialized()) { - throw newUninitializedMessageException(result); - } - return result; - } - - @java.lang.Override - public io.github.dfa1.vortex.proto.EncodingProtos.RunEndMetadata buildPartial() { - io.github.dfa1.vortex.proto.EncodingProtos.RunEndMetadata result = new io.github.dfa1.vortex.proto.EncodingProtos.RunEndMetadata(this); - if (bitField0_ != 0) { buildPartial0(result); } - onBuilt(); - return result; - } - - private void buildPartial0(io.github.dfa1.vortex.proto.EncodingProtos.RunEndMetadata result) { - int from_bitField0_ = bitField0_; - if (((from_bitField0_ & 0x00000001) != 0)) { - result.endsPtype_ = endsPtype_; - } - if (((from_bitField0_ & 0x00000002) != 0)) { - result.numRuns_ = numRuns_; - } - if (((from_bitField0_ & 0x00000004) != 0)) { - result.offset_ = offset_; - } - } - - @java.lang.Override - public Builder mergeFrom(com.google.protobuf.Message other) { - if (other instanceof io.github.dfa1.vortex.proto.EncodingProtos.RunEndMetadata) { - return mergeFrom((io.github.dfa1.vortex.proto.EncodingProtos.RunEndMetadata)other); - } else { - super.mergeFrom(other); - return this; - } - } - - public Builder mergeFrom(io.github.dfa1.vortex.proto.EncodingProtos.RunEndMetadata other) { - if (other == io.github.dfa1.vortex.proto.EncodingProtos.RunEndMetadata.getDefaultInstance()) return this; - if (other.endsPtype_ != 0) { - setEndsPtypeValue(other.getEndsPtypeValue()); - } - if (other.getNumRuns() != 0L) { - setNumRuns(other.getNumRuns()); - } - if (other.getOffset() != 0L) { - setOffset(other.getOffset()); - } - this.mergeUnknownFields(other.getUnknownFields()); - onChanged(); - return this; - } - - @java.lang.Override - public final boolean isInitialized() { - return true; - } - - @java.lang.Override - public Builder mergeFrom( - com.google.protobuf.CodedInputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws java.io.IOException { - if (extensionRegistry == null) { - throw new java.lang.NullPointerException(); - } - try { - boolean done = false; - while (!done) { - int tag = input.readTag(); - switch (tag) { - case 0: - done = true; - break; - case 8: { - endsPtype_ = input.readEnum(); - bitField0_ |= 0x00000001; - break; - } // case 8 - case 16: { - numRuns_ = input.readUInt64(); - bitField0_ |= 0x00000002; - break; - } // case 16 - case 24: { - offset_ = input.readUInt64(); - bitField0_ |= 0x00000004; - break; - } // case 24 - default: { - if (!super.parseUnknownField(input, extensionRegistry, tag)) { - done = true; // was an endgroup tag - } - break; - } // default: - } // switch (tag) - } // while (!done) - } catch (com.google.protobuf.InvalidProtocolBufferException e) { - throw e.unwrapIOException(); - } finally { - onChanged(); - } // finally - return this; - } - private int bitField0_; - - private int endsPtype_ = 0; - /** - * .vortex.dtype.PType ends_ptype = 1; - * @return The enum numeric value on the wire for endsPtype. - */ - @java.lang.Override public int getEndsPtypeValue() { - return endsPtype_; - } - /** - * .vortex.dtype.PType ends_ptype = 1; - * @param value The enum numeric value on the wire for endsPtype to set. - * @return This builder for chaining. - */ - public Builder setEndsPtypeValue(int value) { - endsPtype_ = value; - bitField0_ |= 0x00000001; - onChanged(); - return this; - } - /** - * .vortex.dtype.PType ends_ptype = 1; - * @return The endsPtype. - */ - @java.lang.Override - public io.github.dfa1.vortex.proto.DTypeProtos.PType getEndsPtype() { - io.github.dfa1.vortex.proto.DTypeProtos.PType result = io.github.dfa1.vortex.proto.DTypeProtos.PType.forNumber(endsPtype_); - return result == null ? io.github.dfa1.vortex.proto.DTypeProtos.PType.UNRECOGNIZED : result; - } - /** - * .vortex.dtype.PType ends_ptype = 1; - * @param value The endsPtype to set. - * @throws IllegalArgumentException if UNRECOGNIZED is provided. - * @return This builder for chaining. - */ - public Builder setEndsPtype(io.github.dfa1.vortex.proto.DTypeProtos.PType value) { - if (value == null) { throw new NullPointerException(); } - bitField0_ |= 0x00000001; - endsPtype_ = value.getNumber(); - onChanged(); - return this; - } - /** - * .vortex.dtype.PType ends_ptype = 1; - * @return This builder for chaining. - */ - public Builder clearEndsPtype() { - bitField0_ = (bitField0_ & ~0x00000001); - endsPtype_ = 0; - onChanged(); - return this; - } - - private long numRuns_ ; - /** - * uint64 num_runs = 2; - * @return The numRuns. - */ - @java.lang.Override - public long getNumRuns() { - return numRuns_; - } - /** - * uint64 num_runs = 2; - * @param value The numRuns to set. - * @return This builder for chaining. - */ - public Builder setNumRuns(long value) { - - numRuns_ = value; - bitField0_ |= 0x00000002; - onChanged(); - return this; - } - /** - * uint64 num_runs = 2; - * @return This builder for chaining. - */ - public Builder clearNumRuns() { - bitField0_ = (bitField0_ & ~0x00000002); - numRuns_ = 0L; - onChanged(); - return this; - } - - private long offset_ ; - /** - * uint64 offset = 3; - * @return The offset. - */ - @java.lang.Override - public long getOffset() { - return offset_; - } - /** - * uint64 offset = 3; - * @param value The offset to set. - * @return This builder for chaining. - */ - public Builder setOffset(long value) { - - offset_ = value; - bitField0_ |= 0x00000004; - onChanged(); - return this; - } - /** - * uint64 offset = 3; - * @return This builder for chaining. - */ - public Builder clearOffset() { - bitField0_ = (bitField0_ & ~0x00000004); - offset_ = 0L; - onChanged(); - return this; - } - - // @@protoc_insertion_point(builder_scope:vortex.encodings.RunEndMetadata) - } - - // @@protoc_insertion_point(class_scope:vortex.encodings.RunEndMetadata) - private static final io.github.dfa1.vortex.proto.EncodingProtos.RunEndMetadata DEFAULT_INSTANCE; - static { - DEFAULT_INSTANCE = new io.github.dfa1.vortex.proto.EncodingProtos.RunEndMetadata(); - } - - public static io.github.dfa1.vortex.proto.EncodingProtos.RunEndMetadata getDefaultInstance() { - return DEFAULT_INSTANCE; - } - - private static final com.google.protobuf.Parser - PARSER = new com.google.protobuf.AbstractParser() { - @java.lang.Override - public RunEndMetadata parsePartialFrom( - com.google.protobuf.CodedInputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws com.google.protobuf.InvalidProtocolBufferException { - Builder builder = newBuilder(); - try { - builder.mergeFrom(input, extensionRegistry); - } catch (com.google.protobuf.InvalidProtocolBufferException e) { - throw e.setUnfinishedMessage(builder.buildPartial()); - } catch (com.google.protobuf.UninitializedMessageException e) { - throw e.asInvalidProtocolBufferException().setUnfinishedMessage(builder.buildPartial()); - } catch (java.io.IOException e) { - throw new com.google.protobuf.InvalidProtocolBufferException(e) - .setUnfinishedMessage(builder.buildPartial()); - } - return builder.buildPartial(); - } - }; - - public static com.google.protobuf.Parser parser() { - return PARSER; - } - - @java.lang.Override - public com.google.protobuf.Parser getParserForType() { - return PARSER; - } - - @java.lang.Override - public io.github.dfa1.vortex.proto.EncodingProtos.RunEndMetadata getDefaultInstanceForType() { - return DEFAULT_INSTANCE; - } - - } - - public interface ALPMetadataOrBuilder extends - // @@protoc_insertion_point(interface_extends:vortex.encodings.ALPMetadata) - com.google.protobuf.MessageOrBuilder { - - /** - * uint32 exp_e = 1; - * @return The expE. - */ - int getExpE(); - - /** - * uint32 exp_f = 2; - * @return The expF. - */ - int getExpF(); - - /** - * optional .vortex.encodings.PatchesMetadata patches = 3; - * @return Whether the patches field is set. - */ - boolean hasPatches(); - /** - * optional .vortex.encodings.PatchesMetadata patches = 3; - * @return The patches. - */ - io.github.dfa1.vortex.proto.EncodingProtos.PatchesMetadata getPatches(); - /** - * optional .vortex.encodings.PatchesMetadata patches = 3; - */ - io.github.dfa1.vortex.proto.EncodingProtos.PatchesMetadataOrBuilder getPatchesOrBuilder(); - } - /** - * Protobuf type {@code vortex.encodings.ALPMetadata} - */ - public static final class ALPMetadata extends - com.google.protobuf.GeneratedMessage implements - // @@protoc_insertion_point(message_implements:vortex.encodings.ALPMetadata) - ALPMetadataOrBuilder { - private static final long serialVersionUID = 0L; - static { - com.google.protobuf.RuntimeVersion.validateProtobufGencodeVersion( - com.google.protobuf.RuntimeVersion.RuntimeDomain.PUBLIC, - /* major= */ 4, - /* minor= */ 35, - /* patch= */ 0, - /* suffix= */ "", - "ALPMetadata"); - } - // Use ALPMetadata.newBuilder() to construct. - private ALPMetadata(com.google.protobuf.GeneratedMessage.Builder builder) { - super(builder); - } - private ALPMetadata() { - } - - public static final com.google.protobuf.Descriptors.Descriptor - getDescriptor() { - return io.github.dfa1.vortex.proto.EncodingProtos.internal_static_vortex_encodings_ALPMetadata_descriptor; - } - - @java.lang.Override - public com.google.protobuf.Descriptors.Descriptor getDescriptorForType() { - return io.github.dfa1.vortex.proto.EncodingProtos.internal_static_vortex_encodings_ALPMetadata_descriptor; - } - - @java.lang.Override - protected com.google.protobuf.GeneratedMessage.FieldAccessorTable - internalGetFieldAccessorTable() { - return io.github.dfa1.vortex.proto.EncodingProtos.internal_static_vortex_encodings_ALPMetadata_fieldAccessorTable - .ensureFieldAccessorsInitialized( - io.github.dfa1.vortex.proto.EncodingProtos.ALPMetadata.class, io.github.dfa1.vortex.proto.EncodingProtos.ALPMetadata.Builder.class); - } - - private int bitField0_; - public static final int EXP_E_FIELD_NUMBER = 1; - private int expE_ = 0; - /** - * uint32 exp_e = 1; - * @return The expE. - */ - @java.lang.Override - public int getExpE() { - return expE_; - } - - public static final int EXP_F_FIELD_NUMBER = 2; - private int expF_ = 0; - /** - * uint32 exp_f = 2; - * @return The expF. - */ - @java.lang.Override - public int getExpF() { - return expF_; - } - - public static final int PATCHES_FIELD_NUMBER = 3; - private io.github.dfa1.vortex.proto.EncodingProtos.PatchesMetadata patches_; - /** - * optional .vortex.encodings.PatchesMetadata patches = 3; - * @return Whether the patches field is set. - */ - @java.lang.Override - public boolean hasPatches() { - return ((bitField0_ & 0x00000001) != 0); - } - /** - * optional .vortex.encodings.PatchesMetadata patches = 3; - * @return The patches. - */ - @java.lang.Override - public io.github.dfa1.vortex.proto.EncodingProtos.PatchesMetadata getPatches() { - return patches_ == null ? io.github.dfa1.vortex.proto.EncodingProtos.PatchesMetadata.getDefaultInstance() : patches_; - } - /** - * optional .vortex.encodings.PatchesMetadata patches = 3; - */ - @java.lang.Override - public io.github.dfa1.vortex.proto.EncodingProtos.PatchesMetadataOrBuilder getPatchesOrBuilder() { - return patches_ == null ? io.github.dfa1.vortex.proto.EncodingProtos.PatchesMetadata.getDefaultInstance() : patches_; - } - - private byte memoizedIsInitialized = -1; - @java.lang.Override - public final boolean isInitialized() { - byte isInitialized = memoizedIsInitialized; - if (isInitialized == 1) return true; - if (isInitialized == 0) return false; - - memoizedIsInitialized = 1; - return true; - } - - @java.lang.Override - public void writeTo(com.google.protobuf.CodedOutputStream output) - throws java.io.IOException { - if (expE_ != 0) { - output.writeUInt32(1, expE_); - } - if (expF_ != 0) { - output.writeUInt32(2, expF_); - } - if (((bitField0_ & 0x00000001) != 0)) { - output.writeMessage(3, getPatches()); - } - getUnknownFields().writeTo(output); - } - private int computeSerializedSize_0() { - int size = 0; - if (expE_ != 0) { - size += com.google.protobuf.CodedOutputStream - .computeUInt32Size(1, expE_); - } - if (expF_ != 0) { - size += com.google.protobuf.CodedOutputStream - .computeUInt32Size(2, expF_); - } - if (((bitField0_ & 0x00000001) != 0)) { - size += com.google.protobuf.CodedOutputStream - .computeMessageSize(3, getPatches()); - } - return size; - } - @java.lang.Override - public int getSerializedSize() { - int size = memoizedSize; - if (size != -1) return size; - - size = 0; - size += computeSerializedSize_0(); - size += getUnknownFields().getSerializedSize(); - memoizedSize = size; - return size; - } - - @java.lang.Override - public boolean equals(final java.lang.Object obj) { - if (obj == this) { - return true; - } - if (!(obj instanceof io.github.dfa1.vortex.proto.EncodingProtos.ALPMetadata)) { - return super.equals(obj); - } - io.github.dfa1.vortex.proto.EncodingProtos.ALPMetadata other = (io.github.dfa1.vortex.proto.EncodingProtos.ALPMetadata) obj; - - if (getExpE() - != other.getExpE()) return false; - if (getExpF() - != other.getExpF()) return false; - if (hasPatches() != other.hasPatches()) return false; - if (hasPatches()) { - if (!getPatches() - .equals(other.getPatches())) return false; - } - if (!getUnknownFields().equals(other.getUnknownFields())) return false; - return true; - } - - @java.lang.Override - public int hashCode() { - if (memoizedHashCode != 0) { - return memoizedHashCode; - } - int hash = 41; - hash = (19 * hash) + getDescriptor().hashCode(); - hash = (37 * hash) + EXP_E_FIELD_NUMBER; - hash = (53 * hash) + getExpE(); - hash = (37 * hash) + EXP_F_FIELD_NUMBER; - hash = (53 * hash) + getExpF(); - if (hasPatches()) { - hash = (37 * hash) + PATCHES_FIELD_NUMBER; - hash = (53 * hash) + getPatches().hashCode(); - } - hash = (29 * hash) + getUnknownFields().hashCode(); - memoizedHashCode = hash; - return hash; - } - - public static io.github.dfa1.vortex.proto.EncodingProtos.ALPMetadata parseFrom( - java.nio.ByteBuffer data) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.ALPMetadata parseFrom( - java.nio.ByteBuffer data, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data, extensionRegistry); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.ALPMetadata parseFrom( - com.google.protobuf.ByteString data) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.ALPMetadata parseFrom( - com.google.protobuf.ByteString data, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data, extensionRegistry); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.ALPMetadata parseFrom(byte[] data) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.ALPMetadata parseFrom( - byte[] data, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data, extensionRegistry); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.ALPMetadata parseFrom(java.io.InputStream input) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseWithIOException(PARSER, input); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.ALPMetadata parseFrom( - java.io.InputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseWithIOException(PARSER, input, extensionRegistry); - } - - public static io.github.dfa1.vortex.proto.EncodingProtos.ALPMetadata parseDelimitedFrom(java.io.InputStream input) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseDelimitedWithIOException(PARSER, input); - } - - public static io.github.dfa1.vortex.proto.EncodingProtos.ALPMetadata parseDelimitedFrom( - java.io.InputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseDelimitedWithIOException(PARSER, input, extensionRegistry); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.ALPMetadata parseFrom( - com.google.protobuf.CodedInputStream input) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseWithIOException(PARSER, input); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.ALPMetadata parseFrom( - com.google.protobuf.CodedInputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseWithIOException(PARSER, input, extensionRegistry); - } - - @java.lang.Override - public Builder newBuilderForType() { return newBuilder(); } - public static Builder newBuilder() { - return DEFAULT_INSTANCE.toBuilder(); - } - public static Builder newBuilder(io.github.dfa1.vortex.proto.EncodingProtos.ALPMetadata prototype) { - return DEFAULT_INSTANCE.toBuilder().mergeFrom(prototype); - } - @java.lang.Override - public Builder toBuilder() { - return this == DEFAULT_INSTANCE - ? new Builder() : new Builder().mergeFrom(this); - } - - @java.lang.Override - protected Builder newBuilderForType( - com.google.protobuf.GeneratedMessage.BuilderParent parent) { - Builder builder = new Builder(parent); - return builder; - } - /** - * Protobuf type {@code vortex.encodings.ALPMetadata} - */ - public static final class Builder extends - com.google.protobuf.GeneratedMessage.Builder implements - // @@protoc_insertion_point(builder_implements:vortex.encodings.ALPMetadata) - io.github.dfa1.vortex.proto.EncodingProtos.ALPMetadataOrBuilder { - public static final com.google.protobuf.Descriptors.Descriptor - getDescriptor() { - return io.github.dfa1.vortex.proto.EncodingProtos.internal_static_vortex_encodings_ALPMetadata_descriptor; - } - - @java.lang.Override - protected com.google.protobuf.GeneratedMessage.FieldAccessorTable - internalGetFieldAccessorTable() { - return io.github.dfa1.vortex.proto.EncodingProtos.internal_static_vortex_encodings_ALPMetadata_fieldAccessorTable - .ensureFieldAccessorsInitialized( - io.github.dfa1.vortex.proto.EncodingProtos.ALPMetadata.class, io.github.dfa1.vortex.proto.EncodingProtos.ALPMetadata.Builder.class); - } - - // Construct using io.github.dfa1.vortex.proto.EncodingProtos.ALPMetadata.newBuilder() - private Builder() { - maybeForceBuilderInitialization(); - } - - private Builder( - com.google.protobuf.GeneratedMessage.BuilderParent parent) { - super(parent); - maybeForceBuilderInitialization(); - } - private void maybeForceBuilderInitialization() { - if (com.google.protobuf.GeneratedMessage - .alwaysUseFieldBuilders) { - internalGetPatchesFieldBuilder(); - } - } - @java.lang.Override - public Builder clear() { - super.clear(); - bitField0_ = 0; - expE_ = 0; - expF_ = 0; - patches_ = null; - if (patchesBuilder_ != null) { - patchesBuilder_.dispose(); - patchesBuilder_ = null; - } - return this; - } - - @java.lang.Override - public com.google.protobuf.Descriptors.Descriptor - getDescriptorForType() { - return io.github.dfa1.vortex.proto.EncodingProtos.internal_static_vortex_encodings_ALPMetadata_descriptor; - } - - @java.lang.Override - public io.github.dfa1.vortex.proto.EncodingProtos.ALPMetadata getDefaultInstanceForType() { - return io.github.dfa1.vortex.proto.EncodingProtos.ALPMetadata.getDefaultInstance(); - } - - @java.lang.Override - public io.github.dfa1.vortex.proto.EncodingProtos.ALPMetadata build() { - io.github.dfa1.vortex.proto.EncodingProtos.ALPMetadata result = buildPartial(); - if (!result.isInitialized()) { - throw newUninitializedMessageException(result); - } - return result; - } - - @java.lang.Override - public io.github.dfa1.vortex.proto.EncodingProtos.ALPMetadata buildPartial() { - io.github.dfa1.vortex.proto.EncodingProtos.ALPMetadata result = new io.github.dfa1.vortex.proto.EncodingProtos.ALPMetadata(this); - if (bitField0_ != 0) { buildPartial0(result); } - onBuilt(); - return result; - } - - private void buildPartial0(io.github.dfa1.vortex.proto.EncodingProtos.ALPMetadata result) { - int from_bitField0_ = bitField0_; - if (((from_bitField0_ & 0x00000001) != 0)) { - result.expE_ = expE_; - } - if (((from_bitField0_ & 0x00000002) != 0)) { - result.expF_ = expF_; - } - int to_bitField0_ = 0; - if (((from_bitField0_ & 0x00000004) != 0)) { - result.patches_ = patchesBuilder_ == null - ? patches_ - : patchesBuilder_.build(); - to_bitField0_ |= 0x00000001; - } - result.bitField0_ |= to_bitField0_; - } - - @java.lang.Override - public Builder mergeFrom(com.google.protobuf.Message other) { - if (other instanceof io.github.dfa1.vortex.proto.EncodingProtos.ALPMetadata) { - return mergeFrom((io.github.dfa1.vortex.proto.EncodingProtos.ALPMetadata)other); - } else { - super.mergeFrom(other); - return this; - } - } - - public Builder mergeFrom(io.github.dfa1.vortex.proto.EncodingProtos.ALPMetadata other) { - if (other == io.github.dfa1.vortex.proto.EncodingProtos.ALPMetadata.getDefaultInstance()) return this; - if (other.getExpE() != 0) { - setExpE(other.getExpE()); - } - if (other.getExpF() != 0) { - setExpF(other.getExpF()); - } - if (other.hasPatches()) { - mergePatches(other.getPatches()); - } - this.mergeUnknownFields(other.getUnknownFields()); - onChanged(); - return this; - } - - @java.lang.Override - public final boolean isInitialized() { - return true; - } - - @java.lang.Override - public Builder mergeFrom( - com.google.protobuf.CodedInputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws java.io.IOException { - if (extensionRegistry == null) { - throw new java.lang.NullPointerException(); - } - try { - boolean done = false; - while (!done) { - int tag = input.readTag(); - switch (tag) { - case 0: - done = true; - break; - case 8: { - expE_ = input.readUInt32(); - bitField0_ |= 0x00000001; - break; - } // case 8 - case 16: { - expF_ = input.readUInt32(); - bitField0_ |= 0x00000002; - break; - } // case 16 - case 26: { - input.readMessage( - internalGetPatchesFieldBuilder().getBuilder(), - extensionRegistry); - bitField0_ |= 0x00000004; - break; - } // case 26 - default: { - if (!super.parseUnknownField(input, extensionRegistry, tag)) { - done = true; // was an endgroup tag - } - break; - } // default: - } // switch (tag) - } // while (!done) - } catch (com.google.protobuf.InvalidProtocolBufferException e) { - throw e.unwrapIOException(); - } finally { - onChanged(); - } // finally - return this; - } - private int bitField0_; - - private int expE_ ; - /** - * uint32 exp_e = 1; - * @return The expE. - */ - @java.lang.Override - public int getExpE() { - return expE_; - } - /** - * uint32 exp_e = 1; - * @param value The expE to set. - * @return This builder for chaining. - */ - public Builder setExpE(int value) { - - expE_ = value; - bitField0_ |= 0x00000001; - onChanged(); - return this; - } - /** - * uint32 exp_e = 1; - * @return This builder for chaining. - */ - public Builder clearExpE() { - bitField0_ = (bitField0_ & ~0x00000001); - expE_ = 0; - onChanged(); - return this; - } - - private int expF_ ; - /** - * uint32 exp_f = 2; - * @return The expF. - */ - @java.lang.Override - public int getExpF() { - return expF_; - } - /** - * uint32 exp_f = 2; - * @param value The expF to set. - * @return This builder for chaining. - */ - public Builder setExpF(int value) { - - expF_ = value; - bitField0_ |= 0x00000002; - onChanged(); - return this; - } - /** - * uint32 exp_f = 2; - * @return This builder for chaining. - */ - public Builder clearExpF() { - bitField0_ = (bitField0_ & ~0x00000002); - expF_ = 0; - onChanged(); - return this; - } - - private io.github.dfa1.vortex.proto.EncodingProtos.PatchesMetadata patches_; - private com.google.protobuf.SingleFieldBuilder< - io.github.dfa1.vortex.proto.EncodingProtos.PatchesMetadata, io.github.dfa1.vortex.proto.EncodingProtos.PatchesMetadata.Builder, io.github.dfa1.vortex.proto.EncodingProtos.PatchesMetadataOrBuilder> patchesBuilder_; - /** - * optional .vortex.encodings.PatchesMetadata patches = 3; - * @return Whether the patches field is set. - */ - public boolean hasPatches() { - return ((bitField0_ & 0x00000004) != 0); - } - /** - * optional .vortex.encodings.PatchesMetadata patches = 3; - * @return The patches. - */ - public io.github.dfa1.vortex.proto.EncodingProtos.PatchesMetadata getPatches() { - if (patchesBuilder_ == null) { - return patches_ == null ? io.github.dfa1.vortex.proto.EncodingProtos.PatchesMetadata.getDefaultInstance() : patches_; - } else { - return patchesBuilder_.getMessage(); - } - } - /** - * optional .vortex.encodings.PatchesMetadata patches = 3; - */ - public Builder setPatches(io.github.dfa1.vortex.proto.EncodingProtos.PatchesMetadata value) { - if (patchesBuilder_ == null) { - if (value == null) { - throw new NullPointerException(); - } - patches_ = value; - } else { - patchesBuilder_.setMessage(value); - } - bitField0_ |= 0x00000004; - onChanged(); - return this; - } - /** - * optional .vortex.encodings.PatchesMetadata patches = 3; - */ - public Builder setPatches( - io.github.dfa1.vortex.proto.EncodingProtos.PatchesMetadata.Builder builderForValue) { - if (patchesBuilder_ == null) { - patches_ = builderForValue.build(); - } else { - patchesBuilder_.setMessage(builderForValue.build()); - } - bitField0_ |= 0x00000004; - onChanged(); - return this; - } - /** - * optional .vortex.encodings.PatchesMetadata patches = 3; - */ - public Builder mergePatches(io.github.dfa1.vortex.proto.EncodingProtos.PatchesMetadata value) { - if (patchesBuilder_ == null) { - if (((bitField0_ & 0x00000004) != 0) && - patches_ != null && - patches_ != io.github.dfa1.vortex.proto.EncodingProtos.PatchesMetadata.getDefaultInstance()) { - getPatchesBuilder().mergeFrom(value); - } else { - patches_ = value; - } - } else { - patchesBuilder_.mergeFrom(value); - } - if (patches_ != null) { - bitField0_ |= 0x00000004; - onChanged(); - } - return this; - } - /** - * optional .vortex.encodings.PatchesMetadata patches = 3; - */ - public Builder clearPatches() { - bitField0_ = (bitField0_ & ~0x00000004); - patches_ = null; - if (patchesBuilder_ != null) { - patchesBuilder_.dispose(); - patchesBuilder_ = null; - } - onChanged(); - return this; - } - /** - * optional .vortex.encodings.PatchesMetadata patches = 3; - */ - public io.github.dfa1.vortex.proto.EncodingProtos.PatchesMetadata.Builder getPatchesBuilder() { - bitField0_ |= 0x00000004; - onChanged(); - return internalGetPatchesFieldBuilder().getBuilder(); - } - /** - * optional .vortex.encodings.PatchesMetadata patches = 3; - */ - public io.github.dfa1.vortex.proto.EncodingProtos.PatchesMetadataOrBuilder getPatchesOrBuilder() { - if (patchesBuilder_ != null) { - return patchesBuilder_.getMessageOrBuilder(); - } else { - return patches_ == null ? - io.github.dfa1.vortex.proto.EncodingProtos.PatchesMetadata.getDefaultInstance() : patches_; - } - } - /** - * optional .vortex.encodings.PatchesMetadata patches = 3; - */ - private com.google.protobuf.SingleFieldBuilder< - io.github.dfa1.vortex.proto.EncodingProtos.PatchesMetadata, io.github.dfa1.vortex.proto.EncodingProtos.PatchesMetadata.Builder, io.github.dfa1.vortex.proto.EncodingProtos.PatchesMetadataOrBuilder> - internalGetPatchesFieldBuilder() { - if (patchesBuilder_ == null) { - patchesBuilder_ = new com.google.protobuf.SingleFieldBuilder< - io.github.dfa1.vortex.proto.EncodingProtos.PatchesMetadata, io.github.dfa1.vortex.proto.EncodingProtos.PatchesMetadata.Builder, io.github.dfa1.vortex.proto.EncodingProtos.PatchesMetadataOrBuilder>( - getPatches(), - getParentForChildren(), - isClean()); - patches_ = null; - } - return patchesBuilder_; - } - - // @@protoc_insertion_point(builder_scope:vortex.encodings.ALPMetadata) - } - - // @@protoc_insertion_point(class_scope:vortex.encodings.ALPMetadata) - private static final io.github.dfa1.vortex.proto.EncodingProtos.ALPMetadata DEFAULT_INSTANCE; - static { - DEFAULT_INSTANCE = new io.github.dfa1.vortex.proto.EncodingProtos.ALPMetadata(); - } - - public static io.github.dfa1.vortex.proto.EncodingProtos.ALPMetadata getDefaultInstance() { - return DEFAULT_INSTANCE; - } - - private static final com.google.protobuf.Parser - PARSER = new com.google.protobuf.AbstractParser() { - @java.lang.Override - public ALPMetadata parsePartialFrom( - com.google.protobuf.CodedInputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws com.google.protobuf.InvalidProtocolBufferException { - Builder builder = newBuilder(); - try { - builder.mergeFrom(input, extensionRegistry); - } catch (com.google.protobuf.InvalidProtocolBufferException e) { - throw e.setUnfinishedMessage(builder.buildPartial()); - } catch (com.google.protobuf.UninitializedMessageException e) { - throw e.asInvalidProtocolBufferException().setUnfinishedMessage(builder.buildPartial()); - } catch (java.io.IOException e) { - throw new com.google.protobuf.InvalidProtocolBufferException(e) - .setUnfinishedMessage(builder.buildPartial()); - } - return builder.buildPartial(); - } - }; - - public static com.google.protobuf.Parser parser() { - return PARSER; - } - - @java.lang.Override - public com.google.protobuf.Parser getParserForType() { - return PARSER; - } - - @java.lang.Override - public io.github.dfa1.vortex.proto.EncodingProtos.ALPMetadata getDefaultInstanceForType() { - return DEFAULT_INSTANCE; - } - - } - - public interface BitPackedMetadataOrBuilder extends - // @@protoc_insertion_point(interface_extends:vortex.encodings.BitPackedMetadata) - com.google.protobuf.MessageOrBuilder { - - /** - * uint32 bit_width = 1; - * @return The bitWidth. - */ - int getBitWidth(); - - /** - * uint32 offset = 2; - * @return The offset. - */ - int getOffset(); - - /** - * optional .vortex.encodings.PatchesMetadata patches = 3; - * @return Whether the patches field is set. - */ - boolean hasPatches(); - /** - * optional .vortex.encodings.PatchesMetadata patches = 3; - * @return The patches. - */ - io.github.dfa1.vortex.proto.EncodingProtos.PatchesMetadata getPatches(); - /** - * optional .vortex.encodings.PatchesMetadata patches = 3; - */ - io.github.dfa1.vortex.proto.EncodingProtos.PatchesMetadataOrBuilder getPatchesOrBuilder(); - } - /** - * Protobuf type {@code vortex.encodings.BitPackedMetadata} - */ - public static final class BitPackedMetadata extends - com.google.protobuf.GeneratedMessage implements - // @@protoc_insertion_point(message_implements:vortex.encodings.BitPackedMetadata) - BitPackedMetadataOrBuilder { - private static final long serialVersionUID = 0L; - static { - com.google.protobuf.RuntimeVersion.validateProtobufGencodeVersion( - com.google.protobuf.RuntimeVersion.RuntimeDomain.PUBLIC, - /* major= */ 4, - /* minor= */ 35, - /* patch= */ 0, - /* suffix= */ "", - "BitPackedMetadata"); - } - // Use BitPackedMetadata.newBuilder() to construct. - private BitPackedMetadata(com.google.protobuf.GeneratedMessage.Builder builder) { - super(builder); - } - private BitPackedMetadata() { - } - - public static final com.google.protobuf.Descriptors.Descriptor - getDescriptor() { - return io.github.dfa1.vortex.proto.EncodingProtos.internal_static_vortex_encodings_BitPackedMetadata_descriptor; - } - - @java.lang.Override - public com.google.protobuf.Descriptors.Descriptor getDescriptorForType() { - return io.github.dfa1.vortex.proto.EncodingProtos.internal_static_vortex_encodings_BitPackedMetadata_descriptor; - } - - @java.lang.Override - protected com.google.protobuf.GeneratedMessage.FieldAccessorTable - internalGetFieldAccessorTable() { - return io.github.dfa1.vortex.proto.EncodingProtos.internal_static_vortex_encodings_BitPackedMetadata_fieldAccessorTable - .ensureFieldAccessorsInitialized( - io.github.dfa1.vortex.proto.EncodingProtos.BitPackedMetadata.class, io.github.dfa1.vortex.proto.EncodingProtos.BitPackedMetadata.Builder.class); - } - - private int bitField0_; - public static final int BIT_WIDTH_FIELD_NUMBER = 1; - private int bitWidth_ = 0; - /** - * uint32 bit_width = 1; - * @return The bitWidth. - */ - @java.lang.Override - public int getBitWidth() { - return bitWidth_; - } - - public static final int OFFSET_FIELD_NUMBER = 2; - private int offset_ = 0; - /** - * uint32 offset = 2; - * @return The offset. - */ - @java.lang.Override - public int getOffset() { - return offset_; - } - - public static final int PATCHES_FIELD_NUMBER = 3; - private io.github.dfa1.vortex.proto.EncodingProtos.PatchesMetadata patches_; - /** - * optional .vortex.encodings.PatchesMetadata patches = 3; - * @return Whether the patches field is set. - */ - @java.lang.Override - public boolean hasPatches() { - return ((bitField0_ & 0x00000001) != 0); - } - /** - * optional .vortex.encodings.PatchesMetadata patches = 3; - * @return The patches. - */ - @java.lang.Override - public io.github.dfa1.vortex.proto.EncodingProtos.PatchesMetadata getPatches() { - return patches_ == null ? io.github.dfa1.vortex.proto.EncodingProtos.PatchesMetadata.getDefaultInstance() : patches_; - } - /** - * optional .vortex.encodings.PatchesMetadata patches = 3; - */ - @java.lang.Override - public io.github.dfa1.vortex.proto.EncodingProtos.PatchesMetadataOrBuilder getPatchesOrBuilder() { - return patches_ == null ? io.github.dfa1.vortex.proto.EncodingProtos.PatchesMetadata.getDefaultInstance() : patches_; - } - - private byte memoizedIsInitialized = -1; - @java.lang.Override - public final boolean isInitialized() { - byte isInitialized = memoizedIsInitialized; - if (isInitialized == 1) return true; - if (isInitialized == 0) return false; - - memoizedIsInitialized = 1; - return true; - } - - @java.lang.Override - public void writeTo(com.google.protobuf.CodedOutputStream output) - throws java.io.IOException { - if (bitWidth_ != 0) { - output.writeUInt32(1, bitWidth_); - } - if (offset_ != 0) { - output.writeUInt32(2, offset_); - } - if (((bitField0_ & 0x00000001) != 0)) { - output.writeMessage(3, getPatches()); - } - getUnknownFields().writeTo(output); - } - private int computeSerializedSize_0() { - int size = 0; - if (bitWidth_ != 0) { - size += com.google.protobuf.CodedOutputStream - .computeUInt32Size(1, bitWidth_); - } - if (offset_ != 0) { - size += com.google.protobuf.CodedOutputStream - .computeUInt32Size(2, offset_); - } - if (((bitField0_ & 0x00000001) != 0)) { - size += com.google.protobuf.CodedOutputStream - .computeMessageSize(3, getPatches()); - } - return size; - } - @java.lang.Override - public int getSerializedSize() { - int size = memoizedSize; - if (size != -1) return size; - - size = 0; - size += computeSerializedSize_0(); - size += getUnknownFields().getSerializedSize(); - memoizedSize = size; - return size; - } - - @java.lang.Override - public boolean equals(final java.lang.Object obj) { - if (obj == this) { - return true; - } - if (!(obj instanceof io.github.dfa1.vortex.proto.EncodingProtos.BitPackedMetadata)) { - return super.equals(obj); - } - io.github.dfa1.vortex.proto.EncodingProtos.BitPackedMetadata other = (io.github.dfa1.vortex.proto.EncodingProtos.BitPackedMetadata) obj; - - if (getBitWidth() - != other.getBitWidth()) return false; - if (getOffset() - != other.getOffset()) return false; - if (hasPatches() != other.hasPatches()) return false; - if (hasPatches()) { - if (!getPatches() - .equals(other.getPatches())) return false; - } - if (!getUnknownFields().equals(other.getUnknownFields())) return false; - return true; - } - - @java.lang.Override - public int hashCode() { - if (memoizedHashCode != 0) { - return memoizedHashCode; - } - int hash = 41; - hash = (19 * hash) + getDescriptor().hashCode(); - hash = (37 * hash) + BIT_WIDTH_FIELD_NUMBER; - hash = (53 * hash) + getBitWidth(); - hash = (37 * hash) + OFFSET_FIELD_NUMBER; - hash = (53 * hash) + getOffset(); - if (hasPatches()) { - hash = (37 * hash) + PATCHES_FIELD_NUMBER; - hash = (53 * hash) + getPatches().hashCode(); - } - hash = (29 * hash) + getUnknownFields().hashCode(); - memoizedHashCode = hash; - return hash; - } - - public static io.github.dfa1.vortex.proto.EncodingProtos.BitPackedMetadata parseFrom( - java.nio.ByteBuffer data) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.BitPackedMetadata parseFrom( - java.nio.ByteBuffer data, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data, extensionRegistry); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.BitPackedMetadata parseFrom( - com.google.protobuf.ByteString data) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.BitPackedMetadata parseFrom( - com.google.protobuf.ByteString data, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data, extensionRegistry); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.BitPackedMetadata parseFrom(byte[] data) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.BitPackedMetadata parseFrom( - byte[] data, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data, extensionRegistry); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.BitPackedMetadata parseFrom(java.io.InputStream input) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseWithIOException(PARSER, input); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.BitPackedMetadata parseFrom( - java.io.InputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseWithIOException(PARSER, input, extensionRegistry); - } - - public static io.github.dfa1.vortex.proto.EncodingProtos.BitPackedMetadata parseDelimitedFrom(java.io.InputStream input) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseDelimitedWithIOException(PARSER, input); - } - - public static io.github.dfa1.vortex.proto.EncodingProtos.BitPackedMetadata parseDelimitedFrom( - java.io.InputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseDelimitedWithIOException(PARSER, input, extensionRegistry); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.BitPackedMetadata parseFrom( - com.google.protobuf.CodedInputStream input) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseWithIOException(PARSER, input); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.BitPackedMetadata parseFrom( - com.google.protobuf.CodedInputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseWithIOException(PARSER, input, extensionRegistry); - } - - @java.lang.Override - public Builder newBuilderForType() { return newBuilder(); } - public static Builder newBuilder() { - return DEFAULT_INSTANCE.toBuilder(); - } - public static Builder newBuilder(io.github.dfa1.vortex.proto.EncodingProtos.BitPackedMetadata prototype) { - return DEFAULT_INSTANCE.toBuilder().mergeFrom(prototype); - } - @java.lang.Override - public Builder toBuilder() { - return this == DEFAULT_INSTANCE - ? new Builder() : new Builder().mergeFrom(this); - } - - @java.lang.Override - protected Builder newBuilderForType( - com.google.protobuf.GeneratedMessage.BuilderParent parent) { - Builder builder = new Builder(parent); - return builder; - } - /** - * Protobuf type {@code vortex.encodings.BitPackedMetadata} - */ - public static final class Builder extends - com.google.protobuf.GeneratedMessage.Builder implements - // @@protoc_insertion_point(builder_implements:vortex.encodings.BitPackedMetadata) - io.github.dfa1.vortex.proto.EncodingProtos.BitPackedMetadataOrBuilder { - public static final com.google.protobuf.Descriptors.Descriptor - getDescriptor() { - return io.github.dfa1.vortex.proto.EncodingProtos.internal_static_vortex_encodings_BitPackedMetadata_descriptor; - } - - @java.lang.Override - protected com.google.protobuf.GeneratedMessage.FieldAccessorTable - internalGetFieldAccessorTable() { - return io.github.dfa1.vortex.proto.EncodingProtos.internal_static_vortex_encodings_BitPackedMetadata_fieldAccessorTable - .ensureFieldAccessorsInitialized( - io.github.dfa1.vortex.proto.EncodingProtos.BitPackedMetadata.class, io.github.dfa1.vortex.proto.EncodingProtos.BitPackedMetadata.Builder.class); - } - - // Construct using io.github.dfa1.vortex.proto.EncodingProtos.BitPackedMetadata.newBuilder() - private Builder() { - maybeForceBuilderInitialization(); - } - - private Builder( - com.google.protobuf.GeneratedMessage.BuilderParent parent) { - super(parent); - maybeForceBuilderInitialization(); - } - private void maybeForceBuilderInitialization() { - if (com.google.protobuf.GeneratedMessage - .alwaysUseFieldBuilders) { - internalGetPatchesFieldBuilder(); - } - } - @java.lang.Override - public Builder clear() { - super.clear(); - bitField0_ = 0; - bitWidth_ = 0; - offset_ = 0; - patches_ = null; - if (patchesBuilder_ != null) { - patchesBuilder_.dispose(); - patchesBuilder_ = null; - } - return this; - } - - @java.lang.Override - public com.google.protobuf.Descriptors.Descriptor - getDescriptorForType() { - return io.github.dfa1.vortex.proto.EncodingProtos.internal_static_vortex_encodings_BitPackedMetadata_descriptor; - } - - @java.lang.Override - public io.github.dfa1.vortex.proto.EncodingProtos.BitPackedMetadata getDefaultInstanceForType() { - return io.github.dfa1.vortex.proto.EncodingProtos.BitPackedMetadata.getDefaultInstance(); - } - - @java.lang.Override - public io.github.dfa1.vortex.proto.EncodingProtos.BitPackedMetadata build() { - io.github.dfa1.vortex.proto.EncodingProtos.BitPackedMetadata result = buildPartial(); - if (!result.isInitialized()) { - throw newUninitializedMessageException(result); - } - return result; - } - - @java.lang.Override - public io.github.dfa1.vortex.proto.EncodingProtos.BitPackedMetadata buildPartial() { - io.github.dfa1.vortex.proto.EncodingProtos.BitPackedMetadata result = new io.github.dfa1.vortex.proto.EncodingProtos.BitPackedMetadata(this); - if (bitField0_ != 0) { buildPartial0(result); } - onBuilt(); - return result; - } - - private void buildPartial0(io.github.dfa1.vortex.proto.EncodingProtos.BitPackedMetadata result) { - int from_bitField0_ = bitField0_; - if (((from_bitField0_ & 0x00000001) != 0)) { - result.bitWidth_ = bitWidth_; - } - if (((from_bitField0_ & 0x00000002) != 0)) { - result.offset_ = offset_; - } - int to_bitField0_ = 0; - if (((from_bitField0_ & 0x00000004) != 0)) { - result.patches_ = patchesBuilder_ == null - ? patches_ - : patchesBuilder_.build(); - to_bitField0_ |= 0x00000001; - } - result.bitField0_ |= to_bitField0_; - } - - @java.lang.Override - public Builder mergeFrom(com.google.protobuf.Message other) { - if (other instanceof io.github.dfa1.vortex.proto.EncodingProtos.BitPackedMetadata) { - return mergeFrom((io.github.dfa1.vortex.proto.EncodingProtos.BitPackedMetadata)other); - } else { - super.mergeFrom(other); - return this; - } - } - - public Builder mergeFrom(io.github.dfa1.vortex.proto.EncodingProtos.BitPackedMetadata other) { - if (other == io.github.dfa1.vortex.proto.EncodingProtos.BitPackedMetadata.getDefaultInstance()) return this; - if (other.getBitWidth() != 0) { - setBitWidth(other.getBitWidth()); - } - if (other.getOffset() != 0) { - setOffset(other.getOffset()); - } - if (other.hasPatches()) { - mergePatches(other.getPatches()); - } - this.mergeUnknownFields(other.getUnknownFields()); - onChanged(); - return this; - } - - @java.lang.Override - public final boolean isInitialized() { - return true; - } - - @java.lang.Override - public Builder mergeFrom( - com.google.protobuf.CodedInputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws java.io.IOException { - if (extensionRegistry == null) { - throw new java.lang.NullPointerException(); - } - try { - boolean done = false; - while (!done) { - int tag = input.readTag(); - switch (tag) { - case 0: - done = true; - break; - case 8: { - bitWidth_ = input.readUInt32(); - bitField0_ |= 0x00000001; - break; - } // case 8 - case 16: { - offset_ = input.readUInt32(); - bitField0_ |= 0x00000002; - break; - } // case 16 - case 26: { - input.readMessage( - internalGetPatchesFieldBuilder().getBuilder(), - extensionRegistry); - bitField0_ |= 0x00000004; - break; - } // case 26 - default: { - if (!super.parseUnknownField(input, extensionRegistry, tag)) { - done = true; // was an endgroup tag - } - break; - } // default: - } // switch (tag) - } // while (!done) - } catch (com.google.protobuf.InvalidProtocolBufferException e) { - throw e.unwrapIOException(); - } finally { - onChanged(); - } // finally - return this; - } - private int bitField0_; - - private int bitWidth_ ; - /** - * uint32 bit_width = 1; - * @return The bitWidth. - */ - @java.lang.Override - public int getBitWidth() { - return bitWidth_; - } - /** - * uint32 bit_width = 1; - * @param value The bitWidth to set. - * @return This builder for chaining. - */ - public Builder setBitWidth(int value) { - - bitWidth_ = value; - bitField0_ |= 0x00000001; - onChanged(); - return this; - } - /** - * uint32 bit_width = 1; - * @return This builder for chaining. - */ - public Builder clearBitWidth() { - bitField0_ = (bitField0_ & ~0x00000001); - bitWidth_ = 0; - onChanged(); - return this; - } - - private int offset_ ; - /** - * uint32 offset = 2; - * @return The offset. - */ - @java.lang.Override - public int getOffset() { - return offset_; - } - /** - * uint32 offset = 2; - * @param value The offset to set. - * @return This builder for chaining. - */ - public Builder setOffset(int value) { - - offset_ = value; - bitField0_ |= 0x00000002; - onChanged(); - return this; - } - /** - * uint32 offset = 2; - * @return This builder for chaining. - */ - public Builder clearOffset() { - bitField0_ = (bitField0_ & ~0x00000002); - offset_ = 0; - onChanged(); - return this; - } - - private io.github.dfa1.vortex.proto.EncodingProtos.PatchesMetadata patches_; - private com.google.protobuf.SingleFieldBuilder< - io.github.dfa1.vortex.proto.EncodingProtos.PatchesMetadata, io.github.dfa1.vortex.proto.EncodingProtos.PatchesMetadata.Builder, io.github.dfa1.vortex.proto.EncodingProtos.PatchesMetadataOrBuilder> patchesBuilder_; - /** - * optional .vortex.encodings.PatchesMetadata patches = 3; - * @return Whether the patches field is set. - */ - public boolean hasPatches() { - return ((bitField0_ & 0x00000004) != 0); - } - /** - * optional .vortex.encodings.PatchesMetadata patches = 3; - * @return The patches. - */ - public io.github.dfa1.vortex.proto.EncodingProtos.PatchesMetadata getPatches() { - if (patchesBuilder_ == null) { - return patches_ == null ? io.github.dfa1.vortex.proto.EncodingProtos.PatchesMetadata.getDefaultInstance() : patches_; - } else { - return patchesBuilder_.getMessage(); - } - } - /** - * optional .vortex.encodings.PatchesMetadata patches = 3; - */ - public Builder setPatches(io.github.dfa1.vortex.proto.EncodingProtos.PatchesMetadata value) { - if (patchesBuilder_ == null) { - if (value == null) { - throw new NullPointerException(); - } - patches_ = value; - } else { - patchesBuilder_.setMessage(value); - } - bitField0_ |= 0x00000004; - onChanged(); - return this; - } - /** - * optional .vortex.encodings.PatchesMetadata patches = 3; - */ - public Builder setPatches( - io.github.dfa1.vortex.proto.EncodingProtos.PatchesMetadata.Builder builderForValue) { - if (patchesBuilder_ == null) { - patches_ = builderForValue.build(); - } else { - patchesBuilder_.setMessage(builderForValue.build()); - } - bitField0_ |= 0x00000004; - onChanged(); - return this; - } - /** - * optional .vortex.encodings.PatchesMetadata patches = 3; - */ - public Builder mergePatches(io.github.dfa1.vortex.proto.EncodingProtos.PatchesMetadata value) { - if (patchesBuilder_ == null) { - if (((bitField0_ & 0x00000004) != 0) && - patches_ != null && - patches_ != io.github.dfa1.vortex.proto.EncodingProtos.PatchesMetadata.getDefaultInstance()) { - getPatchesBuilder().mergeFrom(value); - } else { - patches_ = value; - } - } else { - patchesBuilder_.mergeFrom(value); - } - if (patches_ != null) { - bitField0_ |= 0x00000004; - onChanged(); - } - return this; - } - /** - * optional .vortex.encodings.PatchesMetadata patches = 3; - */ - public Builder clearPatches() { - bitField0_ = (bitField0_ & ~0x00000004); - patches_ = null; - if (patchesBuilder_ != null) { - patchesBuilder_.dispose(); - patchesBuilder_ = null; - } - onChanged(); - return this; - } - /** - * optional .vortex.encodings.PatchesMetadata patches = 3; - */ - public io.github.dfa1.vortex.proto.EncodingProtos.PatchesMetadata.Builder getPatchesBuilder() { - bitField0_ |= 0x00000004; - onChanged(); - return internalGetPatchesFieldBuilder().getBuilder(); - } - /** - * optional .vortex.encodings.PatchesMetadata patches = 3; - */ - public io.github.dfa1.vortex.proto.EncodingProtos.PatchesMetadataOrBuilder getPatchesOrBuilder() { - if (patchesBuilder_ != null) { - return patchesBuilder_.getMessageOrBuilder(); - } else { - return patches_ == null ? - io.github.dfa1.vortex.proto.EncodingProtos.PatchesMetadata.getDefaultInstance() : patches_; - } - } - /** - * optional .vortex.encodings.PatchesMetadata patches = 3; - */ - private com.google.protobuf.SingleFieldBuilder< - io.github.dfa1.vortex.proto.EncodingProtos.PatchesMetadata, io.github.dfa1.vortex.proto.EncodingProtos.PatchesMetadata.Builder, io.github.dfa1.vortex.proto.EncodingProtos.PatchesMetadataOrBuilder> - internalGetPatchesFieldBuilder() { - if (patchesBuilder_ == null) { - patchesBuilder_ = new com.google.protobuf.SingleFieldBuilder< - io.github.dfa1.vortex.proto.EncodingProtos.PatchesMetadata, io.github.dfa1.vortex.proto.EncodingProtos.PatchesMetadata.Builder, io.github.dfa1.vortex.proto.EncodingProtos.PatchesMetadataOrBuilder>( - getPatches(), - getParentForChildren(), - isClean()); - patches_ = null; - } - return patchesBuilder_; - } - - // @@protoc_insertion_point(builder_scope:vortex.encodings.BitPackedMetadata) - } - - // @@protoc_insertion_point(class_scope:vortex.encodings.BitPackedMetadata) - private static final io.github.dfa1.vortex.proto.EncodingProtos.BitPackedMetadata DEFAULT_INSTANCE; - static { - DEFAULT_INSTANCE = new io.github.dfa1.vortex.proto.EncodingProtos.BitPackedMetadata(); - } - - public static io.github.dfa1.vortex.proto.EncodingProtos.BitPackedMetadata getDefaultInstance() { - return DEFAULT_INSTANCE; - } - - private static final com.google.protobuf.Parser - PARSER = new com.google.protobuf.AbstractParser() { - @java.lang.Override - public BitPackedMetadata parsePartialFrom( - com.google.protobuf.CodedInputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws com.google.protobuf.InvalidProtocolBufferException { - Builder builder = newBuilder(); - try { - builder.mergeFrom(input, extensionRegistry); - } catch (com.google.protobuf.InvalidProtocolBufferException e) { - throw e.setUnfinishedMessage(builder.buildPartial()); - } catch (com.google.protobuf.UninitializedMessageException e) { - throw e.asInvalidProtocolBufferException().setUnfinishedMessage(builder.buildPartial()); - } catch (java.io.IOException e) { - throw new com.google.protobuf.InvalidProtocolBufferException(e) - .setUnfinishedMessage(builder.buildPartial()); - } - return builder.buildPartial(); - } - }; - - public static com.google.protobuf.Parser parser() { - return PARSER; - } - - @java.lang.Override - public com.google.protobuf.Parser getParserForType() { - return PARSER; - } - - @java.lang.Override - public io.github.dfa1.vortex.proto.EncodingProtos.BitPackedMetadata getDefaultInstanceForType() { - return DEFAULT_INSTANCE; - } - - } - - public interface SequenceMetadataOrBuilder extends - // @@protoc_insertion_point(interface_extends:vortex.encodings.SequenceMetadata) - com.google.protobuf.MessageOrBuilder { - - /** - * .vortex.scalar.ScalarValue base = 1; - * @return Whether the base field is set. - */ - boolean hasBase(); - /** - * .vortex.scalar.ScalarValue base = 1; - * @return The base. - */ - io.github.dfa1.vortex.proto.ScalarProtos.ScalarValue getBase(); - /** - * .vortex.scalar.ScalarValue base = 1; - */ - io.github.dfa1.vortex.proto.ScalarProtos.ScalarValueOrBuilder getBaseOrBuilder(); - - /** - * .vortex.scalar.ScalarValue multiplier = 2; - * @return Whether the multiplier field is set. - */ - boolean hasMultiplier(); - /** - * .vortex.scalar.ScalarValue multiplier = 2; - * @return The multiplier. - */ - io.github.dfa1.vortex.proto.ScalarProtos.ScalarValue getMultiplier(); - /** - * .vortex.scalar.ScalarValue multiplier = 2; - */ - io.github.dfa1.vortex.proto.ScalarProtos.ScalarValueOrBuilder getMultiplierOrBuilder(); - } - /** - * Protobuf type {@code vortex.encodings.SequenceMetadata} - */ - public static final class SequenceMetadata extends - com.google.protobuf.GeneratedMessage implements - // @@protoc_insertion_point(message_implements:vortex.encodings.SequenceMetadata) - SequenceMetadataOrBuilder { - private static final long serialVersionUID = 0L; - static { - com.google.protobuf.RuntimeVersion.validateProtobufGencodeVersion( - com.google.protobuf.RuntimeVersion.RuntimeDomain.PUBLIC, - /* major= */ 4, - /* minor= */ 35, - /* patch= */ 0, - /* suffix= */ "", - "SequenceMetadata"); - } - // Use SequenceMetadata.newBuilder() to construct. - private SequenceMetadata(com.google.protobuf.GeneratedMessage.Builder builder) { - super(builder); - } - private SequenceMetadata() { - } - - public static final com.google.protobuf.Descriptors.Descriptor - getDescriptor() { - return io.github.dfa1.vortex.proto.EncodingProtos.internal_static_vortex_encodings_SequenceMetadata_descriptor; - } - - @java.lang.Override - public com.google.protobuf.Descriptors.Descriptor getDescriptorForType() { - return io.github.dfa1.vortex.proto.EncodingProtos.internal_static_vortex_encodings_SequenceMetadata_descriptor; - } - - @java.lang.Override - protected com.google.protobuf.GeneratedMessage.FieldAccessorTable - internalGetFieldAccessorTable() { - return io.github.dfa1.vortex.proto.EncodingProtos.internal_static_vortex_encodings_SequenceMetadata_fieldAccessorTable - .ensureFieldAccessorsInitialized( - io.github.dfa1.vortex.proto.EncodingProtos.SequenceMetadata.class, io.github.dfa1.vortex.proto.EncodingProtos.SequenceMetadata.Builder.class); - } - - private int bitField0_; - public static final int BASE_FIELD_NUMBER = 1; - private io.github.dfa1.vortex.proto.ScalarProtos.ScalarValue base_; - /** - * .vortex.scalar.ScalarValue base = 1; - * @return Whether the base field is set. - */ - @java.lang.Override - public boolean hasBase() { - return ((bitField0_ & 0x00000001) != 0); - } - /** - * .vortex.scalar.ScalarValue base = 1; - * @return The base. - */ - @java.lang.Override - public io.github.dfa1.vortex.proto.ScalarProtos.ScalarValue getBase() { - return base_ == null ? io.github.dfa1.vortex.proto.ScalarProtos.ScalarValue.getDefaultInstance() : base_; - } - /** - * .vortex.scalar.ScalarValue base = 1; - */ - @java.lang.Override - public io.github.dfa1.vortex.proto.ScalarProtos.ScalarValueOrBuilder getBaseOrBuilder() { - return base_ == null ? io.github.dfa1.vortex.proto.ScalarProtos.ScalarValue.getDefaultInstance() : base_; - } - - public static final int MULTIPLIER_FIELD_NUMBER = 2; - private io.github.dfa1.vortex.proto.ScalarProtos.ScalarValue multiplier_; - /** - * .vortex.scalar.ScalarValue multiplier = 2; - * @return Whether the multiplier field is set. - */ - @java.lang.Override - public boolean hasMultiplier() { - return ((bitField0_ & 0x00000002) != 0); - } - /** - * .vortex.scalar.ScalarValue multiplier = 2; - * @return The multiplier. - */ - @java.lang.Override - public io.github.dfa1.vortex.proto.ScalarProtos.ScalarValue getMultiplier() { - return multiplier_ == null ? io.github.dfa1.vortex.proto.ScalarProtos.ScalarValue.getDefaultInstance() : multiplier_; - } - /** - * .vortex.scalar.ScalarValue multiplier = 2; - */ - @java.lang.Override - public io.github.dfa1.vortex.proto.ScalarProtos.ScalarValueOrBuilder getMultiplierOrBuilder() { - return multiplier_ == null ? io.github.dfa1.vortex.proto.ScalarProtos.ScalarValue.getDefaultInstance() : multiplier_; - } - - private byte memoizedIsInitialized = -1; - @java.lang.Override - public final boolean isInitialized() { - byte isInitialized = memoizedIsInitialized; - if (isInitialized == 1) return true; - if (isInitialized == 0) return false; - - memoizedIsInitialized = 1; - return true; - } - - @java.lang.Override - public void writeTo(com.google.protobuf.CodedOutputStream output) - throws java.io.IOException { - if (((bitField0_ & 0x00000001) != 0)) { - output.writeMessage(1, getBase()); - } - if (((bitField0_ & 0x00000002) != 0)) { - output.writeMessage(2, getMultiplier()); - } - getUnknownFields().writeTo(output); - } - private int computeSerializedSize_0() { - int size = 0; - if (((bitField0_ & 0x00000001) != 0)) { - size += com.google.protobuf.CodedOutputStream - .computeMessageSize(1, getBase()); - } - if (((bitField0_ & 0x00000002) != 0)) { - size += com.google.protobuf.CodedOutputStream - .computeMessageSize(2, getMultiplier()); - } - return size; - } - @java.lang.Override - public int getSerializedSize() { - int size = memoizedSize; - if (size != -1) return size; - - size = 0; - size += computeSerializedSize_0(); - size += getUnknownFields().getSerializedSize(); - memoizedSize = size; - return size; - } - - @java.lang.Override - public boolean equals(final java.lang.Object obj) { - if (obj == this) { - return true; - } - if (!(obj instanceof io.github.dfa1.vortex.proto.EncodingProtos.SequenceMetadata)) { - return super.equals(obj); - } - io.github.dfa1.vortex.proto.EncodingProtos.SequenceMetadata other = (io.github.dfa1.vortex.proto.EncodingProtos.SequenceMetadata) obj; - - if (hasBase() != other.hasBase()) return false; - if (hasBase()) { - if (!getBase() - .equals(other.getBase())) return false; - } - if (hasMultiplier() != other.hasMultiplier()) return false; - if (hasMultiplier()) { - if (!getMultiplier() - .equals(other.getMultiplier())) return false; - } - if (!getUnknownFields().equals(other.getUnknownFields())) return false; - return true; - } - - @java.lang.Override - public int hashCode() { - if (memoizedHashCode != 0) { - return memoizedHashCode; - } - int hash = 41; - hash = (19 * hash) + getDescriptor().hashCode(); - if (hasBase()) { - hash = (37 * hash) + BASE_FIELD_NUMBER; - hash = (53 * hash) + getBase().hashCode(); - } - if (hasMultiplier()) { - hash = (37 * hash) + MULTIPLIER_FIELD_NUMBER; - hash = (53 * hash) + getMultiplier().hashCode(); - } - hash = (29 * hash) + getUnknownFields().hashCode(); - memoizedHashCode = hash; - return hash; - } - - public static io.github.dfa1.vortex.proto.EncodingProtos.SequenceMetadata parseFrom( - java.nio.ByteBuffer data) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.SequenceMetadata parseFrom( - java.nio.ByteBuffer data, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data, extensionRegistry); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.SequenceMetadata parseFrom( - com.google.protobuf.ByteString data) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.SequenceMetadata parseFrom( - com.google.protobuf.ByteString data, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data, extensionRegistry); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.SequenceMetadata parseFrom(byte[] data) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.SequenceMetadata parseFrom( - byte[] data, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data, extensionRegistry); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.SequenceMetadata parseFrom(java.io.InputStream input) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseWithIOException(PARSER, input); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.SequenceMetadata parseFrom( - java.io.InputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseWithIOException(PARSER, input, extensionRegistry); - } - - public static io.github.dfa1.vortex.proto.EncodingProtos.SequenceMetadata parseDelimitedFrom(java.io.InputStream input) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseDelimitedWithIOException(PARSER, input); - } - - public static io.github.dfa1.vortex.proto.EncodingProtos.SequenceMetadata parseDelimitedFrom( - java.io.InputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseDelimitedWithIOException(PARSER, input, extensionRegistry); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.SequenceMetadata parseFrom( - com.google.protobuf.CodedInputStream input) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseWithIOException(PARSER, input); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.SequenceMetadata parseFrom( - com.google.protobuf.CodedInputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseWithIOException(PARSER, input, extensionRegistry); - } - - @java.lang.Override - public Builder newBuilderForType() { return newBuilder(); } - public static Builder newBuilder() { - return DEFAULT_INSTANCE.toBuilder(); - } - public static Builder newBuilder(io.github.dfa1.vortex.proto.EncodingProtos.SequenceMetadata prototype) { - return DEFAULT_INSTANCE.toBuilder().mergeFrom(prototype); - } - @java.lang.Override - public Builder toBuilder() { - return this == DEFAULT_INSTANCE - ? new Builder() : new Builder().mergeFrom(this); - } - - @java.lang.Override - protected Builder newBuilderForType( - com.google.protobuf.GeneratedMessage.BuilderParent parent) { - Builder builder = new Builder(parent); - return builder; - } - /** - * Protobuf type {@code vortex.encodings.SequenceMetadata} - */ - public static final class Builder extends - com.google.protobuf.GeneratedMessage.Builder implements - // @@protoc_insertion_point(builder_implements:vortex.encodings.SequenceMetadata) - io.github.dfa1.vortex.proto.EncodingProtos.SequenceMetadataOrBuilder { - public static final com.google.protobuf.Descriptors.Descriptor - getDescriptor() { - return io.github.dfa1.vortex.proto.EncodingProtos.internal_static_vortex_encodings_SequenceMetadata_descriptor; - } - - @java.lang.Override - protected com.google.protobuf.GeneratedMessage.FieldAccessorTable - internalGetFieldAccessorTable() { - return io.github.dfa1.vortex.proto.EncodingProtos.internal_static_vortex_encodings_SequenceMetadata_fieldAccessorTable - .ensureFieldAccessorsInitialized( - io.github.dfa1.vortex.proto.EncodingProtos.SequenceMetadata.class, io.github.dfa1.vortex.proto.EncodingProtos.SequenceMetadata.Builder.class); - } - - // Construct using io.github.dfa1.vortex.proto.EncodingProtos.SequenceMetadata.newBuilder() - private Builder() { - maybeForceBuilderInitialization(); - } - - private Builder( - com.google.protobuf.GeneratedMessage.BuilderParent parent) { - super(parent); - maybeForceBuilderInitialization(); - } - private void maybeForceBuilderInitialization() { - if (com.google.protobuf.GeneratedMessage - .alwaysUseFieldBuilders) { - internalGetBaseFieldBuilder(); - internalGetMultiplierFieldBuilder(); - } - } - @java.lang.Override - public Builder clear() { - super.clear(); - bitField0_ = 0; - base_ = null; - if (baseBuilder_ != null) { - baseBuilder_.dispose(); - baseBuilder_ = null; - } - multiplier_ = null; - if (multiplierBuilder_ != null) { - multiplierBuilder_.dispose(); - multiplierBuilder_ = null; - } - return this; - } - - @java.lang.Override - public com.google.protobuf.Descriptors.Descriptor - getDescriptorForType() { - return io.github.dfa1.vortex.proto.EncodingProtos.internal_static_vortex_encodings_SequenceMetadata_descriptor; - } - - @java.lang.Override - public io.github.dfa1.vortex.proto.EncodingProtos.SequenceMetadata getDefaultInstanceForType() { - return io.github.dfa1.vortex.proto.EncodingProtos.SequenceMetadata.getDefaultInstance(); - } - - @java.lang.Override - public io.github.dfa1.vortex.proto.EncodingProtos.SequenceMetadata build() { - io.github.dfa1.vortex.proto.EncodingProtos.SequenceMetadata result = buildPartial(); - if (!result.isInitialized()) { - throw newUninitializedMessageException(result); - } - return result; - } - - @java.lang.Override - public io.github.dfa1.vortex.proto.EncodingProtos.SequenceMetadata buildPartial() { - io.github.dfa1.vortex.proto.EncodingProtos.SequenceMetadata result = new io.github.dfa1.vortex.proto.EncodingProtos.SequenceMetadata(this); - if (bitField0_ != 0) { buildPartial0(result); } - onBuilt(); - return result; - } - - private void buildPartial0(io.github.dfa1.vortex.proto.EncodingProtos.SequenceMetadata result) { - int from_bitField0_ = bitField0_; - int to_bitField0_ = 0; - if (((from_bitField0_ & 0x00000001) != 0)) { - result.base_ = baseBuilder_ == null - ? base_ - : baseBuilder_.build(); - to_bitField0_ |= 0x00000001; - } - if (((from_bitField0_ & 0x00000002) != 0)) { - result.multiplier_ = multiplierBuilder_ == null - ? multiplier_ - : multiplierBuilder_.build(); - to_bitField0_ |= 0x00000002; - } - result.bitField0_ |= to_bitField0_; - } - - @java.lang.Override - public Builder mergeFrom(com.google.protobuf.Message other) { - if (other instanceof io.github.dfa1.vortex.proto.EncodingProtos.SequenceMetadata) { - return mergeFrom((io.github.dfa1.vortex.proto.EncodingProtos.SequenceMetadata)other); - } else { - super.mergeFrom(other); - return this; - } - } - - public Builder mergeFrom(io.github.dfa1.vortex.proto.EncodingProtos.SequenceMetadata other) { - if (other == io.github.dfa1.vortex.proto.EncodingProtos.SequenceMetadata.getDefaultInstance()) return this; - if (other.hasBase()) { - mergeBase(other.getBase()); - } - if (other.hasMultiplier()) { - mergeMultiplier(other.getMultiplier()); - } - this.mergeUnknownFields(other.getUnknownFields()); - onChanged(); - return this; - } - - @java.lang.Override - public final boolean isInitialized() { - return true; - } - - @java.lang.Override - public Builder mergeFrom( - com.google.protobuf.CodedInputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws java.io.IOException { - if (extensionRegistry == null) { - throw new java.lang.NullPointerException(); - } - try { - boolean done = false; - while (!done) { - int tag = input.readTag(); - switch (tag) { - case 0: - done = true; - break; - case 10: { - input.readMessage( - internalGetBaseFieldBuilder().getBuilder(), - extensionRegistry); - bitField0_ |= 0x00000001; - break; - } // case 10 - case 18: { - input.readMessage( - internalGetMultiplierFieldBuilder().getBuilder(), - extensionRegistry); - bitField0_ |= 0x00000002; - break; - } // case 18 - default: { - if (!super.parseUnknownField(input, extensionRegistry, tag)) { - done = true; // was an endgroup tag - } - break; - } // default: - } // switch (tag) - } // while (!done) - } catch (com.google.protobuf.InvalidProtocolBufferException e) { - throw e.unwrapIOException(); - } finally { - onChanged(); - } // finally - return this; - } - private int bitField0_; - - private io.github.dfa1.vortex.proto.ScalarProtos.ScalarValue base_; - private com.google.protobuf.SingleFieldBuilder< - io.github.dfa1.vortex.proto.ScalarProtos.ScalarValue, io.github.dfa1.vortex.proto.ScalarProtos.ScalarValue.Builder, io.github.dfa1.vortex.proto.ScalarProtos.ScalarValueOrBuilder> baseBuilder_; - /** - * .vortex.scalar.ScalarValue base = 1; - * @return Whether the base field is set. - */ - public boolean hasBase() { - return ((bitField0_ & 0x00000001) != 0); - } - /** - * .vortex.scalar.ScalarValue base = 1; - * @return The base. - */ - public io.github.dfa1.vortex.proto.ScalarProtos.ScalarValue getBase() { - if (baseBuilder_ == null) { - return base_ == null ? io.github.dfa1.vortex.proto.ScalarProtos.ScalarValue.getDefaultInstance() : base_; - } else { - return baseBuilder_.getMessage(); - } - } - /** - * .vortex.scalar.ScalarValue base = 1; - */ - public Builder setBase(io.github.dfa1.vortex.proto.ScalarProtos.ScalarValue value) { - if (baseBuilder_ == null) { - if (value == null) { - throw new NullPointerException(); - } - base_ = value; - } else { - baseBuilder_.setMessage(value); - } - bitField0_ |= 0x00000001; - onChanged(); - return this; - } - /** - * .vortex.scalar.ScalarValue base = 1; - */ - public Builder setBase( - io.github.dfa1.vortex.proto.ScalarProtos.ScalarValue.Builder builderForValue) { - if (baseBuilder_ == null) { - base_ = builderForValue.build(); - } else { - baseBuilder_.setMessage(builderForValue.build()); - } - bitField0_ |= 0x00000001; - onChanged(); - return this; - } - /** - * .vortex.scalar.ScalarValue base = 1; - */ - public Builder mergeBase(io.github.dfa1.vortex.proto.ScalarProtos.ScalarValue value) { - if (baseBuilder_ == null) { - if (((bitField0_ & 0x00000001) != 0) && - base_ != null && - base_ != io.github.dfa1.vortex.proto.ScalarProtos.ScalarValue.getDefaultInstance()) { - getBaseBuilder().mergeFrom(value); - } else { - base_ = value; - } - } else { - baseBuilder_.mergeFrom(value); - } - if (base_ != null) { - bitField0_ |= 0x00000001; - onChanged(); - } - return this; - } - /** - * .vortex.scalar.ScalarValue base = 1; - */ - public Builder clearBase() { - bitField0_ = (bitField0_ & ~0x00000001); - base_ = null; - if (baseBuilder_ != null) { - baseBuilder_.dispose(); - baseBuilder_ = null; - } - onChanged(); - return this; - } - /** - * .vortex.scalar.ScalarValue base = 1; - */ - public io.github.dfa1.vortex.proto.ScalarProtos.ScalarValue.Builder getBaseBuilder() { - bitField0_ |= 0x00000001; - onChanged(); - return internalGetBaseFieldBuilder().getBuilder(); - } - /** - * .vortex.scalar.ScalarValue base = 1; - */ - public io.github.dfa1.vortex.proto.ScalarProtos.ScalarValueOrBuilder getBaseOrBuilder() { - if (baseBuilder_ != null) { - return baseBuilder_.getMessageOrBuilder(); - } else { - return base_ == null ? - io.github.dfa1.vortex.proto.ScalarProtos.ScalarValue.getDefaultInstance() : base_; - } - } - /** - * .vortex.scalar.ScalarValue base = 1; - */ - private com.google.protobuf.SingleFieldBuilder< - io.github.dfa1.vortex.proto.ScalarProtos.ScalarValue, io.github.dfa1.vortex.proto.ScalarProtos.ScalarValue.Builder, io.github.dfa1.vortex.proto.ScalarProtos.ScalarValueOrBuilder> - internalGetBaseFieldBuilder() { - if (baseBuilder_ == null) { - baseBuilder_ = new com.google.protobuf.SingleFieldBuilder< - io.github.dfa1.vortex.proto.ScalarProtos.ScalarValue, io.github.dfa1.vortex.proto.ScalarProtos.ScalarValue.Builder, io.github.dfa1.vortex.proto.ScalarProtos.ScalarValueOrBuilder>( - getBase(), - getParentForChildren(), - isClean()); - base_ = null; - } - return baseBuilder_; - } - - private io.github.dfa1.vortex.proto.ScalarProtos.ScalarValue multiplier_; - private com.google.protobuf.SingleFieldBuilder< - io.github.dfa1.vortex.proto.ScalarProtos.ScalarValue, io.github.dfa1.vortex.proto.ScalarProtos.ScalarValue.Builder, io.github.dfa1.vortex.proto.ScalarProtos.ScalarValueOrBuilder> multiplierBuilder_; - /** - * .vortex.scalar.ScalarValue multiplier = 2; - * @return Whether the multiplier field is set. - */ - public boolean hasMultiplier() { - return ((bitField0_ & 0x00000002) != 0); - } - /** - * .vortex.scalar.ScalarValue multiplier = 2; - * @return The multiplier. - */ - public io.github.dfa1.vortex.proto.ScalarProtos.ScalarValue getMultiplier() { - if (multiplierBuilder_ == null) { - return multiplier_ == null ? io.github.dfa1.vortex.proto.ScalarProtos.ScalarValue.getDefaultInstance() : multiplier_; - } else { - return multiplierBuilder_.getMessage(); - } - } - /** - * .vortex.scalar.ScalarValue multiplier = 2; - */ - public Builder setMultiplier(io.github.dfa1.vortex.proto.ScalarProtos.ScalarValue value) { - if (multiplierBuilder_ == null) { - if (value == null) { - throw new NullPointerException(); - } - multiplier_ = value; - } else { - multiplierBuilder_.setMessage(value); - } - bitField0_ |= 0x00000002; - onChanged(); - return this; - } - /** - * .vortex.scalar.ScalarValue multiplier = 2; - */ - public Builder setMultiplier( - io.github.dfa1.vortex.proto.ScalarProtos.ScalarValue.Builder builderForValue) { - if (multiplierBuilder_ == null) { - multiplier_ = builderForValue.build(); - } else { - multiplierBuilder_.setMessage(builderForValue.build()); - } - bitField0_ |= 0x00000002; - onChanged(); - return this; - } - /** - * .vortex.scalar.ScalarValue multiplier = 2; - */ - public Builder mergeMultiplier(io.github.dfa1.vortex.proto.ScalarProtos.ScalarValue value) { - if (multiplierBuilder_ == null) { - if (((bitField0_ & 0x00000002) != 0) && - multiplier_ != null && - multiplier_ != io.github.dfa1.vortex.proto.ScalarProtos.ScalarValue.getDefaultInstance()) { - getMultiplierBuilder().mergeFrom(value); - } else { - multiplier_ = value; - } - } else { - multiplierBuilder_.mergeFrom(value); - } - if (multiplier_ != null) { - bitField0_ |= 0x00000002; - onChanged(); - } - return this; - } - /** - * .vortex.scalar.ScalarValue multiplier = 2; - */ - public Builder clearMultiplier() { - bitField0_ = (bitField0_ & ~0x00000002); - multiplier_ = null; - if (multiplierBuilder_ != null) { - multiplierBuilder_.dispose(); - multiplierBuilder_ = null; - } - onChanged(); - return this; - } - /** - * .vortex.scalar.ScalarValue multiplier = 2; - */ - public io.github.dfa1.vortex.proto.ScalarProtos.ScalarValue.Builder getMultiplierBuilder() { - bitField0_ |= 0x00000002; - onChanged(); - return internalGetMultiplierFieldBuilder().getBuilder(); - } - /** - * .vortex.scalar.ScalarValue multiplier = 2; - */ - public io.github.dfa1.vortex.proto.ScalarProtos.ScalarValueOrBuilder getMultiplierOrBuilder() { - if (multiplierBuilder_ != null) { - return multiplierBuilder_.getMessageOrBuilder(); - } else { - return multiplier_ == null ? - io.github.dfa1.vortex.proto.ScalarProtos.ScalarValue.getDefaultInstance() : multiplier_; - } - } - /** - * .vortex.scalar.ScalarValue multiplier = 2; - */ - private com.google.protobuf.SingleFieldBuilder< - io.github.dfa1.vortex.proto.ScalarProtos.ScalarValue, io.github.dfa1.vortex.proto.ScalarProtos.ScalarValue.Builder, io.github.dfa1.vortex.proto.ScalarProtos.ScalarValueOrBuilder> - internalGetMultiplierFieldBuilder() { - if (multiplierBuilder_ == null) { - multiplierBuilder_ = new com.google.protobuf.SingleFieldBuilder< - io.github.dfa1.vortex.proto.ScalarProtos.ScalarValue, io.github.dfa1.vortex.proto.ScalarProtos.ScalarValue.Builder, io.github.dfa1.vortex.proto.ScalarProtos.ScalarValueOrBuilder>( - getMultiplier(), - getParentForChildren(), - isClean()); - multiplier_ = null; - } - return multiplierBuilder_; - } - - // @@protoc_insertion_point(builder_scope:vortex.encodings.SequenceMetadata) - } - - // @@protoc_insertion_point(class_scope:vortex.encodings.SequenceMetadata) - private static final io.github.dfa1.vortex.proto.EncodingProtos.SequenceMetadata DEFAULT_INSTANCE; - static { - DEFAULT_INSTANCE = new io.github.dfa1.vortex.proto.EncodingProtos.SequenceMetadata(); - } - - public static io.github.dfa1.vortex.proto.EncodingProtos.SequenceMetadata getDefaultInstance() { - return DEFAULT_INSTANCE; - } - - private static final com.google.protobuf.Parser - PARSER = new com.google.protobuf.AbstractParser() { - @java.lang.Override - public SequenceMetadata parsePartialFrom( - com.google.protobuf.CodedInputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws com.google.protobuf.InvalidProtocolBufferException { - Builder builder = newBuilder(); - try { - builder.mergeFrom(input, extensionRegistry); - } catch (com.google.protobuf.InvalidProtocolBufferException e) { - throw e.setUnfinishedMessage(builder.buildPartial()); - } catch (com.google.protobuf.UninitializedMessageException e) { - throw e.asInvalidProtocolBufferException().setUnfinishedMessage(builder.buildPartial()); - } catch (java.io.IOException e) { - throw new com.google.protobuf.InvalidProtocolBufferException(e) - .setUnfinishedMessage(builder.buildPartial()); - } - return builder.buildPartial(); - } - }; - - public static com.google.protobuf.Parser parser() { - return PARSER; - } - - @java.lang.Override - public com.google.protobuf.Parser getParserForType() { - return PARSER; - } - - @java.lang.Override - public io.github.dfa1.vortex.proto.EncodingProtos.SequenceMetadata getDefaultInstanceForType() { - return DEFAULT_INSTANCE; - } - - } - - public interface VarBinMetadataOrBuilder extends - // @@protoc_insertion_point(interface_extends:vortex.encodings.VarBinMetadata) - com.google.protobuf.MessageOrBuilder { - - /** - * .vortex.dtype.PType offsets_ptype = 1; - * @return The enum numeric value on the wire for offsetsPtype. - */ - int getOffsetsPtypeValue(); - /** - * .vortex.dtype.PType offsets_ptype = 1; - * @return The offsetsPtype. - */ - io.github.dfa1.vortex.proto.DTypeProtos.PType getOffsetsPtype(); - } - /** - * Protobuf type {@code vortex.encodings.VarBinMetadata} - */ - public static final class VarBinMetadata extends - com.google.protobuf.GeneratedMessage implements - // @@protoc_insertion_point(message_implements:vortex.encodings.VarBinMetadata) - VarBinMetadataOrBuilder { - private static final long serialVersionUID = 0L; - static { - com.google.protobuf.RuntimeVersion.validateProtobufGencodeVersion( - com.google.protobuf.RuntimeVersion.RuntimeDomain.PUBLIC, - /* major= */ 4, - /* minor= */ 35, - /* patch= */ 0, - /* suffix= */ "", - "VarBinMetadata"); - } - // Use VarBinMetadata.newBuilder() to construct. - private VarBinMetadata(com.google.protobuf.GeneratedMessage.Builder builder) { - super(builder); - } - private VarBinMetadata() { - offsetsPtype_ = 0; - } - - public static final com.google.protobuf.Descriptors.Descriptor - getDescriptor() { - return io.github.dfa1.vortex.proto.EncodingProtos.internal_static_vortex_encodings_VarBinMetadata_descriptor; - } - - @java.lang.Override - public com.google.protobuf.Descriptors.Descriptor getDescriptorForType() { - return io.github.dfa1.vortex.proto.EncodingProtos.internal_static_vortex_encodings_VarBinMetadata_descriptor; - } - - @java.lang.Override - protected com.google.protobuf.GeneratedMessage.FieldAccessorTable - internalGetFieldAccessorTable() { - return io.github.dfa1.vortex.proto.EncodingProtos.internal_static_vortex_encodings_VarBinMetadata_fieldAccessorTable - .ensureFieldAccessorsInitialized( - io.github.dfa1.vortex.proto.EncodingProtos.VarBinMetadata.class, io.github.dfa1.vortex.proto.EncodingProtos.VarBinMetadata.Builder.class); - } - - public static final int OFFSETS_PTYPE_FIELD_NUMBER = 1; - private int offsetsPtype_ = 0; - /** - * .vortex.dtype.PType offsets_ptype = 1; - * @return The enum numeric value on the wire for offsetsPtype. - */ - @java.lang.Override public int getOffsetsPtypeValue() { - return offsetsPtype_; - } - /** - * .vortex.dtype.PType offsets_ptype = 1; - * @return The offsetsPtype. - */ - @java.lang.Override public io.github.dfa1.vortex.proto.DTypeProtos.PType getOffsetsPtype() { - io.github.dfa1.vortex.proto.DTypeProtos.PType result = io.github.dfa1.vortex.proto.DTypeProtos.PType.forNumber(offsetsPtype_); - return result == null ? io.github.dfa1.vortex.proto.DTypeProtos.PType.UNRECOGNIZED : result; - } - - private byte memoizedIsInitialized = -1; - @java.lang.Override - public final boolean isInitialized() { - byte isInitialized = memoizedIsInitialized; - if (isInitialized == 1) return true; - if (isInitialized == 0) return false; - - memoizedIsInitialized = 1; - return true; - } - - @java.lang.Override - public void writeTo(com.google.protobuf.CodedOutputStream output) - throws java.io.IOException { - if (offsetsPtype_ != io.github.dfa1.vortex.proto.DTypeProtos.PType.U8.getNumber()) { - output.writeEnum(1, offsetsPtype_); - } - getUnknownFields().writeTo(output); - } - private int computeSerializedSize_0() { - int size = 0; - if (offsetsPtype_ != io.github.dfa1.vortex.proto.DTypeProtos.PType.U8.getNumber()) { - size += com.google.protobuf.CodedOutputStream - .computeEnumSize(1, offsetsPtype_); - } - return size; - } - @java.lang.Override - public int getSerializedSize() { - int size = memoizedSize; - if (size != -1) return size; - - size = 0; - size += computeSerializedSize_0(); - size += getUnknownFields().getSerializedSize(); - memoizedSize = size; - return size; - } - - @java.lang.Override - public boolean equals(final java.lang.Object obj) { - if (obj == this) { - return true; - } - if (!(obj instanceof io.github.dfa1.vortex.proto.EncodingProtos.VarBinMetadata)) { - return super.equals(obj); - } - io.github.dfa1.vortex.proto.EncodingProtos.VarBinMetadata other = (io.github.dfa1.vortex.proto.EncodingProtos.VarBinMetadata) obj; - - if (offsetsPtype_ != other.offsetsPtype_) return false; - if (!getUnknownFields().equals(other.getUnknownFields())) return false; - return true; - } - - @java.lang.Override - public int hashCode() { - if (memoizedHashCode != 0) { - return memoizedHashCode; - } - int hash = 41; - hash = (19 * hash) + getDescriptor().hashCode(); - hash = (37 * hash) + OFFSETS_PTYPE_FIELD_NUMBER; - hash = (53 * hash) + offsetsPtype_; - hash = (29 * hash) + getUnknownFields().hashCode(); - memoizedHashCode = hash; - return hash; - } - - public static io.github.dfa1.vortex.proto.EncodingProtos.VarBinMetadata parseFrom( - java.nio.ByteBuffer data) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.VarBinMetadata parseFrom( - java.nio.ByteBuffer data, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data, extensionRegistry); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.VarBinMetadata parseFrom( - com.google.protobuf.ByteString data) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.VarBinMetadata parseFrom( - com.google.protobuf.ByteString data, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data, extensionRegistry); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.VarBinMetadata parseFrom(byte[] data) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.VarBinMetadata parseFrom( - byte[] data, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data, extensionRegistry); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.VarBinMetadata parseFrom(java.io.InputStream input) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseWithIOException(PARSER, input); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.VarBinMetadata parseFrom( - java.io.InputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseWithIOException(PARSER, input, extensionRegistry); - } - - public static io.github.dfa1.vortex.proto.EncodingProtos.VarBinMetadata parseDelimitedFrom(java.io.InputStream input) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseDelimitedWithIOException(PARSER, input); - } - - public static io.github.dfa1.vortex.proto.EncodingProtos.VarBinMetadata parseDelimitedFrom( - java.io.InputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseDelimitedWithIOException(PARSER, input, extensionRegistry); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.VarBinMetadata parseFrom( - com.google.protobuf.CodedInputStream input) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseWithIOException(PARSER, input); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.VarBinMetadata parseFrom( - com.google.protobuf.CodedInputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseWithIOException(PARSER, input, extensionRegistry); - } - - @java.lang.Override - public Builder newBuilderForType() { return newBuilder(); } - public static Builder newBuilder() { - return DEFAULT_INSTANCE.toBuilder(); - } - public static Builder newBuilder(io.github.dfa1.vortex.proto.EncodingProtos.VarBinMetadata prototype) { - return DEFAULT_INSTANCE.toBuilder().mergeFrom(prototype); - } - @java.lang.Override - public Builder toBuilder() { - return this == DEFAULT_INSTANCE - ? new Builder() : new Builder().mergeFrom(this); - } - - @java.lang.Override - protected Builder newBuilderForType( - com.google.protobuf.GeneratedMessage.BuilderParent parent) { - Builder builder = new Builder(parent); - return builder; - } - /** - * Protobuf type {@code vortex.encodings.VarBinMetadata} - */ - public static final class Builder extends - com.google.protobuf.GeneratedMessage.Builder implements - // @@protoc_insertion_point(builder_implements:vortex.encodings.VarBinMetadata) - io.github.dfa1.vortex.proto.EncodingProtos.VarBinMetadataOrBuilder { - public static final com.google.protobuf.Descriptors.Descriptor - getDescriptor() { - return io.github.dfa1.vortex.proto.EncodingProtos.internal_static_vortex_encodings_VarBinMetadata_descriptor; - } - - @java.lang.Override - protected com.google.protobuf.GeneratedMessage.FieldAccessorTable - internalGetFieldAccessorTable() { - return io.github.dfa1.vortex.proto.EncodingProtos.internal_static_vortex_encodings_VarBinMetadata_fieldAccessorTable - .ensureFieldAccessorsInitialized( - io.github.dfa1.vortex.proto.EncodingProtos.VarBinMetadata.class, io.github.dfa1.vortex.proto.EncodingProtos.VarBinMetadata.Builder.class); - } - - // Construct using io.github.dfa1.vortex.proto.EncodingProtos.VarBinMetadata.newBuilder() - private Builder() { - - } - - private Builder( - com.google.protobuf.GeneratedMessage.BuilderParent parent) { - super(parent); - - } - @java.lang.Override - public Builder clear() { - super.clear(); - bitField0_ = 0; - offsetsPtype_ = 0; - return this; - } - - @java.lang.Override - public com.google.protobuf.Descriptors.Descriptor - getDescriptorForType() { - return io.github.dfa1.vortex.proto.EncodingProtos.internal_static_vortex_encodings_VarBinMetadata_descriptor; - } - - @java.lang.Override - public io.github.dfa1.vortex.proto.EncodingProtos.VarBinMetadata getDefaultInstanceForType() { - return io.github.dfa1.vortex.proto.EncodingProtos.VarBinMetadata.getDefaultInstance(); - } - - @java.lang.Override - public io.github.dfa1.vortex.proto.EncodingProtos.VarBinMetadata build() { - io.github.dfa1.vortex.proto.EncodingProtos.VarBinMetadata result = buildPartial(); - if (!result.isInitialized()) { - throw newUninitializedMessageException(result); - } - return result; - } - - @java.lang.Override - public io.github.dfa1.vortex.proto.EncodingProtos.VarBinMetadata buildPartial() { - io.github.dfa1.vortex.proto.EncodingProtos.VarBinMetadata result = new io.github.dfa1.vortex.proto.EncodingProtos.VarBinMetadata(this); - if (bitField0_ != 0) { buildPartial0(result); } - onBuilt(); - return result; - } - - private void buildPartial0(io.github.dfa1.vortex.proto.EncodingProtos.VarBinMetadata result) { - int from_bitField0_ = bitField0_; - if (((from_bitField0_ & 0x00000001) != 0)) { - result.offsetsPtype_ = offsetsPtype_; - } - } - - @java.lang.Override - public Builder mergeFrom(com.google.protobuf.Message other) { - if (other instanceof io.github.dfa1.vortex.proto.EncodingProtos.VarBinMetadata) { - return mergeFrom((io.github.dfa1.vortex.proto.EncodingProtos.VarBinMetadata)other); - } else { - super.mergeFrom(other); - return this; - } - } - - public Builder mergeFrom(io.github.dfa1.vortex.proto.EncodingProtos.VarBinMetadata other) { - if (other == io.github.dfa1.vortex.proto.EncodingProtos.VarBinMetadata.getDefaultInstance()) return this; - if (other.offsetsPtype_ != 0) { - setOffsetsPtypeValue(other.getOffsetsPtypeValue()); - } - this.mergeUnknownFields(other.getUnknownFields()); - onChanged(); - return this; - } - - @java.lang.Override - public final boolean isInitialized() { - return true; - } - - @java.lang.Override - public Builder mergeFrom( - com.google.protobuf.CodedInputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws java.io.IOException { - if (extensionRegistry == null) { - throw new java.lang.NullPointerException(); - } - try { - boolean done = false; - while (!done) { - int tag = input.readTag(); - switch (tag) { - case 0: - done = true; - break; - case 8: { - offsetsPtype_ = input.readEnum(); - bitField0_ |= 0x00000001; - break; - } // case 8 - default: { - if (!super.parseUnknownField(input, extensionRegistry, tag)) { - done = true; // was an endgroup tag - } - break; - } // default: - } // switch (tag) - } // while (!done) - } catch (com.google.protobuf.InvalidProtocolBufferException e) { - throw e.unwrapIOException(); - } finally { - onChanged(); - } // finally - return this; - } - private int bitField0_; - - private int offsetsPtype_ = 0; - /** - * .vortex.dtype.PType offsets_ptype = 1; - * @return The enum numeric value on the wire for offsetsPtype. - */ - @java.lang.Override public int getOffsetsPtypeValue() { - return offsetsPtype_; - } - /** - * .vortex.dtype.PType offsets_ptype = 1; - * @param value The enum numeric value on the wire for offsetsPtype to set. - * @return This builder for chaining. - */ - public Builder setOffsetsPtypeValue(int value) { - offsetsPtype_ = value; - bitField0_ |= 0x00000001; - onChanged(); - return this; - } - /** - * .vortex.dtype.PType offsets_ptype = 1; - * @return The offsetsPtype. - */ - @java.lang.Override - public io.github.dfa1.vortex.proto.DTypeProtos.PType getOffsetsPtype() { - io.github.dfa1.vortex.proto.DTypeProtos.PType result = io.github.dfa1.vortex.proto.DTypeProtos.PType.forNumber(offsetsPtype_); - return result == null ? io.github.dfa1.vortex.proto.DTypeProtos.PType.UNRECOGNIZED : result; - } - /** - * .vortex.dtype.PType offsets_ptype = 1; - * @param value The offsetsPtype to set. - * @throws IllegalArgumentException if UNRECOGNIZED is provided. - * @return This builder for chaining. - */ - public Builder setOffsetsPtype(io.github.dfa1.vortex.proto.DTypeProtos.PType value) { - if (value == null) { throw new NullPointerException(); } - bitField0_ |= 0x00000001; - offsetsPtype_ = value.getNumber(); - onChanged(); - return this; - } - /** - * .vortex.dtype.PType offsets_ptype = 1; - * @return This builder for chaining. - */ - public Builder clearOffsetsPtype() { - bitField0_ = (bitField0_ & ~0x00000001); - offsetsPtype_ = 0; - onChanged(); - return this; - } - - // @@protoc_insertion_point(builder_scope:vortex.encodings.VarBinMetadata) - } - - // @@protoc_insertion_point(class_scope:vortex.encodings.VarBinMetadata) - private static final io.github.dfa1.vortex.proto.EncodingProtos.VarBinMetadata DEFAULT_INSTANCE; - static { - DEFAULT_INSTANCE = new io.github.dfa1.vortex.proto.EncodingProtos.VarBinMetadata(); - } - - public static io.github.dfa1.vortex.proto.EncodingProtos.VarBinMetadata getDefaultInstance() { - return DEFAULT_INSTANCE; - } - - private static final com.google.protobuf.Parser - PARSER = new com.google.protobuf.AbstractParser() { - @java.lang.Override - public VarBinMetadata parsePartialFrom( - com.google.protobuf.CodedInputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws com.google.protobuf.InvalidProtocolBufferException { - Builder builder = newBuilder(); - try { - builder.mergeFrom(input, extensionRegistry); - } catch (com.google.protobuf.InvalidProtocolBufferException e) { - throw e.setUnfinishedMessage(builder.buildPartial()); - } catch (com.google.protobuf.UninitializedMessageException e) { - throw e.asInvalidProtocolBufferException().setUnfinishedMessage(builder.buildPartial()); - } catch (java.io.IOException e) { - throw new com.google.protobuf.InvalidProtocolBufferException(e) - .setUnfinishedMessage(builder.buildPartial()); - } - return builder.buildPartial(); - } - }; - - public static com.google.protobuf.Parser parser() { - return PARSER; - } - - @java.lang.Override - public com.google.protobuf.Parser getParserForType() { - return PARSER; - } - - @java.lang.Override - public io.github.dfa1.vortex.proto.EncodingProtos.VarBinMetadata getDefaultInstanceForType() { - return DEFAULT_INSTANCE; - } - - } - - public interface FSSTMetadataOrBuilder extends - // @@protoc_insertion_point(interface_extends:vortex.encodings.FSSTMetadata) - com.google.protobuf.MessageOrBuilder { - - /** - * .vortex.dtype.PType uncompressed_lengths_ptype = 1; - * @return The enum numeric value on the wire for uncompressedLengthsPtype. - */ - int getUncompressedLengthsPtypeValue(); - /** - * .vortex.dtype.PType uncompressed_lengths_ptype = 1; - * @return The uncompressedLengthsPtype. - */ - io.github.dfa1.vortex.proto.DTypeProtos.PType getUncompressedLengthsPtype(); - - /** - * .vortex.dtype.PType codes_offsets_ptype = 2; - * @return The enum numeric value on the wire for codesOffsetsPtype. - */ - int getCodesOffsetsPtypeValue(); - /** - * .vortex.dtype.PType codes_offsets_ptype = 2; - * @return The codesOffsetsPtype. - */ - io.github.dfa1.vortex.proto.DTypeProtos.PType getCodesOffsetsPtype(); - } - /** - * Protobuf type {@code vortex.encodings.FSSTMetadata} - */ - public static final class FSSTMetadata extends - com.google.protobuf.GeneratedMessage implements - // @@protoc_insertion_point(message_implements:vortex.encodings.FSSTMetadata) - FSSTMetadataOrBuilder { - private static final long serialVersionUID = 0L; - static { - com.google.protobuf.RuntimeVersion.validateProtobufGencodeVersion( - com.google.protobuf.RuntimeVersion.RuntimeDomain.PUBLIC, - /* major= */ 4, - /* minor= */ 35, - /* patch= */ 0, - /* suffix= */ "", - "FSSTMetadata"); - } - // Use FSSTMetadata.newBuilder() to construct. - private FSSTMetadata(com.google.protobuf.GeneratedMessage.Builder builder) { - super(builder); - } - private FSSTMetadata() { - uncompressedLengthsPtype_ = 0; - codesOffsetsPtype_ = 0; - } - - public static final com.google.protobuf.Descriptors.Descriptor - getDescriptor() { - return io.github.dfa1.vortex.proto.EncodingProtos.internal_static_vortex_encodings_FSSTMetadata_descriptor; - } - - @java.lang.Override - public com.google.protobuf.Descriptors.Descriptor getDescriptorForType() { - return io.github.dfa1.vortex.proto.EncodingProtos.internal_static_vortex_encodings_FSSTMetadata_descriptor; - } - - @java.lang.Override - protected com.google.protobuf.GeneratedMessage.FieldAccessorTable - internalGetFieldAccessorTable() { - return io.github.dfa1.vortex.proto.EncodingProtos.internal_static_vortex_encodings_FSSTMetadata_fieldAccessorTable - .ensureFieldAccessorsInitialized( - io.github.dfa1.vortex.proto.EncodingProtos.FSSTMetadata.class, io.github.dfa1.vortex.proto.EncodingProtos.FSSTMetadata.Builder.class); - } - - public static final int UNCOMPRESSED_LENGTHS_PTYPE_FIELD_NUMBER = 1; - private int uncompressedLengthsPtype_ = 0; - /** - * .vortex.dtype.PType uncompressed_lengths_ptype = 1; - * @return The enum numeric value on the wire for uncompressedLengthsPtype. - */ - @java.lang.Override public int getUncompressedLengthsPtypeValue() { - return uncompressedLengthsPtype_; - } - /** - * .vortex.dtype.PType uncompressed_lengths_ptype = 1; - * @return The uncompressedLengthsPtype. - */ - @java.lang.Override public io.github.dfa1.vortex.proto.DTypeProtos.PType getUncompressedLengthsPtype() { - io.github.dfa1.vortex.proto.DTypeProtos.PType result = io.github.dfa1.vortex.proto.DTypeProtos.PType.forNumber(uncompressedLengthsPtype_); - return result == null ? io.github.dfa1.vortex.proto.DTypeProtos.PType.UNRECOGNIZED : result; - } - - public static final int CODES_OFFSETS_PTYPE_FIELD_NUMBER = 2; - private int codesOffsetsPtype_ = 0; - /** - * .vortex.dtype.PType codes_offsets_ptype = 2; - * @return The enum numeric value on the wire for codesOffsetsPtype. - */ - @java.lang.Override public int getCodesOffsetsPtypeValue() { - return codesOffsetsPtype_; - } - /** - * .vortex.dtype.PType codes_offsets_ptype = 2; - * @return The codesOffsetsPtype. - */ - @java.lang.Override public io.github.dfa1.vortex.proto.DTypeProtos.PType getCodesOffsetsPtype() { - io.github.dfa1.vortex.proto.DTypeProtos.PType result = io.github.dfa1.vortex.proto.DTypeProtos.PType.forNumber(codesOffsetsPtype_); - return result == null ? io.github.dfa1.vortex.proto.DTypeProtos.PType.UNRECOGNIZED : result; - } - - private byte memoizedIsInitialized = -1; - @java.lang.Override - public final boolean isInitialized() { - byte isInitialized = memoizedIsInitialized; - if (isInitialized == 1) return true; - if (isInitialized == 0) return false; - - memoizedIsInitialized = 1; - return true; - } - - @java.lang.Override - public void writeTo(com.google.protobuf.CodedOutputStream output) - throws java.io.IOException { - if (uncompressedLengthsPtype_ != io.github.dfa1.vortex.proto.DTypeProtos.PType.U8.getNumber()) { - output.writeEnum(1, uncompressedLengthsPtype_); - } - if (codesOffsetsPtype_ != io.github.dfa1.vortex.proto.DTypeProtos.PType.U8.getNumber()) { - output.writeEnum(2, codesOffsetsPtype_); - } - getUnknownFields().writeTo(output); - } - private int computeSerializedSize_0() { - int size = 0; - if (uncompressedLengthsPtype_ != io.github.dfa1.vortex.proto.DTypeProtos.PType.U8.getNumber()) { - size += com.google.protobuf.CodedOutputStream - .computeEnumSize(1, uncompressedLengthsPtype_); - } - if (codesOffsetsPtype_ != io.github.dfa1.vortex.proto.DTypeProtos.PType.U8.getNumber()) { - size += com.google.protobuf.CodedOutputStream - .computeEnumSize(2, codesOffsetsPtype_); - } - return size; - } - @java.lang.Override - public int getSerializedSize() { - int size = memoizedSize; - if (size != -1) return size; - - size = 0; - size += computeSerializedSize_0(); - size += getUnknownFields().getSerializedSize(); - memoizedSize = size; - return size; - } - - @java.lang.Override - public boolean equals(final java.lang.Object obj) { - if (obj == this) { - return true; - } - if (!(obj instanceof io.github.dfa1.vortex.proto.EncodingProtos.FSSTMetadata)) { - return super.equals(obj); - } - io.github.dfa1.vortex.proto.EncodingProtos.FSSTMetadata other = (io.github.dfa1.vortex.proto.EncodingProtos.FSSTMetadata) obj; - - if (uncompressedLengthsPtype_ != other.uncompressedLengthsPtype_) return false; - if (codesOffsetsPtype_ != other.codesOffsetsPtype_) return false; - if (!getUnknownFields().equals(other.getUnknownFields())) return false; - return true; - } - - @java.lang.Override - public int hashCode() { - if (memoizedHashCode != 0) { - return memoizedHashCode; - } - int hash = 41; - hash = (19 * hash) + getDescriptor().hashCode(); - hash = (37 * hash) + UNCOMPRESSED_LENGTHS_PTYPE_FIELD_NUMBER; - hash = (53 * hash) + uncompressedLengthsPtype_; - hash = (37 * hash) + CODES_OFFSETS_PTYPE_FIELD_NUMBER; - hash = (53 * hash) + codesOffsetsPtype_; - hash = (29 * hash) + getUnknownFields().hashCode(); - memoizedHashCode = hash; - return hash; - } - - public static io.github.dfa1.vortex.proto.EncodingProtos.FSSTMetadata parseFrom( - java.nio.ByteBuffer data) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.FSSTMetadata parseFrom( - java.nio.ByteBuffer data, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data, extensionRegistry); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.FSSTMetadata parseFrom( - com.google.protobuf.ByteString data) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.FSSTMetadata parseFrom( - com.google.protobuf.ByteString data, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data, extensionRegistry); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.FSSTMetadata parseFrom(byte[] data) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.FSSTMetadata parseFrom( - byte[] data, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data, extensionRegistry); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.FSSTMetadata parseFrom(java.io.InputStream input) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseWithIOException(PARSER, input); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.FSSTMetadata parseFrom( - java.io.InputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseWithIOException(PARSER, input, extensionRegistry); - } - - public static io.github.dfa1.vortex.proto.EncodingProtos.FSSTMetadata parseDelimitedFrom(java.io.InputStream input) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseDelimitedWithIOException(PARSER, input); - } - - public static io.github.dfa1.vortex.proto.EncodingProtos.FSSTMetadata parseDelimitedFrom( - java.io.InputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseDelimitedWithIOException(PARSER, input, extensionRegistry); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.FSSTMetadata parseFrom( - com.google.protobuf.CodedInputStream input) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseWithIOException(PARSER, input); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.FSSTMetadata parseFrom( - com.google.protobuf.CodedInputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseWithIOException(PARSER, input, extensionRegistry); - } - - @java.lang.Override - public Builder newBuilderForType() { return newBuilder(); } - public static Builder newBuilder() { - return DEFAULT_INSTANCE.toBuilder(); - } - public static Builder newBuilder(io.github.dfa1.vortex.proto.EncodingProtos.FSSTMetadata prototype) { - return DEFAULT_INSTANCE.toBuilder().mergeFrom(prototype); - } - @java.lang.Override - public Builder toBuilder() { - return this == DEFAULT_INSTANCE - ? new Builder() : new Builder().mergeFrom(this); - } - - @java.lang.Override - protected Builder newBuilderForType( - com.google.protobuf.GeneratedMessage.BuilderParent parent) { - Builder builder = new Builder(parent); - return builder; - } - /** - * Protobuf type {@code vortex.encodings.FSSTMetadata} - */ - public static final class Builder extends - com.google.protobuf.GeneratedMessage.Builder implements - // @@protoc_insertion_point(builder_implements:vortex.encodings.FSSTMetadata) - io.github.dfa1.vortex.proto.EncodingProtos.FSSTMetadataOrBuilder { - public static final com.google.protobuf.Descriptors.Descriptor - getDescriptor() { - return io.github.dfa1.vortex.proto.EncodingProtos.internal_static_vortex_encodings_FSSTMetadata_descriptor; - } - - @java.lang.Override - protected com.google.protobuf.GeneratedMessage.FieldAccessorTable - internalGetFieldAccessorTable() { - return io.github.dfa1.vortex.proto.EncodingProtos.internal_static_vortex_encodings_FSSTMetadata_fieldAccessorTable - .ensureFieldAccessorsInitialized( - io.github.dfa1.vortex.proto.EncodingProtos.FSSTMetadata.class, io.github.dfa1.vortex.proto.EncodingProtos.FSSTMetadata.Builder.class); - } - - // Construct using io.github.dfa1.vortex.proto.EncodingProtos.FSSTMetadata.newBuilder() - private Builder() { - - } - - private Builder( - com.google.protobuf.GeneratedMessage.BuilderParent parent) { - super(parent); - - } - @java.lang.Override - public Builder clear() { - super.clear(); - bitField0_ = 0; - uncompressedLengthsPtype_ = 0; - codesOffsetsPtype_ = 0; - return this; - } - - @java.lang.Override - public com.google.protobuf.Descriptors.Descriptor - getDescriptorForType() { - return io.github.dfa1.vortex.proto.EncodingProtos.internal_static_vortex_encodings_FSSTMetadata_descriptor; - } - - @java.lang.Override - public io.github.dfa1.vortex.proto.EncodingProtos.FSSTMetadata getDefaultInstanceForType() { - return io.github.dfa1.vortex.proto.EncodingProtos.FSSTMetadata.getDefaultInstance(); - } - - @java.lang.Override - public io.github.dfa1.vortex.proto.EncodingProtos.FSSTMetadata build() { - io.github.dfa1.vortex.proto.EncodingProtos.FSSTMetadata result = buildPartial(); - if (!result.isInitialized()) { - throw newUninitializedMessageException(result); - } - return result; - } - - @java.lang.Override - public io.github.dfa1.vortex.proto.EncodingProtos.FSSTMetadata buildPartial() { - io.github.dfa1.vortex.proto.EncodingProtos.FSSTMetadata result = new io.github.dfa1.vortex.proto.EncodingProtos.FSSTMetadata(this); - if (bitField0_ != 0) { buildPartial0(result); } - onBuilt(); - return result; - } - - private void buildPartial0(io.github.dfa1.vortex.proto.EncodingProtos.FSSTMetadata result) { - int from_bitField0_ = bitField0_; - if (((from_bitField0_ & 0x00000001) != 0)) { - result.uncompressedLengthsPtype_ = uncompressedLengthsPtype_; - } - if (((from_bitField0_ & 0x00000002) != 0)) { - result.codesOffsetsPtype_ = codesOffsetsPtype_; - } - } - - @java.lang.Override - public Builder mergeFrom(com.google.protobuf.Message other) { - if (other instanceof io.github.dfa1.vortex.proto.EncodingProtos.FSSTMetadata) { - return mergeFrom((io.github.dfa1.vortex.proto.EncodingProtos.FSSTMetadata)other); - } else { - super.mergeFrom(other); - return this; - } - } - - public Builder mergeFrom(io.github.dfa1.vortex.proto.EncodingProtos.FSSTMetadata other) { - if (other == io.github.dfa1.vortex.proto.EncodingProtos.FSSTMetadata.getDefaultInstance()) return this; - if (other.uncompressedLengthsPtype_ != 0) { - setUncompressedLengthsPtypeValue(other.getUncompressedLengthsPtypeValue()); - } - if (other.codesOffsetsPtype_ != 0) { - setCodesOffsetsPtypeValue(other.getCodesOffsetsPtypeValue()); - } - this.mergeUnknownFields(other.getUnknownFields()); - onChanged(); - return this; - } - - @java.lang.Override - public final boolean isInitialized() { - return true; - } - - @java.lang.Override - public Builder mergeFrom( - com.google.protobuf.CodedInputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws java.io.IOException { - if (extensionRegistry == null) { - throw new java.lang.NullPointerException(); - } - try { - boolean done = false; - while (!done) { - int tag = input.readTag(); - switch (tag) { - case 0: - done = true; - break; - case 8: { - uncompressedLengthsPtype_ = input.readEnum(); - bitField0_ |= 0x00000001; - break; - } // case 8 - case 16: { - codesOffsetsPtype_ = input.readEnum(); - bitField0_ |= 0x00000002; - break; - } // case 16 - default: { - if (!super.parseUnknownField(input, extensionRegistry, tag)) { - done = true; // was an endgroup tag - } - break; - } // default: - } // switch (tag) - } // while (!done) - } catch (com.google.protobuf.InvalidProtocolBufferException e) { - throw e.unwrapIOException(); - } finally { - onChanged(); - } // finally - return this; - } - private int bitField0_; - - private int uncompressedLengthsPtype_ = 0; - /** - * .vortex.dtype.PType uncompressed_lengths_ptype = 1; - * @return The enum numeric value on the wire for uncompressedLengthsPtype. - */ - @java.lang.Override public int getUncompressedLengthsPtypeValue() { - return uncompressedLengthsPtype_; - } - /** - * .vortex.dtype.PType uncompressed_lengths_ptype = 1; - * @param value The enum numeric value on the wire for uncompressedLengthsPtype to set. - * @return This builder for chaining. - */ - public Builder setUncompressedLengthsPtypeValue(int value) { - uncompressedLengthsPtype_ = value; - bitField0_ |= 0x00000001; - onChanged(); - return this; - } - /** - * .vortex.dtype.PType uncompressed_lengths_ptype = 1; - * @return The uncompressedLengthsPtype. - */ - @java.lang.Override - public io.github.dfa1.vortex.proto.DTypeProtos.PType getUncompressedLengthsPtype() { - io.github.dfa1.vortex.proto.DTypeProtos.PType result = io.github.dfa1.vortex.proto.DTypeProtos.PType.forNumber(uncompressedLengthsPtype_); - return result == null ? io.github.dfa1.vortex.proto.DTypeProtos.PType.UNRECOGNIZED : result; - } - /** - * .vortex.dtype.PType uncompressed_lengths_ptype = 1; - * @param value The uncompressedLengthsPtype to set. - * @throws IllegalArgumentException if UNRECOGNIZED is provided. - * @return This builder for chaining. - */ - public Builder setUncompressedLengthsPtype(io.github.dfa1.vortex.proto.DTypeProtos.PType value) { - if (value == null) { throw new NullPointerException(); } - bitField0_ |= 0x00000001; - uncompressedLengthsPtype_ = value.getNumber(); - onChanged(); - return this; - } - /** - * .vortex.dtype.PType uncompressed_lengths_ptype = 1; - * @return This builder for chaining. - */ - public Builder clearUncompressedLengthsPtype() { - bitField0_ = (bitField0_ & ~0x00000001); - uncompressedLengthsPtype_ = 0; - onChanged(); - return this; - } - - private int codesOffsetsPtype_ = 0; - /** - * .vortex.dtype.PType codes_offsets_ptype = 2; - * @return The enum numeric value on the wire for codesOffsetsPtype. - */ - @java.lang.Override public int getCodesOffsetsPtypeValue() { - return codesOffsetsPtype_; - } - /** - * .vortex.dtype.PType codes_offsets_ptype = 2; - * @param value The enum numeric value on the wire for codesOffsetsPtype to set. - * @return This builder for chaining. - */ - public Builder setCodesOffsetsPtypeValue(int value) { - codesOffsetsPtype_ = value; - bitField0_ |= 0x00000002; - onChanged(); - return this; - } - /** - * .vortex.dtype.PType codes_offsets_ptype = 2; - * @return The codesOffsetsPtype. - */ - @java.lang.Override - public io.github.dfa1.vortex.proto.DTypeProtos.PType getCodesOffsetsPtype() { - io.github.dfa1.vortex.proto.DTypeProtos.PType result = io.github.dfa1.vortex.proto.DTypeProtos.PType.forNumber(codesOffsetsPtype_); - return result == null ? io.github.dfa1.vortex.proto.DTypeProtos.PType.UNRECOGNIZED : result; - } - /** - * .vortex.dtype.PType codes_offsets_ptype = 2; - * @param value The codesOffsetsPtype to set. - * @throws IllegalArgumentException if UNRECOGNIZED is provided. - * @return This builder for chaining. - */ - public Builder setCodesOffsetsPtype(io.github.dfa1.vortex.proto.DTypeProtos.PType value) { - if (value == null) { throw new NullPointerException(); } - bitField0_ |= 0x00000002; - codesOffsetsPtype_ = value.getNumber(); - onChanged(); - return this; - } - /** - * .vortex.dtype.PType codes_offsets_ptype = 2; - * @return This builder for chaining. - */ - public Builder clearCodesOffsetsPtype() { - bitField0_ = (bitField0_ & ~0x00000002); - codesOffsetsPtype_ = 0; - onChanged(); - return this; - } - - // @@protoc_insertion_point(builder_scope:vortex.encodings.FSSTMetadata) - } - - // @@protoc_insertion_point(class_scope:vortex.encodings.FSSTMetadata) - private static final io.github.dfa1.vortex.proto.EncodingProtos.FSSTMetadata DEFAULT_INSTANCE; - static { - DEFAULT_INSTANCE = new io.github.dfa1.vortex.proto.EncodingProtos.FSSTMetadata(); - } - - public static io.github.dfa1.vortex.proto.EncodingProtos.FSSTMetadata getDefaultInstance() { - return DEFAULT_INSTANCE; - } - - private static final com.google.protobuf.Parser - PARSER = new com.google.protobuf.AbstractParser() { - @java.lang.Override - public FSSTMetadata parsePartialFrom( - com.google.protobuf.CodedInputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws com.google.protobuf.InvalidProtocolBufferException { - Builder builder = newBuilder(); - try { - builder.mergeFrom(input, extensionRegistry); - } catch (com.google.protobuf.InvalidProtocolBufferException e) { - throw e.setUnfinishedMessage(builder.buildPartial()); - } catch (com.google.protobuf.UninitializedMessageException e) { - throw e.asInvalidProtocolBufferException().setUnfinishedMessage(builder.buildPartial()); - } catch (java.io.IOException e) { - throw new com.google.protobuf.InvalidProtocolBufferException(e) - .setUnfinishedMessage(builder.buildPartial()); - } - return builder.buildPartial(); - } - }; - - public static com.google.protobuf.Parser parser() { - return PARSER; - } - - @java.lang.Override - public com.google.protobuf.Parser getParserForType() { - return PARSER; - } - - @java.lang.Override - public io.github.dfa1.vortex.proto.EncodingProtos.FSSTMetadata getDefaultInstanceForType() { - return DEFAULT_INSTANCE; - } - - } - - public interface DeltaMetadataOrBuilder extends - // @@protoc_insertion_point(interface_extends:vortex.encodings.DeltaMetadata) - com.google.protobuf.MessageOrBuilder { - - /** - * uint64 deltas_len = 1; - * @return The deltasLen. - */ - long getDeltasLen(); - - /** - * uint32 offset = 2; - * @return The offset. - */ - int getOffset(); - } - /** - * Protobuf type {@code vortex.encodings.DeltaMetadata} - */ - public static final class DeltaMetadata extends - com.google.protobuf.GeneratedMessage implements - // @@protoc_insertion_point(message_implements:vortex.encodings.DeltaMetadata) - DeltaMetadataOrBuilder { - private static final long serialVersionUID = 0L; - static { - com.google.protobuf.RuntimeVersion.validateProtobufGencodeVersion( - com.google.protobuf.RuntimeVersion.RuntimeDomain.PUBLIC, - /* major= */ 4, - /* minor= */ 35, - /* patch= */ 0, - /* suffix= */ "", - "DeltaMetadata"); - } - // Use DeltaMetadata.newBuilder() to construct. - private DeltaMetadata(com.google.protobuf.GeneratedMessage.Builder builder) { - super(builder); - } - private DeltaMetadata() { - } - - public static final com.google.protobuf.Descriptors.Descriptor - getDescriptor() { - return io.github.dfa1.vortex.proto.EncodingProtos.internal_static_vortex_encodings_DeltaMetadata_descriptor; - } - - @java.lang.Override - public com.google.protobuf.Descriptors.Descriptor getDescriptorForType() { - return io.github.dfa1.vortex.proto.EncodingProtos.internal_static_vortex_encodings_DeltaMetadata_descriptor; - } - - @java.lang.Override - protected com.google.protobuf.GeneratedMessage.FieldAccessorTable - internalGetFieldAccessorTable() { - return io.github.dfa1.vortex.proto.EncodingProtos.internal_static_vortex_encodings_DeltaMetadata_fieldAccessorTable - .ensureFieldAccessorsInitialized( - io.github.dfa1.vortex.proto.EncodingProtos.DeltaMetadata.class, io.github.dfa1.vortex.proto.EncodingProtos.DeltaMetadata.Builder.class); - } - - public static final int DELTAS_LEN_FIELD_NUMBER = 1; - private long deltasLen_ = 0L; - /** - * uint64 deltas_len = 1; - * @return The deltasLen. - */ - @java.lang.Override - public long getDeltasLen() { - return deltasLen_; - } - - public static final int OFFSET_FIELD_NUMBER = 2; - private int offset_ = 0; - /** - * uint32 offset = 2; - * @return The offset. - */ - @java.lang.Override - public int getOffset() { - return offset_; - } - - private byte memoizedIsInitialized = -1; - @java.lang.Override - public final boolean isInitialized() { - byte isInitialized = memoizedIsInitialized; - if (isInitialized == 1) return true; - if (isInitialized == 0) return false; - - memoizedIsInitialized = 1; - return true; - } - - @java.lang.Override - public void writeTo(com.google.protobuf.CodedOutputStream output) - throws java.io.IOException { - if (deltasLen_ != 0L) { - output.writeUInt64(1, deltasLen_); - } - if (offset_ != 0) { - output.writeUInt32(2, offset_); - } - getUnknownFields().writeTo(output); - } - private int computeSerializedSize_0() { - int size = 0; - if (deltasLen_ != 0L) { - size += com.google.protobuf.CodedOutputStream - .computeUInt64Size(1, deltasLen_); - } - if (offset_ != 0) { - size += com.google.protobuf.CodedOutputStream - .computeUInt32Size(2, offset_); - } - return size; - } - @java.lang.Override - public int getSerializedSize() { - int size = memoizedSize; - if (size != -1) return size; - - size = 0; - size += computeSerializedSize_0(); - size += getUnknownFields().getSerializedSize(); - memoizedSize = size; - return size; - } - - @java.lang.Override - public boolean equals(final java.lang.Object obj) { - if (obj == this) { - return true; - } - if (!(obj instanceof io.github.dfa1.vortex.proto.EncodingProtos.DeltaMetadata)) { - return super.equals(obj); - } - io.github.dfa1.vortex.proto.EncodingProtos.DeltaMetadata other = (io.github.dfa1.vortex.proto.EncodingProtos.DeltaMetadata) obj; - - if (getDeltasLen() - != other.getDeltasLen()) return false; - if (getOffset() - != other.getOffset()) return false; - if (!getUnknownFields().equals(other.getUnknownFields())) return false; - return true; - } - - @java.lang.Override - public int hashCode() { - if (memoizedHashCode != 0) { - return memoizedHashCode; - } - int hash = 41; - hash = (19 * hash) + getDescriptor().hashCode(); - hash = (37 * hash) + DELTAS_LEN_FIELD_NUMBER; - hash = (53 * hash) + com.google.protobuf.Internal.hashLong( - getDeltasLen()); - hash = (37 * hash) + OFFSET_FIELD_NUMBER; - hash = (53 * hash) + getOffset(); - hash = (29 * hash) + getUnknownFields().hashCode(); - memoizedHashCode = hash; - return hash; - } - - public static io.github.dfa1.vortex.proto.EncodingProtos.DeltaMetadata parseFrom( - java.nio.ByteBuffer data) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.DeltaMetadata parseFrom( - java.nio.ByteBuffer data, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data, extensionRegistry); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.DeltaMetadata parseFrom( - com.google.protobuf.ByteString data) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.DeltaMetadata parseFrom( - com.google.protobuf.ByteString data, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data, extensionRegistry); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.DeltaMetadata parseFrom(byte[] data) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.DeltaMetadata parseFrom( - byte[] data, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data, extensionRegistry); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.DeltaMetadata parseFrom(java.io.InputStream input) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseWithIOException(PARSER, input); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.DeltaMetadata parseFrom( - java.io.InputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseWithIOException(PARSER, input, extensionRegistry); - } - - public static io.github.dfa1.vortex.proto.EncodingProtos.DeltaMetadata parseDelimitedFrom(java.io.InputStream input) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseDelimitedWithIOException(PARSER, input); - } - - public static io.github.dfa1.vortex.proto.EncodingProtos.DeltaMetadata parseDelimitedFrom( - java.io.InputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseDelimitedWithIOException(PARSER, input, extensionRegistry); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.DeltaMetadata parseFrom( - com.google.protobuf.CodedInputStream input) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseWithIOException(PARSER, input); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.DeltaMetadata parseFrom( - com.google.protobuf.CodedInputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseWithIOException(PARSER, input, extensionRegistry); - } - - @java.lang.Override - public Builder newBuilderForType() { return newBuilder(); } - public static Builder newBuilder() { - return DEFAULT_INSTANCE.toBuilder(); - } - public static Builder newBuilder(io.github.dfa1.vortex.proto.EncodingProtos.DeltaMetadata prototype) { - return DEFAULT_INSTANCE.toBuilder().mergeFrom(prototype); - } - @java.lang.Override - public Builder toBuilder() { - return this == DEFAULT_INSTANCE - ? new Builder() : new Builder().mergeFrom(this); - } - - @java.lang.Override - protected Builder newBuilderForType( - com.google.protobuf.GeneratedMessage.BuilderParent parent) { - Builder builder = new Builder(parent); - return builder; - } - /** - * Protobuf type {@code vortex.encodings.DeltaMetadata} - */ - public static final class Builder extends - com.google.protobuf.GeneratedMessage.Builder implements - // @@protoc_insertion_point(builder_implements:vortex.encodings.DeltaMetadata) - io.github.dfa1.vortex.proto.EncodingProtos.DeltaMetadataOrBuilder { - public static final com.google.protobuf.Descriptors.Descriptor - getDescriptor() { - return io.github.dfa1.vortex.proto.EncodingProtos.internal_static_vortex_encodings_DeltaMetadata_descriptor; - } - - @java.lang.Override - protected com.google.protobuf.GeneratedMessage.FieldAccessorTable - internalGetFieldAccessorTable() { - return io.github.dfa1.vortex.proto.EncodingProtos.internal_static_vortex_encodings_DeltaMetadata_fieldAccessorTable - .ensureFieldAccessorsInitialized( - io.github.dfa1.vortex.proto.EncodingProtos.DeltaMetadata.class, io.github.dfa1.vortex.proto.EncodingProtos.DeltaMetadata.Builder.class); - } - - // Construct using io.github.dfa1.vortex.proto.EncodingProtos.DeltaMetadata.newBuilder() - private Builder() { - - } - - private Builder( - com.google.protobuf.GeneratedMessage.BuilderParent parent) { - super(parent); - - } - @java.lang.Override - public Builder clear() { - super.clear(); - bitField0_ = 0; - deltasLen_ = 0L; - offset_ = 0; - return this; - } - - @java.lang.Override - public com.google.protobuf.Descriptors.Descriptor - getDescriptorForType() { - return io.github.dfa1.vortex.proto.EncodingProtos.internal_static_vortex_encodings_DeltaMetadata_descriptor; - } - - @java.lang.Override - public io.github.dfa1.vortex.proto.EncodingProtos.DeltaMetadata getDefaultInstanceForType() { - return io.github.dfa1.vortex.proto.EncodingProtos.DeltaMetadata.getDefaultInstance(); - } - - @java.lang.Override - public io.github.dfa1.vortex.proto.EncodingProtos.DeltaMetadata build() { - io.github.dfa1.vortex.proto.EncodingProtos.DeltaMetadata result = buildPartial(); - if (!result.isInitialized()) { - throw newUninitializedMessageException(result); - } - return result; - } - - @java.lang.Override - public io.github.dfa1.vortex.proto.EncodingProtos.DeltaMetadata buildPartial() { - io.github.dfa1.vortex.proto.EncodingProtos.DeltaMetadata result = new io.github.dfa1.vortex.proto.EncodingProtos.DeltaMetadata(this); - if (bitField0_ != 0) { buildPartial0(result); } - onBuilt(); - return result; - } - - private void buildPartial0(io.github.dfa1.vortex.proto.EncodingProtos.DeltaMetadata result) { - int from_bitField0_ = bitField0_; - if (((from_bitField0_ & 0x00000001) != 0)) { - result.deltasLen_ = deltasLen_; - } - if (((from_bitField0_ & 0x00000002) != 0)) { - result.offset_ = offset_; - } - } - - @java.lang.Override - public Builder mergeFrom(com.google.protobuf.Message other) { - if (other instanceof io.github.dfa1.vortex.proto.EncodingProtos.DeltaMetadata) { - return mergeFrom((io.github.dfa1.vortex.proto.EncodingProtos.DeltaMetadata)other); - } else { - super.mergeFrom(other); - return this; - } - } - - public Builder mergeFrom(io.github.dfa1.vortex.proto.EncodingProtos.DeltaMetadata other) { - if (other == io.github.dfa1.vortex.proto.EncodingProtos.DeltaMetadata.getDefaultInstance()) return this; - if (other.getDeltasLen() != 0L) { - setDeltasLen(other.getDeltasLen()); - } - if (other.getOffset() != 0) { - setOffset(other.getOffset()); - } - this.mergeUnknownFields(other.getUnknownFields()); - onChanged(); - return this; - } - - @java.lang.Override - public final boolean isInitialized() { - return true; - } - - @java.lang.Override - public Builder mergeFrom( - com.google.protobuf.CodedInputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws java.io.IOException { - if (extensionRegistry == null) { - throw new java.lang.NullPointerException(); - } - try { - boolean done = false; - while (!done) { - int tag = input.readTag(); - switch (tag) { - case 0: - done = true; - break; - case 8: { - deltasLen_ = input.readUInt64(); - bitField0_ |= 0x00000001; - break; - } // case 8 - case 16: { - offset_ = input.readUInt32(); - bitField0_ |= 0x00000002; - break; - } // case 16 - default: { - if (!super.parseUnknownField(input, extensionRegistry, tag)) { - done = true; // was an endgroup tag - } - break; - } // default: - } // switch (tag) - } // while (!done) - } catch (com.google.protobuf.InvalidProtocolBufferException e) { - throw e.unwrapIOException(); - } finally { - onChanged(); - } // finally - return this; - } - private int bitField0_; - - private long deltasLen_ ; - /** - * uint64 deltas_len = 1; - * @return The deltasLen. - */ - @java.lang.Override - public long getDeltasLen() { - return deltasLen_; - } - /** - * uint64 deltas_len = 1; - * @param value The deltasLen to set. - * @return This builder for chaining. - */ - public Builder setDeltasLen(long value) { - - deltasLen_ = value; - bitField0_ |= 0x00000001; - onChanged(); - return this; - } - /** - * uint64 deltas_len = 1; - * @return This builder for chaining. - */ - public Builder clearDeltasLen() { - bitField0_ = (bitField0_ & ~0x00000001); - deltasLen_ = 0L; - onChanged(); - return this; - } - - private int offset_ ; - /** - * uint32 offset = 2; - * @return The offset. - */ - @java.lang.Override - public int getOffset() { - return offset_; - } - /** - * uint32 offset = 2; - * @param value The offset to set. - * @return This builder for chaining. - */ - public Builder setOffset(int value) { - - offset_ = value; - bitField0_ |= 0x00000002; - onChanged(); - return this; - } - /** - * uint32 offset = 2; - * @return This builder for chaining. - */ - public Builder clearOffset() { - bitField0_ = (bitField0_ & ~0x00000002); - offset_ = 0; - onChanged(); - return this; - } - - // @@protoc_insertion_point(builder_scope:vortex.encodings.DeltaMetadata) - } - - // @@protoc_insertion_point(class_scope:vortex.encodings.DeltaMetadata) - private static final io.github.dfa1.vortex.proto.EncodingProtos.DeltaMetadata DEFAULT_INSTANCE; - static { - DEFAULT_INSTANCE = new io.github.dfa1.vortex.proto.EncodingProtos.DeltaMetadata(); - } - - public static io.github.dfa1.vortex.proto.EncodingProtos.DeltaMetadata getDefaultInstance() { - return DEFAULT_INSTANCE; - } - - private static final com.google.protobuf.Parser - PARSER = new com.google.protobuf.AbstractParser() { - @java.lang.Override - public DeltaMetadata parsePartialFrom( - com.google.protobuf.CodedInputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws com.google.protobuf.InvalidProtocolBufferException { - Builder builder = newBuilder(); - try { - builder.mergeFrom(input, extensionRegistry); - } catch (com.google.protobuf.InvalidProtocolBufferException e) { - throw e.setUnfinishedMessage(builder.buildPartial()); - } catch (com.google.protobuf.UninitializedMessageException e) { - throw e.asInvalidProtocolBufferException().setUnfinishedMessage(builder.buildPartial()); - } catch (java.io.IOException e) { - throw new com.google.protobuf.InvalidProtocolBufferException(e) - .setUnfinishedMessage(builder.buildPartial()); - } - return builder.buildPartial(); - } - }; - - public static com.google.protobuf.Parser parser() { - return PARSER; - } - - @java.lang.Override - public com.google.protobuf.Parser getParserForType() { - return PARSER; - } - - @java.lang.Override - public io.github.dfa1.vortex.proto.EncodingProtos.DeltaMetadata getDefaultInstanceForType() { - return DEFAULT_INSTANCE; - } - - } - - public interface DecimalMetadataOrBuilder extends - // @@protoc_insertion_point(interface_extends:vortex.encodings.DecimalMetadata) - com.google.protobuf.MessageOrBuilder { - - /** - * 
-     * DecimalType: I8=0, I16=1, I32=2, I64=3, I128=4, I256=5
-     *
- * - * int32 values_type = 1; - * @return The valuesType. - */ - int getValuesType(); - } - /** - * Protobuf type {@code vortex.encodings.DecimalMetadata} - */ - public static final class DecimalMetadata extends - com.google.protobuf.GeneratedMessage implements - // @@protoc_insertion_point(message_implements:vortex.encodings.DecimalMetadata) - DecimalMetadataOrBuilder { - private static final long serialVersionUID = 0L; - static { - com.google.protobuf.RuntimeVersion.validateProtobufGencodeVersion( - com.google.protobuf.RuntimeVersion.RuntimeDomain.PUBLIC, - /* major= */ 4, - /* minor= */ 35, - /* patch= */ 0, - /* suffix= */ "", - "DecimalMetadata"); - } - // Use DecimalMetadata.newBuilder() to construct. - private DecimalMetadata(com.google.protobuf.GeneratedMessage.Builder builder) { - super(builder); - } - private DecimalMetadata() { - } - - public static final com.google.protobuf.Descriptors.Descriptor - getDescriptor() { - return io.github.dfa1.vortex.proto.EncodingProtos.internal_static_vortex_encodings_DecimalMetadata_descriptor; - } - - @java.lang.Override - public com.google.protobuf.Descriptors.Descriptor getDescriptorForType() { - return io.github.dfa1.vortex.proto.EncodingProtos.internal_static_vortex_encodings_DecimalMetadata_descriptor; - } - - @java.lang.Override - protected com.google.protobuf.GeneratedMessage.FieldAccessorTable - internalGetFieldAccessorTable() { - return io.github.dfa1.vortex.proto.EncodingProtos.internal_static_vortex_encodings_DecimalMetadata_fieldAccessorTable - .ensureFieldAccessorsInitialized( - io.github.dfa1.vortex.proto.EncodingProtos.DecimalMetadata.class, io.github.dfa1.vortex.proto.EncodingProtos.DecimalMetadata.Builder.class); - } - - public static final int VALUES_TYPE_FIELD_NUMBER = 1; - private int valuesType_ = 0; - /** - * 
-     * DecimalType: I8=0, I16=1, I32=2, I64=3, I128=4, I256=5
-     *
- * - * int32 values_type = 1; - * @return The valuesType. - */ - @java.lang.Override - public int getValuesType() { - return valuesType_; - } - - private byte memoizedIsInitialized = -1; - @java.lang.Override - public final boolean isInitialized() { - byte isInitialized = memoizedIsInitialized; - if (isInitialized == 1) return true; - if (isInitialized == 0) return false; - - memoizedIsInitialized = 1; - return true; - } - - @java.lang.Override - public void writeTo(com.google.protobuf.CodedOutputStream output) - throws java.io.IOException { - if (valuesType_ != 0) { - output.writeInt32(1, valuesType_); - } - getUnknownFields().writeTo(output); - } - private int computeSerializedSize_0() { - int size = 0; - if (valuesType_ != 0) { - size += com.google.protobuf.CodedOutputStream - .computeInt32Size(1, valuesType_); - } - return size; - } - @java.lang.Override - public int getSerializedSize() { - int size = memoizedSize; - if (size != -1) return size; - - size = 0; - size += computeSerializedSize_0(); - size += getUnknownFields().getSerializedSize(); - memoizedSize = size; - return size; - } - - @java.lang.Override - public boolean equals(final java.lang.Object obj) { - if (obj == this) { - return true; - } - if (!(obj instanceof io.github.dfa1.vortex.proto.EncodingProtos.DecimalMetadata)) { - return super.equals(obj); - } - io.github.dfa1.vortex.proto.EncodingProtos.DecimalMetadata other = (io.github.dfa1.vortex.proto.EncodingProtos.DecimalMetadata) obj; - - if (getValuesType() - != other.getValuesType()) return false; - if (!getUnknownFields().equals(other.getUnknownFields())) return false; - return true; - } - - @java.lang.Override - public int hashCode() { - if (memoizedHashCode != 0) { - return memoizedHashCode; - } - int hash = 41; - hash = (19 * hash) + getDescriptor().hashCode(); - hash = (37 * hash) + VALUES_TYPE_FIELD_NUMBER; - hash = (53 * hash) + getValuesType(); - hash = (29 * hash) + getUnknownFields().hashCode(); - memoizedHashCode = hash; - return hash; - } - - public static io.github.dfa1.vortex.proto.EncodingProtos.DecimalMetadata parseFrom( - java.nio.ByteBuffer data) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.DecimalMetadata parseFrom( - java.nio.ByteBuffer data, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data, extensionRegistry); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.DecimalMetadata parseFrom( - com.google.protobuf.ByteString data) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.DecimalMetadata parseFrom( - com.google.protobuf.ByteString data, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data, extensionRegistry); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.DecimalMetadata parseFrom(byte[] data) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.DecimalMetadata parseFrom( - byte[] data, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data, extensionRegistry); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.DecimalMetadata parseFrom(java.io.InputStream input) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseWithIOException(PARSER, input); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.DecimalMetadata parseFrom( - java.io.InputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseWithIOException(PARSER, input, extensionRegistry); - } - - public static io.github.dfa1.vortex.proto.EncodingProtos.DecimalMetadata parseDelimitedFrom(java.io.InputStream input) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseDelimitedWithIOException(PARSER, input); - } - - public static io.github.dfa1.vortex.proto.EncodingProtos.DecimalMetadata parseDelimitedFrom( - java.io.InputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseDelimitedWithIOException(PARSER, input, extensionRegistry); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.DecimalMetadata parseFrom( - com.google.protobuf.CodedInputStream input) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseWithIOException(PARSER, input); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.DecimalMetadata parseFrom( - com.google.protobuf.CodedInputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseWithIOException(PARSER, input, extensionRegistry); - } - - @java.lang.Override - public Builder newBuilderForType() { return newBuilder(); } - public static Builder newBuilder() { - return DEFAULT_INSTANCE.toBuilder(); - } - public static Builder newBuilder(io.github.dfa1.vortex.proto.EncodingProtos.DecimalMetadata prototype) { - return DEFAULT_INSTANCE.toBuilder().mergeFrom(prototype); - } - @java.lang.Override - public Builder toBuilder() { - return this == DEFAULT_INSTANCE - ? new Builder() : new Builder().mergeFrom(this); - } - - @java.lang.Override - protected Builder newBuilderForType( - com.google.protobuf.GeneratedMessage.BuilderParent parent) { - Builder builder = new Builder(parent); - return builder; - } - /** - * Protobuf type {@code vortex.encodings.DecimalMetadata} - */ - public static final class Builder extends - com.google.protobuf.GeneratedMessage.Builder implements - // @@protoc_insertion_point(builder_implements:vortex.encodings.DecimalMetadata) - io.github.dfa1.vortex.proto.EncodingProtos.DecimalMetadataOrBuilder { - public static final com.google.protobuf.Descriptors.Descriptor - getDescriptor() { - return io.github.dfa1.vortex.proto.EncodingProtos.internal_static_vortex_encodings_DecimalMetadata_descriptor; - } - - @java.lang.Override - protected com.google.protobuf.GeneratedMessage.FieldAccessorTable - internalGetFieldAccessorTable() { - return io.github.dfa1.vortex.proto.EncodingProtos.internal_static_vortex_encodings_DecimalMetadata_fieldAccessorTable - .ensureFieldAccessorsInitialized( - io.github.dfa1.vortex.proto.EncodingProtos.DecimalMetadata.class, io.github.dfa1.vortex.proto.EncodingProtos.DecimalMetadata.Builder.class); - } - - // Construct using io.github.dfa1.vortex.proto.EncodingProtos.DecimalMetadata.newBuilder() - private Builder() { - - } - - private Builder( - com.google.protobuf.GeneratedMessage.BuilderParent parent) { - super(parent); - - } - @java.lang.Override - public Builder clear() { - super.clear(); - bitField0_ = 0; - valuesType_ = 0; - return this; - } - - @java.lang.Override - public com.google.protobuf.Descriptors.Descriptor - getDescriptorForType() { - return io.github.dfa1.vortex.proto.EncodingProtos.internal_static_vortex_encodings_DecimalMetadata_descriptor; - } - - @java.lang.Override - public io.github.dfa1.vortex.proto.EncodingProtos.DecimalMetadata getDefaultInstanceForType() { - return io.github.dfa1.vortex.proto.EncodingProtos.DecimalMetadata.getDefaultInstance(); - } - - @java.lang.Override - public io.github.dfa1.vortex.proto.EncodingProtos.DecimalMetadata build() { - io.github.dfa1.vortex.proto.EncodingProtos.DecimalMetadata result = buildPartial(); - if (!result.isInitialized()) { - throw newUninitializedMessageException(result); - } - return result; - } - - @java.lang.Override - public io.github.dfa1.vortex.proto.EncodingProtos.DecimalMetadata buildPartial() { - io.github.dfa1.vortex.proto.EncodingProtos.DecimalMetadata result = new io.github.dfa1.vortex.proto.EncodingProtos.DecimalMetadata(this); - if (bitField0_ != 0) { buildPartial0(result); } - onBuilt(); - return result; - } - - private void buildPartial0(io.github.dfa1.vortex.proto.EncodingProtos.DecimalMetadata result) { - int from_bitField0_ = bitField0_; - if (((from_bitField0_ & 0x00000001) != 0)) { - result.valuesType_ = valuesType_; - } - } - - @java.lang.Override - public Builder mergeFrom(com.google.protobuf.Message other) { - if (other instanceof io.github.dfa1.vortex.proto.EncodingProtos.DecimalMetadata) { - return mergeFrom((io.github.dfa1.vortex.proto.EncodingProtos.DecimalMetadata)other); - } else { - super.mergeFrom(other); - return this; - } - } - - public Builder mergeFrom(io.github.dfa1.vortex.proto.EncodingProtos.DecimalMetadata other) { - if (other == io.github.dfa1.vortex.proto.EncodingProtos.DecimalMetadata.getDefaultInstance()) return this; - if (other.getValuesType() != 0) { - setValuesType(other.getValuesType()); - } - this.mergeUnknownFields(other.getUnknownFields()); - onChanged(); - return this; - } - - @java.lang.Override - public final boolean isInitialized() { - return true; - } - - @java.lang.Override - public Builder mergeFrom( - com.google.protobuf.CodedInputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws java.io.IOException { - if (extensionRegistry == null) { - throw new java.lang.NullPointerException(); - } - try { - boolean done = false; - while (!done) { - int tag = input.readTag(); - switch (tag) { - case 0: - done = true; - break; - case 8: { - valuesType_ = input.readInt32(); - bitField0_ |= 0x00000001; - break; - } // case 8 - default: { - if (!super.parseUnknownField(input, extensionRegistry, tag)) { - done = true; // was an endgroup tag - } - break; - } // default: - } // switch (tag) - } // while (!done) - } catch (com.google.protobuf.InvalidProtocolBufferException e) { - throw e.unwrapIOException(); - } finally { - onChanged(); - } // finally - return this; - } - private int bitField0_; - - private int valuesType_ ; - /** - * 
-       * DecimalType: I8=0, I16=1, I32=2, I64=3, I128=4, I256=5
-       *
- * - * int32 values_type = 1; - * @return The valuesType. - */ - @java.lang.Override - public int getValuesType() { - return valuesType_; - } - /** - * 
-       * DecimalType: I8=0, I16=1, I32=2, I64=3, I128=4, I256=5
-       *
- * - * int32 values_type = 1; - * @param value The valuesType to set. - * @return This builder for chaining. - */ - public Builder setValuesType(int value) { - - valuesType_ = value; - bitField0_ |= 0x00000001; - onChanged(); - return this; - } - /** - * 
-       * DecimalType: I8=0, I16=1, I32=2, I64=3, I128=4, I256=5
-       *
- * - * int32 values_type = 1; - * @return This builder for chaining. - */ - public Builder clearValuesType() { - bitField0_ = (bitField0_ & ~0x00000001); - valuesType_ = 0; - onChanged(); - return this; - } - - // @@protoc_insertion_point(builder_scope:vortex.encodings.DecimalMetadata) - } - - // @@protoc_insertion_point(class_scope:vortex.encodings.DecimalMetadata) - private static final io.github.dfa1.vortex.proto.EncodingProtos.DecimalMetadata DEFAULT_INSTANCE; - static { - DEFAULT_INSTANCE = new io.github.dfa1.vortex.proto.EncodingProtos.DecimalMetadata(); - } - - public static io.github.dfa1.vortex.proto.EncodingProtos.DecimalMetadata getDefaultInstance() { - return DEFAULT_INSTANCE; - } - - private static final com.google.protobuf.Parser - PARSER = new com.google.protobuf.AbstractParser() { - @java.lang.Override - public DecimalMetadata parsePartialFrom( - com.google.protobuf.CodedInputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws com.google.protobuf.InvalidProtocolBufferException { - Builder builder = newBuilder(); - try { - builder.mergeFrom(input, extensionRegistry); - } catch (com.google.protobuf.InvalidProtocolBufferException e) { - throw e.setUnfinishedMessage(builder.buildPartial()); - } catch (com.google.protobuf.UninitializedMessageException e) { - throw e.asInvalidProtocolBufferException().setUnfinishedMessage(builder.buildPartial()); - } catch (java.io.IOException e) { - throw new com.google.protobuf.InvalidProtocolBufferException(e) - .setUnfinishedMessage(builder.buildPartial()); - } - return builder.buildPartial(); - } - }; - - public static com.google.protobuf.Parser parser() { - return PARSER; - } - - @java.lang.Override - public com.google.protobuf.Parser getParserForType() { - return PARSER; - } - - @java.lang.Override - public io.github.dfa1.vortex.proto.EncodingProtos.DecimalMetadata getDefaultInstanceForType() { - return DEFAULT_INSTANCE; - } - - } - - public interface DecimalBytePartsMetadataOrBuilder extends - // @@protoc_insertion_point(interface_extends:vortex.encodings.DecimalBytePartsMetadata) - com.google.protobuf.MessageOrBuilder { - - /** - * .vortex.dtype.PType zeroth_child_ptype = 1; - * @return The enum numeric value on the wire for zerothChildPtype. - */ - int getZerothChildPtypeValue(); - /** - * .vortex.dtype.PType zeroth_child_ptype = 1; - * @return The zerothChildPtype. - */ - io.github.dfa1.vortex.proto.DTypeProtos.PType getZerothChildPtype(); - - /** - * uint32 lower_part_count = 2; - * @return The lowerPartCount. - */ - int getLowerPartCount(); - } - /** - * Protobuf type {@code vortex.encodings.DecimalBytePartsMetadata} - */ - public static final class DecimalBytePartsMetadata extends - com.google.protobuf.GeneratedMessage implements - // @@protoc_insertion_point(message_implements:vortex.encodings.DecimalBytePartsMetadata) - DecimalBytePartsMetadataOrBuilder { - private static final long serialVersionUID = 0L; - static { - com.google.protobuf.RuntimeVersion.validateProtobufGencodeVersion( - com.google.protobuf.RuntimeVersion.RuntimeDomain.PUBLIC, - /* major= */ 4, - /* minor= */ 35, - /* patch= */ 0, - /* suffix= */ "", - "DecimalBytePartsMetadata"); - } - // Use DecimalBytePartsMetadata.newBuilder() to construct. - private DecimalBytePartsMetadata(com.google.protobuf.GeneratedMessage.Builder builder) { - super(builder); - } - private DecimalBytePartsMetadata() { - zerothChildPtype_ = 0; - } - - public static final com.google.protobuf.Descriptors.Descriptor - getDescriptor() { - return io.github.dfa1.vortex.proto.EncodingProtos.internal_static_vortex_encodings_DecimalBytePartsMetadata_descriptor; - } - - @java.lang.Override - public com.google.protobuf.Descriptors.Descriptor getDescriptorForType() { - return io.github.dfa1.vortex.proto.EncodingProtos.internal_static_vortex_encodings_DecimalBytePartsMetadata_descriptor; - } - - @java.lang.Override - protected com.google.protobuf.GeneratedMessage.FieldAccessorTable - internalGetFieldAccessorTable() { - return io.github.dfa1.vortex.proto.EncodingProtos.internal_static_vortex_encodings_DecimalBytePartsMetadata_fieldAccessorTable - .ensureFieldAccessorsInitialized( - io.github.dfa1.vortex.proto.EncodingProtos.DecimalBytePartsMetadata.class, io.github.dfa1.vortex.proto.EncodingProtos.DecimalBytePartsMetadata.Builder.class); - } - - public static final int ZEROTH_CHILD_PTYPE_FIELD_NUMBER = 1; - private int zerothChildPtype_ = 0; - /** - * .vortex.dtype.PType zeroth_child_ptype = 1; - * @return The enum numeric value on the wire for zerothChildPtype. - */ - @java.lang.Override public int getZerothChildPtypeValue() { - return zerothChildPtype_; - } - /** - * .vortex.dtype.PType zeroth_child_ptype = 1; - * @return The zerothChildPtype. - */ - @java.lang.Override public io.github.dfa1.vortex.proto.DTypeProtos.PType getZerothChildPtype() { - io.github.dfa1.vortex.proto.DTypeProtos.PType result = io.github.dfa1.vortex.proto.DTypeProtos.PType.forNumber(zerothChildPtype_); - return result == null ? io.github.dfa1.vortex.proto.DTypeProtos.PType.UNRECOGNIZED : result; - } - - public static final int LOWER_PART_COUNT_FIELD_NUMBER = 2; - private int lowerPartCount_ = 0; - /** - * uint32 lower_part_count = 2; - * @return The lowerPartCount. - */ - @java.lang.Override - public int getLowerPartCount() { - return lowerPartCount_; - } - - private byte memoizedIsInitialized = -1; - @java.lang.Override - public final boolean isInitialized() { - byte isInitialized = memoizedIsInitialized; - if (isInitialized == 1) return true; - if (isInitialized == 0) return false; - - memoizedIsInitialized = 1; - return true; - } - - @java.lang.Override - public void writeTo(com.google.protobuf.CodedOutputStream output) - throws java.io.IOException { - if (zerothChildPtype_ != io.github.dfa1.vortex.proto.DTypeProtos.PType.U8.getNumber()) { - output.writeEnum(1, zerothChildPtype_); - } - if (lowerPartCount_ != 0) { - output.writeUInt32(2, lowerPartCount_); - } - getUnknownFields().writeTo(output); - } - private int computeSerializedSize_0() { - int size = 0; - if (zerothChildPtype_ != io.github.dfa1.vortex.proto.DTypeProtos.PType.U8.getNumber()) { - size += com.google.protobuf.CodedOutputStream - .computeEnumSize(1, zerothChildPtype_); - } - if (lowerPartCount_ != 0) { - size += com.google.protobuf.CodedOutputStream - .computeUInt32Size(2, lowerPartCount_); - } - return size; - } - @java.lang.Override - public int getSerializedSize() { - int size = memoizedSize; - if (size != -1) return size; - - size = 0; - size += computeSerializedSize_0(); - size += getUnknownFields().getSerializedSize(); - memoizedSize = size; - return size; - } - - @java.lang.Override - public boolean equals(final java.lang.Object obj) { - if (obj == this) { - return true; - } - if (!(obj instanceof io.github.dfa1.vortex.proto.EncodingProtos.DecimalBytePartsMetadata)) { - return super.equals(obj); - } - io.github.dfa1.vortex.proto.EncodingProtos.DecimalBytePartsMetadata other = (io.github.dfa1.vortex.proto.EncodingProtos.DecimalBytePartsMetadata) obj; - - if (zerothChildPtype_ != other.zerothChildPtype_) return false; - if (getLowerPartCount() - != other.getLowerPartCount()) return false; - if (!getUnknownFields().equals(other.getUnknownFields())) return false; - return true; - } - - @java.lang.Override - public int hashCode() { - if (memoizedHashCode != 0) { - return memoizedHashCode; - } - int hash = 41; - hash = (19 * hash) + getDescriptor().hashCode(); - hash = (37 * hash) + ZEROTH_CHILD_PTYPE_FIELD_NUMBER; - hash = (53 * hash) + zerothChildPtype_; - hash = (37 * hash) + LOWER_PART_COUNT_FIELD_NUMBER; - hash = (53 * hash) + getLowerPartCount(); - hash = (29 * hash) + getUnknownFields().hashCode(); - memoizedHashCode = hash; - return hash; - } - - public static io.github.dfa1.vortex.proto.EncodingProtos.DecimalBytePartsMetadata parseFrom( - java.nio.ByteBuffer data) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.DecimalBytePartsMetadata parseFrom( - java.nio.ByteBuffer data, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data, extensionRegistry); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.DecimalBytePartsMetadata parseFrom( - com.google.protobuf.ByteString data) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.DecimalBytePartsMetadata parseFrom( - com.google.protobuf.ByteString data, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data, extensionRegistry); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.DecimalBytePartsMetadata parseFrom(byte[] data) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.DecimalBytePartsMetadata parseFrom( - byte[] data, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data, extensionRegistry); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.DecimalBytePartsMetadata parseFrom(java.io.InputStream input) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseWithIOException(PARSER, input); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.DecimalBytePartsMetadata parseFrom( - java.io.InputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseWithIOException(PARSER, input, extensionRegistry); - } - - public static io.github.dfa1.vortex.proto.EncodingProtos.DecimalBytePartsMetadata parseDelimitedFrom(java.io.InputStream input) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseDelimitedWithIOException(PARSER, input); - } - - public static io.github.dfa1.vortex.proto.EncodingProtos.DecimalBytePartsMetadata parseDelimitedFrom( - java.io.InputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseDelimitedWithIOException(PARSER, input, extensionRegistry); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.DecimalBytePartsMetadata parseFrom( - com.google.protobuf.CodedInputStream input) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseWithIOException(PARSER, input); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.DecimalBytePartsMetadata parseFrom( - com.google.protobuf.CodedInputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseWithIOException(PARSER, input, extensionRegistry); - } - - @java.lang.Override - public Builder newBuilderForType() { return newBuilder(); } - public static Builder newBuilder() { - return DEFAULT_INSTANCE.toBuilder(); - } - public static Builder newBuilder(io.github.dfa1.vortex.proto.EncodingProtos.DecimalBytePartsMetadata prototype) { - return DEFAULT_INSTANCE.toBuilder().mergeFrom(prototype); - } - @java.lang.Override - public Builder toBuilder() { - return this == DEFAULT_INSTANCE - ? new Builder() : new Builder().mergeFrom(this); - } - - @java.lang.Override - protected Builder newBuilderForType( - com.google.protobuf.GeneratedMessage.BuilderParent parent) { - Builder builder = new Builder(parent); - return builder; - } - /** - * Protobuf type {@code vortex.encodings.DecimalBytePartsMetadata} - */ - public static final class Builder extends - com.google.protobuf.GeneratedMessage.Builder implements - // @@protoc_insertion_point(builder_implements:vortex.encodings.DecimalBytePartsMetadata) - io.github.dfa1.vortex.proto.EncodingProtos.DecimalBytePartsMetadataOrBuilder { - public static final com.google.protobuf.Descriptors.Descriptor - getDescriptor() { - return io.github.dfa1.vortex.proto.EncodingProtos.internal_static_vortex_encodings_DecimalBytePartsMetadata_descriptor; - } - - @java.lang.Override - protected com.google.protobuf.GeneratedMessage.FieldAccessorTable - internalGetFieldAccessorTable() { - return io.github.dfa1.vortex.proto.EncodingProtos.internal_static_vortex_encodings_DecimalBytePartsMetadata_fieldAccessorTable - .ensureFieldAccessorsInitialized( - io.github.dfa1.vortex.proto.EncodingProtos.DecimalBytePartsMetadata.class, io.github.dfa1.vortex.proto.EncodingProtos.DecimalBytePartsMetadata.Builder.class); - } - - // Construct using io.github.dfa1.vortex.proto.EncodingProtos.DecimalBytePartsMetadata.newBuilder() - private Builder() { - - } - - private Builder( - com.google.protobuf.GeneratedMessage.BuilderParent parent) { - super(parent); - - } - @java.lang.Override - public Builder clear() { - super.clear(); - bitField0_ = 0; - zerothChildPtype_ = 0; - lowerPartCount_ = 0; - return this; - } - - @java.lang.Override - public com.google.protobuf.Descriptors.Descriptor - getDescriptorForType() { - return io.github.dfa1.vortex.proto.EncodingProtos.internal_static_vortex_encodings_DecimalBytePartsMetadata_descriptor; - } - - @java.lang.Override - public io.github.dfa1.vortex.proto.EncodingProtos.DecimalBytePartsMetadata getDefaultInstanceForType() { - return io.github.dfa1.vortex.proto.EncodingProtos.DecimalBytePartsMetadata.getDefaultInstance(); - } - - @java.lang.Override - public io.github.dfa1.vortex.proto.EncodingProtos.DecimalBytePartsMetadata build() { - io.github.dfa1.vortex.proto.EncodingProtos.DecimalBytePartsMetadata result = buildPartial(); - if (!result.isInitialized()) { - throw newUninitializedMessageException(result); - } - return result; - } - - @java.lang.Override - public io.github.dfa1.vortex.proto.EncodingProtos.DecimalBytePartsMetadata buildPartial() { - io.github.dfa1.vortex.proto.EncodingProtos.DecimalBytePartsMetadata result = new io.github.dfa1.vortex.proto.EncodingProtos.DecimalBytePartsMetadata(this); - if (bitField0_ != 0) { buildPartial0(result); } - onBuilt(); - return result; - } - - private void buildPartial0(io.github.dfa1.vortex.proto.EncodingProtos.DecimalBytePartsMetadata result) { - int from_bitField0_ = bitField0_; - if (((from_bitField0_ & 0x00000001) != 0)) { - result.zerothChildPtype_ = zerothChildPtype_; - } - if (((from_bitField0_ & 0x00000002) != 0)) { - result.lowerPartCount_ = lowerPartCount_; - } - } - - @java.lang.Override - public Builder mergeFrom(com.google.protobuf.Message other) { - if (other instanceof io.github.dfa1.vortex.proto.EncodingProtos.DecimalBytePartsMetadata) { - return mergeFrom((io.github.dfa1.vortex.proto.EncodingProtos.DecimalBytePartsMetadata)other); - } else { - super.mergeFrom(other); - return this; - } - } - - public Builder mergeFrom(io.github.dfa1.vortex.proto.EncodingProtos.DecimalBytePartsMetadata other) { - if (other == io.github.dfa1.vortex.proto.EncodingProtos.DecimalBytePartsMetadata.getDefaultInstance()) return this; - if (other.zerothChildPtype_ != 0) { - setZerothChildPtypeValue(other.getZerothChildPtypeValue()); - } - if (other.getLowerPartCount() != 0) { - setLowerPartCount(other.getLowerPartCount()); - } - this.mergeUnknownFields(other.getUnknownFields()); - onChanged(); - return this; - } - - @java.lang.Override - public final boolean isInitialized() { - return true; - } - - @java.lang.Override - public Builder mergeFrom( - com.google.protobuf.CodedInputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws java.io.IOException { - if (extensionRegistry == null) { - throw new java.lang.NullPointerException(); - } - try { - boolean done = false; - while (!done) { - int tag = input.readTag(); - switch (tag) { - case 0: - done = true; - break; - case 8: { - zerothChildPtype_ = input.readEnum(); - bitField0_ |= 0x00000001; - break; - } // case 8 - case 16: { - lowerPartCount_ = input.readUInt32(); - bitField0_ |= 0x00000002; - break; - } // case 16 - default: { - if (!super.parseUnknownField(input, extensionRegistry, tag)) { - done = true; // was an endgroup tag - } - break; - } // default: - } // switch (tag) - } // while (!done) - } catch (com.google.protobuf.InvalidProtocolBufferException e) { - throw e.unwrapIOException(); - } finally { - onChanged(); - } // finally - return this; - } - private int bitField0_; - - private int zerothChildPtype_ = 0; - /** - * .vortex.dtype.PType zeroth_child_ptype = 1; - * @return The enum numeric value on the wire for zerothChildPtype. - */ - @java.lang.Override public int getZerothChildPtypeValue() { - return zerothChildPtype_; - } - /** - * .vortex.dtype.PType zeroth_child_ptype = 1; - * @param value The enum numeric value on the wire for zerothChildPtype to set. - * @return This builder for chaining. - */ - public Builder setZerothChildPtypeValue(int value) { - zerothChildPtype_ = value; - bitField0_ |= 0x00000001; - onChanged(); - return this; - } - /** - * .vortex.dtype.PType zeroth_child_ptype = 1; - * @return The zerothChildPtype. - */ - @java.lang.Override - public io.github.dfa1.vortex.proto.DTypeProtos.PType getZerothChildPtype() { - io.github.dfa1.vortex.proto.DTypeProtos.PType result = io.github.dfa1.vortex.proto.DTypeProtos.PType.forNumber(zerothChildPtype_); - return result == null ? io.github.dfa1.vortex.proto.DTypeProtos.PType.UNRECOGNIZED : result; - } - /** - * .vortex.dtype.PType zeroth_child_ptype = 1; - * @param value The zerothChildPtype to set. - * @throws IllegalArgumentException if UNRECOGNIZED is provided. - * @return This builder for chaining. - */ - public Builder setZerothChildPtype(io.github.dfa1.vortex.proto.DTypeProtos.PType value) { - if (value == null) { throw new NullPointerException(); } - bitField0_ |= 0x00000001; - zerothChildPtype_ = value.getNumber(); - onChanged(); - return this; - } - /** - * .vortex.dtype.PType zeroth_child_ptype = 1; - * @return This builder for chaining. - */ - public Builder clearZerothChildPtype() { - bitField0_ = (bitField0_ & ~0x00000001); - zerothChildPtype_ = 0; - onChanged(); - return this; - } - - private int lowerPartCount_ ; - /** - * uint32 lower_part_count = 2; - * @return The lowerPartCount. - */ - @java.lang.Override - public int getLowerPartCount() { - return lowerPartCount_; - } - /** - * uint32 lower_part_count = 2; - * @param value The lowerPartCount to set. - * @return This builder for chaining. - */ - public Builder setLowerPartCount(int value) { - - lowerPartCount_ = value; - bitField0_ |= 0x00000002; - onChanged(); - return this; - } - /** - * uint32 lower_part_count = 2; - * @return This builder for chaining. - */ - public Builder clearLowerPartCount() { - bitField0_ = (bitField0_ & ~0x00000002); - lowerPartCount_ = 0; - onChanged(); - return this; - } - - // @@protoc_insertion_point(builder_scope:vortex.encodings.DecimalBytePartsMetadata) - } - - // @@protoc_insertion_point(class_scope:vortex.encodings.DecimalBytePartsMetadata) - private static final io.github.dfa1.vortex.proto.EncodingProtos.DecimalBytePartsMetadata DEFAULT_INSTANCE; - static { - DEFAULT_INSTANCE = new io.github.dfa1.vortex.proto.EncodingProtos.DecimalBytePartsMetadata(); - } - - public static io.github.dfa1.vortex.proto.EncodingProtos.DecimalBytePartsMetadata getDefaultInstance() { - return DEFAULT_INSTANCE; - } - - private static final com.google.protobuf.Parser - PARSER = new com.google.protobuf.AbstractParser() { - @java.lang.Override - public DecimalBytePartsMetadata parsePartialFrom( - com.google.protobuf.CodedInputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws com.google.protobuf.InvalidProtocolBufferException { - Builder builder = newBuilder(); - try { - builder.mergeFrom(input, extensionRegistry); - } catch (com.google.protobuf.InvalidProtocolBufferException e) { - throw e.setUnfinishedMessage(builder.buildPartial()); - } catch (com.google.protobuf.UninitializedMessageException e) { - throw e.asInvalidProtocolBufferException().setUnfinishedMessage(builder.buildPartial()); - } catch (java.io.IOException e) { - throw new com.google.protobuf.InvalidProtocolBufferException(e) - .setUnfinishedMessage(builder.buildPartial()); - } - return builder.buildPartial(); - } - }; - - public static com.google.protobuf.Parser parser() { - return PARSER; - } - - @java.lang.Override - public com.google.protobuf.Parser getParserForType() { - return PARSER; - } - - @java.lang.Override - public io.github.dfa1.vortex.proto.EncodingProtos.DecimalBytePartsMetadata getDefaultInstanceForType() { - return DEFAULT_INSTANCE; - } - - } - - public interface DateTimePartsMetadataOrBuilder extends - // @@protoc_insertion_point(interface_extends:vortex.encodings.DateTimePartsMetadata) - com.google.protobuf.MessageOrBuilder { - - /** - * .vortex.dtype.PType days_ptype = 1; - * @return The enum numeric value on the wire for daysPtype. - */ - int getDaysPtypeValue(); - /** - * .vortex.dtype.PType days_ptype = 1; - * @return The daysPtype. - */ - io.github.dfa1.vortex.proto.DTypeProtos.PType getDaysPtype(); - - /** - * .vortex.dtype.PType seconds_ptype = 2; - * @return The enum numeric value on the wire for secondsPtype. - */ - int getSecondsPtypeValue(); - /** - * .vortex.dtype.PType seconds_ptype = 2; - * @return The secondsPtype. - */ - io.github.dfa1.vortex.proto.DTypeProtos.PType getSecondsPtype(); - - /** - * .vortex.dtype.PType subseconds_ptype = 3; - * @return The enum numeric value on the wire for subsecondsPtype. - */ - int getSubsecondsPtypeValue(); - /** - * .vortex.dtype.PType subseconds_ptype = 3; - * @return The subsecondsPtype. - */ - io.github.dfa1.vortex.proto.DTypeProtos.PType getSubsecondsPtype(); - } - /** - * Protobuf type {@code vortex.encodings.DateTimePartsMetadata} - */ - public static final class DateTimePartsMetadata extends - com.google.protobuf.GeneratedMessage implements - // @@protoc_insertion_point(message_implements:vortex.encodings.DateTimePartsMetadata) - DateTimePartsMetadataOrBuilder { - private static final long serialVersionUID = 0L; - static { - com.google.protobuf.RuntimeVersion.validateProtobufGencodeVersion( - com.google.protobuf.RuntimeVersion.RuntimeDomain.PUBLIC, - /* major= */ 4, - /* minor= */ 35, - /* patch= */ 0, - /* suffix= */ "", - "DateTimePartsMetadata"); - } - // Use DateTimePartsMetadata.newBuilder() to construct. - private DateTimePartsMetadata(com.google.protobuf.GeneratedMessage.Builder builder) { - super(builder); - } - private DateTimePartsMetadata() { - daysPtype_ = 0; - secondsPtype_ = 0; - subsecondsPtype_ = 0; - } - - public static final com.google.protobuf.Descriptors.Descriptor - getDescriptor() { - return io.github.dfa1.vortex.proto.EncodingProtos.internal_static_vortex_encodings_DateTimePartsMetadata_descriptor; - } - - @java.lang.Override - public com.google.protobuf.Descriptors.Descriptor getDescriptorForType() { - return io.github.dfa1.vortex.proto.EncodingProtos.internal_static_vortex_encodings_DateTimePartsMetadata_descriptor; - } - - @java.lang.Override - protected com.google.protobuf.GeneratedMessage.FieldAccessorTable - internalGetFieldAccessorTable() { - return io.github.dfa1.vortex.proto.EncodingProtos.internal_static_vortex_encodings_DateTimePartsMetadata_fieldAccessorTable - .ensureFieldAccessorsInitialized( - io.github.dfa1.vortex.proto.EncodingProtos.DateTimePartsMetadata.class, io.github.dfa1.vortex.proto.EncodingProtos.DateTimePartsMetadata.Builder.class); - } - - public static final int DAYS_PTYPE_FIELD_NUMBER = 1; - private int daysPtype_ = 0; - /** - * .vortex.dtype.PType days_ptype = 1; - * @return The enum numeric value on the wire for daysPtype. - */ - @java.lang.Override public int getDaysPtypeValue() { - return daysPtype_; - } - /** - * .vortex.dtype.PType days_ptype = 1; - * @return The daysPtype. - */ - @java.lang.Override public io.github.dfa1.vortex.proto.DTypeProtos.PType getDaysPtype() { - io.github.dfa1.vortex.proto.DTypeProtos.PType result = io.github.dfa1.vortex.proto.DTypeProtos.PType.forNumber(daysPtype_); - return result == null ? io.github.dfa1.vortex.proto.DTypeProtos.PType.UNRECOGNIZED : result; - } - - public static final int SECONDS_PTYPE_FIELD_NUMBER = 2; - private int secondsPtype_ = 0; - /** - * .vortex.dtype.PType seconds_ptype = 2; - * @return The enum numeric value on the wire for secondsPtype. - */ - @java.lang.Override public int getSecondsPtypeValue() { - return secondsPtype_; - } - /** - * .vortex.dtype.PType seconds_ptype = 2; - * @return The secondsPtype. - */ - @java.lang.Override public io.github.dfa1.vortex.proto.DTypeProtos.PType getSecondsPtype() { - io.github.dfa1.vortex.proto.DTypeProtos.PType result = io.github.dfa1.vortex.proto.DTypeProtos.PType.forNumber(secondsPtype_); - return result == null ? io.github.dfa1.vortex.proto.DTypeProtos.PType.UNRECOGNIZED : result; - } - - public static final int SUBSECONDS_PTYPE_FIELD_NUMBER = 3; - private int subsecondsPtype_ = 0; - /** - * .vortex.dtype.PType subseconds_ptype = 3; - * @return The enum numeric value on the wire for subsecondsPtype. - */ - @java.lang.Override public int getSubsecondsPtypeValue() { - return subsecondsPtype_; - } - /** - * .vortex.dtype.PType subseconds_ptype = 3; - * @return The subsecondsPtype. - */ - @java.lang.Override public io.github.dfa1.vortex.proto.DTypeProtos.PType getSubsecondsPtype() { - io.github.dfa1.vortex.proto.DTypeProtos.PType result = io.github.dfa1.vortex.proto.DTypeProtos.PType.forNumber(subsecondsPtype_); - return result == null ? io.github.dfa1.vortex.proto.DTypeProtos.PType.UNRECOGNIZED : result; - } - - private byte memoizedIsInitialized = -1; - @java.lang.Override - public final boolean isInitialized() { - byte isInitialized = memoizedIsInitialized; - if (isInitialized == 1) return true; - if (isInitialized == 0) return false; - - memoizedIsInitialized = 1; - return true; - } - - @java.lang.Override - public void writeTo(com.google.protobuf.CodedOutputStream output) - throws java.io.IOException { - if (daysPtype_ != io.github.dfa1.vortex.proto.DTypeProtos.PType.U8.getNumber()) { - output.writeEnum(1, daysPtype_); - } - if (secondsPtype_ != io.github.dfa1.vortex.proto.DTypeProtos.PType.U8.getNumber()) { - output.writeEnum(2, secondsPtype_); - } - if (subsecondsPtype_ != io.github.dfa1.vortex.proto.DTypeProtos.PType.U8.getNumber()) { - output.writeEnum(3, subsecondsPtype_); - } - getUnknownFields().writeTo(output); - } - private int computeSerializedSize_0() { - int size = 0; - if (daysPtype_ != io.github.dfa1.vortex.proto.DTypeProtos.PType.U8.getNumber()) { - size += com.google.protobuf.CodedOutputStream - .computeEnumSize(1, daysPtype_); - } - if (secondsPtype_ != io.github.dfa1.vortex.proto.DTypeProtos.PType.U8.getNumber()) { - size += com.google.protobuf.CodedOutputStream - .computeEnumSize(2, secondsPtype_); - } - if (subsecondsPtype_ != io.github.dfa1.vortex.proto.DTypeProtos.PType.U8.getNumber()) { - size += com.google.protobuf.CodedOutputStream - .computeEnumSize(3, subsecondsPtype_); - } - return size; - } - @java.lang.Override - public int getSerializedSize() { - int size = memoizedSize; - if (size != -1) return size; - - size = 0; - size += computeSerializedSize_0(); - size += getUnknownFields().getSerializedSize(); - memoizedSize = size; - return size; - } - - @java.lang.Override - public boolean equals(final java.lang.Object obj) { - if (obj == this) { - return true; - } - if (!(obj instanceof io.github.dfa1.vortex.proto.EncodingProtos.DateTimePartsMetadata)) { - return super.equals(obj); - } - io.github.dfa1.vortex.proto.EncodingProtos.DateTimePartsMetadata other = (io.github.dfa1.vortex.proto.EncodingProtos.DateTimePartsMetadata) obj; - - if (daysPtype_ != other.daysPtype_) return false; - if (secondsPtype_ != other.secondsPtype_) return false; - if (subsecondsPtype_ != other.subsecondsPtype_) return false; - if (!getUnknownFields().equals(other.getUnknownFields())) return false; - return true; - } - - @java.lang.Override - public int hashCode() { - if (memoizedHashCode != 0) { - return memoizedHashCode; - } - int hash = 41; - hash = (19 * hash) + getDescriptor().hashCode(); - hash = (37 * hash) + DAYS_PTYPE_FIELD_NUMBER; - hash = (53 * hash) + daysPtype_; - hash = (37 * hash) + SECONDS_PTYPE_FIELD_NUMBER; - hash = (53 * hash) + secondsPtype_; - hash = (37 * hash) + SUBSECONDS_PTYPE_FIELD_NUMBER; - hash = (53 * hash) + subsecondsPtype_; - hash = (29 * hash) + getUnknownFields().hashCode(); - memoizedHashCode = hash; - return hash; - } - - public static io.github.dfa1.vortex.proto.EncodingProtos.DateTimePartsMetadata parseFrom( - java.nio.ByteBuffer data) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.DateTimePartsMetadata parseFrom( - java.nio.ByteBuffer data, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data, extensionRegistry); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.DateTimePartsMetadata parseFrom( - com.google.protobuf.ByteString data) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.DateTimePartsMetadata parseFrom( - com.google.protobuf.ByteString data, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data, extensionRegistry); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.DateTimePartsMetadata parseFrom(byte[] data) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.DateTimePartsMetadata parseFrom( - byte[] data, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data, extensionRegistry); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.DateTimePartsMetadata parseFrom(java.io.InputStream input) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseWithIOException(PARSER, input); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.DateTimePartsMetadata parseFrom( - java.io.InputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseWithIOException(PARSER, input, extensionRegistry); - } - - public static io.github.dfa1.vortex.proto.EncodingProtos.DateTimePartsMetadata parseDelimitedFrom(java.io.InputStream input) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseDelimitedWithIOException(PARSER, input); - } - - public static io.github.dfa1.vortex.proto.EncodingProtos.DateTimePartsMetadata parseDelimitedFrom( - java.io.InputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseDelimitedWithIOException(PARSER, input, extensionRegistry); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.DateTimePartsMetadata parseFrom( - com.google.protobuf.CodedInputStream input) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseWithIOException(PARSER, input); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.DateTimePartsMetadata parseFrom( - com.google.protobuf.CodedInputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseWithIOException(PARSER, input, extensionRegistry); - } - - @java.lang.Override - public Builder newBuilderForType() { return newBuilder(); } - public static Builder newBuilder() { - return DEFAULT_INSTANCE.toBuilder(); - } - public static Builder newBuilder(io.github.dfa1.vortex.proto.EncodingProtos.DateTimePartsMetadata prototype) { - return DEFAULT_INSTANCE.toBuilder().mergeFrom(prototype); - } - @java.lang.Override - public Builder toBuilder() { - return this == DEFAULT_INSTANCE - ? new Builder() : new Builder().mergeFrom(this); - } - - @java.lang.Override - protected Builder newBuilderForType( - com.google.protobuf.GeneratedMessage.BuilderParent parent) { - Builder builder = new Builder(parent); - return builder; - } - /** - * Protobuf type {@code vortex.encodings.DateTimePartsMetadata} - */ - public static final class Builder extends - com.google.protobuf.GeneratedMessage.Builder implements - // @@protoc_insertion_point(builder_implements:vortex.encodings.DateTimePartsMetadata) - io.github.dfa1.vortex.proto.EncodingProtos.DateTimePartsMetadataOrBuilder { - public static final com.google.protobuf.Descriptors.Descriptor - getDescriptor() { - return io.github.dfa1.vortex.proto.EncodingProtos.internal_static_vortex_encodings_DateTimePartsMetadata_descriptor; - } - - @java.lang.Override - protected com.google.protobuf.GeneratedMessage.FieldAccessorTable - internalGetFieldAccessorTable() { - return io.github.dfa1.vortex.proto.EncodingProtos.internal_static_vortex_encodings_DateTimePartsMetadata_fieldAccessorTable - .ensureFieldAccessorsInitialized( - io.github.dfa1.vortex.proto.EncodingProtos.DateTimePartsMetadata.class, io.github.dfa1.vortex.proto.EncodingProtos.DateTimePartsMetadata.Builder.class); - } - - // Construct using io.github.dfa1.vortex.proto.EncodingProtos.DateTimePartsMetadata.newBuilder() - private Builder() { - - } - - private Builder( - com.google.protobuf.GeneratedMessage.BuilderParent parent) { - super(parent); - - } - @java.lang.Override - public Builder clear() { - super.clear(); - bitField0_ = 0; - daysPtype_ = 0; - secondsPtype_ = 0; - subsecondsPtype_ = 0; - return this; - } - - @java.lang.Override - public com.google.protobuf.Descriptors.Descriptor - getDescriptorForType() { - return io.github.dfa1.vortex.proto.EncodingProtos.internal_static_vortex_encodings_DateTimePartsMetadata_descriptor; - } - - @java.lang.Override - public io.github.dfa1.vortex.proto.EncodingProtos.DateTimePartsMetadata getDefaultInstanceForType() { - return io.github.dfa1.vortex.proto.EncodingProtos.DateTimePartsMetadata.getDefaultInstance(); - } - - @java.lang.Override - public io.github.dfa1.vortex.proto.EncodingProtos.DateTimePartsMetadata build() { - io.github.dfa1.vortex.proto.EncodingProtos.DateTimePartsMetadata result = buildPartial(); - if (!result.isInitialized()) { - throw newUninitializedMessageException(result); - } - return result; - } - - @java.lang.Override - public io.github.dfa1.vortex.proto.EncodingProtos.DateTimePartsMetadata buildPartial() { - io.github.dfa1.vortex.proto.EncodingProtos.DateTimePartsMetadata result = new io.github.dfa1.vortex.proto.EncodingProtos.DateTimePartsMetadata(this); - if (bitField0_ != 0) { buildPartial0(result); } - onBuilt(); - return result; - } - - private void buildPartial0(io.github.dfa1.vortex.proto.EncodingProtos.DateTimePartsMetadata result) { - int from_bitField0_ = bitField0_; - if (((from_bitField0_ & 0x00000001) != 0)) { - result.daysPtype_ = daysPtype_; - } - if (((from_bitField0_ & 0x00000002) != 0)) { - result.secondsPtype_ = secondsPtype_; - } - if (((from_bitField0_ & 0x00000004) != 0)) { - result.subsecondsPtype_ = subsecondsPtype_; - } - } - - @java.lang.Override - public Builder mergeFrom(com.google.protobuf.Message other) { - if (other instanceof io.github.dfa1.vortex.proto.EncodingProtos.DateTimePartsMetadata) { - return mergeFrom((io.github.dfa1.vortex.proto.EncodingProtos.DateTimePartsMetadata)other); - } else { - super.mergeFrom(other); - return this; - } - } - - public Builder mergeFrom(io.github.dfa1.vortex.proto.EncodingProtos.DateTimePartsMetadata other) { - if (other == io.github.dfa1.vortex.proto.EncodingProtos.DateTimePartsMetadata.getDefaultInstance()) return this; - if (other.daysPtype_ != 0) { - setDaysPtypeValue(other.getDaysPtypeValue()); - } - if (other.secondsPtype_ != 0) { - setSecondsPtypeValue(other.getSecondsPtypeValue()); - } - if (other.subsecondsPtype_ != 0) { - setSubsecondsPtypeValue(other.getSubsecondsPtypeValue()); - } - this.mergeUnknownFields(other.getUnknownFields()); - onChanged(); - return this; - } - - @java.lang.Override - public final boolean isInitialized() { - return true; - } - - @java.lang.Override - public Builder mergeFrom( - com.google.protobuf.CodedInputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws java.io.IOException { - if (extensionRegistry == null) { - throw new java.lang.NullPointerException(); - } - try { - boolean done = false; - while (!done) { - int tag = input.readTag(); - switch (tag) { - case 0: - done = true; - break; - case 8: { - daysPtype_ = input.readEnum(); - bitField0_ |= 0x00000001; - break; - } // case 8 - case 16: { - secondsPtype_ = input.readEnum(); - bitField0_ |= 0x00000002; - break; - } // case 16 - case 24: { - subsecondsPtype_ = input.readEnum(); - bitField0_ |= 0x00000004; - break; - } // case 24 - default: { - if (!super.parseUnknownField(input, extensionRegistry, tag)) { - done = true; // was an endgroup tag - } - break; - } // default: - } // switch (tag) - } // while (!done) - } catch (com.google.protobuf.InvalidProtocolBufferException e) { - throw e.unwrapIOException(); - } finally { - onChanged(); - } // finally - return this; - } - private int bitField0_; - - private int daysPtype_ = 0; - /** - * .vortex.dtype.PType days_ptype = 1; - * @return The enum numeric value on the wire for daysPtype. - */ - @java.lang.Override public int getDaysPtypeValue() { - return daysPtype_; - } - /** - * .vortex.dtype.PType days_ptype = 1; - * @param value The enum numeric value on the wire for daysPtype to set. - * @return This builder for chaining. - */ - public Builder setDaysPtypeValue(int value) { - daysPtype_ = value; - bitField0_ |= 0x00000001; - onChanged(); - return this; - } - /** - * .vortex.dtype.PType days_ptype = 1; - * @return The daysPtype. - */ - @java.lang.Override - public io.github.dfa1.vortex.proto.DTypeProtos.PType getDaysPtype() { - io.github.dfa1.vortex.proto.DTypeProtos.PType result = io.github.dfa1.vortex.proto.DTypeProtos.PType.forNumber(daysPtype_); - return result == null ? io.github.dfa1.vortex.proto.DTypeProtos.PType.UNRECOGNIZED : result; - } - /** - * .vortex.dtype.PType days_ptype = 1; - * @param value The daysPtype to set. - * @throws IllegalArgumentException if UNRECOGNIZED is provided. - * @return This builder for chaining. - */ - public Builder setDaysPtype(io.github.dfa1.vortex.proto.DTypeProtos.PType value) { - if (value == null) { throw new NullPointerException(); } - bitField0_ |= 0x00000001; - daysPtype_ = value.getNumber(); - onChanged(); - return this; - } - /** - * .vortex.dtype.PType days_ptype = 1; - * @return This builder for chaining. - */ - public Builder clearDaysPtype() { - bitField0_ = (bitField0_ & ~0x00000001); - daysPtype_ = 0; - onChanged(); - return this; - } - - private int secondsPtype_ = 0; - /** - * .vortex.dtype.PType seconds_ptype = 2; - * @return The enum numeric value on the wire for secondsPtype. - */ - @java.lang.Override public int getSecondsPtypeValue() { - return secondsPtype_; - } - /** - * .vortex.dtype.PType seconds_ptype = 2; - * @param value The enum numeric value on the wire for secondsPtype to set. - * @return This builder for chaining. - */ - public Builder setSecondsPtypeValue(int value) { - secondsPtype_ = value; - bitField0_ |= 0x00000002; - onChanged(); - return this; - } - /** - * .vortex.dtype.PType seconds_ptype = 2; - * @return The secondsPtype. - */ - @java.lang.Override - public io.github.dfa1.vortex.proto.DTypeProtos.PType getSecondsPtype() { - io.github.dfa1.vortex.proto.DTypeProtos.PType result = io.github.dfa1.vortex.proto.DTypeProtos.PType.forNumber(secondsPtype_); - return result == null ? io.github.dfa1.vortex.proto.DTypeProtos.PType.UNRECOGNIZED : result; - } - /** - * .vortex.dtype.PType seconds_ptype = 2; - * @param value The secondsPtype to set. - * @throws IllegalArgumentException if UNRECOGNIZED is provided. - * @return This builder for chaining. - */ - public Builder setSecondsPtype(io.github.dfa1.vortex.proto.DTypeProtos.PType value) { - if (value == null) { throw new NullPointerException(); } - bitField0_ |= 0x00000002; - secondsPtype_ = value.getNumber(); - onChanged(); - return this; - } - /** - * .vortex.dtype.PType seconds_ptype = 2; - * @return This builder for chaining. - */ - public Builder clearSecondsPtype() { - bitField0_ = (bitField0_ & ~0x00000002); - secondsPtype_ = 0; - onChanged(); - return this; - } - - private int subsecondsPtype_ = 0; - /** - * .vortex.dtype.PType subseconds_ptype = 3; - * @return The enum numeric value on the wire for subsecondsPtype. - */ - @java.lang.Override public int getSubsecondsPtypeValue() { - return subsecondsPtype_; - } - /** - * .vortex.dtype.PType subseconds_ptype = 3; - * @param value The enum numeric value on the wire for subsecondsPtype to set. - * @return This builder for chaining. - */ - public Builder setSubsecondsPtypeValue(int value) { - subsecondsPtype_ = value; - bitField0_ |= 0x00000004; - onChanged(); - return this; - } - /** - * .vortex.dtype.PType subseconds_ptype = 3; - * @return The subsecondsPtype. - */ - @java.lang.Override - public io.github.dfa1.vortex.proto.DTypeProtos.PType getSubsecondsPtype() { - io.github.dfa1.vortex.proto.DTypeProtos.PType result = io.github.dfa1.vortex.proto.DTypeProtos.PType.forNumber(subsecondsPtype_); - return result == null ? io.github.dfa1.vortex.proto.DTypeProtos.PType.UNRECOGNIZED : result; - } - /** - * .vortex.dtype.PType subseconds_ptype = 3; - * @param value The subsecondsPtype to set. - * @throws IllegalArgumentException if UNRECOGNIZED is provided. - * @return This builder for chaining. - */ - public Builder setSubsecondsPtype(io.github.dfa1.vortex.proto.DTypeProtos.PType value) { - if (value == null) { throw new NullPointerException(); } - bitField0_ |= 0x00000004; - subsecondsPtype_ = value.getNumber(); - onChanged(); - return this; - } - /** - * .vortex.dtype.PType subseconds_ptype = 3; - * @return This builder for chaining. - */ - public Builder clearSubsecondsPtype() { - bitField0_ = (bitField0_ & ~0x00000004); - subsecondsPtype_ = 0; - onChanged(); - return this; - } - - // @@protoc_insertion_point(builder_scope:vortex.encodings.DateTimePartsMetadata) - } - - // @@protoc_insertion_point(class_scope:vortex.encodings.DateTimePartsMetadata) - private static final io.github.dfa1.vortex.proto.EncodingProtos.DateTimePartsMetadata DEFAULT_INSTANCE; - static { - DEFAULT_INSTANCE = new io.github.dfa1.vortex.proto.EncodingProtos.DateTimePartsMetadata(); - } - - public static io.github.dfa1.vortex.proto.EncodingProtos.DateTimePartsMetadata getDefaultInstance() { - return DEFAULT_INSTANCE; - } - - private static final com.google.protobuf.Parser - PARSER = new com.google.protobuf.AbstractParser() { - @java.lang.Override - public DateTimePartsMetadata parsePartialFrom( - com.google.protobuf.CodedInputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws com.google.protobuf.InvalidProtocolBufferException { - Builder builder = newBuilder(); - try { - builder.mergeFrom(input, extensionRegistry); - } catch (com.google.protobuf.InvalidProtocolBufferException e) { - throw e.setUnfinishedMessage(builder.buildPartial()); - } catch (com.google.protobuf.UninitializedMessageException e) { - throw e.asInvalidProtocolBufferException().setUnfinishedMessage(builder.buildPartial()); - } catch (java.io.IOException e) { - throw new com.google.protobuf.InvalidProtocolBufferException(e) - .setUnfinishedMessage(builder.buildPartial()); - } - return builder.buildPartial(); - } - }; - - public static com.google.protobuf.Parser parser() { - return PARSER; - } - - @java.lang.Override - public com.google.protobuf.Parser getParserForType() { - return PARSER; - } - - @java.lang.Override - public io.github.dfa1.vortex.proto.EncodingProtos.DateTimePartsMetadata getDefaultInstanceForType() { - return DEFAULT_INSTANCE; - } - - } - - public interface ZstdFrameMetadataOrBuilder extends - // @@protoc_insertion_point(interface_extends:vortex.encodings.ZstdFrameMetadata) - com.google.protobuf.MessageOrBuilder { - - /** - * uint64 uncompressed_size = 1; - * @return The uncompressedSize. - */ - long getUncompressedSize(); - - /** - * uint64 n_values = 2; - * @return The nValues. - */ - long getNValues(); - } - /** - * Protobuf type {@code vortex.encodings.ZstdFrameMetadata} - */ - public static final class ZstdFrameMetadata extends - com.google.protobuf.GeneratedMessage implements - // @@protoc_insertion_point(message_implements:vortex.encodings.ZstdFrameMetadata) - ZstdFrameMetadataOrBuilder { - private static final long serialVersionUID = 0L; - static { - com.google.protobuf.RuntimeVersion.validateProtobufGencodeVersion( - com.google.protobuf.RuntimeVersion.RuntimeDomain.PUBLIC, - /* major= */ 4, - /* minor= */ 35, - /* patch= */ 0, - /* suffix= */ "", - "ZstdFrameMetadata"); - } - // Use ZstdFrameMetadata.newBuilder() to construct. - private ZstdFrameMetadata(com.google.protobuf.GeneratedMessage.Builder builder) { - super(builder); - } - private ZstdFrameMetadata() { - } - - public static final com.google.protobuf.Descriptors.Descriptor - getDescriptor() { - return io.github.dfa1.vortex.proto.EncodingProtos.internal_static_vortex_encodings_ZstdFrameMetadata_descriptor; - } - - @java.lang.Override - public com.google.protobuf.Descriptors.Descriptor getDescriptorForType() { - return io.github.dfa1.vortex.proto.EncodingProtos.internal_static_vortex_encodings_ZstdFrameMetadata_descriptor; - } - - @java.lang.Override - protected com.google.protobuf.GeneratedMessage.FieldAccessorTable - internalGetFieldAccessorTable() { - return io.github.dfa1.vortex.proto.EncodingProtos.internal_static_vortex_encodings_ZstdFrameMetadata_fieldAccessorTable - .ensureFieldAccessorsInitialized( - io.github.dfa1.vortex.proto.EncodingProtos.ZstdFrameMetadata.class, io.github.dfa1.vortex.proto.EncodingProtos.ZstdFrameMetadata.Builder.class); - } - - public static final int UNCOMPRESSED_SIZE_FIELD_NUMBER = 1; - private long uncompressedSize_ = 0L; - /** - * uint64 uncompressed_size = 1; - * @return The uncompressedSize. - */ - @java.lang.Override - public long getUncompressedSize() { - return uncompressedSize_; - } - - public static final int N_VALUES_FIELD_NUMBER = 2; - private long nValues_ = 0L; - /** - * uint64 n_values = 2; - * @return The nValues. - */ - @java.lang.Override - public long getNValues() { - return nValues_; - } - - private byte memoizedIsInitialized = -1; - @java.lang.Override - public final boolean isInitialized() { - byte isInitialized = memoizedIsInitialized; - if (isInitialized == 1) return true; - if (isInitialized == 0) return false; - - memoizedIsInitialized = 1; - return true; - } - - @java.lang.Override - public void writeTo(com.google.protobuf.CodedOutputStream output) - throws java.io.IOException { - if (uncompressedSize_ != 0L) { - output.writeUInt64(1, uncompressedSize_); - } - if (nValues_ != 0L) { - output.writeUInt64(2, nValues_); - } - getUnknownFields().writeTo(output); - } - private int computeSerializedSize_0() { - int size = 0; - if (uncompressedSize_ != 0L) { - size += com.google.protobuf.CodedOutputStream - .computeUInt64Size(1, uncompressedSize_); - } - if (nValues_ != 0L) { - size += com.google.protobuf.CodedOutputStream - .computeUInt64Size(2, nValues_); - } - return size; - } - @java.lang.Override - public int getSerializedSize() { - int size = memoizedSize; - if (size != -1) return size; - - size = 0; - size += computeSerializedSize_0(); - size += getUnknownFields().getSerializedSize(); - memoizedSize = size; - return size; - } - - @java.lang.Override - public boolean equals(final java.lang.Object obj) { - if (obj == this) { - return true; - } - if (!(obj instanceof io.github.dfa1.vortex.proto.EncodingProtos.ZstdFrameMetadata)) { - return super.equals(obj); - } - io.github.dfa1.vortex.proto.EncodingProtos.ZstdFrameMetadata other = (io.github.dfa1.vortex.proto.EncodingProtos.ZstdFrameMetadata) obj; - - if (getUncompressedSize() - != other.getUncompressedSize()) return false; - if (getNValues() - != other.getNValues()) return false; - if (!getUnknownFields().equals(other.getUnknownFields())) return false; - return true; - } - - @java.lang.Override - public int hashCode() { - if (memoizedHashCode != 0) { - return memoizedHashCode; - } - int hash = 41; - hash = (19 * hash) + getDescriptor().hashCode(); - hash = (37 * hash) + UNCOMPRESSED_SIZE_FIELD_NUMBER; - hash = (53 * hash) + com.google.protobuf.Internal.hashLong( - getUncompressedSize()); - hash = (37 * hash) + N_VALUES_FIELD_NUMBER; - hash = (53 * hash) + com.google.protobuf.Internal.hashLong( - getNValues()); - hash = (29 * hash) + getUnknownFields().hashCode(); - memoizedHashCode = hash; - return hash; - } - - public static io.github.dfa1.vortex.proto.EncodingProtos.ZstdFrameMetadata parseFrom( - java.nio.ByteBuffer data) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.ZstdFrameMetadata parseFrom( - java.nio.ByteBuffer data, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data, extensionRegistry); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.ZstdFrameMetadata parseFrom( - com.google.protobuf.ByteString data) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.ZstdFrameMetadata parseFrom( - com.google.protobuf.ByteString data, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data, extensionRegistry); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.ZstdFrameMetadata parseFrom(byte[] data) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.ZstdFrameMetadata parseFrom( - byte[] data, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data, extensionRegistry); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.ZstdFrameMetadata parseFrom(java.io.InputStream input) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseWithIOException(PARSER, input); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.ZstdFrameMetadata parseFrom( - java.io.InputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseWithIOException(PARSER, input, extensionRegistry); - } - - public static io.github.dfa1.vortex.proto.EncodingProtos.ZstdFrameMetadata parseDelimitedFrom(java.io.InputStream input) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseDelimitedWithIOException(PARSER, input); - } - - public static io.github.dfa1.vortex.proto.EncodingProtos.ZstdFrameMetadata parseDelimitedFrom( - java.io.InputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseDelimitedWithIOException(PARSER, input, extensionRegistry); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.ZstdFrameMetadata parseFrom( - com.google.protobuf.CodedInputStream input) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseWithIOException(PARSER, input); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.ZstdFrameMetadata parseFrom( - com.google.protobuf.CodedInputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseWithIOException(PARSER, input, extensionRegistry); - } - - @java.lang.Override - public Builder newBuilderForType() { return newBuilder(); } - public static Builder newBuilder() { - return DEFAULT_INSTANCE.toBuilder(); - } - public static Builder newBuilder(io.github.dfa1.vortex.proto.EncodingProtos.ZstdFrameMetadata prototype) { - return DEFAULT_INSTANCE.toBuilder().mergeFrom(prototype); - } - @java.lang.Override - public Builder toBuilder() { - return this == DEFAULT_INSTANCE - ? new Builder() : new Builder().mergeFrom(this); - } - - @java.lang.Override - protected Builder newBuilderForType( - com.google.protobuf.GeneratedMessage.BuilderParent parent) { - Builder builder = new Builder(parent); - return builder; - } - /** - * Protobuf type {@code vortex.encodings.ZstdFrameMetadata} - */ - public static final class Builder extends - com.google.protobuf.GeneratedMessage.Builder implements - // @@protoc_insertion_point(builder_implements:vortex.encodings.ZstdFrameMetadata) - io.github.dfa1.vortex.proto.EncodingProtos.ZstdFrameMetadataOrBuilder { - public static final com.google.protobuf.Descriptors.Descriptor - getDescriptor() { - return io.github.dfa1.vortex.proto.EncodingProtos.internal_static_vortex_encodings_ZstdFrameMetadata_descriptor; - } - - @java.lang.Override - protected com.google.protobuf.GeneratedMessage.FieldAccessorTable - internalGetFieldAccessorTable() { - return io.github.dfa1.vortex.proto.EncodingProtos.internal_static_vortex_encodings_ZstdFrameMetadata_fieldAccessorTable - .ensureFieldAccessorsInitialized( - io.github.dfa1.vortex.proto.EncodingProtos.ZstdFrameMetadata.class, io.github.dfa1.vortex.proto.EncodingProtos.ZstdFrameMetadata.Builder.class); - } - - // Construct using io.github.dfa1.vortex.proto.EncodingProtos.ZstdFrameMetadata.newBuilder() - private Builder() { - - } - - private Builder( - com.google.protobuf.GeneratedMessage.BuilderParent parent) { - super(parent); - - } - @java.lang.Override - public Builder clear() { - super.clear(); - bitField0_ = 0; - uncompressedSize_ = 0L; - nValues_ = 0L; - return this; - } - - @java.lang.Override - public com.google.protobuf.Descriptors.Descriptor - getDescriptorForType() { - return io.github.dfa1.vortex.proto.EncodingProtos.internal_static_vortex_encodings_ZstdFrameMetadata_descriptor; - } - - @java.lang.Override - public io.github.dfa1.vortex.proto.EncodingProtos.ZstdFrameMetadata getDefaultInstanceForType() { - return io.github.dfa1.vortex.proto.EncodingProtos.ZstdFrameMetadata.getDefaultInstance(); - } - - @java.lang.Override - public io.github.dfa1.vortex.proto.EncodingProtos.ZstdFrameMetadata build() { - io.github.dfa1.vortex.proto.EncodingProtos.ZstdFrameMetadata result = buildPartial(); - if (!result.isInitialized()) { - throw newUninitializedMessageException(result); - } - return result; - } - - @java.lang.Override - public io.github.dfa1.vortex.proto.EncodingProtos.ZstdFrameMetadata buildPartial() { - io.github.dfa1.vortex.proto.EncodingProtos.ZstdFrameMetadata result = new io.github.dfa1.vortex.proto.EncodingProtos.ZstdFrameMetadata(this); - if (bitField0_ != 0) { buildPartial0(result); } - onBuilt(); - return result; - } - - private void buildPartial0(io.github.dfa1.vortex.proto.EncodingProtos.ZstdFrameMetadata result) { - int from_bitField0_ = bitField0_; - if (((from_bitField0_ & 0x00000001) != 0)) { - result.uncompressedSize_ = uncompressedSize_; - } - if (((from_bitField0_ & 0x00000002) != 0)) { - result.nValues_ = nValues_; - } - } - - @java.lang.Override - public Builder mergeFrom(com.google.protobuf.Message other) { - if (other instanceof io.github.dfa1.vortex.proto.EncodingProtos.ZstdFrameMetadata) { - return mergeFrom((io.github.dfa1.vortex.proto.EncodingProtos.ZstdFrameMetadata)other); - } else { - super.mergeFrom(other); - return this; - } - } - - public Builder mergeFrom(io.github.dfa1.vortex.proto.EncodingProtos.ZstdFrameMetadata other) { - if (other == io.github.dfa1.vortex.proto.EncodingProtos.ZstdFrameMetadata.getDefaultInstance()) return this; - if (other.getUncompressedSize() != 0L) { - setUncompressedSize(other.getUncompressedSize()); - } - if (other.getNValues() != 0L) { - setNValues(other.getNValues()); - } - this.mergeUnknownFields(other.getUnknownFields()); - onChanged(); - return this; - } - - @java.lang.Override - public final boolean isInitialized() { - return true; - } - - @java.lang.Override - public Builder mergeFrom( - com.google.protobuf.CodedInputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws java.io.IOException { - if (extensionRegistry == null) { - throw new java.lang.NullPointerException(); - } - try { - boolean done = false; - while (!done) { - int tag = input.readTag(); - switch (tag) { - case 0: - done = true; - break; - case 8: { - uncompressedSize_ = input.readUInt64(); - bitField0_ |= 0x00000001; - break; - } // case 8 - case 16: { - nValues_ = input.readUInt64(); - bitField0_ |= 0x00000002; - break; - } // case 16 - default: { - if (!super.parseUnknownField(input, extensionRegistry, tag)) { - done = true; // was an endgroup tag - } - break; - } // default: - } // switch (tag) - } // while (!done) - } catch (com.google.protobuf.InvalidProtocolBufferException e) { - throw e.unwrapIOException(); - } finally { - onChanged(); - } // finally - return this; - } - private int bitField0_; - - private long uncompressedSize_ ; - /** - * uint64 uncompressed_size = 1; - * @return The uncompressedSize. - */ - @java.lang.Override - public long getUncompressedSize() { - return uncompressedSize_; - } - /** - * uint64 uncompressed_size = 1; - * @param value The uncompressedSize to set. - * @return This builder for chaining. - */ - public Builder setUncompressedSize(long value) { - - uncompressedSize_ = value; - bitField0_ |= 0x00000001; - onChanged(); - return this; - } - /** - * uint64 uncompressed_size = 1; - * @return This builder for chaining. - */ - public Builder clearUncompressedSize() { - bitField0_ = (bitField0_ & ~0x00000001); - uncompressedSize_ = 0L; - onChanged(); - return this; - } - - private long nValues_ ; - /** - * uint64 n_values = 2; - * @return The nValues. - */ - @java.lang.Override - public long getNValues() { - return nValues_; - } - /** - * uint64 n_values = 2; - * @param value The nValues to set. - * @return This builder for chaining. - */ - public Builder setNValues(long value) { - - nValues_ = value; - bitField0_ |= 0x00000002; - onChanged(); - return this; - } - /** - * uint64 n_values = 2; - * @return This builder for chaining. - */ - public Builder clearNValues() { - bitField0_ = (bitField0_ & ~0x00000002); - nValues_ = 0L; - onChanged(); - return this; - } - - // @@protoc_insertion_point(builder_scope:vortex.encodings.ZstdFrameMetadata) - } - - // @@protoc_insertion_point(class_scope:vortex.encodings.ZstdFrameMetadata) - private static final io.github.dfa1.vortex.proto.EncodingProtos.ZstdFrameMetadata DEFAULT_INSTANCE; - static { - DEFAULT_INSTANCE = new io.github.dfa1.vortex.proto.EncodingProtos.ZstdFrameMetadata(); - } - - public static io.github.dfa1.vortex.proto.EncodingProtos.ZstdFrameMetadata getDefaultInstance() { - return DEFAULT_INSTANCE; - } - - private static final com.google.protobuf.Parser - PARSER = new com.google.protobuf.AbstractParser() { - @java.lang.Override - public ZstdFrameMetadata parsePartialFrom( - com.google.protobuf.CodedInputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws com.google.protobuf.InvalidProtocolBufferException { - Builder builder = newBuilder(); - try { - builder.mergeFrom(input, extensionRegistry); - } catch (com.google.protobuf.InvalidProtocolBufferException e) { - throw e.setUnfinishedMessage(builder.buildPartial()); - } catch (com.google.protobuf.UninitializedMessageException e) { - throw e.asInvalidProtocolBufferException().setUnfinishedMessage(builder.buildPartial()); - } catch (java.io.IOException e) { - throw new com.google.protobuf.InvalidProtocolBufferException(e) - .setUnfinishedMessage(builder.buildPartial()); - } - return builder.buildPartial(); - } - }; - - public static com.google.protobuf.Parser parser() { - return PARSER; - } - - @java.lang.Override - public com.google.protobuf.Parser getParserForType() { - return PARSER; - } - - @java.lang.Override - public io.github.dfa1.vortex.proto.EncodingProtos.ZstdFrameMetadata getDefaultInstanceForType() { - return DEFAULT_INSTANCE; - } - - } - - public interface ZstdMetadataOrBuilder extends - // @@protoc_insertion_point(interface_extends:vortex.encodings.ZstdMetadata) - com.google.protobuf.MessageOrBuilder { - - /** - * uint32 dictionary_size = 1; - * @return The dictionarySize. - */ - int getDictionarySize(); - - /** - * repeated .vortex.encodings.ZstdFrameMetadata frames = 2; - */ - java.util.List - getFramesList(); - /** - * repeated .vortex.encodings.ZstdFrameMetadata frames = 2; - */ - io.github.dfa1.vortex.proto.EncodingProtos.ZstdFrameMetadata getFrames(int index); - /** - * repeated .vortex.encodings.ZstdFrameMetadata frames = 2; - */ - int getFramesCount(); - /** - * repeated .vortex.encodings.ZstdFrameMetadata frames = 2; - */ - java.util.List - getFramesOrBuilderList(); - /** - * repeated .vortex.encodings.ZstdFrameMetadata frames = 2; - */ - io.github.dfa1.vortex.proto.EncodingProtos.ZstdFrameMetadataOrBuilder getFramesOrBuilder( - int index); - } - /** - * Protobuf type {@code vortex.encodings.ZstdMetadata} - */ - public static final class ZstdMetadata extends - com.google.protobuf.GeneratedMessage implements - // @@protoc_insertion_point(message_implements:vortex.encodings.ZstdMetadata) - ZstdMetadataOrBuilder { - private static final long serialVersionUID = 0L; - static { - com.google.protobuf.RuntimeVersion.validateProtobufGencodeVersion( - com.google.protobuf.RuntimeVersion.RuntimeDomain.PUBLIC, - /* major= */ 4, - /* minor= */ 35, - /* patch= */ 0, - /* suffix= */ "", - "ZstdMetadata"); - } - // Use ZstdMetadata.newBuilder() to construct. - private ZstdMetadata(com.google.protobuf.GeneratedMessage.Builder builder) { - super(builder); - } - private ZstdMetadata() { - frames_ = java.util.Collections.emptyList(); - } - - public static final com.google.protobuf.Descriptors.Descriptor - getDescriptor() { - return io.github.dfa1.vortex.proto.EncodingProtos.internal_static_vortex_encodings_ZstdMetadata_descriptor; - } - - @java.lang.Override - public com.google.protobuf.Descriptors.Descriptor getDescriptorForType() { - return io.github.dfa1.vortex.proto.EncodingProtos.internal_static_vortex_encodings_ZstdMetadata_descriptor; - } - - @java.lang.Override - protected com.google.protobuf.GeneratedMessage.FieldAccessorTable - internalGetFieldAccessorTable() { - return io.github.dfa1.vortex.proto.EncodingProtos.internal_static_vortex_encodings_ZstdMetadata_fieldAccessorTable - .ensureFieldAccessorsInitialized( - io.github.dfa1.vortex.proto.EncodingProtos.ZstdMetadata.class, io.github.dfa1.vortex.proto.EncodingProtos.ZstdMetadata.Builder.class); - } - - public static final int DICTIONARY_SIZE_FIELD_NUMBER = 1; - private int dictionarySize_ = 0; - /** - * uint32 dictionary_size = 1; - * @return The dictionarySize. - */ - @java.lang.Override - public int getDictionarySize() { - return dictionarySize_; - } - - public static final int FRAMES_FIELD_NUMBER = 2; - @SuppressWarnings("serial") - private java.util.List frames_; - /** - * repeated .vortex.encodings.ZstdFrameMetadata frames = 2; - */ - @java.lang.Override - public java.util.List getFramesList() { - return frames_; - } - /** - * repeated .vortex.encodings.ZstdFrameMetadata frames = 2; - */ - @java.lang.Override - public java.util.List - getFramesOrBuilderList() { - return frames_; - } - /** - * repeated .vortex.encodings.ZstdFrameMetadata frames = 2; - */ - @java.lang.Override - public int getFramesCount() { - return frames_.size(); - } - /** - * repeated .vortex.encodings.ZstdFrameMetadata frames = 2; - */ - @java.lang.Override - public io.github.dfa1.vortex.proto.EncodingProtos.ZstdFrameMetadata getFrames(int index) { - return frames_.get(index); - } - /** - * repeated .vortex.encodings.ZstdFrameMetadata frames = 2; - */ - @java.lang.Override - public io.github.dfa1.vortex.proto.EncodingProtos.ZstdFrameMetadataOrBuilder getFramesOrBuilder( - int index) { - return frames_.get(index); - } - - private byte memoizedIsInitialized = -1; - @java.lang.Override - public final boolean isInitialized() { - byte isInitialized = memoizedIsInitialized; - if (isInitialized == 1) return true; - if (isInitialized == 0) return false; - - memoizedIsInitialized = 1; - return true; - } - - @java.lang.Override - public void writeTo(com.google.protobuf.CodedOutputStream output) - throws java.io.IOException { - if (dictionarySize_ != 0) { - output.writeUInt32(1, dictionarySize_); - } - for (int i = 0; i < frames_.size(); i++) { - output.writeMessage(2, frames_.get(i)); - } - getUnknownFields().writeTo(output); - } - private int computeSerializedSize_0() { - int size = 0; - if (dictionarySize_ != 0) { - size += com.google.protobuf.CodedOutputStream - .computeUInt32Size(1, dictionarySize_); - } - - { - final int count = frames_.size(); - for (int i = 0; i < count; i++) { - size += com.google.protobuf.CodedOutputStream - .computeMessageSizeNoTag(frames_.get(i)); - } - size += 1 * count; - } - return size; - } - @java.lang.Override - public int getSerializedSize() { - int size = memoizedSize; - if (size != -1) return size; - - size = 0; - size += computeSerializedSize_0(); - size += getUnknownFields().getSerializedSize(); - memoizedSize = size; - return size; - } - - @java.lang.Override - public boolean equals(final java.lang.Object obj) { - if (obj == this) { - return true; - } - if (!(obj instanceof io.github.dfa1.vortex.proto.EncodingProtos.ZstdMetadata)) { - return super.equals(obj); - } - io.github.dfa1.vortex.proto.EncodingProtos.ZstdMetadata other = (io.github.dfa1.vortex.proto.EncodingProtos.ZstdMetadata) obj; - - if (getDictionarySize() - != other.getDictionarySize()) return false; - if (!getFramesList() - .equals(other.getFramesList())) return false; - if (!getUnknownFields().equals(other.getUnknownFields())) return false; - return true; - } - - @java.lang.Override - public int hashCode() { - if (memoizedHashCode != 0) { - return memoizedHashCode; - } - int hash = 41; - hash = (19 * hash) + getDescriptor().hashCode(); - hash = (37 * hash) + DICTIONARY_SIZE_FIELD_NUMBER; - hash = (53 * hash) + getDictionarySize(); - if (getFramesCount() > 0) { - hash = (37 * hash) + FRAMES_FIELD_NUMBER; - hash = (53 * hash) + getFramesList().hashCode(); - } - hash = (29 * hash) + getUnknownFields().hashCode(); - memoizedHashCode = hash; - return hash; - } - - public static io.github.dfa1.vortex.proto.EncodingProtos.ZstdMetadata parseFrom( - java.nio.ByteBuffer data) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.ZstdMetadata parseFrom( - java.nio.ByteBuffer data, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data, extensionRegistry); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.ZstdMetadata parseFrom( - com.google.protobuf.ByteString data) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.ZstdMetadata parseFrom( - com.google.protobuf.ByteString data, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data, extensionRegistry); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.ZstdMetadata parseFrom(byte[] data) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.ZstdMetadata parseFrom( - byte[] data, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data, extensionRegistry); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.ZstdMetadata parseFrom(java.io.InputStream input) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseWithIOException(PARSER, input); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.ZstdMetadata parseFrom( - java.io.InputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseWithIOException(PARSER, input, extensionRegistry); - } - - public static io.github.dfa1.vortex.proto.EncodingProtos.ZstdMetadata parseDelimitedFrom(java.io.InputStream input) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseDelimitedWithIOException(PARSER, input); - } - - public static io.github.dfa1.vortex.proto.EncodingProtos.ZstdMetadata parseDelimitedFrom( - java.io.InputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseDelimitedWithIOException(PARSER, input, extensionRegistry); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.ZstdMetadata parseFrom( - com.google.protobuf.CodedInputStream input) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseWithIOException(PARSER, input); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.ZstdMetadata parseFrom( - com.google.protobuf.CodedInputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseWithIOException(PARSER, input, extensionRegistry); - } - - @java.lang.Override - public Builder newBuilderForType() { return newBuilder(); } - public static Builder newBuilder() { - return DEFAULT_INSTANCE.toBuilder(); - } - public static Builder newBuilder(io.github.dfa1.vortex.proto.EncodingProtos.ZstdMetadata prototype) { - return DEFAULT_INSTANCE.toBuilder().mergeFrom(prototype); - } - @java.lang.Override - public Builder toBuilder() { - return this == DEFAULT_INSTANCE - ? new Builder() : new Builder().mergeFrom(this); - } - - @java.lang.Override - protected Builder newBuilderForType( - com.google.protobuf.GeneratedMessage.BuilderParent parent) { - Builder builder = new Builder(parent); - return builder; - } - /** - * Protobuf type {@code vortex.encodings.ZstdMetadata} - */ - public static final class Builder extends - com.google.protobuf.GeneratedMessage.Builder implements - // @@protoc_insertion_point(builder_implements:vortex.encodings.ZstdMetadata) - io.github.dfa1.vortex.proto.EncodingProtos.ZstdMetadataOrBuilder { - public static final com.google.protobuf.Descriptors.Descriptor - getDescriptor() { - return io.github.dfa1.vortex.proto.EncodingProtos.internal_static_vortex_encodings_ZstdMetadata_descriptor; - } - - @java.lang.Override - protected com.google.protobuf.GeneratedMessage.FieldAccessorTable - internalGetFieldAccessorTable() { - return io.github.dfa1.vortex.proto.EncodingProtos.internal_static_vortex_encodings_ZstdMetadata_fieldAccessorTable - .ensureFieldAccessorsInitialized( - io.github.dfa1.vortex.proto.EncodingProtos.ZstdMetadata.class, io.github.dfa1.vortex.proto.EncodingProtos.ZstdMetadata.Builder.class); - } - - // Construct using io.github.dfa1.vortex.proto.EncodingProtos.ZstdMetadata.newBuilder() - private Builder() { - - } - - private Builder( - com.google.protobuf.GeneratedMessage.BuilderParent parent) { - super(parent); - - } - @java.lang.Override - public Builder clear() { - super.clear(); - bitField0_ = 0; - dictionarySize_ = 0; - if (framesBuilder_ == null) { - frames_ = java.util.Collections.emptyList(); - } else { - frames_ = null; - framesBuilder_.clear(); - } - bitField0_ = (bitField0_ & ~0x00000002); - return this; - } - - @java.lang.Override - public com.google.protobuf.Descriptors.Descriptor - getDescriptorForType() { - return io.github.dfa1.vortex.proto.EncodingProtos.internal_static_vortex_encodings_ZstdMetadata_descriptor; - } - - @java.lang.Override - public io.github.dfa1.vortex.proto.EncodingProtos.ZstdMetadata getDefaultInstanceForType() { - return io.github.dfa1.vortex.proto.EncodingProtos.ZstdMetadata.getDefaultInstance(); - } - - @java.lang.Override - public io.github.dfa1.vortex.proto.EncodingProtos.ZstdMetadata build() { - io.github.dfa1.vortex.proto.EncodingProtos.ZstdMetadata result = buildPartial(); - if (!result.isInitialized()) { - throw newUninitializedMessageException(result); - } - return result; - } - - @java.lang.Override - public io.github.dfa1.vortex.proto.EncodingProtos.ZstdMetadata buildPartial() { - io.github.dfa1.vortex.proto.EncodingProtos.ZstdMetadata result = new io.github.dfa1.vortex.proto.EncodingProtos.ZstdMetadata(this); - buildPartialRepeatedFields(result); - if (bitField0_ != 0) { buildPartial0(result); } - onBuilt(); - return result; - } - - private void buildPartialRepeatedFields(io.github.dfa1.vortex.proto.EncodingProtos.ZstdMetadata result) { - if (framesBuilder_ == null) { - if (((bitField0_ & 0x00000002) != 0)) { - frames_ = java.util.Collections.unmodifiableList(frames_); - bitField0_ = (bitField0_ & ~0x00000002); - } - result.frames_ = frames_; - } else { - result.frames_ = framesBuilder_.build(); - } - } - - private void buildPartial0(io.github.dfa1.vortex.proto.EncodingProtos.ZstdMetadata result) { - int from_bitField0_ = bitField0_; - if (((from_bitField0_ & 0x00000001) != 0)) { - result.dictionarySize_ = dictionarySize_; - } - } - - @java.lang.Override - public Builder mergeFrom(com.google.protobuf.Message other) { - if (other instanceof io.github.dfa1.vortex.proto.EncodingProtos.ZstdMetadata) { - return mergeFrom((io.github.dfa1.vortex.proto.EncodingProtos.ZstdMetadata)other); - } else { - super.mergeFrom(other); - return this; - } - } - - public Builder mergeFrom(io.github.dfa1.vortex.proto.EncodingProtos.ZstdMetadata other) { - if (other == io.github.dfa1.vortex.proto.EncodingProtos.ZstdMetadata.getDefaultInstance()) return this; - if (other.getDictionarySize() != 0) { - setDictionarySize(other.getDictionarySize()); - } - if (framesBuilder_ == null) { - if (!other.frames_.isEmpty()) { - if (frames_.isEmpty()) { - frames_ = other.frames_; - bitField0_ = (bitField0_ & ~0x00000002); - } else { - ensureFramesIsMutable(); - frames_.addAll(other.frames_); - } - onChanged(); - } - } else { - if (!other.frames_.isEmpty()) { - if (framesBuilder_.isEmpty()) { - framesBuilder_.dispose(); - framesBuilder_ = null; - frames_ = other.frames_; - bitField0_ = (bitField0_ & ~0x00000002); - framesBuilder_ = - com.google.protobuf.GeneratedMessage.alwaysUseFieldBuilders ? - internalGetFramesFieldBuilder() : null; - } else { - framesBuilder_.addAllMessages(other.frames_); - } - } - } - this.mergeUnknownFields(other.getUnknownFields()); - onChanged(); - return this; - } - - @java.lang.Override - public final boolean isInitialized() { - return true; - } - - @java.lang.Override - public Builder mergeFrom( - com.google.protobuf.CodedInputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws java.io.IOException { - if (extensionRegistry == null) { - throw new java.lang.NullPointerException(); - } - try { - boolean done = false; - while (!done) { - int tag = input.readTag(); - switch (tag) { - case 0: - done = true; - break; - case 8: { - dictionarySize_ = input.readUInt32(); - bitField0_ |= 0x00000001; - break; - } // case 8 - case 18: { - io.github.dfa1.vortex.proto.EncodingProtos.ZstdFrameMetadata m = - input.readMessage( - io.github.dfa1.vortex.proto.EncodingProtos.ZstdFrameMetadata.parser(), - extensionRegistry); - if (framesBuilder_ == null) { - ensureFramesIsMutable(); - frames_.add(m); - } else { - framesBuilder_.addMessage(m); - } - break; - } // case 18 - default: { - if (!super.parseUnknownField(input, extensionRegistry, tag)) { - done = true; // was an endgroup tag - } - break; - } // default: - } // switch (tag) - } // while (!done) - } catch (com.google.protobuf.InvalidProtocolBufferException e) { - throw e.unwrapIOException(); - } finally { - onChanged(); - } // finally - return this; - } - private int bitField0_; - - private int dictionarySize_ ; - /** - * uint32 dictionary_size = 1; - * @return The dictionarySize. - */ - @java.lang.Override - public int getDictionarySize() { - return dictionarySize_; - } - /** - * uint32 dictionary_size = 1; - * @param value The dictionarySize to set. - * @return This builder for chaining. - */ - public Builder setDictionarySize(int value) { - - dictionarySize_ = value; - bitField0_ |= 0x00000001; - onChanged(); - return this; - } - /** - * uint32 dictionary_size = 1; - * @return This builder for chaining. - */ - public Builder clearDictionarySize() { - bitField0_ = (bitField0_ & ~0x00000001); - dictionarySize_ = 0; - onChanged(); - return this; - } - - private java.util.List frames_ = - java.util.Collections.emptyList(); - private void ensureFramesIsMutable() { - if (!((bitField0_ & 0x00000002) != 0)) { - frames_ = new java.util.ArrayList(frames_); - bitField0_ |= 0x00000002; - } - } - - private com.google.protobuf.RepeatedFieldBuilder< - io.github.dfa1.vortex.proto.EncodingProtos.ZstdFrameMetadata, io.github.dfa1.vortex.proto.EncodingProtos.ZstdFrameMetadata.Builder, io.github.dfa1.vortex.proto.EncodingProtos.ZstdFrameMetadataOrBuilder> framesBuilder_; - - /** - * repeated .vortex.encodings.ZstdFrameMetadata frames = 2; - */ - public java.util.List getFramesList() { - if (framesBuilder_ == null) { - return java.util.Collections.unmodifiableList(frames_); - } else { - return framesBuilder_.getMessageList(); - } - } - /** - * repeated .vortex.encodings.ZstdFrameMetadata frames = 2; - */ - public int getFramesCount() { - if (framesBuilder_ == null) { - return frames_.size(); - } else { - return framesBuilder_.getCount(); - } - } - /** - * repeated .vortex.encodings.ZstdFrameMetadata frames = 2; - */ - public io.github.dfa1.vortex.proto.EncodingProtos.ZstdFrameMetadata getFrames(int index) { - if (framesBuilder_ == null) { - return frames_.get(index); - } else { - return framesBuilder_.getMessage(index); - } - } - /** - * repeated .vortex.encodings.ZstdFrameMetadata frames = 2; - */ - public Builder setFrames( - int index, io.github.dfa1.vortex.proto.EncodingProtos.ZstdFrameMetadata value) { - if (framesBuilder_ == null) { - if (value == null) { - throw new NullPointerException(); - } - ensureFramesIsMutable(); - frames_.set(index, value); - onChanged(); - } else { - framesBuilder_.setMessage(index, value); - } - return this; - } - /** - * repeated .vortex.encodings.ZstdFrameMetadata frames = 2; - */ - public Builder setFrames( - int index, io.github.dfa1.vortex.proto.EncodingProtos.ZstdFrameMetadata.Builder builderForValue) { - if (framesBuilder_ == null) { - ensureFramesIsMutable(); - frames_.set(index, builderForValue.build()); - onChanged(); - } else { - framesBuilder_.setMessage(index, builderForValue.build()); - } - return this; - } - /** - * repeated .vortex.encodings.ZstdFrameMetadata frames = 2; - */ - public Builder addFrames(io.github.dfa1.vortex.proto.EncodingProtos.ZstdFrameMetadata value) { - if (framesBuilder_ == null) { - if (value == null) { - throw new NullPointerException(); - } - ensureFramesIsMutable(); - frames_.add(value); - onChanged(); - } else { - framesBuilder_.addMessage(value); - } - return this; - } - /** - * repeated .vortex.encodings.ZstdFrameMetadata frames = 2; - */ - public Builder addFrames( - int index, io.github.dfa1.vortex.proto.EncodingProtos.ZstdFrameMetadata value) { - if (framesBuilder_ == null) { - if (value == null) { - throw new NullPointerException(); - } - ensureFramesIsMutable(); - frames_.add(index, value); - onChanged(); - } else { - framesBuilder_.addMessage(index, value); - } - return this; - } - /** - * repeated .vortex.encodings.ZstdFrameMetadata frames = 2; - */ - public Builder addFrames( - io.github.dfa1.vortex.proto.EncodingProtos.ZstdFrameMetadata.Builder builderForValue) { - if (framesBuilder_ == null) { - ensureFramesIsMutable(); - frames_.add(builderForValue.build()); - onChanged(); - } else { - framesBuilder_.addMessage(builderForValue.build()); - } - return this; - } - /** - * repeated .vortex.encodings.ZstdFrameMetadata frames = 2; - */ - public Builder addFrames( - int index, io.github.dfa1.vortex.proto.EncodingProtos.ZstdFrameMetadata.Builder builderForValue) { - if (framesBuilder_ == null) { - ensureFramesIsMutable(); - frames_.add(index, builderForValue.build()); - onChanged(); - } else { - framesBuilder_.addMessage(index, builderForValue.build()); - } - return this; - } - /** - * repeated .vortex.encodings.ZstdFrameMetadata frames = 2; - */ - public Builder addAllFrames( - java.lang.Iterable values) { - if (framesBuilder_ == null) { - ensureFramesIsMutable(); - com.google.protobuf.AbstractMessageLite.Builder.addAll( - values, frames_); - onChanged(); - } else { - framesBuilder_.addAllMessages(values); - } - return this; - } - /** - * repeated .vortex.encodings.ZstdFrameMetadata frames = 2; - */ - public Builder clearFrames() { - if (framesBuilder_ == null) { - frames_ = java.util.Collections.emptyList(); - bitField0_ = (bitField0_ & ~0x00000002); - onChanged(); - } else { - framesBuilder_.clear(); - } - return this; - } - /** - * repeated .vortex.encodings.ZstdFrameMetadata frames = 2; - */ - public Builder removeFrames(int index) { - if (framesBuilder_ == null) { - ensureFramesIsMutable(); - frames_.remove(index); - onChanged(); - } else { - framesBuilder_.remove(index); - } - return this; - } - /** - * repeated .vortex.encodings.ZstdFrameMetadata frames = 2; - */ - public io.github.dfa1.vortex.proto.EncodingProtos.ZstdFrameMetadata.Builder getFramesBuilder( - int index) { - return internalGetFramesFieldBuilder().getBuilder(index); - } - /** - * repeated .vortex.encodings.ZstdFrameMetadata frames = 2; - */ - public io.github.dfa1.vortex.proto.EncodingProtos.ZstdFrameMetadataOrBuilder getFramesOrBuilder( - int index) { - if (framesBuilder_ == null) { - return frames_.get(index); } else { - return framesBuilder_.getMessageOrBuilder(index); - } - } - /** - * repeated .vortex.encodings.ZstdFrameMetadata frames = 2; - */ - public java.util.List - getFramesOrBuilderList() { - if (framesBuilder_ != null) { - return framesBuilder_.getMessageOrBuilderList(); - } else { - return java.util.Collections.unmodifiableList(frames_); - } - } - /** - * repeated .vortex.encodings.ZstdFrameMetadata frames = 2; - */ - public io.github.dfa1.vortex.proto.EncodingProtos.ZstdFrameMetadata.Builder addFramesBuilder() { - return internalGetFramesFieldBuilder().addBuilder( - io.github.dfa1.vortex.proto.EncodingProtos.ZstdFrameMetadata.getDefaultInstance()); - } - /** - * repeated .vortex.encodings.ZstdFrameMetadata frames = 2; - */ - public io.github.dfa1.vortex.proto.EncodingProtos.ZstdFrameMetadata.Builder addFramesBuilder( - int index) { - return internalGetFramesFieldBuilder().addBuilder( - index, io.github.dfa1.vortex.proto.EncodingProtos.ZstdFrameMetadata.getDefaultInstance()); - } - /** - * repeated .vortex.encodings.ZstdFrameMetadata frames = 2; - */ - public java.util.List - getFramesBuilderList() { - return internalGetFramesFieldBuilder().getBuilderList(); - } - private com.google.protobuf.RepeatedFieldBuilder< - io.github.dfa1.vortex.proto.EncodingProtos.ZstdFrameMetadata, io.github.dfa1.vortex.proto.EncodingProtos.ZstdFrameMetadata.Builder, io.github.dfa1.vortex.proto.EncodingProtos.ZstdFrameMetadataOrBuilder> - internalGetFramesFieldBuilder() { - if (framesBuilder_ == null) { - framesBuilder_ = new com.google.protobuf.RepeatedFieldBuilder< - io.github.dfa1.vortex.proto.EncodingProtos.ZstdFrameMetadata, io.github.dfa1.vortex.proto.EncodingProtos.ZstdFrameMetadata.Builder, io.github.dfa1.vortex.proto.EncodingProtos.ZstdFrameMetadataOrBuilder>( - frames_, - ((bitField0_ & 0x00000002) != 0), - getParentForChildren(), - isClean()); - frames_ = null; - } - return framesBuilder_; - } - - // @@protoc_insertion_point(builder_scope:vortex.encodings.ZstdMetadata) - } - - // @@protoc_insertion_point(class_scope:vortex.encodings.ZstdMetadata) - private static final io.github.dfa1.vortex.proto.EncodingProtos.ZstdMetadata DEFAULT_INSTANCE; - static { - DEFAULT_INSTANCE = new io.github.dfa1.vortex.proto.EncodingProtos.ZstdMetadata(); - } - - public static io.github.dfa1.vortex.proto.EncodingProtos.ZstdMetadata getDefaultInstance() { - return DEFAULT_INSTANCE; - } - - private static final com.google.protobuf.Parser - PARSER = new com.google.protobuf.AbstractParser() { - @java.lang.Override - public ZstdMetadata parsePartialFrom( - com.google.protobuf.CodedInputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws com.google.protobuf.InvalidProtocolBufferException { - Builder builder = newBuilder(); - try { - builder.mergeFrom(input, extensionRegistry); - } catch (com.google.protobuf.InvalidProtocolBufferException e) { - throw e.setUnfinishedMessage(builder.buildPartial()); - } catch (com.google.protobuf.UninitializedMessageException e) { - throw e.asInvalidProtocolBufferException().setUnfinishedMessage(builder.buildPartial()); - } catch (java.io.IOException e) { - throw new com.google.protobuf.InvalidProtocolBufferException(e) - .setUnfinishedMessage(builder.buildPartial()); - } - return builder.buildPartial(); - } - }; - - public static com.google.protobuf.Parser parser() { - return PARSER; - } - - @java.lang.Override - public com.google.protobuf.Parser getParserForType() { - return PARSER; - } - - @java.lang.Override - public io.github.dfa1.vortex.proto.EncodingProtos.ZstdMetadata getDefaultInstanceForType() { - return DEFAULT_INSTANCE; - } - - } - - public interface RLEMetadataOrBuilder extends - // @@protoc_insertion_point(interface_extends:vortex.encodings.RLEMetadata) - com.google.protobuf.MessageOrBuilder { - - /** - * uint64 values_len = 1; - * @return The valuesLen. - */ - long getValuesLen(); - - /** - * uint64 indices_len = 2; - * @return The indicesLen. - */ - long getIndicesLen(); - - /** - * .vortex.dtype.PType indices_ptype = 3; - * @return The enum numeric value on the wire for indicesPtype. - */ - int getIndicesPtypeValue(); - /** - * .vortex.dtype.PType indices_ptype = 3; - * @return The indicesPtype. - */ - io.github.dfa1.vortex.proto.DTypeProtos.PType getIndicesPtype(); - - /** - * uint64 values_idx_offsets_len = 4; - * @return The valuesIdxOffsetsLen. - */ - long getValuesIdxOffsetsLen(); - - /** - * .vortex.dtype.PType values_idx_offsets_ptype = 5; - * @return The enum numeric value on the wire for valuesIdxOffsetsPtype. - */ - int getValuesIdxOffsetsPtypeValue(); - /** - * .vortex.dtype.PType values_idx_offsets_ptype = 5; - * @return The valuesIdxOffsetsPtype. - */ - io.github.dfa1.vortex.proto.DTypeProtos.PType getValuesIdxOffsetsPtype(); - - /** - * uint64 offset = 6; - * @return The offset. - */ - long getOffset(); - } - /** - * Protobuf type {@code vortex.encodings.RLEMetadata} - */ - public static final class RLEMetadata extends - com.google.protobuf.GeneratedMessage implements - // @@protoc_insertion_point(message_implements:vortex.encodings.RLEMetadata) - RLEMetadataOrBuilder { - private static final long serialVersionUID = 0L; - static { - com.google.protobuf.RuntimeVersion.validateProtobufGencodeVersion( - com.google.protobuf.RuntimeVersion.RuntimeDomain.PUBLIC, - /* major= */ 4, - /* minor= */ 35, - /* patch= */ 0, - /* suffix= */ "", - "RLEMetadata"); - } - // Use RLEMetadata.newBuilder() to construct. - private RLEMetadata(com.google.protobuf.GeneratedMessage.Builder builder) { - super(builder); - } - private RLEMetadata() { - indicesPtype_ = 0; - valuesIdxOffsetsPtype_ = 0; - } - - public static final com.google.protobuf.Descriptors.Descriptor - getDescriptor() { - return io.github.dfa1.vortex.proto.EncodingProtos.internal_static_vortex_encodings_RLEMetadata_descriptor; - } - - @java.lang.Override - public com.google.protobuf.Descriptors.Descriptor getDescriptorForType() { - return io.github.dfa1.vortex.proto.EncodingProtos.internal_static_vortex_encodings_RLEMetadata_descriptor; - } - - @java.lang.Override - protected com.google.protobuf.GeneratedMessage.FieldAccessorTable - internalGetFieldAccessorTable() { - return io.github.dfa1.vortex.proto.EncodingProtos.internal_static_vortex_encodings_RLEMetadata_fieldAccessorTable - .ensureFieldAccessorsInitialized( - io.github.dfa1.vortex.proto.EncodingProtos.RLEMetadata.class, io.github.dfa1.vortex.proto.EncodingProtos.RLEMetadata.Builder.class); - } - - public static final int VALUES_LEN_FIELD_NUMBER = 1; - private long valuesLen_ = 0L; - /** - * uint64 values_len = 1; - * @return The valuesLen. - */ - @java.lang.Override - public long getValuesLen() { - return valuesLen_; - } - - public static final int INDICES_LEN_FIELD_NUMBER = 2; - private long indicesLen_ = 0L; - /** - * uint64 indices_len = 2; - * @return The indicesLen. - */ - @java.lang.Override - public long getIndicesLen() { - return indicesLen_; - } - - public static final int INDICES_PTYPE_FIELD_NUMBER = 3; - private int indicesPtype_ = 0; - /** - * .vortex.dtype.PType indices_ptype = 3; - * @return The enum numeric value on the wire for indicesPtype. - */ - @java.lang.Override public int getIndicesPtypeValue() { - return indicesPtype_; - } - /** - * .vortex.dtype.PType indices_ptype = 3; - * @return The indicesPtype. - */ - @java.lang.Override public io.github.dfa1.vortex.proto.DTypeProtos.PType getIndicesPtype() { - io.github.dfa1.vortex.proto.DTypeProtos.PType result = io.github.dfa1.vortex.proto.DTypeProtos.PType.forNumber(indicesPtype_); - return result == null ? io.github.dfa1.vortex.proto.DTypeProtos.PType.UNRECOGNIZED : result; - } - - public static final int VALUES_IDX_OFFSETS_LEN_FIELD_NUMBER = 4; - private long valuesIdxOffsetsLen_ = 0L; - /** - * uint64 values_idx_offsets_len = 4; - * @return The valuesIdxOffsetsLen. - */ - @java.lang.Override - public long getValuesIdxOffsetsLen() { - return valuesIdxOffsetsLen_; - } - - public static final int VALUES_IDX_OFFSETS_PTYPE_FIELD_NUMBER = 5; - private int valuesIdxOffsetsPtype_ = 0; - /** - * .vortex.dtype.PType values_idx_offsets_ptype = 5; - * @return The enum numeric value on the wire for valuesIdxOffsetsPtype. - */ - @java.lang.Override public int getValuesIdxOffsetsPtypeValue() { - return valuesIdxOffsetsPtype_; - } - /** - * .vortex.dtype.PType values_idx_offsets_ptype = 5; - * @return The valuesIdxOffsetsPtype. - */ - @java.lang.Override public io.github.dfa1.vortex.proto.DTypeProtos.PType getValuesIdxOffsetsPtype() { - io.github.dfa1.vortex.proto.DTypeProtos.PType result = io.github.dfa1.vortex.proto.DTypeProtos.PType.forNumber(valuesIdxOffsetsPtype_); - return result == null ? io.github.dfa1.vortex.proto.DTypeProtos.PType.UNRECOGNIZED : result; - } - - public static final int OFFSET_FIELD_NUMBER = 6; - private long offset_ = 0L; - /** - * uint64 offset = 6; - * @return The offset. - */ - @java.lang.Override - public long getOffset() { - return offset_; - } - - private byte memoizedIsInitialized = -1; - @java.lang.Override - public final boolean isInitialized() { - byte isInitialized = memoizedIsInitialized; - if (isInitialized == 1) return true; - if (isInitialized == 0) return false; - - memoizedIsInitialized = 1; - return true; - } - - @java.lang.Override - public void writeTo(com.google.protobuf.CodedOutputStream output) - throws java.io.IOException { - if (valuesLen_ != 0L) { - output.writeUInt64(1, valuesLen_); - } - if (indicesLen_ != 0L) { - output.writeUInt64(2, indicesLen_); - } - if (indicesPtype_ != io.github.dfa1.vortex.proto.DTypeProtos.PType.U8.getNumber()) { - output.writeEnum(3, indicesPtype_); - } - if (valuesIdxOffsetsLen_ != 0L) { - output.writeUInt64(4, valuesIdxOffsetsLen_); - } - if (valuesIdxOffsetsPtype_ != io.github.dfa1.vortex.proto.DTypeProtos.PType.U8.getNumber()) { - output.writeEnum(5, valuesIdxOffsetsPtype_); - } - if (offset_ != 0L) { - output.writeUInt64(6, offset_); - } - getUnknownFields().writeTo(output); - } - private int computeSerializedSize_0() { - int size = 0; - if (valuesLen_ != 0L) { - size += com.google.protobuf.CodedOutputStream - .computeUInt64Size(1, valuesLen_); - } - if (indicesLen_ != 0L) { - size += com.google.protobuf.CodedOutputStream - .computeUInt64Size(2, indicesLen_); - } - if (indicesPtype_ != io.github.dfa1.vortex.proto.DTypeProtos.PType.U8.getNumber()) { - size += com.google.protobuf.CodedOutputStream - .computeEnumSize(3, indicesPtype_); - } - if (valuesIdxOffsetsLen_ != 0L) { - size += com.google.protobuf.CodedOutputStream - .computeUInt64Size(4, valuesIdxOffsetsLen_); - } - if (valuesIdxOffsetsPtype_ != io.github.dfa1.vortex.proto.DTypeProtos.PType.U8.getNumber()) { - size += com.google.protobuf.CodedOutputStream - .computeEnumSize(5, valuesIdxOffsetsPtype_); - } - if (offset_ != 0L) { - size += com.google.protobuf.CodedOutputStream - .computeUInt64Size(6, offset_); - } - return size; - } - @java.lang.Override - public int getSerializedSize() { - int size = memoizedSize; - if (size != -1) return size; - - size = 0; - size += computeSerializedSize_0(); - size += getUnknownFields().getSerializedSize(); - memoizedSize = size; - return size; - } - - @java.lang.Override - public boolean equals(final java.lang.Object obj) { - if (obj == this) { - return true; - } - if (!(obj instanceof io.github.dfa1.vortex.proto.EncodingProtos.RLEMetadata)) { - return super.equals(obj); - } - io.github.dfa1.vortex.proto.EncodingProtos.RLEMetadata other = (io.github.dfa1.vortex.proto.EncodingProtos.RLEMetadata) obj; - - if (getValuesLen() - != other.getValuesLen()) return false; - if (getIndicesLen() - != other.getIndicesLen()) return false; - if (indicesPtype_ != other.indicesPtype_) return false; - if (getValuesIdxOffsetsLen() - != other.getValuesIdxOffsetsLen()) return false; - if (valuesIdxOffsetsPtype_ != other.valuesIdxOffsetsPtype_) return false; - if (getOffset() - != other.getOffset()) return false; - if (!getUnknownFields().equals(other.getUnknownFields())) return false; - return true; - } - - @java.lang.Override - public int hashCode() { - if (memoizedHashCode != 0) { - return memoizedHashCode; - } - int hash = 41; - hash = (19 * hash) + getDescriptor().hashCode(); - hash = (37 * hash) + VALUES_LEN_FIELD_NUMBER; - hash = (53 * hash) + com.google.protobuf.Internal.hashLong( - getValuesLen()); - hash = (37 * hash) + INDICES_LEN_FIELD_NUMBER; - hash = (53 * hash) + com.google.protobuf.Internal.hashLong( - getIndicesLen()); - hash = (37 * hash) + INDICES_PTYPE_FIELD_NUMBER; - hash = (53 * hash) + indicesPtype_; - hash = (37 * hash) + VALUES_IDX_OFFSETS_LEN_FIELD_NUMBER; - hash = (53 * hash) + com.google.protobuf.Internal.hashLong( - getValuesIdxOffsetsLen()); - hash = (37 * hash) + VALUES_IDX_OFFSETS_PTYPE_FIELD_NUMBER; - hash = (53 * hash) + valuesIdxOffsetsPtype_; - hash = (37 * hash) + OFFSET_FIELD_NUMBER; - hash = (53 * hash) + com.google.protobuf.Internal.hashLong( - getOffset()); - hash = (29 * hash) + getUnknownFields().hashCode(); - memoizedHashCode = hash; - return hash; - } - - public static io.github.dfa1.vortex.proto.EncodingProtos.RLEMetadata parseFrom( - java.nio.ByteBuffer data) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.RLEMetadata parseFrom( - java.nio.ByteBuffer data, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data, extensionRegistry); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.RLEMetadata parseFrom( - com.google.protobuf.ByteString data) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.RLEMetadata parseFrom( - com.google.protobuf.ByteString data, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data, extensionRegistry); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.RLEMetadata parseFrom(byte[] data) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.RLEMetadata parseFrom( - byte[] data, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data, extensionRegistry); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.RLEMetadata parseFrom(java.io.InputStream input) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseWithIOException(PARSER, input); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.RLEMetadata parseFrom( - java.io.InputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseWithIOException(PARSER, input, extensionRegistry); - } - - public static io.github.dfa1.vortex.proto.EncodingProtos.RLEMetadata parseDelimitedFrom(java.io.InputStream input) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseDelimitedWithIOException(PARSER, input); - } - - public static io.github.dfa1.vortex.proto.EncodingProtos.RLEMetadata parseDelimitedFrom( - java.io.InputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseDelimitedWithIOException(PARSER, input, extensionRegistry); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.RLEMetadata parseFrom( - com.google.protobuf.CodedInputStream input) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseWithIOException(PARSER, input); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.RLEMetadata parseFrom( - com.google.protobuf.CodedInputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseWithIOException(PARSER, input, extensionRegistry); - } - - @java.lang.Override - public Builder newBuilderForType() { return newBuilder(); } - public static Builder newBuilder() { - return DEFAULT_INSTANCE.toBuilder(); - } - public static Builder newBuilder(io.github.dfa1.vortex.proto.EncodingProtos.RLEMetadata prototype) { - return DEFAULT_INSTANCE.toBuilder().mergeFrom(prototype); - } - @java.lang.Override - public Builder toBuilder() { - return this == DEFAULT_INSTANCE - ? new Builder() : new Builder().mergeFrom(this); - } - - @java.lang.Override - protected Builder newBuilderForType( - com.google.protobuf.GeneratedMessage.BuilderParent parent) { - Builder builder = new Builder(parent); - return builder; - } - /** - * Protobuf type {@code vortex.encodings.RLEMetadata} - */ - public static final class Builder extends - com.google.protobuf.GeneratedMessage.Builder implements - // @@protoc_insertion_point(builder_implements:vortex.encodings.RLEMetadata) - io.github.dfa1.vortex.proto.EncodingProtos.RLEMetadataOrBuilder { - public static final com.google.protobuf.Descriptors.Descriptor - getDescriptor() { - return io.github.dfa1.vortex.proto.EncodingProtos.internal_static_vortex_encodings_RLEMetadata_descriptor; - } - - @java.lang.Override - protected com.google.protobuf.GeneratedMessage.FieldAccessorTable - internalGetFieldAccessorTable() { - return io.github.dfa1.vortex.proto.EncodingProtos.internal_static_vortex_encodings_RLEMetadata_fieldAccessorTable - .ensureFieldAccessorsInitialized( - io.github.dfa1.vortex.proto.EncodingProtos.RLEMetadata.class, io.github.dfa1.vortex.proto.EncodingProtos.RLEMetadata.Builder.class); - } - - // Construct using io.github.dfa1.vortex.proto.EncodingProtos.RLEMetadata.newBuilder() - private Builder() { - - } - - private Builder( - com.google.protobuf.GeneratedMessage.BuilderParent parent) { - super(parent); - - } - @java.lang.Override - public Builder clear() { - super.clear(); - bitField0_ = 0; - valuesLen_ = 0L; - indicesLen_ = 0L; - indicesPtype_ = 0; - valuesIdxOffsetsLen_ = 0L; - valuesIdxOffsetsPtype_ = 0; - offset_ = 0L; - return this; - } - - @java.lang.Override - public com.google.protobuf.Descriptors.Descriptor - getDescriptorForType() { - return io.github.dfa1.vortex.proto.EncodingProtos.internal_static_vortex_encodings_RLEMetadata_descriptor; - } - - @java.lang.Override - public io.github.dfa1.vortex.proto.EncodingProtos.RLEMetadata getDefaultInstanceForType() { - return io.github.dfa1.vortex.proto.EncodingProtos.RLEMetadata.getDefaultInstance(); - } - - @java.lang.Override - public io.github.dfa1.vortex.proto.EncodingProtos.RLEMetadata build() { - io.github.dfa1.vortex.proto.EncodingProtos.RLEMetadata result = buildPartial(); - if (!result.isInitialized()) { - throw newUninitializedMessageException(result); - } - return result; - } - - @java.lang.Override - public io.github.dfa1.vortex.proto.EncodingProtos.RLEMetadata buildPartial() { - io.github.dfa1.vortex.proto.EncodingProtos.RLEMetadata result = new io.github.dfa1.vortex.proto.EncodingProtos.RLEMetadata(this); - if (bitField0_ != 0) { buildPartial0(result); } - onBuilt(); - return result; - } - - private void buildPartial0(io.github.dfa1.vortex.proto.EncodingProtos.RLEMetadata result) { - int from_bitField0_ = bitField0_; - if (((from_bitField0_ & 0x00000001) != 0)) { - result.valuesLen_ = valuesLen_; - } - if (((from_bitField0_ & 0x00000002) != 0)) { - result.indicesLen_ = indicesLen_; - } - if (((from_bitField0_ & 0x00000004) != 0)) { - result.indicesPtype_ = indicesPtype_; - } - if (((from_bitField0_ & 0x00000008) != 0)) { - result.valuesIdxOffsetsLen_ = valuesIdxOffsetsLen_; - } - if (((from_bitField0_ & 0x00000010) != 0)) { - result.valuesIdxOffsetsPtype_ = valuesIdxOffsetsPtype_; - } - if (((from_bitField0_ & 0x00000020) != 0)) { - result.offset_ = offset_; - } - } - - @java.lang.Override - public Builder mergeFrom(com.google.protobuf.Message other) { - if (other instanceof io.github.dfa1.vortex.proto.EncodingProtos.RLEMetadata) { - return mergeFrom((io.github.dfa1.vortex.proto.EncodingProtos.RLEMetadata)other); - } else { - super.mergeFrom(other); - return this; - } - } - - public Builder mergeFrom(io.github.dfa1.vortex.proto.EncodingProtos.RLEMetadata other) { - if (other == io.github.dfa1.vortex.proto.EncodingProtos.RLEMetadata.getDefaultInstance()) return this; - if (other.getValuesLen() != 0L) { - setValuesLen(other.getValuesLen()); - } - if (other.getIndicesLen() != 0L) { - setIndicesLen(other.getIndicesLen()); - } - if (other.indicesPtype_ != 0) { - setIndicesPtypeValue(other.getIndicesPtypeValue()); - } - if (other.getValuesIdxOffsetsLen() != 0L) { - setValuesIdxOffsetsLen(other.getValuesIdxOffsetsLen()); - } - if (other.valuesIdxOffsetsPtype_ != 0) { - setValuesIdxOffsetsPtypeValue(other.getValuesIdxOffsetsPtypeValue()); - } - if (other.getOffset() != 0L) { - setOffset(other.getOffset()); - } - this.mergeUnknownFields(other.getUnknownFields()); - onChanged(); - return this; - } - - @java.lang.Override - public final boolean isInitialized() { - return true; - } - - @java.lang.Override - public Builder mergeFrom( - com.google.protobuf.CodedInputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws java.io.IOException { - if (extensionRegistry == null) { - throw new java.lang.NullPointerException(); - } - try { - boolean done = false; - while (!done) { - int tag = input.readTag(); - switch (tag) { - case 0: - done = true; - break; - case 8: { - valuesLen_ = input.readUInt64(); - bitField0_ |= 0x00000001; - break; - } // case 8 - case 16: { - indicesLen_ = input.readUInt64(); - bitField0_ |= 0x00000002; - break; - } // case 16 - case 24: { - indicesPtype_ = input.readEnum(); - bitField0_ |= 0x00000004; - break; - } // case 24 - case 32: { - valuesIdxOffsetsLen_ = input.readUInt64(); - bitField0_ |= 0x00000008; - break; - } // case 32 - case 40: { - valuesIdxOffsetsPtype_ = input.readEnum(); - bitField0_ |= 0x00000010; - break; - } // case 40 - case 48: { - offset_ = input.readUInt64(); - bitField0_ |= 0x00000020; - break; - } // case 48 - default: { - if (!super.parseUnknownField(input, extensionRegistry, tag)) { - done = true; // was an endgroup tag - } - break; - } // default: - } // switch (tag) - } // while (!done) - } catch (com.google.protobuf.InvalidProtocolBufferException e) { - throw e.unwrapIOException(); - } finally { - onChanged(); - } // finally - return this; - } - private int bitField0_; - - private long valuesLen_ ; - /** - * uint64 values_len = 1; - * @return The valuesLen. - */ - @java.lang.Override - public long getValuesLen() { - return valuesLen_; - } - /** - * uint64 values_len = 1; - * @param value The valuesLen to set. - * @return This builder for chaining. - */ - public Builder setValuesLen(long value) { - - valuesLen_ = value; - bitField0_ |= 0x00000001; - onChanged(); - return this; - } - /** - * uint64 values_len = 1; - * @return This builder for chaining. - */ - public Builder clearValuesLen() { - bitField0_ = (bitField0_ & ~0x00000001); - valuesLen_ = 0L; - onChanged(); - return this; - } - - private long indicesLen_ ; - /** - * uint64 indices_len = 2; - * @return The indicesLen. - */ - @java.lang.Override - public long getIndicesLen() { - return indicesLen_; - } - /** - * uint64 indices_len = 2; - * @param value The indicesLen to set. - * @return This builder for chaining. - */ - public Builder setIndicesLen(long value) { - - indicesLen_ = value; - bitField0_ |= 0x00000002; - onChanged(); - return this; - } - /** - * uint64 indices_len = 2; - * @return This builder for chaining. - */ - public Builder clearIndicesLen() { - bitField0_ = (bitField0_ & ~0x00000002); - indicesLen_ = 0L; - onChanged(); - return this; - } - - private int indicesPtype_ = 0; - /** - * .vortex.dtype.PType indices_ptype = 3; - * @return The enum numeric value on the wire for indicesPtype. - */ - @java.lang.Override public int getIndicesPtypeValue() { - return indicesPtype_; - } - /** - * .vortex.dtype.PType indices_ptype = 3; - * @param value The enum numeric value on the wire for indicesPtype to set. - * @return This builder for chaining. - */ - public Builder setIndicesPtypeValue(int value) { - indicesPtype_ = value; - bitField0_ |= 0x00000004; - onChanged(); - return this; - } - /** - * .vortex.dtype.PType indices_ptype = 3; - * @return The indicesPtype. - */ - @java.lang.Override - public io.github.dfa1.vortex.proto.DTypeProtos.PType getIndicesPtype() { - io.github.dfa1.vortex.proto.DTypeProtos.PType result = io.github.dfa1.vortex.proto.DTypeProtos.PType.forNumber(indicesPtype_); - return result == null ? io.github.dfa1.vortex.proto.DTypeProtos.PType.UNRECOGNIZED : result; - } - /** - * .vortex.dtype.PType indices_ptype = 3; - * @param value The indicesPtype to set. - * @throws IllegalArgumentException if UNRECOGNIZED is provided. - * @return This builder for chaining. - */ - public Builder setIndicesPtype(io.github.dfa1.vortex.proto.DTypeProtos.PType value) { - if (value == null) { throw new NullPointerException(); } - bitField0_ |= 0x00000004; - indicesPtype_ = value.getNumber(); - onChanged(); - return this; - } - /** - * .vortex.dtype.PType indices_ptype = 3; - * @return This builder for chaining. - */ - public Builder clearIndicesPtype() { - bitField0_ = (bitField0_ & ~0x00000004); - indicesPtype_ = 0; - onChanged(); - return this; - } - - private long valuesIdxOffsetsLen_ ; - /** - * uint64 values_idx_offsets_len = 4; - * @return The valuesIdxOffsetsLen. - */ - @java.lang.Override - public long getValuesIdxOffsetsLen() { - return valuesIdxOffsetsLen_; - } - /** - * uint64 values_idx_offsets_len = 4; - * @param value The valuesIdxOffsetsLen to set. - * @return This builder for chaining. - */ - public Builder setValuesIdxOffsetsLen(long value) { - - valuesIdxOffsetsLen_ = value; - bitField0_ |= 0x00000008; - onChanged(); - return this; - } - /** - * uint64 values_idx_offsets_len = 4; - * @return This builder for chaining. - */ - public Builder clearValuesIdxOffsetsLen() { - bitField0_ = (bitField0_ & ~0x00000008); - valuesIdxOffsetsLen_ = 0L; - onChanged(); - return this; - } - - private int valuesIdxOffsetsPtype_ = 0; - /** - * .vortex.dtype.PType values_idx_offsets_ptype = 5; - * @return The enum numeric value on the wire for valuesIdxOffsetsPtype. - */ - @java.lang.Override public int getValuesIdxOffsetsPtypeValue() { - return valuesIdxOffsetsPtype_; - } - /** - * .vortex.dtype.PType values_idx_offsets_ptype = 5; - * @param value The enum numeric value on the wire for valuesIdxOffsetsPtype to set. - * @return This builder for chaining. - */ - public Builder setValuesIdxOffsetsPtypeValue(int value) { - valuesIdxOffsetsPtype_ = value; - bitField0_ |= 0x00000010; - onChanged(); - return this; - } - /** - * .vortex.dtype.PType values_idx_offsets_ptype = 5; - * @return The valuesIdxOffsetsPtype. - */ - @java.lang.Override - public io.github.dfa1.vortex.proto.DTypeProtos.PType getValuesIdxOffsetsPtype() { - io.github.dfa1.vortex.proto.DTypeProtos.PType result = io.github.dfa1.vortex.proto.DTypeProtos.PType.forNumber(valuesIdxOffsetsPtype_); - return result == null ? io.github.dfa1.vortex.proto.DTypeProtos.PType.UNRECOGNIZED : result; - } - /** - * .vortex.dtype.PType values_idx_offsets_ptype = 5; - * @param value The valuesIdxOffsetsPtype to set. - * @throws IllegalArgumentException if UNRECOGNIZED is provided. - * @return This builder for chaining. - */ - public Builder setValuesIdxOffsetsPtype(io.github.dfa1.vortex.proto.DTypeProtos.PType value) { - if (value == null) { throw new NullPointerException(); } - bitField0_ |= 0x00000010; - valuesIdxOffsetsPtype_ = value.getNumber(); - onChanged(); - return this; - } - /** - * .vortex.dtype.PType values_idx_offsets_ptype = 5; - * @return This builder for chaining. - */ - public Builder clearValuesIdxOffsetsPtype() { - bitField0_ = (bitField0_ & ~0x00000010); - valuesIdxOffsetsPtype_ = 0; - onChanged(); - return this; - } - - private long offset_ ; - /** - * uint64 offset = 6; - * @return The offset. - */ - @java.lang.Override - public long getOffset() { - return offset_; - } - /** - * uint64 offset = 6; - * @param value The offset to set. - * @return This builder for chaining. - */ - public Builder setOffset(long value) { - - offset_ = value; - bitField0_ |= 0x00000020; - onChanged(); - return this; - } - /** - * uint64 offset = 6; - * @return This builder for chaining. - */ - public Builder clearOffset() { - bitField0_ = (bitField0_ & ~0x00000020); - offset_ = 0L; - onChanged(); - return this; - } - - // @@protoc_insertion_point(builder_scope:vortex.encodings.RLEMetadata) - } - - // @@protoc_insertion_point(class_scope:vortex.encodings.RLEMetadata) - private static final io.github.dfa1.vortex.proto.EncodingProtos.RLEMetadata DEFAULT_INSTANCE; - static { - DEFAULT_INSTANCE = new io.github.dfa1.vortex.proto.EncodingProtos.RLEMetadata(); - } - - public static io.github.dfa1.vortex.proto.EncodingProtos.RLEMetadata getDefaultInstance() { - return DEFAULT_INSTANCE; - } - - private static final com.google.protobuf.Parser - PARSER = new com.google.protobuf.AbstractParser() { - @java.lang.Override - public RLEMetadata parsePartialFrom( - com.google.protobuf.CodedInputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws com.google.protobuf.InvalidProtocolBufferException { - Builder builder = newBuilder(); - try { - builder.mergeFrom(input, extensionRegistry); - } catch (com.google.protobuf.InvalidProtocolBufferException e) { - throw e.setUnfinishedMessage(builder.buildPartial()); - } catch (com.google.protobuf.UninitializedMessageException e) { - throw e.asInvalidProtocolBufferException().setUnfinishedMessage(builder.buildPartial()); - } catch (java.io.IOException e) { - throw new com.google.protobuf.InvalidProtocolBufferException(e) - .setUnfinishedMessage(builder.buildPartial()); - } - return builder.buildPartial(); - } - }; - - public static com.google.protobuf.Parser parser() { - return PARSER; - } - - @java.lang.Override - public com.google.protobuf.Parser getParserForType() { - return PARSER; - } - - @java.lang.Override - public io.github.dfa1.vortex.proto.EncodingProtos.RLEMetadata getDefaultInstanceForType() { - return DEFAULT_INSTANCE; - } - - } - - public interface ListMetadataOrBuilder extends - // @@protoc_insertion_point(interface_extends:vortex.encodings.ListMetadata) - com.google.protobuf.MessageOrBuilder { - - /** - * uint64 elements_len = 1; - * @return The elementsLen. - */ - long getElementsLen(); - - /** - * .vortex.dtype.PType offset_ptype = 2; - * @return The enum numeric value on the wire for offsetPtype. - */ - int getOffsetPtypeValue(); - /** - * .vortex.dtype.PType offset_ptype = 2; - * @return The offsetPtype. - */ - io.github.dfa1.vortex.proto.DTypeProtos.PType getOffsetPtype(); - } - /** - * Protobuf type {@code vortex.encodings.ListMetadata} - */ - public static final class ListMetadata extends - com.google.protobuf.GeneratedMessage implements - // @@protoc_insertion_point(message_implements:vortex.encodings.ListMetadata) - ListMetadataOrBuilder { - private static final long serialVersionUID = 0L; - static { - com.google.protobuf.RuntimeVersion.validateProtobufGencodeVersion( - com.google.protobuf.RuntimeVersion.RuntimeDomain.PUBLIC, - /* major= */ 4, - /* minor= */ 35, - /* patch= */ 0, - /* suffix= */ "", - "ListMetadata"); - } - // Use ListMetadata.newBuilder() to construct. - private ListMetadata(com.google.protobuf.GeneratedMessage.Builder builder) { - super(builder); - } - private ListMetadata() { - offsetPtype_ = 0; - } - - public static final com.google.protobuf.Descriptors.Descriptor - getDescriptor() { - return io.github.dfa1.vortex.proto.EncodingProtos.internal_static_vortex_encodings_ListMetadata_descriptor; - } - - @java.lang.Override - public com.google.protobuf.Descriptors.Descriptor getDescriptorForType() { - return io.github.dfa1.vortex.proto.EncodingProtos.internal_static_vortex_encodings_ListMetadata_descriptor; - } - - @java.lang.Override - protected com.google.protobuf.GeneratedMessage.FieldAccessorTable - internalGetFieldAccessorTable() { - return io.github.dfa1.vortex.proto.EncodingProtos.internal_static_vortex_encodings_ListMetadata_fieldAccessorTable - .ensureFieldAccessorsInitialized( - io.github.dfa1.vortex.proto.EncodingProtos.ListMetadata.class, io.github.dfa1.vortex.proto.EncodingProtos.ListMetadata.Builder.class); - } - - public static final int ELEMENTS_LEN_FIELD_NUMBER = 1; - private long elementsLen_ = 0L; - /** - * uint64 elements_len = 1; - * @return The elementsLen. - */ - @java.lang.Override - public long getElementsLen() { - return elementsLen_; - } - - public static final int OFFSET_PTYPE_FIELD_NUMBER = 2; - private int offsetPtype_ = 0; - /** - * .vortex.dtype.PType offset_ptype = 2; - * @return The enum numeric value on the wire for offsetPtype. - */ - @java.lang.Override public int getOffsetPtypeValue() { - return offsetPtype_; - } - /** - * .vortex.dtype.PType offset_ptype = 2; - * @return The offsetPtype. - */ - @java.lang.Override public io.github.dfa1.vortex.proto.DTypeProtos.PType getOffsetPtype() { - io.github.dfa1.vortex.proto.DTypeProtos.PType result = io.github.dfa1.vortex.proto.DTypeProtos.PType.forNumber(offsetPtype_); - return result == null ? io.github.dfa1.vortex.proto.DTypeProtos.PType.UNRECOGNIZED : result; - } - - private byte memoizedIsInitialized = -1; - @java.lang.Override - public final boolean isInitialized() { - byte isInitialized = memoizedIsInitialized; - if (isInitialized == 1) return true; - if (isInitialized == 0) return false; - - memoizedIsInitialized = 1; - return true; - } - - @java.lang.Override - public void writeTo(com.google.protobuf.CodedOutputStream output) - throws java.io.IOException { - if (elementsLen_ != 0L) { - output.writeUInt64(1, elementsLen_); - } - if (offsetPtype_ != io.github.dfa1.vortex.proto.DTypeProtos.PType.U8.getNumber()) { - output.writeEnum(2, offsetPtype_); - } - getUnknownFields().writeTo(output); - } - private int computeSerializedSize_0() { - int size = 0; - if (elementsLen_ != 0L) { - size += com.google.protobuf.CodedOutputStream - .computeUInt64Size(1, elementsLen_); - } - if (offsetPtype_ != io.github.dfa1.vortex.proto.DTypeProtos.PType.U8.getNumber()) { - size += com.google.protobuf.CodedOutputStream - .computeEnumSize(2, offsetPtype_); - } - return size; - } - @java.lang.Override - public int getSerializedSize() { - int size = memoizedSize; - if (size != -1) return size; - - size = 0; - size += computeSerializedSize_0(); - size += getUnknownFields().getSerializedSize(); - memoizedSize = size; - return size; - } - - @java.lang.Override - public boolean equals(final java.lang.Object obj) { - if (obj == this) { - return true; - } - if (!(obj instanceof io.github.dfa1.vortex.proto.EncodingProtos.ListMetadata)) { - return super.equals(obj); - } - io.github.dfa1.vortex.proto.EncodingProtos.ListMetadata other = (io.github.dfa1.vortex.proto.EncodingProtos.ListMetadata) obj; - - if (getElementsLen() - != other.getElementsLen()) return false; - if (offsetPtype_ != other.offsetPtype_) return false; - if (!getUnknownFields().equals(other.getUnknownFields())) return false; - return true; - } - - @java.lang.Override - public int hashCode() { - if (memoizedHashCode != 0) { - return memoizedHashCode; - } - int hash = 41; - hash = (19 * hash) + getDescriptor().hashCode(); - hash = (37 * hash) + ELEMENTS_LEN_FIELD_NUMBER; - hash = (53 * hash) + com.google.protobuf.Internal.hashLong( - getElementsLen()); - hash = (37 * hash) + OFFSET_PTYPE_FIELD_NUMBER; - hash = (53 * hash) + offsetPtype_; - hash = (29 * hash) + getUnknownFields().hashCode(); - memoizedHashCode = hash; - return hash; - } - - public static io.github.dfa1.vortex.proto.EncodingProtos.ListMetadata parseFrom( - java.nio.ByteBuffer data) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.ListMetadata parseFrom( - java.nio.ByteBuffer data, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data, extensionRegistry); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.ListMetadata parseFrom( - com.google.protobuf.ByteString data) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.ListMetadata parseFrom( - com.google.protobuf.ByteString data, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data, extensionRegistry); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.ListMetadata parseFrom(byte[] data) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.ListMetadata parseFrom( - byte[] data, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data, extensionRegistry); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.ListMetadata parseFrom(java.io.InputStream input) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseWithIOException(PARSER, input); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.ListMetadata parseFrom( - java.io.InputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseWithIOException(PARSER, input, extensionRegistry); - } - - public static io.github.dfa1.vortex.proto.EncodingProtos.ListMetadata parseDelimitedFrom(java.io.InputStream input) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseDelimitedWithIOException(PARSER, input); - } - - public static io.github.dfa1.vortex.proto.EncodingProtos.ListMetadata parseDelimitedFrom( - java.io.InputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseDelimitedWithIOException(PARSER, input, extensionRegistry); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.ListMetadata parseFrom( - com.google.protobuf.CodedInputStream input) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseWithIOException(PARSER, input); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.ListMetadata parseFrom( - com.google.protobuf.CodedInputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseWithIOException(PARSER, input, extensionRegistry); - } - - @java.lang.Override - public Builder newBuilderForType() { return newBuilder(); } - public static Builder newBuilder() { - return DEFAULT_INSTANCE.toBuilder(); - } - public static Builder newBuilder(io.github.dfa1.vortex.proto.EncodingProtos.ListMetadata prototype) { - return DEFAULT_INSTANCE.toBuilder().mergeFrom(prototype); - } - @java.lang.Override - public Builder toBuilder() { - return this == DEFAULT_INSTANCE - ? new Builder() : new Builder().mergeFrom(this); - } - - @java.lang.Override - protected Builder newBuilderForType( - com.google.protobuf.GeneratedMessage.BuilderParent parent) { - Builder builder = new Builder(parent); - return builder; - } - /** - * Protobuf type {@code vortex.encodings.ListMetadata} - */ - public static final class Builder extends - com.google.protobuf.GeneratedMessage.Builder implements - // @@protoc_insertion_point(builder_implements:vortex.encodings.ListMetadata) - io.github.dfa1.vortex.proto.EncodingProtos.ListMetadataOrBuilder { - public static final com.google.protobuf.Descriptors.Descriptor - getDescriptor() { - return io.github.dfa1.vortex.proto.EncodingProtos.internal_static_vortex_encodings_ListMetadata_descriptor; - } - - @java.lang.Override - protected com.google.protobuf.GeneratedMessage.FieldAccessorTable - internalGetFieldAccessorTable() { - return io.github.dfa1.vortex.proto.EncodingProtos.internal_static_vortex_encodings_ListMetadata_fieldAccessorTable - .ensureFieldAccessorsInitialized( - io.github.dfa1.vortex.proto.EncodingProtos.ListMetadata.class, io.github.dfa1.vortex.proto.EncodingProtos.ListMetadata.Builder.class); - } - - // Construct using io.github.dfa1.vortex.proto.EncodingProtos.ListMetadata.newBuilder() - private Builder() { - - } - - private Builder( - com.google.protobuf.GeneratedMessage.BuilderParent parent) { - super(parent); - - } - @java.lang.Override - public Builder clear() { - super.clear(); - bitField0_ = 0; - elementsLen_ = 0L; - offsetPtype_ = 0; - return this; - } - - @java.lang.Override - public com.google.protobuf.Descriptors.Descriptor - getDescriptorForType() { - return io.github.dfa1.vortex.proto.EncodingProtos.internal_static_vortex_encodings_ListMetadata_descriptor; - } - - @java.lang.Override - public io.github.dfa1.vortex.proto.EncodingProtos.ListMetadata getDefaultInstanceForType() { - return io.github.dfa1.vortex.proto.EncodingProtos.ListMetadata.getDefaultInstance(); - } - - @java.lang.Override - public io.github.dfa1.vortex.proto.EncodingProtos.ListMetadata build() { - io.github.dfa1.vortex.proto.EncodingProtos.ListMetadata result = buildPartial(); - if (!result.isInitialized()) { - throw newUninitializedMessageException(result); - } - return result; - } - - @java.lang.Override - public io.github.dfa1.vortex.proto.EncodingProtos.ListMetadata buildPartial() { - io.github.dfa1.vortex.proto.EncodingProtos.ListMetadata result = new io.github.dfa1.vortex.proto.EncodingProtos.ListMetadata(this); - if (bitField0_ != 0) { buildPartial0(result); } - onBuilt(); - return result; - } - - private void buildPartial0(io.github.dfa1.vortex.proto.EncodingProtos.ListMetadata result) { - int from_bitField0_ = bitField0_; - if (((from_bitField0_ & 0x00000001) != 0)) { - result.elementsLen_ = elementsLen_; - } - if (((from_bitField0_ & 0x00000002) != 0)) { - result.offsetPtype_ = offsetPtype_; - } - } - - @java.lang.Override - public Builder mergeFrom(com.google.protobuf.Message other) { - if (other instanceof io.github.dfa1.vortex.proto.EncodingProtos.ListMetadata) { - return mergeFrom((io.github.dfa1.vortex.proto.EncodingProtos.ListMetadata)other); - } else { - super.mergeFrom(other); - return this; - } - } - - public Builder mergeFrom(io.github.dfa1.vortex.proto.EncodingProtos.ListMetadata other) { - if (other == io.github.dfa1.vortex.proto.EncodingProtos.ListMetadata.getDefaultInstance()) return this; - if (other.getElementsLen() != 0L) { - setElementsLen(other.getElementsLen()); - } - if (other.offsetPtype_ != 0) { - setOffsetPtypeValue(other.getOffsetPtypeValue()); - } - this.mergeUnknownFields(other.getUnknownFields()); - onChanged(); - return this; - } - - @java.lang.Override - public final boolean isInitialized() { - return true; - } - - @java.lang.Override - public Builder mergeFrom( - com.google.protobuf.CodedInputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws java.io.IOException { - if (extensionRegistry == null) { - throw new java.lang.NullPointerException(); - } - try { - boolean done = false; - while (!done) { - int tag = input.readTag(); - switch (tag) { - case 0: - done = true; - break; - case 8: { - elementsLen_ = input.readUInt64(); - bitField0_ |= 0x00000001; - break; - } // case 8 - case 16: { - offsetPtype_ = input.readEnum(); - bitField0_ |= 0x00000002; - break; - } // case 16 - default: { - if (!super.parseUnknownField(input, extensionRegistry, tag)) { - done = true; // was an endgroup tag - } - break; - } // default: - } // switch (tag) - } // while (!done) - } catch (com.google.protobuf.InvalidProtocolBufferException e) { - throw e.unwrapIOException(); - } finally { - onChanged(); - } // finally - return this; - } - private int bitField0_; - - private long elementsLen_ ; - /** - * uint64 elements_len = 1; - * @return The elementsLen. - */ - @java.lang.Override - public long getElementsLen() { - return elementsLen_; - } - /** - * uint64 elements_len = 1; - * @param value The elementsLen to set. - * @return This builder for chaining. - */ - public Builder setElementsLen(long value) { - - elementsLen_ = value; - bitField0_ |= 0x00000001; - onChanged(); - return this; - } - /** - * uint64 elements_len = 1; - * @return This builder for chaining. - */ - public Builder clearElementsLen() { - bitField0_ = (bitField0_ & ~0x00000001); - elementsLen_ = 0L; - onChanged(); - return this; - } - - private int offsetPtype_ = 0; - /** - * .vortex.dtype.PType offset_ptype = 2; - * @return The enum numeric value on the wire for offsetPtype. - */ - @java.lang.Override public int getOffsetPtypeValue() { - return offsetPtype_; - } - /** - * .vortex.dtype.PType offset_ptype = 2; - * @param value The enum numeric value on the wire for offsetPtype to set. - * @return This builder for chaining. - */ - public Builder setOffsetPtypeValue(int value) { - offsetPtype_ = value; - bitField0_ |= 0x00000002; - onChanged(); - return this; - } - /** - * .vortex.dtype.PType offset_ptype = 2; - * @return The offsetPtype. - */ - @java.lang.Override - public io.github.dfa1.vortex.proto.DTypeProtos.PType getOffsetPtype() { - io.github.dfa1.vortex.proto.DTypeProtos.PType result = io.github.dfa1.vortex.proto.DTypeProtos.PType.forNumber(offsetPtype_); - return result == null ? io.github.dfa1.vortex.proto.DTypeProtos.PType.UNRECOGNIZED : result; - } - /** - * .vortex.dtype.PType offset_ptype = 2; - * @param value The offsetPtype to set. - * @throws IllegalArgumentException if UNRECOGNIZED is provided. - * @return This builder for chaining. - */ - public Builder setOffsetPtype(io.github.dfa1.vortex.proto.DTypeProtos.PType value) { - if (value == null) { throw new NullPointerException(); } - bitField0_ |= 0x00000002; - offsetPtype_ = value.getNumber(); - onChanged(); - return this; - } - /** - * .vortex.dtype.PType offset_ptype = 2; - * @return This builder for chaining. - */ - public Builder clearOffsetPtype() { - bitField0_ = (bitField0_ & ~0x00000002); - offsetPtype_ = 0; - onChanged(); - return this; - } - - // @@protoc_insertion_point(builder_scope:vortex.encodings.ListMetadata) - } - - // @@protoc_insertion_point(class_scope:vortex.encodings.ListMetadata) - private static final io.github.dfa1.vortex.proto.EncodingProtos.ListMetadata DEFAULT_INSTANCE; - static { - DEFAULT_INSTANCE = new io.github.dfa1.vortex.proto.EncodingProtos.ListMetadata(); - } - - public static io.github.dfa1.vortex.proto.EncodingProtos.ListMetadata getDefaultInstance() { - return DEFAULT_INSTANCE; - } - - private static final com.google.protobuf.Parser - PARSER = new com.google.protobuf.AbstractParser() { - @java.lang.Override - public ListMetadata parsePartialFrom( - com.google.protobuf.CodedInputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws com.google.protobuf.InvalidProtocolBufferException { - Builder builder = newBuilder(); - try { - builder.mergeFrom(input, extensionRegistry); - } catch (com.google.protobuf.InvalidProtocolBufferException e) { - throw e.setUnfinishedMessage(builder.buildPartial()); - } catch (com.google.protobuf.UninitializedMessageException e) { - throw e.asInvalidProtocolBufferException().setUnfinishedMessage(builder.buildPartial()); - } catch (java.io.IOException e) { - throw new com.google.protobuf.InvalidProtocolBufferException(e) - .setUnfinishedMessage(builder.buildPartial()); - } - return builder.buildPartial(); - } - }; - - public static com.google.protobuf.Parser parser() { - return PARSER; - } - - @java.lang.Override - public com.google.protobuf.Parser getParserForType() { - return PARSER; - } - - @java.lang.Override - public io.github.dfa1.vortex.proto.EncodingProtos.ListMetadata getDefaultInstanceForType() { - return DEFAULT_INSTANCE; - } - - } - - public interface ListViewMetadataOrBuilder extends - // @@protoc_insertion_point(interface_extends:vortex.encodings.ListViewMetadata) - com.google.protobuf.MessageOrBuilder { - - /** - * uint64 elements_len = 1; - * @return The elementsLen. - */ - long getElementsLen(); - - /** - * .vortex.dtype.PType offset_ptype = 2; - * @return The enum numeric value on the wire for offsetPtype. - */ - int getOffsetPtypeValue(); - /** - * .vortex.dtype.PType offset_ptype = 2; - * @return The offsetPtype. - */ - io.github.dfa1.vortex.proto.DTypeProtos.PType getOffsetPtype(); - - /** - * .vortex.dtype.PType size_ptype = 3; - * @return The enum numeric value on the wire for sizePtype. - */ - int getSizePtypeValue(); - /** - * .vortex.dtype.PType size_ptype = 3; - * @return The sizePtype. - */ - io.github.dfa1.vortex.proto.DTypeProtos.PType getSizePtype(); - } - /** - * Protobuf type {@code vortex.encodings.ListViewMetadata} - */ - public static final class ListViewMetadata extends - com.google.protobuf.GeneratedMessage implements - // @@protoc_insertion_point(message_implements:vortex.encodings.ListViewMetadata) - ListViewMetadataOrBuilder { - private static final long serialVersionUID = 0L; - static { - com.google.protobuf.RuntimeVersion.validateProtobufGencodeVersion( - com.google.protobuf.RuntimeVersion.RuntimeDomain.PUBLIC, - /* major= */ 4, - /* minor= */ 35, - /* patch= */ 0, - /* suffix= */ "", - "ListViewMetadata"); - } - // Use ListViewMetadata.newBuilder() to construct. - private ListViewMetadata(com.google.protobuf.GeneratedMessage.Builder builder) { - super(builder); - } - private ListViewMetadata() { - offsetPtype_ = 0; - sizePtype_ = 0; - } - - public static final com.google.protobuf.Descriptors.Descriptor - getDescriptor() { - return io.github.dfa1.vortex.proto.EncodingProtos.internal_static_vortex_encodings_ListViewMetadata_descriptor; - } - - @java.lang.Override - public com.google.protobuf.Descriptors.Descriptor getDescriptorForType() { - return io.github.dfa1.vortex.proto.EncodingProtos.internal_static_vortex_encodings_ListViewMetadata_descriptor; - } - - @java.lang.Override - protected com.google.protobuf.GeneratedMessage.FieldAccessorTable - internalGetFieldAccessorTable() { - return io.github.dfa1.vortex.proto.EncodingProtos.internal_static_vortex_encodings_ListViewMetadata_fieldAccessorTable - .ensureFieldAccessorsInitialized( - io.github.dfa1.vortex.proto.EncodingProtos.ListViewMetadata.class, io.github.dfa1.vortex.proto.EncodingProtos.ListViewMetadata.Builder.class); - } - - public static final int ELEMENTS_LEN_FIELD_NUMBER = 1; - private long elementsLen_ = 0L; - /** - * uint64 elements_len = 1; - * @return The elementsLen. - */ - @java.lang.Override - public long getElementsLen() { - return elementsLen_; - } - - public static final int OFFSET_PTYPE_FIELD_NUMBER = 2; - private int offsetPtype_ = 0; - /** - * .vortex.dtype.PType offset_ptype = 2; - * @return The enum numeric value on the wire for offsetPtype. - */ - @java.lang.Override public int getOffsetPtypeValue() { - return offsetPtype_; - } - /** - * .vortex.dtype.PType offset_ptype = 2; - * @return The offsetPtype. - */ - @java.lang.Override public io.github.dfa1.vortex.proto.DTypeProtos.PType getOffsetPtype() { - io.github.dfa1.vortex.proto.DTypeProtos.PType result = io.github.dfa1.vortex.proto.DTypeProtos.PType.forNumber(offsetPtype_); - return result == null ? io.github.dfa1.vortex.proto.DTypeProtos.PType.UNRECOGNIZED : result; - } - - public static final int SIZE_PTYPE_FIELD_NUMBER = 3; - private int sizePtype_ = 0; - /** - * .vortex.dtype.PType size_ptype = 3; - * @return The enum numeric value on the wire for sizePtype. - */ - @java.lang.Override public int getSizePtypeValue() { - return sizePtype_; - } - /** - * .vortex.dtype.PType size_ptype = 3; - * @return The sizePtype. - */ - @java.lang.Override public io.github.dfa1.vortex.proto.DTypeProtos.PType getSizePtype() { - io.github.dfa1.vortex.proto.DTypeProtos.PType result = io.github.dfa1.vortex.proto.DTypeProtos.PType.forNumber(sizePtype_); - return result == null ? io.github.dfa1.vortex.proto.DTypeProtos.PType.UNRECOGNIZED : result; - } - - private byte memoizedIsInitialized = -1; - @java.lang.Override - public final boolean isInitialized() { - byte isInitialized = memoizedIsInitialized; - if (isInitialized == 1) return true; - if (isInitialized == 0) return false; - - memoizedIsInitialized = 1; - return true; - } - - @java.lang.Override - public void writeTo(com.google.protobuf.CodedOutputStream output) - throws java.io.IOException { - if (elementsLen_ != 0L) { - output.writeUInt64(1, elementsLen_); - } - if (offsetPtype_ != io.github.dfa1.vortex.proto.DTypeProtos.PType.U8.getNumber()) { - output.writeEnum(2, offsetPtype_); - } - if (sizePtype_ != io.github.dfa1.vortex.proto.DTypeProtos.PType.U8.getNumber()) { - output.writeEnum(3, sizePtype_); - } - getUnknownFields().writeTo(output); - } - private int computeSerializedSize_0() { - int size = 0; - if (elementsLen_ != 0L) { - size += com.google.protobuf.CodedOutputStream - .computeUInt64Size(1, elementsLen_); - } - if (offsetPtype_ != io.github.dfa1.vortex.proto.DTypeProtos.PType.U8.getNumber()) { - size += com.google.protobuf.CodedOutputStream - .computeEnumSize(2, offsetPtype_); - } - if (sizePtype_ != io.github.dfa1.vortex.proto.DTypeProtos.PType.U8.getNumber()) { - size += com.google.protobuf.CodedOutputStream - .computeEnumSize(3, sizePtype_); - } - return size; - } - @java.lang.Override - public int getSerializedSize() { - int size = memoizedSize; - if (size != -1) return size; - - size = 0; - size += computeSerializedSize_0(); - size += getUnknownFields().getSerializedSize(); - memoizedSize = size; - return size; - } - - @java.lang.Override - public boolean equals(final java.lang.Object obj) { - if (obj == this) { - return true; - } - if (!(obj instanceof io.github.dfa1.vortex.proto.EncodingProtos.ListViewMetadata)) { - return super.equals(obj); - } - io.github.dfa1.vortex.proto.EncodingProtos.ListViewMetadata other = (io.github.dfa1.vortex.proto.EncodingProtos.ListViewMetadata) obj; - - if (getElementsLen() - != other.getElementsLen()) return false; - if (offsetPtype_ != other.offsetPtype_) return false; - if (sizePtype_ != other.sizePtype_) return false; - if (!getUnknownFields().equals(other.getUnknownFields())) return false; - return true; - } - - @java.lang.Override - public int hashCode() { - if (memoizedHashCode != 0) { - return memoizedHashCode; - } - int hash = 41; - hash = (19 * hash) + getDescriptor().hashCode(); - hash = (37 * hash) + ELEMENTS_LEN_FIELD_NUMBER; - hash = (53 * hash) + com.google.protobuf.Internal.hashLong( - getElementsLen()); - hash = (37 * hash) + OFFSET_PTYPE_FIELD_NUMBER; - hash = (53 * hash) + offsetPtype_; - hash = (37 * hash) + SIZE_PTYPE_FIELD_NUMBER; - hash = (53 * hash) + sizePtype_; - hash = (29 * hash) + getUnknownFields().hashCode(); - memoizedHashCode = hash; - return hash; - } - - public static io.github.dfa1.vortex.proto.EncodingProtos.ListViewMetadata parseFrom( - java.nio.ByteBuffer data) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.ListViewMetadata parseFrom( - java.nio.ByteBuffer data, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data, extensionRegistry); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.ListViewMetadata parseFrom( - com.google.protobuf.ByteString data) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.ListViewMetadata parseFrom( - com.google.protobuf.ByteString data, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data, extensionRegistry); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.ListViewMetadata parseFrom(byte[] data) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.ListViewMetadata parseFrom( - byte[] data, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data, extensionRegistry); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.ListViewMetadata parseFrom(java.io.InputStream input) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseWithIOException(PARSER, input); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.ListViewMetadata parseFrom( - java.io.InputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseWithIOException(PARSER, input, extensionRegistry); - } - - public static io.github.dfa1.vortex.proto.EncodingProtos.ListViewMetadata parseDelimitedFrom(java.io.InputStream input) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseDelimitedWithIOException(PARSER, input); - } - - public static io.github.dfa1.vortex.proto.EncodingProtos.ListViewMetadata parseDelimitedFrom( - java.io.InputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseDelimitedWithIOException(PARSER, input, extensionRegistry); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.ListViewMetadata parseFrom( - com.google.protobuf.CodedInputStream input) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseWithIOException(PARSER, input); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.ListViewMetadata parseFrom( - com.google.protobuf.CodedInputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseWithIOException(PARSER, input, extensionRegistry); - } - - @java.lang.Override - public Builder newBuilderForType() { return newBuilder(); } - public static Builder newBuilder() { - return DEFAULT_INSTANCE.toBuilder(); - } - public static Builder newBuilder(io.github.dfa1.vortex.proto.EncodingProtos.ListViewMetadata prototype) { - return DEFAULT_INSTANCE.toBuilder().mergeFrom(prototype); - } - @java.lang.Override - public Builder toBuilder() { - return this == DEFAULT_INSTANCE - ? new Builder() : new Builder().mergeFrom(this); - } - - @java.lang.Override - protected Builder newBuilderForType( - com.google.protobuf.GeneratedMessage.BuilderParent parent) { - Builder builder = new Builder(parent); - return builder; - } - /** - * Protobuf type {@code vortex.encodings.ListViewMetadata} - */ - public static final class Builder extends - com.google.protobuf.GeneratedMessage.Builder implements - // @@protoc_insertion_point(builder_implements:vortex.encodings.ListViewMetadata) - io.github.dfa1.vortex.proto.EncodingProtos.ListViewMetadataOrBuilder { - public static final com.google.protobuf.Descriptors.Descriptor - getDescriptor() { - return io.github.dfa1.vortex.proto.EncodingProtos.internal_static_vortex_encodings_ListViewMetadata_descriptor; - } - - @java.lang.Override - protected com.google.protobuf.GeneratedMessage.FieldAccessorTable - internalGetFieldAccessorTable() { - return io.github.dfa1.vortex.proto.EncodingProtos.internal_static_vortex_encodings_ListViewMetadata_fieldAccessorTable - .ensureFieldAccessorsInitialized( - io.github.dfa1.vortex.proto.EncodingProtos.ListViewMetadata.class, io.github.dfa1.vortex.proto.EncodingProtos.ListViewMetadata.Builder.class); - } - - // Construct using io.github.dfa1.vortex.proto.EncodingProtos.ListViewMetadata.newBuilder() - private Builder() { - - } - - private Builder( - com.google.protobuf.GeneratedMessage.BuilderParent parent) { - super(parent); - - } - @java.lang.Override - public Builder clear() { - super.clear(); - bitField0_ = 0; - elementsLen_ = 0L; - offsetPtype_ = 0; - sizePtype_ = 0; - return this; - } - - @java.lang.Override - public com.google.protobuf.Descriptors.Descriptor - getDescriptorForType() { - return io.github.dfa1.vortex.proto.EncodingProtos.internal_static_vortex_encodings_ListViewMetadata_descriptor; - } - - @java.lang.Override - public io.github.dfa1.vortex.proto.EncodingProtos.ListViewMetadata getDefaultInstanceForType() { - return io.github.dfa1.vortex.proto.EncodingProtos.ListViewMetadata.getDefaultInstance(); - } - - @java.lang.Override - public io.github.dfa1.vortex.proto.EncodingProtos.ListViewMetadata build() { - io.github.dfa1.vortex.proto.EncodingProtos.ListViewMetadata result = buildPartial(); - if (!result.isInitialized()) { - throw newUninitializedMessageException(result); - } - return result; - } - - @java.lang.Override - public io.github.dfa1.vortex.proto.EncodingProtos.ListViewMetadata buildPartial() { - io.github.dfa1.vortex.proto.EncodingProtos.ListViewMetadata result = new io.github.dfa1.vortex.proto.EncodingProtos.ListViewMetadata(this); - if (bitField0_ != 0) { buildPartial0(result); } - onBuilt(); - return result; - } - - private void buildPartial0(io.github.dfa1.vortex.proto.EncodingProtos.ListViewMetadata result) { - int from_bitField0_ = bitField0_; - if (((from_bitField0_ & 0x00000001) != 0)) { - result.elementsLen_ = elementsLen_; - } - if (((from_bitField0_ & 0x00000002) != 0)) { - result.offsetPtype_ = offsetPtype_; - } - if (((from_bitField0_ & 0x00000004) != 0)) { - result.sizePtype_ = sizePtype_; - } - } - - @java.lang.Override - public Builder mergeFrom(com.google.protobuf.Message other) { - if (other instanceof io.github.dfa1.vortex.proto.EncodingProtos.ListViewMetadata) { - return mergeFrom((io.github.dfa1.vortex.proto.EncodingProtos.ListViewMetadata)other); - } else { - super.mergeFrom(other); - return this; - } - } - - public Builder mergeFrom(io.github.dfa1.vortex.proto.EncodingProtos.ListViewMetadata other) { - if (other == io.github.dfa1.vortex.proto.EncodingProtos.ListViewMetadata.getDefaultInstance()) return this; - if (other.getElementsLen() != 0L) { - setElementsLen(other.getElementsLen()); - } - if (other.offsetPtype_ != 0) { - setOffsetPtypeValue(other.getOffsetPtypeValue()); - } - if (other.sizePtype_ != 0) { - setSizePtypeValue(other.getSizePtypeValue()); - } - this.mergeUnknownFields(other.getUnknownFields()); - onChanged(); - return this; - } - - @java.lang.Override - public final boolean isInitialized() { - return true; - } - - @java.lang.Override - public Builder mergeFrom( - com.google.protobuf.CodedInputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws java.io.IOException { - if (extensionRegistry == null) { - throw new java.lang.NullPointerException(); - } - try { - boolean done = false; - while (!done) { - int tag = input.readTag(); - switch (tag) { - case 0: - done = true; - break; - case 8: { - elementsLen_ = input.readUInt64(); - bitField0_ |= 0x00000001; - break; - } // case 8 - case 16: { - offsetPtype_ = input.readEnum(); - bitField0_ |= 0x00000002; - break; - } // case 16 - case 24: { - sizePtype_ = input.readEnum(); - bitField0_ |= 0x00000004; - break; - } // case 24 - default: { - if (!super.parseUnknownField(input, extensionRegistry, tag)) { - done = true; // was an endgroup tag - } - break; - } // default: - } // switch (tag) - } // while (!done) - } catch (com.google.protobuf.InvalidProtocolBufferException e) { - throw e.unwrapIOException(); - } finally { - onChanged(); - } // finally - return this; - } - private int bitField0_; - - private long elementsLen_ ; - /** - * uint64 elements_len = 1; - * @return The elementsLen. - */ - @java.lang.Override - public long getElementsLen() { - return elementsLen_; - } - /** - * uint64 elements_len = 1; - * @param value The elementsLen to set. - * @return This builder for chaining. - */ - public Builder setElementsLen(long value) { - - elementsLen_ = value; - bitField0_ |= 0x00000001; - onChanged(); - return this; - } - /** - * uint64 elements_len = 1; - * @return This builder for chaining. - */ - public Builder clearElementsLen() { - bitField0_ = (bitField0_ & ~0x00000001); - elementsLen_ = 0L; - onChanged(); - return this; - } - - private int offsetPtype_ = 0; - /** - * .vortex.dtype.PType offset_ptype = 2; - * @return The enum numeric value on the wire for offsetPtype. - */ - @java.lang.Override public int getOffsetPtypeValue() { - return offsetPtype_; - } - /** - * .vortex.dtype.PType offset_ptype = 2; - * @param value The enum numeric value on the wire for offsetPtype to set. - * @return This builder for chaining. - */ - public Builder setOffsetPtypeValue(int value) { - offsetPtype_ = value; - bitField0_ |= 0x00000002; - onChanged(); - return this; - } - /** - * .vortex.dtype.PType offset_ptype = 2; - * @return The offsetPtype. - */ - @java.lang.Override - public io.github.dfa1.vortex.proto.DTypeProtos.PType getOffsetPtype() { - io.github.dfa1.vortex.proto.DTypeProtos.PType result = io.github.dfa1.vortex.proto.DTypeProtos.PType.forNumber(offsetPtype_); - return result == null ? io.github.dfa1.vortex.proto.DTypeProtos.PType.UNRECOGNIZED : result; - } - /** - * .vortex.dtype.PType offset_ptype = 2; - * @param value The offsetPtype to set. - * @throws IllegalArgumentException if UNRECOGNIZED is provided. - * @return This builder for chaining. - */ - public Builder setOffsetPtype(io.github.dfa1.vortex.proto.DTypeProtos.PType value) { - if (value == null) { throw new NullPointerException(); } - bitField0_ |= 0x00000002; - offsetPtype_ = value.getNumber(); - onChanged(); - return this; - } - /** - * .vortex.dtype.PType offset_ptype = 2; - * @return This builder for chaining. - */ - public Builder clearOffsetPtype() { - bitField0_ = (bitField0_ & ~0x00000002); - offsetPtype_ = 0; - onChanged(); - return this; - } - - private int sizePtype_ = 0; - /** - * .vortex.dtype.PType size_ptype = 3; - * @return The enum numeric value on the wire for sizePtype. - */ - @java.lang.Override public int getSizePtypeValue() { - return sizePtype_; - } - /** - * .vortex.dtype.PType size_ptype = 3; - * @param value The enum numeric value on the wire for sizePtype to set. - * @return This builder for chaining. - */ - public Builder setSizePtypeValue(int value) { - sizePtype_ = value; - bitField0_ |= 0x00000004; - onChanged(); - return this; - } - /** - * .vortex.dtype.PType size_ptype = 3; - * @return The sizePtype. - */ - @java.lang.Override - public io.github.dfa1.vortex.proto.DTypeProtos.PType getSizePtype() { - io.github.dfa1.vortex.proto.DTypeProtos.PType result = io.github.dfa1.vortex.proto.DTypeProtos.PType.forNumber(sizePtype_); - return result == null ? io.github.dfa1.vortex.proto.DTypeProtos.PType.UNRECOGNIZED : result; - } - /** - * .vortex.dtype.PType size_ptype = 3; - * @param value The sizePtype to set. - * @throws IllegalArgumentException if UNRECOGNIZED is provided. - * @return This builder for chaining. - */ - public Builder setSizePtype(io.github.dfa1.vortex.proto.DTypeProtos.PType value) { - if (value == null) { throw new NullPointerException(); } - bitField0_ |= 0x00000004; - sizePtype_ = value.getNumber(); - onChanged(); - return this; - } - /** - * .vortex.dtype.PType size_ptype = 3; - * @return This builder for chaining. - */ - public Builder clearSizePtype() { - bitField0_ = (bitField0_ & ~0x00000004); - sizePtype_ = 0; - onChanged(); - return this; - } - - // @@protoc_insertion_point(builder_scope:vortex.encodings.ListViewMetadata) - } - - // @@protoc_insertion_point(class_scope:vortex.encodings.ListViewMetadata) - private static final io.github.dfa1.vortex.proto.EncodingProtos.ListViewMetadata DEFAULT_INSTANCE; - static { - DEFAULT_INSTANCE = new io.github.dfa1.vortex.proto.EncodingProtos.ListViewMetadata(); - } - - public static io.github.dfa1.vortex.proto.EncodingProtos.ListViewMetadata getDefaultInstance() { - return DEFAULT_INSTANCE; - } - - private static final com.google.protobuf.Parser - PARSER = new com.google.protobuf.AbstractParser() { - @java.lang.Override - public ListViewMetadata parsePartialFrom( - com.google.protobuf.CodedInputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws com.google.protobuf.InvalidProtocolBufferException { - Builder builder = newBuilder(); - try { - builder.mergeFrom(input, extensionRegistry); - } catch (com.google.protobuf.InvalidProtocolBufferException e) { - throw e.setUnfinishedMessage(builder.buildPartial()); - } catch (com.google.protobuf.UninitializedMessageException e) { - throw e.asInvalidProtocolBufferException().setUnfinishedMessage(builder.buildPartial()); - } catch (java.io.IOException e) { - throw new com.google.protobuf.InvalidProtocolBufferException(e) - .setUnfinishedMessage(builder.buildPartial()); - } - return builder.buildPartial(); - } - }; - - public static com.google.protobuf.Parser parser() { - return PARSER; - } - - @java.lang.Override - public com.google.protobuf.Parser getParserForType() { - return PARSER; - } - - @java.lang.Override - public io.github.dfa1.vortex.proto.EncodingProtos.ListViewMetadata getDefaultInstanceForType() { - return DEFAULT_INSTANCE; - } - - } - - public interface ALPRDMetadataOrBuilder extends - // @@protoc_insertion_point(interface_extends:vortex.encodings.ALPRDMetadata) - com.google.protobuf.MessageOrBuilder { - - /** - * uint32 right_bit_width = 1; - * @return The rightBitWidth. - */ - int getRightBitWidth(); - - /** - * uint32 dict_len = 2; - * @return The dictLen. - */ - int getDictLen(); - - /** - * repeated uint32 dict = 3; - * @return A list containing the dict. - */ - java.util.List getDictList(); - /** - * repeated uint32 dict = 3; - * @return The count of dict. - */ - int getDictCount(); - /** - * repeated uint32 dict = 3; - * @param index The index of the element to return. - * @return The dict at the given index. - */ - int getDict(int index); - - /** - * .vortex.dtype.PType left_parts_ptype = 4; - * @return The enum numeric value on the wire for leftPartsPtype. - */ - int getLeftPartsPtypeValue(); - /** - * .vortex.dtype.PType left_parts_ptype = 4; - * @return The leftPartsPtype. - */ - io.github.dfa1.vortex.proto.DTypeProtos.PType getLeftPartsPtype(); - - /** - * optional .vortex.encodings.PatchesMetadata patches = 5; - * @return Whether the patches field is set. - */ - boolean hasPatches(); - /** - * optional .vortex.encodings.PatchesMetadata patches = 5; - * @return The patches. - */ - io.github.dfa1.vortex.proto.EncodingProtos.PatchesMetadata getPatches(); - /** - * optional .vortex.encodings.PatchesMetadata patches = 5; - */ - io.github.dfa1.vortex.proto.EncodingProtos.PatchesMetadataOrBuilder getPatchesOrBuilder(); - } - /** - * Protobuf type {@code vortex.encodings.ALPRDMetadata} - */ - public static final class ALPRDMetadata extends - com.google.protobuf.GeneratedMessage implements - // @@protoc_insertion_point(message_implements:vortex.encodings.ALPRDMetadata) - ALPRDMetadataOrBuilder { - private static final long serialVersionUID = 0L; - static { - com.google.protobuf.RuntimeVersion.validateProtobufGencodeVersion( - com.google.protobuf.RuntimeVersion.RuntimeDomain.PUBLIC, - /* major= */ 4, - /* minor= */ 35, - /* patch= */ 0, - /* suffix= */ "", - "ALPRDMetadata"); - } - // Use ALPRDMetadata.newBuilder() to construct. - private ALPRDMetadata(com.google.protobuf.GeneratedMessage.Builder builder) { - super(builder); - } - private ALPRDMetadata() { - dict_ = emptyIntList(); - leftPartsPtype_ = 0; - } - - public static final com.google.protobuf.Descriptors.Descriptor - getDescriptor() { - return io.github.dfa1.vortex.proto.EncodingProtos.internal_static_vortex_encodings_ALPRDMetadata_descriptor; - } - - @java.lang.Override - public com.google.protobuf.Descriptors.Descriptor getDescriptorForType() { - return io.github.dfa1.vortex.proto.EncodingProtos.internal_static_vortex_encodings_ALPRDMetadata_descriptor; - } - - @java.lang.Override - protected com.google.protobuf.GeneratedMessage.FieldAccessorTable - internalGetFieldAccessorTable() { - return io.github.dfa1.vortex.proto.EncodingProtos.internal_static_vortex_encodings_ALPRDMetadata_fieldAccessorTable - .ensureFieldAccessorsInitialized( - io.github.dfa1.vortex.proto.EncodingProtos.ALPRDMetadata.class, io.github.dfa1.vortex.proto.EncodingProtos.ALPRDMetadata.Builder.class); - } - - private int bitField0_; - public static final int RIGHT_BIT_WIDTH_FIELD_NUMBER = 1; - private int rightBitWidth_ = 0; - /** - * uint32 right_bit_width = 1; - * @return The rightBitWidth. - */ - @java.lang.Override - public int getRightBitWidth() { - return rightBitWidth_; - } - - public static final int DICT_LEN_FIELD_NUMBER = 2; - private int dictLen_ = 0; - /** - * uint32 dict_len = 2; - * @return The dictLen. - */ - @java.lang.Override - public int getDictLen() { - return dictLen_; - } - - public static final int DICT_FIELD_NUMBER = 3; - @SuppressWarnings("serial") - private com.google.protobuf.Internal.IntList dict_ = - emptyIntList(); - /** - * repeated uint32 dict = 3; - * @return A list containing the dict. - */ - @java.lang.Override - public java.util.List - getDictList() { - return dict_; - } - /** - * repeated uint32 dict = 3; - * @return The count of dict. - */ - public int getDictCount() { - return dict_.size(); - } - /** - * repeated uint32 dict = 3; - * @param index The index of the element to return. - * @return The dict at the given index. - */ - public int getDict(int index) { - return dict_.getInt(index); - } - private int dictMemoizedSerializedSize = -1; - - public static final int LEFT_PARTS_PTYPE_FIELD_NUMBER = 4; - private int leftPartsPtype_ = 0; - /** - * .vortex.dtype.PType left_parts_ptype = 4; - * @return The enum numeric value on the wire for leftPartsPtype. - */ - @java.lang.Override public int getLeftPartsPtypeValue() { - return leftPartsPtype_; - } - /** - * .vortex.dtype.PType left_parts_ptype = 4; - * @return The leftPartsPtype. - */ - @java.lang.Override public io.github.dfa1.vortex.proto.DTypeProtos.PType getLeftPartsPtype() { - io.github.dfa1.vortex.proto.DTypeProtos.PType result = io.github.dfa1.vortex.proto.DTypeProtos.PType.forNumber(leftPartsPtype_); - return result == null ? io.github.dfa1.vortex.proto.DTypeProtos.PType.UNRECOGNIZED : result; - } - - public static final int PATCHES_FIELD_NUMBER = 5; - private io.github.dfa1.vortex.proto.EncodingProtos.PatchesMetadata patches_; - /** - * optional .vortex.encodings.PatchesMetadata patches = 5; - * @return Whether the patches field is set. - */ - @java.lang.Override - public boolean hasPatches() { - return ((bitField0_ & 0x00000001) != 0); - } - /** - * optional .vortex.encodings.PatchesMetadata patches = 5; - * @return The patches. - */ - @java.lang.Override - public io.github.dfa1.vortex.proto.EncodingProtos.PatchesMetadata getPatches() { - return patches_ == null ? io.github.dfa1.vortex.proto.EncodingProtos.PatchesMetadata.getDefaultInstance() : patches_; - } - /** - * optional .vortex.encodings.PatchesMetadata patches = 5; - */ - @java.lang.Override - public io.github.dfa1.vortex.proto.EncodingProtos.PatchesMetadataOrBuilder getPatchesOrBuilder() { - return patches_ == null ? io.github.dfa1.vortex.proto.EncodingProtos.PatchesMetadata.getDefaultInstance() : patches_; - } - - private byte memoizedIsInitialized = -1; - @java.lang.Override - public final boolean isInitialized() { - byte isInitialized = memoizedIsInitialized; - if (isInitialized == 1) return true; - if (isInitialized == 0) return false; - - memoizedIsInitialized = 1; - return true; - } - - @java.lang.Override - public void writeTo(com.google.protobuf.CodedOutputStream output) - throws java.io.IOException { - getSerializedSize(); - if (rightBitWidth_ != 0) { - output.writeUInt32(1, rightBitWidth_); - } - if (dictLen_ != 0) { - output.writeUInt32(2, dictLen_); - } - if (getDictList().size() > 0) { - output.writeUInt32NoTag(26); - output.writeUInt32NoTag(dictMemoizedSerializedSize); - } - for (int i = 0; i < dict_.size(); i++) { - output.writeUInt32NoTag(dict_.getInt(i)); - } - if (leftPartsPtype_ != io.github.dfa1.vortex.proto.DTypeProtos.PType.U8.getNumber()) { - output.writeEnum(4, leftPartsPtype_); - } - if (((bitField0_ & 0x00000001) != 0)) { - output.writeMessage(5, getPatches()); - } - getUnknownFields().writeTo(output); - } - private int computeSerializedSize_0() { - int size = 0; - if (rightBitWidth_ != 0) { - size += com.google.protobuf.CodedOutputStream - .computeUInt32Size(1, rightBitWidth_); - } - if (dictLen_ != 0) { - size += com.google.protobuf.CodedOutputStream - .computeUInt32Size(2, dictLen_); - } - { - int dataSize = 0; - for (int i = 0; i < dict_.size(); i++) { - dataSize += com.google.protobuf.CodedOutputStream - .computeUInt32SizeNoTag(dict_.getInt(i)); - } - size += dataSize; - if (!getDictList().isEmpty()) { - size += 1; - size += com.google.protobuf.CodedOutputStream - .computeInt32SizeNoTag(dataSize); - } - dictMemoizedSerializedSize = dataSize; - } - if (leftPartsPtype_ != io.github.dfa1.vortex.proto.DTypeProtos.PType.U8.getNumber()) { - size += com.google.protobuf.CodedOutputStream - .computeEnumSize(4, leftPartsPtype_); - } - if (((bitField0_ & 0x00000001) != 0)) { - size += com.google.protobuf.CodedOutputStream - .computeMessageSize(5, getPatches()); - } - return size; - } - @java.lang.Override - public int getSerializedSize() { - int size = memoizedSize; - if (size != -1) return size; - - size = 0; - size += computeSerializedSize_0(); - size += getUnknownFields().getSerializedSize(); - memoizedSize = size; - return size; - } - - @java.lang.Override - public boolean equals(final java.lang.Object obj) { - if (obj == this) { - return true; - } - if (!(obj instanceof io.github.dfa1.vortex.proto.EncodingProtos.ALPRDMetadata)) { - return super.equals(obj); - } - io.github.dfa1.vortex.proto.EncodingProtos.ALPRDMetadata other = (io.github.dfa1.vortex.proto.EncodingProtos.ALPRDMetadata) obj; - - if (getRightBitWidth() - != other.getRightBitWidth()) return false; - if (getDictLen() - != other.getDictLen()) return false; - if (!getDictList() - .equals(other.getDictList())) return false; - if (leftPartsPtype_ != other.leftPartsPtype_) return false; - if (hasPatches() != other.hasPatches()) return false; - if (hasPatches()) { - if (!getPatches() - .equals(other.getPatches())) return false; - } - if (!getUnknownFields().equals(other.getUnknownFields())) return false; - return true; - } - - @java.lang.Override - public int hashCode() { - if (memoizedHashCode != 0) { - return memoizedHashCode; - } - int hash = 41; - hash = (19 * hash) + getDescriptor().hashCode(); - hash = (37 * hash) + RIGHT_BIT_WIDTH_FIELD_NUMBER; - hash = (53 * hash) + getRightBitWidth(); - hash = (37 * hash) + DICT_LEN_FIELD_NUMBER; - hash = (53 * hash) + getDictLen(); - if (getDictCount() > 0) { - hash = (37 * hash) + DICT_FIELD_NUMBER; - hash = (53 * hash) + getDictList().hashCode(); - } - hash = (37 * hash) + LEFT_PARTS_PTYPE_FIELD_NUMBER; - hash = (53 * hash) + leftPartsPtype_; - if (hasPatches()) { - hash = (37 * hash) + PATCHES_FIELD_NUMBER; - hash = (53 * hash) + getPatches().hashCode(); - } - hash = (29 * hash) + getUnknownFields().hashCode(); - memoizedHashCode = hash; - return hash; - } - - public static io.github.dfa1.vortex.proto.EncodingProtos.ALPRDMetadata parseFrom( - java.nio.ByteBuffer data) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.ALPRDMetadata parseFrom( - java.nio.ByteBuffer data, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data, extensionRegistry); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.ALPRDMetadata parseFrom( - com.google.protobuf.ByteString data) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.ALPRDMetadata parseFrom( - com.google.protobuf.ByteString data, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data, extensionRegistry); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.ALPRDMetadata parseFrom(byte[] data) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.ALPRDMetadata parseFrom( - byte[] data, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data, extensionRegistry); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.ALPRDMetadata parseFrom(java.io.InputStream input) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseWithIOException(PARSER, input); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.ALPRDMetadata parseFrom( - java.io.InputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseWithIOException(PARSER, input, extensionRegistry); - } - - public static io.github.dfa1.vortex.proto.EncodingProtos.ALPRDMetadata parseDelimitedFrom(java.io.InputStream input) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseDelimitedWithIOException(PARSER, input); - } - - public static io.github.dfa1.vortex.proto.EncodingProtos.ALPRDMetadata parseDelimitedFrom( - java.io.InputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseDelimitedWithIOException(PARSER, input, extensionRegistry); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.ALPRDMetadata parseFrom( - com.google.protobuf.CodedInputStream input) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseWithIOException(PARSER, input); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.ALPRDMetadata parseFrom( - com.google.protobuf.CodedInputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseWithIOException(PARSER, input, extensionRegistry); - } - - @java.lang.Override - public Builder newBuilderForType() { return newBuilder(); } - public static Builder newBuilder() { - return DEFAULT_INSTANCE.toBuilder(); - } - public static Builder newBuilder(io.github.dfa1.vortex.proto.EncodingProtos.ALPRDMetadata prototype) { - return DEFAULT_INSTANCE.toBuilder().mergeFrom(prototype); - } - @java.lang.Override - public Builder toBuilder() { - return this == DEFAULT_INSTANCE - ? new Builder() : new Builder().mergeFrom(this); - } - - @java.lang.Override - protected Builder newBuilderForType( - com.google.protobuf.GeneratedMessage.BuilderParent parent) { - Builder builder = new Builder(parent); - return builder; - } - /** - * Protobuf type {@code vortex.encodings.ALPRDMetadata} - */ - public static final class Builder extends - com.google.protobuf.GeneratedMessage.Builder implements - // @@protoc_insertion_point(builder_implements:vortex.encodings.ALPRDMetadata) - io.github.dfa1.vortex.proto.EncodingProtos.ALPRDMetadataOrBuilder { - public static final com.google.protobuf.Descriptors.Descriptor - getDescriptor() { - return io.github.dfa1.vortex.proto.EncodingProtos.internal_static_vortex_encodings_ALPRDMetadata_descriptor; - } - - @java.lang.Override - protected com.google.protobuf.GeneratedMessage.FieldAccessorTable - internalGetFieldAccessorTable() { - return io.github.dfa1.vortex.proto.EncodingProtos.internal_static_vortex_encodings_ALPRDMetadata_fieldAccessorTable - .ensureFieldAccessorsInitialized( - io.github.dfa1.vortex.proto.EncodingProtos.ALPRDMetadata.class, io.github.dfa1.vortex.proto.EncodingProtos.ALPRDMetadata.Builder.class); - } - - // Construct using io.github.dfa1.vortex.proto.EncodingProtos.ALPRDMetadata.newBuilder() - private Builder() { - maybeForceBuilderInitialization(); - } - - private Builder( - com.google.protobuf.GeneratedMessage.BuilderParent parent) { - super(parent); - maybeForceBuilderInitialization(); - } - private void maybeForceBuilderInitialization() { - if (com.google.protobuf.GeneratedMessage - .alwaysUseFieldBuilders) { - internalGetPatchesFieldBuilder(); - } - } - @java.lang.Override - public Builder clear() { - super.clear(); - bitField0_ = 0; - rightBitWidth_ = 0; - dictLen_ = 0; - dict_ = emptyIntList(); - leftPartsPtype_ = 0; - patches_ = null; - if (patchesBuilder_ != null) { - patchesBuilder_.dispose(); - patchesBuilder_ = null; - } - return this; - } - - @java.lang.Override - public com.google.protobuf.Descriptors.Descriptor - getDescriptorForType() { - return io.github.dfa1.vortex.proto.EncodingProtos.internal_static_vortex_encodings_ALPRDMetadata_descriptor; - } - - @java.lang.Override - public io.github.dfa1.vortex.proto.EncodingProtos.ALPRDMetadata getDefaultInstanceForType() { - return io.github.dfa1.vortex.proto.EncodingProtos.ALPRDMetadata.getDefaultInstance(); - } - - @java.lang.Override - public io.github.dfa1.vortex.proto.EncodingProtos.ALPRDMetadata build() { - io.github.dfa1.vortex.proto.EncodingProtos.ALPRDMetadata result = buildPartial(); - if (!result.isInitialized()) { - throw newUninitializedMessageException(result); - } - return result; - } - - @java.lang.Override - public io.github.dfa1.vortex.proto.EncodingProtos.ALPRDMetadata buildPartial() { - io.github.dfa1.vortex.proto.EncodingProtos.ALPRDMetadata result = new io.github.dfa1.vortex.proto.EncodingProtos.ALPRDMetadata(this); - if (bitField0_ != 0) { buildPartial0(result); } - onBuilt(); - return result; - } - - private void buildPartial0(io.github.dfa1.vortex.proto.EncodingProtos.ALPRDMetadata result) { - int from_bitField0_ = bitField0_; - if (((from_bitField0_ & 0x00000001) != 0)) { - result.rightBitWidth_ = rightBitWidth_; - } - if (((from_bitField0_ & 0x00000002) != 0)) { - result.dictLen_ = dictLen_; - } - if (((from_bitField0_ & 0x00000004) != 0)) { - dict_.makeImmutable(); - result.dict_ = dict_; - } - if (((from_bitField0_ & 0x00000008) != 0)) { - result.leftPartsPtype_ = leftPartsPtype_; - } - int to_bitField0_ = 0; - if (((from_bitField0_ & 0x00000010) != 0)) { - result.patches_ = patchesBuilder_ == null - ? patches_ - : patchesBuilder_.build(); - to_bitField0_ |= 0x00000001; - } - result.bitField0_ |= to_bitField0_; - } - - @java.lang.Override - public Builder mergeFrom(com.google.protobuf.Message other) { - if (other instanceof io.github.dfa1.vortex.proto.EncodingProtos.ALPRDMetadata) { - return mergeFrom((io.github.dfa1.vortex.proto.EncodingProtos.ALPRDMetadata)other); - } else { - super.mergeFrom(other); - return this; - } - } - - public Builder mergeFrom(io.github.dfa1.vortex.proto.EncodingProtos.ALPRDMetadata other) { - if (other == io.github.dfa1.vortex.proto.EncodingProtos.ALPRDMetadata.getDefaultInstance()) return this; - if (other.getRightBitWidth() != 0) { - setRightBitWidth(other.getRightBitWidth()); - } - if (other.getDictLen() != 0) { - setDictLen(other.getDictLen()); - } - if (!other.dict_.isEmpty()) { - if (dict_.isEmpty()) { - dict_ = other.dict_; - dict_.makeImmutable(); - bitField0_ |= 0x00000004; - } else { - ensureDictIsMutable(); - dict_.addAll(other.dict_); - } - onChanged(); - } - if (other.leftPartsPtype_ != 0) { - setLeftPartsPtypeValue(other.getLeftPartsPtypeValue()); - } - if (other.hasPatches()) { - mergePatches(other.getPatches()); - } - this.mergeUnknownFields(other.getUnknownFields()); - onChanged(); - return this; - } - - @java.lang.Override - public final boolean isInitialized() { - return true; - } - - @java.lang.Override - public Builder mergeFrom( - com.google.protobuf.CodedInputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws java.io.IOException { - if (extensionRegistry == null) { - throw new java.lang.NullPointerException(); - } - try { - boolean done = false; - while (!done) { - int tag = input.readTag(); - switch (tag) { - case 0: - done = true; - break; - case 8: { - rightBitWidth_ = input.readUInt32(); - bitField0_ |= 0x00000001; - break; - } // case 8 - case 16: { - dictLen_ = input.readUInt32(); - bitField0_ |= 0x00000002; - break; - } // case 16 - case 24: { - int v = input.readUInt32(); - ensureDictIsMutable(); - dict_.addInt(v); - break; - } // case 24 - case 26: { - int length = input.readRawVarint32(); - int limit = input.pushLimit(length); - ensureDictIsMutable(); - while (input.getBytesUntilLimit() > 0) { - dict_.addInt(input.readUInt32()); - } - input.popLimit(limit); - break; - } // case 26 - case 32: { - leftPartsPtype_ = input.readEnum(); - bitField0_ |= 0x00000008; - break; - } // case 32 - case 42: { - input.readMessage( - internalGetPatchesFieldBuilder().getBuilder(), - extensionRegistry); - bitField0_ |= 0x00000010; - break; - } // case 42 - default: { - if (!super.parseUnknownField(input, extensionRegistry, tag)) { - done = true; // was an endgroup tag - } - break; - } // default: - } // switch (tag) - } // while (!done) - } catch (com.google.protobuf.InvalidProtocolBufferException e) { - throw e.unwrapIOException(); - } finally { - onChanged(); - } // finally - return this; - } - private int bitField0_; - - private int rightBitWidth_ ; - /** - * uint32 right_bit_width = 1; - * @return The rightBitWidth. - */ - @java.lang.Override - public int getRightBitWidth() { - return rightBitWidth_; - } - /** - * uint32 right_bit_width = 1; - * @param value The rightBitWidth to set. - * @return This builder for chaining. - */ - public Builder setRightBitWidth(int value) { - - rightBitWidth_ = value; - bitField0_ |= 0x00000001; - onChanged(); - return this; - } - /** - * uint32 right_bit_width = 1; - * @return This builder for chaining. - */ - public Builder clearRightBitWidth() { - bitField0_ = (bitField0_ & ~0x00000001); - rightBitWidth_ = 0; - onChanged(); - return this; - } - - private int dictLen_ ; - /** - * uint32 dict_len = 2; - * @return The dictLen. - */ - @java.lang.Override - public int getDictLen() { - return dictLen_; - } - /** - * uint32 dict_len = 2; - * @param value The dictLen to set. - * @return This builder for chaining. - */ - public Builder setDictLen(int value) { - - dictLen_ = value; - bitField0_ |= 0x00000002; - onChanged(); - return this; - } - /** - * uint32 dict_len = 2; - * @return This builder for chaining. - */ - public Builder clearDictLen() { - bitField0_ = (bitField0_ & ~0x00000002); - dictLen_ = 0; - onChanged(); - return this; - } - - private com.google.protobuf.Internal.IntList dict_ = emptyIntList(); - private void ensureDictIsMutable() { - if (!dict_.isModifiable()) { - dict_ = makeMutableCopy(dict_); - } - bitField0_ |= 0x00000004; - } - /** - * repeated uint32 dict = 3; - * @return A list containing the dict. - */ - public java.util.List - getDictList() { - dict_.makeImmutable(); - return dict_; - } - /** - * repeated uint32 dict = 3; - * @return The count of dict. - */ - public int getDictCount() { - return dict_.size(); - } - /** - * repeated uint32 dict = 3; - * @param index The index of the element to return. - * @return The dict at the given index. - */ - public int getDict(int index) { - return dict_.getInt(index); - } - /** - * repeated uint32 dict = 3; - * @param index The index to set the value at. - * @param value The dict to set. - * @return This builder for chaining. - */ - public Builder setDict( - int index, int value) { - - ensureDictIsMutable(); - dict_.setInt(index, value); - bitField0_ |= 0x00000004; - onChanged(); - return this; - } - /** - * repeated uint32 dict = 3; - * @param value The dict to add. - * @return This builder for chaining. - */ - public Builder addDict(int value) { - - ensureDictIsMutable(); - dict_.addInt(value); - bitField0_ |= 0x00000004; - onChanged(); - return this; - } - /** - * repeated uint32 dict = 3; - * @param values The dict to add. - * @return This builder for chaining. - */ - public Builder addAllDict( - java.lang.Iterable values) { - ensureDictIsMutable(); - com.google.protobuf.AbstractMessageLite.Builder.addAll( - values, dict_); - bitField0_ |= 0x00000004; - onChanged(); - return this; - } - /** - * repeated uint32 dict = 3; - * @return This builder for chaining. - */ - public Builder clearDict() { - dict_ = emptyIntList(); - bitField0_ = (bitField0_ & ~0x00000004); - onChanged(); - return this; - } - - private int leftPartsPtype_ = 0; - /** - * .vortex.dtype.PType left_parts_ptype = 4; - * @return The enum numeric value on the wire for leftPartsPtype. - */ - @java.lang.Override public int getLeftPartsPtypeValue() { - return leftPartsPtype_; - } - /** - * .vortex.dtype.PType left_parts_ptype = 4; - * @param value The enum numeric value on the wire for leftPartsPtype to set. - * @return This builder for chaining. - */ - public Builder setLeftPartsPtypeValue(int value) { - leftPartsPtype_ = value; - bitField0_ |= 0x00000008; - onChanged(); - return this; - } - /** - * .vortex.dtype.PType left_parts_ptype = 4; - * @return The leftPartsPtype. - */ - @java.lang.Override - public io.github.dfa1.vortex.proto.DTypeProtos.PType getLeftPartsPtype() { - io.github.dfa1.vortex.proto.DTypeProtos.PType result = io.github.dfa1.vortex.proto.DTypeProtos.PType.forNumber(leftPartsPtype_); - return result == null ? io.github.dfa1.vortex.proto.DTypeProtos.PType.UNRECOGNIZED : result; - } - /** - * .vortex.dtype.PType left_parts_ptype = 4; - * @param value The leftPartsPtype to set. - * @throws IllegalArgumentException if UNRECOGNIZED is provided. - * @return This builder for chaining. - */ - public Builder setLeftPartsPtype(io.github.dfa1.vortex.proto.DTypeProtos.PType value) { - if (value == null) { throw new NullPointerException(); } - bitField0_ |= 0x00000008; - leftPartsPtype_ = value.getNumber(); - onChanged(); - return this; - } - /** - * .vortex.dtype.PType left_parts_ptype = 4; - * @return This builder for chaining. - */ - public Builder clearLeftPartsPtype() { - bitField0_ = (bitField0_ & ~0x00000008); - leftPartsPtype_ = 0; - onChanged(); - return this; - } - - private io.github.dfa1.vortex.proto.EncodingProtos.PatchesMetadata patches_; - private com.google.protobuf.SingleFieldBuilder< - io.github.dfa1.vortex.proto.EncodingProtos.PatchesMetadata, io.github.dfa1.vortex.proto.EncodingProtos.PatchesMetadata.Builder, io.github.dfa1.vortex.proto.EncodingProtos.PatchesMetadataOrBuilder> patchesBuilder_; - /** - * optional .vortex.encodings.PatchesMetadata patches = 5; - * @return Whether the patches field is set. - */ - public boolean hasPatches() { - return ((bitField0_ & 0x00000010) != 0); - } - /** - * optional .vortex.encodings.PatchesMetadata patches = 5; - * @return The patches. - */ - public io.github.dfa1.vortex.proto.EncodingProtos.PatchesMetadata getPatches() { - if (patchesBuilder_ == null) { - return patches_ == null ? io.github.dfa1.vortex.proto.EncodingProtos.PatchesMetadata.getDefaultInstance() : patches_; - } else { - return patchesBuilder_.getMessage(); - } - } - /** - * optional .vortex.encodings.PatchesMetadata patches = 5; - */ - public Builder setPatches(io.github.dfa1.vortex.proto.EncodingProtos.PatchesMetadata value) { - if (patchesBuilder_ == null) { - if (value == null) { - throw new NullPointerException(); - } - patches_ = value; - } else { - patchesBuilder_.setMessage(value); - } - bitField0_ |= 0x00000010; - onChanged(); - return this; - } - /** - * optional .vortex.encodings.PatchesMetadata patches = 5; - */ - public Builder setPatches( - io.github.dfa1.vortex.proto.EncodingProtos.PatchesMetadata.Builder builderForValue) { - if (patchesBuilder_ == null) { - patches_ = builderForValue.build(); - } else { - patchesBuilder_.setMessage(builderForValue.build()); - } - bitField0_ |= 0x00000010; - onChanged(); - return this; - } - /** - * optional .vortex.encodings.PatchesMetadata patches = 5; - */ - public Builder mergePatches(io.github.dfa1.vortex.proto.EncodingProtos.PatchesMetadata value) { - if (patchesBuilder_ == null) { - if (((bitField0_ & 0x00000010) != 0) && - patches_ != null && - patches_ != io.github.dfa1.vortex.proto.EncodingProtos.PatchesMetadata.getDefaultInstance()) { - getPatchesBuilder().mergeFrom(value); - } else { - patches_ = value; - } - } else { - patchesBuilder_.mergeFrom(value); - } - if (patches_ != null) { - bitField0_ |= 0x00000010; - onChanged(); - } - return this; - } - /** - * optional .vortex.encodings.PatchesMetadata patches = 5; - */ - public Builder clearPatches() { - bitField0_ = (bitField0_ & ~0x00000010); - patches_ = null; - if (patchesBuilder_ != null) { - patchesBuilder_.dispose(); - patchesBuilder_ = null; - } - onChanged(); - return this; - } - /** - * optional .vortex.encodings.PatchesMetadata patches = 5; - */ - public io.github.dfa1.vortex.proto.EncodingProtos.PatchesMetadata.Builder getPatchesBuilder() { - bitField0_ |= 0x00000010; - onChanged(); - return internalGetPatchesFieldBuilder().getBuilder(); - } - /** - * optional .vortex.encodings.PatchesMetadata patches = 5; - */ - public io.github.dfa1.vortex.proto.EncodingProtos.PatchesMetadataOrBuilder getPatchesOrBuilder() { - if (patchesBuilder_ != null) { - return patchesBuilder_.getMessageOrBuilder(); - } else { - return patches_ == null ? - io.github.dfa1.vortex.proto.EncodingProtos.PatchesMetadata.getDefaultInstance() : patches_; - } - } - /** - * optional .vortex.encodings.PatchesMetadata patches = 5; - */ - private com.google.protobuf.SingleFieldBuilder< - io.github.dfa1.vortex.proto.EncodingProtos.PatchesMetadata, io.github.dfa1.vortex.proto.EncodingProtos.PatchesMetadata.Builder, io.github.dfa1.vortex.proto.EncodingProtos.PatchesMetadataOrBuilder> - internalGetPatchesFieldBuilder() { - if (patchesBuilder_ == null) { - patchesBuilder_ = new com.google.protobuf.SingleFieldBuilder< - io.github.dfa1.vortex.proto.EncodingProtos.PatchesMetadata, io.github.dfa1.vortex.proto.EncodingProtos.PatchesMetadata.Builder, io.github.dfa1.vortex.proto.EncodingProtos.PatchesMetadataOrBuilder>( - getPatches(), - getParentForChildren(), - isClean()); - patches_ = null; - } - return patchesBuilder_; - } - - // @@protoc_insertion_point(builder_scope:vortex.encodings.ALPRDMetadata) - } - - // @@protoc_insertion_point(class_scope:vortex.encodings.ALPRDMetadata) - private static final io.github.dfa1.vortex.proto.EncodingProtos.ALPRDMetadata DEFAULT_INSTANCE; - static { - DEFAULT_INSTANCE = new io.github.dfa1.vortex.proto.EncodingProtos.ALPRDMetadata(); - } - - public static io.github.dfa1.vortex.proto.EncodingProtos.ALPRDMetadata getDefaultInstance() { - return DEFAULT_INSTANCE; - } - - private static final com.google.protobuf.Parser - PARSER = new com.google.protobuf.AbstractParser() { - @java.lang.Override - public ALPRDMetadata parsePartialFrom( - com.google.protobuf.CodedInputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws com.google.protobuf.InvalidProtocolBufferException { - Builder builder = newBuilder(); - try { - builder.mergeFrom(input, extensionRegistry); - } catch (com.google.protobuf.InvalidProtocolBufferException e) { - throw e.setUnfinishedMessage(builder.buildPartial()); - } catch (com.google.protobuf.UninitializedMessageException e) { - throw e.asInvalidProtocolBufferException().setUnfinishedMessage(builder.buildPartial()); - } catch (java.io.IOException e) { - throw new com.google.protobuf.InvalidProtocolBufferException(e) - .setUnfinishedMessage(builder.buildPartial()); - } - return builder.buildPartial(); - } - }; - - public static com.google.protobuf.Parser parser() { - return PARSER; - } - - @java.lang.Override - public com.google.protobuf.Parser getParserForType() { - return PARSER; - } - - @java.lang.Override - public io.github.dfa1.vortex.proto.EncodingProtos.ALPRDMetadata getDefaultInstanceForType() { - return DEFAULT_INSTANCE; - } - - } - - public interface PcoPageInfoOrBuilder extends - // @@protoc_insertion_point(interface_extends:vortex.encodings.PcoPageInfo) - com.google.protobuf.MessageOrBuilder { - - /** - * uint32 n_values = 1; - * @return The nValues. - */ - int getNValues(); - } - /** - * Protobuf type {@code vortex.encodings.PcoPageInfo} - */ - public static final class PcoPageInfo extends - com.google.protobuf.GeneratedMessage implements - // @@protoc_insertion_point(message_implements:vortex.encodings.PcoPageInfo) - PcoPageInfoOrBuilder { - private static final long serialVersionUID = 0L; - static { - com.google.protobuf.RuntimeVersion.validateProtobufGencodeVersion( - com.google.protobuf.RuntimeVersion.RuntimeDomain.PUBLIC, - /* major= */ 4, - /* minor= */ 35, - /* patch= */ 0, - /* suffix= */ "", - "PcoPageInfo"); - } - // Use PcoPageInfo.newBuilder() to construct. - private PcoPageInfo(com.google.protobuf.GeneratedMessage.Builder builder) { - super(builder); - } - private PcoPageInfo() { - } - - public static final com.google.protobuf.Descriptors.Descriptor - getDescriptor() { - return io.github.dfa1.vortex.proto.EncodingProtos.internal_static_vortex_encodings_PcoPageInfo_descriptor; - } - - @java.lang.Override - public com.google.protobuf.Descriptors.Descriptor getDescriptorForType() { - return io.github.dfa1.vortex.proto.EncodingProtos.internal_static_vortex_encodings_PcoPageInfo_descriptor; - } - - @java.lang.Override - protected com.google.protobuf.GeneratedMessage.FieldAccessorTable - internalGetFieldAccessorTable() { - return io.github.dfa1.vortex.proto.EncodingProtos.internal_static_vortex_encodings_PcoPageInfo_fieldAccessorTable - .ensureFieldAccessorsInitialized( - io.github.dfa1.vortex.proto.EncodingProtos.PcoPageInfo.class, io.github.dfa1.vortex.proto.EncodingProtos.PcoPageInfo.Builder.class); - } - - public static final int N_VALUES_FIELD_NUMBER = 1; - private int nValues_ = 0; - /** - * uint32 n_values = 1; - * @return The nValues. - */ - @java.lang.Override - public int getNValues() { - return nValues_; - } - - private byte memoizedIsInitialized = -1; - @java.lang.Override - public final boolean isInitialized() { - byte isInitialized = memoizedIsInitialized; - if (isInitialized == 1) return true; - if (isInitialized == 0) return false; - - memoizedIsInitialized = 1; - return true; - } - - @java.lang.Override - public void writeTo(com.google.protobuf.CodedOutputStream output) - throws java.io.IOException { - if (nValues_ != 0) { - output.writeUInt32(1, nValues_); - } - getUnknownFields().writeTo(output); - } - private int computeSerializedSize_0() { - int size = 0; - if (nValues_ != 0) { - size += com.google.protobuf.CodedOutputStream - .computeUInt32Size(1, nValues_); - } - return size; - } - @java.lang.Override - public int getSerializedSize() { - int size = memoizedSize; - if (size != -1) return size; - - size = 0; - size += computeSerializedSize_0(); - size += getUnknownFields().getSerializedSize(); - memoizedSize = size; - return size; - } - - @java.lang.Override - public boolean equals(final java.lang.Object obj) { - if (obj == this) { - return true; - } - if (!(obj instanceof io.github.dfa1.vortex.proto.EncodingProtos.PcoPageInfo)) { - return super.equals(obj); - } - io.github.dfa1.vortex.proto.EncodingProtos.PcoPageInfo other = (io.github.dfa1.vortex.proto.EncodingProtos.PcoPageInfo) obj; - - if (getNValues() - != other.getNValues()) return false; - if (!getUnknownFields().equals(other.getUnknownFields())) return false; - return true; - } - - @java.lang.Override - public int hashCode() { - if (memoizedHashCode != 0) { - return memoizedHashCode; - } - int hash = 41; - hash = (19 * hash) + getDescriptor().hashCode(); - hash = (37 * hash) + N_VALUES_FIELD_NUMBER; - hash = (53 * hash) + getNValues(); - hash = (29 * hash) + getUnknownFields().hashCode(); - memoizedHashCode = hash; - return hash; - } - - public static io.github.dfa1.vortex.proto.EncodingProtos.PcoPageInfo parseFrom( - java.nio.ByteBuffer data) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.PcoPageInfo parseFrom( - java.nio.ByteBuffer data, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data, extensionRegistry); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.PcoPageInfo parseFrom( - com.google.protobuf.ByteString data) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.PcoPageInfo parseFrom( - com.google.protobuf.ByteString data, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data, extensionRegistry); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.PcoPageInfo parseFrom(byte[] data) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.PcoPageInfo parseFrom( - byte[] data, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data, extensionRegistry); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.PcoPageInfo parseFrom(java.io.InputStream input) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseWithIOException(PARSER, input); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.PcoPageInfo parseFrom( - java.io.InputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseWithIOException(PARSER, input, extensionRegistry); - } - - public static io.github.dfa1.vortex.proto.EncodingProtos.PcoPageInfo parseDelimitedFrom(java.io.InputStream input) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseDelimitedWithIOException(PARSER, input); - } - - public static io.github.dfa1.vortex.proto.EncodingProtos.PcoPageInfo parseDelimitedFrom( - java.io.InputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseDelimitedWithIOException(PARSER, input, extensionRegistry); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.PcoPageInfo parseFrom( - com.google.protobuf.CodedInputStream input) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseWithIOException(PARSER, input); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.PcoPageInfo parseFrom( - com.google.protobuf.CodedInputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseWithIOException(PARSER, input, extensionRegistry); - } - - @java.lang.Override - public Builder newBuilderForType() { return newBuilder(); } - public static Builder newBuilder() { - return DEFAULT_INSTANCE.toBuilder(); - } - public static Builder newBuilder(io.github.dfa1.vortex.proto.EncodingProtos.PcoPageInfo prototype) { - return DEFAULT_INSTANCE.toBuilder().mergeFrom(prototype); - } - @java.lang.Override - public Builder toBuilder() { - return this == DEFAULT_INSTANCE - ? new Builder() : new Builder().mergeFrom(this); - } - - @java.lang.Override - protected Builder newBuilderForType( - com.google.protobuf.GeneratedMessage.BuilderParent parent) { - Builder builder = new Builder(parent); - return builder; - } - /** - * Protobuf type {@code vortex.encodings.PcoPageInfo} - */ - public static final class Builder extends - com.google.protobuf.GeneratedMessage.Builder implements - // @@protoc_insertion_point(builder_implements:vortex.encodings.PcoPageInfo) - io.github.dfa1.vortex.proto.EncodingProtos.PcoPageInfoOrBuilder { - public static final com.google.protobuf.Descriptors.Descriptor - getDescriptor() { - return io.github.dfa1.vortex.proto.EncodingProtos.internal_static_vortex_encodings_PcoPageInfo_descriptor; - } - - @java.lang.Override - protected com.google.protobuf.GeneratedMessage.FieldAccessorTable - internalGetFieldAccessorTable() { - return io.github.dfa1.vortex.proto.EncodingProtos.internal_static_vortex_encodings_PcoPageInfo_fieldAccessorTable - .ensureFieldAccessorsInitialized( - io.github.dfa1.vortex.proto.EncodingProtos.PcoPageInfo.class, io.github.dfa1.vortex.proto.EncodingProtos.PcoPageInfo.Builder.class); - } - - // Construct using io.github.dfa1.vortex.proto.EncodingProtos.PcoPageInfo.newBuilder() - private Builder() { - - } - - private Builder( - com.google.protobuf.GeneratedMessage.BuilderParent parent) { - super(parent); - - } - @java.lang.Override - public Builder clear() { - super.clear(); - bitField0_ = 0; - nValues_ = 0; - return this; - } - - @java.lang.Override - public com.google.protobuf.Descriptors.Descriptor - getDescriptorForType() { - return io.github.dfa1.vortex.proto.EncodingProtos.internal_static_vortex_encodings_PcoPageInfo_descriptor; - } - - @java.lang.Override - public io.github.dfa1.vortex.proto.EncodingProtos.PcoPageInfo getDefaultInstanceForType() { - return io.github.dfa1.vortex.proto.EncodingProtos.PcoPageInfo.getDefaultInstance(); - } - - @java.lang.Override - public io.github.dfa1.vortex.proto.EncodingProtos.PcoPageInfo build() { - io.github.dfa1.vortex.proto.EncodingProtos.PcoPageInfo result = buildPartial(); - if (!result.isInitialized()) { - throw newUninitializedMessageException(result); - } - return result; - } - - @java.lang.Override - public io.github.dfa1.vortex.proto.EncodingProtos.PcoPageInfo buildPartial() { - io.github.dfa1.vortex.proto.EncodingProtos.PcoPageInfo result = new io.github.dfa1.vortex.proto.EncodingProtos.PcoPageInfo(this); - if (bitField0_ != 0) { buildPartial0(result); } - onBuilt(); - return result; - } - - private void buildPartial0(io.github.dfa1.vortex.proto.EncodingProtos.PcoPageInfo result) { - int from_bitField0_ = bitField0_; - if (((from_bitField0_ & 0x00000001) != 0)) { - result.nValues_ = nValues_; - } - } - - @java.lang.Override - public Builder mergeFrom(com.google.protobuf.Message other) { - if (other instanceof io.github.dfa1.vortex.proto.EncodingProtos.PcoPageInfo) { - return mergeFrom((io.github.dfa1.vortex.proto.EncodingProtos.PcoPageInfo)other); - } else { - super.mergeFrom(other); - return this; - } - } - - public Builder mergeFrom(io.github.dfa1.vortex.proto.EncodingProtos.PcoPageInfo other) { - if (other == io.github.dfa1.vortex.proto.EncodingProtos.PcoPageInfo.getDefaultInstance()) return this; - if (other.getNValues() != 0) { - setNValues(other.getNValues()); - } - this.mergeUnknownFields(other.getUnknownFields()); - onChanged(); - return this; - } - - @java.lang.Override - public final boolean isInitialized() { - return true; - } - - @java.lang.Override - public Builder mergeFrom( - com.google.protobuf.CodedInputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws java.io.IOException { - if (extensionRegistry == null) { - throw new java.lang.NullPointerException(); - } - try { - boolean done = false; - while (!done) { - int tag = input.readTag(); - switch (tag) { - case 0: - done = true; - break; - case 8: { - nValues_ = input.readUInt32(); - bitField0_ |= 0x00000001; - break; - } // case 8 - default: { - if (!super.parseUnknownField(input, extensionRegistry, tag)) { - done = true; // was an endgroup tag - } - break; - } // default: - } // switch (tag) - } // while (!done) - } catch (com.google.protobuf.InvalidProtocolBufferException e) { - throw e.unwrapIOException(); - } finally { - onChanged(); - } // finally - return this; - } - private int bitField0_; - - private int nValues_ ; - /** - * uint32 n_values = 1; - * @return The nValues. - */ - @java.lang.Override - public int getNValues() { - return nValues_; - } - /** - * uint32 n_values = 1; - * @param value The nValues to set. - * @return This builder for chaining. - */ - public Builder setNValues(int value) { - - nValues_ = value; - bitField0_ |= 0x00000001; - onChanged(); - return this; - } - /** - * uint32 n_values = 1; - * @return This builder for chaining. - */ - public Builder clearNValues() { - bitField0_ = (bitField0_ & ~0x00000001); - nValues_ = 0; - onChanged(); - return this; - } - - // @@protoc_insertion_point(builder_scope:vortex.encodings.PcoPageInfo) - } - - // @@protoc_insertion_point(class_scope:vortex.encodings.PcoPageInfo) - private static final io.github.dfa1.vortex.proto.EncodingProtos.PcoPageInfo DEFAULT_INSTANCE; - static { - DEFAULT_INSTANCE = new io.github.dfa1.vortex.proto.EncodingProtos.PcoPageInfo(); - } - - public static io.github.dfa1.vortex.proto.EncodingProtos.PcoPageInfo getDefaultInstance() { - return DEFAULT_INSTANCE; - } - - private static final com.google.protobuf.Parser - PARSER = new com.google.protobuf.AbstractParser() { - @java.lang.Override - public PcoPageInfo parsePartialFrom( - com.google.protobuf.CodedInputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws com.google.protobuf.InvalidProtocolBufferException { - Builder builder = newBuilder(); - try { - builder.mergeFrom(input, extensionRegistry); - } catch (com.google.protobuf.InvalidProtocolBufferException e) { - throw e.setUnfinishedMessage(builder.buildPartial()); - } catch (com.google.protobuf.UninitializedMessageException e) { - throw e.asInvalidProtocolBufferException().setUnfinishedMessage(builder.buildPartial()); - } catch (java.io.IOException e) { - throw new com.google.protobuf.InvalidProtocolBufferException(e) - .setUnfinishedMessage(builder.buildPartial()); - } - return builder.buildPartial(); - } - }; - - public static com.google.protobuf.Parser parser() { - return PARSER; - } - - @java.lang.Override - public com.google.protobuf.Parser getParserForType() { - return PARSER; - } - - @java.lang.Override - public io.github.dfa1.vortex.proto.EncodingProtos.PcoPageInfo getDefaultInstanceForType() { - return DEFAULT_INSTANCE; - } - - } - - public interface PcoChunkInfoOrBuilder extends - // @@protoc_insertion_point(interface_extends:vortex.encodings.PcoChunkInfo) - com.google.protobuf.MessageOrBuilder { - - /** - * repeated .vortex.encodings.PcoPageInfo pages = 1; - */ - java.util.List - getPagesList(); - /** - * repeated .vortex.encodings.PcoPageInfo pages = 1; - */ - io.github.dfa1.vortex.proto.EncodingProtos.PcoPageInfo getPages(int index); - /** - * repeated .vortex.encodings.PcoPageInfo pages = 1; - */ - int getPagesCount(); - /** - * repeated .vortex.encodings.PcoPageInfo pages = 1; - */ - java.util.List - getPagesOrBuilderList(); - /** - * repeated .vortex.encodings.PcoPageInfo pages = 1; - */ - io.github.dfa1.vortex.proto.EncodingProtos.PcoPageInfoOrBuilder getPagesOrBuilder( - int index); - } - /** - * Protobuf type {@code vortex.encodings.PcoChunkInfo} - */ - public static final class PcoChunkInfo extends - com.google.protobuf.GeneratedMessage implements - // @@protoc_insertion_point(message_implements:vortex.encodings.PcoChunkInfo) - PcoChunkInfoOrBuilder { - private static final long serialVersionUID = 0L; - static { - com.google.protobuf.RuntimeVersion.validateProtobufGencodeVersion( - com.google.protobuf.RuntimeVersion.RuntimeDomain.PUBLIC, - /* major= */ 4, - /* minor= */ 35, - /* patch= */ 0, - /* suffix= */ "", - "PcoChunkInfo"); - } - // Use PcoChunkInfo.newBuilder() to construct. - private PcoChunkInfo(com.google.protobuf.GeneratedMessage.Builder builder) { - super(builder); - } - private PcoChunkInfo() { - pages_ = java.util.Collections.emptyList(); - } - - public static final com.google.protobuf.Descriptors.Descriptor - getDescriptor() { - return io.github.dfa1.vortex.proto.EncodingProtos.internal_static_vortex_encodings_PcoChunkInfo_descriptor; - } - - @java.lang.Override - public com.google.protobuf.Descriptors.Descriptor getDescriptorForType() { - return io.github.dfa1.vortex.proto.EncodingProtos.internal_static_vortex_encodings_PcoChunkInfo_descriptor; - } - - @java.lang.Override - protected com.google.protobuf.GeneratedMessage.FieldAccessorTable - internalGetFieldAccessorTable() { - return io.github.dfa1.vortex.proto.EncodingProtos.internal_static_vortex_encodings_PcoChunkInfo_fieldAccessorTable - .ensureFieldAccessorsInitialized( - io.github.dfa1.vortex.proto.EncodingProtos.PcoChunkInfo.class, io.github.dfa1.vortex.proto.EncodingProtos.PcoChunkInfo.Builder.class); - } - - public static final int PAGES_FIELD_NUMBER = 1; - @SuppressWarnings("serial") - private java.util.List pages_; - /** - * repeated .vortex.encodings.PcoPageInfo pages = 1; - */ - @java.lang.Override - public java.util.List getPagesList() { - return pages_; - } - /** - * repeated .vortex.encodings.PcoPageInfo pages = 1; - */ - @java.lang.Override - public java.util.List - getPagesOrBuilderList() { - return pages_; - } - /** - * repeated .vortex.encodings.PcoPageInfo pages = 1; - */ - @java.lang.Override - public int getPagesCount() { - return pages_.size(); - } - /** - * repeated .vortex.encodings.PcoPageInfo pages = 1; - */ - @java.lang.Override - public io.github.dfa1.vortex.proto.EncodingProtos.PcoPageInfo getPages(int index) { - return pages_.get(index); - } - /** - * repeated .vortex.encodings.PcoPageInfo pages = 1; - */ - @java.lang.Override - public io.github.dfa1.vortex.proto.EncodingProtos.PcoPageInfoOrBuilder getPagesOrBuilder( - int index) { - return pages_.get(index); - } - - private byte memoizedIsInitialized = -1; - @java.lang.Override - public final boolean isInitialized() { - byte isInitialized = memoizedIsInitialized; - if (isInitialized == 1) return true; - if (isInitialized == 0) return false; - - memoizedIsInitialized = 1; - return true; - } - - @java.lang.Override - public void writeTo(com.google.protobuf.CodedOutputStream output) - throws java.io.IOException { - for (int i = 0; i < pages_.size(); i++) { - output.writeMessage(1, pages_.get(i)); - } - getUnknownFields().writeTo(output); - } - private int computeSerializedSize_0() { - int size = 0; - - { - final int count = pages_.size(); - for (int i = 0; i < count; i++) { - size += com.google.protobuf.CodedOutputStream - .computeMessageSizeNoTag(pages_.get(i)); - } - size += 1 * count; - } - return size; - } - @java.lang.Override - public int getSerializedSize() { - int size = memoizedSize; - if (size != -1) return size; - - size = 0; - size += computeSerializedSize_0(); - size += getUnknownFields().getSerializedSize(); - memoizedSize = size; - return size; - } - - @java.lang.Override - public boolean equals(final java.lang.Object obj) { - if (obj == this) { - return true; - } - if (!(obj instanceof io.github.dfa1.vortex.proto.EncodingProtos.PcoChunkInfo)) { - return super.equals(obj); - } - io.github.dfa1.vortex.proto.EncodingProtos.PcoChunkInfo other = (io.github.dfa1.vortex.proto.EncodingProtos.PcoChunkInfo) obj; - - if (!getPagesList() - .equals(other.getPagesList())) return false; - if (!getUnknownFields().equals(other.getUnknownFields())) return false; - return true; - } - - @java.lang.Override - public int hashCode() { - if (memoizedHashCode != 0) { - return memoizedHashCode; - } - int hash = 41; - hash = (19 * hash) + getDescriptor().hashCode(); - if (getPagesCount() > 0) { - hash = (37 * hash) + PAGES_FIELD_NUMBER; - hash = (53 * hash) + getPagesList().hashCode(); - } - hash = (29 * hash) + getUnknownFields().hashCode(); - memoizedHashCode = hash; - return hash; - } - - public static io.github.dfa1.vortex.proto.EncodingProtos.PcoChunkInfo parseFrom( - java.nio.ByteBuffer data) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.PcoChunkInfo parseFrom( - java.nio.ByteBuffer data, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data, extensionRegistry); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.PcoChunkInfo parseFrom( - com.google.protobuf.ByteString data) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.PcoChunkInfo parseFrom( - com.google.protobuf.ByteString data, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data, extensionRegistry); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.PcoChunkInfo parseFrom(byte[] data) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.PcoChunkInfo parseFrom( - byte[] data, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data, extensionRegistry); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.PcoChunkInfo parseFrom(java.io.InputStream input) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseWithIOException(PARSER, input); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.PcoChunkInfo parseFrom( - java.io.InputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseWithIOException(PARSER, input, extensionRegistry); - } - - public static io.github.dfa1.vortex.proto.EncodingProtos.PcoChunkInfo parseDelimitedFrom(java.io.InputStream input) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseDelimitedWithIOException(PARSER, input); - } - - public static io.github.dfa1.vortex.proto.EncodingProtos.PcoChunkInfo parseDelimitedFrom( - java.io.InputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseDelimitedWithIOException(PARSER, input, extensionRegistry); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.PcoChunkInfo parseFrom( - com.google.protobuf.CodedInputStream input) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseWithIOException(PARSER, input); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.PcoChunkInfo parseFrom( - com.google.protobuf.CodedInputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseWithIOException(PARSER, input, extensionRegistry); - } - - @java.lang.Override - public Builder newBuilderForType() { return newBuilder(); } - public static Builder newBuilder() { - return DEFAULT_INSTANCE.toBuilder(); - } - public static Builder newBuilder(io.github.dfa1.vortex.proto.EncodingProtos.PcoChunkInfo prototype) { - return DEFAULT_INSTANCE.toBuilder().mergeFrom(prototype); - } - @java.lang.Override - public Builder toBuilder() { - return this == DEFAULT_INSTANCE - ? new Builder() : new Builder().mergeFrom(this); - } - - @java.lang.Override - protected Builder newBuilderForType( - com.google.protobuf.GeneratedMessage.BuilderParent parent) { - Builder builder = new Builder(parent); - return builder; - } - /** - * Protobuf type {@code vortex.encodings.PcoChunkInfo} - */ - public static final class Builder extends - com.google.protobuf.GeneratedMessage.Builder implements - // @@protoc_insertion_point(builder_implements:vortex.encodings.PcoChunkInfo) - io.github.dfa1.vortex.proto.EncodingProtos.PcoChunkInfoOrBuilder { - public static final com.google.protobuf.Descriptors.Descriptor - getDescriptor() { - return io.github.dfa1.vortex.proto.EncodingProtos.internal_static_vortex_encodings_PcoChunkInfo_descriptor; - } - - @java.lang.Override - protected com.google.protobuf.GeneratedMessage.FieldAccessorTable - internalGetFieldAccessorTable() { - return io.github.dfa1.vortex.proto.EncodingProtos.internal_static_vortex_encodings_PcoChunkInfo_fieldAccessorTable - .ensureFieldAccessorsInitialized( - io.github.dfa1.vortex.proto.EncodingProtos.PcoChunkInfo.class, io.github.dfa1.vortex.proto.EncodingProtos.PcoChunkInfo.Builder.class); - } - - // Construct using io.github.dfa1.vortex.proto.EncodingProtos.PcoChunkInfo.newBuilder() - private Builder() { - - } - - private Builder( - com.google.protobuf.GeneratedMessage.BuilderParent parent) { - super(parent); - - } - @java.lang.Override - public Builder clear() { - super.clear(); - bitField0_ = 0; - if (pagesBuilder_ == null) { - pages_ = java.util.Collections.emptyList(); - } else { - pages_ = null; - pagesBuilder_.clear(); - } - bitField0_ = (bitField0_ & ~0x00000001); - return this; - } - - @java.lang.Override - public com.google.protobuf.Descriptors.Descriptor - getDescriptorForType() { - return io.github.dfa1.vortex.proto.EncodingProtos.internal_static_vortex_encodings_PcoChunkInfo_descriptor; - } - - @java.lang.Override - public io.github.dfa1.vortex.proto.EncodingProtos.PcoChunkInfo getDefaultInstanceForType() { - return io.github.dfa1.vortex.proto.EncodingProtos.PcoChunkInfo.getDefaultInstance(); - } - - @java.lang.Override - public io.github.dfa1.vortex.proto.EncodingProtos.PcoChunkInfo build() { - io.github.dfa1.vortex.proto.EncodingProtos.PcoChunkInfo result = buildPartial(); - if (!result.isInitialized()) { - throw newUninitializedMessageException(result); - } - return result; - } - - @java.lang.Override - public io.github.dfa1.vortex.proto.EncodingProtos.PcoChunkInfo buildPartial() { - io.github.dfa1.vortex.proto.EncodingProtos.PcoChunkInfo result = new io.github.dfa1.vortex.proto.EncodingProtos.PcoChunkInfo(this); - buildPartialRepeatedFields(result); - if (bitField0_ != 0) { buildPartial0(result); } - onBuilt(); - return result; - } - - private void buildPartialRepeatedFields(io.github.dfa1.vortex.proto.EncodingProtos.PcoChunkInfo result) { - if (pagesBuilder_ == null) { - if (((bitField0_ & 0x00000001) != 0)) { - pages_ = java.util.Collections.unmodifiableList(pages_); - bitField0_ = (bitField0_ & ~0x00000001); - } - result.pages_ = pages_; - } else { - result.pages_ = pagesBuilder_.build(); - } - } - - private void buildPartial0(io.github.dfa1.vortex.proto.EncodingProtos.PcoChunkInfo result) { - int from_bitField0_ = bitField0_; - } - - @java.lang.Override - public Builder mergeFrom(com.google.protobuf.Message other) { - if (other instanceof io.github.dfa1.vortex.proto.EncodingProtos.PcoChunkInfo) { - return mergeFrom((io.github.dfa1.vortex.proto.EncodingProtos.PcoChunkInfo)other); - } else { - super.mergeFrom(other); - return this; - } - } - - public Builder mergeFrom(io.github.dfa1.vortex.proto.EncodingProtos.PcoChunkInfo other) { - if (other == io.github.dfa1.vortex.proto.EncodingProtos.PcoChunkInfo.getDefaultInstance()) return this; - if (pagesBuilder_ == null) { - if (!other.pages_.isEmpty()) { - if (pages_.isEmpty()) { - pages_ = other.pages_; - bitField0_ = (bitField0_ & ~0x00000001); - } else { - ensurePagesIsMutable(); - pages_.addAll(other.pages_); - } - onChanged(); - } - } else { - if (!other.pages_.isEmpty()) { - if (pagesBuilder_.isEmpty()) { - pagesBuilder_.dispose(); - pagesBuilder_ = null; - pages_ = other.pages_; - bitField0_ = (bitField0_ & ~0x00000001); - pagesBuilder_ = - com.google.protobuf.GeneratedMessage.alwaysUseFieldBuilders ? - internalGetPagesFieldBuilder() : null; - } else { - pagesBuilder_.addAllMessages(other.pages_); - } - } - } - this.mergeUnknownFields(other.getUnknownFields()); - onChanged(); - return this; - } - - @java.lang.Override - public final boolean isInitialized() { - return true; - } - - @java.lang.Override - public Builder mergeFrom( - com.google.protobuf.CodedInputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws java.io.IOException { - if (extensionRegistry == null) { - throw new java.lang.NullPointerException(); - } - try { - boolean done = false; - while (!done) { - int tag = input.readTag(); - switch (tag) { - case 0: - done = true; - break; - case 10: { - io.github.dfa1.vortex.proto.EncodingProtos.PcoPageInfo m = - input.readMessage( - io.github.dfa1.vortex.proto.EncodingProtos.PcoPageInfo.parser(), - extensionRegistry); - if (pagesBuilder_ == null) { - ensurePagesIsMutable(); - pages_.add(m); - } else { - pagesBuilder_.addMessage(m); - } - break; - } // case 10 - default: { - if (!super.parseUnknownField(input, extensionRegistry, tag)) { - done = true; // was an endgroup tag - } - break; - } // default: - } // switch (tag) - } // while (!done) - } catch (com.google.protobuf.InvalidProtocolBufferException e) { - throw e.unwrapIOException(); - } finally { - onChanged(); - } // finally - return this; - } - private int bitField0_; - - private java.util.List pages_ = - java.util.Collections.emptyList(); - private void ensurePagesIsMutable() { - if (!((bitField0_ & 0x00000001) != 0)) { - pages_ = new java.util.ArrayList(pages_); - bitField0_ |= 0x00000001; - } - } - - private com.google.protobuf.RepeatedFieldBuilder< - io.github.dfa1.vortex.proto.EncodingProtos.PcoPageInfo, io.github.dfa1.vortex.proto.EncodingProtos.PcoPageInfo.Builder, io.github.dfa1.vortex.proto.EncodingProtos.PcoPageInfoOrBuilder> pagesBuilder_; - - /** - * repeated .vortex.encodings.PcoPageInfo pages = 1; - */ - public java.util.List getPagesList() { - if (pagesBuilder_ == null) { - return java.util.Collections.unmodifiableList(pages_); - } else { - return pagesBuilder_.getMessageList(); - } - } - /** - * repeated .vortex.encodings.PcoPageInfo pages = 1; - */ - public int getPagesCount() { - if (pagesBuilder_ == null) { - return pages_.size(); - } else { - return pagesBuilder_.getCount(); - } - } - /** - * repeated .vortex.encodings.PcoPageInfo pages = 1; - */ - public io.github.dfa1.vortex.proto.EncodingProtos.PcoPageInfo getPages(int index) { - if (pagesBuilder_ == null) { - return pages_.get(index); - } else { - return pagesBuilder_.getMessage(index); - } - } - /** - * repeated .vortex.encodings.PcoPageInfo pages = 1; - */ - public Builder setPages( - int index, io.github.dfa1.vortex.proto.EncodingProtos.PcoPageInfo value) { - if (pagesBuilder_ == null) { - if (value == null) { - throw new NullPointerException(); - } - ensurePagesIsMutable(); - pages_.set(index, value); - onChanged(); - } else { - pagesBuilder_.setMessage(index, value); - } - return this; - } - /** - * repeated .vortex.encodings.PcoPageInfo pages = 1; - */ - public Builder setPages( - int index, io.github.dfa1.vortex.proto.EncodingProtos.PcoPageInfo.Builder builderForValue) { - if (pagesBuilder_ == null) { - ensurePagesIsMutable(); - pages_.set(index, builderForValue.build()); - onChanged(); - } else { - pagesBuilder_.setMessage(index, builderForValue.build()); - } - return this; - } - /** - * repeated .vortex.encodings.PcoPageInfo pages = 1; - */ - public Builder addPages(io.github.dfa1.vortex.proto.EncodingProtos.PcoPageInfo value) { - if (pagesBuilder_ == null) { - if (value == null) { - throw new NullPointerException(); - } - ensurePagesIsMutable(); - pages_.add(value); - onChanged(); - } else { - pagesBuilder_.addMessage(value); - } - return this; - } - /** - * repeated .vortex.encodings.PcoPageInfo pages = 1; - */ - public Builder addPages( - int index, io.github.dfa1.vortex.proto.EncodingProtos.PcoPageInfo value) { - if (pagesBuilder_ == null) { - if (value == null) { - throw new NullPointerException(); - } - ensurePagesIsMutable(); - pages_.add(index, value); - onChanged(); - } else { - pagesBuilder_.addMessage(index, value); - } - return this; - } - /** - * repeated .vortex.encodings.PcoPageInfo pages = 1; - */ - public Builder addPages( - io.github.dfa1.vortex.proto.EncodingProtos.PcoPageInfo.Builder builderForValue) { - if (pagesBuilder_ == null) { - ensurePagesIsMutable(); - pages_.add(builderForValue.build()); - onChanged(); - } else { - pagesBuilder_.addMessage(builderForValue.build()); - } - return this; - } - /** - * repeated .vortex.encodings.PcoPageInfo pages = 1; - */ - public Builder addPages( - int index, io.github.dfa1.vortex.proto.EncodingProtos.PcoPageInfo.Builder builderForValue) { - if (pagesBuilder_ == null) { - ensurePagesIsMutable(); - pages_.add(index, builderForValue.build()); - onChanged(); - } else { - pagesBuilder_.addMessage(index, builderForValue.build()); - } - return this; - } - /** - * repeated .vortex.encodings.PcoPageInfo pages = 1; - */ - public Builder addAllPages( - java.lang.Iterable values) { - if (pagesBuilder_ == null) { - ensurePagesIsMutable(); - com.google.protobuf.AbstractMessageLite.Builder.addAll( - values, pages_); - onChanged(); - } else { - pagesBuilder_.addAllMessages(values); - } - return this; - } - /** - * repeated .vortex.encodings.PcoPageInfo pages = 1; - */ - public Builder clearPages() { - if (pagesBuilder_ == null) { - pages_ = java.util.Collections.emptyList(); - bitField0_ = (bitField0_ & ~0x00000001); - onChanged(); - } else { - pagesBuilder_.clear(); - } - return this; - } - /** - * repeated .vortex.encodings.PcoPageInfo pages = 1; - */ - public Builder removePages(int index) { - if (pagesBuilder_ == null) { - ensurePagesIsMutable(); - pages_.remove(index); - onChanged(); - } else { - pagesBuilder_.remove(index); - } - return this; - } - /** - * repeated .vortex.encodings.PcoPageInfo pages = 1; - */ - public io.github.dfa1.vortex.proto.EncodingProtos.PcoPageInfo.Builder getPagesBuilder( - int index) { - return internalGetPagesFieldBuilder().getBuilder(index); - } - /** - * repeated .vortex.encodings.PcoPageInfo pages = 1; - */ - public io.github.dfa1.vortex.proto.EncodingProtos.PcoPageInfoOrBuilder getPagesOrBuilder( - int index) { - if (pagesBuilder_ == null) { - return pages_.get(index); } else { - return pagesBuilder_.getMessageOrBuilder(index); - } - } - /** - * repeated .vortex.encodings.PcoPageInfo pages = 1; - */ - public java.util.List - getPagesOrBuilderList() { - if (pagesBuilder_ != null) { - return pagesBuilder_.getMessageOrBuilderList(); - } else { - return java.util.Collections.unmodifiableList(pages_); - } - } - /** - * repeated .vortex.encodings.PcoPageInfo pages = 1; - */ - public io.github.dfa1.vortex.proto.EncodingProtos.PcoPageInfo.Builder addPagesBuilder() { - return internalGetPagesFieldBuilder().addBuilder( - io.github.dfa1.vortex.proto.EncodingProtos.PcoPageInfo.getDefaultInstance()); - } - /** - * repeated .vortex.encodings.PcoPageInfo pages = 1; - */ - public io.github.dfa1.vortex.proto.EncodingProtos.PcoPageInfo.Builder addPagesBuilder( - int index) { - return internalGetPagesFieldBuilder().addBuilder( - index, io.github.dfa1.vortex.proto.EncodingProtos.PcoPageInfo.getDefaultInstance()); - } - /** - * repeated .vortex.encodings.PcoPageInfo pages = 1; - */ - public java.util.List - getPagesBuilderList() { - return internalGetPagesFieldBuilder().getBuilderList(); - } - private com.google.protobuf.RepeatedFieldBuilder< - io.github.dfa1.vortex.proto.EncodingProtos.PcoPageInfo, io.github.dfa1.vortex.proto.EncodingProtos.PcoPageInfo.Builder, io.github.dfa1.vortex.proto.EncodingProtos.PcoPageInfoOrBuilder> - internalGetPagesFieldBuilder() { - if (pagesBuilder_ == null) { - pagesBuilder_ = new com.google.protobuf.RepeatedFieldBuilder< - io.github.dfa1.vortex.proto.EncodingProtos.PcoPageInfo, io.github.dfa1.vortex.proto.EncodingProtos.PcoPageInfo.Builder, io.github.dfa1.vortex.proto.EncodingProtos.PcoPageInfoOrBuilder>( - pages_, - ((bitField0_ & 0x00000001) != 0), - getParentForChildren(), - isClean()); - pages_ = null; - } - return pagesBuilder_; - } - - // @@protoc_insertion_point(builder_scope:vortex.encodings.PcoChunkInfo) - } - - // @@protoc_insertion_point(class_scope:vortex.encodings.PcoChunkInfo) - private static final io.github.dfa1.vortex.proto.EncodingProtos.PcoChunkInfo DEFAULT_INSTANCE; - static { - DEFAULT_INSTANCE = new io.github.dfa1.vortex.proto.EncodingProtos.PcoChunkInfo(); - } - - public static io.github.dfa1.vortex.proto.EncodingProtos.PcoChunkInfo getDefaultInstance() { - return DEFAULT_INSTANCE; - } - - private static final com.google.protobuf.Parser - PARSER = new com.google.protobuf.AbstractParser() { - @java.lang.Override - public PcoChunkInfo parsePartialFrom( - com.google.protobuf.CodedInputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws com.google.protobuf.InvalidProtocolBufferException { - Builder builder = newBuilder(); - try { - builder.mergeFrom(input, extensionRegistry); - } catch (com.google.protobuf.InvalidProtocolBufferException e) { - throw e.setUnfinishedMessage(builder.buildPartial()); - } catch (com.google.protobuf.UninitializedMessageException e) { - throw e.asInvalidProtocolBufferException().setUnfinishedMessage(builder.buildPartial()); - } catch (java.io.IOException e) { - throw new com.google.protobuf.InvalidProtocolBufferException(e) - .setUnfinishedMessage(builder.buildPartial()); - } - return builder.buildPartial(); - } - }; - - public static com.google.protobuf.Parser parser() { - return PARSER; - } - - @java.lang.Override - public com.google.protobuf.Parser getParserForType() { - return PARSER; - } - - @java.lang.Override - public io.github.dfa1.vortex.proto.EncodingProtos.PcoChunkInfo getDefaultInstanceForType() { - return DEFAULT_INSTANCE; - } - - } - - public interface PcoMetadataOrBuilder extends - // @@protoc_insertion_point(interface_extends:vortex.encodings.PcoMetadata) - com.google.protobuf.MessageOrBuilder { - - /** - * bytes header = 1; - * @return The header. - */ - com.google.protobuf.ByteString getHeader(); - - /** - * repeated .vortex.encodings.PcoChunkInfo chunks = 2; - */ - java.util.List - getChunksList(); - /** - * repeated .vortex.encodings.PcoChunkInfo chunks = 2; - */ - io.github.dfa1.vortex.proto.EncodingProtos.PcoChunkInfo getChunks(int index); - /** - * repeated .vortex.encodings.PcoChunkInfo chunks = 2; - */ - int getChunksCount(); - /** - * repeated .vortex.encodings.PcoChunkInfo chunks = 2; - */ - java.util.List - getChunksOrBuilderList(); - /** - * repeated .vortex.encodings.PcoChunkInfo chunks = 2; - */ - io.github.dfa1.vortex.proto.EncodingProtos.PcoChunkInfoOrBuilder getChunksOrBuilder( - int index); - } - /** - * Protobuf type {@code vortex.encodings.PcoMetadata} - */ - public static final class PcoMetadata extends - com.google.protobuf.GeneratedMessage implements - // @@protoc_insertion_point(message_implements:vortex.encodings.PcoMetadata) - PcoMetadataOrBuilder { - private static final long serialVersionUID = 0L; - static { - com.google.protobuf.RuntimeVersion.validateProtobufGencodeVersion( - com.google.protobuf.RuntimeVersion.RuntimeDomain.PUBLIC, - /* major= */ 4, - /* minor= */ 35, - /* patch= */ 0, - /* suffix= */ "", - "PcoMetadata"); - } - // Use PcoMetadata.newBuilder() to construct. - private PcoMetadata(com.google.protobuf.GeneratedMessage.Builder builder) { - super(builder); - } - private PcoMetadata() { - header_ = com.google.protobuf.ByteString.EMPTY; - chunks_ = java.util.Collections.emptyList(); - } - - public static final com.google.protobuf.Descriptors.Descriptor - getDescriptor() { - return io.github.dfa1.vortex.proto.EncodingProtos.internal_static_vortex_encodings_PcoMetadata_descriptor; - } - - @java.lang.Override - public com.google.protobuf.Descriptors.Descriptor getDescriptorForType() { - return io.github.dfa1.vortex.proto.EncodingProtos.internal_static_vortex_encodings_PcoMetadata_descriptor; - } - - @java.lang.Override - protected com.google.protobuf.GeneratedMessage.FieldAccessorTable - internalGetFieldAccessorTable() { - return io.github.dfa1.vortex.proto.EncodingProtos.internal_static_vortex_encodings_PcoMetadata_fieldAccessorTable - .ensureFieldAccessorsInitialized( - io.github.dfa1.vortex.proto.EncodingProtos.PcoMetadata.class, io.github.dfa1.vortex.proto.EncodingProtos.PcoMetadata.Builder.class); - } - - public static final int HEADER_FIELD_NUMBER = 1; - private com.google.protobuf.ByteString header_ = com.google.protobuf.ByteString.EMPTY; - /** - * bytes header = 1; - * @return The header. - */ - @java.lang.Override - public com.google.protobuf.ByteString getHeader() { - return header_; - } - - public static final int CHUNKS_FIELD_NUMBER = 2; - @SuppressWarnings("serial") - private java.util.List chunks_; - /** - * repeated .vortex.encodings.PcoChunkInfo chunks = 2; - */ - @java.lang.Override - public java.util.List getChunksList() { - return chunks_; - } - /** - * repeated .vortex.encodings.PcoChunkInfo chunks = 2; - */ - @java.lang.Override - public java.util.List - getChunksOrBuilderList() { - return chunks_; - } - /** - * repeated .vortex.encodings.PcoChunkInfo chunks = 2; - */ - @java.lang.Override - public int getChunksCount() { - return chunks_.size(); - } - /** - * repeated .vortex.encodings.PcoChunkInfo chunks = 2; - */ - @java.lang.Override - public io.github.dfa1.vortex.proto.EncodingProtos.PcoChunkInfo getChunks(int index) { - return chunks_.get(index); - } - /** - * repeated .vortex.encodings.PcoChunkInfo chunks = 2; - */ - @java.lang.Override - public io.github.dfa1.vortex.proto.EncodingProtos.PcoChunkInfoOrBuilder getChunksOrBuilder( - int index) { - return chunks_.get(index); - } - - private byte memoizedIsInitialized = -1; - @java.lang.Override - public final boolean isInitialized() { - byte isInitialized = memoizedIsInitialized; - if (isInitialized == 1) return true; - if (isInitialized == 0) return false; - - memoizedIsInitialized = 1; - return true; - } - - @java.lang.Override - public void writeTo(com.google.protobuf.CodedOutputStream output) - throws java.io.IOException { - if (!header_.isEmpty()) { - output.writeBytes(1, header_); - } - for (int i = 0; i < chunks_.size(); i++) { - output.writeMessage(2, chunks_.get(i)); - } - getUnknownFields().writeTo(output); - } - private int computeSerializedSize_0() { - int size = 0; - if (!header_.isEmpty()) { - size += com.google.protobuf.CodedOutputStream - .computeBytesSize(1, header_); - } - - { - final int count = chunks_.size(); - for (int i = 0; i < count; i++) { - size += com.google.protobuf.CodedOutputStream - .computeMessageSizeNoTag(chunks_.get(i)); - } - size += 1 * count; - } - return size; - } - @java.lang.Override - public int getSerializedSize() { - int size = memoizedSize; - if (size != -1) return size; - - size = 0; - size += computeSerializedSize_0(); - size += getUnknownFields().getSerializedSize(); - memoizedSize = size; - return size; - } - - @java.lang.Override - public boolean equals(final java.lang.Object obj) { - if (obj == this) { - return true; - } - if (!(obj instanceof io.github.dfa1.vortex.proto.EncodingProtos.PcoMetadata)) { - return super.equals(obj); - } - io.github.dfa1.vortex.proto.EncodingProtos.PcoMetadata other = (io.github.dfa1.vortex.proto.EncodingProtos.PcoMetadata) obj; - - if (!getHeader() - .equals(other.getHeader())) return false; - if (!getChunksList() - .equals(other.getChunksList())) return false; - if (!getUnknownFields().equals(other.getUnknownFields())) return false; - return true; - } - - @java.lang.Override - public int hashCode() { - if (memoizedHashCode != 0) { - return memoizedHashCode; - } - int hash = 41; - hash = (19 * hash) + getDescriptor().hashCode(); - hash = (37 * hash) + HEADER_FIELD_NUMBER; - hash = (53 * hash) + getHeader().hashCode(); - if (getChunksCount() > 0) { - hash = (37 * hash) + CHUNKS_FIELD_NUMBER; - hash = (53 * hash) + getChunksList().hashCode(); - } - hash = (29 * hash) + getUnknownFields().hashCode(); - memoizedHashCode = hash; - return hash; - } - - public static io.github.dfa1.vortex.proto.EncodingProtos.PcoMetadata parseFrom( - java.nio.ByteBuffer data) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.PcoMetadata parseFrom( - java.nio.ByteBuffer data, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data, extensionRegistry); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.PcoMetadata parseFrom( - com.google.protobuf.ByteString data) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.PcoMetadata parseFrom( - com.google.protobuf.ByteString data, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data, extensionRegistry); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.PcoMetadata parseFrom(byte[] data) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.PcoMetadata parseFrom( - byte[] data, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data, extensionRegistry); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.PcoMetadata parseFrom(java.io.InputStream input) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseWithIOException(PARSER, input); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.PcoMetadata parseFrom( - java.io.InputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseWithIOException(PARSER, input, extensionRegistry); - } - - public static io.github.dfa1.vortex.proto.EncodingProtos.PcoMetadata parseDelimitedFrom(java.io.InputStream input) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseDelimitedWithIOException(PARSER, input); - } - - public static io.github.dfa1.vortex.proto.EncodingProtos.PcoMetadata parseDelimitedFrom( - java.io.InputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseDelimitedWithIOException(PARSER, input, extensionRegistry); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.PcoMetadata parseFrom( - com.google.protobuf.CodedInputStream input) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseWithIOException(PARSER, input); - } - public static io.github.dfa1.vortex.proto.EncodingProtos.PcoMetadata parseFrom( - com.google.protobuf.CodedInputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseWithIOException(PARSER, input, extensionRegistry); - } - - @java.lang.Override - public Builder newBuilderForType() { return newBuilder(); } - public static Builder newBuilder() { - return DEFAULT_INSTANCE.toBuilder(); - } - public static Builder newBuilder(io.github.dfa1.vortex.proto.EncodingProtos.PcoMetadata prototype) { - return DEFAULT_INSTANCE.toBuilder().mergeFrom(prototype); - } - @java.lang.Override - public Builder toBuilder() { - return this == DEFAULT_INSTANCE - ? new Builder() : new Builder().mergeFrom(this); - } - - @java.lang.Override - protected Builder newBuilderForType( - com.google.protobuf.GeneratedMessage.BuilderParent parent) { - Builder builder = new Builder(parent); - return builder; - } - /** - * Protobuf type {@code vortex.encodings.PcoMetadata} - */ - public static final class Builder extends - com.google.protobuf.GeneratedMessage.Builder implements - // @@protoc_insertion_point(builder_implements:vortex.encodings.PcoMetadata) - io.github.dfa1.vortex.proto.EncodingProtos.PcoMetadataOrBuilder { - public static final com.google.protobuf.Descriptors.Descriptor - getDescriptor() { - return io.github.dfa1.vortex.proto.EncodingProtos.internal_static_vortex_encodings_PcoMetadata_descriptor; - } - - @java.lang.Override - protected com.google.protobuf.GeneratedMessage.FieldAccessorTable - internalGetFieldAccessorTable() { - return io.github.dfa1.vortex.proto.EncodingProtos.internal_static_vortex_encodings_PcoMetadata_fieldAccessorTable - .ensureFieldAccessorsInitialized( - io.github.dfa1.vortex.proto.EncodingProtos.PcoMetadata.class, io.github.dfa1.vortex.proto.EncodingProtos.PcoMetadata.Builder.class); - } - - // Construct using io.github.dfa1.vortex.proto.EncodingProtos.PcoMetadata.newBuilder() - private Builder() { - - } - - private Builder( - com.google.protobuf.GeneratedMessage.BuilderParent parent) { - super(parent); - - } - @java.lang.Override - public Builder clear() { - super.clear(); - bitField0_ = 0; - header_ = com.google.protobuf.ByteString.EMPTY; - if (chunksBuilder_ == null) { - chunks_ = java.util.Collections.emptyList(); - } else { - chunks_ = null; - chunksBuilder_.clear(); - } - bitField0_ = (bitField0_ & ~0x00000002); - return this; - } - - @java.lang.Override - public com.google.protobuf.Descriptors.Descriptor - getDescriptorForType() { - return io.github.dfa1.vortex.proto.EncodingProtos.internal_static_vortex_encodings_PcoMetadata_descriptor; - } - - @java.lang.Override - public io.github.dfa1.vortex.proto.EncodingProtos.PcoMetadata getDefaultInstanceForType() { - return io.github.dfa1.vortex.proto.EncodingProtos.PcoMetadata.getDefaultInstance(); - } - - @java.lang.Override - public io.github.dfa1.vortex.proto.EncodingProtos.PcoMetadata build() { - io.github.dfa1.vortex.proto.EncodingProtos.PcoMetadata result = buildPartial(); - if (!result.isInitialized()) { - throw newUninitializedMessageException(result); - } - return result; - } - - @java.lang.Override - public io.github.dfa1.vortex.proto.EncodingProtos.PcoMetadata buildPartial() { - io.github.dfa1.vortex.proto.EncodingProtos.PcoMetadata result = new io.github.dfa1.vortex.proto.EncodingProtos.PcoMetadata(this); - buildPartialRepeatedFields(result); - if (bitField0_ != 0) { buildPartial0(result); } - onBuilt(); - return result; - } - - private void buildPartialRepeatedFields(io.github.dfa1.vortex.proto.EncodingProtos.PcoMetadata result) { - if (chunksBuilder_ == null) { - if (((bitField0_ & 0x00000002) != 0)) { - chunks_ = java.util.Collections.unmodifiableList(chunks_); - bitField0_ = (bitField0_ & ~0x00000002); - } - result.chunks_ = chunks_; - } else { - result.chunks_ = chunksBuilder_.build(); - } - } - - private void buildPartial0(io.github.dfa1.vortex.proto.EncodingProtos.PcoMetadata result) { - int from_bitField0_ = bitField0_; - if (((from_bitField0_ & 0x00000001) != 0)) { - result.header_ = header_; - } - } - - @java.lang.Override - public Builder mergeFrom(com.google.protobuf.Message other) { - if (other instanceof io.github.dfa1.vortex.proto.EncodingProtos.PcoMetadata) { - return mergeFrom((io.github.dfa1.vortex.proto.EncodingProtos.PcoMetadata)other); - } else { - super.mergeFrom(other); - return this; - } - } - - public Builder mergeFrom(io.github.dfa1.vortex.proto.EncodingProtos.PcoMetadata other) { - if (other == io.github.dfa1.vortex.proto.EncodingProtos.PcoMetadata.getDefaultInstance()) return this; - if (!other.getHeader().isEmpty()) { - setHeader(other.getHeader()); - } - if (chunksBuilder_ == null) { - if (!other.chunks_.isEmpty()) { - if (chunks_.isEmpty()) { - chunks_ = other.chunks_; - bitField0_ = (bitField0_ & ~0x00000002); - } else { - ensureChunksIsMutable(); - chunks_.addAll(other.chunks_); - } - onChanged(); - } - } else { - if (!other.chunks_.isEmpty()) { - if (chunksBuilder_.isEmpty()) { - chunksBuilder_.dispose(); - chunksBuilder_ = null; - chunks_ = other.chunks_; - bitField0_ = (bitField0_ & ~0x00000002); - chunksBuilder_ = - com.google.protobuf.GeneratedMessage.alwaysUseFieldBuilders ? - internalGetChunksFieldBuilder() : null; - } else { - chunksBuilder_.addAllMessages(other.chunks_); - } - } - } - this.mergeUnknownFields(other.getUnknownFields()); - onChanged(); - return this; - } - - @java.lang.Override - public final boolean isInitialized() { - return true; - } - - @java.lang.Override - public Builder mergeFrom( - com.google.protobuf.CodedInputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws java.io.IOException { - if (extensionRegistry == null) { - throw new java.lang.NullPointerException(); - } - try { - boolean done = false; - while (!done) { - int tag = input.readTag(); - switch (tag) { - case 0: - done = true; - break; - case 10: { - header_ = input.readBytes(); - bitField0_ |= 0x00000001; - break; - } // case 10 - case 18: { - io.github.dfa1.vortex.proto.EncodingProtos.PcoChunkInfo m = - input.readMessage( - io.github.dfa1.vortex.proto.EncodingProtos.PcoChunkInfo.parser(), - extensionRegistry); - if (chunksBuilder_ == null) { - ensureChunksIsMutable(); - chunks_.add(m); - } else { - chunksBuilder_.addMessage(m); - } - break; - } // case 18 - default: { - if (!super.parseUnknownField(input, extensionRegistry, tag)) { - done = true; // was an endgroup tag - } - break; - } // default: - } // switch (tag) - } // while (!done) - } catch (com.google.protobuf.InvalidProtocolBufferException e) { - throw e.unwrapIOException(); - } finally { - onChanged(); - } // finally - return this; - } - private int bitField0_; - - private com.google.protobuf.ByteString header_ = com.google.protobuf.ByteString.EMPTY; - /** - * bytes header = 1; - * @return The header. - */ - @java.lang.Override - public com.google.protobuf.ByteString getHeader() { - return header_; - } - /** - * bytes header = 1; - * @param value The header to set. - * @return This builder for chaining. - */ - public Builder setHeader(com.google.protobuf.ByteString value) { - if (value == null) { throw new NullPointerException(); } - header_ = value; - bitField0_ |= 0x00000001; - onChanged(); - return this; - } - /** - * bytes header = 1; - * @return This builder for chaining. - */ - public Builder clearHeader() { - bitField0_ = (bitField0_ & ~0x00000001); - header_ = getDefaultInstance().getHeader(); - onChanged(); - return this; - } - - private java.util.List chunks_ = - java.util.Collections.emptyList(); - private void ensureChunksIsMutable() { - if (!((bitField0_ & 0x00000002) != 0)) { - chunks_ = new java.util.ArrayList(chunks_); - bitField0_ |= 0x00000002; - } - } - - private com.google.protobuf.RepeatedFieldBuilder< - io.github.dfa1.vortex.proto.EncodingProtos.PcoChunkInfo, io.github.dfa1.vortex.proto.EncodingProtos.PcoChunkInfo.Builder, io.github.dfa1.vortex.proto.EncodingProtos.PcoChunkInfoOrBuilder> chunksBuilder_; - - /** - * repeated .vortex.encodings.PcoChunkInfo chunks = 2; - */ - public java.util.List getChunksList() { - if (chunksBuilder_ == null) { - return java.util.Collections.unmodifiableList(chunks_); - } else { - return chunksBuilder_.getMessageList(); - } - } - /** - * repeated .vortex.encodings.PcoChunkInfo chunks = 2; - */ - public int getChunksCount() { - if (chunksBuilder_ == null) { - return chunks_.size(); - } else { - return chunksBuilder_.getCount(); - } - } - /** - * repeated .vortex.encodings.PcoChunkInfo chunks = 2; - */ - public io.github.dfa1.vortex.proto.EncodingProtos.PcoChunkInfo getChunks(int index) { - if (chunksBuilder_ == null) { - return chunks_.get(index); - } else { - return chunksBuilder_.getMessage(index); - } - } - /** - * repeated .vortex.encodings.PcoChunkInfo chunks = 2; - */ - public Builder setChunks( - int index, io.github.dfa1.vortex.proto.EncodingProtos.PcoChunkInfo value) { - if (chunksBuilder_ == null) { - if (value == null) { - throw new NullPointerException(); - } - ensureChunksIsMutable(); - chunks_.set(index, value); - onChanged(); - } else { - chunksBuilder_.setMessage(index, value); - } - return this; - } - /** - * repeated .vortex.encodings.PcoChunkInfo chunks = 2; - */ - public Builder setChunks( - int index, io.github.dfa1.vortex.proto.EncodingProtos.PcoChunkInfo.Builder builderForValue) { - if (chunksBuilder_ == null) { - ensureChunksIsMutable(); - chunks_.set(index, builderForValue.build()); - onChanged(); - } else { - chunksBuilder_.setMessage(index, builderForValue.build()); - } - return this; - } - /** - * repeated .vortex.encodings.PcoChunkInfo chunks = 2; - */ - public Builder addChunks(io.github.dfa1.vortex.proto.EncodingProtos.PcoChunkInfo value) { - if (chunksBuilder_ == null) { - if (value == null) { - throw new NullPointerException(); - } - ensureChunksIsMutable(); - chunks_.add(value); - onChanged(); - } else { - chunksBuilder_.addMessage(value); - } - return this; - } - /** - * repeated .vortex.encodings.PcoChunkInfo chunks = 2; - */ - public Builder addChunks( - int index, io.github.dfa1.vortex.proto.EncodingProtos.PcoChunkInfo value) { - if (chunksBuilder_ == null) { - if (value == null) { - throw new NullPointerException(); - } - ensureChunksIsMutable(); - chunks_.add(index, value); - onChanged(); - } else { - chunksBuilder_.addMessage(index, value); - } - return this; - } - /** - * repeated .vortex.encodings.PcoChunkInfo chunks = 2; - */ - public Builder addChunks( - io.github.dfa1.vortex.proto.EncodingProtos.PcoChunkInfo.Builder builderForValue) { - if (chunksBuilder_ == null) { - ensureChunksIsMutable(); - chunks_.add(builderForValue.build()); - onChanged(); - } else { - chunksBuilder_.addMessage(builderForValue.build()); - } - return this; - } - /** - * repeated .vortex.encodings.PcoChunkInfo chunks = 2; - */ - public Builder addChunks( - int index, io.github.dfa1.vortex.proto.EncodingProtos.PcoChunkInfo.Builder builderForValue) { - if (chunksBuilder_ == null) { - ensureChunksIsMutable(); - chunks_.add(index, builderForValue.build()); - onChanged(); - } else { - chunksBuilder_.addMessage(index, builderForValue.build()); - } - return this; - } - /** - * repeated .vortex.encodings.PcoChunkInfo chunks = 2; - */ - public Builder addAllChunks( - java.lang.Iterable values) { - if (chunksBuilder_ == null) { - ensureChunksIsMutable(); - com.google.protobuf.AbstractMessageLite.Builder.addAll( - values, chunks_); - onChanged(); - } else { - chunksBuilder_.addAllMessages(values); - } - return this; - } - /** - * repeated .vortex.encodings.PcoChunkInfo chunks = 2; - */ - public Builder clearChunks() { - if (chunksBuilder_ == null) { - chunks_ = java.util.Collections.emptyList(); - bitField0_ = (bitField0_ & ~0x00000002); - onChanged(); - } else { - chunksBuilder_.clear(); - } - return this; - } - /** - * repeated .vortex.encodings.PcoChunkInfo chunks = 2; - */ - public Builder removeChunks(int index) { - if (chunksBuilder_ == null) { - ensureChunksIsMutable(); - chunks_.remove(index); - onChanged(); - } else { - chunksBuilder_.remove(index); - } - return this; - } - /** - * repeated .vortex.encodings.PcoChunkInfo chunks = 2; - */ - public io.github.dfa1.vortex.proto.EncodingProtos.PcoChunkInfo.Builder getChunksBuilder( - int index) { - return internalGetChunksFieldBuilder().getBuilder(index); - } - /** - * repeated .vortex.encodings.PcoChunkInfo chunks = 2; - */ - public io.github.dfa1.vortex.proto.EncodingProtos.PcoChunkInfoOrBuilder getChunksOrBuilder( - int index) { - if (chunksBuilder_ == null) { - return chunks_.get(index); } else { - return chunksBuilder_.getMessageOrBuilder(index); - } - } - /** - * repeated .vortex.encodings.PcoChunkInfo chunks = 2; - */ - public java.util.List - getChunksOrBuilderList() { - if (chunksBuilder_ != null) { - return chunksBuilder_.getMessageOrBuilderList(); - } else { - return java.util.Collections.unmodifiableList(chunks_); - } - } - /** - * repeated .vortex.encodings.PcoChunkInfo chunks = 2; - */ - public io.github.dfa1.vortex.proto.EncodingProtos.PcoChunkInfo.Builder addChunksBuilder() { - return internalGetChunksFieldBuilder().addBuilder( - io.github.dfa1.vortex.proto.EncodingProtos.PcoChunkInfo.getDefaultInstance()); - } - /** - * repeated .vortex.encodings.PcoChunkInfo chunks = 2; - */ - public io.github.dfa1.vortex.proto.EncodingProtos.PcoChunkInfo.Builder addChunksBuilder( - int index) { - return internalGetChunksFieldBuilder().addBuilder( - index, io.github.dfa1.vortex.proto.EncodingProtos.PcoChunkInfo.getDefaultInstance()); - } - /** - * repeated .vortex.encodings.PcoChunkInfo chunks = 2; - */ - public java.util.List - getChunksBuilderList() { - return internalGetChunksFieldBuilder().getBuilderList(); - } - private com.google.protobuf.RepeatedFieldBuilder< - io.github.dfa1.vortex.proto.EncodingProtos.PcoChunkInfo, io.github.dfa1.vortex.proto.EncodingProtos.PcoChunkInfo.Builder, io.github.dfa1.vortex.proto.EncodingProtos.PcoChunkInfoOrBuilder> - internalGetChunksFieldBuilder() { - if (chunksBuilder_ == null) { - chunksBuilder_ = new com.google.protobuf.RepeatedFieldBuilder< - io.github.dfa1.vortex.proto.EncodingProtos.PcoChunkInfo, io.github.dfa1.vortex.proto.EncodingProtos.PcoChunkInfo.Builder, io.github.dfa1.vortex.proto.EncodingProtos.PcoChunkInfoOrBuilder>( - chunks_, - ((bitField0_ & 0x00000002) != 0), - getParentForChildren(), - isClean()); - chunks_ = null; - } - return chunksBuilder_; - } - - // @@protoc_insertion_point(builder_scope:vortex.encodings.PcoMetadata) - } - - // @@protoc_insertion_point(class_scope:vortex.encodings.PcoMetadata) - private static final io.github.dfa1.vortex.proto.EncodingProtos.PcoMetadata DEFAULT_INSTANCE; - static { - DEFAULT_INSTANCE = new io.github.dfa1.vortex.proto.EncodingProtos.PcoMetadata(); - } - - public static io.github.dfa1.vortex.proto.EncodingProtos.PcoMetadata getDefaultInstance() { - return DEFAULT_INSTANCE; - } - - private static final com.google.protobuf.Parser - PARSER = new com.google.protobuf.AbstractParser() { - @java.lang.Override - public PcoMetadata parsePartialFrom( - com.google.protobuf.CodedInputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws com.google.protobuf.InvalidProtocolBufferException { - Builder builder = newBuilder(); - try { - builder.mergeFrom(input, extensionRegistry); - } catch (com.google.protobuf.InvalidProtocolBufferException e) { - throw e.setUnfinishedMessage(builder.buildPartial()); - } catch (com.google.protobuf.UninitializedMessageException e) { - throw e.asInvalidProtocolBufferException().setUnfinishedMessage(builder.buildPartial()); - } catch (java.io.IOException e) { - throw new com.google.protobuf.InvalidProtocolBufferException(e) - .setUnfinishedMessage(builder.buildPartial()); - } - return builder.buildPartial(); - } - }; - - public static com.google.protobuf.Parser parser() { - return PARSER; - } - - @java.lang.Override - public com.google.protobuf.Parser getParserForType() { - return PARSER; - } - - @java.lang.Override - public io.github.dfa1.vortex.proto.EncodingProtos.PcoMetadata getDefaultInstanceForType() { - return DEFAULT_INSTANCE; - } - - } - - private static final com.google.protobuf.Descriptors.Descriptor - internal_static_vortex_encodings_PatchesMetadata_descriptor; - private static final - com.google.protobuf.GeneratedMessage.FieldAccessorTable - internal_static_vortex_encodings_PatchesMetadata_fieldAccessorTable; - private static final com.google.protobuf.Descriptors.Descriptor - internal_static_vortex_encodings_SparseMetadata_descriptor; - private static final - com.google.protobuf.GeneratedMessage.FieldAccessorTable - internal_static_vortex_encodings_SparseMetadata_fieldAccessorTable; - private static final com.google.protobuf.Descriptors.Descriptor - internal_static_vortex_encodings_DictMetadata_descriptor; - private static final - com.google.protobuf.GeneratedMessage.FieldAccessorTable - internal_static_vortex_encodings_DictMetadata_fieldAccessorTable; - private static final com.google.protobuf.Descriptors.Descriptor - internal_static_vortex_encodings_RunEndMetadata_descriptor; - private static final - com.google.protobuf.GeneratedMessage.FieldAccessorTable - internal_static_vortex_encodings_RunEndMetadata_fieldAccessorTable; - private static final com.google.protobuf.Descriptors.Descriptor - internal_static_vortex_encodings_ALPMetadata_descriptor; - private static final - com.google.protobuf.GeneratedMessage.FieldAccessorTable - internal_static_vortex_encodings_ALPMetadata_fieldAccessorTable; - private static final com.google.protobuf.Descriptors.Descriptor - internal_static_vortex_encodings_BitPackedMetadata_descriptor; - private static final - com.google.protobuf.GeneratedMessage.FieldAccessorTable - internal_static_vortex_encodings_BitPackedMetadata_fieldAccessorTable; - private static final com.google.protobuf.Descriptors.Descriptor - internal_static_vortex_encodings_SequenceMetadata_descriptor; - private static final - com.google.protobuf.GeneratedMessage.FieldAccessorTable - internal_static_vortex_encodings_SequenceMetadata_fieldAccessorTable; - private static final com.google.protobuf.Descriptors.Descriptor - internal_static_vortex_encodings_VarBinMetadata_descriptor; - private static final - com.google.protobuf.GeneratedMessage.FieldAccessorTable - internal_static_vortex_encodings_VarBinMetadata_fieldAccessorTable; - private static final com.google.protobuf.Descriptors.Descriptor - internal_static_vortex_encodings_FSSTMetadata_descriptor; - private static final - com.google.protobuf.GeneratedMessage.FieldAccessorTable - internal_static_vortex_encodings_FSSTMetadata_fieldAccessorTable; - private static final com.google.protobuf.Descriptors.Descriptor - internal_static_vortex_encodings_DeltaMetadata_descriptor; - private static final - com.google.protobuf.GeneratedMessage.FieldAccessorTable - internal_static_vortex_encodings_DeltaMetadata_fieldAccessorTable; - private static final com.google.protobuf.Descriptors.Descriptor - internal_static_vortex_encodings_DecimalMetadata_descriptor; - private static final - com.google.protobuf.GeneratedMessage.FieldAccessorTable - internal_static_vortex_encodings_DecimalMetadata_fieldAccessorTable; - private static final com.google.protobuf.Descriptors.Descriptor - internal_static_vortex_encodings_DecimalBytePartsMetadata_descriptor; - private static final - com.google.protobuf.GeneratedMessage.FieldAccessorTable - internal_static_vortex_encodings_DecimalBytePartsMetadata_fieldAccessorTable; - private static final com.google.protobuf.Descriptors.Descriptor - internal_static_vortex_encodings_DateTimePartsMetadata_descriptor; - private static final - com.google.protobuf.GeneratedMessage.FieldAccessorTable - internal_static_vortex_encodings_DateTimePartsMetadata_fieldAccessorTable; - private static final com.google.protobuf.Descriptors.Descriptor - internal_static_vortex_encodings_ZstdFrameMetadata_descriptor; - private static final - com.google.protobuf.GeneratedMessage.FieldAccessorTable - internal_static_vortex_encodings_ZstdFrameMetadata_fieldAccessorTable; - private static final com.google.protobuf.Descriptors.Descriptor - internal_static_vortex_encodings_ZstdMetadata_descriptor; - private static final - com.google.protobuf.GeneratedMessage.FieldAccessorTable - internal_static_vortex_encodings_ZstdMetadata_fieldAccessorTable; - private static final com.google.protobuf.Descriptors.Descriptor - internal_static_vortex_encodings_RLEMetadata_descriptor; - private static final - com.google.protobuf.GeneratedMessage.FieldAccessorTable - internal_static_vortex_encodings_RLEMetadata_fieldAccessorTable; - private static final com.google.protobuf.Descriptors.Descriptor - internal_static_vortex_encodings_ListMetadata_descriptor; - private static final - com.google.protobuf.GeneratedMessage.FieldAccessorTable - internal_static_vortex_encodings_ListMetadata_fieldAccessorTable; - private static final com.google.protobuf.Descriptors.Descriptor - internal_static_vortex_encodings_ListViewMetadata_descriptor; - private static final - com.google.protobuf.GeneratedMessage.FieldAccessorTable - internal_static_vortex_encodings_ListViewMetadata_fieldAccessorTable; - private static final com.google.protobuf.Descriptors.Descriptor - internal_static_vortex_encodings_ALPRDMetadata_descriptor; - private static final - com.google.protobuf.GeneratedMessage.FieldAccessorTable - internal_static_vortex_encodings_ALPRDMetadata_fieldAccessorTable; - private static final com.google.protobuf.Descriptors.Descriptor - internal_static_vortex_encodings_PcoPageInfo_descriptor; - private static final - com.google.protobuf.GeneratedMessage.FieldAccessorTable - internal_static_vortex_encodings_PcoPageInfo_fieldAccessorTable; - private static final com.google.protobuf.Descriptors.Descriptor - internal_static_vortex_encodings_PcoChunkInfo_descriptor; - private static final - com.google.protobuf.GeneratedMessage.FieldAccessorTable - internal_static_vortex_encodings_PcoChunkInfo_fieldAccessorTable; - private static final com.google.protobuf.Descriptors.Descriptor - internal_static_vortex_encodings_PcoMetadata_descriptor; - private static final - com.google.protobuf.GeneratedMessage.FieldAccessorTable - internal_static_vortex_encodings_PcoMetadata_fieldAccessorTable; - - public static com.google.protobuf.Descriptors.FileDescriptor - getDescriptor() { - return descriptor; - } - private static final com.google.protobuf.Descriptors.FileDescriptor - descriptor; - static { - java.lang.String[] descriptorData = { - "\n\017encodings.proto\022\020vortex.encodings\032\014sca" + - "lar.proto\032\013dtype.proto\"\231\002\n\017PatchesMetada" + - "ta\022\013\n\003len\030\001 \001(\004\022\016\n\006offset\030\002 \001(\004\022*\n\rindic" + - "es_ptype\030\003 \001(\0162\023.vortex.dtype.PType\022\036\n\021c" + - "hunk_offsets_len\030\004 \001(\004H\000\210\001\001\0225\n\023chunk_off" + - "sets_ptype\030\005 \001(\0162\023.vortex.dtype.PTypeH\001\210" + - "\001\001\022 \n\023offset_within_chunk\030\006 \001(\004H\002\210\001\001B\024\n\022" + - "_chunk_offsets_lenB\026\n\024_chunk_offsets_pty" + - "peB\026\n\024_offset_within_chunk\"D\n\016SparseMeta" + - "data\0222\n\007patches\030\001 \001(\0132!.vortex.encodings" + - ".PatchesMetadata\"\300\001\n\014DictMetadata\022\022\n\nval" + - "ues_len\030\001 \001(\r\022(\n\013codes_ptype\030\002 \001(\0162\023.vor" + - "tex.dtype.PType\022\036\n\021is_nullable_codes\030\003 \001" + - "(\010H\000\210\001\001\022\"\n\025all_values_referenced\030\004 \001(\010H\001" + - "\210\001\001B\024\n\022_is_nullable_codesB\030\n\026_all_values" + - "_referenced\"[\n\016RunEndMetadata\022\'\n\nends_pt" + - "ype\030\001 \001(\0162\023.vortex.dtype.PType\022\020\n\010num_ru" + - "ns\030\002 \001(\004\022\016\n\006offset\030\003 \001(\004\"p\n\013ALPMetadata\022" + - "\r\n\005exp_e\030\001 \001(\r\022\r\n\005exp_f\030\002 \001(\r\0227\n\007patches" + - "\030\003 \001(\0132!.vortex.encodings.PatchesMetadat" + - "aH\000\210\001\001B\n\n\010_patches\"{\n\021BitPackedMetadata\022" + - "\021\n\tbit_width\030\001 \001(\r\022\016\n\006offset\030\002 \001(\r\0227\n\007pa" + - "tches\030\003 \001(\0132!.vortex.encodings.PatchesMe" + - "tadataH\000\210\001\001B\n\n\010_patches\"l\n\020SequenceMetad" + - "ata\022(\n\004base\030\001 \001(\0132\032.vortex.scalar.Scalar" + - "Value\022.\n\nmultiplier\030\002 \001(\0132\032.vortex.scala" + - "r.ScalarValue\"<\n\016VarBinMetadata\022*\n\roffse" + - "ts_ptype\030\001 \001(\0162\023.vortex.dtype.PType\"y\n\014F" + - "SSTMetadata\0227\n\032uncompressed_lengths_ptyp" + - "e\030\001 \001(\0162\023.vortex.dtype.PType\0220\n\023codes_of" + - "fsets_ptype\030\002 \001(\0162\023.vortex.dtype.PType\"3" + - "\n\rDeltaMetadata\022\022\n\ndeltas_len\030\001 \001(\004\022\016\n\006o" + - "ffset\030\002 \001(\r\"&\n\017DecimalMetadata\022\023\n\013values" + - "_type\030\001 \001(\005\"e\n\030DecimalBytePartsMetadata\022" + - "/\n\022zeroth_child_ptype\030\001 \001(\0162\023.vortex.dty" + - "pe.PType\022\030\n\020lower_part_count\030\002 \001(\r\"\233\001\n\025D" + - "ateTimePartsMetadata\022\'\n\ndays_ptype\030\001 \001(\016" + - "2\023.vortex.dtype.PType\022*\n\rseconds_ptype\030\002" + - " \001(\0162\023.vortex.dtype.PType\022-\n\020subseconds_" + - "ptype\030\003 \001(\0162\023.vortex.dtype.PType\"@\n\021Zstd" + - "FrameMetadata\022\031\n\021uncompressed_size\030\001 \001(\004" + - "\022\020\n\010n_values\030\002 \001(\004\"\\\n\014ZstdMetadata\022\027\n\017di" + - "ctionary_size\030\001 \001(\r\0223\n\006frames\030\002 \003(\0132#.vo" + - "rtex.encodings.ZstdFrameMetadata\"\311\001\n\013RLE" + - "Metadata\022\022\n\nvalues_len\030\001 \001(\004\022\023\n\013indices_" + - "len\030\002 \001(\004\022*\n\rindices_ptype\030\003 \001(\0162\023.vorte" + - "x.dtype.PType\022\036\n\026values_idx_offsets_len\030" + - "\004 \001(\004\0225\n\030values_idx_offsets_ptype\030\005 \001(\0162" + - "\023.vortex.dtype.PType\022\016\n\006offset\030\006 \001(\004\"O\n\014" + - "ListMetadata\022\024\n\014elements_len\030\001 \001(\004\022)\n\014of" + - "fset_ptype\030\002 \001(\0162\023.vortex.dtype.PType\"|\n" + - "\020ListViewMetadata\022\024\n\014elements_len\030\001 \001(\004\022" + - ")\n\014offset_ptype\030\002 \001(\0162\023.vortex.dtype.PTy" + - "pe\022\'\n\nsize_ptype\030\003 \001(\0162\023.vortex.dtype.PT" + - "ype\"\274\001\n\rALPRDMetadata\022\027\n\017right_bit_width" + - "\030\001 \001(\r\022\020\n\010dict_len\030\002 \001(\r\022\014\n\004dict\030\003 \003(\r\022-" + - "\n\020left_parts_ptype\030\004 \001(\0162\023.vortex.dtype." + - "PType\0227\n\007patches\030\005 \001(\0132!.vortex.encoding" + - "s.PatchesMetadataH\000\210\001\001B\n\n\010_patches\"\037\n\013Pc" + - "oPageInfo\022\020\n\010n_values\030\001 \001(\r\"<\n\014PcoChunkI" + - "nfo\022,\n\005pages\030\001 \003(\0132\035.vortex.encodings.Pc" + - "oPageInfo\"M\n\013PcoMetadata\022\016\n\006header\030\001 \001(\014" + - "\022.\n\006chunks\030\002 \003(\0132\036.vortex.encodings.PcoC" + - "hunkInfoB-\n\033io.github.dfa1.vortex.protoB" + - "\016EncodingProtosb\006proto3" - }; - descriptor = com.google.protobuf.Descriptors.FileDescriptor - .internalBuildGeneratedFileFrom(descriptorData, - new com.google.protobuf.Descriptors.FileDescriptor[] { - io.github.dfa1.vortex.proto.ScalarProtos.getDescriptor(), - io.github.dfa1.vortex.proto.DTypeProtos.getDescriptor(), - }); - internal_static_vortex_encodings_PatchesMetadata_descriptor = - getDescriptor().getMessageType(0); - internal_static_vortex_encodings_PatchesMetadata_fieldAccessorTable = new - com.google.protobuf.GeneratedMessage.FieldAccessorTable( - internal_static_vortex_encodings_PatchesMetadata_descriptor, - new java.lang.String[] { "Len", "Offset", "IndicesPtype", "ChunkOffsetsLen", "ChunkOffsetsPtype", "OffsetWithinChunk", }); - internal_static_vortex_encodings_SparseMetadata_descriptor = - getDescriptor().getMessageType(1); - internal_static_vortex_encodings_SparseMetadata_fieldAccessorTable = new - com.google.protobuf.GeneratedMessage.FieldAccessorTable( - internal_static_vortex_encodings_SparseMetadata_descriptor, - new java.lang.String[] { "Patches", }); - internal_static_vortex_encodings_DictMetadata_descriptor = - getDescriptor().getMessageType(2); - internal_static_vortex_encodings_DictMetadata_fieldAccessorTable = new - com.google.protobuf.GeneratedMessage.FieldAccessorTable( - internal_static_vortex_encodings_DictMetadata_descriptor, - new java.lang.String[] { "ValuesLen", "CodesPtype", "IsNullableCodes", "AllValuesReferenced", }); - internal_static_vortex_encodings_RunEndMetadata_descriptor = - getDescriptor().getMessageType(3); - internal_static_vortex_encodings_RunEndMetadata_fieldAccessorTable = new - com.google.protobuf.GeneratedMessage.FieldAccessorTable( - internal_static_vortex_encodings_RunEndMetadata_descriptor, - new java.lang.String[] { "EndsPtype", "NumRuns", "Offset", }); - internal_static_vortex_encodings_ALPMetadata_descriptor = - getDescriptor().getMessageType(4); - internal_static_vortex_encodings_ALPMetadata_fieldAccessorTable = new - com.google.protobuf.GeneratedMessage.FieldAccessorTable( - internal_static_vortex_encodings_ALPMetadata_descriptor, - new java.lang.String[] { "ExpE", "ExpF", "Patches", }); - internal_static_vortex_encodings_BitPackedMetadata_descriptor = - getDescriptor().getMessageType(5); - internal_static_vortex_encodings_BitPackedMetadata_fieldAccessorTable = new - com.google.protobuf.GeneratedMessage.FieldAccessorTable( - internal_static_vortex_encodings_BitPackedMetadata_descriptor, - new java.lang.String[] { "BitWidth", "Offset", "Patches", }); - internal_static_vortex_encodings_SequenceMetadata_descriptor = - getDescriptor().getMessageType(6); - internal_static_vortex_encodings_SequenceMetadata_fieldAccessorTable = new - com.google.protobuf.GeneratedMessage.FieldAccessorTable( - internal_static_vortex_encodings_SequenceMetadata_descriptor, - new java.lang.String[] { "Base", "Multiplier", }); - internal_static_vortex_encodings_VarBinMetadata_descriptor = - getDescriptor().getMessageType(7); - internal_static_vortex_encodings_VarBinMetadata_fieldAccessorTable = new - com.google.protobuf.GeneratedMessage.FieldAccessorTable( - internal_static_vortex_encodings_VarBinMetadata_descriptor, - new java.lang.String[] { "OffsetsPtype", }); - internal_static_vortex_encodings_FSSTMetadata_descriptor = - getDescriptor().getMessageType(8); - internal_static_vortex_encodings_FSSTMetadata_fieldAccessorTable = new - com.google.protobuf.GeneratedMessage.FieldAccessorTable( - internal_static_vortex_encodings_FSSTMetadata_descriptor, - new java.lang.String[] { "UncompressedLengthsPtype", "CodesOffsetsPtype", }); - internal_static_vortex_encodings_DeltaMetadata_descriptor = - getDescriptor().getMessageType(9); - internal_static_vortex_encodings_DeltaMetadata_fieldAccessorTable = new - com.google.protobuf.GeneratedMessage.FieldAccessorTable( - internal_static_vortex_encodings_DeltaMetadata_descriptor, - new java.lang.String[] { "DeltasLen", "Offset", }); - internal_static_vortex_encodings_DecimalMetadata_descriptor = - getDescriptor().getMessageType(10); - internal_static_vortex_encodings_DecimalMetadata_fieldAccessorTable = new - com.google.protobuf.GeneratedMessage.FieldAccessorTable( - internal_static_vortex_encodings_DecimalMetadata_descriptor, - new java.lang.String[] { "ValuesType", }); - internal_static_vortex_encodings_DecimalBytePartsMetadata_descriptor = - getDescriptor().getMessageType(11); - internal_static_vortex_encodings_DecimalBytePartsMetadata_fieldAccessorTable = new - com.google.protobuf.GeneratedMessage.FieldAccessorTable( - internal_static_vortex_encodings_DecimalBytePartsMetadata_descriptor, - new java.lang.String[] { "ZerothChildPtype", "LowerPartCount", }); - internal_static_vortex_encodings_DateTimePartsMetadata_descriptor = - getDescriptor().getMessageType(12); - internal_static_vortex_encodings_DateTimePartsMetadata_fieldAccessorTable = new - com.google.protobuf.GeneratedMessage.FieldAccessorTable( - internal_static_vortex_encodings_DateTimePartsMetadata_descriptor, - new java.lang.String[] { "DaysPtype", "SecondsPtype", "SubsecondsPtype", }); - internal_static_vortex_encodings_ZstdFrameMetadata_descriptor = - getDescriptor().getMessageType(13); - internal_static_vortex_encodings_ZstdFrameMetadata_fieldAccessorTable = new - com.google.protobuf.GeneratedMessage.FieldAccessorTable( - internal_static_vortex_encodings_ZstdFrameMetadata_descriptor, - new java.lang.String[] { "UncompressedSize", "NValues", }); - internal_static_vortex_encodings_ZstdMetadata_descriptor = - getDescriptor().getMessageType(14); - internal_static_vortex_encodings_ZstdMetadata_fieldAccessorTable = new - com.google.protobuf.GeneratedMessage.FieldAccessorTable( - internal_static_vortex_encodings_ZstdMetadata_descriptor, - new java.lang.String[] { "DictionarySize", "Frames", }); - internal_static_vortex_encodings_RLEMetadata_descriptor = - getDescriptor().getMessageType(15); - internal_static_vortex_encodings_RLEMetadata_fieldAccessorTable = new - com.google.protobuf.GeneratedMessage.FieldAccessorTable( - internal_static_vortex_encodings_RLEMetadata_descriptor, - new java.lang.String[] { "ValuesLen", "IndicesLen", "IndicesPtype", "ValuesIdxOffsetsLen", "ValuesIdxOffsetsPtype", "Offset", }); - internal_static_vortex_encodings_ListMetadata_descriptor = - getDescriptor().getMessageType(16); - internal_static_vortex_encodings_ListMetadata_fieldAccessorTable = new - com.google.protobuf.GeneratedMessage.FieldAccessorTable( - internal_static_vortex_encodings_ListMetadata_descriptor, - new java.lang.String[] { "ElementsLen", "OffsetPtype", }); - internal_static_vortex_encodings_ListViewMetadata_descriptor = - getDescriptor().getMessageType(17); - internal_static_vortex_encodings_ListViewMetadata_fieldAccessorTable = new - com.google.protobuf.GeneratedMessage.FieldAccessorTable( - internal_static_vortex_encodings_ListViewMetadata_descriptor, - new java.lang.String[] { "ElementsLen", "OffsetPtype", "SizePtype", }); - internal_static_vortex_encodings_ALPRDMetadata_descriptor = - getDescriptor().getMessageType(18); - internal_static_vortex_encodings_ALPRDMetadata_fieldAccessorTable = new - com.google.protobuf.GeneratedMessage.FieldAccessorTable( - internal_static_vortex_encodings_ALPRDMetadata_descriptor, - new java.lang.String[] { "RightBitWidth", "DictLen", "Dict", "LeftPartsPtype", "Patches", }); - internal_static_vortex_encodings_PcoPageInfo_descriptor = - getDescriptor().getMessageType(19); - internal_static_vortex_encodings_PcoPageInfo_fieldAccessorTable = new - com.google.protobuf.GeneratedMessage.FieldAccessorTable( - internal_static_vortex_encodings_PcoPageInfo_descriptor, - new java.lang.String[] { "NValues", }); - internal_static_vortex_encodings_PcoChunkInfo_descriptor = - getDescriptor().getMessageType(20); - internal_static_vortex_encodings_PcoChunkInfo_fieldAccessorTable = new - com.google.protobuf.GeneratedMessage.FieldAccessorTable( - internal_static_vortex_encodings_PcoChunkInfo_descriptor, - new java.lang.String[] { "Pages", }); - internal_static_vortex_encodings_PcoMetadata_descriptor = - getDescriptor().getMessageType(21); - internal_static_vortex_encodings_PcoMetadata_fieldAccessorTable = new - com.google.protobuf.GeneratedMessage.FieldAccessorTable( - internal_static_vortex_encodings_PcoMetadata_descriptor, - new java.lang.String[] { "Header", "Chunks", }); - descriptor.resolveAllFeaturesImmutable(); - io.github.dfa1.vortex.proto.ScalarProtos.getDescriptor(); - io.github.dfa1.vortex.proto.DTypeProtos.getDescriptor(); - } - - // @@protoc_insertion_point(outer_class_scope) -} diff --git a/core/src/main/java/io/github/dfa1/vortex/proto/ScalarProtos.java b/core/src/main/java/io/github/dfa1/vortex/proto/ScalarProtos.java deleted file mode 100644 index 4928ca2..0000000 --- a/core/src/main/java/io/github/dfa1/vortex/proto/ScalarProtos.java +++ /dev/null @@ -1,3648 +0,0 @@ -// Generated by the protocol buffer compiler. DO NOT EDIT! -// NO CHECKED-IN PROTOBUF GENCODE -// source: scalar.proto -// Protobuf Java Version: 4.35.0 - -package io.github.dfa1.vortex.proto; - -@com.google.protobuf.Generated -public final class ScalarProtos extends com.google.protobuf.GeneratedFile { - private ScalarProtos() {} - static { - com.google.protobuf.RuntimeVersion.validateProtobufGencodeVersion( - com.google.protobuf.RuntimeVersion.RuntimeDomain.PUBLIC, - /* major= */ 4, - /* minor= */ 35, - /* patch= */ 0, - /* suffix= */ "", - "ScalarProtos"); - } - public static void registerAllExtensions( - com.google.protobuf.ExtensionRegistryLite registry) { - } - - public static void registerAllExtensions( - com.google.protobuf.ExtensionRegistry registry) { - registerAllExtensions( - (com.google.protobuf.ExtensionRegistryLite) registry); - } - public interface ScalarOrBuilder extends - // @@protoc_insertion_point(interface_extends:vortex.scalar.Scalar) - com.google.protobuf.MessageOrBuilder { - - /** - * .vortex.dtype.DType dtype = 1; - * @return Whether the dtype field is set. - */ - boolean hasDtype(); - /** - * .vortex.dtype.DType dtype = 1; - * @return The dtype. - */ - io.github.dfa1.vortex.proto.DTypeProtos.DType getDtype(); - /** - * .vortex.dtype.DType dtype = 1; - */ - io.github.dfa1.vortex.proto.DTypeProtos.DTypeOrBuilder getDtypeOrBuilder(); - - /** - * .vortex.scalar.ScalarValue value = 2; - * @return Whether the value field is set. - */ - boolean hasValue(); - /** - * .vortex.scalar.ScalarValue value = 2; - * @return The value. - */ - io.github.dfa1.vortex.proto.ScalarProtos.ScalarValue getValue(); - /** - * .vortex.scalar.ScalarValue value = 2; - */ - io.github.dfa1.vortex.proto.ScalarProtos.ScalarValueOrBuilder getValueOrBuilder(); - } - /** - * Protobuf type {@code vortex.scalar.Scalar} - */ - public static final class Scalar extends - com.google.protobuf.GeneratedMessage implements - // @@protoc_insertion_point(message_implements:vortex.scalar.Scalar) - ScalarOrBuilder { - private static final long serialVersionUID = 0L; - static { - com.google.protobuf.RuntimeVersion.validateProtobufGencodeVersion( - com.google.protobuf.RuntimeVersion.RuntimeDomain.PUBLIC, - /* major= */ 4, - /* minor= */ 35, - /* patch= */ 0, - /* suffix= */ "", - "Scalar"); - } - // Use Scalar.newBuilder() to construct. - private Scalar(com.google.protobuf.GeneratedMessage.Builder builder) { - super(builder); - } - private Scalar() { - } - - public static final com.google.protobuf.Descriptors.Descriptor - getDescriptor() { - return io.github.dfa1.vortex.proto.ScalarProtos.internal_static_vortex_scalar_Scalar_descriptor; - } - - @java.lang.Override - public com.google.protobuf.Descriptors.Descriptor getDescriptorForType() { - return io.github.dfa1.vortex.proto.ScalarProtos.internal_static_vortex_scalar_Scalar_descriptor; - } - - @java.lang.Override - protected com.google.protobuf.GeneratedMessage.FieldAccessorTable - internalGetFieldAccessorTable() { - return io.github.dfa1.vortex.proto.ScalarProtos.internal_static_vortex_scalar_Scalar_fieldAccessorTable - .ensureFieldAccessorsInitialized( - io.github.dfa1.vortex.proto.ScalarProtos.Scalar.class, io.github.dfa1.vortex.proto.ScalarProtos.Scalar.Builder.class); - } - - private int bitField0_; - public static final int DTYPE_FIELD_NUMBER = 1; - private io.github.dfa1.vortex.proto.DTypeProtos.DType dtype_; - /** - * .vortex.dtype.DType dtype = 1; - * @return Whether the dtype field is set. - */ - @java.lang.Override - public boolean hasDtype() { - return ((bitField0_ & 0x00000001) != 0); - } - /** - * .vortex.dtype.DType dtype = 1; - * @return The dtype. - */ - @java.lang.Override - public io.github.dfa1.vortex.proto.DTypeProtos.DType getDtype() { - return dtype_ == null ? io.github.dfa1.vortex.proto.DTypeProtos.DType.getDefaultInstance() : dtype_; - } - /** - * .vortex.dtype.DType dtype = 1; - */ - @java.lang.Override - public io.github.dfa1.vortex.proto.DTypeProtos.DTypeOrBuilder getDtypeOrBuilder() { - return dtype_ == null ? io.github.dfa1.vortex.proto.DTypeProtos.DType.getDefaultInstance() : dtype_; - } - - public static final int VALUE_FIELD_NUMBER = 2; - private io.github.dfa1.vortex.proto.ScalarProtos.ScalarValue value_; - /** - * .vortex.scalar.ScalarValue value = 2; - * @return Whether the value field is set. - */ - @java.lang.Override - public boolean hasValue() { - return ((bitField0_ & 0x00000002) != 0); - } - /** - * .vortex.scalar.ScalarValue value = 2; - * @return The value. - */ - @java.lang.Override - public io.github.dfa1.vortex.proto.ScalarProtos.ScalarValue getValue() { - return value_ == null ? io.github.dfa1.vortex.proto.ScalarProtos.ScalarValue.getDefaultInstance() : value_; - } - /** - * .vortex.scalar.ScalarValue value = 2; - */ - @java.lang.Override - public io.github.dfa1.vortex.proto.ScalarProtos.ScalarValueOrBuilder getValueOrBuilder() { - return value_ == null ? io.github.dfa1.vortex.proto.ScalarProtos.ScalarValue.getDefaultInstance() : value_; - } - - private byte memoizedIsInitialized = -1; - @java.lang.Override - public final boolean isInitialized() { - byte isInitialized = memoizedIsInitialized; - if (isInitialized == 1) return true; - if (isInitialized == 0) return false; - - memoizedIsInitialized = 1; - return true; - } - - @java.lang.Override - public void writeTo(com.google.protobuf.CodedOutputStream output) - throws java.io.IOException { - if (((bitField0_ & 0x00000001) != 0)) { - output.writeMessage(1, getDtype()); - } - if (((bitField0_ & 0x00000002) != 0)) { - output.writeMessage(2, getValue()); - } - getUnknownFields().writeTo(output); - } - private int computeSerializedSize_0() { - int size = 0; - if (((bitField0_ & 0x00000001) != 0)) { - size += com.google.protobuf.CodedOutputStream - .computeMessageSize(1, getDtype()); - } - if (((bitField0_ & 0x00000002) != 0)) { - size += com.google.protobuf.CodedOutputStream - .computeMessageSize(2, getValue()); - } - return size; - } - @java.lang.Override - public int getSerializedSize() { - int size = memoizedSize; - if (size != -1) return size; - - size = 0; - size += computeSerializedSize_0(); - size += getUnknownFields().getSerializedSize(); - memoizedSize = size; - return size; - } - - @java.lang.Override - public boolean equals(final java.lang.Object obj) { - if (obj == this) { - return true; - } - if (!(obj instanceof io.github.dfa1.vortex.proto.ScalarProtos.Scalar)) { - return super.equals(obj); - } - io.github.dfa1.vortex.proto.ScalarProtos.Scalar other = (io.github.dfa1.vortex.proto.ScalarProtos.Scalar) obj; - - if (hasDtype() != other.hasDtype()) return false; - if (hasDtype()) { - if (!getDtype() - .equals(other.getDtype())) return false; - } - if (hasValue() != other.hasValue()) return false; - if (hasValue()) { - if (!getValue() - .equals(other.getValue())) return false; - } - if (!getUnknownFields().equals(other.getUnknownFields())) return false; - return true; - } - - @java.lang.Override - public int hashCode() { - if (memoizedHashCode != 0) { - return memoizedHashCode; - } - int hash = 41; - hash = (19 * hash) + getDescriptor().hashCode(); - if (hasDtype()) { - hash = (37 * hash) + DTYPE_FIELD_NUMBER; - hash = (53 * hash) + getDtype().hashCode(); - } - if (hasValue()) { - hash = (37 * hash) + VALUE_FIELD_NUMBER; - hash = (53 * hash) + getValue().hashCode(); - } - hash = (29 * hash) + getUnknownFields().hashCode(); - memoizedHashCode = hash; - return hash; - } - - public static io.github.dfa1.vortex.proto.ScalarProtos.Scalar parseFrom( - java.nio.ByteBuffer data) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data); - } - public static io.github.dfa1.vortex.proto.ScalarProtos.Scalar parseFrom( - java.nio.ByteBuffer data, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data, extensionRegistry); - } - public static io.github.dfa1.vortex.proto.ScalarProtos.Scalar parseFrom( - com.google.protobuf.ByteString data) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data); - } - public static io.github.dfa1.vortex.proto.ScalarProtos.Scalar parseFrom( - com.google.protobuf.ByteString data, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data, extensionRegistry); - } - public static io.github.dfa1.vortex.proto.ScalarProtos.Scalar parseFrom(byte[] data) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data); - } - public static io.github.dfa1.vortex.proto.ScalarProtos.Scalar parseFrom( - byte[] data, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data, extensionRegistry); - } - public static io.github.dfa1.vortex.proto.ScalarProtos.Scalar parseFrom(java.io.InputStream input) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseWithIOException(PARSER, input); - } - public static io.github.dfa1.vortex.proto.ScalarProtos.Scalar parseFrom( - java.io.InputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseWithIOException(PARSER, input, extensionRegistry); - } - - public static io.github.dfa1.vortex.proto.ScalarProtos.Scalar parseDelimitedFrom(java.io.InputStream input) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseDelimitedWithIOException(PARSER, input); - } - - public static io.github.dfa1.vortex.proto.ScalarProtos.Scalar parseDelimitedFrom( - java.io.InputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseDelimitedWithIOException(PARSER, input, extensionRegistry); - } - public static io.github.dfa1.vortex.proto.ScalarProtos.Scalar parseFrom( - com.google.protobuf.CodedInputStream input) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseWithIOException(PARSER, input); - } - public static io.github.dfa1.vortex.proto.ScalarProtos.Scalar parseFrom( - com.google.protobuf.CodedInputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseWithIOException(PARSER, input, extensionRegistry); - } - - @java.lang.Override - public Builder newBuilderForType() { return newBuilder(); } - public static Builder newBuilder() { - return DEFAULT_INSTANCE.toBuilder(); - } - public static Builder newBuilder(io.github.dfa1.vortex.proto.ScalarProtos.Scalar prototype) { - return DEFAULT_INSTANCE.toBuilder().mergeFrom(prototype); - } - @java.lang.Override - public Builder toBuilder() { - return this == DEFAULT_INSTANCE - ? new Builder() : new Builder().mergeFrom(this); - } - - @java.lang.Override - protected Builder newBuilderForType( - com.google.protobuf.GeneratedMessage.BuilderParent parent) { - Builder builder = new Builder(parent); - return builder; - } - /** - * Protobuf type {@code vortex.scalar.Scalar} - */ - public static final class Builder extends - com.google.protobuf.GeneratedMessage.Builder implements - // @@protoc_insertion_point(builder_implements:vortex.scalar.Scalar) - io.github.dfa1.vortex.proto.ScalarProtos.ScalarOrBuilder { - public static final com.google.protobuf.Descriptors.Descriptor - getDescriptor() { - return io.github.dfa1.vortex.proto.ScalarProtos.internal_static_vortex_scalar_Scalar_descriptor; - } - - @java.lang.Override - protected com.google.protobuf.GeneratedMessage.FieldAccessorTable - internalGetFieldAccessorTable() { - return io.github.dfa1.vortex.proto.ScalarProtos.internal_static_vortex_scalar_Scalar_fieldAccessorTable - .ensureFieldAccessorsInitialized( - io.github.dfa1.vortex.proto.ScalarProtos.Scalar.class, io.github.dfa1.vortex.proto.ScalarProtos.Scalar.Builder.class); - } - - // Construct using io.github.dfa1.vortex.proto.ScalarProtos.Scalar.newBuilder() - private Builder() { - maybeForceBuilderInitialization(); - } - - private Builder( - com.google.protobuf.GeneratedMessage.BuilderParent parent) { - super(parent); - maybeForceBuilderInitialization(); - } - private void maybeForceBuilderInitialization() { - if (com.google.protobuf.GeneratedMessage - .alwaysUseFieldBuilders) { - internalGetDtypeFieldBuilder(); - internalGetValueFieldBuilder(); - } - } - @java.lang.Override - public Builder clear() { - super.clear(); - bitField0_ = 0; - dtype_ = null; - if (dtypeBuilder_ != null) { - dtypeBuilder_.dispose(); - dtypeBuilder_ = null; - } - value_ = null; - if (valueBuilder_ != null) { - valueBuilder_.dispose(); - valueBuilder_ = null; - } - return this; - } - - @java.lang.Override - public com.google.protobuf.Descriptors.Descriptor - getDescriptorForType() { - return io.github.dfa1.vortex.proto.ScalarProtos.internal_static_vortex_scalar_Scalar_descriptor; - } - - @java.lang.Override - public io.github.dfa1.vortex.proto.ScalarProtos.Scalar getDefaultInstanceForType() { - return io.github.dfa1.vortex.proto.ScalarProtos.Scalar.getDefaultInstance(); - } - - @java.lang.Override - public io.github.dfa1.vortex.proto.ScalarProtos.Scalar build() { - io.github.dfa1.vortex.proto.ScalarProtos.Scalar result = buildPartial(); - if (!result.isInitialized()) { - throw newUninitializedMessageException(result); - } - return result; - } - - @java.lang.Override - public io.github.dfa1.vortex.proto.ScalarProtos.Scalar buildPartial() { - io.github.dfa1.vortex.proto.ScalarProtos.Scalar result = new io.github.dfa1.vortex.proto.ScalarProtos.Scalar(this); - if (bitField0_ != 0) { buildPartial0(result); } - onBuilt(); - return result; - } - - private void buildPartial0(io.github.dfa1.vortex.proto.ScalarProtos.Scalar result) { - int from_bitField0_ = bitField0_; - int to_bitField0_ = 0; - if (((from_bitField0_ & 0x00000001) != 0)) { - result.dtype_ = dtypeBuilder_ == null - ? dtype_ - : dtypeBuilder_.build(); - to_bitField0_ |= 0x00000001; - } - if (((from_bitField0_ & 0x00000002) != 0)) { - result.value_ = valueBuilder_ == null - ? value_ - : valueBuilder_.build(); - to_bitField0_ |= 0x00000002; - } - result.bitField0_ |= to_bitField0_; - } - - @java.lang.Override - public Builder mergeFrom(com.google.protobuf.Message other) { - if (other instanceof io.github.dfa1.vortex.proto.ScalarProtos.Scalar) { - return mergeFrom((io.github.dfa1.vortex.proto.ScalarProtos.Scalar)other); - } else { - super.mergeFrom(other); - return this; - } - } - - public Builder mergeFrom(io.github.dfa1.vortex.proto.ScalarProtos.Scalar other) { - if (other == io.github.dfa1.vortex.proto.ScalarProtos.Scalar.getDefaultInstance()) return this; - if (other.hasDtype()) { - mergeDtype(other.getDtype()); - } - if (other.hasValue()) { - mergeValue(other.getValue()); - } - this.mergeUnknownFields(other.getUnknownFields()); - onChanged(); - return this; - } - - @java.lang.Override - public final boolean isInitialized() { - return true; - } - - @java.lang.Override - public Builder mergeFrom( - com.google.protobuf.CodedInputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws java.io.IOException { - if (extensionRegistry == null) { - throw new java.lang.NullPointerException(); - } - try { - boolean done = false; - while (!done) { - int tag = input.readTag(); - switch (tag) { - case 0: - done = true; - break; - case 10: { - input.readMessage( - internalGetDtypeFieldBuilder().getBuilder(), - extensionRegistry); - bitField0_ |= 0x00000001; - break; - } // case 10 - case 18: { - input.readMessage( - internalGetValueFieldBuilder().getBuilder(), - extensionRegistry); - bitField0_ |= 0x00000002; - break; - } // case 18 - default: { - if (!super.parseUnknownField(input, extensionRegistry, tag)) { - done = true; // was an endgroup tag - } - break; - } // default: - } // switch (tag) - } // while (!done) - } catch (com.google.protobuf.InvalidProtocolBufferException e) { - throw e.unwrapIOException(); - } finally { - onChanged(); - } // finally - return this; - } - private int bitField0_; - - private io.github.dfa1.vortex.proto.DTypeProtos.DType dtype_; - private com.google.protobuf.SingleFieldBuilder< - io.github.dfa1.vortex.proto.DTypeProtos.DType, io.github.dfa1.vortex.proto.DTypeProtos.DType.Builder, io.github.dfa1.vortex.proto.DTypeProtos.DTypeOrBuilder> dtypeBuilder_; - /** - * .vortex.dtype.DType dtype = 1; - * @return Whether the dtype field is set. - */ - public boolean hasDtype() { - return ((bitField0_ & 0x00000001) != 0); - } - /** - * .vortex.dtype.DType dtype = 1; - * @return The dtype. - */ - public io.github.dfa1.vortex.proto.DTypeProtos.DType getDtype() { - if (dtypeBuilder_ == null) { - return dtype_ == null ? io.github.dfa1.vortex.proto.DTypeProtos.DType.getDefaultInstance() : dtype_; - } else { - return dtypeBuilder_.getMessage(); - } - } - /** - * .vortex.dtype.DType dtype = 1; - */ - public Builder setDtype(io.github.dfa1.vortex.proto.DTypeProtos.DType value) { - if (dtypeBuilder_ == null) { - if (value == null) { - throw new NullPointerException(); - } - dtype_ = value; - } else { - dtypeBuilder_.setMessage(value); - } - bitField0_ |= 0x00000001; - onChanged(); - return this; - } - /** - * .vortex.dtype.DType dtype = 1; - */ - public Builder setDtype( - io.github.dfa1.vortex.proto.DTypeProtos.DType.Builder builderForValue) { - if (dtypeBuilder_ == null) { - dtype_ = builderForValue.build(); - } else { - dtypeBuilder_.setMessage(builderForValue.build()); - } - bitField0_ |= 0x00000001; - onChanged(); - return this; - } - /** - * .vortex.dtype.DType dtype = 1; - */ - public Builder mergeDtype(io.github.dfa1.vortex.proto.DTypeProtos.DType value) { - if (dtypeBuilder_ == null) { - if (((bitField0_ & 0x00000001) != 0) && - dtype_ != null && - dtype_ != io.github.dfa1.vortex.proto.DTypeProtos.DType.getDefaultInstance()) { - getDtypeBuilder().mergeFrom(value); - } else { - dtype_ = value; - } - } else { - dtypeBuilder_.mergeFrom(value); - } - if (dtype_ != null) { - bitField0_ |= 0x00000001; - onChanged(); - } - return this; - } - /** - * .vortex.dtype.DType dtype = 1; - */ - public Builder clearDtype() { - bitField0_ = (bitField0_ & ~0x00000001); - dtype_ = null; - if (dtypeBuilder_ != null) { - dtypeBuilder_.dispose(); - dtypeBuilder_ = null; - } - onChanged(); - return this; - } - /** - * .vortex.dtype.DType dtype = 1; - */ - public io.github.dfa1.vortex.proto.DTypeProtos.DType.Builder getDtypeBuilder() { - bitField0_ |= 0x00000001; - onChanged(); - return internalGetDtypeFieldBuilder().getBuilder(); - } - /** - * .vortex.dtype.DType dtype = 1; - */ - public io.github.dfa1.vortex.proto.DTypeProtos.DTypeOrBuilder getDtypeOrBuilder() { - if (dtypeBuilder_ != null) { - return dtypeBuilder_.getMessageOrBuilder(); - } else { - return dtype_ == null ? - io.github.dfa1.vortex.proto.DTypeProtos.DType.getDefaultInstance() : dtype_; - } - } - /** - * .vortex.dtype.DType dtype = 1; - */ - private com.google.protobuf.SingleFieldBuilder< - io.github.dfa1.vortex.proto.DTypeProtos.DType, io.github.dfa1.vortex.proto.DTypeProtos.DType.Builder, io.github.dfa1.vortex.proto.DTypeProtos.DTypeOrBuilder> - internalGetDtypeFieldBuilder() { - if (dtypeBuilder_ == null) { - dtypeBuilder_ = new com.google.protobuf.SingleFieldBuilder< - io.github.dfa1.vortex.proto.DTypeProtos.DType, io.github.dfa1.vortex.proto.DTypeProtos.DType.Builder, io.github.dfa1.vortex.proto.DTypeProtos.DTypeOrBuilder>( - getDtype(), - getParentForChildren(), - isClean()); - dtype_ = null; - } - return dtypeBuilder_; - } - - private io.github.dfa1.vortex.proto.ScalarProtos.ScalarValue value_; - private com.google.protobuf.SingleFieldBuilder< - io.github.dfa1.vortex.proto.ScalarProtos.ScalarValue, io.github.dfa1.vortex.proto.ScalarProtos.ScalarValue.Builder, io.github.dfa1.vortex.proto.ScalarProtos.ScalarValueOrBuilder> valueBuilder_; - /** - * .vortex.scalar.ScalarValue value = 2; - * @return Whether the value field is set. - */ - public boolean hasValue() { - return ((bitField0_ & 0x00000002) != 0); - } - /** - * .vortex.scalar.ScalarValue value = 2; - * @return The value. - */ - public io.github.dfa1.vortex.proto.ScalarProtos.ScalarValue getValue() { - if (valueBuilder_ == null) { - return value_ == null ? io.github.dfa1.vortex.proto.ScalarProtos.ScalarValue.getDefaultInstance() : value_; - } else { - return valueBuilder_.getMessage(); - } - } - /** - * .vortex.scalar.ScalarValue value = 2; - */ - public Builder setValue(io.github.dfa1.vortex.proto.ScalarProtos.ScalarValue value) { - if (valueBuilder_ == null) { - if (value == null) { - throw new NullPointerException(); - } - value_ = value; - } else { - valueBuilder_.setMessage(value); - } - bitField0_ |= 0x00000002; - onChanged(); - return this; - } - /** - * .vortex.scalar.ScalarValue value = 2; - */ - public Builder setValue( - io.github.dfa1.vortex.proto.ScalarProtos.ScalarValue.Builder builderForValue) { - if (valueBuilder_ == null) { - value_ = builderForValue.build(); - } else { - valueBuilder_.setMessage(builderForValue.build()); - } - bitField0_ |= 0x00000002; - onChanged(); - return this; - } - /** - * .vortex.scalar.ScalarValue value = 2; - */ - public Builder mergeValue(io.github.dfa1.vortex.proto.ScalarProtos.ScalarValue value) { - if (valueBuilder_ == null) { - if (((bitField0_ & 0x00000002) != 0) && - value_ != null && - value_ != io.github.dfa1.vortex.proto.ScalarProtos.ScalarValue.getDefaultInstance()) { - getValueBuilder().mergeFrom(value); - } else { - value_ = value; - } - } else { - valueBuilder_.mergeFrom(value); - } - if (value_ != null) { - bitField0_ |= 0x00000002; - onChanged(); - } - return this; - } - /** - * .vortex.scalar.ScalarValue value = 2; - */ - public Builder clearValue() { - bitField0_ = (bitField0_ & ~0x00000002); - value_ = null; - if (valueBuilder_ != null) { - valueBuilder_.dispose(); - valueBuilder_ = null; - } - onChanged(); - return this; - } - /** - * .vortex.scalar.ScalarValue value = 2; - */ - public io.github.dfa1.vortex.proto.ScalarProtos.ScalarValue.Builder getValueBuilder() { - bitField0_ |= 0x00000002; - onChanged(); - return internalGetValueFieldBuilder().getBuilder(); - } - /** - * .vortex.scalar.ScalarValue value = 2; - */ - public io.github.dfa1.vortex.proto.ScalarProtos.ScalarValueOrBuilder getValueOrBuilder() { - if (valueBuilder_ != null) { - return valueBuilder_.getMessageOrBuilder(); - } else { - return value_ == null ? - io.github.dfa1.vortex.proto.ScalarProtos.ScalarValue.getDefaultInstance() : value_; - } - } - /** - * .vortex.scalar.ScalarValue value = 2; - */ - private com.google.protobuf.SingleFieldBuilder< - io.github.dfa1.vortex.proto.ScalarProtos.ScalarValue, io.github.dfa1.vortex.proto.ScalarProtos.ScalarValue.Builder, io.github.dfa1.vortex.proto.ScalarProtos.ScalarValueOrBuilder> - internalGetValueFieldBuilder() { - if (valueBuilder_ == null) { - valueBuilder_ = new com.google.protobuf.SingleFieldBuilder< - io.github.dfa1.vortex.proto.ScalarProtos.ScalarValue, io.github.dfa1.vortex.proto.ScalarProtos.ScalarValue.Builder, io.github.dfa1.vortex.proto.ScalarProtos.ScalarValueOrBuilder>( - getValue(), - getParentForChildren(), - isClean()); - value_ = null; - } - return valueBuilder_; - } - - // @@protoc_insertion_point(builder_scope:vortex.scalar.Scalar) - } - - // @@protoc_insertion_point(class_scope:vortex.scalar.Scalar) - private static final io.github.dfa1.vortex.proto.ScalarProtos.Scalar DEFAULT_INSTANCE; - static { - DEFAULT_INSTANCE = new io.github.dfa1.vortex.proto.ScalarProtos.Scalar(); - } - - public static io.github.dfa1.vortex.proto.ScalarProtos.Scalar getDefaultInstance() { - return DEFAULT_INSTANCE; - } - - private static final com.google.protobuf.Parser - PARSER = new com.google.protobuf.AbstractParser() { - @java.lang.Override - public Scalar parsePartialFrom( - com.google.protobuf.CodedInputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws com.google.protobuf.InvalidProtocolBufferException { - Builder builder = newBuilder(); - try { - builder.mergeFrom(input, extensionRegistry); - } catch (com.google.protobuf.InvalidProtocolBufferException e) { - throw e.setUnfinishedMessage(builder.buildPartial()); - } catch (com.google.protobuf.UninitializedMessageException e) { - throw e.asInvalidProtocolBufferException().setUnfinishedMessage(builder.buildPartial()); - } catch (java.io.IOException e) { - throw new com.google.protobuf.InvalidProtocolBufferException(e) - .setUnfinishedMessage(builder.buildPartial()); - } - return builder.buildPartial(); - } - }; - - public static com.google.protobuf.Parser parser() { - return PARSER; - } - - @java.lang.Override - public com.google.protobuf.Parser getParserForType() { - return PARSER; - } - - @java.lang.Override - public io.github.dfa1.vortex.proto.ScalarProtos.Scalar getDefaultInstanceForType() { - return DEFAULT_INSTANCE; - } - - } - - public interface ScalarValueOrBuilder extends - // @@protoc_insertion_point(interface_extends:vortex.scalar.ScalarValue) - com.google.protobuf.MessageOrBuilder { - - /** - * .google.protobuf.NullValue null_value = 1; - * @return Whether the nullValue field is set. - */ - boolean hasNullValue(); - /** - * .google.protobuf.NullValue null_value = 1; - * @return The enum numeric value on the wire for nullValue. - */ - int getNullValueValue(); - /** - * .google.protobuf.NullValue null_value = 1; - * @return The nullValue. - */ - com.google.protobuf.NullValue getNullValue(); - - /** - * bool bool_value = 2; - * @return Whether the boolValue field is set. - */ - boolean hasBoolValue(); - /** - * bool bool_value = 2; - * @return The boolValue. - */ - boolean getBoolValue(); - - /** - * sint64 int64_value = 3; - * @return Whether the int64Value field is set. - */ - boolean hasInt64Value(); - /** - * sint64 int64_value = 3; - * @return The int64Value. - */ - long getInt64Value(); - - /** - * uint64 uint64_value = 4; - * @return Whether the uint64Value field is set. - */ - boolean hasUint64Value(); - /** - * uint64 uint64_value = 4; - * @return The uint64Value. - */ - long getUint64Value(); - - /** - * float f32_value = 5; - * @return Whether the f32Value field is set. - */ - boolean hasF32Value(); - /** - * float f32_value = 5; - * @return The f32Value. - */ - float getF32Value(); - - /** - * double f64_value = 6; - * @return Whether the f64Value field is set. - */ - boolean hasF64Value(); - /** - * double f64_value = 6; - * @return The f64Value. - */ - double getF64Value(); - - /** - * string string_value = 7; - * @return Whether the stringValue field is set. - */ - boolean hasStringValue(); - /** - * string string_value = 7; - * @return The stringValue. - */ - java.lang.String getStringValue(); - /** - * string string_value = 7; - * @return The bytes for stringValue. - */ - com.google.protobuf.ByteString - getStringValueBytes(); - - /** - * bytes bytes_value = 8; - * @return Whether the bytesValue field is set. - */ - boolean hasBytesValue(); - /** - * bytes bytes_value = 8; - * @return The bytesValue. - */ - com.google.protobuf.ByteString getBytesValue(); - - /** - * .vortex.scalar.ListValue list_value = 9; - * @return Whether the listValue field is set. - */ - boolean hasListValue(); - /** - * .vortex.scalar.ListValue list_value = 9; - * @return The listValue. - */ - io.github.dfa1.vortex.proto.ScalarProtos.ListValue getListValue(); - /** - * .vortex.scalar.ListValue list_value = 9; - */ - io.github.dfa1.vortex.proto.ScalarProtos.ListValueOrBuilder getListValueOrBuilder(); - - /** - * uint64 f16_value = 10; - * @return Whether the f16Value field is set. - */ - boolean hasF16Value(); - /** - * uint64 f16_value = 10; - * @return The f16Value. - */ - long getF16Value(); - - /** - * 
-     * Variant scalars carry a row-specific nested scalar.
-     * See RFC 0015: https://github.com/vortex-data/rfcs/blob/develop/accepted/0015-variant-type.md
-     *
- * - * .vortex.scalar.Scalar variant_value = 11; - * @return Whether the variantValue field is set. - */ - boolean hasVariantValue(); - /** - * 
-     * Variant scalars carry a row-specific nested scalar.
-     * See RFC 0015: https://github.com/vortex-data/rfcs/blob/develop/accepted/0015-variant-type.md
-     *
- * - * .vortex.scalar.Scalar variant_value = 11; - * @return The variantValue. - */ - io.github.dfa1.vortex.proto.ScalarProtos.Scalar getVariantValue(); - /** - * 
-     * Variant scalars carry a row-specific nested scalar.
-     * See RFC 0015: https://github.com/vortex-data/rfcs/blob/develop/accepted/0015-variant-type.md
-     *
- * - * .vortex.scalar.Scalar variant_value = 11; - */ - io.github.dfa1.vortex.proto.ScalarProtos.ScalarOrBuilder getVariantValueOrBuilder(); - - io.github.dfa1.vortex.proto.ScalarProtos.ScalarValue.KindCase getKindCase(); - } - /** - * Protobuf type {@code vortex.scalar.ScalarValue} - */ - public static final class ScalarValue extends - com.google.protobuf.GeneratedMessage implements - // @@protoc_insertion_point(message_implements:vortex.scalar.ScalarValue) - ScalarValueOrBuilder { - private static final long serialVersionUID = 0L; - static { - com.google.protobuf.RuntimeVersion.validateProtobufGencodeVersion( - com.google.protobuf.RuntimeVersion.RuntimeDomain.PUBLIC, - /* major= */ 4, - /* minor= */ 35, - /* patch= */ 0, - /* suffix= */ "", - "ScalarValue"); - } - // Use ScalarValue.newBuilder() to construct. - private ScalarValue(com.google.protobuf.GeneratedMessage.Builder builder) { - super(builder); - } - private ScalarValue() { - } - - public static final com.google.protobuf.Descriptors.Descriptor - getDescriptor() { - return io.github.dfa1.vortex.proto.ScalarProtos.internal_static_vortex_scalar_ScalarValue_descriptor; - } - - @java.lang.Override - public com.google.protobuf.Descriptors.Descriptor getDescriptorForType() { - return io.github.dfa1.vortex.proto.ScalarProtos.internal_static_vortex_scalar_ScalarValue_descriptor; - } - - @java.lang.Override - protected com.google.protobuf.GeneratedMessage.FieldAccessorTable - internalGetFieldAccessorTable() { - return io.github.dfa1.vortex.proto.ScalarProtos.internal_static_vortex_scalar_ScalarValue_fieldAccessorTable - .ensureFieldAccessorsInitialized( - io.github.dfa1.vortex.proto.ScalarProtos.ScalarValue.class, io.github.dfa1.vortex.proto.ScalarProtos.ScalarValue.Builder.class); - } - - private int kindCase_ = 0; - @SuppressWarnings("serial") - private java.lang.Object kind_; - public enum KindCase - implements com.google.protobuf.Internal.EnumLite, - com.google.protobuf.AbstractMessage.InternalOneOfEnum { - NULL_VALUE(1), - BOOL_VALUE(2), - INT64_VALUE(3), - UINT64_VALUE(4), - F32_VALUE(5), - F64_VALUE(6), - STRING_VALUE(7), - BYTES_VALUE(8), - LIST_VALUE(9), - F16_VALUE(10), - VARIANT_VALUE(11), - KIND_NOT_SET(0); - private final int value; - private KindCase(int value) { - this.value = value; - } - /** - * @param value The number of the enum to look for. - * @return The enum associated with the given number. - * @deprecated Use {@link #forNumber(int)} instead. - */ - @java.lang.Deprecated - public static KindCase valueOf(int value) { - return forNumber(value); - } - - public static KindCase forNumber(int value) { - switch (value) { - case 1: return NULL_VALUE; - case 2: return BOOL_VALUE; - case 3: return INT64_VALUE; - case 4: return UINT64_VALUE; - case 5: return F32_VALUE; - case 6: return F64_VALUE; - case 7: return STRING_VALUE; - case 8: return BYTES_VALUE; - case 9: return LIST_VALUE; - case 10: return F16_VALUE; - case 11: return VARIANT_VALUE; - case 0: return KIND_NOT_SET; - default: return null; - } - } - public int getNumber() { - return this.value; - } - }; - - public KindCase - getKindCase() { - return KindCase.forNumber( - kindCase_); - } - - public static final int NULL_VALUE_FIELD_NUMBER = 1; - /** - * .google.protobuf.NullValue null_value = 1; - * @return Whether the nullValue field is set. - */ - public boolean hasNullValue() { - return kindCase_ == 1; - } - /** - * .google.protobuf.NullValue null_value = 1; - * @return The enum numeric value on the wire for nullValue. - */ - public int getNullValueValue() { - if (kindCase_ == 1) { - return (java.lang.Integer) kind_; - } - return 0; - } - /** - * .google.protobuf.NullValue null_value = 1; - * @return The nullValue. - */ - public com.google.protobuf.NullValue getNullValue() { - if (kindCase_ == 1) { - com.google.protobuf.NullValue result = com.google.protobuf.NullValue.forNumber( - (java.lang.Integer) kind_); - return result == null ? com.google.protobuf.NullValue.UNRECOGNIZED : result; - } - return com.google.protobuf.NullValue.NULL_VALUE; - } - - public static final int BOOL_VALUE_FIELD_NUMBER = 2; - /** - * bool bool_value = 2; - * @return Whether the boolValue field is set. - */ - @java.lang.Override - public boolean hasBoolValue() { - return kindCase_ == 2; - } - /** - * bool bool_value = 2; - * @return The boolValue. - */ - @java.lang.Override - public boolean getBoolValue() { - if (kindCase_ == 2) { - return (java.lang.Boolean) kind_; - } - return false; - } - - public static final int INT64_VALUE_FIELD_NUMBER = 3; - /** - * sint64 int64_value = 3; - * @return Whether the int64Value field is set. - */ - @java.lang.Override - public boolean hasInt64Value() { - return kindCase_ == 3; - } - /** - * sint64 int64_value = 3; - * @return The int64Value. - */ - @java.lang.Override - public long getInt64Value() { - if (kindCase_ == 3) { - return (java.lang.Long) kind_; - } - return 0L; - } - - public static final int UINT64_VALUE_FIELD_NUMBER = 4; - /** - * uint64 uint64_value = 4; - * @return Whether the uint64Value field is set. - */ - @java.lang.Override - public boolean hasUint64Value() { - return kindCase_ == 4; - } - /** - * uint64 uint64_value = 4; - * @return The uint64Value. - */ - @java.lang.Override - public long getUint64Value() { - if (kindCase_ == 4) { - return (java.lang.Long) kind_; - } - return 0L; - } - - public static final int F32_VALUE_FIELD_NUMBER = 5; - /** - * float f32_value = 5; - * @return Whether the f32Value field is set. - */ - @java.lang.Override - public boolean hasF32Value() { - return kindCase_ == 5; - } - /** - * float f32_value = 5; - * @return The f32Value. - */ - @java.lang.Override - public float getF32Value() { - if (kindCase_ == 5) { - return (java.lang.Float) kind_; - } - return 0F; - } - - public static final int F64_VALUE_FIELD_NUMBER = 6; - /** - * double f64_value = 6; - * @return Whether the f64Value field is set. - */ - @java.lang.Override - public boolean hasF64Value() { - return kindCase_ == 6; - } - /** - * double f64_value = 6; - * @return The f64Value. - */ - @java.lang.Override - public double getF64Value() { - if (kindCase_ == 6) { - return (java.lang.Double) kind_; - } - return 0D; - } - - public static final int STRING_VALUE_FIELD_NUMBER = 7; - /** - * string string_value = 7; - * @return Whether the stringValue field is set. - */ - public boolean hasStringValue() { - return kindCase_ == 7; - } - /** - * string string_value = 7; - * @return The stringValue. - */ - public java.lang.String getStringValue() { - if (kindCase_ != 7) { - return ""; - } - java.lang.Object ref = kind_; - if (ref instanceof java.lang.String) { - return (java.lang.String) ref; - } else { - com.google.protobuf.ByteString bs = - (com.google.protobuf.ByteString) ref; - java.lang.String s = bs.toStringUtf8(); - kind_ = s; - return s; - } - } - /** - * string string_value = 7; - * @return The bytes for stringValue. - */ - public com.google.protobuf.ByteString - getStringValueBytes() { - if (kindCase_ != 7) { - return com.google.protobuf.ByteString.copyFromUtf8(""); - } - java.lang.Object ref = kind_; - if (ref instanceof java.lang.String) { - com.google.protobuf.ByteString b = - com.google.protobuf.ByteString.copyFromUtf8( - (java.lang.String) ref); - kind_ = b; - return b; - } else { - return (com.google.protobuf.ByteString) ref; - } - } - - public static final int BYTES_VALUE_FIELD_NUMBER = 8; - /** - * bytes bytes_value = 8; - * @return Whether the bytesValue field is set. - */ - @java.lang.Override - public boolean hasBytesValue() { - return kindCase_ == 8; - } - /** - * bytes bytes_value = 8; - * @return The bytesValue. - */ - @java.lang.Override - public com.google.protobuf.ByteString getBytesValue() { - if (kindCase_ == 8) { - return (com.google.protobuf.ByteString) kind_; - } - return com.google.protobuf.ByteString.EMPTY; - } - - public static final int LIST_VALUE_FIELD_NUMBER = 9; - /** - * .vortex.scalar.ListValue list_value = 9; - * @return Whether the listValue field is set. - */ - @java.lang.Override - public boolean hasListValue() { - return kindCase_ == 9; - } - /** - * .vortex.scalar.ListValue list_value = 9; - * @return The listValue. - */ - @java.lang.Override - public io.github.dfa1.vortex.proto.ScalarProtos.ListValue getListValue() { - if (kindCase_ == 9) { - return (io.github.dfa1.vortex.proto.ScalarProtos.ListValue) kind_; - } - return io.github.dfa1.vortex.proto.ScalarProtos.ListValue.getDefaultInstance(); - } - /** - * .vortex.scalar.ListValue list_value = 9; - */ - @java.lang.Override - public io.github.dfa1.vortex.proto.ScalarProtos.ListValueOrBuilder getListValueOrBuilder() { - if (kindCase_ == 9) { - return (io.github.dfa1.vortex.proto.ScalarProtos.ListValue) kind_; - } - return io.github.dfa1.vortex.proto.ScalarProtos.ListValue.getDefaultInstance(); - } - - public static final int F16_VALUE_FIELD_NUMBER = 10; - /** - * uint64 f16_value = 10; - * @return Whether the f16Value field is set. - */ - @java.lang.Override - public boolean hasF16Value() { - return kindCase_ == 10; - } - /** - * uint64 f16_value = 10; - * @return The f16Value. - */ - @java.lang.Override - public long getF16Value() { - if (kindCase_ == 10) { - return (java.lang.Long) kind_; - } - return 0L; - } - - public static final int VARIANT_VALUE_FIELD_NUMBER = 11; - /** - * 
-     * Variant scalars carry a row-specific nested scalar.
-     * See RFC 0015: https://github.com/vortex-data/rfcs/blob/develop/accepted/0015-variant-type.md
-     *
- * - * .vortex.scalar.Scalar variant_value = 11; - * @return Whether the variantValue field is set. - */ - @java.lang.Override - public boolean hasVariantValue() { - return kindCase_ == 11; - } - /** - * 
-     * Variant scalars carry a row-specific nested scalar.
-     * See RFC 0015: https://github.com/vortex-data/rfcs/blob/develop/accepted/0015-variant-type.md
-     *
- * - * .vortex.scalar.Scalar variant_value = 11; - * @return The variantValue. - */ - @java.lang.Override - public io.github.dfa1.vortex.proto.ScalarProtos.Scalar getVariantValue() { - if (kindCase_ == 11) { - return (io.github.dfa1.vortex.proto.ScalarProtos.Scalar) kind_; - } - return io.github.dfa1.vortex.proto.ScalarProtos.Scalar.getDefaultInstance(); - } - /** - * 
-     * Variant scalars carry a row-specific nested scalar.
-     * See RFC 0015: https://github.com/vortex-data/rfcs/blob/develop/accepted/0015-variant-type.md
-     *
- * - * .vortex.scalar.Scalar variant_value = 11; - */ - @java.lang.Override - public io.github.dfa1.vortex.proto.ScalarProtos.ScalarOrBuilder getVariantValueOrBuilder() { - if (kindCase_ == 11) { - return (io.github.dfa1.vortex.proto.ScalarProtos.Scalar) kind_; - } - return io.github.dfa1.vortex.proto.ScalarProtos.Scalar.getDefaultInstance(); - } - - private byte memoizedIsInitialized = -1; - @java.lang.Override - public final boolean isInitialized() { - byte isInitialized = memoizedIsInitialized; - if (isInitialized == 1) return true; - if (isInitialized == 0) return false; - - memoizedIsInitialized = 1; - return true; - } - - @java.lang.Override - public void writeTo(com.google.protobuf.CodedOutputStream output) - throws java.io.IOException { - if (kindCase_ == 1) { - output.writeEnum(1, ((java.lang.Integer) kind_)); - } - if (kindCase_ == 2) { - output.writeBool( - 2, (boolean)((java.lang.Boolean) kind_)); - } - if (kindCase_ == 3) { - output.writeSInt64( - 3, (long)((java.lang.Long) kind_)); - } - if (kindCase_ == 4) { - output.writeUInt64( - 4, (long)((java.lang.Long) kind_)); - } - if (kindCase_ == 5) { - output.writeFloat( - 5, (float)((java.lang.Float) kind_)); - } - if (kindCase_ == 6) { - output.writeDouble( - 6, (double)((java.lang.Double) kind_)); - } - if (kindCase_ == 7) { - com.google.protobuf.GeneratedMessage.writeString(output, 7, kind_); - } - if (kindCase_ == 8) { - output.writeBytes( - 8, (com.google.protobuf.ByteString) kind_); - } - if (kindCase_ == 9) { - output.writeMessage(9, (io.github.dfa1.vortex.proto.ScalarProtos.ListValue) kind_); - } - if (kindCase_ == 10) { - output.writeUInt64( - 10, (long)((java.lang.Long) kind_)); - } - if (kindCase_ == 11) { - output.writeMessage(11, (io.github.dfa1.vortex.proto.ScalarProtos.Scalar) kind_); - } - getUnknownFields().writeTo(output); - } - private int computeSerializedSize_0() { - int size = 0; - if (kindCase_ == 1) { - size += com.google.protobuf.CodedOutputStream - .computeEnumSize(1, ((java.lang.Integer) kind_)); - } - if (kindCase_ == 2) { - size += com.google.protobuf.CodedOutputStream - .computeBoolSize( - 2, (boolean)((java.lang.Boolean) kind_)); - } - if (kindCase_ == 3) { - size += com.google.protobuf.CodedOutputStream - .computeSInt64Size( - 3, (long)((java.lang.Long) kind_)); - } - if (kindCase_ == 4) { - size += com.google.protobuf.CodedOutputStream - .computeUInt64Size( - 4, (long)((java.lang.Long) kind_)); - } - if (kindCase_ == 5) { - size += com.google.protobuf.CodedOutputStream - .computeFloatSize( - 5, (float)((java.lang.Float) kind_)); - } - if (kindCase_ == 6) { - size += com.google.protobuf.CodedOutputStream - .computeDoubleSize( - 6, (double)((java.lang.Double) kind_)); - } - if (kindCase_ == 7) { - size += com.google.protobuf.GeneratedMessage.computeStringSize(7, kind_); - } - if (kindCase_ == 8) { - size += com.google.protobuf.CodedOutputStream - .computeBytesSize( - 8, (com.google.protobuf.ByteString) kind_); - } - if (kindCase_ == 9) { - size += com.google.protobuf.CodedOutputStream - .computeMessageSize(9, (io.github.dfa1.vortex.proto.ScalarProtos.ListValue) kind_); - } - if (kindCase_ == 10) { - size += com.google.protobuf.CodedOutputStream - .computeUInt64Size( - 10, (long)((java.lang.Long) kind_)); - } - if (kindCase_ == 11) { - size += com.google.protobuf.CodedOutputStream - .computeMessageSize(11, (io.github.dfa1.vortex.proto.ScalarProtos.Scalar) kind_); - } - return size; - } - @java.lang.Override - public int getSerializedSize() { - int size = memoizedSize; - if (size != -1) return size; - - size = 0; - size += computeSerializedSize_0(); - size += getUnknownFields().getSerializedSize(); - memoizedSize = size; - return size; - } - - @java.lang.Override - public boolean equals(final java.lang.Object obj) { - if (obj == this) { - return true; - } - if (!(obj instanceof io.github.dfa1.vortex.proto.ScalarProtos.ScalarValue)) { - return super.equals(obj); - } - io.github.dfa1.vortex.proto.ScalarProtos.ScalarValue other = (io.github.dfa1.vortex.proto.ScalarProtos.ScalarValue) obj; - - if (!getKindCase().equals(other.getKindCase())) return false; - switch (kindCase_) { - case 1: - if (getNullValueValue() - != other.getNullValueValue()) return false; - break; - case 2: - if (getBoolValue() - != other.getBoolValue()) return false; - break; - case 3: - if (getInt64Value() - != other.getInt64Value()) return false; - break; - case 4: - if (getUint64Value() - != other.getUint64Value()) return false; - break; - case 5: - if (java.lang.Float.floatToIntBits(getF32Value()) - != java.lang.Float.floatToIntBits( - other.getF32Value())) return false; - break; - case 6: - if (java.lang.Double.doubleToLongBits(getF64Value()) - != java.lang.Double.doubleToLongBits( - other.getF64Value())) return false; - break; - case 7: - if (!getStringValue() - .equals(other.getStringValue())) return false; - break; - case 8: - if (!getBytesValue() - .equals(other.getBytesValue())) return false; - break; - case 9: - if (!getListValue() - .equals(other.getListValue())) return false; - break; - case 10: - if (getF16Value() - != other.getF16Value()) return false; - break; - case 11: - if (!getVariantValue() - .equals(other.getVariantValue())) return false; - break; - case 0: - default: - } - if (!getUnknownFields().equals(other.getUnknownFields())) return false; - return true; - } - - @java.lang.Override - public int hashCode() { - if (memoizedHashCode != 0) { - return memoizedHashCode; - } - int hash = 41; - hash = (19 * hash) + getDescriptor().hashCode(); - switch (kindCase_) { - case 1: - hash = (37 * hash) + NULL_VALUE_FIELD_NUMBER; - hash = (53 * hash) + getNullValueValue(); - break; - case 2: - hash = (37 * hash) + BOOL_VALUE_FIELD_NUMBER; - hash = (53 * hash) + com.google.protobuf.Internal.hashBoolean( - getBoolValue()); - break; - case 3: - hash = (37 * hash) + INT64_VALUE_FIELD_NUMBER; - hash = (53 * hash) + com.google.protobuf.Internal.hashLong( - getInt64Value()); - break; - case 4: - hash = (37 * hash) + UINT64_VALUE_FIELD_NUMBER; - hash = (53 * hash) + com.google.protobuf.Internal.hashLong( - getUint64Value()); - break; - case 5: - hash = (37 * hash) + F32_VALUE_FIELD_NUMBER; - hash = (53 * hash) + java.lang.Float.floatToIntBits( - getF32Value()); - break; - case 6: - hash = (37 * hash) + F64_VALUE_FIELD_NUMBER; - hash = (53 * hash) + com.google.protobuf.Internal.hashLong( - java.lang.Double.doubleToLongBits(getF64Value())); - break; - case 7: - hash = (37 * hash) + STRING_VALUE_FIELD_NUMBER; - hash = (53 * hash) + getStringValue().hashCode(); - break; - case 8: - hash = (37 * hash) + BYTES_VALUE_FIELD_NUMBER; - hash = (53 * hash) + getBytesValue().hashCode(); - break; - case 9: - hash = (37 * hash) + LIST_VALUE_FIELD_NUMBER; - hash = (53 * hash) + getListValue().hashCode(); - break; - case 10: - hash = (37 * hash) + F16_VALUE_FIELD_NUMBER; - hash = (53 * hash) + com.google.protobuf.Internal.hashLong( - getF16Value()); - break; - case 11: - hash = (37 * hash) + VARIANT_VALUE_FIELD_NUMBER; - hash = (53 * hash) + getVariantValue().hashCode(); - break; - case 0: - default: - } - hash = (29 * hash) + getUnknownFields().hashCode(); - memoizedHashCode = hash; - return hash; - } - - public static io.github.dfa1.vortex.proto.ScalarProtos.ScalarValue parseFrom( - java.nio.ByteBuffer data) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data); - } - public static io.github.dfa1.vortex.proto.ScalarProtos.ScalarValue parseFrom( - java.nio.ByteBuffer data, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data, extensionRegistry); - } - public static io.github.dfa1.vortex.proto.ScalarProtos.ScalarValue parseFrom( - com.google.protobuf.ByteString data) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data); - } - public static io.github.dfa1.vortex.proto.ScalarProtos.ScalarValue parseFrom( - com.google.protobuf.ByteString data, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data, extensionRegistry); - } - public static io.github.dfa1.vortex.proto.ScalarProtos.ScalarValue parseFrom(byte[] data) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data); - } - public static io.github.dfa1.vortex.proto.ScalarProtos.ScalarValue parseFrom( - byte[] data, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data, extensionRegistry); - } - public static io.github.dfa1.vortex.proto.ScalarProtos.ScalarValue parseFrom(java.io.InputStream input) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseWithIOException(PARSER, input); - } - public static io.github.dfa1.vortex.proto.ScalarProtos.ScalarValue parseFrom( - java.io.InputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseWithIOException(PARSER, input, extensionRegistry); - } - - public static io.github.dfa1.vortex.proto.ScalarProtos.ScalarValue parseDelimitedFrom(java.io.InputStream input) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseDelimitedWithIOException(PARSER, input); - } - - public static io.github.dfa1.vortex.proto.ScalarProtos.ScalarValue parseDelimitedFrom( - java.io.InputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseDelimitedWithIOException(PARSER, input, extensionRegistry); - } - public static io.github.dfa1.vortex.proto.ScalarProtos.ScalarValue parseFrom( - com.google.protobuf.CodedInputStream input) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseWithIOException(PARSER, input); - } - public static io.github.dfa1.vortex.proto.ScalarProtos.ScalarValue parseFrom( - com.google.protobuf.CodedInputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseWithIOException(PARSER, input, extensionRegistry); - } - - @java.lang.Override - public Builder newBuilderForType() { return newBuilder(); } - public static Builder newBuilder() { - return DEFAULT_INSTANCE.toBuilder(); - } - public static Builder newBuilder(io.github.dfa1.vortex.proto.ScalarProtos.ScalarValue prototype) { - return DEFAULT_INSTANCE.toBuilder().mergeFrom(prototype); - } - @java.lang.Override - public Builder toBuilder() { - return this == DEFAULT_INSTANCE - ? new Builder() : new Builder().mergeFrom(this); - } - - @java.lang.Override - protected Builder newBuilderForType( - com.google.protobuf.GeneratedMessage.BuilderParent parent) { - Builder builder = new Builder(parent); - return builder; - } - /** - * Protobuf type {@code vortex.scalar.ScalarValue} - */ - public static final class Builder extends - com.google.protobuf.GeneratedMessage.Builder implements - // @@protoc_insertion_point(builder_implements:vortex.scalar.ScalarValue) - io.github.dfa1.vortex.proto.ScalarProtos.ScalarValueOrBuilder { - public static final com.google.protobuf.Descriptors.Descriptor - getDescriptor() { - return io.github.dfa1.vortex.proto.ScalarProtos.internal_static_vortex_scalar_ScalarValue_descriptor; - } - - @java.lang.Override - protected com.google.protobuf.GeneratedMessage.FieldAccessorTable - internalGetFieldAccessorTable() { - return io.github.dfa1.vortex.proto.ScalarProtos.internal_static_vortex_scalar_ScalarValue_fieldAccessorTable - .ensureFieldAccessorsInitialized( - io.github.dfa1.vortex.proto.ScalarProtos.ScalarValue.class, io.github.dfa1.vortex.proto.ScalarProtos.ScalarValue.Builder.class); - } - - // Construct using io.github.dfa1.vortex.proto.ScalarProtos.ScalarValue.newBuilder() - private Builder() { - - } - - private Builder( - com.google.protobuf.GeneratedMessage.BuilderParent parent) { - super(parent); - - } - @java.lang.Override - public Builder clear() { - super.clear(); - bitField0_ = 0; - if (listValueBuilder_ != null) { - listValueBuilder_.clear(); - } - if (variantValueBuilder_ != null) { - variantValueBuilder_.clear(); - } - kindCase_ = 0; - kind_ = null; - return this; - } - - @java.lang.Override - public com.google.protobuf.Descriptors.Descriptor - getDescriptorForType() { - return io.github.dfa1.vortex.proto.ScalarProtos.internal_static_vortex_scalar_ScalarValue_descriptor; - } - - @java.lang.Override - public io.github.dfa1.vortex.proto.ScalarProtos.ScalarValue getDefaultInstanceForType() { - return io.github.dfa1.vortex.proto.ScalarProtos.ScalarValue.getDefaultInstance(); - } - - @java.lang.Override - public io.github.dfa1.vortex.proto.ScalarProtos.ScalarValue build() { - io.github.dfa1.vortex.proto.ScalarProtos.ScalarValue result = buildPartial(); - if (!result.isInitialized()) { - throw newUninitializedMessageException(result); - } - return result; - } - - @java.lang.Override - public io.github.dfa1.vortex.proto.ScalarProtos.ScalarValue buildPartial() { - io.github.dfa1.vortex.proto.ScalarProtos.ScalarValue result = new io.github.dfa1.vortex.proto.ScalarProtos.ScalarValue(this); - if (bitField0_ != 0) { buildPartial0(result); } - buildPartialOneofs(result); - onBuilt(); - return result; - } - - private void buildPartial0(io.github.dfa1.vortex.proto.ScalarProtos.ScalarValue result) { - int from_bitField0_ = bitField0_; - } - - private void buildPartialOneofs(io.github.dfa1.vortex.proto.ScalarProtos.ScalarValue result) { - result.kindCase_ = kindCase_; - result.kind_ = this.kind_; - if (kindCase_ == 9 && - listValueBuilder_ != null) { - result.kind_ = listValueBuilder_.build(); - } - if (kindCase_ == 11 && - variantValueBuilder_ != null) { - result.kind_ = variantValueBuilder_.build(); - } - } - - @java.lang.Override - public Builder mergeFrom(com.google.protobuf.Message other) { - if (other instanceof io.github.dfa1.vortex.proto.ScalarProtos.ScalarValue) { - return mergeFrom((io.github.dfa1.vortex.proto.ScalarProtos.ScalarValue)other); - } else { - super.mergeFrom(other); - return this; - } - } - - public Builder mergeFrom(io.github.dfa1.vortex.proto.ScalarProtos.ScalarValue other) { - if (other == io.github.dfa1.vortex.proto.ScalarProtos.ScalarValue.getDefaultInstance()) return this; - switch (other.getKindCase()) { - case NULL_VALUE: { - setNullValueValue(other.getNullValueValue()); - break; - } - case BOOL_VALUE: { - setBoolValue(other.getBoolValue()); - break; - } - case INT64_VALUE: { - setInt64Value(other.getInt64Value()); - break; - } - case UINT64_VALUE: { - setUint64Value(other.getUint64Value()); - break; - } - case F32_VALUE: { - setF32Value(other.getF32Value()); - break; - } - case F64_VALUE: { - setF64Value(other.getF64Value()); - break; - } - case STRING_VALUE: { - kindCase_ = 7; - kind_ = other.kind_; - onChanged(); - break; - } - case BYTES_VALUE: { - setBytesValue(other.getBytesValue()); - break; - } - case LIST_VALUE: { - mergeListValue(other.getListValue()); - break; - } - case F16_VALUE: { - setF16Value(other.getF16Value()); - break; - } - case VARIANT_VALUE: { - mergeVariantValue(other.getVariantValue()); - break; - } - case KIND_NOT_SET: { - break; - } - } - this.mergeUnknownFields(other.getUnknownFields()); - onChanged(); - return this; - } - - @java.lang.Override - public final boolean isInitialized() { - return true; - } - - @java.lang.Override - public Builder mergeFrom( - com.google.protobuf.CodedInputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws java.io.IOException { - if (extensionRegistry == null) { - throw new java.lang.NullPointerException(); - } - try { - boolean done = false; - while (!done) { - int tag = input.readTag(); - switch (tag) { - case 0: - done = true; - break; - case 8: { - int rawValue = input.readEnum(); - kindCase_ = 1; - kind_ = rawValue; - break; - } // case 8 - case 16: { - kind_ = input.readBool(); - kindCase_ = 2; - break; - } // case 16 - case 24: { - kind_ = input.readSInt64(); - kindCase_ = 3; - break; - } // case 24 - case 32: { - kind_ = input.readUInt64(); - kindCase_ = 4; - break; - } // case 32 - case 45: { - kind_ = input.readFloat(); - kindCase_ = 5; - break; - } // case 45 - case 49: { - kind_ = input.readDouble(); - kindCase_ = 6; - break; - } // case 49 - case 58: { - kindCase_ = 7; - kind_ = input.readStringRequireUtf8(); - break; - } // case 58 - case 66: { - kind_ = input.readBytes(); - kindCase_ = 8; - break; - } // case 66 - case 74: { - input.readMessage( - internalGetListValueFieldBuilder().getBuilder(), - extensionRegistry); - kindCase_ = 9; - break; - } // case 74 - case 80: { - kind_ = input.readUInt64(); - kindCase_ = 10; - break; - } // case 80 - case 90: { - input.readMessage( - internalGetVariantValueFieldBuilder().getBuilder(), - extensionRegistry); - kindCase_ = 11; - break; - } // case 90 - default: { - if (!super.parseUnknownField(input, extensionRegistry, tag)) { - done = true; // was an endgroup tag - } - break; - } // default: - } // switch (tag) - } // while (!done) - } catch (com.google.protobuf.InvalidProtocolBufferException e) { - throw e.unwrapIOException(); - } finally { - onChanged(); - } // finally - return this; - } - private int kindCase_ = 0; - private java.lang.Object kind_; - public KindCase - getKindCase() { - return KindCase.forNumber( - kindCase_); - } - - public Builder clearKind() { - kindCase_ = 0; - kind_ = null; - onChanged(); - return this; - } - - private int bitField0_; - - /** - * .google.protobuf.NullValue null_value = 1; - * @return Whether the nullValue field is set. - */ - @java.lang.Override - public boolean hasNullValue() { - return kindCase_ == 1; - } - /** - * .google.protobuf.NullValue null_value = 1; - * @return The enum numeric value on the wire for nullValue. - */ - @java.lang.Override - public int getNullValueValue() { - if (kindCase_ == 1) { - return ((java.lang.Integer) kind_).intValue(); - } - return 0; - } - /** - * .google.protobuf.NullValue null_value = 1; - * @param value The enum numeric value on the wire for nullValue to set. - * @return This builder for chaining. - */ - public Builder setNullValueValue(int value) { - kindCase_ = 1; - kind_ = value; - onChanged(); - return this; - } - /** - * .google.protobuf.NullValue null_value = 1; - * @return The nullValue. - */ - @java.lang.Override - public com.google.protobuf.NullValue getNullValue() { - if (kindCase_ == 1) { - com.google.protobuf.NullValue result = com.google.protobuf.NullValue.forNumber( - (java.lang.Integer) kind_); - return result == null ? com.google.protobuf.NullValue.UNRECOGNIZED : result; - } - return com.google.protobuf.NullValue.NULL_VALUE; - } - /** - * .google.protobuf.NullValue null_value = 1; - * @param value The nullValue to set. - * @throws IllegalArgumentException if UNRECOGNIZED is provided. - * @return This builder for chaining. - */ - public Builder setNullValue(com.google.protobuf.NullValue value) { - if (value == null) { throw new NullPointerException(); } - kindCase_ = 1; - kind_ = value.getNumber(); - onChanged(); - return this; - } - /** - * .google.protobuf.NullValue null_value = 1; - * @return This builder for chaining. - */ - public Builder clearNullValue() { - if (kindCase_ == 1) { - kindCase_ = 0; - kind_ = null; - onChanged(); - } - return this; - } - - /** - * bool bool_value = 2; - * @return Whether the boolValue field is set. - */ - public boolean hasBoolValue() { - return kindCase_ == 2; - } - /** - * bool bool_value = 2; - * @return The boolValue. - */ - public boolean getBoolValue() { - if (kindCase_ == 2) { - return (java.lang.Boolean) kind_; - } - return false; - } - /** - * bool bool_value = 2; - * @param value The boolValue to set. - * @return This builder for chaining. - */ - public Builder setBoolValue(boolean value) { - - kindCase_ = 2; - kind_ = value; - onChanged(); - return this; - } - /** - * bool bool_value = 2; - * @return This builder for chaining. - */ - public Builder clearBoolValue() { - if (kindCase_ == 2) { - kindCase_ = 0; - kind_ = null; - onChanged(); - } - return this; - } - - /** - * sint64 int64_value = 3; - * @return Whether the int64Value field is set. - */ - public boolean hasInt64Value() { - return kindCase_ == 3; - } - /** - * sint64 int64_value = 3; - * @return The int64Value. - */ - public long getInt64Value() { - if (kindCase_ == 3) { - return (java.lang.Long) kind_; - } - return 0L; - } - /** - * sint64 int64_value = 3; - * @param value The int64Value to set. - * @return This builder for chaining. - */ - public Builder setInt64Value(long value) { - - kindCase_ = 3; - kind_ = value; - onChanged(); - return this; - } - /** - * sint64 int64_value = 3; - * @return This builder for chaining. - */ - public Builder clearInt64Value() { - if (kindCase_ == 3) { - kindCase_ = 0; - kind_ = null; - onChanged(); - } - return this; - } - - /** - * uint64 uint64_value = 4; - * @return Whether the uint64Value field is set. - */ - public boolean hasUint64Value() { - return kindCase_ == 4; - } - /** - * uint64 uint64_value = 4; - * @return The uint64Value. - */ - public long getUint64Value() { - if (kindCase_ == 4) { - return (java.lang.Long) kind_; - } - return 0L; - } - /** - * uint64 uint64_value = 4; - * @param value The uint64Value to set. - * @return This builder for chaining. - */ - public Builder setUint64Value(long value) { - - kindCase_ = 4; - kind_ = value; - onChanged(); - return this; - } - /** - * uint64 uint64_value = 4; - * @return This builder for chaining. - */ - public Builder clearUint64Value() { - if (kindCase_ == 4) { - kindCase_ = 0; - kind_ = null; - onChanged(); - } - return this; - } - - /** - * float f32_value = 5; - * @return Whether the f32Value field is set. - */ - public boolean hasF32Value() { - return kindCase_ == 5; - } - /** - * float f32_value = 5; - * @return The f32Value. - */ - public float getF32Value() { - if (kindCase_ == 5) { - return (java.lang.Float) kind_; - } - return 0F; - } - /** - * float f32_value = 5; - * @param value The f32Value to set. - * @return This builder for chaining. - */ - public Builder setF32Value(float value) { - - kindCase_ = 5; - kind_ = value; - onChanged(); - return this; - } - /** - * float f32_value = 5; - * @return This builder for chaining. - */ - public Builder clearF32Value() { - if (kindCase_ == 5) { - kindCase_ = 0; - kind_ = null; - onChanged(); - } - return this; - } - - /** - * double f64_value = 6; - * @return Whether the f64Value field is set. - */ - public boolean hasF64Value() { - return kindCase_ == 6; - } - /** - * double f64_value = 6; - * @return The f64Value. - */ - public double getF64Value() { - if (kindCase_ == 6) { - return (java.lang.Double) kind_; - } - return 0D; - } - /** - * double f64_value = 6; - * @param value The f64Value to set. - * @return This builder for chaining. - */ - public Builder setF64Value(double value) { - - kindCase_ = 6; - kind_ = value; - onChanged(); - return this; - } - /** - * double f64_value = 6; - * @return This builder for chaining. - */ - public Builder clearF64Value() { - if (kindCase_ == 6) { - kindCase_ = 0; - kind_ = null; - onChanged(); - } - return this; - } - - /** - * string string_value = 7; - * @return Whether the stringValue field is set. - */ - @java.lang.Override - public boolean hasStringValue() { - return kindCase_ == 7; - } - /** - * string string_value = 7; - * @return The stringValue. - */ - @java.lang.Override - public java.lang.String getStringValue() { - if (kindCase_ != 7) { - return ""; - } - java.lang.Object ref = kind_; - if (!(ref instanceof java.lang.String)) { - com.google.protobuf.ByteString bs = - (com.google.protobuf.ByteString) ref; - java.lang.String s = bs.toStringUtf8(); - kind_ = s; - return s; - } else { - return (java.lang.String) ref; - } - } - /** - * string string_value = 7; - * @return The bytes for stringValue. - */ - @java.lang.Override - public com.google.protobuf.ByteString - getStringValueBytes() { - if (kindCase_ != 7) { - return com.google.protobuf.ByteString.copyFromUtf8( ""); - } - java.lang.Object ref = kind_; - if (ref instanceof String) { - com.google.protobuf.ByteString b = - com.google.protobuf.ByteString.copyFromUtf8( - (java.lang.String) ref); - kind_ = b; - return b; - } else { - return (com.google.protobuf.ByteString) ref; - } - } - /** - * string string_value = 7; - * @param value The stringValue to set. - * @return This builder for chaining. - */ - public Builder setStringValue( - java.lang.String value) { - if (value == null) { throw new NullPointerException(); } - kindCase_ = 7; - kind_ = value; - onChanged(); - return this; - } - /** - * string string_value = 7; - * @return This builder for chaining. - */ - public Builder clearStringValue() { - if (kindCase_ == 7) { - kindCase_ = 0; - kind_ = null; - onChanged(); - } - return this; - } - /** - * string string_value = 7; - * @param value The bytes for stringValue to set. - * @return This builder for chaining. - */ - public Builder setStringValueBytes( - com.google.protobuf.ByteString value) { - if (value == null) { throw new NullPointerException(); } - checkByteStringIsUtf8(value); - kindCase_ = 7; - kind_ = value; - onChanged(); - return this; - } - - /** - * bytes bytes_value = 8; - * @return Whether the bytesValue field is set. - */ - public boolean hasBytesValue() { - return kindCase_ == 8; - } - /** - * bytes bytes_value = 8; - * @return The bytesValue. - */ - public com.google.protobuf.ByteString getBytesValue() { - if (kindCase_ == 8) { - return (com.google.protobuf.ByteString) kind_; - } - return com.google.protobuf.ByteString.EMPTY; - } - /** - * bytes bytes_value = 8; - * @param value The bytesValue to set. - * @return This builder for chaining. - */ - public Builder setBytesValue(com.google.protobuf.ByteString value) { - if (value == null) { throw new NullPointerException(); } - kindCase_ = 8; - kind_ = value; - onChanged(); - return this; - } - /** - * bytes bytes_value = 8; - * @return This builder for chaining. - */ - public Builder clearBytesValue() { - if (kindCase_ == 8) { - kindCase_ = 0; - kind_ = null; - onChanged(); - } - return this; - } - - private com.google.protobuf.SingleFieldBuilder< - io.github.dfa1.vortex.proto.ScalarProtos.ListValue, io.github.dfa1.vortex.proto.ScalarProtos.ListValue.Builder, io.github.dfa1.vortex.proto.ScalarProtos.ListValueOrBuilder> listValueBuilder_; - /** - * .vortex.scalar.ListValue list_value = 9; - * @return Whether the listValue field is set. - */ - @java.lang.Override - public boolean hasListValue() { - return kindCase_ == 9; - } - /** - * .vortex.scalar.ListValue list_value = 9; - * @return The listValue. - */ - @java.lang.Override - public io.github.dfa1.vortex.proto.ScalarProtos.ListValue getListValue() { - if (listValueBuilder_ == null) { - if (kindCase_ == 9) { - return (io.github.dfa1.vortex.proto.ScalarProtos.ListValue) kind_; - } - return io.github.dfa1.vortex.proto.ScalarProtos.ListValue.getDefaultInstance(); - } else { - if (kindCase_ == 9) { - return listValueBuilder_.getMessage(); - } - return io.github.dfa1.vortex.proto.ScalarProtos.ListValue.getDefaultInstance(); - } - } - /** - * .vortex.scalar.ListValue list_value = 9; - */ - public Builder setListValue(io.github.dfa1.vortex.proto.ScalarProtos.ListValue value) { - if (listValueBuilder_ == null) { - if (value == null) { - throw new NullPointerException(); - } - kind_ = value; - onChanged(); - } else { - listValueBuilder_.setMessage(value); - } - kindCase_ = 9; - return this; - } - /** - * .vortex.scalar.ListValue list_value = 9; - */ - public Builder setListValue( - io.github.dfa1.vortex.proto.ScalarProtos.ListValue.Builder builderForValue) { - if (listValueBuilder_ == null) { - kind_ = builderForValue.build(); - onChanged(); - } else { - listValueBuilder_.setMessage(builderForValue.build()); - } - kindCase_ = 9; - return this; - } - /** - * .vortex.scalar.ListValue list_value = 9; - */ - public Builder mergeListValue(io.github.dfa1.vortex.proto.ScalarProtos.ListValue value) { - if (listValueBuilder_ == null) { - if (kindCase_ == 9 && - kind_ != io.github.dfa1.vortex.proto.ScalarProtos.ListValue.getDefaultInstance()) { - kind_ = io.github.dfa1.vortex.proto.ScalarProtos.ListValue.newBuilder((io.github.dfa1.vortex.proto.ScalarProtos.ListValue) kind_) - .mergeFrom(value).buildPartial(); - } else { - kind_ = value; - } - onChanged(); - } else { - if (kindCase_ == 9) { - listValueBuilder_.mergeFrom(value); - } else { - listValueBuilder_.setMessage(value); - } - } - kindCase_ = 9; - return this; - } - /** - * .vortex.scalar.ListValue list_value = 9; - */ - public Builder clearListValue() { - if (listValueBuilder_ == null) { - if (kindCase_ == 9) { - kindCase_ = 0; - kind_ = null; - onChanged(); - } - } else { - if (kindCase_ == 9) { - kindCase_ = 0; - kind_ = null; - } - listValueBuilder_.clear(); - } - return this; - } - /** - * .vortex.scalar.ListValue list_value = 9; - */ - public io.github.dfa1.vortex.proto.ScalarProtos.ListValue.Builder getListValueBuilder() { - return internalGetListValueFieldBuilder().getBuilder(); - } - /** - * .vortex.scalar.ListValue list_value = 9; - */ - @java.lang.Override - public io.github.dfa1.vortex.proto.ScalarProtos.ListValueOrBuilder getListValueOrBuilder() { - if ((kindCase_ == 9) && (listValueBuilder_ != null)) { - return listValueBuilder_.getMessageOrBuilder(); - } else { - if (kindCase_ == 9) { - return (io.github.dfa1.vortex.proto.ScalarProtos.ListValue) kind_; - } - return io.github.dfa1.vortex.proto.ScalarProtos.ListValue.getDefaultInstance(); - } - } - /** - * .vortex.scalar.ListValue list_value = 9; - */ - private com.google.protobuf.SingleFieldBuilder< - io.github.dfa1.vortex.proto.ScalarProtos.ListValue, io.github.dfa1.vortex.proto.ScalarProtos.ListValue.Builder, io.github.dfa1.vortex.proto.ScalarProtos.ListValueOrBuilder> - internalGetListValueFieldBuilder() { - if (listValueBuilder_ == null) { - if (!(kindCase_ == 9)) { - kind_ = io.github.dfa1.vortex.proto.ScalarProtos.ListValue.getDefaultInstance(); - } - listValueBuilder_ = new com.google.protobuf.SingleFieldBuilder< - io.github.dfa1.vortex.proto.ScalarProtos.ListValue, io.github.dfa1.vortex.proto.ScalarProtos.ListValue.Builder, io.github.dfa1.vortex.proto.ScalarProtos.ListValueOrBuilder>( - (io.github.dfa1.vortex.proto.ScalarProtos.ListValue) kind_, - getParentForChildren(), - isClean()); - kind_ = null; - } - kindCase_ = 9; - onChanged(); - return listValueBuilder_; - } - - /** - * uint64 f16_value = 10; - * @return Whether the f16Value field is set. - */ - public boolean hasF16Value() { - return kindCase_ == 10; - } - /** - * uint64 f16_value = 10; - * @return The f16Value. - */ - public long getF16Value() { - if (kindCase_ == 10) { - return (java.lang.Long) kind_; - } - return 0L; - } - /** - * uint64 f16_value = 10; - * @param value The f16Value to set. - * @return This builder for chaining. - */ - public Builder setF16Value(long value) { - - kindCase_ = 10; - kind_ = value; - onChanged(); - return this; - } - /** - * uint64 f16_value = 10; - * @return This builder for chaining. - */ - public Builder clearF16Value() { - if (kindCase_ == 10) { - kindCase_ = 0; - kind_ = null; - onChanged(); - } - return this; - } - - private com.google.protobuf.SingleFieldBuilder< - io.github.dfa1.vortex.proto.ScalarProtos.Scalar, io.github.dfa1.vortex.proto.ScalarProtos.Scalar.Builder, io.github.dfa1.vortex.proto.ScalarProtos.ScalarOrBuilder> variantValueBuilder_; - /** - * 
-       * Variant scalars carry a row-specific nested scalar.
-       * See RFC 0015: https://github.com/vortex-data/rfcs/blob/develop/accepted/0015-variant-type.md
-       *
- * - * .vortex.scalar.Scalar variant_value = 11; - * @return Whether the variantValue field is set. - */ - @java.lang.Override - public boolean hasVariantValue() { - return kindCase_ == 11; - } - /** - * 
-       * Variant scalars carry a row-specific nested scalar.
-       * See RFC 0015: https://github.com/vortex-data/rfcs/blob/develop/accepted/0015-variant-type.md
-       *
- * - * .vortex.scalar.Scalar variant_value = 11; - * @return The variantValue. - */ - @java.lang.Override - public io.github.dfa1.vortex.proto.ScalarProtos.Scalar getVariantValue() { - if (variantValueBuilder_ == null) { - if (kindCase_ == 11) { - return (io.github.dfa1.vortex.proto.ScalarProtos.Scalar) kind_; - } - return io.github.dfa1.vortex.proto.ScalarProtos.Scalar.getDefaultInstance(); - } else { - if (kindCase_ == 11) { - return variantValueBuilder_.getMessage(); - } - return io.github.dfa1.vortex.proto.ScalarProtos.Scalar.getDefaultInstance(); - } - } - /** - * 
-       * Variant scalars carry a row-specific nested scalar.
-       * See RFC 0015: https://github.com/vortex-data/rfcs/blob/develop/accepted/0015-variant-type.md
-       *
- * - * .vortex.scalar.Scalar variant_value = 11; - */ - public Builder setVariantValue(io.github.dfa1.vortex.proto.ScalarProtos.Scalar value) { - if (variantValueBuilder_ == null) { - if (value == null) { - throw new NullPointerException(); - } - kind_ = value; - onChanged(); - } else { - variantValueBuilder_.setMessage(value); - } - kindCase_ = 11; - return this; - } - /** - * 
-       * Variant scalars carry a row-specific nested scalar.
-       * See RFC 0015: https://github.com/vortex-data/rfcs/blob/develop/accepted/0015-variant-type.md
-       *
- * - * .vortex.scalar.Scalar variant_value = 11; - */ - public Builder setVariantValue( - io.github.dfa1.vortex.proto.ScalarProtos.Scalar.Builder builderForValue) { - if (variantValueBuilder_ == null) { - kind_ = builderForValue.build(); - onChanged(); - } else { - variantValueBuilder_.setMessage(builderForValue.build()); - } - kindCase_ = 11; - return this; - } - /** - * 
-       * Variant scalars carry a row-specific nested scalar.
-       * See RFC 0015: https://github.com/vortex-data/rfcs/blob/develop/accepted/0015-variant-type.md
-       *
- * - * .vortex.scalar.Scalar variant_value = 11; - */ - public Builder mergeVariantValue(io.github.dfa1.vortex.proto.ScalarProtos.Scalar value) { - if (variantValueBuilder_ == null) { - if (kindCase_ == 11 && - kind_ != io.github.dfa1.vortex.proto.ScalarProtos.Scalar.getDefaultInstance()) { - kind_ = io.github.dfa1.vortex.proto.ScalarProtos.Scalar.newBuilder((io.github.dfa1.vortex.proto.ScalarProtos.Scalar) kind_) - .mergeFrom(value).buildPartial(); - } else { - kind_ = value; - } - onChanged(); - } else { - if (kindCase_ == 11) { - variantValueBuilder_.mergeFrom(value); - } else { - variantValueBuilder_.setMessage(value); - } - } - kindCase_ = 11; - return this; - } - /** - * 
-       * Variant scalars carry a row-specific nested scalar.
-       * See RFC 0015: https://github.com/vortex-data/rfcs/blob/develop/accepted/0015-variant-type.md
-       *
- * - * .vortex.scalar.Scalar variant_value = 11; - */ - public Builder clearVariantValue() { - if (variantValueBuilder_ == null) { - if (kindCase_ == 11) { - kindCase_ = 0; - kind_ = null; - onChanged(); - } - } else { - if (kindCase_ == 11) { - kindCase_ = 0; - kind_ = null; - } - variantValueBuilder_.clear(); - } - return this; - } - /** - * 
-       * Variant scalars carry a row-specific nested scalar.
-       * See RFC 0015: https://github.com/vortex-data/rfcs/blob/develop/accepted/0015-variant-type.md
-       *
- * - * .vortex.scalar.Scalar variant_value = 11; - */ - public io.github.dfa1.vortex.proto.ScalarProtos.Scalar.Builder getVariantValueBuilder() { - return internalGetVariantValueFieldBuilder().getBuilder(); - } - /** - * 
-       * Variant scalars carry a row-specific nested scalar.
-       * See RFC 0015: https://github.com/vortex-data/rfcs/blob/develop/accepted/0015-variant-type.md
-       *
- * - * .vortex.scalar.Scalar variant_value = 11; - */ - @java.lang.Override - public io.github.dfa1.vortex.proto.ScalarProtos.ScalarOrBuilder getVariantValueOrBuilder() { - if ((kindCase_ == 11) && (variantValueBuilder_ != null)) { - return variantValueBuilder_.getMessageOrBuilder(); - } else { - if (kindCase_ == 11) { - return (io.github.dfa1.vortex.proto.ScalarProtos.Scalar) kind_; - } - return io.github.dfa1.vortex.proto.ScalarProtos.Scalar.getDefaultInstance(); - } - } - /** - * 
-       * Variant scalars carry a row-specific nested scalar.
-       * See RFC 0015: https://github.com/vortex-data/rfcs/blob/develop/accepted/0015-variant-type.md
-       *
- * - * .vortex.scalar.Scalar variant_value = 11; - */ - private com.google.protobuf.SingleFieldBuilder< - io.github.dfa1.vortex.proto.ScalarProtos.Scalar, io.github.dfa1.vortex.proto.ScalarProtos.Scalar.Builder, io.github.dfa1.vortex.proto.ScalarProtos.ScalarOrBuilder> - internalGetVariantValueFieldBuilder() { - if (variantValueBuilder_ == null) { - if (!(kindCase_ == 11)) { - kind_ = io.github.dfa1.vortex.proto.ScalarProtos.Scalar.getDefaultInstance(); - } - variantValueBuilder_ = new com.google.protobuf.SingleFieldBuilder< - io.github.dfa1.vortex.proto.ScalarProtos.Scalar, io.github.dfa1.vortex.proto.ScalarProtos.Scalar.Builder, io.github.dfa1.vortex.proto.ScalarProtos.ScalarOrBuilder>( - (io.github.dfa1.vortex.proto.ScalarProtos.Scalar) kind_, - getParentForChildren(), - isClean()); - kind_ = null; - } - kindCase_ = 11; - onChanged(); - return variantValueBuilder_; - } - - // @@protoc_insertion_point(builder_scope:vortex.scalar.ScalarValue) - } - - // @@protoc_insertion_point(class_scope:vortex.scalar.ScalarValue) - private static final io.github.dfa1.vortex.proto.ScalarProtos.ScalarValue DEFAULT_INSTANCE; - static { - DEFAULT_INSTANCE = new io.github.dfa1.vortex.proto.ScalarProtos.ScalarValue(); - } - - public static io.github.dfa1.vortex.proto.ScalarProtos.ScalarValue getDefaultInstance() { - return DEFAULT_INSTANCE; - } - - private static final com.google.protobuf.Parser - PARSER = new com.google.protobuf.AbstractParser() { - @java.lang.Override - public ScalarValue parsePartialFrom( - com.google.protobuf.CodedInputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws com.google.protobuf.InvalidProtocolBufferException { - Builder builder = newBuilder(); - try { - builder.mergeFrom(input, extensionRegistry); - } catch (com.google.protobuf.InvalidProtocolBufferException e) { - throw e.setUnfinishedMessage(builder.buildPartial()); - } catch (com.google.protobuf.UninitializedMessageException e) { - throw e.asInvalidProtocolBufferException().setUnfinishedMessage(builder.buildPartial()); - } catch (java.io.IOException e) { - throw new com.google.protobuf.InvalidProtocolBufferException(e) - .setUnfinishedMessage(builder.buildPartial()); - } - return builder.buildPartial(); - } - }; - - public static com.google.protobuf.Parser parser() { - return PARSER; - } - - @java.lang.Override - public com.google.protobuf.Parser getParserForType() { - return PARSER; - } - - @java.lang.Override - public io.github.dfa1.vortex.proto.ScalarProtos.ScalarValue getDefaultInstanceForType() { - return DEFAULT_INSTANCE; - } - - } - - public interface ListValueOrBuilder extends - // @@protoc_insertion_point(interface_extends:vortex.scalar.ListValue) - com.google.protobuf.MessageOrBuilder { - - /** - * repeated .vortex.scalar.ScalarValue values = 1; - */ - java.util.List - getValuesList(); - /** - * repeated .vortex.scalar.ScalarValue values = 1; - */ - io.github.dfa1.vortex.proto.ScalarProtos.ScalarValue getValues(int index); - /** - * repeated .vortex.scalar.ScalarValue values = 1; - */ - int getValuesCount(); - /** - * repeated .vortex.scalar.ScalarValue values = 1; - */ - java.util.List - getValuesOrBuilderList(); - /** - * repeated .vortex.scalar.ScalarValue values = 1; - */ - io.github.dfa1.vortex.proto.ScalarProtos.ScalarValueOrBuilder getValuesOrBuilder( - int index); - } - /** - * Protobuf type {@code vortex.scalar.ListValue} - */ - public static final class ListValue extends - com.google.protobuf.GeneratedMessage implements - // @@protoc_insertion_point(message_implements:vortex.scalar.ListValue) - ListValueOrBuilder { - private static final long serialVersionUID = 0L; - static { - com.google.protobuf.RuntimeVersion.validateProtobufGencodeVersion( - com.google.protobuf.RuntimeVersion.RuntimeDomain.PUBLIC, - /* major= */ 4, - /* minor= */ 35, - /* patch= */ 0, - /* suffix= */ "", - "ListValue"); - } - // Use ListValue.newBuilder() to construct. - private ListValue(com.google.protobuf.GeneratedMessage.Builder builder) { - super(builder); - } - private ListValue() { - values_ = java.util.Collections.emptyList(); - } - - public static final com.google.protobuf.Descriptors.Descriptor - getDescriptor() { - return io.github.dfa1.vortex.proto.ScalarProtos.internal_static_vortex_scalar_ListValue_descriptor; - } - - @java.lang.Override - public com.google.protobuf.Descriptors.Descriptor getDescriptorForType() { - return io.github.dfa1.vortex.proto.ScalarProtos.internal_static_vortex_scalar_ListValue_descriptor; - } - - @java.lang.Override - protected com.google.protobuf.GeneratedMessage.FieldAccessorTable - internalGetFieldAccessorTable() { - return io.github.dfa1.vortex.proto.ScalarProtos.internal_static_vortex_scalar_ListValue_fieldAccessorTable - .ensureFieldAccessorsInitialized( - io.github.dfa1.vortex.proto.ScalarProtos.ListValue.class, io.github.dfa1.vortex.proto.ScalarProtos.ListValue.Builder.class); - } - - public static final int VALUES_FIELD_NUMBER = 1; - @SuppressWarnings("serial") - private java.util.List values_; - /** - * repeated .vortex.scalar.ScalarValue values = 1; - */ - @java.lang.Override - public java.util.List getValuesList() { - return values_; - } - /** - * repeated .vortex.scalar.ScalarValue values = 1; - */ - @java.lang.Override - public java.util.List - getValuesOrBuilderList() { - return values_; - } - /** - * repeated .vortex.scalar.ScalarValue values = 1; - */ - @java.lang.Override - public int getValuesCount() { - return values_.size(); - } - /** - * repeated .vortex.scalar.ScalarValue values = 1; - */ - @java.lang.Override - public io.github.dfa1.vortex.proto.ScalarProtos.ScalarValue getValues(int index) { - return values_.get(index); - } - /** - * repeated .vortex.scalar.ScalarValue values = 1; - */ - @java.lang.Override - public io.github.dfa1.vortex.proto.ScalarProtos.ScalarValueOrBuilder getValuesOrBuilder( - int index) { - return values_.get(index); - } - - private byte memoizedIsInitialized = -1; - @java.lang.Override - public final boolean isInitialized() { - byte isInitialized = memoizedIsInitialized; - if (isInitialized == 1) return true; - if (isInitialized == 0) return false; - - memoizedIsInitialized = 1; - return true; - } - - @java.lang.Override - public void writeTo(com.google.protobuf.CodedOutputStream output) - throws java.io.IOException { - for (int i = 0; i < values_.size(); i++) { - output.writeMessage(1, values_.get(i)); - } - getUnknownFields().writeTo(output); - } - private int computeSerializedSize_0() { - int size = 0; - - { - final int count = values_.size(); - for (int i = 0; i < count; i++) { - size += com.google.protobuf.CodedOutputStream - .computeMessageSizeNoTag(values_.get(i)); - } - size += 1 * count; - } - return size; - } - @java.lang.Override - public int getSerializedSize() { - int size = memoizedSize; - if (size != -1) return size; - - size = 0; - size += computeSerializedSize_0(); - size += getUnknownFields().getSerializedSize(); - memoizedSize = size; - return size; - } - - @java.lang.Override - public boolean equals(final java.lang.Object obj) { - if (obj == this) { - return true; - } - if (!(obj instanceof io.github.dfa1.vortex.proto.ScalarProtos.ListValue)) { - return super.equals(obj); - } - io.github.dfa1.vortex.proto.ScalarProtos.ListValue other = (io.github.dfa1.vortex.proto.ScalarProtos.ListValue) obj; - - if (!getValuesList() - .equals(other.getValuesList())) return false; - if (!getUnknownFields().equals(other.getUnknownFields())) return false; - return true; - } - - @java.lang.Override - public int hashCode() { - if (memoizedHashCode != 0) { - return memoizedHashCode; - } - int hash = 41; - hash = (19 * hash) + getDescriptor().hashCode(); - if (getValuesCount() > 0) { - hash = (37 * hash) + VALUES_FIELD_NUMBER; - hash = (53 * hash) + getValuesList().hashCode(); - } - hash = (29 * hash) + getUnknownFields().hashCode(); - memoizedHashCode = hash; - return hash; - } - - public static io.github.dfa1.vortex.proto.ScalarProtos.ListValue parseFrom( - java.nio.ByteBuffer data) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data); - } - public static io.github.dfa1.vortex.proto.ScalarProtos.ListValue parseFrom( - java.nio.ByteBuffer data, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data, extensionRegistry); - } - public static io.github.dfa1.vortex.proto.ScalarProtos.ListValue parseFrom( - com.google.protobuf.ByteString data) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data); - } - public static io.github.dfa1.vortex.proto.ScalarProtos.ListValue parseFrom( - com.google.protobuf.ByteString data, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data, extensionRegistry); - } - public static io.github.dfa1.vortex.proto.ScalarProtos.ListValue parseFrom(byte[] data) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data); - } - public static io.github.dfa1.vortex.proto.ScalarProtos.ListValue parseFrom( - byte[] data, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws com.google.protobuf.InvalidProtocolBufferException { - return PARSER.parseFrom(data, extensionRegistry); - } - public static io.github.dfa1.vortex.proto.ScalarProtos.ListValue parseFrom(java.io.InputStream input) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseWithIOException(PARSER, input); - } - public static io.github.dfa1.vortex.proto.ScalarProtos.ListValue parseFrom( - java.io.InputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseWithIOException(PARSER, input, extensionRegistry); - } - - public static io.github.dfa1.vortex.proto.ScalarProtos.ListValue parseDelimitedFrom(java.io.InputStream input) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseDelimitedWithIOException(PARSER, input); - } - - public static io.github.dfa1.vortex.proto.ScalarProtos.ListValue parseDelimitedFrom( - java.io.InputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseDelimitedWithIOException(PARSER, input, extensionRegistry); - } - public static io.github.dfa1.vortex.proto.ScalarProtos.ListValue parseFrom( - com.google.protobuf.CodedInputStream input) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseWithIOException(PARSER, input); - } - public static io.github.dfa1.vortex.proto.ScalarProtos.ListValue parseFrom( - com.google.protobuf.CodedInputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws java.io.IOException { - return com.google.protobuf.GeneratedMessage - .parseWithIOException(PARSER, input, extensionRegistry); - } - - @java.lang.Override - public Builder newBuilderForType() { return newBuilder(); } - public static Builder newBuilder() { - return DEFAULT_INSTANCE.toBuilder(); - } - public static Builder newBuilder(io.github.dfa1.vortex.proto.ScalarProtos.ListValue prototype) { - return DEFAULT_INSTANCE.toBuilder().mergeFrom(prototype); - } - @java.lang.Override - public Builder toBuilder() { - return this == DEFAULT_INSTANCE - ? new Builder() : new Builder().mergeFrom(this); - } - - @java.lang.Override - protected Builder newBuilderForType( - com.google.protobuf.GeneratedMessage.BuilderParent parent) { - Builder builder = new Builder(parent); - return builder; - } - /** - * Protobuf type {@code vortex.scalar.ListValue} - */ - public static final class Builder extends - com.google.protobuf.GeneratedMessage.Builder implements - // @@protoc_insertion_point(builder_implements:vortex.scalar.ListValue) - io.github.dfa1.vortex.proto.ScalarProtos.ListValueOrBuilder { - public static final com.google.protobuf.Descriptors.Descriptor - getDescriptor() { - return io.github.dfa1.vortex.proto.ScalarProtos.internal_static_vortex_scalar_ListValue_descriptor; - } - - @java.lang.Override - protected com.google.protobuf.GeneratedMessage.FieldAccessorTable - internalGetFieldAccessorTable() { - return io.github.dfa1.vortex.proto.ScalarProtos.internal_static_vortex_scalar_ListValue_fieldAccessorTable - .ensureFieldAccessorsInitialized( - io.github.dfa1.vortex.proto.ScalarProtos.ListValue.class, io.github.dfa1.vortex.proto.ScalarProtos.ListValue.Builder.class); - } - - // Construct using io.github.dfa1.vortex.proto.ScalarProtos.ListValue.newBuilder() - private Builder() { - - } - - private Builder( - com.google.protobuf.GeneratedMessage.BuilderParent parent) { - super(parent); - - } - @java.lang.Override - public Builder clear() { - super.clear(); - bitField0_ = 0; - if (valuesBuilder_ == null) { - values_ = java.util.Collections.emptyList(); - } else { - values_ = null; - valuesBuilder_.clear(); - } - bitField0_ = (bitField0_ & ~0x00000001); - return this; - } - - @java.lang.Override - public com.google.protobuf.Descriptors.Descriptor - getDescriptorForType() { - return io.github.dfa1.vortex.proto.ScalarProtos.internal_static_vortex_scalar_ListValue_descriptor; - } - - @java.lang.Override - public io.github.dfa1.vortex.proto.ScalarProtos.ListValue getDefaultInstanceForType() { - return io.github.dfa1.vortex.proto.ScalarProtos.ListValue.getDefaultInstance(); - } - - @java.lang.Override - public io.github.dfa1.vortex.proto.ScalarProtos.ListValue build() { - io.github.dfa1.vortex.proto.ScalarProtos.ListValue result = buildPartial(); - if (!result.isInitialized()) { - throw newUninitializedMessageException(result); - } - return result; - } - - @java.lang.Override - public io.github.dfa1.vortex.proto.ScalarProtos.ListValue buildPartial() { - io.github.dfa1.vortex.proto.ScalarProtos.ListValue result = new io.github.dfa1.vortex.proto.ScalarProtos.ListValue(this); - buildPartialRepeatedFields(result); - if (bitField0_ != 0) { buildPartial0(result); } - onBuilt(); - return result; - } - - private void buildPartialRepeatedFields(io.github.dfa1.vortex.proto.ScalarProtos.ListValue result) { - if (valuesBuilder_ == null) { - if (((bitField0_ & 0x00000001) != 0)) { - values_ = java.util.Collections.unmodifiableList(values_); - bitField0_ = (bitField0_ & ~0x00000001); - } - result.values_ = values_; - } else { - result.values_ = valuesBuilder_.build(); - } - } - - private void buildPartial0(io.github.dfa1.vortex.proto.ScalarProtos.ListValue result) { - int from_bitField0_ = bitField0_; - } - - @java.lang.Override - public Builder mergeFrom(com.google.protobuf.Message other) { - if (other instanceof io.github.dfa1.vortex.proto.ScalarProtos.ListValue) { - return mergeFrom((io.github.dfa1.vortex.proto.ScalarProtos.ListValue)other); - } else { - super.mergeFrom(other); - return this; - } - } - - public Builder mergeFrom(io.github.dfa1.vortex.proto.ScalarProtos.ListValue other) { - if (other == io.github.dfa1.vortex.proto.ScalarProtos.ListValue.getDefaultInstance()) return this; - if (valuesBuilder_ == null) { - if (!other.values_.isEmpty()) { - if (values_.isEmpty()) { - values_ = other.values_; - bitField0_ = (bitField0_ & ~0x00000001); - } else { - ensureValuesIsMutable(); - values_.addAll(other.values_); - } - onChanged(); - } - } else { - if (!other.values_.isEmpty()) { - if (valuesBuilder_.isEmpty()) { - valuesBuilder_.dispose(); - valuesBuilder_ = null; - values_ = other.values_; - bitField0_ = (bitField0_ & ~0x00000001); - valuesBuilder_ = - com.google.protobuf.GeneratedMessage.alwaysUseFieldBuilders ? - internalGetValuesFieldBuilder() : null; - } else { - valuesBuilder_.addAllMessages(other.values_); - } - } - } - this.mergeUnknownFields(other.getUnknownFields()); - onChanged(); - return this; - } - - @java.lang.Override - public final boolean isInitialized() { - return true; - } - - @java.lang.Override - public Builder mergeFrom( - com.google.protobuf.CodedInputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws java.io.IOException { - if (extensionRegistry == null) { - throw new java.lang.NullPointerException(); - } - try { - boolean done = false; - while (!done) { - int tag = input.readTag(); - switch (tag) { - case 0: - done = true; - break; - case 10: { - io.github.dfa1.vortex.proto.ScalarProtos.ScalarValue m = - input.readMessage( - io.github.dfa1.vortex.proto.ScalarProtos.ScalarValue.parser(), - extensionRegistry); - if (valuesBuilder_ == null) { - ensureValuesIsMutable(); - values_.add(m); - } else { - valuesBuilder_.addMessage(m); - } - break; - } // case 10 - default: { - if (!super.parseUnknownField(input, extensionRegistry, tag)) { - done = true; // was an endgroup tag - } - break; - } // default: - } // switch (tag) - } // while (!done) - } catch (com.google.protobuf.InvalidProtocolBufferException e) { - throw e.unwrapIOException(); - } finally { - onChanged(); - } // finally - return this; - } - private int bitField0_; - - private java.util.List values_ = - java.util.Collections.emptyList(); - private void ensureValuesIsMutable() { - if (!((bitField0_ & 0x00000001) != 0)) { - values_ = new java.util.ArrayList(values_); - bitField0_ |= 0x00000001; - } - } - - private com.google.protobuf.RepeatedFieldBuilder< - io.github.dfa1.vortex.proto.ScalarProtos.ScalarValue, io.github.dfa1.vortex.proto.ScalarProtos.ScalarValue.Builder, io.github.dfa1.vortex.proto.ScalarProtos.ScalarValueOrBuilder> valuesBuilder_; - - /** - * repeated .vortex.scalar.ScalarValue values = 1; - */ - public java.util.List getValuesList() { - if (valuesBuilder_ == null) { - return java.util.Collections.unmodifiableList(values_); - } else { - return valuesBuilder_.getMessageList(); - } - } - /** - * repeated .vortex.scalar.ScalarValue values = 1; - */ - public int getValuesCount() { - if (valuesBuilder_ == null) { - return values_.size(); - } else { - return valuesBuilder_.getCount(); - } - } - /** - * repeated .vortex.scalar.ScalarValue values = 1; - */ - public io.github.dfa1.vortex.proto.ScalarProtos.ScalarValue getValues(int index) { - if (valuesBuilder_ == null) { - return values_.get(index); - } else { - return valuesBuilder_.getMessage(index); - } - } - /** - * repeated .vortex.scalar.ScalarValue values = 1; - */ - public Builder setValues( - int index, io.github.dfa1.vortex.proto.ScalarProtos.ScalarValue value) { - if (valuesBuilder_ == null) { - if (value == null) { - throw new NullPointerException(); - } - ensureValuesIsMutable(); - values_.set(index, value); - onChanged(); - } else { - valuesBuilder_.setMessage(index, value); - } - return this; - } - /** - * repeated .vortex.scalar.ScalarValue values = 1; - */ - public Builder setValues( - int index, io.github.dfa1.vortex.proto.ScalarProtos.ScalarValue.Builder builderForValue) { - if (valuesBuilder_ == null) { - ensureValuesIsMutable(); - values_.set(index, builderForValue.build()); - onChanged(); - } else { - valuesBuilder_.setMessage(index, builderForValue.build()); - } - return this; - } - /** - * repeated .vortex.scalar.ScalarValue values = 1; - */ - public Builder addValues(io.github.dfa1.vortex.proto.ScalarProtos.ScalarValue value) { - if (valuesBuilder_ == null) { - if (value == null) { - throw new NullPointerException(); - } - ensureValuesIsMutable(); - values_.add(value); - onChanged(); - } else { - valuesBuilder_.addMessage(value); - } - return this; - } - /** - * repeated .vortex.scalar.ScalarValue values = 1; - */ - public Builder addValues( - int index, io.github.dfa1.vortex.proto.ScalarProtos.ScalarValue value) { - if (valuesBuilder_ == null) { - if (value == null) { - throw new NullPointerException(); - } - ensureValuesIsMutable(); - values_.add(index, value); - onChanged(); - } else { - valuesBuilder_.addMessage(index, value); - } - return this; - } - /** - * repeated .vortex.scalar.ScalarValue values = 1; - */ - public Builder addValues( - io.github.dfa1.vortex.proto.ScalarProtos.ScalarValue.Builder builderForValue) { - if (valuesBuilder_ == null) { - ensureValuesIsMutable(); - values_.add(builderForValue.build()); - onChanged(); - } else { - valuesBuilder_.addMessage(builderForValue.build()); - } - return this; - } - /** - * repeated .vortex.scalar.ScalarValue values = 1; - */ - public Builder addValues( - int index, io.github.dfa1.vortex.proto.ScalarProtos.ScalarValue.Builder builderForValue) { - if (valuesBuilder_ == null) { - ensureValuesIsMutable(); - values_.add(index, builderForValue.build()); - onChanged(); - } else { - valuesBuilder_.addMessage(index, builderForValue.build()); - } - return this; - } - /** - * repeated .vortex.scalar.ScalarValue values = 1; - */ - public Builder addAllValues( - java.lang.Iterable values) { - if (valuesBuilder_ == null) { - ensureValuesIsMutable(); - com.google.protobuf.AbstractMessageLite.Builder.addAll( - values, values_); - onChanged(); - } else { - valuesBuilder_.addAllMessages(values); - } - return this; - } - /** - * repeated .vortex.scalar.ScalarValue values = 1; - */ - public Builder clearValues() { - if (valuesBuilder_ == null) { - values_ = java.util.Collections.emptyList(); - bitField0_ = (bitField0_ & ~0x00000001); - onChanged(); - } else { - valuesBuilder_.clear(); - } - return this; - } - /** - * repeated .vortex.scalar.ScalarValue values = 1; - */ - public Builder removeValues(int index) { - if (valuesBuilder_ == null) { - ensureValuesIsMutable(); - values_.remove(index); - onChanged(); - } else { - valuesBuilder_.remove(index); - } - return this; - } - /** - * repeated .vortex.scalar.ScalarValue values = 1; - */ - public io.github.dfa1.vortex.proto.ScalarProtos.ScalarValue.Builder getValuesBuilder( - int index) { - return internalGetValuesFieldBuilder().getBuilder(index); - } - /** - * repeated .vortex.scalar.ScalarValue values = 1; - */ - public io.github.dfa1.vortex.proto.ScalarProtos.ScalarValueOrBuilder getValuesOrBuilder( - int index) { - if (valuesBuilder_ == null) { - return values_.get(index); } else { - return valuesBuilder_.getMessageOrBuilder(index); - } - } - /** - * repeated .vortex.scalar.ScalarValue values = 1; - */ - public java.util.List - getValuesOrBuilderList() { - if (valuesBuilder_ != null) { - return valuesBuilder_.getMessageOrBuilderList(); - } else { - return java.util.Collections.unmodifiableList(values_); - } - } - /** - * repeated .vortex.scalar.ScalarValue values = 1; - */ - public io.github.dfa1.vortex.proto.ScalarProtos.ScalarValue.Builder addValuesBuilder() { - return internalGetValuesFieldBuilder().addBuilder( - io.github.dfa1.vortex.proto.ScalarProtos.ScalarValue.getDefaultInstance()); - } - /** - * repeated .vortex.scalar.ScalarValue values = 1; - */ - public io.github.dfa1.vortex.proto.ScalarProtos.ScalarValue.Builder addValuesBuilder( - int index) { - return internalGetValuesFieldBuilder().addBuilder( - index, io.github.dfa1.vortex.proto.ScalarProtos.ScalarValue.getDefaultInstance()); - } - /** - * repeated .vortex.scalar.ScalarValue values = 1; - */ - public java.util.List - getValuesBuilderList() { - return internalGetValuesFieldBuilder().getBuilderList(); - } - private com.google.protobuf.RepeatedFieldBuilder< - io.github.dfa1.vortex.proto.ScalarProtos.ScalarValue, io.github.dfa1.vortex.proto.ScalarProtos.ScalarValue.Builder, io.github.dfa1.vortex.proto.ScalarProtos.ScalarValueOrBuilder> - internalGetValuesFieldBuilder() { - if (valuesBuilder_ == null) { - valuesBuilder_ = new com.google.protobuf.RepeatedFieldBuilder< - io.github.dfa1.vortex.proto.ScalarProtos.ScalarValue, io.github.dfa1.vortex.proto.ScalarProtos.ScalarValue.Builder, io.github.dfa1.vortex.proto.ScalarProtos.ScalarValueOrBuilder>( - values_, - ((bitField0_ & 0x00000001) != 0), - getParentForChildren(), - isClean()); - values_ = null; - } - return valuesBuilder_; - } - - // @@protoc_insertion_point(builder_scope:vortex.scalar.ListValue) - } - - // @@protoc_insertion_point(class_scope:vortex.scalar.ListValue) - private static final io.github.dfa1.vortex.proto.ScalarProtos.ListValue DEFAULT_INSTANCE; - static { - DEFAULT_INSTANCE = new io.github.dfa1.vortex.proto.ScalarProtos.ListValue(); - } - - public static io.github.dfa1.vortex.proto.ScalarProtos.ListValue getDefaultInstance() { - return DEFAULT_INSTANCE; - } - - private static final com.google.protobuf.Parser - PARSER = new com.google.protobuf.AbstractParser() { - @java.lang.Override - public ListValue parsePartialFrom( - com.google.protobuf.CodedInputStream input, - com.google.protobuf.ExtensionRegistryLite extensionRegistry) - throws com.google.protobuf.InvalidProtocolBufferException { - Builder builder = newBuilder(); - try { - builder.mergeFrom(input, extensionRegistry); - } catch (com.google.protobuf.InvalidProtocolBufferException e) { - throw e.setUnfinishedMessage(builder.buildPartial()); - } catch (com.google.protobuf.UninitializedMessageException e) { - throw e.asInvalidProtocolBufferException().setUnfinishedMessage(builder.buildPartial()); - } catch (java.io.IOException e) { - throw new com.google.protobuf.InvalidProtocolBufferException(e) - .setUnfinishedMessage(builder.buildPartial()); - } - return builder.buildPartial(); - } - }; - - public static com.google.protobuf.Parser parser() { - return PARSER; - } - - @java.lang.Override - public com.google.protobuf.Parser getParserForType() { - return PARSER; - } - - @java.lang.Override - public io.github.dfa1.vortex.proto.ScalarProtos.ListValue getDefaultInstanceForType() { - return DEFAULT_INSTANCE; - } - - } - - private static final com.google.protobuf.Descriptors.Descriptor - internal_static_vortex_scalar_Scalar_descriptor; - private static final - com.google.protobuf.GeneratedMessage.FieldAccessorTable - internal_static_vortex_scalar_Scalar_fieldAccessorTable; - private static final com.google.protobuf.Descriptors.Descriptor - internal_static_vortex_scalar_ScalarValue_descriptor; - private static final - com.google.protobuf.GeneratedMessage.FieldAccessorTable - internal_static_vortex_scalar_ScalarValue_fieldAccessorTable; - private static final com.google.protobuf.Descriptors.Descriptor - internal_static_vortex_scalar_ListValue_descriptor; - private static final - com.google.protobuf.GeneratedMessage.FieldAccessorTable - internal_static_vortex_scalar_ListValue_fieldAccessorTable; - - public static com.google.protobuf.Descriptors.FileDescriptor - getDescriptor() { - return descriptor; - } - private static final com.google.protobuf.Descriptors.FileDescriptor - descriptor; - static { - java.lang.String[] descriptorData = { - "\n\014scalar.proto\022\rvortex.scalar\032\013dtype.pro" + - "to\032\034google/protobuf/struct.proto\"W\n\006Scal" + - "ar\022\"\n\005dtype\030\001 \001(\0132\023.vortex.dtype.DType\022)" + - "\n\005value\030\002 \001(\0132\032.vortex.scalar.ScalarValu" + - "e\"\332\002\n\013ScalarValue\0220\n\nnull_value\030\001 \001(\0162\032." + - "google.protobuf.NullValueH\000\022\024\n\nbool_valu" + - "e\030\002 \001(\010H\000\022\025\n\013int64_value\030\003 \001(\022H\000\022\026\n\014uint" + - "64_value\030\004 \001(\004H\000\022\023\n\tf32_value\030\005 \001(\002H\000\022\023\n" + - "\tf64_value\030\006 \001(\001H\000\022\026\n\014string_value\030\007 \001(\t" + - "H\000\022\025\n\013bytes_value\030\010 \001(\014H\000\022.\n\nlist_value\030" + - "\t \001(\0132\030.vortex.scalar.ListValueH\000\022\023\n\tf16" + - "_value\030\n \001(\004H\000\022.\n\rvariant_value\030\013 \001(\0132\025." + - "vortex.scalar.ScalarH\000B\006\n\004kind\"7\n\tListVa" + - "lue\022*\n\006values\030\001 \003(\0132\032.vortex.scalar.Scal" + - "arValueB+\n\033io.github.dfa1.vortex.protoB\014" + - "ScalarProtosb\006proto3" - }; - descriptor = com.google.protobuf.Descriptors.FileDescriptor - .internalBuildGeneratedFileFrom(descriptorData, - new com.google.protobuf.Descriptors.FileDescriptor[] { - io.github.dfa1.vortex.proto.DTypeProtos.getDescriptor(), - com.google.protobuf.StructProto.getDescriptor(), - }); - internal_static_vortex_scalar_Scalar_descriptor = - getDescriptor().getMessageType(0); - internal_static_vortex_scalar_Scalar_fieldAccessorTable = new - com.google.protobuf.GeneratedMessage.FieldAccessorTable( - internal_static_vortex_scalar_Scalar_descriptor, - new java.lang.String[] { "Dtype", "Value", }); - internal_static_vortex_scalar_ScalarValue_descriptor = - getDescriptor().getMessageType(1); - internal_static_vortex_scalar_ScalarValue_fieldAccessorTable = new - com.google.protobuf.GeneratedMessage.FieldAccessorTable( - internal_static_vortex_scalar_ScalarValue_descriptor, - new java.lang.String[] { "NullValue", "BoolValue", "Int64Value", "Uint64Value", "F32Value", "F64Value", "StringValue", "BytesValue", "ListValue", "F16Value", "VariantValue", "Kind", }); - internal_static_vortex_scalar_ListValue_descriptor = - getDescriptor().getMessageType(2); - internal_static_vortex_scalar_ListValue_fieldAccessorTable = new - com.google.protobuf.GeneratedMessage.FieldAccessorTable( - internal_static_vortex_scalar_ListValue_descriptor, - new java.lang.String[] { "Values", }); - descriptor.resolveAllFeaturesImmutable(); - io.github.dfa1.vortex.proto.DTypeProtos.getDescriptor(); - com.google.protobuf.StructProto.getDescriptor(); - } - - // @@protoc_insertion_point(outer_class_scope) -} diff --git a/core/src/main/proto/dtype.proto b/core/src/main/proto/dtype.proto deleted file mode 100644 index 2017c9f..0000000 --- a/core/src/main/proto/dtype.proto +++ /dev/null @@ -1,106 +0,0 @@ -// SPDX-License-Identifier: Apache-2.0 -// SPDX-FileCopyrightText: Copyright the Vortex contributors - -syntax = "proto3"; - -package vortex.dtype; - -option java_package = "io.github.dfa1.vortex.proto"; -option java_outer_classname = "DTypeProtos"; - -enum PType { - U8 = 0; - U16 = 1; - U32 = 2; - U64 = 3; - I8 = 4; - I16 = 5; - I32 = 6; - I64 = 7; - F16 = 8; - F32 = 9; - F64 = 10; -} - -message Null {} - -message Bool { - bool nullable = 1; -} - -message Primitive { - PType type = 1; - bool nullable = 2; -} - -message Decimal { - uint32 precision = 1; - int32 scale = 2; - bool nullable = 3; -} - -message Utf8 { - bool nullable = 1; -} - -message Binary { - bool nullable = 1; -} - -message Struct { - repeated string names = 1; - repeated DType dtypes = 2; - bool nullable = 3; -} - -message List { - DType element_type = 1; - bool nullable = 2; -} - -message FixedSizeList { - DType element_type = 1; - uint32 size = 2; - bool nullable = 3; -} - -message Extension { - string id = 1; - DType storage_dtype = 2; - optional bytes metadata = 3; -} - -message Variant { - bool nullable = 1; -} - -message Union { - bool nullable = 4; -} - -message DType { - oneof dtype_type { - Null null = 1; - Bool bool = 2; - Primitive primitive = 3; - Decimal decimal = 4; - Utf8 utf8 = 5; - Binary binary = 6; - Struct struct = 7; - List list = 8; - Extension extension = 9; - FixedSizeList fixed_size_list = 10; // This is after `Extension` for backwards compatibility. - Variant variant = 11; - Union union = 12; - } -} - -message Field { - oneof field_type { - string name = 1; - } -} - -message FieldPath { - repeated Field path = 1; -} diff --git a/core/src/main/proto/encodings.proto b/core/src/main/proto/encodings.proto deleted file mode 100644 index cd5f2c3..0000000 --- a/core/src/main/proto/encodings.proto +++ /dev/null @@ -1,134 +0,0 @@ -// SPDX-License-Identifier: Apache-2.0 -syntax = "proto3"; - -package vortex.encodings; - -option java_package = "io.github.dfa1.vortex.proto"; -option java_outer_classname = "EncodingProtos"; - -import "scalar.proto"; -import "dtype.proto"; - -message PatchesMetadata { - uint64 len = 1; - uint64 offset = 2; - vortex.dtype.PType indices_ptype = 3; - optional uint64 chunk_offsets_len = 4; - optional vortex.dtype.PType chunk_offsets_ptype = 5; - optional uint64 offset_within_chunk = 6; -} - -message SparseMetadata { - PatchesMetadata patches = 1; -} - -message DictMetadata { - uint32 values_len = 1; - vortex.dtype.PType codes_ptype = 2; - optional bool is_nullable_codes = 3; - optional bool all_values_referenced = 4; -} - -message RunEndMetadata { - vortex.dtype.PType ends_ptype = 1; - uint64 num_runs = 2; - uint64 offset = 3; -} - -message ALPMetadata { - uint32 exp_e = 1; - uint32 exp_f = 2; - optional PatchesMetadata patches = 3; -} - -message BitPackedMetadata { - uint32 bit_width = 1; - uint32 offset = 2; - optional PatchesMetadata patches = 3; -} - -message SequenceMetadata { - vortex.scalar.ScalarValue base = 1; - vortex.scalar.ScalarValue multiplier = 2; -} - -message VarBinMetadata { - vortex.dtype.PType offsets_ptype = 1; -} - -message FSSTMetadata { - vortex.dtype.PType uncompressed_lengths_ptype = 1; - vortex.dtype.PType codes_offsets_ptype = 2; -} - -message DeltaMetadata { - uint64 deltas_len = 1; - uint32 offset = 2; -} - -message DecimalMetadata { - // DecimalType: I8=0, I16=1, I32=2, I64=3, I128=4, I256=5 - int32 values_type = 1; -} - -message DecimalBytePartsMetadata { - vortex.dtype.PType zeroth_child_ptype = 1; - uint32 lower_part_count = 2; -} - -message DateTimePartsMetadata { - vortex.dtype.PType days_ptype = 1; - vortex.dtype.PType seconds_ptype = 2; - vortex.dtype.PType subseconds_ptype = 3; -} - -message ZstdFrameMetadata { - uint64 uncompressed_size = 1; - uint64 n_values = 2; -} - -message ZstdMetadata { - uint32 dictionary_size = 1; - repeated ZstdFrameMetadata frames = 2; -} - -message RLEMetadata { - uint64 values_len = 1; - uint64 indices_len = 2; - vortex.dtype.PType indices_ptype = 3; - uint64 values_idx_offsets_len = 4; - vortex.dtype.PType values_idx_offsets_ptype = 5; - uint64 offset = 6; -} - -message ListMetadata { - uint64 elements_len = 1; - vortex.dtype.PType offset_ptype = 2; -} - -message ListViewMetadata { - uint64 elements_len = 1; - vortex.dtype.PType offset_ptype = 2; - vortex.dtype.PType size_ptype = 3; -} - -message ALPRDMetadata { - uint32 right_bit_width = 1; - uint32 dict_len = 2; - repeated uint32 dict = 3; - vortex.dtype.PType left_parts_ptype = 4; - optional PatchesMetadata patches = 5; -} - -message PcoPageInfo { - uint32 n_values = 1; -} - -message PcoChunkInfo { - repeated PcoPageInfo pages = 1; -} - -message PcoMetadata { - bytes header = 1; - repeated PcoChunkInfo chunks = 2; -} diff --git a/core/src/main/proto/scalar.proto b/core/src/main/proto/scalar.proto deleted file mode 100644 index 2d3924e..0000000 --- a/core/src/main/proto/scalar.proto +++ /dev/null @@ -1,39 +0,0 @@ -// SPDX-License-Identifier: Apache-2.0 -// SPDX-FileCopyrightText: Copyright the Vortex contributors - -syntax = "proto3"; - -package vortex.scalar; - -option java_package = "io.github.dfa1.vortex.proto"; -option java_outer_classname = "ScalarProtos"; - -import "dtype.proto"; -import "google/protobuf/struct.proto"; - -message Scalar { - vortex.dtype.DType dtype = 1; - ScalarValue value = 2; -} - -message ScalarValue { - oneof kind { - google.protobuf.NullValue null_value = 1; - bool bool_value = 2; - sint64 int64_value = 3; - uint64 uint64_value = 4; - float f32_value = 5; - double f64_value = 6; - string string_value = 7; - bytes bytes_value = 8; - ListValue list_value = 9; - uint64 f16_value = 10; - // Variant scalars carry a row-specific nested scalar. - // See RFC 0015: https://github.com/vortex-data/rfcs/blob/develop/accepted/0015-variant-type.md - Scalar variant_value = 11; - } -} - -message ListValue { - repeated ScalarValue values = 1; -} diff --git a/core/src/main/resources/META-INF/services/io.github.dfa1.vortex.encoding.Encoding b/core/src/main/resources/META-INF/services/io.github.dfa1.vortex.encoding.Encoding deleted file mode 100644 index 54fa5bd..0000000 --- a/core/src/main/resources/META-INF/services/io.github.dfa1.vortex.encoding.Encoding +++ /dev/null @@ -1,33 +0,0 @@ -io.github.dfa1.vortex.encoding.AlpEncoding -io.github.dfa1.vortex.encoding.AlpRdEncoding -io.github.dfa1.vortex.encoding.BitpackedEncoding -io.github.dfa1.vortex.encoding.BoolEncoding -io.github.dfa1.vortex.encoding.ByteBoolEncoding -io.github.dfa1.vortex.encoding.ChunkedEncoding -io.github.dfa1.vortex.encoding.ConstantEncoding -io.github.dfa1.vortex.encoding.DateTimePartsEncoding -io.github.dfa1.vortex.encoding.DecimalBytePartsEncoding -io.github.dfa1.vortex.encoding.DecimalEncoding -io.github.dfa1.vortex.encoding.DeltaEncoding -io.github.dfa1.vortex.encoding.DictEncoding -io.github.dfa1.vortex.encoding.ExtEncoding -io.github.dfa1.vortex.encoding.FixedSizeListEncoding -io.github.dfa1.vortex.encoding.FrameOfReferenceEncoding -io.github.dfa1.vortex.encoding.FsstEncoding -io.github.dfa1.vortex.encoding.ListEncoding -io.github.dfa1.vortex.encoding.ListViewEncoding -io.github.dfa1.vortex.encoding.NullEncoding -io.github.dfa1.vortex.encoding.PcoEncoding -io.github.dfa1.vortex.encoding.PrimitiveEncoding -io.github.dfa1.vortex.encoding.RleEncoding -io.github.dfa1.vortex.encoding.RunEndEncoding -io.github.dfa1.vortex.encoding.SequenceEncoding -io.github.dfa1.vortex.encoding.SparseEncoding -io.github.dfa1.vortex.encoding.StructEncoding -io.github.dfa1.vortex.encoding.VarBinEncoding -io.github.dfa1.vortex.encoding.VarBinViewEncoding -io.github.dfa1.vortex.encoding.ZigZagEncoding -io.github.dfa1.vortex.encoding.ZstdEncoding -io.github.dfa1.vortex.encoding.MaskedEncoding -io.github.dfa1.vortex.encoding.PatchedEncoding -io.github.dfa1.vortex.encoding.VariantEncoding diff --git a/core/src/test/java/io/github/dfa1/vortex/core/PTypeTest.java b/core/src/test/java/io/github/dfa1/vortex/core/PTypeTest.java deleted file mode 100644 index 741ccfc..0000000 --- a/core/src/test/java/io/github/dfa1/vortex/core/PTypeTest.java +++ /dev/null @@ -1,50 +0,0 @@ -package io.github.dfa1.vortex.core; - -import org.junit.jupiter.params.ParameterizedTest; -import org.junit.jupiter.params.provider.CsvSource; -import org.junit.jupiter.params.provider.EnumSource; - -import static org.assertj.core.api.Assertions.assertThat; - -class PTypeTest { - - @ParameterizedTest - @CsvSource({ - "U8, 1", "I8, 1", - "U16, 2", "I16, 2", "F16, 2", - "U32, 4", "I32, 4", "F32, 4", - "U64, 8", "I64, 8", "F64, 8", - }) - void byteSize(PType ptype, int expected) { - // Given / When / Then - assertThat(ptype.byteSize()).isEqualTo(expected); - } - - @ParameterizedTest - @EnumSource(value = PType.class, names = {"F16", "F32", "F64"}) - void isFloating_trueForFloats(PType ptype) { - // Given / When / Then - assertThat(ptype.isFloating()).isTrue(); - } - - @ParameterizedTest - @EnumSource(value = PType.class, names = {"U8", "U16", "U32", "U64", "I8", "I16", "I32", "I64"}) - void isFloating_falseForIntegers(PType ptype) { - // Given / When / Then - assertThat(ptype.isFloating()).isFalse(); - } - - @ParameterizedTest - @EnumSource(value = PType.class, names = {"I8", "I16", "I32", "I64", "F16", "F32", "F64"}) - void isSigned_trueForSignedAndFloats(PType ptype) { - // Given / When / Then - assertThat(ptype.isSigned()).isTrue(); - } - - @ParameterizedTest - @EnumSource(value = PType.class, names = {"U8", "U16", "U32", "U64"}) - void isSigned_falseForUnsigned(PType ptype) { - // Given / When / Then - assertThat(ptype.isSigned()).isFalse(); - } -} diff --git a/core/src/test/java/io/github/dfa1/vortex/core/array/LongArrayTest.java b/core/src/test/java/io/github/dfa1/vortex/core/array/LongArrayTest.java deleted file mode 100644 index fd0fbb2..0000000 --- a/core/src/test/java/io/github/dfa1/vortex/core/array/LongArrayTest.java +++ /dev/null @@ -1,76 +0,0 @@ -package io.github.dfa1.vortex.core.array; - -import io.github.dfa1.vortex.core.ArrayStats; -import io.github.dfa1.vortex.core.DType; -import io.github.dfa1.vortex.core.PType; -import org.junit.jupiter.api.Test; - -import java.lang.foreign.Arena; -import java.lang.foreign.MemorySegment; -import java.lang.foreign.ValueLayout; -import java.nio.ByteOrder; - -import static org.assertj.core.api.Assertions.assertThat; - -class LongArrayTest { - - private static final ValueLayout.OfLong LE_LONG = - ValueLayout.JAVA_LONG_UNALIGNED.withOrder(ByteOrder.LITTLE_ENDIAN); - - private static LongArray of(long... values) { - MemorySegment seg = Arena.ofAuto().allocate((long) values.length * 8, 8); - for (int i = 0; i < values.length; i++) { - seg.setAtIndex(LE_LONG, i, values[i]); - } - DType dtype = new DType.Primitive(PType.I64, false); - return new LongArray(dtype, values.length, seg, ArrayStats.empty()); - } - - @Test - void fold_sum_returnsCorrectTotal() { - // Given - LongArray sut = of(1L, 2L, 3L, 4L, 5L); - - // When - long result = sut.fold(0L, Long::sum); - - // Then - assertThat(result).isEqualTo(15L); - } - - @Test - void fold_max_returnsLargestValue() { - // Given - LongArray sut = of(3L, 1L, 4L, 1L, 5L, 9L, 2L); - - // When - long result = sut.fold(Long.MIN_VALUE, Math::max); - - // Then - assertThat(result).isEqualTo(9L); - } - - @Test - void fold_min_returnsSmallestValue() { - // Given - LongArray sut = of(3L, 1L, 4L, 1L, 5L, 9L, 2L); - - // When - long result = sut.fold(Long.MAX_VALUE, Math::min); - - // Then - assertThat(result).isEqualTo(1L); - } - - @Test - void fold_emptyArray_returnsIdentity() { - // Given - LongArray sut = of(); - - // When - long result = sut.fold(42L, Long::sum); - - // Then - assertThat(result).isEqualTo(42L); - } -} diff --git a/core/src/test/java/io/github/dfa1/vortex/core/array/VarBinArrayTest.java b/core/src/test/java/io/github/dfa1/vortex/core/array/VarBinArrayTest.java deleted file mode 100644 index a41e8cc..0000000 --- a/core/src/test/java/io/github/dfa1/vortex/core/array/VarBinArrayTest.java +++ /dev/null @@ -1,210 +0,0 @@ -package io.github.dfa1.vortex.core.array; - -import io.github.dfa1.vortex.core.ArrayStats; -import io.github.dfa1.vortex.core.DType; -import io.github.dfa1.vortex.core.PType; -import org.junit.jupiter.api.Nested; -import org.junit.jupiter.api.Test; - -import java.lang.foreign.MemorySegment; -import java.nio.ByteBuffer; -import java.nio.ByteOrder; -import java.nio.charset.StandardCharsets; -import java.util.ArrayList; -import java.util.List; - -import static org.assertj.core.api.Assertions.assertThat; -import static org.assertj.core.api.Assertions.assertThatThrownBy; - -class VarBinArrayTest { - - private static final DType UTF8 = new DType.Utf8(false); - - @Nested - class Regular { - - @Test - void length_returnsElementCount() { - // Given - VarBinArray sut = of("a", "bb", "ccc"); - - // When / Then - assertThat(sut.length()).isEqualTo(3L); - } - - @Test - void dtype_returnsConstructedDtype() { - // Given - VarBinArray sut = of("x"); - - // When / Then - assertThat(sut.dtype()).isEqualTo(UTF8); - } - - @Test - void getBytes_returnsCorrectBytes() { - // Given - VarBinArray sut = of("hello", "world"); - - // When / Then - assertThat(sut.getString(0)).isEqualTo("hello"); - assertThat(sut.getString(1)).isEqualTo("world"); - } - - @Test - void getByteLength_returnsCorrectLengths() { - // Given - VarBinArray sut = of("hi", "there", "!"); - - // When / Then - assertThat(sut.getByteLength(0)).isEqualTo(2); - assertThat(sut.getByteLength(1)).isEqualTo(5); - assertThat(sut.getByteLength(2)).isEqualTo(1); - } - - @Test - void forEachByteLength_visitsAllLengths() { - // Given - VarBinArray sut = of("ab", "c", "def"); - List lengths = new ArrayList<>(); - - // When - sut.forEachByteLength(lengths::add); - - // Then - assertThat(lengths).containsExactly(2, 1, 3); - } - - @Test - void getBytes_emptyString_returnsEmptyArray() { - // Given - VarBinArray sut = of("", "x", ""); - - // When / Then - assertThat(sut.getBytes(0)).isEmpty(); - assertThat(sut.getByteLength(0)).isZero(); - assertThat(sut.getByteLength(2)).isZero(); - } - - @Test - void empty_zeroElements() { - // Given - VarBinArray sut = of(); - - // When / Then - assertThat(sut.length()).isZero(); - } - - @Test - void buffer_invalidIndex_throws() { - // Given - VarBinArray sut = of("a"); - - // When / Then - assertThatThrownBy(() -> sut.buffer(1)) - .isInstanceOf(IndexOutOfBoundsException.class); - } - - @Test - void child_invalidIndex_throws() { - // Given - VarBinArray sut = of("a"); - - // When / Then - assertThatThrownBy(() -> sut.child(1)) - .isInstanceOf(IndexOutOfBoundsException.class); - } - } - - @Nested - class Dict { - - @Test - void getBytes_resolvesViaDict() { - // Given — dict=["foo","bar"], codes=[1,0,1] - VarBinArray sut = ofDict(new String[]{"foo", "bar"}, new int[]{1, 0, 1}); - - // When / Then - assertThat(sut.getString(0)).isEqualTo("bar"); - assertThat(sut.getString(1)).isEqualTo("foo"); - assertThat(sut.getString(2)).isEqualTo("bar"); - } - - @Test - void getByteLength_resolvesViaDict() { - // Given — dict=["hi","there"], codes=[0,1,0] - VarBinArray sut = ofDict(new String[]{"hi", "there"}, new int[]{0, 1, 0}); - - // When / Then - assertThat(sut.getByteLength(0)).isEqualTo(2); - assertThat(sut.getByteLength(1)).isEqualTo(5); - assertThat(sut.getByteLength(2)).isEqualTo(2); - } - - @Test - void forEachByteLength_resolvesViaDict() { - // Given - VarBinArray sut = ofDict(new String[]{"a", "bbb"}, new int[]{1, 0, 1}); - List lengths = new ArrayList<>(); - - // When - sut.forEachByteLength(lengths::add); - - // Then - assertThat(lengths).containsExactly(3, 1, 3); - } - - @Test - void child_inDictMode_throws() { - // Given - VarBinArray sut = ofDict(new String[]{"x"}, new int[]{0}); - - // When / Then - assertThatThrownBy(() -> sut.child(0)) - .isInstanceOf(IllegalStateException.class); - } - - private static VarBinArray ofDict(String[] dictValues, int[] codes) { - byte[] allBytes = String.join("", dictValues).getBytes(StandardCharsets.UTF_8); - MemorySegment dictValBytes = MemorySegment.ofArray(allBytes); - - int[] offs = new int[dictValues.length + 1]; - for (int i = 0; i < dictValues.length; i++) { - offs[i + 1] = offs[i] + dictValues[i].getBytes(StandardCharsets.UTF_8).length; - } - MemorySegment dictValOffsets = leInts(offs); - - MemorySegment dictCodes = leInts(codes); - - return VarBinArray.ofDict(UTF8, codes.length, - dictValBytes, dictValOffsets, PType.I32, - dictCodes, PType.I32, ArrayStats.empty()); - } - - private static MemorySegment leInts(int[] values) { - ByteBuffer bb = ByteBuffer.allocate(values.length * 4).order(ByteOrder.LITTLE_ENDIAN); - for (int v : values) { - bb.putInt(v); - } - return MemorySegment.ofArray(bb.array()); - } - } - - private static VarBinArray of(String... values) { - byte[] allBytes = String.join("", values).getBytes(StandardCharsets.UTF_8); - MemorySegment bytes = MemorySegment.ofArray(allBytes); - - int[] offs = new int[values.length + 1]; - for (int i = 0; i < values.length; i++) { - offs[i + 1] = offs[i] + values[i].getBytes(StandardCharsets.UTF_8).length; - } - ByteBuffer bb = ByteBuffer.allocate(offs.length * 4).order(ByteOrder.LITTLE_ENDIAN); - for (int o : offs) { - bb.putInt(o); - } - MemorySegment offsetsSeg = MemorySegment.ofArray(bb.array()); - DType i32 = new DType.Primitive(PType.I32, false); - IntArray offsetsArr = new IntArray(i32, offs.length, offsetsSeg, ArrayStats.empty()); - return new VarBinArray(UTF8, values.length, bytes, offsetsArr, PType.I32, ArrayStats.empty()); - } -} diff --git a/core/src/test/java/io/github/dfa1/vortex/encoding/AlpEncodingTest.java b/core/src/test/java/io/github/dfa1/vortex/encoding/AlpEncodingTest.java deleted file mode 100644 index 59a3124..0000000 --- a/core/src/test/java/io/github/dfa1/vortex/encoding/AlpEncodingTest.java +++ /dev/null @@ -1,283 +0,0 @@ -package io.github.dfa1.vortex.encoding; - -import io.github.dfa1.vortex.proto.EncodingProtos; -import io.github.dfa1.vortex.core.array.Array; -import io.github.dfa1.vortex.core.ArrayStats; -import org.junit.jupiter.api.Nested; -import org.junit.jupiter.api.Test; -import org.junit.jupiter.params.ParameterizedTest; -import org.junit.jupiter.params.provider.CsvSource; - -import java.lang.foreign.MemorySegment; -import java.nio.ByteBuffer; -import java.nio.ByteOrder; - -import static org.assertj.core.api.Assertions.assertThat; -import static org.assertj.core.api.Assertions.within; - -class AlpEncodingTest { - - - @Nested - class Decode { - - @Test - void decode_f64_noPatches() { - // Given — encode 1.23 with exp_e=2, exp_f=1: encoded = round(1.23 * 100 / 10) = 12 - // decode: 12 * 10^1 * 10^-2 = 12 * 10 * 0.01 = 1.2 - // Use exp_e=0, exp_f=2: encoded = round(1.23 * 1 / 100) ... let's use known values - // encode: value * F10[e] * IF10[f] then round; decode: encoded * F10[f] * IF10[e] - // Use e=2, f=0: encoded = round(1.23 * 100 * 1.0) = 123; decode = 123 * 1.0 * 0.01 = 1.23 - int expE = 2, expF = 0; - long[] encoded = {123L, 456L, 789L}; - double[] expected = {1.23, 4.56, 7.89}; - - DecodeContext ctx = buildAlpCtxF64(expE, expF, encoded, null, null); - AlpEncoding sut = new AlpEncoding(); - - // When - Array result = sut.decode(ctx); - - // Then - assertThat(result.length()).isEqualTo(encoded.length); - var layout = PTypeIO.LE_DOUBLE; - for (int i = 0; i < expected.length; i++) { - assertThat(result.buffer(0).get(layout, (long) i * 8)) - .as("index %d", i).isCloseTo(expected[i], within(1e-9)); - } - } - - @Test - void decode_f64_withPatches() { - // Given — 5 values, encoded = [100, 0, 200, 0, 300] with e=2,f=0 → [1.0, 0.0, 2.0, 0.0, 3.0] - // patches at [1, 3] with real values [Double.NaN, Double.POSITIVE_INFINITY] - int expE = 2, expF = 0; - long[] encoded = {100L, 0L, 200L, 0L, 300L}; - long[] patchIndices = {1L, 3L}; - double[] patchValues = {Double.NaN, Double.POSITIVE_INFINITY}; - - DecodeContext ctx = buildAlpCtxF64(expE, expF, encoded, patchIndices, patchValues); - AlpEncoding sut = new AlpEncoding(); - - // When - Array result = sut.decode(ctx); - - // Then - var layout = PTypeIO.LE_DOUBLE; - assertThat(result.buffer(0).get(layout, 0L)).isCloseTo(1.0, within(1e-9)); - assertThat(result.buffer(0).get(layout, 8L)).isNaN(); - assertThat(result.buffer(0).get(layout, 16L)).isCloseTo(2.0, within(1e-9)); - assertThat(result.buffer(0).get(layout, 24L)).isInfinite(); - assertThat(result.buffer(0).get(layout, 32L)).isCloseTo(3.0, within(1e-9)); - } - - @ParameterizedTest - @CsvSource({"0,0", "1,0", "2,1", "3,2", "4,3"}) - void decode_f64_exponentCombinations(int expE, int expF) { - // Given — encode 42 with given exponents, then verify round-trip - // encode: encoded = round(42.0 * F10[e] * IF10[f]) - double value = 42.0; - double[] f10 = {1e0, 1e1, 1e2, 1e3, 1e4}; - double[] if10 = {1e-0, 1e-1, 1e-2, 1e-3, 1e-4}; - long encVal = Math.round(value * f10[expE] * if10[expF]); - long[] encoded = {encVal}; - - DecodeContext ctx = buildAlpCtxF64(expE, expF, encoded, null, null); - AlpEncoding sut = new AlpEncoding(); - - // When - Array result = sut.decode(ctx); - - // Then - var layout = PTypeIO.LE_DOUBLE; - double decoded = result.buffer(0).get(layout, 0L); - assertThat(decoded).isCloseTo(value, within(1e-6)); - } - - @Test - void decode_f32_noPatches() { - // Given — e=1, f=0: decode = encoded * 1.0 * 0.1 - int expE = 1, expF = 0; - int[] encoded = {10, 25, 100}; - float[] expected = {1.0f, 2.5f, 10.0f}; - - DecodeContext ctx = buildAlpCtxF32(expE, expF, encoded); - AlpEncoding sut = new AlpEncoding(); - - // When - Array result = sut.decode(ctx); - - // Then - var layout = PTypeIO.LE_FLOAT; - for (int i = 0; i < expected.length; i++) { - assertThat(result.buffer(0).get(layout, (long) i * 4)) - .as("index %d", i).isCloseTo(expected[i], within(1e-6f)); - } - } - - private static DecodeContext buildAlpCtxF64( - int expE, int expF, - long[] encodedVals, - long[] patchIndices, - double[] patchValues - ) { - EncodingProtos.ALPMetadata.Builder metaBuilder = EncodingProtos.ALPMetadata.newBuilder() - .setExpE(expE).setExpF(expF); - - if (patchIndices != null) { - EncodingProtos.PatchesMetadata pm = EncodingProtos.PatchesMetadata.newBuilder() - .setLen(patchIndices.length) - .setOffset(0) - .setIndicesPtype(io.github.dfa1.vortex.proto.DTypeProtos.PType.U32) - .build(); - metaBuilder.setPatches(pm); - } - - byte[] metaBytes = metaBuilder.build().toByteArray(); - - byte[] encBuf = new byte[encodedVals.length * 8]; - ByteBuffer bb = ByteBuffer.wrap(encBuf).order(ByteOrder.LITTLE_ENDIAN); - for (long v : encodedVals) { - bb.putLong(v); - } - - ArrayNode encNode = ArrayNode.of(EncodingId.VORTEX_PRIMITIVE, null, - new ArrayNode[0], new int[]{0}, ArrayStats.empty()); - - MemorySegment[] segments; - ArrayNode[] children; - - if (patchIndices != null) { - byte[] idxBuf = new byte[patchIndices.length * 4]; - ByteBuffer ib = ByteBuffer.wrap(idxBuf).order(ByteOrder.LITTLE_ENDIAN); - for (long v : patchIndices) { - ib.putInt((int) v); - } - - byte[] valBuf = new byte[patchValues.length * 8]; - ByteBuffer vb = ByteBuffer.wrap(valBuf).order(ByteOrder.LITTLE_ENDIAN); - for (double v : patchValues) { - vb.putDouble(v); - } - - ArrayNode idxNode = ArrayNode.of(EncodingId.VORTEX_PRIMITIVE, null, - new ArrayNode[0], new int[]{1}, ArrayStats.empty()); - ArrayNode valNode = ArrayNode.of(EncodingId.VORTEX_PRIMITIVE, null, - new ArrayNode[0], new int[]{2}, ArrayStats.empty()); - - children = new ArrayNode[]{encNode, idxNode, valNode}; - segments = new MemorySegment[]{ - MemorySegment.ofArray(encBuf), - MemorySegment.ofArray(idxBuf), - MemorySegment.ofArray(valBuf) - }; - } else { - children = new ArrayNode[]{encNode}; - segments = new MemorySegment[]{MemorySegment.ofArray(encBuf)}; - } - - ArrayNode alpNode = ArrayNode.of(EncodingId.VORTEX_ALP, - ByteBuffer.wrap(metaBytes), children, new int[0], ArrayStats.empty()); - - EncodingRegistry registry = TestRegistry.of(new AlpEncoding(), new PrimitiveEncoding()); - - return new DecodeContext(alpNode, DTypes.F64, encodedVals.length, segments, registry, java.lang.foreign.Arena.global()); - } - - private static DecodeContext buildAlpCtxF32( - int expE, int expF, - int[] encodedVals - ) { - EncodingProtos.ALPMetadata.Builder metaBuilder = EncodingProtos.ALPMetadata.newBuilder() - .setExpE(expE).setExpF(expF); - - byte[] metaBytes = metaBuilder.build().toByteArray(); - - byte[] encBuf = new byte[encodedVals.length * 4]; - ByteBuffer bb = ByteBuffer.wrap(encBuf).order(ByteOrder.LITTLE_ENDIAN); - for (int v : encodedVals) { - bb.putInt(v); - } - - ArrayNode encNode = ArrayNode.of(EncodingId.VORTEX_PRIMITIVE, null, - new ArrayNode[0], new int[]{0}, ArrayStats.empty()); - - ArrayNode alpNode = ArrayNode.of(EncodingId.VORTEX_ALP, - ByteBuffer.wrap(metaBytes), new ArrayNode[]{encNode}, new int[0], ArrayStats.empty()); - - MemorySegment[] segments = {MemorySegment.ofArray(encBuf)}; - - EncodingRegistry registry = TestRegistry.of(new AlpEncoding(), new PrimitiveEncoding()); - - return new DecodeContext(alpNode, DTypes.F32, encodedVals.length, segments, registry, java.lang.foreign.Arena.global()); - } - } - - @Nested - class Encode { - - @Test - void encode_f32_roundTrip_noPatches() { - // Given - float[] values = {1.0f, 2.5f, 3.75f, 10.0f, 0.1f}; - AlpEncoding sut = new AlpEncoding(); - - EncodingRegistry registry = TestRegistry.withPrimitive(sut); - - // When - EncodeResult encoded = sut.encode(DTypes.F32, values); - DecodeContext ctx = EncodeTestHelper.toDecodeContext(encoded, values.length, DTypes.F32, registry); - Array result = sut.decode(ctx); - - // Then - var layout = PTypeIO.LE_FLOAT; - for (int i = 0; i < values.length; i++) { - assertThat(result.buffer(0).get(layout, (long) i * 4)) - .as("index %d", i).isCloseTo(values[i], within(1e-6f)); - } - } - - @Test - void encode_f32_roundTrip_withPatches() { - // Given — Float.NaN and Float.POSITIVE_INFINITY can't be ALP-encoded; must become patches - float[] values = {1.0f, Float.NaN, 2.5f, Float.POSITIVE_INFINITY, 3.0f}; - AlpEncoding sut = new AlpEncoding(); - - EncodingRegistry registry = TestRegistry.withPrimitive(sut); - - // When - EncodeResult encoded = sut.encode(DTypes.F32, values); - DecodeContext ctx = EncodeTestHelper.toDecodeContext(encoded, values.length, DTypes.F32, registry); - Array result = sut.decode(ctx); - - // Then - var layout = PTypeIO.LE_FLOAT; - assertThat(result.buffer(0).get(layout, 0L)).isCloseTo(1.0f, within(1e-6f)); - assertThat(result.buffer(0).get(layout, 4L)).isNaN(); - assertThat(result.buffer(0).get(layout, 8L)).isCloseTo(2.5f, within(1e-6f)); - assertThat(result.buffer(0).get(layout, 12L)).isInfinite(); - assertThat(result.buffer(0).get(layout, 16L)).isCloseTo(3.0f, within(1e-6f)); - } - - @Test - void encode_f64_roundTrip_noPatches() { - // Given - double[] values = {1.23, 4.56, 7.89, 0.001, 100.0}; - AlpEncoding sut = new AlpEncoding(); - - EncodingRegistry registry = TestRegistry.withPrimitive(sut); - - // When - EncodeResult encoded = sut.encode(DTypes.F64, values); - DecodeContext ctx = EncodeTestHelper.toDecodeContext(encoded, values.length, DTypes.F64, registry); - Array result = sut.decode(ctx); - - // Then - var layout = PTypeIO.LE_DOUBLE; - for (int i = 0; i < values.length; i++) { - assertThat(result.buffer(0).get(layout, (long) i * 8)) - .as("index %d", i).isCloseTo(values[i], within(1e-9)); - } - } - } -} diff --git a/core/src/test/java/io/github/dfa1/vortex/encoding/BitpackedEncodingPatchesTest.java b/core/src/test/java/io/github/dfa1/vortex/encoding/BitpackedEncodingPatchesTest.java deleted file mode 100644 index c842da0..0000000 --- a/core/src/test/java/io/github/dfa1/vortex/encoding/BitpackedEncodingPatchesTest.java +++ /dev/null @@ -1,84 +0,0 @@ -package io.github.dfa1.vortex.encoding; - -import io.github.dfa1.vortex.proto.DTypeProtos; -import io.github.dfa1.vortex.proto.EncodingProtos; -import io.github.dfa1.vortex.core.array.Array; -import io.github.dfa1.vortex.core.ArrayStats; -import org.junit.jupiter.api.Nested; -import org.junit.jupiter.api.Test; - -import java.lang.foreign.Arena; -import java.lang.foreign.MemorySegment; -import java.nio.ByteBuffer; -import java.nio.ByteOrder; - -import static org.assertj.core.api.Assertions.assertThat; - -class BitpackedEncodingPatchesTest { - - @Nested - class Decode { - - @Test - void decode_appliesPatches_overridingBitPackedValues() { - // Given — bit-pack [10, 20, 30, 40, 50] via the production encoder (bitWidth = 6), - // then attach synthetic patches metadata that rewrites indices [1, 3] with [777, 999]. - int[] base = {10, 20, 30, 40, 50}; - BitpackedEncoding sut = new BitpackedEncoding(); - EncodeResult packed = sut.encode(DTypes.I32, base); - - MemorySegment packedSeg = packed.buffers().getFirst(); - byte[] packedBytes = packedSeg.toArray(java.lang.foreign.ValueLayout.JAVA_BYTE); - - // Build new BitPackedMetadata that re-uses the packed bytes but advertises patches. - EncodingProtos.PatchesMetadata patches = EncodingProtos.PatchesMetadata.newBuilder() - .setLen(2) - .setOffset(0) - .setIndicesPtype(DTypeProtos.PType.U32) - .build(); - byte[] metaBytes = EncodingProtos.BitPackedMetadata.newBuilder() - .setBitWidth(6) // matches encoder's choice for max=50 - .setOffset(0) - .setPatches(patches) - .build() - .toByteArray(); - - byte[] idxBuf = new byte[2 * 4]; - ByteBuffer.wrap(idxBuf).order(ByteOrder.LITTLE_ENDIAN).putInt(1).putInt(3); - byte[] valBuf = new byte[2 * 4]; - ByteBuffer.wrap(valBuf).order(ByteOrder.LITTLE_ENDIAN).putInt(777).putInt(999); - - ArrayNode idxNode = ArrayNode.of(EncodingId.VORTEX_PRIMITIVE, null, - new ArrayNode[0], new int[]{1}, ArrayStats.empty()); - ArrayNode valNode = ArrayNode.of(EncodingId.VORTEX_PRIMITIVE, null, - new ArrayNode[0], new int[]{2}, ArrayStats.empty()); - ArrayNode bpNode = ArrayNode.of(EncodingId.FASTLANES_BITPACKED, - ByteBuffer.wrap(metaBytes), - new ArrayNode[]{idxNode, valNode}, - new int[]{0}, - ArrayStats.empty()); - - MemorySegment[] segments = { - MemorySegment.ofArray(packedBytes), - MemorySegment.ofArray(idxBuf), - MemorySegment.ofArray(valBuf) - }; - - EncodingRegistry registry = TestRegistry.of(new BitpackedEncoding(), new PrimitiveEncoding()); - - DecodeContext ctx = new DecodeContext( - bpNode, DTypes.I32, base.length, segments, registry, Arena.global()); - - // When - Array result = sut.decode(ctx); - - // Then - assertThat(result.length()).isEqualTo(base.length); - assertThat(result.buffer(0).get(PTypeIO.LE_INT, 0L)).isEqualTo(10); - assertThat(result.buffer(0).get(PTypeIO.LE_INT, 4L)).isEqualTo(777); - assertThat(result.buffer(0).get(PTypeIO.LE_INT, 8L)).isEqualTo(30); - assertThat(result.buffer(0).get(PTypeIO.LE_INT, 12L)).isEqualTo(999); - assertThat(result.buffer(0).get(PTypeIO.LE_INT, 16L)).isEqualTo(50); - } - } -} diff --git a/core/src/test/java/io/github/dfa1/vortex/encoding/BitpackedEncodingTest.java b/core/src/test/java/io/github/dfa1/vortex/encoding/BitpackedEncodingTest.java deleted file mode 100644 index d7f38e5..0000000 --- a/core/src/test/java/io/github/dfa1/vortex/encoding/BitpackedEncodingTest.java +++ /dev/null @@ -1,80 +0,0 @@ -package io.github.dfa1.vortex.encoding; - -import io.github.dfa1.vortex.core.array.Array; -import org.junit.jupiter.api.Nested; -import org.junit.jupiter.params.ParameterizedTest; -import org.junit.jupiter.params.provider.Arguments; -import org.junit.jupiter.params.provider.MethodSource; - -import java.util.stream.Stream; - -import static org.assertj.core.api.Assertions.assertThat; - -/// Property: encode then decode is lossless for unsigned integer types. -class BitpackedEncodingTest { - - - @Nested - class Encode { - - @ParameterizedTest(name = "{0}") - @MethodSource("u32Arrays") - void encodeDecode_u32_isLossless(String name, int[] data) { - // Given - var sut = new BitpackedEncoding(); - EncodingRegistry registry = TestRegistry.of(sut); - - // When - EncodeResult encoded = sut.encode(DTypes.U32, data); - DecodeContext ctx = EncodeTestHelper.toDecodeContext(encoded, data.length, DTypes.U32, registry); - Array result = sut.decode(ctx); - - // Then - assertThat(result.length()).isEqualTo(data.length); - var le = PTypeIO.LE_INT; - for (int i = 0; i < data.length; i++) { - assertThat(result.buffer(0).get(le, (long) i * 4)).as("index %d", i).isEqualTo(data[i]); - } - } - - @ParameterizedTest(name = "{0}") - @MethodSource("u64Arrays") - void encodeDecode_u64_isLossless(String name, long[] data) { - // Given - var sut = new BitpackedEncoding(); - EncodingRegistry registry = TestRegistry.of(sut); - - // When - EncodeResult encoded = sut.encode(DTypes.U64, data); - DecodeContext ctx = EncodeTestHelper.toDecodeContext(encoded, data.length, DTypes.U64, registry); - Array result = sut.decode(ctx); - - // Then - assertThat(result.length()).isEqualTo(data.length); - var le = PTypeIO.LE_LONG; - for (int i = 0; i < data.length; i++) { - assertThat(result.buffer(0).get(le, (long) i * 8)).as("index %d", i).isEqualTo(data[i]); - } - } - - static Stream u32Arrays() { - return Stream.of( - Arguments.of("empty", new int[]{}), - Arguments.of("single", new int[]{0}), - Arguments.of("all-zeros", new int[]{0, 0, 0, 0, 0}), - Arguments.of("small-values", new int[]{1, 2, 3, 4, 5, 6, 7}), - Arguments.of("mixed", new int[]{0, 7, 63, 255, 1023, 65535}), - Arguments.of("max-unsigned", new int[]{-1, -1, -1}) // 0xFFFFFFFF - ); - } - - static Stream u64Arrays() { - return Stream.of( - Arguments.of("empty", new long[]{}), - Arguments.of("single", new long[]{0L}), - Arguments.of("small-values", new long[]{1L, 2L, 3L, 4L, 5L}), - Arguments.of("large-values", new long[]{0L, 0xFFFFL, 0xFFFFFFL, 0xFFFFFFFFL}) - ); - } - } -} diff --git a/core/src/test/java/io/github/dfa1/vortex/encoding/BoolEncodingTest.java b/core/src/test/java/io/github/dfa1/vortex/encoding/BoolEncodingTest.java deleted file mode 100644 index 8fdc042..0000000 --- a/core/src/test/java/io/github/dfa1/vortex/encoding/BoolEncodingTest.java +++ /dev/null @@ -1,68 +0,0 @@ -package io.github.dfa1.vortex.encoding; - -import io.github.dfa1.vortex.core.array.Array; -import io.github.dfa1.vortex.core.array.BoolArray; -import org.junit.jupiter.api.Nested; -import org.junit.jupiter.params.ParameterizedTest; -import org.junit.jupiter.params.provider.MethodSource; - -import java.lang.foreign.ValueLayout; -import java.util.stream.Stream; - -import static org.assertj.core.api.Assertions.assertThat; - -/// Property: encode then decode is lossless for boolean arrays of all lengths (including non-multiples of 8). -class BoolEncodingTest { - - @Nested - class Encode { - - @ParameterizedTest - @MethodSource("boolArrays") - void encodeDecode_isLossless(boolean[] data) { - // Given - var sut = new BoolEncoding(); - EncodingRegistry registry = TestRegistry.of(sut); - - // When - EncodeResult encoded = sut.encode(DTypes.BOOL, data); - DecodeContext ctx = EncodeTestHelper.toDecodeContext(encoded, data.length, DTypes.BOOL, registry); - Array result = sut.decode(ctx); - - // Then - assertThat(result).isInstanceOf(BoolArray.class); - assertThat(result.length()).isEqualTo(data.length); - for (int i = 0; i < data.length; i++) { - byte byteVal = result.buffer(0).get(ValueLayout.JAVA_BYTE, i / 8); - boolean decoded = ((byteVal >>> (i % 8)) & 1) == 1; - assertThat(decoded).as("index %d", i).isEqualTo(data[i]); - } - } - - @ParameterizedTest - @MethodSource("boolArrays") - void encodedSize_isPackedBits(boolean[] data) { - // Given - var sut = new BoolEncoding(); - - // When - EncodeResult encoded = sut.encode(DTypes.BOOL, data); - - // Then — bit-packed: ceiling(n/8) bytes, always ≤ n bytes raw - long totalBytes = encoded.buffers().stream().mapToLong(java.lang.foreign.MemorySegment::byteSize).sum(); - assertThat(totalBytes).isEqualTo((long) (data.length + 7) / 8); - } - - static Stream boolArrays() { - return Stream.of( - new boolean[]{}, - new boolean[]{false}, - new boolean[]{true}, - new boolean[]{false, true, false, true, false, true, false, true}, - new boolean[]{true, true, true, false, false, false, true, false, true}, - new boolean[]{false, false, false, false, false, false, false}, - new boolean[]{true, true, true, true, true, true, true, true, true} - ); - } - } -} diff --git a/core/src/test/java/io/github/dfa1/vortex/encoding/ByteBoolEncodingTest.java b/core/src/test/java/io/github/dfa1/vortex/encoding/ByteBoolEncodingTest.java deleted file mode 100644 index 00288c5..0000000 --- a/core/src/test/java/io/github/dfa1/vortex/encoding/ByteBoolEncodingTest.java +++ /dev/null @@ -1,97 +0,0 @@ -package io.github.dfa1.vortex.encoding; - -import io.github.dfa1.vortex.core.DType; -import io.github.dfa1.vortex.core.array.Array; -import io.github.dfa1.vortex.core.array.BoolArray; -import org.junit.jupiter.api.Nested; -import org.junit.jupiter.params.ParameterizedTest; -import org.junit.jupiter.params.provider.Arguments; -import org.junit.jupiter.params.provider.MethodSource; - -import java.lang.foreign.Arena; -import java.lang.foreign.MemorySegment; -import java.util.stream.Stream; - -import static org.assertj.core.api.Assertions.assertThat; - -class ByteBoolEncodingTest { - - private static final DType BOOL_DTYPE = new DType.Bool(false); - - @Nested - class Encode { - - @ParameterizedTest - @MethodSource("boolArrays") - void encodeDecode_isLossless(boolean[] data) { - // Given - var sut = new ByteBoolEncoding(); - EncodingRegistry registry = EncodingRegistry.empty(); - registry.register(sut); - - // When - EncodeResult encoded = sut.encode(BOOL_DTYPE, data); - DecodeContext ctx = EncodeTestHelper.toDecodeContext(encoded, data.length, BOOL_DTYPE, registry); - Array result = sut.decode(ctx); - - // Then - assertThat(result.length()).isEqualTo(data.length); - BoolArray boolArr = (BoolArray) result; - for (int i = 0; i < data.length; i++) { - assertThat(boolArr.getBoolean(i)).as("index %d", i).isEqualTo(data[i]); - } - } - - static Stream boolArrays() { - return Stream.of( - new boolean[]{}, - new boolean[]{false}, - new boolean[]{true}, - new boolean[]{true, false, true, false, true}, - new boolean[]{false, false, false, false}, - new boolean[]{true, true, true, true, true, true, true, true, true} - ); - } - } - - @Nested - class Decode { - - @ParameterizedTest(name = "{0}") - @MethodSource("cases") - void decode_byteBool_packsToBitArray(String name, byte[] input, boolean[] expected) { - // Given - DecodeContext ctx = buildCtx(input); - var sut = new ByteBoolEncoding(); - - // When - var result = sut.decode(ctx); - - // Then - assertThat(result).isInstanceOf(BoolArray.class); - assertThat(result.length()).isEqualTo(expected.length); - BoolArray boolArr = (BoolArray) result; - for (int i = 0; i < expected.length; i++) { - assertThat(boolArr.getBoolean(i)).as("index %d", i).isEqualTo(expected[i]); - } - } - - static Stream cases() { - return Stream.of( - Arguments.of("all false", new byte[]{0, 0, 0}, new boolean[]{false, false, false}), - Arguments.of("all true", new byte[]{1, 42, (byte) 0xFF}, new boolean[]{true, true, true}), - Arguments.of("mixed", new byte[]{0, 1, 0, 1}, new boolean[]{false, true, false, true}), - Arguments.of("empty", new byte[]{}, new boolean[]{}) - ); - } - - private static DecodeContext buildCtx(byte[] byteValues) { - MemorySegment buf = MemorySegment.ofArray(byteValues); - ArrayNode node = ArrayNode.of(EncodingId.VORTEX_BYTEBOOL, null, new ArrayNode[0], new int[]{0}, null); - EncodingRegistry registry = EncodingRegistry.empty(); - registry.register(new ByteBoolEncoding()); - return new DecodeContext(node, BOOL_DTYPE, byteValues.length, new MemorySegment[]{buf}, registry, - Arena.ofAuto()); - } - } -} diff --git a/core/src/test/java/io/github/dfa1/vortex/encoding/CascadingCompressorTest.java b/core/src/test/java/io/github/dfa1/vortex/encoding/CascadingCompressorTest.java deleted file mode 100644 index e7d103b..0000000 --- a/core/src/test/java/io/github/dfa1/vortex/encoding/CascadingCompressorTest.java +++ /dev/null @@ -1,164 +0,0 @@ -package io.github.dfa1.vortex.encoding; - -import io.github.dfa1.vortex.core.array.DoubleArray; -import io.github.dfa1.vortex.core.array.LongArray; -import org.junit.jupiter.api.Nested; -import org.junit.jupiter.api.Test; - -import java.lang.foreign.MemorySegment; -import java.util.List; -import java.util.Set; - -import static org.assertj.core.api.Assertions.assertThat; - -class CascadingCompressorTest { - - - private static final List ALL_CODECS = List.of( - new AlpEncoding(), new FrameOfReferenceEncoding(), new DictEncoding(), - new BitpackedEncoding(), new PrimitiveEncoding()); - - @Nested - class Compress { - - @Test - void depth0_picksTerminalOnly_forF64() { - // Given - double[] values = new double[4096]; - for (int i = 0; i < values.length; i++) { - values[i] = i * 1.5; - } - CompressorContext ctx = CompressorContext.ofDepth(0); - CascadingCompressor sut = new CascadingCompressor(ALL_CODECS, ctx); - - // When - EncodeResult result = sut.encode(DTypes.F64, values); - - // Then - result should be a valid non-null encode result - assertThat(result).isNotNull(); - assertThat(result.rootNode()).isNotNull(); - assertThat(result.buffers()).isNotEmpty(); - } - - @Test - void depth1_alpPlusBitpacked_producesSmallResult_forF64() { - // Given: OHLC-style prices — ALP-encodable, small range → bitpackable residuals - double[] values = new double[4096]; - for (int i = 0; i < values.length; i++) { - values[i] = 100.0 + (i % 50) * 0.01; - } - CompressorContext ctx = CompressorContext.ofDepth(1); - CascadingCompressor sut = new CascadingCompressor(ALL_CODECS, ctx); - - // When - EncodeResult result = sut.encode(DTypes.F64, values); - - // Then - cascaded result should be smaller than raw primitive (4096 * 8 = 32768 bytes) - long totalBytes = result.buffers().stream().mapToLong(MemorySegment::byteSize).sum(); - assertThat(totalBytes).isLessThan(4096L * 8); - } - - @Test - void excludedEncodings_areSkipped() { - // Given: exclude AlpEncoding — should not be selected for DTypes.F64 - double[] values = new double[512]; - for (int i = 0; i < values.length; i++) { - values[i] = i * 1.5; - } - CompressorContext ctx = new CompressorContext(1, Set.of(EncodingId.VORTEX_ALP), 42L, 64, 0.1); - CascadingCompressor sut = new CascadingCompressor(ALL_CODECS, ctx); - - // When - EncodeResult result = sut.encode(DTypes.F64, values); - - // Then - ALP node should not appear in the tree - assertThat(containsEncoding(result.rootNode(), EncodingId.VORTEX_ALP)).isFalse(); - } - - private boolean containsEncoding(EncodeNode node, EncodingId id) { - if (node.encodingId().equals(id)) { - return true; - } - for (EncodeNode child : node.children()) { - if (containsEncoding(child, id)) { - return true; - } - } - return false; - } - } - - @Nested - class RoundTrip { - - @Test - void alpBitpacked_f64() { - // Given - double[] values = new double[1024]; - for (int i = 0; i < values.length; i++) { - values[i] = 150.0 + (i % 100) * 0.01; - } - CompressorContext ctx = CompressorContext.ofDepth(1); - CascadingCompressor sut = new CascadingCompressor(ALL_CODECS, ctx); - - // When - EncodeResult result = sut.encode(DTypes.F64, values); - EncodingRegistry registry = EncodingRegistry.empty(); - ALL_CODECS.forEach(registry::register); - DecodeContext decodeCtx = EncodeTestHelper.toDecodeContext(result, values.length, DTypes.F64, registry); - DoubleArray decoded = (DoubleArray) registry.decode(decodeCtx); - - // Then - for (int i = 0; i < values.length; i++) { - assertThat(decoded.getDouble(i)).isEqualTo(values[i]); - } - } - - @Test - void forBitpacked_i64() { - // Given: integers in narrow range → FoR reduces to small residuals → bitpackable - long[] values = new long[1024]; - for (int i = 0; i < values.length; i++) { - values[i] = 1_000_000L + (i % 200); - } - CompressorContext ctx = CompressorContext.ofDepth(1); - CascadingCompressor sut = new CascadingCompressor(ALL_CODECS, ctx); - - // When - EncodeResult result = sut.encode(DTypes.I64, values); - EncodingRegistry registry = EncodingRegistry.empty(); - ALL_CODECS.forEach(registry::register); - DecodeContext decodeCtx = EncodeTestHelper.toDecodeContext(result, values.length, DTypes.I64, registry); - LongArray decoded = (LongArray) registry.decode(decodeCtx); - - // Then - for (int i = 0; i < values.length; i++) { - assertThat(decoded.getLong(i)).isEqualTo(values[i]); - } - } - - @Test - void dictBitpacked_i32() { - // Given: low-cardinality int column - int[] values = new int[2048]; - for (int i = 0; i < values.length; i++) { - values[i] = i % 10; - } - CompressorContext ctx = CompressorContext.ofDepth(1); - CascadingCompressor sut = new CascadingCompressor(ALL_CODECS, ctx); - - // When - EncodeResult result = sut.encode(DTypes.I32, values); - EncodingRegistry registry = EncodingRegistry.empty(); - ALL_CODECS.forEach(registry::register); - DecodeContext decodeCtx = EncodeTestHelper.toDecodeContext(result, values.length, DTypes.I32, registry); - io.github.dfa1.vortex.core.array.IntArray decoded = - (io.github.dfa1.vortex.core.array.IntArray) registry.decode(decodeCtx); - - // Then - for (int i = 0; i < values.length; i++) { - assertThat(decoded.getInt(i)).isEqualTo(values[i]); - } - } - } -} diff --git a/core/src/test/java/io/github/dfa1/vortex/encoding/ChunkedEncodingTest.java b/core/src/test/java/io/github/dfa1/vortex/encoding/ChunkedEncodingTest.java deleted file mode 100644 index 1ae4737..0000000 --- a/core/src/test/java/io/github/dfa1/vortex/encoding/ChunkedEncodingTest.java +++ /dev/null @@ -1,202 +0,0 @@ -package io.github.dfa1.vortex.encoding; - -import io.github.dfa1.vortex.core.DType; -import io.github.dfa1.vortex.core.PType; -import io.github.dfa1.vortex.core.VortexException; -import io.github.dfa1.vortex.core.array.Array; -import org.junit.jupiter.api.Nested; -import org.junit.jupiter.api.Test; - -import java.lang.foreign.Arena; -import java.lang.foreign.MemorySegment; -import java.lang.foreign.ValueLayout; -import java.nio.ByteOrder; -import java.util.List; - -import static org.assertj.core.api.Assertions.assertThat; -import static org.assertj.core.api.Assertions.assertThatThrownBy; - -class ChunkedEncodingTest { - - private static final ValueLayout.OfLong LE_LONG = - ValueLayout.JAVA_LONG_UNALIGNED.withOrder(ByteOrder.LITTLE_ENDIAN); - - @Nested - class Encode { - - @Test - void roundTrip_twoChunks_i64_preservesValues() { - // Given - long[] chunk0 = {10L, 20L, 30L}; - long[] chunk1 = {40L, 50L}; - DType i64 = new DType.Primitive(PType.I64, false); - var sut = new ChunkedEncoding(); - EncodingRegistry registry = EncodingRegistry.empty(); - registry.register(sut); - registry.register(new PrimitiveEncoding()); - ChunkedData data = new ChunkedData(List.of(chunk0, chunk1), new long[]{3, 2}); - - // When - EncodeResult encoded = sut.encode(i64, data); - DecodeContext ctx = EncodeTestHelper.toDecodeContext(encoded, 5L, i64, registry); - Array result = sut.decode(ctx); - - // Then - assertThat(result.length()).isEqualTo(5); - assertThat(result.buffer(0).get(LE_LONG, 0L)).isEqualTo(10L); - assertThat(result.buffer(0).get(LE_LONG, 8L)).isEqualTo(20L); - assertThat(result.buffer(0).get(LE_LONG, 16L)).isEqualTo(30L); - assertThat(result.buffer(0).get(LE_LONG, 24L)).isEqualTo(40L); - assertThat(result.buffer(0).get(LE_LONG, 32L)).isEqualTo(50L); - } - - @Test - void encodeNode_hasNoDirectBuffers_offsetsAsFirstChild() { - // Given - long[] chunk0 = {1L, 2L}; - DType i64 = new DType.Primitive(PType.I64, false); - var sut = new ChunkedEncoding(); - ChunkedData data = new ChunkedData(List.of(chunk0), new long[]{2}); - - // When - EncodeResult result = sut.encode(i64, data); - - // Then - assertThat(result.rootNode().bufferIndices()).isEmpty(); - assertThat(result.rootNode().children()).hasSize(2); // offsets + 1 chunk - assertThat(result.buffers()).hasSize(2); // offsets buf + chunk buf - } - - @Test - void mismatchedLengths_throws() { - // Given - var sut = new ChunkedEncoding(); - DType i64 = new DType.Primitive(PType.I64, false); - - // When / Then - assertThatThrownBy(() -> new ChunkedData(List.of(new long[]{1L}), new long[]{1, 2})) - .isInstanceOf(IllegalArgumentException.class); - } - } - - @Nested - class Decode { - - @Test - void roundTrip_twoChunks_concatenatesValues() { - // Given - long[] chunk0 = {10L, 20L, 30L}; - long[] chunk1 = {40L, 50L}; - DType i64 = new DType.Primitive(PType.I64, false); - DType u64 = new DType.Primitive(PType.U64, false); - - EncodingRegistry registry = EncodingRegistry.empty(); - var sut = new ChunkedEncoding(); - registry.register(sut); - registry.register(new PrimitiveEncoding()); - - // Build chunk_offsets segment: [0, 3, 5] as U64 LE - EncodeResult offsetsResult = new PrimitiveEncoding().encode(u64, new long[]{0L, 3L, 5L}); - // Build chunk0 segment - EncodeResult chunk0Result = new PrimitiveEncoding().encode(i64, chunk0); - // Build chunk1 segment - EncodeResult chunk1Result = new PrimitiveEncoding().encode(i64, chunk1); - - // Collect all buffers: [offsets_buf, chunk0_buf, chunk1_buf] - MemorySegment[] allBufs = { - offsetsResult.buffers().getFirst(), - chunk0Result.buffers().getFirst(), - chunk1Result.buffers().getFirst() - }; - - // Build ArrayNode tree: - // root: ChunkedEncoding, children=[offsetsNode, chunk0Node, chunk1Node], buffers=[] - // offsetsNode: PrimitiveEncoding, bufferIndices=[0] - // chunk0Node: PrimitiveEncoding, bufferIndices=[1] - // chunk1Node: PrimitiveEncoding, bufferIndices=[2] - ArrayNode offsetsNode = toArrayNode(offsetsResult.rootNode()); - ArrayNode chunk0Node = toArrayNode(remapped(chunk0Result.rootNode(), 1)); - ArrayNode chunk1Node = toArrayNode(remapped(chunk1Result.rootNode(), 2)); - ArrayNode root = ArrayNode.of( - EncodingId.VORTEX_CHUNKED, null, - new ArrayNode[]{offsetsNode, chunk0Node, chunk1Node}, - new int[]{}, null); - - DecodeContext ctx = new DecodeContext(root, i64, 5L, allBufs, registry, Arena.ofAuto()); - - // When - Array result = sut.decode(ctx); - - // Then - assertThat(result.length()).isEqualTo(5); - assertThat(result.buffer(0).get(LE_LONG, 0L)).isEqualTo(10L); - assertThat(result.buffer(0).get(LE_LONG, 8L)).isEqualTo(20L); - assertThat(result.buffer(0).get(LE_LONG, 16L)).isEqualTo(30L); - assertThat(result.buffer(0).get(LE_LONG, 24L)).isEqualTo(40L); - assertThat(result.buffer(0).get(LE_LONG, 32L)).isEqualTo(50L); - } - - @Test - void singleChunk_returnsSameValues() { - // Given - long[] data = {1L, 2L, 3L}; - DType i64 = new DType.Primitive(PType.I64, false); - DType u64 = new DType.Primitive(PType.U64, false); - - EncodingRegistry registry = EncodingRegistry.empty(); - registry.register(new ChunkedEncoding()); - registry.register(new PrimitiveEncoding()); - - EncodeResult offsetsResult = new PrimitiveEncoding().encode(u64, new long[]{0L, 3L}); - EncodeResult chunkResult = new PrimitiveEncoding().encode(i64, data); - - MemorySegment[] allBufs = { - offsetsResult.buffers().getFirst(), - chunkResult.buffers().getFirst() - }; - - ArrayNode root = ArrayNode.of( - EncodingId.VORTEX_CHUNKED, null, - new ArrayNode[]{toArrayNode(offsetsResult.rootNode()), toArrayNode(remapped(chunkResult.rootNode(), 1))}, - new int[]{}, null); - - DecodeContext ctx = new DecodeContext(root, i64, 3L, allBufs, registry, Arena.ofAuto()); - - // When - Array result = new ChunkedEncoding().decode(ctx); - - // Then - assertThat(result.length()).isEqualTo(3); - for (int i = 0; i < 3; i++) { - assertThat(result.buffer(0).get(LE_LONG, (long) i * 8)).isEqualTo(data[i]); - } - } - - @Test - void noChildren_throws() { - // Given - DType i64 = new DType.Primitive(PType.I64, false); - EncodingRegistry registry = EncodingRegistry.empty(); - registry.register(new ChunkedEncoding()); - ArrayNode root = ArrayNode.of(EncodingId.VORTEX_CHUNKED, null, new ArrayNode[]{}, new int[]{}, null); - DecodeContext ctx = new DecodeContext(root, i64, 0L, new MemorySegment[]{}, registry, Arena.ofAuto()); - - // When / Then - assertThatThrownBy(() -> new ChunkedEncoding().decode(ctx)) - .isInstanceOf(VortexException.class) - .hasMessageContaining("at least one child"); - } - } - - private static ArrayNode toArrayNode(EncodeNode enc) { - ArrayNode[] children = new ArrayNode[enc.children().length]; - for (int i = 0; i < children.length; i++) { - children[i] = toArrayNode(enc.children()[i]); - } - return ArrayNode.of(enc.encodingId(), enc.metadata(), children, enc.bufferIndices(), null); - } - - private static EncodeNode remapped(EncodeNode node, int offset) { - return EncodeNode.remapBufferIndices(node, offset); - } -} diff --git a/core/src/test/java/io/github/dfa1/vortex/encoding/ConstantEncodingTest.java b/core/src/test/java/io/github/dfa1/vortex/encoding/ConstantEncodingTest.java deleted file mode 100644 index 9e4ca56..0000000 --- a/core/src/test/java/io/github/dfa1/vortex/encoding/ConstantEncodingTest.java +++ /dev/null @@ -1,78 +0,0 @@ -package io.github.dfa1.vortex.encoding; - -import io.github.dfa1.vortex.core.array.Array; -import org.junit.jupiter.api.Nested; -import org.junit.jupiter.params.ParameterizedTest; -import org.junit.jupiter.params.provider.Arguments; -import org.junit.jupiter.params.provider.MethodSource; - -import java.util.stream.Stream; - -import static org.assertj.core.api.Assertions.assertThat; - -/// Property: encode then decode is lossless for constant (all-equal) arrays. -class ConstantEncodingTest { - - @Nested - class Encode { - - @ParameterizedTest - @MethodSource("i32ConstantArrays") - void encodeDecode_i32_isLossless(int[] data) { - // Given - var sut = new ConstantEncoding(); - EncodingRegistry registry = TestRegistry.of(sut); - var le = PTypeIO.LE_INT; - - // When - EncodeResult encoded = sut.encode(DTypes.I32, data); - DecodeContext ctx = EncodeTestHelper.toDecodeContext(encoded, data.length, DTypes.I32, registry); - Array result = sut.decode(ctx); - - // Then - assertThat(result.length()).isEqualTo(data.length); - for (int i = 0; i < data.length; i++) { - assertThat(result.buffer(0).get(le, (long) i * 4)).as("index %d", i).isEqualTo(data[i]); - } - } - - @ParameterizedTest - @MethodSource("i64ConstantArrays") - void encodeDecode_i64_isLossless(long[] data) { - // Given - var sut = new ConstantEncoding(); - EncodingRegistry registry = TestRegistry.of(sut); - var le = PTypeIO.LE_LONG; - - // When - EncodeResult encoded = sut.encode(DTypes.I64, data); - DecodeContext ctx = EncodeTestHelper.toDecodeContext(encoded, data.length, DTypes.I64, registry); - Array result = sut.decode(ctx); - - // Then - assertThat(result.length()).isEqualTo(data.length); - for (int i = 0; i < data.length; i++) { - assertThat(result.buffer(0).get(le, (long) i * 8)).as("index %d", i).isEqualTo(data[i]); - } - } - - static Stream i32ConstantArrays() { - return Stream.of( - Arguments.of((Object) new int[]{0}), - Arguments.of((Object) new int[]{42, 42, 42}), - Arguments.of((Object) new int[]{Integer.MIN_VALUE, Integer.MIN_VALUE, Integer.MIN_VALUE}), - Arguments.of((Object) new int[]{Integer.MAX_VALUE, Integer.MAX_VALUE}), - Arguments.of((Object) new int[]{-1, -1, -1, -1, -1}) - ); - } - - static Stream i64ConstantArrays() { - return Stream.of( - Arguments.of((Object) new long[]{0L}), - Arguments.of((Object) new long[]{100L, 100L, 100L}), - Arguments.of((Object) new long[]{Long.MIN_VALUE, Long.MIN_VALUE}), - Arguments.of((Object) new long[]{Long.MAX_VALUE, Long.MAX_VALUE, Long.MAX_VALUE}) - ); - } - } -} diff --git a/core/src/test/java/io/github/dfa1/vortex/encoding/DTypes.java b/core/src/test/java/io/github/dfa1/vortex/encoding/DTypes.java deleted file mode 100644 index ae4c1da..0000000 --- a/core/src/test/java/io/github/dfa1/vortex/encoding/DTypes.java +++ /dev/null @@ -1,35 +0,0 @@ -package io.github.dfa1.vortex.encoding; - -import io.github.dfa1.vortex.core.DType; -import io.github.dfa1.vortex.core.PType; - -/// Shared DType constants for encoding tests. -final class DTypes { - - static final DType I8 = new DType.Primitive(PType.I8, false); - static final DType I16 = new DType.Primitive(PType.I16, false); - static final DType I32 = new DType.Primitive(PType.I32, false); - static final DType I64 = new DType.Primitive(PType.I64, false); - static final DType U8 = new DType.Primitive(PType.U8, false); - static final DType U16 = new DType.Primitive(PType.U16, false); - static final DType U32 = new DType.Primitive(PType.U32, false); - static final DType U64 = new DType.Primitive(PType.U64, false); - static final DType F32 = new DType.Primitive(PType.F32, false); - static final DType F64 = new DType.Primitive(PType.F64, false); - - static final DType I32_N = new DType.Primitive(PType.I32, true); - static final DType I64_N = new DType.Primitive(PType.I64, true); - static final DType F64_N = new DType.Primitive(PType.F64, true); - - static final DType BOOL = new DType.Bool(false); - static final DType BOOL_N = new DType.Bool(true); - static final DType UTF8 = new DType.Utf8(false); - static final DType UTF8_N = new DType.Utf8(true); - static final DType BINARY = new DType.Binary(false); - static final DType BINARY_N = new DType.Binary(true); - static final DType NULL = new DType.Null(true); - - static final DType LIST_I32 = new DType.List(I32, false); - - private DTypes() {} -} diff --git a/core/src/test/java/io/github/dfa1/vortex/encoding/DateTimePartsEncodingTest.java b/core/src/test/java/io/github/dfa1/vortex/encoding/DateTimePartsEncodingTest.java deleted file mode 100644 index 418ec3f..0000000 --- a/core/src/test/java/io/github/dfa1/vortex/encoding/DateTimePartsEncodingTest.java +++ /dev/null @@ -1,234 +0,0 @@ -package io.github.dfa1.vortex.encoding; - -import io.github.dfa1.vortex.core.ArrayStats; -import io.github.dfa1.vortex.core.DType; -import io.github.dfa1.vortex.core.PType; -import io.github.dfa1.vortex.core.array.GenericArray; -import io.github.dfa1.vortex.core.array.LongArray; -import org.junit.jupiter.api.Nested; -import org.junit.jupiter.api.Test; -import org.junit.jupiter.params.ParameterizedTest; -import org.junit.jupiter.params.provider.EnumSource; - -import java.lang.foreign.Arena; -import java.lang.foreign.MemorySegment; -import java.nio.ByteBuffer; -import java.nio.ByteOrder; - -import static org.assertj.core.api.Assertions.assertThat; -import static org.assertj.core.api.Assertions.assertThatThrownBy; - -class DateTimePartsEncodingTest { - - private static final DType EXT_TIMESTAMP_MS = timestampDType(TimeUnit.Milliseconds, false); - private static final DType EXT_TIMESTAMP_NS = timestampDType(TimeUnit.Nanoseconds, false); - private static final DType EXT_TIMESTAMP_S = timestampDType(TimeUnit.Seconds, false); - - private static DType timestampDType(TimeUnit unit, boolean nullable) { - // Rust hand-rolled: byte[0]=unit tag, bytes[1-2]=tz_len u16 LE (0 = no tz) - ByteBuffer meta = ByteBuffer.allocate(3).order(ByteOrder.LITTLE_ENDIAN); - meta.put((byte) unit.ordinal()); - meta.putShort((short) 0); // no timezone - meta.flip(); - return new DType.Extension("vortex.timestamp", - new DType.Primitive(PType.I64, nullable), meta, nullable); - } - - private static ArrayNode toArrayNode(EncodeNode node) { - ArrayNode[] children = new ArrayNode[node.children().length]; - for (int i = 0; i < children.length; i++) { - children[i] = toArrayNode(node.children()[i]); - } - return ArrayNode.of(node.encodingId(), node.metadata(), children, node.bufferIndices(), ArrayStats.empty()); - } - - private static EncodingRegistry registry() { - EncodingRegistry r = EncodingRegistry.empty(); - r.register(new DateTimePartsEncoding()); - r.register(new PrimitiveEncoding()); - return r; - } - - @Nested - class Encode { - - @Test - void accepts_extensionDtype_true() { - // Given - DateTimePartsEncoding sut = new DateTimePartsEncoding(); - - // When / Then - assertThat(sut.accepts(EXT_TIMESTAMP_MS)).isTrue(); - } - - @Test - void accepts_primitiveDtype_false() { - // Given - DateTimePartsEncoding sut = new DateTimePartsEncoding(); - - // When / Then - assertThat(sut.accepts(DTypes.I64)).isFalse(); - } - - @Test - void encode_producesThreeChildren_noBuffersAtRoot() { - // Given - long[] timestamps = {0L, 86_400_000L}; - DateTimePartsData data = new DateTimePartsData(timestamps, false); - DateTimePartsEncoding sut = new DateTimePartsEncoding(); - - // When - EncodeResult result = sut.encode(EXT_TIMESTAMP_MS, data); - - // Then - assertThat(result.rootNode().encodingId()).isEqualTo(EncodingId.VORTEX_DATETIMEPARTS); - assertThat(result.rootNode().bufferIndices()).isEmpty(); - assertThat(result.rootNode().children()).hasSize(3); - } - - @Test - void encode_missingMetadata_throws() { - // Given - DType noMeta = new DType.Extension("vortex.timestamp", - new DType.Primitive(PType.I64, false), null, false); - DateTimePartsEncoding sut = new DateTimePartsEncoding(); - DateTimePartsData data = new DateTimePartsData(new long[]{0L}, false); - - // When / Then - assertThatThrownBy(() -> sut.encode(noMeta, data)) - .hasMessageContaining("extension metadata missing"); - } - } - - @Nested - class Decode { - - @Test - void roundTrip_milliseconds_preservesDaysSecondsSubseconds() { - // Given - // 1970-01-02 01:02:03.456 UTC in millis - long msPerDay = 86_400_000L; - long ts = msPerDay + (3723L * 1000L) + 456L; - long[] timestamps = {ts}; - DateTimePartsData data = new DateTimePartsData(timestamps, false); - DateTimePartsEncoding sut = new DateTimePartsEncoding(); - - // When - EncodeResult result = sut.encode(EXT_TIMESTAMP_MS, data); - MemorySegment[] bufs = result.buffers().toArray(MemorySegment[]::new); - DecodeContext ctx = new DecodeContext( - toArrayNode(result.rootNode()), EXT_TIMESTAMP_MS, 1, bufs, registry(), Arena.global()); - GenericArray decoded = (GenericArray) sut.decode(ctx); - - // Then - assertThat(decoded.length()).isEqualTo(1); - LongArray days = (LongArray) decoded.child(0); - LongArray seconds = (LongArray) decoded.child(1); - LongArray subseconds = (LongArray) decoded.child(2); - assertThat(days.getLong(0)).isEqualTo(1L); - assertThat(seconds.getLong(0)).isEqualTo(3723L); - assertThat(subseconds.getLong(0)).isEqualTo(456L); - } - - @Test - void roundTrip_nanoseconds_preservesSubsecondPrecision() { - // Given - // 1970-01-02 01:02:03.456789123 UTC in nanos - long nsPerDay = 86_400_000_000_000L; - long ts = nsPerDay + (3723L * 1_000_000_000L) + 456_789_123L; - long[] timestamps = {ts}; - DateTimePartsData data = new DateTimePartsData(timestamps, false); - DateTimePartsEncoding sut = new DateTimePartsEncoding(); - - // When - EncodeResult result = sut.encode(EXT_TIMESTAMP_NS, data); - MemorySegment[] bufs = result.buffers().toArray(MemorySegment[]::new); - DecodeContext ctx = new DecodeContext( - toArrayNode(result.rootNode()), EXT_TIMESTAMP_NS, 1, bufs, registry(), Arena.global()); - GenericArray decoded = (GenericArray) sut.decode(ctx); - - // Then - LongArray days = (LongArray) decoded.child(0); - LongArray seconds = (LongArray) decoded.child(1); - LongArray subseconds = (LongArray) decoded.child(2); - assertThat(days.getLong(0)).isEqualTo(1L); - assertThat(seconds.getLong(0)).isEqualTo(3723L); - assertThat(subseconds.getLong(0)).isEqualTo(456_789_123L); - } - - @Test - void roundTrip_epoch_allZero() { - // Given - long[] timestamps = {0L}; - DateTimePartsData data = new DateTimePartsData(timestamps, false); - DateTimePartsEncoding sut = new DateTimePartsEncoding(); - - // When - EncodeResult result = sut.encode(EXT_TIMESTAMP_MS, data); - MemorySegment[] bufs = result.buffers().toArray(MemorySegment[]::new); - DecodeContext ctx = new DecodeContext( - toArrayNode(result.rootNode()), EXT_TIMESTAMP_MS, 1, bufs, registry(), Arena.global()); - GenericArray decoded = (GenericArray) sut.decode(ctx); - - // Then - LongArray days = (LongArray) decoded.child(0); - LongArray seconds = (LongArray) decoded.child(1); - LongArray subseconds = (LongArray) decoded.child(2); - assertThat(days.getLong(0)).isEqualTo(0L); - assertThat(seconds.getLong(0)).isEqualTo(0L); - assertThat(subseconds.getLong(0)).isEqualTo(0L); - } - - @Test - void roundTrip_multipleTimestamps_allRowsPreserved() { - // Given - long msPerDay = 86_400_000L; - long[] timestamps = {0L, msPerDay, msPerDay + 1000L, msPerDay + 1001L}; - DateTimePartsData data = new DateTimePartsData(timestamps, false); - DateTimePartsEncoding sut = new DateTimePartsEncoding(); - - // When - EncodeResult result = sut.encode(EXT_TIMESTAMP_MS, data); - MemorySegment[] bufs = result.buffers().toArray(MemorySegment[]::new); - DecodeContext ctx = new DecodeContext( - toArrayNode(result.rootNode()), EXT_TIMESTAMP_MS, 4, bufs, registry(), Arena.global()); - GenericArray decoded = (GenericArray) sut.decode(ctx); - - // Then - assertThat(decoded.length()).isEqualTo(4); - LongArray days = (LongArray) decoded.child(0); - assertThat(days.getLong(0)).isEqualTo(0L); - assertThat(days.getLong(1)).isEqualTo(1L); - assertThat(days.getLong(2)).isEqualTo(1L); - assertThat(days.getLong(3)).isEqualTo(1L); - LongArray subseconds = (LongArray) decoded.child(2); - assertThat(subseconds.getLong(2)).isEqualTo(0L); - assertThat(subseconds.getLong(3)).isEqualTo(1L); - } - - @ParameterizedTest - @EnumSource(value = TimeUnit.class, names = {"Nanoseconds", "Microseconds", "Milliseconds", "Seconds"}) - void roundTrip_allUnits_epochIsZero(TimeUnit unit) { - // Given - DType dtype = timestampDType(unit, false); - long[] timestamps = {0L}; - DateTimePartsData data = new DateTimePartsData(timestamps, false); - DateTimePartsEncoding sut = new DateTimePartsEncoding(); - - // When - EncodeResult result = sut.encode(dtype, data); - MemorySegment[] bufs = result.buffers().toArray(MemorySegment[]::new); - DecodeContext ctx = new DecodeContext( - toArrayNode(result.rootNode()), dtype, 1, bufs, registry(), Arena.global()); - GenericArray decoded = (GenericArray) sut.decode(ctx); - - // Then - LongArray days = (LongArray) decoded.child(0); - LongArray seconds = (LongArray) decoded.child(1); - LongArray subseconds = (LongArray) decoded.child(2); - assertThat(days.getLong(0)).isZero(); - assertThat(seconds.getLong(0)).isZero(); - assertThat(subseconds.getLong(0)).isZero(); - } - } -} diff --git a/core/src/test/java/io/github/dfa1/vortex/encoding/DecimalBytePartsEncodingTest.java b/core/src/test/java/io/github/dfa1/vortex/encoding/DecimalBytePartsEncodingTest.java deleted file mode 100644 index e3f08b8..0000000 --- a/core/src/test/java/io/github/dfa1/vortex/encoding/DecimalBytePartsEncodingTest.java +++ /dev/null @@ -1,75 +0,0 @@ -package io.github.dfa1.vortex.encoding; - -import io.github.dfa1.vortex.core.DType; -import io.github.dfa1.vortex.core.array.Array; -import io.github.dfa1.vortex.proto.EncodingProtos; -import org.junit.jupiter.api.Nested; -import org.junit.jupiter.api.Test; - - -import static org.assertj.core.api.Assertions.assertThat; - -class DecimalBytePartsEncodingTest { - - @Nested - class Encode { - - @Test - void roundTrip_longArray_preservesMspValues() throws Exception { - // Given - long[] values = {1L, -2L, 3L}; - DType dtype = new DType.Decimal((byte) 18, (byte) 0, false); - var sut = new DecimalBytePartsEncoding(); - EncodingRegistry registry = TestRegistry.withPrimitive(sut); - - // When - EncodeResult encoded = sut.encode(dtype, values); - DecodeContext ctx = EncodeTestHelper.toDecodeContext(encoded, values.length, dtype, registry); - Array result = sut.decode(ctx); - - // Then - assertThat(result.length()).isEqualTo(values.length); - Array msp = result.child(0); - assertThat(msp.length()).isEqualTo(values.length); - var le = PTypeIO.LE_LONG; - for (int i = 0; i < values.length; i++) { - assertThat(msp.buffer(0).get(le, (long) i * 8)).isEqualTo(values[i]); - } - } - - @Test - void encodeNode_hasNoBuffers_andOneMspChild() { - // Given - long[] values = {10L, 20L}; - DType dtype = new DType.Decimal((byte) 18, (byte) 0, false); - var sut = new DecimalBytePartsEncoding(); - - // When - EncodeResult result = sut.encode(dtype, values); - - // Then - assertThat(result.rootNode().bufferIndices()).isEmpty(); - assertThat(result.rootNode().children()).hasSize(1); - assertThat(result.buffers()).hasSize(1); - } - - @Test - void metadata_zerothChildPtype_isI64_lowerPartCountIsZero() throws Exception { - // Given - long[] values = {42L}; - DType dtype = new DType.Decimal((byte) 18, (byte) 0, false); - var sut = new DecimalBytePartsEncoding(); - - // When - EncodeResult result = sut.encode(dtype, values); - - // Then - byte[] metaBytes = new byte[result.rootNode().metadata().remaining()]; - result.rootNode().metadata().duplicate().get(metaBytes); - EncodingProtos.DecimalBytePartsMetadata meta = - EncodingProtos.DecimalBytePartsMetadata.parseFrom(metaBytes); - assertThat(meta.getZerothChildPtypeValue()).isEqualTo(7); // I64 ordinal - assertThat(meta.getLowerPartCount()).isEqualTo(0); - } - } -} diff --git a/core/src/test/java/io/github/dfa1/vortex/encoding/DecimalEncodingTest.java b/core/src/test/java/io/github/dfa1/vortex/encoding/DecimalEncodingTest.java deleted file mode 100644 index 14f32e0..0000000 --- a/core/src/test/java/io/github/dfa1/vortex/encoding/DecimalEncodingTest.java +++ /dev/null @@ -1,106 +0,0 @@ -package io.github.dfa1.vortex.encoding; - -import io.github.dfa1.vortex.core.DType; -import io.github.dfa1.vortex.core.PType; -import io.github.dfa1.vortex.core.VortexException; -import io.github.dfa1.vortex.core.array.Array; -import io.github.dfa1.vortex.proto.EncodingProtos; -import org.junit.jupiter.api.Nested; -import org.junit.jupiter.api.Test; -import org.junit.jupiter.params.ParameterizedTest; -import org.junit.jupiter.params.provider.CsvSource; - -import java.lang.foreign.Arena; -import java.lang.foreign.MemorySegment; - -import static org.assertj.core.api.Assertions.assertThat; -import static org.assertj.core.api.Assertions.assertThatThrownBy; - -class DecimalEncodingTest { - - @Nested - class Encode { - - @Test - void roundTrip_i64Precision_preservesBuffer() throws Exception { - // Given - long[] values = {100L, -200L, 300L}; - MemorySegment input = TestSegments.leLongs(values); - DType dtype = new DType.Decimal((byte) 18, (byte) 2, false); - var sut = new DecimalEncoding(); - EncodingRegistry registry = TestRegistry.of(sut); - - // When - EncodeResult encoded = sut.encode(dtype, input); - DecodeContext ctx = EncodeTestHelper.toDecodeContext(encoded, values.length, dtype, registry); - Array result = sut.decode(ctx); - - // Then - assertThat(result.length()).isEqualTo(values.length); - for (int i = 0; i < values.length; i++) { - assertThat(result.buffer(0).get(PTypeIO.LE_LONG, (long) i * 8)).isEqualTo(values[i]); - } - } - - @Test - void accepts_decimalDtype_true_primitiveReturnsFalse() { - // Given - var sut = new DecimalEncoding(); - - // When / Then - assertThat(sut.accepts(new DType.Decimal((byte) 18, (byte) 2, false))).isTrue(); - assertThat(sut.accepts(new DType.Primitive(PType.I64, false))).isFalse(); - } - - @ParameterizedTest(name = "precision={0} → valuesType={1}") - @CsvSource({ - "1, 0", - "2, 0", - "3, 1", - "4, 1", - "5, 2", - "9, 2", - "10, 3", - "18, 3", - "19, 4", - "38, 4", - "39, 5", - }) - void valuesType_matchesPrecisionBoundaries(int precision, int expectedValuesType) throws Exception { - // Given - int byteWidth = switch (expectedValuesType) { - case 0 -> 1; - case 1 -> 2; - case 2 -> 4; - case 3 -> 8; - case 4 -> 16; - default -> 32; - }; - MemorySegment input = Arena.ofAuto().allocate(byteWidth); - DType dtype = new DType.Decimal((byte) precision, (byte) 0, false); - var sut = new DecimalEncoding(); - - // When - EncodeResult encoded = sut.encode(dtype, input); - - // Then - byte[] metaBytes = new byte[encoded.rootNode().metadata().remaining()]; - encoded.rootNode().metadata().duplicate().get(metaBytes); - EncodingProtos.DecimalMetadata meta = EncodingProtos.DecimalMetadata.parseFrom(metaBytes); - assertThat(meta.getValuesType()).isEqualTo(expectedValuesType); - } - - @Test - void invalidBufferSize_throws() { - // Given - MemorySegment input = Arena.ofAuto().allocate(7); // 7 not divisible by 8 (I64) - DType dtype = new DType.Decimal((byte) 18, (byte) 0, false); - var sut = new DecimalEncoding(); - - // When / Then - assertThatThrownBy(() -> sut.encode(dtype, input)) - .isInstanceOf(VortexException.class) - .hasMessageContaining("not multiple of byteWidth"); - } - } -} diff --git a/core/src/test/java/io/github/dfa1/vortex/encoding/DeltaEncodingTest.java b/core/src/test/java/io/github/dfa1/vortex/encoding/DeltaEncodingTest.java deleted file mode 100644 index 67bb174..0000000 --- a/core/src/test/java/io/github/dfa1/vortex/encoding/DeltaEncodingTest.java +++ /dev/null @@ -1,109 +0,0 @@ -package io.github.dfa1.vortex.encoding; - -import io.github.dfa1.vortex.core.array.Array; -import org.junit.jupiter.api.Nested; -import org.junit.jupiter.params.ParameterizedTest; -import org.junit.jupiter.params.provider.Arguments; -import org.junit.jupiter.params.provider.MethodSource; - -import java.util.stream.Stream; - -import static org.assertj.core.api.Assertions.assertThat; - -/// Property: encode then decode is lossless; monotonic sequences compress smaller than raw. -class DeltaEncodingTest { - - @Nested - class Encode { - - @ParameterizedTest - @MethodSource("i64Arrays") - void encodeDecode_i64_isLossless(long[] data) { - // Given - var sut = new DeltaEncoding(); - EncodingRegistry registry = TestRegistry.withPrimitive(sut); - - // When - EncodeResult encoded = sut.encode(DTypes.I64, data); - DecodeContext ctx = EncodeTestHelper.toDecodeContext(encoded, data.length, DTypes.I64, registry); - Array result = sut.decode(ctx); - - // Then - assertThat(result.length()).isEqualTo(data.length); - var le = PTypeIO.LE_LONG; - for (int i = 0; i < data.length; i++) { - assertThat(result.buffer(0).get(le, (long) i * 8)).as("index %d", i).isEqualTo(data[i]); - } - } - - @ParameterizedTest - @MethodSource("i32Arrays") - void encodeDecode_i32_isLossless(int[] data) { - // Given - var sut = new DeltaEncoding(); - EncodingRegistry registry = TestRegistry.withPrimitive(sut); - - // When - EncodeResult encoded = sut.encode(DTypes.I32, data); - DecodeContext ctx = EncodeTestHelper.toDecodeContext(encoded, data.length, DTypes.I32, registry); - Array result = sut.decode(ctx); - - // Then - assertThat(result.length()).isEqualTo(data.length); - var le = PTypeIO.LE_INT; - for (int i = 0; i < data.length; i++) { - assertThat(result.buffer(0).get(le, (long) i * 4)).as("index %d", i).isEqualTo(data[i]); - } - } - - @ParameterizedTest(name = "{0}") - @MethodSource("monotoneI64Arrays") - void encodeDecode_monotoneI64_isLossless(String name, long[] data) { - // Given - var sut = new DeltaEncoding(); - EncodingRegistry registry = TestRegistry.withPrimitive(sut); - - // When - EncodeResult encoded = sut.encode(DTypes.I64, data); - DecodeContext ctx = EncodeTestHelper.toDecodeContext(encoded, data.length, DTypes.I64, registry); - Array result = sut.decode(ctx); - - // Then - assertThat(result.length()).isEqualTo(data.length); - var le = PTypeIO.LE_LONG; - for (int i = 0; i < data.length; i++) { - assertThat(result.buffer(0).get(le, (long) i * 8)).as("index %d", i).isEqualTo(data[i]); - } - } - - static Stream i64Arrays() { - return Stream.of( - new long[]{0}, - new long[]{Long.MIN_VALUE}, - new long[]{0, 1, 2, 3, 4, 5, 6, 7}, - new long[]{100, 200, 300, 400, 500}, - new long[]{-100, -50, 0, 50, 100}, - new long[]{1000, 999, 998, 997, 996} - ); - } - - static Stream i32Arrays() { - return Stream.of( - new int[]{0}, - new int[]{Integer.MIN_VALUE}, - new int[]{0, 1, 2, 3, 4, 5, 6, 7}, - new int[]{10, 20, 30, 40, 50}, - new int[]{-5, -4, -3, -2, -1, 0} - ); - } - - static Stream monotoneI64Arrays() { - return Stream.of( - // strictly monotone with constant delta → very compressible - Arguments.of("ascending-1", new long[]{0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15}), - Arguments.of("ascending-100", new long[]{0, 100, 200, 300, 400, 500, 600, 700, 800, 900}), - Arguments.of("descending", new long[]{1000, 999, 998, 997, 996, 995, 994, 993, 992, 991}) - ); - } - } -} diff --git a/core/src/test/java/io/github/dfa1/vortex/encoding/DictEncodingTest.java b/core/src/test/java/io/github/dfa1/vortex/encoding/DictEncodingTest.java deleted file mode 100644 index f0f5a0b..0000000 --- a/core/src/test/java/io/github/dfa1/vortex/encoding/DictEncodingTest.java +++ /dev/null @@ -1,173 +0,0 @@ -package io.github.dfa1.vortex.encoding; - -import io.github.dfa1.vortex.core.array.Array; -import io.github.dfa1.vortex.core.array.VarBinArray; -import org.junit.jupiter.api.Nested; -import org.junit.jupiter.api.Test; -import org.junit.jupiter.params.ParameterizedTest; -import org.junit.jupiter.params.provider.Arguments; -import org.junit.jupiter.params.provider.MethodSource; - -import java.lang.foreign.MemorySegment; -import java.util.stream.Stream; - -import static org.assertj.core.api.Assertions.assertThat; - -/// Property: encode then decode is lossless; low-cardinality data compresses smaller than raw. -class DictEncodingTest { - - - @Nested - class Encode { - - @ParameterizedTest - @MethodSource("i32Arrays") - void encodeDecode_i32_isLossless(int[] data) { - // Given - var sut = new DictEncoding(); - EncodingRegistry registry = TestRegistry.withPrimitive(sut); - - // When - EncodeResult encoded = sut.encode(DTypes.I32, data); - DecodeContext ctx = EncodeTestHelper.toDecodeContext(encoded, data.length, DTypes.I32, registry); - Array result = sut.decode(ctx); - - // Then - assertThat(result.length()).isEqualTo(data.length); - var le = PTypeIO.LE_INT; - for (int i = 0; i < data.length; i++) { - assertThat(result.buffer(0).get(le, (long) i * 4)).as("index %d", i).isEqualTo(data[i]); - } - } - - @ParameterizedTest - @MethodSource("i64Arrays") - void encodeDecode_i64_isLossless(long[] data) { - // Given - var sut = new DictEncoding(); - EncodingRegistry registry = TestRegistry.withPrimitive(sut); - - // When - EncodeResult encoded = sut.encode(DTypes.I64, data); - DecodeContext ctx = EncodeTestHelper.toDecodeContext(encoded, data.length, DTypes.I64, registry); - Array result = sut.decode(ctx); - - // Then - assertThat(result.length()).isEqualTo(data.length); - var le = PTypeIO.LE_LONG; - for (int i = 0; i < data.length; i++) { - assertThat(result.buffer(0).get(le, (long) i * 8)).as("index %d", i).isEqualTo(data[i]); - } - } - - @ParameterizedTest(name = "{0}") - @MethodSource("repetitiveI32Arrays") - void encodedSize_lowCardinality_compressesWellVsRaw(String name, int[] data) { - // Given - var sut = new DictEncoding(); - - // When - EncodeResult encoded = sut.encode(DTypes.I32, data); - - // Then — dict-encoded size < raw size (n * 4 bytes) - long encodedBytes = encoded.buffers().stream().mapToLong(MemorySegment::byteSize).sum(); - long rawBytes = (long) data.length * 4; - assertThat(encodedBytes).isLessThan(rawBytes); - } - - @ParameterizedTest(name = "{0}") - @MethodSource("utf8Arrays") - void encodeDecode_utf8_isLossless(String name, String[] data) { - // Given - var sut = new DictEncoding(); - EncodingRegistry registry = TestRegistry.withPrimitive(sut); - registry.register(new VarBinEncoding()); - - // When - EncodeResult encoded = sut.encode(DTypes.UTF8, data); - DecodeContext ctx = EncodeTestHelper.toDecodeContext(encoded, data.length, DTypes.UTF8, registry); - Array result = sut.decode(ctx); - - // Then - assertThat(result).isInstanceOf(VarBinArray.class); - VarBinArray arr = (VarBinArray) result; - assertThat(arr.length()).isEqualTo(data.length); - for (int i = 0; i < data.length; i++) { - String actual = arr.getString(i); - assertThat(actual).as("index %d", i).isEqualTo(data[i]); - } - } - - @Test - void encodedSize_lowCardinalityUtf8_compressesWellVsRaw() { - // Given - var sut = new DictEncoding(); - String[] symbols = {"AAPL", "GOOG", "MSFT"}; - String[] data = repeat(symbols, 1000); // 3 000 rows, 3 unique ~4-byte symbols - - // When - EncodeResult encoded = sut.encode(DTypes.UTF8, data); - - // Then — dict overhead << repeating every string verbatim - long encodedBytes = encoded.buffers().stream().mapToLong(MemorySegment::byteSize).sum(); - long rawBytes = 3000L * 5; // avg 5 bytes/symbol (4 chars + 1 for overhead) - assertThat(encodedBytes).isLessThan(rawBytes); - } - - static Stream i32Arrays() { - return Stream.of( - new int[]{0}, - new int[]{1, 2, 3}, - new int[]{0, 1, 2, 0, 1, 2, 0, 1, 2}, - new int[]{42, 42, 42, 42, 42}, - new int[]{Integer.MIN_VALUE, Integer.MAX_VALUE, 0, Integer.MIN_VALUE, Integer.MAX_VALUE} - ); - } - - static Stream i64Arrays() { - return Stream.of( - new long[]{0L}, - new long[]{Long.MIN_VALUE, Long.MAX_VALUE, 0L, Long.MIN_VALUE}, - new long[]{1L, 2L, 1L, 2L, 1L, 2L, 1L, 2L} - ); - } - - static Stream repetitiveI32Arrays() { - return Stream.of( - // 2 unique values repeated many times → codes + dict much smaller than raw - Arguments.of("binary-100", - repeat(new int[]{0, 1}, 50)), - Arguments.of("single-value-50", - repeat(new int[]{42}, 50)), - Arguments.of("three-values-60", - repeat(new int[]{10, 20, 30}, 20)) - ); - } - - static Stream utf8Arrays() { - return Stream.of( - Arguments.of("single", new String[]{"hello"}), - Arguments.of("all-unique", new String[]{"a", "b", "c"}), - Arguments.of("repeated", new String[]{"AAPL", "GOOG", "AAPL", "MSFT", "GOOG", "AAPL"}), - Arguments.of("unicode", new String[]{"café", "naïve", "café", "résumé", "naïve"}), - Arguments.of("empty-string", new String[]{"", "x", "", "y", ""}) - ); - } - - private static int[] repeat(int[] pattern, int times) { - int[] result = new int[pattern.length * times]; - for (int i = 0; i < times; i++) { - System.arraycopy(pattern, 0, result, i * pattern.length, pattern.length); - } - return result; - } - - private static String[] repeat(String[] pattern, int times) { - String[] result = new String[pattern.length * times]; - for (int i = 0; i < times; i++) { - System.arraycopy(pattern, 0, result, i * pattern.length, pattern.length); - } - return result; - } - } -} diff --git a/core/src/test/java/io/github/dfa1/vortex/encoding/EncodeTestHelper.java b/core/src/test/java/io/github/dfa1/vortex/encoding/EncodeTestHelper.java deleted file mode 100644 index f154341..0000000 --- a/core/src/test/java/io/github/dfa1/vortex/encoding/EncodeTestHelper.java +++ /dev/null @@ -1,31 +0,0 @@ -package io.github.dfa1.vortex.encoding; - -import java.lang.foreign.Arena; -import java.lang.foreign.MemorySegment; -import java.util.List; - -/// Converts an [EncodeResult] into a [DecodeContext] for roundtrip tests. -final class EncodeTestHelper { - - private EncodeTestHelper() { - // no instances - } - - static DecodeContext toDecodeContext( - EncodeResult result, long rowCount, io.github.dfa1.vortex.core.DType dtype, - EncodingRegistry registry - ) { - List buffers = result.buffers(); - MemorySegment[] segments = buffers.toArray(new MemorySegment[0]); - ArrayNode root = toArrayNode(result.rootNode()); - return new DecodeContext(root, dtype, rowCount, segments, registry, Arena.ofAuto()); - } - - private static ArrayNode toArrayNode(EncodeNode enc) { - ArrayNode[] children = new ArrayNode[enc.children().length]; - for (int i = 0; i < children.length; i++) { - children[i] = toArrayNode(enc.children()[i]); - } - return ArrayNode.of(enc.encodingId(), enc.metadata(), children, enc.bufferIndices(), null); - } -} diff --git a/core/src/test/java/io/github/dfa1/vortex/encoding/EncodingIdTest.java b/core/src/test/java/io/github/dfa1/vortex/encoding/EncodingIdTest.java deleted file mode 100644 index d6eb179..0000000 --- a/core/src/test/java/io/github/dfa1/vortex/encoding/EncodingIdTest.java +++ /dev/null @@ -1,73 +0,0 @@ -package io.github.dfa1.vortex.encoding; - -import io.github.dfa1.vortex.core.VortexException; -import org.junit.jupiter.api.Nested; -import org.junit.jupiter.api.Test; -import org.junit.jupiter.params.ParameterizedTest; -import org.junit.jupiter.params.provider.EnumSource; - -import static org.assertj.core.api.Assertions.assertThat; -import static org.assertj.core.api.Assertions.assertThatThrownBy; - -class EncodingIdTest { - - @Nested - class From { - - @ParameterizedTest - @EnumSource(EncodingId.class) - void from_knownId_roundTrips(EncodingId id) { - // Given / When - EncodingId result = EncodingId.from(id.id()); - - // Then - assertThat(result).isSameAs(id); - } - - @Test - void from_unknownId_throwsVortexException() { - // Given / When / Then - assertThatThrownBy(() -> EncodingId.from("vortex.does.not.exist")) - .isInstanceOf(VortexException.class) - .hasMessageContaining("vortex.does.not.exist"); - } - } - - @Nested - class TryFrom { - - @ParameterizedTest - @EnumSource(EncodingId.class) - void tryFrom_knownId_returnsMatchingConstant(EncodingId id) { - // Given / When - EncodingId result = EncodingId.tryFrom(id.id()); - - // Then - assertThat(result).isSameAs(id); - } - - @Test - void tryFrom_unknownId_returnsNull() { - // Given / When / Then - assertThat(EncodingId.tryFrom("supermario")).isNull(); - } - } - - @Nested - class Properties { - - @ParameterizedTest - @EnumSource(EncodingId.class) - void id_isNonBlankString(EncodingId id) { - // Given / When / Then - assertThat(id.id()).isNotBlank(); - } - - @ParameterizedTest - @EnumSource(EncodingId.class) - void toString_equalsId(EncodingId id) { - // Given / When / Then - assertThat(id.toString()).isEqualTo(id.id()); - } - } -} diff --git a/core/src/test/java/io/github/dfa1/vortex/encoding/EncodingRegistryTest.java b/core/src/test/java/io/github/dfa1/vortex/encoding/EncodingRegistryTest.java deleted file mode 100644 index fc7dd39..0000000 --- a/core/src/test/java/io/github/dfa1/vortex/encoding/EncodingRegistryTest.java +++ /dev/null @@ -1,160 +0,0 @@ -package io.github.dfa1.vortex.encoding; - -import io.github.dfa1.vortex.core.ArrayStats; -import io.github.dfa1.vortex.core.DType; -import io.github.dfa1.vortex.core.PType; -import io.github.dfa1.vortex.core.VortexException; -import io.github.dfa1.vortex.core.array.Array; -import io.github.dfa1.vortex.core.array.UnknownArray; -import org.junit.jupiter.api.Test; - -import java.lang.foreign.Arena; -import java.lang.foreign.MemorySegment; -import java.nio.ByteBuffer; - -import static org.assertj.core.api.Assertions.assertThat; -import static org.assertj.core.api.Assertions.assertThatThrownBy; - -class EncodingRegistryTest { - - private static final DType I32 = new DType.Primitive(PType.I32, false); - - @Test - void empty() { - // Given - EncodingRegistry sut = EncodingRegistry.empty(); - - // When - boolean result1 = sut.hasEncoding(EncodingId.VORTEX_DECIMAL); - - // Then - assertThat(result1).isFalse(); - - // When - sut.register(new DecimalEncoding()); - boolean result2 = sut.hasEncoding(EncodingId.VORTEX_DECIMAL); - - // Then - assertThat(result2).isTrue(); - } - - @Test - void duplicateIdThrows() { - // Given - EncodingRegistry sut = EncodingRegistry.empty(); - sut.register(new DecimalEncoding()); - - // When / Then - assertThatThrownBy(() -> sut.register(new DecimalEncoding())) - .isInstanceOf(VortexException.class) - .hasMessageContaining("already registered"); - } - - @Test - void all() { - // Given - EncodingRegistry sut = EncodingRegistry.loadAll(); - - // When - for (EncodingId encodingId : EncodingId.values()) { - assertThat(sut.hasEncoding(encodingId)).describedAs("%s".formatted(encodingId)).isTrue(); - } - } - - @Test - void decodeUnknownEncodingThrowsByDefault() { - // Given - EncodingRegistry sut = EncodingRegistry.empty(); - ArrayNode node = new UnknownArrayNode("some.unknown", - ByteBuffer.allocate(0), new ArrayNode[0], new int[0], ArrayStats.empty()); - DecodeContext ctx = new DecodeContext(node, I32, 0L, - new MemorySegment[0], sut, Arena.ofAuto()); - - // When / Then - assertThatThrownBy(() -> sut.decode(ctx)) - .isInstanceOf(VortexException.class) - .hasMessageContaining("some.unknown"); - } - - @Test - void decodeKnownEncodingWithoutDecoderThrowsByDefault() { - // Given — EncodingId is known but no Encoding registered for it - EncodingRegistry sut = EncodingRegistry.empty(); - ArrayNode node = ArrayNode.of(EncodingId.VORTEX_PRIMITIVE, - ByteBuffer.allocate(0), new ArrayNode[0], new int[0], ArrayStats.empty()); - DecodeContext ctx = new DecodeContext(node, I32, 0L, - new MemorySegment[0], sut, Arena.ofAuto()); - - // When / Then - assertThatThrownBy(() -> sut.decode(ctx)) - .isInstanceOf(VortexException.class) - .hasMessageContaining("vortex.primitive"); - } - - @Test - void decodeKnownEncodingWithoutDecoderReturnsUnknownArrayWhenAllowed() { - // Given — EncodingId is known but no Encoding registered; allowUnknown covers this too - EncodingRegistry sut = EncodingRegistry.empty().allowUnknown(); - ArrayNode node = ArrayNode.of(EncodingId.VORTEX_PRIMITIVE, - ByteBuffer.allocate(0), new ArrayNode[0], new int[0], ArrayStats.empty()); - DecodeContext ctx = new DecodeContext(node, I32, 0L, - new MemorySegment[0], sut, Arena.ofAuto()); - - // When - Array result = sut.decode(ctx); - - // Then - assertThat(result).isInstanceOf(UnknownArray.class); - assertThat(((UnknownArray) result).encodingId()).isEqualTo("vortex.primitive"); - } - - @Test - void decodeUnknownEncodingReturnsUnknownArrayWhenAllowed() { - // Given - EncodingRegistry sut = EncodingRegistry.empty().allowUnknown(); - ByteBuffer metadata = ByteBuffer.wrap(new byte[]{1, 2, 3}); - MemorySegment buf = Arena.ofAuto().allocate(4); - buf.set(java.lang.foreign.ValueLayout.JAVA_INT, 0, 42); - ArrayNode node = new UnknownArrayNode("some.unknown", - metadata, new ArrayNode[0], new int[]{0}, ArrayStats.empty()); - DecodeContext ctx = new DecodeContext(node, I32, 5L, - new MemorySegment[]{buf}, sut, Arena.ofAuto()); - - // When - Array result = sut.decode(ctx); - - // Then - assertThat(result).isInstanceOf(UnknownArray.class); - UnknownArray unknown = (UnknownArray) result; - assertThat(unknown.encodingId()).isEqualTo("some.unknown"); - assertThat(unknown.dtype()).isEqualTo(I32); - assertThat(unknown.length()).isEqualTo(5L); - assertThat(unknown.metadata()).isEqualTo(metadata); - assertThat(unknown.buffers()).hasSize(1); - assertThat(unknown.buffers()[0].get(java.lang.foreign.ValueLayout.JAVA_INT, 0)).isEqualTo(42); - assertThat(unknown.children()).isEmpty(); - } - - @Test - void decodeUnknownEncodingWrapsChildrenAsUnknown() { - // Given - EncodingRegistry sut = EncodingRegistry.empty().allowUnknown(); - // Child uses a known id (vortex.primitive); allow-unknown still wraps it unknown because - // its parent is unknown — mirrors Rust decode_foreign in vortex-array/src/serde.rs:380. - ArrayNode child = ArrayNode.of(EncodingId.VORTEX_PRIMITIVE, - ByteBuffer.allocate(0), new ArrayNode[0], new int[0], ArrayStats.empty()); - ArrayNode parent = new UnknownArrayNode("some.unknown", - ByteBuffer.allocate(0), new ArrayNode[]{child}, new int[0], ArrayStats.empty()); - DecodeContext ctx = new DecodeContext(parent, I32, 0L, - new MemorySegment[0], sut, Arena.ofAuto()); - - // When - Array result = sut.decode(ctx); - - // Then - UnknownArray unknown = (UnknownArray) result; - assertThat(unknown.children()).hasSize(1); - assertThat(unknown.children()[0]).isInstanceOf(UnknownArray.class); - assertThat(((UnknownArray) unknown.children()[0]).encodingId()).isEqualTo("vortex.primitive"); - } -} diff --git a/core/src/test/java/io/github/dfa1/vortex/encoding/ExtEncodingTest.java b/core/src/test/java/io/github/dfa1/vortex/encoding/ExtEncodingTest.java deleted file mode 100644 index bfe2197..0000000 --- a/core/src/test/java/io/github/dfa1/vortex/encoding/ExtEncodingTest.java +++ /dev/null @@ -1,116 +0,0 @@ -package io.github.dfa1.vortex.encoding; - -import io.github.dfa1.vortex.core.DType; -import io.github.dfa1.vortex.core.PType; -import io.github.dfa1.vortex.core.array.LongArray; -import org.junit.jupiter.api.Nested; -import org.junit.jupiter.api.Test; - -import java.lang.foreign.Arena; -import java.lang.foreign.MemorySegment; - -import static org.assertj.core.api.Assertions.assertThat; - -class ExtEncodingTest { - - @Nested - class Encode { - - @Test - void accepts_extensionDtype_returnsTrue() { - // Given - DType extDType = new DType.Extension("vortex.timestamp", - new DType.Primitive(PType.I64, false), null, false); - var sut = new ExtEncoding(); - - // When / Then - assertThat(sut.accepts(extDType)).isTrue(); - } - - @Test - void accepts_primitiveDtype_returnsFalse() { - // Given - var sut = new ExtEncoding(); - - // When / Then - assertThat(sut.accepts(new DType.Primitive(PType.I64, false))).isFalse(); - } - - @Test - void encode_extensionWrappingI64_roundTrips() { - // Given - long[] data = {100L, 200L, 300L, 400L}; - DType storageDType = new DType.Primitive(PType.I64, false); - DType extDType = new DType.Extension("vortex.timestamp", storageDType, null, false); - var sut = new ExtEncoding(); - - // When - EncodeResult result = sut.encode(extDType, data); - - // Then — root is ext, child is primitive - assertThat(result.rootNode().encodingId()).isEqualTo(EncodingId.VORTEX_EXT); - assertThat(result.rootNode().children()).hasSize(1); - assertThat(result.rootNode().children()[0].encodingId()).isEqualTo(EncodingId.VORTEX_PRIMITIVE); - - // Decode back - EncodingRegistry registry = TestRegistry.of(new PrimitiveEncoding(), new ExtEncoding()); - ArrayNode rootNode = encodeNodeToArrayNode(result.rootNode()); - DecodeContext ctx = new DecodeContext( - rootNode, extDType, data.length, - result.buffers().toArray(MemorySegment[]::new), - registry, Arena.ofAuto()); - var decoded = sut.decode(ctx); - - assertThat(decoded).isInstanceOf(LongArray.class); - for (int i = 0; i < data.length; i++) { - LongArray longArray = (LongArray) decoded; - assertThat(longArray.getLong(i)).isEqualTo(data[i]); - } - } - - private ArrayNode encodeNodeToArrayNode(EncodeNode n) { - ArrayNode[] children = new ArrayNode[n.children().length]; - for (int i = 0; i < children.length; i++) { - children[i] = encodeNodeToArrayNode(n.children()[i]); - } - return ArrayNode.of(n.encodingId(), n.metadata(), children, n.bufferIndices(), null); - } - } - - @Nested - class Decode { - - @Test - void decode_extensionWrappingI64_returnsStorageArray() { - // Given - long[] values = {10L, 20L, 30L, 40L}; - MemorySegment buf = TestSegments.leLongs(values); - - DType storageDType = new DType.Primitive(PType.I64, false); - DType extDType = new DType.Extension("vortex.timestamp", storageDType, null, false); - - // child node: vortex.primitive with buffer index 0 - ArrayNode primitiveNode = ArrayNode.of(EncodingId.VORTEX_PRIMITIVE, null, new ArrayNode[0], new int[]{0}, null); - // parent node: vortex.ext, no buffers, one child - ArrayNode extNode = ArrayNode.of(EncodingId.VORTEX_EXT, null, new ArrayNode[]{primitiveNode}, new int[0], null); - - EncodingRegistry registry = TestRegistry.of(new PrimitiveEncoding(), new ExtEncoding()); - - DecodeContext ctx = new DecodeContext( - extNode, extDType, values.length, new MemorySegment[]{buf}, registry, Arena.ofAuto()); - - var sut = new ExtEncoding(); - - // When - var result = sut.decode(ctx); - - // Then - assertThat(result).isInstanceOf(LongArray.class); - assertThat(result.length()).isEqualTo((long) values.length); - for (int i = 0; i < values.length; i++) { - LongArray longArray = (LongArray) result; - assertThat(longArray.getLong(i)).isEqualTo(values[i]); - } - } - } -} diff --git a/core/src/test/java/io/github/dfa1/vortex/encoding/FixedSizeListEncodingTest.java b/core/src/test/java/io/github/dfa1/vortex/encoding/FixedSizeListEncodingTest.java deleted file mode 100644 index 521f97b..0000000 --- a/core/src/test/java/io/github/dfa1/vortex/encoding/FixedSizeListEncodingTest.java +++ /dev/null @@ -1,139 +0,0 @@ -package io.github.dfa1.vortex.encoding; - -import io.github.dfa1.vortex.core.ArrayStats; -import io.github.dfa1.vortex.core.DType; -import io.github.dfa1.vortex.core.array.FixedSizeListArray; -import io.github.dfa1.vortex.core.array.IntArray; -import org.junit.jupiter.api.Nested; -import org.junit.jupiter.api.Test; - -import java.lang.foreign.Arena; -import java.lang.foreign.MemorySegment; - -import static org.assertj.core.api.Assertions.assertThat; -import static org.assertj.core.api.Assertions.assertThatThrownBy; - -class FixedSizeListEncodingTest { - - - private static ArrayNode toArrayNode(EncodeNode node) { - ArrayNode[] children = new ArrayNode[node.children().length]; - for (int i = 0; i < children.length; i++) { - children[i] = toArrayNode(node.children()[i]); - } - return ArrayNode.of(node.encodingId(), node.metadata(), children, node.bufferIndices(), ArrayStats.empty()); - } - - private static EncodingRegistry registry() { - EncodingRegistry r = EncodingRegistry.empty(); - r.register(new FixedSizeListEncoding()); - r.register(new PrimitiveEncoding()); - return r; - } - - @Nested - class Encode { - - @Test - void accepts_fixedSizeListDtype_true() { - // Given - FixedSizeListEncoding sut = new FixedSizeListEncoding(); - DType.FixedSizeList dtype = new DType.FixedSizeList(DTypes.I32, 3, false); - - // When / Then - assertThat(sut.accepts(dtype)).isTrue(); - } - - @Test - void accepts_primitiveDtype_false() { - // Given - FixedSizeListEncoding sut = new FixedSizeListEncoding(); - - // When / Then - assertThat(sut.accepts(DTypes.I32)).isFalse(); - } - - @Test - void encode_producesOneChild_noBuffers() { - // Given - DType.FixedSizeList dtype = new DType.FixedSizeList(DTypes.I32, 2, false); - int[] elements = {1, 2, 3, 4}; - FixedSizeListData data = new FixedSizeListData(elements, 2); - FixedSizeListEncoding sut = new FixedSizeListEncoding(); - - // When - EncodeResult result = sut.encode(dtype, data); - - // Then - assertThat(result.rootNode().encodingId()).isEqualTo(EncodingId.VORTEX_FIXED_SIZE_LIST); - assertThat(result.rootNode().bufferIndices()).isEmpty(); - assertThat(result.rootNode().children()).hasSize(1); - } - } - - @Nested - class Decode { - - @Test - void roundTrip_i32Elements_preservesValues() { - // Given - DType.FixedSizeList dtype = new DType.FixedSizeList(DTypes.I32, 3, false); - int[] elements = {10, 20, 30, 40, 50, 60}; - FixedSizeListData data = new FixedSizeListData(elements, 2); - FixedSizeListEncoding sut = new FixedSizeListEncoding(); - - // When - EncodeResult result = sut.encode(dtype, data); - MemorySegment[] bufs = result.buffers().toArray(MemorySegment[]::new); - DecodeContext ctx = new DecodeContext( - toArrayNode(result.rootNode()), dtype, 2, bufs, registry(), Arena.global()); - FixedSizeListArray decoded = (FixedSizeListArray) sut.decode(ctx); - - // Then - assertThat(decoded.length()).isEqualTo(2); - assertThat(decoded.fixedSize()).isEqualTo(3); - IntArray elems = (IntArray) decoded.elements(); - assertThat(elems.length()).isEqualTo(6); - for (int i = 0; i < elements.length; i++) { - assertThat(elems.getInt(i)).isEqualTo(elements[i]); - } - } - - @Test - void roundTrip_fixedSizeOne_preservesValues() { - // Given - DType.FixedSizeList dtype = new DType.FixedSizeList(DTypes.I32, 1, false); - int[] elements = {7, 8, 9}; - FixedSizeListData data = new FixedSizeListData(elements, 3); - FixedSizeListEncoding sut = new FixedSizeListEncoding(); - - // When - EncodeResult result = sut.encode(dtype, data); - MemorySegment[] bufs = result.buffers().toArray(MemorySegment[]::new); - DecodeContext ctx = new DecodeContext( - toArrayNode(result.rootNode()), dtype, 3, bufs, registry(), Arena.global()); - FixedSizeListArray decoded = (FixedSizeListArray) sut.decode(ctx); - - // Then - assertThat(decoded.length()).isEqualTo(3); - assertThat(decoded.fixedSize()).isEqualTo(1); - IntArray elems = (IntArray) decoded.elements(); - for (int i = 0; i < elements.length; i++) { - assertThat(elems.getInt(i)).isEqualTo(elements[i]); - } - } - - @Test - void decode_wrongDtype_throws() { - // Given - FixedSizeListEncoding sut = new FixedSizeListEncoding(); - ArrayNode node = ArrayNode.of(EncodingId.VORTEX_FIXED_SIZE_LIST, null, - new ArrayNode[0], new int[0], ArrayStats.empty()); - DecodeContext ctx = new DecodeContext(node, DTypes.I32, 0, new MemorySegment[0], registry(), Arena.global()); - - // When / Then - assertThatThrownBy(() -> sut.decode(ctx)) - .hasMessageContaining("DType.FixedSizeList"); - } - } -} diff --git a/core/src/test/java/io/github/dfa1/vortex/encoding/FrameOfReferenceEncodingTest.java b/core/src/test/java/io/github/dfa1/vortex/encoding/FrameOfReferenceEncodingTest.java deleted file mode 100644 index 673fd77..0000000 --- a/core/src/test/java/io/github/dfa1/vortex/encoding/FrameOfReferenceEncodingTest.java +++ /dev/null @@ -1,257 +0,0 @@ -package io.github.dfa1.vortex.encoding; - -import io.github.dfa1.vortex.proto.ScalarProtos; -import io.github.dfa1.vortex.core.array.Array; -import io.github.dfa1.vortex.core.ArrayStats; -import io.github.dfa1.vortex.core.DType; -import io.github.dfa1.vortex.core.PType; -import io.github.dfa1.vortex.core.array.MaskedArray; -import org.junit.jupiter.api.Nested; -import org.junit.jupiter.api.Test; -import org.junit.jupiter.params.ParameterizedTest; -import org.junit.jupiter.params.provider.MethodSource; -import org.junit.jupiter.params.provider.ValueSource; - -import java.lang.foreign.MemorySegment; -import java.nio.ByteBuffer; -import java.nio.ByteOrder; -import java.util.stream.Stream; - -import static org.assertj.core.api.Assertions.assertThat; - -class FrameOfReferenceEncodingTest { - - - @Nested - class Decode { - - @Test - void decode_i64_addsReferenceToResiduals() { - // Given - long reference = 1000L; - long[] residuals = {0, 1, 2, 3, 4}; - long[] expected = {1000, 1001, 1002, 1003, 1004}; - - DecodeContext ctx = buildForContext(DTypes.I64, reference, residuals, PType.I64); - FrameOfReferenceEncoding sut = new FrameOfReferenceEncoding(); - - // When - Array result = sut.decode(ctx); - - // Then - assertThat(result.length()).isEqualTo(residuals.length); - var layout = PTypeIO.LE_LONG; - for (int i = 0; i < expected.length; i++) { - assertThat(result.buffer(0).get(layout, (long) i * 8)) - .as("index %d", i) - .isEqualTo(expected[i]); - } - } - - @Test - void decode_i32_addsReferenceToResiduals() { - // Given - long reference = -100L; - long[] residuals = {0, 5, 10, 15}; - int[] expected = {-100, -95, -90, -85}; - - DecodeContext ctx = buildForContext(DTypes.I32, reference, residuals, PType.I32); - FrameOfReferenceEncoding sut = new FrameOfReferenceEncoding(); - - // When - Array result = sut.decode(ctx); - - // Then - assertThat(result.length()).isEqualTo(residuals.length); - var layout = PTypeIO.LE_INT; - for (int i = 0; i < expected.length; i++) { - assertThat(result.buffer(0).get(layout, (long) i * 4)) - .as("index %d", i) - .isEqualTo(expected[i]); - } - } - - @Test - void decode_zeroReference_returnsChildUnchanged() { - // Given — reference == 0, should skip the add entirely - long[] residuals = {7, 8, 9}; - DecodeContext ctx = buildForContext(DTypes.I64, 0L, residuals, PType.I64); - FrameOfReferenceEncoding sut = new FrameOfReferenceEncoding(); - - // When - Array result = sut.decode(ctx); - - // Then — values unchanged - var layout = PTypeIO.LE_LONG; - for (int i = 0; i < residuals.length; i++) { - assertThat(result.buffer(0).get(layout, (long) i * 8)).isEqualTo(residuals[i]); - } - } - - @ParameterizedTest - @ValueSource(longs = {Long.MIN_VALUE, Long.MAX_VALUE, -1L, 1L}) - void decode_wrappingAdd_i64(long reference) { - // Given — wrapping arithmetic: MAX + 1 wraps to MIN - long[] residuals = {1L}; - DecodeContext ctx = buildForContext(DTypes.I64, reference, residuals, PType.I64); - FrameOfReferenceEncoding sut = new FrameOfReferenceEncoding(); - - // When - Array result = sut.decode(ctx); - - // Then - var layout = PTypeIO.LE_LONG; - long got = result.buffer(0).get(layout, 0L); - assertThat(got).isEqualTo(residuals[0] + reference); - } - - @Test - void decode_nullableResiduals_returnsMaskedArrayWithCorrectValues() { - // Given — 4 I32 residuals; positions 1 and 3 are null (validity: 0b00000101 = 0x05) - // Residuals: [0, 0, 5, 0], reference: 100 → valid outputs: [100, ?, 105, ?] - long reference = 100L; - long[] residuals = {0, 0, 5, 0}; - MemorySegment validitySeg = MemorySegment.ofArray(new byte[]{0x05}); // bits 0,2 - - byte[] residualBytes = new byte[residuals.length * 4]; - ByteBuffer bb = ByteBuffer.wrap(residualBytes).order(ByteOrder.LITTLE_ENDIAN); - for (long v : residuals) { - bb.putInt((int) v); - } - - ArrayNode validityNode = ArrayNode.of( - EncodingId.VORTEX_BOOL, null, new ArrayNode[0], new int[]{1}, ArrayStats.empty()); - ArrayNode primNode = ArrayNode.of( - EncodingId.VORTEX_PRIMITIVE, null, new ArrayNode[]{validityNode}, new int[]{0}, ArrayStats.empty()); - byte[] metaBytes = ScalarProtos.ScalarValue.newBuilder().setInt64Value(reference).build().toByteArray(); - ArrayNode forNode = ArrayNode.of( - EncodingId.FASTLANES_FOR, ByteBuffer.wrap(metaBytes), new ArrayNode[]{primNode}, new int[0], ArrayStats.empty()); - - EncodingRegistry registry = TestRegistry.of(new FrameOfReferenceEncoding(), new PrimitiveEncoding(), new BoolEncoding()); - - MemorySegment[] segments = {MemorySegment.ofArray(residualBytes), validitySeg}; - DecodeContext ctx = new DecodeContext( - forNode, DTypes.I32, residuals.length, segments, registry, java.lang.foreign.Arena.global()); - FrameOfReferenceEncoding sut = new FrameOfReferenceEncoding(); - - // When - Array result = sut.decode(ctx); - - // Then — MaskedArray; reference added to valid positions only - assertThat(result).isInstanceOf(MaskedArray.class); - MaskedArray masked = (MaskedArray) result; - assertThat(masked.isValid(0)).isTrue(); - assertThat(masked.isValid(1)).isFalse(); - assertThat(masked.isValid(2)).isTrue(); - assertThat(masked.isValid(3)).isFalse(); - var layout = PTypeIO.LE_INT; - assertThat(masked.child(0).buffer(0).get(layout, 0L)).isEqualTo(100); - assertThat(masked.child(0).buffer(0).get(layout, 8L)).isEqualTo(105); - } - - private static DecodeContext buildForContext( - DType dtype, long reference, long[] residuals, PType ptype - ) { - byte[] metaBytes = ScalarProtos.ScalarValue.newBuilder() - .setInt64Value(reference) - .build() - .toByteArray(); - - int elemBytes = ptype.byteSize(); - byte[] childBytes = new byte[residuals.length * elemBytes]; - ByteBuffer bb = ByteBuffer.wrap(childBytes).order(ByteOrder.LITTLE_ENDIAN); - for (long v : residuals) { - switch (ptype) { - case I32, U32 -> bb.putInt((int) v); - case I64, U64 -> bb.putLong(v); - default -> throw new UnsupportedOperationException(ptype.name()); - } - } - - ArrayNode childNode = ArrayNode.of( - EncodingId.VORTEX_PRIMITIVE, - null, - new ArrayNode[0], - new int[]{0}, - ArrayStats.empty() - ); - - ArrayNode forNode = ArrayNode.of( - EncodingId.FASTLANES_FOR, - ByteBuffer.wrap(metaBytes), - new ArrayNode[]{childNode}, - new int[0], - ArrayStats.empty() - ); - - MemorySegment[] segments = {MemorySegment.ofArray(childBytes)}; - - EncodingRegistry registry = TestRegistry.of(new FrameOfReferenceEncoding(), new PrimitiveEncoding()); - - return new DecodeContext(forNode, dtype, residuals.length, segments, registry, java.lang.foreign.Arena.global()); - } - } - - @Nested - class Encode { - - @ParameterizedTest - @MethodSource("i64Arrays") - void encodeDecode_i64_isLossless(long[] data) { - // Given - var sut = new FrameOfReferenceEncoding(); - EncodingRegistry registry = TestRegistry.withPrimitive(sut); - var le = PTypeIO.LE_LONG; - - // When - EncodeResult encoded = sut.encode(DTypes.I64, data); - DecodeContext ctx = EncodeTestHelper.toDecodeContext(encoded, data.length, DTypes.I64, registry); - Array result = sut.decode(ctx); - - // Then - assertThat(result.length()).isEqualTo(data.length); - for (int i = 0; i < data.length; i++) { - assertThat(result.buffer(0).get(le, (long) i * 8)).as("index %d", i).isEqualTo(data[i]); - } - } - - @ParameterizedTest - @MethodSource("i32Arrays") - void encodeDecode_i32_isLossless(int[] data) { - // Given - var sut = new FrameOfReferenceEncoding(); - EncodingRegistry registry = TestRegistry.withPrimitive(sut); - var le = PTypeIO.LE_INT; - - // When - EncodeResult encoded = sut.encode(DTypes.I32, data); - DecodeContext ctx = EncodeTestHelper.toDecodeContext(encoded, data.length, DTypes.I32, registry); - Array result = sut.decode(ctx); - - // Then - assertThat(result.length()).isEqualTo(data.length); - for (int i = 0; i < data.length; i++) { - assertThat(result.buffer(0).get(le, (long) i * 4)).as("index %d", i).isEqualTo(data[i]); - } - } - - static Stream i64Arrays() { - return Stream.of( - new long[]{0L}, - new long[]{1000L, 1001L, 1002L, 1003L}, - new long[]{-500L, -499L, -498L}, - new long[]{Long.MIN_VALUE, Long.MIN_VALUE + 1L, Long.MIN_VALUE + 2L}, - new long[]{42L, 42L, 42L} - ); - } - - static Stream i32Arrays() { - return Stream.of( - new int[]{0}, - new int[]{100, 101, 102, 103}, - new int[]{-10, -9, -8, -7}, - new int[]{Integer.MIN_VALUE, Integer.MIN_VALUE + 1} - ); - } - } -} diff --git a/core/src/test/java/io/github/dfa1/vortex/encoding/FsstEncodingTest.java b/core/src/test/java/io/github/dfa1/vortex/encoding/FsstEncodingTest.java deleted file mode 100644 index e6164f0..0000000 --- a/core/src/test/java/io/github/dfa1/vortex/encoding/FsstEncodingTest.java +++ /dev/null @@ -1,285 +0,0 @@ -package io.github.dfa1.vortex.encoding; - -import io.github.dfa1.vortex.core.DType; -import io.github.dfa1.vortex.core.PType; -import io.github.dfa1.vortex.core.VortexException; -import io.github.dfa1.vortex.core.array.VarBinArray; -import io.github.dfa1.vortex.proto.DTypeProtos; -import io.github.dfa1.vortex.proto.EncodingProtos; -import org.junit.jupiter.api.Nested; -import org.junit.jupiter.api.Test; -import org.junit.jupiter.params.ParameterizedTest; -import org.junit.jupiter.params.provider.Arguments; -import org.junit.jupiter.params.provider.MethodSource; - -import java.lang.foreign.Arena; -import java.lang.foreign.MemorySegment; -import java.lang.foreign.ValueLayout; -import java.nio.ByteBuffer; -import java.nio.charset.StandardCharsets; -import java.util.stream.Stream; - -import static org.assertj.core.api.Assertions.assertThat; -import static org.assertj.core.api.Assertions.assertThatThrownBy; - -class FsstEncodingTest { - - private static final DType UTF8 = new DType.Utf8(false); - private static final DType BINARY = new DType.Binary(false); - private static final DType I32 = new DType.Primitive(PType.I32, false); - - @Nested - class Encode { - - @Test - void accepts_utf8_true() { - // Given - var sut = new FsstEncoding(); - - // When / Then - assertThat(sut.accepts(UTF8)).isTrue(); - } - - @Test - void accepts_binary_true() { - // Given - var sut = new FsstEncoding(); - - // When / Then - assertThat(sut.accepts(BINARY)).isTrue(); - } - - @Test - void accepts_primitive_false() { - // Given - var sut = new FsstEncoding(); - - // When / Then - assertThat(sut.accepts(I32)).isFalse(); - } - - @ParameterizedTest(name = "{0}") - @MethodSource("io.github.dfa1.vortex.encoding.FsstEncodingTest$Encode#stringArrays") - void encode_thenDecode_roundtripsAllStrings(String name, String[] values) { - // Given - var sut = new FsstEncoding(); - Arena arena = Arena.ofAuto(); - - // When - EncodeResult result = sut.encode(UTF8, values); - MemorySegment[] bufs = result.buffers().toArray(MemorySegment[]::new); - ArrayNode node = toArrayNode(result.rootNode()); - EncodingRegistry registry = EncodingRegistry.empty(); - registry.register(new PrimitiveEncoding()); - registry.register(sut); - DecodeContext ctx = new DecodeContext(node, UTF8, values.length, bufs, registry, arena); - var decoded = (VarBinArray) sut.decode(ctx); - - // Then - assertThat(decoded.length()).isEqualTo(values.length); - for (int i = 0; i < values.length; i++) { - assertThat(decoded.getString(i)).as("index %d", i).isEqualTo(values[i]); - } - } - - private static ArrayNode toArrayNode(EncodeNode node) { - ArrayNode[] children = new ArrayNode[node.children().length]; - for (int i = 0; i < children.length; i++) { - children[i] = toArrayNode(node.children()[i]); - } - return ArrayNode.of(node.encodingId(), node.metadata(), children, node.bufferIndices(), null); - } - - static Stream stringArrays() { - return Stream.of( - Arguments.of("empty-array", new String[0]), - Arguments.of("single-empty-string", new String[]{""}), - Arguments.of("short-strings", new String[]{"hi", "ok", "no"}), - Arguments.of("repeated-bigram", new String[]{"aaaa", "aaaa", "aaaa"}), - Arguments.of("long-strings", new String[]{"the quick brown fox jumps over the lazy dog"}), - Arguments.of("mixed-lengths", new String[]{"a", "hello", "this is a longer string than twelve"}), - Arguments.of("repeated-short", repeat("ab", 50)), - Arguments.of("all-empty", new String[]{"", "", "", ""}), - Arguments.of("unicode", new String[]{"héllo", "wörld", "こんにちは"}) - ); - } - - private static String[] repeat(String s, int n) { - String[] arr = new String[n]; - java.util.Arrays.fill(arr, s); - return arr; - } - } - - @Nested - class Decode { - - @Test - void decode_singleByteSymbol_decompressesCorrectly() { - // Given: symbol 0 = 'A' (LE u64 = 0x41, length 1); string "AA" → codes [0, 0] - var sut = new FsstEncoding(); - DecodeContext ctx = buildCtx( - new long[]{0x41L}, - new byte[]{1}, - new byte[]{0x00, 0x00}, - new int[]{2}, - new int[]{0, 2}, - 1 - ); - - // When - var result = sut.decode(ctx); - - // Then - assertThat(result).isInstanceOf(VarBinArray.class); - VarBinArray vba = (VarBinArray) result; - assertThat(vba.length()).isEqualTo(1); - assertThat(vba.getBytes(0)).isEqualTo("AA".getBytes(StandardCharsets.UTF_8)); - } - - @Test - void decode_escapeByte_decompressesCorrectly() { - // Given: no symbols; string "XY" → ESCAPE 'X' ESCAPE 'Y' - var sut = new FsstEncoding(); - DecodeContext ctx = buildCtx( - new long[0], - new byte[0], - new byte[]{(byte) 0xFF, 0x58, (byte) 0xFF, 0x59}, - new int[]{2}, - new int[]{0, 4}, - 1 - ); - - // When - var result = sut.decode(ctx); - - // Then - assertThat(result).isInstanceOf(VarBinArray.class); - VarBinArray vba = (VarBinArray) result; - assertThat(vba.length()).isEqualTo(1); - assertThat(vba.getBytes(0)).isEqualTo("XY".getBytes(StandardCharsets.UTF_8)); - } - - @Test - void decode_multiByteSymbol_decompressesCorrectly() { - // Given: symbol 0 = "ab" (LE u64 = 0x6261, length 2); string "ab" → code [0] - var sut = new FsstEncoding(); - DecodeContext ctx = buildCtx( - new long[]{0x6261L}, - new byte[]{2}, - new byte[]{0x00}, - new int[]{2}, - new int[]{0, 1}, - 1 - ); - - // When - var result = sut.decode(ctx); - - // Then - assertThat(result).isInstanceOf(VarBinArray.class); - VarBinArray vba = (VarBinArray) result; - assertThat(vba.length()).isEqualTo(1); - assertThat(vba.getBytes(0)).isEqualTo("ab".getBytes(StandardCharsets.UTF_8)); - } - - @Test - void decode_multipleStrings_decompressesAll() { - // Given: symbol 0 = 'H'; strings ["H", "HH", "!"] where "!" uses ESCAPE - // compressed: [0x00] | [0x00, 0x00] | [0xFF, 0x21] - var sut = new FsstEncoding(); - DecodeContext ctx = buildCtx( - new long[]{0x48L}, - new byte[]{1}, - new byte[]{0x00, 0x00, 0x00, (byte) 0xFF, 0x21}, - new int[]{1, 2, 1}, - new int[]{0, 1, 3, 5}, - 3 - ); - - // When - var result = sut.decode(ctx); - - // Then - assertThat(result).isInstanceOf(VarBinArray.class); - VarBinArray vba = (VarBinArray) result; - assertThat(vba.length()).isEqualTo(3); - assertThat(vba.getBytes(0)).isEqualTo("H".getBytes(StandardCharsets.UTF_8)); - assertThat(vba.getBytes(1)).isEqualTo("HH".getBytes(StandardCharsets.UTF_8)); - assertThat(vba.getBytes(2)).isEqualTo("!".getBytes(StandardCharsets.UTF_8)); - } - - @Test - void decode_missingMetadata_throwsVortexException() { - // Given - var sut = new FsstEncoding(); - ArrayNode node = ArrayNode.of(EncodingId.VORTEX_FSST, null, new ArrayNode[0], new int[0], null); - DecodeContext ctx = new DecodeContext(node, UTF8, 0, new MemorySegment[0], - buildRegistry(), Arena.ofAuto()); - - // When / Then - assertThatThrownBy(() -> sut.decode(ctx)) - .isInstanceOf(VortexException.class); - } - - private static DecodeContext buildCtx( - long[] symbols, byte[] symLens, byte[] compressed, - int[] uncompLens, int[] codesOffsets, long n - ) { - Arena arena = Arena.ofAuto(); - - // Buffer 0: symbol table (8 bytes per symbol, LE u64) - MemorySegment symBuf = arena.allocate(Math.max(symbols.length * 8L, 1), 8); - for (int i = 0; i < symbols.length; i++) { - symBuf.setAtIndex(PTypeIO.LE_LONG, i, symbols[i]); - } - - // Buffer 1: symbol lengths (1 byte per symbol) - MemorySegment symLenBuf = arena.allocate(Math.max(symLens.length, 1)); - for (int i = 0; i < symLens.length; i++) { - symLenBuf.set(ValueLayout.JAVA_BYTE, i, symLens[i]); - } - - // Buffer 2: compressed bytes - MemorySegment compBuf = arena.allocate(Math.max(compressed.length, 1)); - for (int i = 0; i < compressed.length; i++) { - compBuf.set(ValueLayout.JAVA_BYTE, i, compressed[i]); - } - - // Buffer 3: uncompressed_lengths (I32) - MemorySegment uncompLenBuf = arena.allocate((long) uncompLens.length * Integer.BYTES, Integer.BYTES); - for (int i = 0; i < uncompLens.length; i++) { - uncompLenBuf.setAtIndex(PTypeIO.LE_INT, i, uncompLens[i]); - } - - // Buffer 4: codes_offsets (I32, n+1 elements) - MemorySegment codesOffBuf = arena.allocate((long) codesOffsets.length * Integer.BYTES, Integer.BYTES); - for (int i = 0; i < codesOffsets.length; i++) { - codesOffBuf.setAtIndex(PTypeIO.LE_INT, i, codesOffsets[i]); - } - - MemorySegment[] segs = {symBuf, symLenBuf, compBuf, uncompLenBuf, codesOffBuf}; - - byte[] metaBytes = EncodingProtos.FSSTMetadata.newBuilder() - .setUncompressedLengthsPtype(DTypeProtos.PType.forNumber(PType.I32.ordinal())) - .setCodesOffsetsPtype(DTypeProtos.PType.forNumber(PType.I32.ordinal())) - .build().toByteArray(); - - ArrayNode uncompLensNode = ArrayNode.of( - EncodingId.VORTEX_PRIMITIVE, null, new ArrayNode[0], new int[]{3}, null); - ArrayNode codesOffNode = ArrayNode.of( - EncodingId.VORTEX_PRIMITIVE, null, new ArrayNode[0], new int[]{4}, null); - ArrayNode root = ArrayNode.of( - EncodingId.VORTEX_FSST, ByteBuffer.wrap(metaBytes), - new ArrayNode[]{uncompLensNode, codesOffNode}, new int[]{0, 1, 2}, null); - - return new DecodeContext(root, UTF8, n, segs, buildRegistry(), arena); - } - - private static EncodingRegistry buildRegistry() { - EncodingRegistry registry = EncodingRegistry.empty(); - registry.register(new PrimitiveEncoding()); - return registry; - } - } -} diff --git a/core/src/test/java/io/github/dfa1/vortex/encoding/LeBitReaderTest.java b/core/src/test/java/io/github/dfa1/vortex/encoding/LeBitReaderTest.java deleted file mode 100644 index 094dcd6..0000000 --- a/core/src/test/java/io/github/dfa1/vortex/encoding/LeBitReaderTest.java +++ /dev/null @@ -1,225 +0,0 @@ -package io.github.dfa1.vortex.encoding; - -import org.junit.jupiter.api.Nested; -import org.junit.jupiter.api.Test; -import org.junit.jupiter.params.ParameterizedTest; -import org.junit.jupiter.params.provider.CsvSource; - -import java.lang.foreign.Arena; -import java.lang.foreign.MemorySegment; -import java.lang.foreign.ValueLayout; - -import io.github.dfa1.vortex.core.VortexException; - -import static org.assertj.core.api.Assertions.assertThat; -import static org.assertj.core.api.Assertions.assertThatThrownBy; - -class LeBitReaderTest { - - private static LeBitReader readerOf(byte... bytes) { - try (Arena arena = Arena.ofConfined()) { - MemorySegment seg = arena.allocate(bytes.length); - for (int i = 0; i < bytes.length; i++) { - seg.set(ValueLayout.JAVA_BYTE, i, bytes[i]); - } - // Use auto arena so segment outlives this call - MemorySegment copy = Arena.ofAuto().allocate(bytes.length); - MemorySegment.copy(seg, 0, copy, 0, bytes.length); - return new LeBitReader(copy); - } - } - - @Nested - class ReadBits { - - @Test - void zero_bits_returns_zero() { - // Given - var sut = readerOf((byte) 0xFF); - - // When - long result = sut.readBits(0); - - // Then - assertThat(result).isZero(); - assertThat(sut.bitsConsumed()).isZero(); - } - - @Test - void single_byte_low_nibble() { - // Given — 0b10110011 = 0xB3 - var sut = readerOf((byte) 0xB3); - - // When - long result = sut.readBits(4); - - // Then — LSB first: bits [3:0] = 0b0011 = 3 - assertThat(result).isEqualTo(3L); - assertThat(sut.bitsConsumed()).isEqualTo(4); - } - - @Test - void single_byte_high_nibble() { - // Given — 0xB3 = 0b10110011 - var sut = readerOf((byte) 0xB3); - - // When — skip low nibble, read high nibble - sut.readBits(4); - long result = sut.readBits(4); - - // Then — bits [7:4] = 0b1011 = 11 - assertThat(result).isEqualTo(11L); - } - - @Test - void full_byte() { - // Given - var sut = readerOf((byte) 0xA7); - - // When - long result = sut.readBits(8); - - // Then - assertThat(result).isEqualTo(0xA7L); - } - - @Test - void cross_byte_boundary() { - // Given — bytes: 0b11110000, 0b00001111 = 0xF0, 0x0F - var sut = readerOf((byte) 0xF0, (byte) 0x0F); - - // When — read 4 bits (high nibble of 0xF0 = 0b1111=0xF), then 4 bits (low nibble of 0x0F = 0b1111=0xF... wait) - // Actually: 0xF0 = 0b11110000. LSB-first: bits[0..3]=0b0000=0, bits[4..7]=0b1111=15 - // 0x0F = 0b00001111. bits[8..11]=0b1111=15, bits[12..15]=0b0000=0 - sut.readBits(4); // consume low nibble of 0xF0 (= 0) - - // Read 8 bits across byte boundary: high nibble of 0xF0 + low nibble of 0x0F - long result = sut.readBits(8); - - // Then — 0b11110000 low4=0b1111, 0x0F high4=0b1111 → combined: 0b1111_1111 = 0xFF... wait - // bits [4..7] of 0xF0 = (0xF0 >> 4) & 0xF = 0xF - // bits [0..3] of 0x0F = 0x0F & 0xF = 0xF - // 8-bit result: lower 4 from 0xF0 high nibble, upper 4 from 0x0F low nibble = 0xFF - assertThat(result).isEqualTo(0xFFL); - } - - @ParameterizedTest - @CsvSource({ - "0, 0, 0", - "1, 1, 1", - "127, 7, 127", - "255, 8, 255", - }) - void read_n_bits_from_single_byte(int byteVal, int bitsToRead, long expected) { - // Given - var sut = readerOf((byte) byteVal); - - // When - long result = sut.readBits(bitsToRead); - - // Then - assertThat(result).isEqualTo(expected); - } - - @Test - void readPastEnd_throwsVortexException() { - // Given — 1-byte buffer - var sut = readerOf((byte) 0xFF); - sut.readBits(8); // consume all 8 bits - - // When / Then — reading 1 more bit must throw VortexException, not IOOBE - assertThatThrownBy(() -> sut.readBits(1)) - .isInstanceOf(VortexException.class) - .hasMessageContaining("truncated"); - } - - @Test - void sequential_reads_advance_position() { - // Given — 0xAB = 0b10101011 - var sut = readerOf((byte) 0xAB); - - // When - long b0 = sut.readBits(1); // bit 0 = 1 - long b1 = sut.readBits(1); // bit 1 = 1 - long b2 = sut.readBits(1); // bit 2 = 0 - long b3 = sut.readBits(1); // bit 3 = 1 - - // Then — 0xAB = 0b10101011, LSB first: 1,1,0,1,0,1,0,1 - assertThat(b0).isEqualTo(1); - assertThat(b1).isEqualTo(1); - assertThat(b2).isEqualTo(0); - assertThat(b3).isEqualTo(1); - assertThat(sut.bitsConsumed()).isEqualTo(4); - } - - @Test - void read_16_bits_across_two_bytes() { - // Given — 0x34, 0x12 → LE 16-bit = 0x1234 - var sut = readerOf((byte) 0x34, (byte) 0x12); - - // When - long result = sut.readBits(16); - - // Then - assertThat(result).isEqualTo(0x1234L); - } - - @Test - void read_32_bits_across_four_bytes() { - // Given — LE layout: bytes 0x78, 0x56, 0x34, 0x12 → 0x12345678 - var sut = readerOf((byte) 0x78, (byte) 0x56, (byte) 0x34, (byte) 0x12); - - // When - long result = sut.readBits(32); - - // Then - assertThat(result).isEqualTo(0x12345678L); - } - } - - @Nested - class AlignToByte { - - @Test - void already_aligned_is_noop() { - // Given - var sut = readerOf((byte) 0xFF, (byte) 0xAB); - sut.readBits(8); // consume exactly one byte - - // When - sut.alignToByte(); - - // Then — byte offset unchanged - assertThat(sut.byteOffset()).isEqualTo(1); - assertThat(sut.bitsConsumed()).isEqualTo(8); - } - - @Test - void partial_byte_advances_to_next_byte() { - // Given - var sut = readerOf((byte) 0xFF, (byte) 0xAB); - sut.readBits(3); // consume 3 bits of first byte - - // When - sut.alignToByte(); - - // Then — aligned to byte 1 - assertThat(sut.byteOffset()).isEqualTo(1); - assertThat(sut.bitsConsumed()).isEqualTo(8); - } - - @Test - void next_read_after_align_reads_clean_byte() { - // Given - var sut = readerOf((byte) 0xFF, (byte) 0xAB); - sut.readBits(3); - sut.alignToByte(); - - // When - long result = sut.readBits(8); - - // Then - assertThat(result).isEqualTo(0xABL); - } - } -} diff --git a/core/src/test/java/io/github/dfa1/vortex/encoding/ListEncodingTest.java b/core/src/test/java/io/github/dfa1/vortex/encoding/ListEncodingTest.java deleted file mode 100644 index db11800..0000000 --- a/core/src/test/java/io/github/dfa1/vortex/encoding/ListEncodingTest.java +++ /dev/null @@ -1,179 +0,0 @@ -package io.github.dfa1.vortex.encoding; - -import io.github.dfa1.vortex.core.ArrayStats; -import io.github.dfa1.vortex.core.array.IntArray; -import io.github.dfa1.vortex.core.array.ListArray; -import io.github.dfa1.vortex.core.array.LongArray; -import org.junit.jupiter.api.Nested; -import org.junit.jupiter.api.Test; - -import java.lang.foreign.Arena; -import java.lang.foreign.MemorySegment; - -import static org.assertj.core.api.Assertions.assertThat; -import static org.assertj.core.api.Assertions.assertThatThrownBy; - -class ListEncodingTest { - - - private static ArrayNode toArrayNode(EncodeNode node) { - ArrayNode[] children = new ArrayNode[node.children().length]; - for (int i = 0; i < children.length; i++) { - children[i] = toArrayNode(node.children()[i]); - } - return ArrayNode.of(node.encodingId(), node.metadata(), children, node.bufferIndices(), ArrayStats.empty()); - } - - private static EncodingRegistry registry() { - EncodingRegistry r = EncodingRegistry.empty(); - r.register(new ListEncoding()); - r.register(new PrimitiveEncoding()); - return r; - } - - @Nested - class Encode { - - @Test - void accepts_listDtype_true() { - // Given - ListEncoding sut = new ListEncoding(); - - // When / Then - assertThat(sut.accepts(DTypes.LIST_I32)).isTrue(); - } - - @Test - void accepts_primitiveDtype_false() { - // Given - ListEncoding sut = new ListEncoding(); - - // When / Then - assertThat(sut.accepts(DTypes.I32)).isFalse(); - } - - @Test - void encode_producesTwoChildren_noBuffers() { - // Given - int[] elements = {1, 2, 3, 4, 5}; - long[] offsets = {0, 2, 5}; - ListData data = new ListData(elements, offsets, 2); - ListEncoding sut = new ListEncoding(); - - // When - EncodeResult result = sut.encode(DTypes.LIST_I32, data); - - // Then - assertThat(result.rootNode().encodingId()).isEqualTo(EncodingId.VORTEX_LIST); - assertThat(result.rootNode().bufferIndices()).isEmpty(); - assertThat(result.rootNode().children()).hasSize(2); - } - } - - @Nested - class Decode { - - @Test - void roundTrip_i32Elements_preservesValues() { - // Given - int[] elements = {10, 20, 30, 40, 50}; - long[] offsets = {0, 2, 3, 5}; - ListData data = new ListData(elements, offsets, 3); - ListEncoding sut = new ListEncoding(); - - // When - EncodeResult result = sut.encode(DTypes.LIST_I32, data); - MemorySegment[] bufs = result.buffers().toArray(MemorySegment[]::new); - DecodeContext ctx = new DecodeContext( - toArrayNode(result.rootNode()), DTypes.LIST_I32, 3, bufs, registry(), Arena.global()); - ListArray decoded = (ListArray) sut.decode(ctx); - - // Then - assertThat(decoded.length()).isEqualTo(3); - IntArray decodedElems = (IntArray) decoded.elements(); - assertThat(decodedElems.length()).isEqualTo(5); - for (int i = 0; i < elements.length; i++) { - assertThat(decodedElems.getInt(i)).isEqualTo(elements[i]); - } - LongArray decodedOffsets = (LongArray) decoded.offsets(); - assertThat(decodedOffsets.length()).isEqualTo(4); - for (int i = 0; i < offsets.length; i++) { - assertThat(decodedOffsets.getLong(i)).isEqualTo(offsets[i]); - } - } - - @Test - void roundTrip_emptyLists_preservesOffsets() { - // Given - int[] elements = {}; - long[] offsets = {0, 0, 0}; - ListData data = new ListData(elements, offsets, 2); - ListEncoding sut = new ListEncoding(); - - // When - EncodeResult result = sut.encode(DTypes.LIST_I32, data); - MemorySegment[] bufs = result.buffers().toArray(MemorySegment[]::new); - DecodeContext ctx = new DecodeContext( - toArrayNode(result.rootNode()), DTypes.LIST_I32, 2, bufs, registry(), Arena.global()); - ListArray decoded = (ListArray) sut.decode(ctx); - - // Then - assertThat(decoded.length()).isEqualTo(2); - assertThat(decoded.elements().length()).isEqualTo(0); - assertThat(decoded.offsets().length()).isEqualTo(3); - } - - @Test - void roundTrip_singleList_preservesValues() { - // Given - int[] elements = {7, 8, 9}; - long[] offsets = {0, 3}; - ListData data = new ListData(elements, offsets, 1); - ListEncoding sut = new ListEncoding(); - - // When - EncodeResult result = sut.encode(DTypes.LIST_I32, data); - MemorySegment[] bufs = result.buffers().toArray(MemorySegment[]::new); - DecodeContext ctx = new DecodeContext( - toArrayNode(result.rootNode()), DTypes.LIST_I32, 1, bufs, registry(), Arena.global()); - ListArray decoded = (ListArray) sut.decode(ctx); - - // Then - assertThat(decoded.length()).isEqualTo(1); - IntArray decodedElems = (IntArray) decoded.elements(); - assertThat(decodedElems.length()).isEqualTo(3); - for (int i = 0; i < elements.length; i++) { - assertThat(decodedElems.getInt(i)).isEqualTo(elements[i]); - } - } - - @Test - void decode_wrongDtype_throws() { - // Given - ListEncoding sut = new ListEncoding(); - ArrayNode node = ArrayNode.of(EncodingId.VORTEX_LIST, null, - new ArrayNode[0], new int[0], ArrayStats.empty()); - DecodeContext ctx = new DecodeContext(node, DTypes.I32, 0, new MemorySegment[0], registry(), Arena.global()); - - // When / Then - assertThatThrownBy(() -> sut.decode(ctx)) - .hasMessageContaining("DType.List"); - } - - @Test - void decode_wrongChildCount_throws() { - // Given - ListEncoding sut = new ListEncoding(); - ArrayNode child = ArrayNode.of(EncodingId.VORTEX_PRIMITIVE, null, - new ArrayNode[0], new int[0], ArrayStats.empty()); - ArrayNode node = ArrayNode.of(EncodingId.VORTEX_LIST, - java.nio.ByteBuffer.wrap(new byte[0]), - new ArrayNode[]{child}, new int[0], ArrayStats.empty()); - DecodeContext ctx = new DecodeContext(node, DTypes.LIST_I32, 0, new MemorySegment[0], registry(), Arena.global()); - - // When / Then - assertThatThrownBy(() -> sut.decode(ctx)) - .hasMessageContaining("expected 2 or 3 children"); - } - } -} diff --git a/core/src/test/java/io/github/dfa1/vortex/encoding/ListViewEncodingTest.java b/core/src/test/java/io/github/dfa1/vortex/encoding/ListViewEncodingTest.java deleted file mode 100644 index 8988927..0000000 --- a/core/src/test/java/io/github/dfa1/vortex/encoding/ListViewEncodingTest.java +++ /dev/null @@ -1,179 +0,0 @@ -package io.github.dfa1.vortex.encoding; - -import io.github.dfa1.vortex.core.ArrayStats; -import io.github.dfa1.vortex.core.array.IntArray; -import io.github.dfa1.vortex.core.array.ListViewArray; -import org.junit.jupiter.api.Nested; -import org.junit.jupiter.api.Test; - -import java.lang.foreign.Arena; -import java.lang.foreign.MemorySegment; - -import static org.assertj.core.api.Assertions.assertThat; -import static org.assertj.core.api.Assertions.assertThatThrownBy; - -class ListViewEncodingTest { - - - private static ArrayNode toArrayNode(EncodeNode node) { - ArrayNode[] children = new ArrayNode[node.children().length]; - for (int i = 0; i < children.length; i++) { - children[i] = toArrayNode(node.children()[i]); - } - return ArrayNode.of(node.encodingId(), node.metadata(), children, node.bufferIndices(), ArrayStats.empty()); - } - - private static EncodingRegistry registry() { - return TestRegistry.of(new ListViewEncoding(), new PrimitiveEncoding()); - } - - @Nested - class Encode { - - @Test - void accepts_listDtype_true() { - // Given - ListViewEncoding sut = new ListViewEncoding(); - - // When / Then - assertThat(sut.accepts(DTypes.LIST_I32)).isTrue(); - } - - @Test - void accepts_primitiveDtype_false() { - // Given - ListViewEncoding sut = new ListViewEncoding(); - - // When / Then - assertThat(sut.accepts(DTypes.I32)).isFalse(); - } - - @Test - void encode_producesThreeChildren_noBuffers() { - // Given - int[] elements = {1, 2, 3, 4, 5}; - int[] offsets = {0, 2}; - int[] sizes = {2, 3}; - ListViewData data = new ListViewData(elements, offsets, sizes, 2); - ListViewEncoding sut = new ListViewEncoding(); - - // When - EncodeResult result = sut.encode(DTypes.LIST_I32, data); - - // Then - assertThat(result.rootNode().encodingId()).isEqualTo(EncodingId.VORTEX_LISTVIEW); - assertThat(result.rootNode().bufferIndices()).isEmpty(); - assertThat(result.rootNode().children()).hasSize(3); - } - } - - @Nested - class Decode { - - @Test - void roundTrip_i32Elements_preservesValues() { - // Given - int[] elements = {10, 20, 30, 40, 50}; - int[] offsets = {0, 2, 3}; - int[] sizes = {2, 1, 2}; - ListViewData data = new ListViewData(elements, offsets, sizes, 3); - ListViewEncoding sut = new ListViewEncoding(); - - // When - EncodeResult result = sut.encode(DTypes.LIST_I32, data); - MemorySegment[] bufs = result.buffers().toArray(MemorySegment[]::new); - DecodeContext ctx = new DecodeContext( - toArrayNode(result.rootNode()), DTypes.LIST_I32, 3, bufs, registry(), Arena.global()); - ListViewArray decoded = (ListViewArray) sut.decode(ctx); - - // Then - assertThat(decoded.length()).isEqualTo(3); - IntArray decodedElems = (IntArray) decoded.elements(); - assertThat(decodedElems.length()).isEqualTo(5); - for (int i = 0; i < elements.length; i++) { - assertThat(decodedElems.getInt(i)).isEqualTo(elements[i]); - } - IntArray decodedOffsets = (IntArray) decoded.offsets(); - assertThat(decodedOffsets.length()).isEqualTo(3); - IntArray decodedSizes = (IntArray) decoded.sizes(); - assertThat(decodedSizes.length()).isEqualTo(3); - } - - @Test - void roundTrip_emptyLists_preservesZeroSizes() { - // Given - int[] elements = {}; - int[] offsets = {0, 0}; - int[] sizes = {0, 0}; - ListViewData data = new ListViewData(elements, offsets, sizes, 2); - ListViewEncoding sut = new ListViewEncoding(); - - // When - EncodeResult result = sut.encode(DTypes.LIST_I32, data); - MemorySegment[] bufs = result.buffers().toArray(MemorySegment[]::new); - DecodeContext ctx = new DecodeContext( - toArrayNode(result.rootNode()), DTypes.LIST_I32, 2, bufs, registry(), Arena.global()); - ListViewArray decoded = (ListViewArray) sut.decode(ctx); - - // Then - assertThat(decoded.length()).isEqualTo(2); - assertThat(decoded.elements().length()).isEqualTo(0); - assertThat(decoded.offsets().length()).isEqualTo(2); - assertThat(decoded.sizes().length()).isEqualTo(2); - } - - @Test - void roundTrip_singleList_preservesValues() { - // Given - int[] elements = {7, 8, 9}; - int[] offsets = {0}; - int[] sizes = {3}; - ListViewData data = new ListViewData(elements, offsets, sizes, 1); - ListViewEncoding sut = new ListViewEncoding(); - - // When - EncodeResult result = sut.encode(DTypes.LIST_I32, data); - MemorySegment[] bufs = result.buffers().toArray(MemorySegment[]::new); - DecodeContext ctx = new DecodeContext( - toArrayNode(result.rootNode()), DTypes.LIST_I32, 1, bufs, registry(), Arena.global()); - ListViewArray decoded = (ListViewArray) sut.decode(ctx); - - // Then - assertThat(decoded.length()).isEqualTo(1); - IntArray decodedElems = (IntArray) decoded.elements(); - assertThat(decodedElems.length()).isEqualTo(3); - for (int i = 0; i < elements.length; i++) { - assertThat(decodedElems.getInt(i)).isEqualTo(elements[i]); - } - } - - @Test - void decode_wrongDtype_throws() { - // Given - ListViewEncoding sut = new ListViewEncoding(); - ArrayNode node = ArrayNode.of(EncodingId.VORTEX_LISTVIEW, null, - new ArrayNode[0], new int[0], ArrayStats.empty()); - DecodeContext ctx = TestDecodeContexts.of(node, DTypes.I32).registry(registry()).build(); - - // When / Then - assertThatThrownBy(() -> sut.decode(ctx)) - .hasMessageContaining("DType.List"); - } - - @Test - void decode_wrongChildCount_throws() { - // Given - ListViewEncoding sut = new ListViewEncoding(); - ArrayNode child = ArrayNode.of(EncodingId.VORTEX_PRIMITIVE, null, - new ArrayNode[0], new int[0], ArrayStats.empty()); - ArrayNode node = ArrayNode.of(EncodingId.VORTEX_LISTVIEW, - java.nio.ByteBuffer.wrap(new byte[0]), - new ArrayNode[]{child}, new int[0], ArrayStats.empty()); - DecodeContext ctx = TestDecodeContexts.of(node, DTypes.LIST_I32).registry(registry()).build(); - - // When / Then - assertThatThrownBy(() -> sut.decode(ctx)) - .hasMessageContaining("expected 3 or 4 children"); - } - } -} diff --git a/core/src/test/java/io/github/dfa1/vortex/encoding/MaskedEncodingTest.java b/core/src/test/java/io/github/dfa1/vortex/encoding/MaskedEncodingTest.java deleted file mode 100644 index b865d6c..0000000 --- a/core/src/test/java/io/github/dfa1/vortex/encoding/MaskedEncodingTest.java +++ /dev/null @@ -1,196 +0,0 @@ -package io.github.dfa1.vortex.encoding; - -import io.github.dfa1.vortex.core.DType; -import io.github.dfa1.vortex.core.PType; -import io.github.dfa1.vortex.core.VortexException; -import io.github.dfa1.vortex.core.array.Array; -import io.github.dfa1.vortex.core.array.MaskedArray; -import org.junit.jupiter.api.Nested; -import org.junit.jupiter.api.Test; - -import java.lang.foreign.Arena; -import java.lang.foreign.MemorySegment; -import java.util.ArrayList; -import java.util.List; - -import static org.assertj.core.api.Assertions.assertThat; -import static org.assertj.core.api.Assertions.assertThatThrownBy; - -class MaskedEncodingTest { - - @Nested - class Decode { - - @Test - void oneChild_noValidity_allPositionsValid() { - // Given - var sut = new MaskedEncoding(); - EncodingRegistry registry = buildRegistry(); - DType i32Nullable = new DType.Primitive(PType.I32, true); - EncodeResult ctx = maskedResult(new int[]{10, 20, 30}, null); - - // When - Array result = sut.decode(EncodeTestHelper.toDecodeContext(ctx, 3L, i32Nullable, registry)); - - // Then - assertThat(result).isInstanceOf(MaskedArray.class); - MaskedArray masked = (MaskedArray) result; - assertThat(masked.length()).isEqualTo(3); - assertThat(masked.isValid(0)).isTrue(); - assertThat(masked.isValid(1)).isTrue(); - assertThat(masked.isValid(2)).isTrue(); - } - - @Test - void twoChildren_withValidity_masksNulls() { - // Given - var sut = new MaskedEncoding(); - EncodingRegistry registry = buildRegistry(); - DType i32Nullable = new DType.Primitive(PType.I32, true); - EncodeResult ctx = maskedResult(new int[]{1, 2, 3, 4, 5}, - new boolean[]{true, false, true, false, true}); - - // When - Array result = sut.decode(EncodeTestHelper.toDecodeContext(ctx, 5L, i32Nullable, registry)); - - // Then - assertThat(result).isInstanceOf(MaskedArray.class); - MaskedArray masked = (MaskedArray) result; - assertThat(masked.length()).isEqualTo(5); - assertThat(masked.isValid(0)).isTrue(); - assertThat(masked.isValid(1)).isFalse(); - assertThat(masked.isValid(2)).isTrue(); - assertThat(masked.isValid(3)).isFalse(); - assertThat(masked.isValid(4)).isTrue(); - } - - @Test - void dtype_isNullable() { - // Given - var sut = new MaskedEncoding(); - EncodingRegistry registry = buildRegistry(); - DType i32Nullable = new DType.Primitive(PType.I32, true); - EncodeResult ctx = maskedResult(new int[]{1, 2, 3}, null); - - // When - Array result = sut.decode(EncodeTestHelper.toDecodeContext(ctx, 3L, i32Nullable, registry)); - - // Then - assertThat(result.dtype().nullable()).isTrue(); - } - - @Test - void buffer_delegatesToChild() { - // Given - var sut = new MaskedEncoding(); - EncodingRegistry registry = buildRegistry(); - DType i32Nullable = new DType.Primitive(PType.I32, true); - EncodeResult ctx = maskedResult(new int[]{7, 8, 9}, null); - - // When - Array result = sut.decode(EncodeTestHelper.toDecodeContext(ctx, 3L, i32Nullable, registry)); - - // Then — buffer(0) works and contains child values - assertThat(result.buffer(0).get(PTypeIO.LE_INT, 0L)).isEqualTo(7); - assertThat(result.buffer(0).get(PTypeIO.LE_INT, 4L)).isEqualTo(8); - assertThat(result.buffer(0).get(PTypeIO.LE_INT, 8L)).isEqualTo(9); - } - - @Test - void buffersPresentThrows() { - // Given - var sut = new MaskedEncoding(); - EncodingRegistry registry = buildRegistry(); - DType i32Nullable = new DType.Primitive(PType.I32, true); - - // Build a node with an unexpected buffer index - EncodeNode childNode = EncodeNode.leaf(EncodingId.VORTEX_PRIMITIVE, 0); - EncodeNode maskedNode = new EncodeNode( - EncodingId.VORTEX_MASKED, null, - new EncodeNode[]{childNode}, new int[]{1}); - MemorySegment dummyBuf = Arena.ofAuto().allocate(4); - EncodeResult result = new EncodeResult(maskedNode, List.of(dummyBuf, dummyBuf), null, null); - - // When / Then - assertThatThrownBy(() -> - sut.decode(EncodeTestHelper.toDecodeContext(result, 1L, i32Nullable, registry))) - .isInstanceOf(VortexException.class) - .hasMessageContaining("expected 0 buffers"); - } - - @Test - void zeroChildrenThrows() { - // Given - var sut = new MaskedEncoding(); - EncodingRegistry registry = buildRegistry(); - DType i32Nullable = new DType.Primitive(PType.I32, true); - - EncodeNode maskedNode = new EncodeNode( - EncodingId.VORTEX_MASKED, null, new EncodeNode[]{}, new int[]{}); - EncodeResult result = new EncodeResult(maskedNode, List.of(), null, null); - - // When / Then - assertThatThrownBy(() -> - sut.decode(EncodeTestHelper.toDecodeContext(result, 0L, i32Nullable, registry))) - .isInstanceOf(VortexException.class) - .hasMessageContaining("expected 1 or 2 children"); - } - - @Test - void threeChildrenThrows() { - // Given - var sut = new MaskedEncoding(); - EncodingRegistry registry = buildRegistry(); - DType i32Nullable = new DType.Primitive(PType.I32, true); - - PrimitiveEncoding primitiveEncoding = new PrimitiveEncoding(); - DType i32 = new DType.Primitive(PType.I32, false); - EncodeResult childResult = primitiveEncoding.encode(i32, new int[]{1}); - EncodeNode childNode = childResult.rootNode(); - EncodeNode maskedNode = new EncodeNode( - EncodingId.VORTEX_MASKED, null, - new EncodeNode[]{childNode, childNode, childNode}, new int[]{}); - List bufs = new ArrayList<>(childResult.buffers()); - EncodeResult result = new EncodeResult(maskedNode, bufs, null, null); - - // When / Then - assertThatThrownBy(() -> - sut.decode(EncodeTestHelper.toDecodeContext(result, 1L, i32Nullable, registry))) - .isInstanceOf(VortexException.class) - .hasMessageContaining("expected 1 or 2 children"); - } - } - - // ── helpers ─────────────────────────────────────────────────────────────── - - private static EncodingRegistry buildRegistry() { - EncodingRegistry registry = TestRegistry.of(new MaskedEncoding(), new PrimitiveEncoding(), new BoolEncoding()); - return registry; - } - - /// Build a synthetic masked EncodeResult wrapping an I32 child and optional validity. - private static EncodeResult maskedResult(int[] values, boolean[] validity) { - PrimitiveEncoding primitiveEncoding = new PrimitiveEncoding(); - DType i32 = new DType.Primitive(PType.I32, false); - EncodeResult childResult = primitiveEncoding.encode(i32, values); - - List allBuffers = new ArrayList<>(childResult.buffers()); - EncodeNode[] children; - - if (validity == null) { - children = new EncodeNode[]{childResult.rootNode()}; - } else { - BoolEncoding boolEncoding = new BoolEncoding(); - DType boolDtype = new DType.Bool(false); - EncodeResult validityResult = boolEncoding.encode(boolDtype, validity); - EncodeNode remapped = EncodeNode.remapBufferIndices( - validityResult.rootNode(), childResult.buffers().size()); - allBuffers.addAll(validityResult.buffers()); - children = new EncodeNode[]{childResult.rootNode(), remapped}; - } - - EncodeNode maskedNode = new EncodeNode( - EncodingId.VORTEX_MASKED, null, children, new int[]{}); - return new EncodeResult(maskedNode, allBuffers, null, null); - } -} diff --git a/core/src/test/java/io/github/dfa1/vortex/encoding/NullEncodingTest.java b/core/src/test/java/io/github/dfa1/vortex/encoding/NullEncodingTest.java deleted file mode 100644 index aae84c9..0000000 --- a/core/src/test/java/io/github/dfa1/vortex/encoding/NullEncodingTest.java +++ /dev/null @@ -1,79 +0,0 @@ -package io.github.dfa1.vortex.encoding; - -import io.github.dfa1.vortex.core.DType; -import io.github.dfa1.vortex.core.array.NullArray; -import org.junit.jupiter.api.Nested; -import org.junit.jupiter.api.Test; - -import java.lang.foreign.Arena; -import java.lang.foreign.MemorySegment; - -import static org.assertj.core.api.Assertions.assertThat; - -class NullEncodingTest { - - private static final DType NULL_DTYPE = new DType.Null(true); - - @Nested - class Encode { - - @Test - void encode_producesEmptyNode() { - // Given - var sut = new NullEncoding(); - - // When - EncodeResult result = sut.encode(NULL_DTYPE, null); - - // Then - assertThat(result.rootNode().encodingId()).isEqualTo(EncodingId.VORTEX_NULL); - assertThat(result.rootNode().children()).isEmpty(); - assertThat(result.buffers()).isEmpty(); - } - - @Test - void encode_thenDecode_roundTrips() { - // Given - long rowCount = 10L; - var sut = new NullEncoding(); - - // When - EncodeResult encoded = sut.encode(NULL_DTYPE, null); - ArrayNode node = ArrayNode.of(encoded.rootNode().encodingId(), null, new ArrayNode[0], new int[0], null); - DecodeContext ctx = new DecodeContext(node, NULL_DTYPE, rowCount, new MemorySegment[0], - EncodingRegistry.empty(), Arena.ofAuto()); - - // Then - var decoded = sut.decode(ctx); - assertThat(decoded).isInstanceOf(NullArray.class); - assertThat(decoded.length()).isEqualTo(rowCount); - } - } - - @Nested - class Decode { - - @Test - void decode_nullArray_returnsNullArrayWithCorrectLength() { - // Given - long rowCount = 42L; - DecodeContext ctx = buildNullCtx(rowCount); - var sut = new NullEncoding(); - - // When - var result = sut.decode(ctx); - - // Then - assertThat(result).isInstanceOf(NullArray.class); - assertThat(result.length()).isEqualTo(rowCount); - assertThat(result.dtype()).isEqualTo(NULL_DTYPE); - } - - private static DecodeContext buildNullCtx(long rowCount) { - ArrayNode node = ArrayNode.of(EncodingId.VORTEX_NULL, null, new ArrayNode[0], new int[0], null); - EncodingRegistry registry = EncodingRegistry.empty(); - registry.register(new NullEncoding()); - return new DecodeContext(node, NULL_DTYPE, rowCount, new MemorySegment[0], registry, Arena.ofAuto()); - } - } -} diff --git a/core/src/test/java/io/github/dfa1/vortex/encoding/PcoEncodingTest.java b/core/src/test/java/io/github/dfa1/vortex/encoding/PcoEncodingTest.java deleted file mode 100644 index 4aecc24..0000000 --- a/core/src/test/java/io/github/dfa1/vortex/encoding/PcoEncodingTest.java +++ /dev/null @@ -1,812 +0,0 @@ -package io.github.dfa1.vortex.encoding; - -import com.google.protobuf.ByteString; -import io.github.dfa1.vortex.core.DType; -import io.github.dfa1.vortex.core.PType; -import io.github.dfa1.vortex.core.VortexException; -import io.github.dfa1.vortex.core.array.LongArray; -import io.github.dfa1.vortex.core.array.MaskedArray; -import io.github.dfa1.vortex.proto.EncodingProtos; -import org.junit.jupiter.api.Nested; -import org.junit.jupiter.api.Test; -import org.junit.jupiter.params.ParameterizedTest; -import org.junit.jupiter.params.provider.EnumSource; -import org.junit.jupiter.params.provider.MethodSource; -import org.junit.jupiter.params.provider.ValueSource; - -import java.lang.foreign.Arena; -import java.lang.foreign.MemorySegment; -import java.lang.foreign.ValueLayout; -import java.nio.ByteBuffer; -import java.util.Random; -import java.util.stream.Stream; - -import static org.assertj.core.api.Assertions.assertThat; -import static org.assertj.core.api.Assertions.assertThatThrownBy; - -class PcoEncodingTest { - - private static ByteBuffer validMetaBuffer() { - EncodingProtos.PcoMetadata meta = EncodingProtos.PcoMetadata.newBuilder() - .setHeader(ByteString.copyFrom(new byte[]{PcoEncoding.PCO_FORMAT_MAJOR, PcoEncoding.PCO_FORMAT_MINOR})) - .build(); - return ByteBuffer.wrap(meta.toByteArray()); - } - - private static DecodeContext ctxWith(ByteBuffer meta, DType dtype, long rowCount, MemorySegment[] buffers) { - ArrayNode node = ArrayNode.of(EncodingId.VORTEX_PCO, meta, new ArrayNode[0], - bufferIndices(buffers.length), null); - return new DecodeContext(node, dtype, rowCount, buffers, EncodingRegistry.empty(), Arena.ofAuto()); - } - - /// Build a nullable DecodeContext: validity buffer at index 0, pco buffers at indices 1..N. - /// Validity is a bit-packed Bool array (LSB-first, 1=valid). - private static DecodeContext ctxWithValidity(ByteBuffer meta, DType dtype, long rowCount, - MemorySegment validityBuf, MemorySegment[] pcoBuffers) { - MemorySegment[] allBuffers = new MemorySegment[1 + pcoBuffers.length]; - allBuffers[0] = validityBuf; - System.arraycopy(pcoBuffers, 0, allBuffers, 1, pcoBuffers.length); - - ArrayNode validityNode = ArrayNode.of(EncodingId.VORTEX_BOOL, null, new ArrayNode[0], - new int[]{0}, null); - - int[] pcoBufferIndices = new int[pcoBuffers.length]; - for (int i = 0; i < pcoBuffers.length; i++) { - pcoBufferIndices[i] = i + 1; - } - ArrayNode pcoNode = ArrayNode.of(EncodingId.VORTEX_PCO, meta, new ArrayNode[]{validityNode}, - pcoBufferIndices, null); - - EncodingRegistry registry = TestRegistry.of(new BoolEncoding()); - return new DecodeContext(pcoNode, dtype, rowCount, allBuffers, registry, Arena.ofAuto()); - } - - private static int[] bufferIndices(int n) { - int[] idx = new int[n]; - for (int i = 0; i < n; i++) { - idx[i] = i; - } - return idx; - } - - private static MemorySegment segmentOf(byte... bytes) { - MemorySegment seg = Arena.ofAuto().allocate(bytes.length); - for (int i = 0; i < bytes.length; i++) { - seg.set(ValueLayout.JAVA_BYTE, i, bytes[i]); - } - return seg; - } - - /// Build a PcoMetadata proto with one chunk containing one page of {@code nValues} values. - private static ByteBuffer metaWithOneChunk(int nValues) { - EncodingProtos.PcoMetadata meta = EncodingProtos.PcoMetadata.newBuilder() - .setHeader(ByteString.copyFrom(new byte[]{PcoEncoding.PCO_FORMAT_MAJOR, PcoEncoding.PCO_FORMAT_MINOR})) - .addChunks(EncodingProtos.PcoChunkInfo.newBuilder() - .addPages(EncodingProtos.PcoPageInfo.newBuilder().setNValues(nValues).build()) - .build()) - .build(); - return ByteBuffer.wrap(meta.toByteArray()); - } - - /// Chunk-meta bytes for Classic mode, Consecutive delta at {@code order}, ansSizeLog=0, nBins=0. - /// - /// Bit layout (LSB-first per byte): - /// byte0: mode_nibble=0, delta_nibble=1 - /// byte1: order (3b), secondary_uses_delta=0 (1b), ansSizeLog=0 (4b) - /// byte2–3: nBins=0 (15b), align padding - private static MemorySegment chunkMetaConsecutive(int order) { - return segmentOf( - (byte) 0x10, // mode=0, delta_variant=1 - (byte) order, // order[2:0], secondary=0, ansSizeLog=0 (order ≤ 7) - (byte) 0x00, // nBins bits16-23 = 0 - (byte) 0x00 // nBins bits24-30 = 0, padding - ); - } - - /// Page bytes: {@code order} LE-U64 moments, then 4 zero ANS-state slots (0 bits each). - private static MemorySegment pageWithMoments(long... moments) { - byte[] buf = new byte[moments.length * Long.BYTES]; - java.nio.ByteBuffer bb = java.nio.ByteBuffer.wrap(buf).order(java.nio.ByteOrder.LITTLE_ENDIAN); - for (long m : moments) { - bb.putLong(m); - } - return segmentOf(buf); - } - - @Nested - class EncodingIdTest { - - @Test - void encodingId_isVortexPco() { - // Given / When / Then - assertThat(new PcoEncoding().encodingId()).isEqualTo(EncodingId.VORTEX_PCO); - } - } - - @Nested - class Encode { - - @Test - void encode_throwsVortexException() { - // Given - var sut = new PcoEncoding(); - DType dtype = new DType.Primitive(PType.I64, false); - - // When / Then - assertThatThrownBy(() -> sut.encode(dtype, new long[]{1L, 2L, 3L})) - .isInstanceOf(VortexException.class) - .hasMessageContaining("not implemented"); - } - } - - @Nested - class Decode { - - @Test - void decode_nullMetadata_throwsMissingMeta() { - // Given - var sut = new PcoEncoding(); - DecodeContext ctx = ctxWith(null, new DType.Primitive(PType.I64, false), 0, new MemorySegment[0]); - - // When / Then - assertThatThrownBy(() -> sut.decode(ctx)) - .isInstanceOf(VortexException.class) - .hasMessageContaining("missing PcoMetadata"); - } - - @Test - void decode_invalidHeaderVersion_throwsUnsupported() { - // Given - var sut = new PcoEncoding(); - EncodingProtos.PcoMetadata meta = EncodingProtos.PcoMetadata.newBuilder() - .setHeader(ByteString.copyFrom(new byte[]{0x03, 0x00})) - .build(); - DecodeContext ctx = ctxWith(ByteBuffer.wrap(meta.toByteArray()), - new DType.Primitive(PType.I64, false), 0, new MemorySegment[0]); - - // When / Then - assertThatThrownBy(() -> sut.decode(ctx)) - .isInstanceOf(VortexException.class) - .hasMessageContaining("unsupported pco format version 03.00"); - } - - @Test - void decode_nonPrimitiveDtype_throws() { - // Given - var sut = new PcoEncoding(); - DecodeContext ctx = ctxWith(validMetaBuffer(), new DType.Utf8(false), 0, new MemorySegment[0]); - - // When / Then - assertThatThrownBy(() -> sut.decode(ctx)) - .isInstanceOf(VortexException.class) - .hasMessageContaining("Primitive dtype"); - } - - @Test - void decode_unsupportedPtype_throws() { - // Given — F16 not supported by pco - var sut = new PcoEncoding(); - DecodeContext ctx = ctxWith(validMetaBuffer(), new DType.Primitive(PType.F16, false), 0, - new MemorySegment[0]); - - // When / Then - assertThatThrownBy(() -> sut.decode(ctx)) - .isInstanceOf(VortexException.class) - .hasMessageContaining("unsupported ptype"); - } - - @ParameterizedTest - @EnumSource(value = PType.class, names = {"I16", "U16", "I32", "U32", "F32", "I64", "U64", "F64"}) - void decode_zeroChunks_returnsEmptyArray(PType ptype) { - // Given — valid metadata with 0 chunks, 0 rows, any supported ptype - var sut = new PcoEncoding(); - DecodeContext ctx = ctxWith(validMetaBuffer(), new DType.Primitive(ptype, false), 0, - new MemorySegment[0]); - - // When - var result = sut.decode(ctx); - - // Then - assertThat(result.length()).isZero(); - } - - @Test - void decode_consecutiveDelta_order1_singleValue_decodes() { - // Given — U64 sequence [42] encoded with Classic mode, Consecutive delta order=1. - // With pageN=1 and order=1, decodedN=0: the single output value is the moment itself. - var sut = new PcoEncoding(); - DecodeContext ctx = ctxWith( - metaWithOneChunk(1), - new DType.Primitive(PType.U64, false), - 1, - new MemorySegment[]{chunkMetaConsecutive(1), pageWithMoments(42L)}); - - // When - var result = sut.decode(ctx); - - // Then - assertThat(result.length()).isEqualTo(1); - assertThat(((LongArray) result).getLong(0)).isEqualTo(42L); - } - - @Test - void decode_consecutiveDelta_order2_twoValues_decodes() { - // Given — U64 sequence [10, 17] encoded with Consecutive delta order=2. - // With pageN=2, order=2: decodedN=0; moments=[m0=10, m1=delta1=7]. - // Expected reconstruction: [m0, m0+m1] = [10, 17]. - var sut = new PcoEncoding(); - DecodeContext ctx = ctxWith( - metaWithOneChunk(2), - new DType.Primitive(PType.U64, false), - 2, - new MemorySegment[]{chunkMetaConsecutive(2), pageWithMoments(10L, 7L)}); - - // When - var result = sut.decode(ctx); - - // Then - assertThat(result.length()).isEqualTo(2); - assertThat(((LongArray) result).getLong(0)).isEqualTo(10L); - assertThat(((LongArray) result).getLong(1)).isEqualTo(17L); - } - - @Test - void decode_multiPage_singleChunk_decodes() { - // Given — 1 chunk, 2 pages each containing 1 value (Consecutive order=1). - // buffers: [chunkMeta, page0, page1]; page0 moment=10→value 10, page1 moment=20→value 20. - var sut = new PcoEncoding(); - EncodingProtos.PcoMetadata meta = EncodingProtos.PcoMetadata.newBuilder() - .setHeader(ByteString.copyFrom(new byte[]{PcoEncoding.PCO_FORMAT_MAJOR, PcoEncoding.PCO_FORMAT_MINOR})) - .addChunks(EncodingProtos.PcoChunkInfo.newBuilder() - .addPages(EncodingProtos.PcoPageInfo.newBuilder().setNValues(1).build()) - .addPages(EncodingProtos.PcoPageInfo.newBuilder().setNValues(1).build()) - .build()) - .build(); - DecodeContext ctx = ctxWith( - ByteBuffer.wrap(meta.toByteArray()), - new DType.Primitive(PType.U64, false), - 2, - new MemorySegment[]{chunkMetaConsecutive(1), pageWithMoments(10L), pageWithMoments(20L)}); - - // When - var result = sut.decode(ctx); - - // Then - assertThat(result.length()).isEqualTo(2); - assertThat(((LongArray) result).getLong(0)).isEqualTo(10L); - assertThat(((LongArray) result).getLong(1)).isEqualTo(20L); - } - - @Test - void decode_multiChunk_decodes() { - // Given — 2 chunks each with 1 page containing 1 value (Consecutive order=1). - // buffers: [chunkMeta0, page0, chunkMeta1, page1]; values=[100, 200]. - var sut = new PcoEncoding(); - EncodingProtos.PcoMetadata meta = EncodingProtos.PcoMetadata.newBuilder() - .setHeader(ByteString.copyFrom(new byte[]{PcoEncoding.PCO_FORMAT_MAJOR, PcoEncoding.PCO_FORMAT_MINOR})) - .addChunks(EncodingProtos.PcoChunkInfo.newBuilder() - .addPages(EncodingProtos.PcoPageInfo.newBuilder().setNValues(1).build()) - .build()) - .addChunks(EncodingProtos.PcoChunkInfo.newBuilder() - .addPages(EncodingProtos.PcoPageInfo.newBuilder().setNValues(1).build()) - .build()) - .build(); - DecodeContext ctx = ctxWith( - ByteBuffer.wrap(meta.toByteArray()), - new DType.Primitive(PType.U64, false), - 2, - new MemorySegment[]{ - chunkMetaConsecutive(1), pageWithMoments(100L), - chunkMetaConsecutive(1), pageWithMoments(200L)}); - - // When - var result = sut.decode(ctx); - - // Then - assertThat(result.length()).isEqualTo(2); - assertThat(((LongArray) result).getLong(0)).isEqualTo(100L); - assertThat(((LongArray) result).getLong(1)).isEqualTo(200L); - } - } - - @Nested - class DecodeNullable { - - @Test - void decode_nullable_someNulls_scattersCorrectly() { - // Given — U64 sequence: 3 total rows, validity=[true,false,true], valid values=[100,200]. - // Validity bits LSB-first: bit0=1, bit1=0, bit2=1 → byte 0x05. - // Pco encodes only valid values: 1 chunk, 2 pages of nValues=1 (Consecutive order=1). - var sut = new PcoEncoding(); - EncodingProtos.PcoMetadata meta = EncodingProtos.PcoMetadata.newBuilder() - .setHeader(ByteString.copyFrom(new byte[]{PcoEncoding.PCO_FORMAT_MAJOR, PcoEncoding.PCO_FORMAT_MINOR})) - .addChunks(EncodingProtos.PcoChunkInfo.newBuilder() - .addPages(EncodingProtos.PcoPageInfo.newBuilder().setNValues(1).build()) - .addPages(EncodingProtos.PcoPageInfo.newBuilder().setNValues(1).build()) - .build()) - .build(); - MemorySegment validityBuf = segmentOf((byte) 0x05); // bits: 1,0,1 - DecodeContext ctx = ctxWithValidity( - ByteBuffer.wrap(meta.toByteArray()), - new DType.Primitive(PType.U64, true), - 3, - validityBuf, - new MemorySegment[]{chunkMetaConsecutive(1), pageWithMoments(100L), pageWithMoments(200L)}); - - // When - var result = sut.decode(ctx); - - // Then — MaskedArray with 3 slots; positions 0 and 2 valid, position 1 null - assertThat(result).isInstanceOf(MaskedArray.class); - assertThat(result.length()).isEqualTo(3); - MaskedArray masked = (MaskedArray) result; - assertThat(masked.isValid(0)).isTrue(); - assertThat(masked.isValid(1)).isFalse(); - assertThat(masked.isValid(2)).isTrue(); - assertThat(((LongArray) masked.child(0)).getLong(0)).isEqualTo(100L); - assertThat(((LongArray) masked.child(0)).getLong(2)).isEqualTo(200L); - } - - @Test - void decode_nullable_allNull_returnsAllZeroed() { - // Given — 2 total rows, validity=[false,false], validCount=0. Pco has 0 chunks. - // Validity bits LSB-first: 0x00. - var sut = new PcoEncoding(); - MemorySegment validityBuf = segmentOf((byte) 0x00); - DecodeContext ctx = ctxWithValidity( - validMetaBuffer(), - new DType.Primitive(PType.U64, true), - 2, - validityBuf, - new MemorySegment[0]); - - // When - var result = sut.decode(ctx); - - // Then — MaskedArray, length 2, both null, values zeroed - assertThat(result).isInstanceOf(MaskedArray.class); - assertThat(result.length()).isEqualTo(2); - MaskedArray masked = (MaskedArray) result; - assertThat(masked.isValid(0)).isFalse(); - assertThat(masked.isValid(1)).isFalse(); - assertThat(((LongArray) masked.child(0)).getLong(0)).isZero(); - assertThat(((LongArray) masked.child(0)).getLong(1)).isZero(); - } - - @Test - void decode_nullable_allValid_returnsMaskedWithAllValues() { - // Given — 2 total rows, validity=[true,true], valid values=[10,20]. - // Validity bits: 0x03. - var sut = new PcoEncoding(); - MemorySegment validityBuf = segmentOf((byte) 0x03); // bits: 1,1 - DecodeContext ctx = ctxWithValidity( - metaWithOneChunk(2), - new DType.Primitive(PType.U64, true), - 2, - validityBuf, - new MemorySegment[]{chunkMetaConsecutive(2), pageWithMoments(10L, 10L)}); - - // When - var result = sut.decode(ctx); - - // Then — MaskedArray, all valid, values [10, 20] - assertThat(result).isInstanceOf(MaskedArray.class); - assertThat(result.length()).isEqualTo(2); - MaskedArray masked = (MaskedArray) result; - assertThat(masked.isValid(0)).isTrue(); - assertThat(masked.isValid(1)).isTrue(); - assertThat(((LongArray) masked.child(0)).getLong(0)).isEqualTo(10L); - assertThat(((LongArray) masked.child(0)).getLong(1)).isEqualTo(20L); - } - } - - /// Build chunk meta for Classic mode + Conv1 delta by packing bits LSB-first. - /// - /// Layout: mode(4b)=0, delta(4b)=3, quantization(5b), bias_latent(64b), - /// order-1(5b), weights[order×32b], ansSizeLog(4b)=0, nBins(15b)=0, align. - /// bias_latent = bias ^ Long.MIN_VALUE; each weight_latent = weight ^ 0x80000000L. - private static MemorySegment chunkMetaConv1(int quantization, long biasLatent, - int order, long[] weightLatents) { - java.util.BitSet bits = new java.util.BitSet(); - int pos = 0; - // mode nibble = 0 (Classic) - pos += 4; - // delta nibble = 3 (Conv1): bits = 0b0011 - bits.set(pos); bits.set(pos + 1); pos += 4; - // quantization (5 bits) - for (int i = 0; i < 5; i++) { - if (((quantization >> i) & 1) != 0) { - bits.set(pos); - } - pos++; - } - // bias latent (64 bits, LSB first) - for (int i = 0; i < 64; i++) { - if (((biasLatent >> i) & 1L) != 0L) { - bits.set(pos); - } - pos++; - } - // order-1 (5 bits) - for (int i = 0; i < 5; i++) { - if ((((order - 1) >> i) & 1) != 0) { - bits.set(pos); - } - pos++; - } - // weight latents (order × 32 bits) - for (long wl : weightLatents) { - for (int i = 0; i < 32; i++) { - if (((wl >> i) & 1L) != 0L) { - bits.set(pos); - } - pos++; - } - } - // ansSizeLog (4 bits) = 0 - pos += 4; - // nBins (15 bits) = 0 - pos += 15; - int byteLen = (pos + 7) / 8; - byte[] buf = new byte[byteLen]; - for (int i = 0; i < pos; i++) { - if (bits.get(i)) { - buf[i / 8] |= (byte) (1 << (i % 8)); - } - } - return segmentOf(buf); - } - - /// Chunk-meta bytes for Classic mode + Lookback delta with windowNLog=1 (windowN=2), stateNLog=0 (stateN=1), - /// deltaAnsSizeLog=0, primaryAnsSizeLog=0, no bins. - /// - /// Bit layout: - /// byte0: mode=0[3:0], delta=2[7:4] → 0x20 - /// bytes 1-6: windowNLog-1(5b)=0, stateNLog(4b)=0, secondary(1b)=0, - /// deltaAnsSizeLog(4b)=0, nDeltaBins(15b)=0, - /// primaryAnsSizeLog(4b)=0, nBins(15b)=0, align → all 0x00 - private static MemorySegment chunkMetaLookback() { - return segmentOf((byte) 0x20, (byte) 0x00, (byte) 0x00, (byte) 0x00, (byte) 0x00, (byte) 0x00, (byte) 0x00); - } - - /// Page bytes for Lookback with stateN=1, U64, deltaAnsSizeLog=0, primaryAnsSizeLog=0. - /// Format: 8 bytes (one 64-bit initial state). No ANS state bits (sizeLog=0). No decoded bits. - private static MemorySegment lookbackPage(long initialState) { - byte[] buf = new byte[Long.BYTES]; - java.nio.ByteBuffer.wrap(buf).order(java.nio.ByteOrder.LITTLE_ENDIAN).putLong(initialState); - return segmentOf(buf); - } - - @Nested - class DecodeConv1 { - - @Test - void decode_conv1_order1_zeroPrediction_statePassedThrough() { - // Given — I32, pageN=2, order=1, bias=0, weight=0 → prediction always 0. - // State raw = value ^ 0x80000000: for value=5, state_raw=0x80000005. - // Residual from degenerate tANS=0 → decoded = 0 ^ mid_i32(0x80000000) = 0x80000000 - // → fromLatentOrdered(0x80000000, I32) = 0x80000000 ^ 0x80000000 = 0. - // Expected output: [5, 0]. - var sut = new PcoEncoding(); - long biasLatent = Long.MIN_VALUE; // encodes bias=0: raw ^ MIN_VALUE = 0 - long weightLatent = 0x80000000L; // encodes weight=0: (int)(raw ^ 0x80000000) = 0 - MemorySegment chunkMeta = chunkMetaConv1(0, biasLatent, 1, new long[]{weightLatent}); - // Page: 1 × 32-bit state = 0x80000005 (encodes value 5), no residual bits. - MemorySegment page = segmentOf((byte) 0x05, (byte) 0x00, (byte) 0x00, (byte) 0x80); - DecodeContext ctx = ctxWith( - metaWithOneChunk(2), - new DType.Primitive(PType.I32, false), - 2, - new MemorySegment[]{chunkMeta, page}); - - // When - var result = sut.decode(ctx); - - // Then - assertThat(result.length()).isEqualTo(2); - assertThat(((io.github.dfa1.vortex.core.array.IntArray) result).getInt(0)).isEqualTo(5); - assertThat(((io.github.dfa1.vortex.core.array.IntArray) result).getInt(1)).isZero(); - } - } - - @Nested - class DecodeLookback { - - @Test - void decode_lookback_corruptIndexZero_throwsVortexException() { - // Given — Classic+Lookback, windowN=2, stateN=1, degenerate ANS (0 bins). - // Degenerate tANS always outputs lower=0; lb=0 is out of [1, windowN=2]. - // pageN=2: stateN=1 initial value + 1 decoded value with lb=0. - var sut = new PcoEncoding(); - DecodeContext ctx = ctxWith( - metaWithOneChunk(2), - new DType.Primitive(PType.U64, false), - 2, - new MemorySegment[]{chunkMetaLookback(), lookbackPage(0L)}); - - // When / Then - assertThatThrownBy(() -> sut.decode(ctx)) - .isInstanceOf(VortexException.class) - .hasMessageContaining("corrupt lookback index 0"); - } - - @Test - void decode_lookback_stateNExceedsPageN_throwsVortexException() { - // Given — stateN=2 (stateNLog=1) but pageN=1 → decodeN = 1-2 = -1 → corrupt. - // chunkMeta bit layout (all LE, LSB-first): - // byte0: mode=0[3:0], delta=2[7:4] → 0x20 - // byte1: windowNLog-1(5b)=0, stateNLog[0](1b)=1 → 0x20 - // byte2: stateNLog[1..3](3b)=0, secondary(1b)=0, deltaAnsSizeLog(4b)=0 → 0x00 - // bytes3-6: nDeltaBins(15b)=0, primaryAnsSizeLog(4b)=0, nBins(15b)=0 → 0x00 - var sut = new PcoEncoding(); - MemorySegment chunkMeta = segmentOf( - (byte) 0x20, (byte) 0x20, (byte) 0x00, (byte) 0x00, (byte) 0x00, (byte) 0x00, (byte) 0x00); - DecodeContext ctx = ctxWith( - metaWithOneChunk(1), - new DType.Primitive(PType.U64, false), - 1, - new MemorySegment[]{chunkMeta, segmentOf((byte) 0x00)}); - - // When / Then - assertThatThrownBy(() -> sut.decode(ctx)) - .isInstanceOf(VortexException.class) - .hasMessageContaining("stateN"); - } - - @Test - void decode_lookback_singleInitialValue_returnsIt() { - // Given — pageN=1, stateN=1, decodeN=0: only the initial state value; no decoded values. - var sut = new PcoEncoding(); - DecodeContext ctx = ctxWith( - metaWithOneChunk(1), - new DType.Primitive(PType.U64, false), - 1, - new MemorySegment[]{chunkMetaLookback(), lookbackPage(42L)}); - - // When - var result = sut.decode(ctx); - - // Then - assertThat(result.length()).isEqualTo(1); - assertThat(((LongArray) result).getLong(0)).isEqualTo(42L); - } - } - - @Nested - class DecodeLookbackDecodeN { - - @Test - void lookback_decodeNExceedsMax_throwsVortexException() { - // Given — stateN=1, pageN=(1<<23)+2 → decodeN=(1<<23)+1 > cap 1<<23. - // Check fires before arena.allocate; page only needs 8 bytes (1×64-bit initial state). - var sut = new PcoEncoding(); - int pageN = (1 << 23) + 2; - DecodeContext ctx = ctxWith( - metaWithOneChunk(pageN), - new DType.Primitive(PType.U64, false), - pageN, - new MemorySegment[]{chunkMetaLookback(), segmentOf(new byte[8])}); - - // When / Then - assertThatThrownBy(() -> sut.decode(ctx)) - .isInstanceOf(VortexException.class) - .hasMessageContaining("decodeN"); - } - } - - @Nested - class DecodeLookbackStateNWindow { - - @Test - void lookback_stateNExceedsWindowN_throwsVortexException() { - // Given — windowNLog=1 (windowN=2), stateNLog=2 (stateN=4) → stateN > windowN. - // pageN=4 (≥ stateN=4 passes the pageN sut.decode(ctx)) - .isInstanceOf(VortexException.class) - .hasMessageContaining("stateN"); - } - } - - @Nested - class DecodeLookbackWindowNLog { - - @Test - void lookback_windowNLogExceedsMax_throwsVortexException() { - // Given — mode=0 (Classic), delta=2 (Lookback), windowNLog=25 > max 24. - // Bit layout (LSB-first after byte0): - // byte0: mode=0[3:0], delta=2[7:4] → 0x20 - // byte1: windowNLog-1(5b)=24=0b11000 → 0x18; stateNLog bits start at bit5 - // bytes2-6: all 0x00 - var sut = new PcoEncoding(); - MemorySegment chunkMeta = segmentOf( - (byte) 0x20, (byte) 0x18, (byte) 0x00, (byte) 0x00, (byte) 0x00, (byte) 0x00, (byte) 0x00); - DecodeContext ctx = ctxWith( - metaWithOneChunk(1), - new DType.Primitive(PType.U64, false), - 1, - new MemorySegment[]{chunkMeta, segmentOf((byte) 0x00)}); - - // When / Then - assertThatThrownBy(() -> sut.decode(ctx)) - .isInstanceOf(VortexException.class) - .hasMessageContaining("windowNLog"); - } - } - - @Nested - class DecodeDict { - - @Test - void dict_nUniqueExceedsMax_throwsVortexException() { - // Given — mode=4 (Dict), nUnique=65537 > max 65536. - // Bit layout (LSB-first): mode[3:0]=4, nUnique[28:4]=65537 - // combined = 4 | (65537 << 4) = 0x100014 - // bytes: 0x14, 0x00, 0x10, 0x00, 0x00 - var sut = new PcoEncoding(); - MemorySegment chunkMeta = segmentOf( - (byte) 0x14, (byte) 0x00, (byte) 0x10, (byte) 0x00, (byte) 0x00); - DecodeContext ctx = ctxWith( - metaWithOneChunk(1), - new DType.Primitive(PType.U64, false), - 1, - new MemorySegment[]{chunkMeta, segmentOf((byte) 0x00)}); - - // When / Then - assertThatThrownBy(() -> sut.decode(ctx)) - .isInstanceOf(VortexException.class) - .hasMessageContaining("nUnique"); - } - } - - /// Adversarial coverage: malformed inputs must throw VortexException — never AIOOBE, NPE, or OOM. - @Nested - class Adversarial { - - static Stream chunkMetaBytesProvider() { - Random rng = new Random(0xDEADBEEFL); - return Stream.generate(() -> { - byte[] b = new byte[1 + rng.nextInt(64)]; - rng.nextBytes(b); - return b; - }).limit(50); - } - - static Stream pageBytesProvider() { - Random rng = new Random(0xCAFEBABEL); - return Stream.generate(() -> { - byte[] b = new byte[4 + rng.nextInt(125)]; - rng.nextBytes(b); - return b; - }).limit(50); - } - - /// Random chunk-meta bytes — any exception must be a VortexException, not a JVM crash exception. - @ParameterizedTest - @MethodSource("chunkMetaBytesProvider") - void randomChunkMetaBytes_neverThrowsJvmException(byte[] chunkMetaBytes) { - // Given — valid pco header + 1 chunk with 1 page of 1 value; garbage chunk-meta bytes. - var sut = new PcoEncoding(); - DecodeContext ctx = ctxWith( - metaWithOneChunk(1), - new DType.Primitive(PType.U64, false), - 1, - new MemorySegment[]{segmentOf(chunkMetaBytes), segmentOf((byte) 0x00)}); - - // When / Then — either succeeds or throws VortexException; never AIOOBE/NPE/OOM - try { - sut.decode(ctx); - } catch (VortexException ignored) { - // expected — malformed input - } - } - - /// Random page bytes after a valid Classic-mode chunk meta — must not crash the JVM. - @ParameterizedTest - @MethodSource("pageBytesProvider") - void randomPageBytes_classicMode_neverThrowsJvmException(byte[] pageBytes) { - // Given — Classic mode, delta=NoOp, ansSizeLog=0, nBins=0 chunk meta. - var sut = new PcoEncoding(); - // byte0: mode=0 (bits3:0), deltaVariant=0 (bits7:4) → 0x00 - // byte1: ansSizeLog=0 (bits3:0), nBins low bits = 0 - // bytes 2-3: nBins high bits = 0 - DecodeContext ctx = ctxWith( - metaWithOneChunk(1), - new DType.Primitive(PType.U64, false), - 1, - new MemorySegment[]{segmentOf((byte) 0x00, (byte) 0x00, (byte) 0x00, (byte) 0x00), - segmentOf(pageBytes)}); - - // When / Then - try { - sut.decode(ctx); - } catch (VortexException ignored) { - // expected — malformed page data - } - } - - /// Invalid mode nibbles (5–15) must produce a VortexException naming the mode number. - @ParameterizedTest - @ValueSource(ints = {5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15}) - void invalidModeNibble_throwsVortexException(int modeNibble) { - // Given — chunk meta with unsupported mode nibble in bits[3:0]. - var sut = new PcoEncoding(); - // bits[3:0] = modeNibble, delta nibble doesn't matter (won't be reached) - byte modeByte = (byte) (modeNibble & 0x0F); - DecodeContext ctx = ctxWith( - metaWithOneChunk(1), - new DType.Primitive(PType.U64, false), - 1, - new MemorySegment[]{ - segmentOf(modeByte, (byte) 0x00, (byte) 0x00, (byte) 0x00), - segmentOf((byte) 0x00)}); - - // When / Then - assertThatThrownBy(() -> sut.decode(ctx)) - .isInstanceOf(VortexException.class) - .hasMessageContaining("pco mode " + modeNibble); - } - - /// Invalid delta variants (4–15) must produce a VortexException naming the variant number. - @ParameterizedTest - @ValueSource(ints = {4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15}) - void invalidDeltaVariant_throwsVortexException(int deltaVariant) { - // Given — Classic mode (nibble=0) + invalid delta nibble in bits[7:4]. - var sut = new PcoEncoding(); - // byte0: bits[3:0]=mode=0, bits[7:4]=deltaVariant - byte modeDeltaByte = (byte) ((deltaVariant & 0x0F) << 4); - DecodeContext ctx = ctxWith( - metaWithOneChunk(1), - new DType.Primitive(PType.U64, false), - 1, - new MemorySegment[]{ - segmentOf(modeDeltaByte, (byte) 0x00, (byte) 0x00, (byte) 0x00), - segmentOf((byte) 0x00)}); - - // When / Then - assertThatThrownBy(() -> sut.decode(ctx)) - .isInstanceOf(VortexException.class) - .hasMessageContaining("delta variant " + deltaVariant); - } - - /// Conv1 delta with 64-bit dtype must throw VortexException; pcodec only supports 16/32-bit Conv1. - @ParameterizedTest - @EnumSource(value = PType.class, names = {"I64", "U64", "F64"}) - void conv1Delta_with64BitDtype_throwsVortexException(PType ptype) { - // Given — Conv1 delta variant (nibble=3 in bits[7:4]), Classic mode (nibble=0 in bits[3:0]). - // byte0: bits[3:0]=0 (Classic), bits[7:4]=3 (Conv1) → 0x30 - // Remaining bytes: conv1 bit fields (don't matter — error fires before parsing them). - var sut = new PcoEncoding(); - DecodeContext ctx = ctxWith( - metaWithOneChunk(1), - new DType.Primitive(ptype, false), - 1, - new MemorySegment[]{ - segmentOf((byte) 0x30, (byte) 0x00, (byte) 0x00, (byte) 0x00, - (byte) 0x00, (byte) 0x00, (byte) 0x00, (byte) 0x00, - (byte) 0x00, (byte) 0x00, (byte) 0x00, (byte) 0x00, - (byte) 0x00, (byte) 0x00, (byte) 0x00, (byte) 0x00), - segmentOf((byte) 0x00)}); - - // When / Then - assertThatThrownBy(() -> sut.decode(ctx)) - .isInstanceOf(VortexException.class) - .hasMessageContaining("Conv1"); - } - } -} diff --git a/core/src/test/java/io/github/dfa1/vortex/encoding/PcoTansDecoderTest.java b/core/src/test/java/io/github/dfa1/vortex/encoding/PcoTansDecoderTest.java deleted file mode 100644 index 6311f3d..0000000 --- a/core/src/test/java/io/github/dfa1/vortex/encoding/PcoTansDecoderTest.java +++ /dev/null @@ -1,190 +0,0 @@ -package io.github.dfa1.vortex.encoding; - -import org.junit.jupiter.api.Nested; -import org.junit.jupiter.api.Test; - -import java.lang.foreign.Arena; -import java.lang.foreign.MemorySegment; - -import static org.assertj.core.api.Assertions.assertThat; - -class PcoTansDecoderTest { - - @Nested - class Spread { - - @Test - void matchesSpecRsReferenceOutput() { - // Given — from pcodec spec.rs test: ans_spec_new - // weights=[1,1,3,11], total=16=2^4 → size_log=4 - int[] weights = {1, 1, 3, 11}; - int tableSize = 16; - - // When - int[] stateSymbols = PcoTansDecoder.spreadStateSymbols(4, weights, tableSize); - - // Then — exact output from pcodec spec.rs reference test - assertThat(stateSymbols).containsExactly(0, 3, 2, 3, 2, 3, 3, 3, 3, 1, 3, 2, 3, 3, 3, 3); - } - - @Test - void trivialSingleSymbol_fillsTable() { - // Given — from pcodec spec.rs test: ans_spec_new_trivial (weight=[1]) - int[] weights = {1}; - - // When - int[] stateSymbols = PcoTansDecoder.spreadStateSymbols(0, weights, 1); - - // Then - assertThat(stateSymbols).containsExactly(0); - } - - @Test - void twoEqualWeights_twoElementTable() { - // Given — weights=[2], total=2=2^1 - int[] weights = {2}; - - // When - int[] stateSymbols = PcoTansDecoder.spreadStateSymbols(1, weights, 2); - - // Then - assertThat(stateSymbols).containsExactly(0, 0); - } - } - - @Nested - class Decode { - - @Test - void singleBin_zeroBits_constantOutput() { - // Given — 1 bin: lower=42, offsetBits=0 → every value is exactly 42 - PcoBin[] bins = {new PcoBin(1, 42L, 0)}; - PcoTansDecoder sut = PcoTansDecoder.build(0, bins); - - MemorySegment pageBuf = Arena.ofAuto().allocate(8); // all-zero bytes sufficient - LeBitReader reader = new LeBitReader(pageBuf); - int[] stateIdxs = {0, 0, 0, 0}; - - int n = 5; - MemorySegment out = Arena.ofAuto().allocate((long) n * Long.BYTES); - - // When - sut.decodePage(reader, stateIdxs, n, out, 0L, - new long[PcoTansDecoder.BATCH_N], new int[PcoTansDecoder.BATCH_N]); - - // Then — all raw latent values = 42 - for (int i = 0; i < n; i++) { - assertThat(out.get(PTypeIO.LE_LONG, (long) i * Long.BYTES)) - .as("latent[%d]", i) - .isEqualTo(42L); - } - } - - @Test - void singleBin_oneBitOffset_decodesOffset() { - // Given — 1 bin: lower=10, offsetBits=1 → values are 10 or 11 depending on offset bit - PcoBin[] bins = {new PcoBin(1, 10L, 1)}; - PcoTansDecoder sut = PcoTansDecoder.build(0, bins); - - // Page buffer: all-zero → offset bits all 0 → all values = 10 - MemorySegment pageBuf = Arena.ofAuto().allocate(8); - LeBitReader reader = new LeBitReader(pageBuf); - int[] stateIdxs = {0, 0, 0, 0}; - - int n = 4; - MemorySegment out = Arena.ofAuto().allocate((long) n * Long.BYTES); - - // When - sut.decodePage(reader, stateIdxs, n, out, 0L, - new long[PcoTansDecoder.BATCH_N], new int[PcoTansDecoder.BATCH_N]); - - // Then — offsets all zero → all values = lower + 0 = 10 - for (int i = 0; i < n; i++) { - assertThat(out.get(PTypeIO.LE_LONG, (long) i * Long.BYTES)) - .as("latent[%d]", i) - .isEqualTo(10L); - } - } - - @Test - void degenerateBins_zeroBins_outputZero() { - // Given — 0 bins → degenerate decoder, output = 0 - PcoTansDecoder sut = PcoTansDecoder.build(0, new PcoBin[0]); - - MemorySegment pageBuf = Arena.ofAuto().allocate(8); - LeBitReader reader = new LeBitReader(pageBuf); - int[] stateIdxs = {0, 0, 0, 0}; - - int n = 3; - MemorySegment out = Arena.ofAuto().allocate((long) n * Long.BYTES); - - // When - sut.decodePage(reader, stateIdxs, n, out, 0L, - new long[PcoTansDecoder.BATCH_N], new int[PcoTansDecoder.BATCH_N]); - - // Then — all zero - for (int i = 0; i < n; i++) { - assertThat(out.get(PTypeIO.LE_LONG, (long) i * Long.BYTES)).isZero(); - } - } - - @Test - void nonDegenerate_ansSizeLog1_twoBins_stateTransitionsStayInBounds() { - // Given — ansSizeLog=1 (tableSize=2), 2 bins weight=1 each. - // bitsToRead > 0 for each state → real ANS state transitions occur. - // All-zero page bytes → ansVal=0 each step → states stay at 0. - // Confirms bounds check doesn't false-positive on valid decode. - PcoBin[] bins = {new PcoBin(1, 5L, 0), new PcoBin(1, 100L, 0)}; - PcoTansDecoder sut = PcoTansDecoder.build(1, bins); - - // Page: 2 ANS-state bits (ansSizeLog=1, 4×1b=4 bits) + offset bits; use 8 all-zero bytes. - MemorySegment pageBuf = Arena.ofAuto().allocate(8); - LeBitReader reader = new LeBitReader(pageBuf); - // Initial states read from page header: readBits(1) × 4 = [0, 0, 0, 0] - int[] stateIdxs = {0, 0, 0, 0}; - for (int i = 0; i < PcoTansDecoder.ANS_INTERLEAVING; i++) { - stateIdxs[i] = (int) reader.readBits(1); - } - reader.alignToByte(); - - int n = 8; - MemorySegment out = Arena.ofAuto().allocate((long) n * Long.BYTES); - - // When - sut.decodeBatch(reader, stateIdxs, n, - new long[n], new int[n], out, 0L); - - // Then — all states started at 0 → sym=0 → lower=5 for every value - for (int i = 0; i < n; i++) { - assertThat(out.get(PTypeIO.LE_LONG, (long) i * Long.BYTES)) - .as("latent[%d]", i) - .isEqualTo(5L); - } - } - - @Test - void moreThanOneBatch_decodesCorrectly() { - // Given — 1 bin, lower=7, n=300 (> BATCH_N=256 → two batches) - PcoBin[] bins = {new PcoBin(1, 7L, 0)}; - PcoTansDecoder sut = PcoTansDecoder.build(0, bins); - - MemorySegment pageBuf = Arena.ofAuto().allocate(8); - LeBitReader reader = new LeBitReader(pageBuf); - int[] stateIdxs = {0, 0, 0, 0}; - - int n = 300; - MemorySegment out = Arena.ofAuto().allocate((long) n * Long.BYTES); - - // When - sut.decodePage(reader, stateIdxs, n, out, 0L, - new long[PcoTansDecoder.BATCH_N], new int[PcoTansDecoder.BATCH_N]); - - // Then — all values = 7 - for (int i = 0; i < n; i++) { - assertThat(out.get(PTypeIO.LE_LONG, (long) i * Long.BYTES)) - .as("latent[%d]", i) - .isEqualTo(7L); - } - } - } -} diff --git a/core/src/test/java/io/github/dfa1/vortex/encoding/PrimitiveEncodingTest.java b/core/src/test/java/io/github/dfa1/vortex/encoding/PrimitiveEncodingTest.java deleted file mode 100644 index 8b80d20..0000000 --- a/core/src/test/java/io/github/dfa1/vortex/encoding/PrimitiveEncodingTest.java +++ /dev/null @@ -1,202 +0,0 @@ -package io.github.dfa1.vortex.encoding; - -import io.github.dfa1.vortex.core.ArrayStats; -import io.github.dfa1.vortex.core.DType; -import io.github.dfa1.vortex.core.PType; -import io.github.dfa1.vortex.core.array.Array; -import io.github.dfa1.vortex.core.array.IntArray; -import io.github.dfa1.vortex.core.array.MaskedArray; -import org.junit.jupiter.api.Nested; -import org.junit.jupiter.api.Test; -import org.junit.jupiter.params.ParameterizedTest; -import org.junit.jupiter.params.provider.MethodSource; - -import java.lang.foreign.Arena; -import java.lang.foreign.MemorySegment; -import java.util.stream.Stream; - -import static org.assertj.core.api.Assertions.assertThat; - -/// Property: encode then decode is lossless for all primitive types and array sizes. -class PrimitiveEncodingTest { - - @Nested - class Encode { - - @ParameterizedTest - @MethodSource("longArrays") - void encodeDecode_i64_isLossless(long[] data) { - // Given - DType dtype = new DType.Primitive(PType.I64, false); - var sut = new PrimitiveEncoding(); - EncodingRegistry registry = TestRegistry.of(sut); - - // When - EncodeResult encoded = sut.encode(dtype, data); - DecodeContext ctx = EncodeTestHelper.toDecodeContext(encoded, data.length, dtype, registry); - Array result = sut.decode(ctx); - - // Then — roundtrip lossless - assertThat(result.length()).isEqualTo(data.length); - var le = PTypeIO.LE_LONG; - for (int i = 0; i < data.length; i++) { - assertThat(result.buffer(0).get(le, (long) i * 8)).isEqualTo(data[i]); - } - } - - @ParameterizedTest - @MethodSource("intArrays") - void encodeDecode_i32_isLossless(int[] data) { - // Given - DType dtype = new DType.Primitive(PType.I32, false); - var sut = new PrimitiveEncoding(); - EncodingRegistry registry = TestRegistry.of(sut); - - // When - EncodeResult encoded = sut.encode(dtype, data); - DecodeContext ctx = EncodeTestHelper.toDecodeContext(encoded, data.length, dtype, registry); - Array result = sut.decode(ctx); - - // Then — roundtrip lossless - assertThat(result.length()).isEqualTo(data.length); - var le = PTypeIO.LE_INT; - for (int i = 0; i < data.length; i++) { - assertThat(result.buffer(0).get(le, (long) i * 4)).isEqualTo(data[i]); - } - } - - @ParameterizedTest - @MethodSource("doubleArrays") - void encodeDecode_f64_isLossless(double[] data) { - // Given - DType dtype = new DType.Primitive(PType.F64, false); - var sut = new PrimitiveEncoding(); - EncodingRegistry registry = TestRegistry.of(sut); - - // When - EncodeResult encoded = sut.encode(dtype, data); - DecodeContext ctx = EncodeTestHelper.toDecodeContext(encoded, data.length, dtype, registry); - Array result = sut.decode(ctx); - - // Then — roundtrip lossless - assertThat(result.length()).isEqualTo(data.length); - var le = PTypeIO.LE_DOUBLE; - for (int i = 0; i < data.length; i++) { - assertThat(result.buffer(0).get(le, (long) i * 8)).isEqualTo(data[i]); - } - } - - @ParameterizedTest - @MethodSource("longArrays") - void encodedSize_equalsBytesInBuffer(long[] data) { - // Given - DType dtype = new DType.Primitive(PType.I64, false); - var sut = new PrimitiveEncoding(); - - // When - EncodeResult encoded = sut.encode(dtype, data); - - // Then — no compression: wire size = n * elemBytes - long totalBytes = encoded.buffers().stream().mapToLong(java.lang.foreign.MemorySegment::byteSize).sum(); - assertThat(totalBytes).isEqualTo((long) data.length * 8); - } - - static Stream longArrays() { - return Stream.of( - new long[]{}, - new long[]{0}, - new long[]{Long.MIN_VALUE, Long.MAX_VALUE}, - new long[]{-1, 0, 1, 2, 3, 4, 5}, - new long[]{1, 1000, -1000, 42, Long.MAX_VALUE / 2} - ); - } - - static Stream intArrays() { - return Stream.of( - new int[]{}, - new int[]{0}, - new int[]{Integer.MIN_VALUE, Integer.MAX_VALUE}, - new int[]{-1, 0, 1, 100, -100}, - new int[]{7, 14, 21, 42, 99, -7} - ); - } - - static Stream doubleArrays() { - return Stream.of( - new double[]{}, - new double[]{0.0}, - new double[]{Double.MIN_VALUE, Double.MAX_VALUE}, - new double[]{-1.5, 0.0, 1.5, 3.14159, -2.71828}, - new double[]{1e10, -1e10, 1e-10} - ); - } - } - - @Nested - class Decode { - - @Test - void decode_withValidityChild_returnsMaskedArray() { - // Given — 4 I32 values; positions 1 and 3 are null (validity bitmap: 0b00000101 = 0x05) - int[] raw = {10, 0, 20, 0}; // garbage at null positions - MemorySegment valuesSeg = TestSegments.leInts(raw); - MemorySegment validitySeg = MemorySegment.ofArray(new byte[]{0x05}); // bits 0,2 set - - ArrayNode validityNode = ArrayNode.of( - EncodingId.VORTEX_BOOL, null, new ArrayNode[0], new int[]{1}, ArrayStats.empty()); - ArrayNode primNode = ArrayNode.of( - EncodingId.VORTEX_PRIMITIVE, null, new ArrayNode[]{validityNode}, new int[]{0}, ArrayStats.empty()); - - EncodingRegistry registry = TestRegistry.of(new PrimitiveEncoding(), new BoolEncoding()); - - DType dtype = new DType.Primitive(PType.I32, false); - DecodeContext ctx = new DecodeContext( - primNode, dtype, raw.length, - new MemorySegment[]{valuesSeg, validitySeg}, - registry, Arena.global()); - - PrimitiveEncoding sut = new PrimitiveEncoding(); - - // When - Array result = sut.decode(ctx); - - // Then — returns MaskedArray; only valid positions are usable - assertThat(result).isInstanceOf(MaskedArray.class); - MaskedArray masked = (MaskedArray) result; - assertThat(masked.child(0)).isInstanceOf(IntArray.class); - assertThat(masked.isValid(0)).isTrue(); - assertThat(masked.isValid(1)).isFalse(); - assertThat(masked.isValid(2)).isTrue(); - assertThat(masked.isValid(3)).isFalse(); - IntArray values = (IntArray) masked.child(0); - assertThat(values.getInt(0)).isEqualTo(10); - assertThat(values.getInt(2)).isEqualTo(20); - } - - @Test - void decode_noValidityChild_returnsPlainArray() { - // Given — 3 I32 values; no validity child - int[] raw = {1, 2, 3}; - MemorySegment valuesSeg = TestSegments.leInts(raw); - - ArrayNode primNode = ArrayNode.of( - EncodingId.VORTEX_PRIMITIVE, null, new ArrayNode[0], new int[]{0}, ArrayStats.empty()); - - EncodingRegistry registry = TestRegistry.of(new PrimitiveEncoding()); - - DType dtype = new DType.Primitive(PType.I32, false); - DecodeContext ctx = new DecodeContext( - primNode, dtype, raw.length, - new MemorySegment[]{valuesSeg}, - registry, Arena.global()); - - PrimitiveEncoding sut = new PrimitiveEncoding(); - - // When - Array result = sut.decode(ctx); - - // Then — plain array, not MaskedArray - assertThat(result).isInstanceOf(IntArray.class); - } - } -} diff --git a/core/src/test/java/io/github/dfa1/vortex/encoding/RleEncodingTest.java b/core/src/test/java/io/github/dfa1/vortex/encoding/RleEncodingTest.java deleted file mode 100644 index c3e4cf2..0000000 --- a/core/src/test/java/io/github/dfa1/vortex/encoding/RleEncodingTest.java +++ /dev/null @@ -1,345 +0,0 @@ -package io.github.dfa1.vortex.encoding; - -import io.github.dfa1.vortex.core.ArrayStats; -import io.github.dfa1.vortex.core.DType; -import io.github.dfa1.vortex.core.PType; -import io.github.dfa1.vortex.core.array.Array; -import io.github.dfa1.vortex.core.array.IntArray; -import io.github.dfa1.vortex.core.array.MaskedArray; -import org.junit.jupiter.api.Nested; -import org.junit.jupiter.api.Test; -import org.junit.jupiter.params.ParameterizedTest; -import org.junit.jupiter.params.provider.ValueSource; - -import java.lang.foreign.Arena; -import java.lang.foreign.MemorySegment; -import java.util.ArrayList; -import java.util.List; - -import static org.assertj.core.api.Assertions.assertThat; - -class RleEncodingTest { - - private static EncodingRegistry registry() { - EncodingRegistry r = EncodingRegistry.empty(); - r.register(new RleEncoding()); - r.register(new PrimitiveEncoding()); - return r; - } - - private static KnownArrayNode toArrayNode(EncodeNode enc) { - ArrayNode[] children = new ArrayNode[enc.children().length]; - for (int i = 0; i < children.length; i++) { - children[i] = toArrayNode(enc.children()[i]); - } - return new KnownArrayNode(enc.encodingId(), enc.metadata(), children, enc.bufferIndices(), ArrayStats.empty()); - } - - @Nested - class Encode { - - @Test - void roundTrip_empty_i32() { - // Given - var sut = new RleEncoding(); - DType dtype = new DType.Primitive(PType.I32, false); - - // When - EncodeResult encoded = sut.encode(dtype, new int[0]); - DecodeContext ctx = EncodeTestHelper.toDecodeContext(encoded, 0, dtype, registry()); - Array result = sut.decode(ctx); - - // Then - assertThat(result.length()).isZero(); - } - - @Test - void roundTrip_singleElement_i32() { - // Given - var sut = new RleEncoding(); - DType dtype = new DType.Primitive(PType.I32, false); - int[] data = {42}; - - // When - EncodeResult encoded = sut.encode(dtype, data); - DecodeContext ctx = EncodeTestHelper.toDecodeContext(encoded, data.length, dtype, registry()); - Array result = sut.decode(ctx); - - // Then - assertThat(result.length()).isEqualTo(1); - assertThat(result.buffer(0).get(PTypeIO.LE_INT, 0)).isEqualTo(42); - } - - @Test - void roundTrip_constantArray_i32() { - // Given - var sut = new RleEncoding(); - DType dtype = new DType.Primitive(PType.I32, false); - int n = 2048; - int[] data = new int[n]; - for (int i = 0; i < n; i++) { - data[i] = 99; - } - - // When - EncodeResult encoded = sut.encode(dtype, data); - DecodeContext ctx = EncodeTestHelper.toDecodeContext(encoded, n, dtype, registry()); - Array result = sut.decode(ctx); - - // Then - assertThat(result.length()).isEqualTo(n); - for (int i = 0; i < n; i++) { - assertThat(result.buffer(0).get(PTypeIO.LE_INT, (long) i * 4)).as("index %d", i).isEqualTo(99); - } - } - - @Test - void roundTrip_classicRunLengthData_i32() { - // Given - var sut = new RleEncoding(); - DType dtype = new DType.Primitive(PType.I32, false); - int[] data = {1, 1, 1, 2, 2, 3}; - - // When - EncodeResult encoded = sut.encode(dtype, data); - DecodeContext ctx = EncodeTestHelper.toDecodeContext(encoded, data.length, dtype, registry()); - Array result = sut.decode(ctx); - - // Then - assertThat(result.length()).isEqualTo(data.length); - int[] expected = {1, 1, 1, 2, 2, 3}; - for (int i = 0; i < expected.length; i++) { - assertThat(result.buffer(0).get(PTypeIO.LE_INT, (long) i * 4)).as("index %d", i).isEqualTo(expected[i]); - } - } - - @Test - void roundTrip_multipleChunks_i32() { - // Given: spans 3 chunks - var sut = new RleEncoding(); - DType dtype = new DType.Primitive(PType.I32, false); - int n = 3000; - int[] data = new int[n]; - for (int i = 0; i < n; i++) { - data[i] = i / 100; - } - - // When - EncodeResult encoded = sut.encode(dtype, data); - DecodeContext ctx = EncodeTestHelper.toDecodeContext(encoded, n, dtype, registry()); - Array result = sut.decode(ctx); - - // Then - assertThat(result.length()).isEqualTo(n); - for (int i = 0; i < n; i++) { - assertThat(result.buffer(0).get(PTypeIO.LE_INT, (long) i * 4)).as("index %d", i).isEqualTo(i / 100); - } - } - - @Test - void roundTrip_i64() { - // Given - var sut = new RleEncoding(); - DType dtype = new DType.Primitive(PType.I64, false); - long[] data = {100L, 100L, 200L, 300L, 300L, 300L}; - - // When - EncodeResult encoded = sut.encode(dtype, data); - DecodeContext ctx = EncodeTestHelper.toDecodeContext(encoded, data.length, dtype, registry()); - Array result = sut.decode(ctx); - - // Then - assertThat(result.length()).isEqualTo(data.length); - for (int i = 0; i < data.length; i++) { - assertThat(result.buffer(0).get(PTypeIO.LE_LONG, (long) i * 8)).as("index %d", i).isEqualTo(data[i]); - } - } - - @ParameterizedTest - @ValueSource(ints = {1, 512, 1023, 1024, 1025, 2048, 2049}) - void roundTrip_variousLengths_i32(int n) { - // Given - var sut = new RleEncoding(); - DType dtype = new DType.Primitive(PType.I32, false); - int[] data = new int[n]; - for (int i = 0; i < n; i++) { - data[i] = i / 50; - } - - // When - EncodeResult encoded = sut.encode(dtype, data); - DecodeContext ctx = EncodeTestHelper.toDecodeContext(encoded, n, dtype, registry()); - Array result = sut.decode(ctx); - - // Then - assertThat(result.length()).isEqualTo(n); - for (int i = 0; i < n; i++) { - assertThat(result.buffer(0).get(PTypeIO.LE_INT, (long) i * 4)).as("index %d", i).isEqualTo(i / 50); - } - } - - @Test - void roundTrip_allDifferent_u16() { - // Given: worst case — every consecutive value is unique (no compression) - var sut = new RleEncoding(); - DType dtype = new DType.Primitive(PType.U16, false); - short[] data = new short[256]; - for (int i = 0; i < 256; i++) { - data[i] = (short) i; - } - - // When - EncodeResult encoded = sut.encode(dtype, data); - DecodeContext ctx = EncodeTestHelper.toDecodeContext(encoded, data.length, dtype, registry()); - Array result = sut.decode(ctx); - - // Then - assertThat(result.length()).isEqualTo(data.length); - var le = PTypeIO.LE_SHORT; - for (int i = 0; i < data.length; i++) { - assertThat(Short.toUnsignedInt(result.buffer(0).get(le, (long) i * 2))) - .as("index %d", i).isEqualTo(i); - } - } - - @Test - void roundTrip_negativeValues_i32() { - // Given - var sut = new RleEncoding(); - DType dtype = new DType.Primitive(PType.I32, false); - int[] data = {-3, -3, -1, -1, 0, 0, 5}; - - // When - EncodeResult encoded = sut.encode(dtype, data); - DecodeContext ctx = EncodeTestHelper.toDecodeContext(encoded, data.length, dtype, registry()); - Array result = sut.decode(ctx); - - // Then - assertThat(result.length()).isEqualTo(data.length); - for (int i = 0; i < data.length; i++) { - assertThat(result.buffer(0).get(PTypeIO.LE_INT, (long) i * 4)).as("index %d", i).isEqualTo(data[i]); - } - } - } - - @Nested - class Decode { - - @Test - void decode_exactlyOneChunk_correctLength() { - // Given - var sut = new RleEncoding(); - DType dtype = new DType.Primitive(PType.I32, false); - int[] data = new int[1024]; - for (int i = 0; i < 1024; i++) { - data[i] = i / 10; - } - - // When - EncodeResult encoded = sut.encode(dtype, data); - DecodeContext ctx = EncodeTestHelper.toDecodeContext(encoded, 1024, dtype, registry()); - Array result = sut.decode(ctx); - - // Then - assertThat(result.length()).isEqualTo(1024); - } - - @Test - void decode_crossesChunkBoundary_correctValues() { - // Given: values span the chunk boundary at element 1024 - var sut = new RleEncoding(); - DType dtype = new DType.Primitive(PType.I32, false); - int n = 2048; - int[] data = new int[n]; - for (int i = 0; i < n; i++) { - data[i] = i / 100; - } - - // When - EncodeResult encoded = sut.encode(dtype, data); - DecodeContext ctx = EncodeTestHelper.toDecodeContext(encoded, n, dtype, registry()); - Array result = sut.decode(ctx); - - // Then — verify values near the chunk boundary - for (int i = 1000; i < 1048; i++) { - assertThat(result.buffer(0).get(PTypeIO.LE_INT, (long) i * 4)) - .as("index %d", i).isEqualTo(i / 100); - } - } - - @Test - void decode_nullableIndices_returnsMaskedArrayWithCorrectValidity() { - // Given — encode [10, 10, 20, 20]; then inject a validity bitmap into the indices node - // so positions 1 and 3 are null. Valid outputs: [10, null, 20, null]. - var sut = new RleEncoding(); - DType dtype = new DType.Primitive(PType.I32, false); - int[] data = {10, 10, 20, 20}; - EncodeResult encoded = sut.encode(dtype, data); - - // Original buffers: [values_buf, indices_buf, offsets_buf] - List originalBufs = new ArrayList<>(encoded.buffers()); - // Validity bitmap: bits 0 and 2 set → 0b00000101 = 0x05 → positions 0,2 valid - MemorySegment validityBuf = MemorySegment.ofArray(new byte[]{0x05}); - originalBufs.add(validityBuf); // index 3 - MemorySegment[] segments = originalBufs.toArray(new MemorySegment[0]); - - // Rebuild the ArrayNode tree from the encode result - KnownArrayNode origRoot = toArrayNode(encoded.rootNode()); - // RLE root children: [values(0), indices(1), offsets(2)] - KnownArrayNode origIndices = (KnownArrayNode) origRoot.children()[1]; - // Wrap indices with a validity child pointing to buffer 3 - ArrayNode validityNode = ArrayNode.of( - EncodingId.VORTEX_BOOL, null, new ArrayNode[0], new int[]{3}, ArrayStats.empty()); - ArrayNode nullableIndices = ArrayNode.of( - origIndices.encodingId(), origIndices.metadata(), - new ArrayNode[]{validityNode}, origIndices.bufferIndices(), ArrayStats.empty()); - ArrayNode root = ArrayNode.of( - origRoot.encodingId(), origRoot.metadata(), - new ArrayNode[]{origRoot.children()[0], nullableIndices, origRoot.children()[2]}, - origRoot.bufferIndices(), ArrayStats.empty()); - - EncodingRegistry reg = EncodingRegistry.empty(); - reg.register(sut); - reg.register(new PrimitiveEncoding()); - reg.register(new BoolEncoding()); - DecodeContext ctx = new DecodeContext(root, dtype, data.length, segments, reg, Arena.ofAuto()); - - // When - Array result = sut.decode(ctx); - - // Then — MaskedArray; valid at positions 0 and 2 - assertThat(result).isInstanceOf(MaskedArray.class); - MaskedArray masked = (MaskedArray) result; - assertThat(masked.isValid(0)).isTrue(); - assertThat(masked.isValid(1)).isFalse(); - assertThat(masked.isValid(2)).isTrue(); - assertThat(masked.isValid(3)).isFalse(); - IntArray values = (IntArray) masked.child(0); - assertThat(values.getInt(0)).isEqualTo(10); - assertThat(values.getInt(2)).isEqualTo(20); - } - - @Test - void decode_partialLastChunk_correctLength() { - // Given: 1500 elements — two chunks (1024 full + 476 partial) - var sut = new RleEncoding(); - DType dtype = new DType.Primitive(PType.I32, false); - int n = 1500; - int[] data = new int[n]; - for (int i = 0; i < n; i++) { - data[i] = i / 100; - } - - // When - EncodeResult encoded = sut.encode(dtype, data); - DecodeContext ctx = EncodeTestHelper.toDecodeContext(encoded, n, dtype, registry()); - Array result = sut.decode(ctx); - - // Then - assertThat(result.length()).isEqualTo(n); - for (int i = 0; i < n; i++) { - assertThat(result.buffer(0).get(PTypeIO.LE_INT, (long) i * 4)).as("index %d", i).isEqualTo(i / 100); - } - } - } -} diff --git a/core/src/test/java/io/github/dfa1/vortex/encoding/RunEndEncodingTest.java b/core/src/test/java/io/github/dfa1/vortex/encoding/RunEndEncodingTest.java deleted file mode 100644 index 4cc74c1..0000000 --- a/core/src/test/java/io/github/dfa1/vortex/encoding/RunEndEncodingTest.java +++ /dev/null @@ -1,170 +0,0 @@ -package io.github.dfa1.vortex.encoding; - -import io.github.dfa1.vortex.proto.DTypeProtos; -import io.github.dfa1.vortex.proto.EncodingProtos; -import io.github.dfa1.vortex.core.array.Array; -import io.github.dfa1.vortex.core.ArrayStats; -import io.github.dfa1.vortex.core.DType; -import io.github.dfa1.vortex.core.PType; -import org.junit.jupiter.api.Nested; -import org.junit.jupiter.api.Test; -import org.junit.jupiter.params.ParameterizedTest; -import org.junit.jupiter.params.provider.MethodSource; - -import java.lang.foreign.MemorySegment; -import java.nio.ByteBuffer; -import java.nio.ByteOrder; -import java.util.stream.Stream; - -import static org.assertj.core.api.Assertions.assertThat; - -class RunEndEncodingTest { - - @Nested - class Decode { - - @Test - void decode_singleRun_fillsAllElements() { - // Given — 1 run: ends=[5], values=[42]; output = [42, 42, 42, 42, 42] - long[] ends = {5L}; - long[] values = {42L}; - DecodeContext ctx = buildCtx(DTypes.I64, 5, ends, values, PType.U32, 0L); - RunEndEncoding sut = new RunEndEncoding(); - - // When - Array result = sut.decode(ctx); - - // Then - assertThat(result.length()).isEqualTo(5L); - var layout = PTypeIO.LE_LONG; - for (int i = 0; i < 5; i++) { - assertThat(result.buffer(0).get(layout, (long) i * 8)).isEqualTo(42L); - } - } - - @Test - void decode_multipleRuns_expandsCorrectly() { - // Given — runs: [0,2)=10, [2,5)=20, [5,7)=30 - // ends=[2,5,7], values=[10,20,30] - long[] ends = {2L, 5L, 7L}; - long[] values = {10L, 20L, 30L}; - DecodeContext ctx = buildCtx(DTypes.I64, 7, ends, values, PType.U32, 0L); - RunEndEncoding sut = new RunEndEncoding(); - - // When - Array result = sut.decode(ctx); - - // Then - long[] expected = {10, 10, 20, 20, 20, 30, 30}; - var layout = PTypeIO.LE_LONG; - for (int i = 0; i < expected.length; i++) { - assertThat(result.buffer(0).get(layout, (long) i * 8)) - .as("index %d", i).isEqualTo(expected[i]); - } - } - - @Test - void decode_withOffset_skipsLogicalElements() { - // Given — logical array: [0,3)=10, [3,6)=20; offset=2, rowCount=3 - // output elements [2..5): [10, 20, 20] - long[] ends = {3L, 6L}; - long[] values = {10L, 20L}; - DecodeContext ctx = buildCtx(DTypes.I64, 3, ends, values, PType.U32, 2L); - RunEndEncoding sut = new RunEndEncoding(); - - // When - Array result = sut.decode(ctx); - - // Then - long[] expected = {10L, 20L, 20L}; - var layout = PTypeIO.LE_LONG; - for (int i = 0; i < expected.length; i++) { - assertThat(result.buffer(0).get(layout, (long) i * 8)) - .as("index %d", i).isEqualTo(expected[i]); - } - } - - private static DecodeContext buildCtx( - DType dtype, long rowCount, - long[] ends, long[] values, PType endsPtype, long offset - ) { - byte[] metaBytes = EncodingProtos.RunEndMetadata.newBuilder() - .setEndsPtype(DTypeProtos.PType.forNumber(endsPtype.ordinal())) - .setNumRuns(ends.length) - .setOffset(offset) - .build() - .toByteArray(); - - byte[] endsBuf = toLEBytes(ends, endsPtype); - byte[] valBuf = toLEBytes(values, PType.I64); - - ArrayNode endsNode = ArrayNode.of(EncodingId.VORTEX_PRIMITIVE, null, - new ArrayNode[0], new int[]{0}, ArrayStats.empty()); - ArrayNode valsNode = ArrayNode.of(EncodingId.VORTEX_PRIMITIVE, null, - new ArrayNode[0], new int[]{1}, ArrayStats.empty()); - ArrayNode reNode = ArrayNode.of(EncodingId.VORTEX_RUNEND, - ByteBuffer.wrap(metaBytes), - new ArrayNode[]{endsNode, valsNode}, - new int[0], ArrayStats.empty()); - - MemorySegment[] segments = { - MemorySegment.ofArray(endsBuf), - MemorySegment.ofArray(valBuf) - }; - - EncodingRegistry registry = TestRegistry.of(new RunEndEncoding(), new PrimitiveEncoding()); - - return new DecodeContext(reNode, dtype, rowCount, segments, registry, java.lang.foreign.Arena.global()); - } - - private static byte[] toLEBytes(long[] values, PType ptype) { - int elemBytes = ptype.byteSize(); - byte[] buf = new byte[values.length * elemBytes]; - ByteBuffer bb = ByteBuffer.wrap(buf).order(ByteOrder.LITTLE_ENDIAN); - for (long v : values) { - switch (ptype) { - case U8, I8 -> bb.put((byte) v); - case U16, I16 -> bb.putShort((short) v); - case U32, I32 -> bb.putInt((int) v); - case U64, I64 -> bb.putLong(v); - default -> throw new UnsupportedOperationException(ptype.name()); - } - } - return buf; - } - } - - @Nested - class Encode { - - @ParameterizedTest - @MethodSource("i64Arrays") - void encodeDecode_i64_isLossless(long[] data) { - // Given - var sut = new RunEndEncoding(); - EncodingRegistry registry = TestRegistry.withPrimitive(sut); - var le = PTypeIO.LE_LONG; - - // When - EncodeResult encoded = sut.encode(DTypes.I64, data); - DecodeContext ctx = EncodeTestHelper.toDecodeContext(encoded, data.length, DTypes.I64, registry); - Array result = sut.decode(ctx); - - // Then - assertThat(result.length()).isEqualTo(data.length); - for (int i = 0; i < data.length; i++) { - assertThat(result.buffer(0).get(le, (long) i * 8)).as("index %d", i).isEqualTo(data[i]); - } - } - - static Stream i64Arrays() { - return Stream.of( - new long[]{42L}, - new long[]{1L, 1L, 2L, 2L, 3L}, - new long[]{0L, 0L, 0L, 0L, 0L}, - new long[]{10L, 20L, 10L, 20L}, - new long[]{-1L, -1L, -1L, 0L, 0L, 1L} - ); - } - } -} diff --git a/core/src/test/java/io/github/dfa1/vortex/encoding/SequenceEncodingTest.java b/core/src/test/java/io/github/dfa1/vortex/encoding/SequenceEncodingTest.java deleted file mode 100644 index 8cdc61a..0000000 --- a/core/src/test/java/io/github/dfa1/vortex/encoding/SequenceEncodingTest.java +++ /dev/null @@ -1,275 +0,0 @@ -package io.github.dfa1.vortex.encoding; - -import io.github.dfa1.vortex.core.array.DoubleArray; -import io.github.dfa1.vortex.core.array.FloatArray; -import io.github.dfa1.vortex.core.array.IntArray; -import io.github.dfa1.vortex.core.array.LongArray; -import io.github.dfa1.vortex.proto.EncodingProtos; -import io.github.dfa1.vortex.proto.ScalarProtos; -import io.github.dfa1.vortex.core.DType; -import io.github.dfa1.vortex.core.PType; -import io.github.dfa1.vortex.core.VortexException; -import io.github.dfa1.vortex.core.array.Array; -import org.junit.jupiter.api.Nested; -import org.junit.jupiter.api.Test; -import org.junit.jupiter.params.ParameterizedTest; -import org.junit.jupiter.params.provider.Arguments; -import org.junit.jupiter.params.provider.MethodSource; - -import java.lang.foreign.Arena; -import java.lang.foreign.MemorySegment; -import java.nio.ByteBuffer; -import java.util.stream.Stream; - -import static org.assertj.core.api.Assertions.assertThat; -import static org.assertj.core.api.Assertions.assertThatThrownBy; - -class SequenceEncodingTest { - - private static final DType I64_DTYPE = new DType.Primitive(PType.I64, false); - private static final DType I32_DTYPE = new DType.Primitive(PType.I32, false); - private static final DType F64_DTYPE = new DType.Primitive(PType.F64, false); - private static final DType F32_DTYPE = new DType.Primitive(PType.F32, false); - - @Nested - class Encode { - - @Test - void encodingId_isVortexSequence() { - // Given / When / Then - assertThat(new SequenceEncoding().encodingId()).isEqualTo(EncodingId.VORTEX_SEQUENCE); - } - - @Test - void encode_i64_roundTrips() { - // Given - var sut = new SequenceEncoding(); - long[] data = {10L, 12L, 14L, 16L}; - - // When - EncodeResult result = sut.encode(I64_DTYPE, data); - DecodeContext ctx = encodeResultToCtx(result, I64_DTYPE, data.length); - LongArray decoded = (LongArray) sut.decode(ctx); - - // Then - for (int i = 0; i < data.length; i++) { - assertThat(decoded.getLong(i)).as("index %d", i).isEqualTo(data[i]); - } - } - - @Test - void encode_f64_roundTrips() { - // Given - var sut = new SequenceEncoding(); - double[] data = {1.0, 1.5, 2.0, 2.5}; - - // When - EncodeResult result = sut.encode(F64_DTYPE, data); - DecodeContext ctx = encodeResultToCtx(result, F64_DTYPE, data.length); - DoubleArray decoded = (DoubleArray) sut.decode(ctx); - - // Then - for (int i = 0; i < data.length; i++) { - assertThat(decoded.getDouble(i)).as("index %d", i).isEqualTo(data[i]); - } - } - - @Test - void encode_nonArithmeticSequence_throwsVortexException() { - // Given - var sut = new SequenceEncoding(); - long[] data = {1L, 2L, 4L}; - - // When / Then - assertThatThrownBy(() -> sut.encode(I64_DTYPE, data)) - .isInstanceOf(VortexException.class); - } - - @Test - void encode_nonPrimitiveDtype_throwsVortexException() { - // Given - var sut = new SequenceEncoding(); - - // When / Then - assertThatThrownBy(() -> sut.encode(new DType.Utf8(false), new long[]{1L})) - .isInstanceOf(VortexException.class); - } - - private static DecodeContext encodeResultToCtx(EncodeResult result, DType dtype, long n) { - ByteBuffer meta = result.rootNode().metadata(); - ArrayNode node = ArrayNode.of(EncodingId.VORTEX_SEQUENCE, meta, new ArrayNode[0], new int[0], null); - return new DecodeContext(node, dtype, n, new MemorySegment[0], EncodingRegistry.empty(), Arena.ofAuto()); - } - } - - @Nested - class Decode { - - @ParameterizedTest - @MethodSource("i64Sequences") - void decode_i64_generatesCorrectSequence(long base, long mul, long[] expected) { - // Given - var sut = new SequenceEncoding(); - DecodeContext ctx = makeCtx(intMeta(base, mul), I64_DTYPE, expected.length); - - // When - Array result = sut.decode(ctx); - - // Then - assertThat(result.length()).isEqualTo(expected.length); - for (int i = 0; i < expected.length; i++) { - LongArray longArray = (LongArray) result; - assertThat(longArray.getLong(i)).as("index %d", i).isEqualTo(expected[i]); - } - } - - @ParameterizedTest - @MethodSource("i32Sequences") - void decode_i32_generatesCorrectSequence(long base, long mul, int[] expected) { - // Given - var sut = new SequenceEncoding(); - DecodeContext ctx = makeCtx(intMeta(base, mul), I32_DTYPE, expected.length); - - // When - Array result = sut.decode(ctx); - - // Then - assertThat(result.length()).isEqualTo(expected.length); - for (int i = 0; i < expected.length; i++) { - IntArray longArray = (IntArray) result; - assertThat(longArray.getInt(i)).as("index %d", i).isEqualTo(expected[i]); - } - } - - @Test - void decode_f64_generatesCorrectSequence() { - // Given - var sut = new SequenceEncoding(); - DecodeContext ctx = makeCtx(f64Meta(1.0, 0.5), F64_DTYPE, 4); - - // When - Array result = sut.decode(ctx); - - // Then - assertThat(result.length()).isEqualTo(4); - DoubleArray doubleArray = (DoubleArray) result; - assertThat(doubleArray.getDouble(0)).isEqualTo(1.0); - assertThat(doubleArray.getDouble(1)).isEqualTo(1.5); - assertThat(doubleArray.getDouble(2)).isEqualTo(2.0); - assertThat(doubleArray.getDouble(3)).isEqualTo(2.5); - } - - @Test - void decode_f32_generatesCorrectSequence() { - // Given - var sut = new SequenceEncoding(); - DecodeContext ctx = makeCtx(f32Meta(0.0f, 1.0f), F32_DTYPE, 3); - - // When - Array result = sut.decode(ctx); - - // Then - assertThat(result.length()).isEqualTo(3); - FloatArray floatArray = (FloatArray) result; - assertThat(floatArray.getFloat(0)).isEqualTo(0.0f); - assertThat(floatArray.getFloat(1)).isEqualTo(1.0f); - assertThat(floatArray.getFloat(2)).isEqualTo(2.0f); - } - - @Test - void decode_emptySequence_returnsZeroLengthArray() { - // Given - var sut = new SequenceEncoding(); - DecodeContext ctx = makeCtx(intMeta(0, 1), I64_DTYPE, 0); - - // When - Array result = sut.decode(ctx); - - // Then - assertThat(result.length()).isZero(); - } - - @Test - void decode_missingMetadata_throwsVortexException() { - // Given - var sut = new SequenceEncoding(); - ArrayNode node = ArrayNode.of(EncodingId.VORTEX_SEQUENCE, null, new ArrayNode[0], new int[0], null); - DecodeContext ctx = new DecodeContext(node, I64_DTYPE, 3, new MemorySegment[0], EncodingRegistry.empty(), Arena.ofAuto()); - - // When / Then - assertThatThrownBy(() -> sut.decode(ctx)) - .isInstanceOf(VortexException.class); - } - - @Test - void decode_f16_throwsVortexException() { - // Given - var sut = new SequenceEncoding(); - DType f16 = new DType.Primitive(PType.F16, false); - DecodeContext ctx = makeCtx(intMeta(0, 1), f16, 3); - - // When / Then - assertThatThrownBy(() -> sut.decode(ctx)) - .isInstanceOf(VortexException.class); - } - - @Test - void decode_nonPrimitiveDtype_throwsVortexException() { - // Given - var sut = new SequenceEncoding(); - DType utf8 = new DType.Utf8(false); - DecodeContext ctx = makeCtx(intMeta(0, 1), utf8, 3); - - // When / Then - assertThatThrownBy(() -> sut.decode(ctx)) - .isInstanceOf(VortexException.class); - } - - static Stream i64Sequences() { - return Stream.of( - Arguments.of(0L, 1L, new long[]{0, 1, 2, 3, 4}), - Arguments.of(10L, 2L, new long[]{10, 12, 14, 16}), - Arguments.of(-5L, -1L, new long[]{-5, -6, -7}), - Arguments.of(100L, 0L, new long[]{100, 100, 100}), - Arguments.of(Long.MAX_VALUE, 0L, new long[]{Long.MAX_VALUE}) - ); - } - - static Stream i32Sequences() { - return Stream.of( - Arguments.of(0L, 1L, new int[]{0, 1, 2, 3}), - Arguments.of(100L, -10L, new int[]{100, 90, 80}), - Arguments.of((long) Integer.MAX_VALUE, 0L, new int[]{Integer.MAX_VALUE, Integer.MAX_VALUE}) - ); - } - - private static DecodeContext makeCtx(byte[] meta, DType dtype, long n) { - ArrayNode node = ArrayNode.of( - EncodingId.VORTEX_SEQUENCE, - ByteBuffer.wrap(meta), - new ArrayNode[0], new int[0], null); - return new DecodeContext(node, dtype, n, new MemorySegment[0], EncodingRegistry.empty(), Arena.ofAuto()); - } - - private static byte[] intMeta(long base, long mul) { - return EncodingProtos.SequenceMetadata.newBuilder() - .setBase(ScalarProtos.ScalarValue.newBuilder().setInt64Value(base)) - .setMultiplier(ScalarProtos.ScalarValue.newBuilder().setInt64Value(mul)) - .build().toByteArray(); - } - - private static byte[] f64Meta(double base, double mul) { - return EncodingProtos.SequenceMetadata.newBuilder() - .setBase(ScalarProtos.ScalarValue.newBuilder().setF64Value(base)) - .setMultiplier(ScalarProtos.ScalarValue.newBuilder().setF64Value(mul)) - .build().toByteArray(); - } - - private static byte[] f32Meta(float base, float mul) { - return EncodingProtos.SequenceMetadata.newBuilder() - .setBase(ScalarProtos.ScalarValue.newBuilder().setF32Value(base)) - .setMultiplier(ScalarProtos.ScalarValue.newBuilder().setF32Value(mul)) - .build().toByteArray(); - } - } -} diff --git a/core/src/test/java/io/github/dfa1/vortex/encoding/SparseEncodingTest.java b/core/src/test/java/io/github/dfa1/vortex/encoding/SparseEncodingTest.java deleted file mode 100644 index fcaab17..0000000 --- a/core/src/test/java/io/github/dfa1/vortex/encoding/SparseEncodingTest.java +++ /dev/null @@ -1,463 +0,0 @@ -package io.github.dfa1.vortex.encoding; - -import com.google.protobuf.InvalidProtocolBufferException; -import com.google.protobuf.NullValue; -import io.github.dfa1.vortex.proto.DTypeProtos; -import io.github.dfa1.vortex.proto.EncodingProtos; -import io.github.dfa1.vortex.proto.ScalarProtos; -import io.github.dfa1.vortex.core.array.Array; -import io.github.dfa1.vortex.core.array.BoolArray; -import io.github.dfa1.vortex.core.array.VarBinArray; -import io.github.dfa1.vortex.core.ArrayStats; -import io.github.dfa1.vortex.core.DType; -import io.github.dfa1.vortex.core.PType; -import org.junit.jupiter.api.Nested; -import org.junit.jupiter.api.Test; -import org.junit.jupiter.params.ParameterizedTest; -import org.junit.jupiter.params.provider.ValueSource; - -import java.lang.foreign.Arena; -import java.lang.foreign.MemorySegment; -import java.nio.ByteBuffer; -import java.nio.ByteOrder; -import java.nio.charset.StandardCharsets; -import java.util.List; - -import static org.assertj.core.api.Assertions.assertThat; - -class SparseEncodingTest { - - - @Nested - class Encode { - - @Test - void encode_allZeros_noPatches() throws InvalidProtocolBufferException { - // Given - long[] data = {0L, 0L, 0L, 0L}; - SparseEncoding sut = new SparseEncoding(); - - // When - EncodeResult result = sut.encode(DTypes.I64, data); - - // Then - EncodingProtos.SparseMetadata meta = EncodingProtos.SparseMetadata.parseFrom( - result.rootNode().metadata().array()); - assertThat(meta.getPatches().getLen()).isZero(); - } - - @Test - void encode_withNonZero_createsPatches() throws InvalidProtocolBufferException { - // Given — [0, 10, 0, 50, 0] - long[] data = {0L, 10L, 0L, 50L, 0L}; - SparseEncoding sut = new SparseEncoding(); - - // When - EncodeResult result = sut.encode(DTypes.I64, data); - - // Then - EncodingProtos.SparseMetadata meta = EncodingProtos.SparseMetadata.parseFrom( - result.rootNode().metadata().array()); - assertThat(meta.getPatches().getLen()).isEqualTo(2); - } - - @Test - void encode_roundTrip_i64() { - // Given — sparse long array - long[] data = {0L, 0L, 42L, 0L, 99L, 0L, 0L, 7L}; - SparseEncoding sut = new SparseEncoding(); - - // When - EncodeResult encoded = sut.encode(DTypes.I64, data); - Array decoded = decodeResult(encoded, DTypes.I64, data.length); - - // Then - var layout = PTypeIO.LE_LONG; - for (int i = 0; i < data.length; i++) { - assertThat(decoded.buffer(0).get(layout, (long) i * 8)) - .as("index %d", i).isEqualTo(data[i]); - } - } - - @Test - void encode_roundTrip_f64() { - // Given - double[] data = {0.0, 3.14, 0.0, 0.0, 2.72}; - SparseEncoding sut = new SparseEncoding(); - - // When - EncodeResult encoded = sut.encode(DTypes.F64, data); - Array decoded = decodeResult(encoded, DTypes.F64, data.length); - - // Then - var layout = PTypeIO.LE_DOUBLE; - for (int i = 0; i < data.length; i++) { - assertThat(decoded.buffer(0).get(layout, (long) i * 8)) - .as("index %d", i).isEqualTo(data[i]); - } - } - - @ParameterizedTest - @ValueSource(ints = {0, 1, 100}) - void encode_empty_or_allZero_noPatches(int size) throws InvalidProtocolBufferException { - // Given - long[] data = new long[size]; - SparseEncoding sut = new SparseEncoding(); - - // When - EncodeResult result = sut.encode(DTypes.I64, data); - - // Then - EncodingProtos.SparseMetadata meta = EncodingProtos.SparseMetadata.parseFrom( - result.rootNode().metadata().array()); - assertThat(meta.getPatches().getLen()).isZero(); - } - - private static Array decodeResult(EncodeResult encoded, DType dtype, int n) { - EncodeNode root = encoded.rootNode(); - List bufs = encoded.buffers(); - - MemorySegment[] segments = bufs.toArray(new MemorySegment[0]); - - ArrayNode idxNode = ArrayNode.of(root.children()[0].encodingId(), null, - new ArrayNode[0], root.children()[0].bufferIndices(), ArrayStats.empty()); - ArrayNode valNode = ArrayNode.of(root.children()[1].encodingId(), null, - new ArrayNode[0], root.children()[1].bufferIndices(), ArrayStats.empty()); - ArrayNode sparseNode = ArrayNode.of(root.encodingId(), root.metadata(), - new ArrayNode[]{idxNode, valNode}, root.bufferIndices(), ArrayStats.empty()); - - EncodingRegistry registry = TestRegistry.of(new SparseEncoding(), new PrimitiveEncoding()); - - DecodeContext ctx = new DecodeContext(sparseNode, dtype, n, segments, registry, Arena.global()); - return new SparseEncoding().decode(ctx); - } - } - - @Nested - class Decode { - - @Test - void decode_noPatches_returnsFillValue() { - // Given — 5 elements, fill=99, no patches - long fill = 99L; - DecodeContext ctx = buildSparseCtx(DTypes.I64, 5, fill, PType.U32, new long[0], new long[0]); - SparseEncoding sut = new SparseEncoding(); - - // When - Array result = sut.decode(ctx); - - // Then - assertThat(result.length()).isEqualTo(5L); - var layout = PTypeIO.LE_LONG; - for (int i = 0; i < 5; i++) { - assertThat(result.buffer(0).get(layout, (long) i * 8)) - .as("index %d", i).isEqualTo(fill); - } - } - - @Test - void decode_withPatches_overwritesAtIndices() { - // Given — 8 elements, fill=0, patches at indices [1, 5] with values [10, 50] - long fill = 0L; - long[] patchIndices = {1L, 5L}; - long[] patchValues = {10L, 50L}; - DecodeContext ctx = buildSparseCtx(DTypes.I64, 8, fill, PType.U32, patchIndices, patchValues); - SparseEncoding sut = new SparseEncoding(); - - // When - Array result = sut.decode(ctx); - - // Then - var layout = PTypeIO.LE_LONG; - long[] expected = {0, 10, 0, 0, 0, 50, 0, 0}; - for (int i = 0; i < expected.length; i++) { - assertThat(result.buffer(0).get(layout, (long) i * 8)) - .as("index %d", i).isEqualTo(expected[i]); - } - } - - @Test - void decode_f64_fillAndPatches() { - // Given — 4 F64 elements, fill=NaN bits, patch at index 2 with value 3.14 - double fillVal = Double.NaN; - double patchVal = 3.14; - DecodeContext ctx = buildSparseCtxF64(DTypes.F64, 4, fillVal, new long[]{2L}, new double[]{patchVal}); - SparseEncoding sut = new SparseEncoding(); - - // When - Array result = sut.decode(ctx); - - // Then - var layout = PTypeIO.LE_DOUBLE; - assertThat(result.buffer(0).get(layout, 0L)).isNaN(); - assertThat(result.buffer(0).get(layout, 8L)).isNaN(); - assertThat(result.buffer(0).get(layout, 16L)).isEqualTo(3.14); - assertThat(result.buffer(0).get(layout, 24L)).isNaN(); - } - - @Test - void decode_offsetSubtracted() { - // Given — offset=10, patch index=12 → absolute position = 12 - 10 = 2 - long[] patchIndices = {12L}; - long[] patchValues = {777L}; - DecodeContext ctx = buildSparseCtxWithOffset(DTypes.I64, 5, 0L, PType.U32, patchIndices, patchValues, 10L); - SparseEncoding sut = new SparseEncoding(); - - // When - Array result = sut.decode(ctx); - - // Then - var layout = PTypeIO.LE_LONG; - assertThat(result.buffer(0).get(layout, 16L)).isEqualTo(777L); - } - - // regression: NULL_VALUE fill caused "unexpected scalar kind NULL_VALUE" on nullable cols - @Test - void decode_nullValueFill_treatedAsZero() { - // Given — fill encoded as ScalarValue.NULL_VALUE (as Rust writes for nullable cols) - byte[] nullFill = ScalarProtos.ScalarValue.newBuilder() - .setNullValue(NullValue.NULL_VALUE).build().toByteArray(); - byte[] meta = buildSparseMetaBytes(0, 0L, PType.U32); - DecodeContext ctx = buildCtx(DTypes.I64, 4, nullFill, meta, new byte[0], new byte[0], - new DType.Primitive(PType.U32, false)); - SparseEncoding sut = new SparseEncoding(); - - // When - Array result = sut.decode(ctx); - - // Then - var layout = PTypeIO.LE_LONG; - for (int i = 0; i < 4; i++) { - assertThat(result.buffer(0).get(layout, (long) i * 8)).as("index %d", i).isZero(); - } - } - - // regression: Utf8 dtype caused "expected primitive dtype, got Utf8[nullable=true]" - @Test - void decode_utf8_noPatches_allEmpty() { - // Given — Utf8 sparse, no patches → all positions empty (null fill) - DType utf8 = new DType.Utf8(true); - byte[] nullFill = ScalarProtos.ScalarValue.newBuilder() - .setNullValue(NullValue.NULL_VALUE).build().toByteArray(); - byte[] meta = buildSparseMetaBytes(0, 0L, PType.U32); - DecodeContext ctx = buildCtx(utf8, 3, nullFill, meta, new byte[0], new byte[0], - new DType.Primitive(PType.U32, false)); - SparseEncoding sut = new SparseEncoding(); - - // When - Array result = sut.decode(ctx); - - // Then - assertThat(result.length()).isEqualTo(3L); - VarBinArray varBin = (VarBinArray) result; - for (int i = 0; i < 3; i++) { - assertThat(varBin.getByteLength(i)).as("index %d", i).isZero(); - } - } - - // regression: Utf8 dtype caused "expected primitive dtype, got Utf8[nullable=true]" - @Test - void decode_utf8_withPatches_writesStringsAtIndices() { - // Given — 5 Utf8 elements, patches at [1]="hi" and [3]="bye" - DType utf8 = new DType.Utf8(true); - byte[] nullFill = ScalarProtos.ScalarValue.newBuilder() - .setNullValue(NullValue.NULL_VALUE).build().toByteArray(); - byte[] meta = buildSparseMetaBytes(2, 0L, PType.U32); - - byte[] idxBuf = toLEBytes(new long[]{1L, 3L}, PType.U32); - byte[] strBytes = "hibye".getBytes(StandardCharsets.UTF_8); - byte[] offsets = intLEBytes(new int[]{0, 2, 5}); - byte[] varBinMeta = EncodingProtos.VarBinMetadata.newBuilder() - .setOffsetsPtype(DTypeProtos.PType.forNumber(PType.I32.ordinal())) - .build().toByteArray(); - - ArrayNode offsetsNode = ArrayNode.of(EncodingId.VORTEX_PRIMITIVE, null, - new ArrayNode[0], new int[]{3}, ArrayStats.empty()); - ArrayNode valNode = ArrayNode.of(EncodingId.VORTEX_VARBIN, - ByteBuffer.wrap(varBinMeta), - new ArrayNode[]{offsetsNode}, new int[]{2}, ArrayStats.empty()); - ArrayNode idxNode = ArrayNode.of(EncodingId.VORTEX_PRIMITIVE, null, - new ArrayNode[0], new int[]{1}, ArrayStats.empty()); - ArrayNode sparseNode = ArrayNode.of(EncodingId.VORTEX_SPARSE, - ByteBuffer.wrap(meta), - new ArrayNode[]{idxNode, valNode}, new int[]{0}, ArrayStats.empty()); - - EncodingRegistry registry = TestRegistry.of(new SparseEncoding(), new PrimitiveEncoding(), new VarBinEncoding()); - - MemorySegment[] segments = { - MemorySegment.ofArray(nullFill), - MemorySegment.ofArray(idxBuf), - MemorySegment.ofArray(strBytes), - MemorySegment.ofArray(offsets), - }; - DecodeContext ctx = new DecodeContext(sparseNode, utf8, 5, segments, registry, Arena.global()); - SparseEncoding sut = new SparseEncoding(); - - // When - Array result = sut.decode(ctx); - - // Then - VarBinArray varBin = (VarBinArray) result; - assertThat(varBin.length()).isEqualTo(5L); - assertThat(varBin.getByteLength(0)).isZero(); - assertThat(varBin.getString(1)).isEqualTo("hi"); - assertThat(varBin.getByteLength(2)).isZero(); - assertThat(varBin.getString(3)).isEqualTo("bye"); - assertThat(varBin.getByteLength(4)).isZero(); - } - - // regression: Bool dtype caused "expected primitive dtype, got Bool[nullable=true]" - @Test - void decode_bool_withPatches_setsBitsAtIndices() { - // Given — 6 Bool elements, patches at [2]=true and [5]=true - DType bool = new DType.Bool(true); - byte[] nullFill = ScalarProtos.ScalarValue.newBuilder() - .setNullValue(NullValue.NULL_VALUE).build().toByteArray(); - byte[] meta = buildSparseMetaBytes(2, 0L, PType.U32); - byte[] idxBuf = toLEBytes(new long[]{2L, 5L}, PType.U32); - byte[] boolBits = new byte[]{0b00000011}; - - ArrayNode valNode = ArrayNode.of(EncodingId.VORTEX_BOOL, null, - new ArrayNode[0], new int[]{2}, ArrayStats.empty()); - ArrayNode idxNode = ArrayNode.of(EncodingId.VORTEX_PRIMITIVE, null, - new ArrayNode[0], new int[]{1}, ArrayStats.empty()); - ArrayNode sparseNode = ArrayNode.of(EncodingId.VORTEX_SPARSE, - ByteBuffer.wrap(meta), - new ArrayNode[]{idxNode, valNode}, new int[]{0}, ArrayStats.empty()); - - EncodingRegistry registry = TestRegistry.of(new SparseEncoding(), new PrimitiveEncoding(), new BoolEncoding()); - - MemorySegment[] segments = { - MemorySegment.ofArray(nullFill), - MemorySegment.ofArray(idxBuf), - MemorySegment.ofArray(boolBits), - }; - DecodeContext ctx = new DecodeContext(sparseNode, bool, 6, segments, registry, Arena.global()); - SparseEncoding sut = new SparseEncoding(); - - // When - Array result = sut.decode(ctx); - - // Then - BoolArray boolArr = (BoolArray) result; - assertThat(boolArr.length()).isEqualTo(6L); - assertThat(boolArr.getBoolean(0)).isFalse(); - assertThat(boolArr.getBoolean(1)).isFalse(); - assertThat(boolArr.getBoolean(2)).isTrue(); - assertThat(boolArr.getBoolean(3)).isFalse(); - assertThat(boolArr.getBoolean(4)).isFalse(); - assertThat(boolArr.getBoolean(5)).isTrue(); - } - - private static DecodeContext buildSparseCtx( - DType dtype, long rowCount, long fillLong, PType idxPtype, - long[] patchIndices, long[] patchValues - ) { - return buildSparseCtxWithOffset(dtype, rowCount, fillLong, idxPtype, patchIndices, patchValues, 0L); - } - - private static DecodeContext buildSparseCtxWithOffset( - DType dtype, long rowCount, long fillLong, PType idxPtype, - long[] patchIndices, long[] patchValues, long offset - ) { - byte[] fillBytes = ScalarProtos.ScalarValue.newBuilder() - .setInt64Value(fillLong).build().toByteArray(); - - byte[] metaBytes = buildSparseMetaBytes(patchIndices.length, offset, idxPtype); - - byte[] idxBuf = toLEBytes(patchIndices, idxPtype); - byte[] valBuf = toLEBytes(patchValues, PType.I64); - - return buildCtx(dtype, rowCount, fillBytes, metaBytes, idxBuf, valBuf, - new DType.Primitive(idxPtype, false)); - } - - private static DecodeContext buildSparseCtxF64( - DType dtype, long rowCount, double fillDouble, - long[] patchIndices, double[] patchValues - ) { - byte[] fillBytes = ScalarProtos.ScalarValue.newBuilder() - .setF64Value(fillDouble).build().toByteArray(); - byte[] metaBytes = buildSparseMetaBytes(patchIndices.length, 0L, PType.U32); - - byte[] idxBuf = toLEBytes(patchIndices, PType.U32); - byte[] valBuf = f64LEBytes(patchValues); - - return buildCtx(dtype, rowCount, fillBytes, metaBytes, idxBuf, valBuf, - new DType.Primitive(PType.U32, false)); - } - - private static DecodeContext buildCtx( - DType dtype, long rowCount, - byte[] fillBytes, byte[] metaBytes, - byte[] idxBuf, byte[] valBuf, - DType idxDtype - ) { - ArrayNode idxNode = ArrayNode.of(EncodingId.VORTEX_PRIMITIVE, null, - new ArrayNode[0], new int[]{1}, ArrayStats.empty()); - ArrayNode valNode = ArrayNode.of(EncodingId.VORTEX_PRIMITIVE, null, - new ArrayNode[0], new int[]{2}, ArrayStats.empty()); - ArrayNode sparseNode = ArrayNode.of(EncodingId.VORTEX_SPARSE, - ByteBuffer.wrap(metaBytes), - new ArrayNode[]{idxNode, valNode}, - new int[]{0}, - ArrayStats.empty()); - - MemorySegment[] segments = { - MemorySegment.ofArray(fillBytes), - MemorySegment.ofArray(idxBuf), - MemorySegment.ofArray(valBuf) - }; - - EncodingRegistry registry = TestRegistry.of(new SparseEncoding(), new PrimitiveEncoding()); - - return new DecodeContext(sparseNode, dtype, rowCount, segments, registry, java.lang.foreign.Arena.global()); - } - - private static byte[] buildSparseMetaBytes(long numPatches, long offset, PType idxPtype) { - EncodingProtos.PatchesMetadata patchesMeta = EncodingProtos.PatchesMetadata.newBuilder() - .setLen(numPatches) - .setOffset(offset) - .setIndicesPtype(DTypeProtos.PType.forNumber(idxPtype.ordinal())) - .build(); - return EncodingProtos.SparseMetadata.newBuilder() - .setPatches(patchesMeta) - .build() - .toByteArray(); - } - - private static byte[] toLEBytes(long[] values, PType ptype) { - int elemBytes = ptype.byteSize(); - byte[] buf = new byte[values.length * elemBytes]; - ByteBuffer bb = ByteBuffer.wrap(buf).order(ByteOrder.LITTLE_ENDIAN); - for (long v : values) { - switch (ptype) { - case U8, I8 -> bb.put((byte) v); - case U16, I16 -> bb.putShort((short) v); - case U32, I32 -> bb.putInt((int) v); - case U64, I64 -> bb.putLong(v); - default -> throw new UnsupportedOperationException(ptype.name()); - } - } - return buf; - } - - private static byte[] f64LEBytes(double[] values) { - byte[] buf = new byte[values.length * 8]; - ByteBuffer bb = ByteBuffer.wrap(buf).order(ByteOrder.LITTLE_ENDIAN); - for (double v : values) { - bb.putDouble(v); - } - return buf; - } - - private static byte[] intLEBytes(int[] values) { - byte[] buf = new byte[values.length * 4]; - ByteBuffer bb = ByteBuffer.wrap(buf).order(ByteOrder.LITTLE_ENDIAN); - for (int v : values) { - bb.putInt(v); - } - return buf; - } - } -} diff --git a/core/src/test/java/io/github/dfa1/vortex/encoding/StructEncodingTest.java b/core/src/test/java/io/github/dfa1/vortex/encoding/StructEncodingTest.java deleted file mode 100644 index 3c93a83..0000000 --- a/core/src/test/java/io/github/dfa1/vortex/encoding/StructEncodingTest.java +++ /dev/null @@ -1,178 +0,0 @@ -package io.github.dfa1.vortex.encoding; - -import io.github.dfa1.vortex.core.array.Array; -import io.github.dfa1.vortex.core.array.LongArray; -import io.github.dfa1.vortex.core.array.MaskedArray; -import io.github.dfa1.vortex.core.array.StructArray; -import io.github.dfa1.vortex.core.ArrayStats; -import io.github.dfa1.vortex.core.DType; -import org.junit.jupiter.api.Nested; -import org.junit.jupiter.api.Test; - -import java.lang.foreign.Arena; -import java.lang.foreign.MemorySegment; -import java.util.List; - -import static org.assertj.core.api.Assertions.assertThat; - -class StructEncodingTest { - - private static ArrayNode toArrayNode(EncodeNode node) { - ArrayNode[] children = new ArrayNode[node.children().length]; - for (int i = 0; i < children.length; i++) { - children[i] = toArrayNode(node.children()[i]); - } - return ArrayNode.of(node.encodingId(), node.metadata(), children, node.bufferIndices(), ArrayStats.empty()); - } - - @Nested - class Encode { - - @Test - void accepts_structDtype_trueForStruct_falseForPrimitive() { - // Given - StructEncoding sut = new StructEncoding(); - DType.Struct structDtype = new DType.Struct( - List.of("x"), List.of(DTypes.I64), false); - - // When / Then - assertThat(sut.accepts(structDtype)).isTrue(); - assertThat(sut.accepts(DTypes.I64)).isFalse(); - } - - @Test - void roundTrip_twoI64Fields_preservesValues() { - // Given - long[] ids = {1L, 2L, 3L}; - long[] values = {10L, 20L, 30L}; - DType.Struct dtype = new DType.Struct( - List.of("id", "value"), List.of(DTypes.I64, DTypes.I64), false); - StructData data = new StructData(List.of(ids, values)); - StructEncoding sut = new StructEncoding(); - - // When - EncodeResult result = sut.encode(dtype, data); - - // Then — decode round-trip - MemorySegment[] bufs = result.buffers().toArray(MemorySegment[]::new); - EncodingRegistry registry = TestRegistry.of(new StructEncoding(), new PrimitiveEncoding()); - DecodeContext ctx = new DecodeContext( - toArrayNode(result.rootNode()), dtype, ids.length, bufs, registry, Arena.global()); - StructArray decoded = (StructArray) sut.decode(ctx); - - assertThat(decoded.length()).isEqualTo(ids.length); - assertThat(decoded.fieldCount()).isEqualTo(2); - LongArray idField = (LongArray) decoded.field(0); - LongArray valueField = (LongArray) decoded.field(1); - for (int i = 0; i < ids.length; i++) { - assertThat(idField.getLong(i)).isEqualTo(ids[i]); - assertThat(valueField.getLong(i)).isEqualTo(values[i]); - } - } - - @Test - void singleField_encodeResult_hasOneChildAndNoBuffers() { - // Given - long[] data = {7L, 14L, 21L}; - DType.Struct dtype = new DType.Struct(List.of("v"), List.of(DTypes.I64), false); - StructEncoding sut = new StructEncoding(); - - // When - EncodeResult result = sut.encode(dtype, new StructData(List.of(data))); - - // Then — struct node wraps one field child with remapped buffers - assertThat(result.rootNode().encodingId()).isEqualTo(EncodingId.VORTEX_STRUCT); - assertThat(result.rootNode().children()).hasSize(1); - assertThat(result.rootNode().bufferIndices()).isEmpty(); - assertThat(result.buffers()).hasSize(1); // one buffer for the DTypes.I64 field - } - - @Test - void fieldCountMismatch_throwsVortexException() { - // Given - DType.Struct dtype = new DType.Struct(List.of("a", "b"), List.of(DTypes.I64, DTypes.I64), false); - StructData data = new StructData(List.of(new long[]{1L})); // only 1 field, dtype has 2 - StructEncoding sut = new StructEncoding(); - - // When / Then - org.junit.jupiter.api.Assertions.assertThrows( - io.github.dfa1.vortex.core.VortexException.class, - () -> sut.encode(dtype, data)); - } - } - - @Nested - class Decode { - - @Test - void decode_nonNullableWrapper_oneChild_returnsValues() { - // Given — struct{values: DTypes.I64} (non-nullable, 1 child) - long[] data = {10L, 20L, 30L}; - MemorySegment seg = TestSegments.leLongs(data); - ArrayNode valuesNode = primitiveNode(0); - ArrayNode structNode = ArrayNode.of(EncodingId.VORTEX_STRUCT, null, - new ArrayNode[]{valuesNode}, new int[0], ArrayStats.empty()); - - DecodeContext ctx = buildStructCtx(structNode, new MemorySegment[]{seg}, data.length); - StructEncoding sut = new StructEncoding(); - - // When - Array result = sut.decode(ctx); - - // Then - assertThat(result.length()).isEqualTo(data.length); - for (int i = 0; i < data.length; i++) { - assertThat(result.buffer(0).get(PTypeIO.LE_LONG, (long) i * 8)).isEqualTo(data[i]); - } - } - - @Test - void decode_nullableWrapper_twoChildren_returnsMaskedArray() { - // Given — struct{validity: Bool, values: DTypes.I64} (nullable, 2 children) - long[] data = {7L, 14L, 21L}; - MemorySegment validitySeg = MemorySegment.ofArray(new byte[]{(byte) 0xFF}); // all valid - MemorySegment valuesSeg = TestSegments.leLongs(data); - - ArrayNode validityNode = boolNode(0); // slot 0 = validity bitmap - ArrayNode valuesNode = primitiveNode(1); // slot 1 = actual values - ArrayNode structNode = ArrayNode.of(EncodingId.VORTEX_STRUCT, null, - new ArrayNode[]{validityNode, valuesNode}, new int[0], ArrayStats.empty()); - - EncodingRegistry registry = TestRegistry.of(new StructEncoding(), new PrimitiveEncoding(), new BoolEncoding()); - DecodeContext ctx = new DecodeContext( - structNode, DTypes.I64, data.length, - new MemorySegment[]{validitySeg, valuesSeg}, - registry, Arena.global()); - - StructEncoding sut = new StructEncoding(); - - // When - Array result = sut.decode(ctx); - - // Then — validity preserved; values accessible via inner array - assertThat(result).isInstanceOf(MaskedArray.class); - MaskedArray masked = (MaskedArray) result; - LongArray values = (LongArray) masked.child(0); - assertThat(values.length()).isEqualTo(data.length); - for (int i = 0; i < data.length; i++) { - assertThat(masked.isValid(i)).isTrue(); - assertThat(values.getLong(i)).isEqualTo(data[i]); - } - } - - private static ArrayNode primitiveNode(int bufferIdx) { - return ArrayNode.of(EncodingId.VORTEX_PRIMITIVE, null, new ArrayNode[0], - new int[]{bufferIdx}, ArrayStats.empty()); - } - - private static ArrayNode boolNode(int bufferIdx) { - return ArrayNode.of(EncodingId.VORTEX_BOOL, null, new ArrayNode[0], - new int[]{bufferIdx}, ArrayStats.empty()); - } - - private static DecodeContext buildStructCtx(ArrayNode structNode, MemorySegment[] segs, long rowCount) { - EncodingRegistry registry = TestRegistry.of(new StructEncoding(), new PrimitiveEncoding()); - return new DecodeContext(structNode, DTypes.I64, rowCount, segs, registry, Arena.global()); - } - } -} diff --git a/core/src/test/java/io/github/dfa1/vortex/encoding/TestDecodeContexts.java b/core/src/test/java/io/github/dfa1/vortex/encoding/TestDecodeContexts.java deleted file mode 100644 index 2c3cea8..0000000 --- a/core/src/test/java/io/github/dfa1/vortex/encoding/TestDecodeContexts.java +++ /dev/null @@ -1,49 +0,0 @@ -package io.github.dfa1.vortex.encoding; - -import java.lang.foreign.Arena; -import java.lang.foreign.MemorySegment; - -/// Fluent builder for {@link DecodeContext} used in encoding tests. -/// Defaults: rowCount=0, segments=[], registry=empty, arena=Arena.global(). -final class TestDecodeContexts { - - private final ArrayNode node; - private final io.github.dfa1.vortex.core.DType dtype; - private long rowCount = 0; - private MemorySegment[] segments = new MemorySegment[0]; - private EncodingRegistry registry = EncodingRegistry.empty(); - private Arena arena = Arena.global(); - - private TestDecodeContexts(ArrayNode node, io.github.dfa1.vortex.core.DType dtype) { - this.node = node; - this.dtype = dtype; - } - - static TestDecodeContexts of(ArrayNode node, io.github.dfa1.vortex.core.DType dtype) { - return new TestDecodeContexts(node, dtype); - } - - TestDecodeContexts rowCount(long n) { - this.rowCount = n; - return this; - } - - TestDecodeContexts segments(MemorySegment... segs) { - this.segments = segs; - return this; - } - - TestDecodeContexts registry(EncodingRegistry reg) { - this.registry = reg; - return this; - } - - TestDecodeContexts arena(Arena a) { - this.arena = a; - return this; - } - - DecodeContext build() { - return new DecodeContext(node, dtype, rowCount, segments, registry, arena); - } -} diff --git a/core/src/test/java/io/github/dfa1/vortex/encoding/TestRegistry.java b/core/src/test/java/io/github/dfa1/vortex/encoding/TestRegistry.java deleted file mode 100644 index 74ad3b0..0000000 --- a/core/src/test/java/io/github/dfa1/vortex/encoding/TestRegistry.java +++ /dev/null @@ -1,19 +0,0 @@ -package io.github.dfa1.vortex.encoding; - -/// Static factories for EncodingRegistry instances used in encoding tests. -final class TestRegistry { - - static EncodingRegistry of(Encoding... encodings) { - EncodingRegistry r = EncodingRegistry.empty(); - for (Encoding e : encodings) { - r.register(e); - } - return r; - } - - static EncodingRegistry withPrimitive(Encoding sut) { - return of(sut, new PrimitiveEncoding()); - } - - private TestRegistry() {} -} diff --git a/core/src/test/java/io/github/dfa1/vortex/encoding/TestSegments.java b/core/src/test/java/io/github/dfa1/vortex/encoding/TestSegments.java deleted file mode 100644 index acdcb84..0000000 --- a/core/src/test/java/io/github/dfa1/vortex/encoding/TestSegments.java +++ /dev/null @@ -1,50 +0,0 @@ -package io.github.dfa1.vortex.encoding; - -import java.lang.foreign.Arena; -import java.lang.foreign.MemorySegment; - -/// Static factories for LE-encoded MemorySegments used in encoding tests. -final class TestSegments { - - static MemorySegment leLongs(long... values) { - MemorySegment seg = Arena.ofAuto().allocate((long) values.length * Long.BYTES); - for (int i = 0; i < values.length; i++) { - seg.setAtIndex(PTypeIO.LE_LONG, i, values[i]); - } - return seg; - } - - static MemorySegment leInts(int... values) { - MemorySegment seg = Arena.ofAuto().allocate((long) values.length * Integer.BYTES); - for (int i = 0; i < values.length; i++) { - seg.setAtIndex(PTypeIO.LE_INT, i, values[i]); - } - return seg; - } - - static MemorySegment leDoubles(double... values) { - MemorySegment seg = Arena.ofAuto().allocate((long) values.length * Double.BYTES); - for (int i = 0; i < values.length; i++) { - seg.setAtIndex(PTypeIO.LE_DOUBLE, i, values[i]); - } - return seg; - } - - static MemorySegment leFloats(float... values) { - MemorySegment seg = Arena.ofAuto().allocate((long) values.length * Float.BYTES); - for (int i = 0; i < values.length; i++) { - seg.setAtIndex(PTypeIO.LE_FLOAT, i, values[i]); - } - return seg; - } - - static MemorySegment leShorts(short... values) { - MemorySegment seg = Arena.ofAuto().allocate((long) values.length * Short.BYTES); - for (int i = 0; i < values.length; i++) { - seg.setAtIndex(PTypeIO.LE_SHORT, i, values[i]); - } - return seg; - } - - private TestSegments() {} -} diff --git a/core/src/test/java/io/github/dfa1/vortex/encoding/VarBinEncodingTest.java b/core/src/test/java/io/github/dfa1/vortex/encoding/VarBinEncodingTest.java deleted file mode 100644 index 30ad734..0000000 --- a/core/src/test/java/io/github/dfa1/vortex/encoding/VarBinEncodingTest.java +++ /dev/null @@ -1,159 +0,0 @@ -package io.github.dfa1.vortex.encoding; - -import io.github.dfa1.vortex.core.DType; -import io.github.dfa1.vortex.core.PType; -import io.github.dfa1.vortex.core.VortexException; -import io.github.dfa1.vortex.core.array.VarBinArray; -import org.junit.jupiter.api.Nested; -import org.junit.jupiter.api.Test; - -import java.lang.foreign.Arena; -import java.lang.foreign.MemorySegment; -import java.nio.charset.StandardCharsets; - -import static org.assertj.core.api.Assertions.assertThat; -import static org.assertj.core.api.Assertions.assertThatThrownBy; - -class VarBinEncodingTest { - - private static final DType UTF8 = new DType.Utf8(false); - private static final DType BINARY = new DType.Binary(false); - - @Nested - class Encode { - - @Test - void encodingId_isVortexVarbin() { - // Given / When / Then - assertThat(new VarBinEncoding().encodingId()).isEqualTo(EncodingId.VORTEX_VARBIN); - } - - @Test - void accepts_utf8Dtype_returnsTrue() { - // Given / When / Then - assertThat(new VarBinEncoding().accepts(UTF8)).isTrue(); - } - - @Test - void accepts_binaryDtype_returnsTrue() { - // Given / When / Then - assertThat(new VarBinEncoding().accepts(BINARY)).isTrue(); - } - - @Test - void accepts_primitiveDtype_returnsFalse() { - // Given / When / Then - assertThat(new VarBinEncoding().accepts(new DType.Primitive(PType.I64, false))).isFalse(); - } - - @Test - void encode_singleString_roundTrips() { - // Given - var sut = new VarBinEncoding(); - String[] data = {"hello"}; - - // When - EncodeResult result = sut.encode(UTF8, data); - DecodeContext ctx = EncodeTestHelper.toDecodeContext(result, data.length, UTF8, buildRegistry()); - VarBinArray decoded = (VarBinArray) sut.decode(ctx); - - // Then - assertThat(decoded.length()).isEqualTo(1); - assertThat(decoded.getBytes(0)).isEqualTo("hello".getBytes(StandardCharsets.UTF_8)); - } - - @Test - void encode_multipleStrings_roundTrips() { - // Given - var sut = new VarBinEncoding(); - String[] data = {"foo", "bar", "baz"}; - - // When - EncodeResult result = sut.encode(UTF8, data); - DecodeContext ctx = EncodeTestHelper.toDecodeContext(result, data.length, UTF8, buildRegistry()); - VarBinArray decoded = (VarBinArray) sut.decode(ctx); - - // Then - assertThat(decoded.length()).isEqualTo(3); - for (int i = 0; i < data.length; i++) { - assertThat(decoded.getBytes(i)).isEqualTo(data[i].getBytes(StandardCharsets.UTF_8)); - } - } - - @Test - void encode_unicodeString_roundTrips() { - // Given - var sut = new VarBinEncoding(); - String[] data = {"héllo", "wörld", "日本語"}; - - // When - EncodeResult result = sut.encode(UTF8, data); - DecodeContext ctx = EncodeTestHelper.toDecodeContext(result, data.length, UTF8, buildRegistry()); - VarBinArray decoded = (VarBinArray) sut.decode(ctx); - - // Then - assertThat(decoded.length()).isEqualTo(3); - for (int i = 0; i < data.length; i++) { - assertThat(decoded.getBytes(i)).isEqualTo(data[i].getBytes(StandardCharsets.UTF_8)); - } - } - - @Test - void encode_emptyStringInArray_roundTrips() { - // Given - var sut = new VarBinEncoding(); - String[] data = {"a", "", "b"}; - - // When - EncodeResult result = sut.encode(UTF8, data); - DecodeContext ctx = EncodeTestHelper.toDecodeContext(result, data.length, UTF8, buildRegistry()); - VarBinArray decoded = (VarBinArray) sut.decode(ctx); - - // Then - assertThat(decoded.length()).isEqualTo(3); - assertThat(decoded.getBytes(0)).isEqualTo(new byte[]{'a'}); - assertThat(decoded.getBytes(1)).isEmpty(); - assertThat(decoded.getBytes(2)).isEqualTo(new byte[]{'b'}); - } - - @Test - void encode_emptyArray_producesZeroLengthResult() { - // Given - var sut = new VarBinEncoding(); - String[] data = {}; - - // When - EncodeResult result = sut.encode(UTF8, data); - DecodeContext ctx = EncodeTestHelper.toDecodeContext(result, data.length, UTF8, buildRegistry()); - VarBinArray decoded = (VarBinArray) sut.decode(ctx); - - // Then - assertThat(decoded.length()).isZero(); - } - - private static EncodingRegistry buildRegistry() { - EncodingRegistry registry = EncodingRegistry.empty(); - registry.register(new VarBinEncoding()); - registry.register(new PrimitiveEncoding()); - return registry; - } - } - - @Nested - class Decode { - - @Test - void decode_missingMetadata_throwsVortexException() { - // Given - var sut = new VarBinEncoding(); - ArrayNode node = ArrayNode.of(EncodingId.VORTEX_VARBIN, null, new ArrayNode[0], new int[0], null); - DecodeContext ctx = new DecodeContext(node, UTF8, 3, new MemorySegment[0], - EncodingRegistry.empty(), Arena.ofAuto()); - - // When / Then - assertThatThrownBy(() -> sut.decode(ctx)) - .isInstanceOf(VortexException.class) - .hasMessageContaining("missing metadata"); - } - } -} diff --git a/core/src/test/java/io/github/dfa1/vortex/encoding/VarBinViewEncodingTest.java b/core/src/test/java/io/github/dfa1/vortex/encoding/VarBinViewEncodingTest.java deleted file mode 100644 index b177efc..0000000 --- a/core/src/test/java/io/github/dfa1/vortex/encoding/VarBinViewEncodingTest.java +++ /dev/null @@ -1,171 +0,0 @@ -package io.github.dfa1.vortex.encoding; - -import io.github.dfa1.vortex.core.array.VarBinArray; -import org.junit.jupiter.api.Nested; -import org.junit.jupiter.api.Test; -import org.junit.jupiter.params.ParameterizedTest; -import org.junit.jupiter.params.provider.Arguments; -import org.junit.jupiter.params.provider.MethodSource; - -import java.lang.foreign.Arena; -import java.lang.foreign.MemorySegment; -import java.nio.charset.StandardCharsets; -import java.util.stream.Stream; - -import static org.assertj.core.api.Assertions.assertThat; - -/// Property: decode reconstructs every string value exactly, -/// regardless of inlined vs reference layout. -class VarBinViewEncodingTest { - - @Nested - class Encode { - - @Test - void accepts_utf8_true() { - // Given - var sut = new VarBinViewEncoding(); - - // When / Then - assertThat(sut.accepts(DTypes.UTF8)).isTrue(); - } - - @Test - void accepts_binary_true() { - // Given - var sut = new VarBinViewEncoding(); - - // When / Then - assertThat(sut.accepts(DTypes.BINARY)).isTrue(); - } - - @Test - void accepts_primitive_false() { - // Given - var sut = new VarBinViewEncoding(); - - // When / Then - assertThat(sut.accepts(DTypes.I32)).isFalse(); - } - - @ParameterizedTest(name = "{0}") - @MethodSource("io.github.dfa1.vortex.encoding.VarBinViewEncodingTest$Decode#stringArrays") - void encode_thenDecode_roundtripsAllStrings(String name, String[] values) { - // Given - var sut = new VarBinViewEncoding(); - Arena arena = Arena.ofAuto(); - - // When - EncodeResult result = sut.encode(DTypes.UTF8, values); - MemorySegment[] bufs = result.buffers().toArray(MemorySegment[]::new); - ArrayNode node = ArrayNode.of( - EncodingId.VORTEX_VARBINVIEW, null, new ArrayNode[0], - result.rootNode().bufferIndices(), null); - EncodingRegistry registry = TestRegistry.of(sut); - DecodeContext ctx = new DecodeContext(node, DTypes.UTF8, values.length, bufs, registry, arena); - var decoded = (VarBinArray) sut.decode(ctx); - - // Then - assertThat(decoded.length()).isEqualTo(values.length); - for (int i = 0; i < values.length; i++) { - assertThat(decoded.getString(i)).as("index %d", i).isEqualTo(values[i]); - } - } - } - - @Nested - class Decode { - - @ParameterizedTest(name = "{0}") - @MethodSource("io.github.dfa1.vortex.encoding.VarBinViewEncodingTest$Decode#stringArrays") - void decode_roundtrip_returnsAllStrings(String name, String[] values) { - // Given - Arena arena = Arena.ofAuto(); - long n = values.length; - - // Encode all long strings into one data buffer - byte[][] bytesArr = new byte[values.length][]; - int dataBufLen = 0; - for (int i = 0; i < values.length; i++) { - bytesArr[i] = values[i].getBytes(StandardCharsets.UTF_8); - if (bytesArr[i].length > 12) { - dataBufLen += bytesArr[i].length; - } - } - - MemorySegment dataBuf = arena.allocate(dataBufLen > 0 ? dataBufLen : 1); - MemorySegment views = arena.allocate(n > 0 ? n * 16 : 1); - - int dataOffset = 0; - for (int i = 0; i < values.length; i++) { - byte[] b = bytesArr[i]; - long viewOff = (long) i * 16; - views.set(PTypeIO.LE_INT, viewOff, b.length); - if (b.length <= 12) { - // inlined: data at viewOff+4 - MemorySegment.copy(MemorySegment.ofArray(b), 0, views, viewOff + 4, b.length); - } else { - // reference: buffer_index=0, offset=dataOffset - views.set(PTypeIO.LE_INT, viewOff + 8, 0); // buffer_index - views.set(PTypeIO.LE_INT, viewOff + 12, dataOffset); // offset - MemorySegment.copy(MemorySegment.ofArray(b), 0, dataBuf, dataOffset, b.length); - dataOffset += b.length; - } - } - - // bufferIndices: [0=dataBuf, 1=views] when data buffer needed, else just [0=views] - int[] bufIndices; - MemorySegment[] segBufs; - if (dataBufLen > 0) { - bufIndices = new int[]{0, 1}; - segBufs = new MemorySegment[]{dataBuf, views}; - } else { - bufIndices = new int[]{0}; - segBufs = new MemorySegment[]{views}; - } - - ArrayNode node = ArrayNode.of(EncodingId.VORTEX_VARBINVIEW, null, - new ArrayNode[0], bufIndices, null); - - EncodingRegistry registry = TestRegistry.of(new VarBinViewEncoding()); - - DecodeContext ctx = new DecodeContext(node, DTypes.UTF8, n, segBufs, registry, arena); - var sut = new VarBinViewEncoding(); - - // When - var result = sut.decode(ctx); - - // Then - assertThat(result).isInstanceOf(VarBinArray.class); - assertThat(result.length()).isEqualTo(n); - for (int i = 0; i < values.length; i++) { - VarBinArray varBinArray = (VarBinArray) result; - String decoded = varBinArray.getString(i); - assertThat(decoded).as("index %d", i).isEqualTo(values[i]); - } - } - - static Stream stringArrays() { - return Stream.of( - Arguments.of("empty-array", new String[0]), - Arguments.of("single-empty-string", new String[]{""}), - Arguments.of("short-strings", new String[]{"hi", "ok", "no"}), - Arguments.of("exactly-12-bytes", new String[]{"123456789012"}), // max inlined - Arguments.of("just-over-12-bytes", new String[]{"1234567890123"}), // min reference - Arguments.of("long-strings", new String[]{"the quick brown fox jumps over the lazy dog"}), - Arguments.of("mixed-lengths", new String[]{"a", "hello", "this is a longer string than twelve"}), - Arguments.of("repeated-short", repeat("ab", 50)), - Arguments.of("repeated-long", repeat("this exceeds twelve bytes easily", 10)), - Arguments.of("all-empty", new String[]{"", "", "", ""}), - Arguments.of("unicode", new String[]{"héllo", "wörld", "こんにちは"}) - ); - } - - private static String[] repeat(String s, int n) { - String[] arr = new String[n]; - java.util.Arrays.fill(arr, s); - return arr; - } - } - -} diff --git a/core/src/test/java/io/github/dfa1/vortex/encoding/ZigZagEncodingTest.java b/core/src/test/java/io/github/dfa1/vortex/encoding/ZigZagEncodingTest.java deleted file mode 100644 index 9d51278..0000000 --- a/core/src/test/java/io/github/dfa1/vortex/encoding/ZigZagEncodingTest.java +++ /dev/null @@ -1,147 +0,0 @@ -package io.github.dfa1.vortex.encoding; - -import io.github.dfa1.vortex.core.array.Array; -import io.github.dfa1.vortex.core.array.IntArray; -import org.junit.jupiter.api.Nested; -import org.junit.jupiter.api.Test; -import org.junit.jupiter.params.ParameterizedTest; -import org.junit.jupiter.params.provider.Arguments; -import org.junit.jupiter.params.provider.MethodSource; - -import java.lang.foreign.Arena; -import java.lang.foreign.MemorySegment; -import java.nio.ByteBuffer; -import java.nio.ByteOrder; -import java.util.stream.Stream; - -import static org.assertj.core.api.Assertions.assertThat; - -class ZigZagEncodingTest { - - - @Nested - class Decode { - - @ParameterizedTest(name = "{0}") - @MethodSource("i32Cases") - void decode_i32_zigzagDecodesCorrectly(String name, int[] encoded, int[] expected) { - // Given - DecodeContext ctx = buildI32Ctx(encoded); - var sut = new ZigZagEncoding(); - - // When - var result = sut.decode(ctx); - - // Then - assertThat(result).isInstanceOf(IntArray.class); - assertThat(result.length()).isEqualTo(expected.length); - MemorySegment seg = result.buffer(0); - for (int i = 0; i < expected.length; i++) { - assertThat(seg.get(PTypeIO.LE_INT, (long) i * 4)) - .as("index %d", i).isEqualTo(expected[i]); - } - } - - @Test - void decode_empty_returnsEmptyArray() { - // Given - DecodeContext ctx = buildI32Ctx(new int[]{}); - var sut = new ZigZagEncoding(); - - // When - var result = sut.decode(ctx); - - // Then - assertThat(result.length()).isZero(); - } - - static Stream i32Cases() { - return Stream.of( - // zigzag: 0→0, 1→-1, 2→1, 3→-2, 4→2 - Arguments.of("zeros", new int[]{0, 0, 0}, new int[]{0, 0, 0}), - Arguments.of("mixed", new int[]{0, 1, 2, 3, 4}, new int[]{0, -1, 1, -2, 2}), - Arguments.of("large", new int[]{Integer.MAX_VALUE & ~1, (Integer.MAX_VALUE & ~1) | 1}, - new int[]{Integer.MAX_VALUE / 2, Integer.MIN_VALUE / 2}) - ); - } - - private static DecodeContext buildI32Ctx(int[] encodedUnsigned) { - ByteBuffer buf = ByteBuffer.allocate(encodedUnsigned.length * 4).order(ByteOrder.LITTLE_ENDIAN); - for (int v : encodedUnsigned) { - buf.putInt(v); - } - buf.flip(); - MemorySegment seg = MemorySegment.ofBuffer(buf); - - ArrayNode primitiveNode = ArrayNode.of(EncodingId.VORTEX_PRIMITIVE, null, new ArrayNode[0], new int[]{0}, null); - ArrayNode zigzagNode = ArrayNode.of(EncodingId.VORTEX_ZIGZAG, null, new ArrayNode[]{primitiveNode}, new int[0], null); - - EncodingRegistry registry = TestRegistry.of(new ZigZagEncoding(), new PrimitiveEncoding()); - return new DecodeContext(zigzagNode, DTypes.I32, encodedUnsigned.length, - new MemorySegment[]{seg}, registry, Arena.ofAuto()); - } - } - - @Nested - class Encode { - - @ParameterizedTest - @MethodSource("i32RoundtripArrays") - void encodeDecode_i32_isLossless(int[] data) { - // Given - var sut = new ZigZagEncoding(); - EncodingRegistry registry = TestRegistry.withPrimitive(sut); - var le = PTypeIO.LE_INT; - - // When - EncodeResult encoded = sut.encode(DTypes.I32, data); - DecodeContext ctx = EncodeTestHelper.toDecodeContext(encoded, data.length, DTypes.I32, registry); - Array result = sut.decode(ctx); - - // Then - assertThat(result.length()).isEqualTo(data.length); - for (int i = 0; i < data.length; i++) { - assertThat(result.buffer(0).get(le, (long) i * 4)).as("index %d", i).isEqualTo(data[i]); - } - } - - @ParameterizedTest - @MethodSource("i64RoundtripArrays") - void encodeDecode_i64_isLossless(long[] data) { - // Given - var sut = new ZigZagEncoding(); - EncodingRegistry registry = TestRegistry.withPrimitive(sut); - var le = PTypeIO.LE_LONG; - - // When - EncodeResult encoded = sut.encode(DTypes.I64, data); - DecodeContext ctx = EncodeTestHelper.toDecodeContext(encoded, data.length, DTypes.I64, registry); - Array result = sut.decode(ctx); - - // Then - assertThat(result.length()).isEqualTo(data.length); - for (int i = 0; i < data.length; i++) { - assertThat(result.buffer(0).get(le, (long) i * 8)).as("index %d", i).isEqualTo(data[i]); - } - } - - static Stream i32RoundtripArrays() { - return Stream.of( - new int[]{}, - new int[]{0}, - new int[]{-1, 1, -2, 2}, - new int[]{Integer.MIN_VALUE, Integer.MAX_VALUE, 0}, - new int[]{-100, 0, 100, -1000, 1000} - ); - } - - static Stream i64RoundtripArrays() { - return Stream.of( - new long[]{}, - new long[]{0L}, - new long[]{Long.MIN_VALUE, Long.MAX_VALUE, 0L}, - new long[]{-1L, 1L, -2L, 2L} - ); - } - } -} diff --git a/core/src/test/java/io/github/dfa1/vortex/encoding/ZstdEncodingTest.java b/core/src/test/java/io/github/dfa1/vortex/encoding/ZstdEncodingTest.java deleted file mode 100644 index 4bf9ef1..0000000 --- a/core/src/test/java/io/github/dfa1/vortex/encoding/ZstdEncodingTest.java +++ /dev/null @@ -1,541 +0,0 @@ -package io.github.dfa1.vortex.encoding; - -import com.github.luben.zstd.ZstdCompressCtx; -import io.airlift.compress.v3.zstd.ZstdJavaCompressor; -import io.airlift.compress.v3.zstd.ZstdCompressor; -import io.github.dfa1.vortex.core.DType; -import io.github.dfa1.vortex.core.PType; -import io.github.dfa1.vortex.core.VortexException; -import io.github.dfa1.vortex.core.array.IntArray; -import io.github.dfa1.vortex.core.array.LongArray; -import io.github.dfa1.vortex.core.array.MaskedArray; -import io.github.dfa1.vortex.core.array.VarBinArray; -import io.github.dfa1.vortex.proto.EncodingProtos; -import org.junit.jupiter.api.Nested; -import org.junit.jupiter.api.Test; - -import java.lang.foreign.Arena; -import java.lang.foreign.MemorySegment; -import java.nio.ByteBuffer; -import java.nio.ByteOrder; -import java.nio.charset.StandardCharsets; -import java.util.ArrayList; -import java.util.Arrays; -import java.util.List; - -import static org.assertj.core.api.Assertions.assertThat; -import static org.assertj.core.api.Assertions.assertThatThrownBy; - -class ZstdEncodingTest { - - private static final DType I32 = new DType.Primitive(PType.I32, false); - private static final DType I64 = new DType.Primitive(PType.I64, false); - private static final DType UTF8 = new DType.Utf8(false); - - @Nested - class Encode { - - @Test - void encode_i32_roundTrips() { - // Given - var sut = new ZstdEncoding(); - int[] data = {10, 20, 30, 40}; - - // When - EncodeResult result = sut.encode(I32, data); - DecodeContext ctx = EncodeTestHelper.toDecodeContext(result, data.length, I32, EncodingRegistry.empty()); - IntArray decoded = (IntArray) sut.decode(ctx); - - // Then - assertThat(decoded.length()).isEqualTo(data.length); - for (int i = 0; i < data.length; i++) { - assertThat(decoded.getInt(i)).as("index %d", i).isEqualTo(data[i]); - } - } - - @Test - void encode_i64_roundTrips() { - // Given - var sut = new ZstdEncoding(); - long[] data = {100L, 200L, 300L}; - - // When - EncodeResult result = sut.encode(I64, data); - DecodeContext ctx = EncodeTestHelper.toDecodeContext(result, data.length, I64, EncodingRegistry.empty()); - LongArray decoded = (LongArray) sut.decode(ctx); - - // Then - assertThat(decoded.length()).isEqualTo(data.length); - for (int i = 0; i < data.length; i++) { - assertThat(decoded.getLong(i)).as("index %d", i).isEqualTo(data[i]); - } - } - - @Test - void encode_utf8_roundTrips() { - // Given - var sut = new ZstdEncoding(); - String[] data = {"hello", "world", "zstd"}; - - // When - EncodeResult result = sut.encode(UTF8, data); - DecodeContext ctx = EncodeTestHelper.toDecodeContext(result, data.length, UTF8, EncodingRegistry.empty()); - VarBinArray decoded = (VarBinArray) sut.decode(ctx); - - // Then - assertThat(decoded.length()).isEqualTo(data.length); - for (int i = 0; i < data.length; i++) { - assertThat(decoded.getString(i)).as("index %d", i).isEqualTo(data[i]); - } - } - - @Test - void encode_emptyArray_roundTrips() { - // Given - var sut = new ZstdEncoding(); - int[] data = {}; - - // When - EncodeResult result = sut.encode(I32, data); - DecodeContext ctx = EncodeTestHelper.toDecodeContext(result, data.length, I32, EncodingRegistry.empty()); - IntArray decoded = (IntArray) sut.decode(ctx); - - // Then - assertThat(decoded.length()).isZero(); - } - - @Test - void encode_unsupportedDtype_throwsVortexException() { - // Given - var sut = new ZstdEncoding(); - - // When / Then - assertThatThrownBy(() -> sut.encode(new DType.Null(false), null)) - .isInstanceOf(VortexException.class); - } - } - - @Nested - class Decode { - - @Test - void decode_primitiveI32_singleFrame_roundTrips() { - // Given - var sut = new ZstdEncoding(); - int[] values = {10, 20, 30, 40}; - byte[] raw = toLeBytes(values); - byte[] compressed = compress(raw); - DecodeContext ctx = makeCtx( - metaNoDict(new long[]{raw.length}, new long[]{values.length}), - I32, values.length, compressed - ); - - // When - IntArray result = (IntArray) sut.decode(ctx); - - // Then - assertThat(result.length()).isEqualTo(values.length); - for (int i = 0; i < values.length; i++) { - assertThat(result.getInt(i)).as("index %d", i).isEqualTo(values[i]); - } - } - - @Test - void decode_primitiveI64_singleFrame_roundTrips() { - // Given - var sut = new ZstdEncoding(); - long[] values = {100L, 200L, 300L}; - byte[] raw = toLeBytes(values); - byte[] compressed = compress(raw); - DecodeContext ctx = makeCtx( - metaNoDict(new long[]{raw.length}, new long[]{values.length}), - I64, values.length, compressed - ); - - // When - LongArray result = (LongArray) sut.decode(ctx); - - // Then - assertThat(result.length()).isEqualTo(values.length); - for (int i = 0; i < values.length; i++) { - assertThat(result.getLong(i)).as("index %d", i).isEqualTo(values[i]); - } - } - - @Test - void decode_utf8_singleFrame_roundTrips() { - // Given - var sut = new ZstdEncoding(); - String[] strings = {"hello", "world", "zstd"}; - byte[] raw = toLengthPrefixed(strings); - byte[] compressed = compress(raw); - DecodeContext ctx = makeCtx( - metaNoDict(new long[]{raw.length}, new long[]{strings.length}), - UTF8, strings.length, compressed - ); - - // When - VarBinArray result = (VarBinArray) sut.decode(ctx); - - // Then - assertThat(result.length()).isEqualTo(strings.length); - for (int i = 0; i < strings.length; i++) { - assertThat(result.getString(i)).as("index %d", i).isEqualTo(strings[i]); - } - } - - @Test - void decode_primitiveI32_multipleFrames_roundTrips() { - // Given - var sut = new ZstdEncoding(); - int[] frame0Values = {1, 2, 3}; - int[] frame1Values = {4, 5}; - byte[] raw0 = toLeBytes(frame0Values); - byte[] raw1 = toLeBytes(frame1Values); - byte[] comp0 = compress(raw0); - byte[] comp1 = compress(raw1); - byte[] meta = metaNoDict( - new long[]{raw0.length, raw1.length}, - new long[]{frame0Values.length, frame1Values.length} - ); - DecodeContext ctx = makeCtx(meta, I32, 5, comp0, comp1); - - // When - IntArray result = (IntArray) sut.decode(ctx); - - // Then - assertThat(result.length()).isEqualTo(5); - assertThat(result.getInt(0)).isEqualTo(1); - assertThat(result.getInt(4)).isEqualTo(5); - } - - @Test - void decode_emptyArray_returnsZeroLengthArray() { - // Given - var sut = new ZstdEncoding(); - byte[] meta = metaNoDict(new long[0], new long[0]); - DecodeContext ctx = makeCtx(meta, I32, 0); - - // When - IntArray result = (IntArray) sut.decode(ctx); - - // Then - assertThat(result.length()).isZero(); - } - - @Test - void decode_withDictionary_primitive_roundTrips() { - // Given - var sut = new ZstdEncoding(); - int[] values = {10, 20, 30, 40}; - byte[] raw = toLeBytes(values); - byte[] dictBytes = makeDictFor(raw); - byte[] compressed = compressWithDict(raw, dictBytes); - byte[] meta = EncodingProtos.ZstdMetadata.newBuilder() - .setDictionarySize(dictBytes.length) - .addFrames(EncodingProtos.ZstdFrameMetadata.newBuilder() - .setUncompressedSize(raw.length) - .setNValues(values.length)) - .build().toByteArray(); - // buffer[0]=dict, buffer[1]=frame - DecodeContext ctx = makeDictCtx(meta, I32, values.length, dictBytes, compressed); - - // When - IntArray result = (IntArray) sut.decode(ctx); - - // Then - assertThat(result.length()).isEqualTo(values.length); - for (int i = 0; i < values.length; i++) { - assertThat(result.getInt(i)).as("index %d", i).isEqualTo(values[i]); - } - } - - @Test - void decode_withDictionary_utf8_roundTrips() { - // Given - var sut = new ZstdEncoding(); - String[] strings = {"hello", "world", "zstd"}; - byte[] raw = toLengthPrefixed(strings); - byte[] dictBytes = makeDictFor(raw); - byte[] compressed = compressWithDict(raw, dictBytes); - byte[] meta = EncodingProtos.ZstdMetadata.newBuilder() - .setDictionarySize(dictBytes.length) - .addFrames(EncodingProtos.ZstdFrameMetadata.newBuilder() - .setUncompressedSize(raw.length) - .setNValues(strings.length)) - .build().toByteArray(); - DecodeContext ctx = makeDictCtx(meta, UTF8, strings.length, dictBytes, compressed); - - // When - VarBinArray result = (VarBinArray) sut.decode(ctx); - - // Then - assertThat(result.length()).isEqualTo(strings.length); - for (int i = 0; i < strings.length; i++) { - assertThat(result.getString(i)).as("index %d", i).isEqualTo(strings[i]); - } - } - - @Test - void decode_withDictionary_multipleFrames_roundTrips() { - // Given - var sut = new ZstdEncoding(); - int[] frame0 = {1, 2, 3}; - int[] frame1 = {4, 5}; - byte[] raw0 = toLeBytes(frame0); - byte[] raw1 = toLeBytes(frame1); - byte[] dictBytes = makeDictFor(raw0, raw1); - byte[] comp0 = compressWithDict(raw0, dictBytes); - byte[] comp1 = compressWithDict(raw1, dictBytes); - byte[] meta = EncodingProtos.ZstdMetadata.newBuilder() - .setDictionarySize(dictBytes.length) - .addFrames(EncodingProtos.ZstdFrameMetadata.newBuilder() - .setUncompressedSize(raw0.length).setNValues(frame0.length)) - .addFrames(EncodingProtos.ZstdFrameMetadata.newBuilder() - .setUncompressedSize(raw1.length).setNValues(frame1.length)) - .build().toByteArray(); - DecodeContext ctx = makeDictCtx(meta, I32, 5, dictBytes, comp0, comp1); - - // When - IntArray result = (IntArray) sut.decode(ctx); - - // Then - assertThat(result.length()).isEqualTo(5); - for (int i = 0; i < 3; i++) { - assertThat(result.getInt(i)).isEqualTo(frame0[i]); - } - for (int i = 0; i < 2; i++) { - assertThat(result.getInt(3 + i)).isEqualTo(frame1[i]); - } - } - - @Test - void decode_nullable_primitive_scattersValuesCorrectly() { - // Given - var sut = new ZstdEncoding(); - // validity: [true, false, true, false] — positions 0,2 are valid - boolean[] validityBits = {true, false, true, false}; - // only valid values compressed: 10, 30 - byte[] raw = toLeBytes(new int[]{10, 30}); - byte[] compressed = compress(raw); - DType i32Nullable = new DType.Primitive(PType.I32, true); - DecodeContext ctx = makeNullableCtx( - metaNoDict(new long[]{raw.length}, new long[]{2}), - i32Nullable, 4, validityBits, compressed); - - // When - MaskedArray result = (MaskedArray) sut.decode(ctx); - - // Then - assertThat(result.length()).isEqualTo(4); - assertThat(result.isValid(0)).isTrue(); - assertThat(result.isValid(1)).isFalse(); - assertThat(result.isValid(2)).isTrue(); - assertThat(result.isValid(3)).isFalse(); - IntArray child = (IntArray) result.child(0); - assertThat(child.getInt(0)).isEqualTo(10); - assertThat(child.getInt(2)).isEqualTo(30); - } - - @Test - void decode_nullable_utf8_scattersValuesCorrectly() { - // Given - var sut = new ZstdEncoding(); - // validity: [true, false, true] — positions 0,2 are valid - boolean[] validityBits = {true, false, true}; - // only valid strings compressed - byte[] raw = toLengthPrefixed(new String[]{"hello", "world"}); - byte[] compressed = compress(raw); - DType utf8Nullable = new DType.Utf8(true); - DecodeContext ctx = makeNullableCtx( - metaNoDict(new long[]{raw.length}, new long[]{2}), - utf8Nullable, 3, validityBits, compressed); - - // When - MaskedArray result = (MaskedArray) sut.decode(ctx); - - // Then - assertThat(result.length()).isEqualTo(3); - assertThat(result.isValid(0)).isTrue(); - assertThat(result.isValid(1)).isFalse(); - assertThat(result.isValid(2)).isTrue(); - VarBinArray child = (VarBinArray) result.child(0); - assertThat(child.getString(0)).isEqualTo("hello"); - assertThat(child.getString(2)).isEqualTo("world"); - } - - @Test - void decode_allNull_returnsEmptyMaskedArray() { - // Given - var sut = new ZstdEncoding(); - boolean[] validityBits = {false, false, false}; - // no valid values — zero-length compressed buffer - byte[] raw = new byte[0]; - byte[] compressed = compress(raw); - DType i32Nullable = new DType.Primitive(PType.I32, true); - DecodeContext ctx = makeNullableCtx( - metaNoDict(new long[]{raw.length}, new long[]{0}), - i32Nullable, 3, validityBits, compressed); - - // When - MaskedArray result = (MaskedArray) sut.decode(ctx); - - // Then - assertThat(result.length()).isEqualTo(3); - assertThat(result.isValid(0)).isFalse(); - assertThat(result.isValid(1)).isFalse(); - assertThat(result.isValid(2)).isFalse(); - } - - @Test - void decode_missingMetadata_throwsVortexException() { - // Given - var sut = new ZstdEncoding(); - ArrayNode node = ArrayNode.of(EncodingId.VORTEX_ZSTD, null, new ArrayNode[0], new int[0], null); - DecodeContext ctx = new DecodeContext(node, I32, 0, new MemorySegment[0], - EncodingRegistry.empty(), Arena.ofAuto()); - - // When / Then - assertThatThrownBy(() -> sut.decode(ctx)) - .isInstanceOf(VortexException.class) - .hasMessageContaining("missing metadata"); - } - - private static DecodeContext makeDictCtx( - byte[] meta, DType dtype, long n, byte[] dictBytes, byte[]... compressedFrames - ) { - // buffer[0] = dict, buffer[1..] = frames - MemorySegment[] segments = new MemorySegment[1 + compressedFrames.length]; - segments[0] = MemorySegment.ofArray(dictBytes); - int[] bufIndices = new int[1 + compressedFrames.length]; - bufIndices[0] = 0; - for (int i = 0; i < compressedFrames.length; i++) { - segments[i + 1] = MemorySegment.ofArray(compressedFrames[i]); - bufIndices[i + 1] = i + 1; - } - ArrayNode node = ArrayNode.of(EncodingId.VORTEX_ZSTD, ByteBuffer.wrap(meta), - new ArrayNode[0], bufIndices, null); - return new DecodeContext(node, dtype, n, segments, EncodingRegistry.empty(), Arena.ofAuto()); - } - - private static byte[] makeDictFor(byte[]... samples) { - // Repeat samples to meet zstd's minimum training data requirement (~1 KB) - int total = 0; - for (byte[] s : samples) { - total += s.length; - } - int repeats = Math.max(1, 1024 / Math.max(total, 1)); - byte[][] expanded = new byte[samples.length * repeats][]; - for (int r = 0; r < repeats; r++) { - System.arraycopy(samples, 0, expanded, r * samples.length, samples.length); - } - byte[] dict = new byte[256]; - com.github.luben.zstd.Zstd.trainFromBuffer(expanded, dict); - return dict; - } - - private static byte[] compressWithDict(byte[] data, byte[] dictBytes) { - try (ZstdCompressCtx ctx = new ZstdCompressCtx()) { - ctx.loadDict(dictBytes); - return ctx.compress(data); - } catch (Exception e) { - throw new RuntimeException(e); - } - } - - private static DecodeContext makeNullableCtx( - byte[] meta, DType dtype, long n, boolean[] validityBits, byte[]... compressedFrames - ) { - BoolEncoding boolEncoding = new BoolEncoding(); - EncodeResult validityResult = boolEncoding.encode(new DType.Bool(false), validityBits); - EncodeNode remappedValidity = EncodeNode.remapBufferIndices( - validityResult.rootNode(), compressedFrames.length); - - List allSegments = new ArrayList<>(); - int[] bufIndices = new int[compressedFrames.length]; - for (int i = 0; i < compressedFrames.length; i++) { - allSegments.add(MemorySegment.ofArray(compressedFrames[i])); - bufIndices[i] = i; - } - allSegments.addAll(validityResult.buffers()); - - ArrayNode validityNode = toArrayNode(remappedValidity); - ArrayNode node = ArrayNode.of(EncodingId.VORTEX_ZSTD, ByteBuffer.wrap(meta), - new ArrayNode[]{validityNode}, bufIndices, null); - - EncodingRegistry registry = EncodingRegistry.empty(); - registry.register(new BoolEncoding()); - - return new DecodeContext(node, dtype, n, allSegments.toArray(new MemorySegment[0]), - registry, Arena.ofAuto()); - } - - private static ArrayNode toArrayNode(EncodeNode enc) { - ArrayNode[] children = new ArrayNode[enc.children().length]; - for (int i = 0; i < children.length; i++) { - children[i] = toArrayNode(enc.children()[i]); - } - return ArrayNode.of(enc.encodingId(), enc.metadata(), children, enc.bufferIndices(), null); - } - - private static byte[] metaNoDict(long[] uncompressedSizes, long[] nValues) { - EncodingProtos.ZstdMetadata.Builder builder = EncodingProtos.ZstdMetadata.newBuilder() - .setDictionarySize(0); - for (int i = 0; i < uncompressedSizes.length; i++) { - builder.addFrames(EncodingProtos.ZstdFrameMetadata.newBuilder() - .setUncompressedSize(uncompressedSizes[i]) - .setNValues(nValues[i])); - } - return builder.build().toByteArray(); - } - - private static DecodeContext makeCtx(byte[] meta, DType dtype, long n, byte[]... compressedFrames) { - MemorySegment[] segments = new MemorySegment[compressedFrames.length]; - int[] bufIndices = new int[compressedFrames.length]; - for (int i = 0; i < compressedFrames.length; i++) { - segments[i] = MemorySegment.ofArray(compressedFrames[i]); - bufIndices[i] = i; - } - ArrayNode node = ArrayNode.of(EncodingId.VORTEX_ZSTD, ByteBuffer.wrap(meta), - new ArrayNode[0], bufIndices, null); - return new DecodeContext(node, dtype, n, segments, EncodingRegistry.empty(), Arena.ofAuto()); - } - - private static byte[] compress(byte[] input) { - ZstdCompressor compressor = new ZstdJavaCompressor(); - byte[] out = new byte[compressor.maxCompressedLength(input.length)]; - int len = compressor.compress(input, 0, input.length, out, 0, out.length); - return Arrays.copyOf(out, len); - } - - private static byte[] toLeBytes(int[] values) { - ByteBuffer buf = ByteBuffer.allocate(values.length * 4).order(ByteOrder.LITTLE_ENDIAN); - for (int v : values) { - buf.putInt(v); - } - return buf.array(); - } - - private static byte[] toLeBytes(long[] values) { - ByteBuffer buf = ByteBuffer.allocate(values.length * 8).order(ByteOrder.LITTLE_ENDIAN); - for (long v : values) { - buf.putLong(v); - } - return buf.array(); - } - - private static byte[] toLengthPrefixed(String[] strings) { - int total = 0; - for (String s : strings) { - total += 4 + s.getBytes(StandardCharsets.UTF_8).length; - } - ByteBuffer buf = ByteBuffer.allocate(total).order(ByteOrder.LITTLE_ENDIAN); - for (String s : strings) { - byte[] bytes = s.getBytes(StandardCharsets.UTF_8); - buf.putInt(bytes.length); - buf.put(bytes); - } - return buf.array(); - } - } -} diff --git a/csv/pom.xml b/csv/pom.xml deleted file mode 100644 index 34a9286..0000000 --- a/csv/pom.xml +++ /dev/null @@ -1,42 +0,0 @@ - - - 4.0.0 - - io.github.dfa1.vortex - vortex-java - 0.3.0-SNAPSHOT - - - csv - - - - io.github.dfa1.vortex - reader - - - io.github.dfa1.vortex - writer - - - de.siegmar - fastcsv - - - - org.junit.jupiter - junit-jupiter - test - - - org.assertj - assertj-core - test - - - org.mockito - mockito-junit-jupiter - test - - - diff --git a/csv/src/main/java/io/github/dfa1/vortex/csv/CsvExporter.java b/csv/src/main/java/io/github/dfa1/vortex/csv/CsvExporter.java deleted file mode 100644 index 60eef02..0000000 --- a/csv/src/main/java/io/github/dfa1/vortex/csv/CsvExporter.java +++ /dev/null @@ -1,119 +0,0 @@ -package io.github.dfa1.vortex.csv; - -import de.siegmar.fastcsv.writer.CsvWriter; -import io.github.dfa1.vortex.core.DType; -import io.github.dfa1.vortex.core.VortexException; -import io.github.dfa1.vortex.core.array.BoolArray; -import io.github.dfa1.vortex.core.array.ByteArray; -import io.github.dfa1.vortex.core.array.DoubleArray; -import io.github.dfa1.vortex.core.array.FloatArray; -import io.github.dfa1.vortex.core.array.IntArray; -import io.github.dfa1.vortex.core.array.LongArray; -import io.github.dfa1.vortex.core.array.ShortArray; -import io.github.dfa1.vortex.core.array.VarBinArray; -import io.github.dfa1.vortex.core.array.Array; -import io.github.dfa1.vortex.io.VortexReader; -import io.github.dfa1.vortex.scan.ScanIterator; -import io.github.dfa1.vortex.scan.ScanOptions; -import io.github.dfa1.vortex.scan.ScanResult; - -import java.io.FilterWriter; -import java.io.IOException; -import java.io.Writer; -import java.nio.file.Path; -import java.util.List; - -/// Reads a Vortex file and writes rows to a CSV destination. -/// -/// The header row is derived from [DType.Struct] field names. -/// Only struct root dtype is supported; throws [VortexException] otherwise. -public final class CsvExporter { - - private CsvExporter() { - } - - public static void exportCsv(Path vortexPath, Path csvPath) throws IOException { - exportCsv(vortexPath, csvPath, ExportOptions.defaults()); - } - - public static void exportCsv(Path vortexPath, Path csvPath, ExportOptions options) throws IOException { - try (VortexReader reader = VortexReader.open(vortexPath); - CsvWriter csvWriter = CsvWriter.builder() - .fieldSeparator(options.delimiter()) - .build(csvPath)) { - export(reader, csvWriter, options, ScanOptions.all(), RowPredicate.all()); - } - } - - /// Export to a caller-owned [Writer]; the writer is flushed but not closed. - public static void exportCsv(Path vortexPath, Writer out, ExportOptions options) throws IOException { - exportCsvFiltered(vortexPath, out, options, ScanOptions.all(), RowPredicate.all()); - } - - /// Like [#exportCsv(Path, Writer, ExportOptions)] but with zone-map chunk pruning - /// ([ScanOptions#rowFilter()]) and a row-level predicate applied after decoding. - public static void exportCsvFiltered(Path vortexPath, Writer out, ExportOptions options, - ScanOptions scanOptions, RowPredicate predicate) throws IOException { - Writer shielded = new FilterWriter(out) { - @Override - public void close() { - // do not close the caller-owned writer - } - }; - try (VortexReader reader = VortexReader.open(vortexPath); - CsvWriter csvWriter = CsvWriter.builder() - .fieldSeparator(options.delimiter()) - .build(shielded)) { - export(reader, csvWriter, options, scanOptions, predicate); - } - } - - private static void export(VortexReader reader, CsvWriter csvWriter, ExportOptions options, - ScanOptions scanOptions, RowPredicate predicate) { - if (!(reader.dtype() instanceof DType.Struct schema)) { - throw new VortexException("only struct root dtype supported for CSV export"); - } - List colNames = options.hasProjection() ? options.columns() : schema.fieldNames(); - int colCount = colNames.size(); - - if (options.writeHeader()) { - csvWriter.writeRecord(colNames); - } - - String[] row = new String[colCount]; - try (ScanIterator iter = reader.scan(scanOptions)) { - while (iter.hasNext()) { - ScanResult chunk = iter.next(); - Array[] arrays = new Array[colCount]; - for (int c = 0; c < colCount; c++) { - arrays[c] = chunk.column(colNames.get(c)); - } - long rowCount = chunk.rowCount(); - for (long r = 0; r < rowCount; r++) { - if (!predicate.test(chunk, r)) { - continue; - } - for (int c = 0; c < colCount; c++) { - row[c] = cellValue(arrays[c], r); - } - csvWriter.writeRecord(row); - } - } - } - } - - private static String cellValue(Array arr, long rowIdx) { - return switch (arr) { - case LongArray la -> Long.toString(la.getLong(rowIdx)); - case IntArray ia -> Integer.toString(ia.getInt(rowIdx)); - case ShortArray sa -> Short.toString(sa.getShort(rowIdx)); - case ByteArray ba -> Byte.toString(ba.getByte(rowIdx)); - case DoubleArray da -> Double.toString(da.getDouble(rowIdx)); - case FloatArray fa -> Float.toString(fa.getFloat(rowIdx)); - case BoolArray ba -> Boolean.toString(ba.getBoolean(rowIdx)); - case VarBinArray va -> va.getString(rowIdx); - default -> throw new VortexException( - "unsupported array type for CSV export: " + arr.getClass().getSimpleName()); - }; - } -} diff --git a/csv/src/main/java/io/github/dfa1/vortex/csv/CsvImporter.java b/csv/src/main/java/io/github/dfa1/vortex/csv/CsvImporter.java deleted file mode 100644 index fa72bd0..0000000 --- a/csv/src/main/java/io/github/dfa1/vortex/csv/CsvImporter.java +++ /dev/null @@ -1,206 +0,0 @@ -package io.github.dfa1.vortex.csv; - -import de.siegmar.fastcsv.reader.CsvReader; -import de.siegmar.fastcsv.reader.CsvRecord; -import io.github.dfa1.vortex.core.DType; -import io.github.dfa1.vortex.core.PType; -import io.github.dfa1.vortex.writer.VortexWriter; - -import java.io.IOException; -import java.nio.channels.FileChannel; -import java.nio.file.Path; -import java.nio.file.StandardOpenOption; -import java.util.ArrayList; -import java.util.LinkedHashMap; -import java.util.List; -import java.util.Map; - -/// Parses a CSV file and writes a Vortex file. -/// -/// Column types are inferred in priority order: long → double → boolean → utf8. -/// Provide a schema via [ImportOptions#withSchema] to skip inference. -/// Empty cell values are treated as 0 / false / "" for typed columns. -public final class CsvImporter { - - private CsvImporter() { - } - - public static void importCsv(Path csvPath, Path vortexPath) throws IOException { - importCsv(csvPath, vortexPath, ImportOptions.defaults()); - } - - public static void importCsv(Path csvPath, Path vortexPath, ImportOptions options) throws IOException { - List rows = readAllRows(csvPath, options); - if (rows.isEmpty()) { - throw new IllegalArgumentException("CSV file is empty"); - } - - String[] headers; - int dataStart; - if (options.hasHeader()) { - headers = rows.getFirst(); - dataStart = 1; - } else { - headers = generateHeaders(rows.getFirst().length); - dataStart = 0; - } - - List dataRows = rows.subList(dataStart, rows.size()); - int colCount = headers.length; - - DType.Struct schema; - if (options.schema() != null) { - schema = options.schema(); - } else { - schema = inferSchema(headers, dataRows, colCount); - } - - try (FileChannel channel = FileChannel.open( - vortexPath, StandardOpenOption.CREATE, StandardOpenOption.WRITE, - StandardOpenOption.TRUNCATE_EXISTING); - VortexWriter writer = VortexWriter.create(channel, schema, options.writeOptions())) { - int chunkSize = options.chunkSize(); - int total = dataRows.size(); - for (int start = 0; start < total; start += chunkSize) { - int end = Math.min(start + chunkSize, total); - writer.writeChunk(buildChunk(schema, dataRows.subList(start, end))); - if (options.progressListener() != null) { - options.progressListener().onProgress(end, total); - } - } - } - } - - private static List readAllRows(Path path, ImportOptions options) throws IOException { - List rows = new ArrayList<>(); - try (CsvReader reader = CsvReader.builder() - .fieldSeparator(options.delimiter()) - .ofCsvRecord(path)) { - for (CsvRecord record : reader) { - rows.add(record.getFields().toArray(String[]::new)); - } - } - return rows; - } - - private static String[] generateHeaders(int colCount) { - String[] headers = new String[colCount]; - for (int i = 0; i < colCount; i++) { - headers[i] = "col" + i; - } - return headers; - } - - private static DType.Struct inferSchema(String[] headers, List rows, int colCount) { - List names = List.of(headers); - List types = new ArrayList<>(colCount); - for (int c = 0; c < colCount; c++) { - types.add(inferColumnType(rows, c)); - } - return new DType.Struct(names, types, false); - } - - /// Infers the narrowest type for a column using a single pass. - /// - /// Priority: long → double → bool → utf8. Each flag starts true and can only - /// transition to false. Empty cells are skipped (compatible with any type). - /// An all-empty column is inferred as long (all flags remain true). - /// - /// Integer values always infer as i64 (not i32/i16): CSV has no type annotations, - /// so the widest safe integer is chosen. Use [ImportOptions#withSchema] to force i32/i16. - /// Floating-point values always infer as f64 (not f32) for the same reason. - private static DType inferColumnType(List rows, int colIdx) { - boolean canBeLong = true; - boolean canBeDouble = true; - boolean canBeBool = true; - - for (String[] row : rows) { - String val = safeGet(row, colIdx); - if (val.isEmpty()) { - continue; - } - if (canBeLong) { - try { - Long.parseLong(val); - } catch (NumberFormatException e) { - canBeLong = false; - } - } - if (canBeDouble) { - try { - Double.parseDouble(val); - } catch (NumberFormatException e) { - canBeDouble = false; - } - } - if (canBeBool) { - if (!val.equalsIgnoreCase("true") && !val.equalsIgnoreCase("false")) { - canBeBool = false; - } - } - } - - if (canBeLong) { - return new DType.Primitive(PType.I64, false); - } - if (canBeDouble) { - return new DType.Primitive(PType.F64, false); - } - if (canBeBool) { - return new DType.Bool(false); - } - return new DType.Utf8(false); - } - - private static Map buildChunk(DType.Struct schema, List rows) { - int n = rows.size(); - Map chunk = new LinkedHashMap<>(); - for (int c = 0; c < schema.fieldNames().size(); c++) { - chunk.put(schema.fieldNames().get(c), buildColumn(schema.fieldTypes().get(c), rows, c, n)); - } - return chunk; - } - - private static Object buildColumn(DType dtype, List rows, int colIdx, int n) { - return switch (dtype) { - case DType.Primitive p when p.ptype() == PType.I64 -> { - long[] arr = new long[n]; - for (int i = 0; i < n; i++) { - String v = safeGet(rows.get(i), colIdx); - arr[i] = v.isEmpty() ? 0L : Long.parseLong(v); - } - yield arr; - } - case DType.Primitive p when p.ptype() == PType.F64 -> { - double[] arr = new double[n]; - for (int i = 0; i < n; i++) { - String v = safeGet(rows.get(i), colIdx); - arr[i] = v.isEmpty() ? 0.0 : Double.parseDouble(v); - } - yield arr; - } - case DType.Bool ignored -> { - boolean[] arr = new boolean[n]; - for (int i = 0; i < n; i++) { - arr[i] = Boolean.parseBoolean(safeGet(rows.get(i), colIdx)); - } - yield arr; - } - case DType.Utf8 ignored -> { - String[] arr = new String[n]; - for (int i = 0; i < n; i++) { - arr[i] = safeGet(rows.get(i), colIdx); - } - yield arr; - } - default -> throw new UnsupportedOperationException("unsupported dtype for CSV import: " + dtype); - }; - } - - private static String safeGet(String[] row, int idx) { - if (idx >= row.length || row[idx] == null) { - return ""; - } - return row[idx]; - } -} diff --git a/csv/src/main/java/io/github/dfa1/vortex/csv/ExportOptions.java b/csv/src/main/java/io/github/dfa1/vortex/csv/ExportOptions.java deleted file mode 100644 index 61ea15b..0000000 --- a/csv/src/main/java/io/github/dfa1/vortex/csv/ExportOptions.java +++ /dev/null @@ -1,23 +0,0 @@ -package io.github.dfa1.vortex.csv; - -import java.util.List; - -/// Options controlling Vortex → CSV export. -public record ExportOptions( - char delimiter, - boolean writeHeader, - List columns -) { - public static ExportOptions defaults() { - return new ExportOptions(',', true, List.of()); - } - - /// Restrict output to specific columns (projection). Empty list = all columns. - public ExportOptions withColumns(List cols) { - return new ExportOptions(delimiter, writeHeader, List.copyOf(cols)); - } - - public boolean hasProjection() { - return !columns.isEmpty(); - } -} diff --git a/csv/src/main/java/io/github/dfa1/vortex/csv/ImportOptions.java b/csv/src/main/java/io/github/dfa1/vortex/csv/ImportOptions.java deleted file mode 100644 index 6545c82..0000000 --- a/csv/src/main/java/io/github/dfa1/vortex/csv/ImportOptions.java +++ /dev/null @@ -1,32 +0,0 @@ -package io.github.dfa1.vortex.csv; - -import io.github.dfa1.vortex.core.DType; -import io.github.dfa1.vortex.writer.WriteOptions; - -/// Options controlling CSV → Vortex import. -public record ImportOptions( - char delimiter, - int chunkSize, - boolean hasHeader, - DType.Struct schema, - ProgressListener progressListener, - WriteOptions writeOptions -) { - public static ImportOptions defaults() { - return new ImportOptions(',', 65_536, true, null, null, WriteOptions.cascading(3)); - } - - /// Override the inferred schema. The struct's field names become column names; - /// types control how each CSV column is parsed (positionally). - public ImportOptions withSchema(DType.Struct overrideSchema) { - return new ImportOptions(delimiter, chunkSize, hasHeader, overrideSchema, progressListener, writeOptions); - } - - public ImportOptions withProgressListener(ProgressListener listener) { - return new ImportOptions(delimiter, chunkSize, hasHeader, schema, listener, writeOptions); - } - - public ImportOptions withWriteOptions(WriteOptions options) { - return new ImportOptions(delimiter, chunkSize, hasHeader, schema, progressListener, options); - } -} diff --git a/csv/src/main/java/io/github/dfa1/vortex/csv/ProgressListener.java b/csv/src/main/java/io/github/dfa1/vortex/csv/ProgressListener.java deleted file mode 100644 index 46ad33d..0000000 --- a/csv/src/main/java/io/github/dfa1/vortex/csv/ProgressListener.java +++ /dev/null @@ -1,7 +0,0 @@ -package io.github.dfa1.vortex.csv; - -/// Callback invoked after each chunk is written during CSV import. -@FunctionalInterface -public interface ProgressListener { - void onProgress(long rowsDone, long rowsTotal); -} diff --git a/csv/src/main/java/io/github/dfa1/vortex/csv/RowPredicate.java b/csv/src/main/java/io/github/dfa1/vortex/csv/RowPredicate.java deleted file mode 100644 index 295ae29..0000000 --- a/csv/src/main/java/io/github/dfa1/vortex/csv/RowPredicate.java +++ /dev/null @@ -1,15 +0,0 @@ -package io.github.dfa1.vortex.csv; - -import io.github.dfa1.vortex.scan.ScanResult; - -/// Row-level predicate evaluated against decoded chunk data. -/// Used in conjunction with zone-map pruning: zone-maps skip whole chunks, -/// this predicate filters individual rows within surviving chunks. -@FunctionalInterface -public interface RowPredicate { - boolean test(ScanResult chunk, long rowIndex); - - static RowPredicate all() { - return (_, _) -> true; - } -} diff --git a/csv/src/test/java/io/github/dfa1/vortex/csv/CsvExporterTest.java b/csv/src/test/java/io/github/dfa1/vortex/csv/CsvExporterTest.java deleted file mode 100644 index 730bfd1..0000000 --- a/csv/src/test/java/io/github/dfa1/vortex/csv/CsvExporterTest.java +++ /dev/null @@ -1,94 +0,0 @@ -package io.github.dfa1.vortex.csv; - -import io.github.dfa1.vortex.core.DType; -import io.github.dfa1.vortex.core.PType; -import io.github.dfa1.vortex.writer.VortexWriter; -import io.github.dfa1.vortex.writer.WriteOptions; -import org.junit.jupiter.api.Test; -import org.junit.jupiter.api.io.TempDir; - -import java.io.StringWriter; -import java.nio.channels.FileChannel; -import java.nio.file.Files; -import java.nio.file.Path; -import java.nio.file.StandardOpenOption; -import java.util.List; -import java.util.Map; - -import static org.assertj.core.api.Assertions.assertThat; - -class CsvExporterTest { - - @Test - void exportsToCsvFile(@TempDir Path tmp) throws Exception { - // Given - Path vortex = tmp.resolve("data.vortex"); - DType.Struct schema = new DType.Struct( - List.of("id", "name"), - List.of(new DType.Primitive(PType.I64, false), new DType.Utf8(false)), - false); - try (FileChannel ch = FileChannel.open(vortex, StandardOpenOption.CREATE, StandardOpenOption.WRITE); - VortexWriter writer = VortexWriter.create(ch, schema, WriteOptions.defaults())) { - writer.writeChunk(Map.of("id", new long[]{1L, 2L}, "name", new String[]{"Alice", "Bob"})); - } - Path csv = tmp.resolve("out.csv"); - - // When - CsvExporter.exportCsv(vortex, csv); - - // Then - List lines = Files.readAllLines(csv); - assertThat(lines).hasSize(3); - assertThat(lines.get(0)).isEqualTo("id,name"); - assertThat(lines.get(1)).isEqualTo("1,Alice"); - assertThat(lines.get(2)).isEqualTo("2,Bob"); - } - - @Test - void exportsToWriter(@TempDir Path tmp) throws Exception { - // Given - Path vortex = tmp.resolve("data.vortex"); - DType.Struct schema = new DType.Struct( - List.of("x"), - List.of(new DType.Primitive(PType.F64, false)), - false); - try (FileChannel ch = FileChannel.open(vortex, StandardOpenOption.CREATE, StandardOpenOption.WRITE); - VortexWriter writer = VortexWriter.create(ch, schema, WriteOptions.defaults())) { - writer.writeChunk(Map.of("x", new double[]{1.5, 2.7})); - } - StringWriter out = new StringWriter(); - - // When - CsvExporter.exportCsv(vortex, out, ExportOptions.defaults()); - - // Then - String[] lines = out.toString().split("\r?\n"); - assertThat(lines).hasSize(3); - assertThat(lines[0]).isEqualTo("x"); - assertThat(lines[1]).isEqualTo("1.5"); - assertThat(lines[2]).isEqualTo("2.7"); - } - - @Test - void suppressesHeaderWhenConfigured(@TempDir Path tmp) throws Exception { - // Given - Path vortex = tmp.resolve("data.vortex"); - DType.Struct schema = new DType.Struct( - List.of("id"), - List.of(new DType.Primitive(PType.I64, false)), - false); - try (FileChannel ch = FileChannel.open(vortex, StandardOpenOption.CREATE, StandardOpenOption.WRITE); - VortexWriter writer = VortexWriter.create(ch, schema, WriteOptions.defaults())) { - writer.writeChunk(Map.of("id", new long[]{7L})); - } - Path csv = tmp.resolve("out.csv"); - - // When - CsvExporter.exportCsv(vortex, csv, new ExportOptions(',', false, java.util.List.of())); - - // Then - List lines = Files.readAllLines(csv); - assertThat(lines).hasSize(1); - assertThat(lines.getFirst()).isEqualTo("7"); - } -} diff --git a/csv/src/test/java/io/github/dfa1/vortex/csv/CsvImporterTest.java b/csv/src/test/java/io/github/dfa1/vortex/csv/CsvImporterTest.java deleted file mode 100644 index bf85599..0000000 --- a/csv/src/test/java/io/github/dfa1/vortex/csv/CsvImporterTest.java +++ /dev/null @@ -1,116 +0,0 @@ -package io.github.dfa1.vortex.csv; - -import io.github.dfa1.vortex.core.DType; -import io.github.dfa1.vortex.core.PType; -import io.github.dfa1.vortex.writer.WriteOptions; -import io.github.dfa1.vortex.core.array.LongArray; -import io.github.dfa1.vortex.core.array.VarBinArray; -import io.github.dfa1.vortex.io.VortexReader; -import io.github.dfa1.vortex.scan.ScanIterator; -import io.github.dfa1.vortex.scan.ScanOptions; -import io.github.dfa1.vortex.scan.ScanResult; -import org.junit.jupiter.api.Test; -import org.junit.jupiter.api.io.TempDir; - -import java.nio.file.Files; -import java.nio.file.Path; - -import static org.assertj.core.api.Assertions.assertThat; - -class CsvImporterTest { - - @Test - void infersTypedColumnsAndRoundTrips(@TempDir Path tmp) throws Exception { - // Given - Path csv = tmp.resolve("data.csv"); - Files.writeString(csv, "id,price,active,name\n1,1.5,true,Alice\n2,2.7,false,Bob\n"); - Path vortex = tmp.resolve("data.vortex"); - - // When - CsvImporter.importCsv(csv, vortex); - - // Then - try (VortexReader reader = VortexReader.open(vortex)) { - assertThat(reader.dtype()).isInstanceOf(DType.Struct.class); - DType.Struct schema = (DType.Struct) reader.dtype(); - assertThat(schema.fieldNames()).containsExactly("id", "price", "active", "name"); - assertThat(schema.fieldTypes().get(0)).isEqualTo(new DType.Primitive(PType.I64, false)); - assertThat(schema.fieldTypes().get(1)).isEqualTo(new DType.Primitive(PType.F64, false)); - assertThat(schema.fieldTypes().get(2)).isEqualTo(new DType.Bool(false)); - assertThat(schema.fieldTypes().get(3)).isEqualTo(new DType.Utf8(false)); - - try (ScanIterator iter = reader.scan(ScanOptions.all())) { - assertThat(iter.hasNext()).isTrue(); - ScanResult chunk = iter.next(); - assertThat(chunk.rowCount()).isEqualTo(2); - LongArray ids = chunk.column("id"); - assertThat(ids.getLong(0)).isEqualTo(1L); - assertThat(ids.getLong(1)).isEqualTo(2L); - VarBinArray names = chunk.column("name"); - assertThat(names.getString(0)).isEqualTo("Alice"); - assertThat(names.getString(1)).isEqualTo("Bob"); - } - } - } - - @Test - void usesCustomDelimiter(@TempDir Path tmp) throws Exception { - // Given - Path csv = tmp.resolve("data.csv"); - Files.writeString(csv, "id;name\n1;Alice\n2;Bob\n"); - Path vortex = tmp.resolve("data.vortex"); - - // When - CsvImporter.importCsv(csv, vortex, new ImportOptions(';', 65_536, true, null, null, WriteOptions.defaults())); - - // Then - try (VortexReader reader = VortexReader.open(vortex)) { - DType.Struct schema = (DType.Struct) reader.dtype(); - assertThat(schema.fieldNames()).containsExactly("id", "name"); - } - } - - @Test - void generatesHeadersWhenMissing(@TempDir Path tmp) throws Exception { - // Given - Path csv = tmp.resolve("data.csv"); - Files.writeString(csv, "1,Alice\n2,Bob\n"); - Path vortex = tmp.resolve("data.vortex"); - - // When - CsvImporter.importCsv(csv, vortex, new ImportOptions(',', 65_536, false, null, null, WriteOptions.defaults())); - - // Then - try (VortexReader reader = VortexReader.open(vortex)) { - DType.Struct schema = (DType.Struct) reader.dtype(); - assertThat(schema.fieldNames()).containsExactly("col0", "col1"); - } - } - - @Test - void respectsSchemaOverride(@TempDir Path tmp) throws Exception { - // Given - Path csv = tmp.resolve("data.csv"); - Files.writeString(csv, "value\n42\n99\n"); - Path vortex = tmp.resolve("data.vortex"); - DType.Struct forcedSchema = new DType.Struct( - java.util.List.of("value"), - java.util.List.of(new DType.Utf8(false)), - false); - - // When - CsvImporter.importCsv(csv, vortex, ImportOptions.defaults().withSchema(forcedSchema)); - - // Then - try (VortexReader reader = VortexReader.open(vortex)) { - DType.Struct schema = (DType.Struct) reader.dtype(); - assertThat(schema.fieldTypes().getFirst()).isEqualTo(new DType.Utf8(false)); - try (ScanIterator iter = reader.scan(ScanOptions.all())) { - assertThat(iter.hasNext()).isTrue(); - ScanResult chunk = iter.next(); - VarBinArray values = chunk.column("value"); - assertThat(values.getString(0)).isEqualTo("42"); - } - } - } -} diff --git a/dev/bench/data.js b/dev/bench/data.js new file mode 100644 index 0000000..0a3b7b1 --- /dev/null +++ b/dev/bench/data.js @@ -0,0 +1,242 @@ +window.BENCHMARK_DATA = { + "lastUpdate": 1780695328852, + "repoUrl": "https://github.com/dfa1/vortex-java", + "entries": { + "Benchmark": [ + { + "commit": { + "author": { + "name": "Davide Angelocola", + "username": "dfa1", + "email": "davide.angelocola@gmail.com" + }, + "committer": { + "name": "Davide Angelocola", + "username": "dfa1", + "email": "davide.angelocola@gmail.com" + }, + "id": "a16e919b909af4c29af8b7abcb2627e77ca87c54", + "message": "fix(ci): limit benchmark scope and iterations in CI\n\nPass JMH args through bench script. In CI, run only RustVsJava\nbenchmarks (skip 3GB BigFile setup) with -wi 1 -i 3 -f 1 to\nkeep runtime under 5 minutes.\n\nCo-Authored-By: Claude Sonnet 4.6 ", + "timestamp": "2026-06-04T05:07:35Z", + "url": "https://github.com/dfa1/vortex-java/commit/a16e919b909af4c29af8b7abcb2627e77ca87c54" + }, + "date": 1780551396785, + "tool": "jmh", + "benches": [ + { + "name": "io.github.dfa1.vortex.performance.RustVsJavaReadBenchmark.javaReadCascading", + "value": 37.07964506311847, + "unit": "ops/s", + "extra": "iterations: 3\nforks: 1\nthreads: 1" + }, + { + "name": "io.github.dfa1.vortex.performance.RustVsJavaReadBenchmark.javaReadClose", + "value": 23.62549601208856, + "unit": "ops/s", + "extra": "iterations: 3\nforks: 1\nthreads: 1" + }, + { + "name": "io.github.dfa1.vortex.performance.RustVsJavaReadBenchmark.javaReadSymbol", + "value": 10.773279199525296, + "unit": "ops/s", + "extra": "iterations: 3\nforks: 1\nthreads: 1" + }, + { + "name": "io.github.dfa1.vortex.performance.RustVsJavaReadBenchmark.javaReadVolume", + "value": 37.42012482178311, + "unit": "ops/s", + "extra": "iterations: 3\nforks: 1\nthreads: 1" + }, + { + "name": "io.github.dfa1.vortex.performance.RustVsJavaReadBenchmark.jniReadClose", + "value": 1.999072223630875, + "unit": "ops/s", + "extra": "iterations: 3\nforks: 1\nthreads: 1" + }, + { + "name": "io.github.dfa1.vortex.performance.RustVsJavaReadBenchmark.jniReadSymbol", + "value": 0.6645766567296644, + "unit": "ops/s", + "extra": "iterations: 3\nforks: 1\nthreads: 1" + }, + { + "name": "io.github.dfa1.vortex.performance.RustVsJavaReadBenchmark.jniReadVolume", + "value": 3.964346070290889, + "unit": "ops/s", + "extra": "iterations: 3\nforks: 1\nthreads: 1" + }, + { + "name": "io.github.dfa1.vortex.performance.RustVsJavaWriteBenchmark.javaWrite", + "value": 0.41637285115797257, + "unit": "ops/s", + "extra": "iterations: 3\nforks: 1\nthreads: 1" + }, + { + "name": "io.github.dfa1.vortex.performance.RustVsJavaWriteBenchmark.javaWriteCascading", + "value": 0.33923809206624284, + "unit": "ops/s", + "extra": "iterations: 3\nforks: 1\nthreads: 1" + }, + { + "name": "io.github.dfa1.vortex.performance.RustVsJavaWriteBenchmark.jniWrite", + "value": 0.013067648023707664, + "unit": "ops/s", + "extra": "iterations: 3\nforks: 1\nthreads: 1" + } + ] + }, + { + "commit": { + "author": { + "name": "Davide Angelocola", + "username": "dfa1", + "email": "davide.angelocola@gmail.com" + }, + "committer": { + "name": "Davide Angelocola", + "username": "dfa1", + "email": "davide.angelocola@gmail.com" + }, + "id": "4253ea5ba83e55270250a653ca42451b80dfdb87", + "message": "docs: remove completed JDK 26 build matrix TODO\n\nCo-Authored-By: Claude Sonnet 4.6 ", + "timestamp": "2026-06-04T21:30:30Z", + "url": "https://github.com/dfa1/vortex-java/commit/4253ea5ba83e55270250a653ca42451b80dfdb87" + }, + "date": 1780609267895, + "tool": "jmh", + "benches": [ + { + "name": "io.github.dfa1.vortex.performance.RustVsJavaReadBenchmark.javaReadCascading", + "value": 34.73412673017375, + "unit": "ops/s", + "extra": "iterations: 3\nforks: 1\nthreads: 1" + }, + { + "name": "io.github.dfa1.vortex.performance.RustVsJavaReadBenchmark.javaReadClose", + "value": 23.213366666917082, + "unit": "ops/s", + "extra": "iterations: 3\nforks: 1\nthreads: 1" + }, + { + "name": "io.github.dfa1.vortex.performance.RustVsJavaReadBenchmark.javaReadVolume", + "value": 34.739552051152124, + "unit": "ops/s", + "extra": "iterations: 3\nforks: 1\nthreads: 1" + }, + { + "name": "io.github.dfa1.vortex.performance.RustVsJavaReadBenchmark.jniReadClose", + "value": 1.826788113082755, + "unit": "ops/s", + "extra": "iterations: 3\nforks: 1\nthreads: 1" + }, + { + "name": "io.github.dfa1.vortex.performance.RustVsJavaReadBenchmark.jniReadSymbol", + "value": 0.6157357455992578, + "unit": "ops/s", + "extra": "iterations: 3\nforks: 1\nthreads: 1" + }, + { + "name": "io.github.dfa1.vortex.performance.RustVsJavaReadBenchmark.jniReadVolume", + "value": 3.5239271090875484, + "unit": "ops/s", + "extra": "iterations: 3\nforks: 1\nthreads: 1" + }, + { + "name": "io.github.dfa1.vortex.performance.RustVsJavaWriteBenchmark.javaWrite", + "value": 0.7345223414023522, + "unit": "ops/s", + "extra": "iterations: 3\nforks: 1\nthreads: 1" + }, + { + "name": "io.github.dfa1.vortex.performance.RustVsJavaWriteBenchmark.javaWriteCascading", + "value": 0.3751139782900637, + "unit": "ops/s", + "extra": "iterations: 3\nforks: 1\nthreads: 1" + }, + { + "name": "io.github.dfa1.vortex.performance.RustVsJavaWriteBenchmark.jniWrite", + "value": 0.012917204484283023, + "unit": "ops/s", + "extra": "iterations: 3\nforks: 1\nthreads: 1" + } + ] + }, + { + "commit": { + "author": { + "name": "Davide Angelocola", + "username": "dfa1", + "email": "davide.angelocola@gmail.com" + }, + "committer": { + "name": "Davide Angelocola", + "username": "dfa1", + "email": "davide.angelocola@gmail.com" + }, + "id": "801633b171a0c52e70cb285a1f967264d2108134", + "message": "[maven-release-plugin] prepare for next development iteration", + "timestamp": "2026-06-05T21:21:00Z", + "url": "https://github.com/dfa1/vortex-java/commit/801633b171a0c52e70cb285a1f967264d2108134" + }, + "date": 1780695328390, + "tool": "jmh", + "benches": [ + { + "name": "io.github.dfa1.vortex.performance.RustVsJavaReadBenchmark.javaReadCascading", + "value": 28.908963732082018, + "unit": "ops/s", + "extra": "iterations: 3\nforks: 1\nthreads: 1" + }, + { + "name": "io.github.dfa1.vortex.performance.RustVsJavaReadBenchmark.javaReadClose", + "value": 24.033752498112403, + "unit": "ops/s", + "extra": "iterations: 3\nforks: 1\nthreads: 1" + }, + { + "name": "io.github.dfa1.vortex.performance.RustVsJavaReadBenchmark.javaReadVolume", + "value": 35.60017254176457, + "unit": "ops/s", + "extra": "iterations: 3\nforks: 1\nthreads: 1" + }, + { + "name": "io.github.dfa1.vortex.performance.RustVsJavaReadBenchmark.jniReadClose", + "value": 1.8191862506274543, + "unit": "ops/s", + "extra": "iterations: 3\nforks: 1\nthreads: 1" + }, + { + "name": "io.github.dfa1.vortex.performance.RustVsJavaReadBenchmark.jniReadSymbol", + "value": 0.6149197709205404, + "unit": "ops/s", + "extra": "iterations: 3\nforks: 1\nthreads: 1" + }, + { + "name": "io.github.dfa1.vortex.performance.RustVsJavaReadBenchmark.jniReadVolume", + "value": 3.3779569748272884, + "unit": "ops/s", + "extra": "iterations: 3\nforks: 1\nthreads: 1" + }, + { + "name": "io.github.dfa1.vortex.performance.RustVsJavaWriteBenchmark.javaWrite", + "value": 0.7348825431386921, + "unit": "ops/s", + "extra": "iterations: 3\nforks: 1\nthreads: 1" + }, + { + "name": "io.github.dfa1.vortex.performance.RustVsJavaWriteBenchmark.javaWriteCascading", + "value": 0.257446797431972, + "unit": "ops/s", + "extra": "iterations: 3\nforks: 1\nthreads: 1" + }, + { + "name": "io.github.dfa1.vortex.performance.RustVsJavaWriteBenchmark.jniWrite", + "value": 0.012840988644632963, + "unit": "ops/s", + "extra": "iterations: 3\nforks: 1\nthreads: 1" + } + ] + } + ] + } +} \ No newline at end of file diff --git a/dev/bench/index.html b/dev/bench/index.html new file mode 100644 index 0000000..6c88780 --- /dev/null +++ b/dev/bench/index.html @@ -0,0 +1,281 @@ + + + + + + + Benchmarks + + + + +
+ + + + + + + diff --git a/docs/compatibility.md b/docs/compatibility.md deleted file mode 100644 index 3f0f279..0000000 --- a/docs/compatibility.md +++ /dev/null @@ -1,100 +0,0 @@ -# Compatibility - -Tested against the [Rust reference implementation](https://github.com/spiraldb/vortex) v0.72.0. -For the rest of the API surface (reader, writer, scan, CLI), see [reference.md](reference.md). - -## Encodings - -| Encoding ID | Class | Decode | Encode | Notes | -|------------------------------|------------------------------|--------|--------|-------| -| `vortex.primitive` | `PrimitiveEncoding` | ✅ | ✅ | All `PType` (I8–I64, U8–U64, F32, F64) | -| `vortex.bool` | `BoolEncoding` | ✅ | ✅ | Bool (bit-packed) | -| `vortex.null` | `NullEncoding` | ✅ | ✅ | Null | -| `vortex.bytebool` | `ByteBoolEncoding` | ✅ | ✅ | Bool (byte-per-element) | -| `vortex.zigzag` | `ZigZagEncoding` | ✅ | ✅ | Signed integer PTypes | -| `vortex.constant` | `ConstantEncoding` | ✅ | ✅ | Primitive, Utf8, Binary, Bool, Null, Decimal, Extension | -| `vortex.ext` | `ExtEncoding` | ✅ | ✅ | Extension | -| `vortex.runend` | `RunEndEncoding` | ✅ | ✅ | Primitive, Utf8/Binary, Bool | -| `vortex.varbin` | `VarBinEncoding` | ✅ | ✅ | Utf8, Binary | -| `vortex.varbinview` | `VarBinViewEncoding` | ✅ | ✅ | Utf8, Binary | -| `vortex.alp` | `AlpEncoding` | ✅ | ✅ | F64, F32 | -| `vortex.alprd` | `AlpRdEncoding` | ✅ | ✅ | F64, F32 | -| `vortex.dict` | `DictEncoding` | ✅ | ✅ | Primitive, Utf8/Binary | -| `vortex.sparse` | `SparseEncoding` | ✅ | ✅ | Primitive | -| `vortex.sequence` | `SequenceEncoding` | ✅ | ✅ | Primitive | -| `vortex.struct` | `StructEncoding` | ✅ | ✅ | Struct | -| `vortex.chunked` | `ChunkedEncoding` | ✅ | ✅ | Primitive + Struct concat | -| `vortex.fsst` | `FsstEncoding` | ✅ | ✅ | Utf8, Binary | -| `vortex.list` | `ListEncoding` | ✅ | ✅ | | -| `vortex.listview` | `ListViewEncoding` | ✅ | ✅ | | -| `vortex.fixed_size_list` | `FixedSizeListEncoding` | ✅ | ✅ | | -| `vortex.zstd` | `ZstdEncoding` | ✅ | ✅ | Primitive, Utf8, Binary | -| `vortex.masked` | `MaskedEncoding` | ✅ | ❌ | Encode not yet implemented | -| `vortex.decimal` | `DecimalEncoding` | ✅ | ✅ | | -| `vortex.decimal_byte_parts` | `DecimalBytePartsEncoding` | ✅ | ✅ | | -| `vortex.datetimeparts` | `DateTimePartsEncoding` | ✅ | ✅ | | -| `vortex.pco` | `PcoEncoding` | ✅ | ❌ | Decode: all modes; encode not yet implemented | -| `fastlanes.bitpacked` | `BitpackedEncoding` | ✅ | ✅ | Unsigned integer PTypes | -| `fastlanes.delta` | `DeltaEncoding` | ✅ | ✅ | Integer PTypes | -| `fastlanes.for` | `FrameOfReferenceEncoding` | ✅ | ✅ | Integer PTypes | -| `fastlanes.rle` | `RleEncoding` | ✅ | ✅ | Chunk-based RLE | -| `vortex.patched` | — | ❌ | ❌ | No decoder yet; reads as `UnknownArray` when `allowUnknown()` enabled | -| `vortex.variant` | — | ❌ | ❌ | No decoder yet; reads as `UnknownArray` when `allowUnknown()` enabled | - -### Unknown encodings - -Files containing unrecognised encoding IDs throw `VortexException` by default. Opt in to -passthrough mode to read such files without failing: - -```java -EncodingRegistry registry = EncodingRegistry.loadAll().allowUnknown(); -try (VortexReader vf = VortexReader.open(path, registry)) { - // columns with unknown encodings are returned as UnknownArray -} -``` - -## S3 Fixture Status (v0.72.0) - -Cross-language round-trips tested against Rust-written fixture files hosted at -`s3://vortex-compat-fixtures/v0.72.0/arrays/`. - -| Fixture | Status | -|-------------------------------------|--------| -| `primitives.vortex` | ✅ | -| `alp.vortex` | ✅ | -| `bitpacked.vortex` | ✅ | -| `booleans.vortex` | ✅ | -| `constant.vortex` | ✅ | -| `for.vortex` | ✅ | -| `fsst.vortex` | ✅ | -| `runend.vortex` | ✅ | -| `sequence.vortex` | ✅ | -| `varbin.vortex` | ✅ | -| `struct_nested.vortex` | ✅ | -| `null.vortex` | ✅ | -| `bytebool.vortex` | ✅ | -| `zigzag.vortex` | ✅ | -| `datetime.vortex` | ✅ | -| `dict.vortex` | ✅ | -| `sparse.vortex` | ✅ | -| `varbinview.vortex` | ✅ | -| `chunked.vortex` | ✅ | -| `rle.vortex` | ✅ | -| `alprd.vortex` | ✅ | -| `decimal.vortex` | ✅ | -| `decimal_byte_parts.vortex` | ✅ | -| `datetimeparts.vortex` | ✅ | -| `list.vortex` | ✅ | -| `listview.vortex` | ✅ | -| `fixed_size_list.vortex` | ✅ | -| `zstd.vortex` | ✅ | -| `tpch_lineitem.compact.vortex` | ✅ | -| `tpch_lineitem.regular.vortex` | ✅ | -| `tpch_orders.compact.vortex` | ✅ | -| `tpch_orders.regular.vortex` | ✅ | -| `pco.vortex` | ✅ | -| `clickbench_hits_5k.compact.vortex` | ✅ | -| `clickbench_hits_5k.regular.vortex` | ✅ | -| `masked.vortex` | ❓ | No fixture in v0.72.0 | -| `patched.vortex` | ❓ | No fixture in v0.72.0 | -| `variant.vortex` | ❓ | No fixture in v0.72.0 | diff --git a/docs/explanation.md b/docs/explanation.md deleted file mode 100644 index 886b617..0000000 --- a/docs/explanation.md +++ /dev/null @@ -1,198 +0,0 @@ -# Explanation - -Background reading on design decisions, architecture, and benchmarks. - -## Why pure Java instead of JNI - -The official Vortex ecosystem provides JVM bindings via JNI (bundled native `.so`/`.dylib`). -JNI bindings are fast but add deployment friction: platform-specific artifacts, native build -toolchains, and crash-domain coupling between the JVM and native code. The JAR for -vortex-jni 0.72 is **258MB**. - -This library takes a different approach — 100% Java, no JNI, no `sun.misc.Unsafe`. -It uses the Java FFM API (`MemorySegment` / `Arena`, Java 25+) for zero-copy memory-mapped -reads, making it easier to: - -- embed in any JVM project without native-library management -- build and test on any platform with a standard JDK -- debug and profile with standard JVM tooling - -The total JAR size is less than **1MB**. - -### Why Java 25+ - -The FFM API (`MemorySegment`, `Arena`) was finalized as a standard API in JDK 22 (JEP 454). -Java 25 is the first LTS release to ship FFM as stable — requiring it means no preview flags, -no upgrade risk, and a supported LTS for users. - -## Memory model - -`VortexReader` memory-maps the entire file into one `MemorySegment` (confined `Arena`). -All `Array` buffers returned during a scan are zero-copy slices of that segment — their -lifetime is tied to the `VortexReader`. Close the reader to release the mapped region. - -The iterator-based scan API is load-bearing: `iter.hasNext()` closes the previous chunk's -arena. Access all column data before calling `hasNext()` again. - -For the reader / scan method signatures, see [reference.md#reader-api](reference.md#reader-api). - -## Testing strategy - -Unit tests verify internal correctness (encoding round-trips, edge cases), but the format -has no formal specification — the Rust implementation is the ground truth. Unit tests alone -miss cross-language wire-format bugs: Java can round-trip a value internally while writing -bytes that another implementation cannot decode. - -The `integration` module addresses this by using the Rust JNI reader as a **test oracle**: -Java writes a file, the Rust reader decodes it, and the values are compared exactly. -Seeded random parameterized tests generate large, diverse inputs automatically, -covering edge cases no hand-written test would anticipate. - -This combination caught two real bugs in ALP floating-point encoding: -- Java selected exponents outside the range Rust's decoder accepts (silent data corruption) -- Java's encode round-trip check used a different floating-point associativity than Rust's - decode (`encoded * (F10[f] * IF10[e])` vs `(encoded * F10[f]) * IF10[e]`), passing values - that Rust decoded differently - -Both bugs were invisible to pure-Java tests and would have shipped undetected without the -cross-language oracle. - -## Architecture: fewer layers = faster - -``` - vortex-jni vortex-java - ────────────────────────────── ────────────────────────── - ┌──────────────────────────┐ ┌──────────────────────┐ - │ Java App │ │ Java App │ - │ (BigIntVector.get(i)) │ │ (buffer.getAtIndex) │ - └────────────┬─────────────┘ └──────────┬───────────┘ - │ Arrow Java API │ FFM API - ┌────────────▼─────────────┐ │ (MemorySegment, - │ Apache Arrow (Java) │ │ zero-copy slice) - │ VectorSchemaRoot, … │ │ - └────────────┬─────────────┘ ┌──────────▼───────────┐ - │ Arrow C Data Interface │ OS mmap region │ - │ + JNI boundary crossing │ (file on disk) │ - ┌────────────▼─────────────┐ └──────────────────────┘ - │ Native lib (.so/.dylib) │ - │ Rust decode │ - └────────────┬─────────────┘ - │ mmap / read - ┌────────────▼─────────────┐ - │ OS mmap region │ - │ (file on disk) │ - └──────────────────────────┘ - - 4 layers, 1 JNI crossing, 2 layers, 0 boundary crossings, - Arrow C Data Interface overhead no intermediate format -``` - -The JNI path pays three costs per batch: (1) a JNI boundary crossing to call into native -code, (2) the Arrow C Data Interface handshake to pass decoded buffers back to the JVM as -`ArrowArray`/`ArrowSchema` structs, and (3) materialising the result into Apache Arrow -`VectorSchemaRoot` objects before the application can read a single value. The JIT cannot -inline or optimise across the JNI boundary. - -`vortex-java` eliminates all of that. The FFM API (`MemorySegment`) gives Java code a -typed, bounds-checked view directly into the OS mmap region. Decoding reads bytes directly -from that view with no copies, no intermediate Arrow format, and no boundary crossings. -The JIT sees the full decode path as ordinary Java bytecode. - -## Benchmarks - -JMH throughput (ops/s = full-file scans per second). Higher is better. - -**Environment:** Apple M5, OpenJDK 25, 3 warmup × 3 s, 5 measurement × 5 s, fork 1. - -### OHLC read — 10 M rows, 58.9 MB (Rust-written file, single-column projection) - -| Benchmark | Java (ops/s) | JNI/Rust (ops/s) | Java speedup | -|-----------------|--------------|------------------|--------------| -| close (F64/ALP) | 76.7 ± 0.3 | 50.4 ± 2.8 | **1.5×** | -| volume (I64) | 127.9 ± 2.3 | 52.9 ± 0.6 | **2.4×** | -| symbol (varbin) | 110.4 ± 0.4 | 9.6 ± 0.9 | **11.5×** | - -### OHLC write — 10 M rows - -| Benchmark | Java (ops/s) | JNI/Rust (ops/s) | Java speedup | -|-----------|--------------|------------------|--------------| -| write | 4.4 ± 1.1 | 0.7 ± 0.1 | **6.4×** | - -The Java write is faster but also produces bigger files (more optimization work remains). - -### Big-file scan — 100 M rows × 4 I64 columns, ~3 GB (Rust-written file, all columns) - -| Benchmark | Java (ops/s) | JNI/Rust (ops/s) | Java speedup | -|-----------|--------------|------------------|--------------| -| scan | 20.4 ± 0.9 | 5.7 ± 0.6 | **3.6×** | - -### Parquet vs Vortex read — NYC Yellow Taxi 2024-01, 3 M rows, 19 columns - -Both formats store all 19 columns; projection happens at read time. Both sides scalar decode -(Hardwood disables SIMD on JDK 25; Vortex Java uses FFM scalar reads throughout). -File sizes: Parquet 47.6 MB, Vortex Java 50 MB. - -**Environment:** Apple M5, OpenJDK 25, 5 warmup × 3 s, 5 measurement × 5 s, fork 2. - -Two Parquet variants are measured to isolate format cost from API overhead: -- **batch**: `ColumnReader.nextBatch()` + loop over `getDoubles()`/`getInts()` arrays — apples-to-apples with Vortex's batch fold -- **row-by-row**: `RowReader.next()` + `getDouble("col")` per row — measures the full row-cursor overhead on top of format decode - -| Benchmark | ops/s | vs Parquet batch | -|---|---|---| -| `parquetRead` — batch, 1 col (`trip_distance`) | 166.5 ± 4.0 | baseline | -| `parquetReadRowByRow` — row cursor, 1 col | 67.6 ± 4.4 | 0.41× (2.5× API penalty) | -| `vortexRead` — 1 col (`trip_distance`) | 235.1 ± 6.9 | **1.41×** | -| `parquetReadMultiColumn` — batch, 2 cols (`fare_amount`, `PULocationID`) | 133.0 ± 18.3 | baseline | -| `parquetReadMultiColumnRowByRow` — row cursor, 2 cols | 44.0 ± 2.2 | 0.33× (3× API penalty) | -| `vortexReadMultiColumn` — 2 cols | 122.6 ± 3.3 | 0.92× | - -Single-column: Vortex 1.4× faster than Parquet batch — format advantage is real (mmap -zero-copy + ALP vs Parquet RLE/ZSTD page decode). - -Multi-column: Vortex (122.6) slightly behind Parquet batch (133.0). Known gap: Rust uses a -global dict per column (one tiny dict for all 3 M rows); Java applies dict per 131 K-row -chunk, increasing per-chunk overhead for low-cardinality columns like `PULocationID` (260 -unique values). - -#### Why Vortex is faster on single-column reads - -**1. mmap zero-copy.** -Vortex reads directly from the mmap'd `MemorySegment` — the file bytes _are_ the decode -input, no intermediate copies. Hardwood reads into internal page buffers and materialises -values before batch hand-off. Parquet also pays per-page framing overhead: RLE-encoded -definition/repetition levels, page header parsing, optional dictionary decode. Vortex's -layout is a flat array of encoded values with no per-row framing. - -**2. Typed scatter instead of per-element copy.** -`DictEncoding` expansion uses `getAtIndex`/`setAtIndex` with loop-unswitched elemSize — -a single typed load + store per row. The prior `MemorySegment.copy(8 bytes)` per element -dominated 60% of JFR execution samples on multi-column scans before it was fixed. - -``` -Hardwood parquetRead (per 3 M rows) Vortex vortexRead (per 3 M rows) -──────────────────────────────────── ────────────────────────────────── -47.6 MB on disk 50 MB on disk -+ page header parse × N pages + ALP decode (branch-free ×/+) -+ definition-level RLE decode × 3 M rows + fold() tight loop, no dispatch -``` - -#### Why ZstdEncoding is excluded from the numeric cascade - -Adding `ZstdEncoding` to `CASCADE_CODECS` improves file size (50 MB → 43 MB) because -Zstd out-compresses ALP on some F64 columns. But ZSTD decompression is an order of -magnitude slower than ALP reconstruction or bitpack unpack: single-column read throughput -collapses from 235 to 40 ops/s (6×), falling below Parquet batch (166.5 ops/s). - -The smaller file is not worth the read regression. `ZstdEncoding` is retained in the -codec registry for `Utf8`/`Binary` columns where no faster structural alternative exists, -but it is not a candidate in the numeric cascade. - -## Design principles - -- Zero-copy everywhere via FFM `MemorySegment` -- No JNI, no `sun.misc.Unsafe` ([FFM vs Unsafe](https://inside.java/2025/06/12/ffm-vs-unsafe/)) -- Align with vortex-rust and vortex-go semantics -- Make the JIT happy: constant layouts, predictable strides, no virtual dispatch in hot loops -- Rigorous testing: unit + property-based + cross-language integration -- Tracking [JEP 469](https://openjdk.org/jeps/469) (Vector API) for future SIMD paths diff --git a/docs/how-to.md b/docs/how-to.md deleted file mode 100644 index 6b08d84..0000000 --- a/docs/how-to.md +++ /dev/null @@ -1,243 +0,0 @@ -# How-to guides - -Task-oriented recipes. Each section solves one concrete goal. -For API details (classes, methods, operator tables), see [reference.md](reference.md). -For the design rationale behind the iterator lifecycle, see [explanation.md#memory-model](explanation.md#memory-model). - ---- - -## Build the CLI - -Build the fat jar once; reuse it for every CLI recipe below: - -```bash -./mvnw package -pl cli -am -DskipTests -java -jar cli/target/vortex.jar [args] -``` - -For the full subcommand list, see [reference.md#cli](reference.md#cli). - ---- - -## Count rows - -**API:** - -```java -long total = 0; -try (VortexReader vf = VortexReader.open(Path.of("data.vortex")); - var iter = vf.scan(ScanOptions.all())) { - while (iter.hasNext()) { - total += iter.next().rowCount(); - } -} -System.out.println(total); -``` - -**CLI:** - -```bash -java -jar cli/target/vortex.jar count data.vortex -``` - ---- - -## Inspect file structure - -**API:** - -```java -try (VortexReader vf = VortexReader.open(Path.of("data.vortex"))) { - System.out.println(vf.dtype()); // column names and types - System.out.println(vf.layout()); // layout tree (Struct → Chunked → Flat …) -} -``` - -**CLI:** - -```bash -# column names and types -java -jar cli/target/vortex.jar schema data.vortex - -# full layout tree with encoding IDs, row counts, buffer sizes -java -jar cli/target/vortex.jar inspect data.vortex - -# per-column min/max statistics -java -jar cli/target/vortex.jar stats data.vortex -``` - ---- - -## Project columns - -**API:** - -```java -ScanOptions opts = ScanOptions.all().withColumns("symbol", "price"); - -try (VortexReader vf = VortexReader.open(Path.of("trades.vortex")); - var iter = vf.scan(opts)) { - while (iter.hasNext()) { - var chunk = iter.next(); - // chunk.columns() contains only "symbol" and "price" - } -} -``` - -**CLI:** - -```bash -java -jar cli/target/vortex.jar select trades.vortex symbol price -``` - ---- - -## Filter rows - -**API:** - -```java -RowFilter filter = new RowFilter.Gte("volume", 1_000_000); -ScanOptions opts = ScanOptions.all().withFilter(filter); - -try (VortexReader vf = VortexReader.open(Path.of("trades.vortex")); - var iter = vf.scan(opts)) { - while (iter.hasNext()) { - var chunk = iter.next(); - // only rows where volume >= 1_000_000 - } -} -``` - -Combine filters with `and()`: - -```java -RowFilter filter = new RowFilter.Gte("volume", 1_000_000) - .and(new RowFilter.Lte("price", 200.0)); -``` - -For the supported predicate set and CLI operator syntax, see -[reference.md#rowfilter](reference.md#rowfilter-iogithubdfa1vortexscanrowfilter) -and [reference.md#filter-expression-syntax](reference.md#filter-expression-syntax). - -**CLI:** - -```bash -java -jar cli/target/vortex.jar filter trades.vortex "volume >= 1000000" -``` - ---- - -## Preview the first N rows - -**API:** - -```java -ScanOptions opts = ScanOptions.all().withLimit(10); - -try (VortexReader vf = VortexReader.open(Path.of("data.vortex")); - var iter = vf.scan(opts)) { - while (iter.hasNext()) { - var chunk = iter.next(); - // at most 10 rows total across all chunks - } -} -``` - -**CLI:** - -```bash -# export first 10 rows to CSV -java -jar cli/target/vortex.jar export data.vortex | head -n 11 # 1 header + 10 rows -``` - ---- - -## Convert Parquet to Vortex - -**API:** - -```java -import io.github.dfa1.vortex.parquet.ParquetImporter; - -ParquetImporter.importParquet( - Path.of("data.parquet"), - Path.of("data.vortex") -); -``` - -Project specific columns during conversion: - -```java -import io.github.dfa1.vortex.parquet.ImportOptions; - -ImportOptions opts = ImportOptions.defaults() - .withColumns(List.of("trip_distance", "fare_amount")); - -ParquetImporter.importParquet(Path.of("data.parquet"), Path.of("data.vortex"), opts); -``` - -**CLI:** - -```bash -# output defaults to .vortex -java -jar cli/target/vortex.jar import data.parquet - -# explicit output path -java -jar cli/target/vortex.jar import data.parquet out.vortex -``` - ---- - -## Convert CSV to Vortex - -**CLI only** (CSV has no schema — types are inferred): - -```bash -java -jar cli/target/vortex.jar import data.csv -# writes data.vortex, prints size savings -``` - ---- - -## Export to CSV - -**CLI:** - -```bash -# all columns -java -jar cli/target/vortex.jar export data.vortex > out.csv - -# specific columns -java -jar cli/target/vortex.jar select data.vortex col1 col2 > out.csv - -# filtered rows -java -jar cli/target/vortex.jar filter data.vortex "price >= 100" > out.csv -``` - ---- - -## Read files with unknown encodings - -By default, a file containing an unrecognised encoding ID throws `VortexException`. -Use `allowUnknown()` to read the file anyway — columns with unknown encodings are -returned as `UnknownArray` (opaque, not decodable, but the rest of the file is readable): - -```java -import io.github.dfa1.vortex.encoding.EncodingRegistry; -import io.github.dfa1.vortex.core.array.UnknownArray; - -EncodingRegistry registry = EncodingRegistry.loadAll().allowUnknown(); - -try (VortexReader vf = VortexReader.open(Path.of("future.vortex"), registry); - var iter = vf.scan(ScanOptions.all())) { - while (iter.hasNext()) { - var chunk = iter.next(); - chunk.columns().forEach((name, arr) -> { - if (arr instanceof UnknownArray u) { - System.out.println(name + ": unknown encoding " + u.encodingId()); - } - }); - } -} -``` diff --git a/docs/reference.md b/docs/reference.md deleted file mode 100644 index 0184a16..0000000 --- a/docs/reference.md +++ /dev/null @@ -1,244 +0,0 @@ -# Reference - -API surface, CLI commands, and operator tables. Look here for "what exists and what it accepts." -For task-oriented usage see [how-to.md](how-to.md); for design rationale see [explanation.md](explanation.md). - -- [Core types](#core-types) -- [Reader API](#reader-api) -- [Writer API](#writer-api) -- [Scan API](#scan-api) -- [Encoding registry](#encoding-registry) -- [Parquet / CSV import](#parquet--csv-import) -- [CLI](#cli) -- [Encoding compatibility](compatibility.md) - ---- - -## Core types - -### `PType` (`io.github.dfa1.vortex.core.PType`) - -Physical primitive type — wire-level numeric kind for a column. - -| Constant | Bytes | Notes | -|---|---|---| -| `U8`, `U16`, `U32`, `U64` | 1 / 2 / 4 / 8 | Unsigned integers | -| `I8`, `I16`, `I32`, `I64` | 1 / 2 / 4 / 8 | Signed integers | -| `F16` | 2 | IEEE 754 half — decode not yet supported | -| `F32`, `F64` | 4 / 8 | IEEE 754 single / double | - -Methods: `byteSize()`, `isFloating()`, `isSigned()`. - -### `DType` (`io.github.dfa1.vortex.core.DType`) - -Sealed logical type. All variants take a trailing `boolean nullable`. - -| Record | Constructor | -|---|---| -| `DType.Null` | `new DType.Null(nullable)` | -| `DType.Bool` | `new DType.Bool(nullable)` | -| `DType.Primitive` | `new DType.Primitive(PType, nullable)` | -| `DType.Decimal` | `new DType.Decimal(precision, scale, nullable)` | -| `DType.Utf8` | `new DType.Utf8(nullable)` | -| `DType.Binary` | `new DType.Binary(nullable)` | -| `DType.Struct` | `new DType.Struct(fieldNames, fieldTypes, nullable)` | -| `DType.List` | `new DType.List(elementType, nullable)` | -| `DType.FixedSizeList` | `new DType.FixedSizeList(elementType, fixedSize, nullable)` | -| `DType.Extension` | `new DType.Extension(id, storageDType, metadata, nullable)` | - -Helpers: `nullable()`, `withNullable(boolean)`, `DType.Struct.field(name)`. - ---- - -## Reader API - -### `VortexReader` (`io.github.dfa1.vortex.io.VortexReader`) - -Memory-mapped handle to a Vortex file. Implements `AutoCloseable`. Closing releases the mmap region; -all `Array` buffers obtained during scans become invalid. - -| Method | Returns | Notes | -|---|---|---| -| `static open(Path)` | `VortexReader` | Uses `EncodingRegistry.loadAll()` | -| `static open(Path, EncodingRegistry)` | `VortexReader` | Custom registry (e.g. `allowUnknown()`) | -| `dtype()` | `DType` | Schema (typically `DType.Struct`) | -| `layout()` | `Layout` | Layout tree (Struct → Zoned → Chunked → Flat) | -| `footer()` | `Footer` | Segment specs, encoding specs | -| `version()` | `int` | File format version | -| `fileSize()` | `long` | File size in bytes | -| `registry()` | `EncodingRegistry` | Registry in use | -| `scan(ScanOptions)` | `ScanIterator` | Open a scan | -| `columnStats()` | `Map` | Aggregated min/max per column | -| `slice(offset, length)` | `MemorySegment` | Zero-copy slice of mmap region | -| `close()` | — | Releases mmap | - ---- - -## Writer API - -### `VortexWriter` (`io.github.dfa1.vortex.writer.VortexWriter`) - -Writes a Vortex file. Implements `Closeable`. The file is complete and readable as soon as `close()` returns. - -| Method | Notes | -|---|---| -| `static create(WritableByteChannel, DType.Struct, WriteOptions)` | Default codec set | -| `static create(WritableByteChannel, DType.Struct, WriteOptions, List)` | Custom codec set | -| `writeChunk(Map)` | One batch of rows; each value = `long[]`, `double[]`, `String[]`, `boolean[]`, etc., matching the column `DType` | -| `close()` | Finalizes file (footer, postscript, trailer) | - -### `WriteOptions` (`io.github.dfa1.vortex.writer.WriteOptions`) - -Record: `(int chunkSize, boolean enableZoneMaps, double compressionRatioThreshold, int allowedCascading)`. - -| Factory | Defaults | -|---|---| -| `WriteOptions.defaults()` | `chunkSize=65_536`, `enableZoneMaps=true`, `compressionRatioThreshold=0.90`, `allowedCascading=0` | -| `WriteOptions.cascading(depth)` | Same defaults, `allowedCascading=depth` | - ---- - -## Scan API - -### `ScanOptions` (`io.github.dfa1.vortex.scan.ScanOptions`) - -Record: `(List columns, RowFilter rowFilter, long limit)`. Empty `columns` = read all. `NO_LIMIT` = `Long.MAX_VALUE`. - -| Factory / builder | Effect | -|---|---| -| `ScanOptions.all()` | All columns, no filter, no limit | -| `ScanOptions.columns(String... names)` | Project columns | -| `ScanOptions.limit(long n)` | Limit rows | -| `.withColumns(String... names)` | Project columns (builder) | -| `.withFilter(RowFilter)` | Add zone-map filter | -| `.withLimit(long n)` | Cap rows | -| `.hasProjection()` / `.hasFilter()` / `.hasLimit()` | Predicates | - -### `RowFilter` (`io.github.dfa1.vortex.scan.RowFilter`) - -Sealed predicate used for zone-map pruning (per-chunk min/max). Chunks that cannot match are skipped entirely. - -| Record | Static factory | Builder | -|---|---|---| -| `RowFilter.Gt(column, value)` | `RowFilter.gt(col, val)` | — | -| `RowFilter.Gte(column, value)` | `RowFilter.gte(col, val)` | — | -| `RowFilter.Lt(column, value)` | `RowFilter.lt(col, val)` | — | -| `RowFilter.Lte(column, value)` | `RowFilter.lte(col, val)` | — | -| `RowFilter.Eq(column, value)` | `RowFilter.eq(col, val)` | — | -| `RowFilter.Neq(column, value)` | `RowFilter.neq(col, val)` | — | -| `RowFilter.And(filters)` | `RowFilter.and(f1, f2, …)` | `f1.and(f2)` | - -### `ScanIterator` (`io.github.dfa1.vortex.scan.ScanIterator`) - -Implements `AutoCloseable`. Drives one scan. - -| Method | Notes | -|---|---| -| `hasNext()` | **Closes the previous chunk's arena.** Access all column data first. | -| `next()` | Returns `ScanResult` | -| `close()` | Releases iterator state | - -### `ScanResult` (`io.github.dfa1.vortex.scan.ScanResult`) - -Record: `(long rowCount, Map columns)`. - -| Method | Notes | -|---|---| -| `rowCount()` | Rows in this chunk | -| `columns()` | All columns in this chunk | -| ` column(String name)` | Typed column lookup; throws `VortexException` if unknown | - ---- - -## Encoding registry - -### `EncodingRegistry` (`io.github.dfa1.vortex.encoding`) - -| Method | Notes | -|---|---| -| `static loadAll()` | Loads every `Encoding` via `ServiceLoader` | -| `static empty()` | Empty registry | -| `register(Encoding)` | Add a custom encoding; throws if already registered | -| `hasEncoding(EncodingId)` | Lookup | -| `allowUnknown()` | Switch to passthrough mode — unknown nodes (and their children) decode as `UnknownArray` | -| `isAllowUnknown()` | Predicate | - -To register a custom encoding via `ServiceLoader`, add the fully qualified class name to -`META-INF/services/io.github.dfa1.vortex.encoding.Encoding`. - ---- - -## Parquet / CSV import - -### `ParquetImporter` (`io.github.dfa1.vortex.parquet.ParquetImporter`) - -| Method | Notes | -|---|---| -| `importParquet(Path in, Path out)` | Defaults | -| `importParquet(Path in, Path out, ImportOptions)` | Tuned | - -### `ImportOptions` (`io.github.dfa1.vortex.parquet.ImportOptions`) - -Record: `(int chunkSize, List columns, ProgressListener progressListener, WriteOptions writeOptions)`. - -| Factory / builder | Notes | -|---|---| -| `ImportOptions.defaults()` | `chunkSize=65_536`, no projection, `WriteOptions.cascading(3)` | -| `.withColumns(List)` | Project columns during import | -| `.withProgressListener(listener)` | Progress callbacks | -| `.withWriteOptions(WriteOptions)` | Override write options | -| `.withChunkSize(int)` | Override chunk size | - -CSV import is CLI-only — types are inferred from the data. - ---- - -## CLI - -The `cli` module ships a fat jar with subcommands for inspecting and querying Vortex files. - -```bash -./mvnw package -pl cli -am -DskipTests -java -jar cli/target/vortex.jar [args] -``` - -| Subcommand | Syntax | Description | -|---|---|---| -| `inspect` | `inspect ` | Layout tree, encodings, row counts, buffer sizes | -| `schema` | `schema ` | Column names and types | -| `count` | `count ` | Total row count | -| `stats` | `stats ` | Per-column min/max | -| `export` | `export ` | All columns to CSV on stdout | -| `select` | `select [col2 ...]` | Project columns to CSV | -| `filter` | `filter ""` | Filter rows to CSV | -| `import` | `import [out.vortex]` | Convert CSV or Parquet to Vortex | - -### `filter` expression syntax - -``` - -``` - -| Operator | Meaning | -|---|---| -| `>`, `>=` | Greater than, greater-or-equal | -| `<`, `<=` | Less than, less-or-equal | -| `=`, `==` | Equal | -| `!=` | Not equal | - -Values are parsed as integer, double, boolean, or string (in that order). - ---- - -## File format trailer - -8 bytes at EOF: - -``` -version (u16 LE) | postscriptLen (u16 LE) | magic ("VTXF") -``` - -The postscript is a FlatBuffer blob immediately before the trailer. It points (offset + length) to: -the Footer (FlatBuffer), the DType (Protobuf), and the Layout (FlatBuffer) — each stored elsewhere in the file. - -See [explanation.md#memory-model](explanation.md#memory-model) for the mmap lifecycle. diff --git a/docs/tutorial.md b/docs/tutorial.md deleted file mode 100644 index 5222073..0000000 --- a/docs/tutorial.md +++ /dev/null @@ -1,170 +0,0 @@ -# Tutorial: Your first Vortex file - -This tutorial walks you through writing and reading a Vortex file from scratch. -You will end up with a working Maven project that stores time-series data in Vortex format -and reads it back column by column. - -**Prerequisites:** Java 25+, Maven 3.9+. - ---- - -## 1. Create a Maven project - -```bash -mvn archetype:generate \ - -DgroupId=com.example \ - -DartifactId=vortex-demo \ - -DarchetypeArtifactId=maven-archetype-quickstart \ - -DarchetypeVersion=1.5 \ - -DinteractiveMode=false -cd vortex-demo -``` - -Add the dependency to `pom.xml` (inside ``): - -```xml - - io.github.dfa1 - vortex-java - 0.1.0-SNAPSHOT - -``` - -Set the compiler to Java 25: - -```xml - - 25 - -``` - ---- - -## 2. Define a schema - -A Vortex file is a typed struct — every column has a declared type before any data is written. - -```java -import io.github.dfa1.vortex.core.DType; -import io.github.dfa1.vortex.core.PType; -import java.util.List; - -DType.Struct schema = new DType.Struct( - List.of("timestamp", "symbol", "price", "volume"), - List.of( - new DType.Primitive(PType.I64, false), // unix epoch millis, non-nullable - new DType.Utf8(false), // ticker symbol - new DType.Primitive(PType.F64, false), // trade price - new DType.Primitive(PType.I64, false) // shares traded - ), - false // the struct itself is non-nullable -); -``` - -Passing `true` as the trailing argument makes the column nullable. -See [reference.md#core-types](reference.md#core-types) for the full `DType` / `PType` list. - ---- - -## 3. Write data - -```java -import io.github.dfa1.vortex.writer.VortexWriter; -import io.github.dfa1.vortex.writer.WriteOptions; -import java.nio.channels.FileChannel; -import java.nio.file.Path; -import java.util.Map; -import static java.nio.file.StandardOpenOption.*; - -Path outPath = Path.of("trades.vortex"); - -try (FileChannel ch = FileChannel.open(outPath, CREATE, WRITE, TRUNCATE_EXISTING); - VortexWriter writer = VortexWriter.create(ch, schema, WriteOptions.defaults())) { - - writer.writeChunk(Map.of( - "timestamp", new long[] {1_700_000_000_000L, 1_700_000_001_000L, 1_700_000_002_000L}, - "symbol", new String[] {"AAPL", "AAPL", "MSFT"}, - "price", new double[] {189.95, 190.10, 374.20}, - "volume", new long[] {100L, 250L, 175L} - )); -} -``` - -`writeChunk` takes one batch of rows. Call it multiple times to write multiple chunks — -each chunk is compressed independently and can be skipped during a scan if zone-map -statistics rule it out. - -The file is complete and readable as soon as `VortexWriter` is closed. - ---- - -## 4. Read it back - -```java -import io.github.dfa1.vortex.io.VortexReader; -import io.github.dfa1.vortex.scan.ScanOptions; -import io.github.dfa1.vortex.core.array.DoubleArray; -import io.github.dfa1.vortex.core.array.LongArray; - -try (VortexReader vf = VortexReader.open(outPath); - var iter = vf.scan(ScanOptions.all())) { - - while (iter.hasNext()) { - var chunk = iter.next(); // advances to the next batch - - LongArray ts = chunk.column("timestamp"); - DoubleArray price = chunk.column("price"); - - for (long i = 0; i < chunk.rowCount(); i++) { - System.out.printf("%d %.2f%n", ts.getLong(i), price.getDouble(i)); - } - // ⚠ do not store references past this point — - // iter.hasNext() frees the chunk's memory - } -} -``` - -**Important:** every chunk lives in an off-heap `Arena`. Calling `iter.hasNext()` closes -that arena and releases the memory. Read all values before advancing the iterator. -See [explanation.md#memory-model](explanation.md#memory-model) for why the lifetime works this way. - -Expected output: - -``` -1700000000000 189.95 -1700000001000 190.10 -1700000002000 374.20 -``` - ---- - -## 5. Project columns and limit rows - -Reading every column is wasteful when you only need two. Use `withColumns` to project, and -`withLimit` to stop after `n` rows: - -```java -ScanOptions opts = ScanOptions.all() - .withColumns("symbol", "price") - .withLimit(2); - -try (VortexReader vf = VortexReader.open(outPath); - var iter = vf.scan(opts)) { - - while (iter.hasNext()) { - var chunk = iter.next(); - // chunk only contains "symbol" and "price" - } -} -``` - ---- - -## What's next - -You now have a working write-then-read flow. From here: - -- [how-to.md](how-to.md) — task recipes: filter rows, project columns, convert Parquet, use the CLI -- [reference.md](reference.md) — API surface, CLI subcommands, operator tables -- [compatibility.md](compatibility.md) — which encodings are supported -- [explanation.md](explanation.md) — memory model, testing strategy, benchmarks diff --git a/integration/pom.xml b/integration/pom.xml deleted file mode 100644 index 5d81d39..0000000 --- a/integration/pom.xml +++ /dev/null @@ -1,109 +0,0 @@ - - - 4.0.0 - - io.github.dfa1.vortex - vortex-java - 0.3.0-SNAPSHOT - - - integration - - - true - - - - - io.github.dfa1.vortex - writer - test - - - io.github.dfa1.vortex - parquet - test - - - dev.hardwood - hardwood-core - test - - - io.github.dfa1.vortex - reader - test - - - dev.vortex - vortex-jni - test - - - org.apache.arrow - arrow-vector - test - - - org.apache.arrow - arrow-c-data - test - - - org.apache.arrow - arrow-memory-unsafe - test - - - org.junit.jupiter - junit-jupiter - test - - - org.assertj - assertj-core - test - - - org.slf4j - slf4j-simple - test - - - - - - - org.apache.maven.plugins - maven-surefire-plugin - - - **/*IntegrationTest.java - - - - - org.apache.maven.plugins - maven-failsafe-plugin - - - **/*IntegrationTest.java - - - --add-opens java.base/java.nio=ALL-UNNAMED - --enable-native-access=ALL-UNNAMED - --sun-misc-unsafe-memory-access=allow - - - - - - integration-test - verify - - - - - - - diff --git a/integration/src/test/java/io/github/dfa1/vortex/integration/AllowUnknownIntegrationTest.java b/integration/src/test/java/io/github/dfa1/vortex/integration/AllowUnknownIntegrationTest.java deleted file mode 100644 index 06626b2..0000000 --- a/integration/src/test/java/io/github/dfa1/vortex/integration/AllowUnknownIntegrationTest.java +++ /dev/null @@ -1,145 +0,0 @@ -package io.github.dfa1.vortex.integration; - -import dev.vortex.api.Session; -import dev.vortex.api.VortexWriter; -import dev.vortex.arrow.ArrowAllocation; -import dev.vortex.jni.NativeLoader; -import io.github.dfa1.vortex.core.VortexException; -import io.github.dfa1.vortex.core.array.Array; -import io.github.dfa1.vortex.core.array.UnknownArray; -import io.github.dfa1.vortex.encoding.EncodingRegistry; -import io.github.dfa1.vortex.io.VortexReader; -import io.github.dfa1.vortex.scan.ScanResult; -import org.apache.arrow.c.ArrowArray; -import org.apache.arrow.c.ArrowSchema; -import org.apache.arrow.c.Data; -import org.apache.arrow.memory.BufferAllocator; -import org.apache.arrow.vector.BigIntVector; -import org.apache.arrow.vector.Float8Vector; -import org.apache.arrow.vector.VectorSchemaRoot; -import org.apache.arrow.vector.types.FloatingPointPrecision; -import org.apache.arrow.vector.types.pojo.ArrowType; -import org.apache.arrow.vector.types.pojo.Field; -import org.apache.arrow.vector.types.pojo.Schema; -import org.junit.jupiter.api.Test; -import org.junit.jupiter.api.io.TempDir; - -import java.io.IOException; -import java.nio.file.Path; -import java.util.ArrayList; -import java.util.HashMap; -import java.util.List; -import java.util.Random; - -import static org.assertj.core.api.Assertions.assertThat; -import static org.assertj.core.api.Assertions.assertThatThrownBy; - -/// Verifies EncodingRegistry.allowUnknown() passthrough behaviour end-to-end -/// using a JNI-written file whose encodings are deliberately absent from the registry. -class AllowUnknownIntegrationTest { - - private static final Session SESSION = Session.create(); - private static final BufferAllocator ALLOCATOR = ArrowAllocation.rootAllocator(); - private static final Schema SCHEMA = new Schema(List.of( - Field.notNullable("id", new ArrowType.Int(64, true)), - Field.notNullable("value", new ArrowType.FloatingPoint(FloatingPointPrecision.DOUBLE)) - )); - - static { - NativeLoader.loadJni(); - } - - @Test - void allowUnknown_emptyRegistry_allColumnsReturnUnknownArray(@TempDir Path tmp) throws IOException { - // Given — JNI writes a real file; empty registry has no decoders - Path file = tmp.resolve("test.vtx"); - writeJni(file, 10_000); - - // When - List results; - try (VortexReader vf = VortexReader.open(file, EncodingRegistry.empty().allowUnknown())) { - results = scanAll(vf); - } - - // Then — every column value is UnknownArray; file was readable without throwing - assertThat(results).isNotEmpty(); - long totalRows = results.stream().mapToLong(ScanResult::rowCount).sum(); - assertThat(totalRows).isEqualTo(10_000); - for (ScanResult r : results) { - for (Array col : r.columns().values()) { - assertThat(col) - .describedAs("column should be UnknownArray when no decoder registered") - .isInstanceOf(UnknownArray.class); - } - } - } - - @Test - void strictMode_emptyRegistry_throwsVortexException(@TempDir Path tmp) throws IOException { - // Given - Path file = tmp.resolve("test.vtx"); - writeJni(file, 1_000); - - // When / Then — strict mode throws rather than returning UnknownArray - assertThatThrownBy(() -> { - try (VortexReader vf = VortexReader.open(file, EncodingRegistry.empty())) { - scanAll(vf); - } - }).isInstanceOf(VortexException.class); - } - - @Test - void allowUnknown_loadAllRegistry_noUnknownArrayForSupportedEncodings(@TempDir Path tmp) throws IOException { - // Given — loadAll() has decoders for everything the JNI writer produces - Path file = tmp.resolve("test.vtx"); - writeJni(file, 10_000); - - // When - List results; - try (VortexReader vf = VortexReader.open(file, EncodingRegistry.loadAll().allowUnknown())) { - results = scanAll(vf); - } - - // Then — allowUnknown() is a no-op when all encodings are registered; no UnknownArray - assertThat(results).isNotEmpty(); - for (ScanResult r : results) { - for (Array col : r.columns().values()) { - assertThat(col) - .describedAs("fully-supported file should not produce UnknownArray") - .isNotInstanceOf(UnknownArray.class); - } - } - } - - private static void writeJni(Path file, int rows) throws IOException { - String uri = file.toAbsolutePath().toUri().toString(); - var rng = new Random(42L); - try (VortexWriter writer = VortexWriter.create(SESSION, uri, SCHEMA, new HashMap<>(), ALLOCATOR)) { - try (VectorSchemaRoot root = VectorSchemaRoot.create(SCHEMA, ALLOCATOR)) { - BigIntVector idVec = (BigIntVector) root.getVector("id"); - Float8Vector valVec = (Float8Vector) root.getVector("value"); - idVec.allocateNew(rows); - valVec.allocateNew(rows); - for (int i = 0; i < rows; i++) { - idVec.setSafe(i, i); - valVec.setSafe(i, rng.nextDouble() * 1000.0); - } - root.setRowCount(rows); - try (ArrowArray arr = ArrowArray.allocateNew(ALLOCATOR); - ArrowSchema schema = ArrowSchema.allocateNew(ALLOCATOR)) { - Data.exportVectorSchemaRoot(ALLOCATOR, root, null, arr, schema); - writer.writeBatch(arr.memoryAddress(), schema.memoryAddress()); - } - } - } - } - - private static List scanAll(VortexReader vf) { - var results = new ArrayList(); - var iter = vf.scan(io.github.dfa1.vortex.scan.ScanOptions.all()); - while (iter.hasNext()) { - results.add(iter.next()); - } - return results; - } -} diff --git a/integration/src/test/java/io/github/dfa1/vortex/integration/FileSizeComparisonIntegrationTest.java b/integration/src/test/java/io/github/dfa1/vortex/integration/FileSizeComparisonIntegrationTest.java deleted file mode 100644 index 1c6491c..0000000 --- a/integration/src/test/java/io/github/dfa1/vortex/integration/FileSizeComparisonIntegrationTest.java +++ /dev/null @@ -1,203 +0,0 @@ -package io.github.dfa1.vortex.integration; - -import dev.vortex.api.Session; -import dev.vortex.arrow.ArrowAllocation; -import dev.vortex.jni.NativeLoader; -import io.github.dfa1.vortex.core.DType; -import io.github.dfa1.vortex.core.PType; -import io.github.dfa1.vortex.core.array.LongArray; -import io.github.dfa1.vortex.encoding.EncodingRegistry; -import io.github.dfa1.vortex.io.VortexReader; -import io.github.dfa1.vortex.writer.VortexWriter; -import io.github.dfa1.vortex.writer.WriteOptions; -import org.apache.arrow.c.ArrowArray; -import org.apache.arrow.c.ArrowSchema; -import org.apache.arrow.c.Data; -import org.apache.arrow.memory.BufferAllocator; -import org.apache.arrow.vector.BigIntVector; -import org.apache.arrow.vector.DateDayVector; -import org.apache.arrow.vector.Float8Vector; -import org.apache.arrow.vector.VarCharVector; -import org.apache.arrow.vector.VectorSchemaRoot; -import org.apache.arrow.vector.types.DateUnit; -import org.apache.arrow.vector.types.FloatingPointPrecision; -import org.apache.arrow.vector.types.pojo.ArrowType; -import org.apache.arrow.vector.types.pojo.Field; -import org.apache.arrow.vector.types.pojo.Schema; -import org.junit.jupiter.api.Test; -import org.junit.jupiter.api.io.TempDir; - -import java.io.BufferedWriter; -import java.io.IOException; -import java.nio.channels.FileChannel; -import java.nio.charset.StandardCharsets; -import java.nio.file.Files; -import java.nio.file.Path; -import java.nio.file.StandardOpenOption; -import java.time.LocalDate; -import java.util.HashMap; -import java.util.List; -import java.util.Map; - -import static org.assertj.core.api.Assertions.assertThat; - -/// File-size comparison: CSV (baseline) vs JNI/Rust writer (oracle) vs Java writer (SUT). -/// -/// Schema: symbol(Utf8), date(I32), open/high/low/close(F64), volume(I64) — 7 columns. -/// Both Vortex writers use cascading(3) to match the Rust default compression pipeline. -class FileSizeComparisonIntegrationTest { - - private static final int TOTAL_ROWS = 100_000; - private static final int BATCH_SIZE = 50_000; - - private static final DType.Struct JAVA_SCHEMA = new DType.Struct( - List.of("symbol", "date", "open", "high", "low", "close", "volume"), - List.of( - new DType.Utf8(false), - new DType.Primitive(PType.I32, false), - new DType.Primitive(PType.F64, false), - new DType.Primitive(PType.F64, false), - new DType.Primitive(PType.F64, false), - new DType.Primitive(PType.F64, false), - new DType.Primitive(PType.I64, false) - ), - false - ); - - private static final Schema JNI_SCHEMA = new Schema(List.of( - Field.notNullable("symbol", new ArrowType.Utf8()), - Field.notNullable("date", new ArrowType.Date(DateUnit.DAY)), - Field.notNullable("open", new ArrowType.FloatingPoint(FloatingPointPrecision.DOUBLE)), - Field.notNullable("high", new ArrowType.FloatingPoint(FloatingPointPrecision.DOUBLE)), - Field.notNullable("low", new ArrowType.FloatingPoint(FloatingPointPrecision.DOUBLE)), - Field.notNullable("close", new ArrowType.FloatingPoint(FloatingPointPrecision.DOUBLE)), - Field.notNullable("volume", new ArrowType.Int(64, true)) - )); - - private static final Session SESSION = Session.create(); - private static final BufferAllocator ALLOCATOR = ArrowAllocation.rootAllocator(); - - static { - NativeLoader.loadJni(); - } - - // ── Writers ─────────────────────────────────────────────────────────────── - - private static Path writeCsv(Path dir, List batches) throws IOException { - Path file = dir.resolve("ohlc.csv"); - try (BufferedWriter csv = Files.newBufferedWriter(file)) { - csv.write("symbol,date,open,high,low,close,volume\n"); - for (OhlcGenerator.OhlcBatch b : batches) { - for (int i = 0; i < b.dates().length; i++) { - csv.write(b.symbols()[i] + "," + LocalDate.ofEpochDay(b.dates()[i]) + "," - + b.open()[i] + "," + b.high()[i] + "," - + b.low()[i] + "," + b.close()[i] + "," + b.volume()[i] + "\n"); - } - } - } - return file; - } - - private static Path writeJava(Path dir, List batches) throws IOException { - Path file = dir.resolve("ohlc-java.vtx"); - try (FileChannel ch = FileChannel.open(file, StandardOpenOption.CREATE, StandardOpenOption.WRITE); - VortexWriter writer = VortexWriter.create(ch, JAVA_SCHEMA, WriteOptions.cascading(3))) { - for (OhlcGenerator.OhlcBatch b : batches) { - writer.writeChunk(Map.of( - "symbol", b.symbols(), "date", b.dates(), - "open", b.open(), "high", b.high(), - "low", b.low(), "close", b.close(), "volume", b.volume())); - } - } - return file; - } - - private static Path writeJni(Path dir, List batches) throws IOException { - Path file = dir.resolve("ohlc-jni.vtx"); - String uri = file.toAbsolutePath().toUri().toString(); - try (dev.vortex.api.VortexWriter writer = dev.vortex.api.VortexWriter.create( - SESSION, uri, JNI_SCHEMA, new HashMap<>(), ALLOCATOR)) { - for (OhlcGenerator.OhlcBatch b : batches) { - try (VectorSchemaRoot root = VectorSchemaRoot.create(JNI_SCHEMA, ALLOCATOR)) { - VarCharVector symbolVec = (VarCharVector) root.getVector("symbol"); - DateDayVector dateVec = (DateDayVector) root.getVector("date"); - Float8Vector openVec = (Float8Vector) root.getVector("open"); - Float8Vector highVec = (Float8Vector) root.getVector("high"); - Float8Vector lowVec = (Float8Vector) root.getVector("low"); - Float8Vector closeVec = (Float8Vector) root.getVector("close"); - BigIntVector volVec = (BigIntVector) root.getVector("volume"); - - int n = b.dates().length; - symbolVec.allocateNew(); - dateVec.allocateNew(n); - openVec.allocateNew(n); - highVec.allocateNew(n); - lowVec.allocateNew(n); - closeVec.allocateNew(n); - volVec.allocateNew(n); - - for (int i = 0; i < n; i++) { - symbolVec.setSafe(i, b.symbols()[i].getBytes(StandardCharsets.UTF_8)); - dateVec.setSafe(i, b.dates()[i]); - openVec.setSafe(i, b.open()[i]); - highVec.setSafe(i, b.high()[i]); - lowVec.setSafe(i, b.low()[i]); - closeVec.setSafe(i, b.close()[i]); - volVec.setSafe(i, b.volume()[i]); - } - root.setRowCount(n); - - try (ArrowArray arr = ArrowArray.allocateNew(ALLOCATOR); - ArrowSchema schema = ArrowSchema.allocateNew(ALLOCATOR)) { - Data.exportVectorSchemaRoot(ALLOCATOR, root, null, arr, schema); - writer.writeBatch(arr.memoryAddress(), schema.memoryAddress()); - } - } - } - } - return file; - } - - // ── Test ────────────────────────────────────────────────────────────────── - - @Test - void fileSizeComparison(@TempDir Path tmp) throws IOException { - // Given - List batches = OhlcGenerator.generate(TOTAL_ROWS, BATCH_SIZE); - - // When - Path csvFile = writeCsv(tmp, batches); - Path jniFile = writeJni(tmp, batches); - Path javaFile = writeJava(tmp, batches); - - long csvSize = Files.size(csvFile); - long jniSize = Files.size(jniFile); - long javaSize = Files.size(javaFile); - - // Then — report - System.out.printf( - "[FileSizeComparison] %,d rows CSV=%,d bytes (%.1f MB) JNI=%,d bytes (%.1f MB) Java=%,d bytes (%.1f MB) Java/JNI=%.2fx CSV/Java=%.1fx%n", - TOTAL_ROWS, - csvSize, csvSize / 1_048_576.0, - jniSize, jniSize / 1_048_576.0, - javaSize, javaSize / 1_048_576.0, - (double) javaSize / jniSize, - (double) csvSize / javaSize); - - // Then — Java beats CSV - assertThat(javaSize).isLessThan(csvSize); - - // Then — Java within 2x of JNI (both use cascading(3)) - assertThat(javaSize).isLessThan(jniSize * 2); - - // Then — Java file is readable with correct row count - long totalRows = 0L; - try (VortexReader reader = VortexReader.open(javaFile, EncodingRegistry.loadAll())) { - var iter = reader.scan(io.github.dfa1.vortex.scan.ScanOptions.columns("volume")); - while (iter.hasNext()) { - totalRows += iter.next().column("volume").length(); - } - } - assertThat(totalRows).isEqualTo(TOTAL_ROWS); - } -} diff --git a/integration/src/test/java/io/github/dfa1/vortex/integration/InspectForTest.java b/integration/src/test/java/io/github/dfa1/vortex/integration/InspectForTest.java deleted file mode 100644 index 090a173..0000000 --- a/integration/src/test/java/io/github/dfa1/vortex/integration/InspectForTest.java +++ /dev/null @@ -1,27 +0,0 @@ -package io.github.dfa1.vortex.integration; - - -import io.github.dfa1.vortex.encoding.EncodingRegistry; -import io.github.dfa1.vortex.io.VortexInspector; -import io.github.dfa1.vortex.io.VortexReader; -import org.junit.jupiter.api.Disabled; -import org.junit.jupiter.api.Test; -import java.nio.file.Path; - -class InspectForTest { - @Disabled("requires /tmp/for.vortex and /tmp/rle.vortex to be manually downloaded") - @Test void inspect() throws Exception { - for (String f : new String[]{"/tmp/for.vortex", "/tmp/rle.vortex"}) { - try (VortexReader r = VortexReader.open(Path.of(f), EncodingRegistry.loadAll())) { - System.out.println("=== " + f + " ==="); - System.out.println(VortexInspector.inspect(r)); - var iter = r.scan(io.github.dfa1.vortex.scan.ScanOptions.all()); - if (iter.hasNext()) { - var chunk = iter.next(); - chunk.columns().forEach((name, arr) -> - System.out.println(" col=" + name + " type=" + arr.getClass().getSimpleName() + " dtype=" + arr.dtype())); - } - } - } - } -} diff --git a/integration/src/test/java/io/github/dfa1/vortex/integration/JavaWritesRustReadsIntegrationTest.java b/integration/src/test/java/io/github/dfa1/vortex/integration/JavaWritesRustReadsIntegrationTest.java deleted file mode 100644 index ec71218..0000000 --- a/integration/src/test/java/io/github/dfa1/vortex/integration/JavaWritesRustReadsIntegrationTest.java +++ /dev/null @@ -1,1109 +0,0 @@ -package io.github.dfa1.vortex.integration; - -import dev.vortex.api.DataSource; -import dev.vortex.api.Expression; -import dev.vortex.api.Partition; -import dev.vortex.api.Scan; -import dev.vortex.api.ScanOptions; -import dev.vortex.api.Session; -import dev.vortex.arrow.ArrowAllocation; -import dev.vortex.jni.NativeLoader; -import io.github.dfa1.vortex.core.DType; -import io.github.dfa1.vortex.core.PType; -import io.github.dfa1.vortex.encoding.BoolEncoding; -import io.github.dfa1.vortex.encoding.ByteBoolEncoding; -import io.github.dfa1.vortex.encoding.ConstantEncoding; -import io.github.dfa1.vortex.encoding.FsstEncoding; -import io.github.dfa1.vortex.encoding.ListData; -import io.github.dfa1.vortex.encoding.ListEncoding; -import io.github.dfa1.vortex.encoding.ListViewData; -import io.github.dfa1.vortex.encoding.ListViewEncoding; -import io.github.dfa1.vortex.encoding.NullEncoding; -import io.github.dfa1.vortex.encoding.RleEncoding; -import io.github.dfa1.vortex.encoding.RunEndEncoding; -import io.github.dfa1.vortex.encoding.SparseEncoding; -import io.github.dfa1.vortex.encoding.VarBinEncoding; -import io.github.dfa1.vortex.encoding.VarBinViewEncoding; -import io.github.dfa1.vortex.encoding.ZigZagEncoding; -import io.github.dfa1.vortex.encoding.ZstdEncoding; -import io.github.dfa1.vortex.writer.VortexWriter; -import io.github.dfa1.vortex.writer.WriteOptions; -import org.apache.arrow.memory.BufferAllocator; -import org.apache.arrow.vector.BigIntVector; -import org.apache.arrow.vector.BitVector; -import org.apache.arrow.vector.Float2Vector; -import org.apache.arrow.vector.Float8Vector; -import org.apache.arrow.vector.IntVector; -import org.apache.arrow.vector.VarCharVector; -import org.apache.arrow.vector.VectorSchemaRoot; -import org.apache.arrow.vector.complex.ListVector; -import org.apache.arrow.vector.ipc.ArrowReader; -import org.junit.jupiter.api.Disabled; -import org.junit.jupiter.api.Test; -import org.junit.jupiter.api.io.TempDir; -import org.junit.jupiter.params.ParameterizedTest; -import org.junit.jupiter.params.provider.Arguments; -import org.junit.jupiter.params.provider.MethodSource; - -import java.io.IOException; -import java.io.UncheckedIOException; -import java.nio.channels.FileChannel; -import java.nio.file.Files; -import java.nio.file.Path; -import java.nio.file.StandardOpenOption; -import java.util.ArrayList; -import java.util.Arrays; -import java.util.Comparator; -import java.util.List; -import java.util.Map; -import java.util.stream.Stream; - -import static org.assertj.core.api.Assertions.assertThat; - -/// Cross-compatibility: Java writer → Rust (JNI) reader. -class JavaWritesRustReadsIntegrationTest { - - private static final Session SESSION = Session.create(); - private static final BufferAllocator ALLOCATOR = ArrowAllocation.rootAllocator(); - private static final DType.Struct SCHEMA = new DType.Struct( - List.of("id", "value"), - List.of(new DType.Primitive(PType.I64, false), - new DType.Primitive(PType.F64, false)), - false); - - private static final DType.Struct I32_SCHEMA = new DType.Struct( - List.of("v"), - List.of(new DType.Primitive(PType.I32, false)), - false); - - private static final DType.Struct STRING_SCHEMA = new DType.Struct( - List.of("s"), - List.of(new DType.Utf8(false)), - false); - - private static final DType.Struct TS_SCHEMA = new DType.Struct( - List.of("ts"), - List.of(new DType.Primitive(PType.I64, false)), - false); - - private static final DType.Struct F32_SCHEMA = new DType.Struct( - List.of("v"), - List.of(new DType.Primitive(PType.F32, false)), - false); - - private static final DType.Struct F64_SCHEMA = new DType.Struct( - List.of("v"), - List.of(new DType.Primitive(PType.F64, false)), - false); - - private static final DType.Struct F16_SCHEMA = new DType.Struct( - List.of("v"), - List.of(new DType.Primitive(PType.F16, false)), - false); - - private static final DType.Struct BOOL_SCHEMA = new DType.Struct( - List.of("b"), - List.of(new DType.Bool(false)), - false); - - private static final DType.Struct NULL_SCHEMA = new DType.Struct( - List.of("n"), - List.of(new DType.Null(false)), - false); - - private static final DType.Struct LIST_I64_SCHEMA = new DType.Struct( - List.of("items"), - List.of(new DType.List(new DType.Primitive(PType.I64, false), false)), - false); - - private static final DType.Struct OHLC_SCHEMA = new DType.Struct( - List.of("date", "symbol", "open", "high", "low", "close", "volume"), - List.of( - new DType.Primitive(PType.I32, false), - new DType.Utf8(false), - new DType.Primitive(PType.F64, false), - new DType.Primitive(PType.F64, false), - new DType.Primitive(PType.F64, false), - new DType.Primitive(PType.F64, false), - new DType.Primitive(PType.I64, false)), - false); - - static { - NativeLoader.loadJni(); - } - - private static long[] readLongColumn(Path file, String column) throws IOException { - String uri = file.toAbsolutePath().toUri().toString(); - ScanOptions opts = ScanOptions.builder() - .projection(Expression.select(new String[]{column}, Expression.root())) - .build(); - var longs = new ArrayList(); - DataSource ds = DataSource.open(SESSION, uri); - Scan scan = ds.scan(opts); - while (scan.hasNext()) { - Partition partition = scan.next(); - try (ArrowReader reader = partition.scanArrow(ALLOCATOR)) { - while (reader.loadNextBatch()) { - VectorSchemaRoot root = reader.getVectorSchemaRoot(); - BigIntVector vec = (BigIntVector) root.getVector(column); - for (int i = 0; i < root.getRowCount(); i++) { - longs.add(vec.get(i)); - } - } - } - } - return longs.stream().mapToLong(Long::longValue).toArray(); - } - - private static double[] readDoubleColumn(Path file, String column) throws IOException { - String uri = file.toAbsolutePath().toUri().toString(); - ScanOptions opts = ScanOptions.builder() - .projection(Expression.select(new String[]{column}, Expression.root())) - .build(); - var doubles = new ArrayList(); - DataSource ds = DataSource.open(SESSION, uri); - Scan scan = ds.scan(opts); - while (scan.hasNext()) { - Partition partition = scan.next(); - try (ArrowReader reader = partition.scanArrow(ALLOCATOR)) { - while (reader.loadNextBatch()) { - VectorSchemaRoot root = reader.getVectorSchemaRoot(); - Float8Vector vec = (Float8Vector) root.getVector(column); - for (int i = 0; i < root.getRowCount(); i++) { - doubles.add(vec.get(i)); - } - } - } - } - return doubles.stream().mapToDouble(Double::doubleValue).toArray(); - } - - private static float[] readFloatColumn(Path file, String column) throws IOException { - String uri = file.toAbsolutePath().toUri().toString(); - ScanOptions opts = ScanOptions.builder() - .projection(Expression.select(new String[]{column}, Expression.root())) - .build(); - var floats = new ArrayList(); - DataSource ds = DataSource.open(SESSION, uri); - Scan scan = ds.scan(opts); - while (scan.hasNext()) { - Partition partition = scan.next(); - try (ArrowReader reader = partition.scanArrow(ALLOCATOR)) { - while (reader.loadNextBatch()) { - VectorSchemaRoot root = reader.getVectorSchemaRoot(); - org.apache.arrow.vector.Float4Vector vec = (org.apache.arrow.vector.Float4Vector) root.getVector(column); - for (int i = 0; i < root.getRowCount(); i++) { - floats.add(vec.get(i)); - } - } - } - } - float[] result = new float[floats.size()]; - for (int i = 0; i < result.length; i++) { - result[i] = floats.get(i); - } - return result; - } - - private static short[] readHalfColumn(Path file, String column) throws IOException { - String uri = file.toAbsolutePath().toUri().toString(); - ScanOptions opts = ScanOptions.builder() - .projection(Expression.select(new String[]{column}, Expression.root())) - .build(); - var halfs = new ArrayList(); - DataSource ds = DataSource.open(SESSION, uri); - Scan scan = ds.scan(opts); - while (scan.hasNext()) { - Partition partition = scan.next(); - try (ArrowReader reader = partition.scanArrow(ALLOCATOR)) { - while (reader.loadNextBatch()) { - VectorSchemaRoot root = reader.getVectorSchemaRoot(); - Float2Vector vec = (Float2Vector) root.getVector(column); - for (int i = 0; i < root.getRowCount(); i++) { - halfs.add(vec.get(i)); - } - } - } - } - short[] result = new short[halfs.size()]; - for (int i = 0; i < result.length; i++) { - result[i] = halfs.get(i); - } - return result; - } - - private static int[] readIntColumn(Path file, String column) throws IOException { - String uri = file.toAbsolutePath().toUri().toString(); - ScanOptions opts = ScanOptions.builder() - .projection(Expression.select(new String[]{column}, Expression.root())) - .build(); - var ints = new ArrayList(); - DataSource ds = DataSource.open(SESSION, uri); - Scan scan = ds.scan(opts); - while (scan.hasNext()) { - Partition partition = scan.next(); - try (ArrowReader reader = partition.scanArrow(ALLOCATOR)) { - while (reader.loadNextBatch()) { - VectorSchemaRoot root = reader.getVectorSchemaRoot(); - IntVector vec = (IntVector) root.getVector(column); - for (int i = 0; i < root.getRowCount(); i++) { - ints.add(vec.get(i)); - } - } - } - } - return ints.stream().mapToInt(Integer::intValue).toArray(); - } - - private static String[] readStringColumn(Path file, String column) throws IOException { - String uri = file.toAbsolutePath().toUri().toString(); - ScanOptions opts = ScanOptions.builder() - .projection(Expression.select(new String[]{column}, Expression.root())) - .build(); - var strings = new ArrayList(); - DataSource ds = DataSource.open(SESSION, uri); - Scan scan = ds.scan(opts); - while (scan.hasNext()) { - Partition partition = scan.next(); - try (ArrowReader reader = partition.scanArrow(ALLOCATOR)) { - while (reader.loadNextBatch()) { - VectorSchemaRoot root = reader.getVectorSchemaRoot(); - VarCharVector vec = (VarCharVector) root.getVector(column); - for (int i = 0; i < root.getRowCount(); i++) { - strings.add(vec.getObject(i).toString()); - } - } - } - } - return strings.toArray(String[]::new); - } - - // ── JNI read helpers ────────────────────────────────────────────────────── - - @Test - void javaWriter_jniReader_cascading_ohlc(@TempDir Path tmp) throws IOException { - // Given — OHLC data written with cascading(3): exercises ALP→FOR→bitpacked chain - Path file = tmp.resolve("java_cascade_ohlc.vtx"); - List batches = OhlcGenerator.generate(10_000, 1_000); - - try (var ch = FileChannel.open(file, StandardOpenOption.CREATE, StandardOpenOption.WRITE); - var sut = VortexWriter.create(ch, OHLC_SCHEMA, WriteOptions.cascading(3))) { - for (OhlcGenerator.OhlcBatch b : batches) { - sut.writeChunk(Map.of( - "date", b.dates(), - "symbol", b.symbols(), - "open", b.open(), - "high", b.high(), - "low", b.low(), - "close", b.close(), - "volume", b.volume())); - } - } - - // When - long[] volumes = readLongColumn(file, "volume"); - double[] closes = readDoubleColumn(file, "close"); - - // Then — JNI reader may return chunks in a different order for cascaded files; - // verify all values round-trip correctly regardless of partition order. - long[] expectedVolumes = batches.stream().flatMapToLong(b -> Arrays.stream(b.volume())).toArray(); - double[] expectedCloses = batches.stream().flatMapToDouble(b -> Arrays.stream(b.close())).toArray(); - assertThat(volumes).containsExactlyInAnyOrder(expectedVolumes); - assertThat(closes).containsExactlyInAnyOrder(expectedCloses); - } - - @Test - void javaWriter_jniReader_singleChunk(@TempDir Path tmp) throws IOException { - // Given - Path file = tmp.resolve("java_single.vtx"); - long[] ids = {1L, 2L, 3L}; - double[] vals = {1.1, 2.2, 3.3}; - try (var ch = FileChannel.open(file, StandardOpenOption.CREATE, StandardOpenOption.WRITE); - var sut = VortexWriter.create(ch, SCHEMA, WriteOptions.defaults())) { - sut.writeChunk(Map.of("id", ids, "value", vals)); - } - - // When - long[] decodedIds = readLongColumn(file, "id"); - double[] decodedVals = readDoubleColumn(file, "value"); - - // Then - assertThat(decodedIds).containsExactly(1L, 2L, 3L); - assertThat(decodedVals).containsExactly(1.1, 2.2, 3.3); - } - - @Test - void javaWriter_jniReader_multipleChunks(@TempDir Path tmp) throws IOException { - // Given - Path file = tmp.resolve("java_multi.vtx"); - try (var ch = FileChannel.open(file, StandardOpenOption.CREATE, StandardOpenOption.WRITE); - var sut = VortexWriter.create(ch, SCHEMA, WriteOptions.defaults())) { - sut.writeChunk(Map.of("id", new long[]{1L, 2L}, "value", new double[]{1.1, 2.2})); - sut.writeChunk(Map.of("id", new long[]{3L, 4L, 5L}, "value", new double[]{3.3, 4.4, 5.5})); - } - - // When - long[] decodedIds = readLongColumn(file, "id"); - - // Then — JNI may merge chunks; verify all values present regardless of partition count - assertThat(decodedIds).containsExactly(1L, 2L, 3L, 4L, 5L); - } - - @Test - void javaWriter_jniReader_i32Column(@TempDir Path tmp) throws IOException { - // Given - Path file = tmp.resolve("java_i32.vtx"); - int[] data = {-100, 0, 1, 127, Integer.MAX_VALUE, Integer.MIN_VALUE}; - try (var ch = FileChannel.open(file, StandardOpenOption.CREATE, StandardOpenOption.WRITE); - var sut = VortexWriter.create(ch, I32_SCHEMA, WriteOptions.defaults())) { - // When - sut.writeChunk(Map.of("v", data)); - } - - // Then - int[] decoded = readIntColumn(file, "v"); - assertThat(decoded).containsExactly(data); - } - - @Test - void javaWriter_jniReader_utf8Column(@TempDir Path tmp) throws IOException { - // Given — VarBin encoding (raw bytes + offsets, no dictionary) - Path file = tmp.resolve("java_utf8.vtx"); - String[] data = {"apple", "banana", "cherry", "date", "elderberry"}; - try (var ch = FileChannel.open(file, StandardOpenOption.CREATE, StandardOpenOption.WRITE); - var sut = VortexWriter.create(ch, STRING_SCHEMA, WriteOptions.defaults(), - List.of(new VarBinEncoding()))) { - // When - sut.writeChunk(Map.of("s", data)); - } - - // Then - String[] decoded = readStringColumn(file, "s"); - assertThat(decoded).containsExactly(data); - } - - @Test - void javaWriter_jniReader_fsstUtf8Column(@TempDir Path tmp) throws IOException { - // Given — FSST encoding: bigram symbol table, escape fallback for unmatched bytes - Path file = tmp.resolve("java_fsst_utf8.vtx"); - String[] data = {"apple", "banana", "cherry", "apricot", "avocado", "almond"}; - try (var ch = FileChannel.open(file, StandardOpenOption.CREATE, StandardOpenOption.WRITE); - var sut = VortexWriter.create(ch, STRING_SCHEMA, WriteOptions.defaults(), - List.of(new FsstEncoding()))) { - // When - sut.writeChunk(Map.of("s", data)); - } - - // Then - String[] decoded = readStringColumn(file, "s"); - assertThat(decoded).containsExactly(data); - } - - @Test - void javaWriter_jniReader_varBinViewUtf8Column_inlined(@TempDir Path tmp) throws IOException { - // Given — VarBinView, all strings ≤12 bytes: inlined path (no data buffer) - Path file = tmp.resolve("java_varbinview_inlined.vtx"); - String[] data = {"hi", "yo", "ok", "abc", "short", "exactly12ok!"}; - try (var ch = FileChannel.open(file, StandardOpenOption.CREATE, StandardOpenOption.WRITE); - var sut = VortexWriter.create(ch, STRING_SCHEMA, WriteOptions.defaults(), - List.of(new VarBinViewEncoding()))) { - // When - sut.writeChunk(Map.of("s", data)); - } - - // Then - String[] decoded = readStringColumn(file, "s"); - assertThat(decoded).containsExactly(data); - } - - @Test - void javaWriter_jniReader_varBinViewUtf8Column_referenced(@TempDir Path tmp) throws IOException { - // Given — VarBinView, all strings >12 bytes: reference path (data buffer present) - Path file = tmp.resolve("java_varbinview_referenced.vtx"); - String[] data = {"this is long text", "another long string", "yet another long one", "thirteenchars!"}; - try (var ch = FileChannel.open(file, StandardOpenOption.CREATE, StandardOpenOption.WRITE); - var sut = VortexWriter.create(ch, STRING_SCHEMA, WriteOptions.defaults(), - List.of(new VarBinViewEncoding()))) { - // When - sut.writeChunk(Map.of("s", data)); - } - - // Then - String[] decoded = readStringColumn(file, "s"); - assertThat(decoded).containsExactly(data); - } - - @Test - void javaWriter_jniReader_varBinViewUtf8Column_mixed(@TempDir Path tmp) throws IOException { - // Given — VarBinView, mix of inlined (≤12) and referenced (>12) strings - Path file = tmp.resolve("java_varbinview_mixed.vtx"); - String[] data = {"short", "this is a longer string", "hi", "medium length ok", "x", "another longer string here"}; - try (var ch = FileChannel.open(file, StandardOpenOption.CREATE, StandardOpenOption.WRITE); - var sut = VortexWriter.create(ch, STRING_SCHEMA, WriteOptions.defaults(), - List.of(new VarBinViewEncoding()))) { - // When - sut.writeChunk(Map.of("s", data)); - } - - // Then - String[] decoded = readStringColumn(file, "s"); - assertThat(decoded).containsExactly(data); - } - - /// VarBinView utf8 with arbitrary strings — exercises both inlined and referenced views. - @ParameterizedTest - @MethodSource("varBinViewStringArrayProvider") - void prop_varBinView_utf8_roundTripsViaRust(String[] data) throws IOException { - Path tmp = Files.createTempDirectory("vortex-pbt-varbinview"); - try { - Path file = tmp.resolve("pbt_varbinview_utf8.vtx"); - try (var ch = FileChannel.open(file, StandardOpenOption.CREATE, StandardOpenOption.WRITE); - var sut = VortexWriter.create(ch, STRING_SCHEMA, WriteOptions.defaults(), - List.of(new VarBinViewEncoding()))) { - sut.writeChunk(Map.of("s", data)); - } - String[] decoded = readStringColumn(file, "s"); - assertThat(decoded).containsExactly(data); - } finally { - deleteDir(tmp); - } - } - - @Test - void javaWriter_jniReader_dictEncodedUtf8Column(@TempDir Path tmp) throws IOException { - // Given — DictEncoding (DictLayoutMetadata proto + children[0]=codes, children[1]=VarBin values) - Path file = tmp.resolve("java_dict_utf8.vtx"); - String[] data = {"apple", "banana", "cherry", "date", "elderberry"}; - try (var ch = FileChannel.open(file, StandardOpenOption.CREATE, StandardOpenOption.WRITE); - var sut = VortexWriter.create(ch, STRING_SCHEMA, WriteOptions.defaults())) { - // When — default pipeline selects DictEncoding for Utf8 - sut.writeChunk(Map.of("s", data)); - } - - // Then - String[] decoded = readStringColumn(file, "s"); - assertThat(decoded).containsExactly(data); - } - - @Test - void javaWriter_jniReader_largeChunk_twoFastLanesBlocks(@TempDir Path tmp) throws IOException { - // Given — 2048 rows = exactly 2 full 1024-element FastLanes blocks - Path file = tmp.resolve("java_large.vtx"); - int n = 2048; - long[] ids = new long[n]; - double[] vals = new double[n]; - for (int i = 0; i < n; i++) { - ids[i] = i; - vals[i] = i * 0.5; - } - try (var ch = FileChannel.open(file, StandardOpenOption.CREATE, StandardOpenOption.WRITE); - var sut = VortexWriter.create(ch, SCHEMA, WriteOptions.defaults())) { - // When - sut.writeChunk(Map.of("id", ids, "value", vals)); - } - - // Then - long[] decodedIds = readLongColumn(file, "id"); - assertThat(decodedIds).containsExactly(ids); - } - - @Test - void javaWriter_jniReader_monotonic_i64_cascading(@TempDir Path tmp) throws IOException { - // Given — monotonic timestamps: FOR reduces to constant delta, bitpacked to ~10 bits - Path file = tmp.resolve("java_ts.vtx"); - int n = 5_000; - long[] ts = new long[n]; - for (int i = 0; i < n; i++) { - ts[i] = 1_700_000_000L + (long) i * 1_000; - } - try (var ch = FileChannel.open(file, StandardOpenOption.CREATE, StandardOpenOption.WRITE); - var sut = VortexWriter.create(ch, TS_SCHEMA, WriteOptions.cascading(3))) { - // When - sut.writeChunk(Map.of("ts", ts)); - } - - // Then - long[] decoded = readLongColumn(file, "ts"); - assertThat(decoded).containsExactly(ts); - } - - @Test - void javaWriter_jniReader_cascading_ohlc_columnProjection(@TempDir Path tmp) throws IOException { - // Given - Path file = tmp.resolve("java_cascade_proj.vtx"); - List batches = OhlcGenerator.generate(2_000, 1_000); - try (var ch = FileChannel.open(file, StandardOpenOption.CREATE, StandardOpenOption.WRITE); - var sut = VortexWriter.create(ch, OHLC_SCHEMA, WriteOptions.cascading(3))) { - for (OhlcGenerator.OhlcBatch b : batches) { - sut.writeChunk(Map.of( - "date", b.dates(), "symbol", b.symbols(), - "open", b.open(), "high", b.high(), - "low", b.low(), "close", b.close(), "volume", b.volume())); - } - } - - // When — project only volume - long[] volumes = readLongColumn(file, "volume"); - - // Then - long[] expected = batches.stream().flatMapToLong(b -> Arrays.stream(b.volume())).toArray(); - assertThat(volumes).containsExactlyInAnyOrder(expected); - } - - // ── Parameterized random-data tests ───────────────────────────────────── - - /// Dict utf8 with arbitrary strings (small dict → U8 codes). - /// Validates that random string data survives Java dict-encode → Rust JNI read. - @ParameterizedTest - @MethodSource("asciiStringArrayProvider") - void prop_dictUtf8_ascii_roundTripsViaRust(String[] data) throws IOException { - Path tmp = Files.createTempDirectory("vortex-pbt-ascii"); - try { - Path file = tmp.resolve("pbt_dict_utf8_ascii.vtx"); - try (var ch = FileChannel.open(file, StandardOpenOption.CREATE, StandardOpenOption.WRITE); - var sut = VortexWriter.create(ch, STRING_SCHEMA, WriteOptions.defaults())) { - sut.writeChunk(Map.of("s", data)); - } - String[] decoded = readStringColumn(file, "s"); - assertThat(decoded).containsExactly(data); - } finally { - deleteDir(tmp); - } - } - - /// Dict utf8 with 257+ unique strings → forces U16 codes (crosses U8→U16 boundary at 256). - @ParameterizedTest - @MethodSource("u16DictStringArrayProvider") - void prop_dictUtf8_u16Codes_roundTripsViaRust(String[] data) throws IOException { - Path tmp = Files.createTempDirectory("vortex-pbt-u16"); - try { - Path file = tmp.resolve("pbt_dict_utf8_u16.vtx"); - try (var ch = FileChannel.open(file, StandardOpenOption.CREATE, StandardOpenOption.WRITE); - var sut = VortexWriter.create(ch, STRING_SCHEMA, WriteOptions.defaults())) { - sut.writeChunk(Map.of("s", data)); - } - String[] decoded = readStringColumn(file, "s"); - assertThat(decoded).containsExactly(data); - } finally { - deleteDir(tmp); - } - } - - /// Dict utf8 with unicode strings (multi-byte UTF-8, CJK). - @ParameterizedTest - @MethodSource("unicodeStringArrayProvider") - void prop_dictUtf8_unicode_roundTripsViaRust(String[] data) throws IOException { - Path tmp = Files.createTempDirectory("vortex-pbt-unicode"); - try { - Path file = tmp.resolve("pbt_dict_utf8_unicode.vtx"); - try (var ch = FileChannel.open(file, StandardOpenOption.CREATE, StandardOpenOption.WRITE); - var sut = VortexWriter.create(ch, STRING_SCHEMA, WriteOptions.defaults())) { - sut.writeChunk(Map.of("s", data)); - } - String[] decoded = readStringColumn(file, "s"); - assertThat(decoded).containsExactly(data); - } finally { - deleteDir(tmp); - } - } - - /// I64 column: full Long range (MIN_VALUE, MAX_VALUE, negatives), empty and large arrays. - @ParameterizedTest - @MethodSource("i64ArrayProvider") - void prop_i64_roundTripsViaRust(long[] data) throws IOException { - Path tmp = Files.createTempDirectory("vortex-pbt-i64"); - try { - Path file = tmp.resolve("pbt_i64.vtx"); - try (var ch = FileChannel.open(file, StandardOpenOption.CREATE, StandardOpenOption.WRITE); - var sut = VortexWriter.create(ch, TS_SCHEMA, WriteOptions.defaults())) { - sut.writeChunk(Map.of("ts", data)); - } - long[] decoded = readLongColumn(file, "ts"); - assertThat(decoded).containsExactly(data); - } finally { - deleteDir(tmp); - } - } - - - /// F64 column: finite doubles (ALP encoding), empty and large arrays. - @ParameterizedTest - @MethodSource("f64ArrayProvider") - void prop_f64_roundTripsViaRust(double[] data) throws IOException { - Path tmp = Files.createTempDirectory("vortex-pbt-f64"); - try { - Path file = tmp.resolve("pbt_f64.vtx"); - try (var ch = FileChannel.open(file, StandardOpenOption.CREATE, StandardOpenOption.WRITE); - var sut = VortexWriter.create(ch, F64_SCHEMA, WriteOptions.defaults())) { - sut.writeChunk(Map.of("v", data)); - } - double[] decoded = readDoubleColumn(file, "v"); - assertThat(decoded).containsExactly(data); - } finally { - deleteDir(tmp); - } - } - - /// F32 column: finite floats, empty and large arrays. - @ParameterizedTest - @MethodSource("f32ArrayProvider") - void prop_f32_roundTripsViaRust(float[] data) throws IOException { - Path tmp = Files.createTempDirectory("vortex-pbt-f32"); - try { - Path file = tmp.resolve("pbt_f32.vtx"); - try (var ch = FileChannel.open(file, StandardOpenOption.CREATE, StandardOpenOption.WRITE); - var sut = VortexWriter.create(ch, F32_SCHEMA, WriteOptions.defaults())) { - sut.writeChunk(Map.of("v", data)); - } - float[] decoded = readFloatColumn(file, "v"); - assertThat(decoded).containsExactly(data); - } finally { - deleteDir(tmp); - } - } - - static Stream varBinViewStringArrayProvider() { - return RandomArrays.varBinViewStringArrays(20).map(arr -> Arguments.of((Object) arr)); - } - - static Stream asciiStringArrayProvider() { - return RandomArrays.asciiStringArrays(20).map(arr -> Arguments.of((Object) arr)); - } - - static Stream u16DictStringArrayProvider() { - return RandomArrays.u16DictStringArrays(10).map(arr -> Arguments.of((Object) arr)); - } - - static Stream unicodeStringArrayProvider() { - return RandomArrays.unicodeStringArrays(20).map(arr -> Arguments.of((Object) arr)); - } - - static Stream i64ArrayProvider() { - return RandomArrays.i64Arrays(30); - } - - static Stream f64ArrayProvider() { - return RandomArrays.f64Arrays(30); - } - - static Stream f32ArrayProvider() { - return RandomArrays.f32Arrays(30); - } - - /// F64: NaN and Inf go to ALP patches (raw bits). containsExactly fails for NaN — use bitwise comparison. - @Test - void javaWriter_jniReader_f64_nanAndInf(@TempDir Path tmp) throws IOException { - // Given - Path file = tmp.resolve("f64_nan_inf.vtx"); - double[] data = {Double.NaN, Double.POSITIVE_INFINITY, Double.NEGATIVE_INFINITY, 0.0, 1.5, -1.5}; - try (var ch = FileChannel.open(file, StandardOpenOption.CREATE, StandardOpenOption.WRITE); - var sut = VortexWriter.create(ch, F64_SCHEMA, WriteOptions.defaults())) { - // When - sut.writeChunk(Map.of("v", data)); - } - - // Then - double[] decoded = readDoubleColumn(file, "v"); - assertBitwiseEqualsF64(decoded, data); - } - - /// F32: same as F64 — NaN and Inf go to ALP patches, require bitwise comparison. - @Test - void javaWriter_jniReader_f32_nanAndInf(@TempDir Path tmp) throws IOException { - // Given - Path file = tmp.resolve("f32_nan_inf.vtx"); - float[] data = {Float.NaN, Float.POSITIVE_INFINITY, Float.NEGATIVE_INFINITY, 0.0f, 1.5f, -1.5f}; - try (var ch = FileChannel.open(file, StandardOpenOption.CREATE, StandardOpenOption.WRITE); - var sut = VortexWriter.create(ch, F32_SCHEMA, WriteOptions.defaults())) { - // When - sut.writeChunk(Map.of("v", data)); - } - - // Then - float[] decoded = readFloatColumn(file, "v"); - assertBitwiseEqualsF32(decoded, data); - } - - /// F16: PrimitiveEncoding stores raw short bits — NaN and Inf are just bit patterns. - /// Disabled: vortex-jni 0.72 does not export F16 via Arrow C Data Interface. - @Disabled - @Test - void javaWriter_jniReader_f16_nanAndInf(@TempDir Path tmp) throws IOException { - // Given - Path file = tmp.resolve("f16_nan_inf.vtx"); - // Half-precision: +Inf=0x7C00, -Inf=0xFC00, quiet NaN=0x7E00, 1.0=0x3C00, 0.0=0x0000 - short[] data = {(short) 0x7E00, (short) 0x7C00, (short) 0xFC00, (short) 0x3C00, (short) 0x0000}; - try (var ch = FileChannel.open(file, StandardOpenOption.CREATE, StandardOpenOption.WRITE); - var sut = VortexWriter.create(ch, F16_SCHEMA, WriteOptions.defaults())) { - // When - sut.writeChunk(Map.of("v", data)); - } - - // Then - short[] decoded = readHalfColumn(file, "v"); - assertThat(decoded).containsExactly(data); - } - - @Test - void javaWriter_rustReader_zigzag_i32(@TempDir Path tmp) throws IOException { - // Given — ZigZag: signed integers with negatives exercise both encode paths - Path file = tmp.resolve("java_zigzag_i32.vtx"); - int[] data = {-1000, -1, 0, 1, 127, -127, Integer.MIN_VALUE / 2, Integer.MAX_VALUE / 2}; - try (var ch = FileChannel.open(file, StandardOpenOption.CREATE, StandardOpenOption.WRITE); - var sut = VortexWriter.create(ch, I32_SCHEMA, WriteOptions.defaults(), - List.of(new ZigZagEncoding()))) { - // When - sut.writeChunk(Map.of("v", data)); - } - - // Then - int[] decoded = readIntColumn(file, "v"); - assertThat(decoded).containsExactly(data); - } - - @Test - void javaWriter_rustReader_zigzag_i64(@TempDir Path tmp) throws IOException { - // Given - Path file = tmp.resolve("java_zigzag_i64.vtx"); - long[] data = {Long.MIN_VALUE / 2, -1L, 0L, 1L, Long.MAX_VALUE / 2}; - try (var ch = FileChannel.open(file, StandardOpenOption.CREATE, StandardOpenOption.WRITE); - var sut = VortexWriter.create(ch, TS_SCHEMA, WriteOptions.defaults(), - List.of(new ZigZagEncoding()))) { - // When - sut.writeChunk(Map.of("ts", data)); - } - - // Then - long[] decoded = readLongColumn(file, "ts"); - assertThat(decoded).containsExactly(data); - } - - @Test - void javaWriter_rustReader_runEnd_i32(@TempDir Path tmp) throws IOException { - // Given — RunEnd: runs of same value exercise the run-length proto serialisation - Path file = tmp.resolve("java_runend_i32.vtx"); - int[] data = {10, 10, 10, 20, 20, 30, 30, 30, 30}; - try (var ch = FileChannel.open(file, StandardOpenOption.CREATE, StandardOpenOption.WRITE); - var sut = VortexWriter.create(ch, I32_SCHEMA, WriteOptions.defaults(), - List.of(new RunEndEncoding()))) { - // When - sut.writeChunk(Map.of("v", data)); - } - - // Then - int[] decoded = readIntColumn(file, "v"); - assertThat(decoded).containsExactly(data); - } - - @Test - void javaWriter_rustReader_runEnd_i64(@TempDir Path tmp) throws IOException { - // Given - Path file = tmp.resolve("java_runend_i64.vtx"); - long[] data = {100L, 100L, 200L, 200L, 200L, 300L}; - try (var ch = FileChannel.open(file, StandardOpenOption.CREATE, StandardOpenOption.WRITE); - var sut = VortexWriter.create(ch, TS_SCHEMA, WriteOptions.defaults(), - List.of(new RunEndEncoding()))) { - // When - sut.writeChunk(Map.of("ts", data)); - } - - // Then - long[] decoded = readLongColumn(file, "ts"); - assertThat(decoded).containsExactly(data); - } - - @Test - void javaWriter_rustReader_rle_i32(@TempDir Path tmp) throws IOException { - // Given — FastLanes RLE: chunk-based RLE with offset; exercises chunk boundary proto fields - Path file = tmp.resolve("java_rle_i32.vtx"); - int[] data = {5, 5, 5, 7, 7, 5, 5, 5, 5, 9}; - try (var ch = FileChannel.open(file, StandardOpenOption.CREATE, StandardOpenOption.WRITE); - var sut = VortexWriter.create(ch, I32_SCHEMA, WriteOptions.defaults(), - List.of(new RleEncoding()))) { - // When - sut.writeChunk(Map.of("v", data)); - } - - // Then - int[] decoded = readIntColumn(file, "v"); - assertThat(decoded).containsExactly(data); - } - - @Test - void javaWriter_rustReader_rle_i64(@TempDir Path tmp) throws IOException { - // Given - Path file = tmp.resolve("java_rle_i64.vtx"); - long[] data = {1L, 1L, 1L, 2L, 2L, 2L, 3L, 3L}; - try (var ch = FileChannel.open(file, StandardOpenOption.CREATE, StandardOpenOption.WRITE); - var sut = VortexWriter.create(ch, TS_SCHEMA, WriteOptions.defaults(), - List.of(new RleEncoding()))) { - // When - sut.writeChunk(Map.of("ts", data)); - } - - // Then - long[] decoded = readLongColumn(file, "ts"); - assertThat(decoded).containsExactly(data); - } - - @Test - void javaWriter_rustReader_constant_i32(@TempDir Path tmp) throws IOException { - // Given — Constant: scalar proto serialisation and row count metadata - Path file = tmp.resolve("java_constant_i32.vtx"); - int[] data = {42, 42, 42, 42, 42}; - try (var ch = FileChannel.open(file, StandardOpenOption.CREATE, StandardOpenOption.WRITE); - var sut = VortexWriter.create(ch, I32_SCHEMA, WriteOptions.defaults(), - List.of(new ConstantEncoding()))) { - // When - sut.writeChunk(Map.of("v", data)); - } - - // Then - int[] decoded = readIntColumn(file, "v"); - assertThat(decoded).containsExactly(data); - } - - @Test - void javaWriter_rustReader_sparse_i32(@TempDir Path tmp) throws IOException { - // Given — Sparse: mostly-zero data; exercises patch indices + endianness of offset field - Path file = tmp.resolve("java_sparse_i32.vtx"); - int[] data = {0, 0, 7, 0, 0, 0, 13, 0, 0, 0}; - try (var ch = FileChannel.open(file, StandardOpenOption.CREATE, StandardOpenOption.WRITE); - var sut = VortexWriter.create(ch, I32_SCHEMA, WriteOptions.defaults(), - List.of(new SparseEncoding()))) { - // When - sut.writeChunk(Map.of("v", data)); - } - - // Then - int[] decoded = readIntColumn(file, "v"); - assertThat(decoded).containsExactly(data); - } - - @Test - void javaWriter_rustReader_bool_boolEncoding(@TempDir Path tmp) throws IOException { - // Given — BoolEncoding: bit-packed boolean column - Path file = tmp.resolve("java_bool.vtx"); - boolean[] data = {true, false, true, true, false, false, true}; - try (var ch = FileChannel.open(file, StandardOpenOption.CREATE, StandardOpenOption.WRITE); - var sut = VortexWriter.create(ch, BOOL_SCHEMA, WriteOptions.defaults(), - List.of(new BoolEncoding()))) { - // When - sut.writeChunk(Map.of("b", data)); - } - - // Then - boolean[] decoded = readBoolColumn(file, "b"); - assertThat(decoded).containsExactly(data[0], data[1], data[2], data[3], data[4], data[5], data[6]); - } - - @Test - void javaWriter_rustReader_bool_byteBoolEncoding(@TempDir Path tmp) throws IOException { - // Given — ByteBoolEncoding: one byte per bool (uncompressed), exercises different wire format - Path file = tmp.resolve("java_bytebool.vtx"); - boolean[] data = {false, true, false, true, true}; - try (var ch = FileChannel.open(file, StandardOpenOption.CREATE, StandardOpenOption.WRITE); - var sut = VortexWriter.create(ch, BOOL_SCHEMA, WriteOptions.defaults(), - List.of(new ByteBoolEncoding()))) { - // When - sut.writeChunk(Map.of("b", data)); - } - - // Then - boolean[] decoded = readBoolColumn(file, "b"); - assertThat(decoded).containsExactly(data[0], data[1], data[2], data[3], data[4]); - } - - @Test - void javaWriter_rustReader_nullColumn(@TempDir Path tmp) throws IOException { - // Given — NullEncoding: entire column is null; exercises Null DType + empty EncodeNode - Path file = tmp.resolve("java_null.vtx"); - long rowCount = 7L; - try (var ch = FileChannel.open(file, StandardOpenOption.CREATE, StandardOpenOption.WRITE); - var sut = VortexWriter.create(ch, NULL_SCHEMA, WriteOptions.defaults(), - List.of(new NullEncoding()))) { - // When — data is ignored by NullEncoding; pass long[] to satisfy arrayLength - sut.writeChunk(Map.of("n", new long[(int) rowCount])); - } - - // Then - long count = readRowCount(file); - assertThat(count).isEqualTo(rowCount); - } - - @Test - void javaWriter_rustReader_zstd_i64(@TempDir Path tmp) throws IOException { - // Given — ZstdEncoding: compressed primitive; exercises zstd frame magic + frame header - Path file = tmp.resolve("java_zstd_i64.vtx"); - long[] data = {1L, 2L, 3L, 4L, 1000L, 9999L, Long.MAX_VALUE}; - try (var ch = FileChannel.open(file, StandardOpenOption.CREATE, StandardOpenOption.WRITE); - var sut = VortexWriter.create(ch, TS_SCHEMA, WriteOptions.defaults(), - List.of(new ZstdEncoding()))) { - // When - sut.writeChunk(Map.of("ts", data)); - } - - // Then - long[] decoded = readLongColumn(file, "ts"); - assertThat(decoded).containsExactly(data); - } - - @Test - void javaWriter_rustReader_zstd_utf8(@TempDir Path tmp) throws IOException { - // Given — ZstdEncoding: compressed Utf8; exercises length-prefix framing + zstd magic - Path file = tmp.resolve("java_zstd_utf8.vtx"); - String[] data = {"hello", "world", "from", "zstd"}; - try (var ch = FileChannel.open(file, StandardOpenOption.CREATE, StandardOpenOption.WRITE); - var sut = VortexWriter.create(ch, STRING_SCHEMA, WriteOptions.defaults(), - List.of(new ZstdEncoding()))) { - // When - sut.writeChunk(Map.of("s", data)); - } - - // Then - String[] decoded = readStringColumn(file, "s"); - assertThat(decoded).containsExactly(data); - } - - @Test - void javaWriter_rustReader_list_i64(@TempDir Path tmp) throws IOException { - // Given — ListEncoding: exercises elements_len + offset_ptype proto fields (byte-order risk) - Path file = tmp.resolve("java_list_i64.vtx"); - long[] elements = {10L, 20L, 30L, 40L, 50L, 60L}; - long[] offsets = {0L, 2L, 2L, 5L, 6L}; // lists: [10,20], [], [30,40,50], [60] - ListData data = new ListData(elements, offsets, 4L); - try (var ch = FileChannel.open(file, StandardOpenOption.CREATE, StandardOpenOption.WRITE); - var sut = VortexWriter.create(ch, LIST_I64_SCHEMA, WriteOptions.defaults(), - List.of(new ListEncoding()))) { - // When - sut.writeChunk(Map.of("items", data)); - } - - // Then - long[] flatElements = readListLongColumn(file, "items"); - assertThat(flatElements).containsExactly(elements); - } - - @Test - void javaWriter_rustReader_listView_i64(@TempDir Path tmp) throws IOException { - // Given — ListViewEncoding: offset+size pairs in proto; exercises U16 vs U32 ptype risk - Path file = tmp.resolve("java_listview_i64.vtx"); - long[] elements = {1L, 2L, 3L, 4L, 5L}; - int[] offsets = {0, 2, 2, 4}; // lists start at these positions - int[] sizes = {2, 0, 2, 1}; // list lengths: [1,2], [], [3,4], [5] - ListViewData data = new ListViewData(elements, offsets, sizes, 4L); - try (var ch = FileChannel.open(file, StandardOpenOption.CREATE, StandardOpenOption.WRITE); - var sut = VortexWriter.create(ch, LIST_I64_SCHEMA, WriteOptions.defaults(), - List.of(new ListViewEncoding()))) { - // When - sut.writeChunk(Map.of("items", data)); - } - - // Then — Rust normalises ListView to List on read; verify flattened elements - long[] flatElements = readListLongColumn(file, "items"); - assertThat(flatElements).containsExactly(elements); - } - - // ── new reader helpers ──────────────────────────────────────────────────── - - private static boolean[] readBoolColumn(Path file, String column) throws IOException { - String uri = file.toAbsolutePath().toUri().toString(); - ScanOptions opts = ScanOptions.builder() - .projection(Expression.select(new String[]{column}, Expression.root())) - .build(); - var bools = new ArrayList(); - DataSource ds = DataSource.open(SESSION, uri); - Scan scan = ds.scan(opts); - while (scan.hasNext()) { - Partition partition = scan.next(); - try (ArrowReader reader = partition.scanArrow(ALLOCATOR)) { - while (reader.loadNextBatch()) { - VectorSchemaRoot root = reader.getVectorSchemaRoot(); - BitVector vec = (BitVector) root.getVector(column); - for (int i = 0; i < root.getRowCount(); i++) { - bools.add(!vec.isNull(i) && vec.get(i) == 1); - } - } - } - } - boolean[] result = new boolean[bools.size()]; - for (int i = 0; i < result.length; i++) { - result[i] = bools.get(i); - } - return result; - } - - private static long readRowCount(Path file) throws IOException { - String uri = file.toAbsolutePath().toUri().toString(); - long count = 0; - DataSource ds = DataSource.open(SESSION, uri); - Scan scan = ds.scan(ScanOptions.of()); - while (scan.hasNext()) { - Partition partition = scan.next(); - try (ArrowReader reader = partition.scanArrow(ALLOCATOR)) { - while (reader.loadNextBatch()) { - count += reader.getVectorSchemaRoot().getRowCount(); - } - } - } - return count; - } - - private static long[] readListLongColumn(Path file, String column) throws IOException { - String uri = file.toAbsolutePath().toUri().toString(); - ScanOptions opts = ScanOptions.builder() - .projection(Expression.select(new String[]{column}, Expression.root())) - .build(); - var elements = new ArrayList(); - DataSource ds = DataSource.open(SESSION, uri); - Scan scan = ds.scan(opts); - while (scan.hasNext()) { - Partition partition = scan.next(); - try (ArrowReader reader = partition.scanArrow(ALLOCATOR)) { - while (reader.loadNextBatch()) { - VectorSchemaRoot root = reader.getVectorSchemaRoot(); - ListVector vec = (ListVector) root.getVector(column); - BigIntVector child = (BigIntVector) vec.getDataVector(); - for (int i = 0; i < root.getRowCount(); i++) { - int start = vec.getOffsetBuffer().getInt((long) i * ListVector.OFFSET_WIDTH); - int end = vec.getOffsetBuffer().getInt((long) (i + 1) * ListVector.OFFSET_WIDTH); - for (int j = start; j < end; j++) { - elements.add(child.get(j)); - } - } - } - } - } - return elements.stream().mapToLong(Long::longValue).toArray(); - } - - private static void assertBitwiseEqualsF64(double[] actual, double[] expected) { - assertThat(actual).hasSize(expected.length); - for (int i = 0; i < expected.length; i++) { - assertThat(Double.doubleToRawLongBits(actual[i])) - .as("element %d", i) - .isEqualTo(Double.doubleToRawLongBits(expected[i])); - } - } - - private static void assertBitwiseEqualsF32(float[] actual, float[] expected) { - assertThat(actual).hasSize(expected.length); - for (int i = 0; i < expected.length; i++) { - assertThat(Float.floatToRawIntBits(actual[i])) - .as("element %d", i) - .isEqualTo(Float.floatToRawIntBits(expected[i])); - } - } - - private static void deleteDir(Path dir) throws IOException { - try (var walk = Files.walk(dir)) { - walk.sorted(Comparator.reverseOrder()).forEach(p -> { - try { - Files.delete(p); - } catch (IOException e) { - throw new UncheckedIOException(e); - } - }); - } - } -} diff --git a/integration/src/test/java/io/github/dfa1/vortex/integration/OhlcEncodingInspectionIntegrationTest.java b/integration/src/test/java/io/github/dfa1/vortex/integration/OhlcEncodingInspectionIntegrationTest.java deleted file mode 100644 index aee337c..0000000 --- a/integration/src/test/java/io/github/dfa1/vortex/integration/OhlcEncodingInspectionIntegrationTest.java +++ /dev/null @@ -1,137 +0,0 @@ -package io.github.dfa1.vortex.integration; - -import dev.vortex.api.Session; -import dev.vortex.api.VortexWriter; -import dev.vortex.arrow.ArrowAllocation; -import dev.vortex.jni.NativeLoader; -import io.github.dfa1.vortex.encoding.EncodingRegistry; -import io.github.dfa1.vortex.io.VortexInspector; -import io.github.dfa1.vortex.io.VortexReader; -import org.apache.arrow.c.ArrowArray; -import org.apache.arrow.c.ArrowSchema; -import org.apache.arrow.c.Data; -import org.apache.arrow.memory.BufferAllocator; -import org.apache.arrow.vector.BigIntVector; -import org.apache.arrow.vector.DateDayVector; -import org.apache.arrow.vector.Float8Vector; -import org.apache.arrow.vector.VarCharVector; -import org.apache.arrow.vector.VectorSchemaRoot; -import org.apache.arrow.vector.types.DateUnit; -import org.apache.arrow.vector.types.FloatingPointPrecision; -import org.apache.arrow.vector.types.pojo.ArrowType; -import org.apache.arrow.vector.types.pojo.Field; -import org.apache.arrow.vector.types.pojo.Schema; -import org.junit.jupiter.api.Test; -import org.junit.jupiter.api.io.TempDir; - -import java.io.IOException; -import java.nio.charset.StandardCharsets; -import java.nio.file.Path; -import java.time.LocalDate; -import java.util.HashMap; -import java.util.List; -import java.util.Random; - -import static org.assertj.core.api.Assertions.assertThat; - -/// Inspects the OHLC file (same schema and data as RustVsJavaReadBenchmark) -/// to reveal which encodings Rust chose for each column, especially "volume". -class OhlcEncodingInspectionIntegrationTest { - - private static final int ROWS = 200_000; - private static final int BATCH_SIZE = 50_000; - private static final Session SESSION = Session.create(); - private static final BufferAllocator ALLOCATOR = ArrowAllocation.rootAllocator(); - private static final ArrowType F64 = new ArrowType.FloatingPoint(FloatingPointPrecision.DOUBLE); - private static final Schema OHLC_SCHEMA = new Schema(List.of( - Field.notNullable("date", new ArrowType.Date(DateUnit.DAY)), - Field.notNullable("symbol", ArrowType.Utf8.INSTANCE), - Field.notNullable("open", F64), - Field.notNullable("high", F64), - Field.notNullable("low", F64), - Field.notNullable("close", F64), - Field.notNullable("volume", new ArrowType.Int(64, true)) - )); - - static { - NativeLoader.loadJni(); - } - - @Test - void inspect_ohlcFile_showsColumnEncodings(@TempDir Path tmp) throws IOException { - // Given - Path file = tmp.resolve("ohlc.vtx"); - writeOhlc(file, ROWS, BATCH_SIZE); - - // When - String report; - try (VortexReader vf = VortexReader.open(file, EncodingRegistry.loadAll())) { - report = VortexInspector.inspect(vf); - } - - // Then - System.out.println(report); - assertThat(report).contains("volume"); - assertThat(report).contains("Used encodings:"); - } - - private static double round(double v) { - return Math.round(v * 100.0) / 100.0; - } - - private static void writeOhlc(Path file, int totalRows, int batchSize) throws IOException { - String uri = file.toAbsolutePath().toUri().toString(); - var rng = new Random(42L); - double px = 100.0; - int day = (int) LocalDate.of(2020, 1, 2).toEpochDay(); - - try (VortexWriter writer = VortexWriter.create(SESSION, uri, OHLC_SCHEMA, new HashMap<>(), ALLOCATOR)) { - int rowsLeft = totalRows; - while (rowsLeft > 0) { - int n = Math.min(rowsLeft, batchSize); - try (VectorSchemaRoot root = VectorSchemaRoot.create(OHLC_SCHEMA, ALLOCATOR)) { - DateDayVector dateVec = (DateDayVector) root.getVector("date"); - VarCharVector symbolVec = (VarCharVector) root.getVector("symbol"); - Float8Vector openVec = (Float8Vector) root.getVector("open"); - Float8Vector highVec = (Float8Vector) root.getVector("high"); - Float8Vector lowVec = (Float8Vector) root.getVector("low"); - Float8Vector closeVec = (Float8Vector) root.getVector("close"); - BigIntVector volumeVec = (BigIntVector) root.getVector("volume"); - - dateVec.allocateNew(n); - symbolVec.allocateNew(n); - openVec.allocateNew(n); - highVec.allocateNew(n); - lowVec.allocateNew(n); - closeVec.allocateNew(n); - volumeVec.allocateNew(n); - - for (int i = 0; i < n; i++) { - double ret = rng.nextGaussian() * 0.02; - double o = round(px * (1 + ret * 0.3)); - double c = round(px * (1 + ret)); - double spread = Math.abs(px * rng.nextDouble() * 0.03); - double h = round(Math.max(o, c) + spread); - double l = round(Math.min(o, c) - spread); - dateVec.setSafe(i, day++); - symbolVec.setSafe(i, "ACME".getBytes(StandardCharsets.UTF_8)); - openVec.setSafe(i, o); - highVec.setSafe(i, h); - lowVec.setSafe(i, l); - closeVec.setSafe(i, c); - volumeVec.setSafe(i, Math.max(100_000L, Math.round(1_000_000 + rng.nextGaussian() * 200_000))); - px = c; - } - root.setRowCount(n); - - try (ArrowArray arr = ArrowArray.allocateNew(ALLOCATOR); - ArrowSchema schema = ArrowSchema.allocateNew(ALLOCATOR)) { - Data.exportVectorSchemaRoot(ALLOCATOR, root, null, arr, schema); - writer.writeBatch(arr.memoryAddress(), schema.memoryAddress()); - } - } - rowsLeft -= n; - } - } - } -} diff --git a/integration/src/test/java/io/github/dfa1/vortex/integration/OhlcGenerator.java b/integration/src/test/java/io/github/dfa1/vortex/integration/OhlcGenerator.java deleted file mode 100644 index 7d9aea5..0000000 --- a/integration/src/test/java/io/github/dfa1/vortex/integration/OhlcGenerator.java +++ /dev/null @@ -1,63 +0,0 @@ -package io.github.dfa1.vortex.integration; - -import java.time.LocalDate; -import java.util.ArrayList; -import java.util.Arrays; -import java.util.List; -import java.util.Random; - -class OhlcGenerator { - - private OhlcGenerator() {} - - static final String[] TICKERS = { - "AAPL", "MSFT", "GOOGL", "AMZN", "NVDA", "META", "TSLA", "BRK.B", "JPM", "V", - "UNH", "XOM", "LLY", "JNJ", "MA", "PG", "HD", "MRK", "AVGO", "CVX", - "PEP", "ABBV", "KO", "COST", "WMT", "MCD", "CSCO", "ACN", "BAC", "CRM" - }; - - record OhlcBatch(String[] symbols, int[] dates, double[] open, double[] high, - double[] low, double[] close, long[] volume) {} - - static List generate(int totalRows, int batchSize) { - return generate(totalRows, batchSize, 42L); - } - - static List generate(int totalRows, int batchSize, long seed) { - double[] prices = new double[TICKERS.length]; - Arrays.fill(prices, 100.0); - var rng = new Random(seed); - int startDay = (int) LocalDate.of(2020, 1, 2).toEpochDay(); - int rowsLeft = totalRows; - int rowsDone = 0; - var batches = new ArrayList(); - - while (rowsLeft > 0) { - int n = Math.min(rowsLeft, batchSize); - String[] symbols = new String[n]; - int[] dates = new int[n]; - double[] open = new double[n], high = new double[n], low = new double[n], close = new double[n]; - long[] volume = new long[n]; - for (int i = 0; i < n; i++) { - int ticker = i % TICKERS.length; - double px = prices[ticker]; - double ret = rng.nextGaussian() * 0.02; - double o = Math.round(px * (1 + ret * 0.3) * 100.0) * 0.01; - double c = Math.round(px * (1 + ret) * 100.0) * 0.01; - double spread = Math.abs(px * rng.nextDouble() * 0.03); - symbols[i] = TICKERS[ticker]; - dates[i] = startDay + (rowsDone + i) / TICKERS.length; - open[i] = o; - high[i] = Math.round((Math.max(o, c) + spread) * 100.0) * 0.01; - low[i] = Math.round((Math.min(o, c) - spread) * 100.0) * 0.01; - close[i] = c; - volume[i] = Math.max(100_000L, Math.round(1_000_000 + rng.nextGaussian() * 200_000)); - prices[ticker] = c; - } - batches.add(new OhlcBatch(symbols, dates, open, high, low, close, volume)); - rowsLeft -= n; - rowsDone += n; - } - return batches; - } -} diff --git a/integration/src/test/java/io/github/dfa1/vortex/integration/ParquetImportIntegrationTest.java b/integration/src/test/java/io/github/dfa1/vortex/integration/ParquetImportIntegrationTest.java deleted file mode 100644 index 4b7b18a..0000000 --- a/integration/src/test/java/io/github/dfa1/vortex/integration/ParquetImportIntegrationTest.java +++ /dev/null @@ -1,176 +0,0 @@ -package io.github.dfa1.vortex.integration; - -import dev.hardwood.InputFile; -import dev.hardwood.reader.ParquetFileReader; -import dev.hardwood.reader.RowReader; -import io.github.dfa1.vortex.core.DType; -import io.github.dfa1.vortex.core.array.LongArray; -import io.github.dfa1.vortex.core.array.VarBinArray; -import io.github.dfa1.vortex.io.VortexReader; -import io.github.dfa1.vortex.parquet.ParquetImporter; -import io.github.dfa1.vortex.scan.ScanIterator; -import io.github.dfa1.vortex.scan.ScanOptions; -import io.github.dfa1.vortex.scan.ScanResult; -import org.junit.jupiter.api.Test; -import org.junit.jupiter.api.io.TempDir; - -import java.net.URI; -import java.nio.file.Files; -import java.nio.file.Path; -import java.nio.file.StandardCopyOption; -import java.util.ArrayList; -import java.util.List; - -import static org.assertj.core.api.Assertions.assertThat; -import static org.junit.jupiter.api.Assumptions.assumeTrue; - -/// Round-trip: Parquet (via Hardwood) → Vortex (via ParquetImporter) → VortexReader. -/// -/// Fixture: delta_encoding_optional_column.parquet from apache/parquet-testing. -/// TPC-DS customer table, 100 rows, INT64 + STRING columns, all nullable. -class ParquetImportIntegrationTest { - - private static final String FIXTURE_URL = - "https://raw.githubusercontent.com/apache/parquet-testing/master/data/" - + "delta_encoding_optional_column.parquet"; - - private static final int EXPECTED_ROWS = 100; - - @Test - void rowCountAndColumnNamesMatch(@TempDir Path tmp) throws Exception { - // Given - assumeNetworkAvailable(); - Path parquetFile = download(tmp, "delta_encoding_optional_column.parquet"); - Path vortexFile = tmp.resolve("out.vortex"); - - // When - ParquetImporter.importParquet(parquetFile, vortexFile); - - // Then - List parquetColumns = parquetColumnNames(parquetFile); - try (VortexReader reader = VortexReader.open(vortexFile)) { - assertThat(reader.dtype()).isInstanceOf(DType.Struct.class); - DType.Struct schema = (DType.Struct) reader.dtype(); - assertThat(schema.fieldNames()).containsExactlyElementsOf(parquetColumns); - - long vortexRows = countRows(reader); - assertThat(vortexRows).isEqualTo(EXPECTED_ROWS); - } - } - - @Test - void longColumnValuesMatch(@TempDir Path tmp) throws Exception { - // Given - assumeNetworkAvailable(); - Path parquetFile = download(tmp, "delta_encoding_optional_column.parquet"); - Path vortexFile = tmp.resolve("out.vortex"); - - // When - ParquetImporter.importParquet(parquetFile, vortexFile); - - // Then — c_customer_sk (INT64, nullable): first 3 values are 100, 99, 98 - try (VortexReader reader = VortexReader.open(vortexFile); - ScanIterator iter = reader.scan(ScanOptions.all())) { - assertThat(iter.hasNext()).isTrue(); - ScanResult first = iter.next(); - LongArray col = first.column("c_customer_sk"); - assertThat(col.getLong(0)).isEqualTo(100L); - assertThat(col.getLong(1)).isEqualTo(99L); - assertThat(col.getLong(2)).isEqualTo(98L); - } - } - - @Test - void stringColumnValuesMatch(@TempDir Path tmp) throws Exception { - // Given - assumeNetworkAvailable(); - Path parquetFile = download(tmp, "delta_encoding_optional_column.parquet"); - Path vortexFile = tmp.resolve("out.vortex"); - - // When - ParquetImporter.importParquet(parquetFile, vortexFile); - - // Then — c_first_name (STRING, nullable): first 3 values - try (VortexReader reader = VortexReader.open(vortexFile); - ScanIterator iter = reader.scan(ScanOptions.all())) { - assertThat(iter.hasNext()).isTrue(); - ScanResult first = iter.next(); - VarBinArray col = first.column("c_first_name"); - assertThat(col.getString(0)).isEqualTo("Jeannette"); - assertThat(col.getString(1)).isEqualTo("Austin"); - assertThat(col.getString(2)).isEqualTo("David"); - } - } - - @Test - void parquetAndVortexRowCountsAreEqual(@TempDir Path tmp) throws Exception { - // Given - assumeNetworkAvailable(); - Path parquetFile = download(tmp, "delta_encoding_optional_column.parquet"); - Path vortexFile = tmp.resolve("out.vortex"); - - // When - ParquetImporter.importParquet(parquetFile, vortexFile); - - // Then — same row count when reading both with their native reader - long parquetRows = countParquetRows(parquetFile); - long vortexRows; - try (VortexReader reader = VortexReader.open(vortexFile)) { - vortexRows = countRows(reader); - } - assertThat(vortexRows).isEqualTo(parquetRows); - } - - // ── helpers ────────────────────────────────────────────────────────────── - - private static long countRows(VortexReader reader) { - long total = 0; - try (ScanIterator iter = reader.scan(ScanOptions.all())) { - while (iter.hasNext()) { - total += iter.next().rowCount(); - } - } - return total; - } - - private static long countParquetRows(Path path) throws Exception { - long total = 0; - try (ParquetFileReader parquet = ParquetFileReader.open(InputFile.of(path)); - RowReader rowReader = parquet.rowReader()) { - while (rowReader.hasNext()) { - rowReader.next(); - total++; - } - } - return total; - } - - private static List parquetColumnNames(Path path) throws Exception { - try (ParquetFileReader parquet = ParquetFileReader.open(InputFile.of(path))) { - List names = new ArrayList<>(); - parquet.getFileSchema().getColumns() - .forEach(col -> names.add(col.name())); - return names; - } - } - - private static Path download(Path tmp, String name) throws Exception { - Path cached = Path.of("/tmp/parquet-fixtures", name); - if (Files.exists(cached)) { - return cached; - } - Path dest = tmp.resolve(name); - try (var in = URI.create(FIXTURE_URL).toURL().openStream()) { - Files.copy(in, dest, StandardCopyOption.REPLACE_EXISTING); - } - return dest; - } - - private static void assumeNetworkAvailable() { - try { - URI.create("https://raw.githubusercontent.com").toURL().openStream().close(); - } catch (Exception e) { - assumeTrue(false, "no network"); - } - } -} diff --git a/integration/src/test/java/io/github/dfa1/vortex/integration/PcoFixtureInspectionIntegrationTest.java b/integration/src/test/java/io/github/dfa1/vortex/integration/PcoFixtureInspectionIntegrationTest.java deleted file mode 100644 index ae1b561..0000000 --- a/integration/src/test/java/io/github/dfa1/vortex/integration/PcoFixtureInspectionIntegrationTest.java +++ /dev/null @@ -1,356 +0,0 @@ -package io.github.dfa1.vortex.integration; - -import io.github.dfa1.vortex.core.DType; -import io.github.dfa1.vortex.core.Layout; -import io.github.dfa1.vortex.core.SegmentSpec; -import io.github.dfa1.vortex.encoding.EncodingRegistry; -import io.github.dfa1.vortex.fbs.Array; -import io.github.dfa1.vortex.fbs.ArrayNode; -import io.github.dfa1.vortex.fbs.Buffer; -import io.github.dfa1.vortex.io.VortexReader; -import org.junit.jupiter.api.Test; -import org.junit.jupiter.api.io.TempDir; - -import java.lang.foreign.MemorySegment; -import java.net.URI; -import java.nio.ByteBuffer; -import java.nio.ByteOrder; -import java.nio.file.Files; -import java.nio.file.Path; -import java.nio.file.StandardCopyOption; -import java.util.ArrayList; -import java.util.HashMap; -import java.util.List; -import java.util.Map; -import java.util.TreeMap; - -import static org.junit.jupiter.api.Assumptions.assumeTrue; - -/// Phase 0 scoping for `vortex.pco` decode port. -/// -/// Inspects each pco-blocked fixture: walks layout, locates `vortex.pco`-encoded -/// `ArrayNode`s, hand-parses the `PcoMetadata` proto (no generated stub needed yet), -/// and dumps mode/delta nibbles from the first chunk_meta buffer. -/// -/// Output drives the Phase 5 scope decision (which pco modes / delta encodings to -/// implement first). -class PcoFixtureInspectionIntegrationTest { - - private static final String BASE = "https://vortex-compat-fixtures.s3.amazonaws.com/v0.72.0/arrays/"; - private static final String[] FIXTURES = { - "pco.vortex", - "tpch_lineitem.compact.vortex", - "tpch_orders.compact.vortex", - "clickbench_hits_5k.compact.vortex" - }; - - @Test - void dumpPcoMetadataForAllBlockedFixtures(@TempDir Path tmp) throws Exception { - assumeNetworkAvailable(); - StringBuilder report = new StringBuilder(); - for (String name : FIXTURES) { - Path local = downloadIfMissing(tmp, name); - report.append("=== ").append(name).append(" ===\n"); - inspect(local, report); - report.append('\n'); - } - System.out.println(report); - } - - private static void inspect(Path file, StringBuilder out) throws Exception { - try (VortexReader vf = VortexReader.open(file, EncodingRegistry.empty())) { - out.append("dtype: ").append(formatDType(vf.dtype())).append('\n'); - out.append("size: ").append(vf.fileSize()).append(" bytes\n"); - - List arraySpecs = vf.footer().arraySpecs(); - List segmentSpecs = vf.footer().segmentSpecs(); - - PcoStats stats = new PcoStats(); - walkLayout(vf, vf.layout(), arraySpecs, segmentSpecs, stats, columnPath(vf.dtype())); - - out.append("pco arrays found: ").append(stats.pcoArrayCount).append('\n'); - out.append("pco array column paths: ").append(stats.columnPaths).append('\n'); - out.append("pco array ptypes: ").append(stats.ptypeCounts).append('\n'); - out.append("header version bytes seen: ").append(stats.headerVersions).append('\n'); - out.append("chunks per pco array: ").append(stats.chunksPerArray).append('\n'); - out.append("pages per pco array: ").append(stats.pagesPerArray).append('\n'); - out.append("n_values per page (sample first 10): ").append(stats.firstPageSizes).append('\n'); - out.append("first chunk_meta bytes (hex, first 16) per array:\n"); - for (String hex : stats.firstChunkMetaHexes) { - out.append(" ").append(hex).append('\n'); - } - out.append("decoded leading nibbles (mode | delta as 4b):\n"); - for (String n : stats.firstChunkMetaNibbles) { - out.append(" ").append(n).append('\n'); - } - } - } - - private static List columnPath(DType root) { - List names = new ArrayList<>(); - if (root instanceof DType.Struct s) { - names.addAll(s.fieldNames()); - } else { - names.add(""); - } - return names; - } - - private static void walkLayout(VortexReader vf, Layout layout, List arraySpecs, - List segmentSpecs, PcoStats stats, List colNames) { - walkLayoutInner(vf, layout, arraySpecs, segmentSpecs, stats, colNames, ""); - } - - private static void walkLayoutInner(VortexReader vf, Layout layout, List arraySpecs, - List segmentSpecs, PcoStats stats, - List colNames, String currentPath) { - if (layout.isStruct()) { - for (int i = 0; i < layout.children().size(); i++) { - String name = i < colNames.size() ? colNames.get(i) : "col" + i; - walkLayoutInner(vf, layout.children().get(i), arraySpecs, segmentSpecs, stats, List.of(), name); - } - return; - } - if ((layout.isFlat() || layout.isDict()) && !layout.segments().isEmpty()) { - int segIdx = layout.segments().getFirst(); - SegmentSpec spec = segmentSpecs.get(segIdx); - MemorySegment seg = vf.slice(spec.offset(), spec.length()); - scanFlatSegment(seg, arraySpecs, stats, currentPath); - return; - } - for (Layout child : layout.children()) { - walkLayoutInner(vf, child, arraySpecs, segmentSpecs, stats, colNames, currentPath); - } - } - - private static void scanFlatSegment(MemorySegment seg, List arraySpecs, - PcoStats stats, String columnPath) { - int segLen = (int) seg.byteSize(); - ByteBuffer bb = seg.asByteBuffer().order(ByteOrder.LITTLE_ENDIAN); - int fbLen = bb.getInt(segLen - 4); - int fbStart = segLen - 4 - fbLen; - ByteBuffer fbBuf = bb.slice(fbStart, fbLen).order(ByteOrder.LITTLE_ENDIAN); - Array fbArray = Array.getRootAsArray(fbBuf); - - // Materialize per-buffer offsets (need to read chunk_meta buffer for pco arrays). - int numBuffers = fbArray.buffersLength(); - long[] bufOffsets = new long[numBuffers]; - long[] bufLengths = new long[numBuffers]; - long dataOffset = 0; - for (int i = 0; i < numBuffers; i++) { - Buffer bufDesc = fbArray.buffers(i); - dataOffset += bufDesc.padding(); - bufOffsets[i] = dataOffset; - bufLengths[i] = bufDesc.length(); - dataOffset += bufDesc.length(); - } - - visitNode(fbArray.root(), arraySpecs, seg, bufOffsets, bufLengths, stats, columnPath); - } - - private static void visitNode(ArrayNode node, List arraySpecs, MemorySegment seg, - long[] bufOffsets, long[] bufLengths, PcoStats stats, String columnPath) { - if (node == null) { - return; - } - String encId = arraySpecs.get(node.encoding()); - if ("vortex.pco".equals(encId)) { - recordPco(node, seg, bufOffsets, bufLengths, stats, columnPath); - } - for (int i = 0; i < node.childrenLength(); i++) { - visitNode(node.children(i), arraySpecs, seg, bufOffsets, bufLengths, stats, columnPath); - } - } - - private static void recordPco(ArrayNode node, MemorySegment seg, - long[] bufOffsets, long[] bufLengths, - PcoStats stats, String columnPath) { - stats.pcoArrayCount++; - stats.columnPaths.add(columnPath); - - ByteBuffer meta = node.metadataAsByteBuffer(); - ByteBuffer metaSlice = meta != null ? meta.slice() : null; - PcoMeta pm = metaSlice != null ? parsePcoMetadata(metaSlice) : new PcoMeta(); - stats.chunksPerArray.add(pm.chunkCount); - stats.pagesPerArray.add(pm.pageCount); - if (!pm.firstPageSizes.isEmpty() && stats.firstPageSizes.size() < 10) { - stats.firstPageSizes.add(pm.firstPageSizes.get(0)); - } - if (pm.headerBytes != null) { - stats.headerVersions.merge(hex(pm.headerBytes), 1, Integer::sum); - } - - // First chunk_meta buffer is bufferIndices[0] (Vortex layer puts chunk_metas first). - if (node.buffersLength() > 0) { - int bufIdx = node.buffers(0); - if (bufIdx < bufOffsets.length && bufLengths[bufIdx] > 0) { - int n = (int) Math.min(16, bufLengths[bufIdx]); - byte[] first = new byte[n]; - MemorySegment slice = seg.asSlice(bufOffsets[bufIdx], n); - for (int i = 0; i < n; i++) { - first[i] = slice.get(java.lang.foreign.ValueLayout.JAVA_BYTE, i); - } - stats.firstChunkMetaHexes.add(columnPath + ": " + hex(first)); - int b0 = first[0] & 0xFF; - int lowMode = b0 & 0x0F; - int highDelta = (b0 >>> 4) & 0x0F; - stats.firstChunkMetaNibbles.add(columnPath - + ": mode_nibble=0x" + Integer.toHexString(lowMode) - + " next_nibble=0x" + Integer.toHexString(highDelta)); - } - } - } - - /// Minimal protobuf wire-format parse for PcoMetadata. - /// Schema: - /// message PcoMetadata { bytes header = 1; repeated PcoChunkInfo chunks = 2; } - /// message PcoChunkInfo { repeated PcoPageInfo pages = 1; } - /// message PcoPageInfo { uint32 n_values = 1; } - private static PcoMeta parsePcoMetadata(ByteBuffer buf) { - PcoMeta out = new PcoMeta(); - buf.order(ByteOrder.LITTLE_ENDIAN); - while (buf.hasRemaining()) { - int tag = (int) readVarint(buf); - int fieldNum = tag >>> 3; - int wireType = tag & 0x7; - if (fieldNum == 1 && wireType == 2) { - int len = (int) readVarint(buf); - out.headerBytes = new byte[len]; - buf.get(out.headerBytes); - } else if (fieldNum == 2 && wireType == 2) { - int len = (int) readVarint(buf); - ByteBuffer chunkBuf = buf.slice(); - chunkBuf.limit(len); - buf.position(buf.position() + len); - out.chunkCount++; - parseChunkInfo(chunkBuf, out); - } else { - skipField(buf, wireType); - } - } - return out; - } - - private static void parseChunkInfo(ByteBuffer buf, PcoMeta out) { - while (buf.hasRemaining()) { - int tag = (int) readVarint(buf); - int fieldNum = tag >>> 3; - int wireType = tag & 0x7; - if (fieldNum == 1 && wireType == 2) { - int len = (int) readVarint(buf); - ByteBuffer pageBuf = buf.slice(); - pageBuf.limit(len); - buf.position(buf.position() + len); - out.pageCount++; - parsePageInfo(pageBuf, out); - } else { - skipField(buf, wireType); - } - } - } - - private static void parsePageInfo(ByteBuffer buf, PcoMeta out) { - while (buf.hasRemaining()) { - int tag = (int) readVarint(buf); - int fieldNum = tag >>> 3; - int wireType = tag & 0x7; - if (fieldNum == 1 && wireType == 0) { - int nv = (int) readVarint(buf); - out.firstPageSizes.add(nv); - } else { - skipField(buf, wireType); - } - } - } - - private static void skipField(ByteBuffer buf, int wireType) { - switch (wireType) { - case 0 -> readVarint(buf); - case 1 -> buf.position(buf.position() + 8); - case 2 -> { - int len = (int) readVarint(buf); - buf.position(buf.position() + len); - } - case 5 -> buf.position(buf.position() + 4); - default -> throw new IllegalStateException("unknown wire type " + wireType); - } - } - - private static long readVarint(ByteBuffer buf) { - long result = 0; - int shift = 0; - while (true) { - byte b = buf.get(); - result |= ((long) (b & 0x7F)) << shift; - if ((b & 0x80) == 0) { - return result; - } - shift += 7; - } - } - - private static String hex(byte[] bytes) { - StringBuilder sb = new StringBuilder(bytes.length * 3); - for (int i = 0; i < bytes.length; i++) { - if (i > 0) { - sb.append(' '); - } - sb.append(String.format("%02x", bytes[i] & 0xFF)); - } - return sb.toString(); - } - - private static String formatDType(DType d) { - if (d instanceof DType.Struct s) { - StringBuilder sb = new StringBuilder("struct{"); - for (int i = 0; i < s.fieldNames().size(); i++) { - if (i > 0) { - sb.append(", "); - } - sb.append(s.fieldNames().get(i)).append(':').append(s.fieldTypes().get(i)); - } - return sb.append('}').toString(); - } - return String.valueOf(d); - } - - private static Path downloadIfMissing(Path tmp, String name) throws Exception { - // Reuse /tmp/pco-fixtures cache if present. - Path cached = Path.of("/tmp/pco-fixtures", name); - if (Files.exists(cached)) { - return cached; - } - Path dest = tmp.resolve(name); - try (var in = URI.create(BASE + name).toURL().openStream()) { - Files.copy(in, dest, StandardCopyOption.REPLACE_EXISTING); - } - return dest; - } - - private static void assumeNetworkAvailable() { - try { - URI.create("https://vortex-compat-fixtures.s3.amazonaws.com").toURL().openStream().close(); - } catch (Exception e) { - assumeTrue(false, "no network"); - } - } - - private static final class PcoStats { - int pcoArrayCount; - final List columnPaths = new ArrayList<>(); - final Map ptypeCounts = new TreeMap<>(); - final Map headerVersions = new HashMap<>(); - final List chunksPerArray = new ArrayList<>(); - final List pagesPerArray = new ArrayList<>(); - final List firstPageSizes = new ArrayList<>(); - final List firstChunkMetaHexes = new ArrayList<>(); - final List firstChunkMetaNibbles = new ArrayList<>(); - } - - private static final class PcoMeta { - byte[] headerBytes; - int chunkCount; - int pageCount; - final List firstPageSizes = new ArrayList<>(); - } -} diff --git a/integration/src/test/java/io/github/dfa1/vortex/integration/RandomArrays.java b/integration/src/test/java/io/github/dfa1/vortex/integration/RandomArrays.java deleted file mode 100644 index 8009735..0000000 --- a/integration/src/test/java/io/github/dfa1/vortex/integration/RandomArrays.java +++ /dev/null @@ -1,125 +0,0 @@ -package io.github.dfa1.vortex.integration; - -import java.util.LinkedHashSet; -import java.util.Random; -import java.util.Set; -import java.util.stream.IntStream; -import java.util.stream.Stream; - -final class RandomArrays { - - private static final long SEED = 0xCAFEBABEL; - - static Stream i64Arrays(int count) { - Random rng = new Random(SEED); - return IntStream.range(0, count).mapToObj(i -> { - int size = rng.nextInt(10_001); - long[] arr = new long[size]; - for (int j = 0; j < size; j++) { - arr[j] = rng.nextLong(); - } - return arr; - }); - } - - static Stream f64Arrays(int count) { - Random rng = new Random(SEED); - return IntStream.range(0, count).mapToObj(i -> { - int size = rng.nextInt(5_001); - double[] arr = new double[size]; - for (int j = 0; j < size; j++) { - double d; - do { - d = Double.longBitsToDouble(rng.nextLong()); - } while (!Double.isFinite(d)); - arr[j] = d; - } - return arr; - }); - } - - static Stream f32Arrays(int count) { - Random rng = new Random(SEED); - return IntStream.range(0, count).mapToObj(i -> { - int size = rng.nextInt(5_001); - float[] arr = new float[size]; - for (int j = 0; j < size; j++) { - float f; - do { - f = Float.intBitsToFloat(rng.nextInt()); - } while (!Float.isFinite(f)); - arr[j] = f; - } - return arr; - }); - } - - static Stream asciiStringArrays(int count) { - Random rng = new Random(SEED); - return IntStream.range(0, count).mapToObj(i -> { - int size = rng.nextInt(5_001); - String[] arr = new String[size]; - for (int j = 0; j < size; j++) { - arr[j] = randomAlphaString(rng, 0, 100); - } - return arr; - }); - } - - static Stream varBinViewStringArrays(int count) { - Random rng = new Random(SEED); - return IntStream.range(0, count).mapToObj(i -> { - int size = rng.nextInt(1_001); - String[] arr = new String[size]; - for (int j = 0; j < size; j++) { - arr[j] = randomAlphaString(rng, 0, 30); - } - return arr; - }); - } - - static Stream u16DictStringArrays(int count) { - Random rng = new Random(SEED); - return IntStream.range(0, count).mapToObj(i -> { - int targetSize = 257 + rng.nextInt(5_000 - 257 + 1); - Set unique = new LinkedHashSet<>(); - while (unique.size() < targetSize) { - unique.add(randomAlphaString(rng, 3, 20)); - } - return unique.toArray(String[]::new); - }); - } - - static Stream unicodeStringArrays(int count) { - Random rng = new Random(SEED); - return IntStream.range(0, count).mapToObj(i -> { - int size = rng.nextInt(1_001); - StringBuilder sb = new StringBuilder(); - String[] arr = new String[size]; - for (int j = 0; j < size; j++) { - arr[j] = randomCjkString(rng, 0, 50, sb); - } - return arr; - }); - } - - private static String randomAlphaString(Random rng, int minLen, int maxLen) { - int len = minLen + (maxLen > minLen ? rng.nextInt(maxLen - minLen + 1) : 0); - char[] chars = new char[len]; - for (int i = 0; i < len; i++) { - chars[i] = (char) ('a' + rng.nextInt(26)); - } - return new String(chars); - } - - private static String randomCjkString(Random rng, int minLen, int maxLen, StringBuilder sb) { - int len = minLen + (maxLen > minLen ? rng.nextInt(maxLen - minLen + 1) : 0); - sb.setLength(0); - for (int i = 0; i < len; i++) { - sb.appendCodePoint('一' + rng.nextInt('퟿' - '一' + 1)); - } - return sb.toString(); - } - - private RandomArrays() {} -} diff --git a/integration/src/test/java/io/github/dfa1/vortex/integration/RustJavaReaderComparisonIntegrationTest.java b/integration/src/test/java/io/github/dfa1/vortex/integration/RustJavaReaderComparisonIntegrationTest.java deleted file mode 100644 index f027a07..0000000 --- a/integration/src/test/java/io/github/dfa1/vortex/integration/RustJavaReaderComparisonIntegrationTest.java +++ /dev/null @@ -1,422 +0,0 @@ -package io.github.dfa1.vortex.integration; - -import dev.vortex.api.DataSource; -import dev.vortex.api.Partition; -import dev.vortex.api.Scan; -import dev.vortex.api.ScanOptions; -import dev.vortex.api.Session; -import dev.vortex.arrow.ArrowAllocation; -import dev.vortex.jni.NativeLoader; -import io.github.dfa1.vortex.core.DType; -import io.github.dfa1.vortex.core.PType; -import io.github.dfa1.vortex.core.array.Array; -import io.github.dfa1.vortex.core.array.ByteArray; -import io.github.dfa1.vortex.core.array.DoubleArray; -import io.github.dfa1.vortex.core.array.FloatArray; -import io.github.dfa1.vortex.core.array.IntArray; -import io.github.dfa1.vortex.core.array.LongArray; -import io.github.dfa1.vortex.core.array.MaskedArray; -import io.github.dfa1.vortex.core.array.ShortArray; -import io.github.dfa1.vortex.core.array.VarBinArray; -import io.github.dfa1.vortex.encoding.EncodingRegistry; -import io.github.dfa1.vortex.io.VortexReader; -import io.github.dfa1.vortex.scan.ScanResult; -import org.apache.arrow.memory.BufferAllocator; -import org.apache.arrow.vector.BigIntVector; -import org.apache.arrow.vector.Float4Vector; -import org.apache.arrow.vector.Float8Vector; -import org.apache.arrow.vector.IntVector; -import org.apache.arrow.vector.SmallIntVector; -import org.apache.arrow.vector.TinyIntVector; -import org.apache.arrow.vector.UInt1Vector; -import org.apache.arrow.vector.UInt2Vector; -import org.apache.arrow.vector.UInt4Vector; -import org.apache.arrow.vector.UInt8Vector; -import org.apache.arrow.vector.VarCharVector; -import org.apache.arrow.vector.ipc.ArrowReader; -import org.junit.jupiter.api.io.TempDir; -import org.junit.jupiter.params.ParameterizedTest; -import org.junit.jupiter.params.provider.ValueSource; - -import java.net.URI; -import java.net.http.HttpClient; -import java.net.http.HttpRequest; -import java.net.http.HttpResponse; -import java.nio.file.Path; -import java.util.LinkedHashMap; -import java.util.Map; - -import static org.assertj.core.api.Assertions.assertThat; -import static org.assertj.core.data.Percentage.withPercentage; - -/// Cross-decoder correctness: downloads each S3 fixture once, then compares row counts, -/// numeric column sums, and string byte-length sums from the Rust reader and the -/// Java {@link VortexReader}. -/// -/// Both readers decode the same local bytes — no auth, no network dependency during -/// decode. A mismatch in any column value points to a decoding bug in the Java reader. -class RustJavaReaderComparisonIntegrationTest { - - private static final URI BASE = - URI.create("https://vortex-compat-fixtures.s3.amazonaws.com/v0.72.0/arrays/"); - - private static final Session SESSION = Session.create(); - private static final BufferAllocator ALLOCATOR = ArrowAllocation.rootAllocator(); - private static final HttpClient HTTP = HttpClient.newHttpClient(); - - static { - NativeLoader.loadJni(); - } - - private record Stats(long rowCount, Map numSums, Map strLenSums) {} - - @ParameterizedTest(name = "{0}") - @ValueSource(strings = { - "alp.vortex", - "alprd.vortex", - "bitpacked.vortex", - "booleans.vortex", - "bytebool.vortex", - "chunked.vortex", - "constant.vortex", - "datetime.vortex", - "datetimeparts.vortex", - "decimal.vortex", - "decimal_byte_parts.vortex", - "dict.vortex", - "fixed_size_list.vortex", - "for.vortex", - "fsst.vortex", - "null.vortex", - "primitives.vortex", - "rle.vortex", - "runend.vortex", - "sequence.vortex", - "sparse.vortex", - "struct_nested.vortex", - "varbin.vortex", - "varbinview.vortex", - "zigzag.vortex", - // zstd.vortex excluded: ZstdEncoding dictionary mode not yet implemented - }) - void rust_vs_javaReader_statsMatch(String fixture, @TempDir Path tmp) throws Exception { - // Given - Path local = download(BASE.resolve(fixture), tmp); - - // When - Stats rustStats = rustStats(local); - Stats javaStats = javaStats(local); - - // Then — row counts match - assertThat(javaStats.rowCount()) - .as("row count in %s", fixture) - .isEqualTo(rustStats.rowCount()); - - // Then — numeric column sums match (skip if no numeric cols in Rust output) - if (!rustStats.numSums().isEmpty()) { - assertThat(javaStats.numSums().keySet()) - .as("numeric column names in %s", fixture) - .containsExactlyInAnyOrderElementsOf(rustStats.numSums().keySet()); - for (Map.Entry entry : rustStats.numSums().entrySet()) { - assertThat(javaStats.numSums().get(entry.getKey())) - .describedAs("numeric column '%s' sum in %s", entry.getKey(), fixture) - .isCloseTo(entry.getValue(), withPercentage(0.001)); - } - } - - // Then — string byte-length sums match for cols Rust tracks (Rust may miss LargeVarChar etc.) - if (!rustStats.strLenSums().isEmpty()) { - assertThat(javaStats.strLenSums().keySet()) - .as("string column names in %s", fixture) - .containsAll(rustStats.strLenSums().keySet()); - for (Map.Entry entry : rustStats.strLenSums().entrySet()) { - assertThat(javaStats.strLenSums().get(entry.getKey())) - .describedAs("string column '%s' byte-length sum in %s", entry.getKey(), fixture) - .isEqualTo(entry.getValue()); - } - } - } - - // ── download ───────────────────────────────────────────────────────────── - - private static Path download(URI uri, Path dir) throws Exception { - String name = uri.getPath().substring(uri.getPath().lastIndexOf('/') + 1); - Path file = dir.resolve(name); - HTTP.send( - HttpRequest.newBuilder(uri).GET().build(), - HttpResponse.BodyHandlers.ofFile(file) - ); - return file; - } - - // ── Rust side ──────────────────────────────────────────────────────────── - - private static Stats rustStats(Path file) throws Exception { - Map numSums = new LinkedHashMap<>(); - Map strLenSums = new LinkedHashMap<>(); - long rowCount = 0; - String uri = file.toAbsolutePath().toUri().toString(); - DataSource ds = DataSource.open(SESSION, uri); - Scan scan = ds.scan(ScanOptions.of()); - while (scan.hasNext()) { - Partition partition = scan.next(); - try (ArrowReader reader = partition.scanArrow(ALLOCATOR)) { - while (reader.loadNextBatch()) { - var root = reader.getVectorSchemaRoot(); - rowCount += root.getRowCount(); - for (var field : root.getSchema().getFields()) { - var vec = root.getVector(field.getName()); - switch (vec) { - case BigIntVector v -> { - long s = 0; - for (int i = 0; i < root.getRowCount(); i++) { - if (!v.isNull(i)) { - s += v.get(i); - } - } - numSums.merge(field.getName(), (double) s, Double::sum); - } - case IntVector v -> { - long s = 0; - for (int i = 0; i < root.getRowCount(); i++) { - if (!v.isNull(i)) { - s += v.get(i); - } - } - numSums.merge(field.getName(), (double) s, Double::sum); - } - case SmallIntVector v -> { - long s = 0; - for (int i = 0; i < root.getRowCount(); i++) { - if (!v.isNull(i)) { - s += v.get(i); - } - } - numSums.merge(field.getName(), (double) s, Double::sum); - } - case TinyIntVector v -> { - long s = 0; - for (int i = 0; i < root.getRowCount(); i++) { - if (!v.isNull(i)) { - s += v.get(i); - } - } - numSums.merge(field.getName(), (double) s, Double::sum); - } - case UInt1Vector v -> { - long s = 0; - for (int i = 0; i < root.getRowCount(); i++) { - if (!v.isNull(i)) { - s += v.getValueAsLong(i); - } - } - numSums.merge(field.getName(), (double) s, Double::sum); - } - case Float8Vector v -> { - double s = 0; - for (int i = 0; i < root.getRowCount(); i++) { - if (!v.isNull(i)) { - s += v.get(i); - } - } - numSums.merge(field.getName(), s, Double::sum); - } - case Float4Vector v -> { - double s = 0; - for (int i = 0; i < root.getRowCount(); i++) { - if (!v.isNull(i)) { - s += v.get(i); - } - } - numSums.merge(field.getName(), s, Double::sum); - } - case UInt2Vector v -> { - long s = 0; - for (int i = 0; i < root.getRowCount(); i++) { - if (!v.isNull(i)) { - s += v.getValueAsLong(i); - } - } - numSums.merge(field.getName(), (double) s, Double::sum); - } - case UInt4Vector v -> { - long s = 0; - for (int i = 0; i < root.getRowCount(); i++) { - if (!v.isNull(i)) { - s += v.getValueAsLong(i); - } - } - numSums.merge(field.getName(), (double) s, Double::sum); - } - case UInt8Vector v -> { - long s = 0; - for (int i = 0; i < root.getRowCount(); i++) { - if (!v.isNull(i)) { - s += v.getValueAsLong(i); - } - } - numSums.merge(field.getName(), (double) s, Double::sum); - } - case VarCharVector v -> { - long s = 0; - for (int i = 0; i < root.getRowCount(); i++) { - if (!v.isNull(i)) { - s += v.get(i).length; - } - } - if (s > 0) { - strLenSums.merge(field.getName(), s, Long::sum); - } - } - default -> { - // skip: bool, null, list, struct, extension, etc. - } - } - } - } - } - } - return new Stats(rowCount, numSums, strLenSums); - } - - // ── Java side ───────────────────────────────────────────────────────────── - - private static Stats javaStats(Path file) throws Exception { - Map numSums = new LinkedHashMap<>(); - Map strLenSums = new LinkedHashMap<>(); - long rowCount = 0; - try (VortexReader reader = VortexReader.open(file, EncodingRegistry.loadAll()); - var iter = reader.scan(io.github.dfa1.vortex.scan.ScanOptions.all())) { - while (iter.hasNext()) { - ScanResult chunk = iter.next(); - rowCount += chunk.rowCount(); - for (Map.Entry e : chunk.columns().entrySet()) { - Array arr = e.getValue(); - Double numSum = numericSum(arr); - if (numSum != null) { - numSums.merge(e.getKey(), numSum, Double::sum); - } - Long strLen = stringByteLength(arr); - if (strLen != null && strLen > 0) { - strLenSums.merge(e.getKey(), strLen, Long::sum); - } - } - } - } - return new Stats(rowCount, numSums, strLenSums); - } - - private static Double numericSum(Array arr) { - if (arr instanceof MaskedArray m) { - Array child = m.child(0); - return switch (child) { - case LongArray v -> { - long s = 0; - for (long i = 0; i < v.length(); i++) { - if (m.isValid(i)) { - s += v.getLong(i); - } - } - yield (double) s; - } - case DoubleArray v -> { - double s = 0; - for (long i = 0; i < v.length(); i++) { - if (m.isValid(i)) { - s += v.getDouble(i); - } - } - yield s; - } - case IntArray v -> { - long s = 0; - boolean u32 = v.dtype() instanceof DType.Primitive p && p.ptype() == PType.U32; - for (long i = 0; i < v.length(); i++) { - if (m.isValid(i)) { - s += u32 ? Integer.toUnsignedLong(v.getInt(i)) : v.getInt(i); - } - } - yield (double) s; - } - case FloatArray v -> { - double s = 0; - for (long i = 0; i < v.length(); i++) { - if (m.isValid(i)) { - s += v.getFloat(i); - } - } - yield s; - } - case ShortArray v -> { - long s = 0; - boolean u16 = v.dtype() instanceof DType.Primitive p && p.ptype() == PType.U16; - for (long i = 0; i < v.length(); i++) { - if (m.isValid(i)) { - s += u16 ? Short.toUnsignedLong(v.getShort(i)) : v.getShort(i); - } - } - yield (double) s; - } - case ByteArray v -> { - long s = 0; - boolean u8 = v.dtype() instanceof DType.Primitive p && p.ptype() == PType.U8; - for (long i = 0; i < v.length(); i++) { - if (m.isValid(i)) { - s += u8 ? Byte.toUnsignedLong(v.getByte(i)) : v.getByte(i); - } - } - yield (double) s; - } - default -> null; - }; - } - return switch (arr) { - case LongArray v -> (double) v.fold(0L, Long::sum); - case DoubleArray v -> v.fold(0.0, Double::sum); - case IntArray v -> { - boolean u32 = v.dtype() instanceof DType.Primitive p && p.ptype() == PType.U32; - if (u32) { - long s = 0; - for (long i = 0; i < v.length(); i++) { - s += Integer.toUnsignedLong(v.getInt(i)); - } - yield (double) s; - } - yield (double) v.fold(0, Integer::sum); - } - case FloatArray v -> v.fold(0.0, Double::sum); - case ShortArray v -> { - boolean u16 = v.dtype() instanceof DType.Primitive p && p.ptype() == PType.U16; - if (u16) { - long s = 0; - for (long i = 0; i < v.length(); i++) { - s += Short.toUnsignedLong(v.getShort(i)); - } - yield (double) s; - } - yield (double) v.fold(0L, Long::sum); - } - case ByteArray v -> { - boolean u8 = v.dtype() instanceof DType.Primitive p && p.ptype() == PType.U8; - long s = 0; - for (long i = 0; i < v.length(); i++) { - s += u8 ? Byte.toUnsignedLong(v.getByte(i)) : v.getByte(i); - } - yield (double) s; - } - default -> null; - }; - } - - private static Long stringByteLength(Array arr) { - if (!(arr instanceof VarBinArray v)) { - return null; - } - if (!(v.dtype() instanceof DType.Utf8)) { - return null; // Rust only reports VarChar (UTF-8), skip Binary columns - } - long[] total = {0L}; - v.forEachByteLength(len -> total[0] += len); - return total[0]; - } -} diff --git a/integration/src/test/java/io/github/dfa1/vortex/integration/RustWritesJavaReadsIntegrationTest.java b/integration/src/test/java/io/github/dfa1/vortex/integration/RustWritesJavaReadsIntegrationTest.java deleted file mode 100644 index 5dc1263..0000000 --- a/integration/src/test/java/io/github/dfa1/vortex/integration/RustWritesJavaReadsIntegrationTest.java +++ /dev/null @@ -1,401 +0,0 @@ -package io.github.dfa1.vortex.integration; - -import dev.vortex.api.DataSource; -import dev.vortex.api.Expression; -import dev.vortex.api.Partition; -import dev.vortex.api.Scan; -import dev.vortex.api.ScanOptions; -import dev.vortex.api.Session; -import dev.vortex.api.VortexWriter; -import dev.vortex.arrow.ArrowAllocation; -import dev.vortex.jni.NativeLoader; -import io.github.dfa1.vortex.core.DType; -import io.github.dfa1.vortex.core.PType; -import io.github.dfa1.vortex.core.array.LongArray; -import io.github.dfa1.vortex.encoding.EncodingRegistry; -import io.github.dfa1.vortex.io.VortexReader; -import io.github.dfa1.vortex.scan.ScanResult; -import org.apache.arrow.c.ArrowArray; -import org.apache.arrow.c.ArrowSchema; -import org.apache.arrow.c.Data; -import org.apache.arrow.memory.BufferAllocator; -import org.apache.arrow.vector.BigIntVector; -import org.apache.arrow.vector.Float8Vector; -import org.apache.arrow.vector.VectorSchemaRoot; -import org.apache.arrow.vector.ipc.ArrowReader; -import org.apache.arrow.vector.types.FloatingPointPrecision; -import org.apache.arrow.vector.types.pojo.ArrowType; -import org.apache.arrow.vector.types.pojo.Field; -import org.apache.arrow.vector.types.pojo.Schema; -import org.junit.jupiter.api.Test; -import org.junit.jupiter.api.io.TempDir; - -import java.io.IOException; -import java.lang.foreign.ValueLayout; -import java.net.URI; -import java.nio.ByteOrder; -import java.nio.file.Files; -import java.nio.file.Path; -import java.nio.file.StandardCopyOption; -import java.util.ArrayList; -import java.util.Arrays; -import java.util.HashMap; -import java.util.List; - -import static org.assertj.core.api.Assertions.assertThat; -import static org.junit.jupiter.api.Assumptions.assumeTrue; - -/// Cross-compatibility: Rust (JNI) writer → Java reader. -class RustWritesJavaReadsIntegrationTest { - - private static final String S3_BASE = "https://vortex-compat-fixtures.s3.amazonaws.com/v0.72.0/arrays/"; - - private static final Session SESSION = Session.create(); - private static final BufferAllocator ALLOCATOR = ArrowAllocation.rootAllocator(); - private static final Schema JNI_SCHEMA = new Schema(List.of( - Field.notNullable("id", new ArrowType.Int(64, true)), - Field.notNullable("value", new ArrowType.FloatingPoint(FloatingPointPrecision.DOUBLE)) - )); - private static final Schema NULLABLE_SCHEMA = new Schema(List.of( - Field.nullable("id", new ArrowType.Int(64, true)) - )); - - static { - NativeLoader.loadJni(); - } - - private static void writeJni(Path file, long[] ids, double[] vals) throws IOException { - String uri = file.toAbsolutePath().toUri().toString(); - try (VortexWriter writer = VortexWriter.create(SESSION, uri, JNI_SCHEMA, new HashMap<>(), ALLOCATOR)) { - flushBatch(writer, ids, vals); - } - } - - private static void flushBatch(VortexWriter writer, long[] ids, double[] vals) throws IOException { - int n = ids.length; - try (VectorSchemaRoot root = VectorSchemaRoot.create(JNI_SCHEMA, ALLOCATOR)) { - BigIntVector idVec = (BigIntVector) root.getVector("id"); - Float8Vector valVec = (Float8Vector) root.getVector("value"); - idVec.allocateNew(n); - valVec.allocateNew(n); - for (int i = 0; i < n; i++) { - idVec.setSafe(i, ids[i]); - valVec.setSafe(i, vals[i]); - } - root.setRowCount(n); - try (ArrowArray arr = ArrowArray.allocateNew(ALLOCATOR); - ArrowSchema schema = ArrowSchema.allocateNew(ALLOCATOR)) { - Data.exportVectorSchemaRoot(ALLOCATOR, root, null, arr, schema); - writer.writeBatch(arr.memoryAddress(), schema.memoryAddress()); - } - } - } - - private static List scanAll(VortexReader vf) { - return scanAll(vf, io.github.dfa1.vortex.scan.ScanOptions.all()); - } - - private static List scanAll(VortexReader vf, - io.github.dfa1.vortex.scan.ScanOptions opts) { - var results = new ArrayList(); - var iter = vf.scan(opts); - while (iter.hasNext()) { - results.add(iter.next()); - } - return results; - } - - // ── JNI write helpers ───────────────────────────────────────────────────── - - private static long[] toLongs(ScanResult chunk) { - var layout = ValueLayout.JAVA_LONG_UNALIGNED.withOrder(ByteOrder.LITTLE_ENDIAN); - var arr = chunk.columns().get("id"); - long[] out = new long[(int) arr.length()]; - for (int i = 0; i < out.length; i++) { - out[i] = arr.buffer(0).get(layout, (long) i * Long.BYTES); - } - return out; - } - - private static double[] toDoubles(ScanResult chunk) { - var layout = ValueLayout.JAVA_DOUBLE_UNALIGNED.withOrder(ByteOrder.LITTLE_ENDIAN); - var arr = chunk.columns().get("value"); - double[] out = new double[(int) arr.length()]; - for (int i = 0; i < out.length; i++) { - out[i] = arr.buffer(0).get(layout, (long) i * Double.BYTES); - } - return out; - } - - // ── Java read helpers ───────────────────────────────────────────────────── - - @Test - void jniWriter_javaReader_singleChunk(@TempDir Path tmp) throws IOException { - // Given - Path file = tmp.resolve("jni_single.vtx"); - long[] ids = {1L, 2L, 3L}; - double[] vals = {1.1, 2.2, 3.3}; - writeJni(file, ids, vals); - - // When / Then - try (var vf = VortexReader.open(file, EncodingRegistry.loadAll())) { - List results = scanAll(vf); - assertThat(results).hasSize(1); - assertThat(results.getFirst().rowCount()).isEqualTo(3L); - assertThat(toLongs(results.getFirst())).containsExactly(1L, 2L, 3L); - assertThat(toDoubles(results.getFirst())).containsExactly(1.1, 2.2, 3.3); - } - } - - @Test - void jniWriter_javaReader_multipleChunks(@TempDir Path tmp) throws IOException { - // Given - Path file = tmp.resolve("jni_multi.vtx"); - String uri = file.toAbsolutePath().toUri().toString(); - try (VortexWriter writer = VortexWriter.create(SESSION, uri, JNI_SCHEMA, new HashMap<>(), ALLOCATOR)) { - flushBatch(writer, new long[]{1L, 2L}, new double[]{1.1, 2.2}); - flushBatch(writer, new long[]{3L, 4L, 5L}, new double[]{3.3, 4.4, 5.5}); - } - - // When / Then — JNI may merge small batches; verify total rows and values - try (var vf = VortexReader.open(file, EncodingRegistry.loadAll())) { - List results = scanAll(vf); - long totalRows = results.stream().mapToLong(ScanResult::rowCount).sum(); - assertThat(totalRows).isEqualTo(5L); - long[] allIds = results.stream() - .flatMapToLong(r -> java.util.Arrays.stream(toLongs(r))) - .toArray(); - assertThat(allIds).containsExactly(1L, 2L, 3L, 4L, 5L); - } - } - - @Test - void jniWriter_javaReader_columnProjection(@TempDir Path tmp) throws IOException { - // Given - Path file = tmp.resolve("jni_proj.vtx"); - writeJni(file, new long[]{10L, 20L}, new double[]{0.1, 0.2}); - - // When / Then - try (var vf = VortexReader.open(file, EncodingRegistry.loadAll())) { - List results = scanAll(vf, io.github.dfa1.vortex.scan.ScanOptions.columns("id")); - assertThat(results).hasSize(1); - assertThat(results.getFirst().columns()).containsKey("id"); - assertThat(results.getFirst().columns()).doesNotContainKey("value"); - assertThat(toLongs(results.getFirst())).containsExactly(10L, 20L); - } - } - - @Test - void jniWriter_javaReader_fewUniqueF64Values(@TempDir Path tmp) throws IOException { - // Given — 10_000 rows cycling through only 3 unique F64 values to trigger dict encoding - int n = 10_000; - long[] ids = new long[n]; - double[] vals = new double[n]; - double[] unique = {1.1, 2.2, 3.3}; - for (int i = 0; i < n; i++) { - ids[i] = i; - vals[i] = unique[i % unique.length]; - } - Path file = tmp.resolve("jni_dict.vtx"); - writeJni(file, ids, vals); - - // When / Then - try (var vf = VortexReader.open(file, EncodingRegistry.loadAll())) { - List results = scanAll(vf, io.github.dfa1.vortex.scan.ScanOptions.columns("value")); - long total = results.stream().mapToLong(ScanResult::rowCount).sum(); - assertThat(total).isEqualTo(n); - var layout = ValueLayout.JAVA_DOUBLE_UNALIGNED.withOrder(ByteOrder.LITTLE_ENDIAN); - double sum = 0; - for (ScanResult r : results) { - var arr = r.columns().get("value"); - for (long j = 0; j < arr.length(); j++) { - sum += arr.buffer(0).get(layout, j * Double.BYTES); - } - } - // 10_000 rows: 3333 full cycles of [1.1,2.2,3.3] (=6.6 each) + one 1.1 remainder - assertThat(sum).isCloseTo(21_998.9, org.assertj.core.data.Offset.offset(0.1)); - } - } - - @Test - void jniWriter_nullableColumn_decodesWithoutError(@TempDir Path tmp) throws IOException { - // Given — nullable I64 column; the JNI compressor encodes this as fastlanes.bitpacked - // (not vortex.masked) since the compressor folds validity into the encoding. - // This test verifies end-to-end decoding of nullable schemas does not throw. - Path file = tmp.resolve("nullable.vtx"); - String uri = file.toAbsolutePath().toUri().toString(); - int n = 10_000; - try (VortexWriter writer = VortexWriter.create(SESSION, uri, NULLABLE_SCHEMA, new HashMap<>(), ALLOCATOR)) { - try (VectorSchemaRoot root = VectorSchemaRoot.create(NULLABLE_SCHEMA, ALLOCATOR)) { - BigIntVector idVec = (BigIntVector) root.getVector("id"); - idVec.allocateNew(n); - for (int i = 0; i < n; i++) { - if (i % 5 == 0) { - idVec.setNull(i); - } else { - idVec.setSafe(i, (long) i); - } - } - root.setRowCount(n); - try (ArrowArray arr = ArrowArray.allocateNew(ALLOCATOR); - ArrowSchema schema = ArrowSchema.allocateNew(ALLOCATOR)) { - Data.exportVectorSchemaRoot(ALLOCATOR, root, null, arr, schema); - writer.writeBatch(arr.memoryAddress(), schema.memoryAddress()); - } - } - } - - // When / Then — decodes without error, correct row count - try (var vf = VortexReader.open(file, EncodingRegistry.loadAll())) { - List results = scanAll(vf); - long totalRows = results.stream().mapToLong(ScanResult::rowCount).sum(); - assertThat(totalRows).isEqualTo(n); - assertThat(vf.dtype()).isInstanceOf(DType.Struct.class); - DType colDtype = ((DType.Struct) vf.dtype()).field("id"); - assertThat(colDtype.nullable()).isTrue(); - } - } - - // ── S3 fixture round-trip: Rust-written pco → Java reader ──────────────── - - @Test - void s3_pcoVortex_javaDecodeMatchesJni(@TempDir Path tmp) throws Exception { - // Given — pco.vortex: synthetic file with all pco ptypes; Classic+Consecutive+IntMult+FloatMult modes - assumeNetworkAvailable(); - Path file = downloadIfMissing(tmp, "pco.vortex"); - String col = firstI64Column(file); - - // When - long[] jni = readJniLongColumn(file, col); - long[] java = readJavaLongColumn(file, col); - - // Then — same values (order may differ across chunks) - Arrays.sort(jni); - Arrays.sort(java); - assertThat(java).containsExactly(jni); - } - - @Test - void s3_tpchLineitem_javaDecodeMatchesJni(@TempDir Path tmp) throws Exception { - // Given — tpch_lineitem.compact.vortex: I64/I32/Decimal/date columns, Classic+Consecutive - assumeNetworkAvailable(); - Path file = downloadIfMissing(tmp, "tpch_lineitem.compact.vortex"); - String col = firstI64Column(file); - - // When - long[] jni = readJniLongColumn(file, col); - long[] java = readJavaLongColumn(file, col); - - // Then - Arrays.sort(jni); - Arrays.sort(java); - assertThat(java).containsExactly(jni); - } - - @Test - void s3_tpchOrders_javaDecodeMatchesJni(@TempDir Path tmp) throws Exception { - // Given — tpch_orders.compact.vortex: I64/I32/Decimal/date columns, Classic+Consecutive - assumeNetworkAvailable(); - Path file = downloadIfMissing(tmp, "tpch_orders.compact.vortex"); - String col = firstI64Column(file); - - // When - long[] jni = readJniLongColumn(file, col); - long[] java = readJavaLongColumn(file, col); - - // Then - Arrays.sort(jni); - Arrays.sort(java); - assertThat(java).containsExactly(jni); - } - - @Test - void s3_clickbenchHits5k_javaDecodeMatchesJni(@TempDir Path tmp) throws Exception { - // Given — clickbench_hits_5k.compact.vortex: I16/I32/I64/ts-I64 columns, Classic+Consecutive - assumeNetworkAvailable(); - Path file = downloadIfMissing(tmp, "clickbench_hits_5k.compact.vortex"); - String col = firstI64Column(file); - - // When - long[] jni = readJniLongColumn(file, col); - long[] java = readJavaLongColumn(file, col); - - // Then - Arrays.sort(jni); - Arrays.sort(java); - assertThat(java).containsExactly(jni); - } - - // ── S3 helpers ──────────────────────────────────────────────────────────── - - private static String firstI64Column(Path file) throws IOException { - try (var vf = VortexReader.open(file, EncodingRegistry.empty())) { - if (vf.dtype() instanceof DType.Struct struct) { - for (int i = 0; i < struct.fieldNames().size(); i++) { - if (struct.fieldTypes().get(i) instanceof DType.Primitive(PType pt, boolean _) && pt == PType.I64) { - return struct.fieldNames().get(i); - } - } - } - throw new AssertionError("no I64 column found in " + file.getFileName()); - } - } - - private static long[] readJniLongColumn(Path file, String column) throws IOException { - String uri = file.toAbsolutePath().toUri().toString(); - ScanOptions opts = ScanOptions.builder() - .projection(Expression.select(new String[]{column}, Expression.root())) - .build(); - var longs = new ArrayList(); - DataSource ds = DataSource.open(SESSION, uri); - Scan scan = ds.scan(opts); - while (scan.hasNext()) { - Partition partition = scan.next(); - try (ArrowReader reader = partition.scanArrow(ALLOCATOR)) { - while (reader.loadNextBatch()) { - VectorSchemaRoot root = reader.getVectorSchemaRoot(); - BigIntVector vec = (BigIntVector) root.getVector(column); - for (int i = 0; i < root.getRowCount(); i++) { - longs.add(vec.get(i)); - } - } - } - } - return longs.stream().mapToLong(Long::longValue).toArray(); - } - - private static long[] readJavaLongColumn(Path file, String column) throws IOException { - try (var vf = VortexReader.open(file, EncodingRegistry.loadAll())) { - var longs = new ArrayList(); - var iter = vf.scan(io.github.dfa1.vortex.scan.ScanOptions.columns(column)); - while (iter.hasNext()) { - ScanResult r = iter.next(); - LongArray arr = (LongArray) r.columns().get(column); - for (long i = 0; i < arr.length(); i++) { - longs.add(arr.getLong(i)); - } - } - return longs.stream().mapToLong(Long::longValue).toArray(); - } - } - - private static Path downloadIfMissing(Path tmp, String name) throws Exception { - Path cached = Path.of("/tmp/pco-fixtures", name); - if (Files.exists(cached)) { - return cached; - } - Path dest = tmp.resolve(name); - try (var in = URI.create(S3_BASE + name).toURL().openStream()) { - Files.copy(in, dest, StandardCopyOption.REPLACE_EXISTING); - } - return dest; - } - - private static void assumeNetworkAvailable() { - try { - URI.create("https://vortex-compat-fixtures.s3.amazonaws.com").toURL().openStream().close(); - } catch (Exception e) { - assumeTrue(false, "no network"); - } - } -} diff --git a/integration/src/test/java/io/github/dfa1/vortex/integration/VortexInspectorIntegrationTest.java b/integration/src/test/java/io/github/dfa1/vortex/integration/VortexInspectorIntegrationTest.java deleted file mode 100644 index 2a39594..0000000 --- a/integration/src/test/java/io/github/dfa1/vortex/integration/VortexInspectorIntegrationTest.java +++ /dev/null @@ -1,90 +0,0 @@ -package io.github.dfa1.vortex.integration; - -import dev.vortex.api.Session; -import dev.vortex.api.VortexWriter; -import dev.vortex.arrow.ArrowAllocation; -import dev.vortex.jni.NativeLoader; -import io.github.dfa1.vortex.encoding.EncodingRegistry; -import io.github.dfa1.vortex.io.VortexInspector; -import io.github.dfa1.vortex.io.VortexReader; -import org.apache.arrow.c.ArrowArray; -import org.apache.arrow.c.ArrowSchema; -import org.apache.arrow.c.Data; -import org.apache.arrow.memory.BufferAllocator; -import org.apache.arrow.vector.BigIntVector; -import org.apache.arrow.vector.Float8Vector; -import org.apache.arrow.vector.VectorSchemaRoot; -import org.apache.arrow.vector.types.FloatingPointPrecision; -import org.apache.arrow.vector.types.pojo.ArrowType; -import org.apache.arrow.vector.types.pojo.Field; -import org.apache.arrow.vector.types.pojo.Schema; -import org.junit.jupiter.api.Test; -import org.junit.jupiter.api.io.TempDir; - -import java.io.IOException; -import java.nio.file.Path; -import java.util.HashMap; -import java.util.List; -import java.util.Random; - -import static org.assertj.core.api.Assertions.assertThat; - -/// Verifies that VortexInspector produces a correct report for a JNI-written file. -class VortexInspectorIntegrationTest { - - private static final Session SESSION = Session.create(); - private static final BufferAllocator ALLOCATOR = ArrowAllocation.rootAllocator(); - private static final Schema SCHEMA = new Schema(List.of( - Field.notNullable("id", new ArrowType.Int(64, true)), - Field.notNullable("value", new ArrowType.FloatingPoint(FloatingPointPrecision.DOUBLE)) - )); - - static { - NativeLoader.loadJni(); - } - - @Test - void inspect_showsFileInfoAndEncodings(@TempDir Path tmp) throws IOException { - // Given - Path file = tmp.resolve("inspect.vtx"); - writeJni(file, 50_000); - - // When - String report; - try (VortexReader vf = VortexReader.open(file, EncodingRegistry.loadAll())) { - report = VortexInspector.inspect(vf); - } - - // Then - System.out.println(report); - assertThat(report).contains("Vortex v"); - assertThat(report).contains("id"); - assertThat(report).contains("value"); - assertThat(report).contains("Registered encodings:"); - assertThat(report).contains("Used encodings:"); - assertThat(report).contains("Layout:"); - } - - private static void writeJni(Path file, int rows) throws IOException { - String uri = file.toAbsolutePath().toUri().toString(); - var rng = new Random(42L); - try (VortexWriter writer = VortexWriter.create(SESSION, uri, SCHEMA, new HashMap<>(), ALLOCATOR)) { - try (VectorSchemaRoot root = VectorSchemaRoot.create(SCHEMA, ALLOCATOR)) { - BigIntVector idVec = (BigIntVector) root.getVector("id"); - Float8Vector valVec = (Float8Vector) root.getVector("value"); - idVec.allocateNew(rows); - valVec.allocateNew(rows); - for (int i = 0; i < rows; i++) { - idVec.setSafe(i, i); - valVec.setSafe(i, rng.nextDouble() * 1000.0); - } - root.setRowCount(rows); - try (ArrowArray arr = ArrowArray.allocateNew(ALLOCATOR); - ArrowSchema schema = ArrowSchema.allocateNew(ALLOCATOR)) { - Data.exportVectorSchemaRoot(ALLOCATOR, root, null, arr, schema); - writer.writeBatch(arr.memoryAddress(), schema.memoryAddress()); - } - } - } - } -} diff --git a/jdbc/pom.xml b/jdbc/pom.xml deleted file mode 100644 index 62af106..0000000 --- a/jdbc/pom.xml +++ /dev/null @@ -1,38 +0,0 @@ - - - 4.0.0 - - io.github.dfa1.vortex - vortex-java - 0.3.0-SNAPSHOT - - - jdbc - - - - io.github.dfa1.vortex - reader - - - io.github.dfa1.vortex - writer - - - - org.junit.jupiter - junit-jupiter - test - - - org.assertj - assertj-core - test - - - com.h2database - h2 - test - - - diff --git a/jdbc/src/main/java/io/github/dfa1/vortex/jdbc/JdbcImportOptions.java b/jdbc/src/main/java/io/github/dfa1/vortex/jdbc/JdbcImportOptions.java deleted file mode 100644 index 885644d..0000000 --- a/jdbc/src/main/java/io/github/dfa1/vortex/jdbc/JdbcImportOptions.java +++ /dev/null @@ -1,55 +0,0 @@ -package io.github.dfa1.vortex.jdbc; - -import io.github.dfa1.vortex.writer.WriteOptions; - -/// Options controlling JDBC → Vortex import. -/// -/// @param fetchSize number of rows the JDBC driver fetches per round-trip -/// @param chunkSize number of rows per Vortex chunk written to disk -/// @param writeOptions encoding and compression settings passed to {@link io.github.dfa1.vortex.writer.VortexWriter} -/// @param progressListener optional callback invoked after each full chunk is written; {@code null} disables progress reporting -public record JdbcImportOptions( - int fetchSize, - int chunkSize, - WriteOptions writeOptions, - ProgressListener progressListener -) { - /// Returns a default configuration: fetch size 10 000, chunk size 65 536, cascading compression at level 3, no progress listener. - /// - /// @return default import options - public static JdbcImportOptions defaults() { - return new JdbcImportOptions(10_000, 65_536, WriteOptions.cascading(3), null); - } - - /// Returns a copy with the JDBC fetch size set to {@code size}. - /// - /// @param size number of rows fetched per driver round-trip - /// @return updated options - public JdbcImportOptions withFetchSize(int size) { - return new JdbcImportOptions(size, chunkSize, writeOptions, progressListener); - } - - /// Returns a copy with the Vortex chunk size set to {@code size}. - /// - /// @param size number of rows per written chunk - /// @return updated options - public JdbcImportOptions withChunkSize(int size) { - return new JdbcImportOptions(fetchSize, size, writeOptions, progressListener); - } - - /// Returns a copy with the given write options. - /// - /// @param opts encoding and compression settings - /// @return updated options - public JdbcImportOptions withWriteOptions(WriteOptions opts) { - return new JdbcImportOptions(fetchSize, chunkSize, opts, progressListener); - } - - /// Returns a copy with the given progress listener. - /// - /// @param listener callback invoked after each full chunk; pass {@code null} to disable - /// @return updated options - public JdbcImportOptions withProgressListener(ProgressListener listener) { - return new JdbcImportOptions(fetchSize, chunkSize, writeOptions, listener); - } -} diff --git a/jdbc/src/main/java/io/github/dfa1/vortex/jdbc/JdbcImporter.java b/jdbc/src/main/java/io/github/dfa1/vortex/jdbc/JdbcImporter.java deleted file mode 100644 index 8157bc8..0000000 --- a/jdbc/src/main/java/io/github/dfa1/vortex/jdbc/JdbcImporter.java +++ /dev/null @@ -1,184 +0,0 @@ -package io.github.dfa1.vortex.jdbc; - -import io.github.dfa1.vortex.core.DType; -import io.github.dfa1.vortex.core.PType; -import io.github.dfa1.vortex.writer.VortexWriter; - -import java.io.IOException; -import java.nio.channels.FileChannel; -import java.nio.file.Path; -import java.nio.file.StandardOpenOption; -import java.sql.Connection; -import java.sql.PreparedStatement; -import java.sql.ResultSet; -import java.sql.ResultSetMetaData; -import java.sql.SQLException; -import java.util.ArrayList; -import java.util.Arrays; -import java.util.LinkedHashMap; -import java.util.List; -import java.util.Map; - -/// Reads rows from a JDBC {@link ResultSet} and writes a Vortex file. -/// -/// The schema is derived from {@link ResultSetMetaData} — no type inference is needed. -/// SQL NULL values are mapped to {@code 0}, {@code 0.0}, {@code false}, or {@code ""} depending on column type. -/// Use {@link JdbcImportOptions#withFetchSize} to control driver-side streaming for large tables. -public final class JdbcImporter { - - private JdbcImporter() { - } - - /// Imports all rows from {@code tableName} via {@code SELECT * FROM } using default options. - /// - /// @param conn open JDBC connection - /// @param tableName name of the table to export - /// @param vortexPath destination path; created or truncated - /// @throws SQLException if the query fails - /// @throws IOException if writing the Vortex file fails - public static void importTable(Connection conn, String tableName, Path vortexPath) throws SQLException, IOException { - importQuery(conn, "SELECT * FROM " + tableName, vortexPath); - } - - /// Imports all rows returned by {@code sql} using default options. - /// - /// @param conn open JDBC connection - /// @param sql SELECT statement to execute - /// @param vortexPath destination path; created or truncated - /// @throws SQLException if the query fails - /// @throws IOException if writing the Vortex file fails - public static void importQuery(Connection conn, String sql, Path vortexPath) throws SQLException, IOException { - importQuery(conn, sql, vortexPath, JdbcImportOptions.defaults()); - } - - /// Imports all rows returned by {@code sql} with the given options. - /// - /// @param conn open JDBC connection - /// @param sql SELECT statement to execute - /// @param vortexPath destination path; created or truncated - /// @param options fetch size, chunk size, write options, and optional progress listener - /// @throws SQLException if the query fails - /// @throws IOException if writing the Vortex file fails - public static void importQuery(Connection conn, String sql, Path vortexPath, JdbcImportOptions options) - throws SQLException, IOException { - try (PreparedStatement stmt = conn.prepareStatement(sql, ResultSet.TYPE_FORWARD_ONLY, ResultSet.CONCUR_READ_ONLY)) { - stmt.setFetchSize(options.fetchSize()); - try (ResultSet rs = stmt.executeQuery()) { - ResultSetMetaData meta = rs.getMetaData(); - int colCount = meta.getColumnCount(); - DType.Struct schema = buildSchema(meta, colCount); - try (FileChannel channel = FileChannel.open(vortexPath, - StandardOpenOption.CREATE, StandardOpenOption.WRITE, StandardOpenOption.TRUNCATE_EXISTING); - VortexWriter writer = VortexWriter.create(channel, schema, options.writeOptions())) { - importRows(rs, writer, schema, colCount, options); - } - } - } - } - - private static DType.Struct buildSchema(ResultSetMetaData meta, int colCount) throws SQLException { - List names = new ArrayList<>(colCount); - List types = new ArrayList<>(colCount); - for (int c = 1; c <= colCount; c++) { - boolean nullable = meta.isNullable(c) != ResultSetMetaData.columnNoNulls; - names.add(meta.getColumnLabel(c)); - types.add(SqlTypeToDType.map(meta.getColumnType(c), nullable)); - } - return new DType.Struct(names, types, false); - } - - private static void importRows(ResultSet rs, VortexWriter writer, DType.Struct schema, - int colCount, JdbcImportOptions options) throws SQLException, IOException { - int chunkSize = options.chunkSize(); - Object[] buffers = allocateBuffers(schema, colCount, chunkSize); - int rowsInBuffer = 0; - long totalRows = 0; - - while (rs.next()) { - for (int c = 0; c < colCount; c++) { - fillCell(buffers[c], rowsInBuffer, rs, c + 1, schema.fieldTypes().get(c)); - } - rowsInBuffer++; - if (rowsInBuffer == chunkSize) { - writer.writeChunk(toChunkMap(schema, buffers, rowsInBuffer)); - totalRows += rowsInBuffer; - rowsInBuffer = 0; - buffers = allocateBuffers(schema, colCount, chunkSize); - if (options.progressListener() != null) { - options.progressListener().onProgress(totalRows, -1); - } - } - } - - if (rowsInBuffer > 0) { - writer.writeChunk(toChunkMap(schema, buffers, rowsInBuffer)); - } - } - - private static Object[] allocateBuffers(DType.Struct schema, int colCount, int size) { - Object[] buffers = new Object[colCount]; - for (int c = 0; c < colCount; c++) { - buffers[c] = allocateBuffer(schema.fieldTypes().get(c), size); - } - return buffers; - } - - private static Object allocateBuffer(DType dtype, int size) { - return switch (dtype) { - case DType.Primitive p when p.ptype() == PType.I64 -> new long[size]; - case DType.Primitive p when p.ptype() == PType.I32 -> new int[size]; - case DType.Primitive p when p.ptype() == PType.I16 -> new short[size]; - case DType.Primitive p when p.ptype() == PType.I8 -> new byte[size]; - case DType.Primitive p when p.ptype() == PType.F64 -> new double[size]; - case DType.Primitive p when p.ptype() == PType.F32 -> new float[size]; - case DType.Bool ignored -> new boolean[size]; - case DType.Utf8 ignored -> new String[size]; - default -> throw new UnsupportedOperationException("unsupported dtype: " + dtype); - }; - } - - private static void fillCell(Object buffer, int rowIdx, ResultSet rs, int colIdx, DType dtype) - throws SQLException { - if (dtype instanceof DType.Primitive p) { - switch (p.ptype()) { - case I64 -> ((long[]) buffer)[rowIdx] = rs.getLong(colIdx); - case I32 -> ((int[]) buffer)[rowIdx] = rs.getInt(colIdx); - case I16 -> ((short[]) buffer)[rowIdx] = rs.getShort(colIdx); - case I8 -> ((byte[]) buffer)[rowIdx] = rs.getByte(colIdx); - case F64 -> ((double[]) buffer)[rowIdx] = rs.getDouble(colIdx); - case F32 -> ((float[]) buffer)[rowIdx] = rs.getFloat(colIdx); - default -> throw new UnsupportedOperationException("unsupported primitive type: " + p.ptype()); - } - } else if (dtype instanceof DType.Bool) { - ((boolean[]) buffer)[rowIdx] = rs.getBoolean(colIdx); - } else if (dtype instanceof DType.Utf8) { - String val = rs.getString(colIdx); - ((String[]) buffer)[rowIdx] = val != null ? val : ""; - } else { - throw new UnsupportedOperationException("unsupported dtype: " + dtype); - } - } - - private static Map toChunkMap(DType.Struct schema, Object[] buffers, int rows) { - List names = schema.fieldNames(); - Map chunk = new LinkedHashMap<>(); - for (int c = 0; c < names.size(); c++) { - chunk.put(names.get(c), trimBuffer(buffers[c], rows)); - } - return chunk; - } - - private static Object trimBuffer(Object buffer, int rows) { - return switch (buffer) { - case long[] arr -> rows == arr.length ? arr : Arrays.copyOf(arr, rows); - case int[] arr -> rows == arr.length ? arr : Arrays.copyOf(arr, rows); - case short[] arr -> rows == arr.length ? arr : Arrays.copyOf(arr, rows); - case byte[] arr -> rows == arr.length ? arr : Arrays.copyOf(arr, rows); - case double[] arr -> rows == arr.length ? arr : Arrays.copyOf(arr, rows); - case float[] arr -> rows == arr.length ? arr : Arrays.copyOf(arr, rows); - case boolean[] arr -> rows == arr.length ? arr : Arrays.copyOf(arr, rows); - case String[] arr -> rows == arr.length ? arr : Arrays.copyOf(arr, rows); - default -> throw new UnsupportedOperationException("unsupported buffer type: " + buffer.getClass()); - }; - } -} diff --git a/jdbc/src/main/java/io/github/dfa1/vortex/jdbc/ProgressListener.java b/jdbc/src/main/java/io/github/dfa1/vortex/jdbc/ProgressListener.java deleted file mode 100644 index 5c5dad4..0000000 --- a/jdbc/src/main/java/io/github/dfa1/vortex/jdbc/ProgressListener.java +++ /dev/null @@ -1,12 +0,0 @@ -package io.github.dfa1.vortex.jdbc; - -/// Callback invoked after each chunk is written during JDBC import. -/// {@code rowsTotal} is {@code -1} when the total row count is unknown. -@FunctionalInterface -public interface ProgressListener { - /// Called after each full chunk is flushed to disk. - /// - /// @param rowsDone cumulative number of rows written so far - /// @param rowsTotal total row count, or {@code -1} if unknown - void onProgress(long rowsDone, long rowsTotal); -} diff --git a/jdbc/src/main/java/io/github/dfa1/vortex/jdbc/SqlTypeToDType.java b/jdbc/src/main/java/io/github/dfa1/vortex/jdbc/SqlTypeToDType.java deleted file mode 100644 index c41e937..0000000 --- a/jdbc/src/main/java/io/github/dfa1/vortex/jdbc/SqlTypeToDType.java +++ /dev/null @@ -1,27 +0,0 @@ -package io.github.dfa1.vortex.jdbc; - -import io.github.dfa1.vortex.core.DType; -import io.github.dfa1.vortex.core.PType; - -import java.sql.Types; - -final class SqlTypeToDType { - - private SqlTypeToDType() { - } - - static DType map(int sqlType, boolean nullable) { - return switch (sqlType) { - case Types.BIGINT -> new DType.Primitive(PType.I64, nullable); - case Types.INTEGER -> new DType.Primitive(PType.I32, nullable); - case Types.SMALLINT -> new DType.Primitive(PType.I16, nullable); - case Types.TINYINT -> new DType.Primitive(PType.I8, nullable); - case Types.DOUBLE, Types.DECIMAL, Types.NUMERIC -> new DType.Primitive(PType.F64, nullable); - case Types.REAL, Types.FLOAT -> new DType.Primitive(PType.F32, nullable); - case Types.BOOLEAN, Types.BIT -> new DType.Bool(nullable); - case Types.VARCHAR, Types.CHAR, Types.LONGVARCHAR, - Types.NVARCHAR, Types.NCHAR, Types.LONGNVARCHAR, Types.CLOB -> new DType.Utf8(nullable); - default -> throw new UnsupportedOperationException("unsupported SQL type: " + sqlType); - }; - } -} diff --git a/jdbc/src/test/java/io/github/dfa1/vortex/jdbc/JdbcImporterTest.java b/jdbc/src/test/java/io/github/dfa1/vortex/jdbc/JdbcImporterTest.java deleted file mode 100644 index 245fa39..0000000 --- a/jdbc/src/test/java/io/github/dfa1/vortex/jdbc/JdbcImporterTest.java +++ /dev/null @@ -1,226 +0,0 @@ -package io.github.dfa1.vortex.jdbc; - -import io.github.dfa1.vortex.core.DType; -import io.github.dfa1.vortex.core.PType; -import io.github.dfa1.vortex.core.array.BoolArray; -import io.github.dfa1.vortex.writer.WriteOptions; -import io.github.dfa1.vortex.core.array.DoubleArray; -import io.github.dfa1.vortex.core.array.LongArray; -import io.github.dfa1.vortex.core.array.VarBinArray; -import io.github.dfa1.vortex.io.VortexReader; -import io.github.dfa1.vortex.scan.ScanIterator; -import io.github.dfa1.vortex.scan.ScanOptions; -import io.github.dfa1.vortex.scan.ScanResult; -import org.junit.jupiter.api.AfterEach; -import org.junit.jupiter.api.BeforeEach; -import org.junit.jupiter.api.Nested; -import org.junit.jupiter.api.Test; -import org.junit.jupiter.api.io.TempDir; - -import java.nio.file.Path; -import java.sql.Connection; -import java.sql.DriverManager; -import java.sql.Statement; -import java.util.ArrayList; -import java.util.List; - -import static org.assertj.core.api.Assertions.assertThat; - -class JdbcImporterTest { - - private Connection conn; - - @BeforeEach - void openConnection() throws Exception { - conn = DriverManager.getConnection("jdbc:h2:mem:test;DB_CLOSE_DELAY=-1"); - } - - @AfterEach - void closeConnection() throws Exception { - try (Statement stmt = conn.createStatement()) { - stmt.execute("DROP ALL OBJECTS"); - } - conn.close(); - } - - @Nested - class ImportQuery { - - @Test - void roundTripsAllSupportedTypes(@TempDir Path tmp) throws Exception { - // Given - try (Statement stmt = conn.createStatement()) { - stmt.execute("CREATE TABLE events (id BIGINT NOT NULL, name VARCHAR(100) NOT NULL, score DOUBLE NOT NULL, active BOOLEAN NOT NULL)"); - stmt.execute("INSERT INTO events VALUES (1, 'Alice', 1.5, TRUE)"); - stmt.execute("INSERT INTO events VALUES (2, 'Bob', 2.7, FALSE)"); - } - Path vortex = tmp.resolve("events.vortex"); - - // When - JdbcImporter.importQuery(conn, "SELECT * FROM events ORDER BY id", vortex); - - // Then - try (VortexReader reader = VortexReader.open(vortex)) { - assertThat(reader.dtype()).isInstanceOf(DType.Struct.class); - DType.Struct schema = (DType.Struct) reader.dtype(); - assertThat(schema.fieldNames()).containsExactly("ID", "NAME", "SCORE", "ACTIVE"); - assertThat(schema.fieldTypes().get(0)).isEqualTo(new DType.Primitive(PType.I64, false)); - assertThat(schema.fieldTypes().get(1)).isEqualTo(new DType.Utf8(false)); - assertThat(schema.fieldTypes().get(2)).isEqualTo(new DType.Primitive(PType.F64, false)); - assertThat(schema.fieldTypes().get(3)).isEqualTo(new DType.Bool(false)); - - try (ScanIterator iter = reader.scan(ScanOptions.all())) { - assertThat(iter.hasNext()).isTrue(); - ScanResult chunk = iter.next(); - assertThat(chunk.rowCount()).isEqualTo(2); - - LongArray ids = chunk.column("ID"); - assertThat(ids.getLong(0)).isEqualTo(1L); - assertThat(ids.getLong(1)).isEqualTo(2L); - - VarBinArray names = chunk.column("NAME"); - assertThat(names.getString(0)).isEqualTo("Alice"); - assertThat(names.getString(1)).isEqualTo("Bob"); - - DoubleArray scores = chunk.column("SCORE"); - assertThat(scores.getDouble(0)).isEqualTo(1.5); - assertThat(scores.getDouble(1)).isEqualTo(2.7); - - BoolArray active = chunk.column("ACTIVE"); - assertThat(active.getBoolean(0)).isTrue(); - assertThat(active.getBoolean(1)).isFalse(); - } - } - } - - @Test - void splitsIntoMultipleChunks(@TempDir Path tmp) throws Exception { - // Given - try (Statement stmt = conn.createStatement()) { - stmt.execute("CREATE TABLE nums (n BIGINT NOT NULL)"); - for (int i = 0; i < 5; i++) { - stmt.execute("INSERT INTO nums VALUES (" + i + ")"); - } - } - Path vortex = tmp.resolve("nums.vortex"); - JdbcImportOptions options = JdbcImportOptions.defaults() - .withChunkSize(2) - .withWriteOptions(WriteOptions.defaults()); - - // When - JdbcImporter.importQuery(conn, "SELECT * FROM nums ORDER BY n", vortex, options); - - // Then - try (VortexReader reader = VortexReader.open(vortex)) { - List collected = new ArrayList<>(); - try (ScanIterator iter = reader.scan(ScanOptions.all())) { - while (iter.hasNext()) { - ScanResult chunk = iter.next(); - LongArray arr = chunk.column("N"); - for (long r = 0; r < chunk.rowCount(); r++) { - collected.add(arr.getLong(r)); - } - } - } - assertThat(collected).containsExactly(0L, 1L, 2L, 3L, 4L); - } - } - - @Test - void emptyResultSetProducesEmptyFile(@TempDir Path tmp) throws Exception { - // Given - try (Statement stmt = conn.createStatement()) { - stmt.execute("CREATE TABLE empty_table (id BIGINT NOT NULL)"); - } - Path vortex = tmp.resolve("empty.vortex"); - - // When - JdbcImporter.importQuery(conn, "SELECT * FROM empty_table", vortex); - - // Then - try (VortexReader reader = VortexReader.open(vortex)) { - assertThat(reader.dtype()).isInstanceOf(DType.Struct.class); - try (ScanIterator iter = reader.scan(ScanOptions.all())) { - assertThat(iter.hasNext()).isFalse(); - } - } - } - - @Test - void nullValuesMapToZeroOrEmpty(@TempDir Path tmp) throws Exception { - // Given - try (Statement stmt = conn.createStatement()) { - stmt.execute("CREATE TABLE nullable_cols (n BIGINT, s VARCHAR(50))"); - stmt.execute("INSERT INTO nullable_cols VALUES (NULL, NULL)"); - } - Path vortex = tmp.resolve("nullable.vortex"); - JdbcImportOptions options = JdbcImportOptions.defaults().withWriteOptions(WriteOptions.defaults()); - - // When - JdbcImporter.importQuery(conn, "SELECT * FROM nullable_cols", vortex, options); - - // Then - try (VortexReader reader = VortexReader.open(vortex)) { - try (ScanIterator iter = reader.scan(ScanOptions.all())) { - assertThat(iter.hasNext()).isTrue(); - ScanResult chunk = iter.next(); - LongArray nums = chunk.column("N"); - assertThat(nums.getLong(0)).isEqualTo(0L); - VarBinArray strs = chunk.column("S"); - assertThat(strs.getString(0)).isEqualTo(""); - } - } - } - - @Test - void progressListenerIsInvokedPerChunk(@TempDir Path tmp) throws Exception { - // Given - try (Statement stmt = conn.createStatement()) { - stmt.execute("CREATE TABLE progress_test (n BIGINT NOT NULL)"); - for (int i = 0; i < 10; i++) { - stmt.execute("INSERT INTO progress_test VALUES (" + i + ")"); - } - } - Path vortex = tmp.resolve("progress.vortex"); - List checkpoints = new ArrayList<>(); - JdbcImportOptions options = JdbcImportOptions.defaults() - .withChunkSize(3) - .withProgressListener((done, total) -> checkpoints.add(done)); - - // When - JdbcImporter.importQuery(conn, "SELECT * FROM progress_test", vortex, options); - - // Then — 3 full chunks of 3 rows each trigger the listener; last partial chunk does not - assertThat(checkpoints).containsExactly(3L, 6L, 9L); - } - } - - @Nested - class ImportTable { - - @Test - void selectsAllRowsFromTable(@TempDir Path tmp) throws Exception { - // Given - try (Statement stmt = conn.createStatement()) { - stmt.execute("CREATE TABLE products (id BIGINT NOT NULL, label VARCHAR(50) NOT NULL)"); - stmt.execute("INSERT INTO products VALUES (10, 'Widget')"); - stmt.execute("INSERT INTO products VALUES (20, 'Gadget')"); - } - Path vortex = tmp.resolve("products.vortex"); - - // When - JdbcImporter.importTable(conn, "products", vortex); - - // Then - try (VortexReader reader = VortexReader.open(vortex)) { - DType.Struct schema = (DType.Struct) reader.dtype(); - assertThat(schema.fieldNames()).containsExactly("ID", "LABEL"); - try (ScanIterator iter = reader.scan(ScanOptions.all())) { - assertThat(iter.hasNext()).isTrue(); - ScanResult chunk = iter.next(); - assertThat(chunk.rowCount()).isEqualTo(2); - } - } - } - } -} diff --git a/mvnw b/mvnw deleted file mode 100755 index bd8896b..0000000 --- a/mvnw +++ /dev/null @@ -1,295 +0,0 @@ -#!/bin/sh -# ---------------------------------------------------------------------------- -# Licensed to the Apache Software Foundation (ASF) under one -# or more contributor license agreements. See the NOTICE file -# distributed with this work for additional information -# regarding copyright ownership. The ASF licenses this file -# to you under the Apache License, Version 2.0 (the -# "License"); you may not use this file except in compliance -# with the License. You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, -# software distributed under the License is distributed on an -# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY -# KIND, either express or implied. See the License for the -# specific language governing permissions and limitations -# under the License. -# ---------------------------------------------------------------------------- - -# ---------------------------------------------------------------------------- -# Apache Maven Wrapper startup batch script, version 3.3.4 -# -# Optional ENV vars -# ----------------- -# JAVA_HOME - location of a JDK home dir, required when download maven via java source -# MVNW_REPOURL - repo url base for downloading maven distribution -# MVNW_USERNAME/MVNW_PASSWORD - user and password for downloading maven -# MVNW_VERBOSE - true: enable verbose log; debug: trace the mvnw script; others: silence the output -# ---------------------------------------------------------------------------- - -set -euf -[ "${MVNW_VERBOSE-}" != debug ] || set -x - -# OS specific support. -native_path() { printf %s\\n "$1"; } -case "$(uname)" in -CYGWIN* | MINGW*) - [ -z "${JAVA_HOME-}" ] || JAVA_HOME="$(cygpath --unix "$JAVA_HOME")" - native_path() { cygpath --path --windows "$1"; } - ;; -esac - -# set JAVACMD and JAVACCMD -set_java_home() { - # For Cygwin and MinGW, ensure paths are in Unix format before anything is touched - if [ -n "${JAVA_HOME-}" ]; then - if [ -x "$JAVA_HOME/jre/sh/java" ]; then - # IBM's JDK on AIX uses strange locations for the executables - JAVACMD="$JAVA_HOME/jre/sh/java" - JAVACCMD="$JAVA_HOME/jre/sh/javac" - else - JAVACMD="$JAVA_HOME/bin/java" - JAVACCMD="$JAVA_HOME/bin/javac" - - if [ ! -x "$JAVACMD" ] || [ ! -x "$JAVACCMD" ]; then - echo "The JAVA_HOME environment variable is not defined correctly, so mvnw cannot run." >&2 - echo "JAVA_HOME is set to \"$JAVA_HOME\", but \"\$JAVA_HOME/bin/java\" or \"\$JAVA_HOME/bin/javac\" does not exist." >&2 - return 1 - fi - fi - else - JAVACMD="$( - 'set' +e - 'unset' -f command 2>/dev/null - 'command' -v java - )" || : - JAVACCMD="$( - 'set' +e - 'unset' -f command 2>/dev/null - 'command' -v javac - )" || : - - if [ ! -x "${JAVACMD-}" ] || [ ! -x "${JAVACCMD-}" ]; then - echo "The java/javac command does not exist in PATH nor is JAVA_HOME set, so mvnw cannot run." >&2 - return 1 - fi - fi -} - -# hash string like Java String::hashCode -hash_string() { - str="${1:-}" h=0 - while [ -n "$str" ]; do - char="${str%"${str#?}"}" - h=$(((h * 31 + $(LC_CTYPE=C printf %d "'$char")) % 4294967296)) - str="${str#?}" - done - printf %x\\n $h -} - -verbose() { :; } -[ "${MVNW_VERBOSE-}" != true ] || verbose() { printf %s\\n "${1-}"; } - -die() { - printf %s\\n "$1" >&2 - exit 1 -} - -trim() { - # MWRAPPER-139: - # Trims trailing and leading whitespace, carriage returns, tabs, and linefeeds. - # Needed for removing poorly interpreted newline sequences when running in more - # exotic environments such as mingw bash on Windows. - printf "%s" "${1}" | tr -d '[:space:]' -} - -scriptDir="$(dirname "$0")" -scriptName="$(basename "$0")" - -# parse distributionUrl and optional distributionSha256Sum, requires .mvn/wrapper/maven-wrapper.properties -while IFS="=" read -r key value; do - case "${key-}" in - distributionUrl) distributionUrl=$(trim "${value-}") ;; - distributionSha256Sum) distributionSha256Sum=$(trim "${value-}") ;; - esac -done <"$scriptDir/.mvn/wrapper/maven-wrapper.properties" -[ -n "${distributionUrl-}" ] || die "cannot read distributionUrl property in $scriptDir/.mvn/wrapper/maven-wrapper.properties" - -case "${distributionUrl##*/}" in -maven-mvnd-*bin.*) - MVN_CMD=mvnd.sh _MVNW_REPO_PATTERN=/maven/mvnd/ - case "${PROCESSOR_ARCHITECTURE-}${PROCESSOR_ARCHITEW6432-}:$(uname -a)" in - *AMD64:CYGWIN* | *AMD64:MINGW*) distributionPlatform=windows-amd64 ;; - :Darwin*x86_64) distributionPlatform=darwin-amd64 ;; - :Darwin*arm64) distributionPlatform=darwin-aarch64 ;; - :Linux*x86_64*) distributionPlatform=linux-amd64 ;; - *) - echo "Cannot detect native platform for mvnd on $(uname)-$(uname -m), use pure java version" >&2 - distributionPlatform=linux-amd64 - ;; - esac - distributionUrl="${distributionUrl%-bin.*}-$distributionPlatform.zip" - ;; -maven-mvnd-*) MVN_CMD=mvnd.sh _MVNW_REPO_PATTERN=/maven/mvnd/ ;; -*) MVN_CMD="mvn${scriptName#mvnw}" _MVNW_REPO_PATTERN=/org/apache/maven/ ;; -esac - -# apply MVNW_REPOURL and calculate MAVEN_HOME -# maven home pattern: ~/.m2/wrapper/dists/{apache-maven-,maven-mvnd--}/ -[ -z "${MVNW_REPOURL-}" ] || distributionUrl="$MVNW_REPOURL$_MVNW_REPO_PATTERN${distributionUrl#*"$_MVNW_REPO_PATTERN"}" -distributionUrlName="${distributionUrl##*/}" -distributionUrlNameMain="${distributionUrlName%.*}" -distributionUrlNameMain="${distributionUrlNameMain%-bin}" -MAVEN_USER_HOME="${MAVEN_USER_HOME:-${HOME}/.m2}" -MAVEN_HOME="${MAVEN_USER_HOME}/wrapper/dists/${distributionUrlNameMain-}/$(hash_string "$distributionUrl")" - -exec_maven() { - unset MVNW_VERBOSE MVNW_USERNAME MVNW_PASSWORD MVNW_REPOURL || : - exec "$MAVEN_HOME/bin/$MVN_CMD" "$@" || die "cannot exec $MAVEN_HOME/bin/$MVN_CMD" -} - -if [ -d "$MAVEN_HOME" ]; then - verbose "found existing MAVEN_HOME at $MAVEN_HOME" - exec_maven "$@" -fi - -case "${distributionUrl-}" in -*?-bin.zip | *?maven-mvnd-?*-?*.zip) ;; -*) die "distributionUrl is not valid, must match *-bin.zip or maven-mvnd-*.zip, but found '${distributionUrl-}'" ;; -esac - -# prepare tmp dir -if TMP_DOWNLOAD_DIR="$(mktemp -d)" && [ -d "$TMP_DOWNLOAD_DIR" ]; then - clean() { rm -rf -- "$TMP_DOWNLOAD_DIR"; } - trap clean HUP INT TERM EXIT -else - die "cannot create temp dir" -fi - -mkdir -p -- "${MAVEN_HOME%/*}" - -# Download and Install Apache Maven -verbose "Couldn't find MAVEN_HOME, downloading and installing it ..." -verbose "Downloading from: $distributionUrl" -verbose "Downloading to: $TMP_DOWNLOAD_DIR/$distributionUrlName" - -# select .zip or .tar.gz -if ! command -v unzip >/dev/null; then - distributionUrl="${distributionUrl%.zip}.tar.gz" - distributionUrlName="${distributionUrl##*/}" -fi - -# verbose opt -__MVNW_QUIET_WGET=--quiet __MVNW_QUIET_CURL=--silent __MVNW_QUIET_UNZIP=-q __MVNW_QUIET_TAR='' -[ "${MVNW_VERBOSE-}" != true ] || __MVNW_QUIET_WGET='' __MVNW_QUIET_CURL='' __MVNW_QUIET_UNZIP='' __MVNW_QUIET_TAR=v - -# normalize http auth -case "${MVNW_PASSWORD:+has-password}" in -'') MVNW_USERNAME='' MVNW_PASSWORD='' ;; -has-password) [ -n "${MVNW_USERNAME-}" ] || MVNW_USERNAME='' MVNW_PASSWORD='' ;; -esac - -if [ -z "${MVNW_USERNAME-}" ] && command -v wget >/dev/null; then - verbose "Found wget ... using wget" - wget ${__MVNW_QUIET_WGET:+"$__MVNW_QUIET_WGET"} "$distributionUrl" -O "$TMP_DOWNLOAD_DIR/$distributionUrlName" || die "wget: Failed to fetch $distributionUrl" -elif [ -z "${MVNW_USERNAME-}" ] && command -v curl >/dev/null; then - verbose "Found curl ... using curl" - curl ${__MVNW_QUIET_CURL:+"$__MVNW_QUIET_CURL"} -f -L -o "$TMP_DOWNLOAD_DIR/$distributionUrlName" "$distributionUrl" || die "curl: Failed to fetch $distributionUrl" -elif set_java_home; then - verbose "Falling back to use Java to download" - javaSource="$TMP_DOWNLOAD_DIR/Downloader.java" - targetZip="$TMP_DOWNLOAD_DIR/$distributionUrlName" - cat >"$javaSource" <<-END - public class Downloader extends java.net.Authenticator - { - protected java.net.PasswordAuthentication getPasswordAuthentication() - { - return new java.net.PasswordAuthentication( System.getenv( "MVNW_USERNAME" ), System.getenv( "MVNW_PASSWORD" ).toCharArray() ); - } - public static void main( String[] args ) throws Exception - { - setDefault( new Downloader() ); - java.nio.file.Files.copy( java.net.URI.create( args[0] ).toURL().openStream(), java.nio.file.Paths.get( args[1] ).toAbsolutePath().normalize() ); - } - } - END - # For Cygwin/MinGW, switch paths to Windows format before running javac and java - verbose " - Compiling Downloader.java ..." - "$(native_path "$JAVACCMD")" "$(native_path "$javaSource")" || die "Failed to compile Downloader.java" - verbose " - Running Downloader.java ..." - "$(native_path "$JAVACMD")" -cp "$(native_path "$TMP_DOWNLOAD_DIR")" Downloader "$distributionUrl" "$(native_path "$targetZip")" -fi - -# If specified, validate the SHA-256 sum of the Maven distribution zip file -if [ -n "${distributionSha256Sum-}" ]; then - distributionSha256Result=false - if [ "$MVN_CMD" = mvnd.sh ]; then - echo "Checksum validation is not supported for maven-mvnd." >&2 - echo "Please disable validation by removing 'distributionSha256Sum' from your maven-wrapper.properties." >&2 - exit 1 - elif command -v sha256sum >/dev/null; then - if echo "$distributionSha256Sum $TMP_DOWNLOAD_DIR/$distributionUrlName" | sha256sum -c - >/dev/null 2>&1; then - distributionSha256Result=true - fi - elif command -v shasum >/dev/null; then - if echo "$distributionSha256Sum $TMP_DOWNLOAD_DIR/$distributionUrlName" | shasum -a 256 -c >/dev/null 2>&1; then - distributionSha256Result=true - fi - else - echo "Checksum validation was requested but neither 'sha256sum' or 'shasum' are available." >&2 - echo "Please install either command, or disable validation by removing 'distributionSha256Sum' from your maven-wrapper.properties." >&2 - exit 1 - fi - if [ $distributionSha256Result = false ]; then - echo "Error: Failed to validate Maven distribution SHA-256, your Maven distribution might be compromised." >&2 - echo "If you updated your Maven version, you need to update the specified distributionSha256Sum property." >&2 - exit 1 - fi -fi - -# unzip and move -if command -v unzip >/dev/null; then - unzip ${__MVNW_QUIET_UNZIP:+"$__MVNW_QUIET_UNZIP"} "$TMP_DOWNLOAD_DIR/$distributionUrlName" -d "$TMP_DOWNLOAD_DIR" || die "failed to unzip" -else - tar xzf${__MVNW_QUIET_TAR:+"$__MVNW_QUIET_TAR"} "$TMP_DOWNLOAD_DIR/$distributionUrlName" -C "$TMP_DOWNLOAD_DIR" || die "failed to untar" -fi - -# Find the actual extracted directory name (handles snapshots where filename != directory name) -actualDistributionDir="" - -# First try the expected directory name (for regular distributions) -if [ -d "$TMP_DOWNLOAD_DIR/$distributionUrlNameMain" ]; then - if [ -f "$TMP_DOWNLOAD_DIR/$distributionUrlNameMain/bin/$MVN_CMD" ]; then - actualDistributionDir="$distributionUrlNameMain" - fi -fi - -# If not found, search for any directory with the Maven executable (for snapshots) -if [ -z "$actualDistributionDir" ]; then - # enable globbing to iterate over items - set +f - for dir in "$TMP_DOWNLOAD_DIR"/*; do - if [ -d "$dir" ]; then - if [ -f "$dir/bin/$MVN_CMD" ]; then - actualDistributionDir="$(basename "$dir")" - break - fi - fi - done - set -f -fi - -if [ -z "$actualDistributionDir" ]; then - verbose "Contents of $TMP_DOWNLOAD_DIR:" - verbose "$(ls -la "$TMP_DOWNLOAD_DIR")" - die "Could not find Maven distribution directory in extracted archive" -fi - -verbose "Found extracted Maven distribution directory: $actualDistributionDir" -printf %s\\n "$distributionUrl" >"$TMP_DOWNLOAD_DIR/$actualDistributionDir/mvnw.url" -mv -- "$TMP_DOWNLOAD_DIR/$actualDistributionDir" "$MAVEN_HOME" || [ -d "$MAVEN_HOME" ] || die "fail to move MAVEN_HOME" - -clean || : -exec_maven "$@" diff --git a/mvnw.cmd b/mvnw.cmd deleted file mode 100644 index 5761d94..0000000 --- a/mvnw.cmd +++ /dev/null @@ -1,189 +0,0 @@ -<# : batch portion -@REM ---------------------------------------------------------------------------- -@REM Licensed to the Apache Software Foundation (ASF) under one -@REM or more contributor license agreements. See the NOTICE file -@REM distributed with this work for additional information -@REM regarding copyright ownership. The ASF licenses this file -@REM to you under the Apache License, Version 2.0 (the -@REM "License"); you may not use this file except in compliance -@REM with the License. You may obtain a copy of the License at -@REM -@REM http://www.apache.org/licenses/LICENSE-2.0 -@REM -@REM Unless required by applicable law or agreed to in writing, -@REM software distributed under the License is distributed on an -@REM "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY -@REM KIND, either express or implied. See the License for the -@REM specific language governing permissions and limitations -@REM under the License. -@REM ---------------------------------------------------------------------------- - -@REM ---------------------------------------------------------------------------- -@REM Apache Maven Wrapper startup batch script, version 3.3.4 -@REM -@REM Optional ENV vars -@REM MVNW_REPOURL - repo url base for downloading maven distribution -@REM MVNW_USERNAME/MVNW_PASSWORD - user and password for downloading maven -@REM MVNW_VERBOSE - true: enable verbose log; others: silence the output -@REM ---------------------------------------------------------------------------- - -@IF "%__MVNW_ARG0_NAME__%"=="" (SET __MVNW_ARG0_NAME__=%~nx0) -@SET __MVNW_CMD__= -@SET __MVNW_ERROR__= -@SET __MVNW_PSMODULEP_SAVE=%PSModulePath% -@SET PSModulePath= -@FOR /F "usebackq tokens=1* delims==" %%A IN (`powershell -noprofile "& {$scriptDir='%~dp0'; $script='%__MVNW_ARG0_NAME__%'; icm -ScriptBlock ([Scriptblock]::Create((Get-Content -Raw '%~f0'))) -NoNewScope}"`) DO @( - IF "%%A"=="MVN_CMD" (set __MVNW_CMD__=%%B) ELSE IF "%%B"=="" (echo %%A) ELSE (echo %%A=%%B) -) -@SET PSModulePath=%__MVNW_PSMODULEP_SAVE% -@SET __MVNW_PSMODULEP_SAVE= -@SET __MVNW_ARG0_NAME__= -@SET MVNW_USERNAME= -@SET MVNW_PASSWORD= -@IF NOT "%__MVNW_CMD__%"=="" ("%__MVNW_CMD__%" %*) -@echo Cannot start maven from wrapper >&2 && exit /b 1 -@GOTO :EOF -: end batch / begin powershell #> - -$ErrorActionPreference = "Stop" -if ($env:MVNW_VERBOSE -eq "true") { - $VerbosePreference = "Continue" -} - -# calculate distributionUrl, requires .mvn/wrapper/maven-wrapper.properties -$distributionUrl = (Get-Content -Raw "$scriptDir/.mvn/wrapper/maven-wrapper.properties" | ConvertFrom-StringData).distributionUrl -if (!$distributionUrl) { - Write-Error "cannot read distributionUrl property in $scriptDir/.mvn/wrapper/maven-wrapper.properties" -} - -switch -wildcard -casesensitive ( $($distributionUrl -replace '^.*/','') ) { - "maven-mvnd-*" { - $USE_MVND = $true - $distributionUrl = $distributionUrl -replace '-bin\.[^.]*$',"-windows-amd64.zip" - $MVN_CMD = "mvnd.cmd" - break - } - default { - $USE_MVND = $false - $MVN_CMD = $script -replace '^mvnw','mvn' - break - } -} - -# apply MVNW_REPOURL and calculate MAVEN_HOME -# maven home pattern: ~/.m2/wrapper/dists/{apache-maven-,maven-mvnd--}/ -if ($env:MVNW_REPOURL) { - $MVNW_REPO_PATTERN = if ($USE_MVND -eq $False) { "/org/apache/maven/" } else { "/maven/mvnd/" } - $distributionUrl = "$env:MVNW_REPOURL$MVNW_REPO_PATTERN$($distributionUrl -replace "^.*$MVNW_REPO_PATTERN",'')" -} -$distributionUrlName = $distributionUrl -replace '^.*/','' -$distributionUrlNameMain = $distributionUrlName -replace '\.[^.]*$','' -replace '-bin$','' - -$MAVEN_M2_PATH = "$HOME/.m2" -if ($env:MAVEN_USER_HOME) { - $MAVEN_M2_PATH = "$env:MAVEN_USER_HOME" -} - -if (-not (Test-Path -Path $MAVEN_M2_PATH)) { - New-Item -Path $MAVEN_M2_PATH -ItemType Directory | Out-Null -} - -$MAVEN_WRAPPER_DISTS = $null -if ((Get-Item $MAVEN_M2_PATH).Target[0] -eq $null) { - $MAVEN_WRAPPER_DISTS = "$MAVEN_M2_PATH/wrapper/dists" -} else { - $MAVEN_WRAPPER_DISTS = (Get-Item $MAVEN_M2_PATH).Target[0] + "/wrapper/dists" -} - -$MAVEN_HOME_PARENT = "$MAVEN_WRAPPER_DISTS/$distributionUrlNameMain" -$MAVEN_HOME_NAME = ([System.Security.Cryptography.SHA256]::Create().ComputeHash([byte[]][char[]]$distributionUrl) | ForEach-Object {$_.ToString("x2")}) -join '' -$MAVEN_HOME = "$MAVEN_HOME_PARENT/$MAVEN_HOME_NAME" - -if (Test-Path -Path "$MAVEN_HOME" -PathType Container) { - Write-Verbose "found existing MAVEN_HOME at $MAVEN_HOME" - Write-Output "MVN_CMD=$MAVEN_HOME/bin/$MVN_CMD" - exit $? -} - -if (! $distributionUrlNameMain -or ($distributionUrlName -eq $distributionUrlNameMain)) { - Write-Error "distributionUrl is not valid, must end with *-bin.zip, but found $distributionUrl" -} - -# prepare tmp dir -$TMP_DOWNLOAD_DIR_HOLDER = New-TemporaryFile -$TMP_DOWNLOAD_DIR = New-Item -Itemtype Directory -Path "$TMP_DOWNLOAD_DIR_HOLDER.dir" -$TMP_DOWNLOAD_DIR_HOLDER.Delete() | Out-Null -trap { - if ($TMP_DOWNLOAD_DIR.Exists) { - try { Remove-Item $TMP_DOWNLOAD_DIR -Recurse -Force | Out-Null } - catch { Write-Warning "Cannot remove $TMP_DOWNLOAD_DIR" } - } -} - -New-Item -Itemtype Directory -Path "$MAVEN_HOME_PARENT" -Force | Out-Null - -# Download and Install Apache Maven -Write-Verbose "Couldn't find MAVEN_HOME, downloading and installing it ..." -Write-Verbose "Downloading from: $distributionUrl" -Write-Verbose "Downloading to: $TMP_DOWNLOAD_DIR/$distributionUrlName" - -$webclient = New-Object System.Net.WebClient -if ($env:MVNW_USERNAME -and $env:MVNW_PASSWORD) { - $webclient.Credentials = New-Object System.Net.NetworkCredential($env:MVNW_USERNAME, $env:MVNW_PASSWORD) -} -[Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12 -$webclient.DownloadFile($distributionUrl, "$TMP_DOWNLOAD_DIR/$distributionUrlName") | Out-Null - -# If specified, validate the SHA-256 sum of the Maven distribution zip file -$distributionSha256Sum = (Get-Content -Raw "$scriptDir/.mvn/wrapper/maven-wrapper.properties" | ConvertFrom-StringData).distributionSha256Sum -if ($distributionSha256Sum) { - if ($USE_MVND) { - Write-Error "Checksum validation is not supported for maven-mvnd. `nPlease disable validation by removing 'distributionSha256Sum' from your maven-wrapper.properties." - } - Import-Module $PSHOME\Modules\Microsoft.PowerShell.Utility -Function Get-FileHash - if ((Get-FileHash "$TMP_DOWNLOAD_DIR/$distributionUrlName" -Algorithm SHA256).Hash.ToLower() -ne $distributionSha256Sum) { - Write-Error "Error: Failed to validate Maven distribution SHA-256, your Maven distribution might be compromised. If you updated your Maven version, you need to update the specified distributionSha256Sum property." - } -} - -# unzip and move -Expand-Archive "$TMP_DOWNLOAD_DIR/$distributionUrlName" -DestinationPath "$TMP_DOWNLOAD_DIR" | Out-Null - -# Find the actual extracted directory name (handles snapshots where filename != directory name) -$actualDistributionDir = "" - -# First try the expected directory name (for regular distributions) -$expectedPath = Join-Path "$TMP_DOWNLOAD_DIR" "$distributionUrlNameMain" -$expectedMvnPath = Join-Path "$expectedPath" "bin/$MVN_CMD" -if ((Test-Path -Path $expectedPath -PathType Container) -and (Test-Path -Path $expectedMvnPath -PathType Leaf)) { - $actualDistributionDir = $distributionUrlNameMain -} - -# If not found, search for any directory with the Maven executable (for snapshots) -if (!$actualDistributionDir) { - Get-ChildItem -Path "$TMP_DOWNLOAD_DIR" -Directory | ForEach-Object { - $testPath = Join-Path $_.FullName "bin/$MVN_CMD" - if (Test-Path -Path $testPath -PathType Leaf) { - $actualDistributionDir = $_.Name - } - } -} - -if (!$actualDistributionDir) { - Write-Error "Could not find Maven distribution directory in extracted archive" -} - -Write-Verbose "Found extracted Maven distribution directory: $actualDistributionDir" -Rename-Item -Path "$TMP_DOWNLOAD_DIR/$actualDistributionDir" -NewName $MAVEN_HOME_NAME | Out-Null -try { - Move-Item -Path "$TMP_DOWNLOAD_DIR/$MAVEN_HOME_NAME" -Destination $MAVEN_HOME_PARENT | Out-Null -} catch { - if (! (Test-Path -Path "$MAVEN_HOME" -PathType Container)) { - Write-Error "fail to move MAVEN_HOME" - } -} finally { - try { Remove-Item $TMP_DOWNLOAD_DIR -Recurse -Force | Out-Null } - catch { Write-Warning "Cannot remove $TMP_DOWNLOAD_DIR" } -} - -Write-Output "MVN_CMD=$MAVEN_HOME/bin/$MVN_CMD" diff --git a/parquet/pom.xml b/parquet/pom.xml deleted file mode 100644 index 449fc05..0000000 --- a/parquet/pom.xml +++ /dev/null @@ -1,42 +0,0 @@ - - - 4.0.0 - - io.github.dfa1.vortex - vortex-java - 0.3.0-SNAPSHOT - - - parquet - - - - io.github.dfa1.vortex - reader - - - io.github.dfa1.vortex - writer - - - dev.hardwood - hardwood-core - - - - org.junit.jupiter - junit-jupiter - test - - - org.assertj - assertj-core - test - - - org.mockito - mockito-junit-jupiter - test - - - diff --git a/parquet/src/main/java/io/github/dfa1/vortex/parquet/ImportOptions.java b/parquet/src/main/java/io/github/dfa1/vortex/parquet/ImportOptions.java deleted file mode 100644 index 4be8462..0000000 --- a/parquet/src/main/java/io/github/dfa1/vortex/parquet/ImportOptions.java +++ /dev/null @@ -1,38 +0,0 @@ -package io.github.dfa1.vortex.parquet; - -import io.github.dfa1.vortex.writer.WriteOptions; - -import java.util.List; - -/// Options controlling Parquet → Vortex import. -public record ImportOptions( - int chunkSize, - List columns, - ProgressListener progressListener, - WriteOptions writeOptions -) { - public static ImportOptions defaults() { - return new ImportOptions(131_072, List.of(), null, WriteOptions.cascading(3)); - } - - /// Restrict import to specific columns. Empty list = all columns. - public ImportOptions withColumns(List cols) { - return new ImportOptions(chunkSize, List.copyOf(cols), progressListener, writeOptions); - } - - public boolean hasProjection() { - return !columns.isEmpty(); - } - - public ImportOptions withProgressListener(ProgressListener listener) { - return new ImportOptions(chunkSize, columns, listener, writeOptions); - } - - public ImportOptions withWriteOptions(WriteOptions options) { - return new ImportOptions(chunkSize, columns, progressListener, options); - } - - public ImportOptions withChunkSize(int size) { - return new ImportOptions(size, columns, progressListener, writeOptions); - } -} diff --git a/parquet/src/main/java/io/github/dfa1/vortex/parquet/ParquetImporter.java b/parquet/src/main/java/io/github/dfa1/vortex/parquet/ParquetImporter.java deleted file mode 100644 index 0381029..0000000 --- a/parquet/src/main/java/io/github/dfa1/vortex/parquet/ParquetImporter.java +++ /dev/null @@ -1,276 +0,0 @@ -package io.github.dfa1.vortex.parquet; - -import dev.hardwood.InputFile; -import dev.hardwood.metadata.LogicalType; -import dev.hardwood.metadata.RepetitionType; -import dev.hardwood.reader.ParquetFileReader; -import dev.hardwood.reader.RowReader; -import dev.hardwood.schema.ColumnProjection; -import dev.hardwood.schema.ColumnSchema; -import dev.hardwood.schema.FileSchema; -import io.github.dfa1.vortex.core.DType; -import io.github.dfa1.vortex.core.PType; -import io.github.dfa1.vortex.encoding.DateTimePartsData; -import io.github.dfa1.vortex.encoding.TimeUnit; -import io.github.dfa1.vortex.writer.VortexWriter; - -import java.io.IOException; -import java.nio.ByteBuffer; -import java.nio.ByteOrder; -import java.nio.channels.FileChannel; -import java.nio.file.Path; -import java.nio.file.StandardOpenOption; -import java.util.ArrayList; -import java.util.Arrays; -import java.util.LinkedHashMap; -import java.util.List; -import java.util.Map; - -/// Reads a Parquet file and writes a Vortex file. -/// -/// Only flat schemas (no nested struct/list/map columns) are supported. -/// Nullable columns (OPTIONAL repetition) produce nulls mapped to type defaults: -/// 0 for numeric types, {@code false} for bool, {@code ""} for strings. -/// -/// Supported Parquet physical types: -///
    -///
  • BOOLEAN → Bool
    • -///
  • INT32 (no annotation / IntType 8/16/32, signed or unsigned) → I8/U8/I16/U16/I32/U32
    • -///
  • INT64 (no annotation / IntType 64, signed or unsigned) → I64/U64
    • -///
  • FLOAT → F32
    • -///
  • DOUBLE → F64
    • -///
  • BYTE_ARRAY annotated STRING, ENUM, or JSON → Utf8
    • -///
-/// -/// All other physical types (INT96, FIXED_LEN_BYTE_ARRAY, unannotated BYTE_ARRAY, DECIMAL, -/// DATE, TIME, TIMESTAMP) throw {@link UnsupportedOperationException}. -public final class ParquetImporter { - - private ParquetImporter() { - } - - public static void importParquet(Path parquetPath, Path vortexPath) throws IOException { - importParquet(parquetPath, vortexPath, ImportOptions.defaults()); - } - - public static void importParquet(Path parquetPath, Path vortexPath, ImportOptions options) throws IOException { - try (ParquetFileReader parquet = ParquetFileReader.open(InputFile.of(parquetPath))) { - FileSchema fileSchema = parquet.getFileSchema(); - if (!fileSchema.isFlatSchema()) { - throw new UnsupportedOperationException( - "nested Parquet schemas not yet supported; schema: " + fileSchema); - } - - List allColumns = fileSchema.getColumns(); - List columns = options.hasProjection() - ? filterColumns(allColumns, options.columns()) - : allColumns; - int colCount = columns.size(); - - List names = new ArrayList<>(colCount); - List types = new ArrayList<>(colCount); - for (ColumnSchema col : columns) { - names.add(col.name()); - types.add(mapDType(col)); - } - DType.Struct schema = new DType.Struct(names, types, false); - long totalRows = parquet.getFileMetaData().numRows(); - - ColumnProjection projection = options.hasProjection() - ? ColumnProjection.columns(options.columns().toArray(String[]::new)) - : ColumnProjection.all(); - - try (FileChannel channel = FileChannel.open( - vortexPath, - StandardOpenOption.CREATE, StandardOpenOption.WRITE, StandardOpenOption.TRUNCATE_EXISTING); - VortexWriter writer = VortexWriter.create(channel, schema, options.writeOptions()); - RowReader rowReader = parquet.buildRowReader().projection(projection).build()) { - - int chunkSize = options.chunkSize(); - Object[] buffers = allocateBuffers(columns, chunkSize); - int chunkPos = 0; - long rowsDone = 0; - - while (rowReader.hasNext()) { - rowReader.next(); - fillRow(rowReader, columns, buffers, chunkPos); - chunkPos++; - rowsDone++; - - if (chunkPos == chunkSize) { - writer.writeChunk(buildChunk(columns, types, buffers, chunkPos)); - buffers = allocateBuffers(columns, chunkSize); - chunkPos = 0; - if (options.progressListener() != null) { - options.progressListener().onProgress(rowsDone, totalRows); - } - } - } - - if (chunkPos > 0) { - writer.writeChunk(buildChunk(columns, types, buffers, chunkPos)); - if (options.progressListener() != null) { - options.progressListener().onProgress(rowsDone, totalRows); - } - } - } - } - } - - private static DType mapDType(ColumnSchema col) { - boolean nullable = col.repetitionType() == RepetitionType.OPTIONAL; - return switch (col.type()) { - case BOOLEAN -> new DType.Bool(nullable); - case INT32 -> mapInt32(col.logicalType(), nullable); - case INT64 -> mapInt64(col.logicalType(), nullable); - case FLOAT -> new DType.Primitive(PType.F32, nullable); - case DOUBLE -> new DType.Primitive(PType.F64, nullable); - case BYTE_ARRAY -> mapByteArray(col.logicalType(), nullable, col.name()); - default -> throw new UnsupportedOperationException( - "unsupported Parquet physical type: " + col.type() + " (column: " + col.name() + ")"); - }; - } - - private static DType mapInt32(LogicalType logical, boolean nullable) { - if (logical instanceof LogicalType.IntType it) { - PType ptype = switch (it.bitWidth()) { - case 8 -> it.isSigned() ? PType.I8 : PType.U8; - case 16 -> it.isSigned() ? PType.I16 : PType.U16; - default -> it.isSigned() ? PType.I32 : PType.U32; - }; - return new DType.Primitive(ptype, nullable); - } - return new DType.Primitive(PType.I32, nullable); - } - - private static DType mapInt64(LogicalType logical, boolean nullable) { - if (logical instanceof LogicalType.IntType it) { - return new DType.Primitive(it.isSigned() ? PType.I64 : PType.U64, nullable); - } - if (logical instanceof LogicalType.TimestampType ts) { - return timestampExtension(ts.unit(), nullable); - } - return new DType.Primitive(PType.I64, nullable); - } - - private static DType.Extension timestampExtension(LogicalType.TimeUnit parquetUnit, boolean nullable) { - TimeUnit unit = switch (parquetUnit) { - case MILLIS -> TimeUnit.Milliseconds; - case MICROS -> TimeUnit.Microseconds; - case NANOS -> TimeUnit.Nanoseconds; - }; - // Extension metadata: byte[0]=unit tag, bytes[1-2]=tz_len u16 LE (0 = no tz) - ByteBuffer meta = ByteBuffer.allocate(3).order(ByteOrder.LITTLE_ENDIAN); - meta.put((byte) unit.ordinal()); - meta.putShort((short) 0); - meta.flip(); - return new DType.Extension("vortex.timestamp", new DType.Primitive(PType.I64, nullable), meta, nullable); - } - - private static DType mapByteArray(LogicalType logical, boolean nullable, String colName) { - if (logical instanceof LogicalType.StringType - || logical instanceof LogicalType.EnumType - || logical instanceof LogicalType.JsonType) { - return new DType.Utf8(nullable); - } - throw new UnsupportedOperationException( - "BYTE_ARRAY column '" + colName + "' has no string logical type annotation; " - + "only STRING / ENUM / JSON are supported (got: " + logical + ")"); - } - - private static Object[] allocateBuffers(List columns, int chunkSize) { - Object[] buffers = new Object[columns.size()]; - for (int c = 0; c < columns.size(); c++) { - buffers[c] = allocateBuffer(columns.get(c), chunkSize); - } - return buffers; - } - - private static Object allocateBuffer(ColumnSchema col, int chunkSize) { - return switch (col.type()) { - case BOOLEAN -> new boolean[chunkSize]; - case FLOAT -> new float[chunkSize]; - case DOUBLE -> new double[chunkSize]; - case BYTE_ARRAY -> new String[chunkSize]; - case INT32 -> { - if (col.logicalType() instanceof LogicalType.IntType it) { - yield switch (it.bitWidth()) { - case 8 -> new byte[chunkSize]; - case 16 -> new short[chunkSize]; - default -> new int[chunkSize]; - }; - } - yield new int[chunkSize]; - } - case INT64 -> new long[chunkSize]; // covers plain I64 and timestamps - default -> throw new UnsupportedOperationException("unsupported type: " + col.type()); - }; - } - - private static void fillRow(RowReader reader, List columns, Object[] buffers, int pos) { - for (int c = 0; c < columns.size(); c++) { - ColumnSchema col = columns.get(c); - String name = col.name(); - boolean isNull = col.repetitionType() == RepetitionType.OPTIONAL && reader.isNull(name); - switch (buffers[c]) { - case boolean[] arr -> arr[pos] = !isNull && reader.getBoolean(name); - case float[] arr -> arr[pos] = isNull ? 0f : reader.getFloat(name); - case double[] arr -> arr[pos] = isNull ? 0.0 : reader.getDouble(name); - case long[] arr -> arr[pos] = isNull ? 0L : reader.getLong(name); - case int[] arr -> arr[pos] = isNull ? 0 : reader.getInt(name); - case short[] arr -> arr[pos] = isNull ? (short) 0 : (short) reader.getInt(name); - case byte[] arr -> arr[pos] = isNull ? (byte) 0 : (byte) reader.getInt(name); - case String[] arr -> arr[pos] = isNull ? "" : reader.getString(name); - default -> throw new UnsupportedOperationException( - "unexpected buffer type: " + buffers[c].getClass().getSimpleName()); - } - } - } - - private static Map buildChunk(List columns, List types, - Object[] buffers, int size) { - Map chunk = new LinkedHashMap<>(); - for (int c = 0; c < columns.size(); c++) { - Object buf = trimBuffer(buffers[c], size); - if (types.get(c) instanceof DType.Extension) { - boolean nullable = types.get(c).nullable(); - buf = new DateTimePartsData((long[]) buf, nullable); - } - chunk.put(columns.get(c).name(), buf); - } - return chunk; - } - - private static List filterColumns(List all, List names) { - List result = new ArrayList<>(names.size()); - for (String name : names) { - boolean found = false; - for (ColumnSchema col : all) { - if (col.name().equals(name)) { - result.add(col); - found = true; - break; - } - } - if (!found) { - throw new IllegalArgumentException("column not found in Parquet schema: " + name); - } - } - return result; - } - - private static Object trimBuffer(Object buffer, int size) { - return switch (buffer) { - case boolean[] arr -> size == arr.length ? arr : Arrays.copyOf(arr, size); - case float[] arr -> size == arr.length ? arr : Arrays.copyOf(arr, size); - case double[] arr -> size == arr.length ? arr : Arrays.copyOf(arr, size); - case long[] arr -> size == arr.length ? arr : Arrays.copyOf(arr, size); - case int[] arr -> size == arr.length ? arr : Arrays.copyOf(arr, size); - case short[] arr -> size == arr.length ? arr : Arrays.copyOf(arr, size); - case byte[] arr -> size == arr.length ? arr : Arrays.copyOf(arr, size); - case String[] arr -> size == arr.length ? arr : Arrays.copyOf(arr, size); - default -> throw new UnsupportedOperationException( - "unexpected buffer type: " + buffer.getClass().getSimpleName()); - }; - } -} diff --git a/parquet/src/main/java/io/github/dfa1/vortex/parquet/ProgressListener.java b/parquet/src/main/java/io/github/dfa1/vortex/parquet/ProgressListener.java deleted file mode 100644 index ee43360..0000000 --- a/parquet/src/main/java/io/github/dfa1/vortex/parquet/ProgressListener.java +++ /dev/null @@ -1,7 +0,0 @@ -package io.github.dfa1.vortex.parquet; - -/// Callback invoked after each chunk is written during Parquet import. -@FunctionalInterface -public interface ProgressListener { - void onProgress(long rowsDone, long rowsTotal); -} diff --git a/performance/pom.xml b/performance/pom.xml deleted file mode 100644 index 41f0b91..0000000 --- a/performance/pom.xml +++ /dev/null @@ -1,123 +0,0 @@ - - - 4.0.0 - - io.github.dfa1.vortex - vortex-java - 0.3.0-SNAPSHOT - - - performance - - - true - - - - - io.github.dfa1.vortex - writer - compile - - - io.github.dfa1.vortex - parquet - compile - - - dev.hardwood - hardwood-core - compile - - - io.github.dfa1.vortex - reader - compile - - - dev.vortex - vortex-jni - compile - - - org.apache.arrow - arrow-vector - compile - - - org.apache.arrow - arrow-c-data - compile - - - org.apache.arrow - arrow-memory-unsafe - compile - - - org.openjdk.jmh - jmh-core - compile - - - org.openjdk.jmh - jmh-generator-annprocess - provided - - - org.slf4j - slf4j-simple - runtime - - - - - - - org.apache.maven.plugins - maven-compiler-plugin - - 25 - - - org.openjdk.jmh - jmh-generator-annprocess - - - - - - - org.apache.maven.plugins - maven-shade-plugin - - - package - - shade - - - benchmarks - - - org.openjdk.jmh.Main - - - - - - *:* - - META-INF/*.SF - META-INF/*.DSA - META-INF/*.RSA - - - - - - - - - - diff --git a/performance/src/main/java/io/github/dfa1/vortex/performance/ParquetVsVortexReadBenchmark.java b/performance/src/main/java/io/github/dfa1/vortex/performance/ParquetVsVortexReadBenchmark.java deleted file mode 100644 index 26ff79b..0000000 --- a/performance/src/main/java/io/github/dfa1/vortex/performance/ParquetVsVortexReadBenchmark.java +++ /dev/null @@ -1,243 +0,0 @@ -package io.github.dfa1.vortex.performance; - -import dev.hardwood.InputFile; -import dev.hardwood.reader.ColumnReader; -import dev.hardwood.reader.ColumnReaders; -import dev.hardwood.reader.ParquetFileReader; -import dev.hardwood.reader.RowReader; -import dev.hardwood.schema.ColumnProjection; -import io.github.dfa1.vortex.core.array.DoubleArray; -import io.github.dfa1.vortex.core.array.IntArray; -import io.github.dfa1.vortex.encoding.EncodingRegistry; -import io.github.dfa1.vortex.io.VortexReader; -import io.github.dfa1.vortex.parquet.ParquetImporter; -import io.github.dfa1.vortex.scan.ScanResult; -import org.openjdk.jmh.annotations.Benchmark; -import org.openjdk.jmh.annotations.BenchmarkMode; -import org.openjdk.jmh.annotations.Fork; -import org.openjdk.jmh.annotations.Level; -import org.openjdk.jmh.annotations.Measurement; -import org.openjdk.jmh.annotations.Mode; -import org.openjdk.jmh.annotations.OutputTimeUnit; -import org.openjdk.jmh.annotations.Scope; -import org.openjdk.jmh.annotations.Setup; -import org.openjdk.jmh.annotations.State; -import org.openjdk.jmh.annotations.TearDown; -import org.openjdk.jmh.annotations.Warmup; - -import java.io.IOException; -import java.io.InputStream; -import java.net.URI; -import java.net.http.HttpClient; -import java.net.http.HttpRequest; -import java.net.http.HttpResponse; -import java.nio.file.Files; -import java.nio.file.Path; -import java.util.concurrent.TimeUnit; - -/// Benchmark: Hardwood Parquet read vs Java Vortex read on the same real-world dataset. -/// -/// Dataset: NYC Yellow Taxi 2024-01 (~3M rows). -/// Columns: trip_distance (DOUBLE), fare_amount (DOUBLE), PULocationID (INT32). -/// -/// Two Parquet variants: -/// - {@code parquetRead} / {@code parquetReadMultiColumn}: batch column API -/// ({@link ColumnReader#nextBatch()} + {@link ColumnReader#getDoubles()}) — -/// apples-to-apples comparison with Vortex's batch fold. -/// - {@code parquetReadRowByRow} / {@code parquetReadMultiColumnRowByRow}: -/// Hardwood row cursor ({@link RowReader}) — measures row-oriented API overhead -/// on top of format decode cost. -/// -/// Override the Parquet source with: {@code -Dbench.parquet=/path/to/file.parquet} -/// -/// Run: {@code java -jar performance/target/benchmarks.jar ParquetVsVortexReadBenchmark} -@State(Scope.Benchmark) -@BenchmarkMode(Mode.Throughput) -@OutputTimeUnit(TimeUnit.SECONDS) -@Warmup(iterations = 3, time = 3) -@Measurement(iterations = 5, time = 5) -@Fork(value = 1, jvmArgsAppend = { - "--add-opens", "java.base/java.nio=ALL-UNNAMED", - "--enable-native-access=ALL-UNNAMED", - "--sun-misc-unsafe-memory-access=allow" -}) -public class ParquetVsVortexReadBenchmark { - - private static final String PARQUET_URL = - "https://d37ci6vzurychx.cloudfront.net/trip-data/yellow_tripdata_2024-01.parquet"; - private static final Path CACHE_PATH = - Path.of(System.getProperty("java.io.tmpdir"), "yellow_tripdata_2024-01.parquet"); - private static final Object SETUP_LOCK = new Object(); - private static Path sharedParquetFile; - private static Path sharedVortexFile; - private static int refCount; - - private Path parquetFile; - private Path vortexFile; - private EncodingRegistry registry; - - @Setup(Level.Trial) - public void setup() throws Exception { - registry = EncodingRegistry.loadAll(); - synchronized (SETUP_LOCK) { - if (sharedParquetFile == null) { - String override = System.getProperty("bench.parquet"); - if (override != null && !override.isEmpty()) { - sharedParquetFile = Path.of(override); - System.out.printf("[ParquetVsVortexReadBenchmark] using override: %s%n", sharedParquetFile); - } else { - sharedParquetFile = downloadIfAbsent(CACHE_PATH, PARQUET_URL); - System.out.printf("[ParquetVsVortexReadBenchmark] parquet: %s (%.1f MB)%n", - sharedParquetFile, Files.size(sharedParquetFile) / 1_048_576.0); - } - sharedVortexFile = Files.createTempFile("taxi-bench", ".vortex"); - System.out.print("[ParquetVsVortexReadBenchmark] converting to vortex (all columns)..."); - ParquetImporter.importParquet(sharedParquetFile, sharedVortexFile); - System.out.printf(" done (%.1f MB)%n", Files.size(sharedVortexFile) / 1_048_576.0); - } - refCount++; - parquetFile = sharedParquetFile; - vortexFile = sharedVortexFile; - } - } - - @TearDown(Level.Trial) - public void tearDown() throws IOException { - synchronized (SETUP_LOCK) { - refCount--; - if (refCount == 0) { - Files.deleteIfExists(sharedVortexFile); - sharedVortexFile = null; - sharedParquetFile = null; - } - } - } - - // ── Batch API (apples-to-apples with Vortex) ───────────────────────────── - - /// Hardwood batch: decode trip_distance column in chunks, sum all values. - @Benchmark - public double parquetRead() throws IOException { - double sum = 0.0; - try (ParquetFileReader reader = ParquetFileReader.open(InputFile.of(parquetFile)); - ColumnReader cr = reader.columnReader("trip_distance")) { - while (cr.nextBatch()) { - double[] vals = cr.getDoubles(); - int n = cr.getRecordCount(); - for (int i = 0; i < n; i++) { - sum += vals[i]; - } - } - } - return sum; - } - - /// Hardwood batch: decode fare_amount + PULocationID in chunks, sum both. - @Benchmark - public double parquetReadMultiColumn() throws IOException { - double fareSum = 0.0; - long idSum = 0L; - try (ParquetFileReader reader = ParquetFileReader.open(InputFile.of(parquetFile)); - ColumnReaders crs = reader.columnReaders(ColumnProjection.columns("fare_amount", "PULocationID"))) { - while (crs.nextBatch()) { - int n = crs.getRecordCount(); - double[] fares = crs.getColumnReader("fare_amount").getDoubles(); - int[] locs = crs.getColumnReader("PULocationID").getInts(); - for (int i = 0; i < n; i++) { - fareSum += fares[i]; - idSum += locs[i]; - } - } - } - return fareSum + idSum; - } - - // ── Row cursor API (measures row-oriented overhead on top of decode) ────── - - /// Hardwood row cursor: sum trip_distance, one virtual call per row. - @Benchmark - public double parquetReadRowByRow() throws IOException { - double sum = 0.0; - ColumnProjection projection = ColumnProjection.columns("trip_distance"); - try (ParquetFileReader reader = ParquetFileReader.open(InputFile.of(parquetFile)); - RowReader rows = reader.buildRowReader().projection(projection).build()) { - while (rows.hasNext()) { - rows.next(); - sum += rows.getDouble("trip_distance"); - } - } - return sum; - } - - /// Hardwood row cursor: sum fare_amount + PULocationID, two virtual calls per row. - @Benchmark - public double parquetReadMultiColumnRowByRow() throws IOException { - double fareSum = 0.0; - long idSum = 0L; - ColumnProjection projection = ColumnProjection.columns("fare_amount", "PULocationID"); - try (ParquetFileReader reader = ParquetFileReader.open(InputFile.of(parquetFile)); - RowReader rows = reader.buildRowReader().projection(projection).build()) { - while (rows.hasNext()) { - rows.next(); - fareSum += rows.getDouble("fare_amount"); - idSum += rows.getInt("PULocationID"); - } - } - return fareSum + idSum; - } - - // ── Vortex ──────────────────────────────────────────────────────────────── - - /// Vortex: decode trip_distance in chunks, sum via batch fold. - @Benchmark - public double vortexRead() throws IOException { - double sum = 0.0; - try (VortexReader vr = VortexReader.open(vortexFile, registry)) { - var iter = vr.scan(io.github.dfa1.vortex.scan.ScanOptions.columns("trip_distance")); - while (iter.hasNext()) { - ScanResult r = iter.next(); - DoubleArray col = r.column("trip_distance"); - sum += col.fold(0.0, Double::sum); - } - } - return sum; - } - - /// Vortex: decode fare_amount + PULocationID in chunks, sum both via batch fold. - @Benchmark - public double vortexReadMultiColumn() throws IOException { - double fareSum = 0.0; - long idSum = 0L; - try (VortexReader vr = VortexReader.open(vortexFile, registry)) { - var iter = vr.scan(io.github.dfa1.vortex.scan.ScanOptions.columns("fare_amount", "PULocationID")); - while (iter.hasNext()) { - ScanResult r = iter.next(); - DoubleArray fare = r.column("fare_amount"); - IntArray loc = r.column("PULocationID"); - fareSum += fare.fold(0.0, Double::sum); - idSum += loc.fold(0, Integer::sum); - } - } - return fareSum + idSum; - } - - // ── Setup helper ────────────────────────────────────────────────────────── - - private static Path downloadIfAbsent(Path dest, String url) throws Exception { - if (Files.exists(dest)) { - System.out.printf("[ParquetVsVortexReadBenchmark] cache hit: %s%n", dest); - return dest; - } - System.out.printf("[ParquetVsVortexReadBenchmark] downloading %s...%n", url); - HttpClient client = HttpClient.newBuilder().followRedirects(HttpClient.Redirect.NORMAL).build(); - HttpRequest request = HttpRequest.newBuilder().uri(URI.create(url)).build(); - HttpResponse response = client.send(request, HttpResponse.BodyHandlers.ofInputStream()); - if (response.statusCode() != 200) { - throw new IOException("HTTP " + response.statusCode() + " for " + url); - } - try (InputStream in = response.body()) { - Files.copy(in, dest); - } - return dest; - } -} diff --git a/performance/src/main/java/io/github/dfa1/vortex/performance/RustVsJavaReadBenchmark.java b/performance/src/main/java/io/github/dfa1/vortex/performance/RustVsJavaReadBenchmark.java deleted file mode 100644 index 5013e72..0000000 --- a/performance/src/main/java/io/github/dfa1/vortex/performance/RustVsJavaReadBenchmark.java +++ /dev/null @@ -1,483 +0,0 @@ -package io.github.dfa1.vortex.performance; - -import dev.vortex.api.DataSource; -import dev.vortex.api.Expression; -import dev.vortex.api.Partition; -import dev.vortex.api.Scan; -import dev.vortex.api.ScanOptions; -import dev.vortex.api.Session; -import dev.vortex.arrow.ArrowAllocation; -import dev.vortex.jni.NativeLoader; -import io.github.dfa1.vortex.core.DType; -import io.github.dfa1.vortex.core.PType; -import io.github.dfa1.vortex.core.array.DoubleArray; -import io.github.dfa1.vortex.core.array.LongArray; -import io.github.dfa1.vortex.core.array.VarBinArray; -import io.github.dfa1.vortex.encoding.EncodingRegistry; -import io.github.dfa1.vortex.io.VortexReader; -import io.github.dfa1.vortex.writer.VortexWriter; -import io.github.dfa1.vortex.writer.WriteOptions; -import io.github.dfa1.vortex.scan.ScanResult; -import org.apache.arrow.c.ArrowArray; -import org.apache.arrow.c.ArrowSchema; -import org.apache.arrow.c.Data; -import org.apache.arrow.memory.BufferAllocator; -import org.apache.arrow.vector.BigIntVector; -import org.apache.arrow.vector.DateDayVector; -import org.apache.arrow.vector.Float8Vector; -import org.apache.arrow.vector.VarCharVector; -import org.apache.arrow.vector.VectorSchemaRoot; -import org.apache.arrow.vector.ipc.ArrowReader; -import org.apache.arrow.vector.types.DateUnit; -import org.apache.arrow.vector.types.FloatingPointPrecision; -import org.apache.arrow.vector.types.pojo.ArrowType; -import org.apache.arrow.vector.types.pojo.Field; -import org.apache.arrow.vector.types.pojo.Schema; -import org.openjdk.jmh.annotations.Benchmark; -import org.openjdk.jmh.annotations.BenchmarkMode; -import org.openjdk.jmh.annotations.Fork; -import org.openjdk.jmh.annotations.Level; -import org.openjdk.jmh.annotations.Measurement; -import org.openjdk.jmh.annotations.Mode; -import org.openjdk.jmh.annotations.OutputTimeUnit; -import org.openjdk.jmh.annotations.Scope; -import org.openjdk.jmh.annotations.Setup; -import org.openjdk.jmh.annotations.State; -import org.openjdk.jmh.annotations.TearDown; -import org.openjdk.jmh.annotations.Warmup; - -import java.io.IOException; -import java.nio.channels.FileChannel; -import java.nio.charset.StandardCharsets; -import java.nio.file.Files; -import java.nio.file.Path; -import java.nio.file.StandardOpenOption; -import java.time.LocalDate; -import java.util.Arrays; -import java.util.HashMap; -import java.util.List; -import java.util.Map; -import java.util.Random; -import java.util.concurrent.TimeUnit; - -/// Read benchmark: JNI Vortex reader vs Java reader on the same file. -/// -/// The file is written by the JNI VortexWriter in `@Setup`, giving a realistic -/// file with whatever encodings the Rust implementation chooses. -/// -/// Both benchmarks project onto the "close" column (F64) and sum all values so -/// the JVM can't optimise away the decode work. -/// -/// Run: java -jar performance/target/benchmarks.jar RustVsJavaReadBenchmark -/// -/// To test against a pre-existing JNI-written file: -/// java -Dvortex.bench.ohlc=/path/to/file.vtx -jar target/benchmarks.jar RustVsJavaReadBenchmark -@State(Scope.Benchmark) -@BenchmarkMode(Mode.Throughput) -@OutputTimeUnit(TimeUnit.SECONDS) -@Warmup(iterations = 3, time = 3) -@Measurement(iterations = 5, time = 5) -@Fork(value = 1, jvmArgsAppend = { - "--add-opens", "java.base/java.nio=ALL-UNNAMED", - "--enable-native-access=ALL-UNNAMED", - "--sun-misc-unsafe-memory-access=allow" -}) -public class RustVsJavaReadBenchmark { - - private static final int TOTAL_ROWS = 10_000_000; - private static final int BATCH_SIZE = 50_000; // 200 chunks - private static final ArrowType F64_TYPE = new ArrowType.FloatingPoint(FloatingPointPrecision.DOUBLE); - private static final Schema JNI_SCHEMA = new Schema(List.of( - Field.notNullable("date", new ArrowType.Date(DateUnit.DAY)), - Field.notNullable("symbol", ArrowType.Utf8.INSTANCE), - Field.notNullable("open", F64_TYPE), - Field.notNullable("high", F64_TYPE), - Field.notNullable("low", F64_TYPE), - Field.notNullable("close", F64_TYPE), - Field.notNullable("volume", new ArrowType.Int(64, true)) - )); - private static final Session SESSION = Session.create(); - - static { - NativeLoader.loadJni(); - } - - private static final DType.Struct JAVA_SCHEMA = new DType.Struct( - List.of("date", "symbol", "open", "high", "low", "close", "volume"), - List.of( - new DType.Primitive(PType.I32, false), - new DType.Utf8(false), - new DType.Primitive(PType.F64, false), - new DType.Primitive(PType.F64, false), - new DType.Primitive(PType.F64, false), - new DType.Primitive(PType.F64, false), - new DType.Primitive(PType.I64, false) - ), - false); - - private static final Object FILE_LOCK = new Object(); - private static Path sharedBenchFile; - private static boolean sharedOwnFile; - private static Path sharedCascadingFile; - private static int sharedRefCount; - - private Path benchFile; - private Path cascadingFile; - private EncodingRegistry registry; - private BufferAllocator allocator; - - private static double round(double v) { - return Math.round(v * 100.0) / 100.0; - } - - @Setup(Level.Trial) - public void setup() throws IOException { - registry = EncodingRegistry.loadAll(); - allocator = ArrowAllocation.rootAllocator(); - - synchronized (FILE_LOCK) { - if (sharedBenchFile == null) { - String externalFile = System.getProperty("vortex.bench.ohlc"); - if (externalFile != null && !externalFile.isEmpty()) { - sharedBenchFile = Path.of(externalFile); - sharedOwnFile = false; - System.out.printf("[RustVsJavaReadBenchmark] using external file: %s%n", sharedBenchFile); - } else { - sharedBenchFile = Files.createTempFile("ohlc-bench", ".vtx"); - sharedOwnFile = true; - System.out.printf("[RustVsJavaReadBenchmark] writing %d OHLC rows via JNI...%n", TOTAL_ROWS); - writeJni(sharedBenchFile); - System.out.printf("[RustVsJavaReadBenchmark] file size: %.1f MB%n", - Files.size(sharedBenchFile) / 1_048_576.0); - } - } - if (sharedCascadingFile == null) { - sharedCascadingFile = Files.createTempFile("ohlc-java-cascading", ".vtx"); - System.out.printf("[RustVsJavaReadBenchmark] writing %d OHLC rows (cascading depth 3) via Java...%n", TOTAL_ROWS); - writeJavaCascading(sharedCascadingFile); - System.out.printf("[RustVsJavaReadBenchmark] cascading file size: %.1f MB%n", - Files.size(sharedCascadingFile) / 1_048_576.0); - } - sharedRefCount++; - benchFile = sharedBenchFile; - cascadingFile = sharedCascadingFile; - } - } - - @TearDown(Level.Trial) - public void cleanup() throws IOException { - synchronized (FILE_LOCK) { - sharedRefCount--; - if (sharedRefCount == 0) { - if (sharedOwnFile) { - Files.deleteIfExists(sharedBenchFile); - sharedBenchFile = null; - } - if (sharedCascadingFile != null) { - Files.deleteIfExists(sharedCascadingFile); - sharedCascadingFile = null; - } - } - } - } - - /// JNI read: project on "volume", sum all values. - @Benchmark - public long jniReadVolume() throws IOException { - String uri = benchFile.toAbsolutePath().toUri().toString(); - var opts = ScanOptions.builder() - .projection(Expression.select(new String[]{"volume"}, Expression.root())) - .build(); - - long sum = 0L; - DataSource ds = DataSource.open(SESSION, uri); - Scan scan = ds.scan(opts); - while (scan.hasNext()) { - Partition partition = scan.next(); - try (ArrowReader reader = partition.scanArrow(allocator)) { - while (reader.loadNextBatch()) { - VectorSchemaRoot root = reader.getVectorSchemaRoot(); - BigIntVector volumeVec = (BigIntVector) root.getVector("volume"); - for (int i = 0; i < root.getRowCount(); i++) { - sum += volumeVec.get(i); - } - } - } - } - return sum; - } - - /// JNI read: project on "close" (F64, ALP-encoded), sum all values. - @Benchmark - public double jniReadClose() throws IOException { - String uri = benchFile.toAbsolutePath().toUri().toString(); - var opts = ScanOptions.builder() - .projection(Expression.select(new String[]{"close"}, Expression.root())) - .build(); - - double sum = 0.0; - DataSource ds = DataSource.open(SESSION, uri); - Scan scan = ds.scan(opts); - while (scan.hasNext()) { - Partition partition = scan.next(); - try (ArrowReader reader = partition.scanArrow(allocator)) { - while (reader.loadNextBatch()) { - VectorSchemaRoot root = reader.getVectorSchemaRoot(); - Float8Vector closeVec = (Float8Vector) root.getVector("close"); - for (int i = 0; i < root.getRowCount(); i++) { - sum += closeVec.get(i); - } - } - } - } - return sum; - } - - /// JNI read: project on "symbol" (short UTF-8 string, varbin), sum byte lengths. - @Benchmark - public long jniReadSymbol() throws IOException { - String uri = benchFile.toAbsolutePath().toUri().toString(); - var opts = ScanOptions.builder() - .projection(Expression.select(new String[]{"symbol"}, Expression.root())) - .build(); - - long sum = 0L; - DataSource ds = DataSource.open(SESSION, uri); - Scan scan = ds.scan(opts); - while (scan.hasNext()) { - Partition partition = scan.next(); - try (ArrowReader reader = partition.scanArrow(allocator)) { - while (reader.loadNextBatch()) { - VectorSchemaRoot root = reader.getVectorSchemaRoot(); - VarCharVector symbolVec = (VarCharVector) root.getVector("symbol"); - for (int i = 0; i < root.getRowCount(); i++) { - sum += symbolVec.getObject(i).getBytes().length; - } - } - } - } - return sum; - } - - // ── Cascading (Java-written) benchmarks ────────────────────────────────── - - /// Java read: cascading file (depth 3), project on "volume", sum via fold. - @Benchmark - public long javaReadCascading() throws IOException { - long sum = 0L; - try (VortexReader vf = VortexReader.open(cascadingFile, registry)) { - var iter = vf.scan(io.github.dfa1.vortex.scan.ScanOptions.columns("volume")); - while (iter.hasNext()) { - ScanResult r = iter.next(); - LongArray volume = r.column("volume"); - sum += volume.fold(0L, Long::sum); - } - } - return sum; - } - - // ── JNI file generation ─────────────────────────────────────────────────── - - /// Java read: project on "volume", sum all values via fold. - @Benchmark - public long javaReadVolume() throws IOException { - long sum = 0L; - try (VortexReader vf = VortexReader.open(benchFile, registry)) { - var iter = vf.scan(io.github.dfa1.vortex.scan.ScanOptions.columns("volume")); - while (iter.hasNext()) { - ScanResult r = iter.next(); - LongArray volume = r.column("volume"); - sum += volume.fold(0L, Long::sum); - } - } - return sum; - } - - /// Java read: project on "close" (F64, ALP-encoded), sum via fold. - @Benchmark - public double javaReadClose() throws IOException { - double sum = 0.0; - try (VortexReader vf = VortexReader.open(benchFile, registry)) { - var iter = vf.scan(io.github.dfa1.vortex.scan.ScanOptions.columns("close")); - while (iter.hasNext()) { - ScanResult r = iter.next(); - DoubleArray close = r.column("close"); - sum += close.fold(0.0, Double::sum); - } - } - return sum; - } - - /// Java read: project on "symbol" (short UTF-8 string, varbin), sum byte lengths. - @Benchmark - public long javaReadSymbol() throws IOException { - long[] sum = {0L}; - try (VortexReader vf = VortexReader.open(benchFile, registry)) { - var iter = vf.scan(io.github.dfa1.vortex.scan.ScanOptions.columns("symbol")); - while (iter.hasNext()) { - ScanResult r = iter.next(); - VarBinArray symbol = r.column("symbol"); - symbol.forEachByteLength(v -> sum[0] += v); - } - } - return sum[0]; - } - - // Real Nasdaq tickers — mix of lengths (1–5 chars) for realistic varbin distribution. - private static final String[] NASDAQ_TICKERS = { - "AAPL", "MSFT", "NVDA", "AMZN", "META", "GOOGL", "TSLA", "AVGO", "COST", "NFLX", - "AMD", "ADBE", "QCOM", "PEP", "CSCO", "TXN", "INTC", "CMCSA", "INTU", "AMGN", - "HON", "AMAT", "MU", "LRCX", "KLAC", "MRVL", "PANW", "SNPS", "CDNS", "REGN" - }; - private static final byte[][] TICKER_BYTES; - - static { - TICKER_BYTES = new byte[NASDAQ_TICKERS.length][]; - for (int i = 0; i < NASDAQ_TICKERS.length; i++) { - TICKER_BYTES[i] = NASDAQ_TICKERS[i].getBytes(StandardCharsets.UTF_8); - } - } - - private void writeJavaCascading(Path path) throws IOException { - int[] epochDays = new int[BATCH_SIZE]; - String[] symbols = new String[BATCH_SIZE]; - double[] open = new double[BATCH_SIZE]; - double[] high = new double[BATCH_SIZE]; - double[] low = new double[BATCH_SIZE]; - double[] close = new double[BATCH_SIZE]; - long[] volume = new long[BATCH_SIZE]; - - double[] prices = new double[NASDAQ_TICKERS.length]; - Arrays.fill(prices, 100.0); - var rng = new Random(42L); - int day = (int) LocalDate.of(2020, 1, 2).toEpochDay(); - int rowsLeft = TOTAL_ROWS; - - try (FileChannel ch = FileChannel.open(path, - StandardOpenOption.CREATE, StandardOpenOption.WRITE, StandardOpenOption.TRUNCATE_EXISTING); - VortexWriter writer = VortexWriter.create(ch, JAVA_SCHEMA, WriteOptions.cascading(3))) { - while (rowsLeft > 0) { - int n = Math.min(rowsLeft, BATCH_SIZE); - for (int i = 0; i < n; i++) { - int ticker = i % NASDAQ_TICKERS.length; - double px = prices[ticker]; - double ret = rng.nextGaussian() * 0.02; - double o = round(px * (1 + ret * 0.3)); - double c = round(px * (1 + ret)); - double spread = Math.abs(px * rng.nextDouble() * 0.03); - double h = round(Math.max(o, c) + spread); - double l = round(Math.min(o, c) - spread); - epochDays[i] = day + i / NASDAQ_TICKERS.length; - symbols[i] = NASDAQ_TICKERS[ticker]; - open[i] = o; - high[i] = h; - low[i] = l; - close[i] = c; - volume[i] = Math.max(100_000L, Math.round(1_000_000 + rng.nextGaussian() * 200_000)); - prices[ticker] = c; - } - day += n / NASDAQ_TICKERS.length; - writer.writeChunk(Map.of( - "date", epochDays, - "symbol", symbols, - "open", open, - "high", high, - "low", low, - "close", close, - "volume", volume - )); - rowsLeft -= n; - } - } - } - - private void writeJni(Path path) throws IOException { - String uri = path.toAbsolutePath().toUri().toString(); - try (dev.vortex.api.VortexWriter writer = dev.vortex.api.VortexWriter.create( - SESSION, uri, JNI_SCHEMA, new HashMap<>(), allocator)) { - // reuse arrays — filled per-batch - int[] epochDays = new int[BATCH_SIZE]; - byte[][] symbols = new byte[BATCH_SIZE][]; - double[] open = new double[BATCH_SIZE]; - double[] high = new double[BATCH_SIZE]; - double[] low = new double[BATCH_SIZE]; - double[] close = new double[BATCH_SIZE]; - long[] volume = new long[BATCH_SIZE]; - - // Per-ticker price state so each symbol has independent price evolution. - double[] prices = new double[NASDAQ_TICKERS.length]; - Arrays.fill(prices, 100.0); - - var rng = new Random(42L); - int day = (int) LocalDate.of(2020, 1, 2).toEpochDay(); - int rowsLeft = TOTAL_ROWS; - - while (rowsLeft > 0) { - int n = Math.min(rowsLeft, BATCH_SIZE); - for (int i = 0; i < n; i++) { - int ticker = i % NASDAQ_TICKERS.length; - double px = prices[ticker]; - double ret = rng.nextGaussian() * 0.02; - double o = round(px * (1 + ret * 0.3)); - double c = round(px * (1 + ret)); - double rng2 = Math.abs(px * rng.nextDouble() * 0.03); - double h = round(Math.max(o, c) + rng2); - double l = round(Math.min(o, c) - rng2); - epochDays[i] = day + i / NASDAQ_TICKERS.length; - symbols[i] = TICKER_BYTES[ticker]; - open[i] = o; - high[i] = h; - low[i] = l; - close[i] = c; - volume[i] = Math.max(100_000L, Math.round(1_000_000 + rng.nextGaussian() * 200_000)); - prices[ticker] = c; - } - day += n / NASDAQ_TICKERS.length; - flushJni(writer, epochDays, symbols, open, high, low, close, volume, n); - rowsLeft -= n; - } - } - } - - private void flushJni( - dev.vortex.api.VortexWriter writer, - int[] epochDays, byte[][] symbols, - double[] open, double[] high, double[] low, double[] close, long[] volume, - int n - ) throws IOException { - try (VectorSchemaRoot root = VectorSchemaRoot.create(JNI_SCHEMA, allocator)) { - DateDayVector dateVec = (DateDayVector) root.getVector("date"); - VarCharVector symbolVec = (VarCharVector) root.getVector("symbol"); - Float8Vector openVec = (Float8Vector) root.getVector("open"); - Float8Vector highVec = (Float8Vector) root.getVector("high"); - Float8Vector lowVec = (Float8Vector) root.getVector("low"); - Float8Vector closeVec = (Float8Vector) root.getVector("close"); - BigIntVector volumeVec = (BigIntVector) root.getVector("volume"); - - dateVec.allocateNew(n); - symbolVec.allocateNew(n); - openVec.allocateNew(n); - highVec.allocateNew(n); - lowVec.allocateNew(n); - closeVec.allocateNew(n); - volumeVec.allocateNew(n); - - for (int i = 0; i < n; i++) { - dateVec.setSafe(i, epochDays[i]); - symbolVec.setSafe(i, symbols[i]); - openVec.setSafe(i, open[i]); - highVec.setSafe(i, high[i]); - lowVec.setSafe(i, low[i]); - closeVec.setSafe(i, close[i]); - volumeVec.setSafe(i, volume[i]); - } - root.setRowCount(n); - - try (ArrowArray arr = ArrowArray.allocateNew(allocator); - ArrowSchema schema = ArrowSchema.allocateNew(allocator)) { - Data.exportVectorSchemaRoot(allocator, root, null, arr, schema); - writer.writeBatch(arr.memoryAddress(), schema.memoryAddress()); - } - } - } -} diff --git a/performance/src/main/java/io/github/dfa1/vortex/performance/RustVsJavaWriteBenchmark.java b/performance/src/main/java/io/github/dfa1/vortex/performance/RustVsJavaWriteBenchmark.java deleted file mode 100644 index 6d105cb..0000000 --- a/performance/src/main/java/io/github/dfa1/vortex/performance/RustVsJavaWriteBenchmark.java +++ /dev/null @@ -1,292 +0,0 @@ -package io.github.dfa1.vortex.performance; - -import dev.vortex.api.Session; -import dev.vortex.arrow.ArrowAllocation; -import dev.vortex.jni.NativeLoader; -import io.github.dfa1.vortex.core.DType; -import io.github.dfa1.vortex.core.PType; -import io.github.dfa1.vortex.writer.VortexWriter; -import io.github.dfa1.vortex.writer.WriteOptions; -import org.apache.arrow.c.ArrowArray; -import org.apache.arrow.c.ArrowSchema; -import org.apache.arrow.c.Data; -import org.apache.arrow.memory.BufferAllocator; -import org.apache.arrow.vector.DateDayVector; -import org.apache.arrow.vector.Float8Vector; -import org.apache.arrow.vector.BigIntVector; -import org.apache.arrow.vector.VarCharVector; -import org.apache.arrow.vector.VectorSchemaRoot; -import org.apache.arrow.vector.types.DateUnit; -import org.apache.arrow.vector.types.FloatingPointPrecision; -import org.apache.arrow.vector.types.pojo.ArrowType; -import org.apache.arrow.vector.types.pojo.Field; -import org.apache.arrow.vector.types.pojo.Schema; -import org.openjdk.jmh.annotations.Benchmark; -import org.openjdk.jmh.annotations.BenchmarkMode; -import org.openjdk.jmh.annotations.Fork; -import org.openjdk.jmh.annotations.Level; -import org.openjdk.jmh.annotations.Measurement; -import org.openjdk.jmh.annotations.Mode; -import org.openjdk.jmh.annotations.OutputTimeUnit; -import org.openjdk.jmh.annotations.Scope; -import org.openjdk.jmh.annotations.Setup; -import org.openjdk.jmh.annotations.State; -import org.openjdk.jmh.annotations.TearDown; -import org.openjdk.jmh.annotations.Warmup; - -import java.io.IOException; -import java.nio.channels.FileChannel; -import java.nio.charset.StandardCharsets; -import java.nio.file.Files; -import java.nio.file.Path; -import java.nio.file.StandardOpenOption; -import java.time.LocalDate; -import java.util.Arrays; -import java.util.HashMap; -import java.util.List; -import java.util.Map; -import java.util.Random; -import java.util.concurrent.TimeUnit; - -/// Write benchmark: Java writer vs JNI (Rust) writer on the same OHLC dataset. -/// -/// Schema: date(I32/DATE32), symbol(Utf8), open/high/low/close(F64), volume(I64) — 7 columns. -/// -/// Data is pre-generated in @Setup so only encoding + I/O is measured. -/// Each invocation writes 10 M rows; the file size is returned as the benchmark result -/// so the JVM cannot eliminate the write as dead code. -/// -/// Run: java -jar performance/target/benchmarks.jar RustVsJavaWriteBenchmark -@State(Scope.Benchmark) -@BenchmarkMode(Mode.Throughput) -@OutputTimeUnit(TimeUnit.SECONDS) -@Warmup(iterations = 3, time = 3) -@Measurement(iterations = 5, time = 5) -@Fork(value = 1, jvmArgsAppend = { - "--add-opens", "java.base/java.nio=ALL-UNNAMED", - "--enable-native-access=ALL-UNNAMED", - "--sun-misc-unsafe-memory-access=allow" -}) -public class RustVsJavaWriteBenchmark { - - private static final int TOTAL_ROWS = 10_000_000; - private static final int BATCH_SIZE = 50_000; - private static final int NUM_BATCHES = TOTAL_ROWS / BATCH_SIZE; - - private static final ArrowType F64_TYPE = new ArrowType.FloatingPoint(FloatingPointPrecision.DOUBLE); - private static final Schema JNI_SCHEMA = new Schema(List.of( - Field.notNullable("date", new ArrowType.Date(DateUnit.DAY)), - Field.notNullable("symbol", ArrowType.Utf8.INSTANCE), - Field.notNullable("open", F64_TYPE), - Field.notNullable("high", F64_TYPE), - Field.notNullable("low", F64_TYPE), - Field.notNullable("close", F64_TYPE), - Field.notNullable("volume", new ArrowType.Int(64, true)) - )); - - private static final DType.Struct JAVA_SCHEMA = new DType.Struct( - List.of("date", "symbol", "open", "high", "low", "close", "volume"), - List.of( - new DType.Primitive(PType.I32, false), - new DType.Utf8(false), - new DType.Primitive(PType.F64, false), - new DType.Primitive(PType.F64, false), - new DType.Primitive(PType.F64, false), - new DType.Primitive(PType.F64, false), - new DType.Primitive(PType.I64, false) - ), - false - ); - - // Real Nasdaq tickers — mix of lengths (1–5 chars) for realistic varbin distribution. - private static final String[] NASDAQ_TICKERS = { - "AAPL", "MSFT", "NVDA", "AMZN", "META", "GOOGL", "TSLA", "AVGO", "COST", "NFLX", - "AMD", "ADBE", "QCOM", "PEP", "CSCO", "TXN", "INTC", "CMCSA", "INTU", "AMGN", - "HON", "AMAT", "MU", "LRCX", "KLAC", "MRVL", "PANW", "SNPS", "CDNS", "REGN" - }; - private static final byte[][] TICKER_BYTES; - - static { - TICKER_BYTES = new byte[NASDAQ_TICKERS.length][]; - for (int i = 0; i < NASDAQ_TICKERS.length; i++) { - TICKER_BYTES[i] = NASDAQ_TICKERS[i].getBytes(StandardCharsets.UTF_8); - } - } - - private static final Session SESSION = Session.create(); - - static { - NativeLoader.loadJni(); - } - - // Pre-generated batch data — filled once in @Setup, reused across invocations. - private int[][] batchDates; - private String[][] batchSymbols; - private double[][] batchOpen; - private double[][] batchHigh; - private double[][] batchLow; - private double[][] batchClose; - private long[][] batchVolume; - - private Path jniFile; - private Path javaFile; - private Path javaFileCascading; - private BufferAllocator allocator; - - private static double round(double v) { - return Math.round(v * 100.0) / 100.0; - } - - @Setup(Level.Trial) - public void setup() throws IOException { - allocator = ArrowAllocation.rootAllocator(); - jniFile = Files.createTempFile("ohlc-jni-write", ".vtx"); - javaFile = Files.createTempFile("ohlc-java-write", ".vtx"); - javaFileCascading = Files.createTempFile("ohlc-java-cascading-write", ".vtx"); - - batchDates = new int[NUM_BATCHES][BATCH_SIZE]; - batchSymbols = new String[NUM_BATCHES][BATCH_SIZE]; - batchOpen = new double[NUM_BATCHES][BATCH_SIZE]; - batchHigh = new double[NUM_BATCHES][BATCH_SIZE]; - batchLow = new double[NUM_BATCHES][BATCH_SIZE]; - batchClose = new double[NUM_BATCHES][BATCH_SIZE]; - batchVolume = new long[NUM_BATCHES][BATCH_SIZE]; - - double[] prices = new double[NASDAQ_TICKERS.length]; - Arrays.fill(prices, 100.0); - var rng = new Random(42L); - int day = (int) LocalDate.of(2020, 1, 2).toEpochDay(); - - for (int b = 0; b < NUM_BATCHES; b++) { - for (int i = 0; i < BATCH_SIZE; i++) { - int ticker = i % NASDAQ_TICKERS.length; - double px = prices[ticker]; - double ret = rng.nextGaussian() * 0.02; - double o = round(px * (1 + ret * 0.3)); - double c = round(px * (1 + ret)); - double spread = Math.abs(px * rng.nextDouble() * 0.03); - double h = round(Math.max(o, c) + spread); - double l = round(Math.min(o, c) - spread); - batchDates[b][i] = day + (b * BATCH_SIZE + i) / NASDAQ_TICKERS.length; - batchSymbols[b][i] = NASDAQ_TICKERS[ticker]; - batchOpen[b][i] = o; - batchHigh[b][i] = h; - batchLow[b][i] = l; - batchClose[b][i] = c; - batchVolume[b][i] = Math.max(100_000L, Math.round(1_000_000 + rng.nextGaussian() * 200_000)); - prices[ticker] = c; - } - day += BATCH_SIZE / NASDAQ_TICKERS.length; - } - - System.out.printf("[RustVsJavaWriteBenchmark] data pre-generated: %d rows in %d batches%n", - TOTAL_ROWS, NUM_BATCHES); - } - - @TearDown(Level.Trial) - public void cleanup() throws IOException { - if (Files.exists(javaFile) && Files.exists(javaFileCascading)) { - long plain = Files.size(javaFile); - long cascading = Files.size(javaFileCascading); - System.out.printf("[RustVsJavaWriteBenchmark] cascading/plain size ratio: %.3f (%d B / %d B)%n", - (double) cascading / plain, cascading, plain); - } - Files.deleteIfExists(jniFile); - Files.deleteIfExists(javaFile); - Files.deleteIfExists(javaFileCascading); - } - - /// JNI write: encode and write 10 M rows via Rust VortexWriter. - @Benchmark - public long jniWrite() throws IOException { - try (dev.vortex.api.VortexWriter writer = dev.vortex.api.VortexWriter.create( - SESSION, jniFile.toAbsolutePath().toUri().toString(), JNI_SCHEMA, new HashMap<>(), allocator)) { - for (int b = 0; b < NUM_BATCHES; b++) { - flushJni(writer, b); - } - } - return Files.size(jniFile); - } - - /// Java write with cascading compression (depth 3): ALP → FOR → bitpacked. - @Benchmark - public long javaWriteCascading() throws IOException { - try (FileChannel ch = FileChannel.open(javaFileCascading, - StandardOpenOption.CREATE, StandardOpenOption.WRITE, StandardOpenOption.TRUNCATE_EXISTING); - VortexWriter writer = VortexWriter.create(ch, JAVA_SCHEMA, WriteOptions.cascading(3))) { - for (int b = 0; b < NUM_BATCHES; b++) { - Map chunk = Map.of( - "date", batchDates[b], - "symbol", batchSymbols[b], - "open", batchOpen[b], - "high", batchHigh[b], - "low", batchLow[b], - "close", batchClose[b], - "volume", batchVolume[b] - ); - writer.writeChunk(chunk); - } - } - return Files.size(javaFileCascading); - } - - /// Java write: encode and write 10 M rows via Java VortexWriter. - @Benchmark - public long javaWrite() throws IOException { - try (FileChannel ch = FileChannel.open(javaFile, - StandardOpenOption.CREATE, StandardOpenOption.WRITE, StandardOpenOption.TRUNCATE_EXISTING); - VortexWriter writer = VortexWriter.create(ch, JAVA_SCHEMA, WriteOptions.defaults())) { - for (int b = 0; b < NUM_BATCHES; b++) { - Map chunk = Map.of( - "date", batchDates[b], - "symbol", batchSymbols[b], - "open", batchOpen[b], - "high", batchHigh[b], - "low", batchLow[b], - "close", batchClose[b], - "volume", batchVolume[b] - ); - writer.writeChunk(chunk); - } - } - return Files.size(javaFile); - } - - private void flushJni(dev.vortex.api.VortexWriter writer, int b) throws IOException { - try (VectorSchemaRoot root = VectorSchemaRoot.create(JNI_SCHEMA, allocator)) { - DateDayVector dateVec = (DateDayVector) root.getVector("date"); - VarCharVector symbolVec = (VarCharVector) root.getVector("symbol"); - Float8Vector openVec = (Float8Vector) root.getVector("open"); - Float8Vector highVec = (Float8Vector) root.getVector("high"); - Float8Vector lowVec = (Float8Vector) root.getVector("low"); - Float8Vector closeVec = (Float8Vector) root.getVector("close"); - BigIntVector volVec = (BigIntVector) root.getVector("volume"); - - int n = batchDates[b].length; - dateVec.allocateNew(n); - symbolVec.allocateNew(n); - openVec.allocateNew(n); - highVec.allocateNew(n); - lowVec.allocateNew(n); - closeVec.allocateNew(n); - volVec.allocateNew(n); - - for (int i = 0; i < n; i++) { - dateVec.setSafe(i, batchDates[b][i]); - symbolVec.setSafe(i, TICKER_BYTES[i % NASDAQ_TICKERS.length]); - openVec.setSafe(i, batchOpen[b][i]); - highVec.setSafe(i, batchHigh[b][i]); - lowVec.setSafe(i, batchLow[b][i]); - closeVec.setSafe(i, batchClose[b][i]); - volVec.setSafe(i, batchVolume[b][i]); - } - root.setRowCount(n); - - try (ArrowArray arr = ArrowArray.allocateNew(allocator); - ArrowSchema schema = ArrowSchema.allocateNew(allocator)) { - Data.exportVectorSchemaRoot(allocator, root, null, arr, schema); - writer.writeBatch(arr.memoryAddress(), schema.memoryAddress()); - } - } - } -} diff --git a/performance/src/main/java/io/github/dfa1/vortex/performance/RustWritesJavaReadsBigFileBenchmark.java b/performance/src/main/java/io/github/dfa1/vortex/performance/RustWritesJavaReadsBigFileBenchmark.java deleted file mode 100644 index 81374e7..0000000 --- a/performance/src/main/java/io/github/dfa1/vortex/performance/RustWritesJavaReadsBigFileBenchmark.java +++ /dev/null @@ -1,235 +0,0 @@ -package io.github.dfa1.vortex.performance; - -import dev.vortex.api.DataSource; -import dev.vortex.api.Expression; -import dev.vortex.api.Partition; -import dev.vortex.api.Scan; -import dev.vortex.api.ScanOptions; -import dev.vortex.api.Session; -import dev.vortex.arrow.ArrowAllocation; -import dev.vortex.jni.NativeLoader; -import io.github.dfa1.vortex.core.array.Array; -import io.github.dfa1.vortex.encoding.EncodingRegistry; -import io.github.dfa1.vortex.io.VortexReader; -import io.github.dfa1.vortex.scan.ScanResult; -import org.apache.arrow.c.ArrowArray; -import org.apache.arrow.c.ArrowSchema; -import org.apache.arrow.c.Data; -import org.apache.arrow.memory.BufferAllocator; -import org.apache.arrow.vector.BigIntVector; -import org.apache.arrow.vector.VectorSchemaRoot; -import org.apache.arrow.vector.ipc.ArrowReader; -import org.apache.arrow.vector.types.pojo.ArrowType; -import org.apache.arrow.vector.types.pojo.Field; -import org.apache.arrow.vector.types.pojo.Schema; -import org.openjdk.jmh.annotations.Benchmark; -import org.openjdk.jmh.annotations.BenchmarkMode; -import org.openjdk.jmh.annotations.Fork; -import org.openjdk.jmh.annotations.Level; -import org.openjdk.jmh.annotations.Measurement; -import org.openjdk.jmh.annotations.Mode; -import org.openjdk.jmh.annotations.OutputTimeUnit; -import org.openjdk.jmh.annotations.Scope; -import org.openjdk.jmh.annotations.Setup; -import org.openjdk.jmh.annotations.State; -import org.openjdk.jmh.annotations.TearDown; -import org.openjdk.jmh.annotations.Warmup; - -import java.io.IOException; -import java.lang.foreign.MemorySegment; -import java.lang.foreign.ValueLayout; -import java.nio.ByteOrder; -import java.nio.file.Files; -import java.nio.file.Path; -import java.util.HashMap; -import java.util.List; -import java.util.Random; -import java.util.concurrent.TimeUnit; - -/// Big-file scan benchmark exercising the TODO #12 `SegmentSpec.length` = long fix. -/// -///

Setup: JNI writer produces a >2 GB Vortex file with 4 incompressible I64 columns -/// (random longs defeat bit-packing / FoR so the segments stay large). Measurement: pure-Java -/// `VortexReader` scans every column and sums the first column. -/// -///

The trial setup writes ~3 GB to disk and takes tens of seconds, so this benchmark is -/// intended for occasional runs (e.g. before tagging a release). To skip the JNI write, pass -/// {@code -Dvortex.bench.bigfile=/path/to/existing.vtx} — useful when iterating on the read -/// path against the same fixture. -/// -///

Run: {@code java -jar performance/target/benchmarks.jar RustWritesJavaReadsBigFileBenchmark.javaScan} -@State(Scope.Benchmark) -@BenchmarkMode(Mode.Throughput) -@OutputTimeUnit(TimeUnit.SECONDS) -@Warmup(iterations = 2, time = 5) -@Measurement(iterations = 3, time = 10) -@Fork(value = 1, jvmArgsAppend = { - "--add-opens", "java.base/java.nio=ALL-UNNAMED", - "--enable-native-access=ALL-UNNAMED", - "--sun-misc-unsafe-memory-access=allow" -}) -public class RustWritesJavaReadsBigFileBenchmark { - - private static final ValueLayout.OfLong LE_LONG = - ValueLayout.JAVA_LONG_UNALIGNED.withOrder(ByteOrder.LITTLE_ENDIAN); - - private static final long TARGET_BYTES = 3L * 1024 * 1024 * 1024; // ~3 GB - private static final int BATCH_SIZE = 1_000_000; // 1 M rows/batch → ~32 MB raw per col per batch - private static final int COLUMNS = 4; - private static final long BYTES_PER_ROW = (long) COLUMNS * Long.BYTES; - private static final long TOTAL_ROWS = TARGET_BYTES / BYTES_PER_ROW; - - private static final ArrowType I64_TYPE = new ArrowType.Int(64, true); - private static final Schema JNI_SCHEMA = new Schema(List.of( - Field.notNullable("c0", I64_TYPE), - Field.notNullable("c1", I64_TYPE), - Field.notNullable("c2", I64_TYPE), - Field.notNullable("c3", I64_TYPE) - )); - private static final Session SESSION = Session.create(); - - static { - NativeLoader.loadJni(); - } - - private Path benchFile; - private boolean ownFile; - private EncodingRegistry registry; - private BufferAllocator allocator; - - @Setup(Level.Trial) - public void setup() throws IOException { - registry = EncodingRegistry.loadAll(); - allocator = ArrowAllocation.rootAllocator(); - - String externalFile = System.getProperty("vortex.bench.bigfile"); - if (externalFile != null && !externalFile.isEmpty()) { - benchFile = Path.of(externalFile); - ownFile = false; - System.out.printf("[BigFileBenchmark] using external file: %s (%.2f GB)%n", - benchFile, Files.size(benchFile) / (double) (1L << 30)); - } else { - benchFile = Files.createTempFile("vortex-bigfile-bench", ".vtx"); - ownFile = true; - System.out.printf("[BigFileBenchmark] writing %d rows × %d I64 cols (~%.2f GB)...%n", - TOTAL_ROWS, COLUMNS, TARGET_BYTES / (double) (1L << 30)); - writeJni(benchFile); - System.out.printf("[BigFileBenchmark] file size: %.2f GB%n", - Files.size(benchFile) / (double) (1L << 30)); - } - - long jniSum = scanJni(); - long javaSum = scanJava(); - if (jniSum != javaSum) { - throw new AssertionError( - "Big-file correctness check failed: JNI c0 sum=" + jniSum + " Java c0 sum=" + javaSum); - } - System.out.printf("[BigFileBenchmark] correctness OK: c0 sum=%d%n", javaSum); - } - - @TearDown(Level.Trial) - public void cleanup() throws IOException { - if (ownFile) { - Files.deleteIfExists(benchFile); - } - } - - /// JNI reader scans the first column ("c0") via Arrow C Data Interface and sums every value. - @Benchmark - public long jniScan() throws IOException { - return scanJni(); - } - - /// Java reader scans the first column and sums every value. Sum prevents the JIT - /// from eliding the decode loop. - @Benchmark - public long javaScan() throws IOException { - return scanJava(); - } - - private long scanJni() throws IOException { - String uri = benchFile.toAbsolutePath().toUri().toString(); - var opts = ScanOptions.builder() - .projection(Expression.select(new String[]{"c0"}, Expression.root())) - .build(); - - long sum = 0L; - DataSource ds = DataSource.open(SESSION, uri); - Scan scan = ds.scan(opts); - while (scan.hasNext()) { - Partition partition = scan.next(); - try (ArrowReader reader = partition.scanArrow(allocator)) { - while (reader.loadNextBatch()) { - VectorSchemaRoot root = reader.getVectorSchemaRoot(); - BigIntVector v = (BigIntVector) root.getVector("c0"); - for (int i = 0; i < root.getRowCount(); i++) { - sum += v.get(i); - } - } - } - } - return sum; - } - - private long scanJava() throws IOException { - long sum = 0L; - try (VortexReader vf = VortexReader.open(benchFile, registry)) { - var iter = vf.scan(io.github.dfa1.vortex.scan.ScanOptions.columns("c0")); - while (iter.hasNext()) { - ScanResult r = iter.next(); - Array arr = r.columns().get("c0"); - MemorySegment buf = arr.buffer(0); - long count = buf.byteSize() / Long.BYTES; - for (long i = 0; i < count; i++) { - sum += buf.getAtIndex(LE_LONG, i); - } - } - } - return sum; - } - - private void writeJni(Path path) throws IOException { - String uri = path.toAbsolutePath().toUri().toString(); - var rng = new Random(42L); - long[][] cols = new long[COLUMNS][BATCH_SIZE]; - long rowsLeft = TOTAL_ROWS; - - try (dev.vortex.api.VortexWriter writer = dev.vortex.api.VortexWriter.create( - SESSION, uri, JNI_SCHEMA, new HashMap<>(), allocator)) { - while (rowsLeft > 0) { - int n = (int) Math.min(rowsLeft, BATCH_SIZE); - for (int c = 0; c < COLUMNS; c++) { - for (int i = 0; i < n; i++) { - // Random data defeats bit-packing / FoR so the segments stay large - // and the resulting file exercises the >2 GB read path. - cols[c][i] = rng.nextLong(); - } - } - flushBatch(writer, cols, n); - rowsLeft -= n; - } - } - } - - private void flushBatch(dev.vortex.api.VortexWriter writer, long[][] cols, int n) throws IOException { - try (VectorSchemaRoot root = VectorSchemaRoot.create(JNI_SCHEMA, allocator)) { - BigIntVector[] vecs = new BigIntVector[COLUMNS]; - for (int c = 0; c < COLUMNS; c++) { - vecs[c] = (BigIntVector) root.getVector("c" + c); - vecs[c].allocateNew(n); - } - for (int c = 0; c < COLUMNS; c++) { - for (int i = 0; i < n; i++) { - vecs[c].setSafe(i, cols[c][i]); - } - } - root.setRowCount(n); - - try (ArrowArray arr = ArrowArray.allocateNew(allocator); - ArrowSchema schema = ArrowSchema.allocateNew(allocator)) { - Data.exportVectorSchemaRoot(allocator, root, null, arr, schema); - writer.writeBatch(arr.memoryAddress(), schema.memoryAddress()); - } - } - } -} diff --git a/performance/src/main/java/io/github/dfa1/vortex/performance/TaxiLayoutInspector.java b/performance/src/main/java/io/github/dfa1/vortex/performance/TaxiLayoutInspector.java deleted file mode 100644 index 584307e..0000000 --- a/performance/src/main/java/io/github/dfa1/vortex/performance/TaxiLayoutInspector.java +++ /dev/null @@ -1,259 +0,0 @@ -package io.github.dfa1.vortex.performance; - -import dev.hardwood.InputFile; -import dev.hardwood.metadata.RepetitionType; -import dev.hardwood.reader.ParquetFileReader; -import dev.hardwood.reader.RowReader; -import dev.hardwood.schema.ColumnSchema; -import dev.vortex.api.Session; -import dev.vortex.arrow.ArrowAllocation; -import dev.vortex.jni.NativeLoader; -import io.github.dfa1.vortex.encoding.EncodingRegistry; -import io.github.dfa1.vortex.io.VortexInspector; -import io.github.dfa1.vortex.io.VortexReader; -import io.github.dfa1.vortex.parquet.ParquetImporter; -import org.apache.arrow.c.ArrowArray; -import org.apache.arrow.c.ArrowSchema; -import org.apache.arrow.c.Data; -import org.apache.arrow.memory.BufferAllocator; -import org.apache.arrow.vector.BigIntVector; -import org.apache.arrow.vector.Float8Vector; -import org.apache.arrow.vector.IntVector; -import org.apache.arrow.vector.TimeStampMicroVector; -import org.apache.arrow.vector.VarCharVector; -import org.apache.arrow.vector.VectorSchemaRoot; -import org.apache.arrow.vector.types.FloatingPointPrecision; -import org.apache.arrow.vector.types.TimeUnit; -import org.apache.arrow.vector.types.pojo.ArrowType; -import org.apache.arrow.vector.types.pojo.Field; -import org.apache.arrow.vector.types.pojo.Schema; - -import java.io.IOException; -import java.io.InputStream; -import java.net.URI; -import java.net.http.HttpClient; -import java.net.http.HttpRequest; -import java.net.http.HttpResponse; -import java.nio.charset.StandardCharsets; -import java.nio.file.Files; -import java.nio.file.Path; -import java.util.HashMap; -import java.util.List; - -/// Compare JNI (Rust) vs Java Vortex encoding for the NYC Yellow Taxi 2024-01 dataset. -/// -/// Writes the same Parquet file using both writers, then prints {@code inspect} output -/// so encoding choices can be compared side-by-side. -/// -/// Run: java -cp performance/target/benchmarks.jar \ -/// --enable-native-access=ALL-UNNAMED \ -/// --add-opens java.base/java.nio=ALL-UNNAMED \ -/// io.github.dfa1.vortex.performance.TaxiLayoutInspector -public final class TaxiLayoutInspector { - - private static final String PARQUET_URL = - "https://d37ci6vzurychx.cloudfront.net/trip-data/yellow_tripdata_2024-01.parquet"; - private static final Path CACHE_PATH = - Path.of(System.getProperty("java.io.tmpdir"), "yellow_tripdata_2024-01.parquet"); - private static final int BATCH_SIZE = 65_536; - private static final Session SESSION = Session.create(); - private static final ArrowType F64 = - new ArrowType.FloatingPoint(FloatingPointPrecision.DOUBLE); - private static final ArrowType I32 = new ArrowType.Int(32, true); - private static final ArrowType I64 = new ArrowType.Int(64, true); - private static final ArrowType TS_MICROS = - new ArrowType.Timestamp(TimeUnit.MICROSECOND, null); - - static { - NativeLoader.loadJni(); - } - - // Arrow schema matching all 19 nullable taxi columns - private static final Schema ARROW_SCHEMA = new Schema(List.of( - Field.nullable("VendorID", I32), - Field.nullable("tpep_pickup_datetime", TS_MICROS), - Field.nullable("tpep_dropoff_datetime", TS_MICROS), - Field.nullable("passenger_count", I64), - Field.nullable("trip_distance", F64), - Field.nullable("RatecodeID", I64), - Field.nullable("store_and_fwd_flag", ArrowType.Utf8.INSTANCE), - Field.nullable("PULocationID", I32), - Field.nullable("DOLocationID", I32), - Field.nullable("payment_type", I64), - Field.nullable("fare_amount", F64), - Field.nullable("extra", F64), - Field.nullable("mta_tax", F64), - Field.nullable("tip_amount", F64), - Field.nullable("tolls_amount", F64), - Field.nullable("improvement_surcharge", F64), - Field.nullable("total_amount", F64), - Field.nullable("congestion_surcharge", F64), - Field.nullable("Airport_fee", F64) - )); - - private TaxiLayoutInspector() { - } - - public static void main(String[] args) throws Exception { - Path parquetFile = ensureCached(); - System.out.printf("Parquet: %6.1f MB%n", mb(parquetFile)); - - Path jniVortex = Files.createTempFile("taxi-jni", ".vortex"); - Path javaVortex = Files.createTempFile("taxi-java", ".vortex"); - - try { - System.out.print("Writing JNI Vortex (Rust encoder)..."); - writeJni(parquetFile, jniVortex); - System.out.printf(" %6.1f MB%n", mb(jniVortex)); - - System.out.print("Writing Java Vortex (cascading depth 3)..."); - ParquetImporter.importParquet(parquetFile, javaVortex); - System.out.printf(" %6.1f MB%n", mb(javaVortex)); - - System.out.println("\n══════════════════════════════════════════════════════"); - System.out.println(" JNI / Rust writer"); - System.out.println("══════════════════════════════════════════════════════"); - inspect(jniVortex); - - System.out.println("\n══════════════════════════════════════════════════════"); - System.out.println(" Java writer"); - System.out.println("══════════════════════════════════════════════════════"); - inspect(javaVortex); - } finally { - Files.deleteIfExists(jniVortex); - Files.deleteIfExists(javaVortex); - } - } - - private static void inspect(Path path) throws IOException { - try (VortexReader r = VortexReader.open(path, EncodingRegistry.loadAll())) { - System.out.println(VortexInspector.inspect(r)); - } - } - - private static void writeJni(Path parquetFile, Path vortexFile) throws Exception { - String uri = vortexFile.toAbsolutePath().toUri().toString(); - BufferAllocator allocator = ArrowAllocation.rootAllocator(); - - try (dev.vortex.api.VortexWriter writer = dev.vortex.api.VortexWriter.create( - SESSION, uri, ARROW_SCHEMA, new HashMap<>(), allocator); - ParquetFileReader parquet = ParquetFileReader.open(InputFile.of(parquetFile)); - RowReader rows = parquet.buildRowReader().build()) { - - List cols = parquet.getFileSchema().getColumns(); - boolean[] optional = new boolean[cols.size()]; - for (int i = 0; i < cols.size(); i++) { - optional[i] = cols.get(i).repetitionType() == RepetitionType.OPTIONAL; - } - - // column name arrays for fast lookup - String[] names = cols.stream().map(ColumnSchema::name).toArray(String[]::new); - - try (VectorSchemaRoot root = VectorSchemaRoot.create(ARROW_SCHEMA, allocator)) { - int pos = 0; - allocateVectors(root, BATCH_SIZE); - - while (rows.hasNext()) { - rows.next(); - fillRow(rows, root, names, optional, pos); - pos++; - if (pos == BATCH_SIZE) { - flushBatch(writer, allocator, root, pos); - allocateVectors(root, BATCH_SIZE); - pos = 0; - } - } - if (pos > 0) { - flushBatch(writer, allocator, root, pos); - } - } - } - } - - private static void allocateVectors(VectorSchemaRoot root, int capacity) { - for (var vec : root.getFieldVectors()) { - vec.allocateNew(); - } - } - - private static void fillRow(RowReader rows, VectorSchemaRoot root, - String[] names, boolean[] optional, int pos) { - for (int c = 0; c < names.length; c++) { - String name = names[c]; - boolean isNull = optional[c] && rows.isNull(name); - switch (root.getVector(name)) { - case IntVector v -> { - if (isNull) { - v.setNull(pos); - } else { - v.setSafe(pos, rows.getInt(name)); - } - } - case BigIntVector v -> { - if (isNull) { - v.setNull(pos); - } else { - v.setSafe(pos, rows.getLong(name)); - } - } - case TimeStampMicroVector v -> { - if (isNull) { - v.setNull(pos); - } else { - v.setSafe(pos, rows.getLong(name)); - } - } - case Float8Vector v -> { - if (isNull) { - v.setNull(pos); - } else { - v.setSafe(pos, rows.getDouble(name)); - } - } - case VarCharVector v -> { - if (isNull) { - v.setNull(pos); - } else { - byte[] bytes = rows.getString(name).getBytes(StandardCharsets.UTF_8); - v.setSafe(pos, bytes); - } - } - default -> throw new UnsupportedOperationException( - "unexpected vector type: " + root.getVector(name).getClass().getSimpleName()); - } - } - } - - private static void flushBatch(dev.vortex.api.VortexWriter writer, - BufferAllocator allocator, - VectorSchemaRoot root, int rowCount) throws IOException { - root.setRowCount(rowCount); - try (ArrowArray arr = ArrowArray.allocateNew(allocator); - ArrowSchema schema = ArrowSchema.allocateNew(allocator)) { - Data.exportVectorSchemaRoot(allocator, root, null, arr, schema); - writer.writeBatch(arr.memoryAddress(), schema.memoryAddress()); - } - } - - private static Path ensureCached() throws Exception { - if (Files.exists(CACHE_PATH)) { - System.out.printf("cache hit: %s%n", CACHE_PATH); - return CACHE_PATH; - } - System.out.printf("downloading %s...%n", PARQUET_URL); - HttpClient client = HttpClient.newBuilder().followRedirects(HttpClient.Redirect.NORMAL).build(); - HttpRequest req = HttpRequest.newBuilder().uri(URI.create(PARQUET_URL)).build(); - HttpResponse resp = client.send(req, HttpResponse.BodyHandlers.ofInputStream()); - if (resp.statusCode() != 200) { - throw new IOException("HTTP " + resp.statusCode()); - } - try (InputStream in = resp.body()) { - Files.copy(in, CACHE_PATH); - } - return CACHE_PATH; - } - - private static double mb(Path path) throws IOException { - return Files.size(path) / 1_048_576.0; - } -} diff --git a/pom.xml b/pom.xml deleted file mode 100644 index 41b14e5..0000000 --- a/pom.xml +++ /dev/null @@ -1,397 +0,0 @@ - - - 4.0.0 - - io.github.dfa1.vortex - vortex-java - 0.3.0-SNAPSHOT - pom - - vortex-java - Pure-Java reader/writer for the Vortex columnar file format. Zero-copy memory-mapped reads via the Java FFM API (Java 25+), no JNI, no sun.misc.Unsafe. - https://github.com/dfa1/vortex-java - - - - Apache License, Version 2.0 - https://www.apache.org/licenses/LICENSE-2.0 - repo - - - - - - dfa1 - Davide Angelocola - davide.angelocola@gmail.com - https://github.com/dfa1 - - - - - scm:git:https://github.com/dfa1/vortex-java.git - scm:git:git@github.com:dfa1/vortex-java.git - https://github.com/dfa1/vortex-java - HEAD - - - - core - reader - writer - csv - jdbc - parquet - bom - cli - integration - performance - - - - 25 - UTF-8 - - 3.6 - 1.5.7-6 - 25.2.10 - 4.35.0 - 3.6.0 - 1.0.0.CR1 - 2.3.232 - - 5.11.4 - 3.27.7 - 5.14.2 - 0.72.0 - 19.0.0 - 2.0.18 - 1.37 - - 0.7.0 - 3.2.7 - 3.3.1 - 3.11.2 - - - - - - io.github.dfa1.vortex - core - ${project.version} - - - io.github.dfa1.vortex - reader - ${project.version} - - - io.github.dfa1.vortex - writer - ${project.version} - - - io.github.dfa1.vortex - csv - ${project.version} - - - io.github.dfa1.vortex - jdbc - ${project.version} - - - com.h2database - h2 - ${h2.version} - test - - - io.github.dfa1.vortex - parquet - ${project.version} - - - de.siegmar - fastcsv - ${fastcsv.version} - - - dev.hardwood - hardwood-core - ${hardwood.version} - - - io.airlift - aircompressor-v3 - ${aircompressor.version} - - - com.github.luben - zstd-jni - ${zstd-jni.version} - - - com.google.flatbuffers - flatbuffers-java - ${flatbuffers.version} - - - com.google.protobuf - protobuf-java - ${protobuf.version} - - - org.junit.jupiter - junit-jupiter - ${junit.version} - test - - - org.assertj - assertj-core - ${assertj.version} - test - - - org.mockito - mockito-junit-jupiter - ${mockito.version} - test - - - dev.vortex - vortex-jni - ${vortex-jni.version} - test - - - org.apache.arrow - arrow-vector - ${arrow.version} - test - - - org.apache.arrow - arrow-c-data - ${arrow.version} - test - - - org.apache.arrow - arrow-memory-unsafe - ${arrow.version} - test - - - org.slf4j - slf4j-nop - ${slf4j.version} - test - - - org.slf4j - slf4j-simple - ${slf4j.version} - test - - - org.openjdk.jmh - jmh-core - ${jmh.version} - - - org.openjdk.jmh - jmh-generator-annprocess - ${jmh.version} - - - - - - - - - org.apache.maven.plugins - maven-compiler-plugin - 3.13.0 - - 25 - - - - org.apache.maven.plugins - maven-surefire-plugin - 3.5.2 - - --sun-misc-unsafe-memory-access=allow --enable-native-access=ALL-UNNAMED - - - - org.codehaus.mojo - exec-maven-plugin - 3.5.0 - - - org.apache.maven.plugins - maven-antrun-plugin - 3.1.0 - - - org.codehaus.mojo - build-helper-maven-plugin - 3.6.0 - - - org.apache.maven.plugins - maven-shade-plugin - 3.6.0 - - - org.apache.maven.plugins - maven-failsafe-plugin - 3.5.2 - - - org.apache.maven.plugins - maven-javadoc-plugin - ${maven-javadoc-plugin.version} - - io.github.dfa1.vortex.fbs:io.github.dfa1.vortex.proto - - - - org.apache.maven.plugins - maven-checkstyle-plugin - 3.6.0 - - - validate - - check - - - ${maven.multiModuleProjectDirectory}/checkstyle.xml - true - true - - src/main/java - src/test/java - - **/module-info.java,**/fbs/*.java,**/proto/*.java - - - - - - com.puppycrawl.tools - checkstyle - 13.4.0 - - - - - org.apache.maven.plugins - maven-release-plugin - 3.3.1 - - true - v@{project.version} - false - true - - - - - - - org.apache.maven.plugins - maven-checkstyle-plugin - - - - - - - - - release - - - - org.apache.maven.plugins - maven-source-plugin - ${maven-source-plugin.version} - - - attach-sources - - jar-no-fork - - - - - - org.apache.maven.plugins - maven-javadoc-plugin - ${maven-javadoc-plugin.version} - - - attach-javadocs - - jar - - - - - none - - - - org.apache.maven.plugins - maven-gpg-plugin - ${maven-gpg-plugin.version} - - - sign-artifacts - verify - - sign - - - - --pinentry-mode - loopback - - - - - - - org.sonatype.central - central-publishing-maven-plugin - ${central-publishing-plugin.version} - true - - central - - - - - - - - diff --git a/reader/pom.xml b/reader/pom.xml deleted file mode 100644 index d7e5f5d..0000000 --- a/reader/pom.xml +++ /dev/null @@ -1,41 +0,0 @@ - - - 4.0.0 - - io.github.dfa1.vortex - vortex-java - 0.3.0-SNAPSHOT - - - reader - - - - io.github.dfa1.vortex - core - - - com.google.flatbuffers - flatbuffers-java - - - com.google.protobuf - protobuf-java - - - org.junit.jupiter - junit-jupiter - test - - - org.assertj - assertj-core - test - - - org.mockito - mockito-junit-jupiter - test - - - diff --git a/reader/src/main/java/io/github/dfa1/vortex/io/PostscriptParser.java b/reader/src/main/java/io/github/dfa1/vortex/io/PostscriptParser.java deleted file mode 100644 index 96c4c26..0000000 --- a/reader/src/main/java/io/github/dfa1/vortex/io/PostscriptParser.java +++ /dev/null @@ -1,190 +0,0 @@ -package io.github.dfa1.vortex.io; - -import io.github.dfa1.vortex.core.CompressionScheme; -import io.github.dfa1.vortex.core.DType; -import io.github.dfa1.vortex.core.Footer; -import io.github.dfa1.vortex.core.Layout; -import io.github.dfa1.vortex.core.PType; -import io.github.dfa1.vortex.core.SegmentSpec; -import io.github.dfa1.vortex.core.VortexException; -import io.github.dfa1.vortex.fbs.Binary; -import io.github.dfa1.vortex.fbs.Bool; -import io.github.dfa1.vortex.fbs.Decimal; -import io.github.dfa1.vortex.fbs.Extension; -import io.github.dfa1.vortex.fbs.FixedSizeList; -import io.github.dfa1.vortex.fbs.Postscript; -import io.github.dfa1.vortex.fbs.Primitive; -import io.github.dfa1.vortex.fbs.Struct_; -import io.github.dfa1.vortex.fbs.Type; -import io.github.dfa1.vortex.fbs.Utf8; - -import java.lang.foreign.MemorySegment; -import java.nio.ByteBuffer; -import java.nio.ByteOrder; -import java.util.ArrayList; -import java.util.List; - -final class PostscriptParser { - - private PostscriptParser() { - } - - static ParsedFile parse(ByteBuffer postscriptBuf, MemorySegment fileSegment) { - var ps = Postscript.getRootAsPostscript(postscriptBuf); - - var footerSeg = ps.footer(); - if (footerSeg == null) { - throw new VortexException("postscript missing footer segment"); - } - var layoutSeg = ps.layout(); - if (layoutSeg == null) { - throw new VortexException("postscript missing layout segment"); - } - var dtypeSeg = ps.dtype(); - - ByteBuffer footerBuf = slice(fileSegment, footerSeg.offset(), footerSeg.length()); - ByteBuffer layoutBuf = slice(fileSegment, layoutSeg.offset(), layoutSeg.length()); - ByteBuffer dtypeBuf = (dtypeSeg != null && dtypeSeg.length() > 0) - ? slice(fileSegment, dtypeSeg.offset(), dtypeSeg.length()) - : null; - - return parseBlobs(footerBuf, layoutBuf, dtypeBuf); - } - - static ParsedFile parseBlobs(ByteBuffer footerBuf, ByteBuffer layoutBuf, ByteBuffer dtypeBuf) { - var fbsFooter = io.github.dfa1.vortex.fbs.Footer.getRootAsFooter(footerBuf); - var fbsLayout = io.github.dfa1.vortex.fbs.Layout.getRootAsLayout(layoutBuf); - - Footer footer = convertFooter(fbsFooter); - Layout layout = convertLayout(fbsLayout, footer.layoutSpecs()); - - DType dtype = null; - if (dtypeBuf != null && dtypeBuf.hasRemaining()) { - dtype = convertDType(io.github.dfa1.vortex.fbs.DType.getRootAsDType(dtypeBuf)); - } - - return new ParsedFile(footer, dtype, layout); - } - - private static ByteBuffer slice(MemorySegment seg, long offset, long length) { - return seg.asSlice(offset, length).asByteBuffer().order(ByteOrder.LITTLE_ENDIAN); - } - - static Footer convertFooter(io.github.dfa1.vortex.fbs.Footer f) { - var arraySpecs = new ArrayList(f.arraySpecsLength()); - for (int i = 0; i < f.arraySpecsLength(); i++) { - arraySpecs.add(f.arraySpecs(i).id()); - } - - var layoutSpecs = new ArrayList(f.layoutSpecsLength()); - for (int i = 0; i < f.layoutSpecsLength(); i++) { - layoutSpecs.add(f.layoutSpecs(i).id()); - } - - var segmentSpecs = new ArrayList(f.segmentSpecsLength()); - for (int i = 0; i < f.segmentSpecsLength(); i++) { - var s = f.segmentSpecs(i); - segmentSpecs.add(new SegmentSpec( - s.offset(), s.length(), - (byte) s.alignmentExponent(), - CompressionScheme.of(s._Compression()))); - } - - var compressionSpecs = new ArrayList(f.compressionSpecsLength()); - for (int i = 0; i < f.compressionSpecsLength(); i++) { - compressionSpecs.add(CompressionScheme.of(f.compressionSpecs(i).scheme())); - } - - return new Footer( - List.copyOf(arraySpecs), List.copyOf(layoutSpecs), - List.copyOf(segmentSpecs), List.copyOf(compressionSpecs)); - } - - private static Layout convertLayout(io.github.dfa1.vortex.fbs.Layout l, List layoutSpecs) { - String encodingId = layoutSpecs.get(l.encoding()); - - ByteBuffer metadata = l.metadataAsByteBuffer(); - - var children = new ArrayList(l.childrenLength()); - for (int i = 0; i < l.childrenLength(); i++) { - children.add(convertLayout(l.children(i), layoutSpecs)); - } - - var segments = new ArrayList(l.segmentsLength()); - for (int i = 0; i < l.segmentsLength(); i++) { - segments.add((int) l.segments(i)); - } - - return new Layout(encodingId, l.rowCount(), metadata, List.copyOf(children), List.copyOf(segments)); - } - - private static DType convertDType(io.github.dfa1.vortex.fbs.DType fbs) { - byte typeType = fbs.typeType(); - return switch (typeType) { - case Type.Null -> new DType.Null(true); - case Type.Bool -> new DType.Bool(((Bool) fbs.type(new Bool())).nullable()); - case Type.Primitive -> { - var p = (Primitive) fbs.type(new Primitive()); - yield new DType.Primitive(convertPType(p.ptype()), p.nullable()); - } - case Type.Decimal -> { - var d = (Decimal) fbs.type(new Decimal()); - yield new DType.Decimal((byte) d.precision(), d.scale(), d.nullable()); - } - case Type.Utf8 -> new DType.Utf8(((Utf8) fbs.type(new Utf8())).nullable()); - case Type.Binary -> new DType.Binary(((Binary) fbs.type(new Binary())).nullable()); - case Type.Struct_ -> { - var s = (Struct_) fbs.type(new Struct_()); - var names = new ArrayList(s.namesLength()); - var types = new ArrayList(s.dtypesLength()); - for (int i = 0; i < s.namesLength(); i++) { - names.add(s.names(i)); - } - for (int i = 0; i < s.dtypesLength(); i++) { - types.add(convertDType(s.dtypes(new io.github.dfa1.vortex.fbs.DType(), i))); - } - yield new DType.Struct(List.copyOf(names), List.copyOf(types), s.nullable()); - } - case Type.List -> { - var l = (io.github.dfa1.vortex.fbs.List) fbs.type(new io.github.dfa1.vortex.fbs.List()); - yield new DType.List( - convertDType(l.elementType(new io.github.dfa1.vortex.fbs.DType())), - l.nullable()); - } - case Type.FixedSizeList -> { - var fsl = (FixedSizeList) fbs.type(new FixedSizeList()); - yield new DType.FixedSizeList( - convertDType(fsl.elementType(new io.github.dfa1.vortex.fbs.DType())), - (int) fsl.size(), fsl.nullable()); - } - case Type.Extension -> { - var e = (Extension) fbs.type(new Extension()); - yield new DType.Extension( - e.id(), - convertDType(e.storageDtype(new io.github.dfa1.vortex.fbs.DType())), - e.metadataAsByteBuffer(), false); - } - default -> throw new VortexException("unsupported DType typeType=" + typeType); - }; - } - - private static PType convertPType(int fbsPType) { - return switch (fbsPType) { - case io.github.dfa1.vortex.fbs.PType.U8 -> PType.U8; - case io.github.dfa1.vortex.fbs.PType.U16 -> PType.U16; - case io.github.dfa1.vortex.fbs.PType.U32 -> PType.U32; - case io.github.dfa1.vortex.fbs.PType.U64 -> PType.U64; - case io.github.dfa1.vortex.fbs.PType.I8 -> PType.I8; - case io.github.dfa1.vortex.fbs.PType.I16 -> PType.I16; - case io.github.dfa1.vortex.fbs.PType.I32 -> PType.I32; - case io.github.dfa1.vortex.fbs.PType.I64 -> PType.I64; - case io.github.dfa1.vortex.fbs.PType.F16 -> PType.F16; - case io.github.dfa1.vortex.fbs.PType.F32 -> PType.F32; - case io.github.dfa1.vortex.fbs.PType.F64 -> PType.F64; - default -> throw new VortexException("unrecognized PType=" + fbsPType); - }; - } - - record ParsedFile(Footer footer, DType dtype, Layout layout) { - } -} diff --git a/reader/src/main/java/io/github/dfa1/vortex/io/VortexHandle.java b/reader/src/main/java/io/github/dfa1/vortex/io/VortexHandle.java deleted file mode 100644 index 736afb8..0000000 --- a/reader/src/main/java/io/github/dfa1/vortex/io/VortexHandle.java +++ /dev/null @@ -1,37 +0,0 @@ -package io.github.dfa1.vortex.io; - -import io.github.dfa1.vortex.core.DType; -import io.github.dfa1.vortex.core.Footer; -import io.github.dfa1.vortex.core.Layout; -import io.github.dfa1.vortex.encoding.EncodingRegistry; -import io.github.dfa1.vortex.scan.ScanIterator; -import io.github.dfa1.vortex.scan.ScanOptions; - -import java.io.Closeable; -import java.lang.foreign.MemorySegment; - -/// Common interface for handles to a Vortex file, regardless of storage backend. -/// -/// Implementations: [VortexReader] (memory-mapped local file), [VortexHttpReader] (HTTP Range reads). -public interface VortexHandle extends Closeable { - - DType dtype(); - - Layout layout(); - - Footer footer(); - - int version(); - - long fileSize(); - - EncodingRegistry registry(); - - /// Returns a view of bytes `[offset, offset+length)` within the file. - MemorySegment slice(long offset, long length); - - ScanIterator scan(ScanOptions options); - - @Override - void close(); -} diff --git a/reader/src/main/java/io/github/dfa1/vortex/io/VortexHttpReader.java b/reader/src/main/java/io/github/dfa1/vortex/io/VortexHttpReader.java deleted file mode 100644 index e929957..0000000 --- a/reader/src/main/java/io/github/dfa1/vortex/io/VortexHttpReader.java +++ /dev/null @@ -1,274 +0,0 @@ -package io.github.dfa1.vortex.io; - -import io.github.dfa1.vortex.core.DType; -import io.github.dfa1.vortex.core.Footer; -import io.github.dfa1.vortex.core.Layout; -import io.github.dfa1.vortex.core.VortexException; -import io.github.dfa1.vortex.encoding.EncodingRegistry; -import io.github.dfa1.vortex.fbs.Postscript; -import io.github.dfa1.vortex.scan.ScanIterator; -import io.github.dfa1.vortex.scan.ScanOptions; - -import java.io.IOException; -import java.lang.foreign.Arena; -import java.lang.foreign.MemorySegment; -import java.lang.foreign.ValueLayout; -import java.net.URI; -import java.net.http.HttpClient; -import java.net.http.HttpRequest; -import java.net.http.HttpResponse; -import java.nio.ByteBuffer; -import java.nio.ByteOrder; - -/// Handle to a remote Vortex file read via HTTP Range requests. -/// -/// On open: fetches the last [#TAIL_SIZE] bytes to locate the trailer, postscript, -/// and metadata blobs (footer, layout, dtype). Each [#slice] call fires a targeted -/// Range request; no full-file download occurs. -/// -/// All allocated buffers (segment bytes and encoding outputs) share a single -/// confined [Arena] and are released when this handle is closed. -public final class VortexHttpReader implements VortexHandle { - - /// Tail window fetched on open. 65 KB covers the trailer, postscript, and - /// all metadata blobs for typical Vortex files. - static final int TAIL_SIZE = 65 * 1024; - - private final URI uri; - private final HttpClient client; - private final Arena arena; - private final long fileSize; - private final int version; - private final Footer footer; - private final DType dtype; - private final Layout layout; - private final EncodingRegistry registry; - - private VortexHttpReader( - URI uri, HttpClient client, Arena arena, long fileSize, - int version, Footer footer, DType dtype, Layout layout, - EncodingRegistry registry - ) { - this.uri = uri; - this.client = client; - this.arena = arena; - this.fileSize = fileSize; - this.version = version; - this.footer = footer; - this.dtype = dtype; - this.layout = layout; - this.registry = registry; - } - - public static VortexHttpReader open(URI uri) throws IOException { - return open(uri, EncodingRegistry.loadAll()); - } - - public static VortexHttpReader open(URI uri, EncodingRegistry registry) throws IOException { - HttpClient client = HttpClient.newHttpClient(); - Arena arena = Arena.ofConfined(); - try { - // Single suffix Range request — Content-Range response header gives us fileSize. - // Avoids a separate HEAD round trip. - TailFetch tf = fetchTail(client, uri); - byte[] tail = tf.bytes(); - long tailStart = tf.start(); - long fileSize = tf.fileSize(); - long tailLen = tail.length; - - MemorySegment tailSeg = MemorySegment.ofArray(tail); - long trailerOff = tailLen - VortexReader.TRAILER_SIZE; - - int version = Short.toUnsignedInt(tailSeg.get(VortexReader.LE_SHORT, trailerOff)); - int postscriptLen = Short.toUnsignedInt(tailSeg.get(VortexReader.LE_SHORT, trailerOff + 2)); - checkMagic(tailSeg, trailerOff + 4, uri); - - long psOffInTail = trailerOff - postscriptLen; - if (psOffInTail < 0) { - throw new VortexException( - "postscript (%d bytes) extends beyond %d-byte tail; fetch larger tail" - .formatted(postscriptLen, TAIL_SIZE)); - } - - ByteBuffer postscriptBuf = tailSeg.asSlice(psOffInTail, postscriptLen) - .asByteBuffer().order(ByteOrder.LITTLE_ENDIAN); - - var ps = Postscript.getRootAsPostscript(postscriptBuf); - - var footerSpec = ps.footer(); - if (footerSpec == null) { - throw new VortexException("postscript missing footer segment"); - } - var layoutSpec = ps.layout(); - if (layoutSpec == null) { - throw new VortexException("postscript missing layout segment"); - } - var dtypeSpec = ps.dtype(); - - ByteBuffer footerBuf = fetchBlob(footerSpec.offset(), footerSpec.length(), - tailStart, tail, client, uri); - ByteBuffer layoutBuf = fetchBlob(layoutSpec.offset(), layoutSpec.length(), - tailStart, tail, client, uri); - ByteBuffer dtypeBuf = (dtypeSpec != null && dtypeSpec.length() > 0) - ? fetchBlob(dtypeSpec.offset(), dtypeSpec.length(), tailStart, tail, client, uri) - : null; - - var parsed = PostscriptParser.parseBlobs(footerBuf, layoutBuf, dtypeBuf); - - return new VortexHttpReader( - uri, client, arena, fileSize, version, - parsed.footer(), parsed.dtype(), parsed.layout(), - registry - ); - } catch (Exception e) { - arena.close(); - throw e; - } - } - - @Override - public DType dtype() { - return dtype; - } - - @Override - public Layout layout() { - return layout; - } - - @Override - public Footer footer() { - return footer; - } - - @Override - public int version() { - return version; - } - - @Override - public long fileSize() { - return fileSize; - } - - @Override - public EncodingRegistry registry() { - return registry; - } - - /// Fetches bytes `[offset, offset+length)` via HTTP Range and returns them - /// as an off-heap [MemorySegment] tied to this reader's [Arena]. - @Override - public MemorySegment slice(long offset, long length) { - byte[] bytes; - try { - bytes = fetchRange(client, uri, offset, offset + length - 1); - } catch (IOException e) { - throw new VortexException( - "failed to fetch [%d, %d) from %s: %s".formatted(offset, offset + length, uri, e.getMessage())); - } - MemorySegment seg = arena.allocate(length); - MemorySegment.copy(MemorySegment.ofArray(bytes), 0, seg, 0, length); - return seg.asReadOnly(); - } - - @Override - public ScanIterator scan(ScanOptions options) { - return new ScanIterator(this, options); - } - - @Override - public void close() { - arena.close(); - client.close(); - } - - // ── HTTP helpers ────────────────────────────────────────────────────────── - - private record TailFetch(byte[] bytes, long start, long fileSize) {} - - /// Fetches the last [#TAIL_SIZE] bytes in one request. - /// Parses `Content-Range: bytes start-end/total` to extract file size and tail offset, - /// avoiding a separate HEAD round trip. - private static TailFetch fetchTail(HttpClient client, URI uri) throws IOException { - HttpRequest req = HttpRequest.newBuilder(uri) - .header("Range", "bytes=-" + TAIL_SIZE) - .GET() - .build(); - HttpResponse resp; - try { - resp = client.send(req, HttpResponse.BodyHandlers.ofByteArray()); - } catch (InterruptedException e) { - Thread.currentThread().interrupt(); - throw new IOException("interrupted fetching tail of " + uri, e); - } - - byte[] body = resp.body(); - int status = resp.statusCode(); - - if (status == 206) { - // Content-Range: bytes -/ - String cr = resp.headers().firstValue("Content-Range") - .orElseThrow(() -> new VortexException("206 response missing Content-Range from " + uri)); - String spec = cr.substring("bytes ".length()); // "-/" - int slash = spec.indexOf('/'); - long total = Long.parseLong(spec.substring(slash + 1)); - long start = Long.parseLong(spec.substring(0, spec.indexOf('-'))); - return new TailFetch(body, start, total); - } - - if (status == 200) { - // Server returned full file (no Range support) - return new TailFetch(body, 0L, body.length); - } - - throw new VortexException("HTTP " + status + " fetching tail of " + uri); - } - - private static byte[] fetchRange(HttpClient client, URI uri, long from, long to) throws IOException { - HttpRequest req = HttpRequest.newBuilder(uri) - .header("Range", "bytes=" + from + "-" + to) - .GET() - .build(); - try { - HttpResponse resp = client.send(req, HttpResponse.BodyHandlers.ofByteArray()); - int status = resp.statusCode(); - if (status != 206 && status != 200) { - throw new VortexException("HTTP " + status + " fetching range from " + uri); - } - return resp.body(); - } catch (InterruptedException e) { - Thread.currentThread().interrupt(); - throw new IOException("interrupted fetching range from " + uri, e); - } - } - - private static void checkMagic(MemorySegment seg, long offset, URI uri) { - byte m0 = seg.get(ValueLayout.JAVA_BYTE, offset); - byte m1 = seg.get(ValueLayout.JAVA_BYTE, offset + 1); - byte m2 = seg.get(ValueLayout.JAVA_BYTE, offset + 2); - byte m3 = seg.get(ValueLayout.JAVA_BYTE, offset + 3); - byte[] magic = VortexReader.MAGIC; - if (m0 != magic[0] || m1 != magic[1] || m2 != magic[2] || m3 != magic[3]) { - throw new VortexException( - "invalid magic bytes [%02x %02x %02x %02x] from %s".formatted(m0, m1, m2, m3, uri)); - } - } - - /// Returns a ByteBuffer for a blob at absolute file `offset` of `length` bytes. - /// If the blob falls within the already-fetched `tail`, extracts it directly; - /// otherwise fires an additional Range request. - private static ByteBuffer fetchBlob( - long offset, long length, - long tailStart, byte[] tail, - HttpClient client, URI uri - ) throws IOException { - if (offset >= tailStart) { - int relOffset = (int) (offset - tailStart); - return ByteBuffer.wrap(tail, relOffset, (int) length) - .slice().order(ByteOrder.LITTLE_ENDIAN); - } - byte[] bytes = fetchRange(client, uri, offset, offset + length - 1); - return ByteBuffer.wrap(bytes).order(ByteOrder.LITTLE_ENDIAN); - } -} diff --git a/reader/src/main/java/io/github/dfa1/vortex/io/VortexInspector.java b/reader/src/main/java/io/github/dfa1/vortex/io/VortexInspector.java deleted file mode 100644 index 09dbe62..0000000 --- a/reader/src/main/java/io/github/dfa1/vortex/io/VortexInspector.java +++ /dev/null @@ -1,174 +0,0 @@ -package io.github.dfa1.vortex.io; - -import io.github.dfa1.vortex.core.DType; -import io.github.dfa1.vortex.core.Footer; -import io.github.dfa1.vortex.core.Layout; -import io.github.dfa1.vortex.core.SegmentSpec; - -import java.lang.foreign.MemorySegment; -import java.nio.ByteBuffer; -import java.nio.ByteOrder; -import java.util.LinkedHashSet; -import java.util.List; -import java.util.Set; - -/// Produces a human-readable summary of a Vortex file's structure and encodings. -public final class VortexInspector { - - private VortexInspector() { - } - - public static String inspect(VortexHandle reader) { - Footer footer = reader.footer(); - Layout layout = reader.layout(); - DType dtype = reader.dtype(); - - var sb = new StringBuilder(); - - sb.append("Vortex v").append(reader.version()) - .append(" ").append(formatBytes(reader.fileSize())).append('\n'); - sb.append('\n'); - - sb.append("Schema:\n"); - appendSchema(sb, dtype, " "); - sb.append('\n'); - - sb.append("Registered encodings: ").append(String.join(", ", footer.arraySpecs())).append('\n'); - sb.append('\n'); - - Set usedEncodings = collectUsedEncodings(reader); - sb.append("Used encodings: ").append(String.join(", ", usedEncodings)).append('\n'); - sb.append('\n'); - - int segCount = footer.segmentSpecs().size(); - long totalBytes = footer.segmentSpecs().stream().mapToLong(SegmentSpec::length).sum(); - sb.append("Segments: ").append(segCount) - .append(" total ").append(formatBytes(totalBytes)).append('\n'); - sb.append('\n'); - - sb.append("Layout:\n"); - List colNames = (dtype instanceof DType.Struct s) ? s.fieldNames() : List.of(); - appendLayout(sb, layout, colNames, reader, " "); - - return sb.toString(); - } - - // ── Used encodings ──────────────────────────────────────────────────────── - - private static Set collectUsedEncodings(VortexHandle reader) { - var used = new LinkedHashSet(); - collectLayoutEncodings(reader.layout(), reader, used); - return used; - } - - private static void collectLayoutEncodings(Layout layout, VortexHandle reader, Set used) { - if (layout.isFlat() && !layout.segments().isEmpty()) { - int segIdx = layout.segments().getFirst(); - SegmentSpec spec = reader.footer().segmentSpecs().get(segIdx); - if (spec.compression().code == 0) { - MemorySegment seg = reader.slice(spec.offset(), spec.length()); - peekRootEncoding(seg, reader.footer().arraySpecs(), used); - } - } - for (Layout child : layout.children()) { - collectLayoutEncodings(child, reader, used); - } - } - - /// Reads only the root ArrayNode encoding — ignores child/stats sub-nodes. - private static void peekRootEncoding(MemorySegment seg, List arraySpecs, Set used) { - int segLen = (int) seg.byteSize(); - ByteBuffer bb = seg.asByteBuffer().order(ByteOrder.LITTLE_ENDIAN); - int fbLen = bb.getInt(segLen - 4); - int fbStart = segLen - 4 - fbLen; - ByteBuffer fbBuf = bb.slice(fbStart, fbLen).order(ByteOrder.LITTLE_ENDIAN); - var fbArray = io.github.dfa1.vortex.fbs.Array.getRootAsArray(fbBuf); - if (fbArray.root() != null) { - used.add(arraySpecs.get(fbArray.root().encoding())); - } - } - - // ── Layout tree ─────────────────────────────────────────────────────────── - - @SuppressWarnings("SameParameterValue") - private static void appendLayout(StringBuilder sb, Layout layout, List colNames, - VortexHandle reader, String indent) { - if (layout.isStruct()) { - sb.append(indent).append("struct (").append(layout.rowCount()).append(" rows)\n"); - for (int i = 0; i < layout.children().size(); i++) { - String name = i < colNames.size() ? colNames.get(i) : "col" + i; - Set colEncodings = new LinkedHashSet<>(); - collectLayoutEncodings(layout.children().get(i), reader, colEncodings); - sb.append(indent).append(" ").append(name).append(": "); - appendLayoutInline(sb, layout.children().get(i)); - if (!colEncodings.isEmpty()) { - sb.append(" [").append(String.join(", ", colEncodings)).append("]"); - } - sb.append('\n'); - } - } else { - sb.append(indent); - appendLayoutInline(sb, layout); - sb.append('\n'); - } - } - - private static void appendLayoutInline(StringBuilder sb, Layout layout) { - sb.append(layout.encodingId()).append('(').append(layout.rowCount()).append(" rows)"); - if (layout.children().isEmpty()) { - return; - } - sb.append(" → "); - if (layout.children().size() == 1) { - appendLayoutInline(sb, layout.children().getFirst()); - } else { - sb.append(layout.children().size()).append("× ["); - appendLayoutInline(sb, layout.children().getFirst()); - sb.append("]"); - } - } - - // ── Formatting ──────────────────────────────────────────────────────────── - - @SuppressWarnings("SameParameterValue") - private static void appendSchema(StringBuilder sb, DType dtype, String indent) { - if (dtype instanceof DType.Struct s) { - int maxLen = s.fieldNames().stream().mapToInt(String::length).max().orElse(0); - for (int i = 0; i < s.fieldNames().size(); i++) { - String name = s.fieldNames().get(i); - sb.append(indent).append(name) - .append(" ".repeat(maxLen - name.length() + 1)) - .append(formatDType(s.fieldTypes().get(i))).append('\n'); - } - } else { - sb.append(indent).append(formatDType(dtype)).append('\n'); - } - } - - private static String formatDType(DType dtype) { - return switch (dtype) { - case DType.Primitive(var pt, var nullable) -> pt.name() + (nullable ? "?" : ""); - case DType.Utf8(var nullable) -> "utf8" + (nullable ? "?" : ""); - case DType.Binary(var nullable) -> "binary" + (nullable ? "?" : ""); - case DType.Bool(var nullable) -> "bool" + (nullable ? "?" : ""); - case DType.Null ignored -> "null"; - case DType.Decimal(var p, var s, var nullable) -> "decimal(" + p + "," + s + ")" + (nullable ? "?" : ""); - case DType.Struct ignored -> "struct"; - case DType.List(var elem, var nullable) -> "list<" + formatDType(elem) + ">" + (nullable ? "?" : ""); - case DType.FixedSizeList(var elem, var size, var nullable) -> - "list<" + formatDType(elem) + ">[" + size + "]" + (nullable ? "?" : ""); - case DType.Extension(var id, var storage, var meta, var nullable) -> - "ext<" + id + ">" + (nullable ? "?" : ""); - }; - } - - private static String formatBytes(long bytes) { - if (bytes < 1024) { - return bytes + " B"; - } - if (bytes < 1024 * 1024) { - return String.format("%.1f KB", bytes / 1024.0); - } - return String.format("%.1f MB", bytes / (1024.0 * 1024.0)); - } -} diff --git a/reader/src/main/java/io/github/dfa1/vortex/io/VortexReader.java b/reader/src/main/java/io/github/dfa1/vortex/io/VortexReader.java deleted file mode 100644 index fbbc2aa..0000000 --- a/reader/src/main/java/io/github/dfa1/vortex/io/VortexReader.java +++ /dev/null @@ -1,246 +0,0 @@ -package io.github.dfa1.vortex.io; - -import io.github.dfa1.vortex.core.ArrayStats; -import io.github.dfa1.vortex.core.DType; -import io.github.dfa1.vortex.core.Footer; -import io.github.dfa1.vortex.core.Layout; -import io.github.dfa1.vortex.core.SegmentSpec; -import io.github.dfa1.vortex.core.VortexException; -import io.github.dfa1.vortex.encoding.EncodingRegistry; -import io.github.dfa1.vortex.scan.ScanIterator; -import io.github.dfa1.vortex.scan.ScanOptions; - -import java.io.IOException; -import java.lang.foreign.Arena; -import java.lang.foreign.MemorySegment; -import java.lang.foreign.ValueLayout; -import java.nio.ByteOrder; -import java.nio.channels.FileChannel; -import java.nio.file.Path; -import java.nio.file.StandardOpenOption; -import java.util.ArrayList; -import java.util.LinkedHashMap; -import java.util.List; -import java.util.Map; - -/// Handle to an open Vortex file. Memory-maps the file via the FFM API; -/// all Array buffers returned during scan are slices of this `MemorySegment`. -/// -/// Close this to release the memory-mapped region. -public final class VortexReader implements VortexHandle { - - static final ValueLayout.OfShort LE_SHORT = - ValueLayout.JAVA_SHORT_UNALIGNED.withOrder(ByteOrder.LITTLE_ENDIAN); - static final ValueLayout.OfInt LE_INT = - ValueLayout.JAVA_INT_UNALIGNED.withOrder(ByteOrder.LITTLE_ENDIAN); - static final ValueLayout.OfLong LE_LONG = - ValueLayout.JAVA_LONG_UNALIGNED.withOrder(ByteOrder.LITTLE_ENDIAN); - - static final byte[] MAGIC = {'V', 'T', 'X', 'F'}; - static final int TRAILER_SIZE = 8; - - private final Arena arena; - private final MemorySegment fileSegment; - private final long fileSize; - private final int version; - private final Footer footer; - private final DType dtype; - private final Layout layout; - private final EncodingRegistry registry; - - private VortexReader( - Arena arena, MemorySegment fileSegment, long fileSize, - int version, Footer footer, DType dtype, Layout layout, - EncodingRegistry registry - ) { - this.arena = arena; - this.fileSegment = fileSegment; - this.fileSize = fileSize; - this.version = version; - this.footer = footer; - this.dtype = dtype; - this.layout = layout; - this.registry = registry; - } - - /// Open a Vortex file. Memory-maps the entire file; all subsequent reads - /// are zero-copy slices. Call [#close()] when done. - public static VortexReader open(Path path) throws IOException { - return open(path, EncodingRegistry.loadAll()); - } - - public static VortexReader open(Path path, EncodingRegistry registry) throws IOException { - Arena arena = Arena.ofConfined(); - try (var channel = FileChannel.open(path, StandardOpenOption.READ)) { - long size = channel.size(); - if (size < TRAILER_SIZE) { - throw new VortexException("file too small (" + size + " bytes)"); - } - // The channel is no longer needed after map(): the Arena owns the mapping's - // lifetime. try-with-resources closes the file descriptor while all Array - // buffers remain valid zero-copy slices until arena.close() is called. - var segment = channel.map(FileChannel.MapMode.READ_ONLY, 0, size, arena); - return parse(segment, size, arena, registry); - } catch (Exception e) { - arena.close(); - throw e; - } - } - - private static VortexReader parse( - MemorySegment seg, long size, Arena arena, EncodingRegistry registry - ) { - // 8-byte trailer: version(u16 LE) | postscriptLen(u16 LE) | magic(4) - var trailer = seg.asSlice(size - TRAILER_SIZE, TRAILER_SIZE); - - int version = Short.toUnsignedInt(trailer.get(LE_SHORT, 0)); - int postscriptLen = Short.toUnsignedInt(trailer.get(LE_SHORT, 2)); - - byte m0 = trailer.get(ValueLayout.JAVA_BYTE, 4); - byte m1 = trailer.get(ValueLayout.JAVA_BYTE, 5); - byte m2 = trailer.get(ValueLayout.JAVA_BYTE, 6); - byte m3 = trailer.get(ValueLayout.JAVA_BYTE, 7); - if (m0 != MAGIC[0] || m1 != MAGIC[1] || m2 != MAGIC[2] || m3 != MAGIC[3]) { - throw new VortexException( - "invalid magic bytes [%02x %02x %02x %02x]".formatted(m0, m1, m2, m3)); - } - - long postscriptOffset = size - TRAILER_SIZE - postscriptLen; - var postscriptBuf = seg.asSlice(postscriptOffset, postscriptLen) - .asByteBuffer().order(ByteOrder.LITTLE_ENDIAN); - - var parsed = PostscriptParser.parse(postscriptBuf, seg); - - return new VortexReader( - arena, seg, size, version, - parsed.footer(), parsed.dtype(), parsed.layout(), - registry - ); - } - - @Override - public DType dtype() { - return dtype; - } - - @Override - public Layout layout() { - return layout; - } - - @Override - public Footer footer() { - return footer; - } - - @Override - public int version() { - return version; - } - - @Override - public long fileSize() { - return fileSize; - } - - @Override - public EncodingRegistry registry() { - return registry; - } - - @Override - public ScanIterator scan(ScanOptions options) { - return new ScanIterator(this, options); - } - - /// Aggregated per-column statistics (global min/max across all chunks). - /// Returns an empty map if the root layout is not a struct. - /// Columns with no embedded stats return [ArrayStats#empty()]. - public Map columnStats() { - if (!layout.isStruct() || !(dtype instanceof DType.Struct schema)) { - return Map.of(); - } - List names = schema.fieldNames(); - List colLayouts = layout.children(); - Map result = new LinkedHashMap<>(); - for (int i = 0; i < names.size() && i < colLayouts.size(); i++) { - List flats = new ArrayList<>(); - collectFlats(colLayouts.get(i), flats); - result.put(names.get(i), aggregateStats(flats)); - } - return Map.copyOf(result); - } - - private ArrayStats aggregateStats(List flats) { - Object globalMin = null; - Object globalMax = null; - for (Layout flat : flats) { - ArrayStats s = readFlatStats(flat); - if (s.min() != null) { - globalMin = globalMin == null ? s.min() : minOf(globalMin, s.min()); - } - if (s.max() != null) { - globalMax = globalMax == null ? s.max() : maxOf(globalMax, s.max()); - } - } - if (globalMin == null && globalMax == null) { - return ArrayStats.empty(); - } - return new ArrayStats(globalMin, globalMax, null, null, null, null); - } - - private ArrayStats readFlatStats(Layout flat) { - if (flat.segments().isEmpty()) { - return ArrayStats.empty(); - } - int segIdx = flat.segments().getFirst(); - SegmentSpec spec = footer.segmentSpecs().get(segIdx); - long segLen = spec.length(); - MemorySegment seg = fileSegment.asSlice(spec.offset(), segLen); - int fbLen = seg.get(LE_INT, segLen - 4); - long fbStart = segLen - 4L - fbLen; - var fbBuf = seg.asSlice(fbStart, fbLen).asByteBuffer().order(ByteOrder.LITTLE_ENDIAN); - var fbArray = io.github.dfa1.vortex.fbs.Array.getRootAsArray(fbBuf); - var root = fbArray.root(); - if (root == null) { - return ArrayStats.empty(); - } - return ArrayStats.fromFbs(root.stats()); - } - - private static void collectFlats(Layout layout, List out) { - if (layout.isFlat() || layout.isDict()) { - out.add(layout); - } else if (layout.isZoned() && !layout.children().isEmpty()) { - collectFlats(layout.children().getFirst(), out); - } else if (layout.isChunked()) { - int start = (layout.metadata() != null - && layout.metadata().hasRemaining() - && layout.metadata().get(0) == 1) ? 1 : 0; - for (int i = start; i < layout.children().size(); i++) { - collectFlats(layout.children().get(i), out); - } - } - } - - @SuppressWarnings("unchecked") - private static Object minOf(Object a, Object b) { - return ((Comparable) a).compareTo(b) <= 0 ? a : b; - } - - @SuppressWarnings("unchecked") - private static Object maxOf(Object a, Object b) { - return ((Comparable) a).compareTo(b) >= 0 ? a : b; - } - - /// Zero-copy slice of the memory-mapped file. - @Override - public MemorySegment slice(long offset, long length) { - return fileSegment.asSlice(offset, length); - } - - @Override - public void close() { - arena.close(); - } -} diff --git a/reader/src/main/java/io/github/dfa1/vortex/scan/RowFilter.java b/reader/src/main/java/io/github/dfa1/vortex/scan/RowFilter.java deleted file mode 100644 index fcc6b10..0000000 --- a/reader/src/main/java/io/github/dfa1/vortex/scan/RowFilter.java +++ /dev/null @@ -1,63 +0,0 @@ -package io.github.dfa1.vortex.scan; - -import java.util.List; - -/// Predicate tree for zone-map pruning. Evaluated against per-chunk min/max statistics; -/// chunks where no row can satisfy the filter are skipped entirely. -public sealed interface RowFilter - permits RowFilter.And, RowFilter.Eq, RowFilter.Neq, - RowFilter.Gt, RowFilter.Gte, RowFilter.Lt, RowFilter.Lte { - - static RowFilter and(RowFilter... filters) { - return new And(List.of(filters)); - } - - default RowFilter and(RowFilter other) { - return new And(List.of(this, other)); - } - - static RowFilter eq(String col, Object val) { - return new Eq(col, val); - } - - static RowFilter neq(String col, Object val) { - return new Neq(col, val); - } - - static RowFilter gt(String col, Comparable val) { - return new Gt(col, val); - } - - static RowFilter gte(String col, Comparable val) { - return new Gte(col, val); - } - - static RowFilter lt(String col, Comparable val) { - return new Lt(col, val); - } - - static RowFilter lte(String col, Comparable val) { - return new Lte(col, val); - } - - record And(List filters) implements RowFilter { - } - - record Eq(String column, Object value) implements RowFilter { - } - - record Neq(String column, Object value) implements RowFilter { - } - - record Gt(String column, Comparable value) implements RowFilter { - } - - record Gte(String column, Comparable value) implements RowFilter { - } - - record Lt(String column, Comparable value) implements RowFilter { - } - - record Lte(String column, Comparable value) implements RowFilter { - } -} diff --git a/reader/src/main/java/io/github/dfa1/vortex/scan/ScanIterator.java b/reader/src/main/java/io/github/dfa1/vortex/scan/ScanIterator.java deleted file mode 100644 index 14bb5e5..0000000 --- a/reader/src/main/java/io/github/dfa1/vortex/scan/ScanIterator.java +++ /dev/null @@ -1,564 +0,0 @@ -package io.github.dfa1.vortex.scan; - -import com.google.protobuf.InvalidProtocolBufferException; -import io.github.dfa1.vortex.proto.EncodingProtos; -import io.github.dfa1.vortex.core.array.Array; -import io.github.dfa1.vortex.core.array.StructArray; -import io.github.dfa1.vortex.core.ArrayStats; -import io.github.dfa1.vortex.core.DType; -import io.github.dfa1.vortex.core.Layout; -import io.github.dfa1.vortex.core.PType; -import io.github.dfa1.vortex.core.SegmentSpec; -import io.github.dfa1.vortex.core.array.BoolArray; -import io.github.dfa1.vortex.core.array.MaskedArray; -import io.github.dfa1.vortex.core.array.ByteArray; -import io.github.dfa1.vortex.core.array.EmptyArray; -import io.github.dfa1.vortex.core.array.NullArray; -import io.github.dfa1.vortex.core.array.DoubleArray; -import io.github.dfa1.vortex.core.array.FloatArray; -import io.github.dfa1.vortex.core.array.IntArray; -import io.github.dfa1.vortex.core.array.LongArray; -import io.github.dfa1.vortex.core.array.ShortArray; -import io.github.dfa1.vortex.core.array.VarBinArray; -import io.github.dfa1.vortex.core.VortexException; -import io.github.dfa1.vortex.encoding.EncodingId; -import io.github.dfa1.vortex.io.VortexHandle; - -import java.lang.foreign.Arena; -import java.lang.foreign.MemorySegment; -import java.lang.foreign.SegmentAllocator; -import java.lang.foreign.ValueLayout; -import java.nio.ByteBuffer; -import java.nio.ByteOrder; -import java.util.ArrayList; -import java.util.LinkedHashMap; -import java.util.List; -import java.util.Map; - -/// Iterates over decoded chunks from a [VortexReader]. -/// -/// Usage: -/// ```java -/// try (var iter = file.scan(ScanOptions.all())) { -/// while (iter.hasNext()) { -/// ScanResult chunk = iter.next(); -/// } -/// } -/// ``` -public final class ScanIterator implements AutoCloseable { - - private static final ValueLayout.OfShort LE_SHORT = ValueLayout.JAVA_SHORT_UNALIGNED.withOrder(ByteOrder.LITTLE_ENDIAN); - private static final ValueLayout.OfInt LE_INT = ValueLayout.JAVA_INT_UNALIGNED.withOrder(ByteOrder.LITTLE_ENDIAN); - private static final ValueLayout.OfLong LE_LONG = ValueLayout.JAVA_LONG_UNALIGNED.withOrder(ByteOrder.LITTLE_ENDIAN); - - private final VortexHandle file; - private final ScanOptions options; - private Arena chunkArena; - - private List chunks; - private List projectedNames; - private List projectedDtypes; - private int chunkIndex; - private long rowsReturned; - private ScanResult current; - - public ScanIterator(VortexHandle file, ScanOptions options) { - this.file = file; - this.options = options; - } - - private static void collectFlats(Layout layout, List out) { - if (layout.isFlat()) { - out.add(layout); - } else if (layout.isDict()) { - // Dict layout is a leaf chunk — decoded as a unit (values + codes). - out.add(layout); - } else if (layout.isZoned()) { - // vortex.stats wraps one child (the data layout) — pass through for data - if (!layout.children().isEmpty()) { - collectFlats(layout.children().getFirst(), out); - } - } else if (layout.isChunked()) { - // metadata[0] == 1 means children[0] is the per-chunk stats layout; skip it - int start = (layout.metadata() != null - && layout.metadata().hasRemaining() - && layout.metadata().get(0) == 1) ? 1 : 0; - for (int i = start; i < layout.children().size(); i++) { - collectFlats(layout.children().get(i), out); - } - } - } - - private static List buildChunks(Map> columnFlats) { - if (columnFlats.isEmpty()) { - return List.of(); - } - String[] colNames = columnFlats.keySet().toArray(String[]::new); - int numCols = colNames.length; - int numChunks = columnFlats.values().iterator().next().size(); - var result = new ArrayList(numChunks); - for (int i = 0; i < numChunks; i++) { - Layout[] layouts = new Layout[numCols]; - for (int j = 0; j < numCols; j++) { - layouts[j] = columnFlats.get(colNames[j]).get(i); - } - result.add(new ChunkSpec(layouts[0].rowCount(), colNames, layouts)); - } - return List.copyOf(result); - } - - // ── Layout tree traversal ───────────────────────────────────────────────── - - @SuppressWarnings("unchecked") - private static int compareValues(Object a, Object b) { - try { - return ((Comparable) a).compareTo(b); - } catch (ClassCastException e) { - return 0; - } - } - - public boolean hasNext() { - if (chunks == null) { - initialize(); - } - if (rowsReturned >= options.limit()) { - return false; - } - - while (chunkIndex < chunks.size()) { - ChunkSpec chunk = chunks.get(chunkIndex++); - if (options.hasFilter() && canPruneChunk(chunk, options.rowFilter())) { - continue; - } - - if (chunkArena != null) { - chunkArena.close(); - } - chunkArena = Arena.ofConfined(); - - long remaining = options.limit() - rowsReturned; - long chunkRows = Math.min(chunk.rowCount(), remaining); - Map columns = buildColumnMap(chunk); - if (chunkRows < chunk.rowCount()) { - columns = truncateColumns(columns, chunkRows); - } - current = new ScanResult(chunkRows, columns); - rowsReturned += chunkRows; - return true; - } - // Do not close here — the last chunk's arena stays open until close() is called. - // This preserves data validity for callers that collect results before processing. - return false; - } - - public ScanResult next() { - if (current == null) { - throw new VortexException("call hasNext() first"); - } - return current; - } - - // ── Column map builder ──────────────────────────────────────────────────── - - @Override - public void close() { - if (chunkArena != null) { - chunkArena.close(); - chunkArena = null; - } - } - - // ── Flat segment decoding ───────────────────────────────────────────────── - - private void initialize() { - Layout rootLayout = file.layout(); - DType rootDtype = file.dtype(); - - var columnFlats = new LinkedHashMap>(); - Map columnDtypes = new LinkedHashMap<>(); - - if (rootLayout.isStruct() && rootDtype instanceof DType.Struct structDtype) { - List projection = options.columns(); - for (int i = 0; i < rootLayout.children().size(); i++) { - String colName = structDtype.fieldNames().get(i); - DType colDtype = structDtype.fieldTypes().get(i); - if (!projection.isEmpty() && !projection.contains(colName)) { - continue; - } - var flats = new ArrayList(); - collectFlats(rootLayout.children().get(i), flats); - columnFlats.put(colName, flats); - columnDtypes.put(colName, colDtype); - } - } else { - var flats = new ArrayList(); - collectFlats(rootLayout, flats); - columnFlats.put("_col", flats); - columnDtypes.put("_col", rootDtype); - } - - chunks = buildChunks(columnFlats); - projectedNames = List.copyOf(columnDtypes.keySet()); - projectedDtypes = List.copyOf(columnDtypes.values()); - } - - // ── Zone-map pruning ────────────────────────────────────────────────────── - - // Map.of with 1 or 2 args allocates Map1/Map2 (~2-4 fields) — avoids the - // LinkedHashMap + Map.copyOf pair that would otherwise allocate per chunk. - // Direct array index into ChunkSpec.columnLayouts avoids HashMap.get() per column. - private Map buildColumnMap(ChunkSpec chunk) { - Layout[] layouts = chunk.columnLayouts(); - int n = projectedNames.size(); - if (n == 1) { - Array arr = decodeLayout(layouts[0], projectedDtypes.getFirst(), chunkArena); - if (arr instanceof StructArray sa) { - return expandStruct(sa); - } - return Map.of(projectedNames.getFirst(), arr); - } - if (n == 2) { - return Map.of( - projectedNames.get(0), decodeLayout(layouts[0], projectedDtypes.get(0), chunkArena), - projectedNames.get(1), decodeLayout(layouts[1], projectedDtypes.get(1), chunkArena)); - } - var scratch = new LinkedHashMap(n); - for (int i = 0; i < n; i++) { - scratch.put(projectedNames.get(i), decodeLayout(layouts[i], projectedDtypes.get(i), chunkArena)); - } - return Map.copyOf(scratch); - } - - private static Map expandStruct(StructArray sa) { - DType.Struct sd = (DType.Struct) sa.dtype(); - List names = sd.fieldNames(); - int n = names.size(); - var map = new LinkedHashMap(n); - for (int i = 0; i < n; i++) { - map.put(names.get(i), sa.field(i)); - } - return Map.copyOf(map); - } - - private Array decodeLayout(Layout layout, DType dtype, SegmentAllocator arena) { - if (layout.isFlat()) { - return decodeFlat(layout, dtype, arena); - } - if (layout.isDict()) { - return decodeDictLayout(layout, dtype, arena); - } - if (layout.isZoned() && !layout.children().isEmpty()) { - return decodeLayout(layout.children().getFirst(), dtype, arena); - } - if (layout.isChunked()) { - var flats = new ArrayList(); - collectFlats(layout, flats); - return decodeConcatPrimitive(flats, dtype, layout.rowCount(), arena); - } - throw new VortexException("cannot decode layout " + layout.encodingId()); - } - - private Array decodeConcatPrimitive(List flats, DType dtype, long totalRows, SegmentAllocator arena) { - if (flats.isEmpty()) { - throw new VortexException(EncodingId.VORTEX_CHUNKED, "no flat children"); - } - if (flats.size() == 1) { - return decodeFlat(flats.getFirst(), dtype, arena); - } - PType ptype = ((DType.Primitive) dtype).ptype(); - MemorySegment combined = arena.allocate(totalRows * ptype.byteSize()); - long byteOffset = 0; - for (Layout flat : flats) { - Array chunk = decodeFlat(flat, dtype, arena); - MemorySegment src = chunk.buffer(0); - MemorySegment.copy(src, 0, combined, byteOffset, src.byteSize()); - byteOffset += src.byteSize(); - } - MemorySegment ro = combined.asReadOnly(); - return switch (ptype) { - case I64, U64 -> new LongArray(dtype, totalRows, ro, ArrayStats.empty()); - case I32, U32 -> new IntArray(dtype, totalRows, ro, ArrayStats.empty()); - case F64 -> new DoubleArray(dtype, totalRows, ro, ArrayStats.empty()); - case F32 -> new FloatArray(dtype, totalRows, ro, ArrayStats.empty()); - case I16, U16 -> new ShortArray(dtype, totalRows, ro, ArrayStats.empty()); - case I8, U8 -> new ByteArray(dtype, totalRows, ro, ArrayStats.empty()); - default -> throw new VortexException("unsupported ptype for concat: " + ptype); - }; - } - - private Array decodeFlat(Layout flat, DType dtype, SegmentAllocator arena) { - if (flat.segments().isEmpty()) { - throw new VortexException("no segments"); - } - int segIdx = flat.segments().getFirst(); - SegmentSpec spec = file.footer().segmentSpecs().get(segIdx); - MemorySegment seg = file.slice(spec.offset(), spec.length()); - return file.registry().decodeSegment(seg, file.footer().arraySpecs(), dtype, flat.rowCount(), arena); - } - - private Array decodeDictLayout(Layout dictLayout, DType dtype, SegmentAllocator arena) { - ByteBuffer rawMeta = dictLayout.metadata(); - if (rawMeta == null) { - throw new VortexException(EncodingId.VORTEX_DICT, "layout: missing metadata"); - } - EncodingProtos.DictMetadata meta; - try { - meta = EncodingProtos.DictMetadata.parseFrom(rawMeta.duplicate()); - } catch (InvalidProtocolBufferException e) { - throw new VortexException(EncodingId.VORTEX_DICT, "layout: invalid metadata", e); - } - PType codesPType = PType.values()[meta.getCodesPtype().getNumber()]; - - // child[0] = values layout; child[1] = codes layout - Layout valuesLayout = dictLayout.children().get(0); - Layout codesLayout = dictLayout.children().get(1); - long n = codesLayout.rowCount(); - - Array values = decodeLayout(valuesLayout, dtype, arena); - Array codes = decodeLayout(codesLayout, new DType.Primitive(codesPType, false), arena); - - if (values instanceof VarBinArray) { - MemorySegment valOffsets = values.child(0).buffer(0); - PType valOffPType = ((DType.Primitive) values.child(0).dtype()).ptype(); - return VarBinArray.ofDict(dtype, n, values.buffer(0), valOffsets, valOffPType, - codes.buffer(0), codesPType, ArrayStats.empty()); - } - if (dtype instanceof DType.Primitive pDtype) { - return expandDictPrimitive(values, codes, codesPType, pDtype, n, arena); - } - return expandDictStrings(values, codes, codesPType, dtype, n, arena); - } - - private static Array expandDictPrimitive( - Array values, Array codes, - PType codesPType, DType.Primitive dtype, - long n, SegmentAllocator arena - ) { - PType ptype = dtype.ptype(); - int elemBytes = ptype.byteSize(); - MemorySegment valBuf = values.buffer(0); - MemorySegment codesBuf = codes.buffer(0); - MemorySegment out = arena.allocate(n * elemBytes, elemBytes); - for (long i = 0; i < n; i++) { - long code = readUnsigned(codesBuf, i, codesPType); - MemorySegment.copy(valBuf, code * elemBytes, out, i * elemBytes, elemBytes); - } - return switch (ptype) { - case I32, U32 -> new IntArray(dtype, n, out.asReadOnly(), ArrayStats.empty()); - case I64, U64 -> new LongArray(dtype, n, out.asReadOnly(), ArrayStats.empty()); - case F64 -> new DoubleArray(dtype, n, out.asReadOnly(), ArrayStats.empty()); - case F32 -> new FloatArray(dtype, n, out.asReadOnly(), ArrayStats.empty()); - case I16, U16 -> new ShortArray(dtype, n, out.asReadOnly(), ArrayStats.empty()); - case I8, U8 -> new ByteArray(dtype, n, out.asReadOnly(), ArrayStats.empty()); - default -> throw new VortexException(EncodingId.VORTEX_DICT, - "layout: unsupported ptype for dict expansion: " + ptype); - }; - } - - private static Array expandDictStrings( - Array values, Array codes, - PType codesPType, DType dtype, - long n, SegmentAllocator arena - ) { - MemorySegment valBytes = values.buffer(0); - MemorySegment valOffsets = values.child(0).buffer(0); - PType valOffPType = ((DType.Primitive) values.child(0).dtype()).ptype(); - MemorySegment codesSegs = codes.buffer(0); - - // First pass: total output byte length - long totalBytes = 0L; - for (long i = 0; i < n; i++) { - long code = readUnsigned(codesSegs, i, codesPType); - long start = readUnsigned(valOffsets, code, valOffPType); - long end = readUnsigned(valOffsets, code + 1, valOffPType); - totalBytes += end - start; - } - - MemorySegment outBytes = arena.allocate(totalBytes > 0 ? totalBytes : 1); - MemorySegment outOffsets = arena.allocate((n + 1) * 4L, 4); - outOffsets.setAtIndex(LE_INT, 0, 0); - - long bytePos = 0L; - for (long i = 0; i < n; i++) { - long code = readUnsigned(codesSegs, i, codesPType); - long start = readUnsigned(valOffsets, code, valOffPType); - long end = readUnsigned(valOffsets, code + 1, valOffPType); - long strLen = end - start; - if (strLen > 0) { - MemorySegment.copy(valBytes, start, outBytes, bytePos, strLen); - bytePos += strLen; - } - outOffsets.setAtIndex(LE_INT, i + 1, (int) bytePos); - } - - DType i32 = new DType.Primitive(PType.I32, false); - Array offsetArr = new IntArray(i32, n + 1, outOffsets.asReadOnly(), ArrayStats.empty()); - return new VarBinArray(dtype, n, outBytes.asReadOnly(), offsetArr, PType.I32, ArrayStats.empty()); - } - - private static long readUnsigned(MemorySegment seg, long idx, PType ptype) { - return switch (ptype) { - case U8 -> Byte.toUnsignedLong(seg.get(ValueLayout.JAVA_BYTE, idx)); - case U16 -> Short.toUnsignedLong(seg.get(LE_SHORT, idx * 2)); - case U32 -> Integer.toUnsignedLong(seg.getAtIndex(LE_INT, idx)); - case I32 -> seg.getAtIndex(LE_INT, idx); - case I64, U64 -> seg.getAtIndex(LE_LONG, idx); - default -> throw new VortexException(EncodingId.VORTEX_DICT, "layout: unsupported ptype " + ptype); - }; - } - - // ── Limit truncation ───────────────────────────────────────────────────── - - private static Map truncateColumns(Map columns, long rows) { - var result = new LinkedHashMap(columns.size()); - for (var entry : columns.entrySet()) { - result.put(entry.getKey(), truncateArray(entry.getValue(), rows)); - } - return Map.copyOf(result); - } - - private static Array truncateArray(Array arr, long rows) { - if (arr.length() <= rows) { - return arr; - } - return switch (arr) { - case LongArray a -> new LongArray(a.dtype(), rows, a.buffer(0).asSlice(0, rows * Long.BYTES), ArrayStats.empty()); - case IntArray a -> new IntArray(a.dtype(), rows, a.buffer(0).asSlice(0, rows * Integer.BYTES), ArrayStats.empty()); - case DoubleArray a -> new DoubleArray(a.dtype(), rows, a.buffer(0).asSlice(0, rows * Double.BYTES), ArrayStats.empty()); - case FloatArray a -> new FloatArray(a.dtype(), rows, a.buffer(0).asSlice(0, rows * Float.BYTES), ArrayStats.empty()); - case ShortArray a -> new ShortArray(a.dtype(), rows, a.buffer(0).asSlice(0, rows * Short.BYTES), ArrayStats.empty()); - case ByteArray a -> new ByteArray(a.dtype(), rows, a.buffer(0).asSlice(0, rows), ArrayStats.empty()); - case BoolArray a -> new BoolArray(a.dtype(), rows, a.buffer(0).asSlice(0, (rows + 7) / 8), ArrayStats.empty()); - case NullArray a -> new NullArray(a.dtype(), rows); - case VarBinArray a -> a.truncate(rows); - case MaskedArray a -> { - Array truncChild = truncateArray((Array) a.child(0), rows); - BoolArray v = a.validity(); - BoolArray truncValidity = (v != null) ? (BoolArray) truncateArray(v, rows) : null; - yield new MaskedArray(truncChild, truncValidity); - } - case EmptyArray a -> a; - default -> throw new VortexException("limit: truncation not supported for " + arr.getClass().getSimpleName()); - }; - } - - private boolean canPruneChunk(ChunkSpec chunk, RowFilter filter) { - return switch (filter) { - case RowFilter.And(var filters) -> { - for (RowFilter f : filters) { - if (canPruneChunk(chunk, f)) { - yield true; - } - } - yield false; - } - case RowFilter.Gt(var col, var val) -> { - Layout flat = chunk.layoutFor(col); - if (flat == null) { - yield false; - } - Object max = readFlatStats(flat).max(); - yield max != null && compareValues(max, val) <= 0; - } - case RowFilter.Gte(var col, var val) -> { - Layout flat = chunk.layoutFor(col); - if (flat == null) { - yield false; - } - Object max = readFlatStats(flat).max(); - yield max != null && compareValues(max, val) < 0; - } - case RowFilter.Lt(var col, var val) -> { - Layout flat = chunk.layoutFor(col); - if (flat == null) { - yield false; - } - Object min = readFlatStats(flat).min(); - yield min != null && compareValues(min, val) >= 0; - } - case RowFilter.Lte(var col, var val) -> { - Layout flat = chunk.layoutFor(col); - if (flat == null) { - yield false; - } - Object min = readFlatStats(flat).min(); - yield min != null && compareValues(min, val) > 0; - } - case RowFilter.Eq(var col, var val) -> { - Layout flat = chunk.layoutFor(col); - if (flat == null) { - yield false; - } - ArrayStats stats = readFlatStats(flat); - Object min = stats.min(); - Object max = stats.max(); - if (min == null || max == null) { - yield false; - } - try { - @SuppressWarnings("unchecked") - Comparable cv = (Comparable) val; - yield cv.compareTo(min) < 0 || cv.compareTo(max) > 0; - } catch (ClassCastException e) { - yield false; - } - } - case RowFilter.Neq(var col, var val) -> { - Layout flat = chunk.layoutFor(col); - if (flat == null) { - yield false; - } - ArrayStats stats = readFlatStats(flat); - Object min = stats.min(); - Object max = stats.max(); - if (min == null || max == null) { - yield false; - } - try { - @SuppressWarnings("unchecked") - Comparable cv = (Comparable) val; - yield cv.compareTo(min) == 0 && cv.compareTo(max) == 0; - } catch (ClassCastException e) { - yield false; - } - } - }; - } - - private ArrayStats readFlatStats(Layout flat) { - if (flat.segments().isEmpty()) { - return ArrayStats.empty(); - } - int segIdx = flat.segments().getFirst(); - SegmentSpec spec = file.footer().segmentSpecs().get(segIdx); - long segLen = spec.length(); - MemorySegment seg = file.slice(spec.offset(), segLen); - - // Stats FlatBuffer lives in the segment's last 4+fbLen bytes; reading the whole - // segment as a ByteBuffer would fail for segments larger than 2 GB (ByteBuffer cap). - int fbLen = seg.get(LE_INT, segLen - 4); - long fbStart = segLen - 4L - fbLen; - ByteBuffer fbBuf = seg.asSlice(fbStart, fbLen).asByteBuffer().order(ByteOrder.LITTLE_ENDIAN); - var fbArray = io.github.dfa1.vortex.fbs.Array.getRootAsArray(fbBuf); - - io.github.dfa1.vortex.fbs.ArrayNode root = fbArray.root(); - if (root == null) { - return ArrayStats.empty(); - } - return ArrayStats.fromFbs(root.stats()); - } - - // ── Internal record ─────────────────────────────────────────────────────── - - private record ChunkSpec(long rowCount, String[] columnNames, Layout[] columnLayouts) { - Layout layoutFor(String col) { - for (int i = 0; i < columnNames.length; i++) { - if (columnNames[i].equals(col)) { - return columnLayouts[i]; - } - } - return null; - } - } -} diff --git a/reader/src/main/java/io/github/dfa1/vortex/scan/ScanOptions.java b/reader/src/main/java/io/github/dfa1/vortex/scan/ScanOptions.java deleted file mode 100644 index 67e68ce..0000000 --- a/reader/src/main/java/io/github/dfa1/vortex/scan/ScanOptions.java +++ /dev/null @@ -1,51 +0,0 @@ -package io.github.dfa1.vortex.scan; - -import java.util.List; - -/// Options controlling a file scan. -/// -/// Empty `columns` = read all columns. -/// Null `rowFilter` = no zone-map pruning. -public record ScanOptions( - List columns, - RowFilter rowFilter, - long limit -) { - public static final long NO_LIMIT = Long.MAX_VALUE; - - public static ScanOptions all() { - return new ScanOptions(List.of(), null, NO_LIMIT); - } - - public static ScanOptions columns(String... names) { - return new ScanOptions(List.of(names), null, NO_LIMIT); - } - - public static ScanOptions limit(long limit) { - return new ScanOptions(List.of(), null, limit); - } - - public ScanOptions withColumns(String... names) { - return new ScanOptions(List.of(names), rowFilter, limit); - } - - public ScanOptions withLimit(long limit) { - return new ScanOptions(columns, rowFilter, limit); - } - - public ScanOptions withFilter(RowFilter filter) { - return new ScanOptions(columns, filter, limit); - } - - public boolean hasProjection() { - return !columns.isEmpty(); - } - - public boolean hasFilter() { - return rowFilter != null; - } - - public boolean hasLimit() { - return limit != NO_LIMIT; - } -} diff --git a/reader/src/main/java/io/github/dfa1/vortex/scan/ScanResult.java b/reader/src/main/java/io/github/dfa1/vortex/scan/ScanResult.java deleted file mode 100644 index 24a0195..0000000 --- a/reader/src/main/java/io/github/dfa1/vortex/scan/ScanResult.java +++ /dev/null @@ -1,21 +0,0 @@ -package io.github.dfa1.vortex.scan; - -import io.github.dfa1.vortex.core.array.Array; -import io.github.dfa1.vortex.core.VortexException; - -import java.util.Map; - -/// One decoded chunk returned by [ScanIterator]. -public record ScanResult( - long rowCount, - Map columns -) { - @SuppressWarnings("unchecked") - public T column(String name) { - Array arr = columns.get(name); - if (arr == null) { - throw new VortexException("unknown column: " + name); - } - return (T) arr; - } -} diff --git a/reader/src/test/java/io/github/dfa1/vortex/io/PostscriptParserBigSegmentTest.java b/reader/src/test/java/io/github/dfa1/vortex/io/PostscriptParserBigSegmentTest.java deleted file mode 100644 index 21aa057..0000000 --- a/reader/src/test/java/io/github/dfa1/vortex/io/PostscriptParserBigSegmentTest.java +++ /dev/null @@ -1,60 +0,0 @@ -package io.github.dfa1.vortex.io; - -import com.google.flatbuffers.FlatBufferBuilder; -import io.github.dfa1.vortex.core.Footer; -import io.github.dfa1.vortex.core.SegmentSpec; -import io.github.dfa1.vortex.fbs.ArraySpec; -import io.github.dfa1.vortex.fbs.LayoutSpec; -import org.junit.jupiter.api.Test; - -import java.nio.ByteBuffer; - -import static org.assertj.core.api.Assertions.assertThat; - -class PostscriptParserBigSegmentTest { - - @Test - void convertFooter_preservesSegmentLengthAbove2GB() { - // Given — a FlatBuffer Footer whose only segment_spec advertises a uint32 length - // of 0xC000_0000 (3,221,225,472 bytes ≈ 3 GB). This value is negative when - // interpreted as a signed int — the old code path truncated it silently. - long bigLength = 0xC000_0000L; - long bigOffset = 0x1_0000_0000L; // 4 GB into the file - ByteBuffer fbsFooterBytes = buildMinimalFooter(bigOffset, bigLength); - io.github.dfa1.vortex.fbs.Footer fbsFooter = - io.github.dfa1.vortex.fbs.Footer.getRootAsFooter(fbsFooterBytes); - - // When - Footer footer = PostscriptParser.convertFooter(fbsFooter); - - // Then - assertThat(footer.segmentSpecs()).hasSize(1); - SegmentSpec spec = footer.segmentSpecs().getFirst(); - assertThat(spec.offset()).isEqualTo(bigOffset); - assertThat(spec.length()) - .as("u32 segment length above Integer.MAX_VALUE must not be truncated") - .isEqualTo(bigLength); - assertThat(spec.length()).isGreaterThan(Integer.MAX_VALUE); - } - - private static ByteBuffer buildMinimalFooter(long segOffset, long segLength) { - var fbb = new FlatBufferBuilder(256); - - // Empty array_specs + layout_specs vectors (required tables, no entries). - int asv = io.github.dfa1.vortex.fbs.Footer.createArraySpecsVector(fbb, new int[]{ - ArraySpec.createArraySpec(fbb, fbb.createString("vortex.flat")) - }); - int lsv = io.github.dfa1.vortex.fbs.Footer.createLayoutSpecsVector(fbb, new int[]{ - LayoutSpec.createLayoutSpec(fbb, fbb.createString("vortex.flat")) - }); - - // One segment_spec with the big length. - io.github.dfa1.vortex.fbs.Footer.startSegmentSpecsVector(fbb, 1); - io.github.dfa1.vortex.fbs.SegmentSpec.createSegmentSpec(fbb, segOffset, segLength, 6, 0, 0); - int ssv = fbb.endVector(); - - int off = io.github.dfa1.vortex.fbs.Footer.createFooter(fbb, asv, lsv, ssv, 0, 0); - fbb.finish(off); - return fbb.dataBuffer(); - } -} diff --git a/reader/src/test/java/io/github/dfa1/vortex/io/RealWorldDataIT.java b/reader/src/test/java/io/github/dfa1/vortex/io/RealWorldDataIT.java deleted file mode 100644 index 662be06..0000000 --- a/reader/src/test/java/io/github/dfa1/vortex/io/RealWorldDataIT.java +++ /dev/null @@ -1,53 +0,0 @@ -package io.github.dfa1.vortex.io; - -import io.github.dfa1.vortex.scan.ScanOptions; -import org.junit.jupiter.api.Tag; -import org.junit.jupiter.params.ParameterizedTest; -import org.junit.jupiter.params.provider.ValueSource; - -import java.nio.file.Path; - -import static org.assertj.core.api.Assertions.assertThat; -import static org.junit.jupiter.api.Assumptions.assumeTrue; - -/// Integration test: reads real-world Vortex files from local paths (pre-downloaded fixtures). -/// -/// Run with: mvn test -pl reader -Dtest=RealWorldDataIT -Dvortex.fixtures.dir=/path/to/fixtures -/// -/// Skipped automatically when the fixtures directory is not set or files are missing. -@Tag("integration") -class RealWorldDataIT { - - private static final String FIXTURES_DIR = System.getProperty("vortex.fixtures.dir"); - - @ParameterizedTest(name = "{0}") - @ValueSource(strings = { - "clickbench_hits_5k.regular.vortex", - "clickbench_hits_5k.compact.vortex", - "tpch_lineitem.regular.vortex", - "tpch_lineitem.compact.vortex", - "tpch_orders.regular.vortex", - "tpch_orders.compact.vortex", - }) - void scan_realWorldFile_decodesAllRows(String filename) throws Exception { - // Given - assumeTrue(FIXTURES_DIR != null, "set -Dvortex.fixtures.dir to enable"); - Path file = Path.of(FIXTURES_DIR, filename); - assumeTrue(file.toFile().exists(), "fixture not found: " + file); - - // When - long totalRows = 0; - int chunks = 0; - try (var sut = VortexReader.open(file); - var iter = sut.scan(ScanOptions.all())) { - while (iter.hasNext()) { - totalRows += iter.next().rowCount(); - chunks++; - } - } - - // Then - assertThat(totalRows).as("rowCount for %s", filename).isGreaterThan(0); - assertThat(chunks).as("chunkCount for %s", filename).isGreaterThan(0); - } -} diff --git a/reader/src/test/java/io/github/dfa1/vortex/io/VortexHttpReaderIT.java b/reader/src/test/java/io/github/dfa1/vortex/io/VortexHttpReaderIT.java deleted file mode 100644 index e016f9d..0000000 --- a/reader/src/test/java/io/github/dfa1/vortex/io/VortexHttpReaderIT.java +++ /dev/null @@ -1,218 +0,0 @@ -package io.github.dfa1.vortex.io; - -import io.github.dfa1.vortex.core.DType; -import io.github.dfa1.vortex.core.PType; -import io.github.dfa1.vortex.core.array.ListArray; -import io.github.dfa1.vortex.core.array.ListViewArray; -import io.github.dfa1.vortex.scan.ScanOptions; -import org.junit.jupiter.api.Disabled; -import org.junit.jupiter.api.Tag; -import org.junit.jupiter.api.Test; - -import java.net.URI; -import java.util.List; - -import static org.assertj.core.api.Assertions.assertThat; - -/// Integration test: reads real Vortex files from the public S3 compatibility bucket -/// via HTTP Range requests and validates structure + data. -/// -/// Skipped automatically when the network is unavailable. -@Tag("integration") -class VortexHttpReaderIT { - - private static final URI BASE = URI.create("https://vortex-compat-fixtures.s3.amazonaws.com/v0.72.0/arrays/"); - - private static final URI TPCH_LINEITEM = BASE.resolve("tpch_lineitem.compact.vortex"); - - // for.vortex — frame-of-reference encoding, 10 columns in one flat segment - private static final URI FOR_ARRAY = BASE.resolve("for.vortex"); - - @Test - void open_remoteFile_parsesMetadata() throws Exception { - // Given - - // When - try (var sut = VortexHttpReader.open(TPCH_LINEITEM)) { - - // Then - assertThat(sut.version()).isEqualTo(1); - assertThat(sut.fileSize()).isGreaterThan(VortexReader.TRAILER_SIZE); - assertThat(sut.layout()).isNotNull(); - assertThat(sut.footer()).isNotNull(); - assertThat(sut.dtype()).isNotNull(); - } - } - - @Test - void open_remoteFile_layoutRowCountIsPositive() throws Exception { - // Given - - // When — row count comes from the layout (no data decoding required) - try (var sut = VortexHttpReader.open(FOR_ARRAY)) { - - // Then - assertThat(sut.layout().rowCount()).isGreaterThan(0); - assertThat(sut.footer().segmentSpecs()).isNotEmpty(); - } - } - - @Test - void scan_forVortex_decodesAllRows() throws Exception { - // Given - long totalRows = 0; - - // When - try (var sut = VortexHttpReader.open(FOR_ARRAY); - var iter = sut.scan(ScanOptions.all())) { - while (iter.hasNext()) { - totalRows += iter.next().rowCount(); - } - } - - // Then - assertThat(totalRows).isGreaterThan(0); - } - - // vortex.masked / vortex.patched / vortex.variant: IDs known, decoders not yet implemented. - // No S3 fixture exists in v0.72.0 — enable these tests once fixtures are published. - - @Disabled("no S3 fixture in v0.72.0") - @Test - void scan_maskedVortex_decodesAllRows() throws Exception { - // Given - URI uri = BASE.resolve("masked.vortex"); - - // When - long totalRows = 0; - try (var sut = VortexHttpReader.open(uri); - var iter = sut.scan(ScanOptions.all())) { - while (iter.hasNext()) { - totalRows += iter.next().rowCount(); - } - } - - // Then - assertThat(totalRows).isGreaterThan(0); - } - - @Disabled("no S3 fixture in v0.72.0") - @Test - void scan_patchedVortex_decodesAllRows() throws Exception { - // Given - URI uri = BASE.resolve("patched.vortex"); - - // When - long totalRows = 0; - try (var sut = VortexHttpReader.open(uri); - var iter = sut.scan(ScanOptions.all())) { - while (iter.hasNext()) { - totalRows += iter.next().rowCount(); - } - } - - // Then - assertThat(totalRows).isGreaterThan(0); - } - - @Disabled("no S3 fixture in v0.72.0") - @Test - void scan_variantVortex_decodesAllRows() throws Exception { - // Given - URI uri = BASE.resolve("variant.vortex"); - - // When - long totalRows = 0; - try (var sut = VortexHttpReader.open(uri); - var iter = sut.scan(ScanOptions.all())) { - while (iter.hasNext()) { - totalRows += iter.next().rowCount(); - } - } - - // Then - assertThat(totalRows).isGreaterThan(0); - } - - @Test - void scan_listVortex_decodesListArrayWithCorrectShape() throws Exception { - // Given - URI uri = BASE.resolve("list.vortex"); - - // When - try (var sut = VortexHttpReader.open(uri)) { - DType.Struct schema = (DType.Struct) sut.dtype(); - List names = schema.fieldNames(); - List types = schema.fieldTypes(); - String listColName = null; - for (int i = 0; i < types.size(); i++) { - if (types.get(i) instanceof DType.List) { - listColName = names.get(i); - break; - } - } - assertThat(listColName).as("list.vortex must have a List-typed column").isNotNull(); - - try (var iter = sut.scan(ScanOptions.all())) { - assertThat(iter.hasNext()).isTrue(); - var chunk = iter.next(); - - // Then - ListArray listArray = chunk.column(listColName); - assertThat(listArray.elements().length()) - .as("elements_len must be > 0; 0 indicates silent proto tag mismatch") - .isGreaterThan(0); - assertThat(listArray.offsets().dtype()) - .as("offset_ptype must be I64; U8 indicates silent proto tag mismatch") - .isEqualTo(new DType.Primitive(PType.I64, false)); - assertThat(listArray.offsets().length()) - .as("offsets length must be rowCount + 1") - .isEqualTo(chunk.rowCount() + 1); - } - } - } - - @Test - void scan_listviewVortex_decodesListViewArrayWithCorrectShape() throws Exception { - // Given - URI uri = BASE.resolve("listview.vortex"); - - // When - try (var sut = VortexHttpReader.open(uri)) { - DType.Struct schema = (DType.Struct) sut.dtype(); - List names = schema.fieldNames(); - List types = schema.fieldTypes(); - String listColName = null; - for (int i = 0; i < types.size(); i++) { - if (types.get(i) instanceof DType.List) { - listColName = names.get(i); - break; - } - } - assertThat(listColName).as("listview.vortex must have a List-typed column").isNotNull(); - - try (var iter = sut.scan(ScanOptions.all())) { - assertThat(iter.hasNext()).isTrue(); - var chunk = iter.next(); - - // Then - ListViewArray listViewArray = chunk.column(listColName); - assertThat(listViewArray.elements().length()) - .as("elements_len must be > 0; 0 indicates silent proto tag mismatch") - .isGreaterThan(0); - assertThat(listViewArray.offsets().dtype()) - .as("offset_ptype must be U32; U8 indicates silent proto tag mismatch") - .isEqualTo(new DType.Primitive(PType.U32, false)); - assertThat(listViewArray.sizes().dtype()) - .as("size_ptype must be U32; U8 indicates silent proto tag mismatch") - .isEqualTo(new DType.Primitive(PType.U32, false)); - assertThat(listViewArray.offsets().length()) - .as("offsets length must equal rowCount") - .isEqualTo(chunk.rowCount()); - assertThat(listViewArray.sizes().length()) - .as("sizes length must equal rowCount") - .isEqualTo(chunk.rowCount()); - } - } - } -} diff --git a/reader/src/test/java/io/github/dfa1/vortex/io/VortexReaderTest.java b/reader/src/test/java/io/github/dfa1/vortex/io/VortexReaderTest.java deleted file mode 100644 index 552331b..0000000 --- a/reader/src/test/java/io/github/dfa1/vortex/io/VortexReaderTest.java +++ /dev/null @@ -1,342 +0,0 @@ -package io.github.dfa1.vortex.io; - -import io.github.dfa1.vortex.core.array.Array; -import io.github.dfa1.vortex.core.DType; -import io.github.dfa1.vortex.core.VortexException; -import io.github.dfa1.vortex.core.array.EmptyArray; -import io.github.dfa1.vortex.core.array.UnknownArray; -import io.github.dfa1.vortex.encoding.Encoding; -import io.github.dfa1.vortex.encoding.EncodingId; -import io.github.dfa1.vortex.encoding.EncodingRegistry; -import io.github.dfa1.vortex.encoding.DecodeContext; -import io.github.dfa1.vortex.encoding.EncodeResult; -import io.github.dfa1.vortex.scan.ScanOptions; -import io.github.dfa1.vortex.scan.ScanResult; -import org.junit.jupiter.api.Test; -import org.junit.jupiter.api.io.TempDir; -import org.junit.jupiter.params.ParameterizedTest; -import org.junit.jupiter.params.provider.ValueSource; - -import java.io.IOException; -import java.net.URISyntaxException; -import java.nio.ByteOrder; -import java.nio.file.Files; -import java.nio.file.Path; - -import static org.assertj.core.api.Assertions.assertThat; -import static org.assertj.core.api.Assertions.assertThatThrownBy; - -class VortexReaderTest { - - // --- trailer / magic validation --- - - private static EncodingRegistry buildUniversalStubRegistry() { - var registry = EncodingRegistry.empty(); - Encoding stub = new Encoding() { - @Override - public EncodingId encodingId() { - return EncodingId.VORTEX_PRIMITIVE; - } - - @Override - public EncodeResult encode(DType dtype, Object data) { - throw new UnsupportedOperationException(); - } - - @Override - public Array decode(DecodeContext ctx) { - return EmptyArray.of(ctx.dtype()); - } - }; - for (EncodingId encodingId : EncodingId.values()) { - registry.register(new Encoding() { - @Override - public EncodingId encodingId() { - return encodingId; - } - - @Override - public EncodeResult encode(DType dtype, Object data) { - throw new UnsupportedOperationException(); - } - - @Override - public Array decode(DecodeContext ctx) { - return EmptyArray.of(ctx.dtype()); - } - }); - } - return registry; - } - - @Test - void open_fileTooSmall_throwsVortexException(@TempDir Path tmpDir) throws IOException { - // Given - Path sut = Files.write(tmpDir.resolve("tiny.vortex"), new byte[4]); - - // When / Then - assertThatThrownBy(() -> VortexReader.open(sut)) - .isInstanceOf(VortexException.class) - .hasMessageContaining("file too small"); - } - - // --- real fixtures: full parse --- - - @Test - void open_wrongMagic_throwsVortexException(@TempDir Path tmpDir) throws IOException { - // Given - byte[] bytes = new byte[VortexReader.TRAILER_SIZE]; // exactly 8 bytes - bytes[4] = 'X'; - bytes[5] = 'X'; - bytes[6] = 'X'; - bytes[7] = 'X'; - Path sut = Files.write(tmpDir.resolve("bad_magic.vortex"), bytes); - - // When / Then - assertThatThrownBy(() -> VortexReader.open(sut)) - .isInstanceOf(VortexException.class) - .hasMessageContaining("invalid magic bytes"); - } - - @ParameterizedTest - @ValueSource(strings = { - "primitives.vortex", - "booleans.vortex", - "null.vortex", - "varbin.vortex", - "chunked.vortex" - }) - void open_fixture_parsesSuccessfully(String name) throws URISyntaxException, IOException { - // Given - Path path = fixtureFile(name); - - // When - try (var sut = VortexReader.open(path)) { - - // Then - assertThat(sut.version()).isEqualTo(1); - assertThat(sut.fileSize()).isGreaterThan(VortexReader.TRAILER_SIZE); - assertThat(sut.layout()).isNotNull(); - assertThat(sut.footer()).isNotNull(); - } - } - - @ParameterizedTest - @ValueSource(strings = { - "primitives.vortex", - "booleans.vortex", - "null.vortex", - "varbin.vortex", - "chunked.vortex" - }) - void fixture_hasMagicBytesAtEnd(String name) throws IOException, URISyntaxException { - // Given - byte[] bytes = Files.readAllBytes(fixtureFile(name)); - - // When - byte[] trailerMagic = new byte[]{ - bytes[bytes.length - 4], - bytes[bytes.length - 3], - bytes[bytes.length - 2], - bytes[bytes.length - 1] - }; - - // Then - assertThat(trailerMagic).isEqualTo(VortexReader.MAGIC); - } - - // --- scan --- - - @ParameterizedTest - @ValueSource(strings = { - "primitives.vortex", - "booleans.vortex", - "null.vortex", - "varbin.vortex", - "chunked.vortex" - }) - void fixture_trailerHasExpectedVersion(String name) throws IOException, URISyntaxException { - // Given - byte[] bytes = Files.readAllBytes(fixtureFile(name)); - int trailerStart = bytes.length - VortexReader.TRAILER_SIZE; - - // When - int version = java.nio.ByteBuffer.wrap(bytes, trailerStart, 2) - .order(ByteOrder.LITTLE_ENDIAN) - .getShort() & 0xFFFF; - - // Then - assertThat(version).isEqualTo(1); - } - - @ParameterizedTest - @ValueSource(strings = { - "primitives.vortex", - "booleans.vortex", - "null.vortex", - "varbin.vortex", - "chunked.vortex" - }) - void scan_withNoDecoders_reachesDecodeStep(String name) throws URISyntaxException, IOException { - // Given - Path path = fixtureFile(name); - - // When / Then — layout traversal succeeds; decode fails only on missing decoder - try (var sut = VortexReader.open(path, EncodingRegistry.empty()); - var iter = sut.scan(ScanOptions.all())) { - assertThatThrownBy(iter::hasNext) - .isInstanceOf(VortexException.class) - .hasMessageContaining("no encoding registered"); - } - } - - @ParameterizedTest - @ValueSource(strings = { - "primitives.vortex", - "booleans.vortex", - "null.vortex", - "varbin.vortex", - "chunked.vortex" - }) - void scan_withNoDecoders_allowUnknown_returnsUnknownArray(String name) throws URISyntaxException, IOException { - // Given — empty registry + allowUnknown: every leaf decodes to a passthrough UnknownArray - Path path = fixtureFile(name); - var registry = EncodingRegistry.empty().allowUnknown(); - - // When - try (var sut = VortexReader.open(path, registry); - var iter = sut.scan(ScanOptions.all())) { - - // Then - assertThat(iter.hasNext()).isTrue(); - ScanResult chunk = iter.next(); - assertThat(chunk.rowCount()).isGreaterThan(0); - assertThat(chunk.columns()).isNotEmpty(); - for (Array column : chunk.columns().values()) { - assertThat(column).isInstanceOf(UnknownArray.class); - UnknownArray foreign = (UnknownArray) column; - assertThat(foreign.encodingId()).startsWith("vortex.").describedAs(name); - } - } - } - - @ParameterizedTest - @ValueSource(strings = { - "primitives.vortex", - "booleans.vortex", - "null.vortex", - "varbin.vortex", - "chunked.vortex" - }) - void scan_withStubDecoder_producesAtLeastOneChunk(String name) throws URISyntaxException, IOException { - // Given - Path path = fixtureFile(name); - var registry = buildUniversalStubRegistry(); - - // When - try (var sut = VortexReader.open(path, registry); - var iter = sut.scan(ScanOptions.all())) { - - // Then - assertThat(iter.hasNext()).isTrue(); - ScanResult chunk = iter.next(); - assertThat(chunk.rowCount()).isGreaterThan(0); - assertThat(chunk.columns()).isNotEmpty(); - } - } - - @Test - void scan_chunkedFixture_producesExactlyOneLayoutChunk() throws URISyntaxException, IOException { - // Given — chunked.vortex uses a ChunkedArray *encoding* inside one flat layout node - Path path = fixtureFile("chunked.vortex"); - var registry = buildUniversalStubRegistry(); - int chunkCount = 0; - - // When - try (var sut = VortexReader.open(path, registry); - var iter = sut.scan(ScanOptions.all())) { - while (iter.hasNext()) { - iter.next(); - chunkCount++; - } - } - - // Then - assertThat(chunkCount).isEqualTo(1); - } - - @Test - void scan_withLimit_returnsExactlyNRows() throws URISyntaxException, IOException { - // Given — primitives.vortex has 3 rows; limit=2 forces truncation of the single chunk - Path path = fixtureFile("primitives.vortex"); - long limit = 2; - - // When - long totalRows = 0; - try (var sut = VortexReader.open(path); - var iter = sut.scan(ScanOptions.limit(limit))) { - while (iter.hasNext()) { - ScanResult chunk = iter.next(); - totalRows += chunk.rowCount(); - for (Array col : chunk.columns().values()) { - assertThat(col.length()).isLessThanOrEqualTo(limit); - } - } - } - - // Then - assertThat(totalRows).isEqualTo(limit); - } - - @Test - void scan_withLimitExceedingTotal_returnsAllRows() throws URISyntaxException, IOException { - // Given - Path path = fixtureFile("primitives.vortex"); - long totalWithoutLimit; - try (var sut = VortexReader.open(path); - var iter = sut.scan(ScanOptions.all())) { - totalWithoutLimit = 0; - while (iter.hasNext()) { - totalWithoutLimit += iter.next().rowCount(); - } - } - - // When - long totalWithLimit = 0; - try (var sut = VortexReader.open(path); - var iter = sut.scan(ScanOptions.limit(Long.MAX_VALUE))) { - while (iter.hasNext()) { - totalWithLimit += iter.next().rowCount(); - } - } - - // Then - assertThat(totalWithLimit).isEqualTo(totalWithoutLimit); - } - - @Test - void scan_withLimitZero_returnsNoRows() throws URISyntaxException, IOException { - // Given - Path path = fixtureFile("primitives.vortex"); - - // When - long totalRows = 0; - try (var sut = VortexReader.open(path); - var iter = sut.scan(ScanOptions.limit(0))) { - while (iter.hasNext()) { - totalRows += iter.next().rowCount(); - } - } - - // Then - assertThat(totalRows).isZero(); - } - - // --- helpers --- - - private Path fixtureFile(String name) throws URISyntaxException { - var url = getClass().getResource("/fixtures/" + name); - assertThat(url).as("fixture not found: " + name).isNotNull(); - return Path.of(url.toURI()); - } -} diff --git a/reader/src/test/java/io/github/dfa1/vortex/scan/RowFilterTest.java b/reader/src/test/java/io/github/dfa1/vortex/scan/RowFilterTest.java deleted file mode 100644 index 1baa045..0000000 --- a/reader/src/test/java/io/github/dfa1/vortex/scan/RowFilterTest.java +++ /dev/null @@ -1,115 +0,0 @@ -package io.github.dfa1.vortex.scan; - -import org.junit.jupiter.api.Nested; -import org.junit.jupiter.api.Test; - -import java.util.List; - -import static org.assertj.core.api.Assertions.assertThat; - -class RowFilterTest { - - @Nested - class Factories { - @Test - void gt_createsGtRecord() { - // Given / When - RowFilter sut = RowFilter.gt("price", 100L); - - // Then - assertThat(sut).isEqualTo(new RowFilter.Gt("price", 100L)); - } - - @Test - void gte_createsGteRecord() { - // Given / When - RowFilter sut = RowFilter.gte("price", 100L); - - // Then - assertThat(sut).isEqualTo(new RowFilter.Gte("price", 100L)); - } - - @Test - void lt_createsLtRecord() { - // Given / When - RowFilter sut = RowFilter.lt("price", 500L); - - // Then - assertThat(sut).isEqualTo(new RowFilter.Lt("price", 500L)); - } - - @Test - void lte_createsLteRecord() { - // Given / When - RowFilter sut = RowFilter.lte("price", 500L); - - // Then - assertThat(sut).isEqualTo(new RowFilter.Lte("price", 500L)); - } - - @Test - void eq_createsEqRecord() { - // Given / When - RowFilter sut = RowFilter.eq("status", "open"); - - // Then - assertThat(sut).isEqualTo(new RowFilter.Eq("status", "open")); - } - - @Test - void neq_createsNeqRecord() { - // Given / When - RowFilter sut = RowFilter.neq("status", "closed"); - - // Then - assertThat(sut).isEqualTo(new RowFilter.Neq("status", "closed")); - } - } - - @Nested - class AndComposition { - @Test - void and_instanceMethod_combinesTwoFilters() { - // Given - RowFilter left = RowFilter.gte("price", 100L); - RowFilter right = RowFilter.lte("price", 500L); - - // When - RowFilter sut = left.and(right); - - // Then - assertThat(sut).isInstanceOf(RowFilter.And.class); - assertThat(((RowFilter.And) sut).filters()).isEqualTo(List.of(left, right)); - } - - @Test - void and_instanceMethod_chainsMultiple() { - // Given / When - RowFilter sut = RowFilter.gte("price", 10L) - .and(RowFilter.lte("price", 500L)) - .and(RowFilter.neq("status", "cancelled")); - - // Then — nested And(And(gte, lte), neq) - assertThat(sut).isInstanceOf(RowFilter.And.class); - RowFilter.And outer = (RowFilter.And) sut; - assertThat(outer.filters()).hasSize(2); - assertThat(outer.filters().get(0)).isInstanceOf(RowFilter.And.class); - assertThat(outer.filters().get(1)).isInstanceOf(RowFilter.Neq.class); - } - - @Test - void and_staticMethod_combinesMultiple() { - // Given - RowFilter a = RowFilter.gt("x", 0L); - RowFilter b = RowFilter.lt("x", 100L); - RowFilter c = RowFilter.neq("x", 50L); - - // When - RowFilter sut = RowFilter.and(a, b, c); - - // Then - assertThat(sut).isInstanceOf(RowFilter.And.class); - assertThat(((RowFilter.And) sut).filters()).isEqualTo(List.of(a, b, c)); - } - } -} diff --git a/reader/src/test/java/io/github/dfa1/vortex/scan/ScanOptionsTest.java b/reader/src/test/java/io/github/dfa1/vortex/scan/ScanOptionsTest.java deleted file mode 100644 index 6a6b88f..0000000 --- a/reader/src/test/java/io/github/dfa1/vortex/scan/ScanOptionsTest.java +++ /dev/null @@ -1,71 +0,0 @@ -package io.github.dfa1.vortex.scan; - -import org.junit.jupiter.api.Test; - -import static org.assertj.core.api.Assertions.assertThat; - -class ScanOptionsTest { - - @Test - void withLimit_setsLimit() { - // Given / When - ScanOptions sut = ScanOptions.all().withLimit(10); - - // Then - assertThat(sut.limit()).isEqualTo(10); - assertThat(sut.hasLimit()).isTrue(); - assertThat(sut.columns()).isEmpty(); - assertThat(sut.rowFilter()).isNull(); - } - - @Test - void withColumns_setsProjection() { - // Given / When - ScanOptions sut = ScanOptions.all().withColumns("price", "qty"); - - // Then - assertThat(sut.columns()).containsExactly("price", "qty"); - assertThat(sut.hasProjection()).isTrue(); - assertThat(sut.limit()).isEqualTo(ScanOptions.NO_LIMIT); - } - - @Test - void withFilter_setsFilter() { - // Given - RowFilter filter = new RowFilter.Gte("price", 100); - - // When - ScanOptions sut = ScanOptions.all().withFilter(filter); - - // Then - assertThat(sut.rowFilter()).isEqualTo(filter); - assertThat(sut.hasFilter()).isTrue(); - } - - @Test - void fluent_chainingPreservesAllFields() { - // Given / When - ScanOptions sut = ScanOptions.all() - .withColumns("price", "qty") - .withLimit(10) - .withFilter(new RowFilter.Gte("price", 50)); - - // Then - assertThat(sut.columns()).containsExactly("price", "qty"); - assertThat(sut.limit()).isEqualTo(10); - assertThat(sut.hasFilter()).isTrue(); - } - - @Test - void withMethods_doNotMutateOriginal() { - // Given - ScanOptions original = ScanOptions.all(); - - // When - original.withLimit(5).withColumns("x"); - - // Then - assertThat(original.limit()).isEqualTo(ScanOptions.NO_LIMIT); - assertThat(original.columns()).isEmpty(); - } -} diff --git a/reader/src/test/resources/fixtures/booleans.vortex b/reader/src/test/resources/fixtures/booleans.vortex deleted file mode 100644 index 72060d1..0000000 Binary files a/reader/src/test/resources/fixtures/booleans.vortex and /dev/null differ diff --git a/reader/src/test/resources/fixtures/chunked.vortex b/reader/src/test/resources/fixtures/chunked.vortex deleted file mode 100644 index 91a8ed5..0000000 Binary files a/reader/src/test/resources/fixtures/chunked.vortex and /dev/null differ diff --git a/reader/src/test/resources/fixtures/null.vortex b/reader/src/test/resources/fixtures/null.vortex deleted file mode 100644 index dfa85c4..0000000 Binary files a/reader/src/test/resources/fixtures/null.vortex and /dev/null differ diff --git a/reader/src/test/resources/fixtures/primitives.vortex b/reader/src/test/resources/fixtures/primitives.vortex deleted file mode 100644 index 1459171..0000000 Binary files a/reader/src/test/resources/fixtures/primitives.vortex and /dev/null differ diff --git a/reader/src/test/resources/fixtures/varbin.vortex b/reader/src/test/resources/fixtures/varbin.vortex deleted file mode 100644 index 26e5255..0000000 Binary files a/reader/src/test/resources/fixtures/varbin.vortex and /dev/null differ diff --git a/writer/pom.xml b/writer/pom.xml deleted file mode 100644 index ab3f2b6..0000000 --- a/writer/pom.xml +++ /dev/null @@ -1,47 +0,0 @@ - - - 4.0.0 - - io.github.dfa1.vortex - vortex-java - 0.3.0-SNAPSHOT - - - writer - - - - io.github.dfa1.vortex - core - - - com.google.flatbuffers - flatbuffers-java - - - com.google.protobuf - protobuf-java - - - - io.github.dfa1.vortex - reader - test - - - org.junit.jupiter - junit-jupiter - test - - - org.assertj - assertj-core - test - - - org.mockito - mockito-junit-jupiter - test - - - diff --git a/writer/src/main/java/io/github/dfa1/vortex/writer/VortexWriter.java b/writer/src/main/java/io/github/dfa1/vortex/writer/VortexWriter.java deleted file mode 100644 index 2984ace..0000000 --- a/writer/src/main/java/io/github/dfa1/vortex/writer/VortexWriter.java +++ /dev/null @@ -1,482 +0,0 @@ -package io.github.dfa1.vortex.writer; - -import com.google.flatbuffers.FlatBufferBuilder; -import io.github.dfa1.vortex.core.DType; -import io.github.dfa1.vortex.encoding.AlpEncoding; -import io.github.dfa1.vortex.encoding.BitpackedEncoding; -import io.github.dfa1.vortex.encoding.BoolEncoding; -import io.github.dfa1.vortex.encoding.CascadingCompressor; -import io.github.dfa1.vortex.encoding.CompressorContext; -import io.github.dfa1.vortex.encoding.ConstantEncoding; -import io.github.dfa1.vortex.encoding.DateTimePartsData; -import io.github.dfa1.vortex.encoding.DateTimePartsEncoding; -import io.github.dfa1.vortex.encoding.DictEncoding; -import io.github.dfa1.vortex.encoding.Encoding; -import io.github.dfa1.vortex.encoding.FrameOfReferenceEncoding; -import io.github.dfa1.vortex.encoding.ListData; -import io.github.dfa1.vortex.encoding.ListViewData; -import io.github.dfa1.vortex.encoding.VarBinEncoding; -import io.github.dfa1.vortex.encoding.EncodeNode; -import io.github.dfa1.vortex.encoding.EncodeResult; -import io.github.dfa1.vortex.encoding.EncodingId; -import io.github.dfa1.vortex.encoding.PrimitiveEncoding; -import io.github.dfa1.vortex.encoding.RleEncoding; -import io.github.dfa1.vortex.encoding.RunEndEncoding; -import io.github.dfa1.vortex.fbs.ArraySpec; -import io.github.dfa1.vortex.fbs.Extension; -import io.github.dfa1.vortex.fbs.Footer; -import io.github.dfa1.vortex.fbs.Layout; -import io.github.dfa1.vortex.fbs.LayoutSpec; -import io.github.dfa1.vortex.fbs.Postscript; -import io.github.dfa1.vortex.fbs.PostscriptSegment; -import io.github.dfa1.vortex.fbs.SegmentSpec; -import io.github.dfa1.vortex.fbs.Type; - -import java.io.Closeable; -import java.io.IOException; -import java.lang.foreign.MemorySegment; -import java.nio.ByteBuffer; -import java.nio.ByteOrder; -import java.nio.channels.WritableByteChannel; -import java.util.ArrayList; -import java.util.LinkedHashMap; -import java.util.List; -import java.util.Map; - -/// Writes a Vortex file. -/// -/// Usage: -/// ```java -/// var schema = new DType.Struct(List.of("id", "value"), -/// List.of(new DType.Primitive(PType.I64, false), -/// new DType.Primitive(PType.F64, false)), false); -/// try (var channel = FileChannel.open(path, CREATE, WRITE); -/// var writer = VortexWriter.create(channel, schema, WriteOptions.defaults())) { -/// writer.writeChunk(Map.of("id", idArray, "value", valueArray)); -/// } -/// ``` -public final class VortexWriter implements Closeable { - - private static final int VERSION = 1; - - // Indices into layout_specs list in the Footer - private static final int LAYOUT_FLAT = 0; - private static final int LAYOUT_CHUNKED = 1; - private static final int LAYOUT_STRUCT = 2; - - private static final ByteBuffer MAGIC = ByteBuffer.wrap(new byte[]{'V', 'T', 'X', 'F'}) - .asReadOnlyBuffer(); - - private static final List DEFAULT_CODECS = List.of( - new AlpEncoding(), new PrimitiveEncoding(), new BoolEncoding(), new DictEncoding(), new VarBinEncoding()); - - // ZstdEncoding is intentionally absent from this list. - // On NYC Taxi 2024-01: Zstd wins compression for F64 columns (50 MB → 43 MB) but beats - // ALP and kills decode throughput by 6× (240 → 40 ops/s). ZSTD decompression cannot - // compete with ALP reconstruction or bitpack unpack for numeric columns. - // Use ZstdEncoding only for Utf8/Binary where no faster structural encoding exists. - private static final List CASCADE_CODECS = List.of( - new DateTimePartsEncoding(), - new ConstantEncoding(), - new AlpEncoding(), new FrameOfReferenceEncoding(), new RunEndEncoding(), new RleEncoding(), - new DictEncoding(), new BitpackedEncoding(), - new VarBinEncoding(), new PrimitiveEncoding(), new BoolEncoding()); - - private final WritableByteChannel channel; - private final DType.Struct schema; - private final WriteOptions options; - private final List encodings; - private final List segs = new ArrayList<>(); - private final Map> colChunks = new LinkedHashMap<>(); - private final Map encodingIdx = new LinkedHashMap<>(); - private long bytesWritten = 0; - - private VortexWriter( - WritableByteChannel channel, DType.Struct schema, WriteOptions options, List encodings - ) { - this.channel = channel; - this.schema = schema; - this.options = options; - this.encodings = encodings; - for (String name : schema.fieldNames()) { - colChunks.put(name, new ArrayList<>()); - } - } - - public static VortexWriter create( - WritableByteChannel channel, DType.Struct schema, WriteOptions options - ) { - return new VortexWriter(channel, schema, options, DEFAULT_CODECS); - } - - public static VortexWriter create( - WritableByteChannel channel, DType.Struct schema, WriteOptions options, List encodings - ) { - return new VortexWriter(channel, schema, options, encodings); - } - - private static long arrayLength(Object data) { - return switch (data) { - case byte[] a -> a.length; - case short[] a -> a.length; - case int[] a -> a.length; - case long[] a -> a.length; - case float[] a -> a.length; - case double[] a -> a.length; - case boolean[] a -> a.length; - case String[] a -> a.length; - case ListData d -> d.outerLen(); - case ListViewData d -> d.outerLen(); - case DateTimePartsData d -> d.timestamps().length; - default -> throw new UnsupportedOperationException( - "unsupported data type: " + data.getClass()); - }; - } - - private static ByteBuffer buildDType(DType dtype) { - var fbb = new FlatBufferBuilder(128); - int off = serializeDType(fbb, dtype); - io.github.dfa1.vortex.fbs.DType.finishDTypeBuffer(fbb, off); - return fbb.dataBuffer().slice(fbb.dataBuffer().position(), fbb.dataBuffer().remaining()); - } - - private static int serializeDType(FlatBufferBuilder fbb, DType dtype) { - return switch (dtype) { - case DType.Null __ -> { - io.github.dfa1.vortex.fbs.Null.startNull(fbb); - int inner = io.github.dfa1.vortex.fbs.Null.endNull(fbb); - yield io.github.dfa1.vortex.fbs.DType.createDType(fbb, Type.Null, inner); - } - case DType.Bool b -> { - int inner = io.github.dfa1.vortex.fbs.Bool.createBool(fbb, b.nullable()); - yield io.github.dfa1.vortex.fbs.DType.createDType(fbb, Type.Bool, inner); - } - case DType.Primitive p -> { - int inner = io.github.dfa1.vortex.fbs.Primitive.createPrimitive( - fbb, p.ptype().ordinal(), p.nullable()); - yield io.github.dfa1.vortex.fbs.DType.createDType(fbb, Type.Primitive, inner); - } - case DType.Struct s -> { - // Build child DType tables first (FlatBuffers bottom-up requirement) - int[] fieldOffsets = new int[s.fieldTypes().size()]; - for (int i = 0; i < fieldOffsets.length; i++) { - fieldOffsets[i] = serializeDType(fbb, s.fieldTypes().get(i)); - } - int[] nameOffsets = new int[s.fieldNames().size()]; - for (int i = 0; i < nameOffsets.length; i++) { - nameOffsets[i] = fbb.createString(s.fieldNames().get(i)); - } - int namesVec = io.github.dfa1.vortex.fbs.Struct_.createNamesVector(fbb, nameOffsets); - int dtypesVec = io.github.dfa1.vortex.fbs.Struct_.createDtypesVector(fbb, fieldOffsets); - int inner = io.github.dfa1.vortex.fbs.Struct_.createStruct_( - fbb, namesVec, dtypesVec, s.nullable()); - yield io.github.dfa1.vortex.fbs.DType.createDType(fbb, Type.Struct_, inner); - } - case DType.Utf8 u -> { - int inner = io.github.dfa1.vortex.fbs.Utf8.createUtf8(fbb, u.nullable()); - yield io.github.dfa1.vortex.fbs.DType.createDType(fbb, Type.Utf8, inner); - } - case DType.List l -> { - int elemTypeOff = serializeDType(fbb, l.elementType()); - int inner = io.github.dfa1.vortex.fbs.List.createList(fbb, elemTypeOff, l.nullable()); - yield io.github.dfa1.vortex.fbs.DType.createDType(fbb, Type.List, inner); - } - case DType.Extension e -> { - int idOff = fbb.createString(e.extensionId()); - int storageDtypeOff = serializeDType(fbb, e.storageDType()); - int metaOff = 0; - if (e.metadata() != null && e.metadata().remaining() > 0) { - byte[] metaBytes = new byte[e.metadata().remaining()]; - e.metadata().duplicate().get(metaBytes); - metaOff = Extension.createMetadataVector(fbb, metaBytes); - } - int inner = Extension.createExtension(fbb, idOff, storageDtypeOff, metaOff); - yield io.github.dfa1.vortex.fbs.DType.createDType(fbb, Type.Extension, inner); - } - default -> throw new UnsupportedOperationException("unsupported DType: " + dtype); - }; - } - - private static ByteBuffer buildPostscript( - long footerOff, int footerLen, - long dtypeOff, int dtypeLen, - long layoutOff, int layoutLen - ) { - var fbb = new FlatBufferBuilder(256); - - int footerSegOff = PostscriptSegment.createPostscriptSegment( - fbb, footerOff, footerLen, 0, 0, 0); - int dtypeSegOff = PostscriptSegment.createPostscriptSegment( - fbb, dtypeOff, dtypeLen, 0, 0, 0); - int layoutSegOff = PostscriptSegment.createPostscriptSegment( - fbb, layoutOff, layoutLen, 0, 0, 0); - - int psOff = Postscript.createPostscript(fbb, dtypeSegOff, layoutSegOff, 0, footerSegOff); - Postscript.finishPostscriptBuffer(fbb, psOff); - return fbb.dataBuffer().slice(fbb.dataBuffer().position(), fbb.dataBuffer().remaining()); - } - - // ── Segment encoding ───────────────────────────────────────────────────── - - /// Write one chunk. Each column is encoded by the first registered [Encoding] that accepts its dtype. - public void writeChunk(Map columns) throws IOException { - for (int i = 0; i < schema.fieldNames().size(); i++) { - String colName = schema.fieldNames().get(i); - DType colDtype = schema.fieldTypes().get(i); - Object data = columns.get(colName); - if (data == null) { - throw new IllegalArgumentException("missing column: " + colName); - } - long rowCount = arrayLength(data); - int segIdx = writeSegment(colDtype, data); - colChunks.get(colName).add(new ChunkRef(segIdx, rowCount)); - } - } - - @Override - public void close() throws IOException { - ByteBuffer footerBuf = buildFooter(); - long footerOff = bytesWritten; - write(footerBuf); - - ByteBuffer dtypeBuf = buildDType(schema); - long dtypeOff = bytesWritten; - write(dtypeBuf); - - ByteBuffer layoutBuf = buildLayout(); - long layoutOff = bytesWritten; - write(layoutBuf); - - ByteBuffer psBuf = buildPostscript( - footerOff, footerBuf.capacity(), - dtypeOff, dtypeBuf.capacity(), - layoutOff, layoutBuf.capacity()); - write(psBuf); - - // 8-byte trailer: version(u16 LE) | postscriptLen(u16 LE) | magic(4) - var trailer = ByteBuffer.allocate(8).order(ByteOrder.LITTLE_ENDIAN); - trailer.putShort((short) VERSION); - trailer.putShort((short) psBuf.capacity()); - trailer.put(MAGIC.duplicate()); - trailer.flip(); - channel.write(trailer); - } - - private int writeSegment(DType dtype, Object data) throws IOException { - EncodeResult result; - if (options.allowedCascading() > 0) { - CompressorContext ctx = CompressorContext.ofDepth(options.allowedCascading()); - CascadingCompressor compressor = new CascadingCompressor(CASCADE_CODECS, ctx); - result = compressor.encode(dtype, data); - } else { - Encoding encoding = findEncoding(dtype); - result = encoding.encode(dtype, data); - } - - // Register all encoding IDs found in the node tree - registerEncodingIds(result.rootNode()); - - // Align segment start to 64 bytes so each buffer is Arrow-compatible - long prePad = (64 - bytesWritten % 64) % 64; - if (prePad > 0) { - writePadding((int) prePad); - } - - int segIdx = segs.size(); - long offset = bytesWritten; - - ByteBuffer fbBuf = buildArrayFlatBuffer(result); - - // Segment format: [buffer data...] [FlatBuffer Array bytes] [4-byte LE u32 = fbLen] - int fbLen = fbBuf.remaining(); - for (MemorySegment seg : result.buffers()) { - write(seg); - } - write(fbBuf); - var sizeBuf = ByteBuffer.allocate(4).order(ByteOrder.LITTLE_ENDIAN).putInt(fbLen); - sizeBuf.flip(); - channel.write(sizeBuf); - bytesWritten += 4; - - segs.add(new SegRef(offset, bytesWritten - offset)); - return segIdx; - } - - private void registerEncodingIds(EncodeNode node) { - encodingIdx.computeIfAbsent(node.encodingId(), k -> encodingIdx.size()); - for (EncodeNode child : node.children()) { - registerEncodingIds(child); - } - } - - private Encoding findEncoding(DType dtype) { - for (Encoding c : encodings) { - if (c.accepts(dtype)) { - return c; - } - } - throw new UnsupportedOperationException("no encoding for dtype: " + dtype); - } - - private void write(MemorySegment seg) throws IOException { - ByteBuffer buf = seg.asByteBuffer(); - while (buf.hasRemaining()) { - channel.write(buf); - } - bytesWritten += seg.byteSize(); - } - - private void write(ByteBuffer buf) throws IOException { - buf.rewind(); - while (buf.hasRemaining()) { - channel.write(buf); - } - bytesWritten += buf.capacity(); - } - - private void writePadding(int n) throws IOException { - ByteBuffer pad = ByteBuffer.allocate(n); - while (pad.hasRemaining()) { - channel.write(pad); - } - bytesWritten += n; - } - - private ByteBuffer buildArrayFlatBuffer(EncodeResult result) { - var fbb = new FlatBufferBuilder(256); - - // Stats for root node only (build before root ArrayNode) - int statsOff = 0; - if (result.hasStats()) { - int minVec = io.github.dfa1.vortex.fbs.ArrayStats.createMinVector(fbb, result.statsMin()); - int maxVec = io.github.dfa1.vortex.fbs.ArrayStats.createMaxVector(fbb, result.statsMax()); - io.github.dfa1.vortex.fbs.ArrayStats.startArrayStats(fbb); - io.github.dfa1.vortex.fbs.ArrayStats.addMin(fbb, minVec); - io.github.dfa1.vortex.fbs.ArrayStats.addMax(fbb, maxVec); - statsOff = io.github.dfa1.vortex.fbs.ArrayStats.endArrayStats(fbb); - } - - int rootNodeOff = buildArrayNodeFlatBuffer(fbb, result.rootNode(), statsOff); - - // Buffer struct vector — one entry per buffer in result. - // Layout (LE): padding(u16) | alignment_exponent(u8) | compression(u8) | length(u32) - // FlatBuffers builds backward: iterate in reverse. - var bufs = result.buffers(); - io.github.dfa1.vortex.fbs.Array.startBuffersVector(fbb, bufs.size()); - for (int i = bufs.size() - 1; i >= 0; i--) { - fbb.prep(4, 8); - fbb.putInt((int) bufs.get(i).byteSize()); - fbb.putByte((byte) 0); // compression = None - fbb.putByte((byte) 6); // alignment_exponent = 6 (64-byte alignment) - fbb.putShort((short) 0); // padding = 0 - } - int bufVec = fbb.endVector(); - - int arrayOff = io.github.dfa1.vortex.fbs.Array.createArray(fbb, rootNodeOff, bufVec); - io.github.dfa1.vortex.fbs.Array.finishArrayBuffer(fbb, arrayOff); - return fbb.dataBuffer().slice(fbb.dataBuffer().position(), fbb.dataBuffer().remaining()); - } - - // ── Footer / metadata serialization ────────────────────────────────────── - - private int buildArrayNodeFlatBuffer(FlatBufferBuilder fbb, EncodeNode node, int statsOff) { - // Build children first (FlatBuffers bottom-up: nested objects before parent table) - int[] childOffsets = new int[node.children().length]; - for (int i = 0; i < childOffsets.length; i++) { - childOffsets[i] = buildArrayNodeFlatBuffer(fbb, node.children()[i], 0); - } - - int metaOff = 0; - if (node.metadata() != null && node.metadata().hasRemaining()) { - byte[] metaBytes = new byte[node.metadata().remaining()]; - node.metadata().duplicate().get(metaBytes); - metaOff = io.github.dfa1.vortex.fbs.ArrayNode.createMetadataVector(fbb, metaBytes); - } - - int childVec = 0; - if (childOffsets.length > 0) { - childVec = io.github.dfa1.vortex.fbs.ArrayNode.createChildrenVector(fbb, childOffsets); - } - - int bufIdxVec = io.github.dfa1.vortex.fbs.ArrayNode.createBuffersVector(fbb, node.bufferIndices()); - int encIdx = encodingIdx.get(node.encodingId()); - return io.github.dfa1.vortex.fbs.ArrayNode.createArrayNode( - fbb, encIdx, metaOff, childVec, bufIdxVec, statsOff); - } - - private ByteBuffer buildFooter() { - var fbb = new FlatBufferBuilder(512); - - // array_specs: all encoding IDs used across all written segments, in registration order - EncodingId[] encIds = encodingIdx.entrySet().stream() - .sorted(Map.Entry.comparingByValue()) - .map(Map.Entry::getKey) - .toArray(EncodingId[]::new); - int[] asOffsets = new int[encIds.length]; - for (int i = 0; i < encIds.length; i++) { - asOffsets[i] = ArraySpec.createArraySpec(fbb, fbb.createString(encIds[i].id())); - } - int asv = Footer.createArraySpecsVector(fbb, asOffsets); - - // layout_specs: ["vortex.flat", "vortex.chunked", "vortex.struct"] - int ls0 = LayoutSpec.createLayoutSpec(fbb, fbb.createString("vortex.flat")); - int ls1 = LayoutSpec.createLayoutSpec(fbb, fbb.createString("vortex.chunked")); - int ls2 = LayoutSpec.createLayoutSpec(fbb, fbb.createString("vortex.struct")); - int lsv = Footer.createLayoutSpecsVector(fbb, new int[]{ls0, ls1, ls2}); - - // segment_specs (inline struct vector — write in reverse order) - Footer.startSegmentSpecsVector(fbb, segs.size()); - for (int i = segs.size() - 1; i >= 0; i--) { - SegRef s = segs.get(i); - SegmentSpec.createSegmentSpec(fbb, s.offset(), s.len(), 6, 0, 0); - } - int ssv = fbb.endVector(); - - int off = Footer.createFooter(fbb, asv, lsv, ssv, 0, 0); - fbb.finish(off); - return fbb.dataBuffer().slice(fbb.dataBuffer().position(), fbb.dataBuffer().remaining()); - } - - private ByteBuffer buildLayout() { - var fbb = new FlatBufferBuilder(256); - int colCount = schema.fieldNames().size(); - - // Pass 1: build all Flat layout nodes (children must precede parents in FlatBuffers) - int[][] flatsByCol = new int[colCount][]; - long[] colRowCounts = new long[colCount]; - for (int c = 0; c < colCount; c++) { - String colName = schema.fieldNames().get(c); - List chunks = colChunks.get(colName); - int[] flats = new int[chunks.size()]; - long colRows = 0; - for (int i = 0; i < chunks.size(); i++) { - ChunkRef cr = chunks.get(i); - int segV = Layout.createSegmentsVector(fbb, new long[]{cr.segIdx()}); - flats[i] = Layout.createLayout(fbb, LAYOUT_FLAT, cr.rowCount(), 0, 0, segV); - colRows += cr.rowCount(); - } - flatsByCol[c] = flats; - colRowCounts[c] = colRows; - } - - // Pass 2: build Chunked layout per column - int[] colLayouts = new int[colCount]; - long totalRows = colCount > 0 ? colRowCounts[0] : 0; - for (int c = 0; c < colCount; c++) { - int childV = Layout.createChildrenVector(fbb, flatsByCol[c]); - colLayouts[c] = Layout.createLayout(fbb, LAYOUT_CHUNKED, colRowCounts[c], 0, childV, 0); - } - - // Pass 3: Struct root - int rootChildV = Layout.createChildrenVector(fbb, colLayouts); - int rootLayout = Layout.createLayout(fbb, LAYOUT_STRUCT, totalRows, 0, rootChildV, 0); - Layout.finishLayoutBuffer(fbb, rootLayout); - return fbb.dataBuffer().slice(fbb.dataBuffer().position(), fbb.dataBuffer().remaining()); - } - - private record SegRef(long offset, long len) { - } - - private record ChunkRef(int segIdx, long rowCount) { - } -} diff --git a/writer/src/main/java/io/github/dfa1/vortex/writer/WriteOptions.java b/writer/src/main/java/io/github/dfa1/vortex/writer/WriteOptions.java deleted file mode 100644 index 65b609c..0000000 --- a/writer/src/main/java/io/github/dfa1/vortex/writer/WriteOptions.java +++ /dev/null @@ -1,19 +0,0 @@ -package io.github.dfa1.vortex.writer; - -/// Tuning knobs for the Vortex writer. -public record WriteOptions( - int chunkSize, - boolean enableZoneMaps, - double compressionRatioThreshold, - int allowedCascading -) { - public static WriteOptions defaults() { - return new WriteOptions(65_536, true, 0.90, 0); - } - - /// Enable cascading compression with up to {@code depth} recursive levels. - /// Depth 0 preserves current first-match behaviour. - public static WriteOptions cascading(int depth) { - return new WriteOptions(65_536, true, 0.90, depth); - } -} diff --git a/writer/src/test/java/io/github/dfa1/vortex/writer/BitpackedEncodingTest.java b/writer/src/test/java/io/github/dfa1/vortex/writer/BitpackedEncodingTest.java deleted file mode 100644 index fc69291..0000000 --- a/writer/src/test/java/io/github/dfa1/vortex/writer/BitpackedEncodingTest.java +++ /dev/null @@ -1,196 +0,0 @@ -package io.github.dfa1.vortex.writer; - -import io.github.dfa1.vortex.core.array.Array; -import io.github.dfa1.vortex.core.DType; -import io.github.dfa1.vortex.core.PType; -import io.github.dfa1.vortex.encoding.BitpackedEncoding; -import io.github.dfa1.vortex.encoding.EncodingRegistry; -import io.github.dfa1.vortex.io.VortexReader; -import io.github.dfa1.vortex.scan.ScanOptions; -import io.github.dfa1.vortex.scan.ScanResult; -import org.junit.jupiter.api.Test; -import org.junit.jupiter.api.io.TempDir; -import org.junit.jupiter.params.ParameterizedTest; -import org.junit.jupiter.params.provider.ValueSource; - -import java.io.IOException; -import java.lang.foreign.ValueLayout; -import java.nio.ByteOrder; -import java.nio.channels.FileChannel; -import java.nio.file.Path; -import java.nio.file.StandardOpenOption; -import java.util.ArrayList; -import java.util.List; -import java.util.Map; - -import static org.assertj.core.api.Assertions.assertThat; - -class BitpackedEncodingTest { - - private static final DType.Struct I32_SCHEMA = new DType.Struct( - List.of("value"), - List.of(new DType.Primitive(PType.I32, false)), - false); - - private static List scanAll(VortexReader vf) throws IOException { - var results = new ArrayList(); - var iter = vf.scan(ScanOptions.all()); - while (iter.hasNext()) { - results.add(iter.next()); - } - return results; - } - - private static EncodingRegistry bpRegistry() { - var registry = EncodingRegistry.empty(); - registry.register(new BitpackedEncoding()); - return registry; - } - - @Test - void roundTrip_positiveIntegers(@TempDir Path tmp) throws IOException { - // Given - Path file = tmp.resolve("bp.vtx"); - int[] data = {0, 1, 2, 3, 4, 5, 6, 7}; - - try (var ch = FileChannel.open(file, StandardOpenOption.CREATE, StandardOpenOption.WRITE); - var sut = VortexWriter.create(ch, I32_SCHEMA, WriteOptions.defaults(), - List.of(new BitpackedEncoding()))) { - // When - sut.writeChunk(Map.of("value", data)); - } - - // Then - try (var vf = VortexReader.open(file, bpRegistry())) { - List results = scanAll(vf); - assertThat(results).hasSize(1); - Array arr = results.getFirst().columns().get("value"); - assertThat(arr.length()).isEqualTo(8L); - var layout = ValueLayout.JAVA_INT_UNALIGNED.withOrder(ByteOrder.LITTLE_ENDIAN); - for (int i = 0; i < data.length; i++) { - assertThat(arr.buffer(0).get(layout, (long) i * 4)).isEqualTo(data[i]); - } - } - } - - @Test - void roundTrip_allSameValue(@TempDir Path tmp) throws IOException { - // Given — all identical values - Path file = tmp.resolve("bp_same.vtx"); - int[] data = {42, 42, 42, 42}; - - try (var ch = FileChannel.open(file, StandardOpenOption.CREATE, StandardOpenOption.WRITE); - var sut = VortexWriter.create(ch, I32_SCHEMA, WriteOptions.defaults(), - List.of(new BitpackedEncoding()))) { - // When - sut.writeChunk(Map.of("value", data)); - } - - // Then - try (var vf = VortexReader.open(file, bpRegistry())) { - List results = scanAll(vf); - assertThat(results).hasSize(1); - Array arr = results.getFirst().columns().get("value"); - var layout = ValueLayout.JAVA_INT_UNALIGNED.withOrder(ByteOrder.LITTLE_ENDIAN); - for (int i = 0; i < data.length; i++) { - assertThat(arr.buffer(0).get(layout, (long) i * 4)).isEqualTo(42); - } - } - } - - @Test - void roundTrip_negativeIntegers(@TempDir Path tmp) throws IOException { - // Given — frame-of-reference shifts negative values - Path file = tmp.resolve("bp_neg.vtx"); - int[] data = {-10, -5, 0, 5, 10}; - - try (var ch = FileChannel.open(file, StandardOpenOption.CREATE, StandardOpenOption.WRITE); - var sut = VortexWriter.create(ch, I32_SCHEMA, WriteOptions.defaults(), - List.of(new BitpackedEncoding()))) { - // When - sut.writeChunk(Map.of("value", data)); - } - - // Then - try (var vf = VortexReader.open(file, bpRegistry())) { - List results = scanAll(vf); - assertThat(results).hasSize(1); - Array arr = results.getFirst().columns().get("value"); - var layout = ValueLayout.JAVA_INT_UNALIGNED.withOrder(ByteOrder.LITTLE_ENDIAN); - for (int i = 0; i < data.length; i++) { - assertThat(arr.buffer(0).get(layout, (long) i * 4)).isEqualTo(data[i]); - } - } - } - - // ── Helpers ─────────────────────────────────────────────────────────────── - - @Test - void roundTrip_multipleChunks(@TempDir Path tmp) throws IOException { - // Given - Path file = tmp.resolve("bp_multi.vtx"); - int[] chunk1 = {100, 200, 150}; - int[] chunk2 = {300, 400, 350}; - - try (var ch = FileChannel.open(file, StandardOpenOption.CREATE, StandardOpenOption.WRITE); - var sut = VortexWriter.create(ch, I32_SCHEMA, WriteOptions.defaults(), - List.of(new BitpackedEncoding()))) { - // When - sut.writeChunk(Map.of("value", chunk1)); - sut.writeChunk(Map.of("value", chunk2)); - } - - // Then — process each chunk before advancing; hasNext() closes the previous chunk's arena - try (var vf = VortexReader.open(file, bpRegistry()); - var iter = vf.scan(ScanOptions.all())) { - var layout = ValueLayout.JAVA_INT_UNALIGNED.withOrder(ByteOrder.LITTLE_ENDIAN); - - assertThat(iter.hasNext()).isTrue(); - Array a1 = iter.next().columns().get("value"); - assertThat(a1.length()).isEqualTo(3L); - for (int i = 0; i < chunk1.length; i++) { - assertThat(a1.buffer(0).get(layout, (long) i * 4)).isEqualTo(chunk1[i]); - } - - assertThat(iter.hasNext()).isTrue(); - Array a2 = iter.next().columns().get("value"); - assertThat(a2.length()).isEqualTo(3L); - for (int i = 0; i < chunk2.length; i++) { - assertThat(a2.buffer(0).get(layout, (long) i * 4)).isEqualTo(chunk2[i]); - } - - assertThat(iter.hasNext()).isFalse(); - } - } - - @ParameterizedTest - @ValueSource(ints = {1, 2, 4, 7, 8, 15, 16, 31}) - void roundTrip_bitWidths(int maxVal, @TempDir Path tmp) throws IOException { - // Given — verify correct round-trip for various bit widths - Path file = tmp.resolve("bp_bw_" + maxVal + ".vtx"); - int[] data = new int[maxVal + 1]; - for (int i = 0; i <= maxVal; i++) { - data[i] = i; - } - - try (var ch = FileChannel.open(file, StandardOpenOption.CREATE, StandardOpenOption.WRITE); - var sut = VortexWriter.create(ch, I32_SCHEMA, WriteOptions.defaults(), - List.of(new BitpackedEncoding()))) { - // When - sut.writeChunk(Map.of("value", data)); - } - - // Then - try (var vf = VortexReader.open(file, bpRegistry())) { - List results = scanAll(vf); - assertThat(results).hasSize(1); - Array arr = results.getFirst().columns().get("value"); - var layout = ValueLayout.JAVA_INT_UNALIGNED.withOrder(ByteOrder.LITTLE_ENDIAN); - for (int i = 0; i < data.length; i++) { - assertThat(arr.buffer(0).get(layout, (long) i * 4)) - .as("value at index %d", i) - .isEqualTo(data[i]); - } - } - } -} diff --git a/writer/src/test/java/io/github/dfa1/vortex/writer/DeltaEncodingTest.java b/writer/src/test/java/io/github/dfa1/vortex/writer/DeltaEncodingTest.java deleted file mode 100644 index c0000ee..0000000 --- a/writer/src/test/java/io/github/dfa1/vortex/writer/DeltaEncodingTest.java +++ /dev/null @@ -1,241 +0,0 @@ -package io.github.dfa1.vortex.writer; - -import io.github.dfa1.vortex.core.array.Array; -import io.github.dfa1.vortex.core.DType; -import io.github.dfa1.vortex.core.PType; -import io.github.dfa1.vortex.encoding.EncodingRegistry; -import io.github.dfa1.vortex.encoding.DeltaEncoding; -import io.github.dfa1.vortex.io.VortexReader; -import io.github.dfa1.vortex.scan.ScanOptions; -import io.github.dfa1.vortex.scan.ScanResult; -import org.junit.jupiter.api.Test; -import org.junit.jupiter.api.io.TempDir; -import org.junit.jupiter.params.ParameterizedTest; -import org.junit.jupiter.params.provider.ValueSource; - -import java.io.IOException; -import java.lang.foreign.ValueLayout; -import java.nio.ByteOrder; -import java.nio.channels.FileChannel; -import java.nio.file.Path; -import java.nio.file.StandardOpenOption; -import java.util.ArrayList; -import java.util.List; -import java.util.Map; - -import static org.assertj.core.api.Assertions.assertThat; - -class DeltaEncodingTest { - - private static final DType.Struct I64_SCHEMA = new DType.Struct( - List.of("ts"), - List.of(new DType.Primitive(PType.I64, false)), - false); - - private static List scanAll(VortexReader vf) throws IOException { - var results = new ArrayList(); - var iter = vf.scan(ScanOptions.all()); - while (iter.hasNext()) { - results.add(iter.next()); - } - return results; - } - - private static EncodingRegistry deltaRegistry() { - var registry = EncodingRegistry.empty(); - registry.register(new DeltaEncoding()); - registry.register(new io.github.dfa1.vortex.encoding.PrimitiveEncoding()); - return registry; - } - - @Test - void roundTrip_monotonicIncreasing(@TempDir Path tmp) throws IOException { - // Given - Path file = tmp.resolve("delta_inc.vtx"); - long[] data = {100L, 105L, 110L, 115L, 120L}; - - try (var ch = FileChannel.open(file, StandardOpenOption.CREATE, StandardOpenOption.WRITE); - var sut = VortexWriter.create(ch, I64_SCHEMA, WriteOptions.defaults(), - List.of(new DeltaEncoding()))) { - // When - sut.writeChunk(Map.of("ts", data)); - } - - // Then - try (var vf = VortexReader.open(file, deltaRegistry())) { - List results = scanAll(vf); - assertThat(results).hasSize(1); - Array arr = results.getFirst().columns().get("ts"); - assertThat(arr.length()).isEqualTo(5L); - var layout = ValueLayout.JAVA_LONG_UNALIGNED.withOrder(ByteOrder.LITTLE_ENDIAN); - for (int i = 0; i < data.length; i++) { - assertThat(arr.buffer(0).get(layout, (long) i * 8)).isEqualTo(data[i]); - } - } - } - - @Test - void roundTrip_monotonicDecreasing(@TempDir Path tmp) throws IOException { - // Given - Path file = tmp.resolve("delta_dec.vtx"); - long[] data = {1000L, 990L, 975L, 950L}; - - try (var ch = FileChannel.open(file, StandardOpenOption.CREATE, StandardOpenOption.WRITE); - var sut = VortexWriter.create(ch, I64_SCHEMA, WriteOptions.defaults(), - List.of(new DeltaEncoding()))) { - // When - sut.writeChunk(Map.of("ts", data)); - } - - // Then - try (var vf = VortexReader.open(file, deltaRegistry())) { - List results = scanAll(vf); - Array arr = results.getFirst().columns().get("ts"); - var layout = ValueLayout.JAVA_LONG_UNALIGNED.withOrder(ByteOrder.LITTLE_ENDIAN); - for (int i = 0; i < data.length; i++) { - assertThat(arr.buffer(0).get(layout, (long) i * 8)).isEqualTo(data[i]); - } - } - } - - @Test - void roundTrip_allSameValue(@TempDir Path tmp) throws IOException { - // Given — deltas all zero, bit_width=0 - Path file = tmp.resolve("delta_same.vtx"); - long[] data = {42L, 42L, 42L, 42L}; - - try (var ch = FileChannel.open(file, StandardOpenOption.CREATE, StandardOpenOption.WRITE); - var sut = VortexWriter.create(ch, I64_SCHEMA, WriteOptions.defaults(), - List.of(new DeltaEncoding()))) { - // When - sut.writeChunk(Map.of("ts", data)); - } - - // Then - try (var vf = VortexReader.open(file, deltaRegistry())) { - List results = scanAll(vf); - Array arr = results.getFirst().columns().get("ts"); - var layout = ValueLayout.JAVA_LONG_UNALIGNED.withOrder(ByteOrder.LITTLE_ENDIAN); - for (int i = 0; i < data.length; i++) { - assertThat(arr.buffer(0).get(layout, (long) i * 8)).isEqualTo(42L); - } - } - } - - @Test - void roundTrip_singleElement(@TempDir Path tmp) throws IOException { - // Given — no deltas to store - Path file = tmp.resolve("delta_single.vtx"); - long[] data = {99L}; - - try (var ch = FileChannel.open(file, StandardOpenOption.CREATE, StandardOpenOption.WRITE); - var sut = VortexWriter.create(ch, I64_SCHEMA, WriteOptions.defaults(), - List.of(new DeltaEncoding()))) { - // When - sut.writeChunk(Map.of("ts", data)); - } - - // Then - try (var vf = VortexReader.open(file, deltaRegistry())) { - List results = scanAll(vf); - Array arr = results.getFirst().columns().get("ts"); - assertThat(arr.length()).isEqualTo(1L); - var layout = ValueLayout.JAVA_LONG_UNALIGNED.withOrder(ByteOrder.LITTLE_ENDIAN); - assertThat(arr.buffer(0).get(layout, 0L)).isEqualTo(99L); - } - } - - @Test - void roundTrip_mixedDeltas(@TempDir Path tmp) throws IOException { - // Given — non-monotonic; some deltas negative - Path file = tmp.resolve("delta_mixed.vtx"); - long[] data = {0L, 5L, 7L, 12L, 11L, 15L}; - - try (var ch = FileChannel.open(file, StandardOpenOption.CREATE, StandardOpenOption.WRITE); - var sut = VortexWriter.create(ch, I64_SCHEMA, WriteOptions.defaults(), - List.of(new DeltaEncoding()))) { - // When - sut.writeChunk(Map.of("ts", data)); - } - - // Then - try (var vf = VortexReader.open(file, deltaRegistry())) { - List results = scanAll(vf); - Array arr = results.getFirst().columns().get("ts"); - var layout = ValueLayout.JAVA_LONG_UNALIGNED.withOrder(ByteOrder.LITTLE_ENDIAN); - for (int i = 0; i < data.length; i++) { - assertThat(arr.buffer(0).get(layout, (long) i * 8)).isEqualTo(data[i]); - } - } - } - - // ── Helpers ─────────────────────────────────────────────────────────────── - - @Test - void roundTrip_multipleChunks(@TempDir Path tmp) throws IOException { - // Given - Path file = tmp.resolve("delta_multi.vtx"); - long[] chunk1 = {1000L, 1001L, 1002L}; - long[] chunk2 = {2000L, 2005L, 2010L}; - - try (var ch = FileChannel.open(file, StandardOpenOption.CREATE, StandardOpenOption.WRITE); - var sut = VortexWriter.create(ch, I64_SCHEMA, WriteOptions.defaults(), - List.of(new DeltaEncoding()))) { - // When - sut.writeChunk(Map.of("ts", chunk1)); - sut.writeChunk(Map.of("ts", chunk2)); - } - - // Then — process each chunk before advancing; hasNext() closes the previous chunk's arena - try (var vf = VortexReader.open(file, deltaRegistry()); - var iter = vf.scan(ScanOptions.all())) { - var layout = ValueLayout.JAVA_LONG_UNALIGNED.withOrder(ByteOrder.LITTLE_ENDIAN); - - assertThat(iter.hasNext()).isTrue(); - Array a1 = iter.next().columns().get("ts"); - for (int i = 0; i < chunk1.length; i++) { - assertThat(a1.buffer(0).get(layout, (long) i * 8)).isEqualTo(chunk1[i]); - } - - assertThat(iter.hasNext()).isTrue(); - Array a2 = iter.next().columns().get("ts"); - for (int i = 0; i < chunk2.length; i++) { - assertThat(a2.buffer(0).get(layout, (long) i * 8)).isEqualTo(chunk2[i]); - } - - assertThat(iter.hasNext()).isFalse(); - } - } - - @ParameterizedTest - @ValueSource(ints = {2, 10, 100, 1000}) - void roundTrip_sequentialTimestamps(int n, @TempDir Path tmp) throws IOException { - // Given — simulate monotonic timestamp column - Path file = tmp.resolve("delta_seq_" + n + ".vtx"); - long[] data = new long[n]; - for (int i = 0; i < n; i++) { - data[i] = 1_700_000_000L + i * 1000L; - } - - try (var ch = FileChannel.open(file, StandardOpenOption.CREATE, StandardOpenOption.WRITE); - var sut = VortexWriter.create(ch, I64_SCHEMA, WriteOptions.defaults(), - List.of(new DeltaEncoding()))) { - // When - sut.writeChunk(Map.of("ts", data)); - } - - // Then - try (var vf = VortexReader.open(file, deltaRegistry())) { - List results = scanAll(vf); - assertThat(results).hasSize(1); - Array arr = results.getFirst().columns().get("ts"); - assertThat(arr.length()).isEqualTo((long) n); - var layout = ValueLayout.JAVA_LONG_UNALIGNED.withOrder(ByteOrder.LITTLE_ENDIAN); - for (int i = 0; i < n; i++) { - assertThat(arr.buffer(0).get(layout, (long) i * 8)) - .as("index %d", i) - .isEqualTo(data[i]); - } - } - } -} diff --git a/writer/src/test/java/io/github/dfa1/vortex/writer/DictEncodingTest.java b/writer/src/test/java/io/github/dfa1/vortex/writer/DictEncodingTest.java deleted file mode 100644 index 5309212..0000000 --- a/writer/src/test/java/io/github/dfa1/vortex/writer/DictEncodingTest.java +++ /dev/null @@ -1,149 +0,0 @@ -package io.github.dfa1.vortex.writer; - -import io.github.dfa1.vortex.core.array.Array; -import io.github.dfa1.vortex.core.DType; -import io.github.dfa1.vortex.core.PType; -import io.github.dfa1.vortex.encoding.EncodingRegistry; -import io.github.dfa1.vortex.encoding.DictEncoding; -import io.github.dfa1.vortex.encoding.PrimitiveEncoding; -import io.github.dfa1.vortex.io.VortexReader; -import io.github.dfa1.vortex.scan.ScanOptions; -import io.github.dfa1.vortex.scan.ScanResult; -import org.junit.jupiter.api.Test; -import org.junit.jupiter.api.io.TempDir; - -import java.io.IOException; -import java.lang.foreign.ValueLayout; -import java.nio.ByteOrder; -import java.nio.channels.FileChannel; -import java.nio.file.Path; -import java.nio.file.StandardOpenOption; -import java.util.ArrayList; -import java.util.List; -import java.util.Map; - -import static org.assertj.core.api.Assertions.assertThat; - -class DictEncodingTest { - - private static final DType.Struct SCHEMA = new DType.Struct( - List.of("category"), - List.of(new DType.Primitive(PType.I32, false)), - false); - - private static List scanAll(VortexReader vf, ScanOptions opts) throws IOException { - var results = new ArrayList(); - var iter = vf.scan(opts); - while (iter.hasNext()) { - results.add(iter.next()); - } - return results; - } - - private static EncodingRegistry dictRegistry() { - var registry = EncodingRegistry.empty(); - registry.register(new DictEncoding()); - registry.register(new PrimitiveEncoding()); - return registry; - } - - @Test - void roundTrip_lowCardinality_valuesExpandCorrectly(@TempDir Path tmp) throws IOException { - // Given - Path file = tmp.resolve("dict.vtx"); - int[] data = {1, 2, 1, 3, 2, 1}; - - try (var ch = FileChannel.open(file, StandardOpenOption.CREATE, StandardOpenOption.WRITE); - var sut = VortexWriter.create(ch, SCHEMA, WriteOptions.defaults(), - List.of(new DictEncoding()))) { - // When - sut.writeChunk(Map.of("category", data)); - } - - // Then - var registry = dictRegistry(); - try (var vf = VortexReader.open(file, registry)) { - List results = scanAll(vf, ScanOptions.all()); - assertThat(results).hasSize(1); - Array arr = results.getFirst().columns().get("category"); - assertThat(arr.length()).isEqualTo(6L); - var layout = ValueLayout.JAVA_INT_UNALIGNED.withOrder(ByteOrder.LITTLE_ENDIAN); - var buf = arr.buffer(0); - assertThat(buf.get(layout, 0)).isEqualTo(1); - assertThat(buf.get(layout, 4)).isEqualTo(2); - assertThat(buf.get(layout, 8)).isEqualTo(1); - assertThat(buf.get(layout, 12)).isEqualTo(3); - assertThat(buf.get(layout, 16)).isEqualTo(2); - assertThat(buf.get(layout, 20)).isEqualTo(1); - } - } - - // ── Helpers ─────────────────────────────────────────────────────────────── - - @Test - void roundTrip_singleUniqueValue_u8Codes(@TempDir Path tmp) throws IOException { - // Given — all same value, dict has 1 entry, codes are U8(0) - Path file = tmp.resolve("dict_single.vtx"); - int[] data = {42, 42, 42, 42}; - - try (var ch = FileChannel.open(file, StandardOpenOption.CREATE, StandardOpenOption.WRITE); - var sut = VortexWriter.create(ch, SCHEMA, WriteOptions.defaults(), - List.of(new DictEncoding()))) { - // When - sut.writeChunk(Map.of("category", data)); - } - - // Then - var registry = dictRegistry(); - try (var vf = VortexReader.open(file, registry)) { - List results = scanAll(vf, ScanOptions.all()); - assertThat(results).hasSize(1); - Array arr = results.getFirst().columns().get("category"); - assertThat(arr.length()).isEqualTo(4L); - var layout = ValueLayout.JAVA_INT_UNALIGNED.withOrder(ByteOrder.LITTLE_ENDIAN); - var buf = arr.buffer(0); - for (int i = 0; i < 4; i++) { - assertThat(buf.get(layout, (long) i * 4)).isEqualTo(42); - } - } - } - - @Test - void roundTrip_multipleChunks(@TempDir Path tmp) throws IOException { - // Given - Path file = tmp.resolve("dict_multi.vtx"); - int[] chunk1 = {10, 20, 10}; - int[] chunk2 = {30, 10, 20, 30}; - - try (var ch = FileChannel.open(file, StandardOpenOption.CREATE, StandardOpenOption.WRITE); - var sut = VortexWriter.create(ch, SCHEMA, WriteOptions.defaults(), - List.of(new DictEncoding()))) { - // When - sut.writeChunk(Map.of("category", chunk1)); - sut.writeChunk(Map.of("category", chunk2)); - } - - // Then — process each chunk before advancing; hasNext() closes the previous chunk's arena - try (var vf = VortexReader.open(file, dictRegistry()); - var iter = vf.scan(ScanOptions.all())) { - var layout = ValueLayout.JAVA_INT_UNALIGNED.withOrder(ByteOrder.LITTLE_ENDIAN); - - assertThat(iter.hasNext()).isTrue(); - Array a1 = iter.next().columns().get("category"); - assertThat(a1.length()).isEqualTo(3L); - assertThat(a1.buffer(0).get(layout, 0)).isEqualTo(10); - assertThat(a1.buffer(0).get(layout, 4)).isEqualTo(20); - assertThat(a1.buffer(0).get(layout, 8)).isEqualTo(10); - - assertThat(iter.hasNext()).isTrue(); - Array a2 = iter.next().columns().get("category"); - assertThat(a2.length()).isEqualTo(4L); - assertThat(a2.buffer(0).get(layout, 0)).isEqualTo(30); - assertThat(a2.buffer(0).get(layout, 4)).isEqualTo(10); - assertThat(a2.buffer(0).get(layout, 8)).isEqualTo(20); - assertThat(a2.buffer(0).get(layout, 12)).isEqualTo(30); - - assertThat(iter.hasNext()).isFalse(); - } - } -} diff --git a/writer/src/test/java/io/github/dfa1/vortex/writer/VortexWriterTest.java b/writer/src/test/java/io/github/dfa1/vortex/writer/VortexWriterTest.java deleted file mode 100644 index 53f9073..0000000 --- a/writer/src/test/java/io/github/dfa1/vortex/writer/VortexWriterTest.java +++ /dev/null @@ -1,207 +0,0 @@ -package io.github.dfa1.vortex.writer; - -import io.github.dfa1.vortex.core.array.Array; -import io.github.dfa1.vortex.core.DType; -import io.github.dfa1.vortex.core.PType; -import io.github.dfa1.vortex.core.array.LongArray; -import io.github.dfa1.vortex.encoding.AlpEncoding; -import io.github.dfa1.vortex.encoding.EncodingRegistry; -import io.github.dfa1.vortex.encoding.PrimitiveEncoding; -import io.github.dfa1.vortex.io.VortexReader; -import io.github.dfa1.vortex.scan.ScanOptions; -import io.github.dfa1.vortex.scan.ScanResult; -import org.junit.jupiter.api.Test; -import org.junit.jupiter.api.io.TempDir; - -import java.io.IOException; -import java.lang.foreign.MemorySegment; -import java.lang.foreign.ValueLayout; -import java.nio.ByteOrder; -import java.nio.channels.FileChannel; -import java.nio.file.Path; -import java.nio.file.StandardOpenOption; -import java.util.ArrayList; -import java.util.List; -import java.util.Map; - -import static org.assertj.core.api.Assertions.assertThat; -import static org.assertj.core.api.Assertions.assertThatThrownBy; - -class VortexWriterTest { - - private static final DType.Struct SCHEMA = new DType.Struct( - List.of("id", "value"), - List.of(new DType.Primitive(PType.I64, false), - new DType.Primitive(PType.F64, false)), - false); - - // ── writeChunk validation ───────────────────────────────────────────────── - - private static List scanAll(VortexReader vf, ScanOptions opts) throws IOException { - var results = new ArrayList(); - var iter = vf.scan(opts); - while (iter.hasNext()) { - results.add(iter.next()); - } - return results; - } - - // ── Round-trip: write then read ─────────────────────────────────────────── - - private static EncodingRegistry primitiveRegistry() { - var registry = EncodingRegistry.empty(); - registry.register(new AlpEncoding()); - registry.register(new PrimitiveEncoding()); - return registry; - } - - @Test - void writeChunk_missingColumn_throwsIllegalArgument(@TempDir Path tmp) throws IOException { - // Given - Path file = tmp.resolve("missing.vtx"); - try (var ch = FileChannel.open(file, StandardOpenOption.CREATE, StandardOpenOption.WRITE); - var sut = VortexWriter.create(ch, SCHEMA, WriteOptions.defaults())) { - // When / Then - assertThatThrownBy(() -> sut.writeChunk(Map.of("id", new long[]{1L}))) - .isInstanceOf(IllegalArgumentException.class) - .hasMessageContaining("missing column: value"); - } - } - - @Test - void writeAndRead_singleChunk_returnsCorrectRowCount(@TempDir Path tmp) throws IOException { - // Given - Path file = tmp.resolve("single.vtx"); - long[] ids = {1L, 2L, 3L}; - double[] vals = {1.0, 2.0, 3.0}; - - try (var ch = FileChannel.open(file, StandardOpenOption.CREATE, StandardOpenOption.WRITE); - var sut = VortexWriter.create(ch, SCHEMA, WriteOptions.defaults())) { - // When - sut.writeChunk(Map.of("id", ids, "value", vals)); - } - - // Then - var registry = primitiveRegistry(); - try (var vf = VortexReader.open(file, registry)) { - List results = scanAll(vf, ScanOptions.all()); - assertThat(results).hasSize(1); - assertThat(results.getFirst().rowCount()).isEqualTo(3L); - assertThat(results.getFirst().columns()).containsKeys("id", "value"); - } - } - - @Test - void writeAndRead_multipleChunks_returnsAllChunks(@TempDir Path tmp) throws IOException { - // Given - Path file = tmp.resolve("multi.vtx"); - - try (var ch = FileChannel.open(file, StandardOpenOption.CREATE, StandardOpenOption.WRITE); - var sut = VortexWriter.create(ch, SCHEMA, WriteOptions.defaults())) { - // When - sut.writeChunk(Map.of("id", new long[]{1L, 2L}, "value", new double[]{1.0, 2.0})); - sut.writeChunk(Map.of("id", new long[]{3L, 4L, 5L}, "value", new double[]{3.0, 4.0, 5.0})); - } - - // Then - var registry = primitiveRegistry(); - try (var vf = VortexReader.open(file, registry)) { - List results = scanAll(vf, ScanOptions.all()); - assertThat(results).hasSize(2); - assertThat(results.get(0).rowCount()).isEqualTo(2L); - assertThat(results.get(1).rowCount()).isEqualTo(3L); - } - } - - // ── Helpers ─────────────────────────────────────────────────────────────── - - @Test - void writeAndRead_idValues_decodedCorrectly(@TempDir Path tmp) throws IOException { - // Given - Path file = tmp.resolve("values.vtx"); - long[] ids = {42L, 100L, -1L}; - - try (var ch = FileChannel.open(file, StandardOpenOption.CREATE, StandardOpenOption.WRITE); - var sut = VortexWriter.create(ch, SCHEMA, WriteOptions.defaults())) { - // When - sut.writeChunk(Map.of("id", ids, "value", new double[]{0.0, 0.0, 0.0})); - } - - // Then - var registry = primitiveRegistry(); - try (var vf = VortexReader.open(file, registry)) { - List results = scanAll(vf, ScanOptions.all()); - assertThat(results).hasSize(1); - Array idArray = results.getFirst().columns().get("id"); - assertThat(idArray.length()).isEqualTo(3L); - MemorySegment buf = idArray.buffer(0); - assertThat(buf.get(ValueLayout.JAVA_LONG_UNALIGNED.withOrder(ByteOrder.LITTLE_ENDIAN), 0)).isEqualTo(42L); - assertThat(buf.get(ValueLayout.JAVA_LONG_UNALIGNED.withOrder(ByteOrder.LITTLE_ENDIAN), 8)).isEqualTo(100L); - assertThat(buf.get(ValueLayout.JAVA_LONG_UNALIGNED.withOrder(ByteOrder.LITTLE_ENDIAN), 16)).isEqualTo(-1L); - } - } - - @Test - void scanResult_column_returnsTypedArray(@TempDir Path tmp) throws IOException { - // Given - Path file = tmp.resolve("typed.vtx"); - long[] ids = {10L, 20L, 30L}; - - try (var ch = FileChannel.open(file, StandardOpenOption.CREATE, StandardOpenOption.WRITE); - var sut = VortexWriter.create(ch, SCHEMA, WriteOptions.defaults())) { - sut.writeChunk(Map.of("id", ids, "value", new double[]{1.0, 2.0, 3.0})); - } - - // When - var registry = primitiveRegistry(); - try (var vf = VortexReader.open(file, registry)) { - List results = scanAll(vf, ScanOptions.all()); - LongArray idArray = results.getFirst().column("id"); - - // Then - assertThat(idArray.fold(0L, Long::sum)).isEqualTo(60L); - } - } - - @Test - void scanResult_column_unknownName_throwsVortexException(@TempDir Path tmp) throws IOException { - // Given - Path file = tmp.resolve("unknown.vtx"); - - try (var ch = FileChannel.open(file, StandardOpenOption.CREATE, StandardOpenOption.WRITE); - var sut = VortexWriter.create(ch, SCHEMA, WriteOptions.defaults())) { - sut.writeChunk(Map.of("id", new long[]{1L}, "value", new double[]{1.0})); - } - - // When / Then - var registry = primitiveRegistry(); - try (var vf = VortexReader.open(file, registry)) { - List results = scanAll(vf, ScanOptions.all()); - ScanResult sut = results.getFirst(); - assertThatThrownBy(() -> sut.column("nonexistent")) - .hasMessageContaining("unknown column: nonexistent"); - } - } - - @Test - void writeAndRead_columnProjection_returnsOnlyRequestedColumns(@TempDir Path tmp) - throws IOException { - // Given - Path file = tmp.resolve("proj.vtx"); - - try (var ch = FileChannel.open(file, StandardOpenOption.CREATE, StandardOpenOption.WRITE); - var sut = VortexWriter.create(ch, SCHEMA, WriteOptions.defaults())) { - // When - sut.writeChunk(Map.of("id", new long[]{1L}, "value", new double[]{9.9})); - } - - // Then - var registry = primitiveRegistry(); - try (var vf = VortexReader.open(file, registry)) { - List results = scanAll(vf, ScanOptions.columns("id")); - assertThat(results).hasSize(1); - assertThat(results.getFirst().columns()).containsKey("id"); - assertThat(results.getFirst().columns()).doesNotContainKey("value"); - } - } -} diff --git a/writer/src/test/java/io/github/dfa1/vortex/writer/ZoneMapPruningTest.java b/writer/src/test/java/io/github/dfa1/vortex/writer/ZoneMapPruningTest.java deleted file mode 100644 index 4bb3087..0000000 --- a/writer/src/test/java/io/github/dfa1/vortex/writer/ZoneMapPruningTest.java +++ /dev/null @@ -1,140 +0,0 @@ -package io.github.dfa1.vortex.writer; - -import io.github.dfa1.vortex.core.DType; -import io.github.dfa1.vortex.core.PType; -import io.github.dfa1.vortex.encoding.EncodingRegistry; -import io.github.dfa1.vortex.encoding.PrimitiveEncoding; -import io.github.dfa1.vortex.io.VortexReader; -import io.github.dfa1.vortex.scan.RowFilter; -import io.github.dfa1.vortex.scan.ScanOptions; -import io.github.dfa1.vortex.scan.ScanResult; -import org.junit.jupiter.api.Test; -import org.junit.jupiter.api.io.TempDir; - -import java.io.IOException; -import java.nio.channels.FileChannel; -import java.nio.file.Path; -import java.nio.file.StandardOpenOption; -import java.util.ArrayList; -import java.util.List; -import java.util.Map; - -import static org.assertj.core.api.Assertions.assertThat; - -/// Round-trip tests verifying zone-map chunk pruning via RowFilter. -class ZoneMapPruningTest { - - private static final DType.Struct SCHEMA = new DType.Struct( - List.of("id"), - List.of(new DType.Primitive(PType.I64, false)), - false); - - // Three chunks: id in [1..50], [51..100], [101..150] - private static Path writeThreeChunks(Path tmp) throws IOException { - Path file = tmp.resolve("three_chunks.vtx"); - try (var ch = FileChannel.open(file, StandardOpenOption.CREATE, StandardOpenOption.WRITE); - var sut = VortexWriter.create(ch, SCHEMA, WriteOptions.defaults())) { - sut.writeChunk(Map.of("id", range(1L, 50L))); - sut.writeChunk(Map.of("id", range(51L, 100L))); - sut.writeChunk(Map.of("id", range(101L, 150L))); - } - return file; - } - - private static List scanWith(Path file, RowFilter filter) throws IOException { - var opts = new ScanOptions(List.of(), filter, ScanOptions.NO_LIMIT); - var registry = primitiveRegistry(); - var results = new ArrayList(); - try (var vf = VortexReader.open(file, registry); - var iter = vf.scan(opts)) { - while (iter.hasNext()) { - results.add(iter.next()); - } - } - return results; - } - - private static long[] range(long from, long to) { - long[] arr = new long[(int) (to - from + 1)]; - for (int i = 0; i < arr.length; i++) { - arr[i] = from + i; - } - return arr; - } - - private static EncodingRegistry primitiveRegistry() { - var registry = EncodingRegistry.empty(); - registry.register(new PrimitiveEncoding()); - return registry; - } - - @Test - void gte_prunesChunksBelowThreshold(@TempDir Path tmp) throws IOException { - // Given — chunk 1 max=50, threshold=75 → chunk 1 pruned - Path file = writeThreeChunks(tmp); - - // When - List results = scanWith(file, RowFilter.gte("id", 75L)); - - // Then - assertThat(results).hasSize(2); - assertThat(results.get(0).rowCount()).isEqualTo(50L); // chunk 2 - assertThat(results.get(1).rowCount()).isEqualTo(50L); // chunk 3 - } - - @Test - void lte_prunesChunksAboveThreshold(@TempDir Path tmp) throws IOException { - // Given — chunk 3 min=101, threshold=75 → chunk 3 pruned - Path file = writeThreeChunks(tmp); - - // When - List results = scanWith(file, RowFilter.lte("id", 75L)); - - // Then - assertThat(results).hasSize(2); - assertThat(results.get(0).rowCount()).isEqualTo(50L); // chunk 1 - assertThat(results.get(1).rowCount()).isEqualTo(50L); // chunk 2 - } - - // ── Helpers ─────────────────────────────────────────────────────────────── - - @Test - void eq_prunesChunksExcludingValue(@TempDir Path tmp) throws IOException { - // Given — id=75 only falls in chunk 2 [51..100]; chunks 1 and 3 pruned - Path file = writeThreeChunks(tmp); - - // When - List results = scanWith(file, RowFilter.eq("id", 75L)); - - // Then - assertThat(results).hasSize(1); - assertThat(results.getFirst().rowCount()).isEqualTo(50L); // chunk 2 - } - - @Test - void and_prunesChunksExcludedByAnySubFilter(@TempDir Path tmp) throws IOException { - // Given — AND(id>=51, id<=100): chunk 1 pruned by gte, chunk 3 pruned by lte - Path file = writeThreeChunks(tmp); - - // When - List results = scanWith(file, RowFilter.and( - RowFilter.gte("id", 51L), - RowFilter.lte("id", 100L))); - - // Then - assertThat(results).hasSize(1); - assertThat(results.getFirst().rowCount()).isEqualTo(50L); // chunk 2 - } - - @Test - void noFilter_returnsAllChunks(@TempDir Path tmp) throws IOException { - // Given - Path file = writeThreeChunks(tmp); - - // When - List results = scanWith(file, null); - - // Then - assertThat(results).hasSize(3); - } -}