Skip to content
Permalink

Comparing changes

Choose two branches to see what’s changed or to start a new pull request. If you need to, you can also or learn more about diff comparisons.

Open a pull request

Create a new pull request by comparing changes across two branches. If you need to, you can also . Learn more about diff comparisons here.
base repository: feldera/feldera
Failed to load repositories. Confirm that selected base ref is valid, then try again.
Loading
base: v0.276.0
Choose a base ref
...
head repository: feldera/feldera
Failed to load repositories. Confirm that selected head ref is valid, then try again.
Loading
compare: v0.277.0
Choose a head ref
  • 9 commits
  • 71 files changed
  • 5 contributors

Commits on Mar 31, 2026

  1. dbsp: Add range filter in front of bloom filter 

    This is a very cheap filter that can speed up
ingest significantly.

The idea is to track the min/max for every batch.
For seek_key_exact if the value we're seeking
is not in side of the range we skip the batch.

Some ingest heavy benchmarks:

just bloom filter:

╭────────────────────────┬───────────┬──────────┬───────╮
│ Metric                 │ Value     │ Lower    │ Upper │
├────────────────────────┼───────────┼──────────┼───────┤
│ Throughput (records/s) │ 2695496   │ -        │ -     │
│ Memory                 │ 14.28 GiB │ 1.78 GiB │ -     │
│ Storage                │ 79.45 GiB │ 110 B    │ -     │
│ Uptime [ms]            │ 302742    │ -        │ -     │
│ State Amplification    │ 0.43      │ -        │ -     │
╰────────────────────────┴───────────┴──────────┴───────╯

range+bloom filter:

╭────────────────────────┬────────────┬──────────┬───────╮
│ Metric                 │ Value      │ Lower    │ Upper │
├────────────────────────┼────────────┼──────────┼───────┤
│ Throughput (records/s) │ 4035088    │ -        │ -     │
│ Memory                 │ 23.4 GiB   │ 2.42 GiB │ -     │
│ Storage                │ 112.51 GiB │ 110 B    │ -     │
│ Uptime [ms]            │ 303292     │ -        │ -     │
│ State Amplification    │ 0.41       │ -        │ -     │
╰────────────────────────┴────────────┴──────────┴───────╯

Signed-off-by: Gerd Zellweger <mail@gerdzellweger.com>
    @gz
    gz committed Mar 31, 2026
    Configuration menu
    Copy the full SHA
    f9f18b0 View commit details
    Browse the repository at this point in the history

  2. dbsp: Better stats for filters. 

    We add stats for the range filter. This lead to some refactoring:

Since we now have two filters (with another one on the way)
we consolidate the stats into a single struct that can be
re-used across filters.

It also revealed a performance issue with the current filter
stats. Because this function is extremly hot adding the
hit and miss atomics in CachePadded led to a 25% increase
for the ingest benchmark.

Signed-off-by: Gerd Zellweger <mail@gerdzellweger.com>
    @gz
    gz committed Mar 31, 2026
    Configuration menu
    Copy the full SHA
    7fedd83 View commit details
    Browse the repository at this point in the history

  3. dbsp: Fix FileWSet negation. 

    We didn't load the weight into cursor.diff which caused
some newly written tests to fail

We did an anlaysis and concluded that this method,
while wrong, luckily wasn't used anywhere relevant in real
circuits.

Signed-off-by: Gerd Zellweger <mail@gerdzellweger.com>
    @gz
    gz committed Mar 31, 2026
    Configuration menu
    Copy the full SHA
    85c5d4e View commit details
    Browse the repository at this point in the history

  4. dbsp: Filter refactoring 

    - Remove legacy maybe_contains_key
- Reader no longer "owns" bloom filter directly
- Filtering is handled by BatchFilters (which in the future can support
  multiple different filter impls)

This is supposed to make it easier to customize filters in a follow
up PR.

Signed-off-by: Gerd Zellweger <mail@gerdzellweger.com>
    @gz
    gz committed Mar 31, 2026
    Configuration menu
    Copy the full SHA
    c3d05d1 View commit details
    Browse the repository at this point in the history

  5. [dbsp] Eliminate duplicate code in Consensus implementation. 

    `Consensus` is just `Broadcast<bool>` with a logical-and built in.

Signed-off-by: Ben Pfaff <blp@feldera.com>
    @blp
    blp committed Mar 31, 2026
    Configuration menu
    Copy the full SHA
    e971b47 View commit details
    Browse the repository at this point in the history

  6. ci: Prepare for v0.277.0

    @release-feldera-feldera
    release-feldera-feldera[bot] committed Mar 31, 2026
    Configuration menu
    Copy the full SHA
    6ce8158 View commit details
    Browse the repository at this point in the history

  7. Update vulnerable docs site deps, re-introduce min age for Bun 

    Signed-off-by: Karakatiza666 <bulakh.96@gmail.com>
    @Karakatiza666 @gz
    Karakatiza666 authored and gz committed Mar 31, 2026
    Configuration menu
    Copy the full SHA
    9608884 View commit details
    Browse the repository at this point in the history

  8. [dbsp] Make DevTweaks public so it will get documented. 

    Signed-off-by: Ben Pfaff <blp@feldera.com>
    @blp
    blp committed Mar 31, 2026
    Configuration menu
    Copy the full SHA
    161bafd View commit details
    Browse the repository at this point in the history

  9. [ci] apply automatic fixes 

    Signed-off-by: feldera-bot <feldera-bot@feldera.com>
    @feldera-bot @blp
    feldera-bot authored and blp committed Mar 31, 2026
    Configuration menu
    Copy the full SHA
    68ee6dd View commit details
    Browse the repository at this point in the history
Loading