Skip to content

Testing API to block for compaction#6511

Open
mihaibudiu wants to merge 2 commits into
feldera:mainfrom
mihaibudiu:issue2808
Open

Testing API to block for compaction#6511
mihaibudiu wants to merge 2 commits into
feldera:mainfrom
mihaibudiu:issue2808

Conversation

@mihaibudiu

Copy link
Copy Markdown
Contributor

Fixes #2808

Adds a new DBSP-level API for a circuit to wait for compaction to terminate in all workers.
Adds SQL compiler support for writing tests using this API. I expect all tests that relate to GC will use this API heavily.

Describe Manual Test Plan

Tested the new Java test

Checklist

  • Unit tests added/updated

@mihaibudiu mihaibudiu requested a review from blp June 20, 2026 02:23
Signed-off-by: Mihai Budiu <mbudiu@feldera.com>

@mythical-fred mythical-fred left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM — clean API extension, solid test.

Comment thread crates/dbsp/src/circuit/dbsp_handle.rs Outdated

@mythical-fred mythical-fred left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-approve. DbspError::Runtime swap addresses my prior nit on the timeout-error variant. New per-worker IsCompactionComplete round-trip and Java BlockForCompaction command are clean.

One small doc nit inline.

Comment thread crates/dbsp/src/circuit/dbsp_handle.rs Outdated
/// to a cap of 1 s.
///
/// Returns `Ok(())` when compaction is complete, or
/// Error the timeout expires.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: doc-comment grammar — "Error the timeout expires" should be "Returns Err([DbspError::Runtime]) when the timeout expires." or simply "Errors when the timeout expires." (Missing verb.)

Signed-off-by: Mihai Budiu <mbudiu@feldera.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[DBSP] Instant GC

2 participants