Skip to content

Multihost#6703

Open
blp wants to merge 3 commits into
mainfrom
multihost
Open

Multihost#6703
blp wants to merge 3 commits into
mainfrom
multihost

Conversation

@blp

@blp blp commented Jul 22, 2026

Copy link
Copy Markdown
Member

Describe Manual Test Plan

These are primarily performance optimizations. I'll try them out with some workloads before merging.

Signed-off-by: Ben Pfaff <blp@feldera.com>
@blp
blp requested a review from gz July 22, 2026 23:30
@blp blp self-assigned this Jul 22, 2026
@blp blp added performance rust Pull requests that update Rust code multihost Related to multihost or distributed pipelines labels Jul 22, 2026

@gz gz left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the 2nd commit message seems to end abruptly at:
Since the /stats``


// Write an empty checkpoint file to save the cost of listing
// all the files next time.
backend.write_json(&file_name, &VecDeque::<CheckpointMetadata>::new())?;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i hope this is "safe", it seems to be because we only do this when we couldnt recover any checkpoints anyways(?)

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It should be? There's a theoretical race where something external writes a nonempty file just before we write it. But AFAIK nothing should be doing that.

blp added 2 commits July 22, 2026 16:41
When there is no checkpoint catalog file, Checkpointer::read_checkpoints
scans all of storage looking for checkpoint candidates.  Since the `/status`
and `/coordination/status` APIs call this a lot, it's expensive.

Signed-off-by: Ben Pfaff <blp@feldera.com>
This isn't needed and it's expensive to do I/O at least once a second to
implement `/coordination/status`.

Signed-off-by: Ben Pfaff <blp@feldera.com>
@blp

blp commented Jul 22, 2026

Copy link
Copy Markdown
Member Author

the 2nd commit message seems to end abruptly at: Since the /stats``

Oops, fixed.

@mythical-fred mythical-fred left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Three nice perf/quality wins in one PR — happy to see them all land together.

  • get_status(with_storage_status_details) correctly cuts the Checkpointer::read_checkpoints call out of the SSE polling loop in coordination_status. That listing was previously running on every notify iteration, which is exactly what you'd want to avoid on a hot streaming endpoint.
  • Exchange connection retry: hoisting the failure counter and logging "connected after N failures" turns a repeated info! spam into a single actionable signal. Also nice that backoff_time() is now module-level instead of a nested fn — easier to grep.
  • Checkpointer memoization is the right call for the empty-directory startup path.

One non-blocking nit on that last one:

Checkpointer::read_checkpoints now writes to the backend on the NotFound path. The name still suggests a pure read, so a caller (or future maintainer) reading read_checkpoints(&*backend) won't expect it to mutate storage. Worth either (a) a one-line doc comment on the fn saying "materializes an empty checkpoint file on first call if none exists, to short-circuit the listing on subsequent calls," or (b) renaming to load_checkpoints / factoring the write into a separate ensure_checkpoint_file_exists helper. The race gz asked about is theoretical as blp noted, so no blocker there.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

multihost Related to multihost or distributed pipelines performance rust Pull requests that update Rust code

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants