Skip to content

[adapters] log retention duration config for delta connector#6146

Draft
swanandx wants to merge 1 commit into
mainfrom
issue6027
Draft

[adapters] log retention duration config for delta connector#6146
swanandx wants to merge 1 commit into
mainfrom
issue6027

Conversation

@swanandx
Copy link
Copy Markdown
Member

@swanandx swanandx commented Apr 28, 2026

expose log_retention_duration and enable_expired_log_cleanup for delta sinks so users can use them for configuring the table. Similar to what we do for checkpoint_interval

Fix #6027

Describe Manual Test Plan

Testing manually under diff scenarios, will update later.

Checklist

  • Unit tests added/updated
  • Integration tests added/updated
  • Documentation updated
  • Changelog updated

Breaking Changes?

Mark if you think the answer is yes for any of these components:

Describe Incompatible Changes

We add new fields to delta output config, shouldn't break existing things

@swanandx swanandx marked this pull request as draft April 28, 2026 07:31
Comment thread crates/adapters/src/integrated/delta_table/output.rs Outdated
/// Returns the duration in seconds on success. This mirrors the grammar accepted by `delta-rs`
/// (`crates/core/src/table/config.rs::parse_interval`); we replicate it here to fail fast at
/// config-load time rather than only when the table is created.
fn parse_delta_interval(value: &str) -> Result<u64, String> {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

isn't their function public?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

expose log_retention_duration and enable_expired_log_cleanup for delta
sinks so users can use them for configuring the table. Similar to what we
do for checkpoint_interval

Signed-off-by: Swanand Mulay <73115739+swanandx@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[adapters] log retention config for delta output connector

2 participants