Conversation
| iceberg-catalog-rest = "0.8.0" | ||
| iceberg-datafusion = "0.8.0" | ||
| iceberg = { version = "0.9.1" } | ||
| iceberg-catalog-glue = "0.9.1" |
There was a problem hiding this comment.
User-visible dep changes (iceberg 0.8 -> 0.9, deltalake fork bump, datafusion 51 -> 52) but no entry in docs.feldera.com/docs/changelog.md. Per the project rule, user-visible changes need a changelog entry. Please add one calling out: (a) the iceberg/datafusion/object_store version bumps and the rustls-webpki CVE motivation, (b) the move from gz/delta-rs to the org-owned feldera/delta-rs fork (good move, by the way), (c) the Azure Iceberg storage regression below.
| iceberg-catalog-glue = "0.9.1" | ||
| iceberg-catalog-rest = "0.9.1" | ||
| iceberg-datafusion = "0.9.1" | ||
| iceberg-storage-opendal = { version = "0.9.1", features = ["opendal-s3", "opendal-gcs", "opendal-fs", "opendal-memory"] } |
There was a problem hiding this comment.
Behavior change worth flagging: iceberg 0.8 was pulled in with features = ["storage-all"], which in iceberg-rust 0.8 includes Azure (storage-azdls / abfss://). The new explicit feature list (opendal-s3, opendal-gcs, opendal-fs, opendal-memory) drops Azure, and the matching storage_factory_for_url in crates/iceberg/src/input.rs now returns unsupported storage scheme 'abfss'. If Azure Iceberg tables were ever working for users, this is a silent functional regression at the Cargo level and a hard error at runtime. Two acceptable resolutions: (1) add opendal-azblob to the feature list and an "abfss" | "abfs" | "wasbs" arm to storage_factory_for_url; or (2) explicitly call out "Azure Iceberg storage no longer supported" in the changelog and PR description and confirm with Gerd that this is intended.
|
|
||
| fn factory_kind(f: &Arc<dyn StorageFactory>) -> &'static str { | ||
| // `OpenDalStorageFactory` is `#[non_exhaustive]` and the trait object | ||
| // hides the variant, so use the `Debug` impl to identify it. |
There was a problem hiding this comment.
Test smell: factory_kind identifies variants by parsing format!("{f:?}"). If a future iceberg-storage-opendal release changes the Debug formatting (e.g. wraps the variant), the test will silently misclassify rather than fail loudly. A bit of paranoia would help -- e.g. additionally assert the resulting FileIO can produce an input handle for a no-op URL, or fall back to panic! on the _ arm and key on a more stable signal (the input scheme that was matched, threaded out of storage_factory_for_url). Not a hard blocker, just brittle.
|
|
||
| [profile.release] | ||
| debug = "line-tables-only" | ||
|
|
There was a problem hiding this comment.
Nit: trailing blank line at end of file -- please drop it.
Main goal behind updating them was to bump version of rustls-webpki as < 0.103.13 was affected by some severe vulnerabilities. iceberg's 0.9.1 was required to actually bump rustls-webpki and it had a bit of snowball effect and we had to update others. we ported delta-rs fork into feldera org. Cherry picked our 3 fixes over rust-v0.31.1 tag as that had the required datafusion versions to be compatible with iceberg 0.9.1 Signed-off-by: Swanand Mulay <73115739+swanandx@users.noreply.github.com>
Main goal behind updating them was to bump version of rustls-webpki as < 0.103.13 was affected by some severe vulnerabilities.
iceberg's 0.9.1 was required to actually bump rustls-webpki and it had a bit of snowball effect and we had to update others.
we ported delta-rs fork into feldera org. Cherry picked our 3 fixes over rust-v0.31.0 tag as that had the required datafusion versions to be compatible with iceberg 0.9.1
Fix #6011
Describe Manual Test Plan
Checklist
Breaking Changes?
Not a breaking change