-
Notifications
You must be signed in to change notification settings - Fork 0
Comparing changes
Open a pull request
base repository: djanand/datafusion-python
base: main
head repository: apache/datafusion-python
compare: main
- 11 commits
- 70 files changed
- 8 contributors
Commits on Jul 14, 2026
-
fix: no panic on empty partition in register_record_batches (apache#1627
) * fix: no panic on empty partition in register_record_batches `register_record_batches` read the schema via unchecked `partitions.0[0][0]`, which panics when a partition contains no record batches (e.g. the batches of an empty pyarrow table, whose `to_batches()` returns `[]`). Take the schema from the first available batch across all partitions and return a clear error when there is none, mirroring the empty-table handling added for `from_arrow_table` in apache#613. A non-empty later partition now also works instead of panicking on an empty leading one. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor: address review feedback - Use find_map(|p| p.first()) instead of flatten().next() to make the "first batch in any partition" intent clearer (per @kosiew). - Extend regression test to also assert the empty outer partition list case ([]) raises the same ValueError. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Configuration menu - View commit details
-
Copy full SHA for 699e00e - Browse repository at this point
Copy the full SHA 699e00eView commit details
Commits on Jul 28, 2026
-
Update pinned Rust toolchain action revision (apache#1635)
* fix: replace rust-toolchain action with rustup commands for toolchain setup * fix: pin dtolnay/rust-toolchain sha * pin dtolnay/rust-toolchain sha
Configuration menu - View commit details
-
Copy full SHA for 6d16ecd - Browse repository at this point
Copy the full SHA 6d16ecdView commit details -
Use allowlisted Rust toolchain action revision (apache#1661)
* Use allowlisted Rust toolchain action revision AI Disclosure: This code was written in part by an AI agent.: * Set rust toolchain so developer and CI get a consistent experience * minor clippy fixes
Configuration menu - View commit details
-
Copy full SHA for 5cc3230 - Browse repository at this point
Copy the full SHA 5cc3230View commit details
Commits on Jul 30, 2026
-
Add object_store parameter to register/read file methods for thread-s…
…afe credentials (apache#1625) * Add object_store parameter to register/read file methods Add an optional object_store parameter to register_parquet, read_parquet, register_csv, read_csv, register_json, read_json, register_avro, read_avro, register_arrow, and read_arrow methods. When provided, the store is automatically registered for the URL scheme and host parsed from the path, removing the need to manually call register_object_store separately. This enables thread-safe cloud credential handling without os.environ mutation. Includes unit tests (mock-based) for URL parsing logic and integration tests using LocalFileSystem for end-to-end validation. Closes apache#1624 * Fix file:// URL handling in _register_object_store_for_path - Allow empty netloc for file:// scheme (e.g. file:///tmp/data.parquet) - Require netloc (host/bucket) only for non-file schemes (s3, gs, az, https) - Add doctest +SKIP to register_parquet example referencing nonexistent file - Use Path.as_uri() in end-to-end tests for portable file:// URL construction - Add tests for file:// acceptance and scheme-without-host rejection * Refactor tests to use pytest.mark.parametrize per reviewer feedback Replace class-based test structure with flat parametrized functions to match the repository's testing conventions. --------- Co-authored-by: Tim Saucer <timsaucer@gmail.com>
Configuration menu - View commit details
-
Copy full SHA for 4906d39 - Browse repository at this point
Copy the full SHA 4906d39View commit details -
docs: fix bootstrap instructions in contributor guide (apache#1618)
* docs: add missing cd step in contributor guide bootstrap * docs: fix maturin develop flag typo (-uv -> --uv)
Configuration menu - View commit details
-
Copy full SHA for 3d9583c - Browse repository at this point
Copy the full SHA 3d9583cView commit details
Commits on Aug 3, 2026
-
fix: tighten scalar UDF return type hints (apache#1642)
This PR tightens the scalar UDF annotations in python/datafusion/user_defined.py: the callable return type is bounded to pa.Array the scalar UDF constructor accepts return_field: pa.Field the decorator form accepts return_field: pa.DataType | pa.Field
Configuration menu - View commit details
-
Copy full SHA for 7eaef92 - Browse repository at this point
Copy the full SHA 7eaef92View commit details -
Add is_nan aliases (apache#1643)
Adds datafusion.functions.is_nan(...) as an alias for isnan(...) and adds Expr.is_nan() as the corresponding expression method. Existing isnan() behavior is unchanged. The PR also adds coverage for both the function-level alias and the expression method.
Configuration menu - View commit details
-
Copy full SHA for 4702915 - Browse repository at this point
Copy the full SHA 4702915View commit details
Commits on Aug 6, 2026
-
chore: Update to latest main on upstream DataFusion (apache#1662)
* Update to main on the upstream datafusion to prepare for DF55 release * Deprecate singular external table location AI Disclosure: This code was written in part by an AI agent.: * Fix tests for updated DataFusion behavior AI Disclosure: This code was written in part by an AI agent.: * create table requires a location * Remove unused Python token from Substrait deserialization AI Disclosure: This code was written in part by an AI agent.: * test: cover external table locations binding AI Disclosure: This code was written in part by an AI agent.:
Configuration menu - View commit details
-
Copy full SHA for cc2ec5c - Browse repository at this point
Copy the full SHA cc2ec5cView commit details
Commits on Aug 9, 2026
-
Configuration menu - View commit details
-
Copy full SHA for c577c0f - Browse repository at this point
Copy the full SHA c577c0fView commit details
Commits on Aug 13, 2026
-
Configuration menu - View commit details
-
Copy full SHA for 38422e4 - Browse repository at this point
Copy the full SHA 38422e4View commit details
Commits on Aug 14, 2026
-
chore: Bump release candidate version to 55.0.0rc3 (apache#1686)
* Bump release candidate version * spark slice function now respects inner field name
Configuration menu - View commit details
-
Copy full SHA for 2bd80ec - Browse repository at this point
Copy the full SHA 2bd80ecView commit details
This comparison is taking too long to generate.
Unfortunately it looks like we can’t render this comparison for you right now. It might be too big, or there might be something weird with your repository.
You can try running this command locally to see the comparison on your machine:
git diff main...main