Skip to content

Add FFI query planner support with composable extension codecs #1672

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
timsaucer wants to merge 10 commits into from
+2,751 −219
Closed

Add FFI query planner support with composable extension codecs #1672

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
10 commits
Select commit Hold shift + click to select a range
9ba147c
Add FFI query planner support
timsaucer Jul 31, 2026
47aea03
Add three-library FFI planner example
timsaucer Jul 31, 2026
05ab5da
Update FFI query planner integration
timsaucer Aug 7, 2026
8e02881
Make extension codecs composable
timsaucer Aug 7, 2026
55f43b5
add rat
timsaucer Aug 7, 2026
c1bb4bf
Add atomic SessionContext.with_extensions API
timsaucer Aug 7, 2026
c868447
Add extension-bundle example and with_extensions FFI tests
timsaucer Aug 7, 2026
825b748
Document and test the context-outlives-DataFrame contract
timsaucer Aug 7, 2026
8a75aca
Skip private internal methods in wrapper coverage test
timsaucer Aug 7, 2026
590a4a2
Test planner rebinding and codec precedence in with_extensions
timsaucer Aug 7, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 14 additions & 2 deletions .github/workflows/build.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -186,7 +186,7 @@ jobs:
manylinux: "2_28"

# FFI test wheel only needs to be built once per platform; gate to abi3.
- name: Build FFI test library
- name: Build FFI provider test library
if: matrix.python-tag == 'abi3'
uses: PyO3/maturin-action@v1
with:
Expand All @@ -196,6 +196,16 @@ jobs:
args: --out dist
rustup-components: rust-std

- name: Build FFI query planner test library
if: matrix.python-tag == 'abi3'
uses: PyO3/maturin-action@v1
with:
target: x86_64-unknown-linux-gnu
manylinux: "2_28"
working-directory: examples/datafusion-ffi-query-planner-example
args: --out dist
rustup-components: rust-std

- name: Archive wheels
uses: actions/upload-artifact@v7
with:
Expand All @@ -207,7 +217,9 @@ jobs:
uses: actions/upload-artifact@v7
with:
name: test-ffi-manylinux-x86_64
path: examples/datafusion-ffi-example/dist/*
path: |
examples/datafusion-ffi-example/dist/*
examples/datafusion-ffi-query-planner-example/dist/*

# ============================================
# Build - Linux ARM64
Expand Down
2 changes: 2 additions & 0 deletions .github/workflows/test.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -121,6 +121,8 @@ jobs:
run: |
cd examples/datafusion-ffi-example
uv run --no-project pytest python/tests/_test*.py
cd ../datafusion-ffi-query-planner-example
uv run --no-project pytest python/tests/_test*.py

- name: Run tpchgen-cli to create 1 Gb dataset
if: matrix.wheel-tag == 'abi3'
Expand Down
Loading
Loading