Skip to content

fix: Complete data source form support for all SDK-configurable parameters#6613

Merged
ntkathole merged 1 commit into
feast-dev:masterfrom
ntkathole:fix_datasource_gaps
Jul 23, 2026
Merged

fix: Complete data source form support for all SDK-configurable parameters#6613
ntkathole merged 1 commit into
feast-dev:masterfrom
ntkathole:fix_datasource_gaps

Conversation

@ntkathole

Copy link
Copy Markdown
Member

What this PR does / why we need it:

The Feast UI data source creation/edit forms were missing many configuration parameters that the Python SDK supports. This PR performs a full audit of all 16 data source types and closes the gaps so users can fully configure any data source from the UI without needing to edit Python code.

When creating data sources through the Feast UI, users could only configure a subset of available options. For example:

  • SparkSource only exposed table and path, but not query, file_format, table_format (Iceberg/Delta/Hudi), or date_partition_column
  • Snowflake was missing query and warehouse
  • Kafka/Kinesis were missing message_format/record_format (required for deserialization)
  • File source was missing file_format and s3_endpoint_override
  • Several sources were missing date_partition_column for query optimization

This made it impossible to configure sources like SparkSource(path="...", table_format=IcebergFormat(...)) or RaySource(reader_type="parquet", path="abfss://...") entirely from the UI.

@ntkathole ntkathole self-assigned this Jul 18, 2026
@ntkathole
ntkathole requested a review from a team as a code owner July 18, 2026 07:32
@codecov-commenter

codecov-commenter commented Jul 18, 2026

Copy link
Copy Markdown

⚠️ Please install the 'codecov app svg image' to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

❌ Patch coverage is 55.76923% with 69 lines in your changes missing coverage. Please review.
✅ Project coverage is 45.78%. Comparing base (ddb2e9a) to head (4804c4f).
⚠️ Report is 2 commits behind head on master.

Files with missing lines Patch % Lines
sdk/python/feast/api/registry/rest/data_sources.py 54.41% 28 Missing and 3 partials ⚠️
...k/python/feast/api/registry/rest/saved_datasets.py 70.42% 15 Missing and 6 partials ⚠️
sdk/python/feast/dataset_utils.py 0.00% 17 Missing ⚠️
❗ Your organization needs to install the Codecov GitHub app to enable full functionality.
Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #6613      +/-   ##
==========================================
+ Coverage   45.74%   45.78%   +0.03%     
==========================================
  Files         414      414              
  Lines       49700    49854     +154     
  Branches     7087     7122      +35     
==========================================
+ Hits        22737    22824      +87     
- Misses      25377    25436      +59     
- Partials     1586     1594       +8     
Flag Coverage Δ
go-feature-server 30.58% <ø> (ø)
python-unit 47.03% <55.76%> (+0.03%) ⬆️
Files with missing lines Coverage Δ
sdk/python/feast/dataset_utils.py 0.00% <0.00%> (ø)
...k/python/feast/api/registry/rest/saved_datasets.py 48.51% <70.42%> (+7.51%) ⬆️
sdk/python/feast/api/registry/rest/data_sources.py 63.95% <54.41%> (-5.04%) ⬇️

... and 1 file with indirect coverage changes


Continue to review full report in Codecov by Harness.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 2b41ded...4804c4f. Read the comment docs.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@ntkathole
ntkathole force-pushed the fix_datasource_gaps branch from f948b44 to 7c4fea8 Compare July 18, 2026 07:37
@ntkathole
ntkathole force-pushed the fix_datasource_gaps branch 4 times, most recently from b52cbb3 to b6e4a08 Compare July 23, 2026 10:58
Signed-off-by: ntkathole <nikhilkathole2683@gmail.com>
@ntkathole
ntkathole force-pushed the fix_datasource_gaps branch from b6e4a08 to 4804c4f Compare July 23, 2026 14:47
@ntkathole
ntkathole merged commit 5d0f7d6 into feast-dev:master Jul 23, 2026
18 of 22 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants