Skip to content

fix: Complete data source form support for all SDK-configurable parameters#6613

Open
ntkathole wants to merge 1 commit into
feast-dev:masterfrom
ntkathole:fix_datasource_gaps
Open

fix: Complete data source form support for all SDK-configurable parameters#6613
ntkathole wants to merge 1 commit into
feast-dev:masterfrom
ntkathole:fix_datasource_gaps

Conversation

@ntkathole

Copy link
Copy Markdown
Member

What this PR does / why we need it:

The Feast UI data source creation/edit forms were missing many configuration parameters that the Python SDK supports. This PR performs a full audit of all 16 data source types and closes the gaps so users can fully configure any data source from the UI without needing to edit Python code.

When creating data sources through the Feast UI, users could only configure a subset of available options. For example:

  • SparkSource only exposed table and path, but not query, file_format, table_format (Iceberg/Delta/Hudi), or date_partition_column
  • Snowflake was missing query and warehouse
  • Kafka/Kinesis were missing message_format/record_format (required for deserialization)
  • File source was missing file_format and s3_endpoint_override
  • Several sources were missing date_partition_column for query optimization

This made it impossible to configure sources like SparkSource(path="...", table_format=IcebergFormat(...)) or RaySource(reader_type="parquet", path="abfss://...") entirely from the UI.

@ntkathole ntkathole self-assigned this Jul 18, 2026
@ntkathole
ntkathole requested a review from a team as a code owner July 18, 2026 07:32
@codecov-commenter

codecov-commenter commented Jul 18, 2026

Copy link
Copy Markdown

⚠️ Please install the 'codecov app svg image' to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

❌ Patch coverage is 55.76923% with 69 lines in your changes missing coverage. Please review.
✅ Project coverage is 46.00%. Comparing base (0da4546) to head (b52cbb3).

Files with missing lines Patch % Lines
sdk/python/feast/api/registry/rest/data_sources.py 54.41% 28 Missing and 3 partials ⚠️
...k/python/feast/api/registry/rest/saved_datasets.py 70.42% 15 Missing and 6 partials ⚠️
sdk/python/feast/dataset_utils.py 0.00% 17 Missing ⚠️
❗ Your organization needs to install the Codecov GitHub app to enable full functionality.
Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #6613      +/-   ##
==========================================
+ Coverage   45.97%   46.00%   +0.03%     
==========================================
  Files         412      412              
  Lines       48793    48947     +154     
  Branches     6904     6939      +35     
==========================================
+ Hits        22431    22517      +86     
- Misses      24813    24872      +59     
- Partials     1549     1558       +9     
Flag Coverage Δ
go-feature-server 30.58% <ø> (ø)
python-unit 47.30% <55.76%> (+0.02%) ⬆️
Files with missing lines Coverage Δ
sdk/python/feast/dataset_utils.py 0.00% <0.00%> (ø)
...k/python/feast/api/registry/rest/saved_datasets.py 48.47% <70.42%> (+7.84%) ⬆️
sdk/python/feast/api/registry/rest/data_sources.py 63.95% <54.41%> (-5.04%) ⬇️

Continue to review full report in Codecov by Harness.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 0da4546...b52cbb3. Read the comment docs.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@ntkathole
ntkathole force-pushed the fix_datasource_gaps branch from f948b44 to 7c4fea8 Compare July 18, 2026 07:37
@ntkathole
ntkathole force-pushed the fix_datasource_gaps branch 2 times, most recently from 5ef6de9 to 3031f02 Compare July 18, 2026 09:19
Signed-off-by: ntkathole <nikhilkathole2683@gmail.com>
@ntkathole
ntkathole force-pushed the fix_datasource_gaps branch from 3031f02 to b52cbb3 Compare July 20, 2026 05:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants