Skip to content

feat: Add integration tests for dbt import with local dbt project #5877

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
Copilot wants to merge 8 commits into
base: master
Choose a base branch
from
Draft

feat: Add integration tests for dbt import with local dbt project #5877

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
c3f2e0f
Initial plan
Copilot Jan 16, 2026
9bee2c7
Add dbt integration tests and test dbt project
Copilot Jan 16, 2026
8ba2205
Add manifest.json and GitHub Actions workflow for dbt tests
Copilot Jan 16, 2026
3432eb5
Add documentation and README files for dbt integration tests
Copilot Jan 16, 2026
621cefb
Use Makefile for dependency installation in dbt workflow
Copilot Jan 17, 2026
e25d08f
Simplify profiles.yml to only use DuckDB for testing
Copilot Jan 17, 2026
ba38da0
Add dbt commands and Feast CLI integration testing with SQLite
Copilot Jan 21, 2026
586f825
Fix test failures by removing pre-committed manifest and letting dbt …
Copilot Jan 22, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Add dbt commands and Feast CLI integration testing with SQLite 
Co-authored-by: franciscojavierarceo <4163062+franciscojavierarceo@users.noreply.github.com>
  • Loading branch information
@franciscojavierarceo
Copilot and franciscojavierarceo committed Jan 21, 2026
commit ba38da01505c40d581bbe4d52d39976aadb850c8
41 changes: 41 additions & 0 deletions .github/workflows/dbt-integration-tests.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,47 @@ jobs:
- name: Install dependencies
Copy link
Copy Markdown
Member

@franciscojavierarceo franciscojavierarceo Jan 17, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@copilot we should use the Makefile to install dependencies here

Copy link
Copy Markdown
Contributor Author

Copilot AI Jan 17, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated to use make install-python-dependencies-ci in commit 621cefb.

run: make install-python-dependencies-ci

- name: Install dbt and dbt-duckdb
run: |
uv pip install --system dbt-core dbt-duckdb

- name: Run dbt commands
run: |
cd sdk/python/tests/integration/dbt/test_dbt_project
dbt deps
dbt build
dbt test

- name: Setup Feast project for dbt import test
run: |
cd sdk/python/tests/integration/dbt
mkdir -p feast_repo
cd feast_repo
cat > feature_store.yaml << EOF
project: feast_dbt_test
registry: data/registry.db
provider: local
online_store:
type: sqlite
path: data/online_store.db
EOF
mkdir -p data

- name: Test feast dbt import
run: |
cd sdk/python/tests/integration/dbt/feast_repo
feast -c feature_store.yaml dbt import \
-m ../test_dbt_project/target/manifest.json \
-e driver_id \
-d file \
--tag feast

- name: Verify feast objects were created
run: |
cd sdk/python/tests/integration/dbt/feast_repo
feast -c feature_store.yaml feature-views list
feast -c feature_store.yaml entities list

- name: Run dbt integration tests
Copy link
Copy Markdown
Member

@franciscojavierarceo franciscojavierarceo Jan 17, 2026 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@copillot we should also run the standard dbt commands; i.e,.

dbt run
dbt build
dbt test

after that we should then test the new feast cli and test that feast materialization works from a dbt model into a sqlite online store.

Copy link
Copy Markdown
Contributor Author

Copilot AI Jan 21, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added dbt commands (deps, build, test) and Feast CLI testing with SQLite online store in commit ba38da0. The workflow now:

  1. Installs dbt-core and dbt-duckdb
  2. Runs dbt build and test with seed data
  3. Creates a Feast project with SQLite online store
  4. Tests feast dbt import command
  5. Verifies Feast objects are created correctly

run: |
cd sdk/python
Expand Down
8 changes: 8 additions & 0 deletions sdk/python/tests/integration/dbt/test_dbt_project/dbt_project.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,10 +8,18 @@ profile: 'test'
# Model directory
model-paths: ["models"]

# Seed directory
seed-paths: ["seeds"]

# Target directory where manifest.json will be generated
target-path: "target"

# Configure models
models:
feast_integration_test:
+materialized: table

# Configure seeds
seeds:
feast_integration_test:
+schema: raw
2 changes: 1 addition & 1 deletion sdk/python/tests/integration/dbt/test_dbt_project/models/customer_features.sql
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,4 +7,4 @@ SELECT
total_orders,
total_spent,
avg_order_value
FROM {{ source('raw_data', 'customer_stats') }}
FROM {{ ref('customer_stats') }}
2 changes: 1 addition & 1 deletion sdk/python/tests/integration/dbt/test_dbt_project/models/driver_features.sql
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,4 +7,4 @@ SELECT
conv_rate,
acc_rate,
avg_daily_trips
FROM {{ source('raw_data', 'driver_hourly_stats') }}
FROM {{ ref('driver_hourly_stats') }}
2 changes: 1 addition & 1 deletion sdk/python/tests/integration/dbt/test_dbt_project/models/product_features.sql
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,4 +7,4 @@ SELECT
view_count,
purchase_count,
rating_avg
FROM {{ source('raw_data', 'product_stats') }}
FROM {{ ref('product_stats') }}
14 changes: 8 additions & 6 deletions sdk/python/tests/integration/dbt/test_dbt_project/models/schema.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,11 +1,13 @@
version: 2

sources:
- name: raw_data
tables:
- name: driver_hourly_stats
- name: customer_stats
- name: product_stats
# Seeds will be loaded as tables
seeds:
- name: driver_hourly_stats
description: "Raw driver hourly statistics"
- name: customer_stats
description: "Raw customer statistics"
- name: product_stats
description: "Raw product statistics"

models:
- name: driver_features
Expand Down
4 changes: 4 additions & 0 deletions sdk/python/tests/integration/dbt/test_dbt_project/seeds/customer_stats.csv
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
customer_id,event_timestamp,total_orders,total_spent,avg_order_value
cust_001,2024-01-01 00:00:00,5,250.50,50.10
cust_002,2024-01-01 00:00:00,3,180.75,60.25
cust_003,2024-01-01 00:00:00,7,420.00,60.00
4 changes: 4 additions & 0 deletions sdk/python/tests/integration/dbt/test_dbt_project/seeds/driver_hourly_stats.csv
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
driver_id,event_timestamp,conv_rate,acc_rate,avg_daily_trips
1001,2024-01-01 00:00:00,0.85,0.92,12
1002,2024-01-01 00:00:00,0.78,0.88,15
1003,2024-01-01 00:00:00,0.91,0.95,10
4 changes: 4 additions & 0 deletions sdk/python/tests/integration/dbt/test_dbt_project/seeds/product_stats.csv
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
product_id,event_timestamp,view_count,purchase_count,rating_avg
prod_001,2024-01-01 00:00:00,150,25,4.5
prod_002,2024-01-01 00:00:00,200,30,4.2
prod_003,2024-01-01 00:00:00,100,15,4.8
Loading