Skip to content

Commit b0da945

Browse files
franciscojavierarceodevin-ai-integration[bot]
authored and
Rob Howley
committed
feat: Add data labeling tabs to UI (feast-dev#5410)
* Add GenAI documentation page to Introduction section Co-Authored-By: Francisco Javier Arceo <farceo@redhat.com> * Move GenAI page to getting-started directory and update SUMMARY.md Co-Authored-By: Francisco Javier Arceo <farceo@redhat.com> * Update SUMMARY.md * hell 3.12.7 :wq d unstructured data transformation and Spark integration details to GenAI documentation Co-Authored-By: Francisco Javier Arceo <farceo@redhat.com> * Update genai.md * Rename Document Labeling to Data Labeling with blue icon - Update sidebar navigation to show 'Data Labeling' instead of 'Document Labeling' - Add blue color (#006BB4) to Data Labeling icon to match other navbar icons - Update route from 'document-labeling' to 'data-labeling' - Update page title from 'Document Labeling for RAG' to 'Data Labeling for RAG' - Update custom tab types from DocumentLabeling to DataLabeling - Update test document text to reference 'data labeling functionality' Co-Authored-By: Francisco Javier Arceo <arceofrancisco@gmail.com> Signed-off-by: Devin AI <devin-ai-integration[bot]@users.noreply.github.com> Co-Authored-By: Francisco Javier Arceo <arceofrancisco@gmail.com> * Add tabbed interface to Data Labeling with RAG and Classification tabs - Implement separate RAG and Classification tabs for Data Labeling page - Add RAG Context section with prompt and query text areas - Separate chunk extraction and generation labels into distinct H2 sections - Keep existing 'Label Selected Text' button for chunk extraction - Add long text area for ground truth label in 'Label for Generation' section - Implement Classification tab with CSV data loading and editing functionality - Maintain all existing text selection and highlighting functionality - Follow established UI patterns using EUI components Co-Authored-By: Francisco Javier Arceo <arceofrancisco@gmail.com> Co-Authored-By: Francisco Javier Arceo <arceofrancisco@gmail.com> Signed-off-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * Fix save functionality and improve RagTab layout - Simplify save function with setTimeout to avoid protobuf errors - Improve filename extraction for JSON download - Maintain conditional rendering of RAG Context after document loading - Keep existing layout with Step 1 and Step 2 sections - Preserve 'Label Selected Text' button functionality Signed-off-by: Devin AI <devin-ai-integration[bot]@users.noreply.github.com> Co-Authored-By: Francisco Javier Arceo <arceofrancisco@gmail.com> Co-Authored-By: Francisco Javier Arceo <arceofrancisco@gmail.com> * Fix lint-python and unit-test-ui formatting issues - Fix import sorting in feature_server.py (ruff I001) - Remove trailing comma in RagTab.tsx imports - Resolve CI formatting failures Signed-off-by: Devin AI <devin-ai-integration[bot]@users.noreply.github.com> Co-Authored-By: Francisco Javier Arceo <arceofrancisco@gmail.com> Co-Authored-By: Francisco Javier Arceo <arceofrancisco@gmail.com> --------- Signed-off-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> Signed-off-by: Rob Howley <rhowley@seatgeek.com>
1 parent d183c4b commit b0da945

10 files changed

Lines changed: 1019 additions & 476 deletions

File tree

‎docs/getting-started/genai.md‎

Lines changed: 0 additions & 93 deletions
Original file line numberDiff line numberDiff line change
@@ -65,99 +65,6 @@ Feast supports transformations that can be used to:
6565
* Normalize and preprocess features before serving to LLMs
6666
* Apply custom transformations to adapt features for specific LLM requirements
6767

68-
## Getting Started with Feast for GenAI
69-
70-
### Installation
71-
72-
To use Feast with vector database support, install with the appropriate extras:
73-
74-
```bash
75-
# For Milvus support
76-
pip install feast[milvus,nlp]
77-
78-
# For Elasticsearch support
79-
pip install feast[elasticsearch]
80-
81-
# For Qdrant support
82-
pip install feast[qdrant]
83-
84-
# For SQLite support (Python 3.10 only)
85-
pip install feast[sqlite_vec]
86-
```
87-
88-
### Configuration
89-
90-
Configure your feature store to use a vector database as the online store:
91-
92-
```yaml
93-
project: genai-project
94-
provider: local
95-
registry: data/registry.db
96-
online_store:
97-
type: milvus
98-
path: data/online_store.db
99-
vector_enabled: true
100-
embedding_dim: 384 # Adjust based on your embedding model
101-
index_type: "IVF_FLAT"
102-
103-
offline_store:
104-
type: file
105-
entity_key_serialization_version: 3
106-
```
107-
108-
### Defining Vector Features
109-
110-
Create feature views with vector index support:
111-
112-
```python
113-
from feast import FeatureView, Field, Entity
114-
from feast.types import Array, Float32, String
115-
116-
document = Entity(
117-
name="document_id",
118-
description="Document identifier",
119-
join_keys=["document_id"],
120-
)
121-
122-
document_embeddings = FeatureView(
123-
name="document_embeddings",
124-
entities=[document],
125-
schema=[
126-
Field(
127-
name="vector",
128-
dtype=Array(Float32),
129-
vector_index=True, # Enable vector search
130-
vector_search_metric="COSINE", # Similarity metric
131-
),
132-
Field(name="document_id", dtype=String),
133-
Field(name="content", dtype=String),
134-
],
135-
source=document_source,
136-
ttl=timedelta(days=30),
137-
)
138-
```
139-
140-
### Retrieving Similar Documents
141-
142-
Use the `retrieve_online_documents_v2` method to find similar documents:
143-
144-
```python
145-
# Generate query embedding
146-
query = "How does Feast support vector databases?"
147-
query_embedding = embed_text(query) # Your embedding function
148-
149-
# Retrieve similar documents
150-
context_data = store.retrieve_online_documents_v2(
151-
features=[
152-
"document_embeddings:vector",
153-
"document_embeddings:document_id",
154-
"document_embeddings:content",
155-
],
156-
query=query_embedding,
157-
top_k=3,
158-
distance_metric='COSINE',
159-
).to_df()
160-
```
16168
## Use Cases
16269

16370
### Document Question-Answering

‎sdk/python/feast/feature_server.py‎

Lines changed: 26 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -105,6 +105,11 @@ class ReadDocumentRequest(BaseModel):
105105
file_path: str
106106

107107

108+
class SaveDocumentRequest(BaseModel):
109+
file_path: str
110+
data: dict
111+
112+
108113
def _get_features(request: GetOnlineFeaturesRequest, store: "feast.FeatureStore"):
109114
if request.feature_service:
110115
feature_service = store.get_feature_service(
@@ -375,6 +380,27 @@ async def read_document_endpoint(request: ReadDocumentRequest):
375380
except Exception as e:
376381
return {"error": str(e)}
377382

383+
@app.post("/save-document")
384+
async def save_document_endpoint(request: SaveDocumentRequest):
385+
try:
386+
import json
387+
import os
388+
from pathlib import Path
389+
390+
file_path = Path(request.file_path).resolve()
391+
if not str(file_path).startswith(os.getcwd()):
392+
return {"error": "Invalid file path"}
393+
394+
base_name = file_path.stem
395+
labels_file = file_path.parent / f"{base_name}-labels.json"
396+
397+
with open(labels_file, "w", encoding="utf-8") as file:
398+
json.dump(request.data, file, indent=2, ensure_ascii=False)
399+
400+
return {"success": True, "saved_to": str(labels_file)}
401+
except Exception as e:
402+
return {"error": str(e)}
403+
378404
@app.get("/chat")
379405
async def chat_ui():
380406
# Serve the chat UI

‎sdk/python/feast/ui_server.py‎

Lines changed: 26 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -7,10 +7,16 @@
77
from fastapi import FastAPI, Response
88
from fastapi.middleware.cors import CORSMiddleware
99
from fastapi.staticfiles import StaticFiles
10+
from pydantic import BaseModel
1011

1112
import feast
1213

1314

15+
class SaveDocumentRequest(BaseModel):
16+
file_path: str
17+
data: dict
18+
19+
1420
def get_app(
1521
store: "feast.FeatureStore",
1622
project_id: str,
@@ -76,6 +82,26 @@ def read_registry():
7682
media_type="application/octet-stream",
7783
)
7884

85+
@app.post("/save-document")
86+
async def save_document_endpoint(request: SaveDocumentRequest):
87+
try:
88+
import os
89+
from pathlib import Path
90+
91+
file_path = Path(request.file_path).resolve()
92+
if not str(file_path).startswith(os.getcwd()):
93+
return {"error": "Invalid file path"}
94+
95+
base_name = file_path.stem
96+
labels_file = file_path.parent / f"{base_name}-labels.json"
97+
98+
with open(labels_file, "w", encoding="utf-8") as file:
99+
json.dump(request.data, file, indent=2, ensure_ascii=False)
100+
101+
return {"success": True, "saved_to": str(labels_file)}
102+
except Exception as e:
103+
return {"error": str(e)}
104+
79105
# For all other paths (such as paths that would otherwise be handled by react router), pass to React
80106
@app.api_route("/p/{path_name:path}", methods=["GET"])
81107
def catch_all():

‎ui/src/FeastUISansProviders.tsx‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -147,7 +147,7 @@ const FeastUISansProvidersInner = ({
147147
element={<DatasetInstance />}
148148
/>
149149
<Route
150-
path="document-labeling/"
150+
path="data-labeling/"
151151
element={<DocumentLabelingPage />}
152152
/>
153153
<Route path="permissions/" element={<PermissionsIndex />} />

‎ui/src/custom-tabs/types.ts‎

Lines changed: 6 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -136,18 +136,18 @@ interface DatasetCustomTabRegistrationInterface
136136
}: DatasetCustomTabProps) => JSX.Element;
137137
}
138138

139-
// Type for Document Labeling Custom Tabs
140-
interface DocumentLabelingCustomTabProps {
139+
// Type for Data Labeling Custom Tabs
140+
interface DataLabelingCustomTabProps {
141141
id: string | undefined;
142142
feastObjectQuery: RegularFeatureViewQueryReturnType;
143143
}
144-
interface DocumentLabelingCustomTabRegistrationInterface
144+
interface DataLabelingCustomTabRegistrationInterface
145145
extends CustomTabRegistrationInterface {
146146
Component: ({
147147
id,
148148
feastObjectQuery,
149149
...args
150-
}: DocumentLabelingCustomTabProps) => JSX.Element;
150+
}: DataLabelingCustomTabProps) => JSX.Element;
151151
}
152152

153153
export type {
@@ -171,6 +171,6 @@ export type {
171171
FeatureCustomTabProps,
172172
DatasetCustomTabRegistrationInterface,
173173
DatasetCustomTabProps,
174-
DocumentLabelingCustomTabRegistrationInterface,
175-
DocumentLabelingCustomTabProps,
174+
DataLabelingCustomTabRegistrationInterface,
175+
DataLabelingCustomTabProps,
176176
};

‎ui/src/pages/Sidebar.tsx‎

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -132,13 +132,13 @@ const SideNav = () => {
132132
isSelected: useMatchSubpath(`${baseUrl}/data-set`),
133133
},
134134
{
135-
name: "Document Labeling",
136-
id: htmlIdGenerator("documentLabeling")(),
137-
icon: <EuiIcon type="documentEdit" />,
135+
name: "Data Labeling",
136+
id: htmlIdGenerator("dataLabeling")(),
137+
icon: <EuiIcon type="documentEdit" color="#006BB4" />,
138138
renderItem: (props) => (
139-
<Link {...props} to={`${baseUrl}/document-labeling`} />
139+
<Link {...props} to={`${baseUrl}/data-labeling`} />
140140
),
141-
isSelected: useMatchSubpath(`${baseUrl}/document-labeling`),
141+
isSelected: useMatchSubpath(`${baseUrl}/data-labeling`),
142142
},
143143
{
144144
name: "Permissions",

0 commit comments

Comments
 (0)