-
Notifications
You must be signed in to change notification settings - Fork 3.5k
feat(knowledge): add Ollama embedding provider support #3714
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
teedonk
wants to merge
73
commits into
simstudioai:staging
Choose a base branch
from
teedonk:feat/ollama-embedding-support
base: staging
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from 1 commit
Commits
Show all changes
73 commits
Select commit
Hold shift + click to select a range
255640f
feat(knowledge): add Ollama embedding types
teedonk b043bc2
feat(knowledge): add per-KB dynamic pgvector tables
teedonk 61f05a7
feat(knowledge): add Ollama embedding generation with retry and smart…
teedonk 546dd7c
feat(knowledge): store ollamaBaseUrl in KB config
teedonk 616761d
feat(chunkers): add embeddingModel to ChunkerOptions
teedonk 133f326
feat(chunkers): add model-aware token estimation ratio
teedonk 2693251
feat(knowledge): pass embeddingModel to all chunkers
teedonk 18e7ac2
feat(knowledge): add Ollama chunk size and overlap capping
teedonk 983efc3
feat(knowledge): add Ollama model validation and auto-detect dimension
teedonk 53a1423
feat(knowledge): update KB detail API for Ollama support
teedonk 0b5d218
feat(knowledge): add provider routing and cross-provider score normal…
teedonk 606b70b
feat(knowledge): add Ollama provider selection UI
teedonk da36fcd
feat(knowledge): add Ollama params to create KB hook
teedonk b9e6ab7
test(knowledge): update KB detail tests for Ollama support
teedonk b1e92b8
test(knowledge): update search tests for provider routing
teedonk 3698a04
fix(knowledge): separate validation from runtime model info to preven…
teedonk 988158e
fix(knowledge): parameterize query vector and accept transaction handle
teedonk 166a7f3
fix(knowledge): wrap Ollama delete+insert in transaction with status …
teedonk 2f30934
fix(knowledge): clean up orphaned KB on table creation failure
teedonk f88e9f9
fix(knowledge): replace native select with project Select component
teedonk 863e497
fix(knowledge): sort and trim Ollama results to topK
teedonk 546061e
fix(knowledge): restrict Ollama base URL to localhost and private net…
teedonk 00b3c7d
fix(knowledge): filter deleted documents from Ollama search and dedup…
teedonk 075b005
fix(knowledge): use OLLAMA_URL env var and allow Docker hostnames in …
teedonk ea59193
fix(knowledge): align dynamic table SQL types with shared schema
teedonk ee3cc30
fix(knowledge): remove hardcoded OpenAI defaults from updateKnowledge…
teedonk e6d0a60
fix(knowledge): add enabled field and fix token ratio for Ollama embe…
teedonk 0812f3b
fix(knowledge): remove immutable fields from update schema
teedonk fd8d2b3
fix(knowledge): strengthen SSRF validation for Ollama base URL
teedonk 5c872c4
fix(knowledge): remove dead code and fix Record type in search route
teedonk 4571299
fix(knowledge): add missing dynamic-tables mock in test
teedonk 322dc4e
fix(knowledge): block IPv6-mapped IPv4 SSRF bypass and fix ::1 hostna…
teedonk ef84871
fix(knowledge): use KB embedding model for search and fix single-resu…
teedonk d308fe0
fix(knowledge): preserve ollamaBaseUrl when updating chunkingConfig
teedonk aa452f4
fix(knowledge): validate Ollama auto-detected dimension against bounds
teedonk 8445d7e
merge: resolve conflicts with upstream staging
teedonk 185007a
fix(knowledge): prevent SSRF bypass via hostname prefix matching on d…
teedonk 456eaa4
resolve merge conflict in create-base-modal
teedonk 1570b02
fix(knowledge): validate dimension before sql.raw interpolation
teedonk 0e1dcf7
fix(knowledge): remove any casts in search route
teedonk e2b8189
Merge remote-tracking branch 'origin/staging' into feat/ollama-embedd…
teedonk ea3dd08
fix(knowledge): add missing document filters to Ollama search queries
teedonk 24779a7
fix(knowledge): preserve Ollama embedding table on soft delete
teedonk 547de40
fix(knowledge): wrap BETWEEN compound conditions in parentheses
teedonk 7afb708
fix(knowledge): wrap BETWEEN compound conditions in parentheses
teedonk 2cdb519
fix(knowledge): add retry to Ollama search embedding generation
teedonk 507cc36
docs(knowledge): clarify soft-delete table retention rationale
teedonk 50858d4
fix(knowledge): validate UUID format in kbTableName
teedonk 5bdfe15
chore: merge staging into feat/ollama-embedding-support
teedonk f6d121e
fix(knowledge): hard-delete KB row on creation rollback
teedonk d210669
fix(knowledge): use hardDeleteKnowledgeBase in cleanup path
teedonk ff08fb0
fix(knowledge): align drizzle schema id type to uuid
teedonk 5cebdea
fix(knowledge): clamp single-result distance instead of forcing zero
teedonk 2552edc
fix: use global score normalization across all providers
teedonk c6fde92
fix: validate embedding count matches chunk count before insert
teedonk 71b1769
fix: prevent NaN on Ollama dimension input
teedonk 61d7936
fix: correct overlap chunk size unit in JSDoc comment
teedonk 1991604
fix: only normalize scores when mixing OpenAI and Ollama providers
teedonk dbabedd
fix: batch Ollama embeddings by item count, not cumulative chars
teedonk 0f42820
fix: resolve merge conflicts with staging
teedonk 9e3d8ce
fix: rename kbModelName to avoid duplicate variable declaration
teedonk dec517d
fix: restrict .internal SSRF allowlist to host.docker.internal only
teedonk 551dff9
fix: remove dead generateSearchEmbedding re-export from utils.ts
teedonk 952de73
fix: model name prefix, SSRF re-validation, and chunk KB config lookup
teedonk 97c0c71
fix: route chunk ops to per-KB table for Ollama and tighten SSRF allo…
teedonk 578171c
fix: normalize cross-provider scores per-provider instead of globally
teedonk e89e3c2
fix: null-coalesce embeddingDimension and update stale error message
teedonk c527867
fix: validate resolved Ollama URL including env fallback against SSRF…
teedonk 7bcee72
fix: guard Ollama SSRF check by provider and skip normalization for s…
teedonk 5638d3a
fix: normalize IPv6 hostname brackets and validate resolved Ollama UR…
teedonk 2c8cbb4
fix: use provider-specific token estimation for manual chunks
teedonk 08e2b24
fix: add SSRF guard inside generateEmbeddings and generateSearchEmbed…
teedonk 045824a
fix: null guard on KB lookup and Ollama-aware token estimation in Jso…
teedonk File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
fix: only normalize scores when mixing OpenAI and Ollama providers
- Loading branch information
commit 19916045cc78446ca80afefbb2a3c70eae2b34a6
Some comments aren't visible on the classic Files Changed page.
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.