Repository navigation
[search results, deferred] Score every vector search by cosine similarity, and name scores for it (port of #1598's cosine half) - #1663
Draft
edwinyyyu wants to merge 5 commits into
Conversation
This was referenced Sep 16, 2026
Merged
Closed
Merged
Closed
edwinyyyu
force-pushed
the
port/cosine-scores-main
branch
from
September 17, 2026 17:39
86fde33 to
df06603
Compare
edwinyyyu
force-pushed
the
port/cosine-scores-main
branch
from
October 14, 2026 00:21
fb38a72 to
1982ad3
Compare
edwinyyyu
added a commit
to edwinyyyu/MemMachine
that referenced
this pull request
Oct 14, 2026
…he core final state Open-or-create leaves the vector store interface; the service locator's tests build the cosine-only search engine of MemMachine#1663, and the SQLite stores' tests keep the cosine-only metric tests of MemMachine#1663. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This was referenced Oct 14, 2026
Draft
edwinyyyu
force-pushed
the
port/cosine-scores-main
branch
from
October 15, 2026 05:46
1982ad3 to
251dc60
Compare
xiongzubiao
force-pushed
the
feat/horizontal-scaling
branch
from
October 15, 2026 15:31
101182c to
cf7ae15
Compare
edwinyyyu
force-pushed
the
port/cosine-scores-main
branch
8 times, most recently
from
October 16, 2026 22:08
4134c43 to
1d6e17f
Compare
…uilt by the composition root A vector store was a factory of logical collections, each identified by a (namespace, name) pair and created with its own dimensions, metric and declared schema; Qdrant and Milvus shared one native collection among logical collections of equal configuration, under a name derived from a hash of that configuration, and the collection registry mapped (namespace, name) to it. A store is now one collection, and what it held are its partitions. `VectorStore` is built with its vector store name, dimensions, similarity metric and declared properties, and every partition shares them; a partition's handle answers the metric and the declared properties. `create_partition(key)`, `open_or_create_partition(key)`, `open_partition(key)` and `delete_partition(key)` take one string key, and `purge_deleted_partitions()` replaces `purge_deleted_collections()`. A partition is the records carrying its incarnation in the store's native collection (Qdrant, Milvus) or a pair of tables (the SQLite stores). "Collection" names only the native Qdrant or Milvus collection, whose create and delete calls keep their names. `startup()` prepares the storage a store's partitions share: the native collection and its indexes, or the SQLite tables. Vector store names match `[a-z0-9_]+` and are at most 32 bytes, the rule property keys follow, so every name works on every backend: the Qdrant store names its collection by the vector store name, and the Milvus store by `sys_` followed by it, since Milvus requires a leading letter or underscore and names its own internals with a leading underscore. The hash-derived native names go. A partition key is any non-empty string of at most 255 bytes free of NUL characters, since no store names a native resource after one. In the vector store's utilities, a check that returns a bool is named as a predicate, `is_valid_identifier` and `is_valid_filter`, and a check that raises starts with `validate_`: `require_declared_types`, `require_dimensions`, `require_valid_limit`, `require_valid_query_vector` and `require_valid_score_threshold` become `validate_declared_types`, `validate_dimensions`, `validate_limit`, `validate_query_vector` and `validate_score_threshold`, the Milvus store's `_require_every_key_accepted` becomes `_validate_every_key_accepted`, and the configuration's `_require_retention_floor`, which checks a store's tombstone retention, becomes `_validate_tombstone_retention`. The collection registry becomes the partition registry, keyed by partition key within one store. Its tables, `partition_registry_pt` and `partition_registry_gc`, are shared by every store on a database and keyed by vector store name, so the registries of several stores share one database and nothing else. Its reservations, registrations, unregistration and purge rounds are the collection registry's, with the partition key in place of (namespace, name): the primary key arbitrates a reservation, a confirmation or cancellation acts on one incarnation, a registration fences on `require_current()`, and a purge round claims a tombstone under a lease with a claim generation and counts its attempts. The registry records no configuration per partition: every partition shares its store's dimensions, metric and declared properties. On the registry-backed base, the storage a partition keeps of its own is prepared by `_prepare_partition_storage(key, incarnation)`, between its reservation and its confirmation; the Qdrant and Milvus stores keep nothing per partition, and a purge round works on one incarnation in the store's own collection. The errors follow the vocabulary: `VectorStorePartitionAlreadyExistsError`, `VectorStorePartitionDeletedError`, `VectorStorePartitionHandleStaleError` and `VectorStorePartitionPendingError`. `DatabaseManager.get_vector_store(backend, vector_store_name=, vector_dimensions=, similarity_metric=, indexed_properties=)` builds and caches one store per (backend, vector store name), starting its registry and then the store; asking for a name again with other dimensions, another metric or other keys is a `VectorStoreConfigurationError`. The resource manager starts a purge sweeper for each store the first time it hands the store out. The Qdrant and Milvus backends name their registry database with `partition_registry`, in place of `collection_registry`. The event backend's store is named by a UUIDv5 of its embedder id in the UUIDv5 of its backend's key in a namespace fixed for the event backend, written as 32 hexadecimal digits, so neither the key nor the id is constrained, and stores on different backends never share a name. The event backend opens a session's partition with `open_or_create_partition`, which retries the races of a partition another worker is creating or deleting. Semantic memory's one store is named `semantic_memory` whatever its embedder and holds every org's features in one partition, `semantic_memory`. The SQLite stores change shape only: their registry tables become `vector_store_sqlite_pt` and `vector_store_sqlite_vec_pt`, keyed by vector store name and partition key, the pending-operation log is keyed the same way, a partition's tables and index file are named by the vector store name and a SHA-256 digest of the partition key, and the stores of one backend share its engine. No migration is provided: existing SQLite vector data, and the Qdrant and Milvus collections and registry tables of earlier releases, are left unread. The lifecycle contract the registry-backed stores satisfy becomes the partition lifecycle contract, and the in-memory test store's collection becomes a partition. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
edwinyyyu
force-pushed
the
port/cosine-scores-main
branch
from
October 19, 2026 17:10
1d6e17f to
6f42ba1
Compare
…nd their partitions The design documents came from MemMachine#1733-MemMachine#1736, where a store holds logical collections addressed by namespace and name, each with its own configuration. A store is now one collection, with its dimensions, metric and declared schema fixed at construction, and a partition is one tenant's records in it, addressed by key. The documents say so: - the collection registry document becomes the partition registry document: a registry belongs to one store and is addressed by partition key; its tables are `partition_registry_pt` and `partition_registry_gc`, and a tombstone needs no location, since the incarnation alone finds a dead partition's records in the store's native collection; a decision records that a store is one collection; - the Qdrant and Milvus documents lay out one native collection per store, named by the vector store name (`sys_` and the name on Milvus), created at startup, where a partition's creation makes nothing in the backend; - the overview, isolation, consistency and purge documents speak of partitions and `purge_deleted_partitions`, and the overview describes the store and its partitions. The measurements and their conditions are unchanged; the purge document names the MemMachine#1734 commits they were taken at. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A partition stored every property of a record and filtered on any key, which made a caller's arbitrary keys part of the store's schema: the SQLite stores kept them in a JSON column and filtered with json_extract, Milvus the keys its schema did not declare in a JSON field, and a filter on a key the store never indexed scanned. Since EventMemory routes a filter on an undeclared key to the segment store, the vector store need not hold undeclared keys at all. A partition now stores the properties its store declares and no others. `upsert` raises UndeclaredPropertyKeyError before anything is sent for a record naming an undeclared key, and PropertyTypeMismatchError for a value of another type than its key declares; `query` raises UndeclaredPropertyKeyError for a filter naming an undeclared key and UnsupportedFilterError for a node outside the partition's `supported_filter_nodes`. Both SQLite stores keep one typed, indexed, nullable column per declared key on the records table (sql_columns.py); sqlite-vec 0.1.9 rejects NULL in a vec0 metadata column and a declared key is optional per record, so that store keeps the columns on the records table and hands the KNN a `rowid IN (SELECT ...)` allowlist, evaluating the filter during the search instead of after it. Qdrant drops the JSON copy and keeps a payload field per declared key; Milvus drops its JSON field and keeps its typed field per declared key. Datetimes are stored as microseconds since the epoch where a backend has no datetime type. Since every key a filter may name is now indexed, the Qdrant store creates its collection in strict mode (`unindexed_filtering_retrieve` and `_update` false, Qdrant Cloud's default): a filter on an unindexed key is refused by the server instead of scanned for. A leaf whose value is of another type than its key declares matches nothing, as on the SQL stores; the Qdrant compiler answers it with a filter no point satisfies, since the server would refuse the condition for the field's index, and Milvus no longer compares an int with a float key. Local mode does not record the setting, so a unit test checks the request and integration tests the server's answer. `declared_schema_contract.py` states the contract every backend's test module runs: which records a filtered search admits, over fixtures small enough that every backend searches them exactly, checked after each upsert so an approximate index fails on recall, by name, and not on the filter. On the registry-backed stores, the checks are the base handle's: `upsert` runs validate_declared_properties in place of the type check it ran, and `query` runs validate_supported_filter against the subclass's `supported_filter_nodes`, which each subclass now implements. A datetime column on the SQLite stores has a `tz_<key>` column beside it holding the UTC offset in seconds, written with the value and read by no filter, so a stored datetime is the value written, as on Milvus, Qdrant and the segment store; the microseconds column alone would keep only the instant. The Qdrant and Milvus design documents describe the declared-only properties and Qdrant's strict mode. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ESpWYTmCR7X3bJEpoA8SAn
A declared datetime on the SQLite stores had a `tz_<key>` column beside its microseconds column, holding the UTC offset in seconds that no filter reads. The vector stores keep a datetime property's instant only, as MemMachine#1736 now does for Milvus and MemMachine#1788 for Qdrant, and the segment store keeps the offset, so the column, `offset_column_name`, and the value written to it go: each declared property is one column, and a datetime stays microseconds since the epoch. The tests read a stored datetime back as its instant in UTC, and the roundtrip test checks that the stored instant equals the written one. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ESpWYTmCR7X3bJEpoA8SAn
Every embedder MemMachine ships produces vectors meant to be compared by cosine -- OpenAI hard-coded it, Bedrock defaulted to it, SentenceTransformer only reported what the model declared -- while every layer that touched a score paid for the other three metrics in direction flags, threshold directions, and per-backend tables mapping the enum onto native metric names. `SimilarityMetric` is gone; scores are cosine similarities in [-1, 1] and the names say so: `QueryMatch.score` and `SearchMatch.score` become `cosine_similarity`, and `query(score_threshold=)` becomes `query(min_cosine_similarity=)`, which no longer needs a direction to be meaningful and is refused when not finite, as the threshold was. A store and its partitions no longer have a `similarity_metric`, the schema a partition is registered under no longer records one, and a search engine factory takes the dimensions alone. The Bedrock embedder's `similarity_metric` config key and semantic memory's `vector_similarity_metric` go with it, and the install and configuration docs drop them. The vector graph stores carried a metric per stored embedding, as a companion property beside every vector; that is gone and `Node.embeddings` holds plain vectors. NebulaGraph's `cosine()` cannot take `APPROXIMATE` and its vector indexes offer only L2 and IP. Cosine similarity between unit vectors is their inner product, so embeddings are normalized on the way in and compared with `inner_product()` against an IP index. The cosine half of MemMachine#1598 (`abf92a3a4` on speedkick), re-derived on the one-collection store: the rest of MemMachine#1598 (queries answer record UUIDs and scores, no `get`, semantic memory's `vector_uuid`) and all of MemMachine#1603 are in registry-backed base and its partition handle lose the metric, the stores' own threshold checks become `validate_min_cosine_similarity`, and the design documents describe cosine scoring and a schema without a metric. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ESpWYTmCR7X3bJEpoA8SAn
edwinyyyu
force-pushed
the
port/cosine-scores-main
branch
from
October 19, 2026 17:30
6f42ba1 to
c14c06d
Compare
This was referenced Oct 20, 2026
Open
Draft
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Important
Deferred: nothing is to build on this PR until the choice of similarity metrics is decided. It stays a draft on [declared schema 1/2] (#1628) at
c14c06d6e.Port
The cosine half of #1598, merged into
speedkickasabf92a3a4, re-derived on the partitioned store of [vector store scale-out 6] rather than cherry-picked. The rest of #1598 (queries answer record UUIDs and scores, noget, semantic memory'svector_uuid) and all of #1603 are in #1733 already, so this PR carries only the metric removal and its renames. Changes the original could not make are made here: the registry-backed base and its partition handle lose the metric, the finiteness check of a threshold becomesvalidate_min_cosine_similarity, and the Qdrant and Milvus design documents describe cosine scoring. It sits on [declared schema 1/2] (#1628), so it also takes the metric out of that PR's code: sqlite-vec's and Milvus's score paths and the docstrings that name the threshold.Purpose of the change
The vector store carried four similarity metrics and shipped one. Every embedder here produces vectors meant to be compared by cosine — OpenAI hard-coded it, Bedrock defaulted to it, SentenceTransformer only reported what the model declared — while every layer that touched a score paid for the other three in direction flags, threshold directions, and per-backend tables mapping the enum onto native metric names.
Description
SimilarityMetricis gone. Scores are cosine similarities in[-1, 1], and the names say so:QueryMatch.scoreandSearchMatch.score→cosine_similarity,query(score_threshold=)→query(min_cosine_similarity=), which no longer needs a direction to be meaningful and is refused when not finite, as the threshold was. A store and its partitions no longer have asimilarity_metric, and a search engine factory takes the dimensions alone. The Bedrock embedder'ssimilarity_metricconfig key and semantic memory'svector_similarity_metricgo with it, and the install and configuration docs drop them.Two consequences beyond the vector store
The vector graph stores carried a metric per stored embedding, as a companion property beside every vector. That is gone;
Node.embeddingsholds plain vectors.NebulaGraph's
cosine()cannot takeAPPROXIMATEand its vector indexes offer only L2 and IP. Cosine similarity between unit vectors is their inner product, so embeddings are normalized on the way in and compared withinner_product()against an IP index: cosine ranking that, unlikecosine(), can be approximate. Normalization happens in_vector_to_gql_literal, the one place any vector becomes a GQL literal, so the stored side and the query side cannot disagree. Its tests skip everywhere, CI included — they need a real NebulaGraph server rather than a testcontainer — so this is the least-exercised part of the change and wants a reviewer who can run against a licensed 5.x server.Breaking
A configuration that still names
similarity_metricorvector_similarity_metrichas the key ignored.Stack
24 open PRs: three independent PRs, and the vector store tree of short parallel branches. Every open PR in the tree has
feat/horizontal-scalingas its GitHub base, and the independent PRs havemain. The branches are in a fork, and a pull request can target only this repository's branches, so the on column gives the order the PRs build on each other. A stacked PR's diff on GitHub includes the PRs under it until they merge.Independent of the vector store tree, directly on
main:mainmainmainThe vector store tree. Each PR builds on the one in its on column; PRs on the same parent are parallel branches and do not depend on each other. #1631 is closed, superseded by #1733–#1736, which hold its changes split in four, with review changes since. #1675 is closed, superseded by #1627. #1841 is closed: its request packing is dropped, and its cancellation contract is in #1840. #1828 and #1839 sit side by side on the branch; #1839 and #1840 are the event memory recovery stack, and #1684 heads the event memory stack, which its own PRs index, listed because it builds on #1840, its branch also carrying #1828's commits until one of the two merges; #1871 keeps its own Stack section. Until the PRs under it merge, their changes show in a stacked PR's diff.
mainfeat/horizontal-scaling)feat/horizontal-scalingfeat/horizontal-scaling)feat/horizontal-scalingfeat/horizontal-scaling)feat/horizontal-scalingfeat/horizontal-scaling)feat/horizontal-scalingfeat/horizontal-scaling)feat/horizontal-scalingfeat/horizontal-scaling)feat/horizontal-scalingfeat/horizontal-scaling)feat/horizontal-scalingfeat/horizontal-scaling)feat/horizontal-scalingfeat/horizontal-scaling)feat/horizontal-scalingfeat/horizontal-scaling)feat/horizontal-scalingfeat/horizontal-scaling)feat/horizontal-scalingfeat/horizontal-scalingfeat/horizontal-scaling)feat/horizontal-scalingfeat/horizontal-scaling)feat/horizontal-scalingfeat/horizontal-scaling)feat/horizontal-scalingfeat/horizontal-scalingfeat/horizontal-scalingfeat/horizontal-scalingfeat/horizontal-scalingfeat/horizontal-scalingfeat/horizontal-scalingfeat/horizontal-scalingmemmachine-server migrate, which creates every configured store's schema before servers startfeat/horizontal-scalingThis PR is its one commit,
c14c06d6e, stacked on #1628. It stays a draft until the choice of similarity metrics is decided.Verification
At this PR's head
c14c06d6e, on 2026-10-19, its tree is the previous head6f42ba1ea's with design/vector_store_partition_registry.md listing the lookup as "open", the name it has from #1627 until #1625, where it said "get". Checks:ruff checkandruff format --checkclean;ty checkclean as CI runs it (uv run --frozen --all-extras ty check --project packages/server);uv lock --checkclean. Outside that document the tree is the previous head's, so the test runs below stand.Before the partition registry document's wording fix: At this PR's head
6f42ba1ea, on 2026-10-19, its tree is the previous head1d6e17fdc's with #1824 beneath it, which drops the purge queues' unread logical key, and #1627 dropping the same key from the partition registry's tombstones and removing the collection registry design document its partition registry document replaces. Checks:ruff checkandruff format --checkclean;ty checkclean as CI runs it (uv run --frozen --all-extras ty check --project packages/server);uv lock --checkclean. The unit tests pass, 1134 tests (8 skipped).Before the rebase onto #1824's merge: At this PR's head
1d6e17fdc, on 2026-10-16, its tree is the previous head4134c43fe's with the configuration's check of a providedtombstone_retention_secondsnamed_validate_tombstone_retention, where it was_require_retention_floor. Checks:ruff checkandruff format --checkclean;ty checkclean as CI runs it (uv run --frozen --all-extras ty check --project packages/server);uv lock --checkclean. The unit tests pass, 1134 tests (8 skipped).Before the retention check's rename: At this PR's head
4134c43fe, on 2026-10-16, its tree is the previous head203be42e7's with every vector store check of a value that raises namedvalidate_, asvalidate_partition_keyalready was:validate_declared_types,validate_query_vector,validate_score_threshold,validate_limit, andvalidate_dimensionsfrom the base, and the PRs' own checks in their turn. Checks:ruff checkandruff format --checkclean;ty checkclean as CI runs it (uv run --frozen --all-extras ty check --project packages/server);uv lock --checkclean. The unit tests pass, 1134 tests (8 skipped).Before the raising checks took the
validate_prefix: At this PR's head203be42e7, on 2026-10-16, its tree is the previous head8866cb86a's with #1627's changes: theVectorStoreinterface dropsvector_store_name,vector_dimensions,similarity_metric, andindexed_properties, and the partition handle dropspartition_key; a partition key may be any string of up to 255 bytes without NUL, and the SQLite stores name a partition's tables and index file by a SHA-256 digest of it; and the checks that return a bool areis_valid_identifierandis_valid_filter. Checks:ruff checkandruff format --checkclean;ty checkclean as CI runs it (uv run --frozen --all-extras ty check --project packages/server);uv lock --checkclean. This PR's version of the database manager's check compares the dimensions and declared properties each store was built with. The unit tests pass, 1134 tests (8 skipped).Before the store's properties left its interface: At this PR's head
8866cb86a, on 2026-10-15: its tree is the previous head0cc31da14's without #1627's per-partition schema record (PartitionSchema, the registry's and the SQLite stores'schemacolumn, andVectorStorePartitionSchemaMismatchError), and withrequire_partition_keynamedvalidate_partition_key;ruff checkandruff format --checkclean;ty checkclean as CI runs it (uv run --frozen --all-extras ty check --project packages/server);uv lock --checkclean. This PR's own wording of the record in the design documents anddata_types.py, which dropped the metric from it, goes with it. The unit tests pass, 1128 tests.Before the per-partition schema record went: At this PR's head
0cc31da14, on 2026-10-15: its tree is the previous headdd32e27c4's with the partition lookup namedopen_partitionand the segment store'sclose_partitionkept, since #1625 renames the one and removes the other;ruff checkandruff format --checkclean;ty checkclean as CI runs it (uv run --frozen --all-extras ty check --project packages/server);uv lock --checkclean.Before the lookup's rename moved into #1625: At this PR's head
dd32e27c4, on 2026-10-15: its tree is the previous head3f274304b's without the segment store'sclose_partition;ruff checkandruff format --checkclean;ty checkclean as CI runs it (uv run --frozen --all-extras ty check --project packages/server);uv lock --checkclean.Before #1627's first commit took the removal of
close_partition: At this PR's head3f274304b, on 2026-10-15, restacked on #1627's rewritten history: its tree is the previous head250f6d5a0's;ruff checkandruff format --checkclean;ty checkclean as CI runs it (uv run --frozen --all-extras ty check --project packages/server);uv lock --checkclean.Before #1627's history was rewritten: At this PR's head
250f6d5a0, on 2026-10-15, rebased ontofeat/horizontal-scalingatcf7ae1579, which carriesmainthrough #1774:ruff checkandruff format --checkclean;ty checkclean as CI runs it (uv run --frozen --all-extras ty check --project packages/server);uv lock --checkclean; the reranker manager's and the Cohere reranker's unit tests pass, 17 tests.Before the rebase onto
cf7ae1579: At this PR's head251dc6060, on 2026-10-15, rebased ontofeat/horizontal-scalingwith #1702 and #1670 merged (101182c18):ruff checkandruff format --checkclean;ty checkclean as CI runs it (uv run --frozen --all-extras ty check --project packages/server);uv lock --checkclean.Before the rebase onto #1670's merge: At this PR's head
1982ad3d9, on 2026-10-13, rebuilt on #1702's6631b5c77:ruff checkandruff format --checkclean;ty checkclean as CI runs it (uv run --frozen --all-extras ty check --project packages/server);uv lock --checkclean; the episodic memory and filter tests pass, 476 tests (1 skipped).Before the rebuild on #1702's filter routing: At this PR's head
fb38a726f, on 2026-10-09, onfeat/horizontal-scalingwith #1813 merged:ruff checkandruff format --checkclean;ty checkclean as CI runs it (uv run --frozen --all-extras ty check --project packages/server);uv lock --checkclean.Before the rebase onto #1813's merge: At this PR's head
95afd369f, on 2026-10-09:ruff checkandruff format --checkclean;ty checkclean as CI runs it (uv run --frozen --all-extras ty check --project packages/server);uv lock --checkclean. Its tree is the previous #1628 headd6e25ccd3's, which carried #1663 and #1628 together: the server suite without integration tests last passed on that change at6524b2a62, on 2026-10-08, 2110 tests, and the integration tests of the vector stores, the resource manager, episodic memory, and semantic storage at54b6acbce, on 2026-10-08, 689 tests, against PostgreSQL 16, Neo4j, Qdrant 1.19.1, and Milvus 2.6.24 in containers; the SQLite stores' tests at49798fdda, on 2026-10-09, 166 tests.🤖 Generated with Claude Code
https://claude.ai/code/session_01ESpWYTmCR7X3bJEpoA8SAn