Repository navigation
Conversation
This was referenced Oct 20, 2026
[qdrant options] Let a deployment tune a Qdrant collection's HNSW, optimizers and quantization
#1618
Draft
Draft
edwinyyyu
force-pushed
the
feat/event-memory-partition-handle
branch
from
October 20, 2026 21:07
0b020e6 to
a20dc24
Compare
This was referenced Oct 20, 2026
[event memory 2/6] Rename the segment store to the event memory store (port of #1659's rename)
#1686
Draft
edwinyyyu
force-pushed
the
feat/event-memory-partition-handle
branch
from
October 26, 2026 18:17
a20dc24 to
c494900
Compare
edwinyyyu
force-pushed
the
feat/event-memory-partition-handle
branch
8 times, most recently
from
November 2, 2026 17:57
e12aa88 to
b64ecf3
Compare
EventMemory is built from the segment store, the vector store and the models. It opens or creates a partition by key in both stores, the vector store's first, and deletes a partition from both. EventMemoryPartition is the handle that encodes, queries and forgets; it holds its two store partitions privately, so no caller can pair one key's partition in one store with another key's partition in the other. The service locator builds the EventMemory and opens the session's partition through it. EventBackendParams carries the EventMemory and the handle, and LongTermMemory deletes the session's partition through EventMemory. The reserved timestamp field and text extraction, shared by both classes, move to module level. No behavior changes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…tion A vector store's schema is fixed when the store is built, so EventMemory takes it as a parameter, the same schema the composition root built the vector store with, rather than reading it from each partition it opens. - EventMemoryParams gains `store_schema`. EventMemory checks once, at construction, that the schema declares the fields EventMemory requires, and hands each partition handle the declared fields and the similarity metric. A handle's `similarity_metric` and its score direction come from them. - The service locator builds the vector store and EventMemory from one schema, `event_backend_store_schema`. - `VectorStorePartition` no longer has a `store_schema` property; each store's partitions keep the schema privately. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
edwinyyyu
force-pushed
the
feat/event-memory-partition-handle
branch
from
November 2, 2026 19:34
b64ecf3 to
a6c1aaf
Compare
Deleting a session built its EpisodicMemory, which opened, and so could create, the session's event-backend partitions, and then deleted them through the session's own LongTermMemory, which held the EventMemory only for that. The long-term service locator now builds each backend's memory and deletes a session's storage, both as functions that match on the configuration's backend: the event backend deletes the session's partitions by key through EventMemory, opening neither, and the declarative backend deletes the episodes its DeclarativeMemory of the session matches. The episodic service locator deletes the storage whenever the configuration builds long-term memory. EpisodicMemoryManager closes a cached idle instance, then deletes the stored configuration's storage, and builds no EpisodicMemory. LongTermMemory is the one class that routes requests on the backend: it takes each backend's memory built, holds no EventMemory, partition key, or deleted state, and has neither drop_session_partition nor close, whose body did nothing. EpisodicMemory.delete_session_episodes has no caller and goes; its short-term half only reset in-memory state. The evaluation scripts build the DeclarativeMemory they pass in, and delete a session through delete_long_term_memory_storage. A session saved with `enabled: false` now has its long-term storage deleted too, where the old path returned before deleting anything. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
LongTermMemory has had a single search since its unscored `search` went, so `search_scored` no longer tells it apart from anything. It is `query`, as the calls above and below it are: EpisodicMemory.query_memory and EventMemoryPartition.query. Its two backend branches are `_query_declarative` and `_query_event`. DeclarativeMemory keeps `search_scored`, which it tells apart from its unscored `search`. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
`_declarative_params` was one statement and `_event_params` a block with one caller, each standing between `long_term_memory_params_from_config`'s match and the work it routes to. The work now sits in the match arms, as it does in `delete_long_term_memory_storage` beside it. `_declarative_memory` and `_event_memory` stay, since building and deleting both use them. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Once the service locator built each backend's memory, the two backend params differed only in that memory: a DeclarativeMemory, or an EventMemoryPartition with the episode storage that turns its results into episodes and the reranker that sets its score direction. LongTermMemory now takes the memory itself, with those two as keywords, and each request matches on the memory's type. An EventMemoryPartition without episode storage is refused when the LongTermMemory is built. DeclarativeBackendParams, EventBackendParams, their union, the backend literal, and the session id, which nothing read, go. The service locator's `long_term_memory_from_config` builds the LongTermMemory in each match arm, and the episodic service locator takes it from there. The evaluation helper passes its DeclarativeMemory directly. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The episodic service locator's `delete_episodic_memory_storage` only checked whether the stored configuration builds long-term memory and passed it on. EpisodicMemoryManager.delete_episodic_session now makes that check and calls `delete_long_term_memory_storage` itself, so deleting a session goes from the manager straight to the long-term layer. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
EventMemoryParams carried the schema its vector store was built with, beside the store itself, so the schema had two sources that the composition root had to keep in step. A vector store now answers its own `store_schema`, the schema it was built with, and EventMemory reads its required fields, declared fields, and similarity metric from it. EventMemoryParams no longer takes a schema, and the service locator no longer passes one. Each store returns the schema it already held. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
edwinyyyu
marked this pull request as ready for review
November 3, 2026 17:23
Contributor
Author
|
Self-reviewed (human). |
`event_backend_store_schema` was split out so the vector store and EventMemory could be built from one schema. EventMemory now reads the schema from its vector store, so building the store is the schema's one use, and it is built there again. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The test helpers gave an EventMemory's stand-in vector store the schema read back from the partition's private attribute, and the SQLite stores' `_stored` read the declared properties the same way. The tests build each schema, so they now hand the stand-in store that schema, and `_stored` reads the properties its stores are built with. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…on's storage The drop_session_partition test checked that the delete path leaves reclamation to the background sweeper; the test of deleting an event session's storage by key checks it again. A new manager test deletes a cached idle session whose configuration builds long-term memory: the instance is closed and the configured storage deleted. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
edwinyyyu
requested review from
RonNiles,
leomem,
malatewang and
marvinyu-memverge
November 3, 2026 20:53
marvinyu-memverge
approved these changes
Nov 3, 2026
marvinyu-memverge
left a comment
Collaborator
There was a problem hiding this comment.
Approve at 3384289.
I read event_memory.py, long_term_memory.py and its service locator, the manager's delete path, and the evaluation helpers as whole files at head, and diffed each EventMemoryPartition data method against the base EventMemory.
Verified:
- encode_events, query, forget_events and the methods under them are the base code; the only changes are the names that moved to module level and the similarity metric read from the store's schema instead of the partition.
- open_or_create_partition opens the vector store first and the segment store second, and delete_partition deletes in the same order as the base drop_session_partition; both stores document delete_partition as idempotent, so deleting a session that never created its partitions is safe.
- delete_episodic_session no longer builds an EpisodicMemory: it closes a cached idle instance (whose short-term close is the same reset clear_memory did), then deletes the long-term storage from the stored config. The session row is still there when this runs, since the queued deletion removes it only after the TaskGroup.
- The declarative delete is the same get_matching_episodes + delete_episodes as before; the event delete builds no reranker.
- LongTermMemory's score direction uses the same reranker object the locator hands EventMemory, so the two can't disagree.
- Nothing in src/ or evaluation/ still calls search_scored on LongTermMemory, delete_session_episodes, or the removed params classes.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Purpose of the change
#1840 keeps event memory's two stores consistent through failed ingests and forgets. Its background work finishes what a failed request left in any partition, from one EventMemory the server starts at startup, whether or not a session is open. An EventMemory built around one session's two partitions, as it is before this PR, can't do that.
This PR gives EventMemory the shape of the stores it drives, so that #1840's diff holds the recovery protocol and the move to one EventMemory per deployment, and deletes a session's storage by key:
EventMemoryis built from the segment store, the vector store, and the models (segmenter, deriver, embedder, reranker, metrics factory). It has no data operations:open_or_create_partition(key)opens or creates the key's partition in both stores, the vector store's first, and returns a handle.delete_partition(key)deletes the key's partition from both stores.EventMemoryPartitionis the handle:encode_events,query,forget_events, andsimilarity_metric.EventMemory.open_or_create_partitionbuilds it on one key's partition in each store, and it holds the two privately.store_schema, the schema it was built with. EventMemory checks once, when it is built, that its vector store's schema declares the fields it requires, and its handles take the declared fields and the similarity metric from that schema.VectorStorePartitionno longer has astore_schemaproperty; each store keeps its partitions' schema privately.LongTermMemorytakes the session's handle, in place of both stores, both partitions, and the models.LongTermMemorytakes the session's memory itself: aDeclarativeMemory, or anEventMemoryPartitionwith the episode storage that turns its results into episodes and the reranker that sets its score direction. Each request matches on the memory's type, and anEventMemoryPartitionwithout episode storage is refused when theLongTermMemoryis built.DeclarativeBackendParams,EventBackendParams, and their union go. The long-term service locator'slong_term_memory_from_configbuilds theLongTermMemoryin the arm that matches the configuration's backend, the declarative backend'sDeclarativeMemoryincluded.LongTermMemoryhas no lifecycle methods:drop_session_partitiongoes, and so doesclose, whose body did nothing. Its one search isquery, asEpisodicMemory.query_memoryabove it andEventMemoryPartition.querybelow it are; it wassearch_scored.LongTermMemoryheld the stores only to delete the session's partitions, so deleting a session built its whole memory, which opened, and could create, its partitions. Now the long-term service locator deletes a session's storage from its configuration, with a function that matches on the backend as its build function does: the event backend deletes the session's partitions by key throughEventMemory.delete_partition, opening neither, and the declarative backend deletes the episodes itsDeclarativeMemoryof the session matches.EpisodicMemoryManager.delete_episodic_sessioncloses a cached idle instance, then deletes the long-term storage the stored configuration names whenever that configuration builds long-term memory, and builds noEpisodicMemory.EpisodicMemory.delete_session_episodesgoes; its short-term half only reset in-memory state. The evaluation scripts build theDeclarativeMemorythey pass in, and delete a session through the same function.Behavior that changes:
enabled: falsehas its long-term storage deleted too, where deletion returned before deleting anything.Otherwise the stores are opened and deleted in the same order with the same arguments, and every data operation runs the same code. The names the two classes share (the reserved timestamp field and the text extraction) move to module level.
Stack
24 open PRs: three independent PRs, and the vector store tree of short parallel branches. Every open PR in the tree has
feat/horizontal-scalingas its GitHub base, and the independent PRs havemain. The branches are in a fork, and a pull request can target only this repository's branches, so the on column gives the order the PRs build on each other. A stacked PR's diff on GitHub includes the PRs under it until they merge.Independent of the vector store tree, directly on
main:mainmainmainThe vector store tree. Each PR builds on the one in its on column; PRs on the same parent are parallel branches and do not depend on each other. #1631 is closed, superseded by #1733–#1736, which hold its changes split in four, with review changes since. #1675 is closed, superseded by #1627. #1841 is closed: its request packing is dropped, and its cancellation contract is in #1840. #1828 and #1839 sit side by side on the branch; #1839 and #1840 are the event memory recovery stack, and #1684 heads the event memory stack, which its own PRs index, listed because it builds on #1840, its branch also carrying #1828's commits until one of the two merges; #1871 keeps its own Stack section. Until the PRs under it merge, their changes show in a stacked PR's diff.
mainfeat/horizontal-scaling)feat/horizontal-scalingfeat/horizontal-scaling)feat/horizontal-scalingfeat/horizontal-scaling)feat/horizontal-scalingfeat/horizontal-scaling)feat/horizontal-scalingfeat/horizontal-scaling)feat/horizontal-scalingfeat/horizontal-scaling)feat/horizontal-scalingfeat/horizontal-scaling)feat/horizontal-scalingfeat/horizontal-scaling)feat/horizontal-scalingfeat/horizontal-scaling)feat/horizontal-scalingfeat/horizontal-scaling)feat/horizontal-scalingfeat/horizontal-scaling)feat/horizontal-scalingfeat/horizontal-scalingfeat/horizontal-scaling)feat/horizontal-scalingfeat/horizontal-scaling)feat/horizontal-scalingfeat/horizontal-scaling)feat/horizontal-scalingfeat/horizontal-scalingfeat/horizontal-scalingfeat/horizontal-scalingfeat/horizontal-scalingfeat/horizontal-scalingfeat/horizontal-scalingfeat/horizontal-scalingmemmachine-server migrate, which creates every configured store's schema before servers startfeat/horizontal-scalingThis PR is its 12 commits,
e125c9aeb,a6c1aaf88,378c38cf6,0750fb691,b827f3316,b97e81357,6dc4ad6d5,8319ec6e1,b3357dd24,07122e98b,1b1f8be62,338428956, directly onfeat/horizontal-scaling. #1840 is stacked on it.Verification
At this PR's head
338428956, onfeat/horizontal-scaling's0fa52b38b:ruff checkandruff format --checkare clean,ty checkis clean as CI runs it (uv run --frozen --all-extras ty check --project packages/server), anduv lock --checkpasses.packages/server/server_tests -m 'not integration'): 2356 passed, 8 skipped. Client unit tests (packages/client/client_tests): 259 passed.8319ec6e1, the declarativeLongTermMemorytests against Neo4j, 3 passed, and the vector store tests against Qdrant, Milvus, and the registry's PostgreSQL, 384 passed; the commits after it change neither those tests nor the code they run. The declarative tests' NebulaGraph half needs a running NebulaGraph and was not run.feat/horizontal-scaling's0fa52b38b, where deleting a session builds its memory and so opens its partitions.🤖 Written by Claude Code (Claude Opus 5.5) on behalf of @edwinyyyu.