Skip to content

feat: wire clustering into ingestion pipeline and API #1241

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
o-love wants to merge 17 commits into
base: main
Choose a base branch
from
Open

feat: wire clustering into ingestion pipeline and API #1241

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
17 commits
Select commit Hold shift + click to select a range
e35e791
refactor: modernize storage interface types (list→Sequence, dict→Mapp…
o-love Mar 18, 2026
fd6b594
refactor: propagate AsyncIterator through semantic memory module
o-love Mar 18, 2026
e311c7e
fix: resolve ruff and ty lint errors in storage interface refactor
o-love Mar 18, 2026
f45c502
fix: convert Sequence/Mapping to list/dict at API response boundary
o-love Mar 18, 2026
c8848ca
fix: ruff format router.py ternary expressions
o-love Mar 18, 2026
d9fd612
Merge branch 'main' into storage-interface-refactor
malatewang Mar 26, 2026
a7a2428
Update packages/server/src/memmachine_server/semantic_memory/semantic…
malatewang Mar 26, 2026
8723f41
Update packages/server/src/memmachine_server/common/utils.py
malatewang Mar 26, 2026
95edd70
Fix logger.debug formatting for add_messages method
malatewang Mar 26, 2026
7329565
fix: ruff format long line in merge_async_iterators
o-love Apr 2, 2026
6fdfd35
Merge branch 'main' into storage-interface-refactor
o-love Apr 8, 2026
b83c4ca
Fix semantic ingestion type-check test
o-love Apr 8, 2026
e6c9962
feat: add cluster engine for semantic message grouping
o-love Mar 18, 2026
3846d21
feat: wire clustering into ingestion pipeline and API
o-love Mar 18, 2026
f7ae24e
fix: add get_reranker to _ResourceManager in integration test
o-love Mar 18, 2026
bc00c86
Fix lint errors
o-love Apr 8, 2026
7a8c459
fix: make feature cap test trigger ingestion
o-love Apr 10, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Fix lint errors
  • Loading branch information
@o-love
o-love committed Apr 8, 2026
commit bc00c86315fb4df220b254bb3466a5c8e8df12cf
18 changes: 8 additions & 10 deletions packages/server/server_tests/memmachine_server/semantic_memory/test_semantic_ingestion.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1014,19 +1014,17 @@ async def mock_context_length_error(*args, **kwargs):
await ingestion_service._process_single_set("user-context-overflow")

assert (
await _collect(
semantic_storage.get_history_messages(
set_ids=["user-context-overflow"],
is_ingested=False,
)
await _collect_history_messages(
semantic_storage,
set_ids=["user-context-overflow"],
is_ingested=False,
)
== []
)
assert await _collect(
semantic_storage.get_history_messages(
set_ids=["user-context-overflow"],
is_ingested=True,
)
assert await _collect_history_messages(
semantic_storage,
set_ids=["user-context-overflow"],
is_ingested=True,
) == [message_id]


Expand Down
10 changes: 4 additions & 6 deletions packages/server/src/memmachine_server/semantic_memory/semantic_ingestion.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -353,6 +353,7 @@ async def process_semantic_type(
]

message_content = self._format_cluster_messages(cluster_messages)
citation_ids = [m.uid for m in cluster_messages if m.uid is not None]

try:
commands = await llm_feature_update(
Expand All @@ -364,12 +365,11 @@ async def process_semantic_type(
except Exception as err:
if _is_context_length_exceeded_error(err):
logger.warning(
"Skipping message %s for semantic type %s due to non-retryable context length error",
message.uid,
"Skipping cluster %s for semantic type %s due to non-retryable context length error",
cluster_id,
semantic_category.name,
)
if message.uid not in mark_messages:
mark_messages.append(message.uid)
mark_messages.update(citation_ids)
continue

logger.exception(
Expand All @@ -382,8 +382,6 @@ async def process_semantic_type(

continue

citation_ids = [m.uid for m in cluster_messages if m.uid is not None]

await self._apply_commands(
commands=commands,
set_id=set_id,
Expand Down
Loading