Skip to content

Add temporal scoring#1436

Open
edwinyyyu wants to merge 10 commits into
MemMachine:mainfrom
edwinyyyu:temporal_scoring
Open

Add temporal scoring#1436
edwinyyyu wants to merge 10 commits into
MemMachine:mainfrom
edwinyyyu:temporal_scoring

Conversation

@edwinyyyu
Copy link
Copy Markdown
Contributor

@edwinyyyu edwinyyyu commented Jun 1, 2026 •
edited
Loading

Purpose of the change

Improve ability to process temporal retrieval signals.

Description

Make Context composable. Use Context to hold temporal information.

Does not really stack with reranker or #1444 on locomo.
Temporal match score component weight must be tuned for the embedder.

Approach works on synthetic data with harder temporal queries (works well for deictics, complex time ranges).

Recency/proximity not included (design less finalized). Temporal signal for introducing candidates to the pool not well-explored (requires new tables and indexing).

Add temporal extractors and query planners.

Type of change

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Refactor (does not change functionality, e.g., code style improvements, linting)
  • Documentation update
  • Project Maintenance (updates to build scripts, CI, etc., that do not affect the main project)
  • Security (improves security without changing functionality)

How Has This Been Tested?

Tested on synthetically generated data with more difficult temporal problems.

  • Unit Test
  • Integration Test
  • End-to-end Test
  • Test Script (please provide)
  • Manual verification (list step-by-step instructions)

Checklist

  • I have signed the commit(s) within this pull request
  • My code follows the style guidelines of this project (See STYLE_GUIDE.md)
  • I have performed a self-review of my own code
  • I have commented my code
  • I have made corresponding changes to the documentation
  • My changes generate no new warnings
  • I have added unit tests that prove my fix is effective or that my feature works
  • New and existing unit tests pass locally with my changes
  • Any dependent changes have been merged and published in downstream modules
  • I have checked my code and corrected any misspellings

Maintainer Checklist

  • Confirmed all checks passed
  • Contributor has signed the commit(s)
  • Reviewed the code
  • Run, Tested, and Verified the change(s) work as expected

@edwinyyyu edwinyyyu force-pushed the temporal_scoring branch 8 times, most recently from 8958eda to 428604f Compare June 2, 2026 21:43
@edwinyyyu
WIP
2b536e5 
Signed-off-by: Edwin Yu <edwinyyyu@gmail.com>
edwinyyyu added 9 commits June 4, 2026 11:58
@edwinyyyu
Scoring
a1251ce 
Signed-off-by: Edwin Yu <edwinyyyu@gmail.com>
@edwinyyyu
Fix
283ef43 
Signed-off-by: Edwin Yu <edwinyyyu@gmail.com>
@edwinyyyu
Add extractor-backed temporal query planner
d24a059 
Signed-off-by: Edwin Yu <edwinyyyu@gmail.com>
@edwinyyyu
Add scoring test
c9c7118 
Signed-off-by: Edwin Yu <edwinyyyu@gmail.com>
@edwinyyyu
Add test
1490a5d 
Signed-off-by: Edwin Yu <edwinyyyu@gmail.com>
@edwinyyyu
Temporal reranking layer for EventMemory
aeb200d 
Signed-off-by: Edwin Yu <edwinyyyu@gmail.com>
@edwinyyyu
Dependencies for LLM-free temporal extraction
0c78e4e 
Signed-off-by: Edwin Yu <edwinyyyu@gmail.com>
@edwinyyyu
Add temporal match score weight
c226e83 
Signed-off-by: Edwin Yu <edwinyyyu@gmail.com>
@edwinyyyu
Claude-written extractors and planners
b933eb6 
Signed-off-by: Edwin Yu <edwinyyyu@gmail.com>
@edwinyyyu edwinyyyu requested a review from malatewang June 5, 2026 00:42
@edwinyyyu edwinyyyu marked this pull request as ready for review June 5, 2026 00:42
@edwinyyyu edwinyyyu requested a review from sscargal June 5, 2026 00:42
malatewang
malatewang requested changes Jun 5, 2026
View reviewed changes
Comment thread packages/server/src/memmachine_server/episodic_memory/event_memory/event_memory.py
timestamp_prefix = f"[{formatted_timestamp}] " if formatted_timestamp else ""

match segment.context:
return f"{timestamp_prefix}{EventMemory._context_prefix(segment.context)}"
Copy link
Copy Markdown
Contributor

@malatewang malatewang Jun 5, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks like the segment header only has the prefix, not the value. Not sure if this really can help for LLM.

Copy link
Copy Markdown
Contributor Author

@edwinyyyu edwinyyyu Jun 5, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The segment_header staticmethod is meant to generate a header for content that shares the same context (e.g. a single timestamp and speaker for chunks from the same message)

Comment thread packages/server/src/memmachine_server/temporal/extractor/dateparser_temporal_extractor.py
# DateDataParser takes settings only at construction time, so we
# build a fresh parser per call (its construction cost is small;
# ``RELATIVE_BASE`` is the per-call signal).
parser = DateDataParser(languages=["en"], settings=settings)
Copy link
Copy Markdown
Contributor

@malatewang malatewang Jun 5, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It will be better to detect the language type first.

Copy link
Copy Markdown
Contributor Author

@edwinyyyu edwinyyyu Jun 5, 2026 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We will need a pluggable language detection service then, which needs a new feature, or get the locale from the client.

Comment thread packages/server/src/memmachine_server/temporal/extractor/duckling_temporal_extractor.py
return None


class DucklingTemporalExtractorParams(BaseModel):
Copy link
Copy Markdown
Contributor

@malatewang malatewang Jun 5, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this require configuration update?

Copy link
Copy Markdown
Contributor Author

@edwinyyyu edwinyyyu Jun 5, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This did not exist before, so it needs to be added to configuration for it to work on the server.

Duckling requires a Duckling server (running separately).

Comment thread packages/server/src/memmachine_server/episodic_memory/event_memory/temporal_ranking.py
from memmachine_server.temporal.time_range import TimeRange


def _temporal_anchors_from_context(context: Context | None) -> list[TimeRange]:
Copy link
Copy Markdown
Contributor

@malatewang malatewang Jun 5, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The logic not integrated into the query data path yet?

Copy link
Copy Markdown
Contributor Author

@edwinyyyu edwinyyyu Jun 5, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not yet.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@malatewang malatewang malatewang requested changes

@sscargal sscargal Awaiting requested review from sscargal

Requested changes must be addressed to merge this pull request.

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@edwinyyyu @malatewang