Skip to content

feat: implement SQLitePersistence layer for local memory storage #1258

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
jhawpetoss6-collab wants to merge 1 commit into from
+37 −0
Closed

feat: implement SQLitePersistence layer for local memory storage #1258

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
feat: implement SQLitePersistence layer for local memory storage
  • Loading branch information
Justin Skywork committed Mar 23, 2026
commit 263939e23fcd80570a6f5c2989661e7ec9b11259
37 changes: 37 additions & 0 deletions memmachine/storage/sqlite/persistence.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
import sqlite3

Check failure on line 1 in memmachine/storage/sqlite/persistence.py

View workflow job for this annotation

GitHub Actions / ruff

ruff (D100) 

memmachine/storage/sqlite/persistence.py:1:1: D100 Missing docstring in public module

Check failure on line 1 in memmachine/storage/sqlite/persistence.py

View workflow job for this annotation

GitHub Actions / ruff

ruff (INP001) 

memmachine/storage/sqlite/persistence.py:1:1: INP001 File `memmachine/storage/sqlite/persistence.py` is part of an implicit namespace package. Add an `__init__.py`.
import json

Check failure on line 2 in memmachine/storage/sqlite/persistence.py

View workflow job for this annotation

GitHub Actions / ruff

ruff (I001) 

memmachine/storage/sqlite/persistence.py:1:1: I001 Import block is un-sorted or un-formatted
  help: Organize imports
Comment on lines +1 to +2
Copy link

Copilot AI Mar 24, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This module is missing a top-level module docstring, and the stdlib imports are not sorted (Ruff D100 / I001 will likely fail CI with the repo’s current lint settings). Add a short module docstring and sort imports alphabetically.

Suggested change
import sqlite3
import json
"""SQLite-backed persistence layer for MemMachine agent memory."""
import json
import sqlite3

Copilot uses AI. Check for mistakes.

class SQLitePersistence:
"""
SQLite-based persistence layer for MemMachine.
Provides a lightweight local storage option for agent memory.
"""

Check failure on line 8 in memmachine/storage/sqlite/persistence.py

View workflow job for this annotation

GitHub Actions / ruff

ruff (D205) 

memmachine/storage/sqlite/persistence.py:5:5: D205 1 blank line required between summary line and description
  help: Insert single blank line
def __init__(self, db_path="memory.db"):

Check failure on line 9 in memmachine/storage/sqlite/persistence.py

View workflow job for this annotation

GitHub Actions / ruff

ruff (ANN001) 

memmachine/storage/sqlite/persistence.py:9:24: ANN001 Missing type annotation for function argument `db_path`

Check failure on line 9 in memmachine/storage/sqlite/persistence.py

View workflow job for this annotation

GitHub Actions / ruff

ruff (D107) 

memmachine/storage/sqlite/persistence.py:9:9: D107 Missing docstring in `__init__`

Check failure on line 9 in memmachine/storage/sqlite/persistence.py

View workflow job for this annotation

GitHub Actions / ruff

ruff (ANN204) 

memmachine/storage/sqlite/persistence.py:9:9: ANN204 Missing return type annotation for special method `__init__`
  help: Add return type annotation: `None`
self.conn = sqlite3.connect(db_path)
self.cursor = self.conn.cursor()
self._init_db()

Copy link

Copilot AI Mar 24, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The class opens a SQLite connection/cursor in __init__ but never closes them. This can leave the DB file locked and leak file descriptors in long-running processes or tests. Provide an explicit close() method and/or implement the context manager protocol (__enter__/__exit__) so callers can reliably release the connection.

Suggested change
def close(self):
"""
Close the SQLite cursor and connection.
Safe to call multiple times.
"""
if getattr(self, "cursor", None) is not None:
try:
self.cursor.close()
except sqlite3.Error:
pass
self.cursor = None
if getattr(self, "conn", None) is not None:
try:
self.conn.close()
except sqlite3.Error:
pass
self.conn = None
def __enter__(self):
return self
def __exit__(self, exc_type, exc_val, exc_tb):
self.close()

Copilot uses AI. Check for mistakes.
def _init_db(self):

Check failure on line 14 in memmachine/storage/sqlite/persistence.py

View workflow job for this annotation

GitHub Actions / ruff

ruff (ANN202) 

memmachine/storage/sqlite/persistence.py:14:9: ANN202 Missing return type annotation for private function `_init_db`
  help: Add return type annotation: `None`
self.cursor.execute('''
CREATE TABLE IF NOT EXISTS memory (
id TEXT PRIMARY KEY,
content TEXT,
metadata TEXT,
timestamp DATETIME DEFAULT CURRENT_TIMESTAMP
)
''')

Check failure on line 22 in memmachine/storage/sqlite/persistence.py

View workflow job for this annotation

GitHub Actions / ruff

ruff (Q001) 

memmachine/storage/sqlite/persistence.py:15:29: Q001 Single quote multiline found but double quotes preferred
  help: Replace single multiline quotes with double quotes
self.conn.commit()

def save(self, memory_id, content, metadata=None):

Check failure on line 25 in memmachine/storage/sqlite/persistence.py

View workflow job for this annotation

GitHub Actions / ruff

ruff (ANN201) 

memmachine/storage/sqlite/persistence.py:25:9: ANN201 Missing return type annotation for public function `save`
  help: Add return type annotation: `None`
self.cursor.execute(
"INSERT OR REPLACE INTO memory (id, content, metadata) VALUES (?, ?, ?)",
(memory_id, content, json.dumps(metadata))
)
self.conn.commit()

def load(self, memory_id):
self.cursor.execute("SELECT content, metadata FROM memory WHERE id = ?", (memory_id,))
Comment on lines +9 to +33
Copy link

Copilot AI Mar 24, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ruff is configured to require type annotations (ANN*) for function signatures in this repo. __init__, _init_db, save, and load currently have no parameter/return type hints, which will likely fail CI; please add annotations (including the shape of metadata and the load() return type).

Copilot uses AI. Check for mistakes.
row = self.cursor.fetchone()
Comment on lines +11 to +34
Copy link

Copilot AI Mar 24, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Keeping a single cursor on self.cursor and reusing it across calls makes future concurrency/parallel usage harder and can complicate error handling (e.g., partial transactions on exceptions). Prefer using self.conn.execute(...) / creating a fresh cursor per operation and wrapping writes in a transaction context so failures roll back cleanly.

Suggested change
self.cursor = self.conn.cursor()
self._init_db()
def _init_db(self):
self.cursor.execute('''
CREATE TABLE IF NOT EXISTS memory (
id TEXT PRIMARY KEY,
content TEXT,
metadata TEXT,
timestamp DATETIME DEFAULT CURRENT_TIMESTAMP
)
''')
self.conn.commit()
def save(self, memory_id, content, metadata=None):
self.cursor.execute(
"INSERT OR REPLACE INTO memory (id, content, metadata) VALUES (?, ?, ?)",
(memory_id, content, json.dumps(metadata))
)
self.conn.commit()
def load(self, memory_id):
self.cursor.execute("SELECT content, metadata FROM memory WHERE id = ?", (memory_id,))
row = self.cursor.fetchone()
self._init_db()
def _init_db(self):
with self.conn:
self.conn.execute('''
CREATE TABLE IF NOT EXISTS memory (
id TEXT PRIMARY KEY,
content TEXT,
metadata TEXT,
timestamp DATETIME DEFAULT CURRENT_TIMESTAMP
)
''')
def save(self, memory_id, content, metadata=None):
with self.conn:
self.conn.execute(
"INSERT OR REPLACE INTO memory (id, content, metadata) VALUES (?, ?, ?)",
(memory_id, content, json.dumps(metadata))
)
def load(self, memory_id):
cursor = self.conn.execute(
"SELECT content, metadata FROM memory WHERE id = ?",
(memory_id,)
)
row = cursor.fetchone()
cursor.close()

Copilot uses AI. Check for mistakes.
if row:
return row[0], json.loads(row[1])
return None, None
Comment on lines +14 to +37
Copy link

Copilot AI Mar 24, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This introduces new persistence behavior (schema init + save/load semantics) but there are no accompanying unit tests. Consider adding a small pytest suite that uses a temp sqlite file (or :memory:) to verify round-trip of content/metadata and that INSERT OR REPLACE behaves as intended.

Copilot uses AI. Check for mistakes.
Loading