Skip to content

sqlite: optimize column name creation and text value encoding #61954

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
nodejs-github-bot merged 3 commits into from
Apr 11, 2026
+71 −9
Merged

sqlite: optimize column name creation and text value encoding #61954

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
82de30f
sqlite: use OneByte for ASCII text and internalize col names
thisalihassan Feb 23, 2026
cbcd03f
resolve feedback
thisalihassan Feb 24, 2026
3f8cce7
resolve feedback
thisalihassan Mar 3, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
resolve feedback
  • Loading branch information
@thisalihassan
thisalihassan committed Apr 7, 2026
commit 3f8cce791226466baaa0f4bbab39546097a7d742
21 changes: 12 additions & 9 deletions src/node_sqlite.cc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -66,7 +66,7 @@ using v8::Value;

inline MaybeLocal<String> Utf8StringMaybeOneByte(Isolate* isolate,
std::string_view input) {
int len = static_cast<int>(input.size());
const int len = static_cast<int>(input.size());
if (simdutf::validate_ascii(input.data(), input.size())) {
return String::NewFromOneByte(
isolate,
Expand Down Expand Up @@ -120,7 +120,7 @@ inline MaybeLocal<String> Utf8StringMaybeOneByte(Isolate* isolate,
case SQLITE_TEXT: { \
const char* v = \
reinterpret_cast<const char*>(sqlite3_##from##_text(__VA_ARGS__)); \
int v_len = sqlite3_##from##_bytes(__VA_ARGS__); \
const int v_len = sqlite3_##from##_bytes(__VA_ARGS__); \
(result) = \
Utf8StringMaybeOneByte((isolate), std::string_view(v, v_len)) \
.As<Value>(); \
Expand Down Expand Up @@ -2434,6 +2434,10 @@ StatementSync::~StatementSync() {
void StatementSync::Finalize() {
sqlite3_finalize(statement_);
statement_ = nullptr;
InvalidateColumnNameCache();
}

void StatementSync::InvalidateColumnNameCache() {
cached_column_names_.clear();
Copy link
Copy Markdown
Contributor

@louwers louwers Feb 26, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider putting this in a method.

}

Expand Down Expand Up @@ -2623,17 +2627,16 @@ MaybeLocal<Name> StatementSync::ColumnNameToName(const int column) {
.As<Name>();
}

// Returns cached internalized column name strings for this statement,
// invalidating the cache when SQLite re-prepares the statement (e.g. after
// schema changes like ALTER TABLE) detected via SQLITE_STMTSTATUS_REPREPARE.
// Populates `keys` with cached column names, rebuilding the cache if the
// statement was re-prepared.
bool StatementSync::GetCachedColumnNames(LocalVector<Name>* keys) {
Copy link
Copy Markdown
Member

@anonrig anonrig Feb 24, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you add a comment to this function?

Isolate* isolate = env()->isolate();

int reprepare_count =
sqlite3_stmt_status(statement_, SQLITE_STMTSTATUS_REPREPARE, 0);
const int reprepare_count =
sqlite3_stmt_status(statement_, SQLITE_STMTSTATUS_REPREPARE, false);
if (reprepare_count != cached_column_names_reprepare_count_) {
cached_column_names_.clear();
int num_cols = sqlite3_column_count(statement_);
const int num_cols = sqlite3_column_count(statement_);
if (num_cols == 0) {
cached_column_names_reprepare_count_ = reprepare_count;
return true;
Expand All @@ -2642,7 +2645,7 @@ bool StatementSync::GetCachedColumnNames(LocalVector<Name>* keys) {
for (int i = 0; i < num_cols; ++i) {
Local<Name> key;
if (!ColumnNameToName(i).ToLocal(&key)) {
cached_column_names_.clear();
InvalidateColumnNameCache();
return false;
}
cached_column_names_.emplace_back(Global<Name>(isolate, key));
Expand Down
1 change: 1 addition & 0 deletions src/node_sqlite.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -298,6 +298,7 @@ class StatementSync : public BaseObject {
inline int ResetStatement();
std::vector<v8::Global<v8::Name>> cached_column_names_;
int cached_column_names_reprepare_count_ = -1;
Copy link
Copy Markdown
Contributor

@louwers louwers Feb 26, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider using std::optional.

Copy link
Copy Markdown
Contributor Author

@thisalihassan thisalihassan Feb 27, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the suggestion @louwers but I benchmarked on the iterate path (30 runs) saw a consistent -1% to -2% regression after making these changes:

sqlite/sqlite-prepare-select-all.js statement='SELECT text_column FROM foo LIMIT 1' method='iterate' tableSeedSize=100000 n=100000 *** -2.06 %
sqlite/sqlite-prepare-select-all.js statement='SELECT text_column FROM foo LIMIT 100' method='iterate' tableSeedSize=100000 n=100000 *** -1.42 %

The reason might be std::optional comparison generates more instructions per row than just int

Copy link
Copy Markdown
Contributor

@louwers louwers Feb 27, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Makes sense. Not worth it then.

Copy link
Copy Markdown
Contributor Author

@thisalihassan thisalihassan Feb 27, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah it's confirmed, but very interesting

Copy link
Copy Markdown
Contributor Author

@thisalihassan thisalihassan Mar 3, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @louwers should we merge this PR?

void InvalidateColumnNameCache();
bool BindParams(const v8::FunctionCallbackInfo<v8::Value>& args);
bool BindValue(const v8::Local<v8::Value>& value, const int index);

Expand Down
Loading