Crash report

What happened?

If a Python callback is invoked by SQLite during query preparation or execution -- an authorizer (set_authorizer), progress handler (set_progress_handler), trace callback (set_trace_callback), or user-defined function (create_function) -- and that callback calls con.close while SQLite is still on the C stack, the process hard-crashes.

The problem is that connection_close sets self->db to null (then calls sqlite3_close_v2 -- which defers the actual close until all statements are finalised).

There are two different crash causes from what I can tell.

The first is a null pointer dereference on db.
After the callback returns, self->connection->db is null but the cursor code reads it unconditionally in several places:

• cpython/Modules/_sqlite/cursor.c

  Line 918 in 884ac3e

  set_error_from_db(state, self->connection->db);

• cpython/Modules/_sqlite/cursor.c

  Line 961 in 884ac3e

  self->rowcount += (long)sqlite3_changes(self->connection->db);

• cpython/Modules/_sqlite/cursor.c

  Line 974 in 884ac3e

  lastrowid = sqlite3_last_insert_rowid(self->connection->db);

• cpython/Modules/_sqlite/cursor.c

  Line 1161 in 884ac3e

  self->rowcount = (long)sqlite3_changes(self->connection->db);

• cpython/Modules/_sqlite/cursor.c

  Line 1170 in 884ac3e

  rc = set_error_from_db(self->connection->state, self->connection->db);

All call sites either call set_error_from_db directly or pass db to a SQLite API. set_error_from_db also lacks a null guard:

The second crash path is a use-after-free of callback_context (authorizer / progress / trace)

sqlite3_prepare_v2 can call the authorizer more than once per statement (e.g. once for SQLITE_SELECT, then once per SQLITE_READ). If the authorizer closes the connection (through connection_close), free_callback_contexts is called, which calls decref_callback_context on the in-flight context. Combined with the matching decref at the end of the C wrapper, the context's refcount reaches zero and the struct is freed. The next authorizer invocation then calls incref_callback_context on a freed callback_context.
The same problem applies to progress_callback and trace_callback when those fire more than once per step.

Additionally, because pysqlite_connection_close_impl calls Py_CLEAR(self->statement_cache) before calling into connection_close, and because get_statement_from_cache holds only a borrowed reference to the cache object, closing the connection from within a callback frees that object while the call into it is still ongoing.

Reproducer

This is a reproducer for just one example, the set_authorizer one.

import sqlite3

con = sqlite3.connect(":memory:")
con.execute("CREATE TABLE t (v INTEGER)")

def auth(action, arg1, arg2, db, trigger):
    con.close()
    return sqlite3.SQLITE_OK

con.set_authorizer(auth)
con.execute("SELECT v FROM t")

On CPython main, this results in

Assertion failed: (ctx->refcount > 0), function incref_callback_context, file connection.c, line 1131.
zsh: abort      ./python.exe repro.py

CPython versions tested on:

CPython main branch

Operating systems tested on:

Linux

Output from running 'python -VV' on the command line:

No response

Linked PRs

• gh-145040: safety for callbacks closing the sqlite connection #151031