Skip to content

SQLCipher's OpenSSL provider fetches EVP_MAC / EVP_CIPHER from the global method store on every page operation (performance / OpenSSL 3.x best practice) #597

Description

@josefguenther

Summary

SQLCipher's OpenSSL provider (src/crypto_openssl.c) fetches its cryptographic algorithms from OpenSSL's default-libctx global method store on every single page operation, rather than fetching once and caching the handles in the codec/provider context:

  • sqlcipher_openssl_hmac (src/crypto_openssl.c) calls EVP_MAC_fetch(NULL, "HMAC", NULL) and the matching EVP_MAC_free per HMAC — i.e. per page.
  • sqlcipher_openssl_cipher calls EVP_CipherInit_ex(..., EVP_aes_256_cbc(), ...) per page; the first use of that cipher resolves an implicit fetch from the same global method store.

OpenSSL 3.x's own documentation explicitly recommends fetching algorithms once and reusing the resulting EVP_MAC / EVP_CIPHER objects, precisely because each fetch is a relatively expensive, reference-counted lookup into a process-global store (the "explicit fetching" performance guidance in the OpenSSL 3.x migration notes and the EVP_MAC_fetch(3) / EVP_CIPHER_fetch(3) man pages).

Why it matters

  1. Performance. Fetching per page (instead of per codec context) is the well-known OpenSSL-3 explicit-fetch anti-pattern. A database doing thousands of page reads/writes performs thousands of redundant global-store lookups plus EVP_MAC / EVP_MAC_CTX allocations and frees that could be amortized to a single fetch per connection. On large databases this is measurable per-operation overhead.

  2. Lock contention under concurrency. Those per-page fetches also repeatedly take OpenSSL's internal method-store lock. Under heavy multi-threaded, multi-connection page I/O this is avoidable contention that a fetch-once-and-cache approach eliminates.

No correctness / thread-safety claim: in our testing the per-page fetch lookups are properly serialized by OpenSSL's own internal lock and did not race or corrupt — this is a performance + best-practice issue, not a crash report.

Affected versions / environment

  • SQLCipher: present through the current release 4.16.0 (the OpenSSL provider src/crypto_openssl.c is unchanged across 4.14.0–4.16.0); observed on 4.14.0 community (SQLCIPHER_CRYPTO_OPENSSL, via libsqlite3-sys 0.38).
  • OpenSSL: 3.x (vendored), default library context.
  • Threading: SQLITE_THREADSAFE=1; multi-threaded application with many keyed connections doing concurrent page I/O.

Suggested fix

Fetch the EVP_MAC ("HMAC") and EVP_CIPHER ("AES-256-CBC", plus any digests) once — at provider activation or codec-context initialization (sqlcipher_codec_ctx_init / the provider context) — store the fetched handles, and reuse them for every page op:

  • pass the cached EVP_MAC* to EVP_MAC_CTX_new(...) per op instead of re-EVP_MAC_fetch-ing;
  • pass the cached EVP_CIPHER* to EVP_CipherInit_ex2(...) instead of the legacy EVP_aes_256_cbc() convenience (which re-resolves through the default libctx).

Free the cached objects at context/provider teardown. This matches OpenSSL 3.x best practice, eliminates the per-page global-store lookups (a measurable throughput win on large databases), and removes the per-page lock traffic on the shared store.

Impact

Per-page CPU overhead on every encrypted read and write, plus avoidable contention on OpenSSL's global method store under multi-threaded use. Fixable entirely within SQLCipher's OpenSSL provider, with no change to the on-disk format or the public API.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions