Skip to content

Commit 96913b6

Browse files
aitorvsclaudekarlenDimla
authored
Add sqlcipher-loader module and migrate PIR/autofill (#8009)
Task/Issue URL: https://app.asana.com/1/137249556945/project/488551667048375/task/1213721049904377?focus=true ### Description Introduces a dedicated `sqlcipher-loader` module that centralises SQLCipher native library loading and fixes a JNI deadlock causing a 7x spike in `LIBRARY_LOAD_TIMEOUT_SQLCIPHER` (350 → 2,500/day). **Root cause:** removing PIR's eager `System.loadLibrary("sqlcipher")` call (f6c879e) inadvertently removed the pre-warm that autofill depended on. Autofill's `SqlCipherLibraryLoader` then performed the first full load on a background thread, and when PIR's lazy load raced concurrently, a JNI loading deadlock could occur — triggering the 10s timeout. **Fix:** - New `sqlcipher-loader-api` module exposes a `SqlCipherLoader` interface with a single `waitForLibraryLoad(): Result<Unit>` API. - New `sqlcipher-loader-impl` provides `RealSqlCipherLoader`, which: - Implements `MainProcessLifecycleObserver` to eagerly pre-warm SQLCipher on the IO dispatcher at app startup, before autofill or PIR need it. - Uses a `CompletableDeferred<Unit>` (initialised at construction) so all callers share the same load — concurrent `complete()` calls are no-ops, making the race structurally impossible. - Fires `LIBRARY_LOAD_FAILURE_SQLCIPHER` (daily pixel) if the load throws. - PIR and autofill both now inject `SqlCipherLoader` and call `waitForLibraryLoad()` instead of loading independently. - `SqlCipherLibraryLoader` (autofill-local) and its test deleted. - `sqlCipherAsyncLoading` feature flag and `LIBRARY_LOAD_TIMEOUT_SQLCIPHER` pixel removed — no timeout needed when the load is predictable and early. ### Steps to test this PR _SQLCipher loads correctly_ - [x] Install the app and open a page with a password field — autofill suggestion should appear normally - [x] Open the PIR screen — it should load without errors - [x] Check logcat for `SqlCipher: native library loaded successfully` appearing once at startup (not twice, not on demand) _No regression on autofill_ - [x] Save a login and verify it autofills on the target site - [x] Confirm no `LIBRARY_LOAD_TIMEOUT_SQLCIPHER` or `LIBRARY_LOAD_FAILURE_SQLCIPHER` pixels fire under normal conditions _Verify build_ - [x] `./gradlew :sqlcipher-loader-impl:testDebugUnitTest` - [x] `./gradlew :autofill-impl:testDebugUnitTest` - [x] `./gradlew :pir-impl:testDebugUnitTest` _SQLCipher loads correctly on PIR process_ - [x] Obtain subscription - [x] Start a PIR scan via the PIR dashboard - [x] Logcat should show "SqlCipher: Attempting to load native library loaded on the PIR process” - [x] Logcat should show " SqlCipher-Init: Library load wait completed successfully” - [x] Logcat should show "PIR-DB: sqlcipher native library loaded ok" <!-- CURSOR_SUMMARY --> --- > [!NOTE] > **Medium Risk** > Moderate risk because it changes when/how the SQLCipher native library is loaded and gates creation of encrypted databases for both Autofill and PIR, which can impact data access and startup behavior. > > **Overview** > Centralizes SQLCipher native library loading into a new `sqlcipher-loader` module (`SqlCipherLoader` API + `RealSqlCipherLoader` impl) that eagerly starts async loading via process lifecycle observers and provides a shared `waitForLibraryLoad` for all callers (with timeout/failure pixels). > > Migrates Autofill and PIR secure DB factories to inject `SqlCipherLoader` instead of doing their own loads, deleting Autofill’s local `SqlCipherLibraryLoader` and its feature flag (`sqlCipherAsyncLoading`) and moving the SQLCipher load pixels out of `autofill.json5` into `sqlcipher_loader.json5`. The app wiring is updated to depend on the new modules and to strip ATB params for the new SQLCipher pixels. > > <sup>Written by [Cursor Bugbot](https://cursor.com/dashboard?tab=bugbot) for commit 33c8a5f. This will update automatically on new commits. Configure [here](https://cursor.com/dashboard?tab=bugbot).</sup> <!-- /CURSOR_SUMMARY --> --------- Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com> Co-authored-by: Karl Dimla <klmbdimla@gmail.com>
1 parent b3616e2 commit 96913b6

19 files changed

Lines changed: 519 additions & 564 deletions

File tree

‎PixelDefinitions/pixels/autofill.json5‎

Lines changed: 0 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -384,19 +384,5 @@
384384
"enum": ["true", "false"]
385385
}
386386
]
387-
},
388-
"library_load_timeout_sqlcipher": {
389-
"description": "Fired when the sqlcipher native library takes longer than the timeout period to load asynchronously",
390-
"owners": ["CDRussell"],
391-
"triggers": ["exception"],
392-
"suffixes": ["form_factor"],
393-
"parameters": ["appVersion"]
394-
},
395-
"library_load_failure_sqlcipher": {
396-
"description": "Fired when the sqlcipher native library fails to load asynchronously due to an exception",
397-
"owners": ["CDRussell"],
398-
"triggers": ["exception"],
399-
"suffixes": ["form_factor"],
400-
"parameters": ["appVersion"]
401387
}
402388
}

‎PixelDefinitions/pixels/sqlcipher_loader.json5‎

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,16 @@
1+
{
2+
"library_load_timeout_sqlcipher": {
3+
"description": "Fired when the sqlcipher native library takes longer than the timeout period to load asynchronously",
4+
"owners": ["aitorvs", "CDRussell", "karlenDimla"],
5+
"triggers": ["exception"],
6+
"suffixes": ["form_factor"],
7+
"parameters": ["appVersion"]
8+
},
9+
"library_load_failure_sqlcipher": {
10+
"description": "Fired when the sqlcipher native library fails to load due to an exception",
11+
"owners": ["aitorvs", "CDRussell", "karlenDimla"],
12+
"triggers": ["exception"],
13+
"suffixes": ["form_factor"],
14+
"parameters": ["appVersion"]
15+
}
16+
}

‎app/build.gradle‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -317,6 +317,8 @@ dependencies {
317317
implementation project(':privacy-config-store')
318318
implementation project(':request-interception-api')
319319
implementation project(':request-interception-impl')
320+
implementation project(':sqlcipher-loader-api')
321+
implementation project(':sqlcipher-loader-impl')
320322
internalImplementation project(':privacy-config-internal')
321323

322324
implementation project(':anrs-api')

‎app/src/main/java/com/duckduckgo/app/global/api/PixelParamRemovalInterceptor.kt‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -37,6 +37,7 @@ import com.duckduckgo.newtabpage.impl.pixels.NewTabPixelNames
3737
import com.duckduckgo.remote.messaging.impl.pixels.RemoteMessagingPixelName
3838
import com.duckduckgo.savedsites.impl.SavedSitesPixelName
3939
import com.duckduckgo.site.permissions.impl.SitePermissionsPixelName
40+
import com.duckduckgo.sqlcipher.loader.impl.SqlCipherPixelName
4041
import com.squareup.anvil.annotations.ContributesMultibinding
4142
import okhttp3.Interceptor
4243
import okhttp3.Response
@@ -202,6 +203,8 @@ object PixelInterceptorPixelsRequiringDataCleaning : PixelParamRemovalPlugin {
202203
AppPixelName.GET_DESKTOP_BROWSER_SHARE_DOWNLOAD_LINK_CLICK.pixelName to PixelParameter.removeAtb(),
203204
AppPixelName.GET_DESKTOP_BROWSER_LINK_CLICK.pixelName to PixelParameter.removeAtb(),
204205
AppPixelName.MENU_ACTION_VPN_PRESSED.pixelName to PixelParameter.removeAtb(),
206+
SqlCipherPixelName.LIBRARY_LOAD_FAILURE_SQLCIPHER.pixelName to PixelParameter.removeAtb(),
207+
SqlCipherPixelName.LIBRARY_LOAD_TIMEOUT_SQLCIPHER.pixelName to PixelParameter.removeAtb(),
205208
)
206209
}
207210
}

‎autofill/autofill-api/src/main/java/com/duckduckgo/autofill/api/AutofillFeature.kt‎

Lines changed: 0 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -184,14 +184,6 @@ interface AutofillFeature {
184184
@Toggle.DefaultValue(defaultValue = DefaultFeatureValue.FALSE)
185185
fun sendSanitizedStackTraces(): Toggle
186186

187-
/**
188-
* Controls whether SqlCipher library loading uses async mode or sync fallback.
189-
*
190-
* @return `true` to use async loading (default), `false` to use sync loading fallback
191-
*/
192-
@DefaultValue(Toggle.DefaultFeatureValue.TRUE)
193-
fun sqlCipherAsyncLoading(): Toggle
194-
195187
@Toggle.DefaultValue(Toggle.DefaultFeatureValue.FALSE)
196188
fun readFromHarmony(): Toggle
197189
}

‎autofill/autofill-impl/build.gradle‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -47,7 +47,7 @@ dependencies {
4747
implementation project(':data-store-api')
4848
testImplementation project(':feature-toggles-test')
4949
implementation project(path: ':settings-api') // temporary until we release new settings
50-
implementation project(':library-loader-api')
50+
implementation project(':sqlcipher-loader-api')
5151
implementation project(':saved-sites-api')
5252
implementation project(':cookies-api')
5353

‎autofill/autofill-impl/src/main/java/com/duckduckgo/autofill/impl/pixel/AutofillPixelNames.kt‎

Lines changed: 0 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -72,8 +72,6 @@ import com.duckduckgo.autofill.impl.pixel.AutofillPixelNames.BOOKMARK_IMPORT_FRO
7272
import com.duckduckgo.autofill.impl.pixel.AutofillPixelNames.EMAIL_TOOLTIP_DISMISSED
7373
import com.duckduckgo.autofill.impl.pixel.AutofillPixelNames.EMAIL_USE_ADDRESS
7474
import com.duckduckgo.autofill.impl.pixel.AutofillPixelNames.EMAIL_USE_ALIAS
75-
import com.duckduckgo.autofill.impl.pixel.AutofillPixelNames.LIBRARY_LOAD_FAILURE_SQLCIPHER
76-
import com.duckduckgo.autofill.impl.pixel.AutofillPixelNames.LIBRARY_LOAD_TIMEOUT_SQLCIPHER
7775
import com.duckduckgo.autofill.impl.pixel.AutofillPixelNames.PRODUCT_TELEMETRY_SURFACE_PASSWORDS_OPENED
7876
import com.duckduckgo.autofill.impl.pixel.AutofillPixelNames.PRODUCT_TELEMETRY_SURFACE_PASSWORDS_OPENED_DAILY
7977
import com.duckduckgo.common.utils.plugins.pixel.PixelParamRemovalPlugin
@@ -238,8 +236,6 @@ enum class AutofillPixelNames(override val pixelName: String) : Pixel.PixelName
238236
AUTOFILL_HARMONY_KEY_MISSING("autofill_harmony_key_missing"),
239237
AUTOFILL_PREFERENCES_KEY_MISSING("autofill_preferences_key_missing"),
240238
AUTOFILL_STORE_KEY_ALREADY_EXISTS("autofill_store_key_already_exists"),
241-
LIBRARY_LOAD_TIMEOUT_SQLCIPHER("library_load_timeout_sqlcipher"),
242-
LIBRARY_LOAD_FAILURE_SQLCIPHER("library_load_failure_sqlcipher"),
243239
AUTOFILL_PREFERENCES_UPDATE_KEY_NULL_FILE("autofill_preferences_update_key_null_file"),
244240
AUTOFILL_HARMONY_PREFERENCES_UPDATE_KEY_NULL_FILE("autofill_harmony_preferences_update_key_null_file"),
245241
AUTOFILL_PREFERENCES_GET_KEY_NULL_FILE("autofill_preferences_get_key_null_file"),
@@ -332,8 +328,6 @@ object AutofillPixelsRequiringDataCleaning : PixelParamRemovalPlugin {
332328
BOOKMARK_IMPORT_FROM_GOOGLE_FLOW_CANCELLED.pixelName to PixelParameter.removeAtb(),
333329
BOOKMARK_IMPORT_FROM_GOOGLE_FLOW_EXTRA_CHROME_EXPORT.pixelName to PixelParameter.removeAtb(),
334330

335-
LIBRARY_LOAD_TIMEOUT_SQLCIPHER.pixelName to PixelParameter.removeAtb(),
336-
LIBRARY_LOAD_FAILURE_SQLCIPHER.pixelName to PixelParameter.removeAtb(),
337331
)
338332
}
339333
}

‎autofill/autofill-impl/src/main/java/com/duckduckgo/autofill/impl/securestorage/SecureStorageDatabaseFactory.kt‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -21,6 +21,7 @@ import com.duckduckgo.autofill.store.db.SecureStorageDatabase
2121
import com.duckduckgo.data.store.api.DatabaseProvider
2222
import com.duckduckgo.data.store.api.RoomDatabaseConfig
2323
import com.duckduckgo.di.scopes.AppScope
24+
import com.duckduckgo.sqlcipher.loader.api.SqlCipherLoader
2425
import com.squareup.anvil.annotations.ContributesBinding
2526
import dagger.SingleInstanceIn
2627
import kotlinx.coroutines.sync.Mutex
@@ -43,7 +44,7 @@ interface SecureStorageDatabaseFactory {
4344
class RealSecureStorageDatabaseFactory @Inject constructor(
4445
private val keyProvider: SecureStorageKeyProvider,
4546
private val databaseProvider: DatabaseProvider,
46-
private val sqlCipherLoader: SqlCipherLibraryLoader,
47+
private val sqlCipherLoader: SqlCipherLoader,
4748
) : SecureStorageDatabaseFactory {
4849
private var _database: SecureStorageDatabase? = null
4950
private val mutex = Mutex()

‎autofill/autofill-impl/src/main/java/com/duckduckgo/autofill/impl/securestorage/SqlCipherLibraryLoader.kt‎

Lines changed: 0 additions & 198 deletions
This file was deleted.

‎autofill/autofill-impl/src/test/java/com/duckduckgo/autofill/impl/securestorage/RealSecureStorageDatabaseFactoryTest.kt‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -24,6 +24,7 @@ import com.duckduckgo.autofill.store.db.SecureStorageDatabase
2424
import com.duckduckgo.common.test.CoroutineTestRule
2525
import com.duckduckgo.data.store.api.DatabaseProvider
2626
import com.duckduckgo.data.store.api.RoomDatabaseConfig
27+
import com.duckduckgo.sqlcipher.loader.api.SqlCipherLoader
2728
import kotlinx.coroutines.CoroutineStart
2829
import kotlinx.coroutines.TimeoutCancellationException
2930
import kotlinx.coroutines.async
@@ -50,7 +51,7 @@ class RealSecureStorageDatabaseFactoryTest {
5051
val coroutineTestRule: CoroutineTestRule = CoroutineTestRule()
5152

5253
private val keyProvider: SecureStorageKeyProvider = mock()
53-
private val sqlCipherLoader: SqlCipherLibraryLoader = mock()
54+
private val sqlCipherLoader: SqlCipherLoader = mock()
5455
private val timeoutException: TimeoutCancellationException = mock()
5556

5657
// Use a test DatabaseProvider that creates in-memory databases

0 commit comments

Comments
 (0)