Skip to content

Commit 357d5a8

Browse files
zoraconzoracon
authored
DuckDuckGo Smarter Encryption Update Channel (EFForg#20049)
* Add Architecural Design decision doc for DuckDuckGo Smart Encryption - Update License TXT reflecting changes * Add note about manual ruleset changes ending in contributor guide * add path for new update channel * add path for new update channel * Adding Smarter Encryption Update Channel - Adding documentation explaining new update channel - Add update channel - Add generated bloom files for transparency * Update test condition * Fix branding typos
1 parent cdbd95a commit 357d5a8

8 files changed

Lines changed: 107 additions & 21 deletions

File tree

‎CONTRIBUTING.md‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -142,6 +142,8 @@ HTTPS Everywhere is maintained by a limited set of staff and volunteers. Please
142142

143143
### General Info
144144

145+
**On May 31st, 2021 we will end manual additions to the rulesets.** Please see [this explanation on the future of HTTPSE Rulesets.](`https://github.com/EFForg/https-everywhere/blob/master/docs/adr/duckduckgo-smarter-encryption.md`)
146+
145147
Thanks for your interest in contributing to the HTTPS Everywhere `rulesets`! There's just a few things you should know before jumping in. First some terminology, which will help you understand how exactly `rulesets` are structured and what each one contains:
146148

147149
* `ruleset`: a scope in which `rules`, `targets`, and `tests` are contained. `rulesets` are usually named after the entity which controls the group of `targets` contained in it. There is one `ruleset` per XML file within the `src/chrome/content/rules` directory.

‎LICENSE.txt‎

Lines changed: 5 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -5,8 +5,11 @@ Licensed GPL v2+
55
HTTPS Everywhere Rulesets (src/chrome/content/rules):
66
To the extent copyright applies to the rulesets, they can be used according to GPL v2 or later.
77

8+
The DuckDuckGo Smarter Encryption list, utilized by HTTPS Everywhere at https://www.https-rulesets.org/ddgse, is publicly available under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0. International license.
9+
If you'd like to license the list for commercial use, please reach out to: https://help.duckduckgo.com/duckduckgo-help-pages/company/contact-us/
10+
811
Issue Format Bot (utils/issue-format-bot/*):
912
Copyright © 2017 AJ Jordan, AGPLv3+
1013

11-
The build system incorporates code from Python 3.6
12-
Copyright © 2001-2018 Python Software Foundation; All Rights Reserved
14+
The build system incorporates code from Python 3.6 and Python 3 respectively
15+
Copyright © 2001-2018 Python Software Foundation; All Rights Reserved

‎chromium/background-scripts/update_channels.js‎

Lines changed: 35 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -4,22 +4,41 @@
44

55
(function (exports) {
66

7-
exports.update_channels = [{
8-
name: 'EFF (Full)',
9-
jwk: {
10-
kty: 'RSA',
11-
e: 'AQAB',
12-
n: '1cwvFQu3Kw-Pz8bcEFuV5zx0ZheDsc4Tva7Qv6BL90_sDLqCW79Y543nDkPtNVfFH_89pt2kSPp_IcS5XnYiw6zBQeFuILFw5JpvZt14K0s4' +
13-
'e025Q9CXfhYKIBKT9PnqihwAacjMa6rQb7RTu7XxVvqxRb3b0vx2CR40LSlYZ8H_KpeaUwq2oz-fyrI6LFTeYvbO3ZuLKeK5xV1a32xeTVMF' +
14-
'kIj3LxnQalxq-DRHfj7LRRoTnbRDW4uoDc8aVpLFliuO79jUKbobz4slpiWJ4wjKR_O6OK13HbZUiOSxi8Bms-UqBPOyzbMVpmA7lv_zWdaL' +
15-
'u1IVlVXQyLVbbrqI6llRqfHdcJoEl-eC48AofuB-relQtjTEK_hyBf7sPwrbqAarjRjlyEx6Qy5gTXyxM9attfNAeupYR6jm8LKm6TFpfWky' +
16-
'DxUmj_f5pJMBWNTomV74f8iQ2M18_KWMUDCOf80tR0t21Q1iCWdvA3K_KJn05tTLyumlwwlQijMqRkYuao-CX9L3DJIaB3VPYPTSIPUr7oi1' +
17-
'6agsuamOyiOtlZiRpEvoNg2ksJMZtwnj5xhBQydkdhMW2ZpHDzcLuZlhJYZL_l3_7wuzRM7vpyA9obP92CpZRFJErGZmFxJC93I4U9-0B0wg' +
18-
'-sbyMKGJ5j1BWTnibCklDXtWzXtuiz18EgE'
7+
exports.update_channels = [
8+
{
9+
name: 'EFF (Full)',
10+
jwk: {
11+
kty: 'RSA',
12+
e: 'AQAB',
13+
n: '1cwvFQu3Kw-Pz8bcEFuV5zx0ZheDsc4Tva7Qv6BL90_sDLqCW79Y543nDkPtNVfFH_89pt2kSPp_IcS5XnYiw6zBQeFuILFw5JpvZt14K0s4' +
14+
'e025Q9CXfhYKIBKT9PnqihwAacjMa6rQb7RTu7XxVvqxRb3b0vx2CR40LSlYZ8H_KpeaUwq2oz-fyrI6LFTeYvbO3ZuLKeK5xV1a32xeTVMF' +
15+
'kIj3LxnQalxq-DRHfj7LRRoTnbRDW4uoDc8aVpLFliuO79jUKbobz4slpiWJ4wjKR_O6OK13HbZUiOSxi8Bms-UqBPOyzbMVpmA7lv_zWdaL' +
16+
'u1IVlVXQyLVbbrqI6llRqfHdcJoEl-eC48AofuB-relQtjTEK_hyBf7sPwrbqAarjRjlyEx6Qy5gTXyxM9attfNAeupYR6jm8LKm6TFpfWky' +
17+
'DxUmj_f5pJMBWNTomV74f8iQ2M18_KWMUDCOf80tR0t21Q1iCWdvA3K_KJn05tTLyumlwwlQijMqRkYuao-CX9L3DJIaB3VPYPTSIPUr7oi1' +
18+
'6agsuamOyiOtlZiRpEvoNg2ksJMZtwnj5xhBQydkdhMW2ZpHDzcLuZlhJYZL_l3_7wuzRM7vpyA9obP92CpZRFJErGZmFxJC93I4U9-0B0wg' +
19+
'-sbyMKGJ5j1BWTnibCklDXtWzXtuiz18EgE'
20+
},
21+
update_path_prefix: 'https://www.https-rulesets.org/v1/',
22+
scope: '',
23+
replaces_default_rulesets: true
1924
},
20-
update_path_prefix: 'https://www.https-rulesets.org/v1/',
21-
scope: '',
22-
replaces_default_rulesets: true
23-
}];
25+
{
26+
name: 'DuckDuckGo Smarter Encryption',
27+
format: 'bloom',
28+
jwk: {
29+
kty: 'RSA',
30+
e: 'AQAB',
31+
n: '1cwvFQu3Kw-Pz8bcEFuV5zx0ZheDsc4Tva7Qv6BL90_sDLqCW79Y543nDkPtNVfFH_89pt2kSPp_IcS5XnYiw6zBQeFuILFw5JpvZt14K0s4' +
32+
'e025Q9CXfhYKIBKT9PnqihwAacjMa6rQb7RTu7XxVvqxRb3b0vx2CR40LSlYZ8H_KpeaUwq2oz-fyrI6LFTeYvbO3ZuLKeK5xV1a32xeTVMF' +
33+
'kIj3LxnQalxq-DRHfj7LRRoTnbRDW4uoDc8aVpLFliuO79jUKbobz4slpiWJ4wjKR_O6OK13HbZUiOSxi8Bms-UqBPOyzbMVpmA7lv_zWdaL' +
34+
'u1IVlVXQyLVbbrqI6llRqfHdcJoEl-eC48AofuB-relQtjTEK_hyBf7sPwrbqAarjRjlyEx6Qy5gTXyxM9attfNAeupYR6jm8LKm6TFpfWky' +
35+
'DxUmj_f5pJMBWNTomV74f8iQ2M18_KWMUDCOf80tR0t21Q1iCWdvA3K_KJn05tTLyumlwwlQijMqRkYuao-CX9L3DJIaB3VPYPTSIPUr7oi1' +
36+
'6agsuamOyiOtlZiRpEvoNg2ksJMZtwnj5xhBQydkdhMW2ZpHDzcLuZlhJYZL_l3_7wuzRM7vpyA9obP92CpZRFJErGZmFxJC93I4U9-0B0wg' +
37+
'-sbyMKGJ5j1BWTnibCklDXtWzXtuiz18EgE'
38+
},
39+
update_path_prefix: 'https://www.https-rulesets.org/ddg/',
40+
scope: '',
41+
}
42+
];
2443

2544
})(typeof exports === 'undefined' ? require.scopes.update_channels = {} : exports);

‎chromium/test/update_test.js‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -40,9 +40,9 @@ describe('update.js', function() {
4040
resolve();
4141
}));
4242

43-
if(apply_promises.length == update_channels.length) {
44-
Promise.all(apply_promises).then(() => done());
45-
}
43+
44+
Promise.all(apply_promises).then(() => done());
45+
4646
}});
4747

4848
});

‎docs/adrs/bloom-filter-rule-signing.md‎

Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,28 @@
1+
# Bloom Filters and Async Rust for Ruleset Signing
2+
3+
* Status: Deployed
4+
* Deciders: EFF (@zoracon and @hainish)
5+
* Deploy Date: 2021-03-03
6+
7+
## Context and Problem Statement
8+
9+
With larger ruleset lists to be signed on the DuckDuckGo Update channel, a better way to digest and form ruleset files were needed.
10+
11+
## Decision Drivers
12+
13+
* Bloom filters are able to ingest greater data sets at less memory expense
14+
* Rust is already incorporated in HTTPS Everywhere and is a memory safe language
15+
16+
## Decision Outcome
17+
18+
Created an async Rust script that ingests DuckDuckGo's Smarter Encryption list, compares to the Majestic Million list, and forms a bloom file and associated metadata.
19+
20+
### Consequences and Concerns
21+
22+
An accepted false positive is declared when the filter is generated.
23+
24+
[Comment](https://github.com/EFForg/https-everywhere/pull/19910#issuecomment-771102775)
25+
26+
## Links for Further Context
27+
* [Bloom Filter Script](https://github.com/EFForg/generate-smarter-encryption-bloom-filter)
28+

‎docs/adrs/duckduckgo-smarter-encryption.md‎

Lines changed: 33 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,33 @@
1+
# Incorporating DuckDuckGo Smarter Encryption
2+
3+
* Status: Pending
4+
* Deciders: EFF (@zoracon and @hainish) and DuckDuckGo
5+
* Deploy Date: 2021-04-15
6+
7+
## Context and Problem Statement
8+
9+
With the increased HTTPS traffic, the current model of listed sites that support HTTPS is no longer a maintenance task that makes sense to uphold.
10+
11+
## Decision Drivers
12+
13+
* Firefox has an HTTPS-Only option
14+
* Browsers and websites are moving away from issues that created need for more granular ruleset maintenance.
15+
* Mixed content is now blocked in major browsers
16+
* Different domains for secure connection are now an older habit (i.e. secure.google.com)
17+
* TLS 1.0, 1.1 deprecation
18+
* Chrome’s Manifest V3 will force the extensions to have a ruleset cap. Instead of competing with other extensions like DuckDuckGo, if the user prefers to use HTTPS Everywhere or DuckDuckGo's privacy essentials, we will provide the same coverage.
19+
* DuckDuckGo’s Smarter Encryption covers more domains than our current, more manual model.
20+
21+
## Decision Outcome
22+
23+
We chose to add the DuckDuckGo Smarter Encryption update channel, because it no longer is beneficial to diverse efforts with others with similar goals in this space.
24+
25+
### Consequences and Concerns
26+
27+
* We have many downstream partners supported and unofficial that rely on our current rulesets. This transition gives them time to make the needed decisions on their before we completely switch over to using DuckDuckGo's Smarter Encryption, and sunset our current rulesets in HTTPS Everywhere
28+
* …
29+
30+
## Links for Further Context
31+
32+
* https://spreadprivacy.com/duckduckgo-smarter-encryption/
33+
* https://www.eff.org/deeplinks/2020/11/10-years-https-everywhere

‎utils/sign-bloom/ddgse‎

1.35 MB
Binary file not shown.

‎utils/sign-bloom/ddgse.json‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
{"bitmap_bits":11364392,"k_num":24,"sha256sum":"371bfb628062de163947c1226d99a5f3823e6fe1bc0838d24e0deffad1c96ee2","sip_keys":[["4746789603923246281","9835731802354323261"],["11277193235900924841","7296056854142719726"]]}

0 commit comments

Comments
 (0)