Skip to content

Ruby/Python: Add a BlockMode concept for CryptographicOperations#9157

Merged
alexrford merged 22 commits into
github:mainfrom
alexrford:crypto-op-block-mode
Jun 13, 2022
Merged

Ruby/Python: Add a BlockMode concept for CryptographicOperations#9157
alexrford merged 22 commits into
github:mainfrom
alexrford:crypto-op-block-mode

Conversation

@alexrford
Copy link
Copy Markdown
Contributor

Cryptographic operations that use a block cipher such as AES often require a block mode of operation. A common case for this is in encrypting data of arbitrary length using a block cipher. These block modes can have inherent weaknesses that make them less suited to cryptographic purposes - in particular the electronic codebook (ECB) scheme can reveal patterns in data even if the underlying block cipher is secure.

The existing rb/weak-cryptographic-algorithm query checks for uses of ECB, but the message reported to the user was inaccurate, as in cases such as AES-ECB, it would report AES as being insecure rather than the ECB block mode. This is corrected in this PR by adding a new BlockMode concept with an isWeak member predicate and adding a BlockMode getBlockMode() predicate to CryptographicOperation alongside the existing CryptographicAlgorithm getAlgorithm() predicate.

I've made a similar change to the python version of this query. This mostly changes the cryptodome and cryptography libraries. I'm happy to remove the Python changes, or to move them out to a separate PR if that's more convenient than having everything in this PR.

Whilst implementing this, I noticed that the Ruby and Python CryptographicOperations are conceptually different. Ruby uses it to mean an instantiation of a cryptographic cipher or a use of that cipher (e.g. in encrypting some data), whereas it seems like Python uses it to mean only a use of such a cipher. I think that the Python version of this makes more sense, as the Ruby version seems like it could be unnecessarily noisy.

On another note, CryptographicOperation encompasses quite a lot of different operations, including many where a block mode is not relevant. This means that many implementations are just none(). This might suggest that CryptographicOperation should be subclassed for different cases (e.g. encrypting a stream of data, cryptographic hashing, etc).

@alexrford alexrford requested review from a team as code owners May 13, 2022 15:46
@alexrford alexrford requested a review from a team as a code owner May 16, 2022 14:49
erik-krogh
erik-krogh previously approved these changes May 16, 2022
/**
* Holds if `name` corresponds to a weak block cipher mode of operation.
*/
predicate isWeakBlockMode(string name) { name = "ECB" }
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is technically a breaking change, but it's inside an /internal/ folder, so it's fine.

JS 👍

}

override predicate hasActualResult(Location location, string element, string tag, string value) {
exists(location.getFile().getRelativePath()) and
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's this line for?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copied this from another test but it's not relevant here. Deleted.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it's probably from a Python test. Since we extract dependencies/standard library, sometimes we will get results in code that is not user-written, which we don't want.

hmac
hmac previously approved these changes May 17, 2022
Copy link
Copy Markdown
Contributor

@hmac hmac left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The Ruby changes look good to me! I wonder, could we directly share the CryptographicOperation concept between Python and Ruby by moving it to ConceptsShared.qll?

@alexrford alexrford dismissed stale reviews from hmac and erik-krogh via f92782d May 17, 2022 13:58
@alexrford
Copy link
Copy Markdown
Contributor Author

@hmac thanks for the review. I noticed a regression in the DCA results where I was making an incorrect assumption that ciphers instantiated with OpenSSL::Cipher.new always used CBC, which was not the case when the argument to new included an explicit block mode (e.g. 'bf-ecb') or when the argument specified a stream cipher. I've fixed this in f92782d

The Ruby changes look good to me! I wonder, could we directly share the CryptographicOperation concept between Python and Ruby by moving it to ConceptsShared.qll?

Yeah, I think this makes sense. I held off doing this mostly because Javascript implements weak-cryptographic-algorithm differently to Python and Ruby (the Ruby version of this query was based on the Python version), so the CryptographicOperation concept doesn't currently exist in JS. Naively, I could imagine this concept being useful for JS in some circumstances as well, though I'm not sure if we want to add concepts that don't have any concrete implementations.

@hmac
Copy link
Copy Markdown
Contributor

hmac commented May 17, 2022

The way that shared concepts are structured, JS can (I believe) opt to selectively import the ones they want, as a stop-gap until they can adopt the concept properly. So if we have good alignment across Python and Ruby, I think that's a good enough reason to share the concept. @RasmusWL would you agree?

@RasmusWL
Copy link
Copy Markdown
Member

The Ruby changes look good to me! I wonder, could we directly share the CryptographicOperation concept between Python and Ruby by moving it to ConceptsShared.qll?

Yeah, I think this makes sense. I held off doing this mostly because Javascript implements weak-cryptographic-algorithm differently to Python and Ruby (the Ruby version of this query was based on the Python version), so the CryptographicOperation concept doesn't currently exist in JS. Naively, I could imagine this concept being useful for JS in some circumstances as well, though I'm not sure if we want to add concepts that don't have any concrete implementations.

👋 just following a bit along on the sidelines, since I should be reviewing this PR soon. It's fine to share the concept between Ruby and Python initially, and let JS adopt it later on.

@alexrford
Copy link
Copy Markdown
Contributor Author

Cool, I will have a go at moving this into ConceptsShared.qll then.

Copy link
Copy Markdown
Member

@RasmusWL RasmusWL left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice 🎉 Thanks for porting this to Python as well 💪 A few minor things, otherwise looks good to me 👍

Comment thread python/ql/src/change-notes/2022-05-16-broken-crypto-block-mode.md Outdated
Comment thread python/ql/lib/semmle/python/frameworks/Cryptography.qll Outdated
Comment thread ruby/ql/src/change-notes/2022-05-16-broken-crypto-message.md Outdated
alexrford and others added 4 commits May 19, 2022 14:25
Co-authored-by: Rasmus Wriedt Larsen <rasmuswriedtlarsen@gmail.com>
… modes

Co-authored-by: Rasmus Wriedt Larsen <rasmuswriedtlarsen@gmail.com>
Comment thread javascript/ql/lib/semmle/javascript/internal/ConceptsShared.qll Fixed
Comment thread javascript/ql/lib/semmle/javascript/internal/ConceptsShared.qll Fixed
Comment thread javascript/ql/lib/semmle/javascript/internal/ConceptsShared.qll Fixed
Comment thread javascript/ql/lib/semmle/javascript/internal/ConceptsShared.qll Fixed
Comment thread javascript/ql/lib/semmle/javascript/internal/ConceptsShared.qll Fixed
Comment thread javascript/ql/lib/semmle/javascript/internal/ConceptsShared.qll Fixed
Comment thread javascript/ql/lib/semmle/javascript/internal/ConceptsShared.qll Fixed
Comment thread javascript/ql/lib/semmle/javascript/internal/ConceptsShared.qll Fixed
Comment thread javascript/ql/lib/semmle/javascript/internal/ConceptsShared.qll Fixed
Comment thread javascript/ql/lib/semmle/javascript/internal/ConceptsShared.qll Fixed
@alexrford
Copy link
Copy Markdown
Contributor Author

I've run into a somewhat interesting issue in sharing the CryptographicOperation concept. In the Ruby version of this, we currently have an isWeak() member predicate. This was previously used in the broken crypto query, but has been replaced by (effectively) op.getAlgorithm().isWeak() or op.getBlockMode().isWeak(), which is the same approach used in the Python version of the query in this PR.

This predicate is now unused in Ruby. I think it should probably be deprecated and eventually removed - whilst an operation may be insecure in other ways (e.g. insufficient key size), I think we'd still usually want to expose to a user why the operation is weak - something that a simple predicate on the operation alone can't communicate.

I've approached this in the current PR by:

  • Adding the CryptographicOperation#isWeak() predicate to the shared concept
  • Adding a default implementation to the abstract class CryptographicOperation::Range, which lets us avoid implementing this predicate for Python
  • Immediately deprecating this predicate

It doesn't feel great to add a predicate which is then immediately deprecated, but it seemed like the least bad solution which complied with the deprecation policy.

@hmac
Copy link
Copy Markdown
Contributor

hmac commented May 23, 2022

It doesn't feel great to add a predicate which is then immediately deprecated, but it seemed like the least bad solution which complied with the deprecation policy.

Yeah, this is a shame. I personally don't think we should be following the strict deprecation policy for Ruby while it is in beta, since it makes it harder to clean things up like this.

@alexrford
Copy link
Copy Markdown
Contributor Author

It doesn't feel great to add a predicate which is then immediately deprecated, but it seemed like the least bad solution which complied with the deprecation policy.

Yeah, this is a shame. I personally don't think we should be following the strict deprecation policy for Ruby while it is in beta, since it makes it harder to clean things up like this.

I'd agree - our policy on 1 year deprecation periods applies to stable releases, which arguably means that beta languages should not be included. The policy should perhaps be more explicit about languages in beta.

@RasmusWL has avoided adding the deprecated isWeak() predicate in the latest commit to this branch, which at least avoids us adding the deprecated predicate to Python.

The old code was my own suggestion, that I thought would just work, but
was also slightly skeptical about.

I tested out whether it works with the code below

```codeql
predicate foo(int input, string res) {
  input = 1 and res = "that was one"
}

from int input, string res
where
  input in [1, 2] and
  if foo(input, res)
  then any()
  else res = "not one"
select input, res
```

which gave the 3 results

```
1 |	that was one
1 |	not one
2 |	not one
```

only by rewriting the code to be the one below, did I get down to the 2
results I actually wanted. So I've done the same kind of rewrite in the
commit.

```codeql
predicate foo(int input, string res) {
  input = 1 and res = "that was one"
}

from int input, string res
where
  input in [1, 2] and
  if foo(input, _)
  then foo(input, res)
  else res = "not one"
select input, res
```
Copy link
Copy Markdown
Member

@RasmusWL RasmusWL left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just double checked my suggestion in #9157 (comment), and it was not working as intended, so I just fixed that up. (and happy to have done that before the PR was merged ☺️)

But overall looks good to me now 👍 (not sure whether this is blocked on anything at the moment?)

@alexrford
Copy link
Copy Markdown
Contributor Author

alexrford commented May 30, 2022

I just double checked my suggestion in #9157 (comment), and it was not working as intended, so I just fixed that up. (and happy to have done that before the PR was merged ☺️)

But overall looks good to me now 👍 (not sure whether this is blocked on anything at the moment?)

Thanks for the fix, I see the problem now after reading through the commit message. It's a shame that the any() version has some spurious tuples as I find myself wanting to express "either something that satisfies the given predicate, or otherwise a default value" fairly often and this looked like a neat way to do it without any repetition.

I'll just wait on a final approval from @github/codeql-ruby before merging this.

Copy link
Copy Markdown
Contributor

@aibaars aibaars left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me.

@alexrford alexrford merged commit 8d195e3 into github:main Jun 13, 2022
@alexrford alexrford deleted the crypto-op-block-mode branch June 13, 2022 19:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants