Skip to content

Ql4ql: Quality query tagging. #19931

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
michaelnebel merged 10 commits into from
Jul 17, 2025
Merged

Ql4ql: Quality query tagging. #19931

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
10 commits
Select commit Hold shift + click to select a range
cce1774
Ql4Ql: Re-factor the ql/mising-security-metadata query.
michaelnebel Jun 30, 2025
c46b528
Ql4Ql: Add some quality tag testcases.
michaelnebel Jun 30, 2025
e00b535
Ql4Ql: Add a check for quality tag consistency.
michaelnebel Jun 30, 2025
60a1d02
Ql4Ql: Add MissingQualityMetadata test.
michaelnebel Jun 30, 2025
af1c4e0
Ql4Ql: Share the definition of TestFile between multiple tests.
michaelnebel Jul 1, 2025
f58064e
Ql4Ql: Address review comments.
michaelnebel Jul 2, 2025
b79e2dd
Ql4Ql: Add some more quality tag testcases.
michaelnebel Jul 3, 2025
f810e17
Ql4Ql: Address review comments and update expected test output.
michaelnebel Jul 3, 2025
aefd941
Java/Javascript: Fix violations.
michaelnebel Jul 3, 2025
11c4a63
Quality tags: Clarify the quality sub-category tagging policy.
michaelnebel Jul 3, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Ql4Ql: Address review comments.
  • Loading branch information
@michaelnebel
michaelnebel committed Jul 3, 2025
commit f58064e119500396b54182d5fe91050f5350571e
4 changes: 2 additions & 2 deletions ql/ql/src/codeql_ql/ast/Ast.qll
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -232,8 +232,8 @@ class QueryDoc extends QLDoc {
result = this.getContents().regexpCapture("(?s).*@security\\-severity ([\\d\\.]+)\\s.*", 1)
}

/** Gets the individual @tags for the query. */
string getQueryTags() {
/** Gets the individual @tags for the query, if any. */
string getAQueryTag() {
exists(string tags | tags = this.getContents().regexpCapture("(?s).*@tags ([^@]+)", 1) |
result = tags.splitAt("*").trim() and
result.regexpMatch("[\\w\\s\\-]+")
Expand Down
18 changes: 9 additions & 9 deletions ql/ql/src/queries/style/MissingQualityMetadata.ql
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,18 +10,18 @@

import ql

private predicate hasQualityTag(QueryDoc doc) { doc.getQueryTags() = "quality" }
private predicate hasQualityTag(QueryDoc doc) { doc.getAQueryTag() = "quality" }

private predicate incorrectTopLevelCategorisation(QueryDoc doc) {
count(string s | s = doc.getQueryTags() and s = ["maintainability", "reliability"]) != 1
private predicate correctTopLevelCategorisation(QueryDoc doc) {
strictcount(string s | s = doc.getAQueryTag() and s = ["maintainability", "reliability"]) = 1
}

private predicate reliabilitySubCategory(QueryDoc doc) {
doc.getQueryTags() = ["correctness", "performance", "concurrency", "error-handling"]
doc.getAQueryTag() = ["correctness", "performance", "concurrency", "error-handling"]
}

private predicate maintainabilitySubCategory(QueryDoc doc) {
doc.getQueryTags() = ["readability", "useless-code", "complexity"]
doc.getAQueryTag() = ["readability", "useless-code", "complexity"]
}

from TopLevel t, QueryDoc doc, string msg
Expand All @@ -30,18 +30,18 @@ where
not t.getLocation().getFile() instanceof TestFile and
hasQualityTag(doc) and
(
incorrectTopLevelCategorisation(doc) and
not correctTopLevelCategorisation(doc) and
msg =
"This query file has incorrect top-level categorisation. It should have exactly one top-level category, either `@tags maintainability` or `@tags reliability`."
or
maintainabilitySubCategory(doc) and
not doc.getQueryTags() = "maintainability" and
not doc.getAQueryTag() = "maintainability" and
msg =
"This query file has a sub-category of maintainability but is missing the `@tags maintainability` tag."
or
reliabilitySubCategory(doc) and
not doc.getQueryTags() = "reliability" and
not doc.getAQueryTag() = "reliability" and
msg =
"This query file has a sub-category of reliability but is missing the `@tags reliability` tag."
)
select t, msg
select doc, msg
8 changes: 4 additions & 4 deletions ql/ql/src/queries/style/MissingSecurityMetadata.ql
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
/**
* @name Missing security metadata
* @description Security queries should have both a `@tag security` and a `@security-severity` tag.
* @description Security queries should have both a `@tags security` and a `@security-severity` tag.
* @kind problem
* @problem.severity warning
* @precision very-high
Expand All @@ -11,15 +11,15 @@
import ql

predicate missingSecuritySeverity(QueryDoc doc) {
doc.getQueryTags() = "security" and
doc.getAQueryTag() = "security" and
exists(doc.getQueryPrecision()) and
not exists(doc.getQuerySecuritySeverity())
}

predicate missingSecurityTag(QueryDoc doc) {
exists(doc.getQuerySecuritySeverity()) and
exists(doc.getQueryPrecision()) and
not doc.getQueryTags() = "security"
not doc.getAQueryTag() = "security"
}

from TopLevel t, QueryDoc doc, string msg
Expand All @@ -32,4 +32,4 @@ where
or
missingSecurityTag(doc) and msg = "This query file is missing a `@tags security`."
)
select t, msg
select doc, msg
8 changes: 4 additions & 4 deletions ql/ql/test/queries/style/MissingQualityMetadata/MissingQualityMetadata.expected
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
| testcases/BadQualityMaintainabilityWrongToplevel.ql:1:1:17:13 | TopLevel | This query file has a sub-category of reliability but is missing the `@tags reliability` tag. |
| testcases/BadQualityMultipleTopLevel.ql:1:1:17:13 | TopLevel | This query file has incorrect top-level categorisation. It should have exactly one top-level category, either `@tags maintainability` or `@tags reliability`. |
| testcases/BadQualityNoToplevel.ql:1:1:16:13 | TopLevel | This query file has incorrect top-level categorisation. It should have exactly one top-level category, either `@tags maintainability` or `@tags reliability`. |
| testcases/BadQualityReliabilityWrongToplevel.ql:1:1:17:13 | TopLevel | This query file has a sub-category of maintainability but is missing the `@tags maintainability` tag. |
| testcases/BadQualityMaintainabilityWrongToplevel.ql:1:1:11:3 | QueryDoc | This query file has a sub-category of reliability but is missing the `@tags reliability` tag. |
| testcases/BadQualityMultipleTopLevel.ql:1:1:11:3 | QueryDoc | This query file has incorrect top-level categorisation. It should have exactly one top-level category, either `@tags maintainability` or `@tags reliability`. |
| testcases/BadQualityNoToplevel.ql:1:1:10:3 | QueryDoc | This query file has incorrect top-level categorisation. It should have exactly one top-level category, either `@tags maintainability` or `@tags reliability`. |
| testcases/BadQualityReliabilityWrongToplevel.ql:1:1:11:3 | QueryDoc | This query file has a sub-category of maintainability but is missing the `@tags maintainability` tag. |
4 changes: 2 additions & 2 deletions ql/ql/test/queries/style/MissingSecurityMetadata/MissingSecurityMetadata.expected
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,2 +1,2 @@
| testcases/BadNoSecurity.ql:1:1:16:9 | TopLevel | This query file is missing a `@tags security`. |
| testcases/BadNoSeverity.ql:1:1:16:9 | TopLevel | This query file is missing a `@security-severity` tag. |
| testcases/BadNoSecurity.ql:1:1:10:3 | QueryDoc | This query file is missing a `@tags security`. |
| testcases/BadNoSeverity.ql:1:1:10:3 | QueryDoc | This query file is missing a `@security-severity` tag. |