YAML: Extract comments#21935
Open
MathiasVP wants to merge 14 commits into
Open
Conversation
08b6b0f to
8049dec
Compare
c89f174 to
3a88dd3
Compare
0ec56f6 to
b44f0dc
Compare
Contributor
There was a problem hiding this comment.
Pull request overview
This PR extends the shared YAML extraction model to include YAML comments, adding a new yaml_comments relation/type to the relevant dbschemes and exposing a YamlComment class through the shared YAML QL library and per-language YAML adapters.
Changes:
- Add
yaml_comments(and@yaml_comment) to YAML dbscheme fragments and include comments in@yaml_locatable. - Extend the shared YAML library (
codeql.yaml.Yaml) with a comment base type and aYamlCommentclass. - Add a JavaScript library test exercising
YamlComment.getText().
Show a summary per file
| File | Description |
|---|---|
| unified/ql/lib/unified.dbscheme | Adds yaml_comments and @yaml_comment to the unified YAML schema fragment. |
| shared/yaml/codeql/yaml/Yaml.qll | Introduces comment support in the shared YAML QL API (CommentBase, YamlComment). |
| rust/ql/lib/upgrades/77e9a70be4b0cf5ecb1d4c1d841b2d970715a912/upgrade.properties | Declares a Rust db upgrade for YAML comment extraction. |
| rust/ql/lib/rust.dbscheme | Adds yaml_comments and extends @yaml_locatable for Rust DBs. |
| rust/downgrades/e1bce498ef78280ebe0a32b1d9d6f26c96eaf41f/upgrade.properties | Declares Rust downgrade behavior for yaml_comments. |
| ruby/ql/lib/upgrades/29b7b6fc1982422368cb0a4644fd0c81f993c618/upgrade.properties | Declares a Ruby db upgrade for YAML comment extraction. |
| ruby/ql/lib/ruby.dbscheme | Adds yaml_comments and extends @yaml_locatable for Ruby DBs. |
| ruby/downgrades/d6f4c73dc33d28aebcffd53ba080eeecc99470f5/upgrade.properties | Declares Ruby downgrade behavior for yaml_comments. |
| ql/ql/src/ql.dbscheme | Adds yaml_comments and extends @yaml_locatable for QL extractor DBs. |
| ql/ql/src/codeql_ql/ast/Yaml.qll | Wires YAML comment entities into the QL AST YAML adapter. |
| ql/ql/src/codeql_ql/ast/Ast.qll | Wires YAML comment entities into the broader QL AST YAML module. |
| python/ql/lib/upgrades/eb5fc917c79bb23ce2de4a022f3e566d57a91be9/upgrade.properties | Declares a Python db upgrade for YAML comment extraction. |
| python/ql/lib/semmlecode.python.dbscheme.stats | Updates Python dbscheme stats to include @yaml_comment/yaml_comments. |
| python/ql/lib/semmlecode.python.dbscheme | Adds yaml_comments and extends @yaml_locatable for Python DBs. |
| python/ql/lib/semmle/python/Yaml.qll | Wires YAML comment entities into the Python YAML adapter. |
| python/downgrades/b7745eb2df865c97e50b7803956a82988716e29a/upgrade.properties | Declares Python downgrade behavior for yaml_comments. |
| python/downgrades/b7745eb2df865c97e50b7803956a82988716e29a/semmlecode.python.dbscheme | Provides the downgraded Python dbscheme variant without yaml_comments. |
| javascript/ql/test/library-tests/Comments/YamlComments.ql | Adds a JS library test query selecting YamlComment text. |
| javascript/ql/test/library-tests/Comments/YamlComments.expected | Expected results for the new JS YAML comment test. |
| javascript/ql/test/library-tests/Comments/comments.yml | YAML fixture used by the JS comment test. |
| javascript/ql/lib/upgrades/26a123164be893893e2aa0374d820785decf55af/upgrade.properties | Declares a JavaScript db upgrade for YAML comment extraction. |
| javascript/ql/lib/upgrades/26a123164be893893e2aa0374d820785decf55af/semmlecode.javascript.dbscheme | Upgrade dbscheme snapshot including yaml_comments for JavaScript. |
| javascript/ql/lib/upgrades/26a123164be893893e2aa0374d820785decf55af/old.dbscheme | Prior JavaScript dbscheme snapshot for the upgrade. |
| javascript/ql/lib/semmlecode.javascript.dbscheme.stats | Updates JS dbscheme stats to include @yaml_comment/yaml_comments. |
| javascript/ql/lib/semmlecode.javascript.dbscheme | Adds yaml_comments and extends @yaml_locatable for JS DBs. |
| javascript/ql/lib/semmle/javascript/YAML.qll | Wires YAML comment entities into the JS YAML adapter. |
| javascript/extractor/tests/yaml/output/trap/orig.yml.trap | Updates extractor trap output to include extracted YAML comments. |
| javascript/extractor/tests/yaml/output/trap/emoji_buffer_boundary.yml.trap | Updates extractor trap output to include extracted YAML comments (boundary case). |
| javascript/downgrades/ce4a5f401c03a70b0595e71bdc20612d82fa4e67/upgrade.properties | Declares JavaScript downgrade behavior for yaml_comments. |
| javascript/downgrades/ce4a5f401c03a70b0595e71bdc20612d82fa4e67/semmlecode.javascript.dbscheme | Provides the downgraded JS dbscheme variant. |
| javascript/downgrades/ce4a5f401c03a70b0595e71bdc20612d82fa4e67/old.dbscheme | Current JS dbscheme snapshot used as the downgrade source. |
| actions/ql/lib/codeql/actions/ast/internal/Yaml.qll | Wires YAML comment entities into the Actions YAML adapter. |
| actions/ql/lib/codeql/actions/ast/internal/Ast.qll | Exposes YAML comments as Comment in the Actions AST layer. |
Copilot's findings
- Files reviewed: 40/44 changed files
- Comments generated: 1
Comment on lines
+25
to
+26
| ... # document end comment | ||
| # final comment after document end No newline at end of file |
b44f0dc to
44c8a97
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR adds support for extraction of YAML comments to all the languages which makes use of the YAML extractor.
To keep this PR simple I've decided to not include the new
YamlCommentclass into the existingCommentclasses for each language. I suggest we do that as a per-language follow-up if necessary.(All the force-pushes is my fumbling with up- and downgrade scripts. The DCA runs should still be representative of the changes)