Skip to content

C#: Overlay extraction support. #20425

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
michaelnebel merged 27 commits into from
Nov 12, 2025
Merged

C#: Overlay extraction support. #20425

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
b600255
C#: Enable overlay compilation in lib/qlpack.yml.
michaelnebel Oct 21, 2025
bcc6234
C#: Fix bad join due to overlay compilation.
michaelnebel Oct 29, 2025
b48ce8d
C#: Add overlay builtins to the dbscheme.
michaelnebel Sep 12, 2025
cfb6843
C#: Add upgrade- and downgrade scripts.
michaelnebel Sep 12, 2025
9026a5a
C#: Turn on overlay support in codeql-extractor.yml.
michaelnebel Sep 12, 2025
aa80558
C#: Add functionality to detect overlay mode and integrate in extract…
michaelnebel Sep 12, 2025
cab9d81
C#: Add unit test.
michaelnebel Sep 12, 2025
61f6512
C#: Sprinkle uses of OnlyScaffold to extract less when in overlay mode.
michaelnebel Sep 17, 2025
76ac2df
C#: Write overlay metadata at end of extraction.
michaelnebel Oct 21, 2025
121c150
C#: Define discarding predicates for expressions, statements, locatio…
michaelnebel Oct 23, 2025
8a34421
C#: Add QL overlay tests.
michaelnebel Oct 27, 2025
40035d2
C#: Add a locatable type to the dbscheme.
michaelnebel Oct 27, 2025
8927b0a
C#: Simplify the discarding and also discard type mentions and comments.
michaelnebel Oct 27, 2025
04d33cb
C#: Add discarding for diagnostics and extractor messages.
michaelnebel Oct 28, 2025
1d2f154
C#: Add change-note.
michaelnebel Oct 30, 2025
504bb9c
C#: Only scaffold assemblies in overlay mode, only extract expression…
michaelnebel Nov 3, 2025
d95ebc7
C#: Add using directives and type mentions as star entities.
michaelnebel Nov 3, 2025
7c670cd
C#: Address review comments and make more early returns in Populate.
michaelnebel Nov 7, 2025
1657dfb
C#: Remove expression population safeguard and guard creation of cons…
michaelnebel Nov 7, 2025
9d300e3
C#: Address comments in the QL implementation.
michaelnebel Nov 7, 2025
ded1328
C#: Do not extract comments when scaffolding.
michaelnebel Nov 10, 2025
43118ec
C#: The extraction of the TypeMentions for return type and explicit i…
michaelnebel Nov 10, 2025
0a16cf6
C#: Do not require that comments and type locations are in source in …
michaelnebel Nov 10, 2025
c44b747
C#: Minor code quality improvements.
michaelnebel Nov 10, 2025
d6b7424
C#: Add the same strategy in as in Java for XML element discarding.
michaelnebel Nov 10, 2025
3492811
C#: Add XML overlay tests.
michaelnebel Nov 10, 2025
0541dcc
C#: Add discarding for ASP elements.
michaelnebel Nov 10, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
C#: Do not extract comments when scaffolding.
  • Loading branch information
@michaelnebel
michaelnebel committed Nov 10, 2025
commit ded13281034e284b25631dddc51208b0cc20daa7
2 changes: 1 addition & 1 deletion csharp/extractor/Semmle.Extraction.CSharp/Entities/Base/CachedSymbol.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -109,7 +109,7 @@ public virtual IEnumerable<Location> Locations
/// </summary>
protected void BindComments()
{
if (!Symbol.IsImplicitlyDeclared && IsSourceDeclaration && Symbol.FromSource() && !Context.OnlyScaffold)
if (!Symbol.IsImplicitlyDeclared && IsSourceDeclaration && Symbol.FromSource())
Context.BindComments(this, FullLocation);
}

Expand Down
8 changes: 6 additions & 2 deletions csharp/extractor/Semmle.Extraction.CSharp/Entities/CommentBlock.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,12 +10,12 @@ private CommentBlock(Context cx, Comments.CommentBlock init)

public override void Populate(TextWriter trapFile)
{
trapFile.commentblock(this);
Symbol.CommentLines.ForEach((l, child) => trapFile.commentblock_child(this, l, child));
if (Context.OnlyScaffold)
Copy link
Copy Markdown
Contributor

@hvitved hvitved Nov 10, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would have thought that we could simply move this early return up to the beginning of Populate?

Copy link
Copy Markdown
Contributor

@hvitved hvitved Nov 10, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually, now that I think about it, shouldn't we instead be guarding all the places where we extract comments (such as calls to CommentPopulator.ExtractCommentBlocks,CommentPopulator.ExtractComment, and inside Context.BindComments)?

Copy link
Copy Markdown
Contributor Author

@michaelnebel michaelnebel Nov 10, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would like to move this to the begging on the populate method and then also guard the BindTo method. This should be internally consistent and not cause any dangling references (I will also push the scaffolding guard from CachedEntitiy to the Context for comment binding).
Will add one more commit for this.

Also note that for TypeMentions, the guard has been moved to the top of the populate method (as creating these will not cause any dangling references). Most places in the populate methods the TypeMention creation has been "moved below" the scaffolding early return check (and thus not be created), but I haven't done so in all places as this will create a large amount of extra guarding.

Copy link
Copy Markdown
Contributor Author

@michaelnebel michaelnebel Nov 10, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, I will also put in the guards the places where you suggest to avoid all the string processing before the actual comments are created.

{
return;
}
trapFile.commentblock(this);
Symbol.CommentLines.ForEach((l, child) => trapFile.commentblock_child(this, l, child));
WriteLocationToTrap(trapFile.commentblock_location, this, Context.CreateLocation(Symbol.Location));
}

Expand All @@ -31,6 +31,10 @@ public override void WriteId(EscapingTextWriter trapFile)

public void BindTo(Label entity, CommentBinding binding)
{
if (Context.OnlyScaffold)
{
return;
}
Context.TrapWriter.Writer.commentblock_binding(this, entity, binding);
}

Expand Down
2 changes: 1 addition & 1 deletion csharp/extractor/Semmle.Extraction.CSharp/Entities/CommentLine.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,11 +21,11 @@ private CommentLine(Context cx, Microsoft.CodeAnalysis.Location loc, CommentLine

public override void Populate(TextWriter trapFile)
{
trapFile.commentline(this, Type == CommentLineType.MultilineContinuation ? CommentLineType.Multiline : Type, Text, RawText);
if (Context.OnlyScaffold)
{
return;
}
trapFile.commentline(this, Type == CommentLineType.MultilineContinuation ? CommentLineType.Multiline : Type, Text, RawText);
location = Context.CreateLocation(Location);
WriteLocationToTrap(trapFile.commentline_location, this, location);

Expand Down
4 changes: 4 additions & 0 deletions csharp/extractor/Semmle.Extraction.CSharp/Extractor/Context.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -631,6 +631,10 @@ public Entities.Location CreateLocation(Microsoft.CodeAnalysis.Location? locatio
/// <param name="l">Location of the entity.</param>
public void BindComments(Entity entity, Microsoft.CodeAnalysis.Location? l)
{
if (OnlyScaffold)
{
return;
}
var duplicationGuardKey = GetCurrentTagStackKey();
CommentGenerator.AddElement(entity.Label, duplicationGuardKey, l);
}
Expand Down
8 changes: 8 additions & 0 deletions csharp/extractor/Semmle.Extraction.CSharp/Populators/CommentPopulator.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,10 @@ internal static class CommentPopulator
{
public static void ExtractCommentBlocks(Context cx, CommentProcessor gen)
{
if (cx.OnlyScaffold)
{
return;
}
cx.Try(null, null, () =>
{
gen.GenerateBindings((entity, duplicationGuardKey, block, binding) =>
Expand All @@ -34,6 +38,10 @@ public static void ExtractCommentBlocks(Context cx, CommentProcessor gen)

public static void ExtractComment(Context cx, SyntaxTrivia trivia)
{
if (cx.OnlyScaffold)
{
return;
}
switch (trivia.Kind())
{
case SyntaxKind.SingleLineDocumentationCommentTrivia:
Expand Down