Skip to content

C#: Implement support for path transformers#4212

Merged
hvitved merged 5 commits into
github:rc/1.25from
hvitved:csharp/path-transformers
Sep 22, 2020
Merged

C#: Implement support for path transformers#4212
hvitved merged 5 commits into
github:rc/1.25from
hvitved:csharp/path-transformers

Conversation

@hvitved
Copy link
Copy Markdown
Contributor

@hvitved hvitved commented Sep 4, 2020

Overview

This PR implements support for path transformers (specified using the CODEQL_PATH_TRANSFORMER environment variable) in the C# extractor. Paths are transformed in the following places:

  • When populating the files relation.
  • When populating the folders relation.
  • When adding files to the source archive.
  • When creating TRAP files.

While it is strictly speaking not needed for TRAP files, it maintains the invariant that each source code file in the source archive is mapped to a .trap.gz file at the same relative location.

Testing

macOS

I have tested locally on macOS with this transformer file

#src
/<path-to-codeql-checkout>/csharp/autobuilder//

and this build command

CODEQL_PATH_TRANSFORMER=<path-to-transformer-file> codeql database create --language=csharp --keep-trap --command "dotnet build csharp/CSharp.sln /t:rebuild /p:UseSharedCompilation=false" -- db

Windows

I have tested locally on Windows with this transformer file

#src
C:\<path-to-codeql-checkout>\csharp\autobuilder//

and this build command

set CODEQL_PATH_TRANSFORMER=<path-to-transformer-file>
codeql database create --language=csharp --keep-trap --command "dotnet build csharp/CSharp.sln /t:rebuild /p:UseSharedCompilation=false" -- db

Review

Commit-by-commit review is suggested.

@hvitved hvitved force-pushed the csharp/path-transformers branch from b3a03f4 to 4c5c7d5 Compare September 4, 2020 14:18
Implements the specification at https://wiki.semmle.com/display/SDmaster/project-layout+format
by compiling file path specifications to regular expressions.
@hvitved hvitved force-pushed the csharp/path-transformers branch from d336a02 to d4aa276 Compare September 7, 2020 07:18
@hvitved hvitved added the C# label Sep 7, 2020
@hvitved hvitved force-pushed the csharp/path-transformers branch from d4aa276 to 37f1ce3 Compare September 7, 2020 13:03
@hvitved hvitved marked this pull request as ready for review September 7, 2020 13:03
@hvitved hvitved requested a review from a team as a code owner September 7, 2020 13:03
Comment thread csharp/extractor/Semmle.Extraction/FilePattern.cs Outdated
Comment thread csharp/extractor/Semmle.Extraction/FilePattern.cs Outdated
Comment thread csharp/extractor/Semmle.Extraction/FilePattern.cs Outdated
Comment thread csharp/extractor/Semmle.Extraction/PathTransformer.cs Outdated
Comment thread csharp/extractor/Semmle.Extraction/PathTransformer.cs Outdated
Comment thread csharp/extractor/Semmle.Extraction/PathTransformer.cs
tamasvajk
tamasvajk previously approved these changes Sep 8, 2020
Comment thread csharp/extractor/Semmle.Extraction/Entities/File.cs Outdated
This avoids calling the path transformer for `GeneratedFile`s.
@hvitved
Copy link
Copy Markdown
Contributor Author

hvitved commented Sep 16, 2020

@dave-bartolomeo : Let me know when you have tested these changes, and whether the PR can be merged.

@hvitved hvitved mentioned this pull request Sep 21, 2020
@hvitved hvitved added this to the 1.25.1 milestone Sep 21, 2020
Copy link
Copy Markdown

@dbartol dbartol left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've verified that the 1.24-based version of this seems to work for Microsoft, so this is ready for 1.25.1.

@hvitved hvitved merged commit 83340e2 into github:rc/1.25 Sep 22, 2020
@hvitved hvitved deleted the csharp/path-transformers branch September 22, 2020 07:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants