Skip to content

C#: Cache temporary dependency directory for BMN #3286

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
mbg merged 7 commits into from
Nov 26, 2025
Merged

C#: Cache temporary dependency directory for BMN #3286

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
cf8b7a6
Refactor C# cache content paths into a function
mbg Nov 12, 2025
d854ba6
Pass `FeatureEnablement` to `getDependencyPaths`
mbg Nov 12, 2025
a47d04c
Add FF for extra C# cache contents
mbg Nov 12, 2025
ecaa6db
Include `getCsharpTempDependencyDir` in C# caches if FF is enabled
mbg Nov 13, 2025
f5f9571
Configure temp dependency dir for C# extractor when FF is enabled
mbg Nov 13, 2025
0155561
Merge branch 'main' into mbg/csharp/more-cache-locations
mbg Nov 26, 2025
3e93966
Merge branch 'main' into mbg/csharp/more-cache-locations
henrymercer Nov 26, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Configure temp dependency dir for C# extractor when FF is enabled 
And also clean it up.
  • Loading branch information
@mbg
mbg committed Nov 13, 2025
commit f5f9571d6184d3cf1e2477d1dfc61c7ce88cdf4b
24 changes: 16 additions & 8 deletions lib/analyze-action-post.js
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

13 changes: 9 additions & 4 deletions lib/analyze-action.js
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

21 changes: 15 additions & 6 deletions src/analyze-action-env.test.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -74,11 +74,20 @@ test("analyze action with RAM & threads from environment variables", async (t) =
// wait for the action promise to complete before starting verification.
await analyzeAction.runPromise;

t.assert(runFinalizeStub.calledOnce);
t.deepEqual(runFinalizeStub.firstCall.args[1], "--threads=-1");
t.deepEqual(runFinalizeStub.firstCall.args[2], "--ram=4992");
t.assert(runQueriesStub.calledOnce);
t.deepEqual(runQueriesStub.firstCall.args[2], "--threads=-1");
t.deepEqual(runQueriesStub.firstCall.args[1], "--ram=4992");
t.assert(
runFinalizeStub.calledOnceWith(
sinon.match.any,
sinon.match.any,
"--threads=-1",
"--ram=4992",
),
);
t.assert(
runQueriesStub.calledOnceWith(
sinon.match.any,
"--ram=4992",
"--threads=-1",
),
);
Comment on lines +77 to +91
Copy link
Copy Markdown
Contributor

@henrymercer henrymercer Nov 19, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice! Is it worth also worth putting the RAM and threads arguments in the same order?

Copy link
Copy Markdown
Member Author

@mbg mbg Nov 26, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Probably -- I would lie if I said I didn't get this wrong after initially copying it from one to the other. Not sure if this PR is the best place to do it though.

});
});
21 changes: 15 additions & 6 deletions src/analyze-action-input.test.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -72,11 +72,20 @@ test("analyze action with RAM & threads from action inputs", async (t) => {
// wait for the action promise to complete before starting verification.
await analyzeAction.runPromise;

t.assert(runFinalizeStub.calledOnce);
t.deepEqual(runFinalizeStub.firstCall.args[1], "--threads=-1");
t.deepEqual(runFinalizeStub.firstCall.args[2], "--ram=3012");
t.assert(runQueriesStub.calledOnce);
t.deepEqual(runQueriesStub.firstCall.args[2], "--threads=-1");
t.deepEqual(runQueriesStub.firstCall.args[1], "--ram=3012");
t.assert(
runFinalizeStub.calledOnceWith(
sinon.match.any,
sinon.match.any,
"--threads=-1",
"--ram=3012",
),
);
t.assert(
runQueriesStub.calledOnceWith(
sinon.match.any,
"--ram=3012",
"--threads=-1",
),
);
});
});
28 changes: 18 additions & 10 deletions src/analyze-action-post.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,10 @@ import { getGitHubVersion } from "./api-client";
import { getCodeQL } from "./codeql";
import { getConfig } from "./config-utils";
import * as debugArtifacts from "./debug-artifacts";
import { getJavaTempDependencyDir } from "./dependency-caching";
import {
getCsharpTempDependencyDir,
getJavaTempDependencyDir,
} from "./dependency-caching";
import { EnvVar } from "./environment";
import { getActionsLogger } from "./logging";
import { checkGitHubVersionInRange, getErrorMessage } from "./util";
Expand Down Expand Up @@ -42,17 +45,22 @@ async function runWrapper() {
}
}

// If we analysed Java in build-mode: none, we may have downloaded dependencies
// If we analysed Java or C# in build-mode: none, we may have downloaded dependencies
// to the temp directory. Clean these up so they don't persist unnecessarily
// long on self-hosted runners.
const javaTempDependencyDir = getJavaTempDependencyDir();
if (fs.existsSync(javaTempDependencyDir)) {
try {
fs.rmSync(javaTempDependencyDir, { recursive: true });
} catch (error) {
logger.info(
`Failed to remove temporary Java dependencies directory: ${getErrorMessage(error)}`,
);
const tempDependencyDirs = [
getJavaTempDependencyDir(),
getCsharpTempDependencyDir(),
];
for (const tempDependencyDir of tempDependencyDirs) {
if (fs.existsSync(tempDependencyDir)) {
try {
fs.rmSync(tempDependencyDir, { recursive: true });
} catch (error) {
logger.info(
`Failed to remove temporary dependencies directory: ${getErrorMessage(error)}`,
);
}
}
}
} catch (error) {
Expand Down
1 change: 1 addition & 0 deletions src/analyze-action.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -315,6 +315,7 @@ async function run() {
await runAutobuildIfLegacyGoWorkflow(config, logger);

dbCreationTimings = await runFinalize(
features,
outputDir,
threads,
memory,
Expand Down
22 changes: 19 additions & 3 deletions src/analyze.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,10 @@ import * as analyses from "./analyses";
import { setupCppAutobuild } from "./autobuild";
import { type CodeQL } from "./codeql";
import * as configUtils from "./config-utils";
import { getJavaTempDependencyDir } from "./dependency-caching";
import {
getCsharpTempDependencyDir,
getJavaTempDependencyDir,
} from "./dependency-caching";
import { addDiagnostic, makeDiagnostic } from "./diagnostics";
import {
DiffThunkRange,
Expand Down Expand Up @@ -98,6 +101,7 @@ async function setupPythonExtractor(logger: Logger) {

export async function runExtraction(
codeql: CodeQL,
features: FeatureEnablement,
config: configUtils.Config,
logger: Logger,
) {
Expand All @@ -122,7 +126,7 @@ export async function runExtraction(
await setupCppAutobuild(codeql, logger);
}

// The Java `build-mode: none` extractor places dependencies (.jar files) in the
// The Java and C# `build-mode: none` extractors place dependencies in the
// database scratch directory by default. For dependency caching purposes, we want
// a stable path that caches can be restored into and that we can cache at the
// end of the workflow (i.e. that does not get removed when the scratch directory is).
Expand All @@ -133,6 +137,15 @@ export async function runExtraction(
process.env["CODEQL_EXTRACTOR_JAVA_OPTION_BUILDLESS_DEPENDENCY_DIR"] =
getJavaTempDependencyDir();
}
if (
language === KnownLanguage.csharp &&
config.buildMode === BuildMode.None &&
(await features.getValue(Feature.CsharpCacheBuildModeNone))
) {
process.env[
"CODEQL_EXTRACTOR_CSHARP_OPTION_BUILDLESS_DEPENDENCY_DIR"
] = getCsharpTempDependencyDir();
}

await codeql.extractUsingBuildMode(config, language);
} else {
Expand Down Expand Up @@ -177,13 +190,14 @@ export function dbIsFinalized(

async function finalizeDatabaseCreation(
codeql: CodeQL,
features: FeatureEnablement,
config: configUtils.Config,
threadsFlag: string,
memoryFlag: string,
logger: Logger,
): Promise<DatabaseCreationTimings> {
const extractionStart = performance.now();
await runExtraction(codeql, config, logger);
await runExtraction(codeql, features, config, logger);
const extractionTime = performance.now() - extractionStart;

const trapImportStart = performance.now();
Expand Down Expand Up @@ -597,6 +611,7 @@ export async function runQueries(
}

export async function runFinalize(
features: FeatureEnablement,
outputDir: string,
threadsFlag: string,
memoryFlag: string,
Expand All @@ -615,6 +630,7 @@ export async function runFinalize(

const timings = await finalizeDatabaseCreation(
codeql,
features,
config,
threadsFlag,
memoryFlag,
Expand Down
Loading