Skip to content

CI: try only to fill the compilation cache from main in the compile-queries workflow #11162

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
erik-krogh merged 1 commit into from
Nov 8, 2022
Merged

CI: try only to fill the compilation cache from main in the compile-queries workflow #11162

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 11 additions & 11 deletions .github/workflows/compile-queries.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,21 +24,21 @@ jobs:
run: |
MERGE_BASE=$(git merge-base --fork-point origin/$BASE_BRANCH)
Copy link
Copy Markdown
Contributor

@aibaars aibaars Nov 8, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we can simplify this to simply take the SHA of the first parent of the merge commit. That may actually be better than the "fork point". It also avoids having to fetch the commit history.

Copy link
Copy Markdown
Contributor Author

@erik-krogh erik-krogh Nov 8, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can I get a parent commit if I don't fetch the commit history?
Do you have some code you think would work?

Copy link
Copy Markdown
Contributor

@edoardopirovano edoardopirovano Nov 8, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We have some code in the Foundations team benchmarking infrastructure that does this by calling the GitHub API, you could do something similar here: https://github.com/github/semmle-code/blob/db3086b9699c8c8c9846cd6239b78da431f27dc3/.github/actions/benchmark-prepare/lib/index.js#L55-L70 (with apologies to any external users for the link to our internal code)

Copy link
Copy Markdown
Contributor

@aibaars aibaars Nov 8, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should work too. There must be a simpler way....

git cat-file commit  HEAD | grep '^parent ' | head -1 | cut -d ' ' -f

Copy link
Copy Markdown
Contributor Author

@erik-krogh erik-krogh Nov 9, 2022 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should work too. There must be a simpler way....

I'm not sure there is.
Your code gets the parent of the HEAD commit, which will only work when there's exactly one commit in the PR.

I could use the rest API (I got code doing something similar in DCA already), but I want to keep it simple, so I think I'll keep the current approach.

Copy link
Copy Markdown
Contributor

@aibaars aibaars Nov 9, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The HEAD commit should be a merge commit of the the base and the head branches of a pull request.

Copy link
Copy Markdown
Contributor

@aibaars aibaars Nov 9, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See also: https://docs.github.com/en/actions/using-workflows/events-that-trigger-workflows#pull_request

Copy link
Copy Markdown
Contributor Author

@erik-krogh erik-krogh Nov 9, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I know. But I still don't see how that's useful for getting the merge-base.

I have something better when running on main (´github.sha`).
And when running in a PR I don't see how I can use the HEAD commit, because the parent of the HEAD commit is most of the time not the commit I'm looking for.

echo "merge-base=$MERGE_BASE" >> $GITHUB_ENV
- name: Calculate merge-base - branch
if: ${{ github.event_name != 'pull_request' }}
# using github.sha instead, since we're directly on a branch, and not in a PR
run: |
MERGE_BASE=${{ github.sha }}
echo "merge-base=$MERGE_BASE" >> $GITHUB_ENV
- name: Cache CodeQL query compilation
- name: Read CodeQL query compilation - PR
if: ${{ github.event_name == 'pull_request' }}
uses: actions/cache@v3
with:
path: '*/ql/src/.cache'
# current GH HEAD first, merge-base second, generic third
key: codeql-stable-compile-${{ github.sha }}
key: codeql-compile-pr-${{ github.sha }} # deliberately not using the `compile-compile-main` keys here.
restore-keys: |
codeql-stable-compile-${{ env.merge-base }}
codeql-stable-compile-
codeql-compile-main-${{ env.merge-base }}
codeql-compile-main-
Comment on lines +34 to +35
Copy link
Copy Markdown
Contributor

@aibaars aibaars Nov 8, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
codeql-compile-main-${{ env.merge-base }}
codeql-compile-main-
codeql-compile-${{ github.base_ref }}-${{ env.merge-base }}
codeql-compile-${{ github.base_ref }}-
codeql-compile-refs/heads/main-

- name: Fill CodeQL query compilation cache - main
if: ${{ github.event_name != 'pull_request' }}
uses: actions/cache@v3
with:
path: '*/ql/src/.cache'
key: codeql-compile-main-${{ github.sha }} # just fill on main
Copy link
Copy Markdown
Contributor

@aibaars aibaars Nov 8, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps:

Suggested change
key: codeql-compile-main-${{ github.sha }} # just fill on main
key: codeql-compile-${{ github.ref }}-${{ github.sha }} # just fill on main

- name: Setup CodeQL
uses: ./.github/actions/fetch-codeql
with:
Expand Down