Fetch only current HEAD when deepen clone - #1045
Conversation
🦋 Changeset detectedLatest commit: c7ac06f The changes in this PR will be included in the next version bump. This PR includes changesets to release 6 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
|
This pull request is automatically built and testable in CodeSandbox. To see build info of the built libraries, click here or the icon next to each commit SHA. Latest deployment of this branch, based on commit 9321f1b:
|
Also, I'd like to ask if it makes sense to expose this |
|
Hi @Andarist would you share your thoughts on this? Thanks |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #1045 +/- ##
=======================================
Coverage 81.10% 81.10%
=======================================
Files 54 54
Lines 2265 2265
Branches 679 679
=======================================
Hits 1837 1837
Misses 423 423
Partials 5 5 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
|
|
||
| export async function deepenCloneBy({ by, cwd }: { by: number; cwd: string }) { | ||
| await spawn("git", ["fetch", `--deepen=${by}`], { cwd }); | ||
| await spawn("git", ["fetch", `--deepen=${by}`, "origin", "HEAD"], { cwd }); |
There was a problem hiding this comment.
This doesn't work as expected. It still deepens all the branches - I have tested this locally. The only thing I could figure out that works is using git fetch --depth X and incrementing that X with each fetch. We don't know the initial depth though and it might get tricky to calculate it accurately - one would have to research our options around this.
There was a problem hiding this comment.
This doesn't work as expected. It still deepens all the branches - I have tested this locally
Well, in the many years since I submitted this PR, I've created a fork of changesets for my company that contains this patch and it works as expected for our use case... (and in that fork, it didn't fetch all branches)
There was a problem hiding this comment.
I'm sorry for not getting to this PR sooner. I went back to it now to double-check things... I created a GitHub workflow as a testbed:
Git Shallow Clone Test workflow
name: Git Shallow Clone Test
on:
workflow_dispatch:
jobs:
shallow-clone-and-deepen:
runs-on: ubuntu-latest
steps:
- name: Shallow clone changesets repo
run: |
git clone --depth 1 https://github.com/changesets/changesets.git changesets-repo
cd changesets-repo
- name: Print initial git log
run: |
cd changesets-repo
echo "=== Initial log after shallow clone ==="
git log --all --oneline
- name: Deepen by 1 commit
run: |
cd changesets-repo
git fetch --deepen 1
- name: Print log after deepening
run: |
cd changesets-repo
echo "=== Log after deepening by 1 ==="
git log --all --oneline
shallow-clone-with-branch-fetch:
runs-on: ubuntu-latest
steps:
- name: Shallow clone changesets repo
run: |
git clone --depth 1 https://github.com/changesets/changesets.git changesets-repo
cd changesets-repo
- name: Print initial git log
run: |
cd changesets-repo
echo "=== Initial log after shallow clone ==="
git log --all --oneline
- name: Fetch next branch
run: |
cd changesets-repo
git fetch --depth 1 origin next:next
- name: Print log after fetching branch
run: |
cd changesets-repo
echo "=== Log after fetching next branch ==="
git log --all --oneline
- name: Deepen by 1 commit
run: |
cd changesets-repo
git fetch --deepen 1
- name: Print log after deepening
run: |
cd changesets-repo
echo "=== Log after deepening by 1 ==="
git log --all --oneline
shallow-clone-with-branch-fetch-and-refspec-target:
runs-on: ubuntu-latest
steps:
- name: Shallow clone changesets repo
run: |
git clone --depth 1 https://github.com/changesets/changesets.git changesets-repo
cd changesets-repo
- name: Print initial git log
run: |
cd changesets-repo
echo "=== Initial log after shallow clone ==="
git log --all --oneline
- name: Fetch next branch
run: |
cd changesets-repo
git fetch --depth 1 origin next:next
- name: Print log after fetching branch
run: |
cd changesets-repo
echo "=== Log after fetching next branch ==="
git log --all --oneline
- name: Deepen by 1 commit
run: |
cd changesets-repo
git fetch --deepen 1 origin HEAD
- name: Print log after deepening
run: |
cd changesets-repo
echo "=== Log after deepening by 1 ==="
git log --all --oneline
And I got those results:
shallow-clone-and-deepen
=== Log after deepening by 1 ===
cc28222 Use `bumpVersionsWithWorkspaceProtocolOnly` properly when parsing config (#1535)
4c5a207 Update `js-yaml` to `v4` (#1772)
shallow-clone-with-branch-fetch
=== Log after deepening by 1 ===
cc28222 Use `bumpVersionsWithWorkspaceProtocolOnly` properly when parsing config (#1535)
4c5a207 Update `js-yaml` to `v4` (#1772)
47dcb93 Fix `package.json#repository` field in `release-utils` (#1752)
d189da1 Version Packages (next) (#1653)
shallow-clone-with-branch-fetch-and-refspec-target
=== Log after deepening by 1 ===
cc28222 Use `bumpVersionsWithWorkspaceProtocolOnly` properly when parsing config (#1535)
4c5a207 Update `js-yaml` to `v4` (#1772)
47dcb93 Fix `package.json#repository` field in `release-utils` (#1752)
d189da1 Version Packages (next) (#1653)
There was a problem hiding this comment.
Perhaps the question is... why any extra branches would be fetched by your workflow in the first place? Cause when I simply shallow clone a repository it only deepens the default branch (main), but when I fetch an extra branch before deepening then it deepens both regardless of the extra origin HEAD args
There was a problem hiding this comment.
Hmmm interesting... thanks for diving deeper into this... perhaps it's the git version, or perhaps the different git default config on different machines. One different thing is that at my company it is running inside Gitlab CI (private instance), so perhaps there's a different behavior there... Thanks for looking into this, appreciate your time and effort 🙏
The proposed change doesn't work. --depen=1 origin HEAD doesn't seem to affect which branches are actually deepened.
|
Closing since the fix is applicable for maybe only for my use case and not others |
|
My mistake for approving this, I thought it worked after some googling. |
|
Not a problem, it was a reasonable assumption. I have only found out this might not be doing what it was supposed to when trying to get rid of the hardcoded |
I also face issue #571 at work's CI, but I could work around it by doing this.
The failure I face is at
deepenCloneBy. Because my work's repo is huge (lots of commits, branches, and tags), we only use shallow clone at CI jobs. While integrating with Changesets in CI, runninggit fetch --deepenBy=50will fail because apparently it fetches from all branches and the git server at our CI couldn't handle such a huge request in time (i.e. will timeout), and the job will stuck in infinite loop due to #571.This change makes changesets to only deepen clone from current HEAD only. I feel that it makes sense too, since the purpose of the usage (
getCommitsThatAddFiles) is to deepen current branch until it found the relevant commits.What do you think?
Thanks