Skip to content

Enable commit-specific git fetch for compatibility with external forks. - #83

Open
leander-dsouza wants to merge 3 commits into
mainfrom
leander-dsouza/fetch-non-shallow
Open

leander-dsouza wants to merge 3 commits into
mainfrom
leander-dsouza/fetch-non-shallow

Conversation

@leander-dsouza

@leander-dsouza leander-dsouza commented Nov 10, 2025

Copy link
Copy Markdown
Member

Basic Info

Info Please fill out this column
Ticket(s) this addresses resolves #8
Primary OS tested on Ubuntu
Is this a breaking change? No
Does this PR contain AI generated software? No

Description of contribution in a few bullet points

  • Enabled commit-specific fetching of origin on non-shallow clones containing hashes.
  • This allows us to clone PRs from repositories not tied to the source branch for history (external forks).

Description of how this change was tested

1. Testing against vcs2l old pull requests

  • First, clone the codebase:

    git clone git@github.com:ros-infrastructure/vcs2l.git vcs2l-ext-fork
  • Next, use the following repos.yaml to get the commit hash of the following external fork:

    ---
    repositories:
      vcs2l-ext-fork:
        type: git
        url: 'https://github.com/ros-infrastructure/vcs2l.git'
        version: '4ad2dd14b0033c275e5faff7935c4a7d7581fee4'
  • Finally, update the existing repository:

    vcs import --input repos.yaml

    This will successfully checkout the repository to the specific hash.
    This step fails without this contribution with the following error message:

    === ./vcs2l-ext-fork (git) ===
    Could not checkout ref '4ad2dd14b0033c275e5faff7935c4a7d7581fee4': fatal: reference is not a tree:    4ad2dd14b0033c275e5faff7935c4a7d7581fee4

2. Test against the reproduction step in the parent issue - dirk-thomas/vcstool#258

  • Reproduction measures specified by @timor-raiman

    git clone git@github.com:timor-raiman/action-ros-ci.git -b vcs-issue && cd action-ros-ci
    git fetch origin +refs/pull/3/*:refs/remotes/origin/pr/3/*
    export PRMERGE_SHA=`git rev-parse origin/pr/3/merge`
    rm -rf new-path/ && mkdir -p new-path/
    cat test-vcs-issue.repo | envsubst | vcs import --force --recursive new-path/

@leander-dsouza leander-dsouza changed the title Enable commit-specific git fetch for compatibility with pull requests Enable commit-specific git fetch for compatibility with external forks. Nov 10, 2025
@leander-dsouza
leander-dsouza force-pushed the leander-dsouza/fetch-non-shallow branch 2 times, most recently from 85222a3 to e4d26d4 Compare November 10, 2025 16:58
@codecov-commenter

codecov-commenter commented Nov 10, 2025

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 0% with 14 lines in your changes missing coverage. Please review.
✅ Project coverage is 26.76%. Comparing base (fbd0321) to head (f24838b).

Files with missing lines Patch % Lines
vcs2l/clients/git.py 0.00% 14 Missing ⚠️
Additional details and impacted files
@@           Coverage Diff           @@
##             main      #83   +/-   ##
=======================================
  Coverage   26.76%   26.76%           
=======================================
  Files          31       31           
  Lines        2298     2298           
  Branches      405      408    +3     
=======================================
  Hits          615      615           
  Misses       1623     1623           
  Partials       60       60           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@leander-dsouza
leander-dsouza marked this pull request as ready for review November 10, 2025 17:01
Copilot AI review requested due to automatic review settings November 10, 2025 17:01

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This pull request adds support for fetching specific commit hashes in non-shallow Git import mode, and improves test skipping for VCS tools that may not be installed.

  • Refactored version type determination to work for both shallow and non-shallow modes
  • Added explicit fetch of commit hashes for non-shallow repository imports
  • Added test skipping when SVN or Mercurial tools are not available

Reviewed Changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated no comments.

File Description
vcs2l/clients/git.py Refactored to determine version type earlier and fetch specific commit hashes in non-shallow mode for both existing and new repositories
test/test_commands.py Added utility variables and skip decorator for tests requiring SVN and Mercurial
test/reimport_force.txt Updated expected output to include fetch messages for commit hash imports
test/import.txt Updated expected output to include fetch messages for commit hash imports

Comment thread test/test_commands.py

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The checks added here are part of the fixture setup in __init__.py. What's the motivation to duplicate them here?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

They were initially duplicated as Subversion, and Mercurial ignores were flaky in Windows 2025 in previous workflow runs.

However, they currently work as expected, so I have dropped the additional commit.

@leander-dsouza
leander-dsouza force-pushed the leander-dsouza/fetch-non-shallow branch from e4d26d4 to 9d6d6bc Compare January 26, 2026 02:31
@leander-dsouza
leander-dsouza marked this pull request as draft January 26, 2026 02:31
@leander-dsouza
leander-dsouza marked this pull request as ready for review January 26, 2026 02:33
@leander-dsouza leander-dsouza self-assigned this Jan 26, 2026
@leander-dsouza leander-dsouza added the bug Something isn't working label Feb 21, 2026
@leander-dsouza
leander-dsouza force-pushed the leander-dsouza/fetch-non-shallow branch from 9d6d6bc to 1fef30f Compare March 6, 2026 20:29

@KmoM88 KmoM88 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall LGTM.

I have left a suggestion regarding the execution workflow.

Also as a side note: What do you think about adding a dedicated test case that requests a commit from an unmerged, orphaned branch (simulating a real external fork) to prevent any future regressions?

Nice work, waiting to hear your thoughts.

Comment thread vcs2l/clients/git.py Outdated
output = result_clone['output']

# For non-shallow clones with commit hashes, fetch the specific commit
if not command.shallow and version_type == 'hash':

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For a non-shallow commit-specific clone this is doing a full git clone and then does a git fetch for the target hash. For large repositories, this fetches a lot of unnecessary data.
What would you think about bypassing git clone entirely and using the git init + git fetch <hash> path for all precise tags and hashes (both shallow and non-shallow)?

Adjusting the conditional to redirect any hash or tag to the else block and toggling the --depth 1 flag on the fetch command dynamically if command.shallow is True.

This could save significant bandwidth for repositories with large default branches.

What do you think?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I agree.
Now, hash/tag imports go through git init + git fetch <ref> for both shallow and non-shallow clones, with the --depth 1 added for shallow ones.

One thing I ran into in the hash case is that git fetch origin <hash> does not pull any tags, so git describe --tags breaks. Therefore, I have added the --tags to the fetch command to preserve it, and now subsequent expect .txt files have been updated to match it.

Signed-off-by: Leander Stephen D'Souza <leanderdsouza1234@gmail.com>
Signed-off-by: Leander Stephen D'Souza <leanderdsouza1234@gmail.com>
@leander-dsouza
leander-dsouza force-pushed the leander-dsouza/fetch-non-shallow branch from 1fef30f to c722074 Compare August 20, 2026 17:03
Signed-off-by: Leander Stephen D'Souza <leanderdsouza1234@gmail.com>
@leander-dsouza
leander-dsouza force-pushed the leander-dsouza/fetch-non-shallow branch from c722074 to f24838b Compare August 20, 2026 17:04
@leander-dsouza

Copy link
Copy Markdown
Member Author

Nice work

Thank you <3

Also as a side note: What do you think about adding a dedicated test case that requests a commit from an unmerged, orphaned branch (simulating a real external fork) to prevent any future regressions?

All tests from #72 have been designed to run without an internet connection to ensure compatibility across various operating systems.

So, I do not think we can add a real external fork to validate this behaviour.

@leander-dsouza
leander-dsouza requested a review from KmoM88 August 20, 2026 17:07

@KmoM88 KmoM88 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good job on the contribution, the proposed changes look correct. The logic for fetching commit-specific clones seems correct. Overall LGTM! Thank you!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

git client only fetches commit in shallow mode

5 participants