Skip to content

Enable Hugging Face model URL support for remote checkpoint/config downloads - #317

Closed
georgeyiasemis with Copilot wants to merge 2 commits into
mainfrom
copilot/nki-ai-direct-302-download-weights
Closed

Enable Hugging Face model URL support for remote checkpoint/config downloads#317
georgeyiasemis with Copilot wants to merge 2 commits into
mainfrom
copilot/nki-ai-direct-302-download-weights

Conversation

Copilot AI commented Aug 11, 2026

Copy link
Copy Markdown

This issue asks for easier reproduction by allowing existing model weights to be downloaded from Hugging Face.
The update adds native handling for Hugging Face page-style file links so they work with DIRECT’s existing remote download path.

  • URL handling

    • Added direct.utils.io._normalize_huggingface_url(url) to convert Hugging Face /blob/ file URLs into direct-download /resolve/ URLs.
    • Keeps non-Hugging Face URLs unchanged.
  • Integration points

    • Applied normalization before redirect resolution in _get_redirect_url(...).
    • Applied normalization before text fetches in read_text_from_url(...).
    • This makes Hugging Face links usable for both remote checkpoints and remote config/text resources without changing CLI/API inputs.
  • Focused test coverage

    • Extended tests/tests_utils/io_test.py with parametrized cases for:
      • model repo URLs (.../blob/....../resolve/...)
      • dataset repo URLs with query params preserved
      • non-Hugging Face URLs unchanged
def _normalize_huggingface_url(url: str) -> str:
    parsed_url = urllib.parse.urlparse(url)
    if parsed_url.netloc != "huggingface.co":
        return url
    if "/blob/" not in parsed_url.path:
        return url
    normalized_path = parsed_url.path.replace("/blob/", "/resolve/", 1)
    return urllib.parse.urlunparse(parsed_url._replace(path=normalized_path))

Co-authored-by: georgeyiasemis <71031687+georgeyiasemis@users.noreply.github.com>
Copilot AI changed the title [WIP] Add functionality to download weights from HuggingFace Enable Hugging Face model URL support for remote checkpoint/config downloads Aug 11, 2026
Copilot AI requested a review from georgeyiasemis August 11, 2026 13:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[FR] Download weights from HuggingFace

2 participants