Skip to content

Fix pip config parsing for multiple extra-index-urls#3125

Merged
saikonen merged 8 commits intoNetflix:masterfrom
harkrish-1:fix/pip-config-extra-index-url-parsing
May 6, 2026
Merged

Fix pip config parsing for multiple extra-index-urls#3125
saikonen merged 8 commits intoNetflix:masterfrom
harkrish-1:fix/pip-config-extra-index-url-parsing

Conversation

@harkrish-1
Copy link
Copy Markdown
Contributor

@harkrish-1 harkrish-1 commented Apr 17, 2026

Summary

  • Fix Pip.indices() to correctly parse multiple extra-index-url values from pip config
  • When pip config contains multiple extra-index-urls, pip config list outputs them separated by literal \n characters (backslash + n, not actual newlines)
  • The previous regex (\s+) only split on whitespace, so multiple URLs were treated as a single mangled value
  • The fix adds literal \n as an additional split delimiter: r"\\n|\s+"

Tests

  • Unit tests added/updated
  • Reproduction script provided (required for Core Runtime)
  • CI passes
  • If tests are impractical: explain why below and provide manual evidence above

Non-Goals

AI Tool Usage

  • No AI tools were used in this contribution
  • AI tools were used (describe below)

Used claude code to identify and fix this bug, I completely understand the changes made

When pip config contains multiple extra-index-urls, `pip config list`
outputs them separated by literal `\n` characters. The previous regex
only split on whitespace, so multiple URLs were treated as a single
mangled value. Split on literal `\n` as well.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@greptile-apps
Copy link
Copy Markdown
Contributor

greptile-apps Bot commented Apr 17, 2026

Greptile Summary

This PR fixes Pip.indices() to correctly split multiple extra-index-url values from pip config list output, where pip serializes multi-value entries as a single string with literal (backslash + n) separators rather than actual newlines. A targeted regex change and a corresponding regression test are the only modifications.

  • pip.py: regex updated from \\s+ to r\"\\\ |\\s+\" so literal \ separators are recognized as delimiters alongside whitespace.
  • test_pip_indices.py: new pytest file that mocks _call and asserts both URLs are parsed out correctly from a realistic multi-URL config string.

Confidence Score: 5/5

Safe to merge — the change is a narrow, well-tested regex fix with no impact on other code paths.

The fix is minimal and clearly correct: it adds \ as an additional split delimiter so that pip's literal-backslash-n separator between multiple index URLs is handled properly. The regression test exercises the exact scenario. No existing behavior is altered for the single-URL case.

No files require special attention.

Important Files Changed

Filename Overview
metaflow/plugins/pypi/pip.py Adds \\n as a split delimiter in indices() so multiple extra-index-url values emitted by pip config list as literal backslash-n are parsed correctly.
test/plugins/pip/test_pip_indices.py New regression test that mocks _call and verifies multi-URL parsing with literal \n separators; covers the happy path introduced by the fix.
test/plugins/pip/init.py Empty init file to make the new test directory a proper Python package.

Reviews (9): Last reviewed commit: "Merge branch 'master' into fix/pip-confi..." | Re-trigger Greptile

Comment thread metaflow/plugins/pypi/pip.py Outdated
Regression test for the literal \n splitting fix plus coverage for
single index, extra-index, multiple-index priority, empty config,
and config call failure scenarios.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@harkrish-1 harkrish-1 force-pushed the fix/pip-config-extra-index-url-parsing branch from cd6e827 to 5744f99 Compare April 17, 2026 15:18
@harkrish-1
Copy link
Copy Markdown
Contributor Author

@savingoyal @saikonen please review and add your feedback. Simple one line change, shouldn't need a lot of time to review.

@romain-intel romain-intel requested a review from saikonen April 20, 2026 06:20
Comment thread test/plugins/pip/test_pip_indices.py Outdated
@talsperre
Copy link
Copy Markdown
Collaborator

Also make sure to setup the pre-commit hooks insce your code fails on the pre-commit checks in GitHub actions.

@talsperre
Copy link
Copy Markdown
Collaborator

Also the appropriate location for this test file should be inside test/plugins/pip (https://github.com/Netflix/metaflow/tree/master/test/plugins) - i.e., create a new directory for pip and add this test in there.

@codecov
Copy link
Copy Markdown

codecov Bot commented Apr 22, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
⚠️ Please upload report for BASE (master@1c69004). Learn more about missing BASE report.

Additional details and impacted files
@@            Coverage Diff            @@
##             master    #3125   +/-   ##
=========================================
  Coverage          ?   27.95%           
=========================================
  Files             ?      381           
  Lines             ?    52347           
  Branches          ?     9238           
=========================================
  Hits              ?    14634           
  Misses            ?    36778           
  Partials          ?      935           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Comment thread metaflow/plugins/pypi/pip.py Outdated
harkrish-1 and others added 3 commits April 29, 2026 17:08
- Fix Black formatting in pip.py by splitting long line (88 char limit)
- Move test from test/unit/ to test/plugins/pip/ following repo structure
- Add test/plugins/pip/__init__.py per plugin test organization pattern

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
- Replace unittest.mock.patch with monkeypatch fixture
- Use MagicMock with monkeypatch.setattr() for patching
- Add docstring to helper function
- Add comment sections for clarity (setup, execute, assert)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
@harkrish-1
Copy link
Copy Markdown
Contributor Author

@talsperre @saikonen addressed all comments!

@harkrish-1
Copy link
Copy Markdown
Contributor Author

@talsperre @saikonen Please review this

@saikonen saikonen merged commit 70bb50d into Netflix:master May 6, 2026
41 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants