Skip to content

Fix dev setup issues with pooch data fetch and cache handling #1656

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
LOCEANlloydizard merged 2 commits into from
Apr 18, 2026
+21 −9
Merged

Fix dev setup issues with pooch data fetch and cache handling #1656

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
2c35bc9
Fix dev setup issues with pooch data fetch and cache handling
LOCEANlloydizard Apr 18, 2026
4451b4a
Update conftest.py
LOCEANlloydizard Apr 18, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 1 addition & 5 deletions .ci_helpers/docker/setup-services.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -31,11 +31,7 @@ def get_pooch_data_path() -> Path:
"""Return path to the Pooch test data cache."""
ver = os.getenv("ECHOPYPE_DATA_VERSION", "v0.11.1a2")
cache_dir = Path(pooch.os_cache("echopype")) / ver
if not cache_dir.exists():
raise FileNotFoundError(
f"Pooch cache directory not found: {cache_dir}\n"
"Make sure test data was fetched via conftest.py"
)
cache_dir.mkdir(parents=True, exist_ok=True)
return cache_dir


Expand Down
14 changes: 14 additions & 0 deletions docs/source/contrib_setup.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,20 @@ To create an environment for developing Echopype, we recommend the following ste
pip install -e ".[plot]"
```

For a fresh local setup, enable the Pooch-based test data fetch before running tests.

On Linux/macOS:
```shell
export USE_POOCH=True
```

On Windows PowerShell:
```shell
$env:USE_POOCH="True"
```

This allows the required test data to be downloaded into the local Pooch cache on first run.

:::{tip}
We recommend using Mamba to get around Conda's sometimes slow or stuck behavior when solving dependencies.
See [Mamba's documentation](https://mamba.readthedocs.io/en/latest/) for installation and usage.
Expand Down
9 changes: 5 additions & 4 deletions echopype/tests/conftest.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -69,9 +69,11 @@
base_url=base,
version=ver,
registry=registry,
retry_if_failed=1,
retry_if_failed=5,
)

downloader = pooch.HTTPDownloader(timeout=300)
Comment on lines +72 to +75
Copy link
Copy Markdown
Member

@leewujung leewujung Apr 18, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey @LOCEANlloydizard : I am curious - is this a safety measure because you saw whole bunch of failed attempts?

Copy link
Copy Markdown
Collaborator Author

@LOCEANlloydizard LOCEANlloydizard Apr 18, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, after I deleted the local cache directory, it started downloading again and I got several timeout errors! But I mostly saw this locally, not on CI (if I remember correctly). Maybe it's because the asset files are getting too large..?

In the long term, maybe we could reduce the number of retries and the timeout? or maybe that altogether?


def _unpack(fname, action, pooch_instance):
z = Path(fname)
out = z.parent / z.stem
Expand Down Expand Up @@ -125,12 +127,11 @@ def _unpack(fname, action, pooch_instance):

return str(out)


for b in bundles:
url = base.format(version=ver) + b
print(f"[echopype-ci] fetching bundle: {b}")
print(f"[echopype-ci] → URL: {url}")
EP.fetch(b, processor=_unpack, progressbar=False)
print(f"[echopype-ci] -> URL: {url}")
EP.fetch(b, processor=_unpack, progressbar=False, downloader=downloader)

print(
"\n[echopype-ci] TEST_DATA_FOLDER\n"
Expand Down
1 change: 1 addition & 0 deletions requirements-dev.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,3 +25,4 @@ sphinx_rtd_theme
sphinxcontrib-mermaid
twine
wheel
pooch