Skip to content

CLI plugins#71

Open
avaldebe wants to merge 19 commits into
masterfrom
cli-plugins
Open

CLI plugins#71
avaldebe wants to merge 19 commits into
masterfrom
cli-plugins

Conversation

@avaldebe

@avaldebe avaldebe commented May 9, 2025

Copy link
Copy Markdown
Collaborator

add commands and sub-commands from plugin packages using the package metadata

https://packaging.python.org/en/latest/guides/creating-and-discovering-plugins/#using-package-metadata

@avaldebe avaldebe requested a review from JohnPaton May 9, 2025 15:19
@avaldebe avaldebe force-pushed the cli-plugins branch 2 times, most recently from 79a0af8 to 02a6db2 Compare May 9, 2025 16:13
@avaldebe avaldebe marked this pull request as ready for review May 9, 2025 16:16
@avaldebe

avaldebe commented May 9, 2025

Copy link
Copy Markdown
Collaborator Author

I have a follow up PR that adds a command to just download the metadata and create an extract with only the relevant metadata to the observation files found on a particular directory.
It uses polars to read the metadata CSV and observation Parquet files.
I'll convert the code I'm testing at work into a CLI plugin so it can be enabled as an optional/extra.

@avaldebe avaldebe self-assigned this May 11, 2025
@avaldebe avaldebe mentioned this pull request May 14, 2025
JohnPaton
JohnPaton previously approved these changes Oct 3, 2025

@JohnPaton JohnPaton left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm. What were you planning to use this for?

@avaldebe

avaldebe commented Oct 6, 2025

Copy link
Copy Markdown
Collaborator Author

I want to add a catalog feature that index some of the most relevant metadata for the data contained on each parquet file.
These datasets have so many files, that IO becomes relevant when trying to figure out the contents of each file.
Therefore, the catalog simplifies the filtering/selection of observations without opening all the files...

@avaldebe avaldebe force-pushed the cli-plugins branch 2 times, most recently from 8cb5269 to 4cffc07 Compare October 6, 2025 14:34
@avaldebe avaldebe force-pushed the cli-plugins branch 3 times, most recently from e21e5ae to 5de74ef Compare October 8, 2025 13:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants