Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
38 changes: 0 additions & 38 deletions .circleci/config.yml

This file was deleted.

4 changes: 2 additions & 2 deletions .devcontainer/devcontainer.json
Original file line number Diff line number Diff line change
Expand Up @@ -9,8 +9,8 @@
}
},
"forwardPorts": [6080],
//Pip install the requriements, and then install the pre-commit hooks
"postCreateCommand": "pip install -e .[tests,dev] && pre-commit install",
//Pip install the requirements, and then install the pre-commit hooks
"postCreateCommand": "pip install -e .[tests,dev,docs] && pre-commit install",
"customizations": {
"vscode": {
"extensions": [
Expand Down
74 changes: 74 additions & 0 deletions .github/copilot-instructions.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,74 @@
# GitHub Copilot Instructions for cdflib

This repository contains `cdflib`, a pure Python library for reading and writing NASA Common Data Format (CDF) files. It does not rely on the NASA CDF C library.

## Project Architecture & Core Components

- **Pure Python Implementation**: The core logic uses `numpy` and `struct` to parse binary CDF files. No C extensions are used.
- **Reading (`cdflib/cdfread.py`)**:
- The `CDF` class is the main entry point for reading.
- Supports reading from local files, URLs, and S3 buckets.
- **Key Method**: `CDF.varget(variable_name)` retrieves variable data.
- **Key Method**: `CDF.cdf_info()` returns global file information.
- **Writing (`cdflib/cdfwrite.py`)**:
- The `CDF` class in this module is used to create new CDF files.
- Requires a `cdf_spec` dictionary to define file properties (encoding, majority, etc.).
- **Xarray Integration (`cdflib/xarray/`)**:
- `cdf_to_xarray`: Converts CDF files to `xarray.Dataset` objects, mapping ISTP attributes to xarray conventions.
- `xarray_to_cdf`: Converts `xarray.Dataset` objects back to CDF files.
- **Time Handling (`cdflib/epochs.py`)**:
- `CDFepoch` class handles conversions between CDF time types (CDF_EPOCH, CDF_EPOCH16, TT2000) and Python `datetime`, `numpy.datetime64`, or Unix timestamps.

## Developer Workflows

- **Dependency Management**: Dependencies are defined in `pyproject.toml`.
- Core: `numpy`
- Optional/Test: `xarray`, `astropy`, `hypothesis`, `pytest`
- **Testing**:
- Run tests using `tox` to test across multiple Python versions.
- Run specific tests with `pytest`: `pytest tests/test_cdfread.py`.
- Tests are located in the `tests/` directory.
- **Formatting**:
- The project uses `black` for code formatting and `isort` for import sorting.
- Configuration is in `pyproject.toml`.

## Coding Conventions & Patterns

- **Type Hinting**: Use Python type hints extensively (e.g., `Union[str, Path]`, `npt.ArrayLike`).
- **Path Handling**: Support both `str` and `pathlib.Path` objects for file paths. Internally, paths are often resolved to strings or `Path` objects.
- **Numpy Usage**: Use `numpy` for all array operations. Avoid explicit loops over data where possible.
- **S3 Support**: When handling S3 paths (`s3://`), use the internal `cdflib.s3` module logic.

## Common Code Examples

### Reading a CDF File
```python
import cdflib
cdf_file = cdflib.CDF('/path/to/file.cdf')
info = cdf_file.cdf_info()
data = cdf_file.varget("VariableName")
```

### Converting to Xarray
```python
from cdflib import cdf_to_xarray
ds = cdf_to_xarray('/path/to/file.cdf', to_datetime=True)
```

### Time Conversion
```python
from cdflib.epochs import CDFepoch
# Convert CDF epoch to datetime
dt = CDFepoch.to_datetime(cdf_epoch_value)
# Convert datetime to CDF epoch
epoch = CDFepoch.compute(dt)
```

### Writing a CDF File
```python
from cdflib.cdfwrite import CDF
spec = {'Majority': 'row_major', 'Encoding': 6} # 6 is IBMPC_ENCODING
with CDF('new_file.cdf', cdf_spec=spec) as cdf:
cdf.write_globalattrs(global_attrs)
cdf.write_var(var_spec, var_attrs, var_data)
```
65 changes: 39 additions & 26 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
@@ -1,36 +1,49 @@
name: Run tests
name: Tests

on:
push:
branches:
- 'main'
branches: [ main ]
pull_request:
# Allow manual runs through the web UI
workflow_dispatch:

# Only allow one run per git ref at a time
concurrency:
group: '${{ github.workflow }}-${{ github.ref }}'
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true

jobs:
core:
uses: OpenAstronomy/github-actions-workflows/.github/workflows/tox.yml@v1
with:
submodules: false
coverage: codecov
envs: |
- linux: py39
#- linux: py310
#- linux: py311
- linux: py312
#- windows: py39
- windows: py310
#- windows: py311
#- windows: py312
#- macos: py38
#- macos: py39
#- macos: py310
- macos: py311-online
- macos: py312
- windows: py313-devdeps
tests:
name: ${{ matrix.os }} / ${{ matrix.python-version }}
runs-on: ${{ matrix.os }}
strategy:
fail-fast: false
matrix:
os: [ubuntu-latest, windows-latest, macos-latest]
python-version: ["3.10", "3.11", "3.12", "3.13"]
include:
- os: ubuntu-latest
python-version: "3.13"
tox-env: devdeps

steps:
- uses: actions/checkout@v4

- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python-version }}
cache: 'pip'

- name: Install Tools
run: |
python -m pip install --upgrade pip
python -m pip install tox
- name: Run Tests via Tox
# If a specific tox-env is set in matrix, use it; otherwise use standard py version
run: |
if [ -z "${{ matrix.tox-env }}" ]; then
tox -e py$(echo ${{ matrix.python-version }} | tr -d .)
else
tox -e ${{ matrix.tox-env }}
fi
shell: bash
26 changes: 26 additions & 0 deletions .github/workflows/docs.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
name: docs
on:
push:
branches:
- main
permissions:
contents: write
jobs:
build-and-deploy-docs:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v4

- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: 3.x

- name: Install dependencies
run: |
pip install -e .[tests,dev,docs]
Copy link

Copilot AI Nov 21, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The docs workflow installs unnecessary extras (tests and dev) for building documentation. Only [docs] is needed for mkdocs gh-deploy. Installing extra dependencies increases build time and could potentially introduce version conflicts. Change to pip install -e .[docs].

Suggested change
pip install -e .[tests,dev,docs]
pip install -e .[docs]

Copilot uses AI. Check for mistakes.

- name: Build and deploy
run: |
mkdocs gh-deploy --force
28 changes: 16 additions & 12 deletions .github/workflows/pypi-build.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -4,25 +4,29 @@ on:
release:
types: [created]


jobs:
build:
build-and-publish:
runs-on: ubuntu-latest
# REQUIRED: This permission allows the OIDC token to be generated
permissions:
id-token: write
contents: read

steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v4

- name: Set up Python
uses: actions/setup-python@v3
uses: actions/setup-python@v5
with:
python-version: '3.x'
- name: Install dependencies

- name: Install build dependencies
run: |
python -m pip install --upgrade pip
python -m pip install twine build
- name: Build
python -m pip install build

- name: Build package
run: python -m build

- name: Publish
env:
TWINE_USERNAME: __token__
TWINE_PASSWORD: ${{ secrets.pypi_password }}
run: twine upload dist/*
- name: Publish to PyPI
uses: pypa/gh-action-pypi-publish@release/v1
27 changes: 27 additions & 0 deletions .github/workflows/remote-tests.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
name: Remote Data Tests

on:
# Allow manual trigger via the "Run Workflow" button in UI
workflow_dispatch:
# Run automatically every Monday at 6am UTC
schedule:
- cron: '0 6 * * 1'

jobs:
remote-tests:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4

- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: "3.12"

- name: Install dependencies
run: |
pip install .[tests]

- name: Run Remote Tests
# We explicitly tell pytest to run ONLY the remote_data tests
run: pytest -m remote_data
18 changes: 11 additions & 7 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
@@ -1,3 +1,7 @@
ci:
autofix_prs: false
autoupdate_schedule: "quarterly"
skip: [no-commit-to-branch, ccv]
repos:
- repo: https://github.com/myint/autoflake
rev: v2.1.1
Expand All @@ -10,11 +14,11 @@ repos:
rev: 5.12.0
hooks:
- id: isort
args: ['--sp','setup.cfg']
args: ['--sp','pyproject.toml']
exclude: ".*(.fits|.fts|.fit|.txt|tca.*|extern.*|.rst|.md|.svg)$"

- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v4.4.0
rev: v6.0.0
hooks:
- id: check-ast
- id: check-case-conflict
Expand All @@ -32,8 +36,8 @@ repos:
hooks:
- id: black

- repo: https://github.com/pre-commit/mirrors-mypy
rev: 'v1.3.0'
hooks:
- id: mypy
additional_dependencies: [xarray]
- repo: https://github.com/pre-commit/mirrors-mypy
rev: 'v1.18.2'
hooks:
- id: mypy
additional_dependencies: [xarray]
25 changes: 0 additions & 25 deletions .readthedocs.yml

This file was deleted.

2 changes: 1 addition & 1 deletion LICENSE
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
MIT License

Copyright (c) 2024 Regents of the University of Colorado
Copyright (c) 2025 Regents of the University of Colorado

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
Expand Down
8 changes: 4 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
[![Run tests](https://github.com/MAVENSDC/cdflib/actions/workflows/ci.yml/badge.svg)](https://github.com/MAVENSDC/cdflib/actions/workflows/ci.yml)
[![codecov](https://codecov.io/gh/MAVENSDC/cdflib/branch/master/graph/badge.svg?token=IJ6moGc40e)](https://codecov.io/gh/MAVENSDC/cdflib)
[![Run tests](https://github.com/lasp/cdflib/actions/workflows/ci.yml/badge.svg)](https://github.com/lasp/cdflib/actions/workflows/ci.yml)
[![codecov](https://codecov.io/gh/lasp/cdflib/branch/master/graph/badge.svg?token=IJ6moGc40e)](https://codecov.io/gh/lasp/cdflib)
[![DOI](https://zenodo.org/badge/102912691.svg)](https://zenodo.org/badge/latestdoi/102912691)
[![Documentation Status](https://readthedocs.org/projects/cdflib/badge/?version=latest)](https://cdflib.readthedocs.io/en/latest/?badge=latest)

Expand All @@ -20,6 +20,6 @@ pip install cdflib

## Documentation

The full documentation can be found here:
The full documentation can be found here

[https://cdflib.readthedocs.io/en/latest/](https://cdflib.readthedocs.io/en/latest/)
[https://lasp.github.io/cdflib/](https://lasp.github.io/cdflib/)
Loading
Loading