Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
44 changes: 44 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
name: CI

on:
push:
branches: [main]
pull_request:
workflow_dispatch:

jobs:
test:
name: Offline tests (Python ${{ matrix.python-version }})
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
python-version: ["3.9", "3.10", "3.11", "3.12", "3.13"]
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python-version }}
- name: Install package with dev deps
run: |
python -m pip install --upgrade pip
pip install -e ".[dev]"
- name: Run offline tests
# Network tests download hundreds of MB of real data; they are excluded from CI
# and run manually. Everything else must pass on every supported Python version.
run: pytest -m "not network"

build:
name: Build & check distribution
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with:
python-version: "3.10"
- name: Build sdist and wheel
run: |
python -m pip install --upgrade pip build twine
python -m build
- name: Validate metadata and README rendering
run: twine check dist/*
33 changes: 33 additions & 0 deletions .github/workflows/deploy.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
name: deploy

on: workflow_dispatch

jobs:
deploy:
environment: Deploy
runs-on: ubuntu-latest
permissions:
# Required for PyPI Trusted Publishing (OIDC); no API token/secret needed.
id-token: write
contents: read
steps:
- run: echo "Starting deployment to PyPi"
- uses: actions/checkout@v5
- name: Set up Python 3.10
uses: actions/setup-python@v6
with:
python-version: "3.10"
- name: Install helpers
run: |
python -m pip install --upgrade pip
pip install build twine
- name: Build
# This project uses a pyproject.toml build backend (no setup.py), so build
# with the PEP 517 frontend rather than `setup.py sdist bdist_wheel`.
run: python -m build
- name: Check distribution metadata
run: twine check dist/*
- name: Deploy to PyPi
# Trusted Publishing: PyPI verifies this repo/workflow/environment via OIDC.
uses: pypa/gh-action-pypi-publish@release/v1
- run: echo "Successfully deployed to PyPi"
12 changes: 12 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -9,4 +9,16 @@
.pip/
**/__pycache__/
**/.temp_data/
test.py
.DS_Store
**/.DS_Store

# packaging build artifacts
build/
dist/
*.egg-info/
src/*.egg-info/

# editor / tooling
.vscode/
CLAUDE.md
21 changes: 19 additions & 2 deletions CHANGELOG.MD
Original file line number Diff line number Diff line change
Expand Up @@ -4,13 +4,30 @@ All notable changes to this project will be documented in this file.
The format is based on [Keep a Changelog](http://keepachangelog.com/)
and this project adheres to [Semantic Versioning](http://semver.org/).

## [Unreleased] - yyyy-mm-dd
## [0.1.0] - 2026-06-06

### Added
- Introduce structure for different dataset-specific tools
- ...
- `proxy` and `verify` options on the loaders for downloading behind a proxy / custom TLS
- `Matrix.to_pandas()` to convert a matrix to a labelled DataFrame
- OECD ICIO 2025 edition (`2025-regular`, `2025-extended`) and Eurostat Figaro 2025 edition (`2025`, years 2010-2023)
### Changed
- Renamed the package from `iopy` to `iotables` and adopted a `src/` layout with `pyproject.toml`
- Downloads now use `curl_cffi` (browser TLS impersonation) so Cloudflare-protected OECD files work
- Re-pointed OECD/Figaro download URLs to current hosts (OECD `webfs`, Figaro CIRCABC)
- Cached downloads now live in a user cache dir (`~/.cache/iotables`, override with `IOTABLES_DATA`)
- User-facing argument validation now raises `ValueError` instead of `assert` (so it survives `python -O`)
### Fixed
- OECD `2025-regular` file-id regex and in-zip CSV filenames; cross-edition cache-id collisions
- `remove_downloaded_files(database=...)` deleted the wrong database's files
- `ExioBase` download-failure handler raised `AttributeError` instead of the intended warning
- Loader `db_name` derivation used `str.rstrip('.py')` (a character set, not a suffix)
- Dropped `lru_cache` on `_load_data`, which pinned every loader instance (and its data) in memory
- Downloads are now atomic (written to a `.part` file and moved into place on completion), so an interrupted download no longer leaves a truncated file in the cache
- `download_file` now verifies the downloaded size against `Content-Length` (when advertised and unencoded) and raises on a truncated body
### Other
- Added a `py.typed` marker, `[project.urls]`, Python-version classifiers, and a GitHub Actions CI (offline tests across Python 3.9-3.13 + build/twine check)
- Network tests are marked `network` and moved into fixtures, so the suite collects and the offline tests run without a connection (`pytest -m "not network"`)
___


Expand Down
28 changes: 15 additions & 13 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# iopy
# iotables

## Description

Expand All @@ -14,14 +14,15 @@ Currently the following databases are supported:

## Installation
```bash
pip install git+https://github.com/WWakker/iopy.git
pip install iotables
```

## Structure of input-output data

Input-data model the economy in matrix form. Here, we explain input-output data using OECD data as an example.
THE OECD input-output tables contain inputs and outputs in current million USD for 45 sectors and 66 countries as well
as rest-of-world. China and Mexico data are split into CN1, CN2, MX1, and MX2.
The OECD input-output tables contain inputs and outputs in current million USD for a set of sectors and countries (the
exact counts depend on the edition; see `config.py`) as well as rest-of-world. China and Mexico data are split into CN1,
CN2, MX1, and MX2.

![OECD sctructure](res/OECD_structure.png)
*Source*: OECD
Expand Down Expand Up @@ -173,19 +174,20 @@ This is the Ghosh equation, which defines the relation between production output
### Create an instance of the OECD class

Creating an instance of the OECD class loads the OECD data and gives access to convenient attributes and methods.
An instance can be create as follows, specifiying a year between 1995 and 2018.
An instance can be created as follows, specifying an available year for the chosen version (e.g. 1995-2018 for the
`2021` version, up to 2022 for the `2025` versions).

```python
import iopy
oecd = iopy.OECD(version='2021', year=2018, refresh=False)
import iotables
oecd = iotables.OECD(version='2021', year=2018, refresh=False)
```

Similary, an instance can be created for other data, for example Figaro and ExioBase as

```python
import iopy
figaro = iopy.Figaro(version='2022', year=2020, kind='industry-by-industry')
exio = iopy.ExioBase(version='3.81', year=2022, kind='industry-by-industry')
import iotables
figaro = iotables.Figaro(version='2025', year=2020, kind='industry-by-industry')
exio = iotables.ExioBase(version='3.81', year=2022, kind='industry-by-industry')
```

Creating an instance of a database class downloads and loads the data into memory, creates standard input-output matrices, and gives access to the following attributes and methods:
Expand Down Expand Up @@ -223,11 +225,11 @@ All matrices are extended `numpy.ndarray`'s with attributes `info`, `rows` and `

When running a Leontief or Ghosh shock, the percentage shock to final demand/primary inputs in countries and sectors can be specified as
```python
import iopy
import iotables

oecd = iopy.OECD(version='2021', year=2018)
oecd = iotables.OECD(version='2021', year=2018)

df_l = oecd.leontief_demand_shock(shock=-10, regions=['FR', 'DE], sectors=['01T02', '35'])
df_l = oecd.leontief_demand_shock(shock=-10, regions=['FR', 'DE'], sectors=['01T02', '35'])
df_g = oecd.ghosh_supply_shock(shock=-10, regions=['FR', 'DE'], sectors=['01T02', '35'])
```

Expand Down
6 changes: 0 additions & 6 deletions iopy/__about__.py

This file was deleted.

7 changes: 0 additions & 7 deletions iopy/core/__init__.py

This file was deleted.

Loading
Loading