Add execution of notebooks to CI

.github/workflows/changelog-check.yml

@@ -22,4 +22,4 @@ jobs:
           else
             echo "You must update CHANGELOG.md before merging into main."
             exit 1
-          fi
+          fi

.github/workflows/sphinx.yml

@@ -28,11 +28,26 @@ jobs:
               with:
                     enable-cache: true
 
+            - name: Setup cache
+              uses: actions/cache@v4
+              with:
+                path: |
+                  $TMPDIR/*.pkl
+                  $TMPDIR/*.grib
+                key: danra-${{ hashFiles('danra-book/scripts/utils.py') }}
+
+            - name: Write CDS API key (needed for notebooks/paper-figures.ipynb)
+              run: |
+                echo "url: https://cds.climate.copernicus.eu/api" > ~/.cdsapirc
+                echo "key: ${{ secrets.CDS_API_KEY}}" >> ~/.cdsapirc
+                chmod 600 ~/.cdsapirc
+
             - name: Build Jupyter Book
-              run: uv run jupyter-book build danra-book/
+              run: uv run jupyter-book build -W -n --keep-going danra-book/
 
             - name: Upload Jupyter Book as artifact (for debugging)
               uses: actions/upload-artifact@v4
+              if: always()
               with:
                     name: jupyter-book
                     path: danra-book/_build/html

.zenodo.json

@@ -47,4 +47,4 @@
             "affiliation": "Danish Meteorological Institute"
         }
     ]
-}
+}

danra-book/_config.yml

@@ -6,12 +6,6 @@ author: DMI NWP Team
 logo: logo.png
 copyright: '2025'
 
-# Force re-execution of notebooks on each build.
-# See https://jupyterbook.org/content/execute.html
-execute:
-  exclude_patterns:
-    - 'notebooks/**.ipynb'  # exclude all notebooks in the notebooks folder
-
 # Define the name of the latex output file for PDF builds
 latex:
   latex_documents:
@@ -21,6 +15,11 @@ latex:
 bibtex_bibfiles:
   - references.bib
 
+execute:
+  execute_notebooks: auto
+  allow_errors: false
+  timeout: 7200
+
 # Information about where the book exists on the web
 repository:
   url: https://github.com/dmidk/danradocs # Online location of your book

danra-book/_toc.yml

@@ -15,8 +15,6 @@ parts:
     - file: docs/modelspecs.md
     - file: docs/data-availability.md
     - file: docs/data-details.md
-#    - file: docs/annex.md
-#    - file: docs/references.md
 
   - caption: User Examples
     chapters:

danra-book/about.md

@@ -1,4 +1,4 @@
 # About DANRA
 DANRA is a high-resolution meteorological reanalysis dataset for Denmark and Northwestern Europe developed by the Danish Meteorological Institute (DMI). The dataset covers the period September 1990 to December 2023. It is produced using state of the art numerical models and data assimilation, three-hourly analyses and hourly forecasts on a 2.5 km spatial grid.
 
-The DANRA-Book is a collection of resources for the DANRA project.
+The DANRA-Book is a collection of resources for the DANRA project.

danra-book/docs/data-availability.md

@@ -33,9 +33,9 @@ ds_danra_sl
 
 Alternatively, the data can be accessed directly from the S3 object store:
 
-- [s3://dmi-danra-05/height_levels.zarr](s3://dmi-danra-05/height_levels.zarr)
-- [s3://dmi-danra-05/single_levels.zarr](s3://dmi-danra-05/single_levels.zarr)
-- [s3://dmi-danra-05/pressure_levels.zarr](s3://dmi-danra-05/pressure_levels.zarr)
+- [s3://dmi-danra-05/height_levels.zarr](https://registry.opendata.aws/dmi-danra-05/)
+- [s3://dmi-danra-05/single_levels.zarr](https://registry.opendata.aws/dmi-danra-05/)
+- [s3://dmi-danra-05/pressure_levels.zarr](https://registry.opendata.aws/dmi-danra-05/)
 
 Fetching one of these datasets with Python and xarray using simple loading is as simple as
 ```
@@ -49,4 +49,4 @@ ds_danra_sl = xarray.open_zarr(
 )
 ```
 
-Efforts are currently underway to prepare the DANRA forecasts for public release. Once available, the dataset will provide hourly temporal resolution. Updates and further details will be posted on the website.
+Efforts are currently underway to prepare the DANRA forecasts for public release. Once available, the dataset will provide hourly temporal resolution. Updates and further details will be posted on the website.

danra-book/docs/data-details.md

@@ -1,5 +1,5 @@
 # Details about parameters
-This section describes special parameters in the DANRA dataset. 
+This section describes special parameters in the DANRA dataset.
 
 ## DANRA Parameters
 
@@ -119,8 +119,8 @@ The icing index, <code>icei0m</code>, is a measure of the risk of icing in the a
 | 1    | Trace |
 | 2    | Light |
 | 3    | Moderate |
-| 4    | Severe | 
+| 4    | Severe |
 
 <h3>AROME Hail diagnostic</h3>
 The hail diagnostic, <code>xhail0m</code>, is a measure of the risk of hail. It is the maximum total column graupel in kg/m^2 since previous post-processing. Values above 16 kg/m^2 are considered to be a risk of hail.
-</details>
+</details>

danra-book/docs/description.md

@@ -4,4 +4,4 @@ DANRA is produced using three-dimensional variational (3D-Var) data assimilation
 DANRA runs on a horizontal grid mesh of 800 x 600 at 2.5 km grid size, with 65 hybrid sigma/pressure (model) levels in the vertical, with the bottom level at around 12 meters height and the top level at 10 hPa.
 For data assimilation, quality-controlled local observation data from synoptic stations in Denmark and nearby countries have been added, along with conventional weather observations collected from the Global Telecommunication System (GTS) for synoptic, ship, drift buoy, radiosonde, and aircraft observations, as well as satellite remote sensing data of radiance, atmospheric motion vectors (AMV), scatterometer, and radio occultation.
 Selected atmospheric data are available on model levels and are also interpolated to 14 pressure levels. "Surface or single level" data are also available, containing 2D parameters such as screen-level properties, accumulated precipitation, top-of-atmosphere radiation, and vertical integrals over the entire depth of the atmosphere.
-The DANRA dataset contains high-resolution reanalysis for key meteorological parameters. Analysis data are available at 3-hour frequency, while forecast output will be available at 1-hour frequency (expected November 2025).
+The DANRA dataset contains high-resolution reanalysis for key meteorological parameters. Analysis data are available at 3-hour frequency, while forecast output will be available at 1-hour frequency (expected November 2025).

danra-book/docs/documentation.md

@@ -8,8 +8,4 @@
 
 [Details about parameters](data-details.md)
 
-[Known issues](issues.md)
-
-[Annex](annex.md)
-
-[References](references.md)
+[Known issues](https://github.com/dmidk/danradocs/issues)

danra-book/docs/modelspecs.md

@@ -11,7 +11,7 @@ The boundary conditions for the DANRA reanalysis are derived from the ECMWF Glob
 The DANRA dataset uses a Lambert Conic Conformal projection with the following specifications:
 
 ### CRS WKT
-```wkt
+```
 PROJCRS["DMI HARMONIE DANRA lambert projection",
     BASEGEOGCRS["DMI HARMONIE DANRA lambert CRS",
         DATUM["DMI HARMONIE DANRA lambert datum",
@@ -74,4 +74,4 @@ DANRA has produced in several production streams. Each stream has an associated
 | 2009 | 2010.09.01 - 2014.08.30 |
 | 2013 | 2014.09.01 - 2017.08.30 |
 | 2016 | 2017.09.01 - 2019.11.30 |
-| 2018 | 2019.12.01 - 2023.12.31 |
+| 2018 | 2019.12.01 - 2023.12.31 |