As mentioned on Slack, during Christmas @Camilletti and I had some fun refreshing the ConvGRU baseline already in mlcast.
What bothered me was that all the advanced models we're considering (LDCast, etc.) are generative and produce ensembles by nature, while ConvGRU was purely deterministic.
The fix turned out to be surprisingly simple:
- Feed noise into the decoder instead of zeros
- Forward the decoder N times → N ensemble members
- Replace MSE/MAE with CRPS loss (à la AIFS)
- profit! :)
The architecture stays unchanged. Deterministic training is still possible, so we lose nothing.
Preliminary results on Italian radar data look promising - decent rank histogram and better CRPS than a STEPS ensemble (pysteps):
Current implementation lives here: https://github.com/DSIP-FBK/ConvGRU-Ensemble
Full evaluation is ongoing. If results hold up, I'll open a PR to bring this into mlcast.
(Better names than "ConvGRU-CRPS" are welcome!)
As mentioned on Slack, during Christmas @Camilletti and I had some fun refreshing the ConvGRU baseline already in mlcast.
What bothered me was that all the advanced models we're considering (LDCast, etc.) are generative and produce ensembles by nature, while ConvGRU was purely deterministic.
The fix turned out to be surprisingly simple:
The architecture stays unchanged. Deterministic training is still possible, so we lose nothing.
Preliminary results on Italian radar data look promising - decent rank histogram and better CRPS than a STEPS ensemble (pysteps):
Current implementation lives here: https://github.com/DSIP-FBK/ConvGRU-Ensemble
Full evaluation is ongoing. If results hold up, I'll open a PR to bring this into mlcast.
(Better names than "ConvGRU-CRPS" are welcome!)