Pipeline: managements table → events.json with gap filling (MVP)

[AritraDey-Dev]

Pipeline: managements table → events.json with gap filling (MVP)

Summary

Build an automated pipeline that reads a curated cal/val workbook (e.g. https://docs.google.com/spreadsheets/d/1pXiZUkNP50WXbmAztoEgUJ6rpQNyewmobCuaFHL8UjQ/edit?gid=1394889519#gid=1394889519) and produces one events.json per treatment, filling uncertain dates and missing management info using Akash's priors (ccmmf/scenarios#3).

MVP logic

1. Dates — use min_date if min_date == max_date, else use the midpoint.
2. Missing parameters — sample from management_priors.yaml (e.g. fertilization rate, irrigation volume).
3. Missing events — if a system is irrigated but has no irrigation rows, infer them from crop + location + method, then fill from priors.

Event-type rules (per David)

• Tillage: categorical only (did it happen). Implement/depth optional.
• Irrigation: if irrigated, expect events with min_date/max_date = planting/harvest window + method (sprinkler/drip/furrow/flood). Pipeline fills amount, timing, and canopy-vs-soil application.

Inputs / Output

• In: cal/val workbook + management_priors.yaml + sample_priors.R
• Out: one events.json per treatment

Out of scope (follow-up)

Step 2: refine filled values using nearby monitoring-data event parquet files (5–10 km radius, same crop, median values).

Tasks

• Workbook reader (managements / treatments / sites tabs)
• Date midpoint utility
• Prior-sampling helpers wired to management_priors.yaml
• Missing-event detector (e.g. irrigation gap for irrigated systems)
• events.json writer

Open question

• events.json schema — reuse SIPNET-compatible spec or define fresh?

cc @dlebauer

[dlebauer]

This looks like a solid plan, consistent with what we discussed in Slack.

A few notes:

• The events.json schema is intended to be model agnostic (though at this point v0.1.1,with sipnet as the only use case, it hasn’t been generalized).
• Ultimately, the goal will be to gap-fill based on the management events timeseries generated from trends across monitoring inputs, especially for event dates and irrigation rates

It is not clear if it will be easier to do that first than 'just' use priors. But that workflow would be something like:

• use event parquet files generated by the managements pipelines
• subset to a fixed distance around the site in question, e.g. a radius of 5 or 10 km around the location that the cal/val data was collected
• subset to the crop being managed
• calculate the median date and for event type i
• if applicable (depending on event type), calculate