Current
Data hydration works as follows:
- Fetch data from source inputs (PDFs, cheminements.txt, APIs)
- Parse the data
- Save it to database
Goal
Change this pipeline so hydration happens from snapshots instead of directly from source inputs:
- Save data as a snapshot
- hydrate the backend from that snapshot only
This should make the data aggregation pipeline easier to maintain.
(Attributes planificationPdfJson and horaireCoursPdfJson was a good start but never fully implemented, remove it and create table.s containing snapshots)
Current
Data hydration works as follows:
Goal
Change this pipeline so hydration happens from snapshots instead of directly from source inputs:
This should make the data aggregation pipeline easier to maintain.
(Attributes
planificationPdfJsonandhoraireCoursPdfJsonwas a good start but never fully implemented, remove it and create table.s containing snapshots)