Skip to content

Create simple progress visualisation #41

@thill-odi

Description

@thill-odi

At the moment, the process for tracking progress between stages 1 and 2 of the harvest is extremely manual.

It would be possible to ameliorate this by:

  1. Adding a field in the harvester-stage-2-state which holds the value of updatedLastSeen as a Date rather than epoch value (e.g. updatedLastSeenAsDate)

  2. Creating an index pattern of the form 'harvester*' in Kibana to allow querying across indices.

  3. Creating visualisation comparing the before and after values of 'updated' in the harvester-raw index to give a rough sense of how much processing remains to be done.

Metadata

Metadata

Labels

enhancementNew feature or request

Type

No type
No fields configured for issues without a type.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions