`jetstream preview ...` slow due to re-processing (local) repos

[ncalexan]

I'm trying to use jetstream preview ... to make progress on mozilla/metric-hub#1422. So far, it's been a wretched experience, not least because it's incredibly slow. My command line looks like:

$ jetstream preview --config-repos=/Users/nalexander/Mozilla/metric-hub --config-repos=/Users/nalexander/Mozilla/metric-hub/jetstream --platform fenix --sql-output-dir=output --experiment-slug=newly-dormant-push-programme-phase-1-android-retest

I've profiled my failing run and I find that it spends >100s in with_configs_from. It appears to do the actual work many, many times, i.e., there's no caching or single computation re-used.

In addition, it's grinding over various git operations that are surely not relevant to a local repository: does it really matter what the git commit date is? Couldn't we use the on-disk metadata?

In addition, I need to specify both metric-hub and metric-hub/jetstream. (Took a while to figure that out!) Why?

┆Issue is synchronized with this Jira Task

[ncalexan]

This is even worse than I thought: deep in metric_config_parser, we eagerly invoke git rev-parse HEAD -- <path> for every TOML file to determine a last_modified date. We only use those dates in the jetstream rerun* commands.

Ideally, this would be lazy, but that might have other implications (e.g., we'd need to hang onto repo references, potentially longer than we care to). Alternatively, we could make this a function of the with_configs_from constructor, avoiding the evaluation for commands that we know it's not needed for.