[DENG-10697] Add support for targets and use in query run

[scholtzan]

Description

Related to https://docs.google.com/document/d/18uOGoZaE9xdvAXkTs48Vygsju0XcuQr-9KPjwHmCnD4/edit?tab=t.0#heading=h.uwntr4tagom8

Initial implementation of target-based development workflows, as described in this proposal: https://docs.google.com/document/d/18uOGoZaE9xdvAXkTs48Vygsju0XcuQr-9KPjwHmCnD4/edit?tab=t.0#heading=h.uwntr4tagom8. This is an early proof of concept. Only ./bqetl query run is updated for now to support --target.

New file targets.yaml to define named target environments with a project and dataset prefix.
--target <name> top-level flag to load the target and stores it in click context
./bqetl --target dev query run: rewrites references and writes results to the destination table in the target project (auto-deploys schema if missing)

Examples

Configure a target for your development environment in ~/targets.yaml:

targets:
    dev:
      project_id: dev-sandbox-user  # or moz-fx-data-dev
      dataset_prefix: user_{{ git.branch }}_{{ git.commit }}_{{ artifact.project_id }}

Then use --target dev with commands to run and deploy artifacts to the dev environment:

  ./bqetl --target dev query run telemetry_derived.clients_daily_v6  --parameter=submission_date:DATE:2026-02-22 --write

When using --target, a local copy of the artifacts targeting the target project with dataset prefixes will be created:

• Copies to sql/dev-sandbox-user/user_<branch>_<commit>_moz_fx_data_shared_prod_telemetry_derived/clients_daily_v6/
• Rewrites references (deployed tables → dev, others → prod)
• Deploys schema to dev-sandbox-user.user_<branch>_<commit>_moz_fx_data_shared_prod_telemetry_derived.clients_daily_v6
• Runs query and populates data
• The local copy of the artifacts should not be committed to the repo. .gitignore:

Not yet wired up: backfill, deploy, view publish, routine publish (coming in follow-up PRs).

Related Tickets & Documents

• https://mozilla-hub.atlassian.net/browse/DENG-10697
• Test branch with more functionality built out: main...support-targets

Reviewer, please follow this checklist

[github-actions]

Integration report for "DENG-10697 Add support for targets and use in query run"

sql.diff

No content detected.