feat(worker): expand ${VAR} env references in startup-application config

[nilsmechtel]

Summary

read_startup_applications now recursively runs os.path.expandvars over each parsed startup-application config, so a startup app can reference the worker's own environment — including secrets mounted via valueFrom.secretKeyRef — as ${VAR}.

Motivation

Startup apps can only receive env vars through application_env_vars in their startup-app JSON, and that JSON lives in git-tracked helm values. There was no way to give a startup app a secret without writing the raw value into git. Concretely, model-runner (≥1.15.11) needs HF_READ_TOKEN to download gated HuggingFace weight sources (e.g. facebook/sam3) during package preparation — and that token must not be committed.

With this change the operator can wire a k8s secret through cleanly:

# worker deployment env (like HYPHA_TOKEN today)
- name: HF_READ_TOKEN
  valueFrom: { secretKeyRef: { name: bioengine-secrets, key: HF_READ_TOKEN } }

// startup-applications entry
"application_env_vars": {"*": {"_HF_READ_TOKEN": "${HF_READ_TOKEN}"}}

The token value lives only in bioengine-secrets; git holds only the secretKeyRef and the ${HF_READ_TOKEN} placeholder.

What changed

• bioengine/worker/__main__.py: new _expand_env_in_config (recurses dict/list/str), called on each parsed startup-app config in read_startup_applications.
• Expansion runs only on operator-supplied --startup-applications CLI config, never on tenant deploy_app calls, so sourcing from the worker env is safe. An unset ${VAR} is left verbatim.
• Unit tests in tests/worker/test_startup_app_env_expansion.py.

Testing

• Unit tests cover recursion, type preservation, unset-var passthrough, and the end-to-end read_startup_applications expansion.
• Deployment-side dev-image validation on a live KTH worker to follow before marking ready.

🤖 Generated with Claude Code