-
Notifications
You must be signed in to change notification settings - Fork 2
Expand file tree
/
Copy pathdatabricks.yml
More file actions
151 lines (144 loc) · 7.44 KB
/
Copy pathdatabricks.yml
File metadata and controls
151 lines (144 loc) · 7.44 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
bundle:
name: databricks-code-search
variables:
max_repos:
description: >-
Runaway-config guardrail: resolution fails if config.yaml resolves to more repos
than this. Not a wall-clock control (one huge repo outlasts a hundred small ones).
Raise it here and redeploy rather than rebuilding the wheel.
default: 500
github_token_secret_scope:
description: "Databricks secret scope holding the GitHub token."
default: code-search
github_token_secret_key:
description: "Secret key within the scope for the GitHub token."
default: github_token
index_schedule:
description: "Quartz cron for the indexing job (consumed by the job resource in #7)."
default: "0 0 */12 * * ?"
lakebase_project_name:
description: "Lakebase Autoscaling project id (also the display name)."
default: code-search
lakebase_endpoint_name:
description: "Auto-provisioned READ_WRITE endpoint id under the production branch (Phase-0: `primary`)."
default: primary
catalog_name:
description: "New Postgres-backed UC catalog (mirrors the DB; NOT a pre-existing catalog like main). Underscore, not hyphen."
default: code_search
database_name:
description: "Postgres database name (Lakebase default, Phase-0 verified)."
default: databricks_postgres
pg_version:
description: "Postgres major version (schema-typed int)."
default: 17
app_name:
description: "Databricks App name (must start with mcp-). Consumed by resources/app.yml."
default: mcp-code-search
webui_app_name:
description: "Databricks App name for the web UI (second app). Consumed by resources/webui.yml."
default: webui-code-search
job_run_as_sp:
description: >-
Prod ONLY: client id of the pre-created job run-as service principal (writer).
Per-workspace. Empty default keeps `validate -t dev` green (DABs requires a value
for every declared variable, even ones referenced only in another target). The
role is declared only under targets.prod, so an empty value never reaches dev.
Prod deploy MUST pass --var job_run_as_sp=<client-id> (an empty value would create
a broken NO-LOGIN role) — the prod deploy target enforces this.
default: ""
include:
- resources/*.yml
# The indexing job reads config.yaml from ${workspace.file_path} at runtime, so the file
# MUST reach the workspace. It syncs by default today, but .gitignore exclusions apply
# *silently* to bundle sync (databricks/cli#3547) — an explicit include is cheap,
# self-documenting insurance against a future ignore rule quietly breaking every run.
sync:
include:
- config.yaml
# The webui App installs on the uv path (Python 3.12) and needs BOTH pyproject.toml
# (synced by default) and uv.lock. But uv.lock is gitignored globally (see .gitignore) and
# bundle sync applies .gitignore *silently* (cli#3547) -- without this force-include the App
# would ship no lock, fail to select the uv path, and fall back to pip/3.11 where the
# databricks-code-search wheel (requires-python >=3.12) refuses to install. The wheel is
# force-included for the same insurance reason the .gitignore `!webui/wheels/*.whl` negation
# exists: a future broad `*.whl` ignore must never silently drop it from the App source.
# webui/.python-version pins the interpreter to 3.12 (the minor the lock is resolved against);
# it syncs by default today but is listed here so an ignore rule can never silently drop it and
# let the Apps build fall back to the newest interpreter (3.14), which breaks `uv sync --locked`.
- webui/uv.lock
- webui/wheels/*.whl
- webui/.python-version
# The wheel consumed by the serverless indexing job's environment `dependencies`.
# Built with uv (matches the repo's uv-based toolchain); DABs runs `build` itself.
artifacts:
wheel:
type: whl
build: uv build --wheel
path: .
targets:
dev:
mode: development
default: true
# CI target: a dedicated Lakebase project the integration suite runs against, so CI
# exercises the REAL engine instead of a `pgvector` stand-in. The beta surface this
# repo depends on -- lakebase_ann's `<=>` ordered-index path and lakebase_bm25's
# `ts <@> to_bm25query(...)` scorer -- exists in no Postgres image, so a container can
# prove the fusion plumbing but never the ranking or the query plan.
#
# NOT YET PROVISIONED. `.github/workflows/ci-lakebase.yml` is disabled until this
# project exists and its managed shared_preload_libraries includes
# lakebase_vector,lakebase_text (see docs/runbooks/ci-lakebase.md).
#
# Deliberately a SEPARATE project from dev/prod: each CI run forks a throwaway branch
# and freely creates/drops extensions and tables, which must never touch a
# data-bearing project. Keep its `production` branch un-migrated -- forks inherit
# installed extensions, and test_no_vector_extension_installed asserts the core
# migration leaves none.
ci:
mode: development
variables:
lakebase_project_name: code-search-ci
prod:
mode: production
workspace:
# production mode expects a single, stable deploy path (silences the validate
# recommendation and prevents multiple copies across principals).
root_path: /Workspace/Users/${workspace.current_user.userName}/.bundle/${bundle.name}/${bundle.target}
# Prod-only: the job runs as ${var.job_run_as_sp}, but the bundle syncs to the
# DEPLOYER's home (root_path above). Nothing else establishes that the job SP can READ
# config.yaml there — secret-scope ACLs are granted separately and do not cover
# workspace files — so every prod run would fail at config read with a 403.
# Dev never surfaces this because dev runs as the developer.
# Target-scoped for the same reason postgres_roles is (see the note at the
# job_run_as_sp variable): its "" default would resolve to an empty principal and
# break `validate -t dev` if this block sat at top level.
# The block is the DECLARED source of truth for the workspace-folder ACL --
# DABs reconciles the folder to exactly what is listed here. The deployer entry
# is therefore mandatory, not decorative: without it a prod deploy would strip
# every human/CI identity from the bundle root and lock out all subsequent
# deploys, recoverable only out of band.
permissions:
- level: CAN_MANAGE
user_name: ${workspace.current_user.userName}
- level: CAN_VIEW
service_principal_name: ${var.job_run_as_sp}
# Prod-only: the pre-created job run-as SP's Postgres writer role, declared so it
# exists at `bundle deploy -t prod` BEFORE `scripts/migrate.py --apply-grants` (#5)
# grants it and BEFORE `bundle run`. Dev uses the developer identity for writes, so
# this role is prod-only and dev stays input-free. `postgres_role` == SP client id
# (Phase-0 Task 3); `role_id` is a letter-leading slug.
resources:
postgres_roles:
job_writer:
# `.id` ref (not a literal path) so DABs orders the project before this role.
parent: ${resources.postgres_projects.pg_project.id}/branches/production
role_id: code-search-job-writer
postgres_role: ${var.job_run_as_sp}
identity_type: SERVICE_PRINCIPAL
auth_method: LAKEBASE_OAUTH_V1
# Prod runs the indexing job as the pre-created writer SP (dev uses the
# developer identity, so this overlay is prod-only).
jobs:
code_search_index:
run_as:
service_principal_name: ${var.job_run_as_sp}