feat(gcp): add 37 resource types across thirteen services - #196
Merged
Conversation
Client and server TLS policies, backend authentication configs,
authorization policies, gateway security policies and their nested rules,
and DNS threat detectors. Every one was probed against the live API before
being written, and each of the following compensates for something that
probe found.
gatewaySecurityPolicies is regional. Asked for locations/global it answers
400 "Malformed name", so it stays out of globalResourceTypes alongside
urlLists rather than being inferred from ctx.Location.
The nested rules needed three things the package did not have: a path
builder that renders the parent segment, a wildcard parent for a List with
no parent in context, and a native ID parser. Network Security accepts "-"
as a wildcard parent - verified live - so one call enumerates the rules of
every policy in the region and no walking provisioner is needed. Without
the parser the generic path walker overwrites ResourceType as it goes and a
rule's read addresses ".../locations/{region}/rules/{rule}", which 404s.
Three fields come back as empty values that were never sent:
gatewaySecurityPolicy.tlsInspectionPolicy and rule.applicationMatcher as
empty strings, rule.tlsInspectionEnabled as false. The first two are
stripped on read, because state carrying a value against a declaration that
omits the field reads as drift on every sync; the third is a top-level bool,
where a hasProviderDefault hint reaches it.
dnsThreatDetectors is the one collection here that is not a long-running
operation, so it carries its own OperationConfig. That override is
required, not tidiness: a create whose response has no "/operations/"
segment leaves the operation-id extractor returning nothing, and unlike
delete - which reads an empty operation id as already finished - create
reports the resource as in progress with an empty request id and then polls
a URL that is not an operation.
tlsInspectionPolicies and authzPolicies were probed and deliberately left
out. A TLS inspection policy needs a CA pool with an enabled certificate
authority, which is billable; an authz policy needs a real forwarding rule
or Secure Web Proxy gateway as its target and answers an internal error
without one.
A hub on its own connects nothing. A spoke links one VPC network into its mesh, and the hub then propagates that network's routes to every other spoke it has accepted. Only the VPC-network shape is exposed. linkedVpnTunnels, linkedInterconnectAttachments, linkedRouterApplianceInstances and gateway each require an underlay that bills by the hour, while a hub, an empty VPC and a VPC spoke are all free to hold. The discovery document files spokes under projects.locations, which reads as regional, but a VPC spoke was verified to create, read, patch and delete only under locations/global - so the collection is pinned global and a target's region cannot reach the URL. hub is reported as a full resource path while a Hub resolvable yields a short id, and the field is immutable. Both halves of that translation are here for that reason: expanding on the request without shortening on the response leaves the forma and the stored state permanently disagreeing, and every re-apply then plans a replacement the API refuses anyway. linkedVpcNetwork carries output-only members - vpcNetwork, the proposed*ExportRanges pair, producerVpcSpokes - that a hasProviderDefault hint cannot reach because it only applies to top-level fields, so they are stripped from the response instead.
…ucket
rolloutPlans has insert, get, list and delete and no update method at all,
so every field is immutable: the type is registered without an update
operation rather than with a patch that would 404, and its conformance case
is a replace case. The server also stamps an output-only `number` onto each
stored wave; it sits inside waves[], where a schema hint cannot reach it, so
it is stripped on read. Without that strip every read disagrees with the
declaration - and with no update method the disagreement plans a
replacement on every reconcile rather than a no-op patch. A wave's
includedLocations takes bare zone or region names; a scoped path is refused
outright.
A zone VM extension policy with no instanceSelectors selects every VM in
the zone and installs the named extension on all of them. The schema says
so at the field and the fixtures always carry a labelSelector that matches
nothing. After a patch the API echoes each extensionPolicies entry with a
stringConfig it was never sent, so an empty stringConfig is stripped on
read; an empty pinnedVersion is not, because "" is how a forma says "track
the current release".
globalVmExtensionPolicies is deliberately absent. Its delete is a
POST .../{name}/delete carrying a rolloutInput body, and - verified live -
that call's operation reports DONE while the policy is still there,
disappearing only minutes later when an out-of-band purge rollout catches
up. A delete the plugin cannot observe completing is a delete it cannot
promise.
RegionBackendBucket is the global type's body under regions/{region}, with
one difference the API insists on: loadBalancingScheme is required at
regional scope and an insert without it is refused. That rejection is the
only regional-specific behaviour observed live. The full create/read/patch/
delete cycle was not: the probe's insert finished with
GCS_BUCKET_ACCESS_DENIED because the service account this project is tested
with cannot read a Cloud Storage bucket, and the same failure reproduces
against the already-shipping global BackendBucket with the same bucket.
regionHealthCheckServices, instantSnapshotGroups and
regionInstantSnapshotGroups were probed and dropped: the first is refused
with "HealthCheck as a Service feature is not available for this project",
and the other two need a non-empty disk consistency group, whose instant
snapshots are billed for storage.
A user-managed configuration naming where a Spanner instance's replicas may live. Configuration only, and free: nothing is provisioned or billed until an instance is created against it, and this batch creates none. None of base's three request hooks can build this create body, so the transformer emits the whole envelope itself. The id appears twice - instanceConfigId beside the wrapped object and the full resource path inside it - and a create that omitted instanceConfig.name was refused 400 "Invalid CreateInstanceConfig request." RequestWrapper wraps everything the transformer produced, so it cannot leave a sibling beside the wrapped object, and both the create and the patch envelope need one. The patch field mask goes in the body rather than the query string, so UpdateMaskFromBody is off, and it is fixed at displayName,labels - the only two fields the API will update - rather than derived from what is present, so a forma that drops its labels actually clears them. A read reports optionalReplicas: every replica location GCP offers for the base configuration, each with a display name and a labels map. It is never declarable and is stripped rather than stored on every configuration. parseSpannerNativeID previously demanded parts[2] == "instances" and so rejected an instance configuration's id, which is project-scoped rather than instance-scoped.
A named set of Redis OSS ACL rules that a Memorystore for Redis Cluster attaches. The policy provisions nothing and is billed only through the clusters that reference it, which is why this batch covers it and not GCP::Redis::Cluster. It is the one collection in this API that is not a long-running operation: create answers 200 with the finished policy and no Operation to poll, so it carries its own OperationConfig. Left on the async path a create would report in progress with an empty request id and then poll the bare base URL, which reports on nothing. clusterAclPolicyAttachments, the output-only per-cluster attachment status, is a nested array no schema hint can reach, so it is stripped on read: a policy some cluster later attaches must not start reading as drift against a forma that never mentioned it. The new base hook, ResourceConfig.ReadTreatAsMissing, exists because this API keeps serving a tombstone. aclPolicies.delete answers HTTP 200 and the policy then reads back for another fifteen to twenty seconds with "state": "DELETING" - measured against the live API - before the GET finally 404s. A synchronization landing inside that window read the tombstone as a live resource and put a deleted policy back in inventory, which is exactly what the out-of-band-delete conformance step caught. The hook turns such a read into NotFound. It is read-side rather than a delete-side wait on purpose: the delete really has been accepted, and it is the read that is lying. Every other resource leaves it nil and is unaffected.
README rows and CHANGELOG entries for the thirteen new types, and a disposition row for each new hasProviderDefault annotation. Six new sweep sections, five of which close pre-existing gaps rather than covering only this batch. Nothing swept the Network Security API at all, though four of its types already shipped; nothing swept backend buckets in either scope, and gcloud cannot list the regional ones, so both go over REST. Rollout plans, zonal VM extension policies, Redis ACL policies and Network Connectivity service connection tokens have no gcloud surface either. A gateway security policy DELETE is refused with HTTP 400 while it still has rules, so the rules are collected first, through the same "-" wildcard parent the plugin uses for discovery. Spanner instance configurations need their own pattern. Spanner requires a user-managed configuration's id to begin "custom-", so the fixture is named custom-formae-test-sic-<runID> and SWEEP_RE, anchored at the start of the name, does not match it. SIC_RE matches that one prefixed form; narrowing SWEEP_RE instead would loosen it for every other collection, and a Google-managed configuration has no "custom-" prefix so it can never be caught.
With the test project's IAM gap closed, this type could finally be run end to end - and its update failed. The update fixture had been copied from the global backend-bucket case, which flips enableCdn false -> true. Regional scope refuses that outright: Invalid value for field 'resource.enableCdn': 'true'. CDN is not supported for backend bucket with scope REGION and load balancing scheme: EXTERNAL_MANAGED Cloud CDN is a global, external feature. The field stays declarable because the API reports it back as false on every read, so it carries a provider default rather than being absent - but only false is accepted, and the schema now says so at the field. The update case changes the description instead, which is what a regional backend bucket will actually patch. CRUD 8/8 and discovery 4/4 now pass for this type, so nothing in this batch is unverified any more.
* feat(networkservices): add Cloud Service Mesh, seven types
A mesh, the four route kinds that attach to it, an endpoint policy and a
service load balancing policy. This is the direct consumer of the Network
Security policies added earlier on this branch: an endpoint policy's
clientTlsPolicy, serverTlsPolicy and authorizationPolicy all point at them.
Every collection answers 200 at both locations/global and locations/{region},
and they are separate namespaces - a mesh created globally is absent from the
regional list and 404s on a regional GET. A wrong guess creates unfindable
resources rather than failing, so the scope is pinned in a map with that
reasoning rather than inferred from the target.
Reference fields are full paths on the wire and short names in a forma, and
the short form is refused, so both halves of the translation exist with the
identity pinned in unit tests.
A PATCH here validates the resource from the request body alone rather than
merging over stored state, so an update omitting type, hostnames or rules is
refused as though the stored value did not exist. Those fields are
non-optional and always sent. This API also clears labels on any patch whose
mask does not name them, and omits false booleans the proto3 way so a declared
false reads back absent; both are documented at the fields.
Each route kind avoids a billable prerequisite differently: HTTP redirects,
gRPC injects a fault, TCP uses its original destination, and TLS - the one
kind with no destination-free action - points at a backend service with no
backends. gateways is absent from all four because a gateway allocates Envoy
proxies and bills, so the field could not be verified. ServiceBinding is
absent because its only functional field is deprecated in the API's own
discovery document.
* feat(clouddeploy): add Cloud Deploy, five types
Target, DeliveryPipeline, CustomTargetType, DeployPolicy and Automation - the
declarative half of Cloud Deploy. Releases, rollouts and job runs are
deliberately absent: a release renders through Cloud Build and a rollout
actually deploys, so both cost money, and neither is a desired state rather
than a record of an action already taken.
This is where polling earns its keep. A create accepted with HTTP 200 came
back as a done operation carrying an error - a stage naming its target by full
path is refused with "is not a valid resource ID for resource type
stage.targetId" - and the pipeline never existed. The API also serialises
operations per resource and answers a second concurrent mutation with 409
ABORTED.
The reference forms are mixed, which is the trap: stage.targetId must be a
short id and rejects a full path, while customTarget.customTargetType is a
full path. So a pipeline stage passes its resolvable straight through while
Target expands and shortens.
executionConfigs is not declarable. Cloud Deploy fills it in whether or not it
was sent and rewrites what was sent, adding artifactStorage, executionTimeout
and a defaultPool mirror - nested too deep for a schema hint to tolerate and
too meaningful to strip. An automation's rules come back carrying an
output-only condition two levels down, which is stripped instead.
Discovery needs no walking provisioner: Cloud Deploy accepts "-" as a wildcard
delivery pipeline.
* feat(dataform): add Repository, Workspace, ReleaseConfig, WorkflowConfig
Workflow invocations and compilation results are absent: both execute against
BigQuery. Verified rather than assumed that the shipped types do not - a
workflow config with no cronSchedule left both collections empty, and this
project cannot schedule automatic releases at all.
The whole API is synchronous; no operations collection exists in it. Left on
the async path a create would report in-progress with an empty request id and
then poll the bare base URL forever.
Three fields are immutable in a way nothing in the discovery document says,
and immutable by field mask rather than by value: a PATCH whose mask names
invocationConfig, codeCompilationConfig or kmsKeyName is refused with
"update_mask contains immutable fields" even when the object sent is
byte-identical to the stored one. Since the mask is built from the body, all
three leave update bodies - that is the difference between a working timeZone
change and a rejected one.
internalMetadata, a bookkeeping blob that changes on every write, is stripped
from all four types, along with containingFolder, which can only be changed
through a custom verb and so could never be reconciled. A workspace has no
patch method at all, so it replaces.
The schema module is repository_dataform.pkl: verify-schema rejects duplicate
basenames across services and artifactregistry already ships repository.pkl.
* feat(firestore): add Database
Every mutating call in the Firestore Admin API answers with a long-running
operation and two of the three cannot be polled. The operation a PATCH names
answers 404 "Operation does not exist" on a patch that plainly applied, and
the one a DELETE names carries the deleted database under "response" but never
sets "done" - not false, absent - so an async poller waits forever on a
database already gone. Create needs no polling either: it returns the finished
database inline in about a second. So the type is registered synchronous and
reads the resource back after an update.
Three fields the API volunteers are dropped rather than defaulted. etag is
recomputed on every read of an untouched database - three consecutive GETs
gave three etags with an identical updateTime. earliestVersionTime moves
continuously. realtimeUpdatesMode is reported for every database and refused
on create for Standard-edition ones, so declaring it would put a value in
state that fails the create it came from. enhancedTextSearchQueryMode is
dropped for a different reason: the API returns it on every database and the
v1 discovery document does not mention it.
deleteProtectionState is declared explicitly in the fixture because a database
created with protection enabled cannot be deleted at all, and every run would
leak one. A deleted id is held for about five minutes, so the case exercises
update rather than replace.
Index is absent: its name is server-assigned and the API ignores a
caller-supplied one, so a declared name could never round-trip, and an index
build on an empty collection did not complete in five minutes of polling.
BackupSchedule is absent because retained backup data bills.
The schema module is database_firestore.pkl: verify-schema rejects duplicate
basenames and sql already ships database.pkl.
* feat(parametermanager): add Parameter and ParameterVersion
Global only, and that is a limit of the API rather than a simplification.
Parameter Manager serves each region from its own host,
parametermanager.<region>.rep.googleapis.com, and serves locations/global only
from the plain host. Cross the two and it answers 403 PERMISSION_DENIED, "Read
access to project was denied" - a wrong-host error wearing a missing-IAM-
grant's clothes, which is worth knowing before anyone responds to it by
granting a role. base.APIConfig.BaseURL is one constant string, so the
location segment is pinned to global and a target configured for a region
cannot walk into that 403. What base would need to support the regional host
is written down where the next person will look.
A version's payload is stripped from every response, and that is a security
control rather than a drift fix. The payload is user data and may be a secret,
and unlike Secret Manager - whose API withholds secret material - Parameter
Manager hands it straight back, both on a GET defaulting to view=FULL and in
the create response. So data is declared write-only, the payload is removed on
read, and versions:render, which would additionally resolve Secret Manager
references inside it, is never called from a read path.
The whole API is synchronous. disabled is ignored on create - a create sending
true answers 200 and the version is enabled anyway - so it carries a provider
default and takes effect only through a patch. A parameter with versions
refuses to delete and its delete takes no force flag, so the declared
reference is also what orders the teardown.
* feat(binaryauthorization,apikeys): add Attestor, PlatformPolicy and Key
The attestor's Grafeas note lives in the Container Analysis API, which this
plugin does not ship, and that turns out not to matter: Binary Authorization
does not resolve noteReference at create time, so an attestor pointing at a
note that does not exist - in a project where Container Analysis is not even
enabled - is accepted. That is what makes the type declarable with no
prerequisite in someone else's API.
A public key sent without an id comes back with one the API computed from the
key's DER digest, so id is required rather than defaulted.
asciiArmoredPgpPublicKey is absent from the schema for the harder version of
the same problem: with a PGP key the API overwrites id with the key's
fingerprint, so declared and stored could never agree. Update is a PUT - this
API has no patch, no update mask and no long-running operation anywhere - so
omitting an optional field really does clear it.
PlatformPolicy is why the project-wide projects/{project}/policy singleton is
not modelled: it offers the same expressive power as a real, deletable
collection, where the singleton has only a get and a PUT and a "create" would
mean mutating live admission policy. The platform is a URL segment rather than
a field: v1 exposes only gke, and a field with exactly one legal value the API
never echoes back is an input-only field that reads as drift. ListItemsKey is
set because the response keys its array platformPolicies while the collection
segment is policies.
An API key is the first type here whose creation mints a secret, and the
create call is the only call that returns it: keyString arrives inline in the
completed operation and nowhere else. It is dropped on every read path rather
than merely expected to be absent, the same treatment
compute.vpnTunnels.sharedSecret gets, and getKeyString is never called from a
read. There is no field to declare opaque - the value has no authored
counterpart, only something to refuse to store.
Every key response reports name in project-number form while the target
declares a project id, so the native ID is rebuilt from context or the same
key would be managed under one identity and discovered under another. And
delete is a soft delete: the key stops working and drops out of listings, but
a read keeps answering 200 for thirty days with deleteTime set, so
ReadTreatAsMissing turns such a read into NotFound. Without it every sync
inside that month would put a deleted key back into inventory. That window
also reserves the id, so the case exercises update rather than replace.
* feat(cloudbuild): add BuildTrigger
Creating a trigger starts nothing, which was checked rather than assumed: the
project's builds collection was empty before the first probe trigger and still
empty after nine creations and seven patches. The fixture is doubly inert -
disabled, and with a sourceToBuild URL as its only source, so no webhook, no
Pub/Sub and no repo connection exists that could fire it.
Three behaviours compensated for. The trigger methods are synchronous: create
and patch answer with the trigger and delete with an empty body, unlike this
API's own builds.create which answers with an Operation. It authorizes a PATCH
against the request body's resourceName rather than the URL's, so a replayed
one turns an in-project update into a 403 on another project's path, and that
field is stripped from requests. And it silently drops false booleans, so
disabled:false comes back absent and an approvalConfig of
{approvalRequired:false} comes back as {}; both are restored on read.
A project with no legacy Cloud Build service account - every project created
in recent years - rejects a trigger that names none with a bare HTTP 400
naming no field. Five different source forms and gcloud's own "builds triggers
create manual" all fail identically, and the cause is the missing
serviceAccount. That opaque 400 is the biggest trap in this API and is
documented in the schema.
WorkerPool is absent: a private pool holds provisioned machines billed per
hour. The v2 Connection and Repository types need an external OAuth account.
* feat(bigquery): add RowAccessPolicy, and let Table.datasetId take a resolvable
Row-level access control on a single table: a SQL boolean predicate deciding
which rows a principal may see, so one shared table can serve each tenant only
its own rows instead of a view per tenant. This was dropped from the previous
batch on a missing bigquery.datasets.create grant, which has since landed.
Its identity is not a name but the composite rowAccessPolicyReference
{projectId,datasetId,tableId,policyId}, which every request body must carry
and which the API checks against the URL segment by segment - "Project ID in
URL path and content do not match." So the four flat properties a forma
declares are assembled on the way out and flattened back on the way in.
Update is a PUT with no field mask, and sending one is not ignored: the API
answers `Unknown name "updateMask": Cannot bind query parameter`. A delete
that would leave the table with no policy is refused outright unless
force=true is sent, so the type always sends it - without that flag every
teardown of a single-policy table fails.
grantees is accepted on insert and returned by neither get nor list, so it is
not declared: an input-only field reads as drift on every sync. It is readable
only through a separate getIamPolicy call, and a PUT that omits it clears the
roles/bigquery.filteredDataViewer binding it created. Both facts are recorded
at the field, because together they mean a formae-managed policy grants row
access to nobody until someone binds that role out of band.
Policies hang off a table and BigQuery accepts no wildcard for the dataset or
table segment - datasets/-/tables/- answers 404 blaming the dataset - so
discovery walks datasets, then tables, then policies rather than reporting an
empty inventory.
Table.datasetId was typed plain String, so dataset.res.datasetId failed
evaluation and a table could only name a dataset some other forma had already
created. The sibling Routine already had this right.
* fix: KMS key rings can be destroyed, and two Compute proxies patched wrong
Three defects that a missing conformance case had been hiding, plus the case.
cloudkms keyRings.delete did not exist when GCP::KMS::KeyRing was written, and
the plugin said so in a comment, a doc comment and a unit test - so a forma
could create a key ring and never reclaim it, and the fixture was dropped for
exactly that reason. The method exists now: it answers with an already-
finished Operation, the ring 404s on the very next GET, and the deleted id can
be re-used. The case is back with a -replace companion, since there is still
no keyRings.patch and the id is the only declarable field. Nine key rings
leaked by the nightly before the case was dropped have been reclaimed.
regionTargetHttpsProxies.patch enforces the fingerprint rather than treating
it as advisory - a PATCH without one is refused with "Required field
'resource.fingerprint' not specified" - so optimistic locking here is not an
optimisation, it is the only way an update lands at all, and it was off. Every
update of GCP::Compute::RegionTargetHttpsProxy failed with a 400. tlsEarlyData
also gains hasProviderDefault: the API reports DISABLED back on every read
whether or not it was sent, exactly as the global sibling already documents.
regionTargetHttpProxies has no patch method at all - delete, get, insert, list
and setUrlMap only - but GCP::Compute::RegionTargetHttpProxy claimed
SupportsUpdate, so a PATCH landed on a URL the API does not serve and came
back as Google's HTML 404 page rather than an API error. Its global sibling
does have patch, which is how the regional one came to claim it. A change now
replaces. The region-http-lb case, which uses this type and has no -update
companion, is what let it hide; it still passes.
EssentialContacts::Contact.email is marked createOnly: the API refuses a
masked change to it and the field was annotated as though it were mutable.
Inert today because the type does not support update, but it stops the trap.
* chore: register eight new packages, document the batch, sweep what it creates
Blank imports for apikeys, binaryauthorization, cloudbuild, clouddeploy,
dataform, firestore, networkservices and parametermanager; README rows and
CHANGELOG entries for the 24 new types; a disposition row for each new
hasProviderDefault annotation.
Eight new sweep sections, all REST because none of these collections has a
usable gcloud surface, and all eight sharing one helper rather than eight
near-identical copies - the copies are exactly how the earlier sweeps drifted
apart. Every list URL was dry-run against the live API before being wired in.
Ordering is encoded in the order of the lists, not left to chance: a Cloud
Deploy pipeline refuses to delete while it has an automation, a Dataform
repository while it has workspaces or configs, a Parameter Manager parameter
while it has versions and its delete takes no force flag, and a Network
Services mesh goes after the routes attached to it. Nested collections are
reached through each API's own "-" wildcard parent.
API keys get half a sweep, and it says so. A key delete is a soft delete: the
key stops working and drops out of listings, but a read keeps answering 200
for thirty days and the v2 API has no purge, only undelete. So a tombstoned
key cannot be collected by anything and every conformance run leaves one
behind until it self-purges - free and non-functional, but recorded so nobody
goes looking for the sweep that is missing. What is collected is a live key a
run created and failed to delete, which is the case that would otherwise leave
a working credential lying around.
* fix(testdata): alias the two renamed schema modules explicitly
verify-schema rejects duplicate basenames across services, so this batch's
dataform/repository.pkl and firestore/database.pkl were renamed to
repository_dataform.pkl and database_firestore.pkl - the convention
instance_spanner.pkl and table_bigquery.pkl already follow.
Pkl derives a module's identifier from its filename when the import carries no
alias, so renaming the file silently renamed the identifier and every
`repository.Repository` / `database.Database` reference stopped resolving:
"Cannot find module import `database`". Both now carry an explicit `as` alias,
exactly as testdata/spanner-database-replace.pkl already does.
Neither `pkl eval formae-plugin.pkl` nor verify-schema evaluates a fixture, so
both gates passed while all ten of these cases failed at conformance step 1.
Evaluating every fixture directly is what catches this.
* fix(firestore): Database was undiscoverable, and its update raced its create
Two defects that only a conformance run surfaces, plus the base hook one of
them needed.
databases.list takes no pagination parameters at all. It returns every
database in the project in one response and refuses the ones it does not know:
"?pageSize=100" answers 400 "page_size is not supported." and "?maxResults=100"
answers 400 Unknown name "maxResults". The whole request is refused rather than
the parameter ignored, so the registration's PageSizeParam turned every List
into an error instead of a list - the type was undiscoverable while create,
read, update and delete all worked. Pagination is now disabled for this API.
Firestore also answers a patch issued while a create is still settling with 409
ABORTED, "There are concurrent database changes, please try again." That is
deterministic rather than occasional - a create followed immediately by a patch
reproduces it every time - and it is the exact shape of a reconcile, so the
conformance update step failed on a database that was perfectly healthy.
base already mapped a retryable error to NotStabilized so formae core re-runs
the operation, but only on the delete path, where Cloud SQL needed it. The same
check now guards both update paths, and Firestore classifies the 409. This is
not Firestore-specific: Cloud Deploy answers a concurrent mutation with 409
ABORTED too, and any API that asks for a retry was previously reported as a
terminal failure.
naxty
changed the title
# Conflicts: # README.md
gcloud bigtable instances delete "$instance" --quiet 2>/dev/null || true stderr to /dev/null, exit code swallowed by "|| true". A delete refused for lack of a permission printed "Deleting Bigtable instance: x" and did nothing - output identical to a delete that worked. Four instances survived a sweep that way and billed per node-hour until someone looked at them; the REST DELETE removed all four immediately, which is how the swallowed failure was found. Now over REST with the status checked, the same treatment and for the same reason as the Cloud SQL section directly below it, which already switched away from gcloud after twenty-one instances accumulated behind a delete that always failed. A failure now says so and says the instance is still billing. Bigtable is the expensive one to get wrong: an instance holds nodes and is charged per node-hour whether or not anything reads from it.
…lds the ref
networkconnectivity-spoke was the one case failing in CI, and it failed
deterministically there while passing every local run. The isolated Debug
Conformance run gave the reason, which is not an API error at all:
linkedVpcNetwork.uri has no resolved $value yet (after extract)
linkedVpcNetwork.uri resolved to: https://.../networks/formae-test-nc-spk-net-...
ERR resolving references at execution time changed createOnly fields
[linkedVpcNetwork] on plugin-sdk-test-nc-spoke; a replacement was not
planned, refusing to proceed
The update phase applies in patch mode, so the network the spoke points at is
not in that changeset and its resolvable is still unresolved when the change is
planned; it resolves only at execution time. `createOnly` sat on
`linkedVpcNetwork`, the wrapper object, while the resolvable is on the nested
`uri` - so formae compared the whole object across that resolution, saw a
createOnly field go from unresolved to resolved, and refused.
The hint now sits on `LinkedVpcNetwork.uri`, the scalar that actually carries
the reference, so the comparison is between the resolved value and the stored
one - the same string. That is the shape
`GCP::NetworkConnectivity::InternalRange.network` already had, which is why
that type never hit this. Same immutability, declared at the level where it is
about a value rather than about an object.
It passes locally either way, so it is verified in CI rather than here.
… that holds the ref" This reverts commit 94cc908.
…level field
networkconnectivity-spoke was the last case failing in CI. It failed there
every run and passed here every run, and the reason was never an API error:
linkedVpcNetwork.uri has no resolved $value yet (after extract)
linkedVpcNetwork.uri resolved to: https://.../networks/formae-test-nc-spk-net-...
ERR resolving references at execution time changed createOnly fields
[linkedVpcNetwork]; a replacement was not planned, refusing to proceed
The update phase applies in patch mode, so the network the spoke points at is
not in that changeset and its reference is still unresolved when the change is
planned - it resolves at execution time. formae then checks whether resolving
changed a createOnly field. A property declared `String|formae.Resolvable` is
exempt, because its type says the value is a reference and is expected to
materialise. An object that merely contains one carries no such signal, so the
whole `linkedVpcNetwork` was read as an immutable field mutating, and every
update was refused. Moving the annotation onto the nested `uri` did not help:
the check reports and reasons at the top-level property wherever the hint sits.
So the reference is now a top-level `network: String|formae.Resolvable`, with
`includeExportRanges` and `excludeExportRanges` alongside it, and the
transformers assemble the API's `linkedVpcNetwork{uri,...}` on the way out and
unpack it on the way in. That is the shape `hub` and
`GCP::NetworkConnectivity::InternalRange.network` already had - the two
references in this service that always worked - and it keeps both the
immutability and the dependency edge on the network.
The alternative was one line: drop createOnly and let the API refuse a network
change. Probing killed it. NCC rejects the update *mask*, not the value -
400 changing field "linked_vpc_network" is not allowed: invalid argument
- with a body byte-identical to what was stored, so sending the field would
break every update, and dropping it from the body instead would leave a network
change never reaching the API and drifting forever.
All three fields are createOnly: they are one immutable object on the wire.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR now carries two batches. #198 was squash-merged into this branch (
504a649), so thediff against
mainis 37 new resource types across thirteen services, seven of them entirely newto the plugin, plus five defect fixes on types that already shipped and two additions to
base.The plugin goes from 206 to 243 resource types.
What's here
Network Services is the direct consumer of the Network Security policies in the same PR: an
EndpointPolicyreferencesClientTlsPolicy,ServerTlsPolicyandAuthorizationPolicy.Verification
Every candidate was probed against the live API before any code was written — list, create, poll
the operation to done, GET, PATCH, DELETE — and everything that survived was then run through
conformance. Locally that is 38 cases, 76 runs, all green, plus
region-http-lbas a regressioncheck. CI ran 25 of the new cases on the second batch's head as
github-deploy@and 23 hadcompleted green when it merged.
Read CI on this head, not the local results, as the gate.
Five defects fixed in types that already shipped
Each was found by writing a missing conformance case, which is the point of writing them:
GCP::KMS::KeyRingcould be created and never destroyed.keyRings.deletedid not exist whenthe type was written; the plugin said so in a comment, a doc comment and a unit test, and the
fixture had been dropped for that reason. The method exists now. Nine key rings leaked by the
nightly before the case was dropped have been reclaimed.
GCP::Compute::RegionTargetHttpsProxycould never be updated — that collection enforces thepatch fingerprint and optimistic locking was off, so every update failed with a 400.
GCP::Compute::RegionTargetHttpProxyclaimed a patch method that does not exist — the requestcame back as Google's HTML 404 page.
GCP::BigQuery::Table.datasetIdrejected a resolvable, so a table could only name a datasetsome other forma had already created.
GCP::EssentialContacts::Contact.emailwas annotated mutable; the API refuses a masked change.Two
baseadditions, each driven by a live failureResourceConfig.ReadTreatAsMissing— for an API that keeps serving a tombstone after a 200delete. Memorystore's
aclPolicies.deleteanswers 200 and the policy reads back for another 15–20seconds with
state: DELETING; a sync in that window put a deleted policy back into inventory.OperationConfig.RetryableErrornow guards the update path, not only delete. Firestore answersa patch issued while a create is still settling with 409
ABORTED, "please try again" —deterministic, and the exact shape of a reconcile. Cloud Deploy returns 409
ABORTEDonconcurrent mutations too, so this is a class rather than one API.
Behaviours the probes caught
databases.listtakes no pagination parameters and refuses theones it does not know —
?pageSize=100→ 400page_size is not supported.Every List returned anerror instead of a list while create, read, update and delete all worked.
answers 403
PERMISSION_DENIED, "Read access to project was denied" — a wrong-host error wearing amissing-IAM-grant's clothes. Registered global-only, with what
basewould need written down.invocationConfig,codeCompilationConfigorkmsKeyNameeven when the object is byte-identical,and calls all three plain "Optional" in its own discovery document.
carrying an error, and the pipeline never existed.
grantees(and a PUT omitting it clears the IAM binding it created), Cloud Build'sincludedFiles,Firestore's
tags.Secrets
Two types carry credentials and neither puts one in state.
ApiKeys::Key'skeyStringarrivesinline in the create response and nowhere else — dropped on every read path, and
getKeyStringisnever called from a read.
ParameterManager::ParameterVersion's payload is write-only and strippedon read; unlike Secret Manager, that API hands the payload straight back on a plain GET.
Sweeps
Fourteen new sections. Five close pre-existing gaps: nothing swept the Network Security API at
all though four of its types already shipped, and nothing swept backend buckets in either scope. The
sweep logic was run live and observed collecting the real leftovers a conformance run leaves —
the prerequisites
Destroyspares — with child-before-parent ordering working.API keys get half a sweep, and it says so. A key delete is a soft delete: the key stops working
and drops out of listings, but a read answers 200 for thirty days and the v2 API has no purge, only
undelete. A tombstoned key cannot be collected by anything, so every run leaves one behind until it
self-purges — free and non-functional, but recorded so nobody hunts for the missing sweep.
Spanner needed its own pattern: a user-managed instance configuration's id must begin
custom-, andSWEEP_REis anchored at the start of the name.Dropped after probing — 29 candidates
Billable prerequisite:
tlsInspectionPolicies(needs an enabled CA),instantSnapshotGroups×2,transports,gatewayAdvertisedRoutes, Cloud BuildWorkerPool, FirestoreBackupSchedule,Network Services
gatewaysand the*Extensions. Needs a real external target:authzPolicies,serviceConnectionMaps,pscAuthorizationPolicies, Cloud Build's v2Connection/Repository,ArtifactRegistry
tags. Not a desired state: Cloud Deploy releases/rollouts/jobRuns, DataformworkflowInvocations/compilationResults. Structurally unmodellable: Binary Authorization's
projects/{p}/policysingleton, FirestoreIndex(server-assigned name),ServiceBinding(deprecated field). Feature-gated or region-gated:
regionHealthCheckServices, Eventarcchannels.Broken delete:
globalVmExtensionPolicies— its LRO reports DONE while the policy still exists.Already shipping under another name:
apigateway/configs,monitoring/serviceLevelObjectives.Of the 15 shipped types with no conformance case, one is now covered and the rest were each
confirmed blocked with a reason. Two need only a decision:
EssentialContacts::Contactneeds anallowed contact domain, and
GKEHub::Featureneedsgkehub.googleapis.comenabled plusroles/gkehub.admin.Gates
go build ./...,golangci-lint run ./...(0 issues),make test-unit(0 failures, 50 packages),make verify-schema(243 types, 0 duplicates),pkl eval formae-plugin.pkl,bash -non everytracked
.sh,make lint-reuse(930/930), and every new or changed fixture evaluates — that lastone added after a schema rename silently broke ten fixtures while both pkl gates passed, because
neither evaluates a fixture.