feat: wave 3 — 100 new Azure resource types - #152
Merged
Conversation
`pre-cleanup` has failed on main since #145 with a single line: clean-environment.sh failed at line 189 (exit 1): GROUPS=$(list_groups) and because `pre-cleanup` gates `conformance-tests` via `needs`, no fixture could be verified on any PR while it was red. Line 189 was never the problem. An ERR trap is not inherited by shell functions unless `set -E` is on, so a failure anywhere inside `list_groups` was reported against the *caller* - the command substitution - rather than against the command that actually failed. Three fixes (#146, #147, #149) were aimed at line 189 on the strength of that message. Two changes: - `set -eEuo pipefail`. With errtrace the trap fires at the inner line and names the real command, so the next occurrence is a one-line diagnosis. - `list_groups` now runs its whole body with `set -e` off, not just the `az group list` call. Its status is consumed by `GROUPS=$(list_groups)`, where a non-zero exit is fatal; under `set -e` any failing command in the body aborted the function before its `return 0` could run, so the substitution exited non-zero and killed the script. Listing groups is read-only, so nothing in there needs to be fatal. The ERR trap is also disarmed for that body: a tolerated failure is not an error, and letting it reach the trap prints a GitHub `::error::` annotation on a run that then succeeds. The existing `note:` line already reports it. Verified with a stubbed `az` that writes valid JSON and exits non-zero - the condition a CLEAN_DEBUG trace caught in CI. Before: the caller died. After: the caller survives with the group list intact, and the trap names `az group list` instead of line 189.
Phase 0 of the 100-resource wave. Every resource type needs a client field,
a constructor and a struct-literal entry, and `pkg/client/client.go` is a
single file that all twelve implementation groups would otherwise touch at
once. Landing all of it up front means no group has to edit a shared file,
so the groups cannot conflict with each other.
Six new SDK modules, all pinned:
datafactory/armdatafactory v1.3.0
automation/armautomation v1.0.0
streamanalytics/armstreamanalytics v1.2.0
logic/armlogic v1.2.0
desktopvirtualization/armdesktopvirtualization v1.0.0
apimanagement/armapimanagement v1.1.1
The remaining clients come from modules already required: armoperationalinsights
(Log Analytics depth), armstorage (storage depth), armdataprotection/v3 and
armnetwork/v4 (network manager).
Every one of the 78 constructors was verified by compile probe against a
throwaway module before being written here, rather than inferred from the
service name. The constructor list itself was enumerated from each module's
own source in the module cache, so there were no guesses left to be wrong -
the previous wave got 5 of 17 wrong in one pass, including two modules that
had removed the client entirely.
Network Manager's clients are named generically in armnetwork
(`GroupsClient`, `StaticMembersClient`), so their fields carry a
NetworkManager prefix to say what they actually address.
Adds schema/pkl/automation/ with the account plus its six children: runbook, schedule, job schedule, variable, credential and module. Every top-level field carries an @Azure.FieldHint - an unannotated field is silently dropped from the generated property set, and build, unit tests and verify-schema all stay green while live create fails with the handler's own "X is required". No hasProviderDefault annotation is introduced anywhere in this namespace. A field ARM always answers Get with is declared required instead (the account's skuName, the runbook's two log flags, the schedule's timeZone and isEnabled, the variable's isEncrypted), so it is always present in desired state and always comparable. Fields ARM only echoes when they were supplied stay optional and are read back only when present. Write-only where ARM cannot return the value: the credential password (the response model has no password field at all), the variable value (never returned when encrypted, and a runbook is a legitimate co-writer when not), the module contentLink and the runbook publishContentLink (neither is returned by Get). The advancedSchedule block is deliberately not modelled: it is a plain nested class whose members ARM fills in from the frequency, and hasProviderDefault is a no-op inside one, so those service-chosen members would read back as unexplained drift.
New namespace directory schema/pkl/streamanalytics/ covering the streaming job, its three stream inputs (blob, Event Hub, IoT Hub), its four outputs (blob, Event Hub, Service Bus queue, Azure Table) and a JavaScript UDF. Two shapes worth calling out: - Every datasource credential (storage account key, Event Hubs / Service Bus / IoT Hub shared access policy key) is writeOnly. ARM accepts them on PUT and PATCH but strips them from every response, so a readable field would report drift on every sync. - ARM's polymorphic datasource discriminator is flattened away: each formae type pins one datasource, so its fields sit at the top level of the schema where FieldHint semantics (required/writeOnly) actually apply, rather than inside a nested class. The streaming job models no execution state. Start/Stop is a separate ARM verb pair, a running job cannot be reconciled into a stopped one by a PATCH, and a running job bills streaming units per hour.
… services Adds nine AZURE::DataFactory::* resource types (schemas + provisioners + mock-based integration tests): Factory, IntegrationRuntimeSelfHosted, IntegrationRuntimeAzure, Pipeline, LinkedServiceAzureBlobStorage, LinkedServiceAzureSqlDatabase, LinkedServiceKeyVault, LinkedServiceWeb, LinkedServiceAzureTableStorage Every armdatafactory verb these types use is synchronous — FactoriesClient, LinkedServicesClient and PipelinesClient have no BeginX at all, and the only pollers on IntegrationRuntimesClient are BeginStart/BeginStop for Azure-SSIS — so no resource here creates a poller and Status always echoes success. The five linked services share one envelope: datafactorylinkedservice.go carries the LinkedServiceResource wrapper, the read-back mapping and the CRUD, and each type file contributes only its ARM discriminator plus its connector fields. Precedent: cosmoschild.go, servicebusprops.go. Credentials are always sent as SecureString and declared writeOnly in the schema: ARM returns only a mask, so comparing one on read would report drift forever.
AZURE::Automation::Account, ::Runbook, ::Schedule, ::JobSchedule, ::Variable, ::Credential and ::Module, against armautomation v1.0.0 (api-version 2024-10-23). Every client in this SDK module is named in the singular - AccountClient, not AccountsClient - and none of them is LRO-based: there is no BeginX anywhere except RunbookClient.BeginPublish, which this code deliberately does not drive. So every Status implementation here is the echo form, and create, update and delete all complete inside their own call. JobSchedule is immutable: armautomation offers Create, Get and Delete and no update verb, so every field is createOnly and Update refuses with the cause in StatusMessage rather than a bare transition to Failed - the shape roleassignment.go uses. Every failure path in all seven sets StatusMessage. Read paths drop service-managed state that would otherwise report drift on every sync: the schedule's nextRun and the four *OffsetMinutes, the runbook's jobCount / state / provisioningState, the module's provisioningState and everything else derived from the package ARM downloaded, and the timestamps on all of them. Two service-normalisation traps are handled explicitly: the schedule's year-9999 no-expiry sentinel is treated as unset, and its times are re-emitted as RFC3339 in UTC so an offset-formatted response compares equal. Write-only values are read with the existing opaqueString helper rather than a plain string field, so an opaque property that arrives as a wrapper object rather than plaintext cannot fail the whole json.Unmarshal. Shared helpers live in automationaccount.go (the parent) rather than in a new file: automationChildIDParts parses a child's ARM ID for all six children, and automationChildNativeID builds the ID used as a create fallback.
Adds the PKL schemas and provisioners for the wave-3 Storage depth group: - AZURE::Storage::LocalUser - AZURE::Storage::BlobInventoryPolicy - AZURE::Storage::ObjectReplicationPolicy - AZURE::Storage::BlobContainerImmutabilityPolicy - AZURE::Storage::BlobContainerLegalHold - AZURE::Storage::QueueServiceProperties Two of these are verb-shaped rather than resource-shaped: the immutability policy is written through createOrUpdateImmutabilityPolicy with an If-Match ETag the provisioner reads for itself, and the legal hold has only setLegalHold/clearLegalHold with no Get, so its tag set is read off the parent container and its identifier is synthesised from the container's ARM ID. ObjectReplicationPolicy models ARM's two-sided flow honestly: the destination copy is written first under the name `default`, which mints the policy and rule ids, and the same document is then echoed onto the source account. No hasProviderDefault annotation is introduced anywhere: every optional field either carries a schema default and is always sent and always read back, or is omitted from the read-back unless the service actually reports it. Integration tests for LocalUser and BlobInventoryPolicy land with this commit; the remaining four follow.
streamanalyticsio.go is the one shared helper: ARM models every input as a single streamingjobs/<job>/inputs/<name> resource and every output as a single .../outputs/<name> resource, with the interesting part behind a properties.datasource.type discriminator. Formae splits those into one type per datasource, which leaves seven resources with an identical envelope and seven different datasource bodies. The envelope lives in the helper; each resource file contributes only its datasource builder and serializer, in the same shape as cosmoschild.go and servicebusprops.go. Two rules the datasource contributions respect: - No serializer ever emits a credential. ARM strips AccountKey and SharedAccessPolicyKey from every response, so echoing one back would put a field in state with no ARM value behind it and report drift every phase. - List filters on the discriminator. NewListByStreamingJobPager returns every input (or output) of the job whatever its datasource, so discovery for InputBlob would otherwise claim the job's Event Hub inputs too. The same applies to the JavaScript UDF, which shares its ARM collection with aggregate functions and ML-web-service bindings. The job's Read asks for $expand=transformation: ARM's default GET projection excludes it, and there is no wired TransformationsClient, so the query is a createOnly field of the job rather than its own resource type. Create and delete are LROs; update is a synchronous PATCH that deliberately omits the transformation, which ARM rejects on PATCH. Every failure path sets ProgressResult.StatusMessage.
One base fixture per type plus an -update variant, except the job schedule,
which is immutable and therefore gets -replace: ARM has no update verb, so a
change to any of its fields is a delete-and-recreate.
Every fixture creates its own resource group and Free-tier automation account.
Free includes 500 job minutes a month at no cost and none of these fixtures
runs a job; the tier is quota-limited per subscription, so the account has to
go with the resource group.
Two fixtures reach outside Azure, deliberately and with pinned targets:
- automation-module uses a versioned PowerShell Gallery nupkg
(xActiveDirectory 2.19.0, updating to 2.20.0) because ARM fetches the
package server-side and the import is what the resource does. Both URLs
were verified live.
- automation-job-schedule publishes its runbook from the canonical Azure
quickstart tutorial script, because a job schedule can only attach a
PUBLISHED runbook - a runbook created with an empty draft has nothing to
run. automation-runbook itself omits the content link and uses the empty
draft, so the runbook record is exercised with no outbound fetch at all.
The schedule fixtures use a far-future whole-minute startTime: ARM rejects a
start under five minutes out and truncates whatever it accepts to the minute,
and Pkl has no clock to compute one from.
The job schedule passes no runbook parameters: ARM validates them against the
runbook's declared parameters and the tutorial script declares none.
All fourteen were checked with pkl eval against a local project pinning
@Azure to this tree.
One base plus one -update fixture per type. Each is standalone: it builds its own resource group, the parents its datasource needs (storage account plus blob container or table, Event Hubs namespace plus hub, IoT hub, Service Bus namespace plus queue) and a stopped streaming job. Datasource credentials are placeholders, not real keys. ARM accepts any string there and never returns it, Stream Analytics validates a datasource only when the job starts or when the separate Test operation is called, and none of these jobs is ever started. There is also no way to obtain a real key from a forma: this plugin deliberately does not surface storage account keys or SAS policy keys as properties. Each update fixture changes exactly one in-place-patchable field, never a createOnly one: the job widens its late-arrival window, the blob input and output move their path prefix, the Event Hub and IoT Hub inputs and the Event Hub output repartition on a different column, the Service Bus queue output attaches a second message property, the table output doubles its batch size, and the UDF changes its function body.
…rvices One provisioner serves all five connectors, so the shared CRUD is exercised once through the Key Vault kind and each of the other four gets its own build/read round trip. Asserts what must NOT come back as much as what must: every credential is wrapped in SecureString and absent from the read, and a factory's mixed linked-service pager is filtered by discriminator so one connector never claims another's IDs.
…and runbook
Assert the things a mocked test can actually prove and a live conformance run
would otherwise be the first to notice:
- no operation mints a resume token, because armautomation has no LRO;
- the read path drops every service-managed field (account state, runbook
jobCount / state / provisioningState / logActivityTrace) and folds ARM's
"East US" back to the compact region form desired state carries;
- an omitted publishContentLink becomes an empty draft, since ARM rejects a
runbook carrying neither that nor a draft, and a supplied one suppresses
the draft;
- a write-only property that arrives as the opaque wrapper object rather
than unwrapped plaintext still yields its value;
- the account PATCH does not restate the createOnly location;
- a provider error reaches StatusMessage rather than being dropped.
Committed by the orchestrator, not the implementing agent. The agent stopped mid-task when the session hit its rate limit; this preserves the work in progress so it is not lost, and is expected to be amended or built on when the agent resumes. Not reviewed and not verified - the six-command gate has not been run against this state.
Committed by the orchestrator, not the implementing agent. The agent stopped mid-task when the session hit its rate limit; this preserves the work in progress so it is not lost, and is expected to be amended or built on when the agent resumes. Not reviewed and not verified - the six-command gate has not been run against this state.
Committed by the orchestrator, not the implementing agent. The agent stopped mid-task when the session hit its rate limit; this preserves the work in progress so it is not lost, and is expected to be amended or built on when the agent resumes. Not reviewed and not verified - the six-command gate has not been run against this state.
Committed by the orchestrator, not the implementing agent. The agent stopped mid-task when the session hit its rate limit; this preserves the work in progress so it is not lost, and is expected to be amended or built on when the agent resumes. Not reviewed and not verified - the six-command gate has not been run against this state.
Committed by the orchestrator, not the implementing agent. The agent stopped mid-task when the session hit its rate limit; this preserves the work in progress so it is not lost, and is expected to be amended or built on when the agent resumes. Not reviewed and not verified - the six-command gate has not been run against this state.
Committed by the orchestrator, not the implementing agent. The agent stopped mid-task when the session hit its rate limit; this preserves the work in progress so it is not lost, and is expected to be amended or built on when the agent resumes. Not reviewed and not verified - the six-command gate has not been run against this state.
…ve3-g01-datafactory-core
…ve3-g06-streamanalytics
…ve3-g04-loganalytics-depth
…ve3-g03-automation
…ve3-g11-apim-core
…ve3-g05-storage-depth
…lasses PR #144 changed the rule: schema extraction walks only nested classes that formally extend formae.SubResource, so a field hint on a plain nested class is silently inert. Identity and RepoConfiguration now extend it and carry real hints, which makes `required` on their fields actually enforced. identityPrincipalId and identityTenantId stay flattened to the top level. That is no longer about the nested-provider-default bug — it is because a formae.Resolvable addresses a top-level property by name, and a role assignment granting the factory's managed identity a data role cannot reach a value nested one level down.
PR #144 changed the rule: schema extraction walks only nested classes that formally extend formae.SubResource, so a class without the clause has every field hint on it silently dropped. TestNoOrphanFieldHintClasses now guards it. Covers all nine nested classes in the namespace: StreamingJobSKU and JobTransformation on the job, the per-type serialization blocks on the three inputs and three of the four outputs, and FunctionArgument on the UDF. None of them carries a nullable collection, so the SubResource render path's absent / explicit-null / explicit-empty handling changes nothing here.
AZURE::OperationalInsights::LinkedService was the one type in this group without a -update.pkl. The update re-asserts the same link rather than changing one. ARM's linkedServices body carries nothing but the two target IDs, and `name` (the link kind) is the resource's own path segment, so the only mutable property is the link target - repointing it would mean provisioning a SECOND Log Analytics dedicated cluster: another 500 GB/day capacity reservation and another 1-3 hour wait. What the fixture still exercises is apply-idempotency: a second apply of an unchanged link must neither replace it nor report drift. Both linked-service fixtures are expected to land on .github/conformance-pr-skip.txt for the same reason the cluster fixture does.
…ule and variable
The three types whose read path carries the most service normalisation:
- Schedule: ARM answers with an offset-formatted startTime, a year-9999
no-expiry sentinel rather than a null, and interval as an untyped JSON
number. Asserts the start time comes back as the same instant in the Z form
desired state carries, the sentinel is dropped while a real expiry is kept,
and nextRun plus the four derived *OffsetMinutes never reach the read.
automationScheduleInterval is table-tested across every numeric form the
SDK's untyped interval field can hold.
- JobSchedule: asserts Update REFUSES with the cause on StatusMessage, since
ARM has no update verb and a silent no-op would let core believe a change
landed. Also that the name reaches the URL segment as the jobScheduleId,
that the read falls back to properties.jobScheduleId (ARM leaves the
envelope name unset on this type) without emitting it as a second
property, and that a 404 on delete is success - a replace deletes first.
- Variable: the load-bearing assertion is that the read NEVER emits the
value, including the unencrypted case where ARM does return it. An
encrypted variable never returns one, and a runbook can assign to an
unencrypted one at runtime, so comparing it would report drift the plugin
cannot own.
All three also cover the wrapped-opaque write path and confirm no operation
mints a resume token.
Wave 2 (ed4839e) landed 8 nested classes carrying `@azure.FieldHint` without extending `formae.SubResource`, which makes `main` red on `TestNoOrphanFieldHintClasses` and - more importantly - makes those hints silently inert. Schema extraction only walks nested classes that formally extend `SubResource`, so `required`, `hasProviderDefault`, `createOnly` and `writeOnly` on any of them never reach `Schema.Hints`. alertsmanagement/alertprocessingruleactiongroup.pkl AlertProcessingRuleCondition alertsmanagement/alertprocessingrulesuppression.pkl AlertProcessingRuleSchedule alertsmanagement/prometheusrulegroup.pkl PrometheusRule insights/autoscalesetting.pkl AutoscaleProfile, MetricTrigger, ScaleAction, ScaleCapacity, ScaleRule Same fix as #154: both the `extends` clause and the `@azure.SubResourceHint { apiVersion = module.apiVersion }` annotation, because the clause alone makes a class unrenderable - `Fq.subresourceProps` dereferences a null `findSubResourceHint`. This is a `main` bug rather than a wave-3 one, and it is fixed here only because this PR merges `main` and cannot be green while inheriting it. It is worth cherry-picking to `main` on its own. Note this is the second time the same class of defect has reached `main` in two days: #144 did it to 86 classes, wave 2 to 8 more. `TestNoOrphanFieldHintClasses` catches it, but only once the schema is merged - the CI job that would have caught wave 2's copy is the one this PR is trying to get green. Full gate on the merged tree (wave 2 + wave 3, 292 registered types): make build ok go vet ./... ok make test-unit ok golangci-lint ok make verify-schema ok integration tests ok make verify-fixtures 593/593 REUSE ok conformance-scope 13/13
This was referenced Sep 3, 2026
# Conflicts: # schema/provider-default-dispositions.json
All 7 Automation fixtures failed live conformance. Two distinct symptoms, one
cause:
automation-account [Verify] skuName should match: expected Free, got Basic
automation-credential [CreateOOB] create aacred-test-account failed:
RESPONSE 409: Conflict
The Free tier is quota-limited per subscription, and every Automation fixture
creates its own account in eastus. The first few get Free; after that Azure
either refuses outright (409) or silently hands back Basic, which then reads as
drift on every phase - Verify, Extract, Sync and Update all failed on the same
mismatch.
Basic is the deterministic choice here. It has no per-account charge - it bills
per job minute and per watcher, and these fixtures run no jobs and create no
watchers - and it is not quota-limited, so parallel fixtures cannot starve each
other.
This is my error, not the implementing agent's: the group brief said "use Basic
only if Free rejects something, Free is quota-limited to a handful per
subscription but is what CI should use", so Free was the reasonable reading. The
schema is unchanged and still correct - `skuName` stays `required` rather than
provider-defaulted, precisely because ARM always answers Get with a sku block,
so an omitted value would read as drift either way.
14 fixtures changed (7 types x base + update/replace). Verified locally:
`make verify-fixtures` 593/593, `make test-unit` 0 failures.
Soft-deleted API Management services keep counting against the Consumption
per-subscription service quota until they are purged, so they do not merely hold
a name - they eventually stop new services being created at all.
Observed on 2026-09-03: the subscription had accumulated 31 soft-deleted APIM
services and wave-3 fixtures started failing on create with
RESPONSE 400: MaxConsumptionServicesPerSubscriptionExceeded
Every APIM fixture provisions its own Consumption service, so the quota is
reached quickly once deleted ones stop being reclaimed. Key Vaults have been
purged here from the beginning; APIM was simply missed.
Mirrors the Key Vault block, with two deliberate differences:
- **No `--no-wait`.** A queued vault purge is fine; a queued APIM purge does not
free quota in time to help the run that follows, and the call returns quickly
enough that waiting costs little.
- **The prefix is re-checked locally.** A purge is irreversible, so this only ever
acts on a name the script has verified itself, rather than trusting the
server-side `--query` projection - the same rule `list_groups` follows. This
was not theoretical: the first version of this loop trusted the query, and a
stub that ignored it purged a service the sweep had no business touching. The
test now asserts a non-matching service is skipped by name.
Verified against a stub `az`: two `fpsdt-` services purged, one third-party
service skipped with a reason, sweep still exits 0.
Trap 6 was living in the shared wrapper, which made it everyone's problem at
once.
`createFailure`, `updateFailure`, `deleteFailure` and `statusFailure` set
`ErrorCode` and **discarded `err.Error()`**. Every resource routes its Go-error
path through those four helpers, so an ARM refusal reached core as a bare code
with no text - which is precisely the "51 of the plugin's resources drop the
provider error" symptom, except centralised.
They now set `StatusMessage` from the error, with a regression test that fails if
any of the four stops doing it (verified by reintroducing the bug: the test
catches it and names the helper).
## And the reason is now logged
Even when a provisioner sets `StatusMessage` itself - as the Data Protection
policies do - nothing printed it. From CI a failed operation looks like:
[Create] Apply command should complete successfully: command reached terminal
state: Failed
and the agent log shows the transition to Failed with no cause. During wave 3's
first live run that cost roughly fifteen failures their diagnosis: the only way
to see an ARM error was to reproduce the fixture by hand, and for one class
(`MaxConsumptionServicesPerSubscriptionExceeded`) the error only became visible
because a different fixture happened to surface it through the harness's own
out-of-band create.
`logFailure` now logs operation, native ID, error code and reason at Error level
via `plugin.LoggerFromContext`. It is called from the wrapper rather than from
each provisioner deliberately: every resource is wrapped, so one call site covers
all of them and cannot be forgotten by the next resource added. It fires on the
pass-through path too, where a provisioner returns a failure `ProgressResult`
rather than a Go error - which is the case the four helpers never saw.
Verified: build, test-unit (0 failures), integration tests, verify-schema,
verify-fixtures 593/593, go vet, golangci-lint, REUSE all pass.
Two Logic fixtures gave the explicit ARM error:
RESPONSE 409: EnabledRegionalFreeSkuIntegrationAccountsQuotaExceeded
A subscription may hold exactly ONE Free integration account per region, and the
CRUD lifecycle creates a second, out-of-band account in the SAME region as the
one the forma declares. So a Free fixture needs two Free accounts in one region
and ARM refuses the second.
Group 7 already knew about the per-region limit and carefully spread its seven
account-bearing fixtures across seven regions to avoid colliding with each
other. That was correct as far as it went, and it is not enough: the collision is
*within* a single fixture, between the forma's account and the harness's
out-of-band one, so no amount of spreading helps.
`Basic` removes the constraint. It bills per hour rather than being free, but a
fixture holds an account for a few minutes, so the cost is negligible - the same
trade already made for Automation accounts in this wave.
14 fixtures changed. The schema's own COST guidance still recommends `Free` for
real use, which is right, but now records why conformance fixtures must not use
it, so the next person does not reintroduce the 409 by following the advice one
paragraph up.
Verified: verify-schema, verify-fixtures 593/593, test-unit 0 failures.
Two more failure classes from the 2026-09-03 live run, both addressed with the
mechanisms this repo already has rather than with code changes.
## Timeout arms (6 fixtures)
Five fixtures failed the DISCOVERY phase, not provisioning:
[Discover] resource not discovered: timeout after 5m0s
(7 discovery trigger attempt(s))
data-factory-data-flow, data-factory-linked-service-sql-database,
data-factory-linked-service-table-storage, and both log-analytics-datasource-*.
Worth stating what was ruled out before reaching for the timeout, because a
timeout arm is the easy answer and often the wrong one. Structurally identical
siblings passed: same `listParam` (byte-identical), same ARM type discriminator
(`AzureSqlDatabase`, `AzureTableStorage`, `MappingDataFlow` all correct), same
`discoverable` setting, and for the Data Factory three the same shared List
helper. The agent log shows the discovery cycle being paused and resumed
repeatedly for user changesets. So the resource is created and simply has not
been imported inside the 5 minute default. If they still time out at 15 min the
cause is not timing and the arm should come back out - that is recorded in the
script.
`api-management-service` gets 30/75: a Consumption instance provisions in ~3 min
but DELETE is far slower, and the lifecycle deletes twice (Destroy plus the
out-of-band phase). The job ran 10m23s and still failed on the OOB-delete wait.
Its 28 child fixtures stand up their own services and pass inside the default, so
only the parent's own lifecycle needs the wider budget.
## Two singletons skip-listed
storage-queue-service-properties and storage-blob-container-immutability-policy
fail only `[OOB Del]`, for the same structural reason log-analytics-table already
carries: a singleton whose Read keeps succeeding after an out-of-band delete can
never be seen to disappear. The queue-service singleton is named `default` and
has no create or delete at all - only SetServiceProperties, where "delete" means
reset-to-empty - and the immutability policy is verb-shaped with its state read
off a parent container that survives.
Every phase before OOB-delete passes for both. This is a shape mismatch between a
singleton and a lifecycle that assumes create/destroy symmetry, so no diff fixes
it.
conformance-scope self-check 13/13; verify-fixtures 593/593; test-unit clean.
…manager
Two more classes from the live run, one a real bug and one an environment limit.
## log-analytics-storage-insight-config: a genuine type bug
failed to parse resource properties: json: cannot unmarshal object into Go
struct field logAnalyticsStorageInsightConfigProps.storageAccountKey of type
string
A write-only secret is declared in a forma as `formae.value("...").opaque`, which
renders as an OBJECT, not a string - so a `string` field rejects it before any
ARM call is made. `app-service-certificate` already had the answer for its pfx
blob and password: keep the raw value and unwrap with `opaqueString`, which takes
either a bare string or the wrapper. The field is now `any` and goes through a
small `storageInsightAccountKey` helper.
The existing test never caught this because it passes a bare string, which is
exactly the shape that works. Added a table test covering all three forms - bare
string, `$value` wrapper, `value` wrapper - so the failing shape is the one under
test.
## network-manager: all 8 types cannot pass here
RESPONSE 400: BadRequest
"Cannot have two Network Managers applied to the same object overlapping
scope accesses. Scope id: /subscriptions/<sub>"
Azure permits ONE Network Manager per overlapping scope. Every fixture declares a
manager scoped to the subscription, and the lifecycle creates a second,
out-of-band manager at the same scope - so the fixtures conflict with each other
and with their own OOB manager. The second create at that scope always fails,
whatever the plugin does.
Management group scope is what would separate them, and it is unavailable on this
subscription - no principal holds a role above subscription scope, which is
precisely why group 10 was scoped to subscriptions only. The constraint that made
the group buildable is the same one that makes it untestable in parallel.
The code does work against ARM: `network-manager-static-member` got as far as
[Update] before failing. Handlers, schemas and mocked tests are complete for all
eight. The skip entries record what would let them run again - management-group
scope, or a harness that serialises fixtures declaring an exclusive scope.
Gate: build, test-unit, integration tests, verify-schema, verify-fixtures 593/593,
go vet, golangci-lint, REUSE, conformance-scope 13/13 - all pass.
I got the `--no-wait` call wrong in the previous commit and it showed up immediately. That version purged synchronously, reasoning that a queued purge would not free quota in time to help the run that follows. Measured on the next CI run: each synchronous purge takes ~1.3 min, so clearing a 16-service backlog held `pre-cleanup` for over 17 minutes with **zero resource groups to sweep** - the whole step was waiting on APIM. At steady state every run leaves ~28 soft-deleted services behind, which would have meant ~36 minutes of purging in every single sweep, on a job that otherwise finishes in seconds. The original worry does not survive the timings either: the conformance matrix takes hours, so a purge queued at the start of pre-cleanup completes long before the APIM fixtures need the quota back. The purge itself was working - the backlog went 16 -> 3 while that run's pre-cleanup was still going - so this is purely about not blocking on it. Message changed to "queued purge for N" so the log does not overstate what has finished. Held back from the in-flight run deliberately: pushing this would have cancelled a run that was minutes from starting 76 conformance fixtures, to save a purge that was nearly done. It goes out with the next push.
…ionMethods
Both diagnosed from the ARM error text, which is visible for the first time
because of the wrap.go logging fix - these two had failed silently for two runs.
## api-management-global-schema
ERROR CODE: ValidationError
"XML schemas without targetNamespace are not allowed."
The fixture's XSD declared only the XMLSchema namespace. Added a
`targetNamespace` (and matching default xmlns) so the document is a valid
namespaced schema.
## api-management-authorization-server
ERROR CODE: ValidationError
target: "authorizationMethods"
message: "Invalid value"
The fixture deliberately omitted the three method lists "so ARM applies its own
defaults", which the schema documents as GET. That is not what ARM does for an
implicit-grant server: it validates the field even when the request omits it.
Worth being precise about what was checked - the handler does the right thing
(`apimAuthorizationMethods` returns nil for an empty list, and the SDK omits
nil), so this is not a case of sending `[]` or `null`. The value simply has to be
stated, and the fixture now sets `["GET"]`.
The schema keeps `hasProviderDefault` on the field, which remains correct for
ordinary use where ARM does fill it in; only the fixture changes.
Both were previously invisible: for two runs these reported nothing beyond
"command reached terminal state: Failed", and the quota clearance did not fix
them - they still failed on a completely clean subscription, which is what ruled
the earlier quota hypothesis out for this pair.
verify-fixtures 593/593, test-unit clean.
The Free -> Basic change was necessary but not sufficient, and the new error
logging says exactly why:
"The operation could not be completed because it exceeded your quota for
Automation accounts in your subscription in the region."
The quota is on Automation ACCOUNTS PER REGION, independent of SKU - so moving
off Free fixed the drift that `automation-account` was failing on (it now passes)
and did nothing for the other six. Every fixture stood up its account in eastus,
and the CRUD lifecycle adds a second out-of-band account in the same region, so
seven fixtures wanted ~14 accounts in one region.
Each fixture now owns a region: eastus, eastus2, westus, westus2, centralus,
northeurope, westeurope - two accounts each, which fits. This is the same
approach group 7 took for Logic integration accounts, and it was the right
instinct; it just needed applying here too. Note it does NOT rescue Logic, where
the collision is between a fixture and its own out-of-band account within one
region - that needed the SKU change instead.
Also `timeZone = "UTC"` -> `"Etc/UTC"` in the two schedule fixtures. ARM
normalises the IANA name and answers Get with `Etc/UTC`, so the declared value
read as drift on Verify, Extract and Sync:
[Verify] Property timeZone should match expected value: expected UTC,
got Etc/UTC
verify-fixtures 593/593, test-unit clean.
…tion cap
RESPONSE 400: "Maximum of 20 Consumption SKU services are allowed per
Subscription <sub>."
21 APIM fixtures each stand up their own Consumption service, and the CRUD
lifecycle creates a second out-of-band one - roughly 42 service creations across
a run against a ceiling of 20 live, with deleted services still counting until
their soft-delete is purged. The suite as designed does not fit the
subscription, so these failures are capacity rather than correctness: the same
fixtures pass when they get a service and fail when they cannot, which is
exactly what the last two runs showed.
Six representative fixtures stay, chosen to keep one of each distinct shape
rather than the first six alphabetically:
api-management-service the parent's own lifecycle
api-management-api a first-level child
api-management-api-operation a second-level child
api-management-product the product family's parent
api-management-policy the policy-body XML round-trip
api-management-diagnostic the singleton-child shape
The 15 skipped all have handlers, schemas and mock-based integration tests, and
several were observed passing in earlier runs before the cap was hit - so this
costs live coverage, not confidence in the code. The skip block records the two
things that would bring them back: a raised Consumption-service quota, or
reworking the fixtures to share one long-lived service instead of each creating
its own.
Matrix goes from 76 to 61 fixtures. conformance-scope self-check 13/13.
Two fixes from this batch stay in and are worth keeping visible for whoever
raises the quota later: api-management-global-schema needed a targetNamespace on
its XSD, and api-management-authorization-server needed authorizationMethods
stated explicitly. Both are real fixes to real ARM validation errors; they are
skipped for capacity, not because they are unresolved.
…imeout arm
Three more, all from ARM error text the wrap.go logging finally surfaced.
## Data Protection: policies are immutable (4 fixtures)
"Update of existing policy is not supported."
All four backup policies passed Create, Verify, Extract and Sync and failed only
[Update]. `BackupPoliciesClient` has no PATCH, and its CreateOrUpdate turns out
to be create-only in practice: a PUT against an existing policy is rejected
rather than replacing it. Group 9 had reasonably assumed the PUT would replace
the rule tree.
`backupRules` and `retentionRules` are now `createOnly` alongside the three
identity fields, so a changed rule tree plans as a replace rather than an
in-place update, and the `-update.pkl` fixtures become `-replace.pkl`. That is
the same shape AZURE::Automation::JobSchedule already uses, and the immutability
is documented on each of the four schemas with the ARM message quoted.
## Logic agreement: micHashingAlgorithm
"The agreement 'conformance-as2-agreement' must have 'micHashingAlgorithm'
property set in 'mdnSettings'."
The fixture did set it - to "NotSpecified", which ARM does not count as set.
Changed both occurrences to "SHA2256".
## Reverting my own discovery timeout arm
The previous commit widened the discovery budget for three Data Factory fixtures
and two Log Analytics datasources, on the theory that 5 minutes was simply too
short. It was not:
[Discover] resource not discovered: timeout after 15m0s
(17 discovery trigger attempt(s))
The arm applied and the fixtures still failed, so timing was not the cause. I
wrote into that arm that it should come back out in exactly this case, so it is
out. These five remain failing and are NOT skip-listed - unlike every other skip
in this file, there is no environmental or SDK reason established, and hiding a
possible real discovery bug behind a skip entry would be worse than leaving it
red. One clue for whoever picks it up: the discovery log carries
"Resource group '...-adfdssql-rg-...' could not be found", so a List may be
running against a scope that no longer exists.
The api-management-service arm stays - its OOB-delete genuinely needed longer
than 10 minutes.
Gate: build, test-unit, integration, verify-schema, verify-fixtures 593/593,
conformance-scope 13/13, go vet, golangci-lint all pass.
WorkflowUnsupportedRecurrenceTriggerForResponseAction
The workflow with 'Response' action type should not have triggers with
'recurrence' property defined: 'tick'.
A `Response` action replies to whoever called the workflow, so ARM only allows
it under a trigger that has a caller - an HTTP Request trigger. Our definition
paired it with a Recurrence trigger, which has none.
Swapped the action for `Compose`, which is a data operation with no caller and
no external dependency. The Recurrence trigger stays, and the workflow is still
created `Disabled`, so it never fires and still costs nothing to hold. Using a
Request trigger instead would have worked too, but that publishes a callable
public endpoint for the length of every conformance run.
unable to access host pool with name 'fpsdt-avd-sp-hp-...', please make sure
that you have given the Azure Virtual Desktop service permissions to access
your resource
A scaling plan is evaluated by the Azure Virtual Desktop service, not by the
caller, so that service principal needs its own read access to the host pool it
scales - Microsoft's "Desktop Virtualization Power On Off Contributor" role, or
any role carrying hostpools/read plus sessionHosts write on the scope. The
conformance subscription has no such assignment.
Not a plugin defect: the fixture wires hostPoolArmPath from the host pool it
creates, and the host pool itself creates fine. It is a one-time grant on the
subscription. The skip entry says so, and says to un-skip once it exists.
The real cause of every "[Discover] timeout" in this branch, and it is not
timing. The discovery log for log-analytics-storage-insight-config says:
Discovery finished. The following resources have been discovered:
AZURE::OperationalInsights::StorageInsightConfig 1
Validation of required fields failed error="resource fpsdt-lasic-... of type
AZURE::OperationalInsights::StorageInsightConfig is missing required fields:
[storageAccountKey]"
Discovery found the resource and then threw it away. `writeOnly` says the
provider never returns the value; `required` makes core reject any resource that
lacks it. Together they make a resource permanently undiscoverable, and the
harness reports it as a bare `[Discover] resource not discovered: timeout after
5m0s` with nothing pointing at the cause - which is why my previous commit
mistook it for a slow List and widened the timeout instead.
Twenty wave-3 fields carried both hints. Every one of the 38 write-only fields
that predate this branch is optional and nullable, including
`datafactorylinkedserviceblobstorage.connectionString` - whose sql-database and
table-storage siblings in this branch are two of the failures. That is the
convention, and these twenty now follow it: `required` dropped, property made
nullable. Presence stays ARM's to enforce on the write; there is nothing for
core to check on a read that never returns the value.
This is expected to fix discovery for log-analytics-storage-insight-config, the
three Data Factory fixtures, all seven Stream Analytics inputs/outputs, the five
Logic integration-account children, automation-credential, automation-module and
logic-workflow. It also fixes container-registry-webhook, a wave-2 resource that
carried the same pair but has no conformance-matrix row, so nothing ever
exercised its discovery.
`TestNoFieldIsBothWriteOnlyAndRequired` walks schema/pkl and fails on the
combination. Verified it fails when the pair is reintroduced on
loganalyticsstorageinsightconfig and passes once removed. The SDK's
verify-schema cannot see this, and neither can any other gate.
Gate: verify-schema, verify-fixtures 593/593, test-unit, integration,
conformance-scope, go vet, golangci-lint, reuse lint all pass.
…he wrong form
Both Log Analytics datasource fixtures failed discovery. ARM's answer to our
ListByWorkspace call:
Must specify a valid kind filter. For example,
$filter=kind eq 'windowsPerformanceCounter'
We sent `kind='WindowsEvent'`. ARM rejects that spelling with a 400, so the
pager yielded nothing and the fixture reported the failure as a bare
`[Discover] resource not discovered: timeout after 15m0s`.
The wrong form came from the SDK's generated example file, which is built from a
swagger sample and never executed. The SDK's *live* test - the one that runs
against Azure - uses `kind eq 'WindowsEvent'`, and so does the REST reference.
Following the live test.
The reason this shipped green is worth naming: both mock tests asserted the
broken string.
expected: "kind='WindowsEvent'"
actual : "kind eq 'WindowsEvent'"
A unit test that pins whatever the code currently does cannot catch a contract
error with the service. Both assertions now pin `kind eq '...'`, with a comment
at the assertion saying what it guards so the next edit does not restore the
old form to make a test pass.
The filter value carries no `$filter=` prefix: the SDK does
`reqQP.Set("$filter", filter)`, so a prefix would encode a second one into the
query string.
Gate: build, test-unit, go vet, golangci-lint, reuse lint all pass.
…a flag I broke this myself two commits ago and the failure was silent. b596e46 added `--no-wait` to `az apim deletedservice purge` to stop pre-cleanup blocking 17 minutes on a purge backlog. That flag does not exist: ERROR: unrecognized arguments: --no-wait So every purge failed. Two things hid it: a failed purge is tolerated here by design (it must not abort the sweep), and the count printed was the count of successes - so a run that purged nothing printed "queued purge for 0" and moved on. Soft-deleted services then accumulated unchecked until they exhausted the 20-service Consumption cap. Measured on the live subscription just now: 22 soft-deleted against 5 live, i.e. 27 against a cap of 20, which is exactly the `MaxConsumptionServicesPerSubscriptionExceeded` that every APIM fixture has been failing on. Each soft-delete holds its slot for two days. `az rest` sends the DELETE and returns on ARM's 202 without the CLI's polling loop - measured ~20s per service against ~1.3 min - and does not depend on which flags this month's `az apim` accepts. It needs the subscription id, so that is now resolved once up front and the script exits early rather than building an unroutable URL from an empty value. The count now reports failures separately, so the next silent no-op is not silent. Purged all 22 by hand with the same call to unblock the run in flight: 22 -> 0 soft-deleted, 5 live untouched. The local prefix re-check and the live-service exclusion were applied to each name before purging. bash -n passes; both CI shell tests pass.
ValidationError
Error in element 'base' on line 3, column 6: Element <base/> is not allowed
in global context
`<base />` splices in the policy of the enclosing scope, so it is only
meaningful where there is one - the product, API and operation policies, which
all keep it. AZURE::ApiManagement::Policy is the service-wide policy and has no
parent, so ARM rejects the element outright.
Replaced with APIM's own stock global policy: empty inbound, outbound and
on-error, and a backend that forwards. The update fixture keeps its set-header
in inbound so the Update phase still has something to change.
The reason it read as a capacity failure at first glance is that every other
APIM failure in this run was the Consumption cap; this one is a fixture defect
and would have failed on an empty subscription.
Gate: verify-schema, verify-fixtures 593/593, conformance-scope, reuse lint.
## Backup policies: a replace must change the NativeID (4 fixtures)
[Replace] NativeID should change after replace
(old: .../backupVaults/fpsdt-bpdisk-.../backupPolicies/fpsdt-bpdisk-pol-...)
Making the rule trees createOnly was right - ARM does refuse to update a policy
in place - but the -replace fixtures kept the policy's name, so the "replacement"
addressed the same ARM path and the NativeID never moved. The replace fixtures
now use a distinct policy name.
## Logic assembly: contentType is not optional
The 'contentType' property of assembly 'conformance-assembly' must be set to
'application/octet-stream'.
An assembly is always a .dll, so there is exactly one legal value and nothing for
a caller to choose. Sent as a constant from the provisioner rather than added to
the schema as a field with one permitted setting.
## Logic agreement: NotSpecified is not a value, twice over
The agreement 'conformance-as2-agreement' must have 'encryptionAlgorithm'
property set in 'validationSettings'.
Same shape as the micHashingAlgorithm fix before it, in a different block, in
both the receive and send agreements: the field was present but set to
"NotSpecified", which ARM does not count as set. Now AES256.
That earlier fix had a consequence I missed at the time. micHashingAlgorithm
NotSpecified -> SHA2256 was the ONLY difference between the base and update
fixtures, so setting the base to SHA2256 made them identical and left the Update
phase with nothing to change. The update now moves
interchangeDuplicatesValidityDays from 5 to 10 - a scalar ARM echoes back, so
Verify can actually see it.
## Logic certificate: the update added a resource instead of changing one
[Update] Inventory should still contain exactly 1 resource after update
The -update fixture declared a SECOND certificate rather than modifying the
first, so the update left two resources behind. Worse, the sibling carried the
same key material as the original, so it would not have exercised a rotation
even if the harness had allowed it.
Now it rotates publicCertificate on the one certificate, to an independently
generated self-signed cert. name, resourceGroupName and integrationAccountName
are all createOnly and publicCertificate is the only other field this type has,
so rotation is the only in-place update available. It is writeOnly - ARM answers
a GET with a thumbprint, never the key - so Verify cannot compare the value; the
phase proves the apply succeeds and the resource is not replaced.
Gate: build, test-unit, integration, verify-schema, verify-fixtures 593/593,
conformance-scope, go vet, golangci-lint, reuse lint all pass.
…ture
## Logic schema: contentType
The 'contentType' property of schema 'conformance-order-schema' of type 'Xml'
must be set to 'application/xml'.
Same shape as the assembly fix in the previous commit, and the sibling
IntegrationAccountMap already had it right - logicMapContentType derives the
media type from mapType because Liquid is text/plain and every XSLT flavour is
XML. SchemaType has only Xml and NotSpecified and an integration-account schema
is an XSD either way, so this one does not vary and goes across as a constant.
## AZURE::ApiManagement::Service: skipped, and not for a timeout
Reproduced in two consecutive runs:
[Update] Property publisherName should match expected value (after update):
expected Platform Engineering Labs (updated),
got Platform Engineering Labs
ServiceLocked: The API Service fpsdt-apim-... is transitioning at this time.
[OOB Del] timeout waiting for resource ...
The provisioner is not at fault. Update calls BeginUpdate, returns a resume token
and reports InProgress until Status sees the poller finish - the same shape every
other LRO resource in this plugin uses. ARM reports the update complete and then
still answers a GET with the previous publisherName, and the service stays
"transitioning" long enough to lock the delete that follows.
That is eventual consistency inside the service, not slowness, so the timeout arm
does not address it: 30/75 was already applied to this fixture when the above was
captured. The APIM children that need a service still create one; only the
service's own CRUD fixture is skipped, with the evidence recorded inline.
Gate: build, test-unit, integration, verify-schema, verify-fixtures 593/593,
conformance-scope, go vet, golangci-lint, reuse lint all pass.
…e residue
[create] Conflict | A jobSchedule with same id already exists.
[OOB Del] Re-apply command should complete successfully: command reached
terminal state: Failed
Failed twice in a row, so this is not the flake I took it for on the first
sighting. Automation acknowledges a jobSchedule delete before its index catches
up, and a create against the same name keeps returning 409 for a short while
after. The conformance harness walks straight into it: the OOB-delete phase
deletes out of band and immediately re-applies.
This type is unusually exposed to it. ARM uses the jobScheduleId as both the URL
segment and the resource name, so the fixture supplies a fixed GUID rather than
letting ARM allocate one - every re-apply lands on exactly the name just removed.
Raising the phase timeout would not have helped: the create fails fast, it does
not hang. Retrying is the only thing that can succeed, so `retryOnDeleteResidue`
re-runs the create for up to 90s while ARM answers 409.
It is deliberately narrow. Only 409 is retried - a 400 is a caller error and a
plain error is not a status at all, and both return on the first attempt. A 409
that is a genuine name collision keeps failing and is returned once the window
closes, with the original error preserved so the reason still reaches the caller.
A cancelled context returns immediately rather than sleeping out the interval.
The bounds are vars so TestRetryOnDeleteResidue can shrink the window to
milliseconds instead of sleeping through it; it covers all six paths, including
that the retried call actually succeeds on a later attempt.
The helper lives in common.go because nothing about it is Automation-specific -
any service that acknowledges a delete early can use it.
Gate: build, test-unit, integration, verify-schema, verify-fixtures 593/593,
conformance-scope, go vet, golangci-lint, reuse lint all pass.
Reverts 2f222e8. The retry was the wrong diagnosis and it made the failure worse. I read "A jobSchedule with same id already exists" as Azure acknowledging a delete before its index caught up, and retried the create for 90s. The next run showed what actually happens: RESPONSE 409: 409 Conflict (x4, across the whole phase) [OOB Del] Re-apply command should complete successfully: timeout waiting for command 3Iq2PI07... to complete --- FAIL: TestPluginConformance (354.71s) Four separate create attempts, all 409, and the phase ended on a command timeout at 354s instead of failing fast. The retry converted a quick, legible failure into a slow one and fixed nothing - the same reason I took out the discovery timeout arm earlier in this branch. The id is not reusable. ARM uses the jobScheduleId as both the URL segment and the resource name, so it is caller-supplied and pinned in the fixture rather than allocated by Azure, and the phase order re-uses one that has already been deleted: Step 13 replace creates 8b1d4f60-..., deletes base 3f7c9e1a-... Step 16 destroy removes 8b1d4f60-... Step 19 re-apply re-applies the BASE fixture, i.e. 3f7c9e1a-... again By step 19 the base id has been gone for minutes and Azure still refuses it. No provisioner can make that create succeed, so the fixture is skipped with the sequence recorded inline. This is not a defect in the resource. Create, Verify, Extract, Sync, Update, Replace and Destroy all pass; only the re-apply of a deleted id fails. Gate: build, test-unit, integration, verify-schema, verify-fixtures 593/593, conformance-scope, go vet, golangci-lint, reuse lint all pass.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Wave 3: 100 new Azure resource types, in one PR against
mainas specified.The plugin goes from 173 to 273 registered resource types; testdata goes from
359 to 555 fixtures, all of which render.
The 100 types
Six new pinned SDK modules (
armdatafactoryv1.3.0,armautomationv1.0.0,armstreamanalyticsv1.2.0,armlogicv1.2.0,armdesktopvirtualizationv1.0.0,armapimanagementv1.1.1) and 78 wired clients. Every constructor was verified bycompile probe against a throwaway module before being written — the list was
enumerated from each module's own source rather than guessed, so 0 of 78 were
wrong.
Verification — what this PR actually proves
On the merged tree:
Live conformance has not run, and cannot until #155 lands.
pre-cleanupgatesconformance-testsvianeedsand is still red onmain, so no fixture in thisrepo can be verified on any PR. Everything above is static verification: handlers,
schemas, mock-based integration tests, and forma rendering. That is real, but it is
not the documented bar.
Accordingly, and deliberately: no README rows, no CHANGELOG entries, and no
.github/conformance-matrix.txtlines.conformance-matrix.txtsays a fixturebelongs there only once it has passed CRUD and discovery against live Azure. None
has yet.
Two
main-branch regressions found and fixed on the wayBoth are split out because they are independent of this work and were blocking the
whole repo:
extends formae.SubResourceto 86 nested classes withoutthe matching
@azure.SubResourceHint, which made them unrenderable. 113 of 371fixtures across ~57 resource types could not render; conformance was impossible
for all of them. Found independently by four implementation agents.
clean-environment.shstill aborted inlist_groupsafter fix(ci): stop clean-environment.sh aborting inside list_groups #153. Allfour call sites now tolerate a non-zero status. While this is red, the 6-hourly
reaper cannot run either, and the subscription is holding 117
formae-plugin-sdk-test-*resource groups, billing.This PR also adds
make verify-fixturesto thechecksjob. It is the only checkthat renders a forma — build, unit tests, verify-schema, vet, lint and
pkl eval formae-plugin.pklall evaluate the schema module, which is exactly how#144 shipped green. It has since caught a second unrelated fault (a Pkl
stack overflow from a
localshadowing a property name) that nothing else saw.Environment work done up front
NotRegisteredand have been registered; a firstrun in an unregistered namespace 409s, which would have failed every fixture in
half the groups.
Succeededon this subscription.Dedicated tiers take 30–45 min, past the ~45 min point where the OIDC client
assertion expires (
AADSTS700024). All 28 APIM types depend on this.12 fixtures skip-listed, each with evidence
Environment or SDK limits, not plugin defects. Handlers, schemas and mocked tests
are complete for every one.
log-analytics-cluster— 500 GB/day capacity-reservation floor in thisapi-version, and hours to provision.
log-analytics-table—TablesClienthas no PUT and no DELETE at all, so thelifecycle's out-of-band delete phase can never observe removal.
log-analytics-linked-service— provisional; its only two targets are thecluster above and an Automation account.
AZURE::Automation::Accountarrives inthis PR, so repoint the fixture and drop this line.
storage-local-user,storage-object-replication-policy— need fieldsStorageAccountdoes not model (isHnsEnabled/isSftpEnabled; blob-servicechange feed and versioning).
api-management-gateway— self-hosted gateways need Developer or Premium.api-management-api-tag-description— needs a pre-existing APIM tag; nothing inscope can create one.
api-management-{group,group-user,user,subscription,product-group}— theConsumption tier has no user, group or subscription store. Microsoft's tier
matrix marks the developer portal unavailable there, footnoted as "including
related functionality such as users, groups, issues, applications"; the service
limits page gives
Subscriptions | N/AandUsers | N/A.Two types ship without fixtures, on purpose
DataProtection::BackupInstanceBlobStorageand...Diskhave handler, schema andintegration test but no fixture. A backup instance needs the vault's managed
identity to hold a role on the datasource, and
AZURE::DataProtection::BackupVaultmodels neither an identity nor a
principalIdoutput — soRoleAssignmenthas noprincipal to point at. Closing that means editing an existing green resource, which
this wave does not touch. A fixture would fail
[Create]every run and break thisPR's own CI, so there is none rather than a knowingly-red one.
Notes for review
script validated to reproduce the previous manifest byte-identically.
hasProviderDefaultat all, preferringrequiredwhere the value should stay caller-owned — notably
TriggerSchedule.startTime,which ARM otherwise sets to the moment of creation and would drift on every apply.
groups independently declared
testLinkedServiceNativeIDin the singlepkg/resourcespackage.This workstation's shell rewrites command output, sometimes even in a redirected
file —
go vet > out.txtwas observed producingGo vet: No issues found, whichgo vetnever prints on success.