Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .claude-plugin/marketplace.json
Original file line number Diff line number Diff line change
Expand Up @@ -104,7 +104,7 @@
{
"name": "gtm-audit-pro",
"source": "./gtm-audit-pro",
"description": "AI-assisted GTM + GA4 audit: 72 read-only checkpoints across MeasureU's base 33, consent/privacy, data-layer quality, Google Ads + Meta reconciliation, sGTM delivery and cross-container dedup, performance/security, version history and drift, and offline-conversion / Data Manager API coverage. Produces a scored, tiered-recommendation report. Successor to the audit portion of tidy-gtm (gtm-ai-plugin, gads-gtm-plugin, fix-your-tracking) \u2014 tidy-gtm's write/remediation phase remains the tool for actually applying fixes."
"description": "GTM-only static configuration audits across six quality dimensions, optional background change monitoring, and bounded local Autoresearch candidates. Shared runtime with Fix Your Tracking and GTM AI. No live publishing; live firing and cross-platform reconciliation are not verified."
},
{
"name": "openclaw-tools",
Expand Down Expand Up @@ -132,4 +132,4 @@
"description": "Framework for wiring Meta APIs into a Cloudflare Worker \u2014 spin up Meta Apps and system users, the self-refreshing 60-day token vault, Business Use Case rate limiting, and multi-ad-account fan-out. Covers Marketing API, Conversions API, Business Management, Catalog, Instagram, WhatsApp, Pages and Lead Ads, plus the Meta Ads CLI and hosted Ads MCP server."
}
]
}
}
30 changes: 18 additions & 12 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -723,7 +723,11 @@ Google Ads & GTM automation suite — create conversion actions, wire into GTM,

### gtm-audit-pro ⭐ NEW

AI-assisted, **read-only** GTM + GA4 audit: 72 checkpoints from MeasureU's frozen 33-question base through consent/privacy, data-layer integrity, Google Ads + Meta reconciliation, sGTM delivery and cross-container dedup, performance/security, version history and drift, and offline-conversion / Data Manager API coverage. Outputs a scored report with a tiered (fix-now / next / structural) recommendation per finding.
GTM-only static configuration audits with an optional background change monitor
and bounded Autoresearch candidates. The shared runtime is maintained in Fix Your
Tracking and bundled into GTM AI and this standalone package. This 0.1 rebuild
implements six heuristic quality dimensions; the previously advertised
72-checkpoint engine was not included in the repository and is not claimed here.

```
/plugin install gtm-audit-pro@organized-ai-marketplace
Expand All @@ -733,17 +737,19 @@ AI-assisted, **read-only** GTM + GA4 audit: 72 checkpoints from MeasureU's froze

| Skill | Description |
| --- | --- |
| `gtm-audit-pro` | The audit engine — 72 checkpoints, scored report, tiered recommendations |
| `gtm-mcp-setup` | Five-minute connector preflight (GTM required, Ads/Meta optional) before the first audit |

**Relationship to tidy-gtm.** `gtm-audit-pro` supersedes tidy-gtm's read-only audit phase (duplicate/orphan/naming
detection, tag-trigger-variable correlation, sGTM correlation) — it covers the same ground plus consent, data-layer
integrity, and offline-conversion checks, with a scored report instead of a health number. It never writes to your
container. Once you have findings, `tidy-gtm` (in `gtm-ai-plugin`, `gads-gtm-plugin`, or `fix-your-tracking`) is
still the tool that applies them — renames, dedup, folder reorganization, republish.

**Requires:** GTM MCP (any vendor exposing the standard `gtm_*` tool surface). Google Ads and Meta Ads MCPs are
optional — they unlock ad-account reconciliation but the core audit runs without them.
| `gtm-audit-pro` | Report-only static checks, scored findings, workshop questions |
| `gtm-autoresearch-loop` | Shared change watcher and score/propose/validate/keep-or-revert runtime |

**Relationship to tidy-gtm.** The audit checks references, duplicate configurations,
naming, unused components, legacy UA tags, and folders. Live firing, consent,
server delivery, and cross-platform reconciliation remain unverified. `tidy-gtm`
remains the separately authorized live remediation path. Autoresearch only saves
local metadata-edit candidates; it never imports or publishes.

**Requires:** Node.js 22 and either a complete exported container or a Google
read-only token provider for direct GTM polling. A model command is optional for
candidate optimization. Background mode needs a running host; installing the
plugin alone does not enable it. See [setup](gtm-audit-pro/README.md).

---

Expand Down
2 changes: 1 addition & 1 deletion fix-your-tracking/.claude-plugin/plugin.json
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{
"name": "fix-your-tracking",
"description": "Tracking diagnostics and fixes for analytics implementations",
"version": "1.1.0",
"version": "1.2.0",
"author": {
"name": "Organized AI",
"url": "https://github.com/Organized-AI"
Expand Down
22 changes: 22 additions & 0 deletions fix-your-tracking/.claude/skills/gtm-audit-pro/SKILL.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
---
name: gtm-audit-pro
description: Run a report-only GTM configuration audit before a workshop or after a container change. Inspect references, duplicate configurations, naming, unused components, legacy tags, and folders; save findings and questions.
---

# GTM Audit Skill Pro

Use the bundled sibling `gtm-autoresearch-loop` skill's
[setup reference](../gtm-autoresearch-loop/references/audit-integration.md).
Resolve its `scripts/runtime/cli.mjs` and run `audit` against a complete export or
authorized remote GTM target. Open the saved report and verify target and coverage.

Default to a one-time report-only run. If ongoing monitoring is requested, use
`start` or supervised `watch` with `optimize: false` and verify `status`. If the user
also requests Autoresearch, follow the sibling skill's bounded optimization setup.
No mode imports or publishes a GTM change. Existing `tidy-gtm` remains the separate
remediation workflow for explicitly authorized fixes.

This 0.1 release implements six static quality dimensions. Report skipped checks;
do not claim 72 checkpoints, GA4/ads reconciliation, compliance verification,
live firing validation, or a complete business-event audit. An exported file
source only sees new changes when that file is refreshed.
154 changes: 49 additions & 105 deletions fix-your-tracking/.claude/skills/gtm-autoresearch-loop/SKILL.md
Original file line number Diff line number Diff line change
@@ -1,109 +1,53 @@
---
name: gtm-autoresearch-loop
description: Run a Karpathy-style autonomous optimization loop on a GTM container for a specific client. Scores across 12 dimensions (structural + ads-driven), mutates via Claude CLI, validates invariants, and saves winning configs.
triggers:
- "run gtm loop"
- "autoresearch loop"
- "optimize GTM container"
- "run the loop for"
- "gtm experiment"
description: Audit GTM configuration changes and run a bounded score-propose-validate-keep/revert loop on exported candidates. Use for GTM Autoresearch, automatic re-audits, background container monitoring, or optimizing an audit report.
---

# GTM Autoresearch Loop Skill

Autonomous GTM container optimization using structural scoring + LLM mutations.

## What It Does

Takes a client's GTM container export JSON and enriched ads snapshot, then runs an iterative improve-or-revert loop:

1. **Score** — Evaluate the container across 12 weighted dimensions
2. **Prompt** — Build a targeted mutation prompt focusing on the lowest-scoring dimension
3. **Mutate** — Call Claude CLI to generate JSON operations (add tags, set consent, etc.)
4. **Validate** — 3-tier gate: JSON parse, GTM schema, invariant constraints
5. **Keep/Revert** — Accept improvements, reject regressions
6. **Repeat** — Until plateau (92%+ for 3 rounds), max rounds, or failure limit

## Prerequisites

### Per-Client Setup

Each client needs a directory under `content/gtm-templates/{CLIENT}/`:

```
content/gtm-templates/{CLIENT}/
seed/
{template-name}.json
winning/
manifest.json
```

Each client needs an ads snapshot at `data/signals/{client}-ads-snapshot-enriched.json`.

Each client needs a program contract at `content/gtm-templates/{client}-program.md`.

## Execution

### Step 0: Export GTM Container via MCP

If the Stape GTM MCP server is connected:

1. List GTM accounts and containers
2. Pull the live container version by resource type
3. Assemble the export JSON
4. Save to `content/gtm-templates/{CLIENT}/seed/{publicId}-live.json`

Assembly script:

```bash
npx tsx scripts/export-gtm-container.ts <dump.json> <CLIENT>
```

### Step 1: Prepare Client Data

```bash
mkdir -p content/gtm-templates/{CLIENT}/seed content/gtm-templates/{CLIENT}/winning
cp /path/to/export.json content/gtm-templates/{CLIENT}/seed/{template-name}.json
cp /path/to/snapshot.json data/signals/{client}-ads-snapshot-enriched.json
cp content/gtm-templates/program.md content/gtm-templates/{client}-program.md
```

### Step 2: Run the Loop

```bash
npx tsx scripts/run-gtm-loop.ts content/gtm-templates/{client}-program.md
MAX_ROUNDS=30 npx tsx scripts/run-gtm-loop.ts content/gtm-templates/{client}-program.md
```

### Step 3: Validate the Winner

```bash
npx tsx evals/eval_gtm_signal_quality.ts content/gtm-templates/{CLIENT}/winning/best-*.json \
--enriched-snapshot data/signals/{client}-ads-snapshot-enriched.json
```

## Environment Variables

| Variable | Default | Description |
|----------|---------|-------------|
| `MUTATION_PROVIDER` | `claude` | `claude` or `codex` |
| `MUTATION_MODEL` | `sonnet` | Model for mutations |
| `CLAUDE_PATH` | `/Users/jordaaan/.local/bin/claude` | Path to Claude CLI |
| `MAX_ROUNDS` | `30` | Maximum optimization rounds |

## Scoring Dimensions (12)

| # | Dimension | Weight | Requires Ads Data |
|---|-----------|--------|-------------------|
| 1 | Tag coverage | 0.14 | No |
| 2 | Parameter completeness | 0.10 | No |
| 3 | Deduplication | 0.07 | No |
| 4 | Consent settings | 0.11 | No |
| 5 | Naming conventions | 0.06 | No |
| 6 | Variable hygiene | 0.06 | No |
| 7 | Trigger quality | 0.08 | No |
| 8 | Folder organization | 0.06 | No |
| 9 | Meta Ads alignment | 0.09 | Yes |
| 10 | CAPI coverage | 0.08 | Yes |
| 11 | Funnel integrity | 0.07 | Yes |
| 12 | Google Ads alignment | 0.08 | Yes |
# GTM Autoresearch

Reuse Fix Your Tracking's container-audit criteria and keep/revert workflow with
the bundled dependency-free Node.js 22 runtime. Read
[setup and integration](references/audit-integration.md) before the first run.

## Modes

- **Audit**: inspect a complete GTM export or read the selected remote GTM target;
save findings and workshop questions. No model required.
- **Loop**: ask a configured model command for metadata-only candidate edits;
keep strict score improvements with no per-dimension regression. Stop after
the configured rounds, plateau, or failures.
- **Watch/start**: poll the selected source, wait for stable snapshots, audit each
changed snapshot, and optionally invoke the loop. A running host is required.
Installing the skill alone does not enable monitoring.

The first runtime release implements six static quality dimensions: references,
duplicates, naming, hygiene, legacy UA tags, and folders. These are heuristic
configuration checks, not a complete tracking validation. The former skill
described an unavailable twelve-dimension ads-driven evaluator; this rebuild
does not claim to implement those missing checks. Report skipped coverage.

## Workflow

1. Identify the requested target and mode. Preserve the user's existing scope:
an audit request does not authorize a background service or optimization.
2. Set up a persistent configuration outside the plugin cache using the reference.
For direct remote monitoring, obtain read-only GTM OAuth through the host's
existing credential provider; MCP login alone does not authenticate this runner.
3. Run the initial audit, reopen the report, and verify target and findings.
4. For an authorized loop, configure a trusted model command, validate it on a
sample, and run `loop`. Save baseline, rounds, candidate, and final report.
5. For authorized monitoring, use `start` or a supervised `watch`, then verify
status and a changed source. Provide the status/stop commands. Clearly identify
whether the source is a live GTM workspace, published version, or local file.

The runtime can rename unreferenced components, add folders, and assign folders.
It cannot change tag parameters, consent settings, triggers, or delete components.
It only issues GET requests to GTM and never imports, publishes, or rolls back.
Use the existing `tidy-gtm` remediation workflow separately for authorized live fixes.

## Maintainer source

This directory in Fix Your Tracking is canonical. Run
`python3 gtm-ai-plugin/scripts/sync-autoresearch.py` from the marketplace checkout
to update the identical standalone bundles in GTM AI and GTM Audit Pro.
Run the same command with `--check` to detect drift. Do not edit bundled copies.
Original file line number Diff line number Diff line change
@@ -0,0 +1,146 @@
# Setup and integration

## Requirements and source

Node.js 22 or newer; no npm dependencies. Resolve `scripts/runtime/cli.mjs`
relative to this installed skill, then use its absolute path in commands below.
Keep the configuration, exports, and output directory outside the plugin cache.
The examples use `CLI` to stand for that absolute path; it is not an installed command.

The canonical workflow is reused from Fix Your Tracking. Static checks are based
on its `tidy-gtm/references/audit-checklist.md`. Do not copy its heuristic claims
as verified facts: GA4 event tags (`gaawe`) are not UA tags, and container configuration
alone cannot establish live firing, consent timing, or conversion accuracy.

## One-time audit from an export

Save a GTM export as `container.json`. The runtime accepts its `containerVersion`
object or a direct container object. Require explicit `tag`, `trigger`, `variable`,
`folder`, and `builtInVariable` arrays. If a native export omits an empty collection,
confirm the export is complete before explicitly adding `[]`; do not silently
turn an incomplete MCP response into an empty, passing audit.

Create `audit-config.json` alongside the export:

```json
{
"source": { "type": "file", "path": "container.json" },
"outputDir": ".gtm-audit",
"intervalSeconds": 60,
"stablePolls": 2,
"maxConsecutiveErrors": 3,
"optimize": false
}
```

```sh
node CLI audit audit-config.json
```

Paths are relative to the config file. Reopen `audit.md`, `audit.json`, and
`questions.md` in the returned run directory. Empty inventories are valid but
do not prove required business events exist. Generated reports are private local
files; nothing is sent to recipients automatically.

## Monitor real GTM changes

Replace `source` with the selected numeric API account/container/workspace IDs:

```json
{
"type": "gtm",
"accountId": "123456",
"containerId": "789012",
"workspaceId": "3",
"tokenCommand": ["/absolute/path/to/your-existing-token-provider"]
}
```

The IDs above are examples; use verified IDs, not the `GTM-...` public ID. Omit
`workspaceId` to monitor the published version instead. The workspace source
reads all pages of tags, triggers, variables, folders, built-in variables, and
templates. This release targets web containers; server-specific clients and
transformations are not included in the workspace inventory.

The token command must print only a fresh Google OAuth access token with
`https://www.googleapis.com/auth/tagmanager.readonly` access to the target. It is
called on every capture, so the host provider owns secure storage and refresh.
For a short-lived trial, `tokenEnv` can name an existing environment variable
instead, but it will stop working when that token expires. Do not place tokens
in the JSON config, prompt, or reports. Interactive Stape MCP credentials are not
automatically reusable as Google API tokens. If no direct provider is available,
use a complete exported file for the one-time audit; disclose that file monitoring
does not detect remote edits unless an external exporter refreshes that file.

```sh
node CLI audit audit-config.json
node CLI start audit-config.json
node CLI status audit-config.json
node CLI stop audit-config.json
```

`start` launches a detached local process. It does not install a boot-time service;
the host must remain awake and a supervisor is needed for restart after reboot.
`watch` runs in the foreground for a host service manager. Windows service and
all five agent clients have not been end-to-end verified; any agent with local
Node and command access can invoke the same runtime, while ordinary Desktop chat
needs a local execution host.

Polling is eventual, not an event stream. Two matching captures are required by
default; intermediate edits may be coalesced. API calls are spaced seven seconds
apart by default to reduce quota pressure. Multiple targets share provider quotas.
Use one output root per host for all copies of this plugin: locks are per target
within that root, not distributed across machines or arbitrary output directories.

The watcher saves fingerprints across restarts, ignores its own generated files,
and logs only completion or errors. It retries failed work up to the configured
consecutive-failure limit, then exits nonzero for the supervisor. `stop` waits for
the current capture/round to finish. Use `status` to confirm exit. After a crash,
`unlock` removes a stale lock only when its recorded PID no longer exists; inspect
the process if the OS reused its PID.

## Enable Autoresearch candidates

First verify audit-only mode. For an authorized optimization workflow, add:

```json
{
"optimize": true,
"mutationCommand": ["claude", "-p", "--safe-mode", "--tools", "", "--disallowedTools", "mcp__*", "--output-format", "json", "--no-session-persistence"],
"commandTimeoutMs": 120000,
"loop": { "maxRounds": 5, "maxFailures": 2, "plateauRounds": 2 }
}
```

Check the installed Claude CLI supports these flags before using this example.
It disables customizations and tools; do not use a permissions-bypass flag. Each
round consumes the model account's usage. The model receives container content;
use only a model/provider your organization permits for that data.

Other providers can supply a trusted argv command that reads one JSON request
from stdin and emits only `{"operations": [...]}` to stdout. The request contains
the container, current findings, round number, and allowed operation shapes. Codex,
Hermes, and GrokBot adapters must satisfy that contract; no unverified CLI syntax
is assumed. Commands are administrator-controlled executable code, not a sandbox.

`node CLI loop audit-config.json` runs one bounded optimization. Restart the
watcher after config changes; with `optimize: true`, a new stable snapshot triggers
audit → propose → validate → keep/revert. Changing loop policy also forces a new run.
The runtime saves `optimization.json` and `candidate.json`; it never deploys them.
Candidates must improve the aggregate heuristic score without worsening any
dimension. Referenced-variable and sequenced-tag renames are rejected. Unresolved
reference or functional issues can remain; the output is not a deploy-ready claim.

## Verification

From `scripts/runtime`, run `node --test test/*.test.mjs`. Tests use synthetic
containers, mocked read-only API responses, and local process fixtures. Production
OAuth and a real model need a separate end-to-end smoke test on an authorized
container before workshop distribution.

## Provider references

- [GTM authorization](https://developers.google.com/tag-platform/tag-manager/api/v2/authorization)
- [Workspace tag pagination](https://developers.google.com/tag-platform/tag-manager/api/reference/rest/v2/accounts.containers.workspaces.tags/list)
- [Published container snapshot](https://developers.google.com/tag-platform/tag-manager/api/reference/rest/v2/accounts.containers.versions/live)
- [Claude CLI](https://code.claude.com/docs/en/cli-reference)
Loading