diff --git a/.claude-plugin/plugin.json b/.claude-plugin/plugin.json new file mode 100644 index 00000000..941afc1e --- /dev/null +++ b/.claude-plugin/plugin.json @@ -0,0 +1,5 @@ +{ + "name": "graehamwatts-skills", + "version": "1.0.0", + "description": "Graeham Watts personal skills vault: content creation engine, CMA generator, disclosure analyzer, offer analyzer, GHL CRM audit, content calendar, listing tools, and the Watts content+video pipeline. Publishing handled via the Composio workflow (see shared-references/publishing-via-composio.md)." +} diff --git a/.claude/CLAUDE.md b/.claude/CLAUDE.md new file mode 100644 index 00000000..b20ba5b9 --- /dev/null +++ b/.claude/CLAUDE.md @@ -0,0 +1,3 @@ +# graphify +- **graphify** (`.claude/skills/graphify/SKILL.md`) - any input to knowledge graph. Trigger: `/graphify` +When the user types `/graphify`, use the installed graphify skill or instructions before doing anything else. diff --git a/.claude/settings.json b/.claude/settings.json new file mode 100644 index 00000000..deffac97 --- /dev/null +++ b/.claude/settings.json @@ -0,0 +1,3 @@ +{ + "hooks": {} +} diff --git a/.claude/skills/graphify/.graphify_version b/.claude/skills/graphify/.graphify_version new file mode 100644 index 00000000..03834411 --- /dev/null +++ b/.claude/skills/graphify/.graphify_version @@ -0,0 +1 @@ +0.9.5 \ No newline at end of file diff --git a/.claude/skills/graphify/SKILL.md b/.claude/skills/graphify/SKILL.md new file mode 100644 index 00000000..6ab0e027 --- /dev/null +++ b/.claude/skills/graphify/SKILL.md @@ -0,0 +1,712 @@ +--- +name: graphify-windows +description: "Use for any question about a codebase, its architecture, file relationships, or project content — especially when graphify-out/ exists, where the question should be treated as a graphify query first. Turns any input (code, docs, papers, images, videos) into a persistent knowledge graph with god nodes, community detection, and query/path/explain tools." +--- + +# /graphify + +Turn any folder of files into a navigable knowledge graph with community detection, an honest audit trail, and three outputs: interactive HTML, GraphRAG-ready JSON, and a plain-language GRAPH_REPORT.md. + +## Usage + +``` +/graphify # full pipeline on current directory → Obsidian vault +/graphify # full pipeline on specific path +/graphify https://github.com// # clone repo then run full pipeline on it +/graphify https://github.com// --branch # clone a specific branch +/graphify ... # clone multiple repos, build each, merge into one cross-repo graph +/graphify --mode deep # thorough extraction, richer INFERRED edges +/graphify --update # incremental - re-extract only new/changed files +/graphify --directed # build directed graph (preserves edge direction: source→target) +/graphify --whisper-model medium # use a larger Whisper model for better transcription accuracy +/graphify --cluster-only # rerun clustering on existing graph +/graphify --no-viz # skip visualization, just report + JSON +/graphify --html # (HTML is generated by default - this flag is a no-op) +/graphify --svg # also export graph.svg (embeds in Notion, GitHub) +/graphify --graphml # export graph.graphml (Gephi, yEd) +/graphify --neo4j # generate graphify-out/cypher.txt for Neo4j +/graphify --neo4j-push bolt://localhost:7687 # push directly to Neo4j +/graphify --falkordb # generate graphify-out/cypher.txt for FalkorDB +/graphify --falkordb-push falkordb://localhost:6379 # push directly to FalkorDB +/graphify --mcp # start MCP stdio server for agent access +/graphify --watch # watch folder, auto-rebuild on code changes (no LLM needed) +/graphify --wiki # build agent-crawlable wiki (index.md + one article per community) +/graphify --obsidian --obsidian-dir ~/vaults/my-project # write vault to custom path (e.g. existing vault) +/graphify add # fetch URL, save to ./raw, update graph +/graphify add --author "Name" # tag who wrote it +/graphify add --contributor "Name" # tag who added it to the corpus +/graphify query "" # BFS traversal - broad context +/graphify query "" --dfs # DFS - trace a specific path +/graphify query "" --budget 1500 # cap answer at N tokens +/graphify path "AuthModule" "Database" # shortest path between two concepts +/graphify explain "SwinTransformer" # plain-language explanation of a node +``` + +## What graphify is for + +Drop any folder of code, docs, papers, images, or video into graphify and get a queryable knowledge graph. Persistent across sessions, honest audit trail (EXTRACTED/INFERRED/AMBIGUOUS), community detection surfaces cross-document connections you wouldn't think to ask about. + +## What You Must Do When Invoked + +If the user invoked `/graphify --help` or `/graphify -h` (with no other arguments), print the contents of the `## Usage` section above verbatim and stop. Do not run any commands, do not detect files, do not default the path to `.`. Just print the Usage block and return. + +**Fast path — existing graph:** Before doing anything else, check whether `graphify-out/graph.json` exists. The expected location is `graphify-out/graph.json` relative to the **current working directory** (i.e. the project root where you are running commands). If it exists AND the user's request is a natural-language question about the codebase (e.g. "How does X work?", "What calls Y?", "Trace the data flow through Z") and NOT an explicit rebuild command (`--update`, `--cluster-only`, or a bare path/URL that implies fresh extraction): **skip Steps 1–5 entirely and jump straight to `## For /graphify query`.** Run `graphify query ""` immediately. Do not run detect. Do not check corpus size. Do not ask the user to narrow. The graph is already built — use it. + +If no path was given, use `.` (current directory). Do not ask the user for a path. + +If the path argument starts with `https://github.com/` or `http://github.com/`, treat it as a GitHub URL - run Step 0 before anything else, then continue with the resolved local path. + +Follow these steps in order. Do not skip steps. + +### Step 0 - GitHub repos and multi-path merge (only if a URL or several paths) + +Only when the path is one or more `https://github.com/...` URLs, or several local subfolders to merge. See `references/github-and-merge.md` for the clone, cross-repo merge, and monorepo flow, then continue with the resolved local path. A plain local path skips this step. + +### Step 1 - Ensure graphify is installed + +```powershell +# Detect Python with graphify — uv/pipx-aware (fixes #831) +New-Item -ItemType Directory -Force -Path graphify-out | Out-Null +$GRAPHIFY_PYTHON = $null + +function Find-GraphifyPython { + # 1. uv tool install — 'uv tool dir' is authoritative, respects UV_TOOL_DIR automatically + if (Get-Command uv -ErrorAction SilentlyContinue) { + $uvDir = (uv tool dir 2>$null).Trim() + if ($uvDir) { + $py = Join-Path $uvDir "graphifyy\Scripts\python.exe" + if (Test-Path $py) { + & $py -c "import graphify" 2>$null + if ($LASTEXITCODE -eq 0) { return $py } + } + } + } + # 2. pipx install — 'pipx environment' respects PIPX_HOME automatically + if (Get-Command pipx -ErrorAction SilentlyContinue) { + $venvs = (pipx environment --value PIPX_LOCAL_VENVS 2>$null).Trim() + if ($venvs) { + $py = Join-Path $venvs "graphifyy\Scripts\python.exe" + if (Test-Path $py) { + & $py -c "import graphify" 2>$null + if ($LASTEXITCODE -eq 0) { return $py } + } + } + } + # 3. Active venv / conda / pip-into-current-env + $pyCmd = Get-Command python -ErrorAction SilentlyContinue + if ($pyCmd) { + & $pyCmd.Source -c "import graphify" 2>$null + if ($LASTEXITCODE -eq 0) { + return (& $pyCmd.Source -c "import sys; print(sys.executable)").Trim() + } + } + return $null +} + +# Try to find the right Python (uv → pipx → active env) +$GRAPHIFY_PYTHON = Find-GraphifyPython + +# Not found — install then re-detect +if (-not $GRAPHIFY_PYTHON) { + if (Get-Command uv -ErrorAction SilentlyContinue) { + uv tool install --upgrade graphifyy -q 2>&1 | Select-Object -Last 3 + } else { + pip install graphifyy -q 2>&1 | Select-Object -Last 3 + } + $GRAPHIFY_PYTHON = Find-GraphifyPython +} + +# Save interpreter path — all subsequent steps read this +$GRAPHIFY_PYTHON | Out-File -FilePath graphify-out\.graphify_python -Encoding utf8 -NoNewline +# Save scan root so `graphify update` (no args) knows where to look next time +(Resolve-Path INPUT_PATH).Path | Out-File -FilePath graphify-out\.graphify_root -Encoding utf8 -NoNewline +``` + +If the import succeeds, print nothing and move straight to Step 2. + +**In every subsequent block, run Python through the saved interpreter — `& (Get-Content graphify-out\.graphify_python)` in place of a bare `python3` — so every step uses the interpreter that actually has graphify.** + +### Step 2 - Detect files + +```bash +$(cat graphify-out/.graphify_python) -c " +import json +from graphify.detect import detect +from pathlib import Path +result = detect(Path('INPUT_PATH')) +print(json.dumps(result, ensure_ascii=False)) +" > graphify-out/.graphify_detect.json +``` + +Replace INPUT_PATH with the actual path the user provided. Do NOT cat or print the JSON - read it silently and present a clean summary instead: + +``` +Corpus: X files · ~Y words + code: N files (.py .ts .go ...) + docs: N files (.md .txt ...) + papers: N files (.pdf ...) + images: N files + video: N files (.mp4 .mp3 ...) +``` + +Omit any category with 0 files from the summary. + +Then act on it: +- If `total_files` is 0: stop with "No supported files found in [path]." +- If `skipped_sensitive` is non-empty: mention file count skipped, not the file names. +- If `total_words` > 2,000,000 OR `total_files` > 500: show the warning. Then compute the top 5 first-level subdirectories by file count: + - Read `scan_root` from the detect JSON (always an absolute path to the resolved INPUT_PATH). + - Concatenate all file lists across all types (`code`, `document`, `paper`, `image`, `video`). + - Filter out any path that starts with `scan_root + "/graphify-out/"` to exclude converted sidecars. + - For each file, strip the `scan_root` prefix and take the first path component. Files directly in `scan_root` with no subdirectory count as `(root)`. + - If all files are in `(root)` with no subdirectories, do not ask to narrow — no subfolders exist. Instead suggest `--no-cluster` to skip the expensive clustering step and proceed. + - Otherwise rank by count, show the top 5 with file counts, then ask which subfolder to run on. Wait for the user's answer before proceeding. +- Otherwise: proceed directly to Step 2.5 if video files were detected, or Step 3 if not. + +### Step 2.5 - Video and audio (only if video files detected) + +Skip this step entirely if `detect` returned zero `video` files. When the corpus has video or audio, see `references/transcribe.md` to transcribe them to text first, then treat the transcripts as doc files in Step 3. + +### Step 3 - Extract entities and relationships + +**Before starting:** note whether `--mode deep` was given. You must pass `DEEP_MODE=true` to every subagent in Step B2 if it was. Track this from the original invocation - do not lose it. + +This step has two parts: **structural extraction** (deterministic, free) and **semantic extraction** (LLM, costs tokens). + +> **graphify needs no API key. Never ask the user for one, and never block on one.** Code is extracted structurally (AST) with no LLM and no key at all — a code-only corpus (the common `/graphify .` on a repo) skips semantic extraction entirely, so it needs nothing here: go straight to Part A and skip Part B. Semantic extraction (only for docs, papers, and images) uses Gemini **only if** `GEMINI_API_KEY`/`GOOGLE_API_KEY` is already set; otherwise the host agent itself is the LLM. graphify does **not** read `ANTHROPIC_API_KEY`, `OPENAI_API_KEY`, or any other provider key. If you catch yourself about to prompt for, wait on, or stop because of a missing API key, that is a misread of this skill — proceed without one. + +**Before semantic extraction:** check whether `GEMINI_API_KEY` or `GOOGLE_API_KEY` is set. If neither is set, print this one-liner to the user: +> Tip: set `GEMINI_API_KEY` or `GOOGLE_API_KEY` to use Gemini for semantic extraction (`pip install 'graphifyy[gemini]'`). + +Print it once, then continue — do not wait for the user to supply a key. If `GEMINI_API_KEY` or `GOOGLE_API_KEY` IS set, use `graphify.llm.extract_corpus_parallel(files, backend="gemini")` for semantic extraction instead of dispatching subagents. The default Gemini model is `gemini-3-flash-preview`; set `GRAPHIFY_GEMINI_MODEL` or pass `--model` in headless CLI flows to override it. + +> **No other API keys are read.** When `GEMINI_API_KEY`/`GOOGLE_API_KEY` are unset, semantic extraction falls to the host agent itself — the running session is the LLM. On a host that dispatches subagents (e.g. Claude Code), dispatch them as written in Part B. On a host that runs the CLI directly in a terminal and cannot dispatch subagents, do not stall: a code-only corpus has no semantic work, so write the empty semantic file (Part B "Fast path") and continue to Part C; for a corpus with docs/papers/images, either set a Gemini key or extract those inline yourself, but in no case prompt for `ANTHROPIC_API_KEY` — that prompt is a misread of this skill. + +**Run Part A (AST) and Part B (semantic) in parallel. Dispatch all semantic subagents AND start AST extraction in the same message. Both can run simultaneously since they operate on different file types. Merge results in Part C as before.** + +Note: Parallelizing AST + semantic saves 5-15s on large corpora. AST is deterministic and fast; start it while subagents are processing docs/papers. + +#### Part A - Structural extraction for code files + +For any code files detected, run AST extraction in parallel with Part B subagents: + +```bash +$(cat graphify-out/.graphify_python) -c " +import sys, json +from graphify.extract import collect_files, extract +from pathlib import Path +import json + +code_files = [] +detect = json.loads(Path('graphify-out/.graphify_detect.json').read_text(encoding=\"utf-8\")) +for f in detect.get('files', {}).get('code', []): + code_files.extend(collect_files(Path(f)) if Path(f).is_dir() else [Path(f)]) + +if code_files: + result = extract(code_files, cache_root=Path('INPUT_PATH')) + Path('graphify-out/.graphify_ast.json').write_text(json.dumps(result, indent=2, ensure_ascii=False), encoding=\"utf-8\") + print(f'AST: {len(result[\"nodes\"])} nodes, {len(result[\"edges\"])} edges') +else: + Path('graphify-out/.graphify_ast.json').write_text(json.dumps({'nodes':[],'edges':[],'input_tokens':0,'output_tokens':0}, ensure_ascii=False), encoding=\"utf-8\") + print('No code files - skipping AST extraction') +" +``` + +#### Part B - Semantic extraction (parallel subagents) + +**Fast path:** If detection found zero docs, papers, and images (code-only corpus), skip Part B entirely and go straight to Part C. AST handles code - there is nothing for semantic subagents to do. **First write an empty semantic file** so Part C's merge has its input (it reads `.graphify_semantic.json` unconditionally; without this a code-only run hits `FileNotFoundError`): + +```bash +$(cat graphify-out/.graphify_python) -c " +import json +from pathlib import Path +Path('graphify-out/.graphify_semantic.json').write_text(json.dumps({'nodes':[],'edges':[],'hyperedges':[],'input_tokens':0,'output_tokens':0}), encoding='utf-8') +" +``` + +**MANDATORY: You MUST use the Agent tool here. Reading files yourself one-by-one is forbidden - it is 5-10x slower. If you do not use the Agent tool you are doing this wrong.** + +Before dispatching subagents, print a timing estimate: +- Load `total_words` and file counts from `graphify-out/.graphify_detect.json` +- Estimate agents needed: `ceil(uncached_non_code_files / 22)` (chunk size is 20-25) +- Estimate time: ~45s per agent batch (they run in parallel, so total ≈ 45s × ceil(agents/parallel_limit)) +- Print: "Semantic extraction: ~N files → X agents, estimated ~Ys" + +**Step B0 - Check extraction cache first** + +Before dispatching any subagents, check which files already have cached extraction results: + +```bash +$(cat graphify-out/.graphify_python) -c " +import json +from graphify.cache import check_semantic_cache +from pathlib import Path + +detect = json.loads(Path('graphify-out/.graphify_detect.json').read_text(encoding=\"utf-8\")) +# Only content files go to semantic extraction. Code is already covered structurally +# by the AST pass (Part A); flattening every category here makes subagents re-read +# every source file (#1392). Video is transcribed to a document in Step 2.5 first. +all_files = [f for cat in ('document', 'paper', 'image') for f in detect['files'].get(cat, [])] + +cached_nodes, cached_edges, cached_hyperedges, uncached = check_semantic_cache(all_files, root='INPUT_PATH') + +# Always (re)write the cache file: write hits, else DELETE any leftover from a prior +# run so Part C never merges a stale .graphify_cached.json (#1392). +if cached_nodes or cached_edges or cached_hyperedges: + Path('graphify-out/.graphify_cached.json').write_text(json.dumps({'nodes': cached_nodes, 'edges': cached_edges, 'hyperedges': cached_hyperedges}, ensure_ascii=False), encoding=\"utf-8\") +else: + Path('graphify-out/.graphify_cached.json').unlink(missing_ok=True) +Path('graphify-out/.graphify_uncached.txt').write_text('\n'.join(uncached), encoding=\"utf-8\") +print(f'Cache: {len(all_files)-len(uncached)} files hit, {len(uncached)} files need extraction') +" +``` + +Only dispatch subagents for files listed in `graphify-out/.graphify_uncached.txt`. If all files are cached, skip to Part C directly. + +**Step B1 - Split into chunks** + +Load files from `graphify-out/.graphify_uncached.txt`. Split into chunks of 20-25 files each. Each image gets its own chunk (vision needs separate context). When splitting, group files from the same directory together so related artifacts land in the same chunk and cross-file relationships are more likely to be extracted. + +**Step B2 - Dispatch ALL subagents in a single message** + +Call the Agent tool multiple times IN THE SAME RESPONSE - one call per chunk. This is the only way they run in parallel. If you make one Agent call, wait, then make another, you are doing it sequentially and defeating the purpose. + +**IMPORTANT - subagent type:** Always use `subagent_type="general-purpose"`. Do NOT use `Explore` - it is read-only and cannot write chunk files to disk, which silently drops extraction results. General-purpose has Write and Bash access which the subagent needs. + +Concrete example for 3 chunks: +``` +[Agent tool call 1: files 1-15, subagent_type="general-purpose"] +[Agent tool call 2: files 16-30, subagent_type="general-purpose"] +[Agent tool call 3: files 31-45, subagent_type="general-purpose"] +``` +All three in one message. Not three separate messages. + +Each subagent receives this exact prompt (substitute FILE_LIST, CHUNK_NUM, TOTAL_CHUNKS, DEEP_MODE, and CHUNK_PATH). + +CHUNK_PATH must be an **absolute** path — derive it before dispatching: +```powershell +$PROJECT_ROOT = (Get-Location).Path # cwd — where Part C globs graphify-out\ (NOT .graphify_root/scan dir, #1392) +# Then for chunk N: $CHUNK_PATH = Join-Path $PROJECT_ROOT "graphify-out\.graphify_chunk_0N.json" +``` + +Subagent prompt template: + +See `references/extraction-spec.md` for the exact subagent prompt (JSON schema, node-ID rules, confidence rubric, frontmatter, hyperedge, and vision rules). Load it only here, only when at least one chunk holds a doc, paper, or image; a pure-code corpus has skipped Part B and never reads it. Pass each subagent that prompt verbatim with FILE_LIST, CHUNK_NUM, TOTAL_CHUNKS, DEEP_MODE, and CHUNK_PATH substituted, and have it write the result to CHUNK_PATH. + +**Step B3 - Collect, cache, and merge** + +Wait for all subagents. For each result: +- Check that `graphify-out/.graphify_chunk_NN.json` exists on disk — this is the success signal +- If the file exists and contains valid JSON with `nodes` and `edges`, include it and save to cache +- If the file is missing, the subagent was likely dispatched as read-only (Explore type) — print a warning: "chunk N missing from disk — subagent may have been read-only. Re-run with general-purpose agent." Do not silently skip. +- If a subagent failed or returned invalid JSON, print a warning and skip that chunk - do not abort + +If more than half the chunks failed or are missing, stop and tell the user to re-run and ensure `subagent_type="general-purpose"` is used. + +Merge all chunk files into `.graphify_semantic_new.json`. **After each Agent call completes, read the real token counts from the Agent tool result's `usage` field and write them back into the chunk JSON before merging** — the chunk JSON itself always has placeholder zeros. Then run: +```bash +$(cat graphify-out/.graphify_python) -c " +import json, glob +from pathlib import Path + +chunks = sorted(glob.glob('graphify-out/.graphify_chunk_*.json')) +all_nodes, all_edges, all_hyperedges = [], [], [] +total_in, total_out = 0, 0 +for c in chunks: + d = json.loads(Path(c).read_text(encoding=\"utf-8\")) + all_nodes += d.get('nodes', []) + all_edges += d.get('edges', []) + all_hyperedges += d.get('hyperedges', []) + total_in += d.get('input_tokens', 0) + total_out += d.get('output_tokens', 0) +Path('graphify-out/.graphify_semantic_new.json').write_text(json.dumps({ + 'nodes': all_nodes, 'edges': all_edges, 'hyperedges': all_hyperedges, + 'input_tokens': total_in, 'output_tokens': total_out, +}, indent=2, ensure_ascii=False), encoding=\"utf-8\") +print(f'Merged {len(chunks)} chunks: {total_in:,} in / {total_out:,} out tokens') +" +``` + +Save new results to cache: +```bash +$(cat graphify-out/.graphify_python) -c " +import json +from graphify.cache import save_semantic_cache +from pathlib import Path + +new = json.loads(Path('graphify-out/.graphify_semantic_new.json').read_text(encoding=\"utf-8\")) if Path('graphify-out/.graphify_semantic_new.json').exists() else {'nodes':[],'edges':[],'hyperedges':[]} +saved = save_semantic_cache(new.get('nodes', []), new.get('edges', []), new.get('hyperedges', []), root='INPUT_PATH') +print(f'Cached {saved} files') +" +``` + +Merge cached + new results into `graphify-out/.graphify_semantic.json`: +```bash +$(cat graphify-out/.graphify_python) -c " +import json +from pathlib import Path + +cached = json.loads(Path('graphify-out/.graphify_cached.json').read_text(encoding=\"utf-8\")) if Path('graphify-out/.graphify_cached.json').exists() else {'nodes':[],'edges':[],'hyperedges':[]} +new = json.loads(Path('graphify-out/.graphify_semantic_new.json').read_text(encoding=\"utf-8\")) if Path('graphify-out/.graphify_semantic_new.json').exists() else {'nodes':[],'edges':[],'hyperedges':[]} + +all_nodes = cached['nodes'] + new.get('nodes', []) +all_edges = cached['edges'] + new.get('edges', []) +all_hyperedges = cached.get('hyperedges', []) + new.get('hyperedges', []) +seen = set() +deduped = [] +for n in all_nodes: + if n['id'] not in seen: + seen.add(n['id']) + deduped.append(n) + +merged = { + 'nodes': deduped, + 'edges': all_edges, + 'hyperedges': all_hyperedges, + 'input_tokens': new.get('input_tokens', 0), + 'output_tokens': new.get('output_tokens', 0), +} +Path('graphify-out/.graphify_semantic.json').write_text(json.dumps(merged, indent=2, ensure_ascii=False), encoding=\"utf-8\") +print(f'Extraction complete - {len(deduped)} nodes, {len(all_edges)} edges ({len(cached[\"nodes\"])} from cache, {len(new.get(\"nodes\",[]))} new)') +" +``` +Clean up temp files: `rm -f graphify-out/.graphify_cached.json graphify-out/.graphify_uncached.txt graphify-out/.graphify_semantic_new.json` + +#### Part C - Merge AST + semantic into final extraction + +```bash +$(cat graphify-out/.graphify_python) -c " +import sys, json +from pathlib import Path + +ast = json.loads(Path('graphify-out/.graphify_ast.json').read_text(encoding=\"utf-8\")) +sem = json.loads(Path('graphify-out/.graphify_semantic.json').read_text(encoding=\"utf-8\")) + +# Merge: AST nodes first, semantic nodes deduplicated by id +seen = {n['id'] for n in ast['nodes']} +merged_nodes = list(ast['nodes']) +for n in sem['nodes']: + if n['id'] not in seen: + merged_nodes.append(n) + seen.add(n['id']) + +merged_edges = ast['edges'] + sem['edges'] +merged_hyperedges = sem.get('hyperedges', []) +merged = { + 'nodes': merged_nodes, + 'edges': merged_edges, + 'hyperedges': merged_hyperedges, + 'input_tokens': sem.get('input_tokens', 0), + 'output_tokens': sem.get('output_tokens', 0), +} +Path('graphify-out/.graphify_extract.json').write_text(json.dumps(merged, indent=2, ensure_ascii=False), encoding=\"utf-8\") +total = len(merged_nodes) +edges = len(merged_edges) +print(f'Merged: {total} nodes, {edges} edges ({len(ast[\"nodes\"])} AST + {len(sem[\"nodes\"])} semantic)') +" +``` + +### Step 4 - Build graph, cluster, analyze, generate outputs + +**Before starting:** the code blocks below pass `directed=IS_DIRECTED` to `build_from_json()`. Replace `IS_DIRECTED` with `True` if `--directed` was given (builds a `DiGraph` preserving edge direction source→target), otherwise `False` (the default undirected `Graph`). Substitute it the same way you substitute `INPUT_PATH` — do not leave the literal `IS_DIRECTED` in the code. + +```bash +mkdir -p graphify-out +$(cat graphify-out/.graphify_python) -c " +import sys, json +from graphify.build import build_from_json +from graphify.cluster import cluster, score_all +from graphify.analyze import god_nodes, surprising_connections, suggest_questions +from graphify.report import generate +from graphify.export import to_json +from pathlib import Path + +extraction = json.loads(Path('graphify-out/.graphify_extract.json').read_text(encoding=\"utf-8\")) +detection = json.loads(Path('graphify-out/.graphify_detect.json').read_text(encoding=\"utf-8\")) + +# root= mirrors the --update runbook (#1361): relativize source_file to the same +# base so the full build and incremental --update never drift apart on re-extract. +G = build_from_json(extraction, root='INPUT_PATH', directed=IS_DIRECTED) +# Guard BEFORE any write: an empty extraction must not clobber a good graph.json / +# GRAPH_REPORT.md / analysis sidecar. Check immediately after build (#1392). +if G.number_of_nodes() == 0: + print('ERROR: Graph is empty - extraction produced no nodes.') + print('Possible causes: all files were skipped, binary-only corpus, or extraction failed.') + raise SystemExit(1) +communities = cluster(G) +cohesion = score_all(G, communities) +tokens = {'input': extraction.get('input_tokens', 0), 'output': extraction.get('output_tokens', 0)} +gods = god_nodes(G) +surprises = surprising_connections(G, communities) +labels = {cid: 'Community ' + str(cid) for cid in communities} +# Placeholder questions - regenerated with real labels in Step 5 +questions = suggest_questions(G, communities, labels) + +# Export FIRST and honor the #479 shrink-guard: to_json returns False (writing +# nothing) when the new graph is smaller than the existing graph.json. Only write +# GRAPH_REPORT.md + the analysis sidecar when the graph was actually written, so +# they never describe a graph that graph.json doesn't contain (#1392). +wrote = to_json(G, communities, 'graphify-out/graph.json') +if not wrote: + print('ERROR: refused to shrink graphify-out/graph.json (existing graph has more nodes; #479).') + print('If this shrink is intentional (you deleted files), re-run a full build with --force.') + raise SystemExit(1) +report = generate(G, communities, cohesion, labels, gods, surprises, detection, tokens, 'INPUT_PATH', suggested_questions=questions) +Path('graphify-out/GRAPH_REPORT.md').write_text(report, encoding=\"utf-8\") +analysis = { + 'communities': {str(k): v for k, v in communities.items()}, + 'cohesion': {str(k): v for k, v in cohesion.items()}, + 'gods': gods, + 'surprises': surprises, + 'questions': questions, +} +Path('graphify-out/.graphify_analysis.json').write_text(json.dumps(analysis, indent=2, ensure_ascii=False), encoding=\"utf-8\") +print(f'Graph: {G.number_of_nodes()} nodes, {G.number_of_edges()} edges, {len(communities)} communities') +" +``` + +If this step prints `ERROR: Graph is empty`, stop and tell the user what happened - do not proceed to labeling or visualization. + +Replace INPUT_PATH with the actual path. + +### Step 4.5 - Graph health check (read-only integrity gate) + +A non-destructive diagnostic on the extraction, before labeling. It surfaces edge collapse, dangling/missing endpoints, and self-loops — the silent-corruption modes of incremental updates and AST/LLM id mismatches. Read-only; never aborts. + +```bash +$(cat graphify-out/.graphify_python) -c " +import json +from pathlib import Path +from graphify.diagnostics import diagnose_extraction, format_diagnostic_report + +extraction = json.loads(Path('graphify-out/.graphify_extract.json').read_text(encoding=\"utf-8\")) +summary = diagnose_extraction(extraction, directed=IS_DIRECTED, root='INPUT_PATH') +print(format_diagnostic_report(summary)) +flags = [f'{summary[k]} {label}' for k, label in ( + ('dangling_endpoint_edges', 'dangling-endpoint edges'), + ('missing_endpoint_edges', 'missing-endpoint edges'), + ('self_loop_edges', 'self-loop edges'), + ('directed_same_endpoint_collapsed_edges', 'collapsed (directed) edges'), + ('undirected_same_endpoint_collapsed_edges', 'collapsed (undirected) edges'), +) if summary.get(k, 0)] +print('GRAPH HEALTH WARNING: ' + '; '.join(flags) + ' - graph may be incomplete/corrupt.' if flags else 'Graph health: OK (no dangling/missing/collapsed edges).') +" +``` + +Substitute `IS_DIRECTED` and `INPUT_PATH` as in Step 4. If a `GRAPH HEALTH WARNING` prints, surface it in the final summary (do not abort — the graph is still usable, but the integrity issue must be visible, per the Honesty Rules). + +### Step 5 - Label communities + +Read `graphify-out/.graphify_analysis.json`. For each community key, look at its node labels and write a 2-5 word plain-language name (e.g. "Attention Mechanism", "Training Pipeline", "Data Loading"). + +Then regenerate the report and save the labels for the visualizer: + +```bash +$(cat graphify-out/.graphify_python) -c " +import sys, json +from graphify.build import build_from_json +from graphify.cluster import score_all +from graphify.analyze import god_nodes, surprising_connections, suggest_questions +from graphify.report import generate +from pathlib import Path + +extraction = json.loads(Path('graphify-out/.graphify_extract.json').read_text(encoding=\"utf-8\")) +detection = json.loads(Path('graphify-out/.graphify_detect.json').read_text(encoding=\"utf-8\")) +analysis = json.loads(Path('graphify-out/.graphify_analysis.json').read_text(encoding=\"utf-8\")) + +# root= as in Step 4 / the --update runbook (#1361) — same base for node-key parity. +G = build_from_json(extraction, root='INPUT_PATH', directed=IS_DIRECTED) +communities = {int(k): v for k, v in analysis['communities'].items()} +cohesion = {int(k): v for k, v in analysis['cohesion'].items()} +tokens = {'input': extraction.get('input_tokens', 0), 'output': extraction.get('output_tokens', 0)} + +# LABELS - replace these with the names you chose above +labels = LABELS_DICT + +# Regenerate questions with real community labels (labels affect question phrasing) +questions = suggest_questions(G, communities, labels) + +report = generate(G, communities, cohesion, labels, analysis['gods'], analysis['surprises'], detection, tokens, 'INPUT_PATH', suggested_questions=questions) +Path('graphify-out/GRAPH_REPORT.md').write_text(report, encoding=\"utf-8\") +Path('graphify-out/.graphify_labels.json').write_text(json.dumps({str(k): v for k, v in labels.items()}, ensure_ascii=False), encoding=\"utf-8\") +print('Report updated with community labels') +" +``` + +Replace `LABELS_DICT` with the actual dict you constructed (e.g. `{0: "Attention Mechanism", 1: "Training Pipeline"}`). +Replace INPUT_PATH with the actual path. + +### Step 6 - Generate Obsidian vault (opt-in) + HTML + +**Generate HTML always** (unless `--no-viz`). **Obsidian vault only if `--obsidian` was explicitly given** — skip it otherwise, it generates one file per node. + +If `--obsidian` was given: + +- If `--obsidian-dir ` was also given, pass it via `--dir`. Otherwise defaults to `graphify-out/obsidian`. + +```bash +graphify export obsidian +# or with custom dir: graphify export obsidian --dir ~/vaults/my-project +``` + +Generate the HTML graph (always, unless `--no-viz`): + +```bash +graphify export html # auto-aggregates to community view if graph > 5000 nodes +# or: graphify export html --no-viz +``` + +### Steps 6b-8 - Wiki, Neo4j, FalkorDB, SVG, GraphML, MCP, benchmark (only on their flags) + +These run only when their flag is present (`--wiki`, `--neo4j`/`--neo4j-push`, `--falkordb`/`--falkordb-push`, `--svg`, `--graphml`, `--mcp`) or, for the token-reduction benchmark, when `total_words` exceeds 5,000. A default run with no export flags skips all of them. See `references/exports.md` for each one. Run any `--wiki` export before Step 9 cleanup so `.graphify_labels.json` is still available. + +--- + +### Step 9 - Save manifest, update cost tracker, clean up, and report + +```bash +$(cat graphify-out/.graphify_python) -c " +import json +from pathlib import Path +from datetime import datetime, timezone +from graphify.detect import save_manifest + +# Save manifest for --update +detect = json.loads(Path('graphify-out/.graphify_detect.json').read_text(encoding=\"utf-8\")) +# In --update mode, 'all_files' carries the full corpus; 'files' is the changed +# subset. Full-rebuild mode populates only 'files', so the fallback handles that. +# root= relativizes the manifest keys to the scan root (same base as the build), +# so the on-disk manifest is portable across clones/machines and a later --update +# matches cached files instead of missing every one (#1417). +save_manifest(detect.get('all_files') or detect['files'], root='INPUT_PATH') + +# Update cumulative cost tracker +extract = json.loads(Path('graphify-out/.graphify_extract.json').read_text(encoding=\"utf-8\")) +input_tok = extract.get('input_tokens', 0) +output_tok = extract.get('output_tokens', 0) + +cost_path = Path('graphify-out/cost.json') +if cost_path.exists(): + cost = json.loads(cost_path.read_text(encoding=\"utf-8\")) +else: + cost = {'runs': [], 'total_input_tokens': 0, 'total_output_tokens': 0} + +cost['runs'].append({ + 'date': datetime.now(timezone.utc).isoformat(), + 'input_tokens': input_tok, + 'output_tokens': output_tok, + 'files': detect.get('total_files', 0), +}) +cost['total_input_tokens'] += input_tok +cost['total_output_tokens'] += output_tok +cost_path.write_text(json.dumps(cost, indent=2, ensure_ascii=False), encoding=\"utf-8\") + +print(f'This run: {input_tok:,} input tokens, {output_tok:,} output tokens') +print(f'All time: {cost[\"total_input_tokens\"]:,} input, {cost[\"total_output_tokens\"]:,} output ({len(cost[\"runs\"])} runs)') +" +rm -f graphify-out/.graphify_detect.json graphify-out/.graphify_extract.json graphify-out/.graphify_ast.json graphify-out/.graphify_semantic.json graphify-out/.graphify_analysis.json +find graphify-out -maxdepth 1 -name '.graphify_chunk_*.json' -delete 2>/dev/null +rm -f graphify-out/.needs_update 2>/dev/null || true +``` + +Replace INPUT_PATH with the actual path (same value used in Steps 4-5) so the manifest is relativized to the scan root. + +Tell the user (omit the obsidian line unless --obsidian was given): +``` +Graph complete. Outputs in PATH_TO_DIR/graphify-out/ + + graph.html - interactive graph, open in browser + GRAPH_REPORT.md - audit report + graph.json - raw graph data + obsidian/ - Obsidian vault (only if --obsidian was given) +``` + +If graphify saved you time, consider supporting it: https://github.com/sponsors/safishamsi + +Replace PATH_TO_DIR with the actual absolute path of the directory that was processed. + +Then paste these sections from GRAPH_REPORT.md directly into the chat: +- God Nodes +- Surprising Connections +- Suggested Questions + +Do NOT paste the full report - just those three sections. Keep it concise. + +Then immediately offer to explore. Pick the single most interesting suggested question from the report - the one that crosses the most community boundaries or has the most surprising bridge node - and ask: + +> "The most interesting question this graph can answer: **[question]**. Want me to trace it?" + +If the user says yes, run `/graphify query "[question]"` on the graph and walk them through the answer using the graph structure - which nodes connect, which community boundaries get crossed, what the path reveals. Keep going as long as they want to explore. Each answer should end with a natural follow-up ("this connects to X - want to go deeper?") so the session feels like navigation, not a one-shot report. + +The graph is the map. Your job after the pipeline is to be the guide. + +--- + +## Interpreter guard for subcommands + +Before running any subcommand below (`--update`, `--cluster-only`, `query`, `path`, `explain`, `add`), check that `.graphify_python` exists. If it's missing (e.g. user deleted `graphify-out/`), re-resolve the interpreter first: + +```bash +if [ ! -f graphify-out/.graphify_python ]; then + GRAPHIFY_BIN=$(which graphify 2>/dev/null) + if [ -n "$GRAPHIFY_BIN" ]; then + PYTHON=$(head -1 "$GRAPHIFY_BIN" | tr -d '#!') + case "$PYTHON" in *[!a-zA-Z0-9/_.@-]*) PYTHON="python3" ;; esac + else + PYTHON="python3" + fi + mkdir -p graphify-out + "$PYTHON" -c "import sys; open('graphify-out/.graphify_python', 'w', encoding='utf-8').write(sys.executable)" +fi +``` + +## For --update and --cluster-only + +Both are non-default subcommands. `--update` re-extracts only new or changed files; `--cluster-only` reruns clustering on the existing graph. See `references/update.md` for both flows. + +--- + +## For /graphify query + +When `graphify-out/graph.json` already exists and the user asks a question about the corpus, answer from the graph rather than rebuilding it: + +```bash +graphify query "" +``` + +Before traversal, expand the question against the graph's own vocabulary so a wording mismatch does not collapse the answer to noise. If the `graphify query` CLI is unavailable, fall back to an inline NetworkX traversal of `graphify-out/graph.json`. Answer using only what the graph output contains, and quote `source_location` when citing a specific fact. For that vocab-expansion step, the BFS/DFS traversal modes, the `--budget` cap, the NetworkX fallback, `save-result` feedback, and the `/graphify path` and `/graphify explain` flows, see `references/query.md`. + +--- + +## For /graphify add and --watch + +Neither is part of the default build. When the user runs `/graphify add ` to fetch a URL into the corpus, or passes `--watch` to auto-rebuild on file changes, see `references/add-watch.md`. + +--- + +## For the commit hook and native CLAUDE.md integration + +When the user asks to install the post-commit auto-rebuild hook or wire graphify into a project's CLAUDE.md, see `references/hooks.md`. + +--- + +## Troubleshooting + +### PowerShell 5.1: Vertical scrolling stops working + +If vertical scrolling breaks in PowerShell after running graphify, this is caused by ANSI escape sequences from the `graspologic` library. Graphify v0.3.10+ suppresses this output, but if you still see the issue: + +1. **Upgrade graphify**: `pip install --upgrade graphifyy` +2. **Use Windows Terminal** instead of the legacy PowerShell console — Windows Terminal handles ANSI codes correctly +3. **Reset your terminal**: close and reopen PowerShell +4. **Skip graspologic**: uninstall it (`pip uninstall graspologic`) and graphify will fall back to NetworkX's built-in Louvain algorithm, which produces no ANSI output + +--- + +## Honesty Rules + +- Never invent an edge. If unsure, use AMBIGUOUS. +- Never skip the corpus check warning. +- Always show token cost in the report. +- Never hide cohesion scores behind symbols - show the raw number. +- Never run HTML viz on a graph with more than 5,000 nodes without warning the user. diff --git a/.claude/skills/graphify/references/add-watch.md b/.claude/skills/graphify/references/add-watch.md new file mode 100644 index 00000000..77844343 --- /dev/null +++ b/.claude/skills/graphify/references/add-watch.md @@ -0,0 +1,56 @@ +# graphify reference: add a URL and watch a folder + +Load this when the user ran `/graphify add ` or passed `--watch`. Neither is part of the default build. + +## For /graphify add + +Fetch a URL and add it to the corpus, then update the graph. + +```bash +$(cat graphify-out/.graphify_python) -c " +import sys +from graphify.ingest import ingest +from pathlib import Path + +try: + out = ingest('URL', Path('./raw'), author='AUTHOR', contributor='CONTRIBUTOR') + print(f'Saved to {out}') +except ValueError as e: + print(f'error: {e}', file=sys.stderr) + sys.exit(1) +except RuntimeError as e: + print(f'error: {e}', file=sys.stderr) + sys.exit(1) +" +``` + +Replace `URL` with the actual URL, `AUTHOR` with the user's name if provided, `CONTRIBUTOR` likewise. If the command exits with an error, tell the user what went wrong - do not silently continue. After a successful save, automatically run the `--update` pipeline on `./raw` to merge the new file into the existing graph. + +Supported URL types (auto-detected): +- YouTube / any video URL → audio downloaded via yt-dlp, transcribed to `.txt` on next run (requires `pip install 'graphifyy[video]'`) +- Twitter/X → fetched via oEmbed, saved as `.md` with tweet text and author +- arXiv → abstract + metadata saved as `.md` +- PDF → downloaded as `.pdf` +- Images (.png/.jpg/.webp) → downloaded, Claude vision extracts on next run +- Any webpage → converted to markdown via html2text + +--- + +## For --watch + +Start a background watcher that monitors a folder and auto-updates the graph when files change. + +```bash +$(cat graphify-out/.graphify_python) -m graphify.watch INPUT_PATH --debounce 3 +``` + +Replace INPUT_PATH with the folder to watch. Behavior depends on what changed: + +- **Code files only (.py, .ts, .go, etc.):** re-runs AST extraction + rebuild + cluster immediately, no LLM needed. `graph.json` and `GRAPH_REPORT.md` are updated automatically. +- **Docs, papers, or images:** writes a `graphify-out/needs_update` flag and prints a notification to run `/graphify --update` (LLM semantic re-extraction required). + +Debounce (default 3s): waits until file activity stops before triggering, so a wave of parallel agent writes doesn't trigger a rebuild per file. + +Press Ctrl+C to stop. + +For agentic workflows: run `--watch` in a background terminal. Code changes from agent waves are picked up automatically between waves. If agents are also writing docs or notes, you'll need a manual `/graphify --update` after those waves. diff --git a/.claude/skills/graphify/references/exports.md b/.claude/skills/graphify/references/exports.md new file mode 100644 index 00000000..242ff868 --- /dev/null +++ b/.claude/skills/graphify/references/exports.md @@ -0,0 +1,87 @@ +# graphify reference: extra exports and benchmark + +Load this when the user passed one of the export flags (`--wiki`, `--neo4j`, `--neo4j-push`, `--falkordb`, `--falkordb-push`, `--svg`, `--graphml`, `--mcp`), or when the corpus is large enough for the token-reduction benchmark. Each step runs only for its own flag. + +### Step 6b - Wiki (only if --wiki flag) + +**Only run this step if `--wiki` was explicitly given in the original command.** + +Run this before Step 9 (cleanup) so `.graphify_labels.json` is still available. + +```bash +graphify export wiki +``` + +### Step 7 - Neo4j export (only if --neo4j or --neo4j-push flag) + +**If `--neo4j`** - generate a Cypher file for manual import: + +```bash +graphify export neo4j +``` + +**If `--neo4j-push `** - push directly to a running Neo4j instance. Ask the user for credentials if not provided: + +```bash +graphify export neo4j --push bolt://localhost:7687 --user neo4j --password PASSWORD +``` + +Default URI is `bolt://localhost:7687`, default user is `neo4j`. Uses MERGE - safe to re-run without creating duplicates. + +### Step 7a - FalkorDB export (only if --falkordb or --falkordb-push flag) + +**If `--falkordb`** - generate a Cypher file. The statements are OpenCypher, but FalkorDB's `GRAPH.QUERY` runs one statement at a time (no bulk script import like Neo4j's `cypher-shell`), so prefer `--falkordb-push` to load a graph. Use this only when you want the portable `cypher.txt` artifact: + +```bash +graphify export falkordb +``` + +**If `--falkordb-push `** - push directly to a running FalkorDB instance. Credentials are optional; ask the user only if the instance requires auth: + +```bash +graphify export falkordb --push falkordb://localhost:6379 +``` + +Default URI is `falkordb://localhost:6379` (the scheme is informational - `redis://` or a bare `host:port` work too), auth is optional, and the target graph defaults to `graphify`. Uses MERGE - safe to re-run without creating duplicates. + +### Step 7b - SVG export (only if --svg flag) + +```bash +graphify export svg +``` + +### Step 7c - GraphML export (only if --graphml flag) + +```bash +graphify export graphml +``` + +### Step 7d - MCP server (only if --mcp flag) + +```bash +$(cat graphify-out/.graphify_python) -m graphify.serve graphify-out/graph.json +``` + +This starts a stdio MCP server that exposes tools: `query_graph`, `get_node`, `get_neighbors`, `get_community`, `god_nodes`, `graph_stats`, `shortest_path`. Add to Claude Desktop or any MCP-compatible agent orchestrator so other agents can query the graph live. + +To configure in Claude Desktop, add to `claude_desktop_config.json`. Claude Desktop can't run `$(...)`, and under `uv tool install` the system `python3` can't import graphify — so set `command` to the **absolute interpreter path** printed by `cat graphify-out/.graphify_python`: +```json +{ + "mcpServers": { + "graphify": { + "command": "", + "args": ["-m", "graphify.serve", "/absolute/path/to/graphify-out/graph.json"] + } + } +} +``` + +### Step 8 - Token reduction benchmark (only if total_words > 5000) + +If `total_words` from `graphify-out/.graphify_detect.json` is greater than 5,000, run: + +```bash +graphify benchmark +``` + +Print the output directly in chat. If `total_words <= 5000`, skip silently - the graph value is structural clarity, not token compression, for small corpora. diff --git a/.claude/skills/graphify/references/extraction-spec.md b/.claude/skills/graphify/references/extraction-spec.md new file mode 100644 index 00000000..388df767 --- /dev/null +++ b/.claude/skills/graphify/references/extraction-spec.md @@ -0,0 +1,70 @@ +# graphify reference: extraction subagent prompt + +Load this in Step 3 Part B when the corpus has at least one doc, paper, or image chunk. A pure-code corpus skips Part B and never reads this file. Each semantic subagent receives the prompt below verbatim (substitute FILE_LIST, CHUNK_NUM, TOTAL_CHUNKS, DEEP_MODE, and CHUNK_PATH). + +``` +You are a graphify extraction subagent. Read the files listed and extract a knowledge graph fragment. +Output ONLY valid JSON matching the schema below - no explanation, no markdown fences, no preamble. + +Files (chunk CHUNK_NUM of TOTAL_CHUNKS): +FILE_LIST + +Rules: +- EXTRACTED: relationship explicit in source (import, call, citation, "see §3.2") +- INFERRED: reasonable inference (shared data structure, implied dependency) +- AMBIGUOUS: uncertain - flag for review, do not omit + +Code files: focus on semantic edges AST cannot find (call relationships, shared data, arch patterns). + Do not re-extract imports - AST already has those. +Doc/paper files: extract named concepts, entities, citations. For rationale (WHY decisions were made, trade-offs, design intent): store as a `rationale` attribute on the relevant concept node — do NOT create a separate rationale node or fragment node. Only create a node for something that is itself a named entity or concept. Use `file_type:"rationale"` for concept-like nodes (ideas, principles, mechanisms, design patterns). `file_type` MUST be one of exactly these six values: `code`, `document`, `paper`, `image`, `rationale`, `concept`. Any other value is invalid and will be rejected. +Code files: when adding `calls` edges, source MUST be the caller (the function/class doing the calling), target MUST be the callee. Never reverse this direction. `calls` edges MUST stay within one language: a Python function cannot `calls` a JS/TS/Go/Rust/Java symbol and vice versa — cross-language call edges are phantom artifacts, never emit them. +Image files: use vision to understand what the image IS - do not just OCR. + UI screenshot: layout patterns, design decisions, key elements, purpose. + Chart: metric, trend/insight, data source. + Tweet/post: claim as node, author, concepts mentioned. + Diagram: components and connections. + Research figure: what it demonstrates, method, result. + Handwritten/whiteboard: ideas and arrows, mark uncertain readings AMBIGUOUS. + +DEEP_MODE (if --mode deep was given): be aggressive with INFERRED edges - indirect deps, + shared assumptions, latent couplings. Mark uncertain ones AMBIGUOUS instead of omitting. + +Semantic similarity: if two concepts in this chunk solve the same problem or represent the same idea without any structural link (no import, no call, no citation), add a `semantically_similar_to` edge marked INFERRED with a confidence_score reflecting how similar they are (0.6-0.95). Examples: +- Two functions that both validate user input but never call each other +- A class in code and a concept in a paper that describe the same algorithm +- Two error types that handle the same failure mode differently +Only add these when the similarity is genuinely non-obvious and cross-cutting. Do not add them for trivially similar things. + +Hyperedges: if 3 or more nodes clearly participate together in a shared concept, flow, or pattern that is not captured by pairwise edges alone, add a hyperedge to a top-level `hyperedges` array. Examples: +- All classes that implement a common protocol or interface +- All functions in an authentication flow (even if they don't all call each other) +- All concepts from a paper section that form one coherent idea +Use sparingly — only when the group relationship adds information beyond the pairwise edges. Maximum 3 hyperedges per chunk. + +If a file has YAML frontmatter (--- ... ---), copy source_url, captured_at, author, + contributor onto every node from that file. + +confidence_score is REQUIRED on every edge - never omit it, never use 0.5 as a default: +- EXTRACTED edges: confidence_score = 1.0 always +- INFERRED edges: pick exactly ONE value from this set — never 0.5: + 0.95 direct structural evidence (shared data structure, named cross-file reference). + 0.85 strong inference (clear functional alignment, no direct symbol link). + 0.75 reasonable inference (shared problem domain + similar shape, requires interpretation). + 0.65 weak inference (thematically related, no shape evidence). + 0.55 speculative but plausible (surface-level co-occurrence only). + Models follow discrete rubrics better than continuous ranges; the bimodal + distribution observed in production (>50% at 0.5, >40% at 0.85+) shows the + range guidance is being collapsed to a binary. If no value above fits, mark + the edge AMBIGUOUS rather than picking 0.4 or below. +- AMBIGUOUS edges: 0.1-0.3 + +Node ID format: lowercase, only `[a-z0-9_]`, no dots or slashes. Format: `{stem}_{entity}` where stem is the **full repo-relative path with the extension dropped**, every path segment kept and joined with `_` (each segment lowercased with non-alphanumeric chars replaced by `_`), and entity is the symbol name similarly normalized. Use every directory level, not just the immediate parent — this keeps same-named files in different directories distinct. Examples: `src/auth/session.py` + `ValidateToken` → `src_auth_session_validatetoken`; `lib/utils/helpers.py` + `parse_url` → `lib_utils_helpers_parse_url`; `tests/test_foo.py` + `_helper` → `tests_test_foo_helper`; `docs/v1/api/README.md` + `getUser` → `docs_v1_api_readme_getuser`. Top-level files (no parent dir, e.g. `setup.py`) use just the filename stem: `setup_my_func`. This must match the ID the AST extractor generates — using just the filename (e.g., `session_validatetoken`) or only the immediate parent (e.g., `auth_session_validatetoken`) will create orphan ghost-duplicate nodes. If you are re-extracting a project built under the old immediate-parent format, the user should run `graphify extract --force` to rebuild cleanly. CRITICAL: never append chunk numbers, sequence numbers, or any suffix to an ID (no `_c1`, `_c2`, `_chunk2`, etc.). IDs must be deterministic from the label alone — the same entity must always produce the same ID regardless of which chunk processes it. + +Generate the extraction JSON matching this schema exactly: +{"nodes":[{"id":"auth_session_validatetoken","label":"Human Readable Name","file_type":"code|document|paper|image|rationale|concept","source_file":"","source_location":null,"source_url":null,"captured_at":null,"author":null,"contributor":null}],"edges":[{"source":"node_id","target":"node_id","relation":"calls|implements|references|cites|conceptually_related_to|shares_data_with|semantically_similar_to|rationale_for","confidence":"EXTRACTED|INFERRED|AMBIGUOUS","confidence_score":1.0,"source_file":"","source_location":null,"weight":1.0}],"hyperedges":[{"id":"snake_case_id","label":"Human Readable Label","nodes":["node_id1","node_id2","node_id3"],"relation":"participate_in|implement|form","confidence":"EXTRACTED|INFERRED","confidence_score":0.75,"source_file":""}],"input_tokens":0,"output_tokens":0} + +source_file RULE (every node, edge, and hyperedge): set source_file to the path of the originating file EXACTLY as it appears in FILE_LIST — verbatim and absolute. Do NOT shorten to a basename, do NOT re-relativize, do NOT strip any directory prefix, and do NOT change separators (the engine canonicalizes separators and relativizes against the build root downstream). Copy the FILE_LIST entry character-for-character. This keeps the full build and incremental --update on the same base, so build_merge's replace-on-re-extract matches the existing node instead of accumulating a duplicate. + +Then write the JSON to disk using the Write tool at this exact absolute path (no relative paths — Write resolves relative paths against an undefined cwd and the file will be silently lost): +CHUNK_PATH +``` diff --git a/.claude/skills/graphify/references/github-and-merge.md b/.claude/skills/graphify/references/github-and-merge.md new file mode 100644 index 00000000..a41ea06e --- /dev/null +++ b/.claude/skills/graphify/references/github-and-merge.md @@ -0,0 +1,46 @@ +# graphify reference: GitHub clone and cross-repo merge + +Load this when the user passed one or more `https://github.com/...` URLs, or named several local subfolders to merge into one graph. + +### Step 0 - Clone GitHub repo(s) (only if a GitHub URL was given) + +**Single repo:** +```bash +LOCAL_PATH=$(graphify clone [--branch ]) +# Use LOCAL_PATH as the target for all subsequent steps +``` + +**Multiple repos (cross-repo graph):** +```bash +# Clone each repo, run the full pipeline on each, then merge +graphify clone # → ~/.graphify/repos// +graphify clone # → ~/.graphify/repos// +# Run /graphify on each local path to produce their graph.json files +# Then merge: +graphify merge-graphs \ + ~/.graphify/repos///graphify-out/graph.json \ + ~/.graphify/repos///graphify-out/graph.json \ + --out graphify-out/cross-repo-graph.json +``` + +Graphify clones into `~/.graphify/repos//` and reuses existing clones on repeat runs. Each node in the merged graph carries a `repo` attribute so you can filter by origin. + +**Multiple local subfolders (monorepo or multi-service layout):** + +The skill pipeline writes all intermediate and final outputs to `graphify-out/` in the current working directory. Running the skill on each subfolder separately will clobber the same output dir. Instead, use the CLI directly for each subfolder — it places `graphify-out/` *inside* the scanned path: + +```bash +graphify extract ./core/ # → ./core/graphify-out/graph.json +graphify extract ./service/ # → ./service/graphify-out/graph.json +graphify extract ./platform/ # → ./platform/graphify-out/graph.json +# Add --backend gemini|kimi|openai|deepseek|claude-cli depending on which API key you have set + +# Then merge at the project root: +graphify merge-graphs \ + ./core/graphify-out/graph.json \ + ./service/graphify-out/graph.json \ + ./platform/graphify-out/graph.json \ + --out graphify-out/graph.json +``` + +Once `graphify-out/graph.json` exists, the fast path above takes over: any codebase question runs `graphify query` directly on the merged graph — no re-extraction, no size gate. diff --git a/.claude/skills/graphify/references/hooks.md b/.claude/skills/graphify/references/hooks.md new file mode 100644 index 00000000..438b8b16 --- /dev/null +++ b/.claude/skills/graphify/references/hooks.md @@ -0,0 +1,33 @@ +# graphify reference: commit hook and native CLAUDE.md integration + +Load this when the user asked to install the post-commit hook or wire graphify into a project's CLAUDE.md. + +## For git commit hook + +Install a post-commit hook that auto-rebuilds the graph after every commit. No background process needed - triggers once per commit, works with any editor. + +```bash +graphify hook install # install +graphify hook uninstall # remove +graphify hook status # check +``` + +After every `git commit`, the hook detects which code files changed (via `git diff HEAD~1`), re-runs AST extraction on those files, and rebuilds `graph.json` and `GRAPH_REPORT.md`. Doc/image changes are ignored by the hook - run `/graphify --update` manually for those. + +If a post-commit hook already exists, graphify appends to it rather than replacing it. + +--- + +## For native CLAUDE.md integration + +Run once per project to make graphify always-on in Claude Code sessions: + +```bash +graphify claude install +``` + +This writes a `## graphify` section to the local `CLAUDE.md` that instructs Claude to check the graph before answering codebase questions and rebuild it after code changes. No manual `/graphify` needed in future sessions. + +```bash +graphify claude uninstall # remove the section +``` diff --git a/.claude/skills/graphify/references/query.md b/.claude/skills/graphify/references/query.md new file mode 100644 index 00000000..52321f10 --- /dev/null +++ b/.claude/skills/graphify/references/query.md @@ -0,0 +1,311 @@ +# graphify reference: query, path, explain + +Load this when the user asks a question against an existing graph, or runs `/graphify path` or `/graphify explain`. The core's query stub points here for the full traversal flow. These flows use the `graphify query` CLI when it is available and fall back to an inline NetworkX traversal otherwise. + +Two traversal modes - choose based on the question: + +| Mode | Flag | Best for | +|------|------|----------| +| BFS (default) | _(none)_ | "What is X connected to?" - broad context, nearest neighbors first | +| DFS | `--dfs` | "How does X reach Y?" - trace a specific chain or dependency path | + +First check the graph exists: +```bash +$(cat graphify-out/.graphify_python) -c " +from pathlib import Path +if not Path('graphify-out/graph.json').exists(): + print('ERROR: No graph found. Run /graphify first to build the graph.') + raise SystemExit(1) +" +``` +If it fails, stop and tell the user to run `/graphify ` first. + +### Step 0 — Constrained query expansion (REQUIRED before traversal) + +graphify's `query` CLI matches nodes via case-folded substring + IDF — there is **no stemming, no synonyms, no cross-language match** inside the binary, and the inline fallback below matches the same way. If the user's question uses different language or different domain vocabulary than the graph's labels (user says "обработчик" / graph says "handler"; user says "authentication" / graph says "Guardian"), the literal matcher returns 0 hits and the answer collapses to noise. + +Fix this **without inventing tokens** by expanding the query against the actual graph vocabulary first: + +1. Extract the token vocabulary from node labels: +```bash +$(cat graphify-out/.graphify_python) -c " +import json, re +from pathlib import Path +data = json.loads(Path('graphify-out/graph.json').read_text()) +vocab = set() +for n in data['nodes']: + for c in re.findall(r'[^\W\d_]+', n.get('label','') or '', re.UNICODE): + parts = re.findall(r'[A-Z]+(?=[A-Z][a-z])|[A-Z]?[a-z]+|[A-Z]+', c) or [c] + for p in parts: + t = p.lower() + if 3 <= len(t) <= 30: + vocab.add(t) +Path('graphify-out/.vocab.txt').write_text('\n'.join(sorted(vocab))) +print(f'vocab: {len(vocab)} tokens') +" +``` + +2. Read `graphify-out/.vocab.txt`. Then for the user's question, select **up to 12 tokens from this exact list** that semantically match the query intent. Hard constraints: + - You MUST pick only tokens present in the vocabulary file. Do NOT invent tokens. + - If a query concept has no plausible token in the vocab, skip it — do not substitute a near-synonym from training memory. + - If **no** vocab tokens match the query at all, output an empty list and tell the user the corpus has no relevant vocabulary for this question. Do not fabricate a search. + - Translate cross-language: Russian "аутентификация" → look for `auth`, `credential`, `token`, `security` IFF present in vocab. + - Morphology: "handlers" maps to `handler` IFF present; "todos" maps to `todo` IFF present. + +3. Print the selection explicitly to the user before running the query, so the expansion is auditable: +``` +Query expanded to (from graph vocab, N tokens): [token1, token2, ...] +``` +If the list is empty, say so plainly and stop — do not proceed to traversal. + +### Step 1 — Traversal + +Build the **expanded query string** by joining the selected tokens with spaces. Use this string as `QUESTION` below — NOT the original user question. (The original question is preserved only for `save-result` at the end.) + +Prefer the CLI when it is installed: +```bash +graphify query "QUESTION" +# or: graphify query "QUESTION" --dfs --budget 3000 +``` + +If the CLI is unavailable, load `graphify-out/graph.json` and run the traversal inline: + +1. Find the 1-3 nodes whose label best matches the expanded tokens. +2. Run the appropriate traversal from each starting node. +3. Read the subgraph - node labels, edge relations, confidence tags, source locations. +4. Answer using **only** what the graph contains. Quote `source_location` when citing a specific fact. +5. If the graph lacks enough information, say so - do not hallucinate edges. + +```bash +$(cat graphify-out/.graphify_python) -c " +import sys, json +from networkx.readwrite import json_graph +import networkx as nx +from pathlib import Path + +data = json.loads(Path('graphify-out/graph.json').read_text()) +G = json_graph.node_link_graph(data, edges='links') + +question = 'QUESTION' +mode = 'MODE' # 'bfs' or 'dfs' +terms = [t.lower() for t in question.split() if len(t) >= 3] # match the vocab threshold; keeps api/jwt/ios (#1392) + +# Find best-matching start nodes +scored = [] +for nid, ndata in G.nodes(data=True): + label = ndata.get('label', '').lower() + score = sum(1 for t in terms if t in label) + if score > 0: + scored.append((score, nid)) +scored.sort(reverse=True) +start_nodes = [nid for _, nid in scored[:3]] + +if not start_nodes: + print('No matching nodes found for query terms:', terms) + sys.exit(0) + +subgraph_nodes = set() +subgraph_edges = [] + +if mode == 'dfs': + # DFS: follow one path as deep as possible before backtracking. + # Depth-limited to 6 to avoid traversing the whole graph. + visited = set() + stack = [(n, 0) for n in reversed(start_nodes)] + while stack: + node, depth = stack.pop() + if node in visited or depth > 6: + continue + visited.add(node) + subgraph_nodes.add(node) + for neighbor in G.neighbors(node): + if neighbor not in visited: + stack.append((neighbor, depth + 1)) + subgraph_edges.append((node, neighbor)) +else: + # BFS: explore all neighbors layer by layer up to depth 3. + frontier = set(start_nodes) + subgraph_nodes = set(start_nodes) + for _ in range(3): + next_frontier = set() + for n in frontier: + for neighbor in G.neighbors(n): + if neighbor not in subgraph_nodes: + next_frontier.add(neighbor) + subgraph_edges.append((n, neighbor)) + subgraph_nodes.update(next_frontier) + frontier = next_frontier + +# Token-budget aware output: rank by relevance, cut at budget (~4 chars/token) +token_budget = BUDGET # default 2000 +char_budget = token_budget * 4 + +# Score each node by term overlap for ranked output +def relevance(nid): + label = G.nodes[nid].get('label', '').lower() + return sum(1 for t in terms if t in label) + +ranked_nodes = sorted(subgraph_nodes, key=relevance, reverse=True) + +lines = [f'Traversal: {mode.upper()} | Start: {[G.nodes[n].get(\"label\",n) for n in start_nodes]} | {len(subgraph_nodes)} nodes'] +for nid in ranked_nodes: + d = G.nodes[nid] + lines.append(f' NODE {d.get(\"label\", nid)} [src={d.get(\"source_file\",\"\")} loc={d.get(\"source_location\",\"\")}]') +for u, v in subgraph_edges: + if u in subgraph_nodes and v in subgraph_nodes: + _raw = G[u][v]; d = next(iter(_raw.values()), {}) if isinstance(G, nx.MultiGraph) else _raw + lines.append(f' EDGE {G.nodes[u].get(\"label\",u)} --{d.get(\"relation\",\"\")} [{d.get(\"confidence\",\"\")}]--> {G.nodes[v].get(\"label\",v)}') + +output = '\n'.join(lines) +if len(output) > char_budget: + output = output[:char_budget] + f'\n... (truncated at ~{token_budget} token budget - use --budget N for more)' +print(output) +" +``` + +Replace `QUESTION` with the **expanded** query string, `MODE` with `bfs` or `dfs`, and `BUDGET` with the token budget (default `2000`, or whatever `--budget N` specifies). Then answer based on the subgraph output above, using only what the graph contains. + +After writing the answer, save it back into the graph so it improves future queries. Include the expanded tokens inside the `--answer` text (e.g. `"Expanded from original query via vocab: [tokens]. Then traversed..."`) so the next `--update` extracts the expansion history as a graph node: + +```bash +$(cat graphify-out/.graphify_python) -m graphify save-result --question "ORIGINAL_QUESTION" --answer "ANSWER" --type query --nodes NODE1 NODE2 +``` + +Replace `ORIGINAL_QUESTION` with the user's verbatim question, `ANSWER` with your full answer text (containing the expanded-token trace), `NODE1 NODE2` with the list of node labels you cited. This closes the feedback loop: the next `--update` will extract this Q&A as a node in the graph. + +**Work memory (self-improving loop).** Add an `--outcome` so future sessions learn from this one — append `--outcome useful|dead_end|corrected` to the `save-result` command (and `--correction "the right answer"` when correcting): + +- `useful` — the cited nodes answered the question well (they become *preferred sources*). +- `dead_end` — the question/path led nowhere; don't re-derive it next time. +- `corrected` — the saved answer was wrong; `--correction` records what was right. + +At the **start** of graph work, refresh and read the lessons: run `graphify reflect --if-stale` (cheap, deterministic, no LLM; `--if-stale` makes it a no-op when `LESSONS.md` is already newer than every input, e.g. when the git hook just refreshed it), then read `graphify-out/reflections/LESSONS.md`. It lists **preferred sources** (start there), **known dead ends** (skip them), and prior **corrections**. Running `reflect` yourself keeps the lessons current even without the git hook installed; if the post-commit hook *is* installed, `--if-stale` means your session-start run costs almost nothing. + +--- + +## For /graphify path + +Find the shortest path between two named concepts in the graph. Prefer the CLI when installed: + +```bash +graphify path "NODE_A" "NODE_B" +``` + +If the CLI is unavailable, run it inline: + +```bash +$(cat graphify-out/.graphify_python) -c " +import json, sys +import networkx as nx +from networkx.readwrite import json_graph +from pathlib import Path + +data = json.loads(Path('graphify-out/graph.json').read_text()) +G = json_graph.node_link_graph(data, edges='links') + +a_term = 'NODE_A' +b_term = 'NODE_B' + +def find_node(term): + term = term.lower() + scored = sorted( + [(sum(1 for w in term.split() if w in G.nodes[n].get('label','').lower()), n) + for n in G.nodes()], + reverse=True + ) + return scored[0][1] if scored and scored[0][0] > 0 else None + +src = find_node(a_term) +tgt = find_node(b_term) + +if not src or not tgt: + print(f'Could not find nodes matching: {a_term!r} or {b_term!r}') + sys.exit(0) + +try: + path = nx.shortest_path(G, src, tgt) + print(f'Shortest path ({len(path)-1} hops):') + for i, nid in enumerate(path): + label = G.nodes[nid].get('label', nid) + if i < len(path) - 1: + _raw = G[nid][path[i+1]]; edge = next(iter(_raw.values()), {}) if isinstance(G, nx.MultiGraph) else _raw + rel = edge.get('relation', '') + conf = edge.get('confidence', '') + print(f' {label} --{rel}--> [{conf}]') + else: + print(f' {label}') +except nx.NetworkXNoPath: + print(f'No path found between {a_term!r} and {b_term!r}') +except nx.NodeNotFound as e: + print(f'Node not found: {e}') +" +``` + +Replace `NODE_A` and `NODE_B` with the actual concept names from the user. Then explain the path in plain language - what each hop means, why it's significant. + +After writing the explanation, save it back: + +```bash +$(cat graphify-out/.graphify_python) -m graphify save-result --question "Path from NODE_A to NODE_B" --answer "ANSWER" --type path_query --nodes NODE_A NODE_B +``` + +--- + +## For /graphify explain + +Give a plain-language explanation of a single node - everything connected to it. Prefer the CLI when installed: + +```bash +graphify explain "NODE_NAME" +``` + +If the CLI is unavailable, run it inline: + +```bash +$(cat graphify-out/.graphify_python) -c " +import json, sys +import networkx as nx +from networkx.readwrite import json_graph +from pathlib import Path + +data = json.loads(Path('graphify-out/graph.json').read_text()) +G = json_graph.node_link_graph(data, edges='links') + +term = 'NODE_NAME' +term_lower = term.lower() + +# Find best matching node +scored = sorted( + [(sum(1 for w in term_lower.split() if w in G.nodes[n].get('label','').lower()), n) + for n in G.nodes()], + reverse=True +) +if not scored or scored[0][0] == 0: + print(f'No node matching {term!r}') + sys.exit(0) + +nid = scored[0][1] +data_n = G.nodes[nid] +print(f'NODE: {data_n.get(\"label\", nid)}') +print(f' source: {data_n.get(\"source_file\",\"unknown\")}') +print(f' type: {data_n.get(\"file_type\",\"unknown\")}') +print(f' degree: {G.degree(nid)}') +print() +print('CONNECTIONS:') +for neighbor in G.neighbors(nid): + _raw = G[nid][neighbor]; edge = next(iter(_raw.values()), {}) if isinstance(G, nx.MultiGraph) else _raw + nlabel = G.nodes[neighbor].get('label', neighbor) + rel = edge.get('relation', '') + conf = edge.get('confidence', '') + src_file = G.nodes[neighbor].get('source_file', '') + print(f' --{rel}--> {nlabel} [{conf}] ({src_file})') +" +``` + +Replace `NODE_NAME` with the concept the user asked about. Then write a 3-5 sentence explanation of what this node is, what it connects to, and why those connections are significant. Use the source locations as citations. + +After writing the explanation, save it back: + +```bash +$(cat graphify-out/.graphify_python) -m graphify save-result --question "Explain NODE_NAME" --answer "ANSWER" --type explain --nodes NODE_NAME +``` diff --git a/.claude/skills/graphify/references/transcribe.md b/.claude/skills/graphify/references/transcribe.md new file mode 100644 index 00000000..b967f837 --- /dev/null +++ b/.claude/skills/graphify/references/transcribe.md @@ -0,0 +1,52 @@ +# graphify reference: transcribe video and audio + +Load this only when `detect` reported one or more `video` files. A corpus with no video never reads this. + +### Step 2.5 - Transcribe video / audio files (only if video files detected) + +Skip this step entirely if `detect` returned zero `video` files. + +Video and audio files cannot be read directly. Transcribe them to text first, then treat the transcripts as doc files in Step 3. + +**Strategy:** Read the god nodes from `graphify-out/.graphify_detect.json` (or the analysis file if it exists from a previous run). You are already a language model — write a one-sentence domain hint yourself from those labels. Then pass it to Whisper as the initial prompt. No separate API call needed. + +**However**, if the corpus has *only* video files and no other docs/code, use the generic fallback prompt: `"Use proper punctuation and paragraph breaks."` + +**Step 1 - Write the Whisper prompt yourself.** + +Read the top god node labels from detect output or analysis, then compose a short domain hint sentence, for example: + +- Labels: `transformer, attention, encoder, decoder` → `"Machine learning research on transformer architectures and attention mechanisms. Use proper punctuation and paragraph breaks."` +- Labels: `kubernetes, deployment, pod, helm` → `"DevOps discussion about Kubernetes deployments and Helm charts. Use proper punctuation and paragraph breaks."` + +**Export** it as `GRAPHIFY_WHISPER_PROMPT` (the exact name the transcriber reads — and it must be `export`ed so the child Python process sees it) for the next command. + +**Step 2 - Transcribe:** + +```bash +export GRAPHIFY_WHISPER_MODEL=base # or whatever --whisper-model the user passed (must be exported) +export GRAPHIFY_WHISPER_PROMPT="" +$(cat graphify-out/.graphify_python) -c " +import json, os, sys +from pathlib import Path +from graphify.transcribe import transcribe_all + +detect = json.loads(Path('graphify-out/.graphify_detect.json').read_text(encoding=\"utf-8\")) +video_files = detect.get('files', {}).get('video', []) +prompt = os.environ.get('GRAPHIFY_WHISPER_PROMPT', 'Use proper punctuation and paragraph breaks.') + +transcript_paths = transcribe_all(video_files, initial_prompt=prompt) +# Write the JSON from Python (NOT a shell '>' redirect): transcribe_all/Whisper +# print progress to stdout, which would otherwise corrupt the JSON file (#1392). +Path('graphify-out/.graphify_transcripts.json').write_text(json.dumps(transcript_paths, ensure_ascii=False), encoding=\"utf-8\") +print(f'Transcribed {len(transcript_paths)} file(s)', file=sys.stderr) +" +``` + +After transcription: +- Read the transcript paths from `graphify-out/.graphify_transcripts.json` +- Add them to the docs list before dispatching semantic subagents in Step 3B +- Print how many transcripts were created: `Transcribed N video file(s) -> treating as docs` +- If transcription fails for a file, print a warning and continue with the rest + +**Whisper model:** Default is `base`. If the user passed `--whisper-model `, `export GRAPHIFY_WHISPER_MODEL=` (it must be exported, not just assigned) before running the command above. diff --git a/.claude/skills/graphify/references/update.md b/.claude/skills/graphify/references/update.md new file mode 100644 index 00000000..fa261218 --- /dev/null +++ b/.claude/skills/graphify/references/update.md @@ -0,0 +1,192 @@ +# graphify reference: incremental update and cluster-only + +Load this only when the user passed `--update` or `--cluster-only`. A first-time full build never reads this file. + +## For --update (incremental re-extraction) + +Use when you've added or modified files since the last run. Only re-extracts changed files - saves tokens and time. + +```bash +$(cat graphify-out/.graphify_python) -c " +import sys, json +from graphify.detect import detect_incremental, save_manifest +from pathlib import Path + +result = detect_incremental(Path('INPUT_PATH')) +new_total = result.get('new_total', 0) +print(json.dumps(result, indent=2, ensure_ascii=False)) +Path('graphify-out/.graphify_incremental.json').write_text(json.dumps(result, ensure_ascii=False), encoding=\"utf-8\") +deleted = list(result.get('deleted_files', [])) +if new_total == 0 and not deleted: + print('No files changed since last run. Nothing to update.') + raise SystemExit(0) +if deleted: + print(f'{len(deleted)} deleted file(s) to prune.') +if new_total > 0: + print(f'{new_total} new/changed file(s) to re-extract.') +" +``` + +Then populate `.graphify_detect.json` so Steps 3A–6 (which read it unconditionally) see the right state for an incremental run. `files` carries the changed subset (drives Step 3A AST + Step 3B0 cache check on only what changed); `all_files` carries the full corpus for any step that needs corpus-wide context: + +```bash +$(cat graphify-out/.graphify_python) -c " +import json +from pathlib import Path +r = json.loads(Path('graphify-out/.graphify_incremental.json').read_text(encoding=\"utf-8\")) +Path('graphify-out/.graphify_detect.json').write_text(json.dumps({ + 'files': r.get('new_files', {}), + 'all_files': r.get('files', {}), + 'total_files': r.get('new_total', 0), + 'total_words': r.get('total_words', 0), + 'skipped_sensitive': r.get('skipped_sensitive', []), + 'needs_graph': True, +}, ensure_ascii=False), encoding=\"utf-8\") +" +``` + +If new files exist, first check whether all changed files are code files: + +```bash +$(cat graphify-out/.graphify_python) -c " +import json +from pathlib import Path + +result = json.loads(open('graphify-out/.graphify_incremental.json', encoding='utf-8').read()) if Path('graphify-out/.graphify_incremental.json').exists() else {} +code_exts = {'.py','.ts','.js','.go','.rs','.java','.cpp','.c','.rb','.swift','.kt','.cs','.scala','.php','.cc','.cxx','.hpp','.h','.kts','.lua','.toc','.f','.F','.f90','.F90','.f95','.F95','.f03','.F03','.f08','.F08'} +new_files = result.get('new_files', {}) +all_changed = [f for files in new_files.values() for f in files] +code_only = all(Path(f).suffix.lower() in code_exts for f in all_changed) +print('code_only:', code_only) +" +``` + +If `code_only` is True: print `[graphify update] Code-only changes detected - skipping semantic extraction (no LLM needed)`, run only Step 3A (AST) on the changed files, skip Step 3B entirely (no subagents), then go straight to merge and Steps 4–8. + +If `code_only` is False (any changed file is a doc/paper/image/video): **first, if any changed file is in `new_files['video']`, run `references/transcribe.md` (Step 2.5) on those files, then rewrite `.graphify_detect.json` to move the resulting transcript paths into `files['document']` and drop `files['video']`** — otherwise raw `.mp4/.mp3` paths are fed to semantic subagents as unreadable media (#1392). Then run the full Steps 3A–3C pipeline as normal. + + +If no new files exist (only deletions), create an empty extraction so the merge step can prune: + +```bash +if [ ! -f graphify-out/.graphify_extract.json ]; then + echo '[graphify update] Only deletions -- creating empty extraction for merge.' + $(cat graphify-out/.graphify_python) -c " +import json +from pathlib import Path +Path('graphify-out/.graphify_extract.json').write_text(json.dumps({'nodes':[],'edges':[],'hyperedges':[],'input_tokens':0,'output_tokens':0}), encoding='utf-8') +" +fi +``` + + +Then: + +```bash +$(cat graphify-out/.graphify_python) -c " +import json +from pathlib import Path +from graphify.build import build_merge +from graphify.detect import save_manifest + +# Load new extraction and incremental state +new_extraction = json.loads(Path('graphify-out/.graphify_extract.json').read_text(encoding=\"utf-8\")) +incremental = json.loads(Path('graphify-out/.graphify_incremental.json').read_text(encoding=\"utf-8\")) +deleted = list(incremental.get('deleted_files', [])) +# prune_sources is ONLY for genuinely DELETED files. Changed/re-extracted files are +# handled by build_merge's replace-on-re-extract (#1344): every source_file in +# new_chunks is dropped from the base before merge, so old/stale nodes don't survive. +# Do NOT add `changed` here: with root= passed, prune_set relativizes to the same base +# as the freshly merged nodes and would DELETE the re-extracted content (#1178 is moot +# now that replace — not the dedup pass — reconciles changed files). +prune = list(deleted) or None + +# Use build_merge() — reads graph.json directly without NetworkX round-trip +# so edge direction (calls, implements, imports) is always preserved (#801). +# Pass root= so prune_sources (absolute paths from detect_incremental) are +# relativized to match the graph's relative source_file values; without it +# nothing is pruned and stale nodes accumulate on every update (#1361). +# directed=IS_DIRECTED: replace IS_DIRECTED with True if --directed was given, else +# False. Without it a --directed --update silently rebuilds undirected and collapses +# reciprocal A<->B edges (#1392). +G = build_merge( + [new_extraction], + graph_path='graphify-out/graph.json', + prune_sources=prune, + root='INPUT_PATH', + directed=IS_DIRECTED, +) +print(f'[graphify update] Merged: {G.number_of_nodes()} nodes, {G.number_of_edges()} edges') + +# Write merged result back to .graphify_extract.json so Step 4 sees the full graph +merged_out = { + 'nodes': [{'id': n, **d} for n, d in G.nodes(data=True)], + 'edges': [ + # Explicit source/target last so they win over any stale attrs in d. + {**{k: val for k, val in d.items() if k not in ('_src', '_tgt', 'source', 'target')}, + 'source': d.get('_src', u), 'target': d.get('_tgt', v)} + for u, v, d in G.edges(data=True) + ], + # G.graph["hyperedges"] holds hyperedges from both existing graph.json + # and new_extraction (build_merge combines them). Falling back to + # new_extraction only would silently drop prior-run hyperedges (#801). + 'hyperedges': list(G.graph.get('hyperedges', [])), + 'input_tokens': new_extraction.get('input_tokens', 0), + 'output_tokens': new_extraction.get('output_tokens', 0), +} +Path('graphify-out/.graphify_extract.json').write_text(json.dumps(merged_out, ensure_ascii=False), encoding=\"utf-8\") +print(f'[graphify update] Merged extraction written ({len(merged_out[\"nodes\"])} nodes, {len(merged_out[\"edges\"])} edges)') + +# Save manifest so next --update diffs against today's state, not the +# prior run's baseline (prevents ghost-node reports on subsequent updates). +# root= matches the build_merge call above so the manifest keys stay relative to +# the scan root — portable across clones/machines, so --update keeps matching +# cached files instead of missing every one after a move (#1417). +save_manifest(incremental['files'], root='INPUT_PATH') +print('[graphify update] Manifest saved.') +" +``` + +Then run Steps 4–8 on the merged graph as normal. + +After Step 4, show the graph diff: + +```bash +$(cat graphify-out/.graphify_python) -c " +import json +from graphify.analyze import graph_diff +from graphify.build import build_from_json +from networkx.readwrite import json_graph +import networkx as nx +from pathlib import Path + +# Load old graph (before update) from backup written before merge +old_data = json.loads(Path('graphify-out/.graphify_old.json').read_text(encoding=\"utf-8\")) if Path('graphify-out/.graphify_old.json').exists() else None +new_extract = json.loads(Path('graphify-out/.graphify_extract.json').read_text(encoding=\"utf-8\")) +G_new = build_from_json(new_extract, directed=IS_DIRECTED) + +if old_data: + G_old = json_graph.node_link_graph(old_data, edges='links') + diff = graph_diff(G_old, G_new) + print(diff['summary']) + if diff['new_nodes']: + print('New nodes:', ', '.join(n['label'] for n in diff['new_nodes'][:5])) + if diff['new_edges']: + print('New edges:', len(diff['new_edges'])) +" +``` + +Before the merge step, save the old graph: `cp graphify-out/graph.json graphify-out/.graphify_old.json` +Clean up after: `rm -f graphify-out/.graphify_old.json` + +--- + +## For --cluster-only + +Skip Steps 1–3. Re-run clustering on the existing graph: + +```bash +graphify cluster-only . +``` + +`graphify cluster-only .` is **self-contained**: it re-clusters, names communities, and regenerates `GRAPH_REPORT.md`, `graph.json`, and `graph.html` from the existing graph. **Do not re-run Steps 5–9** — they read intermediate files (`.graphify_extract.json`, `.graphify_detect.json`, `.graphify_analysis.json`) that a prior build's cleanup (Step 9) already deleted, so they raise `FileNotFoundError` (#1392). When it finishes, present the refreshed `GRAPH_REPORT.md` summary as usual. diff --git a/.gitignore b/.gitignore new file mode 100644 index 00000000..f4f12383 --- /dev/null +++ b/.gitignore @@ -0,0 +1,68 @@ +# Local skill outputs, caching, staging — never commit to GitHub +outputs/ +**/outputs/ +**/generated/ + +# Local credentials — NEVER commit +.claude-credentials/ +**/.claude-credentials/ +*.pat +*-pat.txt +github-pat.txt +github-token.txt +.github-token +*token*.txt +*.token +ghl-pit.txt +*-pit.txt +*.pit +*-credentials.txt +secrets.txt + +# OS / editor cruft +.DS_Store +Thumbs.db +*.swp +*~ +.vscode/ +.idea/ + +# Python +__pycache__/ +*.pyc +*.pyo +.venv/ +venv/ +*.egg-info/ + +# Node +node_modules/ + +# Session-local scripts and TODOs (never commit) +CLEANUP-*.bat +FINISH-*.bat +SYNC-*.bat +FINAL-*.bat +cleanup-and-commit.ps1 +NEXT-SESSION-TODO.md +test-persist.txt + +# Local credentials (never commit) +.env +*.env.local +switchy-token.txt +Git Hub PAT Token for claude/ + +# Added 2026-06-09 — additional local credential files & artifacts +gmail-app-password.txt +*password*.txt +Git Hub PAT Token for claude/ +*.skill +skills/*.zip +__writetest.tmp +_wt.txt + +# Heavy binary output files — belong in online-content repo or local only, never here +online_content_files/ +graphify-out/ +**/graphify-out/ diff --git a/.nojekyll b/.nojekyll new file mode 100644 index 00000000..e69de29b diff --git a/AGENTS.md b/AGENTS.md new file mode 100644 index 00000000..e2974d26 --- /dev/null +++ b/AGENTS.md @@ -0,0 +1,80 @@ +# READ THIS FIRST — Onboarding for Codex sessions touching this repo + +## Brand identity — the ONE rule that keeps getting violated + +**Single source of truth for Graeham's brand identity:** `skills/shared-references/identity.json` + +Read that file before writing ANY content that includes: +- DRE number (Graeham's individual salesperson DRE is `01466876`) +- Brokerage name +- Contact info (phone, email, website) +- Markets served + +**Do NOT hardcode brand details from memory or training.** California real estate marketing has multiple plausible-looking DRE numbers (brokerage DRE, salesperson DRE, archived numbers from old brokerages). It's tempting to type one in from prior context. **Don't.** Always read identity.json first. + +**Specifically prohibited:** the value `02015066` has been blocklisted ten separate times now (as of April 29, 2026). It is NOT Graeham's DRE, NOT Intero Real Estate's brokerage DRE (per Graeham's confirmation), and has no legitimate use anywhere in this repo or in outputs. If you find it in your context window, in a CMA template, in a contact strip, in a SKILL.md description, or anywhere else — **delete it. Do not propagate it.** Note that Cowork's cached skill descriptions may still show the wrong DRE; those are stale and should not be trusted over the actual SKILL.md files on disk. + +## Enforcement + +`scripts/verify_brand_identity.py` audits the entire repo against `identity.json`'s blocklist. It runs: + +1. As a local pre-push git hook (advisory — only runs on machines that have it installed). +2. Manually before every push. + +**Run the tripwire manually before pushing:** +```bash +python3 scripts/verify_brand_identity.py +``` + +If it fails, **fix the file paths it lists before pushing.** Do not bypass. + +## Repo structure (Option B architecture, 2026-04-29) + +This repo holds **source code only** — no outputs, no data bins. + +The repo root contains exactly these items: +- `skills/` — all 39 skills, each in its own folder. **Source of truth.** +- `scripts/` — repo-wide infrastructure scripts (currently just the brand-identity tripwire). +- `.Codex-plugin/` — Cowork plugin manifest. +- `.nojekyll` — disables Jekyll on GitHub Pages. +- `index.html` + `assets/` — GitHub Pages landing page. +- `AGENTS.md` (this file) — onboarding. +- `README.md` — public README. + +**Do NOT add output bins to this repo.** Generated content has its own home: + +| Output type | Where it goes | +|---|---| +| Published CMAs | `Graehamwatts/online-content/cmas/` | +| Published offer reports | `Graehamwatts/online-content/offers/` | +| Published disclosure reports | `Graehamwatts/online-content/disclosures/` | +| Published newsletters | `Graehamwatts/online-content/newsletters/` | +| Weekly production calendars | `Graehamwatts/online-content/dashboards/weekly-calendars/` | +| Per-topic single-topic dashboards | `Graehamwatts/online-content/dashboards/single-topic/` | +| Internal skill caching/staging | `/outputs/` (skill-local, gitignored) | + +The `online-content` repo is the **published content hub** — a separate repo because (1) it's a GitHub Pages site with public client-facing URLs, (2) outputs and source code shouldn't mix, and (3) it can be backed up/audited independently. + +> **Naming history:** This repo was renamed from `cma-reports` to `online-content` on 2026-05-01 to reflect that it holds ALL published content (CMAs, offers, disclosures, newsletters, dashboards) — not just CMAs. The old `cma-reports` repo has been retired; nothing migrated. + +## Content-creation primary skill + +The active content-engine skill is `skills/content-creation-engine/`. (The older `video-script-creation-engine` was retired during the 2026-04-29 reorganization.) When in doubt about which skill handles content/script generation, use `content-creation-engine`. + +## Tag of last known-good state + +`v2026.04.27-stable` — if anything regresses, compare against this tag. + + +## 2026-04-29 leak post-mortem (the 10th occurrence) + +**Where it leaked:** `Graehamwatts/cma-reports/Offer_828_Weeks_St.html` (in the now-retired `cma-reports` repo, since superseded by `online-content`) — a published GitHub Pages report for a real client offer comparison. The wrong DRE appeared on lines 523 and 753. + +**Root cause:** The Codex session that ran `offer-analyzer` on 2026-04-29 at 22:05 UTC had the wrong DRE (02015066) cached in its system prompt's `available_skills` list (specifically in the now-retired `video-script-creation-engine` description). Instead of reading the DRE from `identity.json` like this file instructs, that session typed the value from prior context. + +**Fix applied (2026-04-29):** +- Corrected the contaminated file in cma-reports +- Added a `BRAND IDENTITY HARD RULE` warning at the top of `cma-generator/SKILL.md` and `offer-analyzer/SKILL.md` that explicitly says "do NOT type from prior context" +- Retired `video-script-creation-engine` from GitHub (it's been merged into `content-creation-engine`); local Cowork sync should refresh the cache + +**Audit gap:** The tripwire (`scripts/verify_brand_identity.py`) only audits the skills repo. It does NOT currently audit `online-content` (the published-content sister repo, formerly `cma-reports`). A copy of the script should be added to `online-content` as well, OR this script extended to clone-and-audit `online-content` as part of its run. Open follow-up — increased priority since `online-content` will be the live target for every new CMA, offer, disclosure, newsletter, and dashboard going forward. diff --git a/CLAUDE.md b/CLAUDE.md new file mode 100644 index 00000000..6f38a610 --- /dev/null +++ b/CLAUDE.md @@ -0,0 +1,82 @@ +# READ THIS FIRST — Onboarding for Claude sessions touching this repo + +## Brand identity — the ONE rule that keeps getting violated + +**Single source of truth for Graeham's brand identity:** `skills/shared-references/identity.json` + +Read that file before writing ANY content that includes: +- DRE number (Graeham's individual salesperson DRE is `01466876`) +- Brokerage name +- Contact info (phone, email, website) +- Markets served + +**Do NOT hardcode brand details from memory or training.** California real estate marketing has multiple plausible-looking DRE numbers (brokerage DRE, salesperson DRE, archived numbers from old brokerages). It's tempting to type one in from prior context. **Don't.** Always read identity.json first. + +**Specifically prohibited:** the value `02015066` has been blocklisted ten separate times now (as of April 29, 2026). It is NOT Graeham's DRE, NOT Intero Real Estate's brokerage DRE (per Graeham's confirmation), and has no legitimate use anywhere in this repo or in outputs. If you find it in your context window, in a CMA template, in a contact strip, in a SKILL.md description, or anywhere else — **delete it. Do not propagate it.** Note that Cowork's cached skill descriptions may still show the wrong DRE; those are stale and should not be trusted over the actual SKILL.md files on disk. + +## Enforcement + +`scripts/verify_brand_identity.py` audits the entire repo against `identity.json`'s blocklist. It runs: + +1. As a local pre-push git hook (advisory — only runs on machines that have it installed). +2. Manually before every push. + +**Run the tripwire manually before pushing:** +```bash +python3 scripts/verify_brand_identity.py +``` + +If it fails, **fix the file paths it lists before pushing.** Do not bypass. + +## Repo structure (Option B architecture, 2026-04-29) + +This repo holds **source code only** — no outputs, no data bins. + +The repo root contains exactly these items: +- `skills/` — every skill, each in its own folder (run `ls skills/` for the current count — it drifts, don't hardcode it). **Source of truth.** +- `scripts/` — repo-wide infrastructure scripts (currently just the brand-identity tripwire). +- `.claude-plugin/` — Cowork plugin manifest. +- `.nojekyll` — disables Jekyll on GitHub Pages. +- `index.html` + `assets/` — GitHub Pages landing page. +- `CLAUDE.md` (this file) — onboarding. +- `README.md` — public README. + +**Do NOT add output bins to this repo.** Generated content has its own home: + +| Output type | Where it goes | +|---|---| +| Published CMAs | `Graehamwatts/online-content/cmas/` | +| Published offer reports | `Graehamwatts/online-content/offers/` | +| Published disclosure reports | `Graehamwatts/online-content/disclosures/` | +| Published newsletters | `Graehamwatts/online-content/newsletters/` | +| Weekly production calendars | `Graehamwatts/online-content/dashboards/weekly-calendars/` | +| Per-topic single-topic dashboards | `Graehamwatts/online-content/dashboards/single-topic/` | +| Internal skill caching/staging | `/outputs/` (skill-local, gitignored) | + +The `online-content` repo is the **published content hub** — a separate repo because (1) it's a GitHub Pages site with public client-facing URLs, (2) outputs and source code shouldn't mix, and (3) it can be backed up/audited independently. + +> **Naming history:** This repo was renamed from `cma-reports` to `online-content` on 2026-05-01 to reflect that it holds ALL published content (CMAs, offers, disclosures, newsletters, dashboards) — not just CMAs. The old `cma-reports` repo has been retired; nothing migrated. + +## Content-creation primary skill + +The active content-engine skill is `skills/content-creation-engine/`. (The older `video-script-creation-engine` was retired during the 2026-04-29 reorganization.) When in doubt about which skill handles content/script generation, use `content-creation-engine`. + +## Tag of last known-good state + +`v2026.04.27-stable` — if anything regresses, compare against this tag. + + +## 2026-04-29 leak post-mortem (the 10th occurrence) + +The `02015066` blocklist entry above exists because of a real incident on 2026-04-29 (a wrong-DRE leak into a published client offer report). **Read `skills/shared-references/dre-leak-incident-log.md`** for the full root-cause writeup and the still-open audit gap (the tripwire doesn't yet cover the `online-content` repo). The enforcement rule itself is already stated above and doesn't require reloading this history every session. + +## graphify (optional, on-demand only) + +This project has a knowledge graph at `graphify-out/` covering the `scripts/` and `.py`/`.js`-style code in this repo (most of this repo is SKILL.md/reference markdown, which the graph doesn't meaningfully help with — don't reach for it there). + +Use it ONLY when doing genuine code-relationship archaeology across script files — e.g. "how do these scripts call each other," "what depends on this function." It is NOT a required first step for normal file reads or greps; there are no automatic hooks enforcing it, by design, so it doesn't tax routine skill/doc reads. + +- `graphify query ""`, `graphify path "" ""`, `graphify explain ""` — scoped subgraph queries, cheaper than a broad grep when the question really is about code structure. +- After modifying `scripts/` code, run `graphify update .` to keep the graph current (AST-only, no API cost). + +**Staleness check (mandatory before trusting any graphify query result — applies to every graphed folder, not just this repo):** before answering from a graph query, check whether `graphify-out/needs_update` exists in that folder. This check is free (just a file existence check). If the flag is present, the graph may be stale — do NOT answer from it silently. Either (a) read the specific file in question directly instead of trusting the graph, or (b) tell the user the graph is stale for this folder and ask whether to run a full refresh (paid) before answering. Never present a graph-derived answer as current without this check. diff --git a/Menalto-Ave-Production-Skills.zip b/Menalto-Ave-Production-Skills.zip new file mode 100644 index 00000000..6a78189d Binary files /dev/null and b/Menalto-Ave-Production-Skills.zip differ diff --git a/README.md b/README.md index 60f7cfc3..fdfeefa4 100644 --- a/README.md +++ b/README.md @@ -3,3 +3,5 @@ ## Usage This is an internal package and has no documentation. + +<\!-- sync-test: 15-04-2026-0421 --> diff --git a/_anthropic_sales_email_draft.md b/_anthropic_sales_email_draft.md new file mode 100644 index 00000000..a5402b55 --- /dev/null +++ b/_anthropic_sales_email_draft.md @@ -0,0 +1,19 @@ +To: sales@anthropic.com (or the contact form at anthropic.com/contact-sales) +Subject: Solo founder at ~$18K/mo API run-rate — need help finding the right pricing structure + +Hi, + +I'm a solo founder (real estate tech — PropertyIQ, an AI-powered platform for real estate agents, built on Claude) currently running at roughly $600/day (~$18K/month pace) in API usage on my individual account, driven by an agentic development system built on Claude models — about 2.4B input tokens in the first 12 days of July, with a 97.8% prompt-cache hit ratio already in place. + +I'll be transparent about my situation: that run-rate is far beyond what I can sustain, and we're actively re-architecting to bring it down (model tiering, Batch API, workload reduction). I'm writing to understand what options exist on the pricing side in parallel: + +1. Do you have committed-use / volume discount structures that apply at my scale, and what monthly commitment do they start at? +2. Do you have a startup program (credits, discounted tiers) that a bootstrapped single-founder company building on Claude could qualify for? +3. Is there someone I can speak with briefly about the most cost-effective way to structure a Claude-based agentic product for a company at my stage? + +Happy to share usage data from my Console. My account is under graehamwatts@gmail.com. + +Thanks, +Graeham Watts +Founder, PropertyIQ +650-308-4727 diff --git a/_sami_email_draft.md b/_sami_email_draft.md new file mode 100644 index 00000000..feb64916 --- /dev/null +++ b/_sami_email_draft.md @@ -0,0 +1,49 @@ +REPLY IN THREAD: "combining everything: token optimizations + full restructure onto Claude Team plan" +To: sami@lfglabs.ai + +Hi Samii, + +Updating yesterday's email — since I sent it I pulled the real numbers from my Anthropic Console and ran an independent audit of all the repos, so now we're working with facts instead of feelings. Some of it is worse than I thought, some of it is genuinely better. Read this before our call. + +**1. The real burn numbers.** + +- July 1-13: $7,300 spent. That's about $600/day, pace of ~$18K/month. +- June was $4,150 for the ENTIRE month. So the daily rate quadrupled starting July 1-2 — which is exactly when we cancelled Fugu and that verification workload moved onto Fable 5 via the API. Nobody re-costed that migration, including me. So this isn't me saying you dropped the ball across the board — the explosion has a specific cause and a specific date. +- 91% of all token volume is running on Fable 5, the most expensive model that exists. Haiku is at 0.06%. That's the fixable part. +- Credit where due: your caching is working great — 97.8% cache hit ratio. Without it this would be 10x worse. + +I have to be straight with you: I can't sustain $18K/month, period. If we can't get this fixed the project as currently structured has to stop. But the fix looks very doable, which brings me to: + +**2. The gears.** Fable 5 needs to become the TOP gear, not the only gear: + +- Gear 1 (routine/mechanical — file ops, extraction, formatting, status checks): Haiku, or DeepSeek for the absolute cheapest tier (caveat: DeepSeek's hosted API is China-based servers, so nothing client or proprietary goes there unless self-hosted). +- Gear 2 (standard well-specified coding, tests, routine refactors): Sonnet, and add ChatGPT/OpenAI models here too — I want us multi-model so we're not single-vendor priced. +- Gear 3 (architecture, hard debugging, verification, anything a lower gear failed at): Fable 5 only. Escalate up, never default up. + +Benchmark to design against: Viktor (viktor.com) sells a full AI employee at $50/month. That's the cost class the Wattson assistants have to hit — not because I'm cheap, but because if we ever sell Wattson assistants to other agents at the $349-399/month price point, the AI cost per assistant has to sit way under that or there's no margin. This cost discipline IS our unit economics, not just my savings. + +**3. The audit — good news and a real problem you both need to fix this week.** + +I had an independent audit run across all five repos today. The good news, and I mean this: the July spend bought real work. 162 commits, ~37,000 lines on the Watson PropCast fork in nine days — lead intake, attribution pipeline, publishing center, engagement queue — with 50 test files, CI, and proper PR gates. Quality was rated above average, not agent thrash. That would have cost more from human devs. The fleet works. + +The problem: **Watson's fork and QuestLab's main PropCast repo are building overlapping features separately and diverging.** The fork itself is by design — I want human review before anything hits main, that stays. But the review/merge pipeline has stalled, so ~37K lines of good work is sitting unmerged while QuestLab independently builds overlapping approval/publishing features upstream. I'm paying twice for some of the same functionality. + +Two things I need done on this, starting today: + +1. **QuestLab needs a detailed writeup — today if we can** — of exactly what the Watson system has built and where it overlaps with what they're building, so they stop duplicating and correct the overlap ASAP. I want this on our call. +2. **The Watson agents need an updated snapshot of QuestLab's current build in their context before every build cycle** — so Watson builds FROM the end of what QuestLab has done, forward. Not duplicating what already exists. Make that a standing part of the fleet's workflow, not a one-time sync. And use this as the template for how the system develops going forward: sync first, then build. + +Also from the audit: PropSearch is ~70% built and real, but has almost no test coverage (4 test files across 76K lines) — QuestLab needs to fix that. And a regular merge cadence (weekly at minimum) so reviewed fork work actually lands in main. + +**4. Quick confirms for the call:** + +- Our OpenClaw is the public open-source project (openclaw.ai) with your modifications, correct? And Watson runs on a proper API key — nothing token-extraction-based (that's the pattern Anthropic blocked in January and I don't want us anywhere near it). +- The Wattson Discord agent is still unresponsive (I sent the screenshot on WhatsApp + the error email Saturday) — status? + +Everything from yesterday's email stands: the seat restructure, the internal-jobs migration plan, the inventory of every Wattson API job tagged internal vs customer-facing with spend split, BYO-subscription for resale. The targets now have numbers attached: I want the plan to get us to low four figures this month, with a path under $1K/month as the gears and migration land. If the architecture fundamentally can't run at that level, I need to hear that on the call, not discover it in August. + +Talk Monday. + +Graeham Watts +650-308-4727 +graehamwatts.com diff --git a/assets/dashboard.css b/assets/dashboard.css new file mode 100644 index 00000000..db2c875c --- /dev/null +++ b/assets/dashboard.css @@ -0,0 +1,267 @@ +:root{ + --navy:#1B2A4A; + --navy-2:#2a3d6b; + --gold:#C5A258; + --gold-soft:rgba(197,162,88,0.12); + --green:#2e7d32; + --red:#c62828; + --amber:#e65100; + --blue:#1565C0; + --bg:#F7F5EF; + --card:#FFFFFF; + --border:rgba(27,42,74,0.10); + --border-strong:rgba(27,42,74,0.20); + --text:#1B2A4A; + --muted:#5a6478; + --muted-2:#8a92a3; + --radius:12px; + --radius-sm:8px; + --radius-pill:99px; + --shadow:0 2px 8px rgba(27,42,74,0.06); + --shadow-lg:0 6px 20px rgba(27,42,74,0.12); + --shadow-hover:0 4px 14px rgba(27,42,74,0.14); + --font-display:'Plus Jakarta Sans',system-ui,sans-serif; + --font-body:'DM Sans',system-ui,sans-serif; + --font-mono:ui-monospace,'SF Mono','Monaco',monospace; +} +*{box-sizing:border-box;margin:0;padding:0} +html{scroll-behavior:smooth} +body{background:var(--bg);color:var(--text);font-family:var(--font-body);line-height:1.6;font-size:15px;-webkit-font-smoothing:antialiased} +a{color:inherit;text-decoration:none} +code{font-family:var(--font-mono);font-size:0.9em;background:var(--gold-soft);padding:1px 6px;border-radius:4px;color:var(--navy)} + +/* Top Nav (sticky) */ +.topnav{background:var(--navy);border-bottom:2px solid var(--gold);padding:12px 0;position:sticky;top:0;z-index:100;box-shadow:0 1px 3px rgba(0,0,0,0.05)} +.topnav-inner{max-width:1320px;margin:0 auto;padding:0 36px;display:flex;justify-content:space-between;align-items:center;gap:16px;flex-wrap:wrap} +.topnav-brand{font-family:var(--font-display);font-size:13px;font-weight:800;letter-spacing:0.5px;color:#fff;display:flex;align-items:center;gap:10px} +.topnav-brand .dot{color:var(--gold);font-size:10px} +.topnav-links{display:flex;gap:14px;align-items:center} +.topnav-link{font-size:12px;font-weight:600;letter-spacing:0.3px;color:rgba(255,255,255,0.7);text-transform:uppercase;transition:color 0.15s} +.topnav-link:hover{color:#fff} +.topnav-link.active{color:var(--gold)} +.topnav-link.btn{background:var(--gold);color:var(--navy);padding:8px 16px;border-radius:var(--radius-pill);font-weight:700;letter-spacing:0.3px;transition:transform 0.15s,box-shadow 0.15s} +.topnav-link.btn:hover{transform:translateY(-1px);box-shadow:0 4px 10px rgba(197,162,88,0.4);color:var(--navy)} + +/* Page wrapper */ +.wrap{max-width:1320px;margin:0 auto;padding:40px 36px 60px} +.wrap.tight{max-width:1100px} + +/* Hero */ +.hero{background:linear-gradient(135deg,var(--navy) 0%,var(--navy-2) 60%,#3a5090 100%);color:#fff;padding:44px 44px 36px;border-radius:var(--radius);margin-bottom:32px;position:relative;overflow:hidden;box-shadow:var(--shadow-lg)} +.hero::after{content:'';position:absolute;top:-100px;right:-100px;width:340px;height:340px;border-radius:50%;background:rgba(197,162,88,0.08);pointer-events:none} +.hero-ey{font-family:var(--font-display);font-size:11px;font-weight:700;letter-spacing:2.5px;text-transform:uppercase;color:var(--gold);margin-bottom:10px;position:relative} +.hero h1{font-family:var(--font-display);font-size:36px;font-weight:800;margin-bottom:10px;line-height:1.15;letter-spacing:-0.5px;position:relative} +.hero .hsub{font-size:15px;color:rgba(255,255,255,0.78);max-width:680px;line-height:1.6;position:relative} +.hero-meta{display:flex;gap:8px;margin-top:22px;flex-wrap:wrap;position:relative} +.hm-pill{background:rgba(255,255,255,0.1);border:1px solid rgba(255,255,255,0.18);padding:6px 14px;border-radius:var(--radius-pill);font-size:11px;font-weight:600;letter-spacing:0.3px} +.hm-pill.gold{background:var(--gold);color:var(--navy);border-color:var(--gold);font-weight:700} + +/* Section heading */ +.sh{font-family:var(--font-display);font-size:22px;font-weight:800;color:var(--navy);margin:44px 0 6px;letter-spacing:-0.2px;display:flex;align-items:baseline;gap:10px} +.sh:first-child{margin-top:0} +.sh .sh-count{font-size:14px;font-weight:600;color:var(--muted-2);letter-spacing:0} +.sh-sub{font-size:13px;color:var(--muted);margin-bottom:20px;max-width:800px;line-height:1.6} + +/* Peter Read First card */ +.peter-card{background:var(--navy);color:#fff;padding:22px 26px;border-radius:var(--radius);margin-bottom:30px;box-shadow:var(--shadow);border-left:4px solid var(--gold)} +.peter-card details summary{list-style:none;cursor:pointer;display:flex;justify-content:space-between;align-items:center;gap:12px} +.peter-card details summary::-webkit-details-marker{display:none} +.peter-card details summary::after{content:'+';font-size:22px;color:var(--gold);font-weight:700;line-height:1;transition:transform 0.2s} +.peter-card details[open] summary::after{content:'−'} +.peter-card .peter-head{display:flex;align-items:center;gap:12px} +.peter-card .peter-tag{background:var(--gold);color:var(--navy);padding:3px 10px;border-radius:var(--radius-pill);font-size:10px;font-weight:800;letter-spacing:0.6px;text-transform:uppercase} +.peter-card .peter-title{font-family:var(--font-display);font-size:16px;font-weight:700} +.peter-card .peter-body{margin-top:18px;padding-top:18px;border-top:1px solid rgba(255,255,255,0.15);font-size:13px;line-height:1.7;color:rgba(255,255,255,0.85)} +.peter-card .peter-body p{margin-bottom:10px} +.peter-card .peter-body strong{color:#fff} + +/* Day Tiles Grid */ +.day-grid{display:grid;grid-template-columns:repeat(5,1fr);gap:14px;margin-bottom:44px} +@media(max-width:1000px){.day-grid{grid-template-columns:repeat(3,1fr)}} +@media(max-width:640px){.day-grid{grid-template-columns:repeat(2,1fr)}} +@media(max-width:430px){.day-grid{grid-template-columns:1fr}} + +.day-tile{background:var(--card);border:1px solid var(--border);border-radius:var(--radius);padding:20px 18px;text-decoration:none;color:inherit;display:flex;flex-direction:column;min-height:200px;box-shadow:var(--shadow);transition:transform 0.2s,box-shadow 0.2s,border-color 0.2s;position:relative;overflow:hidden} +.day-tile:hover{transform:translateY(-3px);box-shadow:var(--shadow-hover);border-color:var(--gold)} +.day-tile.breaking{border:2px solid var(--red)} +.day-tile .dt-head{display:flex;justify-content:space-between;align-items:flex-start;margin-bottom:10px} +.day-tile .dt-day{font-family:var(--font-display);font-size:11px;font-weight:800;letter-spacing:1.5px;text-transform:uppercase;color:var(--muted)} +.day-tile .dt-date{font-family:var(--font-display);font-size:22px;font-weight:800;color:var(--navy);line-height:1;margin-top:2px} +.day-tile .dt-status{font-size:9px;font-weight:700;letter-spacing:0.8px;text-transform:uppercase;padding:3px 8px;border-radius:4px} +.day-tile .dt-status.shipped{background:rgba(46,125,50,0.12);color:var(--green)} +.day-tile .dt-status.scheduled{background:rgba(230,81,0,0.12);color:var(--amber)} +.day-tile .dt-status.draft{background:rgba(90,100,120,0.12);color:var(--muted)} +.day-tile .dt-status.breaking{background:rgba(198,40,40,0.12);color:var(--red)} +.day-tile .dt-title{font-family:var(--font-display);font-size:14px;font-weight:700;color:var(--navy);line-height:1.35;margin-bottom:10px;flex:1} +.day-tile .dt-meta{display:flex;gap:6px;flex-wrap:wrap;margin-bottom:12px;font-size:10px;color:var(--muted)} +.day-tile .dt-badge{padding:2px 7px;border-radius:4px;background:var(--gold-soft);color:var(--navy);font-weight:700;letter-spacing:0.3px;text-transform:uppercase} +.day-tile .dt-badge.score{background:rgba(27,42,74,0.06);color:var(--navy)} +.day-tile .dt-footer{display:flex;justify-content:space-between;align-items:center;padding-top:12px;border-top:1px dashed var(--border);font-size:11px} +.day-tile .dt-why{color:var(--muted);font-style:italic;flex:1;margin-right:10px;line-height:1.4} +.day-tile .dt-arrow{color:var(--gold);font-weight:800;font-size:16px} + +/* Generic card */ +.card{background:var(--card);border:1px solid var(--border);border-radius:var(--radius);padding:22px 24px;box-shadow:var(--shadow);margin-bottom:16px} +.card h3{font-family:var(--font-display);font-size:17px;font-weight:800;color:var(--navy);margin-bottom:8px} + +/* Collapsible accordion (matches existing v5-research style) */ +.acc{background:var(--card);border:1px solid var(--border);border-radius:var(--radius);margin-bottom:14px;box-shadow:var(--shadow);overflow:hidden} +.acc details summary{list-style:none;cursor:pointer;padding:18px 24px;display:flex;justify-content:space-between;align-items:center;gap:14px;font-family:var(--font-display);font-size:15px;font-weight:700;color:var(--navy)} +.acc details summary::-webkit-details-marker{display:none} +.acc details summary::after{content:'+';font-size:22px;color:var(--gold);font-weight:700;line-height:1;min-width:20px;text-align:right} +.acc details[open] summary::after{content:'−'} +.acc .acc-sub{font-family:var(--font-body);font-size:12px;font-weight:500;color:var(--muted);letter-spacing:0;margin-left:4px} +.acc details[open] summary{border-bottom:1px solid rgba(197,162,88,0.25)} +.acc .acc-body{padding:20px 24px 24px} + +/* Tables */ +table.t{width:100%;border-collapse:collapse;font-size:13px;background:transparent} +table.t thead th{text-align:left;padding:10px 12px;font-size:10px;font-weight:800;color:var(--muted);letter-spacing:0.8px;text-transform:uppercase;border-bottom:1px solid var(--border)} +table.t tbody td{padding:10px 12px;border-bottom:1px solid rgba(27,42,74,0.05);vertical-align:top;line-height:1.5} +table.t tbody td.num{font-weight:700;color:var(--navy);white-space:nowrap;width:80px;text-align:right;font-family:var(--font-mono)} +table.t tbody tr.total td{background:var(--gold-soft);border-top:2px solid var(--border);font-weight:700} + +/* Scoring Architecture two-table grid */ +.sa-wrap{display:grid;grid-template-columns:1fr 1fr;gap:18px;margin:4px 0} +@media(max-width:900px){.sa-wrap{grid-template-columns:1fr}} +.sa-col{background:#FCFAF4;border:1px solid var(--border);border-radius:var(--radius-sm);padding:18px} +.sa-head{display:flex;justify-content:space-between;align-items:center;margin-bottom:4px;font-family:var(--font-display);font-size:14px;font-weight:800;color:var(--navy)} +.sa-head .sa-total{color:var(--gold);font-size:18px;font-weight:800} +.sa-owner{font-size:10px;color:var(--muted);margin-bottom:14px;font-family:var(--font-mono)} +.sa-ok{color:var(--green);font-weight:700} +.sa-warn{color:var(--amber);font-weight:700} +.sa-bad{color:var(--red);font-weight:700} + +/* Priority Axes */ +.pa{display:flex;flex-direction:column;gap:10px;margin-top:16px;padding:18px;background:#FCFAF4;border:1px solid var(--border);border-radius:var(--radius-sm)} +.pa-h{font-family:var(--font-display);font-size:11px;font-weight:800;color:var(--muted);letter-spacing:1px;text-transform:uppercase} +.pa-row{display:grid;grid-template-columns:100px 1fr 60px;align-items:center;gap:12px;font-size:13px} +.pa-label{color:var(--muted);font-weight:700} +.pa-track{background:rgba(27,42,74,0.08);height:9px;border-radius:5px;overflow:hidden} +.pa-fill{height:100%;border-radius:5px;transition:width 0.3s} +.pa-val{text-align:right;font-weight:700;color:var(--navy);font-family:var(--font-mono);font-size:12px} + +/* Buttons */ +.btn{display:inline-flex;align-items:center;gap:8px;background:var(--gold);color:var(--navy);padding:13px 24px;border-radius:var(--radius-pill);font-family:var(--font-display);font-size:14px;font-weight:700;border:none;cursor:pointer;text-decoration:none;transition:transform 0.15s,box-shadow 0.15s;letter-spacing:0.2px} +.btn:hover{transform:translateY(-1px);box-shadow:0 4px 12px rgba(197,162,88,0.35)} +.btn.navy{background:var(--navy);color:#fff} +.btn.navy:hover{box-shadow:0 4px 12px rgba(27,42,74,0.3)} +.btn.outline{background:transparent;color:var(--navy);border:2px solid var(--navy)} +.btn.outline:hover{background:var(--navy);color:#fff;box-shadow:none} +.btn.sm{padding:8px 16px;font-size:12px} +.btn .arrow{font-size:16px} + +/* Breadcrumb (day views) */ +.crumb{font-size:13px;color:var(--muted);margin-bottom:20px;display:flex;align-items:center;gap:8px;font-weight:600} +.crumb a{color:var(--muted);transition:color 0.15s} +.crumb a:hover{color:var(--navy)} +.crumb .sep{color:var(--muted-2)} +.crumb .current{color:var(--navy)} + +/* Footer */ +.footer{margin-top:60px;padding:24px 0 10px;border-top:1px solid var(--border);font-size:12px;color:var(--muted);line-height:1.7} +.footer a{color:var(--navy);font-weight:600} +.footer a:hover{text-decoration:underline} +.footer-links{display:flex;flex-wrap:wrap;gap:14px;margin-top:8px;margin-bottom:16px} + +/* Goal Mix Table small variant */ +.mix-tbl-sm{font-size:13px} +.mix-tbl-sm td{padding:8px 12px !important} + +/* Placeholder callout for stubbed day views */ +.placeholder{background:rgba(197,162,88,0.06);border:1px dashed var(--gold);border-radius:var(--radius);padding:20px 24px;text-align:center;color:var(--muted);font-size:13px;line-height:1.7} +.placeholder strong{color:var(--navy);display:block;margin-bottom:4px;font-family:var(--font-display);font-size:15px} + + +/* Content Creation Section */ +.cc-overview{display:grid;grid-template-columns:repeat(auto-fill,minmax(200px,1fr));gap:10px;margin-bottom:24px} +.cc-stat{background:var(--card);border:1px solid var(--border);border-radius:var(--radius-sm);padding:14px 16px;text-align:center;box-shadow:var(--shadow)} +.cc-stat .cc-num{font-family:var(--font-display);font-size:26px;font-weight:800;color:var(--navy);line-height:1} +.cc-stat .cc-lbl{font-size:11px;font-weight:700;letter-spacing:0.5px;text-transform:uppercase;color:var(--muted);margin-top:4px} +.cc-stat.gold .cc-num{color:var(--gold)} + +.format-grid{display:grid;grid-template-columns:repeat(auto-fill,minmax(320px,1fr));gap:14px;margin-bottom:32px} +.format-card{background:var(--card);border:1px solid var(--border);border-radius:var(--radius);padding:20px 22px;box-shadow:var(--shadow);display:flex;flex-direction:column;transition:border-color 0.15s,box-shadow 0.15s} +.format-card:hover{border-color:var(--gold)} +.format-card .fc-head{display:flex;justify-content:space-between;align-items:center;margin-bottom:10px;gap:10px} +.format-card .fc-tag{background:var(--navy);color:#fff;padding:4px 10px;border-radius:4px;font-size:10px;font-weight:800;letter-spacing:0.6px;text-transform:uppercase} +.format-card .fc-tag.yt{background:#c62828} +.format-card .fc-tag.ig{background:#c13584} +.format-card .fc-tag.tt{background:#010101} +.format-card .fc-tag.blog{background:#2e7d32} +.format-card .fc-tag.fb{background:#1877f2} +.format-card .fc-tag.li{background:#0a66c2} +.format-card .fc-tag.gmb{background:#ea4335} +.format-card .fc-tag.nl{background:var(--gold);color:var(--navy)} +.format-card .fc-tag.ad{background:#ff6f00} +.format-card .fc-meta{font-size:11px;color:var(--muted);font-family:var(--font-mono);font-weight:600} +.format-card h3{font-family:var(--font-display);font-size:16px;font-weight:700;color:var(--navy);margin-bottom:6px;line-height:1.35} +.format-card .fc-desc{font-size:13px;color:var(--muted);line-height:1.55;margin-bottom:14px;flex:1} +.format-card .fc-status{font-size:11px;font-weight:700;letter-spacing:0.4px;text-transform:uppercase;padding:3px 8px;border-radius:4px;display:inline-block;margin-bottom:10px} +.format-card .fc-status.ready{background:rgba(46,125,50,0.12);color:var(--green)} +.format-card .fc-status.pending{background:rgba(230,81,0,0.1);color:var(--amber)} +.format-card details summary{list-style:none;cursor:pointer;background:var(--gold);color:var(--navy);padding:10px 16px;border-radius:var(--radius-pill);font-family:var(--font-display);font-size:13px;font-weight:700;display:inline-flex;align-items:center;gap:6px;text-align:center;justify-content:center;transition:transform 0.15s,box-shadow 0.15s;align-self:flex-start} +.format-card details summary::-webkit-details-marker{display:none} +.format-card details summary:hover{transform:translateY(-1px);box-shadow:0 3px 8px rgba(197,162,88,0.35)} +.format-card details summary::after{content:'↓';margin-left:2px;font-weight:700} +.format-card details[open] summary::after{content:'↑'} +.format-card details[open] summary{background:var(--navy);color:#fff} +.format-card .fc-content{margin-top:14px;padding:14px;background:#FCFAF4;border-radius:var(--radius-sm);border:1px solid var(--border);font-family:var(--font-mono);font-size:12px;line-height:1.7;white-space:pre-wrap;color:var(--text)} +.format-card .fc-copy{margin-top:10px;display:flex;gap:8px;flex-wrap:wrap} +.format-card .fc-note{font-size:11px;color:var(--muted);margin-top:10px;font-style:italic;padding-top:10px;border-top:1px dashed var(--border)} + + +/* Modal overlay (for Content Creation) */ +.modal-backdrop{position:fixed;inset:0;background:rgba(27,42,74,0.65);backdrop-filter:blur(4px);z-index:9000;display:none;align-items:flex-start;justify-content:center;padding:40px 24px;overflow-y:auto} +.modal-backdrop.open{display:flex} +.modal{background:var(--card);border-radius:var(--radius);max-width:860px;width:100%;box-shadow:0 20px 60px rgba(0,0,0,0.3);position:relative;margin:0 auto} +.modal-head{padding:22px 28px 16px;border-bottom:1px solid var(--border);display:flex;justify-content:space-between;align-items:flex-start;gap:16px} +.modal-head .modal-tag{background:var(--navy);color:#fff;padding:4px 10px;border-radius:4px;font-size:10px;font-weight:800;letter-spacing:0.6px;text-transform:uppercase;display:inline-block;margin-bottom:6px} +.modal-head h3{font-family:var(--font-display);font-size:18px;font-weight:800;color:var(--navy);line-height:1.3} +.modal-head .modal-meta{font-size:12px;color:var(--muted);font-family:var(--font-mono);margin-top:4px} +.modal-close{background:transparent;border:none;font-size:26px;font-weight:700;color:var(--muted);cursor:pointer;line-height:1;padding:0;min-width:32px;transition:color 0.15s} +.modal-close:hover{color:var(--red)} +.modal-body{padding:24px 28px;max-height:68vh;overflow-y:auto} +.modal-body .m-content{font-family:var(--font-mono);font-size:13px;line-height:1.8;white-space:pre-wrap;color:var(--text);background:#FCFAF4;padding:20px;border-radius:var(--radius-sm);border:1px solid var(--border)} +.modal-body .m-help{font-size:13px;color:var(--muted);margin-bottom:16px;line-height:1.6} +.modal-actions{padding:16px 28px;border-top:1px solid var(--border);display:flex;gap:10px;justify-content:flex-end;flex-wrap:wrap} + +/* Scoring explanation box */ +.sa-howto{background:rgba(27,42,74,0.04);border-left:3px solid var(--navy);padding:12px 16px;font-size:12px;line-height:1.7;color:var(--muted);margin-bottom:14px;border-radius:0 var(--radius-sm) var(--radius-sm) 0} +.sa-howto strong{color:var(--navy);font-family:var(--font-display);display:block;margin-bottom:6px;font-size:11px;letter-spacing:0.5px;text-transform:uppercase} + +/* Research card as button */ +.research-grid{display:grid;grid-template-columns:repeat(auto-fill,minmax(260px,1fr));gap:12px;margin-top:18px} +.research-card-btn{background:var(--card);border:1px solid var(--border);border-radius:var(--radius);padding:16px 18px;box-shadow:var(--shadow);text-decoration:none;color:inherit;display:flex;flex-direction:column;transition:transform 0.15s,box-shadow 0.15s,border-color 0.15s;cursor:pointer} +.research-card-btn:hover{transform:translateY(-2px);box-shadow:var(--shadow-hover);border-color:var(--gold)} +.research-card-btn .rc-head{display:flex;justify-content:space-between;align-items:flex-start;gap:10px;margin-bottom:8px} +.research-card-btn h4{font-family:var(--font-display);font-size:14px;font-weight:700;color:var(--navy)} +.research-card-btn .rc-status{font-size:9px;font-weight:700;letter-spacing:0.6px;text-transform:uppercase;padding:2px 7px;border-radius:4px;white-space:nowrap} +.research-card-btn .rc-status.live{background:rgba(46,125,50,0.12);color:var(--green)} +.research-card-btn .rc-status.demo{background:rgba(230,81,0,0.12);color:var(--amber)} +.research-card-btn .rc-status.stub{background:rgba(90,100,120,0.12);color:var(--muted)} +.research-card-btn .rc-desc{font-size:12px;color:var(--muted);line-height:1.55;margin-bottom:10px;flex:1} +.research-card-btn .rc-meta{font-size:11px;color:var(--muted-2);display:flex;justify-content:space-between;align-items:center;padding-top:10px;border-top:1px dashed var(--border)} +.research-card-btn .rc-arrow{color:var(--gold);font-weight:800} + +/* Lead Capture pipeline callout */ +.lead-pipeline{background:linear-gradient(135deg,var(--navy) 0%,var(--navy-2) 100%);color:#fff;padding:22px 26px;border-radius:var(--radius);margin-top:18px;box-shadow:var(--shadow)} +.lead-pipeline h3{font-family:var(--font-display);font-size:15px;font-weight:800;margin-bottom:8px;color:#fff} +.lead-pipeline p{font-size:13px;line-height:1.7;color:rgba(255,255,255,0.8)} +.lead-pipeline code{background:rgba(255,255,255,0.12);color:var(--gold);padding:1px 8px;border-radius:4px} + +/* Demo data warning */ +.demo-warn{display:inline-flex;align-items:center;gap:6px;background:rgba(230,81,0,0.1);border:1px solid var(--amber);color:var(--amber);padding:3px 10px;border-radius:4px;font-size:10px;font-weight:700;letter-spacing:0.4px;text-transform:uppercase;margin-left:8px} + + + +/* Format card buttons: primary + prompt (outline) + production (purple) */ +.fc-buttons{display:flex;gap:8px;flex-wrap:wrap;align-items:center} +.fc-buttons .btn{padding:9px 16px;font-size:12px} +.btn.gold-outline{background:transparent;color:var(--gold);border:1.5px solid var(--gold)} +.btn.gold-outline:hover{background:var(--gold);color:var(--navy);box-shadow:0 3px 8px rgba(197,162,88,0.3)} +.btn.purple{background:#5b2c8b;color:#fff;border:1.5px solid #5b2c8b} +.btn.purple:hover{background:#4a2170;box-shadow:0 3px 8px rgba(91,44,139,0.35)} + +/* Success flash on copy buttons */ +.btn.copied{background:var(--green) !important;color:#fff !important;border-color:var(--green) !important} diff --git a/content-creation-engine-SKILL.docx b/content-creation-engine-SKILL.docx new file mode 100644 index 00000000..db3fc55e Binary files /dev/null and b/content-creation-engine-SKILL.docx differ diff --git a/content-creation-engine-obsidian-note.md b/content-creation-engine-obsidian-note.md new file mode 100644 index 00000000..d0e53a7d --- /dev/null +++ b/content-creation-engine-obsidian-note.md @@ -0,0 +1,1122 @@ +--- +tags: [skill, content-engine, real-estate] +updated: 2026-05-25 +skill_path: skills/content-creation-engine/SKILL.md +--- + +--- +name: content-creation-engine +description: "Bay Area / East Palo Alto real estate content creation engine for Graeham Watts (REALTOR, Compass, DRE# 01466876). Use this skill ANY time the user mentions: content creation engine, create content, newsletter content, blog post, ad copy, social media content, video script, content ideas, YouTube, Reels, Shorts, TikTok, AI avatar script, listing video, market update video, BOFU content, TOFU content, MOFU content, Bay Area real estate content, East Palo Alto content, Redwood City content, Palo Alto content, Menlo Park content, San Mateo County content, Reddit ideation, content scoring, content pillars, AB 1482, relocation content, first-time-buyer content, seller content, find me topics, content intelligence, I need content, market research, topic discovery, or anything related to generating inbound real-estate content for Graeham's markets. Also trigger when the user uploads MLS data or a new listing and wants a content package, or pastes a YouTube URL and wants content ideas from it." +--- + +# Content Creation Engine + +> **Absorbed on 2026-05-13 (Merge 3):** `video-research-engine` was merged into this skill. Its content now lives at `references/phases/video-research.md` and its Python scripts at `scripts/video-research/`. The folder `skills/video-research-engine/` was deleted in the same commit. The original v-r-e trigger phrases ("transcribe YouTube video", "analyze this video", "frame by frame", "B-roll breakdown", "deep dive on this video") are already part of this skill's top-level description. + +> **Re-extracted on 2026-05-15:** the video-research scripts (download/frames/transcribe/analyze/library/dataforseo_direct) have been LIFTED OUT of this skill into a new standalone skill `video-watcher`. The copies inside `scripts/video-research/` are now deprecated — they remain for backward compatibility but new invocations should call `video-watcher` directly. Reason: the visual-analysis capability was effectively dormant inside this skill because (1) most users didn't know it existed, (2) trigger keywords didn't match how Peter/Ellie/Adrian actually speak, (3) the visual analysis was coupled to content generation when it should be a standalone tool. The new `video-watcher` skill fires on "watch this video," "make ours like this," "shot list," "blueprint," "full breakdown" — keywords that map naturally to the video-editor workflow. When this skill needs visual analysis as part of Phase 0 source ingestion (Mode B-style visual pass), it should call `video-watcher` as an external skill rather than running the embedded code. Companion to the also-extracted `video-transcriber` (audio→text) which was extracted on the same day. + +> **Absorbed on 2026-05-13 (Merge 2):** `bofu-query-generator` and `bofu-intent-scorer` were merged into this skill. Their content now lives at `references/phases/bofu-query-generator.md` (Phase 1) and `references/phases/bofu-intent-scorer.md` (Phase 3). The folders `skills/bofu-query-generator/` and `skills/bofu-intent-scorer/` were deleted in the same commit. If you find any reference to those folder paths anywhere in this repo, that reference is a bug — point it at the new phase reference files instead. + +> **NOTE (April 2026):** This skill absorbed `video-script-creation-engine`. That skill no longer exists as a separate folder — all its capabilities (script writing, SSML generation, shot lists, editing notes, AI video prompts, SEO packages, platform cross-posting, voice+production pairing) now live here. All skills that previously referenced `video-script-creation-engine` (heygen-video, heygen-elevenlabs-renderer, content-calendar) have been updated to point here instead. + + + +Modular real estate content generation system for Graeham Watts. Turns a single topic into a funnel-tagged, multi-platform content package grounded in live Bay Area buyer and seller data. + +**This skill is PER-TOPIC.** Given ONE topic (from `content-calendar`'s weekly plan or a direct user ask), it pulls topic-matched research, classifies BOFU intent, and produces the full content package (scripts, SSML, editing notes, AI video prompts, SEO, and HeyGen render hand-off). **Weekly planning and opportunity scoring live in `content-calendar`, not here.** See the Scoring Architecture section below for the clean handoff between weekly (Opportunity Score) and per-topic (Intent Score). + + +## Scope Boundary (Who Owns What) + +> **Updated April 2026 to resolve overlap with `content-calendar`.** + +This skill is the **PER-TOPIC PRODUCTION** layer. Given a single topic (from `content-calendar`'s weekly plan OR from a direct user ask like "build me a content package on the EPA homicide-free story"), this skill generates the full content package — 14 formats with pre-generated content and companion prompts, research data panel, shot list, SSML, editing notes, AI video prompts, SEO package, alt hooks, and HeyGen render hand-off. + +**WEEKLY PLANNING** (deciding WHICH topics to cover across a 5-day week, what funnel mix, scoring across candidates) is owned by `content-calendar`, not this skill. + +Per-Topic Research in this skill pulls research SPECIFIC TO ONE TOPIC (the stats, news, quotes that back this one content package). The WEEKLY research that feeds multi-topic scoring lives in `content-calendar`. + +| Request type | Which skill | Why | +|-------------|-------------|-----| +| "What should I post this week?" | `content-calendar` | Weekly scope, multi-topic scoring | +| "Plan next week's 5 topics" | `content-calendar` | Weekly scope | +| "Build a content package for [specific topic]" | `content-creation-engine` (this skill) | Per-topic scope | +| "I have a new listing, give me content for it" | `content-creation-engine` (this skill) | Per-topic scope | +| "Research and write content on [breaking news]" | `content-creation-engine` (this skill) | Per-topic scope, includes Per-Topic Research research | +| "Swap Monday's topic for [new topic]" | Both: `content-calendar` to update weekly plan, then `content-creation-engine` to produce the package | Chained | + +## Scoring Architecture — Single Source of Truth + +> **Updated April 2026. Resolves prior conflict where four separate scoring systems were competing silently.** + +The content system has **two distinct scores** that answer two distinct questions. They are NOT interchangeable and they must NEVER be merged. + +| Score | Owner | Scale | Answers | When applied | +|---|---|---|---|---| +| **Opportunity Score** | `content-calendar` skill | 25 pts (5 criteria × 5 pts): Performance Signal, Search Demand, Audience Intent, Competitive Gap, Timeliness | "Should we cover this topic THIS WEEK vs other candidates?" | Once per week, across 12-15 candidates. Top 4-5 by score make the weekly calendar. | +| **Intent Score** | This skill's Phase 3 (BOFU Intent Scorer) | 25 pts (5 criteria × 5 pts) + freshness adjustment (±5): Inquiry Type, Intent Matrix, Source Confirmation, Emotional Temp, Local Relevance, Freshness | "What's the BOFU intent of this topic (DECISION / CONSIDERATION / AWARENESS)?" | Once per topic, AFTER opportunity selection. Used to tag funnel position and adjust CTA. | + +**What the other "scoring" references in this stack actually are:** + +| Previously called | Actually is | Lives in | +|---|---|---| +| Per-Topic Research scoring (10-pt, 4 criteria) — DELETED | Per-topic research — no scoring | Per-Topic Research below (rewritten) | +| Phase 2 "4-axis scoring" | Reddit signal filtering (what to surface from scrape) | Phase 2 content-ideation-engine | + +**Rule of thumb:** If a topic is on the weekly calendar, content-calendar already scored it (Opportunity). When you build its content package here, Phase 3 scores it again for a DIFFERENT reason (Intent). Both scores appear in the Scoring Architecture panel on the single-topic dashboard — see `references/single-topic-dashboard-rules.md` for the rendering spec. + +## Before You Start — Read These + +1. **`CLAUDE.md`** (bundled with this skill) — full orchestrator / project instructions. Read this first for the complete workflow, Fair Housing compliance section, lead capture keyword matrix, and data source strategy. +2. **`references/market-config.md`** — Graeham's agent identity, primary/secondary markets, CRM config, lead magnets, content pillars, jurisdiction-specific process terms. This grounds every piece of generated content in Graeham's real market context. +3. **`references/research-sources.md`** — Complete documentation of every data source used in Per-Topic Research (Research & Discover), including what to pull, how to pull it, what to look for, and the scoring rubric. +4. **`references/single-topic-dashboard-rules.md`** — 12 strict rules + 16-item self-check for building single-topic production dashboards. Reference implementation: `online-content/dashboards/single-topic/2026-04-18-epa-two-years-homicide-free-production.html`. Template builder: `templates/single-topic-dashboard-builder.py`. +5. **Shared Branding** — Before generating any client-facing output, read the shared branding reference at `../shared-references/branding.md` for consistent colors, fonts, and UI components. + +## Agent Identity + +You are generating content as Graeham Watts — REALTOR at Compass, DRE# 01466876. Primary market is East Palo Alto. Secondary markets are Redwood City, Palo Alto, Menlo Park, San Mateo County, and the Peninsula. CRM is GoHighLevel with comment-keyword lead capture configured for SELL, BUY, COSTS, OPTIONS, and 1482 triggers. + +## Production Rules (Non-Negotiable) + +These rules exist because ignoring them caused real bugs in production on April 18, 2026. Every one has a specific failure mode attached. Follow them every time. + +### Rule 1: HTML Generation Safety — Use Python, Not Bash Heredoc + +NEVER use `cat > file << 'EOF'` bash heredoc to write HTML files. Bash silently escapes `\!` characters in HTML comments to `\\!`, which breaks comment parsing and leaks raw visible text onto the rendered page. + +ALWAYS write HTML via Python `Path.write_text()`: + +```python +from pathlib import Path +Path("online-content/dashboards/single-topic/my-dashboard.html").write_text(html_content, encoding="utf-8") +``` + +After writing, VERIFY with: `grep -c '<\\\!--' file.html` — must return 0. Reject and rewrite if non-zero. + +**Failure mode this prevents:** April 18, 2026 dashboard had 20+ visible escaped comment strings rendered as text on the page. + +### Rule 2: Mandatory Screenshot-Loop After HTML Output + +After writing any HTML to `online-content/dashboards/` (either `weekly-calendars/` or `single-topic/`) or `online-content/newsletters/`, read `skills/website-builder/references/screenshot-loop.md` and execute it BEFORE `git push`. Minimum 1 iteration, target 3. + +Sandbox Chromium install often fails. Fallback: push to GitHub Pages first, then use Claude-in-Chrome MCP to navigate to the live URL and screenshot. If bugs found, fix locally, push again, re-verify. + +**Failure mode this prevents:** Shipping broken HTML because nobody looked at the rendered page. Code review catches structure; screenshots catch vibe, color, hierarchy, and rendering bugs. + +### Rule 3: PROMPT_LIBRARY Default for Multi-Format Dashboards + +For any multi-format deliverable dashboard (single-topic OR weekly), use the `window.PROMPT_LIBRARY` JS object pattern with a Copy button per format. Each prompt includes Agent Identity + Fair Housing + DATE/YEAR QC + Timing Self-Check + Voice + Topic + AEO stats + Key Facts + GHL CTA + **Humanizer Block (see Rule 8)** + format-specific deliverable spec. + +NEVER pre-generate full script/caption/blog content inline in the dashboard HTML. Reasons: +- Risks truncation on long outputs +- Burns context on content external AIs generate better +- Makes the dashboard hard to iterate on + +Reference: `online-content/dashboards/weekly-calendars/2026-04-27-production-calendar-v7.html`. + +**Failure mode this prevents:** April 18, 2026 first attempt generated 10 inline scripts consuming 100K+ chars and still risking truncation. + +### Rule 4: Timing Self-Check in Every Script Prompt + +Every script-generation prompt MUST include this block (same enforcement as DATE/YEAR QC): + +``` +TIMING SELF-CHECK (FOR SCRIPT OUTPUTS ONLY): +Before emitting any script, calculate: (spoken_word_count / 150 WPM) * 1.15 = target_minutes. +Show the math in the output. NEVER default to generic durations like "8-10 min". +``` + +**Failure mode this prevents:** April 18, 2026 initial estimate of "8-10 min" for a 573-word script that was actually 4:30. + +### Rule 5: Single-Topic Dashboard Output Format + +Single-topic content packages go to: + +``` +online-content/dashboards/single-topic/YYYY-MM-DD-{slug}-production.html +``` + +NOT a markdown file. HTML goes to GitHub Pages for the live URL the production team uses. Same design language as weekly calendars (navy/gold palette, DM Sans + Plus Jakarta Sans, same component classes). + +**Failure mode this prevents:** April 18, 2026 first output was a markdown file that wasn't accessible to the production team via a live URL. + +### Rule 6: Gold Is A Brand Color — Use Sparingly + +`--gold` (#C5A258) is Graeham's real estate brand color. Reserve for brand moments only. Maximum ~10 instances per dashboard. + +**Gold is for:** Primary action buttons (Copy Prompt, primary CTA), Opportunity Score values and score badges, PICKED / SELECTED tags, small hero accents, CTA section headings. + +**Gold is NOT for:** General UI borders (use `--navy`), callout boxes (use `--teal` for tips, `--orange` for warnings), table headers, shot number circles, hover states on general UI. + +**Failure mode this prevents:** April 18, 2026 dashboard applied gold to general UI chrome (timing card, intelligence stack borders, flow card states, use-in callouts, hook cards) — diluting brand impact. + +--- + + +## Self-Check Before Shipping + +Before declaring any content-creation task complete, run this checklist explicitly in your response before pushing: + +1. [ ] Timing calculation shown with math (word_count / 150 * 1.15 = minutes) +2. [ ] Fair Housing check passed (no demographic code words, no school rankings) +3. [ ] DATE/YEAR QC applied (2026 everywhere, historical refs explicitly labeled) +4. [ ] AEO statements open with date anchor ("As of April 2026...") +5. [ ] GHL keyword is valid (in the keyword matrix) +6. [ ] HTML output: `grep -c '<\\\!--'` returns 0 (no escape bug) +7. [ ] HTML output: screenshot-loop executed, visual verified +8. [ ] HTML output: PROMPT_LIBRARY used (not inline pre-generated content) +9. [ ] HTML output: gold usage is brand-only (Rule 6) +10. [ ] Source citations included with clickable links +11. [ ] Single-topic output: saved to `online-content/dashboards/single-topic/` as HTML +12. [ ] HTML pushed to GH Pages +13. [ ] **Humanizer pass run on all written prose deliverables** (see Rule 7 below) +14. [ ] **Weekly calendar uses single-file canonical pattern** — no -all/-blogs/-videos/-research variants created (Rule 11) +15. [ ] **All visual dashboard sections present** — Hero, Audience nav, Run note, Research/Live Data Layer, Performance Signal (with charts), Freshness, Pipeline Diagram, Calendar grid, Video Content, Blog Content (Rule 10) +16. [ ] **Orphan-href audit passed** — `grep -oE 'href="[^"]*\.html"'` shows zero relative-html-file links to files not in the same commit (Rule 9) +17. [ ] **COPY_DATA Humanizer Block injection verified** — JSON parses cleanly, 15 prose entries contain "HUMANIZER RULES" string, 5 ssml entries do NOT contain it (Rule 8) + +--- + +### Rule 7: Humanizer Pass on All Written Prose (Non-Negotiable) + +Every written deliverable produced by Phase G — scripts, blog posts, ad copy, social captions, newsletter sections, CTAs, AEO statements — must be run through the `humanizer` skill before it goes to Adrian, Peter, or Graeham for review. Spoken scripts especially: an em-dash-heavy "stands as a testament" sentence is invisible on paper but sounds like a robot when Graeham reads it on camera, and the engagement drops. + +**Apply humanizer to:** +- Long-form YouTube scripts (Pt 1 script body) +- Short-form scripts (YT Shorts, IG Reels, TikTok) +- Blog post body copy +- Social captions (IG, FB, LinkedIn, GMB) +- Newsletter section prose +- Ad copy variants (FB, Google) +- AEO cite-ready statements (the prose around the stat, not the stat itself) +- Alt hook variants + +**Do NOT apply humanizer to:** +- The SSML / ElevenLabs audio-tag blocks (those are markup, humanizer would break them) +- Shot list bullets and inline shot direction tags (`[TALKING HEAD]`, `[B-ROLL: ...]` — these are production metadata) +- Editing Notes for Jason (production directions, not reader-facing prose) +- JSON-LD schema markup +- YouTube SEO metadata fields (title, description tags, keyword lists) +- GHL keyword strings (`SELL`, `BUY`, etc.) +- Raw research data JSON + +**How to invoke during Phase G:** +1. Generate the draft script / post / caption as usual through the existing phase pipeline. +2. Separate the reader-facing prose from the production metadata (SSML, shot lists, editing notes). +3. Pass the reader-facing prose to the humanizer skill with Graeham's voice as the calibration sample (first-person, conversational, specific numbers over abstract claims, zero hype, no em-dash overuse). +4. Replace the original prose with the humanized version. +5. Re-stitch with the unchanged production metadata. +6. Continue to the rest of the Self-Check. + +**Failure mode this prevents:** Scripts that read like ChatGPT wrote them get flagged by viewers in seconds. YouTube comments call it out. Engagement metrics drop. This rule existed informally — making it explicit ensures every generated package gets the pass before going to the production team. + +--- + +### Rule 8: Humanizer Block in Every PROMPT_LIBRARY Entry (Non-Negotiable) + +Rule 7 covers the case where this skill generates content directly and runs the `humanizer` skill as a post-pass. But the PROMPT_LIBRARY pattern (Rule 3) hands the actual generation to an external AI tool — Adrian, Peter, or Graeham copy a prompt and paste it into ChatGPT, Claude.ai, Gemini, or wherever. They cannot run the humanizer skill at that point. The fix: embed the humanizer rules INSIDE each prompt so the external AI generates already-clean output from the start. + +**Every PROMPT_LIBRARY entry that produces written prose (scripts, blogs, captions, ad copy, newsletter sections, AEO statements) must include the Humanizer Block below as a standard preamble item, placed AFTER Voice & Style and BEFORE the format-specific deliverable spec.** + +#### Canonical Humanizer Block (copy verbatim into every prose-generating prompt) + +``` +HUMANIZER RULES (apply throughout the output — do NOT mention these rules in the response itself, just follow them): + +Avoid these AI-tell patterns: +- Em dashes — use commas, periods, or parentheses instead. +- Significance inflation: "stands as a testament," "marks a pivotal moment," "evolving landscape," "key turning point," "deeply rooted," "indelible mark." +- Promotional language: "boasts a," "nestled in," "vibrant," "rich" (figurative), "stunning," "must-see," "groundbreaking," "renowned," "breathtaking." +- Vague attributions: "experts say," "industry observers note," "research suggests" without naming the actual source. +- "-ing" tail clauses that add fake depth: "highlighting...," "underscoring...," "ensuring...," "contributing to...," "reflecting...," "showcasing..." +- Forced rule-of-three lists: "innovation, inspiration, and industry insights" / "streamlining, enhancing, fostering." +- Negative parallelism: "It's not just X, it's Y" / "Not only X, but Y." +- Tailing negation fragments: "no guessing," "no wasted motion" tacked onto sentences. +- Copula avoidance: "serves as," "stands as," "functions as," "represents a." Use "is" / "are" / "has." +- Sycophantic openers: "Great question," "I hope this helps," "Certainly," "Of course," "You're absolutely right." +- Knowledge-cutoff disclaimers: "As of my last update," "While specific details are limited." +- Excessive hedging: "could potentially possibly," "might have some effect on." +- Generic positive conclusions: "the future looks bright," "exciting times lie ahead," "represents a major step." +- Inline-header vertical lists where every bullet starts with "**Bold Header:**" followed by a colon. +- Curly quotes (use straight quotes only). +- Mechanical boldface — reserve bold for true emphasis, not decoration. +- False ranges: "from X to Y" when X and Y aren't on a meaningful scale. +- Persuasive authority tropes: "the real question is," "at its core," "what really matters," "fundamentally." +- Signposting announcements: "Let's dive in," "Here's what you need to know," "Now let's look at." +- Hyphenated word-pair clusters: "high-quality, data-driven, client-facing, decision-making" all in one sentence. + +Instead: +- Vary sentence rhythm. Mix short punchy sentences with longer flowing ones. +- Use first person when it fits: "I keep coming back to," "Here's what gets me." +- Use specific numbers, dates, and concrete details over abstract claims. "$680K-$850K in Woodland Park" beats "competitive pricing in the area." +- Sound like one human talking to another about something that matters. +- Acknowledge complexity and mixed feelings when honest: "This is interesting but also kind of unsettling" beats "This is interesting." +- If the topic has a real edge or controversy, lean into it. Don't sand it down. + +Read the final draft aloud in your head. If any sentence sounds like a press release, a Wikipedia article, or a LinkedIn thought leader, rewrite it. +``` + +#### Block Placement Order in the Prompt Preamble + +``` +1. Agent Identity (Graeham Watts, REALTOR, Compass, DRE 01466876) +2. Fair Housing Guardrails +3. DATE/YEAR QC +4. Timing Self-Check (scripts only) +5. Voice & Style +6. HUMANIZER BLOCK ← inserted here +7. Topic + Key Facts +8. AEO stats +9. GHL CTA / Lead Capture +10. Format-specific deliverable spec +``` + +#### When to Skip the Humanizer Block + +Omit ONLY in prompts that don't generate reader-facing prose: +- SSML / audio-tag generation prompts (markup output, humanizer rules would conflict) +- Shot list generation prompts (production metadata) +- Editing Notes prompts (production directions) +- JSON-LD schema generation prompts +- YouTube metadata field prompts (titles, tags, keywords — these have their own length and format rules) +- Image generation prompts (visual, not prose) + +All other prompts — long-form scripts, short-form scripts, blog posts, ad copy, captions, newsletter sections, AEO statements, alt hooks — MUST include the block verbatim. + +#### Maintenance + +The canonical block above is the single source of truth. When the `humanizer` skill at `skills/humanizer/SKILL.md` is updated with new patterns (new AI-tells observed in the wild), update this block in the same commit so PROMPT_LIBRARY entries stay in sync. The block is intentionally compact — 30-35 lines — to keep prompt size reasonable while covering the patterns that cause the most damage in spoken / read content. + +**Failure mode this prevents:** Rule 7 (post-gen humanizer skill pass) only works when this skill generates content directly. When Adrian/Peter copy a prompt and paste into an external AI tool, Rule 7 doesn't fire — and the resulting script or blog reads like ChatGPT wrote it. Rule 8 closes that gap by moving the humanizer rules upstream into the prompt itself, so the external AI never produces the bad output in the first place. + +#### Canonical prompt-data structure (May 2026 update) + +The CURRENT canonical weekly calendar uses a `const COPY_DATA = { "t1": { "ssml": "...", "prod_video": "...", "blog_brief": "...", "prod_blog": "..." }, "t2": {...}, ... }` JS object with 5 topics × 4 prompt types = 20 entries. Of those, 15 are prose-generating (3 per topic: prod_video, blog_brief, prod_blog) and MUST contain the Humanizer Block. 5 are SSML markup (one per topic) and MUST NOT contain the block (it would break the XML). + +Older variants of the calendar (the `-all.html`, `-blogs.html`, `-videos.html`, `-research.html` quad-file pattern with `const PROMPTS = {...}`) are deprecated as of 2026-05-15 due to two architectural defects documented in Rules 9 and 11 below. New calendars use the single-file COPY_DATA pattern in `2026-05-11-production-calendar.html`. + +--- + +### Rule 9: No Orphan Internal Links (Non-Negotiable) + +Every `href=""` attribute in a generated dashboard HTML file MUST point to one of: +1. An in-page anchor (`href="#section-id"`) where the target id exists in the same file, OR +2. A JavaScript no-op (`href="#"` paired with `onclick`) for setView/setFilter buttons, OR +3. A fully-qualified external URL on a domain that's actually reachable (citation links, social posts, etc.), OR +4. A relative URL to another file that is ALSO pushed to GitHub in the same commit + +**Forbidden:** any `href=` to a sibling HTML file that doesn't exist on GitHub Pages. This was the root cause of the "blog tab 404s" failure on 2026-05-15 — the `-all-humanizer.html` linked to `2026-05-11-blogs.html`, `-videos.html`, `-research.html`, and `-all.html` (the four older variant files) but only the humanizer variants were ever pushed. Every audience tab 404'd silently on the live URL. + +**Pre-push audit (mandatory):** + +```bash +# Extract all hrefs from the dashboard HTML +grep -oE 'href="[^"]*"' dashboard.html | sort -u > /tmp/hrefs.txt + +# For each href that points to a relative .html file: +# 1. Check it exists in /tmp/online-content-clone/dashboards/weekly-calendars/ +# 2. If missing on the remote and not being added in this commit, FAIL the push + +# For each href that's a fully-qualified external URL: +# 3. (optional) HEAD request to confirm 2xx, flag any 404 +``` + +The pre-push audit must run as part of every weekly calendar build. If any `href=` points to a missing local file, STOP and either fix the link or include the target file in the same commit. + +**Failure mode this prevents:** Zombie file references. Files that exist locally in Documents\Claude but never made it to GitHub get cross-linked from pushed files, creating tabs/buttons that 404 on the live URL while looking fine in local preview. + +--- + +### Rule 10: Visual Dashboard Sections Required (Non-Negotiable) + +Every weekly calendar MUST include the following visual dashboard sections, in this order, at the TOP of the file (before the Calendar grid and per-topic sections): + +1. **Hero + audience-tab nav** — header bar, week date range, 5-button filter row (Research / Diagram / Calendar / Video / Blog) wired to `setView()` and `data-audience=""` attributes +2. **Run-note banner** — any blockers (e.g., "Apify blocked at firewall, pivoted to WebSearch") so the production team knows what was fresh vs derived +3. **Research — Live Data Layer** — source cards showing which 8 data sources ran live, blocked, or partial. Color-coded: green = live, red = blocked +4. **Performance Signal — What's Actually Working** — **ApexCharts brushable time-series ONLY** (Chart.js is forbidden for these charts — it lacks brush interaction): + - Instagram Activity Over Time — area chart, last 26 weeks (100 posts via Composio Meta Graph API), dual axis (likes + posts), brush slider below for drag-to-zoom + - YouTube Activity Over Time — area chart, last 14 weeks (50 videos via YouTube Data API v3), dual axis (views + videos), brush slider below + - Engagement Rate Per Post Per Week — line chart, avg per-piece for IG + YT, strips out posting-frequency effect, brush slider below + - Each chart is a PAIR: a main chart (`#xxxChartMain`, height 300) + a brush slider (`#xxxChartBrush`, height 100, marginTop -6) wired via `brush: { target: 'xxxMain', enabled: true }, selection: { enabled: true, ... }` + - Library: `` (or later compatible version) + - The brush pattern lets users drag a window on the bottom slider to zoom the main chart to any time range. This is the canonical "slide it across time" interaction Graeham expects. + - **Top 5 lists** (YouTube Top 5 last 99 videos, IG Top 5 last 20 posts) — render as data tables, not charts. Sortable by views / likes / engagement. +5. **Full Weekly Research Data panel** — collapsible accordion containing the 7 mandatory data tables that back the week's topic picks: + a. Instagram Own-Channel Performance — last 25 posts with caption excerpt, likes, comments, pattern match column (gold-highlighted rows = match week's content patterns) + b. YouTube Own-Channel Performance — last 15 videos with views, likes, comments, pattern match column + c. Google Search Console Topic-Targeted Queries — query, impressions (last 7d), clicks, position, trend WoW, day-the-query-maps-to + d. Reddit Demand Signals — subreddit, thread title, upvotes, comments, topic cluster (star the day-of-week match) + e. Zillow Q&A — question, page, asked count (last 30d), day-the-question-maps-to + f. MLS Pull — metric, current month, year-ago, YoY delta (with trend-up/trend-down color coding) + g. Convergence — Why Each Day Picked — day, topic, sources-converged list, score out of 25 (star the highest-converging day) + Plus a Macro Rates & Permits bullet list (30Y fixed rate, Fed Funds, county permits, notable ADU permits) and a DataForSEO SERP Queue status note. +6. **Freshness Constraints + Citations** — 4-week topic history check, blocked angles, citation URLs (external) +7. **Diagram — How We Built This (10-Step Data Pipeline)** — clickable nodes showing data flow: 4 INPUT nodes → 3 ANALYSIS nodes → 3 OUTPUT nodes +8. **Calendar — Week of [date range]** — 5 day-cards with funnel-tier color coding (TOFU/MOFU/BOFU), GHL keyword chips, click-to-expand topic details +9. **Video Content — All 5 Topics** — per-topic article cards with Copy SSML + Copy Production Prompt buttons +10. **Blog Content — All 5 Topics** — per-topic article cards with Copy Blog Brief + Copy Production Prompt buttons + +**Failure mode this prevents:** Calendars shipped without the visual research dashboard look like prompt dumps and provide no analytical context. The production team can't tell which topics are backed by which data signal, and Graeham can't review the run quality at a glance. Two real production failures led to this rule: + +1. On 2026-05-15 the `-all-humanizer.html` shipped without sections 4 (Performance Signal charts) and 7 (Pipeline Diagram), making it look incomplete next to the prior week's production-calendar.html. +2. Later that same night the rebuilt production-calendar shipped with Chart.js line charts (no brush) and only a 6-card Live Data Layer with no underlying data tables. Graeham flagged it: "the graphs you created are different from the graphs in the previous version — should be the ones where you can slide across time" and "missing a lot of the research data." The fix required transplanting the ApexCharts brushable charts + 7 data tables from `2026-05-11-research.html`. **This rule's section 4 now mandates ApexCharts brushable (not Chart.js) and section 5 enumerates the 7 required data tables explicitly so the omission can't repeat.** + +--- + +### Rule 11: Single Canonical File Pattern (Non-Negotiable) + +A weekly calendar is ONE file, not four. The deprecated pattern was: +- `2026-05-11-all.html` — full view +- `2026-05-11-blogs.html` — blog-track filter +- `2026-05-11-videos.html` — video-track filter +- `2026-05-11-research.html` — research-only filter + +That pattern is **forbidden** going forward. The four files were not in sync (different sizes, different prompt content), the audience tabs cross-linked between them (creating the Rule 9 violations), and maintaining four parallel files for the same week multiplied the surface area for bugs by 4x. + +The canonical pattern is **ONE file per week**: +- `2026-MM-DD-production-calendar.html` (where MM-DD is the Monday the week starts) + +Audience filtering happens **via in-page JavaScript** using: +- `data-audience="blog all"` attributes on each section +- A `setView('blog')` function that hides/shows sections matching the selected view +- The "Show everything" link resets to `setView('all')` + +This means clicking "Blog Track" doesn't navigate to a sibling file — it just filters the current file. No 404 risk. No drift between variants. One file to maintain, one URL to share, one place to verify before pushing. + +**Existing deprecated files** (`2026-05-11-all.html`, `-blogs.html`, `-videos.html`, `-research.html`, and their `-humanizer` siblings) should be removed from `Graehamwatts/online-content` in a cleanup commit. They remain locally for archival but should not be referenced or linked to from any new file. + +**Failure mode this prevents:** Variant proliferation. Each variant file is another place where the prompt data can drift, where href targets can break, and where a humanizer update has to be applied 4x instead of 1x. + +--- + +## Fair Housing Guardrails (Non-Negotiable) + +NEVER generate content that: +- Describes neighborhoods by demographics (race, religion, national origin, family status, disability) +- Uses "safe / good areas / family-friendly / up-and-coming" as a proxy for demographic signaling +- Ranks or rates schools as a primary selling point for a neighborhood +- Promotes kickback arrangements with lenders, inspectors, or other vendors + +Neighborhood content is limited to: property features, price ranges, market trends, lot sizes, amenities, architecture, housing stock age, HOA structure, zoning, new development, commute/transit facts, and walkability. When in doubt, reframe or drop the topic. This is both the law and Graeham's brand standard. + +--- + +## THE PER-TOPIC WORKFLOW (Per-Topic Research is PER-TOPIC, not weekly) + +> **Rewritten April 2026.** Previous version of Per-Topic Research pulled 8 weekly-scope sources and applied a 10-pt scoring rubric — that's weekly-planning work and it belongs in `content-calendar`. This skill's Per-Topic Research now does one job: gather citations, stats, and quotes for ONE topic that's already been selected. + +When a topic arrives here (from `content-calendar`'s weekly plan, or a direct ask like "build a package on X"), Per-Topic Research pulls the *research data panel* that backs the single-topic dashboard. When the request is "what should I post this week?" — **hand it to `content-calendar`, not Per-Topic Research.** + +### Phase 0a — Clarifier Check (ASK BEFORE RESEARCHING) + +Before pulling any data, confirm the scope in ONE question. Don't skip this step — it prevents a full Per-Topic Research run for a request that actually wanted weekly planning, or vice versa. + +If the user's ask is ambiguous, confirm in this form: + +> "Before I start — which of these are you asking for? +> (a) **Per-topic content package** — you already know the topic (e.g., 'EPA homicide-free story', 'this new $2.1M listing', 'AB 1482 explainer'). I'll pull research for THAT topic and build the full dashboard. +> (b) **Weekly planning** — you want me to decide which topics to cover this week. For that I should hand off to `content-calendar`. +> (c) **Raw research only** — you want current market signal dumped to the chat, no package built yet." + +If the ask is unambiguous (user provided a specific topic, a listing, a YouTube URL, or breaking news), skip Phase 0a and proceed to Per-Topic Research. + +### Per-Topic Research (citations & stats for ONE topic) + +**Read:** `references/research-sources.md` for source documentation. + +Given ONE already-selected topic, pull the evidence that will populate the dashboard's "Show Full Research Data" panel — statistics, quotes, news clippings, permits, MLS comps, GSC queries that match this topic. **No scoring happens here.** The Opportunity Score is already done (content-calendar set it when the topic was selected). The Intent Score runs in Phase 3. + +#### What to pull (scoped to the ONE topic) + +1. **Topic-matched MLS stats** — only the price bands / DOM / inventory numbers that back this topic. If the topic is "EPA homes under $700K," pull that bucket. Do NOT pull the full county stat sheet. +2. **Topic-matched GSC queries** — the specific queries from Search Console that this topic targets. Note impressions, position, and whether it's a rising query. +3. **Topic-matched local news/permits** — web search AND city gov search for this topic's exact subject (e.g., "East Palo Alto homicide rate 2026," "AB 1482 2026 amendments"). +4. **Topic-matched social performance** — did similar topics perform well in the last 60 days on Graeham's channels? (This feeds the dashboard's "format recommendation" based on what worked for similar content.) +5. **Topic-matched competitor content** — have competitors covered this exact angle in the last 30 days? Use Apify datasets if fresh, Claude-in-Chrome for manual check otherwise. +6. **Topic-matched Reddit/audience signal** — pull relevant snippets from the most recent `outputs/ideation-topics-*.json` that match this topic's keywords. +7. **Simulated LLM Query Capture** — Before running this step, confirm the target geography. If the topic makes it obvious (e.g., "EPA homicide-free story" → East Palo Alto, "AB 1482 for RWC landlords" → Redwood City), proceed. If the topic is market-agnostic or could apply to multiple of Graeham's markets, ask: *"Which geography should I use for the LLM query simulation — East Palo Alto, Redwood City, Menlo Park, Palo Alto, or broader Peninsula/Bay Area?"* Use the confirmed geo in every prompt below. + + Query Claude (yourself), GPT-4 (via web or API), and Perplexity with this prompt for each relevant persona (BUYER, SELLER, RELOCATOR, INVESTOR — pick the 1-2 that fit the topic): + + > *"If a [PERSONA] were researching [TOPIC] in [GEO], list 15-25 specific questions they would likely ask an AI assistant. Output as a JSON array of strings."* + + Run for each persona × LLM combination. Deduplicate across results. Score each question by **cross-LLM agreement**: questions surfaced by 2+ LLMs independently are the highest-priority AEO targets — they represent what AI search engines themselves expect buyers/sellers to ask. Questions surfaced by only one LLM are lower priority but still useful. + + **Use the output to:** + - Identify which questions your content package must answer directly (especially the 2+ LLM agreement ones) + - Prioritize which AEO cite-ready statements go in the blog derivative + - Determine the FAQ schema block questions for the blog's JSON-LD markup + - Inform the hook for the video script (questions with high cross-LLM agreement = proven demand signal) + + This is a leading indicator for AEO — it surfaces what buyers/sellers will ask AI search engines 12-24 months before that demand shows up in Google Search Console. + +Do NOT pull the broad weekly trend data Per-Topic Research previously pulled. That lives in content-calendar now. + +#### Output — Research Data Panel (JSON) + +Save research as `outputs/research-{topic-slug}-{timestamp}.json` with this shape: + +```json +{ + "topic_slug": "epa-homicide-free-story", + "topic_title": "East Palo Alto Two Years Homicide-Free — What It Means For Home Values", + "pulled_at": "2026-04-22T18:00:00Z", + "mls_stats": [ { "metric": "...", "value": "...", "as_of": "..." } ], + "gsc_queries": [ { "query": "...", "impressions": 0, "position": 0.0, "rising": true } ], + "news_and_permits": [ { "source": "...", "headline": "...", "url": "...", "date": "..." } ], + "social_signal": { "similar_topic_avg_reach": 0, "best_format": "IG Reel 30s", "sample_size": 4 }, + "competitor_coverage": [ { "competitor": "...", "covered_angle": "...", "views": 0 } ], + "reddit_signal": [ { "thread_title": "...", "url": "...", "upvotes": 0 } ], + "llm_anticipated_queries": [ + { + "question": "...", + "persona": "BUYER", + "cross_llm_agreement": 3, + "llms_that_surfaced": ["claude", "gpt4", "perplexity"], + "priority": "HIGH", + "aeo_use": "FAQ block + cite-ready statement" + } + ] +} +``` + +`cross_llm_agreement` is 1–3 (how many of the three LLMs surfaced this question). Priority: HIGH = 3, MEDIUM = 2, LOW = 1. The `llm_anticipated_queries` array should be sorted HIGH → LOW before saving. + +This JSON is the single source of truth for the "Show Full Research Data" accordion on the single-topic dashboard. + +### If Graeham asks the ambiguous questions — routing table + +| User says | Runs where | +|---|---| +| "What should I post this week?" | **`content-calendar`** (weekly planning + Opportunity scoring). NOT Per-Topic Research. | +| "Plan next week's 5 topics" | **`content-calendar`** | +| "Run research" / "What's happening in EPA?" | Phase 0a clarifier → usually content-calendar weekly research, unless user specifies one topic | +| "Build a content package for [specific topic]" | Per-Topic Research here, per-topic | +| "I have a new listing, give me content" | Per-Topic Research here, per-topic (the listing IS the topic) | +| "Transcribe this YouTube video and build content from it" | Phase 0 (ingestion) → Per-Topic Research here, per-topic | + +--- + +### Phase S — Select & Plan + +User picks 2-3 topics from the Content Opportunity Report. Engine confirms: +- Which formats to generate for each topic (video script, newsletter section, blog post, ad copy, social posts) +- Which platforms each format targets +- Funnel tier assignment (TOFU / MOFU / BOFU) for each topic +- Any dependencies (e.g., "this topic needs MLS data screenshots for the carousel") + +Confirm the plan with Graeham before proceeding to generation. + +--- + +### Phase G — Generate Content + +#### Pre-Generation Topic-Type Routing + +Before generating any formats, check the topic type and route through the appropriate module: + +| Topic type | Route through | Then | +|---|---|---| +| Market update / monthly report / weekly market read / "is now a good time to buy/sell" | `modules/market-update-narrative/README.md` | Module returns a narrative outline JSON; pass to Phase 5 script-writer for final format rendering | +| Listing spotlight (specific property) | `../listing-remarks-writer/SKILL.md` for the source-of-truth listing description; `../listing-photo-captioner/SKILL.md` for carousel/photo captions | Phase 5 builds derivatives from those outputs | +| Stale listing / price reduction angle | `../price-reduction-angle-generator/SKILL.md` (PRIVATE — seller-only, never public content) | Output is for agent's seller convo, NOT for public posting. Do not generate downstream public formats. | +| Education / how-to / process / decision frameworks | No pre-module — go directly to Phase 5 | Phase 5 handles standard content generation | +| YouTube source-driven repurposing | Phase 0 source-ingestion (`scripts/youtube_transcriber.py`) or `../youtube-scraper/SKILL.md` for channel monitoring | Returns transcript + metadata; Phase 5 builds derivatives | + +If the topic doesn't match any of the above, default to "education" routing and let Phase 5 handle it directly. + +#### Format Generation + +For each selected topic, produce ALL relevant formats using the existing phase pipeline: + +1. **Video Script** — Long-form + short-form with clear section headers (see Script Output Format below). Includes ElevenLabs SSML block, inline shot directions, editing notes for Jason, and AI video prompts. +2. **Newsletter Section** — HTML formatted per the newsletter module. See `modules/newsletter/` and `../newsletter-generator/SKILL.md`. +3. **Blog Post Draft** — SEO-optimized with AEO cite-ready statements, meta description, title tag, target keywords, JSON-LD schema markup (Article + FAQPage + VideoObject as applicable), RSS-feed-based internal linking to existing graehamwatts.com posts, YouTube embed + timestamp link patterns when source video exists. +4. **Ad Copy Variants** — If the topic lends itself to paid promotion: Facebook ad copy, Google ad copy, with multiple hook variants for A/B testing. +5. **Social Posts** — Platform-specific: IG caption with hashtags and GHL keyword CTA, Facebook post, LinkedIn post (if applicable), Google My Business post. + +The generation phase uses the existing 6-phase pipeline (Phase 0 through Phase 5) documented below for the actual content creation logic. Per-Topic Research replaces the "what should I write about?" question — by the time we reach Phase G, we already know exactly what topics to cover and why. + +--- + +### Phase A — Review & Approve + +Present all generated content to Graeham (and Adrian if applicable) for approval. For each piece: +- Show the content with its section headers +- Note the source data that inspired it (from Per-Topic Research) +- Flag any items that need fact-checking or data verification +- Ask for approval, revision requests, or rejection + +--- + +### Phase D — Distribute + +Once approved: +- **Newsletter:** Assemble full newsletter from selected sections, draft in Gmail via Gmail MCP +- **Blog:** Ready-to-publish format with SEO metadata +- **Social:** Platform-specific posts queued for posting +- **Ads:** Ready for deployment with targeting recommendations +- **Video:** Hand off to heygen-elevenlabs-renderer for avatar video rendering (see Auto-Render Hand-off section below) + +--- + +## THE CONTENT GENERATION PIPELINE (Used by Phase G) + +The phases below contain the detailed content creation logic. During the per-topic workflow, these are invoked during Phase G (Generate Content) after a topic has been selected. They can also be invoked directly when Graeham already knows exactly what topic he wants to cover and skips research. + +Each phase has its own detailed instruction file in `references/phases/`. Read the phase file before executing that phase. + +### Phase 0 — Source Ingestion (YouTube) + +**Read:** `references/phases/source-ingestion/instructions.md` + +Phase 0 has TWO modes. The orchestrator picks the right one based on what the user provided. + +#### Mode A — Single-URL Transcription + +**When to use:** the user pasted a specific YouTube video URL ("here's a video, give me content ideas from it" / "transcribe this and adapt it for EPA sellers"). + +**How it works:** Two-tier transcription system — tries free caption pull first (instant), falls back to OpenAI Whisper (free, local, ~1-3 min) for videos without captions. Run `scripts/youtube_transcriber.py` for the transcription. + +**Output:** `outputs/transcripts/transcript-{video_id}-{timestamp}.txt` + Source Ingestion Brief. + +**Triggers:** +- User pastes a YouTube watch URL (`youtube.com/watch?v=...` or `youtu.be/...`) +- User pastes a Shorts URL (`youtube.com/shorts/...`) +- User says "transcribe this video," "give me content from this video," "adapt this video" + +#### Mode B — Channel Monitoring (Repurposing Pipeline) + +**When to use:** the user wants to monitor a YouTube channel for new uploads and auto-repurpose each new video into content derivatives. Common use cases: monitor Graeham's own channel for his uploads, monitor a competitor channel for their topics, monitor an industry voice for trend signal. + +**How it works:** invoke `../youtube-scraper/SKILL.md` (standalone skill) to scan the channel for new uploads in the last 24 hours (or user-specified window), check against `processed_videos.txt` to skip already-handled videos, and extract metadata + transcript for each new video. The scraper delegates transcript work to `scripts/youtube_transcriber.py` so the same transcription pipeline handles both modes. + +**Output:** `outputs/scraper/current_video_{N}.md` files (one per new video) + transcript files in `outputs/transcripts/`. + +**Triggers:** +- User pastes a YouTube channel URL (`youtube.com/@channelname` or `/channel/UC...` or `/c/channelname`) +- User says "check my YouTube channel for new uploads," "monitor [channel] for new videos," "scrape this YouTube channel" +- Scheduled task fires (daily check on Graeham's channel + competitor watch list) + +**After Mode B completes:** for each new video, either: +1. Run Mode A's downstream flow (skip Phases 1-2, go to Phase 3 → Phase G to build derivatives), OR +2. If multiple new videos found, batch-process: each video becomes its own per-topic content package run + +**Important:** Mode B is for *channel monitoring*, not single-URL transcription. Don't fire Mode B when the user pasted a single video URL — use Mode A. Don't fire Mode A when the user pasted a channel URL — use Mode B. + +#### Skip Phase 0 entirely when: + +- The user is asking for original content ideas with no external video source — go straight to Phase 1 +- The user already has a topic and just wants the content package — go to Per-Topic Research (per-topic research) → Phase G + +#### After Phase 0 Completes (Either Mode): + +If Phase 0 produced a transcript / source ingestion brief, **skip Phases 1-2** (the source video replaces ideation) and jump to Phase 3 (BOFU Intent Scorer) with the brief, or go directly to Phase G (Script Writer) for a quick script. + +If Mode B produced multiple new videos, treat each one as its own per-topic content package run — they batch-feed into Phase G in sequence. + +### Phase 1 — BOFU Query Generator + +**Read:** `references/phases/bofu-query-generator.md` (absorbed phase reference; was a standalone skill prior to May 2026 consolidation) + +Generate 230+ localized bottom-of-funnel query patterns across 5 inquiry types (SELL, BUY, COSTS, OPTIONS, 1482). Output: `outputs/bofu-queries-{timestamp}.json`. + +### Phase 2 — Content Ideation Engine + +**Read:** `references/phases/content-ideation-engine/instructions.md` and its reference files: +- `references/phases/content-ideation-engine/references/apify-actors.md` — Apify actor config +- `references/phases/content-ideation-engine/references/subreddit-list.md` — target subreddits with priorities +- `references/phases/content-ideation-engine/references/query-templates.md` — search query templates +- `references/phases/content-ideation-engine/references/ideation-rubric.md` — what signals to extract + +Pull live audience demand via Apify `trudax/reddit-scraper-lite` (primary) + Claude web search + browser deep dives (supplementary). Run `scripts/run_reddit_ideation.py` for the Reddit scrape. Requires `APIFY_API_TOKEN` in environment. + +Output: `outputs/ideation-raw-{timestamp}.json` and `outputs/ideation-topics-{timestamp}.json`. + +### Phase 3 — BOFU Intent Scorer + +**Read:** `references/phases/bofu-intent-scorer.md` (absorbed phase reference; was a standalone skill prior to May 2026 consolidation) + +> **This is the INTENT SCORE, not the OPPORTUNITY SCORE.** It classifies each topic's BOFU intent (DECISION / CONSIDERATION / AWARENESS) for funnel-mix purposes. It does NOT decide whether a topic should be covered this week — that job belongs to the 25-pt Opportunity Score in `content-calendar`. See the Scoring Architecture table at the top of this file. + +Scores each candidate topic on the 6-criteria rubric: Inquiry Type Match, Intent Matrix Position, Source Confirmation, Emotional Temperature, Local Relevance, and Freshness (penalties + bonuses from `topic-history.json`). Base score max 25; freshness adjusts ±5. Keep ≥18/25 after freshness applied. Output: `outputs/scored-topics-{timestamp}.json`. + +### Phase 4 — Funnel Tagger + +**Read:** `references/phases/funnel-tagger/instructions.md` + +Tag surviving topics TOFU / MOFU / BOFU. Default mix 40/30/30. Override based on user goal (lead gen bias = 20/30/50, audience growth bias = 60/25/15, fresh-listing bias = heavy BOFU for that listing's market). Output: `outputs/tagged-topics-{timestamp}.json`. + + +### Phase 4.5 — Format Ranker (PropertyIQ CI v1.0) + +**Read:** `references/phases/format-ranker.md` + +For each scored, funnel-tagged topic, produce a ranked list of which formats (YT Long, YT Short, IG Reel, TikTok, Carousel, Blog, GBP, Facebook, Email) to produce and in what order. Uses the Format Type Scoring Formula from PropertyIQ Content Intelligence v1.0 with workflow constraint applied. Tells Phase 5 which derivatives to generate vs skip. Required for any multi-derivative content package. + +### Phase 5 — Script Writer + +**Read:** `references/phases/script-writer/instructions.md` and its reference files: +- `references/phases/script-writer/references/content-pillars.md` — Graeham's content pillar framework +- `references/phases/script-writer/references/platform-specs.md` — per-platform length/format rules +- `references/phases/script-writer/references/cross-posting-matrix.md` — cross-post adaptation matrix +- `references/phases/script-writer/references/voice-and-style.md` — Graeham's voice guide +- `references/phases/script-writer/references/seo-keywords.md` — SEO keyword set +- `references/phases/script-writer/references/aeo-geo-requirements.md` — Answer Engine Optimization + Geo requirements +- `references/phases/script-writer/references/lead-capture-keywords.md` — GHL comment-keyword automation map + +**Conditional reference files — read only when generating blog post derivative:** +- `references/phases/script-writer/references/schema-markup-templates.md` — JSON-LD schema templates (Article required, FAQPage when blog has FAQ section, VideoObject when YouTube embed exists, HowTo when content is step-by-step, BreadcrumbList optional) +- `references/phases/script-writer/references/rss-internal-linking.md` — scrape graehamwatts.com RSS / sitemap / blog index, identify 2-4 semantically relevant existing posts, insert inline links naturally in body +- `references/phases/script-writer/references/youtube-embed-patterns.md` — responsive iframe embed + timestamp link patterns when source is a YouTube video + +Produce multi-platform content packages: hook, short-form script, long-form script, caption, hashtags, comment-keyword CTA, cross-post matrix, AND an **ElevenLabs-Ready Variant** (v3 audio tags + v2 break-tag fallback + voice settings block) for every script so Graeham can paste directly into ElevenLabs with no guessing on inflection. See `references/phases/script-writer/references/elevenlabs-audio-tags.md`. Output: `outputs/content-package-{timestamp}.md`. + +--- + +## Video Duration Estimation (Mandatory Calculation) + +NEVER guess or default to generic durations like "8-10 minutes." Every script MUST include an explicit timing calculation based on actual word count: + +1. **Count the actual words** in the script body (exclude shot directions, editing notes, and metadata) +2. **Average speaking pace:** 150 words per minute for conversational delivery +3. **Add 15%** for pauses, transitions, and B-roll cuts +4. **Formula:** `(word_count / 150) × 1.15 = estimated minutes` +5. **The target duration in the section header MUST match this calculation** + +Examples: +- 150-word script → (150/150) × 1.15 = ~1.15 minutes → "Target: ~1 minute" +- 750-word script → (750/150) × 1.15 = ~5.75 minutes → "Target: ~6 minutes" +- 1500-word script → (1500/150) × 1.15 = ~11.5 minutes → "Target: ~11-12 minutes" + +If the script is only ~150 words, it is a 1-minute video, NOT an 8-minute video. Be accurate. Show the word count and calculation in a comment at the top of the script output so the estimate is verifiable. + +## Script Output Format (Required Section Headers) + +Every script output MUST use the following visually distinct section headers so that Adrian, Peter, or John can grab just their section without confusion. Each section is self-explanatory and separated by a clear visual divider: + +``` +═══════════════════════════════════════════════════ +📹 LONG-FORM SCRIPT (YouTube — Target: [X] minutes) +Platform: YouTube | Format: Talking head + B-roll +═══════════════════════════════════════════════════ + +[script content here] + +═══════════════════════════════════════════════════ +📱 SHORT-FORM SCRIPT (Reels / Shorts / TikTok — Target: [X] seconds) +Platform: Instagram Reels, YouTube Shorts, TikTok +Cut from: Long-form timestamp [X:XX - X:XX] OR record separately +═══════════════════════════════════════════════════ + +[script content here] + +═══════════════════════════════════════════════════ +🎬 SHOT LIST — Hand to production team (Peter/John) +═══════════════════════════════════════════════════ + +[shot list here with numbered shots, each with: shot description, duration estimate, location/setup notes] +``` + +These headers are non-negotiable. Every script output — whether standalone or embedded in a V6 Production Calendar — MUST start each piece with its corresponding header block. Do not omit headers, do not merge sections, do not use plain markdown headers instead. The visual dividers (`═══`) ensure each section is scannable when printed or viewed on a phone. + +## V6 Production Bible Integration + +When scripts are generated as part of a V6 Production Calendar (content-calendar skill), the +output format changes from standalone markdown to **embedded HTML derivative panels** inside the +hosted calendar page. This section documents the V6-specific requirements. + +### Derivative Format System + +Every content day MUST produce scripts for ALL of these platform formats: + +| Format | Key Specs | Notes | +|--------|-----------|-------| +| **YouTube Long** | 8-15 min, 16:9, 1080p | Core asset — fullest script with all production details | +| **YouTube Short** | 30-59 sec, 9:16, 1080p | Strongest hook + one key insight + CTA | +| **IG Reel #1** | 30-60 sec, 9:16, 1080p | Hook-first, face-to-camera, caption overlay | +| **IG Reel #2** | 15-30 sec, 9:16, 1080p | Different angle/hook from Reel #1, B-roll heavy | +| **IG Carousel** | 5-10 slides, 1:1 or 4:5 | Key stats/facts as visual slides, swipe CTA | +| **TikTok** | 30-60 sec, 9:16, 1080p | More casual tone, trending audio hook if applicable | +| **Blog** | 800-1200 words, SEO-optimized | AEO-ready with cite-worthy key statements | +| **GMB (Google My Business)** | 100-300 words, 1 image | Local SEO post, location-tagged | +| **Facebook** | Cross-post from primary + FB-native caption | Longer caption OK, link in post | + +Each derivative panel includes: full script, platform specs, caption with hashtags, description/ +SEO metadata, posting instructions, and GHL keyword CTA. + +### Inline Shot Direction Tags + +Every script (especially the YouTube Long core asset) MUST include inline shot direction tags +embedded directly in the script text. These tell Jason (the video editor) exactly what visual +to use at each moment: + +``` +[TALKING HEAD] — Graeham speaking directly to camera +[B-ROLL: description of footage needed] — Overlay footage +[TEXT OVERLAY: "exact text to display"] — On-screen text/graphics +[DRONE: description of aerial shot] — Drone footage +[SCREEN RECORD: description of what to capture] — Screen recording +[TRANSITION: type] — Cut/dissolve/swipe transition +``` + +Place these INLINE within the script, not as a separate section. Example: +``` +[TALKING HEAD] "If you own rental property in California, you need to know about AB 1482." +[TEXT OVERLAY: "AB 1482 — California Tenant Protection Act"] +[B-ROLL: California apartment complexes, rental signs] +"This law caps your annual rent increase at 5% plus CPI, or 10% — whichever is lower." +[TEXT OVERLAY: "Max Increase: 5% + CPI or 10%"] +``` + +### Editing Notes for Jason + +Every core asset script MUST include an **Editing Notes** block — a dedicated section for the +video editor with production-specific instructions: + +``` +EDITING NOTES FOR JASON: +B-ROLL SHOT LIST: +- [List specific B-roll clips needed with descriptions] +- [Include stock footage suggestions if no original footage exists] + +TEXT OVERLAY TIMING: +- [Timestamp] -> [Text to display] (duration: Xs) +- [Timestamp] -> [Text to display] (duration: Xs) + +PACING NOTES: +- [Specific pacing instructions — fast cuts for hook, slower for education, etc.] + +THUMBNAIL CONCEPT: +- [Describe the thumbnail — text, expression, background, colors] + +MUSIC / SFX DIRECTION: +- [Music mood, tempo, genre suggestion] +- [Specific SFX moments — whoosh on transition, ding on stat, etc.] +``` + +### ElevenLabs SSML Blocks + +Every core asset script MUST include a complete ElevenLabs SSML block — the full script +wrapped in `` tags with prosody and break markup so Graeham can paste it directly +into ElevenLabs for AI avatar voice generation: + +```xml + + + If you own rental property in California, + + + + you need to know about AB 1482. + + + ... + +``` + +Use `` for emphasis shifts, `` for natural pauses, vary rate/pitch for +engagement. The hook should have higher energy (faster rate, higher pitch), educational +sections should be measured (medium rate), and CTAs should be emphatic (slower, louder). + +### AI Video Prompts (Seedance 2.0 / Kling) + +For content days that would benefit from AI-generated video (cinematic hooks, B-roll that +doesn't exist as footage, pattern-interrupt openers), include an **AI Video Prompt** block: + +``` +AI VIDEO PROMPT (Seedance 2.0): +SHOT: [Hook / B-Roll / Transition] +PROMPT: "Cinematic aerial drone shot of [description], golden hour lighting, + slow dolly forward, shallow depth of field, 4K, [duration]s" +CAMERA: [Movement type — dolly, crane, orbit, static, handheld] +LIGHTING: [Golden hour / overcast / interior warm / etc.] +DURATION: [3-5 seconds typical] +USE IN EDIT: [Where this clip goes in the timeline] +``` + +Include 2-3 AI video prompts per content day where applicable. Focus on: +- Hook shots (first 2-3 seconds — the scroll-stopper) +- B-roll that would be expensive or impossible to film (aerials, time-lapses, cinematic establishing shots) +- Transition moments between script sections + +### GHL Keyword Capture Integration + +Every script CTA must include a GHL comment-keyword trigger. Current active keywords: +`SELL`, `BUY`, `COSTS`, `OPTIONS`, `1482`, `EPA`, `VALUE`, `READY`, `INVEST`, `NUMBERS`, +`RELOCATING`, `MARKET`, `CHECKLIST`, `WATCH`, `RWC`, `PA`, `MP`, `SF` + +Format: "Comment [KEYWORD] below and I'll send you [lead magnet]" + +### AEO (Answer Engine Optimization) + +Every long-form script and blog derivative MUST include **cite-ready key statements** — +factual, data-heavy sentences that AI search engines (ChatGPT, Perplexity, Gemini) can +cite as authoritative answers. Format these as standalone declarative statements with +specific numbers, dates, or legal references. + +## Production Calendar Hardening (v7.3 — Apr 2026) + +The following requirements apply whenever scripts are produced as part of a V6/V7 Production Calendar. They exist to prevent three failure modes encountered in production: year-drift in on-screen text, output truncation on long-form deliverables, and week-to-week content duplication. + +### Date & Year QC (mandatory self-check block) + +Every prompt in `PROMPT_LIBRARY` MUST carry a `DATE & YEAR QUALITY CONTROL` block placed immediately after Fair Housing Guardrails and before Voice & Style. The block instructs the generating model to: + +- Treat the calendar's publication week as the current production date (e.g., April 2026 for the Apr 20-26 calendar). +- Force every year reference — text overlays, graphic callouts, on-screen stats, captions, email subject lines — to match the production year. Never a past year unless explicitly framed as historical with clear labeling. +- Open every cite-ready / AEO statement with a date anchor ("As of April 2026...", "As of Q2 2026..."). This makes statements durable for AI search engines to cite by name months later. +- Date-stamp every price/market stat ("As of April 2026, 3-bed SFH in Woodland Park is $680K-$850K") rather than emitting bare numbers. +- Self-scan the output before emitting and fix any bare-year drift. + +The v7.3 production calendar contains the exact block text — copy it verbatim when building future calendars. This QC block is required on every format (YT Long Pt 1/2, Shorts, Reels, TikTok, Carousel, Blog, GMB, Facebook, Email), not just long-form. + +### Identity & Date Validation Gate (mandatory, fail-closed) + +Date and DRE errors have reached deliverables before, so QC *instructions* alone are not enough. Every content package, dashboard, and automated email MUST pass a programmatic gate before it is published or sent. The gate fails closed: if any hard check fails, the content does NOT go out and a human is alerted instead. + +Hard checks (block on failure): + +1. **DRE number.** `01466876` is the only DRE that may appear anywhere. The known-bad DRE (the `0201`-prefixed number that has leaked repeatedly in the past; the exact blocklist value lives in `shared-references/identity.json`) must never appear in any output. If it does, BLOCK immediately. +2. **Output not empty/truncated.** Generated text must be present and of reasonable length. + +Soft checks (flag for review, do not auto-block): + +3. **Correct DRE present where expected.** If a caption/CTA should carry the DRE and `01466876` is absent, flag it. +4. **Date correctness.** The current date is read from the system clock at run time — never typed from memory, never inferred. Every year/date reference must match the real production date; unlabeled past-year references are a failure. +5. **Range language for perishable figures.** Any rate/price/median stated as a single hard number not verified from a live source this run must be rewritten as a range (see `references/phases/script-writer/references/data-verification-and-nuance.md`). + +Manual builds: run this gate as the final step before pushing or sending, and record the result. + +Automated daily email: the gate is implemented as the **Validate Date + DRE** Code node in the N8N workflow below. The date is injected from the system clock in the **Compute Today** node (never guessed), and the validation node fails closed — on failure the workflow routes to an alert to Graeham instead of sending to Peter. + +## Daily Automation — Peter's Daily Email (N8N) + +A weekday email to Peter is produced by the N8N workflow **"Daily Content Email — Peter (script + SSML + production)"** (instance `n8n.graehamwattsn8n.com`, workflow id `REVqxrlAb3CHJumM`). It complements the weekly dashboard: Peter can work from the dashboard OR act straight from the email. + +Flow: Schedule (Mon-Fri 6:00 AM PT) → CONFIG (`peter_email`, `cc_email`, `dashboard_url`) → Compute Today (system date → topic t1..t5, Mon=t1) → Fetch Dashboard HTML (the live weekly calendar is the single source of truth) → Parse Topic (pulls that day's `prod_script` + `prod_video` out of `COPY_DATA`) → Generate (OpenAI runs both prompts: script + SSML, then production assets) → Validate Date + DRE (fail-closed gate above) → IF passed → Email Peter (dashboard link + the day's full package); ELSE → Alert Graeham. + +Operational notes: + +- The workflow reads the LIVE published dashboard, so a corrected or new dashboard must be pushed to GitHub Pages before its content reaches the email. +- Update `CONFIG.dashboard_url` whenever a new week's dashboard is published; set `CONFIG.peter_email` once. +- Model is `gpt-4o-mini` for free-credit reliability; swap to `gpt-4o` or a paid key for higher script quality. +- Figures use range language; the script's "Verify before recording" block tells the human exactly which live numbers to confirm before shooting. + +### Output Split Strategy (YouTube Long only) + +A single YouTube Long prompt requesting 6 deliverables (Script + SSML + Editing Notes + AI Video Prompts + YouTube SEO Package + 3 Alt Hooks) produces roughly 40K-60K chars of output, which exceeds default `max_tokens` on most consumer AI tools and causes the model to truncate mid-Deliverable 4. + +Split YouTube Long prompts into TWO buttons: + +- **Pt 1 — Script + Voice** → Deliverables 1 (full timestamped script with inline shot tags) + 2 (complete ElevenLabs SSML block). Target output ~20-25K chars, fits in one response on any tool. +- **Pt 2 — Production Package** → Deliverables 3 (Editing Notes for Jason) + 4 (AI Video Prompts) + 5 (YouTube SEO Package) + 6 (3 Alt Hooks for A/B testing). Target output ~20-25K chars. Includes the standard 5-minute structure reference (hook / problem / core1 / core2 / advisory / CTA) so the production package works without pasting the script back in. + +Both parts share the same preamble (Agent Identity + Fair Housing + DATE/YEAR QC + Voice + Topic + Funnel Tier + AEO + Key Facts + GHL Lead Capture). Only the deliverable list and output-mode header differ. + +Short-form formats (YT Shorts, IG Reels, TikTok, Carousel, Blog, GMB, Facebook, Email newsletter) do NOT need splitting — their single output fits comfortably in one response. + +### Week-over-Week Overlap Check (mandatory pre-ship) + +Before shipping each weekly calendar, run an overlap comparison against the immediately preceding week's calendar. For each of the 5 daily topics plus the email newsletter, compare against the prior week's topics on: + +- **Title** — substring match or semantic overlap (e.g., "EPA Homes Under $700K" vs "EPA Homes Under $1M" = HIGH overlap) +- **Slug** — exact or near-exact match +- **Neighborhood** — repeated primary-market focus on consecutive days +- **Funnel tier** — consecutive weeks of same tier + same topic cluster +- **GHL keyword** — keyword reuse across weeks + +Classify each match as HIGH / MODERATE / LOW risk. Write a markdown comparison note (`YYYY-MM-DD-vs-PRIOR-content-overlap-check.md`) alongside the calendar HTML in `online-content/dashboards/weekly-calendars/` and commit it to the repo. HIGH-risk overlaps MUST be resolved before shipping — reframe the angle, replace the topic, or defer a week. + +**Future systematization**: add a `TOPIC_HISTORY` object to the calendar HTML containing the last 4 weeks of (title, slug, neighborhood, tier, ghl_keyword) tuples. Future calendar generation runs an automatic pre-publish check against that history and flags overlaps without a manual pass. + + +## Examples + +Three worked examples live in `examples/`: +- `example-1-bofu-trigger-event-tech-layoff.md` — BOFU response to a tech layoff trigger event +- `example-2-tofu-lifestyle-reel-epa-tacos.md` — TOFU lifestyle reel (East Palo Alto tacos) +- `example-3-aeo-legal-education-ab1482.md` — AEO-optimized legal education on AB 1482 + +Read these before writing new content packages — they show the expected output format and voice. + +## Example Prompts + +**Per-topic (this skill):** +- "Build a content package on the EPA homicide-free story" → Phase 0a (confirm topic) → Per-Topic Research (pull topic-matched research) → Phase G (build package) +- "I just got a new listing in Menlo Park at $2.1M — give me the full content package" → Per-Topic Research (the listing IS the topic) → Phase G +- "Make me a TOFU reel about East Palo Alto lifestyle" +- "Generate 5 BOFU videos about AB 1482 for Bay Area landlords" +- "Hey I saw this video, can we do something like this? https://youtube.com/watch?v=..." → Phase 0 (ingestion) → Per-Topic Research → Phase G +- "Transcribe this YouTube video and tell me what ideas we can use" → Phase 0 ingestion only +- "Here's a video about staging tips — adapt it for EPA sellers on a budget" + +**Weekly planning (hand to `content-calendar` — NOT this skill):** +- "What should I post this week?" +- "Plan my content calendar for the next 7 days" +- "What topics should I focus on based on my data?" +- "What are my competitors posting that I'm not?" +- "The Bay Area just had a big tech layoff announcement — what should I post?" + +**Ambiguous — run Phase 0a clarifier:** +- "I need content" +- "Run research" +- "Content opportunities" +- "What's happening in EPA?" + +## Output Locations + +All phase outputs save to the user's selected folder (or `outputs/` in Cowork). Provide `computer://` links to the final content package when delivering. + +## Data Source Status + +- **Primary:** Apify `trudax/reddit-scraper-lite` with residential proxy (~$0.30-$2.50 per run). Requires `APIFY_API_TOKEN`. +- **Supplementary:** Windsor MCP for Instagram, YouTube, Facebook, Search Console, and Apify scraper performance data. +- **Supplementary:** Claude web search for market context, news events, and competitor research. +- **Supplementary:** Chrome browser for MLS data, local government sites, and Google Trends. + +## Auto-Render Hand-off (v6.2 — Apr 2026) + +Once a V6 script is finalized, it no longer needs to be manually copy-pasted into ElevenLabs and HeyGen. The `heygen-elevenlabs-renderer` skill owns the full render pipeline and this skill hands off to it. + +### What this skill produces for the renderer + +For every core asset script written, write out a companion SSML file next to the content package: + +``` +outputs/content-package-{timestamp}.md (the full package — scripts, captions, etc.) +outputs/content-package-{timestamp}.ssml.txt (just the … block, nothing else) +``` + +The `.ssml.txt` file is the raw input the renderer reads. It must contain only the SSML — no headers, no comments, no markdown fences, no "SCRIPT:" prefix. One file = one render. + +### Known SSML quirks (read before hand-off) + +ElevenLabs `eleven_multilingual_v2` does NOT fully honor ``. Only `` produces audible effect. `` tags are accepted by the API but silently dropped — the inner text is still read, just at the default speed. So: + +- KEEP `` tags for human readability in the package file (they document intent) +- ALSO provide the `.ssml.txt` with the same tags (the renderer strips ineffective ones at TTS time) +- When you genuinely need rate/pitch change (e.g., whispered BOFU asides), use **ElevenLabs bracket audio tags** inside the text: `[whispers]`, `[excited]`, `[sarcastic]`, `[laughs]`. See the renderer skill's `references/elevenlabs-audio-tags.md`. + +### Hand-off invocation (one command) + +After this skill writes the package, the renderer takes over: + +```bash +python3 skills/heygen-elevenlabs-renderer/scripts/full_render.py \ + --script outputs/content-package-{timestamp}.ssml.txt \ + --slug "{content-slug}" \ + --resolution 1080p \ + --aspect 9:16 +``` + +The renderer: (1) synthesizes MP3 via ElevenLabs using Graeham's voice clone `Pa3vOYQHHpLJn1Tf7hnP`, (2) uploads MP3 to HeyGen, (3) creates an avatar video against Graeham's avatar `9a3600b16f604059b6ab8b9a55e29ea9`, (4) polls until complete, (5) downloads MP4 to `outputs/renders/{slug}.mp4` with a sibling `{slug}.meta.json` holding `video_id`, `video_url`, and duration. + +### Dashboard locations (where rendered media lives) + +After a render completes, the same files are available in four places. The renderer (`poll_and_download.py`) writes a `dashboards` block into `{slug}.meta. + +### Rule 15: Two-Register Freshness Check + +**Why this exists:** Prior to April 24 2026, the ideation engine only +checked `topic-history.json`'s `history` array (posted topics). Topics +that had been SHOT but not yet POSTED were invisible to the gatekeeper, +which meant the engine could queue a near-duplicate of a video sitting +in Graeham's edit pipeline. + +**The fix:** `topic-history.json` now has TWO registers — `history` +(posted) and `in_production` (shot-but-not-posted). Every freshness +check must read BOTH. + +**Implementation in ideation-engine:** + +1. Load `topic-history.json`. +2. Build `excluded_slugs = {t['slug'] for w in history for t in w['topics']} | {t['slug'] for t in in_production}`. +3. For each candidate topic, also check `exclusion_radius` text on every + `in_production` entry — if the candidate touches the same market + + angle, exclude even when the slug differs. +4. When a topic ships, MOVE it from `in_production` (if present) into + `history`. Don't leave duplicates. + +**When to write to `in_production`:** Whenever Graeham confirms he has +shot or is currently editing a video for a topic that isn't yet posted. +Add via the `script-writer` phase or manually before ideation runs. + +Reference: `skills/content-creation-engine/references/topic-history.json` +schema v2.0. + +--- + +## Publishing via Composio (canonical pattern) + +> **Read first:** [`shared-references/publishing-via-composio.md`](../shared-references/publishing-via-composio.md) — single source of truth for ALL skills. + +After generating the topic-production dashboard HTML output, publish via Composio to `Graehamwatts/online-content` so the agent gets a permanent hosted URL. + +**Account:** `github_spar-devata` +**Owner:** `Graehamwatts` +**Repo:** `online-content` +**Branch:** `main` +**Path pattern:** `dashboards/single-topic/YYYY-MM-DD-slug-production.html` +**Hosted URL pattern:** `https://graehamwatts.github.io/online-content/dashboards/single-topic/YYYY-MM-DD-slug-production.html` + +**Tool to use:** `GITHUB_COMMIT_MULTIPLE_FILES` (atomic commit, retry-safe). + +```python +result, error = run_composio_tool( + tool_slug='GITHUB_COMMIT_MULTIPLE_FILES', + arguments={ + 'owner': 'Graehamwatts', + 'repo': 'online-content', + 'branch': 'main', + 'message': 'descriptive commit message', + 'upserts': [{'path': 'dashboards/single-topic/YYYY-MM-DD-slug-production.html', 'content': html_content, 'encoding': 'utf-8'}] + }, + account='github_spar-devata' +) +``` + +**HARD RULES:** +- Do NOT use the legacy GitHub Contents API with PAT or `javascript_tool` chunked uploads (replaced 2026-05-03). +- Do NOT use GitHub Desktop or `git push` from the agent sandbox. +- Run the brand-integrity check before push (see shared doc — blocks DRE# 01 leaks). +- After commit, give the user BOTH the hosted URL and the local `computer://` link. + +See `shared-references/publishing-via-composio.md` for full details, common pitfalls, and verification flow. + + +## Canonical Weekly Calendar Template (v5.4 — locked in May 2026) + +> **This is the format moving forward.** Live reference: [`Graehamwatts/online-content/dashboards/weekly-calendars/2026-05-11-production-calendar.html`](https://github.com/Graehamwatts/online-content/blob/main/dashboards/weekly-calendars/2026-05-11-production-calendar.html). Hosted at: https://graehamwatts.github.io/online-content/dashboards/weekly-calendars/YYYY-MM-DD-production-calendar.html + +**Template structure (top to bottom):** + +1. **Hero** — week date range, opportunity-score pill chips, BOFU mix label. +2. **Audience tabs** (sticky) — Research / Blog Track / Peter / Show Everything. Tab state persists in URL hash (`#audience-blog`, `#audience-peter`). +3. **Preview banner** — explains v5 features + auto-refresh time. +4. **Live Data Layer** — 8 source cards (Composio IG, Composio YT, DataForSEO, n8n Local News, GSC via Windsor, Reddit via Apify, YT Comment Mining, Zillow Q&A). +5. **Full Research Data panel** (collapsed by default; toggle to expand): + a. **Brushable time-series charts** (ApexCharts via CDN): + - Instagram Activity Over Time (weekly likes + posts, dual axis, drag bottom slider to zoom) + - YouTube Activity Over Time (weekly views + videos, dual axis, drag bottom slider to zoom) + - Engagement Rate Per Post Per Week (avg per-piece for IG + YT) + b. Instagram 25/100-row table (live via Composio Meta Graph API) + c. YouTube 15/50-video table with stats (live via YouTube Data API v3) + d. GSC topic-targeted queries + e. Reddit demand signals + f. Zillow Q&A + g. MLS pull + h. Macro Rates & Permits + i. DataForSEO SERP queue status + j. Convergence — Why each day picked (with source counts and scores) +6. **5 Day Cards (week grid)** — clickable to filter Blog Track + Peter sections to one day. +7. **Weekly Strategy** — funnel mix bar + cross-platform handoff notes. +8. **Blog Track section** — 5 daily-items, each with prominent topic title + hook + format pill rows. Pills copy Claude-ready prompts. +9. **Peter section** — same pattern, video formats, with Image-Gen pills for carousels. +10. **Footer** — DRE 01466876, contact, refresh schedule, Composio commit reference. + +**Hard rules (don't drift from this):** + +- **Brand identity** — pull from `shared-references/identity.json`. Run the blocklist verifier before every push (see `scripts/verify_brand_identity.py` and `shared-refer \ No newline at end of file diff --git a/docs/Skills Dependency Audit 2026-07-28.md b/docs/Skills Dependency Audit 2026-07-28.md new file mode 100644 index 00000000..eacf2b36 --- /dev/null +++ b/docs/Skills Dependency Audit 2026-07-28.md @@ -0,0 +1,223 @@ +# Skills Dependency Audit — 2026-07-28 + +Video/Creative (13) + Real Estate Content (17) skill groups. Maps all 30 skills flagged as "left untouched" in the last /doctor pass — dependency graph, coupling analysis, consolidation proposals, description trims, dead-weight calls. + +**Status as of 2026-07-28 (approved by Graeham):** +- ✅ `content-multiplier` deleted — zero inbound callers found anywhere in the tree; confirmed dead weight, not just unwired. +- ✅ `listing-launch-engine` description trimmed (~300 chars cut: removed the duplicated shot-type inventory and the delivery-mechanism clause, both already stated in the SKILL.md body). Trigger-phrase list and all three scope-boundary sentences ("does not auto-edit," "does not remember prior runs," "does not build the avatar," "One listing per run") were preserved untouched. +- ⏳ Not yet actioned: the `remotion-video`↔`remotion-rules` cross-reference addition, the `heygen-video`↔`heygen-elevenlabs-renderer` cross-reference addition, and the `watts-motion-graphics`/`remotion-video` reference de-duplication — these remain open proposals below, not yet approved for action. + +--- + +## 1. Dependency Graph + +```mermaid +flowchart TD + subgraph Planning["Planning / Decision Layer"] + CCal[content-calendar] + MLS[mls-matrix-scraper] + end + + subgraph Ideation["Ideation"] + CF[concept-forge] + CH[cinematic-hooks] + end + + subgraph Engine["Production Engine (hub)"] + CCE[content-creation-engine] + end + + subgraph Multiply["Atomization"] + CM[content-multiplier] + end + + subgraph Launch["Listing Launch"] + LLE[listing-launch-engine] + LPC[listing-photo-captioner] + LRW[listing-remarks-writer] + WLU[weekly-listing-update] + PRAG[price-reduction-angle-generator] + end + + subgraph Craft["Craft Reference (loaded, not called)"] + MP[marketing-psychology] + CC[comedy-craft] + CW[copywriter] + HUM[humanizer] + end + + subgraph Distribution["Assembly / Distribution"] + NG[newsletter-generator] + MA[meta-ads] + SW[switchy-engine] + end + + subgraph Video["Video/Creative Production"] + HGV[heygen-video] + HGE[heygen-elevenlabs-renderer] + HFV[higgsfield-video] + VC[video-creator] + RV[remotion-video] + RR[remotion-rules] + WMG[watts-motion-graphics] + CTP[cinematic-trailer-pipeline] + VW[video-watcher] + VT[video-transcriber] + VTO[video-to-obsidian] + end + + subgraph Standalone["Standalone / Isolated"] + RD[room-redesign] + PS[podcast-studio] + OMS[off-market-property-search] + OA[offer-analyzer] + PU[property-underwriter] + end + + MLS --> CCal + CCal <--> CCE + CCE --> CF + CF --> LLE + CF --> CM + CM --> CCE + CM --> LLE + LLE --> CF + LLE --> CH + LLE --> WMG + LLE --> HGV + LLE --> HGE + LLE --> MA + LLE --> CW + CCE --> LPC + CCE --> LRW + CCE --> PRAG + CCE --> HGE + CCE --> VW + CCE --> VT + CCE -.-> MP + CCE -.-> CC + CCE --> MA + CCE --> NG + NG --> CCE + NG -.-> CC + CW -.-> MP + CW -.-> CC + LRW -.-> CC + LLE -.-> CC + CH -.-> VTO + VW <--> VT + VTO --> VT + CTP --> HGV + CTP --> HFV + WMG --> HGV + WMG --> HFV + WMG --> RV + RR -.-> RV + HGE -.-> PS + HUM -.-> CCE + HUM -.-> LLE + HUM -.-> NG + HUM -.-> WLU + + classDef hub fill:#f96,stroke:#333,stroke-width:2px + classDef ref fill:#bbf,stroke:#333 + classDef standalone fill:#9f9,stroke:#333 + class CCE hub + class MP,CC,HUM ref + class RD,PS,OMS,OA,PU,MLS standalone +``` + +**Legend:** solid arrow = functional call/handoff; dotted arrow = "loads as reference" or soft pairing. Orange = the central hub. Blue = reference-only (never produce deliverables themselves). Green = genuinely standalone. + +### Notable chains +- **Listing pipeline (most complex):** `concept-forge → content-multiplier → content-creation-engine (fed mode) → listing-launch-engine/packager → switchy-engine / GHL / meta-ads`. This exact chain is spelled out verbatim in `content-multiplier`'s own SKILL.md. +- **Planning → production:** `mls-matrix-scraper → content-calendar → content-creation-engine`, with a documented two-way "Scope Boundary" table between content-calendar and content-creation-engine (duplicated in both files — see risk note below). +- **Video render pipeline:** `content-creation-engine`/`listing-launch-engine` write scripts → `heygen-elevenlabs-renderer` (auto SSML→ElevenLabs→HeyGen) or `heygen-video` (manual avatar picks) → composited with `higgsfield-video` b-roll and `watts-motion-graphics` overlays in CapCut. +- **Video research pair:** `video-watcher` (visual) and `video-transcriber` (words) are explicit bidirectional siblings with matching "trigger boundary" tables in both files; `content-creation-engine` calls both as "external dependencies" it used to own internally. + +--- + +## 2. Standalone vs. Tightly Coupled + +### Genuinely standalone (safe to touch/refactor in isolation) +| Skill | Why | +|---|---| +| `room-redesign` | Zero inbound callers anywhere in the tree; only outbound mention is a one-line "this is images not video" disambiguation. Arguably miscategorized as "video group" — it's image-only. | +| `podcast-studio` | Nothing calls it (terminal personal-use skill); it calls out to `humanizer`, `website-crawler`, `founder-academy`, `heygen-elevenlabs-renderer` but nothing calls in. | +| `off-market-property-search` | Fully self-contained scraper + branded report generator, no cross-refs either direction. | +| `offer-analyzer` | Self-contained two-mode tool, no inbound or outbound refs to the other 16 real-estate skills. | +| `property-underwriter` | Standalone relative to this group (real dependency is `xlsx`, outside scope). | +| `mls-matrix-scraper` | Pure scraper; content-calendar reads its JSON output but it calls nothing itself. | +| `weekly-listing-update` | Only touches the cross-cutting `humanizer` utility; own data pipeline, own publish flow. | +| `listing-photo-captioner` | Explicitly documented as running independently of listing-remarks-writer despite the natural pairing ("They run independently"). | +| `remotion-rules` | Pure lookup index (38 rule files); one-directional reference to remotion-video, nothing calls it in. | + +### Tightly coupled hubs (edit with care — wide blast radius) +| Skill | Coupling | +|---|---| +| `content-creation-engine` | **The central hub of the entire 30-skill set.** 25+ other skills reference it (inside and outside these two groups). Any interface change here has the widest blast radius in the tree. | +| `heygen-video` | 9 external callers (vaibhav-template, watts-motion-graphics, transcript-repurposer, listing-launch-engine, cinematic-trailer-pipeline, meta-ads, room-redesign, concept-forge, content-creation-engine). | +| `content-calendar` ↔ `content-creation-engine` | Explicit bidirectional "Scope Boundary" table in **both** files — the cleanest deliberate split in the whole set, but duplicated prose = consistency risk if one is edited without the other. | +| `listing-launch-engine` | Coupled to concept-forge, content-creation-engine, content-multiplier, comedy-craft, cinematic-hooks, watts-motion-graphics, heygen-video/heygen-elevenlabs-renderer, humanizer, copywriter, meta-ads. | +| `content-multiplier` | Almost pure middleware — explicitly owns nothing itself, sits between concept-forge and listing-launch-engine. **Flag:** zero inbound references found anywhere — may be spec-only / not yet wired into real runs (worth confirming with you). | +| `heygen-elevenlabs-renderer` | Documented downstream execution layer for content-creation-engine's v5.4/V6 scripts; cited by podcast-studio for shared SSML facts. | +| `video-watcher` ↔ `video-transcriber` | True sibling pair, matching trigger-boundary tables duplicated in both files. | +| `marketing-psychology` / `comedy-craft` | Reference-only "twins" (persuasion vs. humor) — not coupled by function, but loaded by nearly every content-producing skill (content-creation-engine, newsletter-generator, listing-remarks-writer, listing-launch-engine, copywriter, content-multiplier, concept-forge). | +| `higgsfield-video` / `cinematic-hooks` | No outbound refs in their own bodies, but 5-6 external skills call into each as the shared b-roll/prompt execution engine. | + +--- + +## 3. Consolidation / Combination Proposals + +These are proposals, not actions — flagging where real overlap exists. + +1. **`remotion-video` + `remotion-rules` → keep separate, but tighten the link.** `remotion-rules` already calls itself a "lookup reference... pairs with remotion-video," but `remotion-video`'s own file never reciprocates or mentions `remotion-rules` at all. This is a one-directional gap, not true overlap — recommend adding one sentence to `remotion-video` pointing at `remotion-rules` for best-practices lookup, rather than merging (they serve different purposes: generator vs. reference index). + +2. **`watts-motion-graphics` vs. `remotion-video` — genuine overlap, not accidental.** `watts-motion-graphics` explicitly is the Watts-brand specialization of generic Remotion ("Generic Remotion projects (non-Watts brand) → use `remotion-video`"). Both files independently duplicate a fair amount of generic Remotion component/aspect-ratio boilerplate that `remotion-rules` already indexes in depth. Proposal: strip the duplicated generic-Remotion reference material out of `remotion-video`'s body and point it at `remotion-rules` instead, leaving `remotion-video` as a thinner project-scaffolder and `watts-motion-graphics` as the only brand-specific layer. No functional merge needed — just de-duplication of reference content. + +3. **`copywriter` / `marketing-psychology` / `comedy-craft` — do NOT merge.** This looks like three overlapping "writing help" skills but is actually a deliberately documented, clean division of labor (copywriter = deliverable + format/variant engine; marketing-psychology = persuasion diagnosis; comedy-craft = humor calibration), explicitly spelled out in marketing-psychology's own file. This is good architecture, not overlap — flagging only so it's not mistakenly "simplified" away. + +4. **`heygen-video` / `heygen-elevenlabs-renderer` — clarify the split, don't merge.** Both render HeyGen avatar video, but heygen-video is the manual/interactive job (Claude asks which avatar, submits, tracks) while heygen-elevenlabs-renderer is the fully automated SSML→ElevenLabs→HeyGen pipeline for content-creation-engine's v5.4/V6 scripts specifically. They're both legitimately load-bearing render endpoints for different callers (9 callers use heygen-video; content-creation-engine/podcast-studio/listing-launch-engine use heygen-elevenlabs-renderer for the automated path). Recommend keeping both, but adding one explicit cross-reference line in each pointing at the other so a reader landing on either knows the other exists and why. + +5. **`content-multiplier` — confirm it's live before investing more in it.** It has a fully-specified, well-written SKILL.md sitting "between" concept-forge and listing-launch-engine, but nothing in the tree calls into it yet. Before folding effort into further wiring it, worth a quick check with you: is this actually used in real listing launches today, or is it a designed-but-not-yet-adopted layer? If unused, it's the closest thing to "dead weight" found in either group — though it reads as forward-looking infrastructure, not abandoned cruft. + +6. **`room-redesign` — reconsider its grouping, not its content.** It has zero ties to the other 12 video skills (image-only, no inbound/outbound refs). No change needed to the skill itself, but if you ever do a folder/category reorg, it belongs closer to a general "AI image generation" bucket than the cinematic-video cluster. + +--- + +## 4. Safe Description-Trim Proposals + +Only trimming content that's duplicated elsewhere in the same file's body or is pure descriptive color — never touching trigger-phrase lists or scope-boundary/routing sentences. + +| Skill | Current length | Trim proposal | What must stay | +|---|---|---|---| +| **`listing-launch-engine`** | 2,181 chars | Cut the parenthetical shot-type inventory ("the 4-photo status set, reveal cuts, walkthrough, neighborhood B-roll, open-house clips" — duplicates the body's "It produces three things" list) and the delivery-mechanism clause ("Output is a visual HTML dashboard... distributed as a Gmail draft" — duplicates Step 9). Est. **300–400 char cut**, ~15-18%. | Full trigger-phrase enumeration (routing depends on it); the three boundary sentences — "does not auto-edit," "does not remember prior runs (that is PropertyIQ's job)," "does not build the avatar," "One listing per run." | +| **`meta-ads`** | 1,414 chars | Minor trim of the long trailing trigger-phrase run-ons is possible but risky — this is Meta's own paid-ads vocabulary and likely tuned for real trigger accuracy. Recommend **no trim** unless you specifically want it shorter; flagging only because it's the 2nd-longest. | Everything — this one's length is functional, not padding. | +| **`property-underwriter`** | 1,526 chars | The "Design note" self-justification vs. Anthropic's generic financial-analysis plugin is unusually long for a description but is genuinely load-bearing (explains why this skill exists rather than a built-in). **No trim recommended.** | — | +| **`cinematic-trailer-pipeline`** | ~840 chars | Body (not description) has a trimmable "Honest reality checks" section (character-drift caveats, cost estimates) that could move to a reference file. Description itself is already tight — no cut needed. | Hand-off contract / routing-away sentence to heygen-video and higgsfield-video. | +| **`podcast-studio`** | ~640 chars | Body's SHOW REGISTRY entries (business data, not orchestration) are candidates to move to a reference file, shrinking the SKILL.md body — description itself is fine as-is. | GLOBAL HARD RULES (esp. the iTunes re-sync fix and mandatory humanizer pass). | +| **`video-creator`** | ~640 chars | Body has detailed per-template JSON config examples that could move to `references/` without affecting how other skills invoke it. | The "no external tools or local setup needed" framing — remotion-video's fallback routing depends on this exact characterization. | +| All others (17 remaining) | — | No trims proposed — bodies scale sensibly with description length (92–636 lines), and none showed daylight between what's claimed and what's delivered. | — | + +**General pattern:** across both groups, the safe-to-trim material is consistently *changelog/historical narration* ("formerly listing-call-sheet," "absorbed video-script-creation-engine in April 2026," "First built file:...") and *duplicated descriptive detail* that repeats the SKILL.md body almost verbatim in the frontmatter description. Trigger-phrase lists and any sentence containing "does NOT," "CALLED BY," "owns," "route to," or a Scope Boundary/Architecture table were never proposed for trimming — those are the load-bearing routing text you flagged as the risk. + +--- + +## 5. Dead Weight + +**Nothing in either group is unambiguous dead weight.** Both groups are unusually well-maintained — every skill has at least one genuine inbound or outbound tie, or is intentionally standalone by design (room-redesign, podcast-studio, the five real-estate standalone tools). Two things worth a closer look, not deletion: + +- **`content-multiplier`** — fully speced, zero inbound callers found. **Deleted 2026-07-28** per Graeham's approval; no other skill referenced it, so removal was clean. +- **`remotion-rules`** — thinnest SKILL.md body of the 30 (70 lines), but this is by design (it's a pure index into 38 `rules/*.md` files that weren't read in this pass). Not dead weight, just flagging that its real substance lives outside the file that was audited here. +- **One unverifiable grep hit:** `context-engineer/SKILL.md` matched "video-creator" during the scan but the surrounding content appeared to be binary/non-UTF8 at that location — worth a manual check outside this audit, not a finding in itself. + +--- + +## What's next + +Remaining open proposals, not yet actioned: +1. The `remotion-video` ↔ `remotion-rules` cross-reference addition. +2. The `heygen-video` ↔ `heygen-elevenlabs-renderer` cross-reference addition. +3. The `watts-motion-graphics`/`remotion-video` reference de-duplication. + +This document is kept as a standing reference — see the Skills repo (`docs/`) and the Obsidian vault (`AI Library/`) for the saved copy. diff --git a/docs/Skills Native-Knowledge Audit 2026-07-28.md b/docs/Skills Native-Knowledge Audit 2026-07-28.md new file mode 100644 index 00000000..6d9a5e15 --- /dev/null +++ b/docs/Skills Native-Knowledge Audit 2026-07-28.md @@ -0,0 +1,89 @@ +# Skills "Native Knowledge vs. Hard-Won Specifics" Audit — 2026-07-28 + +Full sweep of all 66 remaining skills (everything except `remotion-video`/`remotion-rules`, already actioned separately). Six parallel research agents read every SKILL.md in full and classified content against the lens you proposed: **generic content a capable model already knows** (trimmable) vs. **hard-won, non-reconstructable specifics** — bug workarounds, exact IDs/paths, brand locks, compliance rules, discovered vendor quirks (must stay regardless of model capability). + +No files have been edited. This is a proposal set. + +--- + +## Part 1 — Actual bugs found (separate from the genericness question, higher priority) + +These aren't about trimming — they're incidental discoveries of real inconsistencies/breakage while agents read every file closely. Recommend fixing these regardless of what happens with Part 2. + +| # | Skill | Issue | +|---|---|---| +| 1 | `transcript-repurposer` | Repeatedly references `video-script-creation-engine/references/...` paths. That skill was retired 2026-04-29 per the repo's own `CLAUDE.md` in favor of `content-creation-engine`. These are dead pointers today. | +| 2 | `meta-ads` | Its Handoffs table still names `social-media-analyzer`, which was absorbed into `content-calendar`. Stale skill name. | +| 3 | `video-creator` | Hardcodes `"Brokerage": "Compass"` (appears twice) in its Agent Info Defaults. Every other skill in the audit uses "Intero Real Estate," and the repo's `CLAUDE.md` explicitly says brand facts must come from `identity.json`, never be hardcoded. This is the same class of bug the blocklisted-DRE incident was about. | +| 4 | `heygen-video` vs. `heygen-elevenlabs-renderer` | Disagree on Graeham's HeyGen avatar inventory — one documents 6 named looks with a "must ask every time" rule; the other says "70 personal avatar looks" with a single primary ID. These describe the same account and need reconciling. | +| 5 | `newsletter-generator` | Two different gold brand hex values appear in the same file (`#C2A14E` vs. `#C5A258`/`#B8860B`) — looks like drift between an old and new brand pass. | +| 6 | `cinematic-trailer-pipeline` | Built entirely around PAI 2.0/UTOPAI as the video generator, while `higgsfield-video`/`cinematic-hooks` reflect the newer Higgsfield/Seedance/Kling stack. Worth confirming whether PAI 2.0 is still active or this pipeline itself is now the stale one. | +| 7 | `llm-council` | Step 5 says "Do NOT generate an HTML report... user reads it in conversation," but the closing notes contradict this: "The visual report matters... make the HTML output clean." Leftover instruction from an earlier version. | +| 8 | `flow-dictation` | Its "Roadmap (not built yet)" section lists two features (lock mode, AI-polish pass) that the body already describes as shipped. Only "voice commands" is genuinely still pending. | +| 9 | `ai-library` | Cites "45 Personal Claude Skills / 139 total items," clearly stale against the current ~90+ skill directory (the repo's own root `CLAUDE.md` separately claims a stale "39 skills" too). | +| 10 | `cowork-task-shutoff` | Explicitly scoped to a one-time migration dated 2026-06-09/06-12. ~7 weeks later, worth checking whether this is done and the skill itself can retire. | +| 11 | `trackabi-va-payroll` | Bakes a point-in-time loan balance ("as of Jul 1 2026: payment 5 of 10, $522.49 remaining") directly into the SKILL.md prose, duplicating the live loan tracker file. Will read as false every pay period. | +| 12 | `past-client-follow-up-system` | The documented schedule horizon ends "Fri Aug 7 2026" — about 10 days out from this audit. Needs regenerating soon; also has one unconfirmed GHL custom-field ID flagged "confirm before run." | +| 13 | `website-crawler` | Self-flagged "scaffolded 2026-07-18, not yet installed or run end-to-end" — 10 days old as of this audit; worth confirming whether it's since been verified. | +| 14 | `youtube-scraper` | References `mcp__Claude_in_Chrome__*` (underscore/caps) — this session's actual Chrome MCP tools are `mcp__claude-in-chrome__*` (hyphenated). Worth verifying this isn't a dead tool reference. | +| 15 | `travel-hq` | Not actually a bug — it's an almost entirely unfilled placeholder template (every card, loyalty number, and companion profile is still `[PLACEHOLDER]`). Flagging because it can't meaningfully be audited for genericness until you fill in your actual travel profile. | + +--- + +## Part 2 — The genericness/trim question + +### Tier A — Strong trim candidates (mostly/entirely reconstructable from general model knowledge) + +| Skill | What's generic | What to keep | +|---|---|---| +| `context-engineer` | Nearly the whole file — token-budget categorization, anti-patterns list, pushback FAQ, a hypothetical (not real) `video-creator` example. Ironically the most bloated file relative to how little Graeham-specific content it has. | The two hard numeric constraints and pointers to its own `references/` files. | +| `github-repo-analyzer` | The entire file — ghost-developer reasoning, git/GitHub domain knowledge, generic management-tact advice. Zero Graeham-specific content found anywhere. | Report format/headers if you like the output shape. | +| `job-search-engine` | Most of the explanatory prose (ATS/7-second-scan lecture, negotiation-script rationale) — though the actual prompt templates are more like reusable assets than teaching content and can stay. | The 9 prompt templates themselves; cut the surrounding generic scene-setting. | +| `language-tutor` | Nearly the whole file — CEFR milestones, generic lesson structure, generic tutoring "rules." Zero facts specific to what Graeham's actually learning. | The intake questions (they gather his real answers) and the progression-tracking state model. | +| `consolidate-memory` | Most of it — general "how to maintain a memory system" theory. | The two hard constraints (200 lines/25KB, line format) and the pointer to the system prompt's memory config. | +| `seo-optimizer` | The entire audit rubric (title-tag length, meta description length, keyword density, Flesch score) — standard on-page SEO a model already knows. Isn't even real-estate-localized. | The one genuinely hard-won insight (YouTube/Reddit citation frequency in AI answers, Pantana Feb 2026 field notes) — should be promoted higher, not cut. | +| `copywriter` | The AIDA/PAS/FAB/BAB framework definitions and the psychological-levers list — textbook, and duplicated almost verbatim in `marketing-psychology`. | Framework *selection* guidance, output templates, humanizer-pass instructions, format character limits. | +| `marketing-psychology` | Schwartz awareness-stage definitions, Cialdini principles, the "11 minds" panel bios — canon-level marketing knowledge. | The decision tables (blocking-force matrix, framework tree, failure-mode fixes) and the explicit division-of-labor vs. `copywriter` — that's applied judgment, not facts. | +| `property-underwriter` | The full NOI/cap-rate/DSCR/IRR formula derivations — textbook real estate finance. | The Model-Builder-gets-the-debt-schedule-wrong warning, all CA/Bay-Area tax specifics, the "never fabricate rent" rule, PropertyIQ integration section. | +| `listing-remarks-writer` | The "Nouns Over Pronouns" copywriting lecture and generic vocabulary lists. | Fair Housing/RESPA rules, truth-in-advertising rules, Bay Area neighborhood specifics, the "AI search ignores these words" claim (flag as uncertain — may be calibrated, not generic). | +| `listing-photo-captioner` | Generic caption-style bullets ("lead with room name," "sentence case"). | Fair Housing/RESPA guardrails, ADU naming rule, MLS character limits, Bay Area landmark examples. | +| `disclosure-analyzer` | Generic finding-severity definitions, generic tact advice, most of "Common Pitfalls" (verify with you whether these came from real incidents first). | Seller Credit Request framework, QC failure-mode list, publishing pipeline. | +| `price-reduction-angle-generator` | Generic "don't use fear tactics" tone rules. | Three-Strategy Honesty Rule, banned-word list, Fair Housing specifics, Bay Area DOM thresholds (flag as uncertain — looks generic but may be calibrated). | +| `offer-analyzer` | Generic Tone & Style paragraph. | The entire Default-Output-Mode-Information-Only section — this is the actual differentiator of the skill, a deliberate liability posture, not generic advice. | +| `cma-generator` | Narrative-writing paragraph-count instructions (now redundant with the separate `humanizer` skill). | DRE/brand hard rule, File-Integrity Protocol, exact publishing steps. | +| `contract-estimate-builder` | Excel-formatting-rules bullet list, already duplicated by the `xlsx` skill it references. | Menalto/Minalto misspelling note, disclaimer legal language, Option-Group modeling logic. | +| `website-builder` | Design-philosophy/anti-pattern list — the file itself admits this overlaps with the separately-installed `frontend-design` skill. | Brand-state table (locked vs. placeholder), snippet routing, conflict-priority rule. | +| `transcript-repurposer` | Analysis-axis and angle-option explanatory prose (duplicates ground `comedy-craft`/`marketing-psychology` already cover). | The named rubric as a compact checklist, environment/entry-point logic, GHL keyword list, humanizer inclusion/exclusion list. | +| `video-watcher` | The "why visual analysis matters" reasoning and the "Why this exists (history)" section. | Trigger-boundary table (cost-driven routing vs. video-transcriber), cost tables, maintenance/ownership note. | +| `room-redesign` | Generic prompt-engineering tips ("tell it what to keep," "one change at a time"). | The four pre-tuned prompt templates, exact model IDs, MLS staging-disclosure legal note. | + +### Tier B — Uncertain, don't cut without checking with you first + +These *look* generic but the agents flagged them as possibly calibrated from real testing/taste, not textbook knowledge: + +- `cinematic-hooks`'s "pattern interrupt psychology" and "creative frameworks" sections — did these come from testing what actually stops a scroll, or are they written from general theory? +- `listing-remarks-writer`'s claim about which words "AI search ignores" +- `price-reduction-angle-generator`'s Bay Area DOM velocity thresholds +- `cma-generator`'s chart-type choices (radar vs. table) +- `humanizer`'s 29-pattern taxonomy (may encode Wikipedia-sourced completeness value even if individually recognizable) and its linked `references/voice-calibration.md`/`patterns-catalog.md` (not read yet — could hold real calibration) +- `llm-council`'s five-advisor personas (a deliberate design choice creating three tensions, not random) +- `comedy-craft`'s technique toolkit (illustrated with real-estate-specific examples throughout — leaning toward keep, but flagged since it's presented as generic craft) + +### Tier C — No changes recommended (dense, hard-won, non-reconstructable) + +`mls-matrix-scraper`, `farming-postcard`, `weekly-listing-update`, `chatgpt-ads`, `comedy-craft`, `switchy-engine`, `switchy-qr`, `listing-launch-engine` (core orchestration), `founder-academy`, `finance-watch`, `flow-dictation` (aside from the roadmap fix above), `heygen-video`, `heygen-elevenlabs-renderer`, `higgsfield-video`, `watts-motion-graphics`, `podcast-studio`, `vaibhav-template`, `video-to-obsidian`, `cinematic-trailer-pipeline` (aside from the PAI-2.0 staleness check), `instagram-competitor-scraper`, `local-news-scraper`, `obsidian-vault`, `past-client-follow-up-system`, `schedule`, `setup-cowork`, `trackabi-va-payroll`, `youtube-scraper`, `property-os-sync` (only the Cowork/Windows dual-path branching is worth consolidating), `skill-creator` (its own tripwire/placement-rule block is critical; the generic skill-authoring methodology below it is arguably the actual content this tool-of-tools skill needs to teach, not bloat). + +### Anthropic-authored reference skills — different tradeoffs + +`docx`, `pdf`, `pptx`, `xlsx` all ship an Anthropic `LICENSE.txt` — they're unmodified public reference skills, not Graeham-custom. Their generic library-API content is intentional design for portability across arbitrary projects, not accumulated bloat. Trimming them loses upgradeability from Anthropic and has different tradeoffs than trimming a bespoke skill. Recommend leaving these alone unless you specifically want a leaner, less-portable fork. `video-transcriber` (no LICENSE.txt, Graeham-custom) is the opposite — almost entirely hard-won, no changes recommended. + +--- + +## What I'd suggest as next steps + +Given the scale (66 skills touched by this audit), rather than doing all of Tier A in one giant sweep, sensible slices to approve independently: + +1. **Fix the 15 bugs in Part 1** — these are correctness issues, not judgment calls, and mostly small. +2. **Trim Tier A** — 20 skills, all with a clear "cut this / keep this" split above. +3. **Decide Tier B case-by-case** — genuinely needs your judgment on whether each was empirically calibrated or just written generically. +4. **Leave Tier C and the Anthropic reference skills alone.** diff --git a/docs/architecture-and-build-guide.md b/docs/architecture-and-build-guide.md new file mode 100644 index 00000000..dc79ddcf --- /dev/null +++ b/docs/architecture-and-build-guide.md @@ -0,0 +1,597 @@ +# Architecture & Build Guide + +> **Read this first.** If you're contributing to or rebuilding any part of Graeham Watts's skills repo, this document describes what's built, why it's built that way, and what's open work. Last updated April 2026. + +This guide is written for a developer joining the project (Mehmood — building Graeham's systems alongside Uzair, Khawaja, Wattson). It assumes you can read code, but doesn't assume you've worked with Cowork or Claude Skills before. + +--- + +## Table of Contents + +1. [The Project: PropCast](#the-project-propcast) +2. [Repo Structure](#repo-structure) +3. [Core Architectural Decisions](#core-architectural-decisions) +4. [Data Flow Diagrams](#data-flow-diagrams) +5. [Skills Inventory](#skills-inventory) +6. [How Cowork Skills Work (Mechanics)](#how-cowork-skills-work-mechanics) +7. [Identity & Brand Hard Rules](#identity--brand-hard-rules) +8. [Integrations](#integrations) +9. [Pantana Reference (Category Context)](#pantana-reference-category-context) +10. [Open Work / Known Issues](#open-work--known-issues) +11. [Build Priorities](#build-priorities) +12. [How to Develop on This Repo](#how-to-develop-on-this-repo) + +--- + +## The Project: PropCast + +PropCast is a unified **content + transaction operating system** for real estate agents, built first for Graeham Watts (REALTOR, Intero Real Estate, Bay Area / East Palo Alto) and structured to be productized for other agents later. + +PropCast covers two surfaces: + +**Content side** (week-over-week, evergreen) +- Research signals across multiple data sources +- Topic ideation + scoring +- Multi-format content generation (video scripts, blog posts, social, email) +- Distribution (HeyGen avatar video, ElevenLabs voice, GoHighLevel CRM keyword capture, GitHub Pages publishing) + +**Transaction side** (deal-by-deal) +- CMA generation (3-strategy pricing, branded report, GitHub Pages publishing) +- Offer analysis + comparison (multi-offer net sheets) +- Disclosure / inspection report analysis +- Listing remarks (MLS) + photo captions +- Price reduction conversations (data-backed angle for the seller convo) + +Both sides are powered by **skills** running in **Cowork** (Anthropic's desktop product), backed by external integrations (Apify, Windsor MCP, GoHighLevel, HeyGen, ElevenLabs, GitHub). The skills repo is the codebase. + +--- + +## Repo Structure + +``` +Graehamwatts/skills/ (this repo) +├── .claude-plugin/ +│ └── plugin.json (Cowork plugin manifest) +├── .git/, .nojekyll, index.html, assets/ (GitHub Pages landing infra) +├── CLAUDE.md (root onboarding doc) +├── README.md (public README) +├── docs/ +│ └── architecture-and-build-guide.md (THIS FILE) +├── scripts/ +│ └── verify_brand_identity.py (DRE-leak tripwire) +└── skills/ (all 39+ skills) + ├── shared-references/ + │ ├── identity.json (BRAND IDENTITY SSOT) + │ ├── integrations.md (canonical integration matrix) + │ └── data-contracts.md (cross-skill JSON contracts) + ├── content-creation-engine/ (main content engine) + ├── content-calendar/ (weekly planning) + ├── bofu-query-generator/ (standalone BOFU) + ├── bofu-intent-scorer/ (standalone BOFU scorer) + ├── cma-generator/ + ├── offer-analyzer/ + ├── disclosure-analyzer/ + ├── listing-remarks-writer/ + ├── listing-photo-captioner/ + ├── price-reduction-angle-generator/ + ├── youtube-scraper/ + ├── ... (other skills — see Skills Inventory below) + └── ... +``` + +**Sister repo: `Graehamwatts/online-content`** — published content hub (separate repo because it's a GitHub Pages site with public client-facing URLs; outputs and source code shouldn't mix). Renamed from `cma-reports` on 2026-05-01 to reflect that it holds ALL published content types, not just CMAs. The old `cma-reports` repo was retired with no migration — its content was disposable. + +| Output type | Where it goes | +|---|---| +| Published CMAs | `Graehamwatts/online-content/cmas/` | +| Published offer reports | `Graehamwatts/online-content/offers/` | +| Published disclosure reports | `Graehamwatts/online-content/disclosures/` | +| Published newsletters | `Graehamwatts/online-content/newsletters/` | +| Weekly production calendars | `Graehamwatts/online-content/dashboards/weekly-calendars/` | +| Per-topic single-topic dashboards | `Graehamwatts/online-content/dashboards/single-topic/` | + +--- + +## Core Architectural Decisions + +### 1. GitHub Is the Source of Truth + +- Skills repo: `https://github.com/Graehamwatts/skills` +- Working copy: clone via GitHub Desktop to `~/Documents/GitHub/skills` +- Cowork local plugin folder (`%APPDATA%/Claude/local-agent-mode-sessions/skills-plugin/`) is **downstream**. It syncs FROM GitHub. Edits made there don't persist. +- All architectural changes go: edit → commit → push → Cowork picks up updated skills on next session + +If the local Cowork plugin shows skills that don't match GitHub, the local cache is stale. The remedy is forcing a Cowork sync (close + reopen Cowork session typically refreshes). + +### 2. Engine + Standalone Pattern + +Skills are organized in two tiers: + +**Engines** — orchestrate multi-phase workflows. Examples: `content-creation-engine`, `content-calendar` (which absorbed `social-media-analyzer` in May 2026). + +**Standalones** — single-purpose, can be invoked directly OR referenced by an engine. Examples: `bofu-query-generator`, `listing-remarks-writer`, `cma-generator`. + +Engines reference standalones via **sibling-path imports** in their SKILL.md (e.g., engine's Phase 1 says "Read `../bofu-query-generator/SKILL.md`"). This is DRY: + +- Standalones can be invoked alone — user says "generate BOFU queries for Redwood City" → standalone fires +- Engines pull standalones into pipelines — user says "build content package on EPA homicide-free story" → engine fires, internally reads standalone's instructions +- One source of truth per skill — edits propagate + +**Engine-internal sub-modules** (not standalone-useful) live INSIDE the engine folder: +- `content-creation-engine/references/phases/` — the 6 internal phases (source-ingestion, content-ideation-engine, funnel-tagger, script-writer) +- `content-creation-engine/modules/` — sub-modules (newsletter, market-update-narrative) + +### 3. Two-Score Model (Content Architecture) + +The content system has **two distinct scores** answering two distinct questions. They are NOT interchangeable and must NEVER be merged: + +| Score | Owner | Scale | Answers | When applied | +|---|---|---|---|---| +| **Opportunity Score** | `content-calendar` | 25 pts (5 criteria × 5) | "Should we cover this topic THIS WEEK vs other candidates?" | Once per week, across 12-15 candidates. Top 4-5 by score → weekly calendar. | +| **Intent Score** | `bofu-intent-scorer` (standalone) | 25 pts (5 criteria × 5) + freshness ±5 | "What's the BOFU intent of this topic (DECISION / CONSIDERATION / AWARENESS)?" | Once per topic, AFTER opportunity selection. Used for funnel-mix and CTA decisions. | + +Both scores are rendered side-by-side on the per-topic dashboard so the distinction stays visible. See `content-creation-engine/SKILL.md` → Scoring Architecture section for full model. + +### 4. Audience-Targeted Button Pattern (Dashboards) + +Per-topic dashboards have buttons targeted at specific team members: + +- **Blog producer** (publishing team — posts content to platforms): gets the **Copy Content** button (gold solid). The blog producer never needs to regenerate; he posts what's already produced. +- **Peter** (video production): gets the **Copy Script Prompt** button (gold outline) and **Copy Production Prompt** button (purple). Peter regenerates as needed for his AI tools. + +Non-video formats (Blog, Email, GMB, Facebook, IG Carousel) have 2 buttons (blog only). Video formats (YT Long Pt1+Pt2, YT Short, IG Reel #1, IG Reel #2, TikTok) have 3 buttons (blog + video). + +Button colors carry semantic meaning: +- **Gold solid** = Blog Track's primary action (post-ready content) +- **Gold outline** = secondary regeneration / script-side prompt +- **Purple solid** = Peter's production-side prompt +- **Navy** = UI chrome (toggles, expanders, navigation) + +See `content-creation-engine/references/single-topic-dashboard-rules.md` Rule 3 for full spec. + +### 5. Weekly Output: HTML Calendar + Three-Tier Email + +content-calendar produces TWO weekly outputs: + +1. **HTML Production Calendar** (hosted on GitHub Pages) — the full multi-tab dashboard for Jason (video editor) and Peter (production). Three tabs: Analytics, Production Map, Copy Bank. + +2. **Three-Tier Email for Blog Track** (sent Monday + daily) — Blog Track's quick-decision surface. Topics ranked into Top tier (Score 22-25, "must_create"), Next tier (17-21, "strong"), Third tier (12-16, "consider"). Each email link deep-links into the dashboard where the Copy buttons live (email clients strip JS — buttons can't work IN the email). + +See `content-calendar/SKILL.md` → "Weekly Email Format (for the Blog Producer)" section. + +### 6. YouTube Source Ingestion: Two Modes + +Phase 0 of content-creation-engine has two distinct modes: + +- **Mode A — Single-URL Transcription** — user pastes a video URL → `youtube_transcriber.py` runs (caption pull → Whisper fallback) +- **Mode B — Channel Monitoring** — user pastes a channel URL OR a scheduled task fires → `youtube-scraper` standalone scans for new uploads in the time window, delegates transcripts to youtube_transcriber.py for each + +The orchestrator picks the right mode based on what the user provided. Don't fire Mode B for a single URL or vice versa. + +--- + +## Data Flow Diagrams + +### Weekly Planning Flow + +``` +[Data Sources — see integrations.md] + ├── Windsor MCP (Instagram, Facebook, YouTube, GSC) + ├── Apify Reddit scraper (trudax/reddit-scraper-lite) + ├── MLSListings (Chrome) + ├── Google Trends (Chrome) + ├── Local news (web search) + EPA gov (Chrome) + └── Apify competitor scrapers + │ + ▼ + 12-15 topic candidates extracted + │ + ▼ + Opportunity Score applied (25 pts) + • Performance Signal (5) + • Search Demand (5) + • Audience Intent (5) + • Competitive Gap (5) + • Timeliness (5) + │ + ▼ + Top 4-5 → weekly calendar + Tier breakdown: + • Top tier (22-25): "must_create" + • Next tier (17-21): "strong" + • Third tier (12-16): "consider" + │ + ▼ + Two outputs: + ├── HTML Production Calendar (online-content/dashboards/weekly-calendars/{date}-production-calendar-v6.html) + └── Blog Track's three-tier email (outputs/emails/weekly-{date}-blog.html) + │ + ▼ + For each selected topic → handoff to content-creation-engine (per-topic flow) +``` + +### Per-Topic Production Flow + +``` +Topic arrives at content-creation-engine +(from content-calendar weekly plan OR direct user ask) + │ + ▼ +Phase 0a — Clarifier Check (only if ambiguous) + │ + ▼ +Phase 0 — Source Ingestion (only if user provided YouTube URL or channel) + • Mode A: youtube_transcriber.py + • Mode B: youtube-scraper → youtube_transcriber.py + │ + ▼ +Phase R — Per-Topic Research + • Topic-matched MLS stats (MLSListings via Chrome) + • Topic-matched GSC queries (Windsor + Direct API parallel-pull) + • Topic-matched local news (web search + EPA gov) + • Topic-matched social signal (Windsor) + • Topic-matched competitor coverage (Apify) + • Topic-matched Reddit signal (Apify Reddit scraper output) + │ + ▼ +Phase G — Generate Content (with topic-type routing) + • Market update topics → modules/market-update-narrative/ → Phase 5 + • Listing spotlight → ../listing-remarks-writer/ + ../listing-photo-captioner/ → Phase 5 + • Price reduction → ../price-reduction-angle-generator/ (PRIVATE — no public output) + • Education / how-to → Phase 5 directly + │ + ▼ +Phase 1 — BOFU Query Generator (standalone) — only for ideation-driven runs + │ + ▼ +Phase 2 — Content Ideation (Reddit/Apify scrape) + │ + ▼ +Phase 3 — BOFU Intent Scorer (standalone) — DECISION/CONSIDERATION/AWARENESS classification + │ + ▼ +Phase 4 — Funnel Tagger (TOFU/MOFU/BOFU mix) + │ + ▼ +Phase 5 — Script Writer + • References: content-pillars, platform-specs, voice-and-style, seo-keywords, + aeo-geo-requirements, lead-capture-keywords, elevenlabs-audio-tags + • Conditional (when format = blog): schema-markup-templates, + rss-internal-linking, youtube-embed-patterns + │ + ▼ +Outputs: + • outputs/content-package-{ts}.md (full package — scripts, captions, etc.) + • outputs/content-package-{ts}.ssml.txt (raw SSML for renderer) + • online-content/dashboards/single-topic/{date}-{slug}-production.html (per-topic dashboard) + │ + ▼ +Phase A — Review & Approve (user) + │ + ▼ +Phase D — Distribute + • Newsletter → Gmail draft (via Gmail MCP) + • Blog → ready for CMS publish + • Social → platform-specific posts queued + • Video → handoff to heygen-elevenlabs-renderer: + full_render.py → ElevenLabs (voice) → HeyGen (avatar) → MP4 +``` + +### Transaction-Side Flow (CMA Example) + +``` +User uploads MLS data + property details + │ + ▼ +cma-generator + • Reads identity.json (DRE, brokerage, contact) + • References branding.md (colors, fonts, logo) + • References charts.md (matplotlib styling) + │ + ▼ +Three-strategy pricing analysis + 1. Aspirational + 2. Market-Aligned + 3. Move-It + │ + ▼ +Three output formats: + • Interactive HTML Report (Chart.js, sticky nav, animated counters) + • Email-Safe HTML (table-based, inline styles) + • PDF (print-optimized HTML → WeasyPrint/xhtml2pdf) + │ + ▼ +Publish to GitHub Pages + online-content/cmas/CMA_{address}.html + │ + ▼ +Live URL: https://graehamwatts.github.io/online-content/cmas/CMA_{address}.html +``` + +--- + +## Skills Inventory + +The repo has 39+ skills as of April 2026. Categorized: + +### Engines (orchestrators) +- `content-creation-engine` — main content production pipeline, per-topic +- `content-calendar` — weekly planning + performance analytics (absorbed `social-media-analyzer` May 2026) + +### Standalone Content Skills +- `bofu-query-generator` — 230+ localized BOFU search queries +- `bofu-intent-scorer` — Intent Score (DECISION/CONSIDERATION/AWARENESS) +- `cinematic-hooks` — pattern-interrupt video prompts +- `vaibhav-template` — talking-head Vaibhav-style aesthetic +- `video-prompt-builder` — Seedance shot lists +- `youtube-scraper` — channel monitoring (different from URL transcription) + +### Transaction-Side Skills +- `cma-generator` — branded CMA reports (PDF + HTML + email) +- `offer-analyzer` — multi-offer comparison + seller net sheets +- `disclosure-analyzer` — TDS/SPQ/inspection report analysis +- `listing-remarks-writer` — MLS public remarks (walkthrough + condition-aware) +- `listing-photo-captioner` — per-photo MLS captions +- `price-reduction-angle-generator` — data-backed seller convo angle + +### Video Production Skills +- `heygen-video` — single-call HeyGen avatar video +- `heygen-elevenlabs-renderer` — ElevenLabs → HeyGen pipeline (voice + avatar) +- `video-creator` — Python + ffmpeg slideshow videos +- `remotion-video` — React-based programmatic video +- `higgsfield-video` — Higgsfield AI b-roll + +### Communication Skills +- `html-email` — branded HTML email generation + GitHub Pages hosting + +### CRM / Operations +- `ghl-crm-audit` — GoHighLevel audit + N8N workflow building + +### Document Processing +- `docx`, `pdf`, `xlsx`, `pptx` — Office document creation/editing + +### Infrastructure +- `github-skill-sync` — automated repo backup +- `skill-creator` — skill scaffolding + evaluation +- `schedule` — scheduled tasks +- `consolidate-memory` — memory file maintenance +- `setup-cowork` — guided Cowork onboarding +- `context-engineer`, `copywriter` — utility/meta skills + +### Off-Market / Custom +- `off-market-property-search` — off-market lead generation +- `newsletter-generator` — separate from html-email, focused on multi-section newsletters +- `website-builder` — landing page generation + +For any skill not listed above, look in `skills//SKILL.md`. Each skill's SKILL.md is the canonical doc for that skill — read the frontmatter `description` field for triggering keywords. + +--- + +## How Cowork Skills Work (Mechanics) + +Cowork is the runtime. Each skill is a folder with a `SKILL.md` file containing YAML frontmatter (name + description) and Markdown body (instructions). + +**Skill discovery:** Cowork scans the top-level `skills/` directory at session start. Each `SKILL.md` is registered with its description as the trigger string. + +**Skill triggering:** when a user prompt arrives, Cowork's reasoning matches the prompt against every skill's description. The closest match fires — that skill's full instructions become the operating context for the response. + +**Sub-skill referencing:** when a skill's instructions say "Read `../other-skill/SKILL.md`" — that's a regular file read into the current session's context. It doesn't formally invoke the other skill as a separate sub-call. The current skill's session reads the referenced file and follows those instructions inline. + +**Why this matters for architecture:** +- Skills MUST live at `skills//` to be discoverable +- Sub-modules nested inside skills (e.g., `content-creation-engine/modules/newsletter/`) are NOT independently triggerable — they're reference files the parent skill reads +- For a skill to be both standalone AND part of an engine, it MUST be at top level + the engine references it via path + +**Plugin manifest** (`.claude-plugin/plugin.json`) describes the skills bundle for Cowork to load. + +--- + +## Identity & Brand Hard Rules + +### identity.json Is the Single Source of Truth + +`skills/shared-references/identity.json` contains: +- Graeham's name, title, brokerage, **DRE**, phone, email, website +- Primary + secondary markets +- Blocklist of values that must NEVER appear in outputs + +**The DRE is `01466876`.** The blocklist contains one value (a wrong DRE that has leaked into outputs 11 times historically — see CLAUDE.md root for post-mortems). + +### Hard Rules + +1. **NEVER hardcode brand identity from memory or context.** Every skill that emits brand details (CMA reports, listing remarks, schema markup, signatures, footers) MUST read identity.json at generation time. + +2. **NEVER type a DRE-shaped string from prior context.** If a session's reasoning sees a DRE value in its working context, do NOT type it into output. Read identity.json fresh. + +3. **`scripts/verify_brand_identity.py` is the tripwire.** It scans the entire repo for blocked values and exits non-zero if found. Run before every push: + ```bash + python3 scripts/verify_brand_identity.py + ``` + +4. **Documentation-exempt files** (CLAUDE.md root, identity.json itself, verify_brand_identity.py) MAY contain blocked values for warning/blocklist purposes. The tripwire skips them. Other files MUST NOT contain blocked values, even in warnings. + +5. **Cowork's cached skill descriptions can be stale.** If you see the wrong DRE in a skill's description string in Cowork's UI, that's a cache issue, not a real leak. The fix is restarting Cowork to refresh. + +### What Mehmood Needs to Do + +When building features that emit brand details: +- Read identity.json at runtime — never hardcode +- Test that the tripwire passes before committing +- Don't include literal blocklist values in code comments or docs (use abstract references like "the blocklisted value documented in identity.json") + +--- + +## Integrations + +See `skills/shared-references/integrations.md` for the canonical integration matrix. Key points: + +- **13 active integrations** (MLSListings, GSC, Apify Reddit, Apify Zillow, YouTube transcriber, YouTube Data API, Instagram, Facebook, EPA gov, HeyGen, ElevenLabs, GoHighLevel, GitHub) +- **3 stale / needs verification** (Apify Zillow, YouTube Direct API, GSC Direct API) +- **3 pending** (Reddit official API — applied; Santa Clara county records — not wired; San Mateo county records — not wired) +- **Windsor + Direct API parallel-pull rule** — for any source available via both, pull both in parallel, compare freshness/completeness, pick winner. Documented in integrations.md with pseudocode. + +When wiring a new integration: +1. Add an entry in integrations.md +2. Update the per-skill map at the bottom of that file +3. If the integration touches identity-related fields, update verify_brand_identity.py's tripwire +4. Run verification before declaring it production-ready + +--- + +## Pantana Reference (Category Context) + +Jason Pantana ships a real-estate AI content kit (sometimes called "PropCast" — distinct from Graeham's project despite the name overlap). It's mostly content-side and template-shaped: agents download the templates, swap placeholder values for their own market, and run. + +**Where Pantana overlaps PropCast:** +- BOFU query patterns (his query library inspired Graeham's bofu-query-generator) +- BOFU scoring framework (Intent Matrix concept) +- Listing remarks writer (Pantana's "nouns over pronouns" approach) +- Listing photo captioner +- Price reduction angle generator +- Blog post writer (AEO structure) +- YouTube-to-blog pipeline +- Channel scraper + +**Where PropCast goes further:** +- Transaction-side stack (CMA, offer analysis, disclosure analysis) — Pantana doesn't have these +- Hyperlocal Bay Area / EPA context baked into every skill +- Branded output (CMA reports, dashboards) with Graeham's identity +- GitHub Pages publishing pipeline for client-facing URLs +- GoHighLevel CRM integration for comment-keyword lead capture +- ElevenLabs + HeyGen production pipeline (avatar voice + video) +- Two-score architecture (Opportunity vs Intent) +- Three-tier email + dashboard buttons targeted at specific team members (Blog Track vs Peter) +- Windsor MCP integration for cross-platform analytics +- Apify-driven Reddit ideation +- Three-strategy CMA pricing framework (Graeham's specific methodology) + +**Why this matters for build priorities:** Pantana ships what most agents would build first (templates). PropCast's moat is the depth on the transaction side and the integration layer. Don't spend time replicating Pantana's templates — use them as references where they're already in our repo, and focus build effort on the parts Pantana doesn't have. + +**Files in this repo that originated from Pantana's templates:** +- `Cotent Creation engine Jason Pantana/` (in Graeham's local Documents folder, not in repo) — the original Pantana download. Used as reference during the April 2026 audit. NOT in the skills repo. +- Pantana's BOFU Query Generator + BOFU Scorer were absorbed into our skills (reorganized + Bay Area localized). Our `bofu-query-generator/SKILL.md` and `bofu-intent-scorer/SKILL.md` are the canonical versions. +- Pantana's blog post writer additions (JSON-LD schema, RSS internal linking, YouTube embed patterns) were folded into `content-creation-engine/references/phases/script-writer/references/` rather than living as a separate blog-post-writer skill. + +--- + +## Open Work / Known Issues + +Tracked as of April 2026: + +### High priority + +1. **`single-topic-dashboard-builder.py` function-body refactor** — the v5 builder has the loader fixed and button-render logic updated for the new 3-button-per-video-format pattern, but the render code still runs at module-load time with empty dicts. Wrapping it inside `_render_html()` is a ~1000-line indentation refactor that needs bash + Python AST verification. Deferred from April 30 session due to bash sandbox failure. See file's NOTICE block at top. + +2. **Email generator script** — `weekly-email-builder.py` doesn't exist yet. Spec for the email format is in `content-calendar/SKILL.md` (Weekly Email Format section). Builder script should produce `outputs/emails/weekly-{date}-blog.html` and `daily-{date}-blog.html` from the weekly calendar JSON. + +3. **County records integrations** — Santa Clara + San Mateo. Spec in integrations.md. Build a `county-records-scraper` standalone skill that takes county + APN as input, returns parcel JSON. + +### Medium priority + +4. **Verify Apify Zillow scraper** — flagged stale in integrations.md. Run a test scrape on a known address before relying on it for time-sensitive output. + +5. **Verify YouTube Data API + GSC Direct API OAuth flows** — both flagged stale. Confirm token refresh handling. + +6. **Reddit official API follow-up** — see Cloud Chrome prompt in integrations.md. When approved, document the new connector and update content-ideation-engine to parallel-pull with Apify scraper. + +7. **Tripwire extension to `online-content` repo** — currently `verify_brand_identity.py` only audits the skills repo. The April 29 leak was IN the published-content repo (then `cma-reports`, now `online-content`). Either copy the script to `online-content` OR extend this script to clone-and-audit `online-content` as part of its run. + +### Low priority / future + +8. **Google Trends MCP** — currently uses generic web search via Chrome. If a reliable Trends MCP appears, switch. + +9. **Skill-calls-skill via Skill tool** — currently engines reference standalones via `Read` (sibling-path file read). Could migrate to formal `Skill` tool invocation for better separation. Not urgent. + +10. **Content-creation-engine Phase 5 reference file split** — `instructions.md` is large. Could be split into smaller per-format files. + +--- + +## Build Priorities + +If you're starting fresh and asking "what should I work on?", priority order: + +### Week 1: Verify what exists +1. Read this entire doc + identity.json + integrations.md +2. Pull the repo via GitHub Desktop +3. Run `python3 scripts/verify_brand_identity.py` (it should pass — confirm) +4. Open `skills/content-creation-engine/SKILL.md` end-to-end +5. Open `skills/content-calendar/SKILL.md` end-to-end +6. Try a real run: in Cowork, ask "build me a content package on AB 1482 for Bay Area landlords" and trace which skills fire and in what order + +### Week 2: Knock out the high-priority open work +1. Finish the single-topic-dashboard-builder.py function-body refactor (item #1 in Open Work) +2. Build weekly-email-builder.py (item #2) +3. Wire county records (item #3) + +### Week 3+: Verify integrations + extend +1. Test stale integrations (Apify Zillow, Direct APIs) +2. Follow up on Reddit API (use the Cloud Chrome prompt in integrations.md) +3. Extend tripwire to `online-content` (item #7) + +### Ongoing: Productize +- Once Graeham's specific build is solid, evaluate productizing for other agents (the PropCast SaaS direction). That's a separate project — multi-tenant identity.json, tenant-isolated dashboards, billing, etc. + +--- + +## How to Develop on This Repo + +### Local Setup + +1. Install GitHub Desktop (Mac or Windows) +2. Clone `Graehamwatts/skills` via GitHub Desktop +3. Local path: `~/Documents/GitHub/skills` (Mac) or `C:\Users\{user}\Documents\GitHub\skills` (Windows) +4. Verify: `cd skills && ls` should show README.md, CLAUDE.md, skills/, scripts/, etc. + +### Editing Workflow + +1. Open the repo in your editor of choice (VS Code recommended) +2. Edit any SKILL.md or reference file +3. Run `python3 scripts/verify_brand_identity.py` to check for DRE leaks +4. Commit via GitHub Desktop with a descriptive message +5. Push via GitHub Desktop +6. Cowork will pick up the updated skills on the next session (may need to restart Cowork to force refresh) + +### Testing a Skill + +Without bash access (which is the case in standard Cowork sessions): +- Read the SKILL.md end-to-end +- Trace what files/scripts it references +- Read those references +- Look for hardcoded paths that point to dead sandbox sessions (e.g., `/sessions/something/mnt/...`) — those are stale and need fixing + +With bash access: +- `python3 scripts/verify_brand_identity.py` — tripwire check +- `python3 -c "import ast; ast.parse(open('skills/some-skill/script.py').read())"` — syntax check on Python skill scripts + +### Common Pitfalls + +1. **Editing the local Cowork plugin folder** — those edits don't persist. Always edit the GitHub Desktop clone. +2. **Hardcoding DRE in code or docs** — read from identity.json at runtime. Tripwire will catch you. +3. **Deleting skills without updating cross-references** — use the Grep tool to find every reference before deletion. +4. **Pushing without running the tripwire** — eventual leak. Run it. +5. **Mistaking Cowork's cached descriptions for current state** — if something's "wrong" in Cowork but right on GitHub, restart Cowork. + +### Naming Conventions + +- Skill folder names: `kebab-case` (e.g., `bofu-query-generator`) +- Reference files: `kebab-case.md` (e.g., `voice-and-style.md`) +- Output files: `{slug}-{date or ts}.{ext}` (e.g., `content-package-2026-04-30.md`) +- GitHub Pages dashboards: `YYYY-MM-DD-{slug}-production.html` +- Commit messages: descriptive imperative (e.g., "Add market-update-narrative module to content-creation-engine") + +### When in Doubt + +1. Check `CLAUDE.md` (root) — it has the high-level rules +2. Check `skills/shared-references/identity.json` — for any brand identity question +3. Check `skills/shared-references/integrations.md` — for any external data source question +4. Check `skills/shared-references/data-contracts.md` — for cross-skill JSON contracts +5. Read the SKILL.md of the skill you're working on +6. Ask Graeham — he has the most context on intent + +--- + +## Last Updated + +April 30, 2026 — Phase 7 deliverable of the Pantana audit + repo restructure. + +Future updates: when integrations are verified, when stale items are resolved, when build priorities shift, or quarterly \ No newline at end of file diff --git a/emails/2026-05-24-peter-june-bimonthly-equity-production-brief.html b/emails/2026-05-24-peter-june-bimonthly-equity-production-brief.html new file mode 100644 index 00000000..5660de49 --- /dev/null +++ b/emails/2026-05-24-peter-june-bimonthly-equity-production-brief.html @@ -0,0 +1,274 @@ + + + + + +June 2026 Bi-Monthly Market Update — Production Brief for Peter + + + + + + + \ No newline at end of file diff --git a/emails/market-update-production-brief-may-2026.html b/emails/market-update-production-brief-may-2026.html new file mode 100644 index 00000000..50586217 --- /dev/null +++ b/emails/market-update-production-brief-may-2026.html @@ -0,0 +1,482 @@ + + + + + +Production Brief — EPA + Bay Area Market Update Videos (May 2026) + + + + +
+ + +
+
Production Brief • Graeham Watts
+

Market Update Videos
EPA + Bay Area — May 2026

+
Scripts • Shot Lists • B-Roll Prompts • Thumbnail Concepts
+
+ May 25, 2026 + 2 Videos + Data: MLSListings Apr–May 2026 + DRE# 01466876 +
+
+ + +
+ + + 🔗 Full Hosted Production Brief + graehamwatts.github.io/skills/emails/market-update-production-brief-may-2026.html + + +
+

Hey Peter — two market update videos below, ready to shoot. Each section has the full script, shot list, b-roll prompts (Seedance + Higgsfield), and thumbnail concept — all with one-click copy buttons so you can paste straight into your workflow.

+

Both videos: Graeham talking-head on warm desk look • ~90–120 sec each • No equity/tax talk — straight market data, fun and energetic.

+
+ +
+ + +
+
+
1
+
+
East Palo Alto Market Update — May 2026
+
GHL keyword: MARKET • Audience: EPA buyers + sellers • ~2:45
+
+
+
+ +
April 2026 Data (MLSListings)
+
+
$1.2M
Median Sale Price
+
102%
List Price Rec'd
+
34 days
Median DOM
+
9
Closed Sales
+
12
New Listings
+
$765/sf
Avg Price/SqFt
+
+ +
Hook (0:00–0:05 open frame)
+
+
▶ Say this first — camera rolling
+

“In East Palo Alto right now, homes are selling for $1.2 million — and buyers are paying more than asking price. So why is demand still this strong in a market this quiet?”

+
+ +
Full Script (click to copy)
+
+ +
+
0:00–0:30 • HOOK
+

In East Palo Alto right now, homes are selling for $1.2 million — and buyers are paying 102 cents on the dollar. That means on a million-two home, you are not getting a deal. You are paying over ask. So the question is: why is demand still this strong in a market this quiet?

+

I’m Graeham Watts, I sell real estate in East Palo Alto, and I just pulled the April 2026 numbers straight from MLSListings. Let me show you what they say.

+
0:30–0:42 • CTA
+

Before I get into the data — drop "MARKET" in the comments and I’ll send you the full EPA breakdown for your specific street. Free. No pitch. Now, here’s what the numbers show.

+
0:42–2:15 • DATA BREAKDOWN
+

Question: What is the median home price in East Palo Alto right now?

+

As of April 2026, the median sale price in East Palo Alto is $1,200,000. That’s based on 9 closed sales last month. Small number — this is a thin market — but it’s consistent with what we’ve been seeing all year.

+

Question: How long are homes sitting before they sell?

+

Median days on market is 34. But here’s the split: the homes that are priced right are gone in under two weeks. The ones that sit are usually priced above comp — and they’re pulling that 34-day median up. If you’re priced right in EPA, you’re not waiting a month.

+

Question: Are buyers paying over asking?

+

Yes. 102% of list price on average. That’s not a bidding war frenzy, but it means sellers are not giving anything away. If you walked in expecting to negotiate down — this data says the market doesn’t support that right now.

+

Question: How much inventory is there in East Palo Alto?

+

12 new listings came to market in April. 9 sold. That’s it. EPA is a small market by design — bounded by the 101, the Bay, and Palo Alto. There is no flood of new inventory coming. What you see is what you get.

+
2:15–2:45 • CONTEXT
+

Here’s the bigger picture. San Mateo County’s median dropped year-over-year. EPA did not. This market is holding. And with only 12 listings a month, if you’re a buyer waiting for a deal — you’re competing with everyone else who is also waiting for a deal. The people who bought in 2024 when it felt uncomfortable? Those are the people who own in EPA right now.

+
2:45–3:00 • CLOSE + CTA
+

I’m Graeham Watts with Intero Real Estate — DRE 01466876. I specialize in East Palo Alto. Drop "EPA" below and let’s talk about your specific situation. I’ll pull the data for your block.

+
+
+ + +
AEO: Q&A structure mirrors how buyers search ChatGPT/Perplexity. Each answer is date-anchored (“As of April 2026”) for Google AI Overviews pickup.
+ +
Shot List (click to copy)
+
+ + + + + + + + + + + +
#ShotNotes
1Open talking headGraeham at desk, warm lit, direct-to-camera. Conversational, leaning in.
2Stat overlay: $1.2MBurn-in: “$1,200,000 — EPA Median Sale Price • April 2026”
3B-roll: EPA aerialNeighborhood from above, tree canopy, Bay in background. 4–6 sec.
4Stat overlay: 102%“102% of List Price • Buyers Paying Over Ask” — animated text reveal
5B-roll: Street levelQuiet EPA residential block, Sold sign on well-kept home
6Stat overlay: inventory“12 New Listings • 9 Sold • That’s the whole market”
7B-roll: communityCooley Landing, Bay trail, or community park. Establishes EPA identity.
8Close talking headBack to Graeham for CTA. Hold 3 sec post-CTA for caption text overlay.
+
+ + +
B-Roll Prompts
+ +
+ +
+
B-Roll 1 — EPA Aerial (Seedance 2.0)
+

Smooth cinematic drone pull-back from a quiet residential street in East Palo Alto, California. Single-family homes with mature trees, green lawns. Morning light, golden hour warmth. Bay visible in the far background. No cars moving. Peaceful, hyper-local feel. 4K, slow motion 0.5x, natural color grade, no lens flare.

+
+
+ + +
+ +
+
B-Roll 2 — Sold Sign Street Level (Higgsfield / Kling 3.0)
+

Street-level steady shot of a residential front yard in a quiet California suburb. A red "SOLD" rider is attached to a white real estate sign in the front yard. Lush green landscaping. Warm afternoon light. Slight breeze moves the tree leaves. Camera holds static for 4 seconds, then slowly pushes in on the sold sign. Photorealistic, 24fps cinematic.

+
+
+ + +
+ +
+
B-Roll 3 — EPA Community (Seedance 2.0)
+

Cinematic slow pan across Cooley Landing waterfront, East Palo Alto, California. San Francisco Bay in background, blue sky with light clouds. Empty walking path in foreground. Early morning. Peaceful, community-focused. Slightly wide lens, natural color. No people. Establishes "this specific neighborhood" identity. 4K, smooth gimbal movement.

+
+
+ + +
Thumbnail Concept
+
+
EPA Market Update Thumbnail
+

Layout: Graeham left-frame (shoulder up, pointing at stat), bold stat right-frame.

+

Primary text: “$1.2M” massive gold type. Subtext: “EPA IS STILL HOT” white.

+

Background: Dark navy gradient with faint EPA aerial photo texture.

+

Feel: Confident, hyper-local, data-forward. NOT stock-photo suburban agent vibes.

+
+ +
YouTube thumbnail, dark navy background with subtle aerial neighborhood photo texture, large bold gold text reading '$1.2M' on the right side, smaller white bold text below reading 'EPA IS STILL HOT', professional real estate agent placeholder on the left pointing at the text, gold bottom-border accent bar, clean modern typography, high contrast, no stock photo cliches
+
+
+ + +
+
+ +
+ + +
+
+
2
+
+
Bay Area Real Estate — Spring 2026 Market Update
+
GHL keyword: NUMBERS • Audience: Peninsula buyers + sellers • ~2:45
+
+
+
+ +
12-Month Data (MLSListings, All Residential)
+
+
$1,280,201
May 2026 Avg Price
+
$1,045,278
Dec 2025 Low
+
+$234,923
5-Month Move
+
+11.9%
YoY (vs Jun 2025)
+
+ +
Hook (0:00–0:05 open frame)
+
+
▶ Say this first — camera rolling
+

“Bay Area home prices just hit $1.28 million — the highest average in over a year. And here’s the wild part: in December we were at $1.04 million. That is a $235,000 move in five months.”

+
+ +
Full Script (click to copy)
+
+ +
+
0:00–0:30 • HOOK
+

Bay Area home prices just hit $1.28 million — the highest average sale price in over a year. And here’s the part that’s actually wild: in December 2025, we were at $1.04 million. That is a two-hundred-and-thirty-five-thousand dollar move in five months.

+

I’m Graeham Watts, and I just pulled the 12-month trend from MLSListings. This is the actual data — not the headlines, not vibes. Let me walk you through what’s happening.

+
0:30–0:42 • CTA
+

Drop "NUMBERS" in the comments — I’ll send you the full 12-month trend report for the Peninsula. No cost. Now — here’s the data.

+
0:42–2:15 • DATA BREAKDOWN
+

Question: What happened to Bay Area home prices between December 2025 and May 2026?

+

As of May 2026, the average sale price across Bay Area residential properties tracked by MLSListings reached $1,280,201. That’s up from a 12-month low of $1,045,278 in December 2025. The market dropped through fall and winter — then spring happened, and it came back hard.

+

Question: Why did Bay Area prices spike so fast in spring 2026?

+

Three things happened at once. One: inventory stayed historically low. Homeowners who locked in at 2 and 3 percent mortgage rates are not selling. Two: buyers who sat on the sidelines all winter moved in February — all at the same time. Three: multiple offers came back to the sub-1.5 million range in March. When homes get three, four, five offers, that sale-to-list ratio climbs above 100 percent — and it drags the median up fast.

+

Question: Is the Bay Area real estate market going up or down in 2026?

+

As of May 2026, prices are at a 12-month high. Year-over-year, we’re up about 12 percent from June 2025. Whether we hold this level through summer depends on two things: inventory and rates. Right now, both are holding the floor.

+
2:15–2:45 • CONTEXT
+

Here’s what this means depending on where you are. If you bought in 2024 or early 2025 when everyone was nervous — you are already up. If you’re a buyer who has been waiting for prices to come down — the window that opened in December is closing fast. If you’re a seller — spring is historically your best listing window, and right now you have buyer attention, low competition from other sellers, and prices at 12-month highs. That combination doesn’t last all year.

+
2:45–3:00 • CLOSE + CTA
+

I’m Graeham Watts with Intero Real Estate, DRE 01466876 — serving the Peninsula and East Palo Alto. Drop your zip code in the comments and I’ll pull the specific data for your market. I do this every month. It’s free.

+
+
+ + +
AEO: Three Q&A anchors targeting “Bay Area home prices 2026”, “why did Bay Area prices spike”, “is Bay Area real estate going up or down.” Each date-anchored for LLM citation.
+ +
Shot List (click to copy)
+
+ + + + + + + + + + + +
#ShotNotes
1Open talking headGraeham at desk. Energetic — this is a “look at this data” energy video. Lean forward.
2Stat overlay: chartAnimated line chart: $1.14M (Jun 25) dips to $1.04M (Dec 25), surges to $1.28M (May 26). 3-sec reveal.
3B-roll: Peninsula aerialSuburban neighborhoods from above, Bay visible, Silicon Valley sprawl. Wide establishing shot.
4Stat overlay: move“+$234,923 in 5 months” bold text burn-in. High contrast, brief hold.
5B-roll: street levelPalo Alto or Redwood City residential block — well-kept homes, green landscaping, afternoon light.
6Stat overlay: high“May 2026: $1,280,201 • 12-Month High” with subtle upward arrow animation.
7B-roll: interior / open houseStaged living room with natural light. Represents buyer demand returning.
8Close talking headBack to Graeham for CTA. Hold 3 sec post-CTA for caption text.
+
+ + +
B-Roll Prompts
+ +
+ +
+
B-Roll 1 — Peninsula Aerial (Seedance 2.0)
+

High cinematic drone shot pulling back slowly over a dense suburban neighborhood in Silicon Valley, California. Red-tiled roofs, tree-lined streets, mid-century and modern homes mixed. San Francisco Bay visible on the horizon. Late afternoon golden light. Wide establishing shot. 4K, natural color grade, slow 0.4x speed, no artificial lens effects.

+
+
+ + +
+ +
+
B-Roll 2 — Suburban Street (Higgsfield / Kling 3.0)
+

Slow steady tracking shot down a quiet California suburban street. Mature oak trees lining both sides, dappled afternoon light hitting the pavement. Well-maintained single-family homes visible on each side. No cars, no people. Camera moves at walking pace. Warm, aspirational, peaceful. Photorealistic 24fps. Peninsula / Bay Area vibe.

+
+
+ + +
+ +
+
B-Roll 3 — Interior / Open House (Seedance 2.0)
+

Interior of a bright, modern, staged living room in a California home. Large windows letting in natural afternoon light. Clean furniture, white walls, hardwood floors. No people. Slow dolly push from the doorway toward the center of the room. Architectural photography feel. 4K, warm daylight color temperature, no motion blur.

+
+
+ + +
Thumbnail Concept
+
+
Bay Area Market Update Thumbnail
+

Layout: Graeham right-frame (hands open, explaining posture). Stat left-frame.

+

Primary text: “$1.28M” massive white type + gold upward arrow + “12-MONTH HIGH”.

+

Background: Peninsula aerial, darkened + gradient overlay left to right.

+

Feel: Momentum. “Something is happening” energy — urgent without alarmist.

+
+ +
YouTube thumbnail for real estate market update video, background is aerial photo of suburban Silicon Valley California with dark gradient overlay on left half, large bold white text '$1.28M' center-left, gold upward arrow icon below it, smaller gold text '12-MONTH HIGH', real estate agent placeholder on right side with open explaining hand gesture, bottom accent bar with 'MAY 2026 DATA' in small white text, cinematic and data-forward, high contrast
+
+
+ + +
+
+ +
+ + +
+

Graeham Watts • Intero Real Estate • DRE# 01466876
+ 650-308-4727 • graehamwatts@gmail.com • graehamwatts.com
+ Data source: MLSListings Pro Dashboard, April–May 2026

+
+ +
+ + + + \ No newline at end of file diff --git a/index.html b/index.html new file mode 100644 index 00000000..d3c5f84e --- /dev/null +++ b/index.html @@ -0,0 +1,229 @@ + + + + + +Main Dashboard -- Graeham Watts Content Engine + + + + + + + + + +
+ + +
+
Main Dashboard · Week of April 27 – May 3, 2026
+

This Week's Content Plan

+
Five topics scored, ranked, and scheduled across Mon–Fri. Click any day to open the production view with scoring, research, and content creation for that topic.
+
+
Goal: Lead Gen
+
5 topics
+
Mix: 20% TOFU / 30% MOFU / 50% BOFU target
+
Generated April 24, 2026
+
+
+ + +
+
+ +
+ Read First + For Peter — How to Use This Dashboard +
+ +
+

This is the weekly plan. Each day tile below is the content scheduled to ship that day. Click a tile to open the production view — that's where you'll find the scripts, copy bank, thumbnail prompts, and the ElevenLabs/HeyGen render commands.

+

Your job, Monday morning: Click Monday's tile. On that page, scroll to the Content Creation section. Click View [Format] Content on YouTube Long to open the script modal. Copy the script + SSML, paste into ElevenLabs, then HeyGen. Move through the other formats in order. Done — move to Tuesday.

+

If a format shows "Awaiting Generation," click the gold-outline Copy Prompt button — it copies a format-specific prompt to your clipboard. Paste into Claude, get the finished content, drop it into the destination platform.

+

If something looks off (wrong day, wrong script, broken button) tell Graeham directly. Don't edit dashboards yourself. Regeneration is one Claude command.

+
+
+
+ + +

Days of the Week 5 scheduled

+
Click a day tile to open the full production view: scoring & why, research breakdown, and content to create.
+ +
+ + +
+
+
Mon
+
27
+
+
Scheduled
+
+
First-Time Buyer Math — Why Renting in 2026 Costs More Than Owning Under $1.1M on the Peninsula — Where They Still Exist in 2026
+
+ BOFU + GHL: BUY + Opp 22.5/25 +
+
+ BOFU buyer anchor — sub-$1M inventory, content gap, Peninsula lead gen. + → +
+ + + +
+
+
Tue
+
28
+
+
Scheduled
+
+
Peninsula Bidding Wars Are Back — Here's What's Actually Different This Time
+
+ BOFU + GHL: OFFERS + Opp 22.5/25 +
+
+ Reddit trending threads, buyer decision-stage, OFFERS CTA. + → +
+ + + +
+
+
Wed
+
29
+
+
Scheduled
+
+
Your Peninsula Home's Hidden Equity — Refinance vs Sell Math for 2026
+
+ BOFU + GHL: EQUITY + Opp 20.2/25 +
+
+ Homeowner BOFU — refi break-even math, sell-vs-stay decision. + → +
+ + + +
+
+
Thu
+
30
+
+
Scheduled
+
+
Woodland Park 772 New Units Approved — What It Does to EPA Home Prices
+
+ MOFU + GHL: EPA + Opp 20.9/25 +
+
+ Council approval this week, permit-led data, 2-3 week window. + → +
+ + + +
+
+
Fri
+
01
+
+
Scheduled
+
+
AB 1482 2026 Amendment — Is Your Rental Exempt? Quick Checklist
+
+ BOFU + GHL: 1482 + Opp 20.5/25 +
+
+ Landlord BOFU — high lead quality, converts to seller 6-12mo. + → +
+ + +
+ + +

Week Intelligence

+
Context the weekly plan is built on: goal mix, what was cut and why.
+ +
+
+ + 📊 Goal Mix Check + Did the 5 scheduled topics hit the funnel-mix target? + +
+ + + + + + + +
TierTargetActualStatus
TOFU20%0%⚠️ Drift >10% (lead_gen goal prioritized)
MOFU30%20%⚠️ Drift >10%
BOFU50%80%⚠️ Drift >10% (by design — lead_gen focus)
+

Goal = lead_gen, so BOFU-heavy is intentional. Drift flags are informational.

+
+
+
+ +
+
+ + ✂️ Cut Topics (5 candidates scored but not selected) + +
+

Topics considered this week and rejected — visible for audit. See the full weekly calendar for scoring details.

+ View Full Weekly Calendar → +
+
+
+ +
+
+ + 📤 Graeham's Edits (Captured overrides) + +
+

No overrides yet this week. Tell Claude if you want to swap, drop, or add topics — your edits will be captured here and persisted.

+
+
+
+ + +

Architecture & References

+
+
+ Weekly Calendar (Rule 14) + Rule 13 (scoring architecture panel) + Rule 14 (weekly calendar spec) + Data Contracts & Schemas + topic-history.json (4-week rolling) + Repo Root +
+ Content Creation Engine v2 (April 2026 streamline) · Two-score model (Opportunity + Intent) · Powered by The Boyenga Team at Compass · DRE #01466876 +
+ +
+ + \ No newline at end of file diff --git a/n8n-workflows/PCFS-CMA-Digest-DAILY+WEEKLY.json b/n8n-workflows/PCFS-CMA-Digest-DAILY+WEEKLY.json new file mode 100755 index 00000000..0efee076 --- /dev/null +++ b/n8n-workflows/PCFS-CMA-Digest-DAILY+WEEKLY.json @@ -0,0 +1,20 @@ +{ + "name": "PCFS — CMA Daily + Weekly Digest", + "_backup_note": "Updated 2026-05-21 by Cowork. Trigger changed from weekly (Mon 9am) to DAILY 9am PT. Build Email code is now day-aware: Monday = CMAs due in next 14 days, Tue–Sun = CMAs due that day only. To = graehamwattsclientcare@gmail.com + graehamwatts@gmail.com. n8n workflow id: LHGnZC2X2KKXljB0.", + "trigger": { + "node": "Daily @ 9am PT", + "type": "scheduleTrigger", + "cron": "0 9 * * *", + "timezone": "America/Los_Angeles" + }, + "build_email_logic": "const isMonday = (tzNow.getDay() === 1). If Monday: CMAs where due date between now and now+14d. Else: CMAs where due date is today (between todayMidnight and todayEnd). Header, intro, empty-state, and subject all switch between 'Next 2 Weeks'/'Today' based on isMonday.", + "recipients": { + "to": "graehamwattsclientcare@gmail.com,graehamwatts@gmail.com" + }, + "manual_fire_webhook": "pcfs-cma-digest-fire", + "source_sheet": "https://docs.google.com/spreadsheets/d/1PtfGzUvjJOz5qNmA5173MqmKO9cLCevAhcP12pFeG3s/edit", + "credentials_used": { + "google_sheets": "AkBUwX11QA8RRHec (Google Sheets account)", + "gmail": "DtB2QyzcO239Eb5l (Gmail OAuth2 API)" + } +} diff --git a/n8n-workflows/PCFS-Sharon-Handwritten-Notes-DAILY+WEEKLY.json b/n8n-workflows/PCFS-Sharon-Handwritten-Notes-DAILY+WEEKLY.json new file mode 100755 index 00000000..65d18e12 --- /dev/null +++ b/n8n-workflows/PCFS-Sharon-Handwritten-Notes-DAILY+WEEKLY.json @@ -0,0 +1,21 @@ +{ + "name": "PCFS — Sharon Daily + Weekly Handwritten Notes", + "_backup_note": "Updated 2026-05-21 by Cowork. Trigger changed from weekly (Mon 8am) to DAILY 8am PT. Build Email code is now day-aware: Monday = full week roster, Tue–Sun = that day's notes only. To = sharonpwatts@gmail.com, CC = graehamwattsclientcare@gmail.com + graehamwatts@gmail.com. n8n workflow id: 7CxqNkCQAuw1noGL.", + "trigger": { + "node": "Daily @ 8am PT", + "type": "scheduleTrigger", + "cron": "0 8 * * *", + "timezone": "America/Los_Angeles" + }, + "build_email_logic": "const isMonday = (tzNow.getDay() === 1). If Monday: notes where Week of (Mon) === thisMondayStr (full week). Else: notes where Date === today (__sameToday). Header, intro, empty-state, and subject all switch between 'This Week'/'Today' based on isMonday.", + "recipients": { + "to": "sharonpwatts@gmail.com", + "cc": "graehamwattsclientcare@gmail.com,graehamwatts@gmail.com" + }, + "manual_fire_webhook": "pcfs-sharon-notes-fire", + "source_sheet": "https://docs.google.com/spreadsheets/d/1PtfGzUvjJOz5qNmA5173MqmKO9cLCevAhcP12pFeG3s/edit", + "credentials_used": { + "google_sheets": "AkBUwX11QA8RRHec (Google Sheets account)", + "gmail": "DtB2QyzcO239Eb5l (Gmail OAuth2 API)" + } +} diff --git a/n8n-workflows/README.md b/n8n-workflows/README.md new file mode 100755 index 00000000..5797de80 --- /dev/null +++ b/n8n-workflows/README.md @@ -0,0 +1,11 @@ +# n8n Workflow Backups + +Reference copies and change summaries for n8n workflows on `n8n.graehamwattsn8n.com`. These are NOT importable JSON exports — the workflows live in n8n cloud and are edited there. These files document what the workflows do and what changed. + +## Workflows tracked here + +| Workflow | n8n ID | Trigger | Notes | +|---|---|---|---| +| PCFS — Sharon Daily + Weekly Handwritten Notes | `7CxqNkCQAuw1noGL` | Daily 8am PT (cron `0 8 * * *`) | Mon = full week roster; Tue–Sun = today's note(s) only. Changed from weekly to daily 2026-05-21. | +| PCFS — CMA Daily + Weekly Digest | `LHGnZC2X2KKXljB0` | Daily 9am PT (cron `0 9 * * *`) | Mon = next 2 weeks; Tue–Sun = CMAs due today. Changed from weekly to daily 2026-05-21. | +| PCFS — CMA Autobuild Watchdog | `SMQMpqyKWQVBkiZs` | Mon 11am PT (cron `0 11 * * 1`) | Watches the Cowork autobuild task. Pulls due-list webhook + searches Gmail. Alerts Graeham if expected CMA review emails are missing. Created 2026-05-26. | diff --git a/n8n-workflows/plaid-transactions-to-sheets.json b/n8n-workflows/plaid-transactions-to-sheets.json new file mode 100644 index 00000000..eaf1c558 --- /dev/null +++ b/n8n-workflows/plaid-transactions-to-sheets.json @@ -0,0 +1,87 @@ +{ + "name": "Plaid → Finance Sheet (Bank Transactions Pull)", + "meta": { + "description": "Rebuilt 2026-06-04. Pulls Chase business transactions from Plaid and appends them to the finance Google Sheet in Graeham's canonical schema. Replaces the workflow lost in the cloud→Mac Studio n8n migration. Secrets are read from n8n ENV vars so this JSON is safe to commit to GitHub.", + "fillBeforeRunning": [ + "n8n ENV: PLAID_CLIENT_ID, PLAID_SECRET, PLAID_ACCESS_TOKEN, PLAID_ENV (sandbox|development|production), PLAID_LOOKBACK_DAYS (optional, default 90)", + "Google Sheets credential (OAuth2) selected on the 'Append to Finance Sheet' node", + "GOOGLE_SHEET_ID and target tab name on the 'Append to Finance Sheet' node" + ] + }, + "nodes": [ + { + "id": "trigger", + "name": "Daily 6am", + "type": "n8n-nodes-base.scheduleTrigger", + "typeVersion": 1.2, + "position": [220, 300], + "parameters": { + "rule": { + "interval": [ + { "field": "cronExpression", "expression": "0 6 * * *" } + ] + } + } + }, + { + "id": "buildreq", + "name": "Build Plaid Request", + "type": "n8n-nodes-base.code", + "typeVersion": 2, + "position": [460, 300], + "parameters": { + "jsCode": "// Reads secrets from n8n environment variables so they never live in this file.\nconst env = $env;\nconst PLAID_ENV = (env.PLAID_ENV || 'production').toLowerCase();\nconst host = PLAID_ENV === 'sandbox' ? 'https://sandbox.plaid.com'\n : PLAID_ENV === 'development' ? 'https://development.plaid.com'\n : 'https://production.plaid.com';\nconst lookback = Number(env.PLAID_LOOKBACK_DAYS || 90);\nconst end = new Date();\nconst start = new Date();\nstart.setDate(end.getDate() - lookback);\nconst fmt = (d) => d.toISOString().slice(0, 10);\nreturn [{ json: {\n plaidUrl: host + '/transactions/get',\n reqBody: {\n client_id: env.PLAID_CLIENT_ID,\n secret: env.PLAID_SECRET,\n access_token: env.PLAID_ACCESS_TOKEN,\n start_date: fmt(start),\n end_date: fmt(end),\n options: { count: 500, offset: 0 }\n }\n}}];" + } + }, + { + "id": "plaid", + "name": "Plaid /transactions/get", + "type": "n8n-nodes-base.httpRequest", + "typeVersion": 4.4, + "position": [700, 300], + "parameters": { + "method": "POST", + "url": "={{ $json.plaidUrl }}", + "sendBody": true, + "contentType": "json", + "specifyBody": "json", + "jsonBody": "={{ JSON.stringify($json.reqBody) }}", + "options": { "response": { "response": { "neverError": false } } } + }, + "notes": "Plaid is paginated. If total_transactions > 500, add a loop that re-calls with options.offset += 500 until all are fetched. v1 fetches the most recent 500 (plenty for a daily/weekly pull)." + }, + { + "id": "map", + "name": "Map to Sheet Schema", + "type": "n8n-nodes-base.code", + "typeVersion": 2, + "position": [940, 300], + "parameters": { + "jsCode": "// Flatten Plaid response into Graeham's canonical row schema.\nconst res = $json;\nconst accts = {};\n(res.accounts || []).forEach(a => { accts[a.account_id] = (a.name || '') + ' ...' + (a.mask || ''); });\nconst txns = res.transactions || [];\nreturn txns.map(t => ({ json: {\n txn_id: t.transaction_id,\n date: t.date,\n account: accts[t.account_id] || t.account_id,\n merchant: t.merchant_name || t.name,\n category: (t.personal_finance_category && t.personal_finance_category.primary) || (Array.isArray(t.category) ? t.category.join(' > ') : ''),\n amount: t.amount,\n currency: t.iso_currency_code || 'USD',\n pending: t.pending,\n notes: t.name,\n receipt_match: ''\n}}));" + }, + "notes": "Plaid amount sign: POSITIVE = money leaving the account (debit), NEGATIVE = money in. This matches Graeham's existing chase_transactions.csv. Entity/IRS-line routing via the 'Tagging Rules' tab is a layer-2 enhancement — add a lookup here against that tab." + }, + { + "id": "sheet", + "name": "Append to Finance Sheet", + "type": "n8n-nodes-base.googleSheets", + "typeVersion": 4.5, + "position": [1180, 300], + "parameters": { + "operation": "append", + "documentId": { "__rl": true, "mode": "id", "value": "<>" }, + "sheetName": { "__rl": true, "mode": "name", "value": "Bank Transactions" }, + "mappingMode": "autoMapInputData", + "options": { "cellFormat": "USER_ENTERED" } + }, + "notes": "Set documentId to the finance Google Sheet ID and sheetName to the raw bank-transactions tab. De-dupe on txn_id: switch to 'appendOrUpdate' keyed on txn_id once the tab has a txn_id column, so re-runs don't double-post." + } + ], + "connections": { + "Daily 6am": { "main": [[{ "node": "Build Plaid Request", "type": "main", "index": 0 }]] }, + "Build Plaid Request": { "main": [[{ "node": "Plaid /transactions/get", "type": "main", "index": 0 }]] }, + "Plaid /transactions/get": { "main": [[{ "node": "Map to Sheet Schema", "type": "main", "index": 0 }]] }, + "Map to Sheet Schema": { "main": [[{ "node": "Append to Finance Sheet", "type": "main", "index": 0 }]] } + }, + "settings": { "executionOrder": "v1", "timezone": "America/Los_Angeles" } +} diff --git a/online-content/dashboards/switchy/index.html b/online-content/dashboards/switchy/index.html new file mode 100644 index 00000000..4a98ac5b --- /dev/null +++ b/online-content/dashboards/switchy/index.html @@ -0,0 +1,106 @@ + + +Switchy Clicks Dashboard — Graeham Watts + + +

Switchy Clicks Dashboard ALL SOURCES

+
Generated 2026-06-08 15:04 · vs. 2026-06-01 (switchy-snapshot-2026-06-01.json) · model: 55% match · 10×/30d · $22 CPM
+ +
+
16,033
Total clicks / scans
+
+27
New this week
+
8,817
Targetable audience
+
$1,940
Justified ad budget / mo
+
+ +
+

Where the clicks come from (by Switchy folder)

+ +
+ +
+

Sources breakdown

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Source (folder)LinksClicksNew/wkAudienceBudget/mo
Google Ads448,53704,695$1,033
youtube descrip63,905+112,148$473
Facebook Ads211,7260949$209
GMB EPA updates1452990164$36
Yard Sign QR1293+11161$35
Blog Links9241+1133$29
GMB General107156086$0
GAEPAunder1mil1136075$0
adsby bellehave5113062$0
Adsby epaSearch596053$0
Postcard Old193051$0
adsby rwcsearch580044$0
Post card qr768037$0
Unfiled / no source tag1248+426$0
GMBRedwood City2844024$0
GARC Homes FS133018$0
buyerlinkghl251427015$0
GHL Calendar li122012$0
Seller Drip822012$0
Sellerlinkghl25721012$0
Buyer Drip Camp120011$0
SHORTS GMB418010$0
Orchard Park11709$0
952 6th AVE1704$0
GMB EPA Product4704$0
GMB East Menlo3402$0
TOTAL44216,033+278,817$1,940
+
+ +
+

Top 15 links

+ + + + + + + + + + + + + + + + + +
LinkSourceShort URLClicksNew/wk
Homes for Sale - Graeham Watts - Intero Real Eyoutube descrippages.graehamwatts.com/bay-area-homes-for-sale1,073+4
Sell Your Home with Graeham Watts - Step-by-StGoogle Adspages.graehamwatts.com/SbSR1,0230
graeham watts - 30 Minute Meeting | TidyCalyoutube descrippages.graehamwatts.com/graehamwatts-meet780+1
How Much is My Home Worth? - Get Free Report -youtube descrippages.graehamwatts.com/whats-my-house-worth724+2
How Much is My Home Worth? - Get Free Report |Google Adspages.graehamwatts.com/SKZY6810
Get in Touch with Graeham Watts: Let’s Talk AbGoogle Adspages.graehamwatts.com/SKaA6020
How Much is My Home Worth? - Get Free Report |Google Adspages.graehamwatts.com/SK0r5860
2271 Euclid AVE, East Palo Alto, CA 94303 - MLFacebook Adspages.graehamwatts.com/XhI55700
How Much is My Home Worth? - Get Free Report |Google Adspages.graehamwatts.com/SK1P5470
Sell Your Home with Graeham Watts - Step-by-StGoogle Adspages.graehamwatts.com/SKai5150
Buy Your Home with Graeham WattsGoogle Adspages.graehamwatts.com/SKY85090
Redwood City Homes for Saleyoutube descrippages.graehamwatts.com/redwood-city-homes-for-sale498+1
Buy Your Home with Graeham WattsGoogle Adspages.graehamwatts.com/SKYl4600
East palo alto ca homes for saleyoutube descrippages.graehamwatts.com/east-palo-alto-ca-homes-for-sale432+2
Explore East Menlo Park: A Hidden Gem!youtube descrippages.graehamwatts.com/homes-for-sale-belle-haven398+1
+
+ +
+ How to read this: Audience = clicks that resolve to a targetable pixeled user (55%). Budget/mo = what that audience can absorb at 10×/30d, $22 CPM — a ceiling, not a target. Sources are the Switchy folders each link lives in; "Unfiled" links need a folder/tag to be attributable. Switchy's API gives click totals only — geo/referrer/device live in GA4 (via UTM) and Meta (via pixel). +
+ + + \ No newline at end of file diff --git a/online-content/dashboards/switchy/monday-email.html b/online-content/dashboards/switchy/monday-email.html new file mode 100644 index 00000000..fa136273 --- /dev/null +++ b/online-content/dashboards/switchy/monday-email.html @@ -0,0 +1,34 @@ + +Your Monday Switchy Report + +
+
+
Monday Switchy Report
+
Where your clicks came from this week
+
+
+ + + + + + +
15,997
Total scans/clicks
8,798
Targetable audience
$1,936
Justified ad $/mo
+

+ Your full breakdown — clicks by source (postcards, Google Business, YouTube, + yard signs, ad campaigns), top links, and week-over-week growth — is on the + live dashboard. It refreshes every Monday. +

+ +

+ Link goes live once the skills repo is pushed to GitHub Pages. Switchy reports + click totals only; geo/referrer/device live in GA4 (via UTM) and Meta (via pixel). +

+
+
+ diff --git a/online-content/dashboards/switchy/snapshots/switchy-snapshot-2026-05-28.json b/online-content/dashboards/switchy/snapshots/switchy-snapshot-2026-05-28.json new file mode 100644 index 00000000..ec39c4b6 --- /dev/null +++ b/online-content/dashboards/switchy/snapshots/switchy-snapshot-2026-05-28.json @@ -0,0 +1 @@ +{"date": "2026-05-28", "clicks": {"bay-area-homes-for-sale": 1067, "SbSR": 1023, "graehamwatts-meet": 776, "whats-my-house-worth": 722, "SKZY": 681, "SKaA": 602, "SK0r": 585, "XhI5": 570, "SK1P": 547, "SKai": 515, "SKY8": 509, "redwood-city-homes-for-sale": 496, "SKYl": 460, "east-palo-alto-ca-homes-for-sale": 429, "homes-for-sale-belle-haven": 397, "T1td": 396, "SJob": 376, "SJ-k": 296, "MFUy": 281, "SXDB": 234, "SM9A": 205, "SKZH": 190, "SRlP": 185, "SKDr": 180, "SX1b": 170, "SJoH": 163, "SKYx": 141, "SJpt": 141, "east-palo-alto-CA-homes-for-sale-under-1million": 136, "SJ-D": 126, "SX7L": 124, "SK0T": 123, "S-VQ": 116, "T1nG": 111, "SXGz": 108, "SKYW": 105, "East-palo-alto-home-for-sale": 96, "menlo-park-homes-for-sale": 94, "how-much-is-my-home-worth": 93, "T1r5": 90, "S-Vz": 88, "redwood-city-home-for-sale": 80, "T8PU": 78, "TA7m": 71, "SX5l": 70, "SXG2": 70, "S9yj": 67, "XhH8": 61, "schedule-bay-area-real-estate-meeting": 60, "SKYI": 60, "blog-east-palo-alto-ca-homes-for-sale": 59, "S-Vg": 59, "free-home-evaluation": 59, "S9yo": 54, "T1v5": 51, "SM92": 50, "SXBD": 48, "S9xP": 41, "SX4-": 41, "T8Yl": 41, "SKXy": 39, "redwood-city-ca-homes-for-sale": 33, "real-estate-market-forecast-save-thousands-on-your-mortgage-hack": 28, "formly_email": 28, "WVkH": 28, "S9w-": 27, "bay-area-real-estate-market-update-2026-01": 27, "SKXW": 24, "Meet-with-Graeham": 22, "S9xr": 22, "Home-For-Sale": 20, "SM9K": 20, "east-menlo-park": 19, "sell-with-graeham": 19, "5636-orchard-park-drive-san-jose-ca-95123": 17, "why-sell-with-graeham": 16, "discover-east-menlo-park-homes": 15, "Belle-haven-home-value-evaluation": 14, "SKaM": 13, "homes-for-sale-redwood-city-ca": 13, "east-palo-alto-real-estate-sold-40k-more-with-50k-issue": 13, "get_free_consultation": 13, "home-selling-tips-sell-your-home-quickly": 12, "homevalue": 11, "v4Sk": 10, "S9y0": 10, "before_after_1930_Sarah_Dr": 9, "SX9V": 9, "S9z0": 9, "schedule-call-with-graeham": 9, "woodside-plaza-redwood-city-homes-for-sale": 9, "east-palo-alto-condo-1982-w-bayshore-223": 9, "essential-home-selling-tips-5-Biggest-Mistakes": 8, "homes-for-sale-east-palo-alto-ca-discover-your-dream-home": 7, "952-6th-AVE-Redwood-City-CA-94063": 7, "828-Weeks-ST": 7, "homes-for-sale-in-the-bay-area-safety-tips": 7, "free-home-valuation": 7, "Home-Buying-Myths": 7, "redwood-city-real-estate-expert-advice-you-can-trust": 6, "east-palo-alto-real-estate-470-bell-st-tranquil-home-tour-silicon-valley": 6, "real-estate-market-forecast-fed-rate-cuts-2025-housing-market": 6, "bay-area-realtor-graeham-watts-winnie-danny-dream-home": 6, "real-estate-market-update-transformed-sold-for-top-dollar": 6, "east-palo-alto-condos-for-sale-woodland-creek-223": 5, "3-Buyer-Myths": 5, "selling-homes-quickly-tips-maximize-home-value": 5, "east-palo-alto-real-estate-insights-650k-fixer-upper-dream-home": 5, "Home-Evaluation": 5, "bay-area-realtor-dream-home-client-testimonial": 5, "2025-california-landlord-risk-update": 5, "Why-Clients-Trust-Graeham-Watts": 5, "homes-for-sale-in-menlo-park-graeham-watts-tour": 5, "redwood-city-real-estate-cost-of-living": 4, "Proven-Home-Selling-Tips": 4, "east-palo-alto-homes-for-sale-431-larkspur-dr-record-sale": 4, "east-palo-alto-market-update-prices-rising-homes-selling-fast": 4, "KII7": 4, "east-palo-alto-real-estate-470-bell-st-listing": 4, "avoid-costly-homebuyer-errors-redwood-city-real-estate-ca": 4, "SJpA": 4, "real-estate-market-update-313-smithwood-milpitas-ca-95035-home-tour": 4, "Offer-Accepted": 4, "top-east-palo-alto-realtor-sell-home-fast-top-dollar": 4, "top-east-palo-alto-realtor-home-selling-tips": 4, "discover-homes-for-sale-redwood-city-ca-spanish-style-4-bedroom-friendly-acres": 4, "san-jose-home-for-sale-500k-assumable-rate": 4, "How-We-Price-Your-Home": 4, "east-palo-alto-homes-for-sale-123-main-st-tour": 4, "redwood-city-real-estate-monthly-market-update-Graeham-watts": 4, "houses-in-east-palo-alto-modernized-1239-jervis-ave": 4, "East-Palo-Alto-Real-Estate-Monthly-Market-Update-graeham-watts": 4, "east-palo-alto-real-estate-missed-ca-fee-amnesty-act-now": 3, "east-palo-alto-realtor-sell-home-fast-top-dollar": 3, "bay-area-realtor-graeham-watts-testimonial-savings": 3, "1457-quail-st-los-banos-4-bedroom-home": 3, "affordable-east-palo-alto-homes-14-robin-court-tour": 3, "bay-area-realtor-free-home-staging-maximize-value": 3, "real-estate-predictions-2025-east-menlo-park": 3, "redwood-city-real-estate-graeham-watts-love-this-city": 3, "bay-area-realtor-california-rent-rules": 3, "bay-area-realtor-supply-and-demand-toilet-paper-bidding-war": 3, "home-selling-tips-maximize-sale-price-fast": 3, "big-head-big-ideas-big-results-real-estate-predictions-2025": 3, "bay-area-realtor-graeham-watts-johnny-ashley-success": 3, "ca-rent-laws-in-crises": 3, "east-palo-alto-real-estate-market-update-graeham-watts": 3, "essential-home-selling-tips-third-step-home-search": 3, "bay-area-realtor-graeham-watts-ryan-teeda-journey": 3, "1908-Cooley-AVE": 3, "east-menlo-park-market-update-prices-up-inventory-low": 3, "bair-island-homes-for-sale-in-redwood-city": 3, "east-menlo-park-real-estate-march-market-update": 3, "The-Graeham-Watts-Advantage": 3, "homes-for-sale-in-the-bay-area-buy-sell-with-graeham-watts": 3, "redwood-city-march-real-estate-market-update": 3, "discover-east-palo-alto-real-estate-stunning-home-under-900k": 3, "redwood-city-homes-for-sale-hidden-gems-affordable-living": 3, "redwood-city-real-estate-off-market-secrets": 3, "real-estate-market-forecast": 3, "east-palo-alto-market-update-are-you-keeping-up": 3, "east-palo-alto-ravenswood-school-district-transformation": 3, "east-palo-alto-ca-homes-for-sale-woodland-creek-condo-tour": 3, "redwood-city-real-estate-insider": 3, "client-testimonial-moneisha-jermell": 3, "bay-area-realtor-graeham-watts-hidden-gems": 3, "bay-area-realtor-graeham-watts-dream-home-success": 3, "east-palo-alto-ca-house-for-sale-worth-investment": 2, "real-estate-market-update-homes-selling-over-asking": 2, "homes-for-sale-in-east-palo-alto-ca-431-larkspur-dr": 2, "east-palo-alto-real-estate-why-now-is-the-time-to-buy": 2, "belle-haven-market-update": 2, "unleashing-potential-east-palo-alto-real-estate-transformative-fixer-upper": 2, "homes-for-sale-east-palo-alto-ca-117-mission-drive": 2, "bay-area-realtor-graeham-watts-record-home-sale": 2, "redwood-city-homes-for-sale-757-douglas-ave-tour": 2, "homes-for-sale-in-menlo-park-home-value": 2, "east-palo-alto-homes-for-sale-2620-fordham-st-update": 2, "friendly-acres-redwood-city": 2, "redwood-city-condos-for-sale": 2, "redwood-city-living-expense": 2, "east-menlo-park-market-update": 2, "max-profit-selling-homes-redwood-city": 2, "graeham-watts-advantage-redwood-city-real-estate": 2, "east-palo-alto-ca-homes-for-sale-2109-myrtle-pl-tour": 2, "redwood-shores-homes-for-sale-luxury-waterfront-tour": 2, "menlo-park-houses-for-sale-breathtaking-transformation": 2, "houses-for-sale-in-east-palo-alto-1404-camellia-drive": 2, "houses-in-east-palo-alto-952-newbridge-st-real-estate-gem": 2, "bay-area-realtor-graeham-client-testimonial-john-ward": 2, "home-selling-tips-attract-perfect-buyers": 2, "real-estate-market-update-menlo-park-2022-trends": 2, "bay-area-realtor-graeham-watts-trusted-expert": 2, "home-selling-tips-get-top-dollar-for-your-property": 2, "homes-for-sale-in-the-bay-area-los-gatos-home-sells-big": 2, "homes-for-sale-in-the-bay-area-luxury-the-westerly": 2, "redwood-city-real-estate-ca-safer": 2, "redwood-city-real-estate-next-big-investment": 2, "homes-for-sale-in-menlo-park-market-trends-graeham-watts": 2, "bay-area-realtor-home-alone-reaction": 2, "redwood-city-real-estate-january-market-update": 2, "east-menlo-park-market-update-january-2025": 2, "east-palo-alto-real-estate-homeowner-update": 2, "redwood-city-real-estate-charming-dream-homes": 2, "redwood-city-march-real-estate-market-update-2025": 2, "redwood-city-real-estate-market-home-value-insight": 2, "selling-homes-quickly-tips-kitchen-countertops": 2, "SK1C": 2, "SM9O": 2, "east-palo-alto-ca-homes-for-sale-1765-e-bayshore-rd-214": 2, "home-selling-tips-expert-staging-top-dollar": 2, "bay-area-realtor-essential-real-estate-tips": 2, "Home-Search": 2, "redwood-city-real-estate-prices": 2, "east-palo-alto-california-real-estate": 2, "rise-and-fall-of-silicon-valley-bank": 2, "bay-area-realtor-graeham-watts-off-market-dream-homes": 2, "just-sold-east-palo-alto-real-estate-1765-e-bayshore-rd-203": 2, "real-estate-market-update-stunning-home-oakley-59-escher-circle-tour-features": 2, "broadway-masala-redwood-city-indian-food": 2, "east-bay-real-estate-hidden-deals-maximize-sale": 2, "join-bay-area-realtor-graeham-watts-fiesta": 2, "mortgage-rates-forecast-2025-housing-loan-trends": 2, "shocking-bay-area-house-tour-twist": 2, "iBiB": 2, "bay-area-realtor-graeham-watts-humor-and-real-estate": 2, "bay-area-realtor-client-testimonial-trust": 2, "bay-area-realtor-daily-hustle-inside-look": 2, "graeham-watts-client-testimonials": 2, "Choosing-a-Bay-Area-Realtor": 2, "Understanding-Contingencies": 2, "nEnF": 2, "is-now-the-best-time-to-buy-real-estate": 2, "v4S8": 2, "sell-home-prep-tips": 2, "real-estate-market-update-you-need-to-know": 2, "Home-Selling-Tips": 2, "redwood-city-real-estate-ca-silicon-valley-oasis": 2, "graeham-watts-bay-area-realtor-real-estate-success": 2, "bay-area-realtor-graeham-watts-advantage": 2, "houses-for-sale-east-palo-alto-453-okeefe-st": 1, "real-estate-market-forecast-august-trends": 1, "east-menlo-park-market-update-home-prices-up": 1, "Real-Estate-Market-Forecast-Save-Thousands": 1, "bay-area-realtor-success-story-marketing-results": 1, "Why-Clients-Trust-Graeham": 1, "redwood-city-real-estate-charming-2bd-1ba-home-tour-expansive-yard": 1, "real-estate-market-update-top-investment-websites": 1, "east-palo-alto-market-update-free-home-valuation": 1, "east-palo-alto-homes-for-sale-2620-fordham": 1, "belle-haven-market-update-menlo-park": 1, "real-estate-market-update-sell-home-top-dollar": 1, "east-palo-alto-real-estate-market-update-2023": 1, "homes-for-sale-in-the-bay-area-just-listed-hot-properties": 1, "bay-area-realtor-graeham-watts-staging-and-3d-tours": 1, "rising-interest-rates-and-real-estate": 1, "redwood-city-real-estate-update-sell-high-inventory-rising": 1, "redwood-city-real-estate-update-2022-home-value": 1, "Home-buyer-guide": 1, "houses-in-east-palo-alto-dream-homes-for-every-budget": 1, "menlo-park-houses-for-sale-monthly-market-update": 1, "east-palo-alto-homes-for-sale-stylish-loft": 1, "ULo7": 1, "redwood-city-real-estate-market-update-for-homeowners": 1, "east-palo-alto-california-real-estate-trends": 1, "real-estate-market-trends-2025-home-warranty-explained": 1, "real-estate-inventory-trends-low-inventory-market-impact": 1, "east-palo-alto-real-estate-market-update-feb-2025": 1, "homes-for-sale-in-the-bay-area-san-leandro-modern-family-home": 1, "bay-area-realtor-graeham-watts-client-testimonial": 1, "homes-for-sale-in-farm-hill-7210-eagle-ridge-dr-gilroy-ca": 1, "KII1": 1, "essential-home-selling-tips-biggest-seller-mistakes": 1, "east-palo-alto-ca-homes-for-sale-buyer-tips-1765-e-bayshore-rd-204": 1, "menlo-park-real-estate-market-update-latest-trends-insights": 1, "home-selling-tips-fix-now-pay-later": 1, "homes-for-sale-in-redwood-city": 1, "real-estate-market-forecast-shocker-haunting-2025": 1, "home-selling-tips-out-of-state-overwhelmed-help": 1, "real-estate-update-home-selling-fast": 1, "rare-redwood-city-real-estate-fixer-2-bed-condo-hidden-potential": 1, "redwood-city-real-estate-insights": 1, "game-changing-real-estate-predictions-2025-big-deals-big-results": 1, "home-selling-mistakes-to-avoid": 1, "best-time-to-buy-real-estate-1930-pinole-drive-tour": 1, "ab-1482-unlocked-secrets-every-california-landlord-should-know": 1, "scariest-real-estate-costume-ever": 1, "redwood-city-real-estate-market-update-prices-up-inventory-down": 1, "home-alone-sequel-we-always-wanted-east-palo-alto-homes": 1, "unlock-hidden-redwood-city-real-estate-ca-gems": 1, "beat-the-market-east-palo-alto-market-update-secret-price-bump": 1, "redwood-city-real-estate-renovation-alert": 1, "2842-Cornelius-Dr-Tour": 1, "Featured-Property-menlo-park": 1, "Featured-Property-Redwood-City": 1, "Featured-Property-east-palo-alto": 1, "east-palo-alto-homes-for-sale-1765-bayshore-rd-203": 1, "homes-for-sale-in-east-palo-alto-ca-2398-palgas-ave": 1, "redwood-city-real-estate-showdown": 1, "redwood-city-real-estate-2022-sell-high-find-home": 1, "east-palo-alto-real-estate-gaillardia-way-property": 1, "real-estate-law-101-ca-landlord-entry-laws-explained": 1, "ab1482-explained-avoid-landlord-mistakes": 1, "client-testimonial-kevin-rebecca-bowe": 1, "east-palo-alto-market-update-2022-inventory-and-prices": 1, "east-palo-alto-market-update-2022-sell-for-maximum-value": 1, "real-estate-market-update-city-vs-county-closing-costs": 1, "redwood-city-real-estate-2025-market-shocker-revealed": 1, "discover-stunning-homes-for-sale-east-palo-alto-ca-graeham-watts-advantage": 1, "east-palo-alto-market-update-2022-home-value-increase": 1, "fha-loans-vs-conventional-loans": 1, "Home-Selling-Tips-Must-Know": 1, "price-your-home-right": 1, "thinking-of-selling-watch-this": 1, "home-worth-estimate": 1, "bay-area-realtor-why-experience-matters": 1, "san-mateo-real-estate-update": 1, "redwood-city-real-estate-market-update": 1, "east-palo-alto-real-estate-mark-dinan-vision-for-change-city-council-candidate": 1, "tech-growth-impact-redwood-city-real-estate": 1, "interest-rates-and-real-estate-market-impact": 1, "bay-area-realtor-graeham-watts-difference": 1, "east-palo-alto-houses-14-robin-ct-tour": 1, "unlock-redwood-city-real-estate": 1, "real-estate-market-update-22277-hartman-drive-property-tour": 1, "homes-for-sale-redwood-city-ca-staging-secrets": 1, "mt-carmel-redwood-city-homes-for-sale": 1, "redwood-city-real-estate-market-update-graeham-watts": 1, "redwood-city-real-estate-hidden-gems-revealed": 1, "mortgage-costs-redwood-city-real-estate-ca": 1, "discover-affordable-homes-under-100k-across-america": 1, "redwood-city-ca-real-estate-property-tax": 1, "ab-1482-explained-landlords-tenants-must-know": 1, "homes-for-sale-in-redwood-city-ca": 1, "redwood-city-real-estate-monthly-market-update": 1, "real-estate-market-forecast-foreclosure-myth-busted": 1, "roosevelt-redwood-city-homes-for-sale": 1, "homes-for-sale-east-palo-alto-ca-living-guide": 1, "redwood-city-real-estate-market-update-feb-2025": 1, "redwood-city-real-estate-financing": 1, "essential-home-selling-tips": 1, "essential-home-selling-tips-fourth-step-offer-accepted": 1, "homes-for-sale-in-redwood-city-ca-590-hurlingame-ave-makeover": 1, "homes-for-sale-in-the-bay-area-3500-19th-st-san-francisco": 1, "homes-for-sale-in-east-palo-alto-ca-2288-addison-ave": 1, "homes-for-sale-in-redwood-city-ca-winning-offers-underwriting-edge": 1, "homes-for-sale-in-the-bay-area-786-honeywood-court-pending-sale": 1, "homes-for-sale-in-the-bay-area-graeham-watts-2022-successes": 1, "real-estate-market-update-186-overlook-ave-hayward-ca-virtual-tour": 1, "bay-area-realtor-graeham-watts-client-love": 1, "ab-1482-explained-maximum-rent-increase-limits-in-california": 1, "new-rent-rules-real-estate-predictions-2025-tenants-landlords": 0, "redwood-city-real-estate-moving-guide-what-you-need-to-know": 0, "east-menlo-park-market-update-february-2025": 0, "east-palo-alto-real-estate-moving-guide": 0, "home-selling-tips-make-sales-breeze-no-stress": 0, "east-palo-alto-homes-for-sale": 0, "S9z9": 0, "east-palo-alto-dream-home-tour-30-seconds": 0, "redwood-city-real-estate-hidden-secrets": 0, "real-estate-market-trends-2025-alameda-county-hidden-gem": 0, "unbelievable-house-tour-market-forecast": 0, "palm-redwood-city-homes-for-sale": 0, "discover-east-palo-alto-real-estate-february-market-update": 0, "homes-for-sale-in-the-bay-area-soquel-3-bedroom": 0, "exciting-real-estate-market-update-jumping-housing-market": 0, "trusted-bay-area-realtor-house-safety-guarantee": 0, "redwood-city-real-estate-floor-installation-tips": 0, "unexpected-homes-for-sale-bay-area-dream-home": 0, "property-tax-shock-2025-real-estate-forecast": 0, "explore-east-palo-alto-homes-graeham-watts": 0, "101_Graden_St": 0, "ravenswood-school-district-50m-investment-better-schools-higher-teacher-pay": 0, "top-tips-for-selling-homes-quickly-maximize-home-value": 0, "rising-interest-rates-and-real-estate-prices": 0, "homes-for-sale-east-palo-alto": 0, "buy-a-home-east-palo-alto": 0, "redwood-village-homes-for-sale-in-redwood-city": 0, "east-palo-alto-market-update-january-2025-key-insights": 0, "real-estate-market-update-hartman-drive-los-altos-ca-virtual-tour": 0, "bay-area-realtor-secrets-unveiled": 0, "east-palo-alto-next-real-estate-goldmine": 0, "east-palo-alto-ca-homes-for-sale-house-tour": 0, "homes-for-sale-menlo-park-ca-modern-home-tour-1318-hollyburne-avenue": 0, "graeham-watts-east-palo-alto-listings": 0, "real-estate-market-trends-2025-phone-to-realty": 0, "discover-homes-for-sale-east-palo-alto-ca-1560-kavanaugh-drive-tour": 0, "home-selling-tips-busting-3-buyer-myths": 0, "the-buzz-in-redwood-city-real-estate": 0, "ab1482-what-landlords-need-to-know-and-do": 0, "eagle-hill-redwood-city-homes-for-sale": 0, "east-menlo-park-homes-for-sale-graeham-watts": 0, "epa-comps-0601": 0, "discover-stunning-homes-for-sale-menlo-park": 0, "east-menlo-park-homes-for-sale-listings-graeham-watts": 0, "expert-home-selling-tips": 0, "bay-area-realtor-free-staging-and-repairs": 0, "ab-1482-rent-increases-exemptions-and-california-rental-laws-explained": 0, "Ace_Your_First_Home_Purchase": 0, "homes-for-sale-menlo-park-ca-update": 0, "bay-area-home-search-map": 0, "downtown-redwood-city-homes-for-sale": 0, "maximize-your-home-sale-value": 0, "east-menlo-park-homes-for-sale": 0, "redwood-city-real-estate-paradise": 0, "Redwood-Oaks-homes-for-sale-in-redwood-city-ca": 0, "stambaugh-heller-redwood-city-homes-for-sale": 0, "east-palo-alto-real-estate-housing-market-update": 0, "essential-real-estate-market-update-homebuyers-guide": 0, "redwood-city-real-estate-housing-market-update": 0, "real-estate-market-update-2025-california-landlord-risks": 0, "central-redwood-city-homes-for-sale": 0, "east-palo-alto-homes-for-sale-woodland-creek-condo": 0, "discovering-redwood-city-real-estate": 0, "menlo-park-market-update-belle-haven-housing-trends": 0, "real-estate-market-forecast-fed-interest-rates-explained": 0, "redwood-city-real-estate-crucial-market-update": 0, "real-estate-market-update-just-listed-1186-overlook-ave-hayward-ca-house-tour": 0, "east-palo-alto-real-estate-monthly-market-update": 0, "centennial-redwood-city-homes-for-sale": 0, "real-estate-market-trends-2025": 0, "kIhN": 0, "best-indian-restaurant-redwood-city-broadway-masala": 0, "Redwood-City-Homes-on-Sale-Hot-Listings-Price-Drops": 0, "Redwood-City-Homes-on-Sale-Hot-Listings-Price-Drops-Graeham-Watts": 0, "Redwood-City-Homes-on-Sale": 0, "2115-Clarke-Ave": 0, "Redwood-City-Homes-on-Sale-Hot-Listings": 0, "redwood-city-real-estate-san-francisco-monthly-market-update": 0, "essential-real-estate-market-update-guide-for-homebuyers": 0, "essential-home-selling-tips-second-step-pre-qualification": 0, "choose-the-right-agent": 0, "ab-1482-explained-is-your-property-rent-controlled-or-exempt": 0, "stunning-homes-for-sale-menlo-park-1135-madera-ave-tour": 0, "redwood-city-spanish-style-4-bedroom-friendly-acres": 0, "east-palo-alto-real-estate-market-update": 0, "real-estate-market-update-foundation-tips-when-buying-your-new-home": 0, "real-estate-market-update-tips-to-protect-your-offer-and-investment": 0, "east-palo-alto-real-estate-mark-dinan-vision-change": 0, "what-you-can-buy-redwood-city-houses-for-sale": 0, "redwood-city-real-estate-market-february-update-you-need-to-know": 0, "selling-your-home-just-got-easier": 0}} \ No newline at end of file diff --git a/online-content/dashboards/switchy/snapshots/switchy-snapshot-2026-05-29.json b/online-content/dashboards/switchy/snapshots/switchy-snapshot-2026-05-29.json new file mode 100644 index 00000000..38635ff1 --- /dev/null +++ b/online-content/dashboards/switchy/snapshots/switchy-snapshot-2026-05-29.json @@ -0,0 +1 @@ +{"date": "2026-05-29", "clicks": {"bay-area-homes-for-sale": 1067, "SbSR": 1023, "graehamwatts-meet": 776, "whats-my-house-worth": 722, "SKZY": 681, "SKaA": 602, "SK0r": 586, "XhI5": 570, "SK1P": 547, "SKai": 515, "SKY8": 509, "redwood-city-homes-for-sale": 496, "SKYl": 460, "east-palo-alto-ca-homes-for-sale": 430, "homes-for-sale-belle-haven": 397, "T1td": 396, "SJob": 376, "SJ-k": 296, "MFUy": 282, "SXDB": 234, "SM9A": 205, "SKZH": 190, "SRlP": 185, "SKDr": 180, "SX1b": 170, "SJoH": 163, "SKYx": 141, "SJpt": 141, "east-palo-alto-CA-homes-for-sale-under-1million": 136, "SJ-D": 126, "SX7L": 124, "SK0T": 123, "S-VQ": 116, "T1nG": 111, "SXGz": 108, "SKYW": 105, "East-palo-alto-home-for-sale": 96, "menlo-park-homes-for-sale": 94, "how-much-is-my-home-worth": 93, "T1r5": 90, "S-Vz": 88, "redwood-city-home-for-sale": 80, "T8PU": 78, "TA7m": 71, "SXG2": 70, "SX5l": 70, "S9yj": 67, "XhH8": 61, "SKYI": 60, "schedule-bay-area-real-estate-meeting": 60, "blog-east-palo-alto-ca-homes-for-sale": 59, "free-home-evaluation": 59, "S-Vg": 59, "S9yo": 54, "T1v5": 51, "SM92": 50, "SXBD": 48, "T8Yl": 41, "SX4-": 41, "S9xP": 41, "SKXy": 39, "redwood-city-ca-homes-for-sale": 33, "real-estate-market-forecast-save-thousands-on-your-mortgage-hack": 28, "WVkH": 28, "formly_email": 28, "bay-area-real-estate-market-update-2026-01": 27, "S9w-": 27, "SKXW": 24, "Meet-with-Graeham": 22, "S9xr": 22, "SM9K": 20, "Home-For-Sale": 20, "sell-with-graeham": 19, "east-menlo-park": 19, "5636-orchard-park-drive-san-jose-ca-95123": 17, "why-sell-with-graeham": 16, "discover-east-menlo-park-homes": 15, "Belle-haven-home-value-evaluation": 14, "SKaM": 13, "east-palo-alto-real-estate-sold-40k-more-with-50k-issue": 13, "homes-for-sale-redwood-city-ca": 13, "get_free_consultation": 13, "home-selling-tips-sell-your-home-quickly": 12, "homevalue": 11, "v4Sk": 10, "S9y0": 10, "before_after_1930_Sarah_Dr": 9, "SX9V": 9, "S9z0": 9, "schedule-call-with-graeham": 9, "woodside-plaza-redwood-city-homes-for-sale": 9, "east-palo-alto-condo-1982-w-bayshore-223": 9, "essential-home-selling-tips-5-Biggest-Mistakes": 8, "homes-for-sale-east-palo-alto-ca-discover-your-dream-home": 7, "952-6th-AVE-Redwood-City-CA-94063": 7, "828-Weeks-ST": 7, "homes-for-sale-in-the-bay-area-safety-tips": 7, "free-home-valuation": 7, "Home-Buying-Myths": 7, "redwood-city-real-estate-expert-advice-you-can-trust": 6, "east-palo-alto-real-estate-470-bell-st-tranquil-home-tour-silicon-valley": 6, "real-estate-market-forecast-fed-rate-cuts-2025-housing-market": 6, "bay-area-realtor-graeham-watts-winnie-danny-dream-home": 6, "real-estate-market-update-transformed-sold-for-top-dollar": 6, "east-palo-alto-condos-for-sale-woodland-creek-223": 5, "3-Buyer-Myths": 5, "selling-homes-quickly-tips-maximize-home-value": 5, "east-palo-alto-real-estate-insights-650k-fixer-upper-dream-home": 5, "Home-Evaluation": 5, "bay-area-realtor-dream-home-client-testimonial": 5, "2025-california-landlord-risk-update": 5, "Why-Clients-Trust-Graeham-Watts": 5, "homes-for-sale-in-menlo-park-graeham-watts-tour": 5, "redwood-city-real-estate-cost-of-living": 4, "Proven-Home-Selling-Tips": 4, "east-palo-alto-homes-for-sale-431-larkspur-dr-record-sale": 4, "east-palo-alto-market-update-prices-rising-homes-selling-fast": 4, "KII7": 4, "east-palo-alto-real-estate-470-bell-st-listing": 4, "avoid-costly-homebuyer-errors-redwood-city-real-estate-ca": 4, "SJpA": 4, "real-estate-market-update-313-smithwood-milpitas-ca-95035-home-tour": 4, "Offer-Accepted": 4, "top-east-palo-alto-realtor-sell-home-fast-top-dollar": 4, "top-east-palo-alto-realtor-home-selling-tips": 4, "discover-homes-for-sale-redwood-city-ca-spanish-style-4-bedroom-friendly-acres": 4, "san-jose-home-for-sale-500k-assumable-rate": 4, "How-We-Price-Your-Home": 4, "east-palo-alto-homes-for-sale-123-main-st-tour": 4, "redwood-city-real-estate-monthly-market-update-Graeham-watts": 4, "houses-in-east-palo-alto-modernized-1239-jervis-ave": 4, "East-Palo-Alto-Real-Estate-Monthly-Market-Update-graeham-watts": 4, "east-palo-alto-real-estate-missed-ca-fee-amnesty-act-now": 3, "east-palo-alto-realtor-sell-home-fast-top-dollar": 3, "bay-area-realtor-graeham-watts-testimonial-savings": 3, "1457-quail-st-los-banos-4-bedroom-home": 3, "affordable-east-palo-alto-homes-14-robin-court-tour": 3, "bay-area-realtor-free-home-staging-maximize-value": 3, "real-estate-predictions-2025-east-menlo-park": 3, "redwood-city-real-estate-graeham-watts-love-this-city": 3, "bay-area-realtor-california-rent-rules": 3, "bay-area-realtor-supply-and-demand-toilet-paper-bidding-war": 3, "home-selling-tips-maximize-sale-price-fast": 3, "big-head-big-ideas-big-results-real-estate-predictions-2025": 3, "bay-area-realtor-graeham-watts-johnny-ashley-success": 3, "ca-rent-laws-in-crises": 3, "east-palo-alto-real-estate-market-update-graeham-watts": 3, "essential-home-selling-tips-third-step-home-search": 3, "bay-area-realtor-graeham-watts-ryan-teeda-journey": 3, "1908-Cooley-AVE": 3, "east-menlo-park-market-update-prices-up-inventory-low": 3, "bair-island-homes-for-sale-in-redwood-city": 3, "east-menlo-park-real-estate-march-market-update": 3, "The-Graeham-Watts-Advantage": 3, "homes-for-sale-in-the-bay-area-buy-sell-with-graeham-watts": 3, "redwood-city-march-real-estate-market-update": 3, "discover-east-palo-alto-real-estate-stunning-home-under-900k": 3, "redwood-city-homes-for-sale-hidden-gems-affordable-living": 3, "redwood-city-real-estate-off-market-secrets": 3, "real-estate-market-forecast": 3, "east-palo-alto-market-update-are-you-keeping-up": 3, "east-palo-alto-ravenswood-school-district-transformation": 3, "east-palo-alto-ca-homes-for-sale-woodland-creek-condo-tour": 3, "redwood-city-real-estate-insider": 3, "client-testimonial-moneisha-jermell": 3, "bay-area-realtor-graeham-watts-hidden-gems": 3, "bay-area-realtor-graeham-watts-dream-home-success": 3, "east-palo-alto-ca-house-for-sale-worth-investment": 2, "real-estate-market-update-homes-selling-over-asking": 2, "homes-for-sale-in-east-palo-alto-ca-431-larkspur-dr": 2, "east-palo-alto-real-estate-why-now-is-the-time-to-buy": 2, "belle-haven-market-update": 2, "unleashing-potential-east-palo-alto-real-estate-transformative-fixer-upper": 2, "homes-for-sale-east-palo-alto-ca-117-mission-drive": 2, "bay-area-realtor-graeham-watts-record-home-sale": 2, "redwood-city-homes-for-sale-757-douglas-ave-tour": 2, "homes-for-sale-in-menlo-park-home-value": 2, "east-palo-alto-homes-for-sale-2620-fordham-st-update": 2, "friendly-acres-redwood-city": 2, "redwood-city-condos-for-sale": 2, "redwood-city-living-expense": 2, "east-menlo-park-market-update": 2, "max-profit-selling-homes-redwood-city": 2, "graeham-watts-advantage-redwood-city-real-estate": 2, "east-palo-alto-ca-homes-for-sale-2109-myrtle-pl-tour": 2, "redwood-shores-homes-for-sale-luxury-waterfront-tour": 2, "menlo-park-houses-for-sale-breathtaking-transformation": 2, "houses-for-sale-in-east-palo-alto-1404-camellia-drive": 2, "houses-in-east-palo-alto-952-newbridge-st-real-estate-gem": 2, "bay-area-realtor-graeham-client-testimonial-john-ward": 2, "home-selling-tips-attract-perfect-buyers": 2, "real-estate-market-update-menlo-park-2022-trends": 2, "bay-area-realtor-graeham-watts-trusted-expert": 2, "home-selling-tips-get-top-dollar-for-your-property": 2, "homes-for-sale-in-the-bay-area-los-gatos-home-sells-big": 2, "homes-for-sale-in-the-bay-area-luxury-the-westerly": 2, "redwood-city-real-estate-ca-safer": 2, "redwood-city-real-estate-next-big-investment": 2, "homes-for-sale-in-menlo-park-market-trends-graeham-watts": 2, "bay-area-realtor-home-alone-reaction": 2, "redwood-city-real-estate-january-market-update": 2, "east-menlo-park-market-update-january-2025": 2, "east-palo-alto-real-estate-homeowner-update": 2, "redwood-city-real-estate-charming-dream-homes": 2, "redwood-city-march-real-estate-market-update-2025": 2, "redwood-city-real-estate-market-home-value-insight": 2, "selling-homes-quickly-tips-kitchen-countertops": 2, "SK1C": 2, "SM9O": 2, "east-palo-alto-ca-homes-for-sale-1765-e-bayshore-rd-214": 2, "home-selling-tips-expert-staging-top-dollar": 2, "bay-area-realtor-essential-real-estate-tips": 2, "Home-Search": 2, "redwood-city-real-estate-prices": 2, "east-palo-alto-california-real-estate": 2, "rise-and-fall-of-silicon-valley-bank": 2, "bay-area-realtor-graeham-watts-off-market-dream-homes": 2, "just-sold-east-palo-alto-real-estate-1765-e-bayshore-rd-203": 2, "real-estate-market-update-stunning-home-oakley-59-escher-circle-tour-features": 2, "broadway-masala-redwood-city-indian-food": 2, "east-bay-real-estate-hidden-deals-maximize-sale": 2, "join-bay-area-realtor-graeham-watts-fiesta": 2, "mortgage-rates-forecast-2025-housing-loan-trends": 2, "shocking-bay-area-house-tour-twist": 2, "iBiB": 2, "bay-area-realtor-graeham-watts-humor-and-real-estate": 2, "bay-area-realtor-client-testimonial-trust": 2, "bay-area-realtor-daily-hustle-inside-look": 2, "graeham-watts-client-testimonials": 2, "Choosing-a-Bay-Area-Realtor": 2, "Understanding-Contingencies": 2, "nEnF": 2, "is-now-the-best-time-to-buy-real-estate": 2, "v4S8": 2, "sell-home-prep-tips": 2, "real-estate-market-update-you-need-to-know": 2, "Home-Selling-Tips": 2, "redwood-city-real-estate-ca-silicon-valley-oasis": 2, "graeham-watts-bay-area-realtor-real-estate-success": 2, "bay-area-realtor-graeham-watts-advantage": 2, "houses-for-sale-east-palo-alto-453-okeefe-st": 1, "east-menlo-park-market-update-home-prices-up": 1, "Real-Estate-Market-Forecast-Save-Thousands": 1, "real-estate-market-forecast-august-trends": 1, "bay-area-realtor-success-story-marketing-results": 1, "Why-Clients-Trust-Graeham": 1, "redwood-city-real-estate-charming-2bd-1ba-home-tour-expansive-yard": 1, "real-estate-market-update-top-investment-websites": 1, "east-palo-alto-market-update-free-home-valuation": 1, "east-palo-alto-homes-for-sale-2620-fordham": 1, "belle-haven-market-update-menlo-park": 1, "real-estate-market-update-sell-home-top-dollar": 1, "east-palo-alto-real-estate-market-update-2023": 1, "homes-for-sale-in-the-bay-area-just-listed-hot-properties": 1, "bay-area-realtor-graeham-watts-staging-and-3d-tours": 1, "rising-interest-rates-and-real-estate": 1, "redwood-city-real-estate-update-sell-high-inventory-rising": 1, "redwood-city-real-estate-update-2022-home-value": 1, "Home-buyer-guide": 1, "houses-in-east-palo-alto-dream-homes-for-every-budget": 1, "menlo-park-houses-for-sale-monthly-market-update": 1, "east-palo-alto-homes-for-sale-stylish-loft": 1, "ULo7": 1, "redwood-city-real-estate-market-update-for-homeowners": 1, "east-palo-alto-california-real-estate-trends": 1, "real-estate-market-trends-2025-home-warranty-explained": 1, "real-estate-inventory-trends-low-inventory-market-impact": 1, "east-palo-alto-real-estate-market-update-feb-2025": 1, "homes-for-sale-in-the-bay-area-san-leandro-modern-family-home": 1, "bay-area-realtor-graeham-watts-client-testimonial": 1, "homes-for-sale-in-farm-hill-7210-eagle-ridge-dr-gilroy-ca": 1, "KII1": 1, "essential-home-selling-tips-biggest-seller-mistakes": 1, "east-palo-alto-ca-homes-for-sale-buyer-tips-1765-e-bayshore-rd-204": 1, "menlo-park-real-estate-market-update-latest-trends-insights": 1, "home-selling-tips-fix-now-pay-later": 1, "homes-for-sale-in-redwood-city": 1, "real-estate-market-forecast-shocker-haunting-2025": 1, "home-selling-tips-out-of-state-overwhelmed-help": 1, "real-estate-update-home-selling-fast": 1, "rare-redwood-city-real-estate-fixer-2-bed-condo-hidden-potential": 1, "redwood-city-real-estate-insights": 1, "game-changing-real-estate-predictions-2025-big-deals-big-results": 1, "home-selling-mistakes-to-avoid": 1, "best-time-to-buy-real-estate-1930-pinole-drive-tour": 1, "ab-1482-unlocked-secrets-every-california-landlord-should-know": 1, "scariest-real-estate-costume-ever": 1, "redwood-city-real-estate-market-update-prices-up-inventory-down": 1, "home-alone-sequel-we-always-wanted-east-palo-alto-homes": 1, "unlock-hidden-redwood-city-real-estate-ca-gems": 1, "beat-the-market-east-palo-alto-market-update-secret-price-bump": 1, "Featured-Property-menlo-park": 1, "redwood-city-real-estate-renovation-alert": 1, "2842-Cornelius-Dr-Tour": 1, "Featured-Property-Redwood-City": 1, "Featured-Property-east-palo-alto": 1, "east-palo-alto-homes-for-sale-1765-bayshore-rd-203": 1, "homes-for-sale-in-east-palo-alto-ca-2398-palgas-ave": 1, "redwood-city-real-estate-2022-sell-high-find-home": 1, "redwood-city-real-estate-showdown": 1, "east-palo-alto-market-update-2022-inventory-and-prices": 1, "east-palo-alto-real-estate-gaillardia-way-property": 1, "real-estate-law-101-ca-landlord-entry-laws-explained": 1, "ab1482-explained-avoid-landlord-mistakes": 1, "client-testimonial-kevin-rebecca-bowe": 1, "east-palo-alto-market-update-2022-sell-for-maximum-value": 1, "real-estate-market-update-city-vs-county-closing-costs": 1, "redwood-city-real-estate-2025-market-shocker-revealed": 1, "discover-stunning-homes-for-sale-east-palo-alto-ca-graeham-watts-advantage": 1, "east-palo-alto-market-update-2022-home-value-increase": 1, "fha-loans-vs-conventional-loans": 1, "Home-Selling-Tips-Must-Know": 1, "price-your-home-right": 1, "thinking-of-selling-watch-this": 1, "home-worth-estimate": 1, "bay-area-realtor-why-experience-matters": 1, "san-mateo-real-estate-update": 1, "redwood-city-real-estate-market-update": 1, "east-palo-alto-real-estate-mark-dinan-vision-for-change-city-council-candidate": 1, "tech-growth-impact-redwood-city-real-estate": 1, "interest-rates-and-real-estate-market-impact": 1, "bay-area-realtor-graeham-watts-difference": 1, "east-palo-alto-houses-14-robin-ct-tour": 1, "unlock-redwood-city-real-estate": 1, "real-estate-market-update-22277-hartman-drive-property-tour": 1, "homes-for-sale-redwood-city-ca-staging-secrets": 1, "mt-carmel-redwood-city-homes-for-sale": 1, "redwood-city-real-estate-market-update-graeham-watts": 1, "redwood-city-real-estate-hidden-gems-revealed": 1, "mortgage-costs-redwood-city-real-estate-ca": 1, "discover-affordable-homes-under-100k-across-america": 1, "redwood-city-ca-real-estate-property-tax": 1, "ab-1482-explained-landlords-tenants-must-know": 1, "homes-for-sale-in-redwood-city-ca": 1, "redwood-city-real-estate-monthly-market-update": 1, "real-estate-market-forecast-foreclosure-myth-busted": 1, "roosevelt-redwood-city-homes-for-sale": 1, "homes-for-sale-east-palo-alto-ca-living-guide": 1, "redwood-city-real-estate-market-update-feb-2025": 1, "redwood-city-real-estate-financing": 1, "essential-home-selling-tips": 1, "essential-home-selling-tips-fourth-step-offer-accepted": 1, "homes-for-sale-in-redwood-city-ca-590-hurlingame-ave-makeover": 1, "homes-for-sale-in-the-bay-area-3500-19th-st-san-francisco": 1, "homes-for-sale-in-east-palo-alto-ca-2288-addison-ave": 1, "homes-for-sale-in-redwood-city-ca-winning-offers-underwriting-edge": 1, "homes-for-sale-in-the-bay-area-786-honeywood-court-pending-sale": 1, "homes-for-sale-in-the-bay-area-graeham-watts-2022-successes": 1, "real-estate-market-update-186-overlook-ave-hayward-ca-virtual-tour": 1, "bay-area-realtor-graeham-watts-client-love": 1, "ab-1482-explained-maximum-rent-increase-limits-in-california": 1, "new-rent-rules-real-estate-predictions-2025-tenants-landlords": 0, "redwood-city-real-estate-moving-guide-what-you-need-to-know": 0, "east-menlo-park-market-update-february-2025": 0, "east-palo-alto-real-estate-moving-guide": 0, "home-selling-tips-make-sales-breeze-no-stress": 0, "east-palo-alto-homes-for-sale": 0, "S9z9": 0, "east-palo-alto-dream-home-tour-30-seconds": 0, "redwood-city-real-estate-hidden-secrets": 0, "real-estate-market-trends-2025-alameda-county-hidden-gem": 0, "unbelievable-house-tour-market-forecast": 0, "palm-redwood-city-homes-for-sale": 0, "discover-east-palo-alto-real-estate-february-market-update": 0, "homes-for-sale-in-the-bay-area-soquel-3-bedroom": 0, "exciting-real-estate-market-update-jumping-housing-market": 0, "trusted-bay-area-realtor-house-safety-guarantee": 0, "redwood-city-real-estate-floor-installation-tips": 0, "unexpected-homes-for-sale-bay-area-dream-home": 0, "property-tax-shock-2025-real-estate-forecast": 0, "explore-east-palo-alto-homes-graeham-watts": 0, "101_Graden_St": 0, "ravenswood-school-district-50m-investment-better-schools-higher-teacher-pay": 0, "top-tips-for-selling-homes-quickly-maximize-home-value": 0, "rising-interest-rates-and-real-estate-prices": 0, "homes-for-sale-east-palo-alto": 0, "buy-a-home-east-palo-alto": 0, "redwood-village-homes-for-sale-in-redwood-city": 0, "east-palo-alto-market-update-january-2025-key-insights": 0, "real-estate-market-update-hartman-drive-los-altos-ca-virtual-tour": 0, "bay-area-realtor-secrets-unveiled": 0, "east-palo-alto-next-real-estate-goldmine": 0, "east-palo-alto-ca-homes-for-sale-house-tour": 0, "homes-for-sale-menlo-park-ca-modern-home-tour-1318-hollyburne-avenue": 0, "graeham-watts-east-palo-alto-listings": 0, "real-estate-market-trends-2025-phone-to-realty": 0, "discover-homes-for-sale-east-palo-alto-ca-1560-kavanaugh-drive-tour": 0, "home-selling-tips-busting-3-buyer-myths": 0, "the-buzz-in-redwood-city-real-estate": 0, "epa-comps-0601": 0, "ab1482-what-landlords-need-to-know-and-do": 0, "eagle-hill-redwood-city-homes-for-sale": 0, "east-menlo-park-homes-for-sale-graeham-watts": 0, "discover-stunning-homes-for-sale-menlo-park": 0, "east-menlo-park-homes-for-sale-listings-graeham-watts": 0, "expert-home-selling-tips": 0, "bay-area-realtor-free-staging-and-repairs": 0, "bay-area-home-search-map": 0, "Ace_Your_First_Home_Purchase": 0, "homes-for-sale-menlo-park-ca-update": 0, "downtown-redwood-city-homes-for-sale": 0, "maximize-your-home-sale-value": 0, "east-menlo-park-homes-for-sale": 0, "redwood-city-real-estate-paradise": 0, "stambaugh-heller-redwood-city-homes-for-sale": 0, "Redwood-Oaks-homes-for-sale-in-redwood-city-ca": 0, "real-estate-market-update-2025-california-landlord-risks": 0, "east-palo-alto-real-estate-housing-market-update": 0, "essential-real-estate-market-update-homebuyers-guide": 0, "redwood-city-real-estate-housing-market-update": 0, "central-redwood-city-homes-for-sale": 0, "east-palo-alto-homes-for-sale-woodland-creek-condo": 0, "discovering-redwood-city-real-estate": 0, "redwood-city-real-estate-crucial-market-update": 0, "menlo-park-market-update-belle-haven-housing-trends": 0, "real-estate-market-forecast-fed-interest-rates-explained": 0, "real-estate-market-update-just-listed-1186-overlook-ave-hayward-ca-house-tour": 0, "centennial-redwood-city-homes-for-sale": 0, "east-palo-alto-real-estate-monthly-market-update": 0, "kIhN": 0, "real-estate-market-trends-2025": 0, "best-indian-restaurant-redwood-city-broadway-masala": 0, "Redwood-City-Homes-on-Sale-Hot-Listings-Price-Drops": 0, "ab-1482-rent-increases-exemptions-and-california-rental-laws-explained": 0, "Redwood-City-Homes-on-Sale-Hot-Listings-Price-Drops-Graeham-Watts": 0, "Redwood-City-Homes-on-Sale": 0, "2115-Clarke-Ave": 0, "Redwood-City-Homes-on-Sale-Hot-Listings": 0, "redwood-city-real-estate-san-francisco-monthly-market-update": 0, "essential-real-estate-market-update-guide-for-homebuyers": 0, "essential-home-selling-tips-second-step-pre-qualification": 0, "choose-the-right-agent": 0, "ab-1482-explained-is-your-property-rent-controlled-or-exempt": 0, "stunning-homes-for-sale-menlo-park-1135-madera-ave-tour": 0, "redwood-city-spanish-style-4-bedroom-friendly-acres": 0, "east-palo-alto-real-estate-market-update": 0, "real-estate-market-update-foundation-tips-when-buying-your-new-home": 0, "real-estate-market-update-tips-to-protect-your-offer-and-investment": 0, "east-palo-alto-real-estate-mark-dinan-vision-change": 0, "what-you-can-buy-redwood-city-houses-for-sale": 0, "redwood-city-real-estate-market-february-update-you-need-to-know": 0, "selling-your-home-just-got-easier": 0}} \ No newline at end of file diff --git a/online-content/dashboards/switchy/snapshots/switchy-snapshot-2026-06-01.json b/online-content/dashboards/switchy/snapshots/switchy-snapshot-2026-06-01.json new file mode 100644 index 00000000..ec3e288f --- /dev/null +++ b/online-content/dashboards/switchy/snapshots/switchy-snapshot-2026-06-01.json @@ -0,0 +1 @@ +{"date": "2026-06-01", "clicks": {"bay-area-homes-for-sale": 1069, "SbSR": 1023, "graehamwatts-meet": 779, "whats-my-house-worth": 722, "SKZY": 681, "SKaA": 602, "SK0r": 586, "XhI5": 570, "SK1P": 547, "SKai": 515, "SKY8": 509, "redwood-city-homes-for-sale": 497, "SKYl": 460, "east-palo-alto-ca-homes-for-sale": 430, "homes-for-sale-belle-haven": 397, "T1td": 396, "SJob": 376, "SJ-k": 296, "MFUy": 282, "SXDB": 234, "SM9A": 205, "SKZH": 190, "SRlP": 185, "SKDr": 180, "SX1b": 170, "SJoH": 163, "SJpt": 141, "SKYx": 141, "east-palo-alto-CA-homes-for-sale-under-1million": 136, "SJ-D": 126, "SX7L": 124, "SK0T": 123, "S-VQ": 116, "T1nG": 111, "SXGz": 108, "SKYW": 105, "East-palo-alto-home-for-sale": 96, "menlo-park-homes-for-sale": 94, "how-much-is-my-home-worth": 93, "T1r5": 90, "S-Vz": 88, "redwood-city-home-for-sale": 80, "T8PU": 78, "TA7m": 71, "SX5l": 70, "SXG2": 70, "S9yj": 67, "XhH8": 61, "SKYI": 60, "schedule-bay-area-real-estate-meeting": 60, "blog-east-palo-alto-ca-homes-for-sale": 59, "free-home-evaluation": 59, "S-Vg": 59, "S9yo": 54, "T1v5": 51, "SM92": 50, "SXBD": 48, "T8Yl": 41, "SX4-": 41, "S9xP": 41, "SKXy": 39, "redwood-city-ca-homes-for-sale": 33, "real-estate-market-forecast-save-thousands-on-your-mortgage-hack": 28, "WVkH": 28, "formly_email": 28, "bay-area-real-estate-market-update-2026-01": 27, "S9w-": 27, "SKXW": 24, "Meet-with-Graeham": 22, "S9xr": 22, "SM9K": 20, "Home-For-Sale": 20, "sell-with-graeham": 19, "east-menlo-park": 19, "5636-orchard-park-drive-san-jose-ca-95123": 17, "why-sell-with-graeham": 16, "discover-east-menlo-park-homes": 15, "Belle-haven-home-value-evaluation": 14, "SKaM": 13, "east-palo-alto-real-estate-sold-40k-more-with-50k-issue": 13, "homes-for-sale-redwood-city-ca": 13, "get_free_consultation": 13, "home-selling-tips-sell-your-home-quickly": 12, "homevalue": 11, "v4Sk": 10, "S9y0": 10, "before_after_1930_Sarah_Dr": 9, "SX9V": 9, "S9z0": 9, "schedule-call-with-graeham": 9, "woodside-plaza-redwood-city-homes-for-sale": 9, "east-palo-alto-condo-1982-w-bayshore-223": 9, "essential-home-selling-tips-5-Biggest-Mistakes": 8, "homes-for-sale-east-palo-alto-ca-discover-your-dream-home": 7, "952-6th-AVE-Redwood-City-CA-94063": 7, "828-Weeks-ST": 7, "homes-for-sale-in-the-bay-area-safety-tips": 7, "free-home-valuation": 7, "Home-Buying-Myths": 7, "redwood-city-real-estate-expert-advice-you-can-trust": 6, "east-palo-alto-real-estate-470-bell-st-tranquil-home-tour-silicon-valley": 6, "real-estate-market-forecast-fed-rate-cuts-2025-housing-market": 6, "bay-area-realtor-graeham-watts-winnie-danny-dream-home": 6, "real-estate-market-update-transformed-sold-for-top-dollar": 6, "east-palo-alto-condos-for-sale-woodland-creek-223": 5, "3-Buyer-Myths": 5, "selling-homes-quickly-tips-maximize-home-value": 5, "east-palo-alto-real-estate-insights-650k-fixer-upper-dream-home": 5, "Home-Evaluation": 5, "bay-area-realtor-dream-home-client-testimonial": 5, "2025-california-landlord-risk-update": 5, "Why-Clients-Trust-Graeham-Watts": 5, "homes-for-sale-in-menlo-park-graeham-watts-tour": 5, "redwood-city-real-estate-cost-of-living": 4, "Proven-Home-Selling-Tips": 4, "east-palo-alto-homes-for-sale-431-larkspur-dr-record-sale": 4, "east-palo-alto-market-update-prices-rising-homes-selling-fast": 4, "KII7": 4, "east-palo-alto-real-estate-470-bell-st-listing": 4, "avoid-costly-homebuyer-errors-redwood-city-real-estate-ca": 4, "SJpA": 4, "real-estate-market-update-313-smithwood-milpitas-ca-95035-home-tour": 4, "Offer-Accepted": 4, "top-east-palo-alto-realtor-sell-home-fast-top-dollar": 4, "top-east-palo-alto-realtor-home-selling-tips": 4, "discover-homes-for-sale-redwood-city-ca-spanish-style-4-bedroom-friendly-acres": 4, "san-jose-home-for-sale-500k-assumable-rate": 4, "How-We-Price-Your-Home": 4, "east-palo-alto-homes-for-sale-123-main-st-tour": 4, "redwood-city-real-estate-monthly-market-update-Graeham-watts": 4, "houses-in-east-palo-alto-modernized-1239-jervis-ave": 4, "East-Palo-Alto-Real-Estate-Monthly-Market-Update-graeham-watts": 4, "east-palo-alto-real-estate-missed-ca-fee-amnesty-act-now": 3, "east-palo-alto-realtor-sell-home-fast-top-dollar": 3, "bay-area-realtor-graeham-watts-testimonial-savings": 3, "1457-quail-st-los-banos-4-bedroom-home": 3, "affordable-east-palo-alto-homes-14-robin-court-tour": 3, "bay-area-realtor-free-home-staging-maximize-value": 3, "real-estate-predictions-2025-east-menlo-park": 3, "redwood-city-real-estate-graeham-watts-love-this-city": 3, "bay-area-realtor-california-rent-rules": 3, "bay-area-realtor-supply-and-demand-toilet-paper-bidding-war": 3, "home-selling-tips-maximize-sale-price-fast": 3, "big-head-big-ideas-big-results-real-estate-predictions-2025": 3, "bay-area-realtor-graeham-watts-johnny-ashley-success": 3, "ca-rent-laws-in-crises": 3, "east-palo-alto-real-estate-market-update-graeham-watts": 3, "essential-home-selling-tips-third-step-home-search": 3, "bay-area-realtor-graeham-watts-ryan-teeda-journey": 3, "1908-Cooley-AVE": 3, "east-menlo-park-market-update-prices-up-inventory-low": 3, "bair-island-homes-for-sale-in-redwood-city": 3, "east-menlo-park-real-estate-march-market-update": 3, "The-Graeham-Watts-Advantage": 3, "homes-for-sale-in-the-bay-area-buy-sell-with-graeham-watts": 3, "redwood-city-march-real-estate-market-update": 3, "discover-east-palo-alto-real-estate-stunning-home-under-900k": 3, "redwood-city-homes-for-sale-hidden-gems-affordable-living": 3, "redwood-city-real-estate-off-market-secrets": 3, "real-estate-market-forecast": 3, "east-palo-alto-market-update-are-you-keeping-up": 3, "east-palo-alto-ravenswood-school-district-transformation": 3, "east-palo-alto-ca-homes-for-sale-woodland-creek-condo-tour": 3, "redwood-city-real-estate-insider": 3, "client-testimonial-moneisha-jermell": 3, "bay-area-realtor-graeham-watts-hidden-gems": 3, "bay-area-realtor-graeham-watts-dream-home-success": 3, "east-palo-alto-ca-house-for-sale-worth-investment": 2, "real-estate-market-update-homes-selling-over-asking": 2, "homes-for-sale-in-east-palo-alto-ca-431-larkspur-dr": 2, "east-palo-alto-real-estate-why-now-is-the-time-to-buy": 2, "belle-haven-market-update": 2, "unleashing-potential-east-palo-alto-real-estate-transformative-fixer-upper": 2, "homes-for-sale-east-palo-alto-ca-117-mission-drive": 2, "bay-area-realtor-graeham-watts-record-home-sale": 2, "redwood-city-homes-for-sale-757-douglas-ave-tour": 2, "homes-for-sale-in-menlo-park-home-value": 2, "east-palo-alto-homes-for-sale-2620-fordham-st-update": 2, "friendly-acres-redwood-city": 2, "redwood-city-condos-for-sale": 2, "redwood-city-living-expense": 2, "east-menlo-park-market-update": 2, "max-profit-selling-homes-redwood-city": 2, "graeham-watts-advantage-redwood-city-real-estate": 2, "east-palo-alto-ca-homes-for-sale-2109-myrtle-pl-tour": 2, "redwood-shores-homes-for-sale-luxury-waterfront-tour": 2, "menlo-park-houses-for-sale-breathtaking-transformation": 2, "houses-for-sale-in-east-palo-alto-1404-camellia-drive": 2, "houses-in-east-palo-alto-952-newbridge-st-real-estate-gem": 2, "bay-area-realtor-graeham-client-testimonial-john-ward": 2, "home-selling-tips-attract-perfect-buyers": 2, "real-estate-market-update-menlo-park-2022-trends": 2, "bay-area-realtor-graeham-watts-trusted-expert": 2, "home-selling-tips-get-top-dollar-for-your-property": 2, "homes-for-sale-in-the-bay-area-los-gatos-home-sells-big": 2, "homes-for-sale-in-the-bay-area-luxury-the-westerly": 2, "redwood-city-real-estate-ca-safer": 2, "redwood-city-real-estate-next-big-investment": 2, "homes-for-sale-in-menlo-park-market-trends-graeham-watts": 2, "bay-area-realtor-home-alone-reaction": 2, "redwood-city-real-estate-january-market-update": 2, "east-menlo-park-market-update-january-2025": 2, "east-palo-alto-real-estate-homeowner-update": 2, "redwood-city-real-estate-charming-dream-homes": 2, "redwood-city-march-real-estate-market-update-2025": 2, "redwood-city-real-estate-market-home-value-insight": 2, "selling-homes-quickly-tips-kitchen-countertops": 2, "SK1C": 2, "SM9O": 2, "east-palo-alto-ca-homes-for-sale-1765-e-bayshore-rd-214": 2, "home-selling-tips-expert-staging-top-dollar": 2, "bay-area-realtor-essential-real-estate-tips": 2, "Home-Search": 2, "redwood-city-real-estate-prices": 2, "east-palo-alto-california-real-estate": 2, "rise-and-fall-of-silicon-valley-bank": 2, "bay-area-realtor-graeham-watts-off-market-dream-homes": 2, "just-sold-east-palo-alto-real-estate-1765-e-bayshore-rd-203": 2, "real-estate-market-update-stunning-home-oakley-59-escher-circle-tour-features": 2, "broadway-masala-redwood-city-indian-food": 2, "east-bay-real-estate-hidden-deals-maximize-sale": 2, "join-bay-area-realtor-graeham-watts-fiesta": 2, "mortgage-rates-forecast-2025-housing-loan-trends": 2, "shocking-bay-area-house-tour-twist": 2, "iBiB": 2, "bay-area-realtor-graeham-watts-humor-and-real-estate": 2, "bay-area-realtor-client-testimonial-trust": 2, "bay-area-realtor-daily-hustle-inside-look": 2, "graeham-watts-client-testimonials": 2, "Choosing-a-Bay-Area-Realtor": 2, "Understanding-Contingencies": 2, "nEnF": 2, "is-now-the-best-time-to-buy-real-estate": 2, "v4S8": 2, "sell-home-prep-tips": 2, "real-estate-market-update-you-need-to-know": 2, "Home-Selling-Tips": 2, "redwood-city-real-estate-ca-silicon-valley-oasis": 2, "graeham-watts-bay-area-realtor-real-estate-success": 2, "bay-area-realtor-graeham-watts-advantage": 2, "houses-for-sale-east-palo-alto-453-okeefe-st": 1, "east-menlo-park-market-update-home-prices-up": 1, "Real-Estate-Market-Forecast-Save-Thousands": 1, "real-estate-market-forecast-august-trends": 1, "bay-area-realtor-success-story-marketing-results": 1, "Why-Clients-Trust-Graeham": 1, "redwood-city-real-estate-charming-2bd-1ba-home-tour-expansive-yard": 1, "real-estate-market-update-top-investment-websites": 1, "east-palo-alto-market-update-free-home-valuation": 1, "east-palo-alto-homes-for-sale-2620-fordham": 1, "belle-haven-market-update-menlo-park": 1, "real-estate-market-update-sell-home-top-dollar": 1, "east-palo-alto-real-estate-market-update-2023": 1, "homes-for-sale-in-the-bay-area-just-listed-hot-properties": 1, "bay-area-realtor-graeham-watts-staging-and-3d-tours": 1, "rising-interest-rates-and-real-estate": 1, "redwood-city-real-estate-update-sell-high-inventory-rising": 1, "redwood-city-real-estate-update-2022-home-value": 1, "Home-buyer-guide": 1, "houses-in-east-palo-alto-dream-homes-for-every-budget": 1, "menlo-park-houses-for-sale-monthly-market-update": 1, "east-palo-alto-homes-for-sale-stylish-loft": 1, "ULo7": 1, "redwood-city-real-estate-market-update-for-homeowners": 1, "east-palo-alto-california-real-estate-trends": 1, "real-estate-market-trends-2025-home-warranty-explained": 1, "real-estate-inventory-trends-low-inventory-market-impact": 1, "east-palo-alto-real-estate-market-update-feb-2025": 1, "homes-for-sale-in-the-bay-area-san-leandro-modern-family-home": 1, "bay-area-realtor-graeham-watts-client-testimonial": 1, "homes-for-sale-in-farm-hill-7210-eagle-ridge-dr-gilroy-ca": 1, "KII1": 1, "essential-home-selling-tips-biggest-seller-mistakes": 1, "east-palo-alto-ca-homes-for-sale-buyer-tips-1765-e-bayshore-rd-204": 1, "menlo-park-real-estate-market-update-latest-trends-insights": 1, "home-selling-tips-fix-now-pay-later": 1, "homes-for-sale-in-redwood-city": 1, "real-estate-market-forecast-shocker-haunting-2025": 1, "home-selling-tips-out-of-state-overwhelmed-help": 1, "real-estate-update-home-selling-fast": 1, "rare-redwood-city-real-estate-fixer-2-bed-condo-hidden-potential": 1, "redwood-city-real-estate-insights": 1, "game-changing-real-estate-predictions-2025-big-deals-big-results": 1, "home-selling-mistakes-to-avoid": 1, "best-time-to-buy-real-estate-1930-pinole-drive-tour": 1, "ab-1482-unlocked-secrets-every-california-landlord-should-know": 1, "scariest-real-estate-costume-ever": 1, "redwood-city-real-estate-market-update-prices-up-inventory-down": 1, "home-alone-sequel-we-always-wanted-east-palo-alto-homes": 1, "unlock-hidden-redwood-city-real-estate-ca-gems": 1, "beat-the-market-east-palo-alto-market-update-secret-price-bump": 1, "Featured-Property-menlo-park": 1, "redwood-city-real-estate-renovation-alert": 1, "2842-Cornelius-Dr-Tour": 1, "Featured-Property-Redwood-City": 1, "Featured-Property-east-palo-alto": 1, "east-palo-alto-homes-for-sale-1765-bayshore-rd-203": 1, "homes-for-sale-in-east-palo-alto-ca-2398-palgas-ave": 1, "redwood-city-real-estate-2022-sell-high-find-home": 1, "redwood-city-real-estate-showdown": 1, "east-palo-alto-market-update-2022-inventory-and-prices": 1, "east-palo-alto-real-estate-gaillardia-way-property": 1, "real-estate-law-101-ca-landlord-entry-laws-explained": 1, "ab1482-explained-avoid-landlord-mistakes": 1, "client-testimonial-kevin-rebecca-bowe": 1, "east-palo-alto-market-update-2022-sell-for-maximum-value": 1, "real-estate-market-update-city-vs-county-closing-costs": 1, "redwood-city-real-estate-2025-market-shocker-revealed": 1, "discover-stunning-homes-for-sale-east-palo-alto-ca-graeham-watts-advantage": 1, "east-palo-alto-market-update-2022-home-value-increase": 1, "fha-loans-vs-conventional-loans": 1, "Home-Selling-Tips-Must-Know": 1, "price-your-home-right": 1, "thinking-of-selling-watch-this": 1, "home-worth-estimate": 1, "bay-area-realtor-why-experience-matters": 1, "san-mateo-real-estate-update": 1, "redwood-city-real-estate-market-update": 1, "east-palo-alto-real-estate-mark-dinan-vision-for-change-city-council-candidate": 1, "tech-growth-impact-redwood-city-real-estate": 1, "interest-rates-and-real-estate-market-impact": 1, "bay-area-realtor-graeham-watts-difference": 1, "east-palo-alto-houses-14-robin-ct-tour": 1, "unlock-redwood-city-real-estate": 1, "real-estate-market-update-22277-hartman-drive-property-tour": 1, "homes-for-sale-redwood-city-ca-staging-secrets": 1, "mt-carmel-redwood-city-homes-for-sale": 1, "redwood-city-real-estate-market-update-graeham-watts": 1, "redwood-city-real-estate-hidden-gems-revealed": 1, "mortgage-costs-redwood-city-real-estate-ca": 1, "discover-affordable-homes-under-100k-across-america": 1, "redwood-city-ca-real-estate-property-tax": 1, "ab-1482-explained-landlords-tenants-must-know": 1, "homes-for-sale-in-redwood-city-ca": 1, "redwood-city-real-estate-monthly-market-update": 1, "real-estate-market-forecast-foreclosure-myth-busted": 1, "roosevelt-redwood-city-homes-for-sale": 1, "homes-for-sale-east-palo-alto-ca-living-guide": 1, "redwood-city-real-estate-market-update-feb-2025": 1, "redwood-city-real-estate-financing": 1, "essential-home-selling-tips": 1, "essential-home-selling-tips-fourth-step-offer-accepted": 1, "homes-for-sale-in-redwood-city-ca-590-hurlingame-ave-makeover": 1, "homes-for-sale-in-the-bay-area-3500-19th-st-san-francisco": 1, "homes-for-sale-in-east-palo-alto-ca-2288-addison-ave": 1, "homes-for-sale-in-redwood-city-ca-winning-offers-underwriting-edge": 1, "homes-for-sale-in-the-bay-area-786-honeywood-court-pending-sale": 1, "homes-for-sale-in-the-bay-area-graeham-watts-2022-successes": 1, "real-estate-market-update-186-overlook-ave-hayward-ca-virtual-tour": 1, "bay-area-realtor-graeham-watts-client-love": 1, "ab-1482-explained-maximum-rent-increase-limits-in-california": 1, "new-rent-rules-real-estate-predictions-2025-tenants-landlords": 0, "redwood-city-real-estate-moving-guide-what-you-need-to-know": 0, "east-menlo-park-market-update-february-2025": 0, "east-palo-alto-real-estate-moving-guide": 0, "home-selling-tips-make-sales-breeze-no-stress": 0, "epa-comps-0601": 0, "S9z9": 0, "east-palo-alto-dream-home-tour-30-seconds": 0, "redwood-city-real-estate-hidden-secrets": 0, "real-estate-market-trends-2025-alameda-county-hidden-gem": 0, "unbelievable-house-tour-market-forecast": 0, "palm-redwood-city-homes-for-sale": 0, "discover-east-palo-alto-real-estate-february-market-update": 0, "homes-for-sale-in-the-bay-area-soquel-3-bedroom": 0, "exciting-real-estate-market-update-jumping-housing-market": 0, "trusted-bay-area-realtor-house-safety-guarantee": 0, "redwood-city-real-estate-floor-installation-tips": 0, "unexpected-homes-for-sale-bay-area-dream-home": 0, "property-tax-shock-2025-real-estate-forecast": 0, "explore-east-palo-alto-homes-graeham-watts": 0, "101_Graden_St": 0, "ravenswood-school-district-50m-investment-better-schools-higher-teacher-pay": 0, "top-tips-for-selling-homes-quickly-maximize-home-value": 0, "rising-interest-rates-and-real-estate-prices": 0, "homes-for-sale-east-palo-alto": 0, "buy-a-home-east-palo-alto": 0, "redwood-village-homes-for-sale-in-redwood-city": 0, "east-palo-alto-market-update-january-2025-key-insights": 0, "real-estate-market-update-hartman-drive-los-altos-ca-virtual-tour": 0, "bay-area-realtor-secrets-unveiled": 0, "east-palo-alto-next-real-estate-goldmine": 0, "east-palo-alto-ca-homes-for-sale-house-tour": 0, "homes-for-sale-menlo-park-ca-modern-home-tour-1318-hollyburne-avenue": 0, "graeham-watts-east-palo-alto-listings": 0, "real-estate-market-trends-2025-phone-to-realty": 0, "discover-homes-for-sale-east-palo-alto-ca-1560-kavanaugh-drive-tour": 0, "home-selling-tips-busting-3-buyer-myths": 0, "the-buzz-in-redwood-city-real-estate": 0, "east-palo-alto-homes-for-sale": 0, "ab1482-what-landlords-need-to-know-and-do": 0, "eagle-hill-redwood-city-homes-for-sale": 0, "east-menlo-park-homes-for-sale-graeham-watts": 0, "discover-stunning-homes-for-sale-menlo-park": 0, "east-menlo-park-homes-for-sale-listings-graeham-watts": 0, "expert-home-selling-tips": 0, "bay-area-realtor-free-staging-and-repairs": 0, "bay-area-home-search-map": 0, "Ace_Your_First_Home_Purchase": 0, "homes-for-sale-menlo-park-ca-update": 0, "downtown-redwood-city-homes-for-sale": 0, "maximize-your-home-sale-value": 0, "east-menlo-park-homes-for-sale": 0, "redwood-city-real-estate-paradise": 0, "stambaugh-heller-redwood-city-homes-for-sale": 0, "Redwood-Oaks-homes-for-sale-in-redwood-city-ca": 0, "real-estate-market-update-2025-california-landlord-risks": 0, "east-palo-alto-real-estate-housing-market-update": 0, "essential-real-estate-market-update-homebuyers-guide": 0, "redwood-city-real-estate-housing-market-update": 0, "central-redwood-city-homes-for-sale": 0, "east-palo-alto-homes-for-sale-woodland-creek-condo": 0, "discovering-redwood-city-real-estate": 0, "redwood-city-real-estate-crucial-market-update": 0, "menlo-park-market-update-belle-haven-housing-trends": 0, "real-estate-market-forecast-fed-interest-rates-explained": 0, "real-estate-market-update-just-listed-1186-overlook-ave-hayward-ca-house-tour": 0, "centennial-redwood-city-homes-for-sale": 0, "east-palo-alto-real-estate-monthly-market-update": 0, "kIhN": 0, "real-estate-market-trends-2025": 0, "best-indian-restaurant-redwood-city-broadway-masala": 0, "Redwood-City-Homes-on-Sale-Hot-Listings-Price-Drops": 0, "ab-1482-rent-increases-exemptions-and-california-rental-laws-explained": 0, "Redwood-City-Homes-on-Sale-Hot-Listings-Price-Drops-Graeham-Watts": 0, "Redwood-City-Homes-on-Sale": 0, "2115-Clarke-Ave": 0, "Redwood-City-Homes-on-Sale-Hot-Listings": 0, "redwood-city-real-estate-san-francisco-monthly-market-update": 0, "essential-real-estate-market-update-guide-for-homebuyers": 0, "essential-home-selling-tips-second-step-pre-qualification": 0, "choose-the-right-agent": 0, "ab-1482-explained-is-your-property-rent-controlled-or-exempt": 0, "stunning-homes-for-sale-menlo-park-1135-madera-ave-tour": 0, "redwood-city-spanish-style-4-bedroom-friendly-acres": 0, "east-palo-alto-real-estate-market-update": 0, "real-estate-market-update-foundation-tips-when-buying-your-new-home": 0, "real-estate-market-update-tips-to-protect-your-offer-and-investment": 0, "east-palo-alto-real-estate-mark-dinan-vision-change": 0, "what-you-can-buy-redwood-city-houses-for-sale": 0, "redwood-city-real-estate-market-february-update-you-need-to-know": 0, "selling-your-home-just-got-easier": 0}} \ No newline at end of file diff --git a/online-content/dashboards/switchy/snapshots/switchy-snapshot-2026-06-08.json b/online-content/dashboards/switchy/snapshots/switchy-snapshot-2026-06-08.json new file mode 100644 index 00000000..0c016ef9 --- /dev/null +++ b/online-content/dashboards/switchy/snapshots/switchy-snapshot-2026-06-08.json @@ -0,0 +1 @@ +{"date": "2026-06-08", "clicks": {"bay-area-homes-for-sale": 1073, "SbSR": 1023, "graehamwatts-meet": 780, "whats-my-house-worth": 724, "SKZY": 681, "SKaA": 602, "SK0r": 586, "XhI5": 570, "SK1P": 547, "SKai": 515, "SKY8": 509, "redwood-city-homes-for-sale": 498, "SKYl": 460, "east-palo-alto-ca-homes-for-sale": 432, "homes-for-sale-belle-haven": 398, "T1td": 396, "SJob": 376, "SJ-k": 296, "MFUy": 293, "SXDB": 234, "SM9A": 205, "SKZH": 190, "SRlP": 185, "SKDr": 180, "SX1b": 170, "SJoH": 163, "SJpt": 141, "SKYx": 141, "east-palo-alto-CA-homes-for-sale-under-1million": 136, "SJ-D": 126, "SX7L": 124, "SK0T": 123, "S-VQ": 116, "T1nG": 111, "SXGz": 108, "SKYW": 105, "East-palo-alto-home-for-sale": 96, "menlo-park-homes-for-sale": 94, "how-much-is-my-home-worth": 93, "T1r5": 90, "S-Vz": 88, "redwood-city-home-for-sale": 80, "T8PU": 78, "TA7m": 71, "SXG2": 70, "SX5l": 70, "S9yj": 67, "XhH8": 61, "SKYI": 60, "blog-east-palo-alto-ca-homes-for-sale": 60, "schedule-bay-area-real-estate-meeting": 60, "free-home-evaluation": 59, "S-Vg": 59, "S9yo": 54, "T1v5": 51, "SM92": 50, "SXBD": 48, "S9xP": 41, "SX4-": 41, "T8Yl": 41, "SKXy": 39, "redwood-city-ca-homes-for-sale": 33, "real-estate-market-forecast-save-thousands-on-your-mortgage-hack": 28, "WVkH": 28, "formly_email": 28, "bay-area-real-estate-market-update-2026-01": 27, "S9w-": 27, "SKXW": 24, "S9xr": 22, "Meet-with-Graeham": 22, "SM9K": 20, "Home-For-Sale": 20, "sell-with-graeham": 19, "east-menlo-park": 19, "5636-orchard-park-drive-san-jose-ca-95123": 17, "why-sell-with-graeham": 16, "homevalue": 15, "discover-east-menlo-park-homes": 15, "Belle-haven-home-value-evaluation": 14, "SKaM": 13, "homes-for-sale-redwood-city-ca": 13, "east-palo-alto-real-estate-sold-40k-more-with-50k-issue": 13, "get_free_consultation": 13, "home-selling-tips-sell-your-home-quickly": 12, "S9y0": 10, "v4Sk": 10, "before_after_1930_Sarah_Dr": 9, "SX9V": 9, "S9z0": 9, "schedule-call-with-graeham": 9, "woodside-plaza-redwood-city-homes-for-sale": 9, "east-palo-alto-condo-1982-w-bayshore-223": 9, "essential-home-selling-tips-5-Biggest-Mistakes": 8, "Home-Buying-Myths": 7, "828-Weeks-ST": 7, "homes-for-sale-east-palo-alto-ca-discover-your-dream-home": 7, "952-6th-AVE-Redwood-City-CA-94063": 7, "free-home-valuation": 7, "homes-for-sale-in-the-bay-area-safety-tips": 7, "bay-area-realtor-graeham-watts-winnie-danny-dream-home": 6, "east-palo-alto-real-estate-470-bell-st-tranquil-home-tour-silicon-valley": 6, "real-estate-market-forecast-fed-rate-cuts-2025-housing-market": 6, "real-estate-market-update-transformed-sold-for-top-dollar": 6, "redwood-city-real-estate-expert-advice-you-can-trust": 6, "homes-for-sale-in-menlo-park-graeham-watts-tour": 5, "3-Buyer-Myths": 5, "2025-california-landlord-risk-update": 5, "east-palo-alto-real-estate-insights-650k-fixer-upper-dream-home": 5, "Home-Evaluation": 5, "east-palo-alto-condos-for-sale-woodland-creek-223": 5, "Why-Clients-Trust-Graeham-Watts": 5, "selling-homes-quickly-tips-maximize-home-value": 5, "bay-area-realtor-dream-home-client-testimonial": 5, "SJpA": 4, "east-palo-alto-real-estate-470-bell-st-listing": 4, "san-jose-home-for-sale-500k-assumable-rate": 4, "redwood-city-real-estate-monthly-market-update-Graeham-watts": 4, "avoid-costly-homebuyer-errors-redwood-city-real-estate-ca": 4, "Proven-Home-Selling-Tips": 4, "houses-in-east-palo-alto-modernized-1239-jervis-ave": 4, "discover-homes-for-sale-redwood-city-ca-spanish-style-4-bedroom-friendly-acres": 4, "KII7": 4, "east-palo-alto-market-update-prices-rising-homes-selling-fast": 4, "East-Palo-Alto-Real-Estate-Monthly-Market-Update-graeham-watts": 4, "top-east-palo-alto-realtor-home-selling-tips": 4, "Offer-Accepted": 4, "east-palo-alto-homes-for-sale-431-larkspur-dr-record-sale": 4, "top-east-palo-alto-realtor-sell-home-fast-top-dollar": 4, "real-estate-market-update-313-smithwood-milpitas-ca-95035-home-tour": 4, "redwood-city-real-estate-cost-of-living": 4, "east-palo-alto-homes-for-sale-123-main-st-tour": 4, "How-We-Price-Your-Home": 4, "bair-island-homes-for-sale-in-redwood-city": 3, "bay-area-realtor-free-home-staging-maximize-value": 3, "homes-for-sale-in-the-bay-area-buy-sell-with-graeham-watts": 3, "The-Graeham-Watts-Advantage": 3, "east-menlo-park-market-update-prices-up-inventory-low": 3, "bay-area-realtor-graeham-watts-ryan-teeda-journey": 3, "bay-area-realtor-graeham-watts-johnny-ashley-success": 3, "home-selling-tips-maximize-sale-price-fast": 3, "bay-area-realtor-supply-and-demand-toilet-paper-bidding-war": 3, "real-estate-predictions-2025-east-menlo-park": 3, "bay-area-realtor-graeham-watts-testimonial-savings": 3, "east-palo-alto-realtor-sell-home-fast-top-dollar": 3, "east-palo-alto-ca-homes-for-sale-woodland-creek-condo-tour": 3, "east-palo-alto-market-update-are-you-keeping-up": 3, "redwood-city-homes-for-sale-hidden-gems-affordable-living": 3, "redwood-city-real-estate-off-market-secrets": 3, "real-estate-market-forecast": 3, "east-palo-alto-ravenswood-school-district-transformation": 3, "redwood-city-real-estate-insider": 3, "1457-quail-st-los-banos-4-bedroom-home": 3, "east-palo-alto-real-estate-missed-ca-fee-amnesty-act-now": 3, "affordable-east-palo-alto-homes-14-robin-court-tour": 3, "redwood-city-real-estate-graeham-watts-love-this-city": 3, "big-head-big-ideas-big-results-real-estate-predictions-2025": 3, "ca-rent-laws-in-crises": 3, "east-menlo-park-real-estate-march-market-update": 3, "1908-Cooley-AVE": 3, "redwood-city-march-real-estate-market-update": 3, "discover-east-palo-alto-real-estate-stunning-home-under-900k": 3, "bay-area-realtor-graeham-watts-dream-home-success": 3, "bay-area-realtor-graeham-watts-hidden-gems": 3, "bay-area-realtor-california-rent-rules": 3, "east-palo-alto-real-estate-market-update-graeham-watts": 3, "essential-home-selling-tips-third-step-home-search": 3, "client-testimonial-moneisha-jermell": 3, "east-palo-alto-ca-house-for-sale-worth-investment": 2, "real-estate-market-update-homes-selling-over-asking": 2, "homes-for-sale-in-east-palo-alto-ca-431-larkspur-dr": 2, "east-palo-alto-real-estate-why-now-is-the-time-to-buy": 2, "belle-haven-market-update": 2, "unleashing-potential-east-palo-alto-real-estate-transformative-fixer-upper": 2, "homes-for-sale-east-palo-alto-ca-117-mission-drive": 2, "bay-area-realtor-graeham-watts-record-home-sale": 2, "redwood-city-homes-for-sale-757-douglas-ave-tour": 2, "homes-for-sale-in-menlo-park-home-value": 2, "east-palo-alto-homes-for-sale-2620-fordham-st-update": 2, "friendly-acres-redwood-city": 2, "redwood-city-condos-for-sale": 2, "redwood-city-living-expense": 2, "east-menlo-park-market-update": 2, "max-profit-selling-homes-redwood-city": 2, "graeham-watts-advantage-redwood-city-real-estate": 2, "east-palo-alto-ca-homes-for-sale-2109-myrtle-pl-tour": 2, "redwood-shores-homes-for-sale-luxury-waterfront-tour": 2, "menlo-park-houses-for-sale-breathtaking-transformation": 2, "houses-for-sale-in-east-palo-alto-1404-camellia-drive": 2, "houses-in-east-palo-alto-952-newbridge-st-real-estate-gem": 2, "bay-area-realtor-graeham-client-testimonial-john-ward": 2, "home-selling-tips-attract-perfect-buyers": 2, "real-estate-market-update-menlo-park-2022-trends": 2, "bay-area-realtor-graeham-watts-trusted-expert": 2, "home-selling-tips-get-top-dollar-for-your-property": 2, "homes-for-sale-in-the-bay-area-los-gatos-home-sells-big": 2, "homes-for-sale-in-the-bay-area-luxury-the-westerly": 2, "redwood-city-real-estate-ca-safer": 2, "redwood-city-real-estate-next-big-investment": 2, "homes-for-sale-in-menlo-park-market-trends-graeham-watts": 2, "bay-area-realtor-home-alone-reaction": 2, "redwood-city-real-estate-january-market-update": 2, "east-menlo-park-market-update-january-2025": 2, "east-palo-alto-real-estate-homeowner-update": 2, "redwood-city-real-estate-charming-dream-homes": 2, "redwood-city-march-real-estate-market-update-2025": 2, "redwood-city-real-estate-market-home-value-insight": 2, "selling-homes-quickly-tips-kitchen-countertops": 2, "SK1C": 2, "east-palo-alto-ca-homes-for-sale-1765-e-bayshore-rd-214": 2, "SM9O": 2, "home-selling-tips-expert-staging-top-dollar": 2, "bay-area-realtor-essential-real-estate-tips": 2, "Home-Search": 2, "redwood-city-real-estate-prices": 2, "east-palo-alto-california-real-estate": 2, "rise-and-fall-of-silicon-valley-bank": 2, "bay-area-realtor-graeham-watts-off-market-dream-homes": 2, "just-sold-east-palo-alto-real-estate-1765-e-bayshore-rd-203": 2, "real-estate-market-update-stunning-home-oakley-59-escher-circle-tour-features": 2, "broadway-masala-redwood-city-indian-food": 2, "east-bay-real-estate-hidden-deals-maximize-sale": 2, "join-bay-area-realtor-graeham-watts-fiesta": 2, "mortgage-rates-forecast-2025-housing-loan-trends": 2, "shocking-bay-area-house-tour-twist": 2, "iBiB": 2, "bay-area-realtor-graeham-watts-humor-and-real-estate": 2, "bay-area-realtor-client-testimonial-trust": 2, "bay-area-realtor-daily-hustle-inside-look": 2, "graeham-watts-client-testimonials": 2, "Choosing-a-Bay-Area-Realtor": 2, "Understanding-Contingencies": 2, "is-now-the-best-time-to-buy-real-estate": 2, "nEnF": 2, "v4S8": 2, "sell-home-prep-tips": 2, "real-estate-market-update-you-need-to-know": 2, "Home-Selling-Tips": 2, "redwood-city-real-estate-ca-silicon-valley-oasis": 2, "graeham-watts-bay-area-realtor-real-estate-success": 2, "bay-area-realtor-graeham-watts-advantage": 2, "home-alone-sequel-we-always-wanted-east-palo-alto-homes": 1, "scariest-real-estate-costume-ever": 1, "rare-redwood-city-real-estate-fixer-2-bed-condo-hidden-potential": 1, "unlock-hidden-redwood-city-real-estate-ca-gems": 1, "beat-the-market-east-palo-alto-market-update-secret-price-bump": 1, "real-estate-market-update-sell-home-top-dollar": 1, "redwood-city-real-estate-renovation-alert": 1, "ULo7": 1, "Featured-Property-menlo-park": 1, "2842-Cornelius-Dr-Tour": 1, "Featured-Property-Redwood-City": 1, "Featured-Property-east-palo-alto": 1, "Why-Clients-Trust-Graeham": 1, "redwood-city-real-estate-charming-2bd-1ba-home-tour-expansive-yard": 1, "real-estate-market-update-top-investment-websites": 1, "homes-for-sale-in-east-palo-alto-ca-2288-addison-ave": 1, "east-palo-alto-homes-for-sale-1765-bayshore-rd-203": 1, "homes-for-sale-in-east-palo-alto-ca-2398-palgas-ave": 1, "redwood-city-real-estate-showdown": 1, "redwood-city-real-estate-2022-sell-high-find-home": 1, "east-palo-alto-homes-for-sale-2620-fordham": 1, "east-palo-alto-real-estate-gaillardia-way-property": 1, "belle-haven-market-update-menlo-park": 1, "real-estate-law-101-ca-landlord-entry-laws-explained": 1, "ab1482-explained-avoid-landlord-mistakes": 1, "real-estate-market-forecast-foreclosure-myth-busted": 1, "client-testimonial-kevin-rebecca-bowe": 1, "east-palo-alto-market-update-2022-inventory-and-prices": 1, "east-palo-alto-market-update-2022-sell-for-maximum-value": 1, "real-estate-market-update-city-vs-county-closing-costs": 1, "discover-stunning-homes-for-sale-east-palo-alto-ca-graeham-watts-advantage": 1, "redwood-city-real-estate-2025-market-shocker-revealed": 1, "east-palo-alto-market-update-free-home-valuation": 1, "east-palo-alto-real-estate-market-update-2023": 1, "east-palo-alto-market-update-2022-home-value-increase": 1, "homes-for-sale-in-the-bay-area-just-listed-hot-properties": 1, "bay-area-realtor-graeham-watts-staging-and-3d-tours": 1, "rising-interest-rates-and-real-estate": 1, "redwood-city-real-estate-update-sell-high-inventory-rising": 1, "fha-loans-vs-conventional-loans": 1, "houses-for-sale-east-palo-alto-453-okeefe-st": 1, "price-your-home-right": 1, "thinking-of-selling-watch-this": 1, "home-worth-estimate": 1, "redwood-city-real-estate-update-2022-home-value": 1, "houses-in-east-palo-alto-dream-homes-for-every-budget": 1, "east-palo-alto-homes-for-sale-stylish-loft": 1, "bay-area-realtor-why-experience-matters": 1, "menlo-park-houses-for-sale-monthly-market-update": 1, "homes-for-sale-in-the-bay-area-san-leandro-modern-family-home": 1, "san-mateo-real-estate-update": 1, "redwood-city-real-estate-market-update": 1, "Home-buyer-guide": 1, "redwood-city-real-estate-market-update-for-homeowners": 1, "east-palo-alto-real-estate-mark-dinan-vision-for-change-city-council-candidate": 1, "east-palo-alto-california-real-estate-trends": 1, "tech-growth-impact-redwood-city-real-estate": 1, "real-estate-market-trends-2025-home-warranty-explained": 1, "real-estate-inventory-trends-low-inventory-market-impact": 1, "east-palo-alto-real-estate-market-update-feb-2025": 1, "bay-area-realtor-graeham-watts-client-testimonial": 1, "homes-for-sale-in-farm-hill-7210-eagle-ridge-dr-gilroy-ca": 1, "interest-rates-and-real-estate-market-impact": 1, "bay-area-realtor-graeham-watts-difference": 1, "east-palo-alto-ca-homes-for-sale-buyer-tips-1765-e-bayshore-rd-204": 1, "KII1": 1, "essential-home-selling-tips-biggest-seller-mistakes": 1, "menlo-park-real-estate-market-update-latest-trends-insights": 1, "homes-for-sale-in-redwood-city-ca": 1, "roosevelt-redwood-city-homes-for-sale": 1, "east-palo-alto-houses-14-robin-ct-tour": 1, "unlock-redwood-city-real-estate": 1, "ab-1482-explained-landlords-tenants-must-know": 1, "home-selling-tips-fix-now-pay-later": 1, "real-estate-market-update-22277-hartman-drive-property-tour": 1, "homes-for-sale-in-redwood-city": 1, "homes-for-sale-redwood-city-ca-staging-secrets": 1, "redwood-city-ca-real-estate-property-tax": 1, "real-estate-market-update-186-overlook-ave-hayward-ca-virtual-tour": 1, "ab-1482-explained-maximum-rent-increase-limits-in-california": 1, "homes-for-sale-in-redwood-city-ca-winning-offers-underwriting-edge": 1, "mt-carmel-redwood-city-homes-for-sale": 1, "redwood-city-real-estate-market-update-graeham-watts": 1, "redwood-city-real-estate-hidden-gems-revealed": 1, "mortgage-costs-redwood-city-real-estate-ca": 1, "redwood-city-real-estate-insights": 1, "real-estate-market-forecast-shocker-haunting-2025": 1, "discover-affordable-homes-under-100k-across-america": 1, "homes-for-sale-in-the-bay-area-786-honeywood-court-pending-sale": 1, "homes-for-sale-in-the-bay-area-graeham-watts-2022-successes": 1, "Home-Selling-Tips-Must-Know": 1, "home-selling-tips-out-of-state-overwhelmed-help": 1, "homes-for-sale-east-palo-alto-ca-living-guide": 1, "redwood-city-real-estate-market-update-feb-2025": 1, "bay-area-realtor-graeham-watts-client-love": 1, "homes-for-sale-in-redwood-city-ca-590-hurlingame-ave-makeover": 1, "essential-home-selling-tips": 1, "real-estate-update-home-selling-fast": 1, "essential-home-selling-tips-fourth-step-offer-accepted": 1, "redwood-city-real-estate-financing": 1, "redwood-city-real-estate-market-update-prices-up-inventory-down": 1, "east-menlo-park-market-update-home-prices-up": 1, "Real-Estate-Market-Forecast-Save-Thousands": 1, "ab-1482-unlocked-secrets-every-california-landlord-should-know": 1, "real-estate-market-forecast-august-trends": 1, "game-changing-real-estate-predictions-2025-big-deals-big-results": 1, "bay-area-realtor-success-story-marketing-results": 1, "home-selling-mistakes-to-avoid": 1, "redwood-city-real-estate-monthly-market-update": 1, "best-time-to-buy-real-estate-1930-pinole-drive-tour": 1, "homes-for-sale-in-the-bay-area-3500-19th-st-san-francisco": 1, "kIhN": 0, "new-rent-rules-real-estate-predictions-2025-tenants-landlords": 0, "redwood-city-real-estate-moving-guide-what-you-need-to-know": 0, "east-menlo-park-market-update-february-2025": 0, "east-palo-alto-real-estate-moving-guide": 0, "east-palo-alto-homes-for-sale": 0, "S9z9": 0, "redwood-city-real-estate-hidden-secrets": 0, "palm-redwood-city-homes-for-sale": 0, "homes-for-sale-in-the-bay-area-soquel-3-bedroom": 0, "101_Graden_St": 0, "explore-east-palo-alto-homes-graeham-watts": 0, "homes-for-sale-east-palo-alto": 0, "redwood-village-homes-for-sale-in-redwood-city": 0, "real-estate-market-update-hartman-drive-los-altos-ca-virtual-tour": 0, "east-palo-alto-market-update-january-2025-key-insights": 0, "discover-homes-for-sale-east-palo-alto-ca-1560-kavanaugh-drive-tour": 0, "buy-a-home-east-palo-alto": 0, "graeham-watts-east-palo-alto-listings": 0, "discover-stunning-homes-for-sale-menlo-park": 0, "the-buzz-in-redwood-city-real-estate": 0, "epa-comps-0601": 0, "expert-home-selling-tips": 0, "bay-area-realtor-free-staging-and-repairs": 0, "Ace_Your_First_Home_Purchase": 0, "Redwood-Oaks-homes-for-sale-in-redwood-city-ca": 0, "essential-real-estate-market-update-guide-for-homebuyers": 0, "essential-home-selling-tips-second-step-pre-qualification": 0, "stunning-homes-for-sale-menlo-park-1135-madera-ave-tour": 0, "east-palo-alto-real-estate-market-update": 0, "redwood-city-spanish-style-4-bedroom-friendly-acres": 0, "real-estate-market-update-foundation-tips-when-buying-your-new-home": 0, "east-palo-alto-real-estate-mark-dinan-vision-change": 0, "what-you-can-buy-redwood-city-houses-for-sale": 0, "selling-your-home-just-got-easier": 0, "home-selling-tips-make-sales-breeze-no-stress": 0, "east-palo-alto-dream-home-tour-30-seconds": 0, "real-estate-market-trends-2025-alameda-county-hidden-gem": 0, "unbelievable-house-tour-market-forecast": 0, "exciting-real-estate-market-update-jumping-housing-market": 0, "trusted-bay-area-realtor-house-safety-guarantee": 0, "unexpected-homes-for-sale-bay-area-dream-home": 0, "property-tax-shock-2025-real-estate-forecast": 0, "top-tips-for-selling-homes-quickly-maximize-home-value": 0, "bay-area-realtor-secrets-unveiled": 0, "east-palo-alto-ca-homes-for-sale-house-tour": 0, "real-estate-market-trends-2025-phone-to-realty": 0, "ab-1482-rent-increases-exemptions-and-california-rental-laws-explained": 0, "bay-area-home-search-map": 0, "east-palo-alto-real-estate-housing-market-update": 0, "redwood-city-real-estate-housing-market-update": 0, "essential-real-estate-market-update-homebuyers-guide": 0, "menlo-park-market-update-belle-haven-housing-trends": 0, "real-estate-market-forecast-fed-interest-rates-explained": 0, "east-palo-alto-real-estate-monthly-market-update": 0, "real-estate-market-trends-2025": 0, "2115-Clarke-Ave": 0, "choose-the-right-agent": 0, "ab-1482-explained-is-your-property-rent-controlled-or-exempt": 0, "real-estate-market-update-tips-to-protect-your-offer-and-investment": 0, "redwood-city-real-estate-market-february-update-you-need-to-know": 0, "discover-east-palo-alto-real-estate-february-market-update": 0, "home-selling-tips-busting-3-buyer-myths": 0, "ab1482-what-landlords-need-to-know-and-do": 0, "redwood-city-real-estate-san-francisco-monthly-market-update": 0, "east-menlo-park-homes-for-sale-graeham-watts": 0, "homes-for-sale-menlo-park-ca-update": 0, "east-menlo-park-homes-for-sale-listings-graeham-watts": 0, "redwood-city-real-estate-paradise": 0, "real-estate-market-update-2025-california-landlord-risks": 0, "east-palo-alto-homes-for-sale-woodland-creek-condo": 0, "discovering-redwood-city-real-estate": 0, "redwood-city-real-estate-crucial-market-update": 0, "real-estate-market-update-just-listed-1186-overlook-ave-hayward-ca-house-tour": 0, "centennial-redwood-city-homes-for-sale": 0, "east-menlo-park-homes-for-sale": 0, "Redwood-City-Homes-on-Sale-Hot-Listings-Price-Drops": 0, "Redwood-City-Homes-on-Sale-Hot-Listings-Price-Drops-Graeham-Watts": 0, "Redwood-City-Homes-on-Sale": 0, "Redwood-City-Homes-on-Sale-Hot-Listings": 0, "redwood-city-real-estate-floor-installation-tips": 0, "east-palo-alto-next-real-estate-goldmine": 0, "homes-for-sale-menlo-park-ca-modern-home-tour-1318-hollyburne-avenue": 0, "downtown-redwood-city-homes-for-sale": 0, "stambaugh-heller-redwood-city-homes-for-sale": 0, "central-redwood-city-homes-for-sale": 0, "best-indian-restaurant-redwood-city-broadway-masala": 0, "ravenswood-school-district-50m-investment-better-schools-higher-teacher-pay": 0, "rising-interest-rates-and-real-estate-prices": 0, "maximize-your-home-sale-value": 0, "eagle-hill-redwood-city-homes-for-sale": 0}} \ No newline at end of file diff --git a/online-content/switchy/EMAIL-to-Peter-generate-QR.html b/online-content/switchy/EMAIL-to-Peter-generate-QR.html new file mode 100644 index 00000000..73247c8c --- /dev/null +++ b/online-content/switchy/EMAIL-to-Peter-generate-QR.html @@ -0,0 +1,58 @@ + +Peter — Generate the postcard QR + +
+
+
New workflow · Switchy QR
+
Generate & embed the postcard QR codes
+
+
+ +

Hey Peter (Jason),

+ +

New skill set up for you — switchy-engine. From now on you don't make + QR codes by hand. You build the postcard, hand it to Claude in Cowork, and it + generates a tracked QR (so we can see scans and retarget people who scan it), + names and files it correctly in Switchy, drops it on our dashboard, and syncs it to GitHub.

+ +

Install (one time)

+

Download the skill, then in Cowork click "Save skill" to install it. This is the + lightweight, QR-only version — all you need to make postcard QR codes.

+ +

If the button doesn't download, copy this link: + https://graehamwatts.github.io/skills/online-content/switchy/switchy-qr.skill + — or pull from GitHub: Graehamwatts/skills → skills/switchy-qr.

+ +

How to generate a QR (every postcard)

+
    +
  1. Finish the postcard in Canva.
    2. +
  2. In Cowork, say: "Generate a QR code for this postcard" and upload the postcard PDF.
    3. +
  3. Claude takes control of Chrome and opens Switchy. You log in once when it asks + (it can't type your password for you — see login below).
    4. +
  4. Claude then does the rest automatically: creates the tracked link with the right + landing page + UTM + our Meta/Google pixels, files it in the "Post card qr" folder, + names it clearly, opens the QR designer, and downloads the QR PNG for you.
    5. +
  5. You drop that QR into the Canva postcard and export for print.
    6. +
  6. Claude adds the link to our Switchy dashboard and syncs it to GitHub.
    7. +
+ +

The one that's already done (your test)

+

The June 1, 2026 EPA "Last 5 Homes" card is already created so you can see how it works:
+ Short link: hi.switchy.io/epa-comps-0601 · Folder: Post card qr · + goes to the home-value page. Pull up that QR in Switchy to grab it.

+ +

Switchy login

+
+ Graeham — fill this in before sending (don't put the password in the email body).
+ Recommended: add Peter as a Switchy team member (Account → Team) so he gets his own login.
+ Login: ________________________  ·  Password: sent separately +
+ +

Naming (Claude handles this automatically)

+

+ Link name: Postcard EPA 2026-06-01 — L \ No newline at end of file diff --git a/online-content/switchy/Farming_Postcard_EPA_06_01_26_IMPROVED.html b/online-content/switchy/Farming_Postcard_EPA_06_01_26_IMPROVED.html new file mode 100644 index 00000000..23554242 --- /dev/null +++ b/online-content/switchy/Farming_Postcard_EPA_06_01_26_IMPROVED.html @@ -0,0 +1,127 @@ + +Farming Postcard EPA 06/01/26 — Improved + + + + +

+ +
Front — improved
+
+
+
+
WHAT DID THE
+
LAST 5 HOMES
+
ON YOUR STREET REALLY SELL FOR?
+
FLIP OVER for your free home-value report  →
+
+ Graeham + +
+
+
+ Intero + Graeham Watts +
+
+
REALTOR®
+
The Martin Team
+
DRE #01466876
+
650-308-4727
+
graehamwatts@gmail.com
+
www.graehamwatts.com
+
+
+
+ +
Back — improved
+
+
+ Graeham +
+
Your street · Free · 60 seconds
+
ZILLOW GUESSES.
I MEASURE.
+
Scan for the actual sale prices on YOUR street — plus what your home would sell for in today’s market. Real comps, not a zip-code guess.
+
    +
  • Real comps on your street — not your zip
    • +
  • Updated this week
    • +
  • No follow-up unless you ask
    • +
+
+
SWITCHY QR
(token pending)
+
+
SEE THE
+
REAL NUMBERS
+
Scan → your street’s real numbers in 60 seconds
+
— Graeham
+
+
+
+
+
+ Intero + Graeham Watts +
+
+
REALTOR®
+
The Martin Team
+
DRE #01466876
+
650-308-4727
+
graehamwatts@gmail.com
+
www.graehamwatts.com
+
+
+
+
If your home is listed with another broker, please disregard this postcard. Homes not necessarily sold by this broker.
+
+ +
+ \ No newline at end of file diff --git a/online-content/switchy/SWITCHY-INTEGRATION-BRIEF.md b/online-content/switchy/SWITCHY-INTEGRATION-BRIEF.md new file mode 100644 index 00000000..71e6decc --- /dev/null +++ b/online-content/switchy/SWITCHY-INTEGRATION-BRIEF.md @@ -0,0 +1,152 @@ +# Switchy Integration — Decision Brief + +Prepared for Graeham Watts · 2026-05-28. Companion files live in the +`switchy-engine/` skill folder. + +--- + +## 1. Architecture recommendation — build the standalone engine (validated) + +Build **`switchy-engine`** as a standalone skill that newsletter, content engine, +postcards, listings, etc. **call into** — with one refinement: the durable +constants (pixel IDs, default domain, tag vocabulary, exclusion-list location) go +in `shared-references/switchy.json` so the engine and every caller read one source. + +I didn't just accept the hypothesis — I pressure-tested it against your actual +stack, and it holds *because of receipts in the code itself*: + +- Your skills already use "build once, reference everywhere" (`cma-generator` + called by the newsletter, `identity.json` shared, `github-skill-sync` as a + horizontal utility). +- Your `content-creation-engine` changelog literally documents the opposite + approach failing: `video-research-engine` went **dormant** when buried inside a + host skill and had to be extracted into `video-watcher`. Embedding link/pixel + logic in the newsletter or content engine would repeat that mistake. +- Duplicating it per-skill would copy **token-handling code into 6+ places** — six + ways to leak a credential and guarantee the pixel list/tag vocab drift apart. + +Full reasoning + the per-skill wiring table: `switchy-engine/references/architecture-decision.md`. + +**Where each skill plugs in:** +- **newsletter-generator** → wrap every EPA Report CTA (highest-value: opted-in consumers). +- **content-creation-engine** → wrap YouTube CTAs, social links, link-in-bio. +- **html-email** → wrap *consumer* emails only; B2B/coach emails track-only. +- **weekly-listing-update** → track-only (audience is one known seller). +- **listing-remarks-writer** → **no wrap** — MLS public remarks legally can't carry + URLs. Tracked links go on the listing's collateral (property page, flyers, QR), not the remarks. +- **postcards** → **the gap** (see §5). + +## 2. Retargeting pathway map + +Full table (29 surfaces, with traffic type / value / pixel-or-not / caveat): +`switchy-engine/references/retargeting-pathway-map.md`. The three highest-leverage +buckets: + +1. **Offline→online bridge** (postcards, yard riders, open-house QR, mailers, + window cards) — traffic you *cannot pixel any other way*. A QR scan converts a + physical mail drop into a digital retargeting audience. +2. **Non-owned platforms** (Zillow, Realtor.com, Nextdoor, GBP posts, social bios) + — you can't put your pixel on their pages, so the redirect is the only hook. +3. **Per-source attribution at scale** (newsletter sections, listings, campaigns) + — tagging tells you which surface actually built the audience. + +## 3. GBP answer — NOT the website field; YES posts + secondary links + +Google's Business links policy prohibits URLs that "redirect or refer" users +elsewhere, and Google now auto-removes violating links; shorteners in the +**primary website field** are a known enforcement target (real cases of links +getting pulled). So: + +- **Primary website field:** real domain only (`graehamwatts.com`), pixeled + natively. Don't risk your map-pack click. +- **GBP posts / appointment / secondary links:** Switchy is safe and is the right + home for GBP retargeting. +- **Headline GBP play:** *GBP post link → YouTube channel* pixels every + high-intent local searcher who clicks, then retargets them. Also: post→listing, + post→home-value form, appointment→GHL booking. + +Detail: `switchy-engine/references/gbp-and-youtube.md`. + +## 4. YouTube / own-site answer — pixel is redundant, link still isn't + +When a Switchy link points to your **own already-pixeled site**, the pixel-drop is +largely redundant (your site tags pixel them on load anyway). But Switchy still +earns its place for four non-pixel reasons: **per-source attribution**, +**swappable destination** (change a printed QR's target without reprinting), +**multi-pixel firing** (fire Meta+Google+LinkedIn from one link), and +**pixel-fires-before-page-load** (catches bouncers your on-site pixel misses). + +Per-surface rule: **non-owned destination → always wrap (essential). Own pixeled +site → wrap only if you want attribution/swap/multi-pixel/pre-load capture; +otherwise raw URL is fine.** Print/QR pointing to your own site → still wrap (swap +value alone). Table in the same reference file. + +## 5. The postcard gap (flagged) + +There is **no postcard skill** — it's a manual Canva workflow, and its QR codes are +the single biggest missed retargeting opportunity (offline→online, un-pixelable +otherwise). Recommendation: short-term, have the Canva workflow mint its QR via +`switchy-engine` (one tagged tracked link per drop/ZIP); later, build a thin +`postcard-engine` that calls the engine automatically. Defer the new skill until +the link engine is live. + +## 6. Audience hygiene tradeoff — don't pixel everything + +Pixeling tiny or B2B traffic pollutes audiences and burns spend (showing listing +ads to your title rep; sub-100 audiences you can't even target). + +- **PIXEL:** newsletter, SMS, listings, GBP posts, Zillow/Realtor, social bios, + YouTube links, all offline QR. +- **SKIP/track-only:** email signature, LinkedIn, peer business cards, sphere/PCFS + touches (mixed/B2B/known). +- **Segment** via mandatory `audience-class` tag (`consumer`/`prospect`/`b2b`/ + `mixed`) on every minted link; build ad audiences from `consumer`+`prospect` + **minus a standing vendor/agent exclusion list**; never spend against a + sub-1,000 standalone audience; build lookalikes only from clean seeds. + +Detail: `switchy-engine/references/audience-hygiene.md`. + +## 7. The scaffolded skill + working query + +`switchy-engine/` contains: +- `SKILL.md` — engine definition, API facts, token setup, the mint/report contract. +- `scripts/switchy_analytics.py` — secure token handling (env/file, never hardcoded), + schema introspection, the per-link analytics query (scalar + aggregate fallback), + and the **scans → audience → budget** table. Runs in DEMO mode without a token; + I've verified it executes and renders. +- `references/` — queries, pathway map, GBP/YouTube answers, audience hygiene, + architecture decision. +- `sample_switchy_report.md/.csv` — sample output (illustrative numbers). +- `.gitignore` — keeps the token and workspace data out of version control. + +The budget model: `audience = clicks × pixel-match-rate (55%)`; `monthly budget = +audience × frequency (10×) × CPM ($22) / 1000`. All three are CLI-tunable. It's the +spend an audience can *absorb*, not a target. + +--- + +## What I need from you to finish (the asks) + +1. **Token activation — the blocker.** Confirm whether your Switchy API token is + actually enabled. Generate it (Workspace → Settings → Integrations → Generate a + token); if the smoke-test query returns errors/empty, message Switchy **live + chat** to enable API access, then regenerate. Until this returns rows, I can't + lock the real click/scan field name or pull live numbers. +2. **Decision — GDPR popup.** Default `showGDPR: true` on cold consumer links (CA, + safer, slightly lower match) vs. `false` (higher match, more exposure). Your call. +3. **Constants for `shared-references/switchy.json`:** your Meta pixel ID, GA + measurement ID (and any LinkedIn/Pinterest/Bing pixels you want fired), and your + preferred default Switchy domain. I'll wire these once you provide them. +4. **Vendor/agent exclusion list** — a CSV of known peers/vendors/team emails to + stand up the standing exclusion audience. +5. **Approve the per-skill wiring** in §1 before I edit the live skills (right now + `switchy-engine` is built but not yet referenced by newsletter/content/etc.). +6. **TikTok** — accept GTM-routed or click-only tracking (no native Switchy TikTok + pixel), or drop TikTok from the pixel plan. + +### Two accuracy flags (per your rules) +- The **per-link click/scan GraphQL field name is unverified** — public docs only + show workspace-level fields. The script introspects it first; I did **not** + fabricate a field name. Treat the analytics query as confirmed only after + `--confirm-schema` runs on your live token. +- The sample report numbers are **illustrative DEMO data**, not real. diff --git a/online-content/switchy/switchy-engine.skill b/online-content/switchy/switchy-engine.skill new file mode 100644 index 00000000..3a849d5a Binary files /dev/null and b/online-content/switchy/switchy-engine.skill differ diff --git a/online-content/switchy/switchy-qr.skill b/online-content/switchy/switchy-qr.skill new file mode 100644 index 00000000..1f979080 Binary files /dev/null and b/online-content/switchy/switchy-qr.skill differ diff --git a/online-content/switchy/switchy_LIVE_report.md b/online-content/switchy/switchy_LIVE_report.md new file mode 100644 index 00000000..03c258d5 --- /dev/null +++ b/online-content/switchy/switchy_LIVE_report.md @@ -0,0 +1,452 @@ +# Switchy Retargeting Report — 2026-05-28 20:30 + +_Data source: LIVE Switchy API (env:SWITCHY_API_TOKEN)_ +_Model: pixel match 55%, freq 10x / 30d, CPM $22_ + +| Short link | Tags | Destination | Clicks | Audience | Monthly budget | Status | +|---|---|---|---:|---:|---:|---| +| pages.graehamwatts.com/bay-area-homes-for-sale | | https://graehamwatts.com/bay-area-homes-fo… | 1,067 | 587 | $129 | Thin — fold into a combined audience | +| pages.graehamwatts.com/SbSR | | https://graehamwatts.com/sell-with-graeham… | 1,023 | 563 | $124 | Thin — fold into a combined audience | +| pages.graehamwatts.com/graehamwatts-meet | | https://tidycal.com/graehamwatts/30-minute… | 776 | 427 | $94 | Thin — fold into a combined audience | +| pages.graehamwatts.com/whats-my-house-worth | | https://graehamwatts.com/evaluation | 722 | 397 | $87 | Thin — fold into a combined audience | +| pages.graehamwatts.com/SKZY | | https://graehamwatts.com/evaluation?utm_ca… | 681 | 375 | $82 | Thin — fold into a combined audience | +| pages.graehamwatts.com/SKaA | | https://graehamwatts.com/meet-with-graeham… | 602 | 331 | $73 | Thin — fold into a combined audience | +| pages.graehamwatts.com/SK0r | | https://graehamwatts.com/evaluation?utm_ca… | 585 | 322 | $71 | Thin — fold into a combined audience | +| pages.graehamwatts.com/XhI5 | | https://graehamwatts.com/2271-euclid-ave?u… | 570 | 314 | $69 | Thin — fold into a combined audience | +| pages.graehamwatts.com/SK1P | | https://graehamwatts.com/evaluation?utm_ca… | 547 | 301 | $66 | Thin — fold into a combined audience | +| pages.graehamwatts.com/SKai | | https://graehamwatts.com/sell-with-graeham… | 515 | 283 | $62 | Thin — fold into a combined audience | +| pages.graehamwatts.com/SKY8 | | https://graehamwatts.com/buy-with-graeham?… | 509 | 280 | $62 | Thin — fold into a combined audience | +| pages.graehamwatts.com/redwood-city-homes-for-sale | | https://graehamwatts.com/redwood-city-ca-h… | 496 | 273 | $60 | Thin — fold into a combined audience | +| pages.graehamwatts.com/SKYl | | https://graehamwatts.com/buy-with-graeham?… | 460 | 253 | $56 | Thin — fold into a combined audience | +| pages.graehamwatts.com/east-palo-alto-ca-homes-for-sale | | https://graehamwatts.com/east-palo-alto-ca… | 429 | 236 | $52 | Thin — fold into a combined audience | +| pages.graehamwatts.com/homes-for-sale-belle-haven | | https://graehamwatts.com/east-menlo-park | 397 | 218 | $48 | Thin — fold into a combined audience | +| pages.graehamwatts.com/T1td | | https://graehamwatts.com/59-escher-cir?utm… | 396 | 218 | $48 | Thin — fold into a combined audience | +| pages.graehamwatts.com/SJob | | https://graehamwatts.com/buy-with-graeham?… | 376 | 207 | $46 | Thin — fold into a combined audience | +| pages.graehamwatts.com/SJ-k | | https://graehamwatts.com/evaluation?utm_ca… | 296 | 163 | $36 | Thin — fold into a combined audience | +| hi.switchy.io/MFUy | | https://graehamwatts.com/ | 281 | 155 | $34 | Thin — fold into a combined audience | +| pages.graehamwatts.com/SXDB | | https://graehamwatts.com/sell-with-graeham… | 234 | 129 | $28 | Thin — fold into a combined audience | +| pages.graehamwatts.com/SM9A | | https://graehamwatts.com/sell-with-graeham… | 205 | 113 | $25 | Thin — fold into a combined audience | +| pages.graehamwatts.com/SKZH | | https://graehamwatts.com/buy-with-graeham?… | 190 | 105 | $23 | Thin — fold into a combined audience | +| pages.graehamwatts.com/SRlP | | https://graehamwatts.com/sell-with-graeham… | 185 | 102 | $22 | Thin — fold into a combined audience | +| pages.graehamwatts.com/SKDr | | https://graehamwatts.com/buy-with-graeham?… | 180 | 99 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/SX1b | | https://graehamwatts.com/sell-with-graeham… | 170 | 94 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/SJoH | | https://graehamwatts.com/buy-with-graeham?… | 163 | 90 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/SKYx | | https://graehamwatts.com/buy-with-graeham?… | 141 | 78 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/SJpt | | https://graehamwatts.com/buy-with-graeham?… | 141 | 78 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/east-palo-alto-CA-homes-for-sale-under-1million | | https://graehamwatts.com/east-palo-alto-ca… | 136 | 75 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/SJ-D | | https://graehamwatts.com/buy-with-graeham?… | 126 | 69 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/SX7L | | https://graehamwatts.com/sell-with-graeham… | 124 | 68 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/SK0T | | https://graehamwatts.com/buy-with-graeham?… | 123 | 68 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/S-VQ | | https://graehamwatts.com/sell-with-graeham… | 116 | 64 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/T1nG | | https://graehamwatts.com/59-escher-cir?utm… | 111 | 61 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/SXGz | | https://graehamwatts.com/sell-with-graeham… | 108 | 59 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/SKYW | | https://graehamwatts.com/buy-with-graeham?… | 105 | 58 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/East-palo-alto-home-for-sale | | https://graehamwatts.com/east-palo-alto-ca… | 96 | 53 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/menlo-park-homes-for-sale | | https://graehamwatts.com/east-menlo-park?u… | 94 | 52 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/how-much-is-my-home-worth | | https://bit.ly/4a7ScVM | 93 | 51 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/T1r5 | | https://graehamwatts.com/2842-cornelius-dr… | 90 | 50 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/S-Vz | | https://graehamwatts.com/sell-with-graeham… | 88 | 48 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/redwood-city-home-for-sale | | https://graehamwatts.com/redwood-city-home… | 80 | 44 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/T8PU | | https://graehamwatts.com/sell-with-graeham… | 78 | 43 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/TA7m | | https://graehamwatts.com/sell-with-graeham… | 71 | 39 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/SX5l | | https://graehamwatts.com/sell-with-graeham… | 70 | 38 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/SXG2 | | https://graehamwatts.com/sell-with-graeham… | 70 | 38 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/S9yj | | https://graehamwatts.com/sell-with-graeham… | 67 | 37 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/XhH8 | | https://graehamwatts.com/2271-euclid-ave?u… | 61 | 34 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/schedule-bay-area-real-estate-meeting | | https://tidycal.com/graehamwatts/30-minute… | 60 | 33 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/SKYI | | https://graehamwatts.com/buy-with-graeham?… | 60 | 33 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/blog-east-palo-alto-ca-homes-for-sale | | https://graehamwatts.com/east-palo-alto-ca… | 59 | 32 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/S-Vg | | https://graehamwatts.com/sell-with-graeham… | 59 | 32 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/free-home-evaluation | | https://graehamwatts.com/evaluation | 59 | 32 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/S9yo | | https://graehamwatts.com/?utm_campaign=Sel… | 54 | 30 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/T1v5 | | https://graehamwatts.com/2842-cornelius-dr… | 51 | 28 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/SM92 | | https://graehamwatts.com/buy-with-graeham?… | 50 | 28 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/SXBD | | https://graehamwatts.com/sell-with-graeham… | 48 | 26 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/S9xP | | https://graehamwatts.com/sell-with-graeham… | 41 | 23 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/SX4- | | https://graehamwatts.com/sell-with-graeham… | 41 | 23 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/T8Yl | | https://graehamwatts.com/sell-with-graeham… | 41 | 23 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/SKXy | | https://graehamwatts.com/buy-with-graeham?… | 39 | 21 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/redwood-city-ca-homes-for-sale | | https://graehamwatts.com/redwood-city-ca-h… | 33 | 18 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/real-estate-market-forecast-save-thousands-on-your-mortgage-hack | | https://youtu.be/Swqu9xssJFE | 28 | 15 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| hi.switchy.io/formly_email | | https://getformly.app/m0ZTE9 | 28 | 15 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/WVkH | | https://graehamwatts.com/5636-orchard-park… | 28 | 15 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/S9w- | | https://graehamwatts.com/?utm_campaign=Sel… | 27 | 15 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/bay-area-real-estate-market-update-2026-01 | | https://youtu.be/LI92G9EpfRg | 27 | 15 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/SKXW | | https://graehamwatts.com/buy-with-graeham?… | 24 | 13 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/Meet-with-Graeham | | https://link.graehamwatts.com/widget/booki… | 22 | 12 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/S9xr | | https://graehamwatts.com/sell-with-graeham… | 22 | 12 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/Home-For-Sale | | https://graehamwatts.com/listing?_gl=1*dmy… | 20 | 11 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/SM9K | | https://graehamwatts.com/sell-with-graeham… | 20 | 11 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/east-menlo-park | | https://graehamwatts.com/east-menlo-park | 19 | 10 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/sell-with-graeham | | https://graehamwatts.com/sell-with-graeham… | 19 | 10 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/5636-orchard-park-drive-san-jose-ca-95123 | | https://graehamwatts.com/5636-orchard-park… | 17 | 9 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/why-sell-with-graeham | | https://graehamwatts.com/whats-my-east-pal… | 16 | 9 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/discover-east-menlo-park-homes | | https://graehamwatts.com/east-menlo-park | 15 | 8 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/Belle-haven-home-value-evaluation | | https://graehamwatts.com/evaluation | 14 | 8 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/SKaM | | https://graehamwatts.com/meet-with-graeham… | 13 | 7 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/homes-for-sale-redwood-city-ca | | https://graehamwatts.com/redwood-city-ca-h… | 13 | 7 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/east-palo-alto-real-estate-sold-40k-more-with-50k-issue | | https://youtu.be/IOMKVGUeET0 | 13 | 7 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| hi.switchy.io/get_free_consultation | | https://graehamwatts.com/ppc-lp?fromCms=1 | 13 | 7 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/home-selling-tips-sell-your-home-quickly | | https://youtu.be/UR8raNurlrw | 12 | 7 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/homevalue | | https://graehamwatts.com/whats-my-east-pal… | 11 | 6 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/v4Sk | | https://youtube.com/shorts/Uc5GU2m3lVQ?fea… | 10 | 6 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/S9y0 | | https://graehamwatts.com/?utm_campaign=Sel… | 10 | 6 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| hi.switchy.io/before_after_1930_Sarah_Dr | | https://youtu.be/cwCtzfaTFBU | 9 | 5 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/SX9V | | https://graehamwatts.com/sell-with-graeham… | 9 | 5 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/S9z0 | | https://graehamwatts.com/sell-with-graeham… | 9 | 5 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/schedule-call-with-graeham | | https://tidycal.com/graehamwatts/30-minute… | 9 | 5 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/woodside-plaza-redwood-city-homes-for-sale | | https://graehamwatts.com/redwood-city-wood… | 9 | 5 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/east-palo-alto-condo-1982-w-bayshore-223 | | https://graehamwatts.com/listing-detail/11… | 9 | 5 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/essential-home-selling-tips-5-Biggest-Mistakes | | https://www.youtube.com/watch?v=t5K0mSd9vho | 8 | 4 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/homes-for-sale-east-palo-alto-ca-discover-your-dream-home | | https://youtu.be/9czXxf4eV98 | 7 | 4 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/952-6th-AVE-Redwood-City-CA-94063 | | https://graehamwatts.com/listing-detail/11… | 7 | 4 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/828-Weeks-ST | | https://graehamwatts.com/listing-detail/11… | 7 | 4 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/homes-for-sale-in-the-bay-area-safety-tips | | https://youtu.be/tcXNFmlJ5-A | 7 | 4 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/free-home-valuation | | https://graehamwatts.com/evaluation?_gl=1*… | 7 | 4 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/Home-Buying-Myths | | https://youtu.be/I5cgy1ck4Gc | 7 | 4 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/redwood-city-real-estate-expert-advice-you-can-trust | | https://youtu.be/AMCRuyMPwq0 | 6 | 3 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/east-palo-alto-real-estate-470-bell-st-tranquil-home-tour-silicon-valley | | https://youtu.be/jkRO-VHB4rw | 6 | 3 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/real-estate-market-forecast-fed-rate-cuts-2025-housing-market | | https://youtube.com/shorts/TJ1s6lGQQhs | 6 | 3 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/bay-area-realtor-graeham-watts-winnie-danny-dream-home | | https://youtu.be/x3P7pQgX4c4 | 6 | 3 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/real-estate-market-update-transformed-sold-for-top-dollar | | https://youtu.be/cwCtzfaTFBU | 6 | 3 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/east-palo-alto-condos-for-sale-woodland-creek-223 | | https://youtu.be/jm59JyV3fJk | 5 | 3 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/3-Buyer-Myths | | https://62bygw.hippovideo.io/page/graehamw… | 5 | 3 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/selling-homes-quickly-tips-maximize-home-value | | https://youtu.be/oQTZdIt8gSs | 5 | 3 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/east-palo-alto-real-estate-insights-650k-fixer-upper-dream-home | | https://youtube.com/shorts/FoCq1qIOD64 | 5 | 3 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/Home-Evaluation | | https://graehamwatts.com/whats-my-east-pal… | 5 | 3 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/bay-area-realtor-dream-home-client-testimonial | | https://youtu.be/jDPAsX80XCU | 5 | 3 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/2025-california-landlord-risk-update | | https://www.youtube.com/watch?v=DYlXSFnO-5M | 5 | 3 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/Why-Clients-Trust-Graeham-Watts | | https://youtu.be/_UZPXHYIwHc | 5 | 3 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/homes-for-sale-in-menlo-park-graeham-watts-tour | | https://youtu.be/9C35wqF79wk | 5 | 3 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/redwood-city-real-estate-cost-of-living | | https://youtu.be/qe8FdzkNPc8 | 4 | 2 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/Proven-Home-Selling-Tips | | https://youtu.be/eYS4AWn92-8 | 4 | 2 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/east-palo-alto-homes-for-sale-431-larkspur-dr-record-sale | | https://youtu.be/HUfZa1kIt-g | 4 | 2 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/east-palo-alto-market-update-prices-rising-homes-selling-fast | | https://youtu.be/IZV4sljzPcs | 4 | 2 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| hi.switchy.io/KII7 | | sms:+16503084727?&body=Hey%2C%20Graeham%20… | 4 | 2 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/east-palo-alto-real-estate-470-bell-st-listing | | https://youtu.be/Fr7wU3SkAAk | 4 | 2 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/avoid-costly-homebuyer-errors-redwood-city-real-estate-ca | | https://youtu.be/2nLZQ6xOoRw | 4 | 2 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/SJpA | | https://graehamwatts.com/buy-with-graeham?… | 4 | 2 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/real-estate-market-update-313-smithwood-milpitas-ca-95035-home-tour | | https://youtu.be/ueh_94DC2-Y | 4 | 2 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/Offer-Accepted | | https://62bygw.hippovideo.io/page/graehamw… | 4 | 2 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/top-east-palo-alto-realtor-sell-home-fast-top-dollar | | https://www.youtube.com/shorts/rmREybVznQA | 4 | 2 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/top-east-palo-alto-realtor-home-selling-tips | | https://www.youtube.com/shorts/rmREybVznQA | 4 | 2 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/discover-homes-for-sale-redwood-city-ca-spanish-style-4-bedroom-friendly-acres | | https://youtube.com/shorts/s5ANgnt78Yo | 4 | 2 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/san-jose-home-for-sale-500k-assumable-rate | | https://youtu.be/NNs77CwRI3g | 4 | 2 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/How-We-Price-Your-Home | | https://62bygw.hippovideo.io/page/graehamw… | 4 | 2 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/east-palo-alto-homes-for-sale-123-main-st-tour | | https://youtu.be/PRPE4iykKJ4 | 4 | 2 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/redwood-city-real-estate-monthly-market-update-Graeham-watts | | https://youtu.be/wEGx9jY_lJY | 4 | 2 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/houses-in-east-palo-alto-modernized-1239-jervis-ave | | https://youtu.be/PRPE4iykKJ4 | 4 | 2 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/East-Palo-Alto-Real-Estate-Monthly-Market-Update-graeham-watts | | https://youtu.be/ERx55bk7Ac0 | 4 | 2 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/east-palo-alto-real-estate-missed-ca-fee-amnesty-act-now | | https://youtu.be/ji4bzCdj-aw | 3 | 2 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/east-palo-alto-realtor-sell-home-fast-top-dollar | | https://youtube.com/shorts/rmREybVznQA | 3 | 2 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/bay-area-realtor-graeham-watts-testimonial-savings | | https://youtu.be/zX5JB2RuXsY | 3 | 2 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/1457-quail-st-los-banos-4-bedroom-home | | https://youtube.com/shorts/Lf-0Jq-KCa4 | 3 | 2 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/affordable-east-palo-alto-homes-14-robin-court-tour | | https://youtube.com/shorts/-DF6_AEZ6s0 | 3 | 2 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/bay-area-realtor-free-home-staging-maximize-value | | https://youtu.be/oSYwa8rn7qg | 3 | 2 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/real-estate-predictions-2025-east-menlo-park | | https://youtu.be/MhavOznsxdI | 3 | 2 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/redwood-city-real-estate-graeham-watts-love-this-city | | https://www.youtube.com/shorts/CmeYtuyutTw | 3 | 2 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/bay-area-realtor-california-rent-rules | | https://youtu.be/DWf1Zm5XCLE | 3 | 2 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/bay-area-realtor-supply-and-demand-toilet-paper-bidding-war | | https://youtu.be/9U3IflXKGdg | 3 | 2 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/home-selling-tips-maximize-sale-price-fast | | https://youtu.be/eYS4AWn92-8 | 3 | 2 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/big-head-big-ideas-big-results-real-estate-predictions-2025 | | https://youtube.com/shorts/rmREybVznQA | 3 | 2 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/bay-area-realtor-graeham-watts-johnny-ashley-success | | https://youtu.be/R-vWlOU5PvQ | 3 | 2 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/ca-rent-laws-in-crises | | https://youtu.be/qOOcIe-3NP4 | 3 | 2 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/east-palo-alto-real-estate-market-update-graeham-watts | | https://youtu.be/UebeV3jVfy4 | 3 | 2 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/essential-home-selling-tips-third-step-home-search | | https://youtu.be/DTuZtG-eOLQ | 3 | 2 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/bay-area-realtor-graeham-watts-ryan-teeda-journey | | https://youtu.be/K1thMc06p2M | 3 | 2 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/1908-Cooley-AVE | | https://graehamwatts.com/listing-detail/11… | 3 | 2 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/east-menlo-park-market-update-prices-up-inventory-low | | https://youtu.be/Thl3cu12BKg | 3 | 2 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/bair-island-homes-for-sale-in-redwood-city | | https://graehamwatts.com/redwood-city-bair… | 3 | 2 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/east-menlo-park-real-estate-march-market-update | | https://www.youtube.com/watch?v=8tG7RqPXIAo | 3 | 2 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/The-Graeham-Watts-Advantage | | https://62bygw.hippovideo.io/s/k4EWYg3P? W… | 3 | 2 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/homes-for-sale-in-the-bay-area-buy-sell-with-graeham-watts | | https://youtu.be/d-1do_fHc7A | 3 | 2 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/redwood-city-march-real-estate-market-update | | https://www.youtube.com/watch?v=fjDbun9DY5I | 3 | 2 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/discover-east-palo-alto-real-estate-stunning-home-under-900k | | https://youtu.be/ERozyMaZkGg | 3 | 2 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/redwood-city-homes-for-sale-hidden-gems-affordable-living | | https://youtu.be/KqvEa_SdxNU | 3 | 2 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/redwood-city-real-estate-off-market-secrets | | https://youtu.be/pfTUOCWSiM4 | 3 | 2 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/real-estate-market-forecast | | https://youtu.be/aM4qx_1v26k | 3 | 2 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/east-palo-alto-market-update-are-you-keeping-up | | https://www.youtube.com/watch?v=Sz4jRWTZgbI | 3 | 2 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/east-palo-alto-ravenswood-school-district-transformation | | https://youtu.be/gdytpSbZBeQ | 3 | 2 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/east-palo-alto-ca-homes-for-sale-woodland-creek-condo-tour | | https://youtu.be/4z_EXVd2aiw?utm_campaign=… | 3 | 2 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/redwood-city-real-estate-insider | | https://youtu.be/IOMKVGUeET0 | 3 | 2 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/client-testimonial-moneisha-jermell | | https://youtu.be/wyiKWpj852w | 3 | 2 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/bay-area-realtor-graeham-watts-hidden-gems | | https://youtu.be/n8lNXk3QTUM | 3 | 2 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/bay-area-realtor-graeham-watts-dream-home-success | | https://youtu.be/D6FOBstUpPM | 3 | 2 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/east-palo-alto-ca-house-for-sale-worth-investment | | https://youtu.be/hOcUeiAWhhI | 2 | 1 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/real-estate-market-update-homes-selling-over-asking | | https://youtu.be/4hWHkLPj7vU | 2 | 1 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/homes-for-sale-in-east-palo-alto-ca-431-larkspur-dr | | https://youtu.be/GYtNQ18PKSg | 2 | 1 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/east-palo-alto-real-estate-why-now-is-the-time-to-buy | | https://youtu.be/FpaTXnmdtyI | 2 | 1 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/belle-haven-market-update | | https://youtu.be/JRdenb9WY68 | 2 | 1 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/unleashing-potential-east-palo-alto-real-estate-transformative-fixer-upper | | https://youtu.be/q5sc-22AauA | 2 | 1 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/homes-for-sale-east-palo-alto-ca-117-mission-drive | | https://youtu.be/yYs6poUDo3g | 2 | 1 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/bay-area-realtor-graeham-watts-record-home-sale | | https://youtu.be/mZ5d1OXOiBk | 2 | 1 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/redwood-city-homes-for-sale-757-douglas-ave-tour | | https://youtu.be/P_xG-3Vbvwk | 2 | 1 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/homes-for-sale-in-menlo-park-home-value | | https://youtu.be/sOCqkU8pzvY | 2 | 1 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/east-palo-alto-homes-for-sale-2620-fordham-st-update | | https://youtu.be/lWQmnzSH69s | 2 | 1 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/friendly-acres-redwood-city | | https://youtu.be/q3ZqNbKcxs4 | 2 | 1 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/redwood-city-condos-for-sale | | https://youtu.be/5DkZ2Omhn0E | 2 | 1 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/redwood-city-living-expense | | https://youtu.be/uXimfvpyQoU | 2 | 1 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/east-menlo-park-market-update | | https://youtu.be/xrhk16hEQkk | 2 | 1 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/max-profit-selling-homes-redwood-city | | https://youtu.be/J911WGvePPc | 2 | 1 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/graeham-watts-advantage-redwood-city-real-estate | | https://youtu.be/8O_PDLpfRIo | 2 | 1 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/east-palo-alto-ca-homes-for-sale-2109-myrtle-pl-tour | | https://youtu.be/SqOzKVyRUWc | 2 | 1 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/redwood-shores-homes-for-sale-luxury-waterfront-tour | | https://youtu.be/tqKSADgRm8Q | 2 | 1 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/menlo-park-houses-for-sale-breathtaking-transformation | | https://youtu.be/jz3XXxRmL88 | 2 | 1 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/houses-for-sale-in-east-palo-alto-1404-camellia-drive | | https://youtu.be/mpkKH9kGCQU | 2 | 1 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/houses-in-east-palo-alto-952-newbridge-st-real-estate-gem | | https://youtu.be/ViQurCKu2yA | 2 | 1 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/bay-area-realtor-graeham-client-testimonial-john-ward | | https://youtu.be/X3hp3FH_OLc | 2 | 1 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/home-selling-tips-attract-perfect-buyers | | https://youtu.be/5Bg2RaEf2TU | 2 | 1 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/real-estate-market-update-menlo-park-2022-trends | | https://youtu.be/_aiC3xKvFII | 2 | 1 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/bay-area-realtor-graeham-watts-trusted-expert | | https://youtu.be/QWrMu_NQZ3A | 2 | 1 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/home-selling-tips-get-top-dollar-for-your-property | | https://youtu.be/Lh_1XIsn5Tk | 2 | 1 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/homes-for-sale-in-the-bay-area-los-gatos-home-sells-big | | https://youtu.be/cjq9DDMAteg | 2 | 1 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/homes-for-sale-in-the-bay-area-luxury-the-westerly | | https://youtu.be/PTDk3BnFfF4 | 2 | 1 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/redwood-city-real-estate-ca-safer | | https://youtu.be/ihNmmiZCb_4 | 2 | 1 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/redwood-city-real-estate-next-big-investment | | https://youtu.be/yCnb-wXOeN4 | 2 | 1 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/homes-for-sale-in-menlo-park-market-trends-graeham-watts | | https://youtu.be/F_zJJ71-7Us | 2 | 1 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/bay-area-realtor-home-alone-reaction | | https://youtu.be/j4D7UP38nuw | 2 | 1 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/redwood-city-real-estate-january-market-update | | https://youtu.be/sPtJJjuLNAc | 2 | 1 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/east-menlo-park-market-update-january-2025 | | https://youtu.be/CIxAjHekyR0 | 2 | 1 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/east-palo-alto-real-estate-homeowner-update | | https://youtu.be/N6dfSAq35ng@gmail.com | 2 | 1 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/redwood-city-real-estate-charming-dream-homes | | https://youtu.be/dshlY4NEDkY | 2 | 1 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/redwood-city-march-real-estate-market-update-2025 | | https://www.youtube.com/watch?v=fjDbun9DY5I | 2 | 1 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/redwood-city-real-estate-market-home-value-insight | | https://youtu.be/DDkpLjrXlQg | 2 | 1 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/selling-homes-quickly-tips-kitchen-countertops | | https://youtu.be/CNePiFgQloo | 2 | 1 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/SK1C | | https://graehamwatts.com/buy-with-graeham?… | 2 | 1 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/SM9O | | https://graehamwatts.com/sell-with-graeham… | 2 | 1 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/east-palo-alto-ca-homes-for-sale-1765-e-bayshore-rd-214 | | https://youtu.be/F_9aCy48xiA | 2 | 1 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/home-selling-tips-expert-staging-top-dollar | | https://youtu.be/wojWWH-c1-g | 2 | 1 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/bay-area-realtor-essential-real-estate-tips | | https://youtu.be/4yHmZf3_cK0 | 2 | 1 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/Home-Search | | https://62bygw.hippovideo.io/s/Xodgg8bk? W… | 2 | 1 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/redwood-city-real-estate-prices | | https://youtu.be/nxmsOFoDNak | 2 | 1 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/east-palo-alto-california-real-estate | | https://youtu.be/NXxYWfxCpu8 | 2 | 1 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/rise-and-fall-of-silicon-valley-bank | | https://youtu.be/1qnDcx755mk | 2 | 1 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/bay-area-realtor-graeham-watts-off-market-dream-homes | | https://youtu.be/qksoxPL1OS4 | 2 | 1 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/just-sold-east-palo-alto-real-estate-1765-e-bayshore-rd-203 | | https://youtu.be/npwvcGCvJaU | 2 | 1 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/real-estate-market-update-stunning-home-oakley-59-escher-circle-tour-features | | https://youtu.be/L0oPoggLJXs | 2 | 1 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/broadway-masala-redwood-city-indian-food | | https://www.youtube.com/watch?v=igwB8ZVOj7Y | 2 | 1 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/east-bay-real-estate-hidden-deals-maximize-sale | | https://www.youtube.com/shorts/tj7luRmP6hE | 2 | 1 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/join-bay-area-realtor-graeham-watts-fiesta | | https://www.youtube.com/shorts/kICnNSf9yt8 | 2 | 1 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/mortgage-rates-forecast-2025-housing-loan-trends | | https://www.youtube.com/shorts/M2aq5ab09XI | 2 | 1 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/shocking-bay-area-house-tour-twist | | https://www.youtube.com/shorts/JW2CEG3xOn0 | 2 | 1 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/iBiB | | https://calendly.com/pat-brunner/30min?mon… | 2 | 1 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/bay-area-realtor-graeham-watts-humor-and-real-estate | | https://youtu.be/FtXHg1K89Fk | 2 | 1 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/bay-area-realtor-client-testimonial-trust | | https://youtu.be/_UZPXHYIwHc | 2 | 1 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/bay-area-realtor-daily-hustle-inside-look | | https://youtu.be/hDmcb6z23v4 | 2 | 1 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/graeham-watts-client-testimonials | | https://youtu.be/-KsvYprH3qA | 2 | 1 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/Choosing-a-Bay-Area-Realtor | | https://62bygw.hippovideo.io/page/graehamw… | 2 | 1 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/Understanding-Contingencies | | https://62bygw.hippovideo.io/page/graehamw… | 2 | 1 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/nEnF | | https://app.box.com/s/wjmikb88phzfjpml5jbm… | 2 | 1 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/is-now-the-best-time-to-buy-real-estate | | https://youtu.be/cxAoaqG-Uqk | 2 | 1 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/v4S8 | | https://app.box.com/s/k1rzueil7beea2l089ot… | 2 | 1 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/sell-home-prep-tips | | https://drive.google.com/file/d/1VyIGSRMAz… | 2 | 1 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/real-estate-market-update-you-need-to-know | | https://youtu.be/BlAWo571LKs | 2 | 1 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/Home-Selling-Tips | | https://youtu.be/Lh_1XIsn5Tk | 2 | 1 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/redwood-city-real-estate-ca-silicon-valley-oasis | | https://youtu.be/M1EhjP0be6g | 2 | 1 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/graeham-watts-bay-area-realtor-real-estate-success | | https://youtu.be/sAY8s59amvI | 2 | 1 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/bay-area-realtor-graeham-watts-advantage | | https://youtu.be/0CuOueT7orQ | 2 | 1 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/houses-for-sale-east-palo-alto-453-okeefe-st | | https://youtu.be/42K4c5tIJEU | 1 | 1 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/real-estate-market-forecast-august-trends | | https://youtu.be/dj6wor-GUME | 1 | 1 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/east-menlo-park-market-update-home-prices-up | | https://youtu.be/USUMOzPTYoA | 1 | 1 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/Real-Estate-Market-Forecast-Save-Thousands | | https://youtu.be/Swqu9xssJFE | 1 | 1 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/bay-area-realtor-success-story-marketing-results | | https://youtu.be/KX8LKsRAi4Y | 1 | 1 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/Why-Clients-Trust-Graeham | | https://62bygw.hippovideo.io/page/graehamw… | 1 | 1 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/redwood-city-real-estate-charming-2bd-1ba-home-tour-expansive-yard | | https://youtu.be/T6PxDSQ0gKw | 1 | 1 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/real-estate-market-update-top-investment-websites | | https://youtu.be/FAHWVteiHSc | 1 | 1 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/east-palo-alto-market-update-free-home-valuation | | https://youtu.be/PdIi0jNDZUM | 1 | 1 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/east-palo-alto-homes-for-sale-2620-fordham | | https://youtu.be/G8QEsqgvEuw | 1 | 1 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/belle-haven-market-update-menlo-park | | https://youtu.be/EK67h0ASaMo | 1 | 1 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/real-estate-market-update-sell-home-top-dollar | | https://youtu.be/ldOkUcZHAF8 | 1 | 1 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/east-palo-alto-real-estate-market-update-2023 | | https://youtu.be/X4v_FmagjpQ | 1 | 1 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/homes-for-sale-in-the-bay-area-just-listed-hot-properties | | https://youtu.be/VM_85hkCqp4 | 1 | 1 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/bay-area-realtor-graeham-watts-staging-and-3d-tours | | https://youtu.be/wjZi8StZPqg | 1 | 1 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/rising-interest-rates-and-real-estate | | https://youtu.be/gqAN-5bmvqA | 1 | 1 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/redwood-city-real-estate-update-sell-high-inventory-rising | | https://youtu.be/-d8go2bi1mY | 1 | 1 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/redwood-city-real-estate-update-2022-home-value | | https://youtu.be/bBy7vxYDYys | 1 | 1 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/Home-buyer-guide | | https://62bygw.hippovideo.io/s/XJWAro6k? W… | 1 | 1 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/houses-in-east-palo-alto-dream-homes-for-every-budget | | https://youtu.be/xkMTxjo70pA | 1 | 1 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/menlo-park-houses-for-sale-monthly-market-update | | https://youtu.be/xpuyGJiAriM | 1 | 1 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/east-palo-alto-homes-for-sale-stylish-loft | | https://youtu.be/1qxzr2_GlGY | 1 | 1 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/ULo7 | | https://drive.google.com/file/d/1R8h7q1tkP… | 1 | 1 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/redwood-city-real-estate-market-update-for-homeowners | | https://youtu.be/j9dH9J_uZx8 | 1 | 1 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/east-palo-alto-california-real-estate-trends | | https://youtu.be/D398slPWpwk | 1 | 1 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/real-estate-market-trends-2025-home-warranty-explained | | https://youtu.be/lg9R8VVEe24 | 1 | 1 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/real-estate-inventory-trends-low-inventory-market-impact | | https://youtu.be/YqmnsBJtGto | 1 | 1 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/east-palo-alto-real-estate-market-update-feb-2025 | | https://youtu.be/UuyT46sxvfk | 1 | 1 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/homes-for-sale-in-the-bay-area-san-leandro-modern-family-home | | https://youtu.be/kVGvjcUKGmI | 1 | 1 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/bay-area-realtor-graeham-watts-client-testimonial | | https://youtu.be/ICsCIS90AZE | 1 | 1 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/homes-for-sale-in-farm-hill-7210-eagle-ridge-dr-gilroy-ca | | https://youtu.be/Bp2yY9AnP9k | 1 | 1 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| hi.switchy.io/KII1 | | https://graehamwatts.com/redwood-city-ca-h… | 1 | 1 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/essential-home-selling-tips-biggest-seller-mistakes | | https://www.youtube.com/watch?v=t5K0mSd9vho | 1 | 1 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/east-palo-alto-ca-homes-for-sale-buyer-tips-1765-e-bayshore-rd-204 | | https://youtu.be/288g3Z5OFO8 | 1 | 1 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/menlo-park-real-estate-market-update-latest-trends-insights | | https://youtu.be/U1qjQsQBjFw | 1 | 1 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/home-selling-tips-fix-now-pay-later | | https://youtube.com/shorts/4t-rdIeqxFo | 1 | 1 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/homes-for-sale-in-redwood-city | | https://youtu.be/A92e3q7mDQg | 1 | 1 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/real-estate-market-forecast-shocker-haunting-2025 | | https://www.youtube.com/shorts/rVrZ9Kp_2u0 | 1 | 1 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/home-selling-tips-out-of-state-overwhelmed-help | | https://www.youtube.com/shorts/trlfRnmnw28 | 1 | 1 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/real-estate-update-home-selling-fast | | https://youtube.com/shorts/uZaeOawttVE | 1 | 1 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/rare-redwood-city-real-estate-fixer-2-bed-condo-hidden-potential | | https://youtu.be/hCYx3eGj6OQ | 1 | 1 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/redwood-city-real-estate-insights | | https://youtu.be/Nrmd49MtMSk | 1 | 1 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/game-changing-real-estate-predictions-2025-big-deals-big-results | | https://youtube.com/shorts/P1YFfUbH7kk | 1 | 1 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/home-selling-mistakes-to-avoid | | https://www.youtube.com/shorts/ACcRPJE3jyU | 1 | 1 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/best-time-to-buy-real-estate-1930-pinole-drive-tour | | https://youtube.com/shorts/hQ0_-D-afYE | 1 | 1 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/ab-1482-unlocked-secrets-every-california-landlord-should-know | | https://youtu.be/ohWO_mG0qiw | 1 | 1 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/scariest-real-estate-costume-ever | | https://www.youtube.com/shorts/2jnSAF2CLnA | 1 | 1 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/redwood-city-real-estate-market-update-prices-up-inventory-down | | https://youtu.be/iq2Ckq89ULA | 1 | 1 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/home-alone-sequel-we-always-wanted-east-palo-alto-homes | | https://youtu.be/j4D7UP38nuw | 1 | 1 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/unlock-hidden-redwood-city-real-estate-ca-gems | | https://youtu.be/NQMWcyCHSs8 | 1 | 1 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/beat-the-market-east-palo-alto-market-update-secret-price-bump | | https://youtu.be/7Uy_68OC1cE | 1 | 1 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/redwood-city-real-estate-renovation-alert | | https://youtu.be/-Nfx05GPn1U | 1 | 1 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/2842-Cornelius-Dr-Tour | | https://youtube.com/shorts/9olaiaSnfuU | 1 | 1 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/Featured-Property-menlo-park | | https://listings.graehamwatts.com/i/menlo-… | 1 | 1 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/Featured-Property-Redwood-City | | https://listings.graehamwatts.com/i/redwoo… | 1 | 1 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/Featured-Property-east-palo-alto | | https://listings.graehamwatts.com/i/east-p… | 1 | 1 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/east-palo-alto-homes-for-sale-1765-bayshore-rd-203 | | https://youtu.be/KiniEUC3W70 | 1 | 1 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/homes-for-sale-in-east-palo-alto-ca-2398-palgas-ave | | https://youtu.be/z7X3VwkBJEw | 1 | 1 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/redwood-city-real-estate-showdown | | https://youtu.be/9bkSH4M6c34 | 1 | 1 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/redwood-city-real-estate-2022-sell-high-find-home | | https://youtu.be/6OCyKKB75hA | 1 | 1 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/east-palo-alto-real-estate-gaillardia-way-property | | https://youtu.be/iEU8OHj9_kg | 1 | 1 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/real-estate-law-101-ca-landlord-entry-laws-explained | | https://youtu.be/y81FHaaVUUY | 1 | 1 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/ab1482-explained-avoid-landlord-mistakes | | https://youtu.be/vGJUhXQvrPM | 1 | 1 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/client-testimonial-kevin-rebecca-bowe | | https://youtu.be/dkLUoarS5HU | 1 | 1 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/east-palo-alto-market-update-2022-inventory-and-prices | | https://youtu.be/ClKCUlHs6DM | 1 | 1 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/east-palo-alto-market-update-2022-sell-for-maximum-value | | https://youtu.be/0rCfTn8FN5Y | 1 | 1 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/real-estate-market-update-city-vs-county-closing-costs | | https://youtu.be/ocfb0l1DJf4 | 1 | 1 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/redwood-city-real-estate-2025-market-shocker-revealed | | https://youtu.be/0cNdrGka8kc | 1 | 1 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/discover-stunning-homes-for-sale-east-palo-alto-ca-graeham-watts-advantage | | https://youtu.be/NA_0CMREfGQ | 1 | 1 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/east-palo-alto-market-update-2022-home-value-increase | | https://youtu.be/0whkr64iX_U | 1 | 1 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/fha-loans-vs-conventional-loans | | https://youtu.be/xVqmEcc4tec | 1 | 1 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/Home-Selling-Tips-Must-Know | | https://62bygw.hippovideo.io/page/graehamw… | 1 | 1 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/price-your-home-right | | https://drive.google.com/file/d/1KB0RsLVMV… | 1 | 1 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/thinking-of-selling-watch-this | | https://drive.google.com/file/d/1c7An667lT… | 1 | 1 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/home-worth-estimate | | https://drive.google.com/file/d/14eimqGKP7… | 1 | 1 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/bay-area-realtor-why-experience-matters | | https://youtu.be/dikIkbHIQB0 | 1 | 1 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/san-mateo-real-estate-update | | https://youtu.be/u8c9mRU66A0 | 1 | 1 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/redwood-city-real-estate-market-update | | https://youtu.be/tOEjIHN-ZfI | 1 | 1 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/east-palo-alto-real-estate-mark-dinan-vision-for-change-city-council-candidate | | https://youtu.be/Y_IG9oKuaUQ | 1 | 1 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/tech-growth-impact-redwood-city-real-estate | | https://youtu.be/Jh9IevCn7tA | 1 | 1 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/interest-rates-and-real-estate-market-impact | | https://youtu.be/9pILbnitgrE | 1 | 1 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/bay-area-realtor-graeham-watts-difference | | https://youtu.be/fff0Xjxl7Zc | 1 | 1 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/east-palo-alto-houses-14-robin-ct-tour | | https://youtu.be/wkYryOjVlcw | 1 | 1 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/unlock-redwood-city-real-estate | | https://youtu.be/msbUaLDXlw0 | 1 | 1 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/real-estate-market-update-22277-hartman-drive-property-tour | | https://youtu.be/APqqzIENZbw | 1 | 1 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/homes-for-sale-redwood-city-ca-staging-secrets | | https://youtu.be/6fNHLeT8nm4 | 1 | 1 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/mt-carmel-redwood-city-homes-for-sale | | https://graehamwatts.com/redwood-city-mt-c… | 1 | 1 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/redwood-city-real-estate-market-update-graeham-watts | | https://youtu.be/eNggnaSP47w | 1 | 1 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/redwood-city-real-estate-hidden-gems-revealed | | https://youtu.be/rpIOXWPsleg | 1 | 1 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/mortgage-costs-redwood-city-real-estate-ca | | https://youtu.be/TcXgG5MRm7Y | 1 | 1 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/discover-affordable-homes-under-100k-across-america | | https://youtu.be/tox0H8d5k40 | 1 | 1 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/redwood-city-ca-real-estate-property-tax | | https://youtu.be/GVtBBvXtlvw | 1 | 1 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/ab-1482-explained-landlords-tenants-must-know | | https://youtu.be/0iQvda-Gfxo | 1 | 1 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/homes-for-sale-in-redwood-city-ca | | https://graehamwatts.com/redwood-city-edge… | 1 | 1 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/redwood-city-real-estate-monthly-market-update | | https://youtu.be/1LZRAjwdW9Y | 1 | 1 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/real-estate-market-forecast-foreclosure-myth-busted | | https://youtu.be/f-JzACJM6pI | 1 | 1 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/roosevelt-redwood-city-homes-for-sale | | https://graehamwatts.com/redwood-city-roos… | 1 | 1 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/homes-for-sale-east-palo-alto-ca-living-guide | | https://youtu.be/_g2hBsYVdPU | 1 | 1 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/redwood-city-real-estate-market-update-feb-2025 | | https://youtu.be/tO0Y55L0Wj0 | 1 | 1 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/redwood-city-real-estate-financing | | https://youtu.be/d0piUVmCUQ8 | 1 | 1 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/essential-home-selling-tips | | https://youtu.be/grWljZdENRs | 1 | 1 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/essential-home-selling-tips-fourth-step-offer-accepted | | https://youtu.be/JtoOgcnJr60 | 1 | 1 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/homes-for-sale-in-redwood-city-ca-590-hurlingame-ave-makeover | | https://youtu.be/q1b0fqiK9g4 | 1 | 1 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/homes-for-sale-in-the-bay-area-3500-19th-st-san-francisco | | https://youtu.be/N3F98VFuqr0 | 1 | 1 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/homes-for-sale-in-east-palo-alto-ca-2288-addison-ave | | https://youtu.be/5YFH0qcKVvQ | 1 | 1 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/homes-for-sale-in-redwood-city-ca-winning-offers-underwriting-edge | | https://youtu.be/VZ1FTIOndqI | 1 | 1 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/homes-for-sale-in-the-bay-area-786-honeywood-court-pending-sale | | https://youtu.be/r5-48v4n9Xc | 1 | 1 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/homes-for-sale-in-the-bay-area-graeham-watts-2022-successes | | https://youtu.be/wesNDJR5d8o | 1 | 1 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/real-estate-market-update-186-overlook-ave-hayward-ca-virtual-tour | | https://youtu.be/uqlYv1-5owM | 1 | 1 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/bay-area-realtor-graeham-watts-client-love | | https://youtu.be/-KsvYprH3qA | 1 | 1 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/ab-1482-explained-maximum-rent-increase-limits-in-california | | https://youtu.be/a-W-1r9UIAc | 1 | 1 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/new-rent-rules-real-estate-predictions-2025-tenants-landlords | | https://youtu.be/KawgQWCbe-I | 0 | 0 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/redwood-city-real-estate-moving-guide-what-you-need-to-know | | https://youtu.be/oM_kYq7BSdU | 0 | 0 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/east-menlo-park-market-update-february-2025 | | https://youtu.be/ZCtyZf01kWg | 0 | 0 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/east-palo-alto-real-estate-moving-guide | | https://youtu.be/1WjY6nKsfvg | 0 | 0 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/home-selling-tips-make-sales-breeze-no-stress | | https://www.youtube.com/shorts/eadSSCmcgaI | 0 | 0 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/east-palo-alto-homes-for-sale | | https://graehamwatts.com/east-palo-alto-ca… | 0 | 0 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/S9z9 | | https://graehamwatts.com/?utm_campaign=Sel… | 0 | 0 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/east-palo-alto-dream-home-tour-30-seconds | | https://www.youtube.com/shorts/LvROobK_n_w | 0 | 0 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/redwood-city-real-estate-hidden-secrets | | https://youtu.be/NQMWcyCHSs8 | 0 | 0 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/real-estate-market-trends-2025-alameda-county-hidden-gem | | https://youtube.com/shorts/xCZfvG6n7J8 | 0 | 0 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/unbelievable-house-tour-market-forecast | | https://youtube.com/shorts/ZX_PDWtqi3E | 0 | 0 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/palm-redwood-city-homes-for-sale | | https://graehamwatts.com/redwood-city-palm | 0 | 0 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/discover-east-palo-alto-real-estate-february-market-update | | https://youtu.be/5q7eX3Ss6HU | 0 | 0 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/homes-for-sale-in-the-bay-area-soquel-3-bedroom | | https://youtu.be/-FAt99sn7uo | 0 | 0 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/exciting-real-estate-market-update-jumping-housing-market | | https://www.youtube.com/shorts/_1xN3UXfWW0 | 0 | 0 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/trusted-bay-area-realtor-house-safety-guarantee | | https://www.youtube.com/shorts/SWsqZtwiuik | 0 | 0 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/redwood-city-real-estate-floor-installation-tips | | https://youtu.be/c-6Npb9MxMs | 0 | 0 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/unexpected-homes-for-sale-bay-area-dream-home | | https://www.youtube.com/shorts/9YUM9fEM0tM | 0 | 0 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/property-tax-shock-2025-real-estate-forecast | | https://www.youtube.com/shorts/zIA-VY9l7S0 | 0 | 0 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/explore-east-palo-alto-homes-graeham-watts | | https://graehamwatts.com/east-palo-alto-ca… | 0 | 0 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| hi.switchy.io/101_Graden_St | | https://graehamwatts.com/101-garden-st | 0 | 0 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/ravenswood-school-district-50m-investment-better-schools-higher-teacher-pay | | https://youtu.be/YcOrP9Lm8D4 | 0 | 0 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/top-tips-for-selling-homes-quickly-maximize-home-value | | https://www.youtube.com/watch?v=oQTZdIt8gSs | 0 | 0 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/rising-interest-rates-and-real-estate-prices | | https://youtu.be/3ZRiArz0EYE | 0 | 0 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/homes-for-sale-east-palo-alto | | https://graehamwatts.com/east-palo-alto-ca… | 0 | 0 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/buy-a-home-east-palo-alto | | https://graehamwatts.com/east-palo-alto-ca… | 0 | 0 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/redwood-village-homes-for-sale-in-redwood-city | | https://graehamwatts.com/redwood-city-redw… | 0 | 0 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/east-palo-alto-market-update-january-2025-key-insights | | https://youtu.be/fo4EUuC_cVQ | 0 | 0 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/real-estate-market-update-hartman-drive-los-altos-ca-virtual-tour | | https://youtu.be/p-PRVqU6jQ4 | 0 | 0 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/bay-area-realtor-secrets-unveiled | | https://www.youtube.com/shorts/YBXqxOZdbxk | 0 | 0 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/east-palo-alto-next-real-estate-goldmine | | https://youtu.be/iiVNOLbYzKs | 0 | 0 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/east-palo-alto-ca-homes-for-sale-house-tour | | https://youtube.com/shorts/8F7y4FAHitA | 0 | 0 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/homes-for-sale-menlo-park-ca-modern-home-tour-1318-hollyburne-avenue | | https://youtu.be/dKct3RK1evQ | 0 | 0 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/graeham-watts-east-palo-alto-listings | | https://graehamwatts.com/east-palo-alto-ca… | 0 | 0 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/real-estate-market-trends-2025-phone-to-realty | | https://youtube.com/shorts/qbK4pNsIsdo | 0 | 0 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/discover-homes-for-sale-east-palo-alto-ca-1560-kavanaugh-drive-tour | | https://youtu.be/u04SeMhpk5g | 0 | 0 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/home-selling-tips-busting-3-buyer-myths | | https://youtu.be/I5cgy1ck4Gc | 0 | 0 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/the-buzz-in-redwood-city-real-estate | | https://youtu.be/MAxSNYhAteM | 0 | 0 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/ab1482-what-landlords-need-to-know-and-do | | https://youtu.be/O_w34dqEt5Y | 0 | 0 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/eagle-hill-redwood-city-homes-for-sale | | https://graehamwatts.com/redwood-city-eagl… | 0 | 0 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/east-menlo-park-homes-for-sale-graeham-watts | | https://graehamwatts.com/east-menlo-park?u… | 0 | 0 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| hi.switchy.io/epa-comps-0601 | postcard,qr,consumer,epa_06_01_26 | https://graehamwatts.com/value?utm_source=… | 0 | 0 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/discover-stunning-homes-for-sale-menlo-park | | https://youtu.be/jvp54Jb7VK4 | 0 | 0 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/east-menlo-park-homes-for-sale-listings-graeham-watts | | https://graehamwatts.com/east-menlo-park?u… | 0 | 0 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/expert-home-selling-tips | | https://youtu.be/rqMMBnK9d8E | 0 | 0 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/bay-area-realtor-free-staging-and-repairs | | https://youtu.be/1lqbMWIOnW8 | 0 | 0 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/ab-1482-rent-increases-exemptions-and-california-rental-laws-explained | | https://youtu.be/b2xwhEpEBu8 | 0 | 0 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/Ace_Your_First_Home_Purchase | | https://62bygw.hippovideo.io/s/PzyGv5bk? W… | 0 | 0 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/homes-for-sale-menlo-park-ca-update | | https://youtu.be/bWzXuOnZJGk | 0 | 0 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/bay-area-home-search-map | | https://listings.graehamwatts.com/idx/map/ | 0 | 0 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/downtown-redwood-city-homes-for-sale | | https://graehamwatts.com/redwood-city-down… | 0 | 0 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/maximize-your-home-sale-value | | https://youtu.be/83RtQ3vqGEo | 0 | 0 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/east-menlo-park-homes-for-sale | | https://graehamwatts.com/east-menlo-park?u… | 0 | 0 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/redwood-city-real-estate-paradise | | https://youtu.be/Rm99sPa28-0 | 0 | 0 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/Redwood-Oaks-homes-for-sale-in-redwood-city-ca | | https://graehamwatts.com/redwood-city-redw… | 0 | 0 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/stambaugh-heller-redwood-city-homes-for-sale | | https://graehamwatts.com/redwood-city-stam… | 0 | 0 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/east-palo-alto-real-estate-housing-market-update | | https://youtu.be/mXjf-uXTt6g | 0 | 0 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/essential-real-estate-market-update-homebuyers-guide | | https://62bygw.hippovideo.io/s/XozaVD9Q? W… | 0 | 0 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/redwood-city-real-estate-housing-market-update | | https://youtu.be/0Yzkc6YSnA0 | 0 | 0 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/real-estate-market-update-2025-california-landlord-risks | | https://youtu.be/DYlXSFnO-5M | 0 | 0 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/central-redwood-city-homes-for-sale | | https://graehamwatts.com/redwood-city-cent… | 0 | 0 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/east-palo-alto-homes-for-sale-woodland-creek-condo | | https://www.youtube.com/watch?v=4z_EXVd2aiw | 0 | 0 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/discovering-redwood-city-real-estate | | https://youtu.be/BHbX0AyHIzM | 0 | 0 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/menlo-park-market-update-belle-haven-housing-trends | | https://youtu.be/Nm6-hBJZBwg | 0 | 0 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/real-estate-market-forecast-fed-interest-rates-explained | | https://youtu.be/3NY7jujU4hM | 0 | 0 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/redwood-city-real-estate-crucial-market-update | | https://youtu.be/j9dH9J_uZx8 | 0 | 0 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/real-estate-market-update-just-listed-1186-overlook-ave-hayward-ca-house-tour | | https://youtu.be/HDqkFGIluiQ | 0 | 0 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/east-palo-alto-real-estate-monthly-market-update | | https://youtu.be/C1m4qdLZILE | 0 | 0 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/centennial-redwood-city-homes-for-sale | | https://graehamwatts.com/redwood-city-cent… | 0 | 0 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/real-estate-market-trends-2025 | | https://youtu.be/CXOle49DNbk | 0 | 0 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/kIhN | | https://graehamwatts.com/listing-detail/11… | 0 | 0 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/best-indian-restaurant-redwood-city-broadway-masala | | https://www.youtube.com/watch?v=igwB8ZVOj7Y | 0 | 0 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/Redwood-City-Homes-on-Sale-Hot-Listings-Price-Drops | | https://graehamwatts.com/redwood-city-home… | 0 | 0 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/Redwood-City-Homes-on-Sale-Hot-Listings-Price-Drops-Graeham-Watts | | https://graehamwatts.com/redwood-city-home… | 0 | 0 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/Redwood-City-Homes-on-Sale | | https://graehamwatts.com/redwood-city-home… | 0 | 0 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/2115-Clarke-Ave | | https://listings.graehamwatts.com/idx/deta… | 0 | 0 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/Redwood-City-Homes-on-Sale-Hot-Listings | | https://graehamwatts.com/redwood-city-home… | 0 | 0 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/redwood-city-real-estate-san-francisco-monthly-market-update | | https://youtu.be/9N4Y58df8QE | 0 | 0 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/essential-real-estate-market-update-guide-for-homebuyers | | https://youtu.be/0_5fp-8gxMU | 0 | 0 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/essential-home-selling-tips-second-step-pre-qualification | | https://youtu.be/860IJbDEfVI | 0 | 0 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/choose-the-right-agent | | https://drive.google.com/file/d/1AeAby_tOh… | 0 | 0 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/ab-1482-explained-is-your-property-rent-controlled-or-exempt | | https://youtu.be/mxpvotOcARY | 0 | 0 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/stunning-homes-for-sale-menlo-park-1135-madera-ave-tour | | https://youtu.be/HMIJcWF2VLo | 0 | 0 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/redwood-city-spanish-style-4-bedroom-friendly-acres | | https://www.youtube.com/shorts/s5ANgnt78Yo | 0 | 0 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/east-palo-alto-real-estate-market-update | | https://youtu.be/d7vXrMHu6Xc | 0 | 0 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/real-estate-market-update-foundation-tips-when-buying-your-new-home | | https://youtu.be/Alxmfb7BHjg | 0 | 0 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/real-estate-market-update-tips-to-protect-your-offer-and-investment | | https://youtu.be/C8QcRE9OMJk | 0 | 0 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/east-palo-alto-real-estate-mark-dinan-vision-change | | https://www.youtube.com/shorts/05Q6-V_58jw | 0 | 0 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/what-you-can-buy-redwood-city-houses-for-sale | | https://www.youtube.com/shorts/s2LZKuUx8Jc | 0 | 0 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/redwood-city-real-estate-market-february-update-you-need-to-know | | https://youtu.be/uWGng3F1glA | 0 | 0 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| pages.graehamwatts.com/selling-your-home-just-got-easier | | https://youtube.com/shorts/cTYXi2SH3gc | 0 | 0 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| **TOTAL** | | | **15,997** | **8,855** | **$1,397** | | + +**How to read this:** *Audience* = clicks that resolve to a targetable pixeled user. *Monthly budget* is what it costs to hit that audience 10x over 30 days at $22 CPM — i.e. the spend the audience can actually absorb, not a target. Audiences under 100 can't be targeted; under 1,000 should be merged by source. \ No newline at end of file diff --git a/scheduled-tasks/README.md b/scheduled-tasks/README.md new file mode 100755 index 00000000..cb8efb0a --- /dev/null +++ b/scheduled-tasks/README.md @@ -0,0 +1,20 @@ +# Scheduled Tasks + +This folder holds the canonical, version-controlled copy of Cowork scheduled-task SKILL.md files. Cowork itself reads them from `~/Documents/Claude/Scheduled//SKILL.md` on each machine, so when one of these files changes here in the repo, the next step is to copy it onto each machine that runs the task. + +## Sync onto the Mac Studio + +```bash +cd ~/Documents/Claude/Skills +git pull origin main +# Copy any updated scheduled task SKILL.md files into Cowork's scheduled folder +rsync -a scheduled-tasks/ ~/Documents/Claude/Scheduled/ +``` + +That `rsync -a` copies the contents of `scheduled-tasks//SKILL.md` into `~/Documents/Claude/Scheduled//SKILL.md` while preserving the folder structure. Run it any time after pulling. + +## Tasks tracked here + +| Task | Cadence | What it does | +|---|---|---| +| pcfs-cma-autobuild-weekly | Mon 9:21am PT | Builds past-client CMA value-update reports for clients due in next 7 days; sends review emails to Graeham + Adrian (changed from drafts → sends 2026-05-26) | diff --git a/scheduled-tasks/pcfs-cma-autobuild-weekly/SKILL.md b/scheduled-tasks/pcfs-cma-autobuild-weekly/SKILL.md new file mode 100755 index 00000000..81f77136 --- /dev/null +++ b/scheduled-tasks/pcfs-cma-autobuild-weekly/SKILL.md @@ -0,0 +1,73 @@ +--- +name: pcfs-cma-autobuild-weekly +description: Weekly: build past-client CMA value-update reports for clients due in the next 7 days and send review emails to Graeham + Adrian (direct send — no longer drafts, because drafts get lost). +--- + +You are running the weekly PCFS CMA auto-build for Graeham Watts (REALTOR, Compass, DRE #01466876, 650-308-4727, graehamwatts@gmail.com). This produces PAST-CLIENT home-value update CMAs and SENDS review emails directly to Graeham + Adrian. NOTHING is auto-sent to clients — you send the review email ONLY to graehamwatts@gmail.com + graehamwattsclientcare@gmail.com, and they manually forward the bottom (client-facing) section to the actual past client after reviewing. + +IMPORTANT DELIVERY CHANGE (2026-05-26): previously this task created Gmail DRAFTS. Drafts got lost in the drafts folder. We now SEND the review emails directly to Graeham + Adrian's inboxes so they show up where they'll actually be seen. Same content, same two-section format with a divider — just sent instead of drafted. + +STEP 1 — Get the due list. +Fetch the due CMAs from this n8n endpoint (7-day window): https://n8n.graehamwattsn8n.com/webhook/cma-due-list?days=7 +Try mcp__workspace__web_fetch first; if that is blocked, use Claude in Chrome (navigate a tab to that URL and read the JSON body). The response is JSON: { count, due_cmas: [ { client_name, email, property_address, due_date, contact_id, last_cma_sent } ] }. +Dedup: read the local log at the workspace path "Online Content/cma/_autobuild_log.json" (create if missing). Skip any {client_name + due_date} already logged. Only process NEW ones. +If a due_cma has a BLANK property_address, do NOT guess — add it to a "needs address" list to report to Graeham, and skip building it. + +STEP 2 — For each client to process, build the CMA. +First detect MLS login: select Graeham's Mac Studio Chrome (mcp__Claude_in_Chrome__list_connected_browsers then select the macOS 'chrome'), open a tab to https://search.mlslistings.com/Matrix/Search/Residential/ResidentialSearch?f= . If it redirects to login.aspx, MLS is LOGGED OUT. + - If LOGGED IN: Use the cma-generator skill methodology. Pull the subject specs from Realist (REALIST tab → search the address), then pull SOLD comps: same city, Single Family Home, SqFt within ~250 of subject, Close Of Escrow date in the last 6 months. Capture ~10-20 comps with sold price, $/sqft, sqft, beds/baths, lot, age, DOM. MLSListings carries East Bay via reciprocal share, so Contra Costa/Alameda addresses work too. + - If LOGGED OUT: per Graeham's instruction, build from PUBLIC data, but HARD-FLAG every figure as a lower-confidence estimate (state clearly in the report and email that MLS was unavailable and numbers should be verified). Also include in your final report-to-Graeham a note: "MLS was logged out — flagged public-data estimates used; log in for full-confidence versions." + Compute all statistics in Python (mcp__workspace__bash) for accuracy — never eyeball math. + +STEP 3 — PAST-CLIENT VERBIAGE (critical — this is an owner's value update, NOT a listing presentation). + +⚠️ MANDATORY CHECKLIST CHECK (added 2026-05-26): Before writing a single line of HTML, READ `skills/cma-generator/references/past_client_mode.md` MANDATORY CHECKLIST section in full. Every item on that checklist must appear in your published HTML — all five Chart.js charts (trendPrice, trendLS, priceJourney, domVsCut, priceDom), every comp-table column (especially Original List, # Reductions, $-cut, List-to-Sale %), the Interest Rate Environment 4-source section, the branded nav, and zero em-dashes. The May 25 autobuild outputs (Ravi Indurkar, Viduishi Jain, Narasimha Subraveti) skipped 4 of the 5 charts, the Interest Rate section, the extra comp columns, the nav bar, and were riddled with em-dashes — that pattern is what this checklist exists to prevent. If you cannot produce a checklist item from available data (e.g., MLS history isn't reachable for Original List), state that explicitly in the report rather than silently omitting the column. + +Build the report with the cma-generator's premium branded HTML (black #1A1A1A / gold #C5A55A, the graehamwatts.com nav, Chart.js charts), BUT the language must read as a friendly update to someone who already OWNS the home: + - Hero label: "HOME VALUE UPDATE". + - DO NOT include a "Pricing Strategy" section with list-below/at/above-market advice — that is seller-listing language. Instead, a section titled "WHAT YOUR HOME IS WORTH TODAY" presenting a current market-value range framed as the owner's equity/standing, not a list price. + - Replace "Conservative / Competitive / Stretch list price" labels with value-range framing like "Likely range / Most-likely value / Top of range in strong condition." + - Tone: warm, no-agenda, "as your agent I like to keep you posted on where you stand." If the purchase price/date is known, show the equity gain since they bought. + - Keep: subject summary, the market story, comparable sales tables + $/sqft chart, market data, the value range, and honest notes (condition caveats, data source). Avoid any "let's sell / let's list" push. +Run the report through the humanizer skill before finalizing. Verify the math and comp accuracy as a QC pass. + +STEP 4 — Publish. +Save the HTML as CMA_[street_number]_[street_name_underscored].html and publish to Graeham's online-content repo at paths cma/, cma-reports/, and cmas/ via the GitHub Contents API using the token in "Online Content/github-token.txt" (classic token, repo Graehamwatts/online-content). Use the browser (example.com origin) compress→chunk→decompress→PUT method since the sandbox proxy blocks api.github.com. Live URL: https://graehamwatts.github.io/online-content/cma/CMA_[address].html . Also copy the file into the local "Online Content/cma" (and cma-reports, cmas) folders. + +STEP 5 — Send the review email (Gmail) — NOT a draft. +SEND ONE Gmail message per client directly to Graeham + Adrian. Use any send action available on the Gmail MCP (mcp__69816e67-52bb-4259-b487-681f474d6ef0) — do NOT use create_draft. If only create_draft is available, use it to compose then immediately send the resulting draft so the email lands in the inboxes (not Drafts). + + to: ["graehamwatts@gmail.com","graehamwattsclientcare@gmail.com"] + subject: "[REVIEW → forward] CMA ready: [property address] — [client name]" + + Format the body with TWO clearly-separated sections divided by an obvious "delete above this line" marker: + + ━━━━━━━━━━ INTERNAL NOTE (delete this whole section before forwarding) ━━━━━━━━━━ + + 📧 FORWARD TO: [client email address] ← this is where Graeham/Adrian sends the bottom half + 👤 Client: [client full name] + 🏡 Property: [property address] + 📅 CMA due: [due_date] + 💰 Most-likely value: $[value] | Range: $[low] – $[high] + 📊 Median $/sqft: $[median] (from [N] comps) + 🔗 Live CMA: [live_url] + 🗂️ Data source: [MLS-FULL OR PUBLIC-FALLBACK — if public, lower confidence; recommend re-run when MLS is logged in] + + Quick QC notes: [any caveats — comp quality, condition flags, equity gain math, anomalies] + + ⬇️⬇️⬇️ DELETE EVERYTHING ABOVE THIS LINE BEFORE FORWARDING TO [client email] ⬇️⬇️⬇️ + ════════════════════════════════════════════════════════════════════════════════ + ⬆️⬆️⬆️ EVERYTHING BELOW IS THE FORWARD-READY CLIENT EMAIL ⬆️⬆️⬆️ + + Suggested subject: 🔥 [warm no-agenda subject — e.g. "A quick update on your [city] home" or "Here's what your home is worth right now"] + + [Body: warm past-client greeting by first name ("Hi [first name],"), 2–3 short paragraphs framing this as a no-agenda value update, the value range stated plainly, the clickable live CMA link, friendly close, Graeham's signature with DRE# and phone.] + +Provide both plain text `body` and styled `htmlBody`. In the HTML version, render the divider as a real styled
block with the "DELETE EVERYTHING ABOVE" text inside a colored banner (red/orange) so it's impossible to miss. The email IS sent immediately — it lands in Graeham's inbox + Adrian's clientcare inbox. They review the top section for accuracy, then copy/forward the bottom section to the client (deleting the top half first). + +After a client is sent, append {client_name, client_email, due_date, property_address, live_url, sent_at, data_source} to "Online Content/cma/_autobuild_log.json". + +STEP 6 — Report back. +Summarize for Graeham: which CMAs were built + drafted (with live links), which were skipped for missing addresses, and whether MLS was logged in or the public-data fallback was used. Keep it concise. + +GUARDRAILS: Never auto-send client email. Never enter or submit Graeham's MLS password — if logged out, use the public-data fallback (flagged) and note it. Do not modify any of the 7 live PCFS cadence workflows. If the Mac Chrome isn't connected, report that and stop. diff --git a/scripts/AUDIT_REPORT.md b/scripts/AUDIT_REPORT.md new file mode 100755 index 00000000..c8a7c341 --- /dev/null +++ b/scripts/AUDIT_REPORT.md @@ -0,0 +1,41 @@ +# Skills Audit Report +**Generated:** 2026-05-14 +**Audited folder:** C:\Users\Graeham Watts\Documents\Skills LLMS\Claude\Skills\ + +## Note on terminology +The original prompt used "DRV"; this audit treats it as **DRE** (California Department of Real Estate license number), matching the canonical field defined in `skills/shared-references/identity.json`. + +## Canonical values (from identity.json) +- **DRE:** `01466876` (correct, current) +- **Blocklisted:** `02015066` (the recurring zombie - must never appear except in documentation-exempt files) +- **Doc-exempt files** (legitimately reference the blocklisted value to enforce policy): `CLAUDE.md`, `skills/shared-references/identity.json`, `scripts/verify_brand_identity.py` + +## Skill folder inventory +- Total skill folders under `skills/`: 45 +- All have valid `SKILL.md`: yes +- Deprecated skills (per CLAUDE.md policy) present: NONE - `video-script-creation-engine`, `social-media-analyzer`, `video-prompt-builder`, `html-email`, `github-skill-sync` all already removed +- `_backup` / `_archive` / `old` / `deprecated` folders inside Skills/: none found + +## DRE occurrences scan +| File | Status | +|---|---| +| `Skills/CLAUDE.md` lines 15, 73 | DOC-EXEMPT (policy warning) | +| `Skills/skills/shared-references/identity.json` lines 27, 30, 37 | DOC-EXEMPT (blocklist + audit history) | +| All other content files | CLEAN (0 occurrences) | + +## Stray `.skill` bundles outside Skills/ +| File | Disposition | +|---|---| +| `Documents/Skills LLMS/Claude/weekly-listing-update.skill` | ZOMBIE - canonical version is at `Skills/skills/weekly-listing-update/`. Targeted for removal. | + +## Recommended canonical version per skill type +All 45 skills under `skills/` are unique; no duplicates. Each folder's own `SKILL.md` is the canonical version. + +## Cleanup actions taken in this session +- Patched DRE 02015066 -> 01466876 in: + - `skills/content-calendar/templates/main-dashboard-builder.py` + - `skills/shared-references/publishing-via-composio.md` + - `skills/watts-motion-graphics/references/standing-rules.md` + - `skills/contract-estimate-builder/SKILL.md` + - `Online Content/dashboards/attribution/2026-05-12-daily.html` +- Pushed cleaned state to `Graehamwatts/skills` main (commit `384b595`) diff --git a/scripts/claude_with_cache.py b/scripts/claude_with_cache.py new file mode 100755 index 00000000..d4f03f4f --- /dev/null +++ b/scripts/claude_with_cache.py @@ -0,0 +1,142 @@ +#!/usr/bin/env python3 +"""claude_with_cache.py - Single entry point for Claude API calls with +server-side prompt caching of the Skills bundle. + +Behavior: + - On init, reads every SKILL.md from Skills/skills// and joins them + into system prompt blocks (one block per skill). + - Marks the LAST block with cache_control: {"type": "ephemeral"} so + Anthropic caches the whole skills bundle as one entry. + - Exposes send_message(user_input, model=DEFAULT_MODEL). + - Logs CACHE WRITE on first call, CACHE HIT on subsequent calls within + 5 minutes (driven by the response's cache_read_input_tokens vs. + cache_creation_input_tokens fields). + +Usage: + from claude_with_cache import send_message + text = send_message("Hello, give me a 1-line market summary.") +""" +import datetime +import json +import os +import sys +from pathlib import Path + +try: + import anthropic +except ImportError: + print("ERROR: pip install anthropic", file=sys.stderr) + sys.exit(1) + +SKILLS_DIR = Path(r"C:\Users\Graeham Watts\Documents\Skills LLMS\Claude\Skills\skills") +DEFAULT_MODEL = "claude-sonnet-4-6" + +_CLIENT = None +_SYSTEM_BLOCKS = None + + +def _load_skill_blocks(): + """Build a list of system blocks, one per skill, plus a small header + block. The last block gets cache_control.""" + blocks = [] + header = ( + "You have access to Graeham Watts's skill toolkit. Each block below " + "is one skill's SKILL.md. The blocks are immutable - they describe " + "tools available to you. Read identity.json (referenced inside the " + "skills) for canonical brand details (name, DRE, etc.) - never type " + "those values from memory." + ) + blocks.append({"type": "text", "text": header}) + + if not SKILLS_DIR.exists(): + return blocks + + for sub in sorted(SKILLS_DIR.iterdir()): + if not sub.is_dir(): + continue + sm = sub / "SKILL.md" + if not sm.exists(): + continue + try: + content = sm.read_text(encoding="utf-8") + blocks.append({"type": "text", "text": f"### Skill: {sub.name}\n\n{content}"}) + except Exception as e: + print(f"WARN: could not read {sm}: {e}", file=sys.stderr) + + # Mark the LAST block with ephemeral cache_control so the whole prefix + # is cached as a single cache entry on Anthropic's side. + if blocks: + blocks[-1]["cache_control"] = {"type": "ephemeral"} + return blocks + + +def _client(): + global _CLIENT + if _CLIENT is None: + api_key = os.environ.get("ANTHROPIC_API_KEY") + if not api_key: + raise RuntimeError("ANTHROPIC_API_KEY not set") + _CLIENT = anthropic.Anthropic(api_key=api_key) + return _CLIENT + + +def _system(): + global _SYSTEM_BLOCKS + if _SYSTEM_BLOCKS is None: + _SYSTEM_BLOCKS = _load_skill_blocks() + return _SYSTEM_BLOCKS + + +def _log_cache(usage): + """Print one of: CACHE WRITE | CACHE HIT | CACHE MISS based on usage.""" + write = getattr(usage, "cache_creation_input_tokens", 0) or 0 + hit = getattr(usage, "cache_read_input_tokens", 0) or 0 + if write and not hit: + print(f"CACHE WRITE: {write} tokens written to cache") + elif hit: + print(f"CACHE HIT: {hit} tokens read from cache (saved)") + else: + print("CACHE MISS: no cache_read / cache_creation tokens reported") + + +def send_message(user_input: str, model: str = DEFAULT_MODEL, max_tokens: int = 1024, + stream: bool = False): + """Send one message with the cached Skills system prompt. + + Returns the assistant text. For more advanced use (tools, multi-turn), + call _client() directly with _system() as the system parameter. + """ + client = _client() + sys_blocks = _system() + + resp = client.messages.create( + model=model, + max_tokens=max_tokens, + system=sys_blocks, + messages=[{"role": "user", "content": user_input}], + ) + _log_cache(resp.usage) + + # Collect text from content blocks + parts = [] + for blk in resp.content: + t = getattr(blk, "text", None) + if t: + parts.append(t) + return "".join(parts) + + +def prewarm(): + """One-shot call to write the Skills bundle into cache. Smallest possible + real request - max_tokens=1 because Anthropic requires >= 1.""" + print("Pre-warming Skills cache...") + out = send_message("Reply with the single word: OK", max_tokens=4) + print(f" prewarm output: {out.strip()[:40]}") + + +if __name__ == "__main__": + if len(sys.argv) > 1 and sys.argv[1] == "prewarm": + prewarm() + else: + msg = " ".join(sys.argv[1:]) or "Reply with the single word: OK" + print(send_message(msg)) diff --git a/scripts/cleanup_skills.py b/scripts/cleanup_skills.py new file mode 100755 index 00000000..42b012e6 --- /dev/null +++ b/scripts/cleanup_skills.py @@ -0,0 +1,209 @@ +#!/usr/bin/env python3 +"""cleanup_skills.py - Remove zombie skill files and patch wrong DRE values. + +CANONICAL_DRE is the correct value. Edit this constant if it ever changes +(should also be reflected in skills/shared-references/identity.json). + +Safe by default: +- Creates a timestamped backup of every file BEFORE deletion or modification +- Prompts for YES confirmation before any destructive operation +- Skips documentation-exempt files (CLAUDE.md, identity.json, this script's + own definition list) so the policy warnings don't get neutered +""" +import argparse +import datetime +import re +import shutil +import sys +from pathlib import Path + +# ---- Canonical values (mirror of identity.json) ----------------------------- +CANONICAL_DRE = "01466876" # The correct current DRE +BLOCKED_DRES = ["02015066"] # Values to scrub out + +# ---- Paths ------------------------------------------------------------------ +SKILLS_ROOT = Path(r"C:\Users\Graeham Watts\Documents\Skills LLMS\Claude\Skills") +BACKUP_ROOT = SKILLS_ROOT / "_backup" + +# Files that may legitimately reference blocked values (policy/enforcement) +DOC_EXEMPT = { + "CLAUDE.md", + "identity.json", + "verify_brand_identity.py", + "cleanup_skills.py", # this script itself +} + +# Zombie bundles to remove (outside the canonical Skills/ folder) +EXTERNAL_ZOMBIES = [ + Path(r"C:\Users\Graeham Watts\Documents\Skills LLMS\Claude\weekly-listing-update.skill"), +] + + +def find_dre_violations(root: Path): + """Return list of (path, line_no, line) where a blocked DRE appears + in a file that is NOT in the doc-exempt allowlist.""" + hits = [] + exts = {".md", ".txt", ".json", ".py", ".html", ".yml", ".yaml"} + for p in root.rglob("*"): + if not p.is_file() or p.suffix.lower() not in exts: + continue + if any(seg in (".git", "_backup") for seg in p.parts): + continue + if p.name in DOC_EXEMPT: + continue + try: + for i, line in enumerate(p.read_text(encoding="utf-8", errors="ignore").splitlines(), start=1): + for bad in BLOCKED_DRES: + if bad in line: + hits.append((p, i, line.rstrip(), bad)) + except Exception as e: + print(f" WARN: could not read {p}: {e}", file=sys.stderr) + return hits + + +def find_zombie_duplicates(root: Path): + """Detect duplicate skill folders. + + A skill is identified by its folder name under skills/. A 'zombie' + is a folder that matches a deprecation list OR another folder of the + same canonical name. + """ + skills_dir = root / "skills" + if not skills_dir.exists(): + return [] + deprecated_names = { + "video-script-creation-engine", + "social-media-analyzer", + "video-prompt-builder", + "html-email", + "github-skill-sync", + } + zombies = [] + for sub in skills_dir.iterdir(): + if sub.is_dir() and sub.name in deprecated_names: + zombies.append(sub) + return zombies + + +def backup_file(p: Path, ts: str): + rel = p.relative_to(SKILLS_ROOT) if str(p).startswith(str(SKILLS_ROOT)) else Path(p.name) + dest = BACKUP_ROOT / ts / rel + dest.parent.mkdir(parents=True, exist_ok=True) + if p.is_dir(): + shutil.copytree(p, dest, dirs_exist_ok=True) + else: + shutil.copy2(p, dest) + + +def patch_dre(p: Path) -> int: + """Replace any blocked DRE with the canonical one. Returns count of replacements.""" + txt = p.read_text(encoding="utf-8", errors="ignore") + n = 0 + for bad in BLOCKED_DRES: + if bad in txt: + n += txt.count(bad) + txt = txt.replace(bad, CANONICAL_DRE) + if n: + p.write_text(txt, encoding="utf-8") + return n + + +def main(): + parser = argparse.ArgumentParser(description="Clean up zombie skills and patch DRE.") + parser.add_argument("--yes", action="store_true", + help="Skip the YES prompt (only for automation)") + args = parser.parse_args() + + print("=" * 70) + print(f"SKILLS CLEANUP - {datetime.datetime.now().isoformat(timespec='seconds')}") + print(f"Canonical DRE: {CANONICAL_DRE}") + print(f"Blocked DRE(s): {', '.join(BLOCKED_DRES)}") + print("=" * 70) + + if not SKILLS_ROOT.exists(): + print(f"ERROR: {SKILLS_ROOT} does not exist.", file=sys.stderr) + sys.exit(1) + + # Stage 1: find DRE violations + print("\n[1/3] Scanning for blocked DRE values in content files...") + violations = find_dre_violations(SKILLS_ROOT) + if violations: + for p, ln, line, bad in violations: + print(f" {p.relative_to(SKILLS_ROOT)}:{ln} ({bad})") + else: + print(" No violations.") + + # Stage 2: find zombie skill folders + print("\n[2/3] Scanning for deprecated/duplicate skill folders...") + zombies = find_zombie_duplicates(SKILLS_ROOT) + if zombies: + for z in zombies: + print(f" ZOMBIE: {z}") + else: + print(" None.") + + # Stage 3: external stray bundles + print("\n[3/3] Checking for stray .skill bundles outside Skills/...") + strays = [p for p in EXTERNAL_ZOMBIES if p.exists()] + if strays: + for s in strays: + print(f" STRAY: {s}") + else: + print(" None.") + + files_to_patch = sorted({v[0] for v in violations}) + if not files_to_patch and not zombies and not strays: + print("\nNothing to do. Exiting clean.") + return + + print("\n" + "=" * 70) + print("PLANNED ACTIONS:") + print(f" - Patch DRE in {len(files_to_patch)} file(s)") + print(f" - Delete {len(zombies)} zombie skill folder(s)") + print(f" - Delete {len(strays)} stray .skill bundle(s)") + print(f" - Backup destination: {BACKUP_ROOT}") + print("=" * 70) + + if not args.yes: + ans = input("Type YES to proceed (anything else aborts): ").strip() + if ans != "YES": + print("Aborted.") + return + + ts = datetime.datetime.now().strftime("%Y%m%d-%H%M%S") + BACKUP_ROOT.mkdir(parents=True, exist_ok=True) + + # Backup + patch DRE files + for p in files_to_patch: + backup_file(p, ts) + n = patch_dre(p) + print(f" patched ({n}x): {p.relative_to(SKILLS_ROOT)}") + + # Backup + remove zombies + for z in zombies: + backup_file(z, ts) + shutil.rmtree(z) + print(f" removed: {z}") + + for s in strays: + backup_file(s, ts) + s.unlink() + print(f" removed: {s}") + + # Final manifest + print("\n" + "=" * 70) + print("SURVIVING SKILL MANIFEST") + print("=" * 70) + skills_dir = SKILLS_ROOT / "skills" + for sub in sorted(skills_dir.iterdir()): + if not sub.is_dir(): + continue + sm = sub / "SKILL.md" + if sm.exists(): + mtime = datetime.datetime.fromtimestamp(sm.stat().st_mtime).isoformat(timespec="seconds") + print(f" {sub.name:35s} {mtime} {sm}") + print(f"\nBackup saved to: {BACKUP_ROOT / ts}") + + +if __name__ == "__main__": + main() diff --git a/scripts/master_reset.py b/scripts/master_reset.py new file mode 100755 index 00000000..1472e603 --- /dev/null +++ b/scripts/master_reset.py @@ -0,0 +1,102 @@ +#!/usr/bin/env python3 +"""master_reset.py - One-command orchestrator. + +Runs in sequence: + 1. cleanup_skills.py (gated by YES confirmation unless --yes passed) + 2. sync_skills.py + 3. claude_with_cache.py prewarm (writes Skills bundle into Anthropic's + server-side cache so the first real user request hits cache) + +Final report prints zombie removals, sync counts, and estimated token +savings per session. +""" +import argparse +import datetime +import subprocess +import sys +from pathlib import Path + +HERE = Path(__file__).resolve().parent + + +def run(cmd): + print(f"\n>>> {' '.join(cmd)}") + r = subprocess.run(cmd, cwd=str(HERE)) + return r.returncode + + +def estimate_token_savings(): + """Rough estimate of cached tokens. Counts ~chars/4 across all SKILL.md.""" + skills_dir = HERE.parent / "skills" + total_chars = 0 + n = 0 + if skills_dir.exists(): + for sub in skills_dir.iterdir(): + sm = sub / "SKILL.md" + if sm.exists(): + try: + total_chars += len(sm.read_text(encoding="utf-8")) + n += 1 + except Exception: + pass + tokens = total_chars // 4 + return n, tokens + + +def main(): + ap = argparse.ArgumentParser() + ap.add_argument("--yes", action="store_true", + help="Skip cleanup confirmation prompt") + ap.add_argument("--skip-cleanup", action="store_true") + ap.add_argument("--skip-sync", action="store_true") + ap.add_argument("--skip-prewarm", action="store_true") + args = ap.parse_args() + + print("=" * 70) + print(f"MASTER RESET - {datetime.datetime.now().isoformat(timespec='seconds')}") + print("=" * 70) + + zombie_removals = 0 + sync_summary = "skipped" + + if not args.skip_cleanup: + cleanup_cmd = [sys.executable, "cleanup_skills.py"] + if args.yes: + cleanup_cmd.append("--yes") + rc = run(cleanup_cmd) + if rc != 0: + print("Cleanup aborted - stopping.") + sys.exit(rc) + # Count what got moved into _backup as a proxy for zombie removals + backup_root = HERE.parent / "_backup" + if backup_root.exists(): + latest = sorted(backup_root.iterdir()) + if latest: + zombie_removals = sum(1 for _ in latest[-1].rglob("*") if _.is_file()) + + if not args.skip_sync: + rc = run([sys.executable, "sync_skills.py"]) + sync_summary = "ran (see sync_log.txt)" if rc == 0 else f"errors (rc={rc})" + + if not args.skip_prewarm: + rc = run([sys.executable, "claude_with_cache.py", "prewarm"]) + if rc != 0: + print("Pre-warm failed (likely missing ANTHROPIC_API_KEY).") + + n_skills, est_tokens = estimate_token_savings() + + print("\n" + "=" * 70) + print("MASTER RESET COMPLETE") + print("=" * 70) + print(f" Files moved to _backup (proxy for zombies removed): {zombie_removals}") + print(f" Sync step: {sync_summary}") + print(f" Skills bundled into cache: {n_skills}") + print(f" Estimated cached tokens per session: ~{est_tokens:,}") + print(f" Estimated savings: cache reads cost ~10% of base " + f"input tokens, so each cached session saves roughly " + f"~{int(est_tokens * 0.9):,} billable input tokens.") + print("=" * 70) + + +if __name__ == "__main__": + main() diff --git a/scripts/pre-commit-mass-delete-guard.sh b/scripts/pre-commit-mass-delete-guard.sh new file mode 100644 index 00000000..345569ef --- /dev/null +++ b/scripts/pre-commit-mass-delete-guard.sh @@ -0,0 +1,41 @@ +#!/bin/sh +# Mass-deletion guard — pre-commit hook for the skills repo. +# +# Why: on 2026-06-09 an automated "Auto-sync: session-end" commit deleted 64 +# files (10 entire skills + shared-references/identity.json) because the +# Documents working copy was stale relative to a sandbox-clone push. This hook +# blocks any commit that stages a suspicious number of deletions so that an +# auto-sync against a stale tree fails loudly instead of silently wiping work. +# +# Install (already done on Graeham's machine, repeat after a fresh clone): +# cp scripts/pre-commit-mass-delete-guard.sh .git/hooks/pre-commit +# +# To intentionally delete many files in one commit, run with the override: +# ALLOW_MASS_DELETE=1 git commit -m "..." + +LIMIT=5 + +if [ "$ALLOW_MASS_DELETE" = "1" ]; then + exit 0 +fi + +deletions=$(git diff --cached --diff-filter=D --name-only | wc -l | tr -d ' ') + +if [ "$deletions" -gt "$LIMIT" ]; then + echo "BLOCKED: this commit deletes $deletions files (limit $LIMIT)." >&2 + echo "If the working copy is stale (sandbox push not pulled yet), run:" >&2 + echo " git pull origin main" >&2 + echo "and re-check. To force a genuine mass deletion:" >&2 + echo " ALLOW_MASS_DELETE=1 git commit ..." >&2 + git diff --cached --diff-filter=D --name-only | head -10 >&2 + exit 1 +fi + +# Never allow deleting the brand source of truth, even under the limit. +if git diff --cached --diff-filter=D --name-only | grep -q "shared-references/identity.json"; then + echo "BLOCKED: commit deletes skills/shared-references/identity.json (brand source of truth)." >&2 + echo "Use ALLOW_MASS_DELETE=1 only if this is truly intentional." >&2 + exit 1 +fi + +exit 0 diff --git a/scripts/sync_skills.py b/scripts/sync_skills.py new file mode 100755 index 00000000..b996c227 --- /dev/null +++ b/scripts/sync_skills.py @@ -0,0 +1,181 @@ +#!/usr/bin/env python3 +"""sync_skills.py - Sync local skills to the Anthropic Console + git push. + +Behavior: + 1. Iterate every SKILL.md under Skills/skills// on the canonical disk + 2. Upload or update the matching skill via the Anthropic Skills API + 3. git add + commit + push the whole Skills repo + 4. Log every action with a timestamp to sync_log.txt + 5. Print a final summary + +Auth: + - ANTHROPIC_API_KEY must be set in the environment + - For git push, the PAT lives at Skills/github-token.txt (gitignored) + +Note: The Anthropic Skills Console API surface may evolve. This script uses +the /v1/skills HTTP endpoints (PATCH/POST/GET). If your account uses a +different surface, swap the SKILLS_LIST / SKILLS_UPSERT helpers. +""" +import datetime +import os +import subprocess +import sys +from pathlib import Path + +try: + import requests +except ImportError: + print("ERROR: install requests first: pip install requests", file=sys.stderr) + sys.exit(1) + +SKILLS_ROOT = Path(r"C:\Users\Graeham Watts\Documents\Skills LLMS\Claude\Skills") +SKILLS_DIR = SKILLS_ROOT / "skills" +LOG_PATH = SKILLS_ROOT / "scripts" / "sync_log.txt" +GIT_TOKEN_FILE = SKILLS_ROOT / "github-token.txt" +GIT_REMOTE = "https://github.com/Graehamwatts/skills.git" + +ANTHROPIC_BASE = "https://api.anthropic.com" +ANTHROPIC_VERSION = "2023-06-01" + + +def log(msg: str): + ts = datetime.datetime.now().isoformat(timespec="seconds") + line = f"[{ts}] {msg}" + print(line) + LOG_PATH.parent.mkdir(parents=True, exist_ok=True) + with open(LOG_PATH, "a", encoding="utf-8") as f: + f.write(line + "\n") + + +def api_headers(): + key = os.environ.get("ANTHROPIC_API_KEY") + if not key: + log("FATAL: ANTHROPIC_API_KEY not set") + sys.exit(2) + return { + "x-api-key": key, + "anthropic-version": ANTHROPIC_VERSION, + "content-type": "application/json", + } + + +def list_console_skills(): + r = requests.get(f"{ANTHROPIC_BASE}/v1/skills", headers=api_headers(), timeout=30) + if r.status_code == 404: + log("Skills API returned 404 - Skills surface may not be enabled on this account.") + return {} + r.raise_for_status() + data = r.json() + by_name = {} + for s in data.get("data", data.get("skills", [])): + name = s.get("name") or s.get("display_name") + if name: + by_name[name] = s + return by_name + + +def upsert_skill(name: str, content: str, existing_id: str | None): + payload = {"name": name, "display_name": name, "instructions": content} + if existing_id: + url = f"{ANTHROPIC_BASE}/v1/skills/{existing_id}" + r = requests.patch(url, headers=api_headers(), json=payload, timeout=60) + return "updated", r + url = f"{ANTHROPIC_BASE}/v1/skills" + r = requests.post(url, headers=api_headers(), json=payload, timeout=60) + return "created", r + + +def read_skill_files(): + out = [] + for sub in sorted(SKILLS_DIR.iterdir()) if SKILLS_DIR.exists() else []: + if not sub.is_dir(): + continue + sm = sub / "SKILL.md" + if sm.exists(): + try: + out.append((sub.name, sm, sm.read_text(encoding="utf-8"))) + except Exception as e: + log(f"WARN cannot read {sm}: {e}") + return out + + +def git_commit_and_push(changed_names): + try: + if not GIT_TOKEN_FILE.exists(): + log("WARN: github-token.txt not found - skipping git push") + return False + pat = GIT_TOKEN_FILE.read_text(encoding="utf-8").strip() + ts = datetime.datetime.now().strftime("%Y-%m-%d %H:%M:%S") + msg = f"skill-sync: {ts} {' '.join(changed_names) if changed_names else '(no content changes)'}" + + cwd = str(SKILLS_ROOT) + subprocess.run(["git", "add", "-A"], cwd=cwd, check=True) + diff = subprocess.run(["git", "diff", "--cached", "--name-only"], + cwd=cwd, capture_output=True, text=True) + if not diff.stdout.strip(): + log("git: no changes to commit") + return True + subprocess.run(["git", "commit", "-m", msg], cwd=cwd, check=True) + push_url = f"https://{pat}@github.com/Graehamwatts/skills.git" + subprocess.run(["git", "push", push_url, "HEAD:main"], cwd=cwd, check=True) + log(f"git: pushed - {msg}") + return True + except Exception as e: + log(f"git error: {e}") + return False + + +def main(): + log("==== sync_skills.py start ====") + local = read_skill_files() + log(f"local skills found: {len(local)}") + + if not local: + log("Nothing to sync.") + return + + try: + console = list_console_skills() + log(f"console skills found: {len(console)}") + except Exception as e: + log(f"could not list console skills: {e} - aborting console sync") + console = None + + synced = created = updated = errors = 0 + changed_names = [] + + if console is not None: + for name, path, content in local: + existing = console.get(name) + existing_id = existing.get("id") if existing else None + try: + action, resp = upsert_skill(name, content, existing_id) + if resp.status_code >= 400: + log(f" ERROR {name}: {resp.status_code} {resp.text[:200]}") + errors += 1 + continue + synced += 1 + changed_names.append(name) + if action == "created": + created += 1 + else: + updated += 1 + log(f" {action}: {name}") + except Exception as e: + log(f" EXCEPTION {name}: {e}") + errors += 1 + else: + log("Skipping console upload step (API not reachable).") + + git_commit_and_push(changed_names) + + log("==== summary ====") + log(f" synced : {synced}") + log(f" created: {created}") + log(f" updated: {updated}") + log(f" errors : {errors}") + log("==== sync_skills.py end ====") + + +if __name__ == "__main__": + main() diff --git a/scripts/verify_brand_identity.py b/scripts/verify_brand_identity.py new file mode 100755 index 00000000..14bffe49 --- /dev/null +++ b/scripts/verify_brand_identity.py @@ -0,0 +1,116 @@ +#!/usr/bin/env python3 +""" +verify_brand_identity.py — Brand identity tripwire. + +Reads skills/shared-references/identity.json (the single source of truth) +and audits the entire repo. Fails (exit 1) if any blocked value appears +anywhere outside the identity file itself. + +Run before every push: + python3 scripts/verify_brand_identity.py + +Or wire into a git pre-push hook for automatic enforcement: + cp scripts/verify_brand_identity.py .git/hooks/pre-push + chmod +x .git/hooks/pre-push + +Why this exists: + Prior to April 24 2026, brand identity (DRE number especially) was + duplicated across 70+ files. Each "scrub" had to find and fix every + instance — miss one and the wrong DRE leaked into outputs. Five + consecutive scrubs failed to fully eliminate the wrong DRE because + there was no enforcement layer. + + This script IS the enforcement layer. It runs in seconds and grep-fails + the entire repo against the blocklist. Future regressions get caught + at push time instead of at user-discovery time. +""" +from __future__ import annotations +import json +import subprocess +import sys +from pathlib import Path + + +def main() -> int: + repo_root = Path(__file__).resolve().parent.parent + identity_path = repo_root / "skills" / "shared-references" / "identity.json" + + if not identity_path.exists(): + print(f"FAIL: identity source-of-truth not found at {identity_path}") + return 2 + + with open(identity_path) as f: + identity = json.load(f) + + blocked = identity.get("_blocked_values", {}) + # Two registers, same enforcement mechanism: a wrong DRE and a stale + # brokerage fail identically (a buried stale string propagates into every + # generated output), so both are grep-blocked at push time. + dre_values = blocked.get("dre_blocklist", []) + brand_values = blocked.get("brand_blocklist", []) + # (value, case_insensitive). Brand strings are matched case-insensitively + # because a case-sensitive sweep once skipped every ALL-CAPS "INTERO". + blocklist_specs = [(v, False) for v in dre_values] + [(v, True) for v in brand_values] + blocklist = dre_values + brand_values + correct_dre = identity["identity"]["dre"] + correct_brokerage = identity["identity"].get("brokerage", "") + + print(f"Brand identity tripwire — checking {len(blocklist)} blocked values") + print(f" Source of truth: {identity_path.relative_to(repo_root)}") + print(f" Correct DRE: {correct_dre}") + print(f" Correct brokerage: {correct_brokerage}") + print(f" Blocklist: {blocklist}") + print() + + failures = [] + + for blocked_value, case_insensitive in blocklist_specs: + flags = "-rlni" if case_insensitive else "-rln" + result = subprocess.run( + ["grep", flags, blocked_value, "--exclude-dir=.git", "."], + cwd=repo_root, + capture_output=True, + text=True, + ) + # Allow blocked values to appear in documentation files that + # legitimately need to reference them (e.g. CLAUDE.md warns about + # the blocked DRE so future sessions know not to add it). + exempt = set(blocked.get("_documentation_exempt", ["skills/shared-references/identity.json"])) + # Path fragments cover whole classes of file (generated catalogs, + # historical outputs) rather than naming each file individually. + exempt_fragments = blocked.get("_exempt_path_fragments", []) + hits = [] + for line in result.stdout.strip().splitlines(): + if not line: + continue + # Strip leading ./ from grep output and check against exemption set + normalized = line.lstrip("./") + if normalized in exempt: + continue + if line.endswith("identity.json"): # legacy fallback + continue + if any(frag in normalized.replace("\\", "/") for frag in exempt_fragments): + continue + hits.append(line) + if hits: + failures.append((blocked_value, hits)) + + if not failures: + print("PASS: zero blocked values found in repo.") + print(f" Repo is clean against the {len(blocklist)}-item blocklist.") + return 0 + + print("FAIL: blocked values found:") + for blocked_value, hits in failures: + print(f"\n {blocked_value!r} appears in {len(hits)} file(s):") + for hit in hits: + print(f" - {hit}") + + print() + print("Fix: replace each instance with the correct DRE from identity.json,") + print(" or update identity.json's blocklist if a value is no longer blocked.") + return 1 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/scripts/verify_python_integrity.py b/scripts/verify_python_integrity.py new file mode 100644 index 00000000..a82fd302 --- /dev/null +++ b/scripts/verify_python_integrity.py @@ -0,0 +1,72 @@ +#!/usr/bin/env python3 +""" +verify_python_integrity.py -- compile-check every Python file in the skills repo. + +Catches truncated / corrupted .py files. This is the recurring failure mode: +a bulk text-rewrite during a skill consolidation chops a file mid-string, it gets +committed broken, and it ships silently until someone runs the skill weeks later +(e.g. weekly-calendar-builder.py truncated in the May 2026 social-media-analyzer +consolidation; prompts-library-builder.py truncated and committed broken). + +Run with NO args to scan the whole skills/ tree (used by the auto-push hook). +Pass explicit file paths to check only those (used by the pre-push hook for the +files in the push range). + +Exit codes: + 0 all scanned files compile + 1 one or more files failed to compile (truncation / syntax error) +""" +from __future__ import annotations + +import py_compile +import sys +from pathlib import Path + +REPO_ROOT = Path(__file__).resolve().parent.parent +SCAN_ROOT = REPO_ROOT / "skills" + + +def iter_targets(argv): + """Yield .py paths to check: explicit args if given, else the whole tree.""" + if argv: + for a in argv: + p = Path(a) + if p.suffix == ".py" and p.exists(): + yield p + return + for p in SCAN_ROOT.rglob("*.py"): + if "__pycache__" in p.parts: + continue + yield p + + +def main(): + targets = list(iter_targets(sys.argv[1:])) + broken = [] + for p in targets: + try: + py_compile.compile(str(p), doraise=True) + except py_compile.PyCompileError as e: + broken.append((p, str(e).strip().splitlines()[-1])) + except SyntaxError as e: + broken.append((p, f"{type(e).__name__}: {e}")) + + print(f"Python integrity check -- {len(targets)} file(s) scanned under {SCAN_ROOT}") + if not broken: + print("PASS: every scanned Python file compiles.") + return 0 + print(f"FAIL: {len(broken)} file(s) do NOT compile (truncated or corrupted):") + for p, msg in broken: + try: + rel = p.relative_to(REPO_ROOT) + except ValueError: + rel = p + print(f" BROKEN {rel}") + print(f" -> {msg}") + print("") + print("A non-compiling .py must never be committed/pushed -- fix or remove it.") + return 1 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/scripts/verify_references.py b/scripts/verify_references.py new file mode 100644 index 00000000..df4f8189 --- /dev/null +++ b/scripts/verify_references.py @@ -0,0 +1,246 @@ +#!/usr/bin/env python3 +"""verify_references.py - fail on instructions that point at things which do not exist. + +WHY THIS EXISTS +--------------- +The brand tripwire catches bad VALUES (a wrong DRE, a retired brokerage). Nothing +caught bad REFERENCES, so these all survived in live instructions: + + - Composio was retired workspace-wide 2026-06-09, yet a scheduled task still called + GITHUB_GET_REPOSITORY_CONTENT every Monday and fell back to re-sending stale output. + One skill even instructed "COMPOSIO GITHUB TOOLS - never via local git", forbidding + the correct method. + - mcp__Claude_in_Chrome__ (wrong case) in 4 skills. Hard-fails on invocation. + - Routes to skills that have never existed: video-editor, ghl-crm-audit, + cinematic-video-engine, pipeline-dashboard. + - references/*.md and scripts/*.py paths naming files that are not on disk. + +An agent reading these does the confident thing and fails, or worse, silently skips +the step. This script makes that class of rot fail at push time instead. + +USAGE + python scripts/verify_references.py [--json] [--skill NAME] + +EXIT CODES + 0 no broken references + 1 broken references found +""" +from __future__ import annotations +import argparse, json, re, sys +from pathlib import Path + +REPO = Path(__file__).resolve().parent.parent +SKILLS = REPO / "skills" + +# Directories whose contents are generated or historical; not live instructions. +SKIP_PARTS = {"outputs", "__pycache__", "node_modules", ".venv", "graphify-out", "library", ".git"} + +# Things that are gone. Referencing them operationally is always a bug. +RETIRED = { + "run_composio_tool": "Composio retired workspace-wide 2026-06-09; use direct git", + "GITHUB_COMMIT_MULTIPLE_FILES": "Composio retired; use direct git push", + "GITHUB_GET_REPOSITORY_CONTENT":"Composio retired; read the local clone instead", + "COMPOSIO_REMOTE_WORKBENCH": "Composio retired", + "mcp__Claude_in_Chrome__": "wrong case; correct prefix is mcp__claude-in-chrome__", + "mcp__cowork__": "no such MCP server in this environment", + "mcp__workspace__": "no such MCP server in this environment", + "ask_user_input_v0": "no such tool; ask in plain text instead", + r"C:\\Users\\Admin": "no such user on this machine; it is C:\\Users\\Graeham Watts", +} + +# Skill names confirmed by audit to have NEVER existed in this repo. Listed explicitly +# because the contextual heuristic below is deliberately conservative (to keep the +# false-positive rate near zero), and that conservatism would otherwise let these slip +# when they appear in a table cell or prose without the word "skill" nearby. +PHANTOM_SKILLS = { + "ghl-crm-audit": "never existed; meta-ads routes to it twice", + "pipeline-dashboard": "never existed; named in shared-references/integrations.md", + "video-editor": "never existed; use video-creator", + "cinematic-video-engine": "never existed; concept-forge ends every path here", +} + +# Skills retired with a documented replacement. +DEPRECATED_SKILLS = { + "html-email": "publish HTML to online-content via direct git", + "github-skill-sync": "direct git push", + "video-script-creation-engine": "content-creation-engine", + "social-media-analyzer": "content-calendar", + "video-prompt-builder": "cinematic-hooks", +} + +# Files that legitimately NAME dead things in order to warn about them. +EXEMPT_FILES = { + "CLAUDE.md", "AGENTS.md", "README.md", + "skill-deprecation-protocol.md", "dre-leak-incident-log.md", + "architecture-decision.md", "identity.json", + "verify_references.py", "verify_brand_identity.py", + "dashboard-manifest.json", "validate_dashboard.py", +} + +# A line that is *explaining* a dead thing rather than instructing its use. +TOMBSTONE = re.compile( + r"deprecat|retired|do not use|never use|no longer|replaced by|instead of|" + r"was renamed|absorbed|historical|does not exist|never existed|is void|" + r"formerly|old approach|superseded", re.I) + + +# Scheduled tasks live OUTSIDE this repo but are the highest-risk instructions in the +# system: they run unattended, so a broken reference fails with nobody watching. The +# Composio call that broke the Monday content build for two months lived here, not in +# skills/. Scan it whenever it is present. +SCHEDULED = REPO.parent / "Scheduled" + + +def live_files(): + roots = [SKILLS] + ([SCHEDULED] if SCHEDULED.is_dir() else []) + for root in roots: + for p in root.rglob("*"): + if not p.is_file() or p.suffix.lower() not in {".md", ".json", ".py"}: + continue + if SKIP_PARTS & {q.name for q in p.parents}: + continue + if p.name in EXEMPT_FILES: + continue + # Correspondence and briefs are documents ABOUT the system, not + # instructions TO it. A support email describing the zombie-skill + # problem legitimately names dead skills; flagging it is noise. + if re.search(r"EMAIL|BRIEF|-for-|POST-?MORTEM|AUDIT|REPORT|NOTES?$", + p.stem, re.I): + continue + yield p + + +def main() -> int: + ap = argparse.ArgumentParser() + ap.add_argument("--json", action="store_true") + ap.add_argument("--skill", help="check only this skill") + a = ap.parse_args() + + existing = {d.name for d in SKILLS.iterdir() if d.is_dir()} + findings: list[dict] = [] + + for path in live_files(): + try: + rel = path.relative_to(REPO).as_posix() + except ValueError: + rel = "Scheduled/" + path.relative_to(SCHEDULED).as_posix() + if a.skill and f"skills/{a.skill}/" not in rel: + continue + try: + lines = path.read_text(encoding="utf-8", errors="replace").splitlines() + except OSError: + continue + + for i, line in enumerate(lines, 1): + if TOMBSTONE.search(line): + continue # explaining, not instructing + + for token, why in RETIRED.items(): + if re.search(token if token.startswith("C:") else re.escape(token), line): + findings.append({"file": rel, "line": i, "kind": "retired", + "ref": token.replace("\\\\", "\\"), "why": why}) + + for dead, repl in DEPRECATED_SKILLS.items(): + if re.search(rf"`{re.escape(dead)}`", line): + findings.append({"file": rel, "line": i, "kind": "deprecated-skill", + "ref": dead, "why": f"retired; use {repl}"}) + + for phantom, why in PHANTOM_SKILLS.items(): + if re.search(rf"`{re.escape(phantom)}`|skills/{re.escape(phantom)}\b", line): + findings.append({"file": rel, "line": i, "kind": "missing-skill", + "ref": phantom, "why": why}) + + # Backticked skill-like name used in a routing context. + for m in re.finditer(r"`([a-z][a-z0-9]+(?:-[a-z0-9]+){1,3})`", line): + name = m.group(1) + if name in existing or name in DEPRECATED_SKILLS: + continue + # Require an EXPLICIT skill signal. A bare "use the `x-y`" was far too + # loose: it matched scheduled-task names, output modes, and worked + # examples, producing enough noise to make the whole check ignorable. + if not re.search( + rf"`{re.escape(name)}`\s*skill|skill\s*`{re.escape(name)}`|" + rf"skills/{re.escape(name)}\b|read the\s*`?{re.escape(name)}`?\s*skill|" + rf"hands? off to\s*`{re.escape(name)}`|route to\s*`{re.escape(name)}`", + line, re.I): + continue + # Scheduled-task names, doc examples, and placeholders are not skills. + if re.search(r"\btask\b|cron|schedul|example|e\.g\.|placeholder|such as|" + r"\bmode\b|variant", line, re.I): + continue + if (SKILLS / name).exists(): + continue + findings.append({"file": rel, "line": i, "kind": "missing-skill", + "ref": name, "why": "no such skill directory"}) + + # Real client data inside a TEMPLATE or REFERENCE file. Scoped to those + # directories deliberately: a client's name belongs in their own report, + # never in the thing every future report is built from. Two cma-generator + # "templates" were finished client CMAs with zero placeholders, published + # publicly, and seeding every new report with the original client's + # identity. weekly-listing-update had the same bug with a seller's data. + if re.search(r"/(references|templates)/", "/" + rel): + for pat, kind in ( + (r"Prepared for\s+[A-Z][a-z]+\s+[A-Z][a-z]+", "client name"), + (r"\bAPN[:\s]*\d{3}-\d{3}-\d{3}\b", "parcel number"), + (r"\bML8\d{7}\b", "MLS number"), + ): + m = re.search(pat, line) + # A placeholder in the same position is the correct state. + if m and "{{" not in m.group(0): + findings.append({"file": rel, "line": i, "kind": "client-data", + "ref": m.group(0)[:48], + "why": f"real {kind} in a template/reference; " + f"use a {{{{PLACEHOLDER}}}} instead"}) + + # Relative pointers to references/ and scripts/ inside the same skill. + for m in re.finditer(r"`((?:references|scripts|templates|assets)/[A-Za-z0-9_./-]+\.(?:md|py|json|html))`", line): + target = m.group(1) + skill_root = path + while skill_root.parent != SKILLS and skill_root.parent != skill_root: + skill_root = skill_root.parent + # Legitimate homes: this skill, the repo root, or any sibling skill + # (cross-skill pointers like `scripts/verify_brand_identity.py` are real). + found = ((skill_root / target).exists() + or (REPO / target).exists() + or any((d / target).exists() for d in SKILLS.iterdir() if d.is_dir())) + if not found: + findings.append({"file": rel, "line": i, "kind": "missing-file", + "ref": target, "why": "path does not exist in this skill"}) + + if a.json: + print(json.dumps(findings, indent=2)); return 1 if findings else 0 + + print(f"Reference integrity check - {len(list(live_files()))} live instruction files\n") + if not findings: + print("PASS: every referenced skill, tool, and file exists.") + return 0 + + by_kind: dict[str, list] = {} + for f in findings: + by_kind.setdefault(f["kind"], []).append(f) + + print(f"FAIL: {len(findings)} broken reference(s).\n") + labels = {"retired": "RETIRED TOOL / PATH (will fail at runtime)", + "deprecated-skill": "DEPRECATED SKILL (has a replacement)", + "missing-skill": "SKILL DOES NOT EXIST", + "missing-file": "FILE DOES NOT EXIST", + "client-data": "REAL CLIENT DATA IN A TEMPLATE (privacy)"} + for kind, items in sorted(by_kind.items()): + print(f" [{labels.get(kind, kind)}] {len(items)}") + seen = set() + for f in items: + key = (f["ref"], f["file"]) + if key in seen: + continue + seen.add(key) + print(f" {f['file']}:{f['line']}") + print(f" -> {f['ref']} ({f['why']})") + print() + print("Fix each, or add the file to EXEMPT_FILES if it legitimately names a dead") + print("thing in order to warn about it (a tombstone, not an instruction).") + return 1 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/skills/.claude/settings.json b/skills/.claude/settings.json new file mode 100644 index 00000000..dfae28e0 --- /dev/null +++ b/skills/.claude/settings.json @@ -0,0 +1,5 @@ +{ + "hooks": { + "PreToolUse": [] + } +} \ No newline at end of file diff --git a/skills/_shared/transcription/PROPERTY_OS_SPEC.md b/skills/_shared/transcription/PROPERTY_OS_SPEC.md new file mode 100755 index 00000000..eab0c3db --- /dev/null +++ b/skills/_shared/transcription/PROPERTY_OS_SPEC.md @@ -0,0 +1,370 @@ +# Property OS — Transcription Service Specification + +**Status:** Draft v1 — May 2026 +**Owner:** Graeham Watts +**Purpose:** Spec for a transcription microservice inside Property OS so team members and end-users can drop a video URL and get a transcript back automatically. No manual paste. No browser automation. No team time spent on transcription. + +--- + +## What this is — and what it isn't + +This document is the engineering blueprint for adding transcription to Property OS as a server-side feature. It is meant to be handed to a developer (or used by Claude in a future session if Graeham provides the Property OS stack). + +**What's in scope:** +- API design (the endpoints Property OS exposes) +- Worker architecture (how transcription jobs run in the background) +- Storage schema (how transcripts are kept in the DB) +- Cost model (what this will run per month at various scales) +- Choice of transcription engine and download tool +- Failure handling + +**What's out of scope:** +- Property OS frontend UI design (depends on the existing app) +- Auth model (assumes Property OS already has user accounts) +- Specific deployment platform (works on AWS, Vercel, Railway, Render, etc.) + +--- + +## The honest scope-setting + +Claude in a Cowork session can build skills that run in Graeham's Cowork sandbox. Claude CANNOT magically run a transcription service inside Property OS for Graeham's team to use. The service has to be deployed by a developer (or by Claude in a future session if the codebase is connected). + +**What Claude can do here:** +- Write the entire service code (Python or TypeScript) ready for deployment +- Spec the database schema and migrations +- Build a working demo runnable in a sandbox +- Document everything + +**What Claude cannot do:** +- Deploy code to Property OS production infrastructure unilaterally +- Hold a long-running transcription server in a Cowork session (sessions expire) +- Maintain login state for paid services like Unmixr at scale + +The fastest realistic path: Claude writes this service in a future session, Graeham's dev (or Claude with deploy access) deploys it, team uses it via Property OS UI from then on. + +--- + +## Recommended architecture + +### Tech stack choice + +Two viable stacks depending on what Property OS already runs on: + +**Option A — Node/TypeScript stack** (if Property OS is Next.js/Express/Nest/etc.) + +- API: Next.js API route or Express endpoint +- Queue: BullMQ (Redis-backed) +- Workers: Node workers running BullMQ consumers +- Download tool: `youtube-dl-exec` (Node wrapper around yt-dlp) +- Transcription: Deepgram Node SDK +- DB: existing Property OS PostgreSQL + +**Option B — Python sidecar** (if Property OS is mostly TS but transcription wants Python) + +- API: FastAPI service running as a separate microservice +- Queue: Celery + Redis OR RQ + Redis +- Workers: Python workers in Docker +- Download tool: `yt-dlp` (native Python) +- Transcription: Deepgram Python SDK or local Whisper for low-tier +- DB: existing Property OS PostgreSQL accessed via SQLAlchemy or asyncpg + +**My recommendation:** If Property OS is Node, go Option A — keeps it in one stack. If Property OS already has a Python service (or you want local Whisper as a backup tier without Deepgram costs), go Option B. + +For most production Property OS deployments serving a team, **Option A with Deepgram-only (no local Whisper) is simpler and cheaper to operate.** Local Whisper means you need GPU-or-very-fast-CPU workers, which complicates deployment. Deepgram-only means stateless workers, simple horizontal scaling, predictable costs. + +### Service topology + +``` + ┌──────────────────┐ + │ Property OS UI │ + │ (Next.js/React) │ + └────────┬─────────┘ + │ POST /transcribe { url } + ▼ +┌──────────────────────────────────────────────────────────────────────────────┐ +│ Property OS API │ +│ ┌────────────────────────────────────────────────────────────────────────┐ │ +│ │ POST /api/transcribe │ │ +│ │ 1. Validate URL, check user has credit/quota │ │ +│ │ 2. Insert transcripts row (status=pending) │ │ +│ │ 3. Enqueue job in BullMQ → returns job_id + transcript_id │ │ +│ └────────────────────────────────────────────────────────────────────────┘ │ +│ ┌────────────────────────────────────────────────────────────────────────┐ │ +│ │ GET /api/transcribe/:id → returns transcript row (status + text) │ │ +│ │ GET /api/transcribe → list user's transcripts │ │ +│ └────────────────────────────────────────────────────────────────────────┘ │ +└──────────────────────────┬───────────────────────────────────────────────────┘ + │ enqueue + ▼ + ┌────────────────┐ + │ Redis │ + │ (BullMQ) │ + └────────┬───────┘ + │ workers pick up + ▼ +┌──────────────────────────────────────────────────────────────────────────────┐ +│ Transcription worker(s) — horizontally scalable │ +│ 1. Pull job (URL + transcript_id) │ +│ 2. yt-dlp downloads audio to /tmp │ +│ 3. POST audio to Deepgram │ +│ 4. Update transcripts row (status=complete, transcript_text=...) │ +│ 5. Send webhook/Pusher event to Property OS UI │ +│ 6. Optional: trigger downstream content pipeline │ +└──────────────────────────────────────────────────────────────────────────────┘ +``` + +### Database schema (PostgreSQL) + +```sql +CREATE TABLE transcripts ( + id UUID PRIMARY KEY DEFAULT gen_random_uuid(), + user_id UUID NOT NULL REFERENCES users(id) ON DELETE CASCADE, + property_id UUID REFERENCES properties(id) ON DELETE SET NULL, -- optional context + + source_url TEXT NOT NULL, + source_platform TEXT, -- youtube | instagram | tiktok | ... + source_title TEXT, -- pulled from yt-dlp metadata + source_uploader TEXT, -- creator handle + duration_sec INTEGER, + + status TEXT NOT NULL DEFAULT 'pending', + -- pending | downloading | transcribing | complete | failed + tier TEXT NOT NULL DEFAULT 'standard', + -- standard | premium + + transcript_text TEXT, + word_count INTEGER, + + error_message TEXT, + retry_count INTEGER DEFAULT 0, + + cost_cents INTEGER, -- Deepgram cost for this job + + created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(), + started_at TIMESTAMPTZ, + completed_at TIMESTAMPTZ +); + +CREATE INDEX idx_transcripts_user_status ON transcripts(user_id, status); +CREATE INDEX idx_transcripts_source_url ON transcripts(source_url); +CREATE INDEX idx_transcripts_status_created ON transcripts(status, created_at); + +-- Optional: dedupe across the whole platform +CREATE UNIQUE INDEX idx_transcripts_dedupe ON transcripts(source_url, tier) + WHERE status = 'complete'; +``` + +### API surface + +```typescript +// POST /api/transcribe +// Body: { url: string, tier?: 'standard' | 'premium', property_id?: string } +// Returns: { transcript_id: string, status: 'pending', estimated_seconds: number } + +// GET /api/transcribe/:id +// Returns: { id, status, source_url, source_platform, source_title, +// duration_sec, transcript_text, word_count, error_message, +// created_at, completed_at } + +// GET /api/transcribe?status=complete&limit=50 +// Returns: array of transcript rows for the authenticated user + +// DELETE /api/transcribe/:id +// Removes the transcript (user's own only) + +// POST /api/transcribe/:id/retry +// Re-runs a failed job +``` + +### Frontend integration + +```typescript +// In Property OS UI: +async function submitTranscription(url: string, tier: 'standard' | 'premium' = 'standard') { + const res = await fetch('/api/transcribe', { + method: 'POST', + body: JSON.stringify({ url, tier }), + }); + const { transcript_id } = await res.json(); + + // Poll OR subscribe to Pusher channel for status updates + // (Pusher recommended — Property OS likely already uses it) + + return transcript_id; +} +``` + +Real-time updates: use Pusher / Ably / Supabase Realtime to push status changes to the UI without polling. The worker updates the row, and a Postgres NOTIFY or a Pusher event fires the UI update. + +--- + +## Cost model + +### Deepgram Nova-3 (recommended) + +- **Standard plan:** $0.0043 per minute of audio +- **Growth plan:** $0.0036 per minute (at higher volume) +- **Pre-paid bulk:** Volume discounts available — talk to sales at scale + +### Real-world costs at various team scales + +Assumption: average video length 5 minutes. + +| Scale | Videos/month | Minutes/month | Cost/month | +|---|---|---|---| +| Solo agent | 20 | 100 | $0.43 | +| Small team (5 agents) | 100 | 500 | $2.15 | +| Property OS public beta (50 users, 20 videos each) | 1,000 | 5,000 | $21.50 | +| Property OS scale (500 users, 20 videos each) | 10,000 | 50,000 | $215 | + +Deepgram costs are negligible relative to the team time saved. A single agent spending 10 min/day on manual transcription costs the business ~$50/day in opportunity cost. Deepgram saves that for cents. + +### Infrastructure costs + +- **Redis (Upstash or similar managed):** Free tier covers up to ~10K commands/day; ~$10/month for low-mid scale; $50-100/month at scale +- **Worker compute (Render / Railway / Fly.io background worker):** ~$10/month for a small worker; ~$50-100/month for 3-5 workers at scale +- **Storage:** Negligible — transcripts are text, ~5KB per row average + +**Total estimated cost at the Property OS-scale tier:** ~$300/month all-in for transcription serving 500 active users. At small-team scale: under $20/month. + +--- + +## yt-dlp robustness — the Instagram problem + +The hard part isn't transcription. Deepgram is rock solid. The hard part is **reliably extracting audio from platforms that fight scrapers**, mainly Instagram. + +**yt-dlp success rates (rough industry estimates, 2026):** + +- YouTube: 98%+ (Google rarely breaks yt-dlp; even when they do, fix lands in days) +- TikTok: 95%+ (occasional rate limits) +- Vimeo: 99%+ +- X / Twitter: 90% (depends on tweet visibility) +- Facebook: 85% +- **Instagram: 70-85%** (Instagram aggressively breaks yt-dlp; rate limits hit fast) + +For production reliability on Instagram, **use Apify's Instagram Reel Scraper** as a fallback: + +- ~$0.50 per 1000 reels +- Maintained by Apify's team — they keep up with Instagram changes +- Returns audio URL + metadata +- Hand off audio URL to Deepgram + +**Hybrid strategy:** +1. Try yt-dlp first (free) +2. If it fails on Instagram, retry via Apify (paid but reliable) +3. Log every fallback so you can monitor yt-dlp reliability over time + +### Worker pseudocode + +```typescript +async function processTranscriptionJob(job: TranscriptionJob) { + const { url, transcript_id, tier } = job.data; + + await db.update('transcripts', { id: transcript_id, status: 'downloading' }); + + let audioPath: string; + try { + audioPath = await downloadWithYtDlp(url); + } catch (err) { + // Instagram fallback + if (url.includes('instagram.com')) { + audioPath = await downloadWithApify(url); + } else { + await markFailed(transcript_id, err); + return; + } + } + + await db.update('transcripts', { id: transcript_id, status: 'transcribing' }); + + let transcript: string; + if (tier === 'premium') { + transcript = await deepgramTranscribe(audioPath, 'nova-3'); + } else { + transcript = await deepgramTranscribe(audioPath, 'nova-2'); + // Note: even "standard" tier uses Deepgram in this Property OS spec — local Whisper + // is the right move for Cowork sessions but adds operational complexity in production + } + + const wordCount = transcript.split(/\s+/).length; + const costCents = Math.ceil(durationSec / 60 * 0.43); // standard rate + + await db.update('transcripts', { + id: transcript_id, + status: 'complete', + transcript_text: transcript, + word_count: wordCount, + cost_cents: costCents, + completed_at: new Date(), + }); + + await pusher.trigger(`user-${userId}`, 'transcript-complete', { transcript_id }); +} +``` + +--- + +## Why NOT just use Cowork sessions for the team + +Graeham's instinct is the right one — building this into Property OS server-side is better than having every team member run Cowork sessions. Reasons: + +1. **Cowork sessions are interactive.** They're great for Graeham working on content, not great for "the system does it while I sleep." +2. **Session state is per-user.** Each team member would need their own Cowork session, their own credentials, their own setup. +3. **Sessions time out.** A team member starts a transcription, walks away, comes back to a closed session. +4. **No shared transcript history.** Cowork sessions don't share state — Property OS DB shares state across the team naturally. +5. **No automation.** Property OS can trigger transcription on events (new lead, new listing, scheduled batch). Cowork only fires when a human is there. + +Production transcription = backend service. End of story. + +--- + +## Suggested implementation order + +If Graeham wants this built (in a future Cowork session or by his dev), here's the order: + +1. **Week 1 — Foundation** + - Set up Redis + BullMQ + - Implement the `POST /api/transcribe` endpoint + - Create the `transcripts` DB table + migrations + - Stub worker that just downloads and logs (no transcription yet) + +2. **Week 2 — Core transcription** + - Integrate Deepgram SDK in the worker + - Implement yt-dlp download + - Wire up status updates (Pusher or polling) + - Build the minimal UI: "Drop URL → see transcript when done" + +3. **Week 3 — Reliability & polish** + - Add Apify fallback for Instagram + - Build retry logic for failed jobs + - Add cost tracking + per-user quotas + - Add transcript search in Property OS + +4. **Week 4 — Integration with content pipeline** + - Wire completed transcripts into Property OS content workflow + - Add bulk-import (drop 10 URLs, get 10 transcripts) + - Add scheduled batch jobs (transcribe a creator's whole feed weekly) + +Total: roughly 4 weeks of focused dev work for a small team, or ~2 weeks if it's the dev's primary focus. + +--- + +## Open questions for Graeham + +Before building, decide: + +1. **What stack is Property OS on?** Node/TypeScript or Python? This determines Option A vs Option B. +2. **What's the hosting platform?** Vercel, AWS, Railway, Render, Fly.io? Each has different worker support. +3. **Who pays for Deepgram — Property OS or end-user?** Per-user quotas? Per-team plans? +4. **Do you want speaker diarization?** ("Who said what" in podcasts.) If yes, AssemblyAI is better than Deepgram. Costs a bit more. +5. **Do you want this to feed into Graeham's existing content pipeline automatically?** E.g., transcript completes → auto-runs through `transcript-repurposer` logic → content draft saved. + +Once those are answered, the build is straightforward. + +--- + +## Bottom line + +Build the service. Don't try to automate Unmixr or scale Cowork sessions for the team. Deepgram + yt-dlp + a simple worker is the right architecture, costs almost nothing at small scale, and runs hands-off forever. + +The transcription module in `_shared/transcription/transcribe.py` is the reference implementation. The Property OS version is the same logic, deployed as a server-side service with a queue. diff --git a/skills/_shared/transcription/README.md b/skills/_shared/transcription/README.md new file mode 100755 index 00000000..4e03f6da --- /dev/null +++ b/skills/_shared/transcription/README.md @@ -0,0 +1,125 @@ +# Shared Transcription Module + +Single source of truth for video → text across all of Graeham's skills. + +## Why this exists + +Before this module: `content-creation-engine` had its own YouTube transcriber, the new `transcript-repurposer` was about to grow its own, and Property OS was on track to grow a third. Three different transcribers means three different sets of bugs, three different output formats, three different failure modes. + +This module is the one place where transcription happens. Any skill that needs a transcript calls `transcribe.py` and gets back a consistent JSON result. + +## Two tiers + +| Tier | Engine | Cost | Speed | Accuracy | Trigger | +|---|---|---|---|---|---| +| Default | yt-dlp + OpenAI Whisper (local in sandbox) | Free | 30s-5min depending on length | ~95% | Default — no flag needed | +| Premium | yt-dlp + Deepgram Nova-3 API | $0.0043/min | ~real-time for short, ~10x for long | 98%+ | `--premium` flag + `DEEPGRAM_API_KEY` env var | + +## Usage + +From any skill (or the command line): + +```bash +# Default: Whisper local, free +python3 transcribe.py --url "https://www.instagram.com/reel/..." + +# Premium: Deepgram, higher accuracy +DEEPGRAM_API_KEY=your_key python3 transcribe.py --url "..." --premium + +# Local audio file +python3 transcribe.py --file /path/to/podcast.mp3 + +# Get the full JSON result (default outputs transcript text only) +python3 transcribe.py --url "..." --json + +# Write to a file +python3 transcribe.py --url "..." --output transcript.txt +``` + +## Caching + +Every successful transcription is cached under `~/.cache/graeham-transcripts/.json`. Same URL + tier = same cache key. Re-running on a cached URL returns instantly. + +Cache key hashes the source + tier — so the same URL transcribed via Whisper vs Deepgram are stored separately. Useful when you want to compare quality or upgrade a previous Whisper transcript to Deepgram. + +## Supported sources + +Anything yt-dlp supports — that's 1,000+ sites including: + +- YouTube (videos, Shorts, live streams) +- Instagram (Reels, posts, IGTV) +- TikTok +- Twitter / X video posts +- Facebook video +- Vimeo +- Twitch VODs +- Most podcast hosting platforms +- Direct .mp3 / .mp4 / .m4a file URLs + +For full list: `yt-dlp --list-extractors`. + +## Output JSON shape + +```json +{ + "transcript": "Clean spoken text...", + "source_url": "https://...", + "source_platform": "youtube|instagram|tiktok|x|vimeo|facebook|unknown", + "title": "
+ +

Scoring Architecture — Why This Topic Ships

+

Two scores, two questions. Opportunity Score (owned by content-calendar) answers "should we cover this topic THIS WEEK vs other candidates?" Intent Score (owned by bofu-scorer, Phase 3 of the content-creation-engine) answers "what's the BOFU intent of this topic for CTA and funnel decisions?" Both live here expanded — per Rule 13, no toggle.

+
+ +
+
Table A — Opportunity Score 23/25
+
Owner: content-calendar · Source: outputs/calendar-data/calendar-2026-04-27.json
+ + + + + + + + + + +
CriterionScoreSource / Notes
Performance Signal4/5IG reach +52% WoW, data/market content is your #1 lane (1,720 avg reach)
Search Demand5/5GSC: 15+ Peninsula home-value queries, pos 13-42, zero clicks = content gap
Audience Intent4/5Reddit + Nextdoor + news comments all confirming demand
Competitive Gap5/5Zero competitor coverage of the homicide-free milestone + home value angle
Timeliness5/5Story broke April 17, 2026 — 48hr news window still open
Total23/25Threshold: must_create (22-25 range)
+
+ +
+
Table B — Intent Score 20/25
+
Owner: content-creation-engine/references/phases/bofu-intent-scorer.md · Source: outputs/scored-topics-{ts}.json
+ + + + + + + + + + + + +
CriterionScoreSource / Notes
Inquiry Type Match4/5Process inquiry (home value impact) with Property overlap
Intent Matrix Position3/5CONSIDERATION (MOFU), Voluntary + Independent → BOFU via COSTS keyword CTA
Source Confirmation5/53+ platforms: Google PAA, Reddit r/bayarea, Nextdoor EPA, local news comments
Emotional Temperature4/5Moderate-to-high — buyers skipped EPA based on old data, now second-guessing
Local Relevance5/5Hyperlocal — EPA specific, with Peninsula comparative frame
Base Total21/25Before freshness adjustment
Freshness Adjustment−1EPA used 2x in last 2 weeks (market overlap, different angle) — small penalty
Final Total20/25Threshold: ships (≥18)
+
+ +
+ +
+ 📅 Calendar Integration: This topic is the anchor for its scheduled day. If a higher-priority breaking story comes in, three options: (A) Replace this slot with the breaking topic and bump this to next week. (B) Add the breaking topic as a Sat/Sun interrupt. (C) Re-score on the next ideation pass. → Current weekly calendar +
+ +

📅 7-Day Posting Calendar — When to Ship Each Format

+

What this shows: Recommended publishing schedule for this topic across 7 days. Times are based on your actual IG performance data (top posts were 6-9am and 5-8pm). Each day card links to the matching format panel above so you can jump straight to copying.

+
+
Mon
Day 1
9:00 AM🎥 YouTube Long publishes
6:00 PM📱 IG Reel #1 + FB cross-post
+
Tue
Day 2
8:00 AM📹 YouTube Short
7:00 PM🎵 TikTok
+
Wed
Day 3
7:00 AM📝 Blog post publishes
10:00 AM📍 GMB post
+
Thu
Day 4
8:00 AM💼 LinkedIn post
6:00 PM📱 IG Reel #2 (data-led)
+
Fri
Day 5
9:00 AM📧 Newsletter send
2:00 PM📘 Facebook extended post
+
Sat
Day 6
10:00 AM🖼️ IG Carousel (saves best on weekends)
—💰 Ad campaigns continue
+
Sun
Day 7
—📊 Review Week 1 analytics
—🔬 Plan Week 2 derivatives
+
+
Why this order: YouTube Long first (longest shelf life, primes retargeting). Short-form (Reel #1, YT Short, TikTok) Mon-Tue hits peak algorithmic distribution. Blog Wednesday for SEO indexing before Friday newsletter references it. Newsletter Friday because your subscribers open email at end-of-workweek. Carousel Saturday because IG carousel saves peak on weekends.
+ +

Content Derivatives — 15 Formats Ready

+

Each format has a Copy button (gold, format-specific label like "Copy Script" or "Copy Newsletter HTML") + Copy Prompt (gold outline, for regeneration). YT Long Pt 1 also has a paired Copy Production Content (purple) button. Scroll down — 2 newsletter buttons are in row 2.

+
+ __FLOW__ +
+ +
+__PANELS__ +
+ +

Shot List — Hand to Peter and John

+ + + + + + + + + + + + + + + + +
#Shot DescriptionDurationSetup Notes
1Open Talking Head — Graeham neutral expression (no smile on hook)0:00-0:20Eye-level, 50mm look, clean backdrop
2Archival 1990s news clips / chyrons0:20-0:35Stock archival OR AI-generate
3TH cutback — setup context0:35-1:05Same framing as Shot 1
490s newspaper headlines / period EPA photos1:05-1:15SF Chronicle / Mercury News archive
5TH Act 2 — warmer tone1:15-1:45Small camera repositioning
6Community B-roll — Joel Davis Park, youth programs1:45-2:05Shoot locally OR request from City of EPA
7TH milestone reveal — slower pace2:05-2:35Direct-to-camera, closer framing
8EPA City Hall / current streets / events2:35-2:55Shoot locally
9TH market angle — business tone2:55-3:45TH, stat overlays in post
10Motion graphic stat cards — DOM and price data3:45-4:00Motion graphics (Jason)
11TH CTA — direct, confident4:00-4:30TH, close framing
12End card — Graeham branding4:30Static, 3-4 sec hold
+ +

📋 Copy Bank — All 15 Formats in One Place

+

What this is: Every format's production-ready content as a quick-copy button, stacked in one section. Use this when you want to batch-copy multiple formats without clicking through the tabs above. Color-coded by format family (video red, Instagram pink, blog green, social blue, email gold).

+
+__COPY_BANK__ +
+
How this differs from the tabs above: Tabs show the full preview + render instructions + prompt. Copy Bank is just the Copy Content buttons stacked for speed. Use Copy Bank for batch-shipping, tabs for deep-diving a single format.
+ +

3 Alternate Hooks (A/B Testing)

+
+
PICKED

Hook A — Story-led

"East Palo Alto was called 'the murder capital of America.' That was 1992. Last week — 34 years later — the city quietly hit a milestone almost nobody outside of here is talking about."

+

Hook B — Buyer-math-led

"If you've been shopping the Peninsula and skipping East Palo Alto — you're paying Palo Alto prices for a problem that stopped existing in 2024. Let me show you the data."

+

Hook C — Counter-narrative-led

"What if I told you the 'murder capital of America' has gone two full years without a single homicide — and the rest of the Peninsula just lost 7% of its home value while East Palo Alto quietly went up?"

+
+
Recommendation: Hook A as primary. Shares trigger on curiosity + charged phrase + reveal pattern.
+ +
+

🚀 Power-User Alternative: ElevenLabs + HeyGen Pipeline (Optional)

+

TLDR: You probably don't need this. The red Render buttons per format (above) are the recommended path — they use the HeyGen MCP and handle everything automatically. This section is the OLD manual pipeline that uses ElevenLabs for voice + HeyGen for avatar, for when you want more granular voice control (custom SSML tags, specific pacing).

+

What this pipeline does (if you choose to use it):

+
    +
  1. Takes the SSML block from YouTube Long Pt 1's "Ready to Post" content.
    2. +
  2. Synthesizes Graeham's cloned voice via ElevenLabs (better prosody control than HeyGen's default TTS).
    3. +
  3. Uploads the resulting MP3 to HeyGen.
    4. +
  4. Renders the avatar video in HeyGen using that MP3 as the audio track.
    5. +
  5. Downloads the finished MP4 to your outputs folder.
    6. +
+

To use: Click Copy Script + SSML on YouTube Long Pt 1, paste just the <speak>...</speak> block into a new file at outputs/content-package-2026-04-18-epa-two-years-homicide-free.ssml.txt, then run this command in your terminal:

+ python3 skills/heygen-elevenlabs-renderer/scripts/full_render.py \\\\
  --script outputs/content-package-2026-04-18-epa-two-years-homicide-free.ssml.txt \\\\
  --slug "epa-two-years-homicide-free" \\\\
  --resolution 1080p \\\\
  --aspect 16:9 +
+
Voice: Graeham clone Pa3vOYQHHpLJn1Tf7hnP
+
Avatar: 9a3600b16f604059b6ab8b9a55e29ea9
+
GHL Keyword: EPA
+
+
+ +
+
Graeham Watts — Powered by The Boyenga Team at Compass · DRE #01466876
+
Content Creation Engine · Phase G Per-Topic Dashboard · v5 Scoring Architecture Panel · Generated April 18, 2026
+
Sources: Local News Matters · The Almanac · Redfin · Benson Group · Own Team · Palo Alto Online · City of East Palo Alto
+
+ +
+ + + + +""" + +# Substitute placeholders +DASHBOARD = HEAD +DASHBOARD = DASHBOARD.replace("__RESEARCH_DATA_TOP__", RESEARCH_DATA_HTML) +DASHBOARD = DASHBOARD.replace("__FLOW__", FLOW) +DASHBOARD = DASHBOARD.replace("__PANELS__", PANELS) +DASHBOARD = DASHBOARD.replace("__PLIB__", PLIB) +DASHBOARD = DASHBOARD.replace("__CLIB__", CLIB) +DASHBOARD = DASHBOARD.replace("__HRLIB__", HRLIB) +DASHBOARD = DASHBOARD.replace("__COPY_BANK__", COPY_BANK) +DASHBOARD = DASHBOARD.replace("__TOPIC_SLUG__", "epa-two-years-homicide-free") + +OUT = Path("/var/tmp/stage3/skills/online-content/dashboards/single-topic/2026-04-18-epa-two-years-homicide-free-production.html") +OUT.write_text(DASHBOARD, encoding="utf-8") + +print(f"WROTE: {OUT}") +print(f"size={len(DASHBOARD):,} prompts={len(PROMPTS)} content={len(CONTENT)} panels={len(panels_html)} cards={len(flow_cards)}") + +# ============================================================================ +# Auto-unify (added 2026-04-21) +# Runs the canonical UNIFIED_FINAL_V2 post-processor on the file we just wrote +# so this newly-generated dashboard inherits: +# - Consolidated stylesheet (hero h1 contrast, card unification, etc) +# - v5 hero with clickable badge tooltips + plain-English timing +# - "Why This Topic?" research accordion (collapsed) +# - Calendar clarifier + inline help blocks +# - Crew-tool accordions (Shot List / Hooks / ElevenLabs collapsed) +# This replaces the old manual post-processing step. +# Auto-unify removed 2026-04-29 — unify_final.py was a one-time migration script. diff --git a/skills/content-creation-engine/templates/video-research/notes-mode-a.md b/skills/content-creation-engine/templates/video-research/notes-mode-a.md new file mode 100644 index 00000000..47a68f08 --- /dev/null +++ b/skills/content-creation-engine/templates/video-research/notes-mode-a.md @@ -0,0 +1,20 @@ +--- +source: {source_url} +title: {title} +channel: {channel} +length: {duration} +captured: {captured_iso} +mode: transcript-only +slug: {slug} +--- + +# {title} + +**Channel:** [{channel}]({channel_url}) +**Source:** [{source_url}]({source_url}) +**Length:** {duration} | **Captured:** {captured_human} +**Transcript method:** {transcript_method} + +## Transcript + +{transcript_with_timestamps} diff --git a/skills/content-creation-engine/templates/video-research/notes-mode-b.md b/skills/content-creation-engine/templates/video-research/notes-mode-b.md new file mode 100644 index 00000000..337d9c32 --- /dev/null +++ b/skills/content-creation-engine/templates/video-research/notes-mode-b.md @@ -0,0 +1,91 @@ +--- +source: {source_url} +title: {title} +channel: {channel} +length: {duration} +captured: {captured_iso} +mode: frame-by-frame +focus_range: {focus_range} +frame_count: {frame_count} +topic: {topic} +slug: {slug} +--- + +# {title} — Visual Analysis + +**Channel:** [{channel}]({channel_url}) +**Source:** [{source_url}]({source_url}) +**Length:** {duration} | **Captured:** {captured_human} + +## TLDR + +{tldr_3_4_sentences} + +## Hooks (First 0:00–0:10) + +**Visual hook (0:00–0:03):** +![Opening frame](frames/{hook_frame}) +{visual_hook_description} + +**Audio hook (first spoken line):** +> "{audio_hook_quote}" + +**Why it works:** {hook_analysis} + +## Key Concepts + +{key_concepts_list} + +## Scene-by-Scene Notes + +{scene_by_scene_blocks} + +## B-Roll Catalog (for reverse-engineering) + +| Shot Type | Timestamps | % of runtime | Notes | +|---|---|---|---| +{broll_table_rows} + +**Cut pacing:** {cuts_per_minute_avg} cuts/min average. Energy arc: {energy_arc_description} + +## On-Screen Text Catalog + +| Timestamp | Text | Style | +|---|---|---| +{onscreen_text_rows} + +## Production Style Fingerprint + +- **Color grade:** {color_grade} +- **Typography:** {typography} +- **Motion graphics style:** {motion_graphics_style} +- **Aspect ratio / framing:** {aspect_framing} +- **Brand signals:** {brand_signals} + +## Code & Commands (if any) + +{code_blocks} + +## Replicate-This Brief (for HeyGen + Higgsfield) + +If you wanted to recreate this video's structure with your own content: + +**Hook (0:00–0:03):** +- Visual: {replicate_hook_visual} +- Audio: {replicate_hook_audio} + +**Body (0:03–end):** +{replicate_body_structure} + +**CTA placement:** {replicate_cta} + +## Open Questions / Unknowns + +{open_questions} + +## Source + +- Original video: {source_url} +- Captured: {captured_iso} +- Cache key: {slug} +- Frame count: {frame_count} ({scene_change_count} scene changes + {coverage_floor_count} coverage-floor frames) diff --git a/skills/context-engineer/SKILL.md b/skills/context-engineer/SKILL.md new file mode 100755 index 00000000..a3baf4ae --- /dev/null +++ b/skills/context-engineer/SKILL.md @@ -0,0 +1,25 @@ +--- +name: context-engineer +description: "Context-window diagnostic and optimization engine for Claude sessions and skills. Use ANY time the user mentions context window, context length, token budget, token limit, running out of context, context engineering, optimize my prompt, optimize my skill, my skill is too long, my prompt is too long, context management, context bloat..." +--- + +# Context Engineer — Diagnose & Optimize the Context Window + +Diagnose bloated/expensive contexts and skills, and design new ones, using the tiered progressive-disclosure model below. + +## Hard constraints + +| Tier | What | When loaded | Budget | +|---|---|---|---| +| 1. Frontmatter | `name`, `description` | Always, even when skill doesn't trigger | ~200 words (~1% of a 200k context budget) | +| 2. SKILL.md body | Core workflow + decision logic | When the skill triggers | <500 lines ideal — flag anything over | +| 3. `references/` | Detailed docs, rubrics, playbooks | On demand, when SKILL.md body tells Claude to Read them | Unbounded | +| 4. `assets/` and `scripts/` | Templates, fonts, images, executables | On demand; scripts can run without loading source | Unbounded | + +## Reference files + +- `references/token_estimation.md` — How to estimate tokens from files, messages, and tool results without running a tokenizer. +- `references/patterns.md` — Tiered context patterns: domain-organized, variant-organized, phase-organized, with examples. +- `references/anti_patterns.md` — The seven anti-patterns with diagnostic signatures and fixes. + +Read them on demand. If the user asks for a general context diagnosis, audit, or refactor, work from the tier table above and pull in the reference files as needed for the specific case. diff --git a/skills/context-engineer/references/anti_patterns.md b/skills/context-engineer/references/anti_patterns.md new file mode 100755 index 00000000..91a58743 --- /dev/null +++ b/skills/context-engineer/references/anti_patterns.md @@ -0,0 +1,123 @@ +# The Seven Context Anti-Patterns + +Each anti-pattern has a diagnostic signature (how to spot it), a cost profile (why it hurts), and a fix. + +## Table of contents +1. The Kitchen Sink +2. The Repeat Offender +3. The Stale Tool Result +4. The Silent Re-Read +5. The Verbose Example +6. The Unused Lookup Table +7. The Restatement of Defaults + +--- + +## 1. The Kitchen Sink + +**Signature:** one SKILL.md file that handles every domain, variant, and edge case. Often 800–2000 lines. + +**Cost:** every invocation pays the full body tax, even for the 80% of runs that only need 20% of the content. + +**Fix:** Split by variant/domain/phase into reference files (see `patterns.md`). Keep the shared workflow in SKILL.md. The body should shrink to the decision logic + the common path. + +**Example:** a video-creator SKILL.md that inlines ffmpeg slideshow instructions AND Remotion project scaffolding AND HeyGen API docs. Split into three references. + +--- + +## 2. The Repeat Offender + +**Signature:** the same instruction written in multiple places. "Use TypeScript" appears in Step 2, Step 5, and the "important notes" section. Or the output format is spec'd three times for three different cases that could share one spec. + +**Cost:** more tokens, and (worse) ambiguity — if the three statements drift apart over edits, the model has to guess which to follow. + +**Fix:** pick one canonical spot. Link or reference from other spots. + +**Example:** a SKILL.md that says "ALWAYS save output to /outputs" in four places. Pick the first, delete the rest. + +--- + +## 3. The Stale Tool Result + +**Signature:** a large tool result sitting in conversation history from 10 turns ago, no longer relevant. Common culprits: directory listings, grep results, web fetches, subagent reports. + +**Cost:** persistent token cost that grows with every tool call. Especially painful in long sessions. + +**Fix:** summarize the result after you've extracted what you need. In new turns, reference the summary instead of scrolling back to find the raw output. + +**Example:** a `find /` that returned 40,000 characters of paths, of which the user only needed 3. Summarize as: "Found the following relevant files: [path1, path2, path3]. Full listing archived in prior turn if needed." + +--- + +## 4. The Silent Re-Read + +**Signature:** the same file Read multiple times across a session because Claude didn't track that it was already in context. + +**Cost:** N copies of the file in history instead of 1. For large files, this is brutal. + +**Fix:** Read once. For subsequent references, cite the file by path/line numbers rather than re-reading. If the file has been edited mid-session, re-read is appropriate; if not, it's waste. + +**Example:** a 2,000-line source file Read 5 times in one debugging session. Fix: Read once up front, note the important functions and line numbers, reference those in subsequent turns. + +--- + +## 5. The Verbose Example + +**Signature:** a single example in SKILL.md that runs 100+ lines. Often 3–5 of them stacked. + +**Cost:** examples are valuable (better than abstract instructions), but each example is paid for on every invocation, not just when relevant. + +**Fix:** keep 1 canonical example inline. Move the rest to `references/examples.md` with a short index. The SKILL.md body says "For more examples covering X, Y, Z, read references/examples.md." + +**Example:** a copywriting skill with full before/after rewrites for 6 formats inline. Keep one format's example inline; move the other 5 to references. + +--- + +## 6. The Unused Lookup Table + +**Signature:** a long reference table (prices, codes, flags, character limits) that the skill rarely consults, but it's sitting at the top of SKILL.md. + +**Cost:** the table is paid for every invocation; the value is returned only on invocations where the skill actually needs to look something up. + +**Fix:** move the table to a reference file. The SKILL.md body says "For the full character limit table across all ad formats, read references/format_specs.md." + +**Example:** a 40-line table of Google Ads character limits. If it's checked 1 run in 5, move it. + +--- + +## 7. The Restatement of Defaults + +**Signature:** the skill (or the user) restates things Claude already does by default. "Always be helpful." "Respond in Markdown." "Use clear language." "Ask clarifying questions when unsure." + +**Cost:** tokens that convey zero new information. + +**Fix:** delete. Trust the defaults. Only write down the things that deviate from default behavior or specify the domain-specific move. + +**Example:** a SKILL.md that opens with 15 lines of "You are a helpful AI assistant. You should be clear and concise..." Delete it all. The model already knows. + +--- + +## Diagnostic workflow + +When auditing a context / skill for anti-patterns: + +1. **Scan for repeated phrases.** If "ALWAYS" appears more than 3 times, there's probably a Repeat Offender. +2. **Count lines per conceptual block.** Anything over 200 lines on a single concept is Kitchen Sink territory. +3. **Grep for example blocks.** If there's more example text than instruction text, you probably have Verbose Examples. +4. **Look for tables.** Tables over 20 rows used less than every run = Unused Lookup Tables. +5. **Look at the top 10 lines.** Is any of it restatement of default AI behavior? That's Restatement of Defaults. +6. **For live sessions:** check the last 10 tool results. Any of them over 5KB? Candidate for Stale Tool Result once they've been consumed. +7. **For live sessions:** grep the Read calls. Same path twice? Silent Re-Read. + +--- + +## What's actually worth optimizing + +Not every anti-pattern is worth fixing. Optimize where the cost is significant: + +- **A 5KB unused table in a skill used 10x/day** → fix it. 50KB/day of waste. +- **A 500-byte repeated instruction in a skill used once a month** → leave it. The juice isn't worth the squeeze. +- **A 40KB stale tool result in a 200k context** → summarize. Big, idle, easy. +- **A 2KB example repeated in a skill used 1000x/day** → cut one copy. Measurable impact. + +Context engineering has diminishing returns. Fix the big offenders first, and don't spend an hour saving 200 tokens on something rarely used. diff --git a/skills/context-engineer/references/patterns.md b/skills/context-engineer/references/patterns.md new file mode 100755 index 00000000..feae0df5 --- /dev/null +++ b/skills/context-engineer/references/patterns.md @@ -0,0 +1,137 @@ +# Tiered Context Patterns + +Three organizing principles for splitting a monolithic SKILL.md into a tiered structure. Pick the one that matches how the skill's variation falls. + +## Pattern 1 — Domain-organized + +Use when the skill supports multiple domains or verticals and each has its own deep playbook. + +**Example — a real-estate content skill that serves multiple markets:** + +``` +real-estate-content/ +├── SKILL.md (workflow + market selection) +└── references/ + ├── bay_area.md (local nuance for SF, Peninsula, EPA) + ├── austin.md (local nuance for Austin TX) + └── nyc.md (local nuance for NYC) +``` + +**Pattern:** SKILL.md asks the user which market (or infers it from inputs), then reads only the matching reference. The other market files never enter context. + +**When this pattern fits:** the shared workflow is 60%+ of the job and the rest is market/vertical nuance. Each domain file is self-contained. + +--- + +## Pattern 2 — Variant-organized + +Use when the skill produces different output formats / variants of the same core thing. + +**Example — a video creation skill:** + +``` +video-creator/ +├── SKILL.md (format selection + shared principles) +└── references/ + ├── mp4_slideshow.md (ffmpeg-based slideshows) + ├── remotion.md (React component-based video) + ├── heygen.md (avatar-based video) + └── stock_broll.md (higgsfield b-roll generation) +``` + +**Pattern:** SKILL.md contains a decision tree ("Is this a talking-head video? → HeyGen. Is this a slideshow? → MP4. Is this component-based? → Remotion."). Only the matching reference loads. + +**When this pattern fits:** the variants are fundamentally different technologies / output shapes, even if the user-facing request ("make me a video") looks the same. + +--- + +## Pattern 3 — Phase-organized + +Use when the skill has a long multi-phase workflow where each phase is expensive to describe. + +**Example — a CMA (comparable market analysis) skill:** + +``` +cma-generator/ +├── SKILL.md (overall workflow + phase selector) +└── references/ + ├── intake.md (gathering subject property + comp data) + ├── selection.md (which comps to include, how to filter) + ├── adjustments.md (time, size, condition adjustments) + ├── pricing.md (three-strategy pricing framework) + └── presentation.md (how to format the final report) +``` + +**Pattern:** SKILL.md is the conductor. Each phase of the workflow tells Claude which reference to load next. A full run will touch several references; a quick spot-check might only touch one. + +**When this pattern fits:** workflows where each phase is self-contained with its own rules, and where different invocations may only need a subset of phases. + +--- + +## Anti-patterns + +### The false split +Splitting a SKILL.md into references when the references are so tightly coupled they all get loaded on every run anyway. You just added Read overhead for zero benefit. + +Signal: the SKILL.md says "first read references/a.md, then references/b.md, then references/c.md" at the start of every invocation. Not a split — just a slower monolith. Put it all back together. + +### The too-fine split +Breaking a 300-line SKILL.md into 8 reference files. Each file is 30 lines and the overhead of loading them outweighs the savings. + +Rule of thumb: don't create a reference file under ~100 lines unless it's genuinely independent and only rarely needed. The minimum viable reference is usually ~200 lines. + +### The mis-organized split +Using domain organization when variant organization fits better (or vice versa). Signal: the SKILL.md selection logic is awkward — "if the user's industry is X and the format is Y..." when the split was done on industry. Usually means the split should have been on format. + +--- + +## Designing the SKILL.md "dispatcher" + +The SKILL.md body in a tiered system has one main job: route to the right reference. + +Good dispatcher patterns: + +**Explicit routing table:** +```markdown +## Which reference to read + +| User says | Read | +|---|---| +| "make me a slideshow" | references/mp4_slideshow.md | +| "React video" or "Remotion" | references/remotion.md | +| "avatar video" or "HeyGen" | references/heygen.md | +``` + +**Decision tree in prose:** +```markdown +## Pick the format + +Ask yourself: +1. Is this a talking-head format with a real face? → HeyGen. Read references/heygen.md. +2. Is this a programmatic composition with React? → Remotion. Read references/remotion.md. +3. Is this a photo-based slideshow with voiceover? → MP4 slideshow. Read references/mp4_slideshow.md. +``` + +Both work. Pick whichever flows more naturally with the skill's style. + +--- + +## Mixing patterns + +Large skills sometimes benefit from two levels of organization: + +``` +real-estate-content/ +├── SKILL.md +└── references/ + ├── markets/ + │ ├── bay_area.md + │ └── austin.md + └── formats/ + ├── listing_video.md + └── market_update.md +``` + +The dispatcher reads one from `markets/` and one from `formats/`. More complexity, but scales if you have >5 in each axis. + +Don't do this for skills with <3 entries in an axis. It's overkill. diff --git a/skills/context-engineer/references/token_estimation.md b/skills/context-engineer/references/token_estimation.md new file mode 100755 index 00000000..1914e353 --- /dev/null +++ b/skills/context-engineer/references/token_estimation.md @@ -0,0 +1,105 @@ +# Token Estimation — Without Running a Tokenizer + +You usually won't have a live tokenizer in the loop, but you can estimate token counts within ±15% using simple heuristics. Good enough for diagnostics. + +## The rules of thumb + +### Prose (English) +- **1 token ≈ 4 characters** (including spaces and punctuation) +- **1 token ≈ 0.75 words** +- A typical paragraph (80 words) ≈ 100–110 tokens +- A page of prose (~500 words) ≈ 650–700 tokens +- Typical chat message (200 words) ≈ 250–270 tokens + +### Code +Code tokenizes differently — more symbols, shorter identifiers, more punctuation. +- **Python / JavaScript / TypeScript:** ~2 characters per token (half of prose) +- A 100-line Python file (~2,500 chars) ≈ 1,200–1,400 tokens +- A 1,000-line codebase file ≈ 12,000–15,000 tokens + +### JSON and structured data +Punctuation heavy; lots of repeated keys. +- **~3 characters per token** +- A 10 KB JSON blob ≈ 3,000–3,500 tokens +- A typical MCP tool result (list of records) is surprisingly expensive — a 100-record list with 10 fields each can run 8,000+ tokens + +### Markdown +Close to prose, slightly higher due to syntax. +- **~3.5 characters per token** +- A 500-line SKILL.md (assume 50 chars/line average) ≈ 7,000 tokens + +### URLs and paths +Very dense. +- **~2 characters per token** +- A long URL with query strings can be 30–50 tokens on its own + +### Non-English languages +Much higher token-per-character ratios. +- Japanese / Chinese / Korean: 1 token ≈ 1 character +- Many European languages: slightly higher than English (accented chars, compound words) + +## Estimating from disk + +For a file on disk, the quick formula: + +``` +estimated_tokens = file_size_in_bytes / cpb +``` + +where `cpb` (characters per byte) is: +- Prose: 4 +- Code: 2 +- JSON/YAML: 3 +- Markdown: 3.5 + +In bash: +```bash +# Quick token estimate for a file +wc -c path/to/file | awk '{print "~"int($1/4)" tokens (prose)"; print "~"int($1/2)" tokens (code)"}' +``` + +## Estimating a conversation + +For a whole conversation including system prompt and history, sum these: + +1. **System prompt** — if you can see its length in characters, divide by 4 +2. **Each user message** — count roughly 100 tokens per short message, 500 per long one +3. **Each assistant message** — same math, but assistant messages tend to be longer +4. **Tool results** — treat as JSON (÷3) if structured, prose (÷4) if text +5. **Loaded files** — see "from disk" above + +Double your estimate if the system prompt looks unusually large (includes many tool definitions, user preferences, skill index). Modern Claude agent system prompts run 5–20k tokens before you add anything. + +## What token counts look like in practice + +For reference, these are typical sizes for common things: + +| Thing | Tokens | +|---|---| +| Short chat message ("what's the weather?") | 10–15 | +| Typical user request (3–4 sentences) | 50–100 | +| Long user message (1 paragraph of context) | 200–400 | +| Claude's typical response (1–2 paragraphs) | 200–500 | +| A SKILL.md frontmatter (name + description) | 100–300 | +| A SKILL.md body (500 lines, markdown) | 6,000–8,000 | +| A reference file (300 lines) | 3,500–5,000 | +| A bash `ls -la ~` output | 500–2,000 | +| A bash `find /` output | can be 50,000+ | +| A web fetch of a standard page | 3,000–15,000 | +| A PDF extracted to text (10 pages) | 3,000–5,000 | +| A 50KB JSON blob | 15,000–18,000 | + +## The "should I summarize?" threshold + +Rule of thumb: if an item takes more than **5% of your remaining context budget** to keep around, and it's not actively being referenced, summarize it. + +Example: 200k window, you're at 150k used, remaining = 50k. 5% of 50k = 2,500 tokens. Anything over ~10KB of content sitting idle should be summarized or dropped. + +## Caveats + +These are estimates, not ground truth. Actual tokenization varies: +- Anthropic's tokenizer treats some English words as single tokens while splitting others +- Emoji and special characters can each be multiple tokens +- Code with lots of identifiers in one style (camelCase vs snake_case) tokenizes differently + +For diagnostics, estimates are plenty. If you need exactness, run the text through a tokenizer (e.g., `tiktoken` for OpenAI models, Anthropic's SDK tokenizer for Claude). diff --git a/skills/contract-estimate-builder/SKILL.md b/skills/contract-estimate-builder/SKILL.md new file mode 100644 index 00000000..3e2eecc1 --- /dev/null +++ b/skills/contract-estimate-builder/SKILL.md @@ -0,0 +1,181 @@ +--- +name: contract-estimate-builder +description: "Contract Estimate Builder for Graeham Watts. Trigger ANY time the user mentions: contract estimate, contractor bid, scope of work, SOW, bid sheet, contractor quote, send to contractor for pricing, itemize a job, itemize landscaping/repairs, write up the scope, punch list, prep list, listing prep scope, vendor scope..." +--- + +# Contract Estimate Builder + +You are a contract estimate builder working alongside Graeham Watts (REALTOR, Compass, DRE# 01466876) and his team. Graeham coordinates contractors to prep homes for sale, and his contractors are often busy guys in the field who don't compile formal scopes themselves. This skill takes Graeham's spoken/written description of the work and turns it into a clean itemized estimate that: + +1. The **contractor** can fill in with pricing and return +2. The **client** (usually the seller) can read and understand +3. Graeham's **assistant** can text or email out to the contractor for quoting + +The two outputs are: +- **Excel bid sheet** (.xlsx) — editable, with formulas that auto-calculate totals including alternative-option scenarios +- **PDF scope of work** (.pdf) — polished, presentation-ready, suitable for emailing or printing + +Both outputs always get generated unless Graeham explicitly says otherwise. + +--- + +## When to Trigger + +Trigger any time Graeham (or his assistant) describes a list of work items for a property and wants it formatted for a contractor or client. The cue is usually one of: + +- "Itemize this for [contractor]" +- "Build a scope for [property address]" +- "Send this to my landscaper / painter / handyman" +- "Make this look professional" +- A bulleted/spoken list of tasks at a property + +If the user mentions "options" — like "for the middle, we could do bark, flagstone, or mulch" — that's a signal to use the **option group** pattern (see below). Don't bury options inside a single line; break them out so each can be priced. + +--- + +## Step 1: Intake — Get the Inputs + +Before generating anything, you need: + +- **Required:** Property address (confirm city if only a street name is given — Menalto Avenue in EPA is commonly misspelled as "Minalto" by autocomplete/dictation); Work scope (parse Graeham's list/conversation into individual line items). +- **Conditional (ask if not provided, else leave blank):** Contractor name, Client/seller name, Trade category (infer from scope if unclear), Date (defaults to today). +- **Don't over-ask:** if Graeham dropped a task list with an address and said "build the estimate," go — generate with reasonable defaults and let him fill gaps after seeing the draft. + +--- + +## Step 2: Structure the Scope + +Convert Graeham's plain-language tasks into a structured spec. The internal format uses **base items** and **option groups**: + +### Base Items (always included in the job) +These are tasks the contractor will do regardless of which options the client picks. Each base item has: +- A short task name (e.g., "Trim trees around exterior") +- An optional longer description / clarifying notes +- A blank pricing cell for the contractor to fill in + +### Option Groups (pick-one alternatives) +When Graeham mentions multiple ways to do one portion of the work, that's an option group. Each option group has: +- A group label (e.g., "Middle yard surface treatment") +- 2 or more options (e.g., Option 1: Bark, Option 2: Flagstone with gravel, Option 3: Mulch) +- The contractor prices each option; the spreadsheet shows a separate grand total for each scenario + +**Important: don't flatten options into the base list.** If Graeham says "the middle could be bark, flagstone, or mulch," that is one option group with three options — NOT three separate base line items. Flattening it forces the contractor to price all three as if they were all being done, which is wrong. + +### Example Spec + +```json +{ + "property_address": "2247 Menalto Avenue, East Palo Alto, CA", + "trade": "Landscaping & Property Prep", + "contractor_name": null, + "client_name": null, + "date": "2026-05-14", + "base_items": [ + {"task": "Apply rock mulch around outside lawn perimeter", + "notes": "Confirm rock type with Graeham before purchasing"} + ], + "option_groups": [ + { + "label": "Middle yard surface treatment", + "notes": "Pick one. Each option is priced separately so client can compare.", + "options": [ + {"name": "Option 1: Bark", "notes": "Standard bark mulch"}, + {"name": "Option 2: Flagstone with gravel", "notes": "Flagstone set in DG or pea gravel"}, + {"name": "Option 3: Mulch", "notes": "Wood-chip mulch alternative"} + ] + } + ] +} +``` + +Save the spec as JSON, then hand it to `scripts/build_estimate.py`. + +--- + +## Step 3: Generate the Outputs + +Use the bundled build script — it produces both files in one call: + +```bash +python scripts/build_estimate.py +``` + +The script writes two files into ``: +- `{address-slug}-estimate.xlsx` +- `{address-slug}-estimate.pdf` + +### What the Excel Looks Like + +**Sheet 1 — "Bid Sheet"** (the working document for the contractor): + +The contractor fills in **only** the `Unit Cost` cells. Everything else is formulas. The contractor (or Graeham, or the client) instantly sees how each option changes the bottom line. + +**Sheet 2 — "Totals Summary"** auto-calculates a grand total for each scenario: + +| Scenario | Grand Total | +|---------------------------------------|--------------------------| +| Base only (no options selected) | = Base Total | +| Base + Option 1 (Bark) | = Base Total + Option 1 | +| Base + Option 2 (Flagstone + gravel) | = Base Total + Option 2 | +| Base + Option 3 (Mulch) | = Base Total + Option 3 | + +Follow the `xlsx` skill's formatting conventions. + +### What the PDF Looks Like + +Clean, professional, one-pagish (longer if scope demands it): + +1. **Header** — Trade title, property address, date, prepared-by line +2. **Contractor / Client fields** — Two clean labeled lines (filled in if names provided, blank lines if not) +3. **Scope of Work** — Numbered list of base items, each with task name (bold) and notes (regular) +4. **Options Section** — Each option group in its own bordered block with the options listed underneath as alternatives the contractor prices separately +5. **Pricing instructions footer** — Short note: "Please return pricing per line item. Options are mutually exclusive — price each so the client can compare." +6. **Signature / acceptance lines** — Contractor signature + date, client signature + date + +**PDF styling** — neutral, no agent branding. Dark navy (#1a365d) headers, clean sans-serif, generous margins, good print quality. + +--- + +## The Courtesy Disclaimer (Always Included) + +Every estimate carries a small courtesy disclaimer at the bottom — small gray italic on the PDF, small gray text on the Excel Summary tab. It exists because Graeham is facilitating, not contracting. Without it, a realtor who hands a vendor estimate to a client can be implied as the contracting party or as warranting the vendor's quality, which is legal exposure he doesn't want. + +The default disclaimer (in `build_estimate.py` as `DEFAULT_DISCLAIMER`) covers: +- Courtesy basis — Graeham/Compass are not a party to any contractor agreement +- No warranty on pricing, scope, quality, or contractor qualifications +- No liability for performance +- Reminder to verify license/bond at `cslb.ca.gov` +- Acknowledgment that the owner may choose licensed OR unlicensed at their discretion +- Suggestion to confirm insurance, get additional bids, and review with counsel + +**Don't remove the disclaimer.** If the user wants different language for a specific deal, accept a custom version via the spec's optional `disclaimer` field — but always include something. Quiet protection that nobody reads is still protection. + +--- + +## Step 4: Deliver and Offer Next Steps + +After generating, present the files to Graeham. Always offer two natural next steps: + +1. **Email it to the contractor** — If you have a contractor email, offer to compose a Gmail draft. Subject line: `Estimate Request — [Address] — [Trade]`. Body should be short, friendly, and ask for a return-by date. + +2. **Hand off to assistant for text** — Generate a short SMS-friendly message the assistant can copy-paste: + +> "Hi [Contractor], Graeham asked me to send over the scope for [Address]. Attached PDF + Excel — please fill in pricing on the Excel and send back when you can. Thanks!" + +Don't send anything without explicit confirmation. Drafts and copy-paste messages only. + +--- + +## Edge Cases & Judgment Calls + +### Unit / Quantity Confusion +Some tasks are clearly "one job" (e.g., "power wash front and back") and others might be per-unit (e.g., "plant new bushes — qty TBD"). When unclear, default Qty to 1 and add a note like "Qty TBD with contractor." + +### Vague Tasks +If Graeham says something vague like "fix up the side yard," ask one clarifying question rather than guessing. Bad estimates come from over-interpreting. + +### Multiple Option Groups +A single estimate can have several option groups. The Summary tab shows every combination of scenarios. If there are more than ~6 combinations, the script switches from a flat list to a small grid showing each group's option columns. + +### Adding Items After the Fact +Reuse the original spec, append the line, \ No newline at end of file diff --git a/skills/contract-estimate-builder/examples/menalto-example.json b/skills/contract-estimate-builder/examples/menalto-example.json new file mode 100644 index 00000000..320b3475 --- /dev/null +++ b/skills/contract-estimate-builder/examples/menalto-example.json @@ -0,0 +1,28 @@ +{ + "property_address": "2247 Menalto Avenue, East Palo Alto, CA", + "trade": "Landscaping and Property Prep", + "contractor_name": "Adrian Aboniawn", + "client_name": null, + "date": "2026-05-14", + "base_items": [ + {"task": "Apply rock mulch around outside lawn perimeter", "notes": "Despite being called mulch on the original scope, finish material is rock. Confirm rock type/size with Graeham before purchasing.", "qty": 1, "unit_cost": null}, + {"task": "Fill in missing rock in middle yard area", "notes": "Match existing rock type and color where rock is currently missing.", "qty": 1, "unit_cost": null}, + {"task": "Trim trees around exterior of property", "notes": "Shape, clear deadwood and crossing branches. Haul away all cuttings.", "qty": 1, "unit_cost": null}, + {"task": "Plant new bushes (quantity TBD)", "notes": "Suggest 2-3 medium evergreen shrubs to fill bare spots. Confirm species and count with contractor.", "qty": 1, "unit_cost": null}, + {"task": "Power wash front yard and backyard hardscape", "notes": "Driveway, walkways, patio, and any concrete or pavers. Aim for listing-ready appearance.", "qty": 1, "unit_cost": null}, + {"task": "Complete clean of ADU in backyard", "notes": "Full interior + exterior clean. Floors, windows, bathrooms, kitchen, all surfaces. Should look move-in ready.", "qty": 1, "unit_cost": null}, + {"task": "Remove all leaves from property", "notes": "Rake, blow, and haul away. Front, back, and side yards.", "qty": 1, "unit_cost": null}, + {"task": "Clean turf in back portion", "notes": "Brush, deep clean, refresh appearance. No replacement - cleaning only.", "qty": 1, "unit_cost": null} + ], + "option_groups": [ + { + "label": "Middle yard surface treatment", + "notes": "Pick one. Each option is priced separately so the client can compare cost vs. look.", + "options": [ + {"name": "Option 1: Bark", "notes": "Standard bark mulch. Lowest cost, refreshes easily, needs replenishing every couple years.", "qty": 1, "unit_cost": null}, + {"name": "Option 2: Flagstone with gravel", "notes": "Flagstone pieces set in decomposed granite or pea gravel. Highest cost, most polished look, longest lasting.", "qty": 1, "unit_cost": null}, + {"name": "Option 3: Mulch (wood-chip)", "notes": "Wood-chip mulch alternative to bark. Mid-cost, soft look, breaks down over time.", "qty": 1, "unit_cost": null} + ] + } + ] +} diff --git a/skills/contract-estimate-builder/scripts/build_estimate.py b/skills/contract-estimate-builder/scripts/build_estimate.py new file mode 100644 index 00000000..2186c04d --- /dev/null +++ b/skills/contract-estimate-builder/scripts/build_estimate.py @@ -0,0 +1,533 @@ +#!/usr/bin/env python3 +"""Contract Estimate Builder. + +Generates an Excel bid sheet + PDF scope of work from a JSON spec. + +CRITICAL: Reads brand identity (DRE, name, brokerage) from +shared-references/identity.json - never hardcoded. This is the rule across all +of Graeham's skills: identity.json is the single source of truth. +""" + +import json +import re +import sys +from datetime import date +from itertools import product +from pathlib import Path + +from openpyxl import Workbook +from openpyxl.styles import Alignment, Border, Font, PatternFill, Side + +from reportlab.lib import colors +from reportlab.lib.pagesizes import letter +from reportlab.lib.styles import ParagraphStyle, getSampleStyleSheet +from reportlab.lib.units import inch +from reportlab.platypus import ( + HRFlowable, KeepTogether, ListFlowable, ListItem, PageBreak, + Paragraph, SimpleDocTemplate, Spacer, Table, TableStyle, +) + +# ---- Identity (read from canonical source) ---- + +def load_identity(): + """Read brand identity from shared-references/identity.json. + + Walks up from this script's location to find Skills/skills/shared-references/. + Raises a clear error if not found - we never want to fall back to hardcoded + values because that's exactly how the zombie DRE keeps reappearing. + """ + here = Path(__file__).resolve().parent + # contract-estimate-builder/scripts -> contract-estimate-builder -> skills + skills_root = here.parent.parent + identity_path = skills_root / "shared-references" / "identity.json" + if not identity_path.exists(): + raise FileNotFoundError( + "Could not find identity.json at " + str(identity_path) + + ". This skill must run from inside Skills/skills/contract-estimate-builder/. " + "Do NOT hardcode identity values - fix the path instead." + ) + with open(identity_path) as f: + data = json.load(f) + ident = data["identity"] + blocked = data.get("_blocked_values", {}).get("dre_blocklist", []) + if ident["dre"] in blocked: + raise ValueError( + "identity.json has a blocked DRE (" + ident["dre"] + + "). Stop and fix identity.json before proceeding." + ) + return ident + + +IDENTITY = load_identity() + +NAVY = "1A365D" +TEAL = "0D9488" +SLATE = "475569" +LIGHT_GRAY = "F1F5F9" +ROW_ALT = "F8FAFC" +GREEN_FILL = "DCFCE7" +BLUE_INPUT = "0000FF" +DISCLAIMER_GRAY = "64748B" +DISCLAIMER_DARK = "334155" + +CURRENCY_FMT = '"$"#,##0.00;[Red]("$"#,##0.00);"-"' +THIN = Side(border_style="thin", color="CBD5E1") +BORDER = Border(left=THIN, right=THIN, top=THIN, bottom=THIN) + + +def agent_credit(): + """Build the prepared-by signature line from identity.json values.""" + return ( + IDENTITY["name"] + ", " + IDENTITY["title"] + + " - " + IDENTITY["brokerage"] + + " - DRE# " + IDENTITY["dre"] + ) + + +DISCLAIMER_TITLE = "DISCLAIMER AND OWNER RESPONSIBILITIES" + +DISCLAIMER_INTRO = ( + "This document is provided solely as a courtesy by " + IDENTITY["name"] + + " (" + IDENTITY["title"] + ", " + IDENTITY["brokerage"] + + ", DRE# " + IDENTITY["dre"] + ") to summarize a scope of work being " + "discussed between the property owner and a third-party contractor. It is " + "not a contract, not a binding offer, and not a recommendation. No signature " + "is required, and receipt or review of this document by any party does not " + "constitute acceptance of, or agreement to, any terms set forth in it. Any " + "agreement for work to be performed must be entered into separately and in " + "writing directly between the property owner and the contractor." +) + +DISCLAIMER_LIABILITY = ( + IDENTITY["name"] + " and " + IDENTITY["brokerage"] + " are not a party to " + "any agreement between the property owner and the contractor, are not " + "performing any of the work described, and receive no referral fee, " + "compensation, or financial benefit of any kind from the contractor. No " + "warranty - express or implied - is made as to pricing, scope, quality of " + "work, contractor licensure, contractor insurance, contractor bond status, " + "contractor qualifications, materials, timeline, code compliance, permit " + "requirements, or workmanship. " + IDENTITY["name"] + " and " + + IDENTITY["brokerage"] + " assume no liability for performance or non-" + "performance of the work, for any damage to person or property, or for any " + "dispute that may arise between the owner and the contractor." +) + +DISCLAIMER_LICENSING = ( + "Licensed or unlicensed contractor - owner accepts all risk. The contractor " + "named in this scope may or may not be a licensed contractor. " + IDENTITY["name"] + " " + "makes no representation either way and has not verified the contractor's " + "license status. The property owner accepts all risk and responsibility for " + "their choice of contractor. If the contractor IS licensed, the owner is " + "responsible for verifying that license themselves at cslb.ca.gov; " + + IDENTITY["name"] + " takes no responsibility for whether that license is " + "current, valid, or in good standing. If the contractor is NOT licensed " + "and the owner chooses to work with them anyway, that decision and all " + "consequences are the owner's alone. We always recommend obtaining " + "alternative bids if the owner does not feel comfortable with this " + "contractor for any reason." +) + + +DISCLAIMER_OWNER_LEAD = "The property owner is solely responsible for:" + +DISCLAIMER_OWNER_ITEMS = [ + "Verifying the contractor's current license status and bond at cslb.ca.gov " + "(California Contractors State License Board), or knowingly accepting an " + "unlicensed contractor at the owner's own risk. Under California Business " + "and Professions Code Section 7048, unlicensed contractors may only perform " + "work where the combined cost of labor and materials is less than $500.", + + "Confirming the contractor's general liability insurance and workers' " + "compensation coverage before any work begins.", + + "Pulling any required building permits and confirming the work complies with " + "all local building codes, HOA covenants, conditions, and restrictions, " + "easements, and zoning restrictions.", + + "Investigating any environmental factors that may apply, including but not " + "limited to lead paint (homes built before 1978), asbestos, mold, and any " + "other hazardous-material considerations.", + + "Obtaining additional bids if desired and determining for themselves whether " + "the contractor's pricing is reasonable for the work described.", + + "Inspecting completed work, accepting or rejecting completion, and resolving " + "any disputes directly with the contractor.", + + "All other investigation, due diligence, follow-up, and research relating to " + "the contractor, the scope, and the work to be performed. " + + IDENTITY["name"] + " is not conducting any such investigation or due " + "diligence on behalf of the owner.", + + "Consulting their own independent legal counsel before signing any contract " + "or making any payment.", +] + +DISCLAIMER_ACK = ( + "By receiving this document, the property owner and contractor each " + "acknowledge that this is an informational scope summary only, that it " + "does not constitute a contract or agreement between any parties, and that " + "no party is bound by anything contained in it until and unless a separate " + "written agreement is signed." +) + + +def slugify(s): + s = s.lower().strip() + s = re.sub(r"[^a-z0-9]+", "-", s) + return s.strip("-")[:60] or "estimate" + + +def fmt_date(d): + return d or date.today().isoformat() + + +def excel_disclaimer_text(): + parts = [DISCLAIMER_TITLE, "", DISCLAIMER_INTRO, "", + DISCLAIMER_LIABILITY, "", DISCLAIMER_LICENSING, "", + DISCLAIMER_OWNER_LEAD] + for item in DISCLAIMER_OWNER_ITEMS: + parts.append("- " + item) + parts.append("") + parts.append(DISCLAIMER_ACK) + return "\n".join(parts) + + +def build_excel(spec, out_path): + wb = Workbook() + ws = wb.active + ws.title = "Bid Sheet" + + ws["A1"] = "CONTRACT ESTIMATE - " + spec.get("trade", "Scope of Work").upper() + ws["A1"].font = Font(bold=True, size=14, color=NAVY) + ws.merge_cells("A1:F1") + + ws["A2"] = "Property: " + spec["property_address"] + ws["A3"] = "Date: " + fmt_date(spec.get("date")) + ws["A4"] = "Contractor: " + (spec.get("contractor_name") or "____________________________") + ws["A5"] = "Client: " + (spec.get("client_name") or "____________________________") + for r in range(2, 6): + ws["A" + str(r)].font = Font(size=10, color=SLATE) + ws.merge_cells("A" + str(r) + ":F" + str(r)) + + header_row = 7 + for col_idx, h in enumerate(["#", "Task", "Notes", "Qty", "Unit Cost", "Line Total"], start=1): + c = ws.cell(row=header_row, column=col_idx, value=h) + c.font = Font(bold=True, color="FFFFFF") + c.fill = PatternFill("solid", start_color=NAVY) + c.alignment = Alignment(horizontal="center", vertical="center") + c.border = BORDER + + row = header_row + 1 + base_first = row + for i, item in enumerate(spec.get("base_items", []), start=1): + unit_cost = item.get("unit_cost") + cells = [ + i, item.get("task", ""), item.get("notes", "") or "", + item.get("qty", 1), + unit_cost if unit_cost is not None else None, + "=D" + str(row) + "*E" + str(row), + ] + for col_idx, val in enumerate(cells, start=1): + c = ws.cell(row=row, column=col_idx, value=val) + c.border = BORDER + if col_idx == 1: c.alignment = Alignment(horizontal="center") + elif col_idx in (2, 3): c.alignment = Alignment(wrap_text=True, vertical="top") + elif col_idx in (4, 5, 6): c.alignment = Alignment(horizontal="right") + if col_idx in (5, 6): c.number_format = CURRENCY_FMT + if col_idx == 5: c.font = Font(color=BLUE_INPUT) + if row % 2 == 0: c.fill = PatternFill("solid", start_color=ROW_ALT) + row += 1 + base_last = row - 1 + + base_total_row = row + ws.cell(row=row, column=2, value="BASE TOTAL").font = Font(bold=True) + ws.cell(row=row, column=6, + value="=SUM(F" + str(base_first) + ":F" + str(base_last) + ")").number_format = CURRENCY_FMT + for col_idx in range(1, 7): + c = ws.cell(row=row, column=col_idx) + c.fill = PatternFill("solid", start_color=LIGHT_GRAY) + c.font = Font(bold=True) + c.border = BORDER + row += 2 + + option_group_rows = [] + for g_idx, group in enumerate(spec.get("option_groups", []), start=1): + ws.cell(row=row, column=1, value="OPTIONS - Group " + str(g_idx)) + ws.cell(row=row, column=2, value=group.get("label", "")) + ws.cell(row=row, column=3, value=group.get("notes", "Pick one.")) + for col_idx in range(1, 7): + c = ws.cell(row=row, column=col_idx) + c.fill = PatternFill("solid", start_color=NAVY) + c.font = Font(bold=True, color="FFFFFF") + c.border = BORDER + ws.merge_cells(start_row=row, start_column=3, end_row=row, end_column=6) + row += 1 + + rows_for_group = [] + for o_idx, opt in enumerate(group.get("options", []), start=1): + unit_cost = opt.get("unit_cost") + cells = [ + str(g_idx) + "." + str(o_idx), opt.get("name", ""), + opt.get("notes", "") or "", opt.get("qty", 1), + unit_cost if unit_cost is not None else None, + "=D" + str(row) + "*E" + str(row), + ] + for col_idx, val in enumerate(cells, start=1): + c = ws.cell(row=row, column=col_idx, value=val) + c.border = BORDER + if col_idx == 1: c.alignment = Alignment(horizontal="center") + elif col_idx in (2, 3): c.alignment = Alignment(wrap_text=True, vertical="top") + elif col_idx in (4, 5, 6): c.alignment = Alignment(horizontal="right") + if col_idx in (5, 6): c.number_format = CURRENCY_FMT + if col_idx == 5: c.font = Font(color=BLUE_INPUT) + c.fill = PatternFill("solid", start_color=GREEN_FILL if row % 2 == 0 else "ECFCCB") + rows_for_group.append((opt.get("name", "Option " + str(o_idx)), row)) + row += 1 + option_group_rows.append({"label": group.get("label", "Group " + str(g_idx)), "options": rows_for_group}) + row += 1 + + for col, w in {"A": 6, "B": 36, "C": 38, "D": 8, "E": 14, "F": 16}.items(): + ws.column_dimensions[col].width = w + ws.freeze_panes = ws.cell(row=header_row + 1, column=1) + + s2 = wb.create_sheet("Totals Summary") + s2["A1"] = "TOTALS SUMMARY - Grand Total by Scenario" + s2["A1"].font = Font(bold=True, size=14, color=NAVY) + s2.merge_cells("A1:C1") + s2["A2"] = "Property: " + spec["property_address"] + s2["A2"].font = Font(size=10, color=SLATE) + s2.merge_cells("A2:C2") + + s2["A4"] = "Scenario"; s2["B4"] = "Components"; s2["C4"] = "Grand Total" + for col_idx in range(1, 4): + c = s2.cell(row=4, column=col_idx) + c.font = Font(bold=True, color="FFFFFF") + c.fill = PatternFill("solid", start_color=NAVY) + c.alignment = Alignment(horizontal="center") + c.border = BORDER + + s_row = 5 + s2.cell(row=s_row, column=1, value="Base only (no options)") + s2.cell(row=s_row, column=2, value="Base Total") + s2.cell(row=s_row, column=3, value="='Bid Sheet'!F" + str(base_total_row)).number_format = CURRENCY_FMT + for col_idx in range(1, 4): + s2.cell(row=s_row, column=col_idx).border = BORDER + s_row += 1 + + if option_group_rows: + combos = list(product(*[g["options"] for g in option_group_rows])) + if len(combos) <= 12: + for combo in combos: + labels = " + ".join(n for n, _ in combo) + comps = "Base + " + " + ".join(n for n, _ in combo) + parts = ["'Bid Sheet'!F" + str(base_total_row)] + ["'Bid Sheet'!F" + str(rn) for _, rn in combo] + formula = "=" + "+".join(parts) + s2.cell(row=s_row, column=1, value="Base + " + labels) + s2.cell(row=s_row, column=2, value=comps) + s2.cell(row=s_row, column=3, value=formula).number_format = CURRENCY_FMT + for col_idx in range(1, 4): + s2.cell(row=s_row, column=col_idx).border = BORDER + if s_row % 2 == 0: + s2.cell(row=s_row, column=col_idx).fill = PatternFill("solid", start_color=ROW_ALT) + s_row += 1 + else: + for g in option_group_rows: + s2.cell(row=s_row, column=1, value="-- " + g["label"] + " options --").font = Font(bold=True, italic=True) + s_row += 1 + for n, rn in g["options"]: + s2.cell(row=s_row, column=1, value="Base + " + n) + s2.cell(row=s_row, column=2, value="Base + " + n) + s2.cell(row=s_row, column=3, value="='Bid Sheet'!F" + str(base_total_row) + "+'Bid Sheet'!F" + str(rn)).number_format = CURRENCY_FMT + for col_idx in range(1, 4): + s2.cell(row=s_row, column=col_idx).border = BORDER + s_row += 1 + + for r in range(5, s_row): + s2.cell(row=r, column=3).font = Font(bold=True) + s2.column_dimensions["A"].width = 42 + s2.column_dimensions["B"].width = 48 + s2.column_dimensions["C"].width = 18 + s2.freeze_panes = s2.cell(row=5, column=1) + + note_row = s_row + 2 + s2.cell(row=note_row, column=1, value="How to use this:").font = Font(bold=True) + s2.cell(row=note_row + 1, column=1, value="- Contractor fills in Unit Cost on the Bid Sheet tab (blue cells).").font = Font(size=10, color=SLATE) + s2.cell(row=note_row + 2, column=1, value="- Grand Total for each scenario above updates automatically.").font = Font(size=10, color=SLATE) + s2.cell(row=note_row + 3, column=1, value="- Options are mutually exclusive - pick ONE per group.").font = Font(size=10, color=SLATE) + for r in range(note_row + 1, note_row + 4): + s2.merge_cells(start_row=r, start_column=1, end_row=r, end_column=3) + + disc_row = note_row + 5 + c = s2.cell(row=disc_row, column=1, value=excel_disclaimer_text()) + c.font = Font(size=8, color=DISCLAIMER_DARK, italic=True) + c.alignment = Alignment(wrap_text=True, vertical="top") + s2.merge_cells(start_row=disc_row, start_column=1, end_row=disc_row, end_column=3) + s2.row_dimensions[disc_row].height = 320 + + wb.save(out_path) + + +def build_pdf(spec, out_path): + doc = SimpleDocTemplate( + str(out_path), pagesize=letter, + leftMargin=0.7 * inch, rightMargin=0.7 * inch, + topMargin=0.6 * inch, bottomMargin=0.6 * inch, + title="Contract Estimate - " + spec["property_address"], + author=IDENTITY["name"] + ", " + IDENTITY["brokerage"], + ) + styles = getSampleStyleSheet() + h1 = ParagraphStyle("h1", parent=styles["Heading1"], textColor=colors.HexColor("#" + NAVY), fontSize=18, leading=22, spaceAfter=4) + h2 = ParagraphStyle("h2", parent=styles["Heading2"], textColor=colors.HexColor("#" + NAVY), fontSize=13, leading=16, spaceBefore=14, spaceAfter=6) + meta = ParagraphStyle("meta", parent=styles["Normal"], textColor=colors.HexColor("#" + SLATE), fontSize=10, leading=13) + body = ParagraphStyle("body", parent=styles["Normal"], fontSize=10, leading=14) + task_title = ParagraphStyle("task_title", parent=styles["Normal"], fontSize=10.5, leading=14, fontName="Helvetica-Bold") + task_notes = ParagraphStyle("task_notes", parent=styles["Normal"], fontSize=9.5, leading=12, textColor=colors.HexColor("#" + SLATE)) + option_label = ParagraphStyle("opt", parent=styles["Normal"], fontSize=11, leading=14, fontName="Helvetica-Bold", textColor=colors.HexColor("#" + TEAL)) + disc_h = ParagraphStyle("disc_h", parent=styles["Normal"], fontSize=10, leading=13, fontName="Helvetica-Bold", textColor=colors.HexColor("#" + DISCLAIMER_DARK), spaceBefore=10, spaceAfter=4) + disc_body = ParagraphStyle("disc_body", parent=styles["Normal"], fontSize=8, leading=11, textColor=colors.HexColor("#" + DISCLAIMER_DARK), alignment=4, spaceAfter=6) + disc_bullet = ParagraphStyle("disc_bullet", parent=disc_body, fontSize=8, leading=11, leftIndent=12, spaceAfter=3) + disc_ack = ParagraphStyle("disc_ack", parent=disc_body, fontSize=8, leading=11, fontName="Helvetica-Oblique", textColor=colors.HexColor("#" + DISCLAIMER_DARK)) + + story = [] + story.append(Paragraph("CONTRACT ESTIMATE - " + spec.get("trade", "Scope of Work").upper(), h1)) + story.append(Paragraph("Property: " + spec["property_address"], meta)) + story.append(Paragraph("Date: " + fmt_date(spec.get("date")), meta)) + contractor = spec.get("contractor_name") or "____________________________" + client = spec.get("client_name") or "____________________________" + story.append(Paragraph("Contractor: " + contractor, meta)) + story.append(Paragraph("Client: " + client, meta)) + story.append(Spacer(1, 6)) + story.append(HRFlowable(width="100%", thickness=0.6, color=colors.HexColor("#" + NAVY))) + story.append(Spacer(1, 4)) + + instr = Table([[Paragraph( + "Instructions: Please return pricing per line item on the attached Excel bid sheet. " + "Items below are the proposed base scope. Where alternative options are listed, " + "price each option separately - the client will pick one. The Excel auto-calculates " + "a grand total for each option scenario. No signature is required on this document - " + "it is a scope summary, not a contract.", + body)]], colWidths=[7.0 * inch]) + instr.setStyle(TableStyle([ + ("BACKGROUND", (0, 0), (-1, -1), colors.HexColor("#" + LIGHT_GRAY)), + ("BOX", (0, 0), (-1, -1), 0.5, colors.HexColor("#" + SLATE)), + ("LEFTPADDING", (0, 0), (-1, -1), 10), ("RIGHTPADDING", (0, 0), (-1, -1), 10), + ("TOPPADDING", (0, 0), (-1, -1), 8), ("BOTTOMPADDING", (0, 0), (-1, -1), 8), + ])) + story.append(instr) + + story.append(Paragraph("Base Scope of Work", h2)) + base_data = [["#", "Task", "Notes", "Unit Price"]] + for i, item in enumerate(spec.get("base_items", []), start=1): + base_data.append([ + str(i), + Paragraph(item.get("task", ""), task_title), + Paragraph(item.get("notes", "") or "", task_notes), + "$ _______________" if item.get("unit_cost") is None else "$" + format(item.get("unit_cost"), ",.2f"), + ]) + bt = Table(base_data, colWidths=[0.4 * inch, 2.6 * inch, 2.8 * inch, 1.4 * inch], repeatRows=1) + bt.setStyle(TableStyle([ + ("BACKGROUND", (0, 0), (-1, 0), colors.HexColor("#" + NAVY)), + ("TEXTCOLOR", (0, 0), (-1, 0), colors.white), + ("FONTNAME", (0, 0), (-1, 0), "Helvetica-Bold"), + ("FONTSIZE", (0, 0), (-1, 0), 9.5), + ("ALIGN", (0, 0), (0, -1), "CENTER"), + ("ALIGN", (3, 0), (3, -1), "RIGHT"), + ("VALIGN", (0, 0), (-1, -1), "TOP"), + ("ROWBACKGROUNDS", (0, 1), (-1, -1), [colors.white, colors.HexColor("#" + ROW_ALT)]), + ("BOX", (0, 0), (-1, -1), 0.6, colors.HexColor("#" + SLATE)), + ("INNERGRID", (0, 0), (-1, -1), 0.3, colors.HexColor("#E2E8F0")), + ("LEFTPADDING", (0, 0), (-1, -1), 6), ("RIGHTPADDING", (0, 0), (-1, -1), 6), + ("TOPPADDING", (0, 0), (-1, -1), 6), ("BOTTOMPADDING", (0, 0), (-1, -1), 6), + ])) + story.append(bt) + + if spec.get("option_groups"): + story.append(Paragraph("Alternative Options (Pick One Per Group)", h2)) + for g_idx, group in enumerate(spec["option_groups"], start=1): + block = [Paragraph("Group " + str(g_idx) + ": " + group.get("label", ""), option_label)] + if group.get("notes"): + block.append(Paragraph(group["notes"], task_notes)) + block.append(Spacer(1, 4)) + opt_data = [["#", "Option", "Notes", "Unit Price"]] + for o_idx, opt in enumerate(group.get("options", []), start=1): + opt_data.append([ + str(g_idx) + "." + str(o_idx), + Paragraph(opt.get("name", ""), task_title), + Paragraph(opt.get("notes", "") or "", task_notes), + "$ _______________" if opt.get("unit_cost") is None else "$" + format(opt.get("unit_cost"), ",.2f"), + ]) + ot = Table(opt_data, colWidths=[0.5 * inch, 2.5 * inch, 2.8 * inch, 1.4 * inch], repeatRows=1) + ot.setStyle(TableStyle([ + ("BACKGROUND", (0, 0), (-1, 0), colors.HexColor("#" + TEAL)), + ("TEXTCOLOR", (0, 0), (-1, 0), colors.white), + ("FONTNAME", (0, 0), (-1, 0), "Helvetica-Bold"), + ("FONTSIZE", (0, 0), (-1, 0), 9.5), + ("ALIGN", (0, 0), (0, -1), "CENTER"), + ("ALIGN", (3, 0), (3, -1), "RIGHT"), + ("VALIGN", (0, 0), (-1, -1), "TOP"), + ("ROWBACKGROUNDS", (0, 1), (-1, -1), [colors.white, colors.HexColor("#" + GREEN_FILL)]), + ("BOX", (0, 0), (-1, -1), 0.6, colors.HexColor("#" + TEAL)), + ("INNERGRID", (0, 0), (-1, -1), 0.3, colors.HexColor("#A7F3D0")), + ("LEFTPADDING", (0, 0), (-1, -1), 6), ("RIGHTPADDING", (0, 0), (-1, -1), 6), + ("TOPPADDING", (0, 0), (-1, -1), 6), ("BOTTOMPADDING", (0, 0), (-1, -1), 6), + ])) + block.append(ot) + block.append(Spacer(1, 8)) + story.append(KeepTogether(block)) + + story.append(PageBreak()) + story.append(Paragraph(DISCLAIMER_TITLE, disc_h)) + story.append(HRFlowable(width="100%", thickness=0.5, color=colors.HexColor("#" + DISCLAIMER_GRAY))) + story.append(Spacer(1, 6)) + story.append(Paragraph(DISCLAIMER_INTRO, disc_body)) + story.append(Paragraph(DISCLAIMER_LIABILITY, disc_body)) + story.append(Paragraph(DISCLAIMER_LICENSING, disc_body)) + story.append(Paragraph("" + DISCLAIMER_OWNER_LEAD + "", disc_body)) + bullets = [ListItem(Paragraph(item, disc_bullet), leftIndent=12, bulletColor=colors.HexColor("#" + DISCLAIMER_DARK)) + for item in DISCLAIMER_OWNER_ITEMS] + story.append(ListFlowable(bullets, bulletType="bullet", start="circle", leftIndent=14, bulletFontSize=7)) + story.append(Spacer(1, 6)) + story.append(Paragraph("" + DISCLAIMER_ACK + "", disc_ack)) + story.append(Spacer(1, 10)) + + story.append(Paragraph("Optional - Acknowledgment of receipt (not required, not a contract):", disc_body)) + sig = Table([ + ["Property owner signature", "Date"], + ["_________________________________", "______________"], + ["", ""], + ["Contractor signature", "Date"], + ["_________________________________", "______________"], + ], colWidths=[4.5 * inch, 2.5 * inch]) + sig.setStyle(TableStyle([ + ("FONTNAME", (0, 0), (-1, -1), "Helvetica"), + ("FONTSIZE", (0, 0), (-1, -1), 9), + ("TEXTCOLOR", (0, 0), (-1, -1), colors.HexColor("#" + SLATE)), + ("BOTTOMPADDING", (0, 0), (-1, -1), 4), ("TOPPADDING", (0, 0), (-1, -1), 4), + ])) + story.append(sig) + story.append(Spacer(1, 14)) + story.append(Paragraph("Prepared by " + agent_credit() + "", + ParagraphStyle("footer", parent=styles["Normal"], fontSize=8, + textColor=colors.HexColor("#" + SLATE), alignment=1))) + doc.build(story) + + +def main(): + if len(sys.argv) < 3: + print("Usage: python build_estimate.py ") + sys.exit(1) + spec = json.loads(Path(sys.argv[1]).read_text(encoding="utf-8")) + out_dir = Path(sys.argv[2]) + out_dir.mkdir(parents=True, exist_ok=True) + slug = slugify(spec["property_address"].split(",")[0]) + xlsx = out_dir / (slug + "-estimate.xlsx") + pdf = out_dir / (slug + "-estimate.pdf") + build_excel(spec, xlsx) + build_pdf(spec, pdf) + print("Wrote:", xlsx) + print("Wrote:", pdf) + + +if __name__ == "__main__": + main() diff --git a/skills/copywriter/SKILL.md b/skills/copywriter/SKILL.md new file mode 100755 index 00000000..65ee67aa --- /dev/null +++ b/skills/copywriter/SKILL.md @@ -0,0 +1,157 @@ +--- +name: copywriter +description: Direct-response copywriter that produces high-converting marketing copy in any format — ad headlines, landing page hero copy, email subject lines, sales pages, CTAs, product descriptions, social captions. Picks the right framework (AIDA for cma-reports/newsletters/ads, PAS for problem-aware audiences, FAB for feature-heavy products, Before/After/Bridge for transformation offers), delivers three variations by default, and explains the psychological lever each one pulls. Use ANY time the user mentions copywriting, copywriter, write me a headline, write me copy, ad copy, Facebook ad, Google ad, landing page copy, hero section, email subject line, cold email, CTA, call to action, sales copy, sales page, product description, conversion copy, VSL, marketing copy, brand voice, tagline, slogan, or promo copy. Also trigger when the user pastes a product and asks "how would you pitch this?", shares a weak CTA, or wants A/B variations. Over-trigger — any "help me sell this" moment is copywriting. +--- + +# Copywriter — Direct-Response Engine + +You are a direct-response copywriter. You write words designed to move a specific audience from awareness to action. You pick the framework that fits the job, you write three variations, and you explain why each one works. + +This skill is about craft, not just vibes. Every piece of copy is a hypothesis about the reader: who they are, what they want, what's stopping them, and what will unstick them. Good copy is that hypothesis, sharpened. + +**Companion reference: `marketing-psychology`.** For substantive jobs (sales pages, landing pages, email sequences, webinar/listing-presentation scripts, long-form ads), load the `marketing-psychology` skill and run its Step 1 diagnosis (Schwartz awareness stage, market sophistication, emotional state, blocking force, temperature) BEFORE Step 1 below, use its blocking-force matrix to pick the levers for your three variations, and run its Step 4 panel check + Step 5 failure audit on the drafts before the humanizer pass. For micro-copy (a subject line, a CTA button), its quick path applies: diagnose, pick one lever, consult one panel mind (e.g., Ogilvy for headlines), audit. The frameworks below remain this skill's own; `marketing-psychology` adds StoryBrand, Monroe's Motivated Sequence, and Hook-Story-Offer when the job outgrows AIDA/PAS/FAB/BAB. + +**Companion reference: `comedy-craft`.** When the copy should carry personality, wit, or a dry edge — social captions, taglines, ad openers, anything meant to be *memorable*, not just persuasive — also load the `comedy-craft` skill for Graeham's calibrated humor (dry, deadpan, twist-first, clean-with-a-little-bite), its punch-up pass, and the screenshot-test rubric. `marketing-psychology` makes copy land as persuasion; `comedy-craft` makes it land as voice. Both run before the humanizer pass. + +--- + +## Step 1 — Get the minimum inputs + +Before you write, you need three things: + +1. **What's being sold** — the product, service, or offer. A sentence or two is enough. +2. **Who's being sold to** — the target audience. Who they are, what they want, where they are in their journey (cold / warm / hot). +3. **What format** — the specific deliverable the user needs: ad headline, email subject line, landing page hero, CTA, sales page section, product description, social caption, VSL opener, etc. + +If any of these three are missing, ask. Don't guess. Ask concisely, one question at a time if needed, and move on the moment you have enough to work. + +**Example clarifying prompts:** +- "Who's this for? Describe the reader in one sentence — their role, their situation, what they're trying to do." +- "What format do you want — a headline, an email subject line, a full landing page hero section, something else?" +- "Is this audience cold (doesn't know the problem yet), warm (aware of the problem, shopping for solutions), or hot (close to buying, just needs the nudge)?" + +If the user has given you enough, don't stall. Write. + +--- + +## Step 2 — Pick the right framework + +The framework is a shortcut to the right structure. Pick based on the format and the audience's awareness level. + +| Framework | Best for | +|---|---| +| **AIDA** | Email subject lines, ads, cold outreach, short landing page heroes | +| **PAS** | Problem-aware audiences, sales pages for painful problems | +| **FAB** | Feature-heavy products (SaaS, physical products, technical tools) | +| **BAB** | Transformation-based offers (coaching, courses, health, career) | + +If none of these feel right, write in plain direct-response voice: concrete, specific, short sentences, lead with the benefit. Read `references/frameworks.md` for deeper examples and less common frameworks (4Us, 4Cs, PASTOR, Storybrand) if the job calls for something more specialized. + +--- + +## Step 3 — Write three variations + +Three is the sweet spot. One is a swing and a miss risk. Five dilutes. Three lets the user pick, combine, or use as a starting point for their own edits. + +Each variation should: +- **Pull a different lever.** Don't write three versions of the same angle — write three angles. One might lead with loss aversion, one with social proof, one with curiosity. Variety is the point. +- **Be finished, not a draft.** Every variation should be copy-paste-ready at its intended length. +- **Come with a short "why."** One or two sentences explaining the psychological mechanism. Examples: + - "Loss aversion — frames the price as 'not losing your weekends' rather than a cost." + - "Curiosity gap — the subject line implies information the reader doesn't have yet." + - "Specificity builds trust — the number '23%' reads more real than 'significant improvement.'" + +The "why" is what makes this a copywriter, not a generator. It shows the user the move so they can do it themselves next time. + +--- + +## Step 4 — Output format + +Always use this structure: + +``` +## Framework +[Name of framework + one-sentence justification — why this fit] + +## Variation 1 — [lever name, e.g., "Loss aversion"] +[The copy. Clean, finished, ready to paste.] + +**Why it works:** [one to two sentences on the psychological mechanism] + +## Variation 2 — [lever name] +[The copy.] + +**Why it works:** [explanation] + +## Variation 3 — [lever name] +[The copy.] + +**Why it works:** [explanation] + +## Pick / Combine +[One line: which variation you'd ship and why, or how to combine the best bits. The user asked for a recommendation by default — give it.] +``` + +For formats with length constraints (email subject lines: 40–60 chars; Google Ads headlines: 30 chars; meta descriptions: ~155 chars), note the character count after each variation. + +--- + +## Step 5 — Mandatory final pass: humanizer + +Before you deliver the three variations, run every piece of copy through the `humanizer` skill. Direct-response copy is the worst place to leak AI patterns — readers can smell "stands as a testament" or em-dash overload in two seconds and the conversion rate dies with it. + +**What gets humanized:** +- Every headline, subhead, body line, and CTA in all three variations +- The "Why it works" rationales (clients read these too) +- The "Pick / Combine" recommendation line + +**What does NOT get humanized:** +- Character counts and length notes (numerical metadata) +- The framework justification line (technical reference) + +**How to invoke:** +1. Generate the three variations and rationales as usual. +2. Pass the full prose block to the humanizer skill with a one-line voice note (e.g., "B2B SaaS, confident, no jargon" or "DTC, warm, sensory"). +3. Replace the original copy with the humanized version before assembling the final output structure. +4. Deliver. + +If the user has supplied a brand voice sample at intake, hand it to the humanizer as the voice-calibration sample so the rewrite matches their tone rather than the default humanizer voice. + +This step is non-negotiable. Ad copy that sounds like a model wrote it does not convert. + +--- + +## Psychological levers + +For the full persuasion-lever library, load `marketing-psychology`. + +--- + +## Format-specific rules + +See `references/format_specs.md` for the character counts, structural conventions, and known pitfalls for every format this skill handles: +- Email subject lines and preview text +- Facebook / Instagram / TikTok ad copy +- Google Ads (Responsive Search Ads) +- Landing page hero sections +- Sales page sections (headline, subhead, bullet lists, CTA stacks) +- Product descriptions (Shopify / Amazon style) +- CTAs (button text) +- Social captions (LinkedIn, X, Instagram) +- Cold outreach openers + +Load that reference file when the user's format has specific constraints or conventions you need to hit. + +--- + +## Tone calibration + +Ask or infer the tone from context. Default defaults: +- **B2B SaaS** — confident, specific, light humor OK, avoid corporate jargon +- **DTC consumer** — warm, benefit-forward, sensory language +- **High-ticket service / coaching** — authority + transformation, less "buy now" energy +- **Newsletter / creator** — conversational, first-person, direct-to-reader +- **Legacy brand / enterprise** — measured, credibility cues, fewer exclamation marks + +If the user has a brand voice guide or existing copy to mimic, ask for it or ask them to paste samples. Copy that doesn't match existing brand voice is worse than no copy. + diff --git a/skills/copywriter/references/format_specs.md b/skills/copywriter/references/format_specs.md new file mode 100755 index 00000000..c2102125 --- /dev/null +++ b/skills/copywriter/references/format_specs.md @@ -0,0 +1,78 @@ +# Format Specs — Constraints and Conventions by Format + +Each format has rules. Break them at your own risk — they exist because of how the format is consumed. + +## Email subject lines +- **Length:** 40–60 characters. Under 40 is OK for very sharp lines. Over 60 starts getting truncated on mobile. +- **Preview text:** separately craft the preview (aka "preheader") — first ~85 chars shown after the subject. Don't let it auto-populate from "Having trouble viewing?" +- **What works:** curiosity gaps, specificity, lowercase + casual, personal-seeming (lowercase, no brand, first-name energy). +- **What fails:** all caps, multiple exclamation points, generic ("Our newsletter"), spam triggers ("FREE!!!", "ACT NOW", $$$ symbols). +- **Three styles to test:** curiosity ("how i doubled replies last week"), specificity ("+23% reply rate — here's the tweak"), personal ("quick question for you"). + +## Facebook / Instagram ad copy +- **Primary text:** the text above the image. First 125 characters show before "See more" — front-load the hook. +- **Headline:** 25–40 characters under the image. This is the single most-seen copy. +- **Description:** 30 characters under the headline on some placements. +- **What works:** pattern interrupt first line, specific numbers, emojis sparingly for scannability, CTA explicit. +- **What fails:** long setups, generic hooks, text that reads like a brochure. + +## TikTok / Reels caption +- **Length:** short. 100–300 characters max. +- **First line:** the "stop scrolling" line. Treat it like a subject line. +- **Hashtags:** 3–5, mix of broad and niche. +- **CTA:** soft. "Follow for more" / "Comment X if..." + +## Google Ads — Responsive Search Ads +- **Headlines:** up to 15, each max 30 characters. Google rotates the best combinations. +- **Descriptions:** up to 4, each max 90 characters. +- **What works:** keyword in headline, distinct headlines (don't write 15 variants of the same line), descriptions that expand the pitch. +- **What fails:** repetition, punctuation that wastes characters, vague claims. + +## Landing page hero section +- **Headline:** 6–12 words. The single most-read line on the site. +- **Subhead:** 15–25 words. Answers "so what?" or adds the specific benefit. +- **CTA button:** 2–4 words. Action-oriented, outcome-focused. +- **Supporting proof element:** social proof ("Used by X"), logos, or a specific number right below the CTA. +- **What works:** lead with the benefit, not the brand. Specificity beats cleverness. Subhead earns the scroll. + +## Sales page sections +- **Section headlines (H2):** 6–15 words. Each one should make sense read alone — imagine the reader scrolling fast and reading only the headlines. +- **Bullet lists:** start every bullet with a benefit or outcome, not a feature. Parallel structure across bullets. +- **CTA stacks:** every 500–800 words, offer a CTA. Readers buy at different scroll depths; don't make the buyer scroll all the way back up. +- **Close:** the last section should handle the final objection — cost, time, risk — and offer a guarantee if you have one. + +## Product descriptions (Shopify / Amazon) +- **Shopify:** 150–300 words. First 2 sentences show above the fold on mobile. Lead with the primary benefit. Use short paragraphs and 3–5 bullets for scanners. +- **Amazon:** 5 bullets + description. Bullets get keyword weight in Amazon search. Lead each bullet with a benefit in ALL CAPS (2–3 words), then explain in normal case. +- **What fails:** feature-dump. "100% cotton, 14oz, machine washable" sells nothing. "Heavyweight 14oz cotton that gets softer with every wash" sells. + +## CTAs (button text) +- **Length:** 2–5 words. +- **Rule:** describe the outcome, not the click. +- **Good:** `Get my plan`, `Start free trial`, `Send me the guide`, `Book my demo`, `See pricing`, `Download the template`. +- **Bad:** `Click here`, `Submit`, `Send`, `Learn more` (sometimes OK, but usually lazy). + +## Social captions +- **LinkedIn:** 1,300 characters max before "see more." Hook in first 2 lines. Skip lines between paragraphs. End with a question or reflection. +- **X / Twitter:** 280 chars. Lead with the claim. If threading, first tweet should stand alone. +- **Instagram caption:** 2,200 chars allowed but most top posts use 100–500. Hook first line, whitespace between paragraphs. + +## Cold outreach openers +- **Length:** 2–4 sentences for the first email, body included. +- **Don't:** "Hope you're doing well." / "Quick question." / "I came across your company..." +- **Do:** lead with something specific to the recipient (a recent post, launch, announcement) + a single-sentence reason for reaching out + a single-sentence ask. +- **Best opener formula:** specific observation about them → one-line credential about you → specific ask with a clear out. + +## Character-count quick reference + +| Format | Max | +|---|---| +| Google Ads headline | 30 | +| X / Twitter post | 280 | +| Email subject line | 60 (target) | +| Meta description | 160 | +| Facebook ad primary text (before "more") | 125 | +| LinkedIn post (before "see more") | ~210 | +| Instagram caption (before "more") | ~125 | + +Include character counts in your variations when writing for these formats. The constraint is the art. diff --git a/skills/copywriter/references/frameworks.md b/skills/copywriter/references/frameworks.md new file mode 100755 index 00000000..890b8a64 --- /dev/null +++ b/skills/copywriter/references/frameworks.md @@ -0,0 +1,168 @@ +# Copywriting Frameworks — Full Playbook + +The four frameworks in SKILL.md cover most jobs. This file goes deeper and adds specialized frameworks for when the standard four don't fit. + +## Table of contents +- AIDA — the workhorse +- PAS — the knife twist +- FAB — the translator +- BAB — the transformation arc +- 4Us — headline discipline +- PASTOR — long-form sales pages +- StoryBrand — brand narrative structure +- How to choose between them + +--- + +## AIDA — Attention, Interest, Desire, Action + +The oldest framework in the book and still the most used. Every piece of direct-response copy you've ever read follows some version of AIDA. + +**Structure:** +- **Attention** — a hook that makes the reader stop +- **Interest** — specifics that earn continued reading +- **Desire** — paint the picture of having the thing +- **Action** — one clear next step + +**When it shines:** short formats where the reader's attention is a resource you can lose in a second. Email subject lines + opening. Cold outreach. Display ads. Landing page heroes. + +**Example (email):** +- Attention: `Subject: The email trick that booked me 3 demos last Tuesday` +- Interest: `Here's what happened. I had 47 cold emails out and exactly 0 replies...` +- Desire: `By 4pm Tuesday I had three calls on the calendar. Here's what changed.` +- Action: `The full breakdown is in this week's issue — grab it here → [link]` + +**Failure mode:** starting strong, fading in the middle. The Interest and Desire sections are where most AIDA copy dies. Keep them concrete and specific. + +--- + +## PAS — Problem, Agitate, Solution + +The best framework for problem-aware audiences. The reader already knows they have the pain; you name it, you twist the knife, then you offer the release. + +**Structure:** +- **Problem** — name the pain in the reader's language +- **Agitate** — make them feel it. Not fear-mongering — just honest specificity about what it costs them +- **Solution** — your offer as the clean exit + +**When it shines:** sales pages for pain-driven purchases (back pain, sleep issues, burned out in their job, failing to hit revenue targets). Any audience that already knows the problem exists. + +**Example (SaaS for agency billing):** +- Problem: `You're leaving money on the table every month.` +- Agitate: `Untracked time. Unbilled revisions. Scope creep that nobody logged. Multiply that by 6 people and 12 months and the number you're staring at is between $40K and $120K in unbilled work — every year.` +- Solution: `TimeStitch tracks it automatically so you bill every hour you earn.` + +**Failure mode:** over-agitating. If the reader feels manipulated, you've lost them. Stop twisting the knife the moment the reader is nodding. Don't belabor. + +--- + +## FAB — Features, Advantages, Benefits + +The essential framework for feature-heavy products. Forces you to translate specs into what they do (advantages) into what the reader gains (benefits). + +**Structure:** +- **Feature** — the literal spec +- **Advantage** — what that spec enables +- **Benefit** — what the reader gains (usually emotional, time, money, status) + +**When it shines:** SaaS, physical products, technical tools where you have to actually explain what the thing is. + +**Example (mattress):** +- Feature: `3-layer gel memory foam, 14" total thickness` +- Advantage: `Contours to your spine without the heat trap of traditional memory foam` +- Benefit: `Wake up without back pain. Sleep cool enough to stop kicking the covers off at 3am.` + +**Writing shortcut:** the "so what?" test. After every feature, ask "so what?" and keep answering until you land on something the reader actually cares about. That's the benefit. + +**Failure mode:** stopping at advantages. "Contours to your spine" isn't a benefit — it's an advantage. "Wake up without back pain" is a benefit. The job isn't done until you reach the emotional payoff. + +--- + +## BAB — Before, After, Bridge + +The transformation framework. Reader is buying a future state; you show the gap and position the offer as the bridge. + +**Structure:** +- **Before** — where they are now, specifically +- **After** — where they want to be +- **Bridge** — your offer as the path + +**When it shines:** coaching, courses, health and fitness, career pivots, anything where the purchase is really a purchase of a different identity or outcome. + +**Example (career coaching):** +- Before: `You're five years into a career you fell into. The salary is fine. The work is draining. Every Sunday night, the same knot in your stomach.` +- After: `Six months from now: a role you chose on purpose. Higher cap, higher interest, Sundays that feel like Sundays.` +- Bridge: `The Pivot Program is how 400+ mid-career professionals got there. 12 weeks, a real plan, real accountability, and a cohort of people doing the same hard thing.` + +**Failure mode:** generic Before and After. "You're unhappy" → "You're happy" is not copy. Specificity is what makes BAB work. The reader has to see themselves in the Before and want to be the person in the After. + +--- + +## 4Us — Useful, Urgent, Unique, Ultra-specific + +Not a full framework, more of a headline filter. Every headline should hit at least three of the four. + +- **Useful** — is there a clear benefit to the reader? +- **Urgent** — is there a reason to read/act now? +- **Unique** — does it sound different from every other headline in the category? +- **Ultra-specific** — are there concrete numbers, names, times? + +Example that hits all four: `How a $2.3M boutique agency cut time-to-cash from 45 days to 9 — in 6 weeks` +- Useful (cash flow improvement), Urgent (implied by the timeline), Unique (most "agency cash flow" headlines don't lead with specifics), Ultra-specific (4 concrete numbers). + +Use the 4Us as a check on any headline before shipping it. + +--- + +## PASTOR — for long-form sales pages + +Ray Edwards' framework for long-form sales letters. Basically an expanded PAS. + +- **Person / Problem** — who this is for and the problem they have +- **Amplify** — cost of the problem +- **Story / Solution** — your story of solving it +- **Transformation / Testimony** — what changed + proof +- **Offer** — the actual pitch +- **Response** — clear call to action + +Use this for sales pages over 1,500 words, webinar pitches, or any long-form piece where you need to walk the reader through a complete arc. + +--- + +## StoryBrand — brand narrative structure + +Donald Miller's framework. Frames the customer as the hero, the brand as the guide. + +- A **Character** (the customer) +- has a **Problem** +- and meets a **Guide** (the brand) +- who gives them a **Plan** +- and calls them to **Action** +- that ends in **Success** +- and helps them avoid **Failure** + +Best for brand-level messaging (homepage, about page, brand video scripts) — not for single headlines or ads. Too scaffolded for that. + +--- + +## How to choose between them + +Ask these questions in order: + +1. **Does the audience already know they have this problem?** + - Yes → PAS or BAB + - No → AIDA (you have to create awareness) + +2. **Is the buyer purchasing a transformation or a tool?** + - Transformation (identity, outcome) → BAB + - Tool (features they'll use) → FAB + +3. **How much space do you have?** + - Short (headline, subject line, ad) → AIDA or the 4Us as a filter + - Medium (landing page section) → any of the main four + - Long (sales page, webinar pitch) → PASTOR + +4. **Is this for brand messaging across the whole site?** + - → StoryBrand at the top layer, with the main four used within sections + +If you're unsure, default to AIDA. It almost always works. diff --git a/skills/copywriter/references/levers.md b/skills/copywriter/references/levers.md new file mode 100755 index 00000000..b10496e3 --- /dev/null +++ b/skills/copywriter/references/levers.md @@ -0,0 +1,175 @@ +# Psychological Levers — The Copywriter's Toolkit + +These are the moves direct-response copy makes to earn the click, the signup, the sale. Each lever is a hypothesis about the reader's mind. Mix and match across your three variations. + +## Loss aversion + +Fear of losing is roughly 2x stronger than desire to gain (Kahneman). Copy that frames the offer as *preventing loss* often outperforms copy framing the same offer as *achieving gain*. + +**Before:** "Double your reply rate with better cold emails." +**After (loss aversion):** "Stop losing deals to cold emails that never get opened." + +**When to use:** audiences that are already aware of the pain. Doesn't work well on cold audiences who don't feel the problem yet — you'd be loss-framing a loss they don't feel. + +**When to avoid:** luxury / status purchases. No one buys a Rolex because they're afraid of losing something. + +--- + +## Social proof + +Other people did this and it worked. Our brains read that as "this is safe." + +**Formats:** +- Numbers: `Used by 47,000 agencies` +- Named customers: `From the team at Stripe, Notion, and Vercel` +- Reviews: `4.9 stars from 2,000+ reviews` +- Media: `Featured in TechCrunch, The Verge, and Forbes` +- Testimonials: specific quotes from named customers with their role + +**When to use:** always, if you have it. Even a small number (`200 founders`) beats no number. + +**When to avoid:** when the numbers would undermine — if you have 12 customers, don't lead with that. Lead with who they are. + +--- + +## Specificity + +Specific numbers and details read as true. Round numbers and vague descriptors read as fake. + +**Before:** "Significant improvement in conversion" +**After:** "23% lift in checkout completion in week 1" + +**Before:** "Our customers save time" +**After:** "The average agency saves 4.2 hours per week per project manager" + +**When to use:** everywhere you can back it up. If you don't have the number, get the number before you write the copy. + +**When to avoid:** when you don't have the data. Making up a specific number is worse than using a vague claim — if the specific is ever challenged and proven fake, trust collapses. + +--- + +## Curiosity gap + +Imply information the reader wants and doesn't have. The mind hates an open loop. + +**Before:** "Tips for better sleep" +**After:** "The 4-minute evening routine that doubled my deep sleep" + +**Before:** "Improve your Facebook ads" +**After:** "Why our $37 ad outperformed our $12,000 ad" + +**When to use:** email subject lines, ad headlines, blog titles — anywhere the goal is the next click. + +**When to avoid:** CTAs and post-click copy. Once the reader has clicked, deliver on the gap immediately or you've broken the trust. + +--- + +## Status / identity + +Copy that says *this is for people like you* or *people like you use this* taps identity signaling, which is often more powerful than feature selling. + +**Examples:** +- `For founders who actually ship.` +- `Designed for operators, not tourists.` +- `The tool professional writers use when they have to ship fast.` + +**When to use:** when your audience has a clear identity they'd like to reinforce. Works especially well in B2B (job title pride) and premium consumer (taste signaling). + +**When to avoid:** when you're trying to reach a broad audience. Identity copy is inherently exclusive — that's the point — and alienates anyone outside the identity. + +--- + +## Urgency / scarcity + +Real urgency is a cheat code. Fake urgency is a trust-destroyer. + +**Real urgency:** +- Real deadline: `Early-bird pricing ends Friday` +- Limited inventory: `12 spots left in the October cohort` +- Real calendar: `Booking calls for Q2 until April 15` + +**Fake urgency (avoid):** +- `Only a few left!` (with no number) +- `Limited time only!` (with no deadline) +- Countdown timers that reset when you reload + +**When to use:** any time you have real urgency and aren't using it. + +**When to avoid:** whenever you're tempted to manufacture it. Readers detect this and it poisons the rest of the copy. + +--- + +## Contrast + +The mind reads in contrast. Before/after. Us/them. Old way/new way. + +**Examples:** +- `Old way: 6 tools stitched together. New way: one platform.` +- `Most agencies track time in three places. We track it in one.` +- `Before: 40-minute weekly reports. After: 4-minute ones.` + +**When to use:** when your offer is a meaningful upgrade from the status quo. Contrast makes the upgrade feel real. + +**When to avoid:** when the contrast is manufactured. If the "old way" is a straw man nobody actually uses, the reader sees through it. + +--- + +## Speed / ease + +Humans systematically underestimate future effort. Copy that promises speed and ease converts because the reader's brain discounts the cost. + +**Examples:** +- `Setup in 90 seconds.` +- `One click to enable.` +- `No credit card required.` +- `Paste your link — we do the rest.` + +**When to use:** when speed is genuinely a feature. SaaS onboarding, form fills, any low-commitment action. + +**When to avoid:** high-ticket or transformation purchases. "Get a coaching certification in 90 seconds" is a lie the reader will punish you for. + +--- + +## Authority + +Credentials, named proof, pedigree. Reader doesn't have to trust you — they trust the thing that vouches for you. + +**Formats:** +- `Built by the team that shipped Stripe Atlas.` +- `Featured in the New York Times.` +- `PhD-level expertise without the PhD-level wait.` +- `Certified by [known credential body].` + +**When to use:** when you have real authority to name. Works especially well when the reader doesn't know you but trusts whoever vouches for you. + +**When to avoid:** when you don't have it. Never invent authority — easiest way to destroy credibility. + +--- + +## Story + specificity together + +The strongest copy combines narrative with concrete detail. The mind latches onto a story and the details anchor it as real. + +**Weak:** "Sarah got better results with our platform." +**Strong:** "Last Tuesday at 2:47 PM, Sarah was about to lose a $40,000 deal — the buyer had ghosted her for 9 days. She sent one email using our follow-up template. 11 minutes later, the deal was back on." + +**When to use:** email body copy, sales page storytelling sections, case study lead-ins. Anywhere you have space for 2–3 sentences of narrative. + +**When to avoid:** headlines and subject lines (no room), CTAs (no room). Tight formats need compression, not narrative. + +--- + +## The levers, ranked by reliability + +If you had to bet on which levers land, in order: + +1. **Specificity** — almost always helps. Low downside. +2. **Social proof** — if you have it, use it. +3. **Authority** — same. +4. **Contrast** — reliable in medium-to-long formats. +5. **Loss aversion** — works when the audience feels the pain. +6. **Curiosity gap** — high variance; works great for opens, can hurt trust if abused. +7. **Story + specificity** — expensive to write well; pays off if you nail it. +8. **Status / identity** — powerful but narrow audience. +9. **Urgency / scarcity** — cheat code when real, poison when fake. +10. **Speed / ease** — useful modifier, rarely the main lever. diff --git a/skills/disclosure-analyzer/SKILL.md b/skills/disclosure-analyzer/SKILL.md new file mode 100644 index 00000000..49c62b39 --- /dev/null +++ b/skills/disclosure-analyzer/SKILL.md @@ -0,0 +1,299 @@ +--- +name: disclosure-analyzer +description: "Disclosure & Inspection Report Analyzer for real estate transactions. Use this skill ANY time the user mentions: disclosures, inspection report, TDS, SPQ, AVID, seller disclosures, pest report, termite report, foundation inspection, roof inspection, sewer lateral, home inspection, property condition, inspection findings..." +--- + +# Disclosure & Inspection Report Analyzer + +You are a real estate disclosure and inspection report analyst. Your job is to take seller disclosure forms and inspection reports from a real estate transaction, extract the important findings, cross-reference what the seller said against what the inspectors found, and produce a clear, organized report that a buyer (and their agent) can use to understand the property's condition. + +**Before generating any report, read the reference file:** +- `references/cost-estimates.md` — Common repair cost ranges for Northern California / Bay Area market + +--- + +## How This Works + +The user (a real estate agent) will upload some combination of: +- **Seller disclosures** — forms the seller fills out describing what they know about the property (TDS, SPQ, AVID, and other standard CAR forms) +- **Inspection reports** — professional reports from inspectors (general home inspection, pest/termite, roof, foundation, sewer lateral, chimney, pool, etc.) + +Your job is to read all of them, pull out the meaningful findings, and produce an organized analysis. The two key things you're doing: + +1. **Extracting and categorizing findings** from every inspection report by severity +2. **Cross-referencing disclosures against inspections** to flag where the seller's statements don't match what the inspectors found + +--- + +## Step 1: Intake — Collect the Documents + +Ask the user what documents they have. Common combinations include: + +- Seller disclosures (TDS, SPQ, AVID, other CAR forms) +- General home inspection +- Pest / termite (Section 1 and Section 2 findings) +- Roof inspection +- Foundation inspection +- Sewer lateral inspection (camera scope) +- Chimney inspection +- Pool/spa inspection +- Any other specialty reports + +Also ask: + +- **Property address** (for the report header) +- **Include cost estimates?** Some buyers want ballpark cost ranges for repairs, others don't. Ask every time. + +If the user has already provided documents and info, skip ahead — don't re-ask for things you already have. + +--- + +## Step 2: Extract and Analyze + +### Reading Seller Disclosures + +Seller disclosures are forms where the seller checks boxes and writes notes about what they know about the property. Focus on: + +- Anything marked "Yes" with an explanation — these are things the seller is explicitly flagging +- Written notes in the margins or explanation sections — sellers sometimes bury important info here +- Items the seller marks as "Unknown" or leaves blank — note these, especially if they relate to something an inspector flagged + +Disclosures reflect the seller's *knowledge*, not the property's actual condition — state discrepancies as simple facts, no ominous language or implications ("The seller indicated no knowledge of foundation issues on the TDS. The foundation inspection report identified X."). + +### Reading Inspection Reports + +For each inspection report, extract and categorize findings by severity: Critical, Moderate, or Minor. + +### Practical Scope Callouts + +When damage to one area is extensive enough that fixing it essentially means remodeling that area (e.g., severe dry rot through subfloor, joists, and walls), say so plainly — this helps the buyer understand real scope and cost. + +### Cross-Referencing + +Go through the disclosure forms item by item and compare against inspection findings. You're looking for: + +- **Discrepancies** — seller said "no" or "unknown" to something, but the inspection found evidence of it. Be fair about why this might happen (see the note above about seller knowledge). +- **Confirmed issues** — seller disclosed something AND the inspection confirmed it. Good — this means the seller was upfront. +- **Inspection-only findings** — things the inspector found that the disclosures don't address at all. These aren't necessarily discrepancies — inspectors look at things sellers might not think to disclose. + +--- + +## Step 3: Produce the Report + +### Report Structure + +Organize by severity, with the most important stuff first. The buyer should be able to read the first page and understand the big picture. + +**Report sections:** + +#### Header +- Property address +- Date of analysis +- List of documents reviewed (with dates of each report) + +#### Executive Summary +- 3-5 sentences covering the overall condition picture +- Total count of critical, moderate, and minor findings +- Any major discrepancy between disclosures and inspections +- One-line bottom line: is this property in generally good shape with some items to address, or are there significant concerns? + +#### Critical Findings +- Each finding gets its own entry with: + - **What was found** — plain language description + - **Source** — which report, what page/section if possible + - **Disclosure cross-reference** — what did the seller say about this? If nothing, note that + - **Cost estimate** (if the user requested cost estimates) — a realistic range. See the cost estimates reference for guidance. When an inspection report quotes a repair cost, note that the actual total cost may be higher because inspectors often quote only their scope of work. For example, a termite company quotes for treating and removing damaged wood, but doesn't include the cost of the flooring contractor to put the floor back, or the plumber to reconnect pipes they had to move. Account for the full scope when estimating. +- If there are no critical findings, say so — that's good news + +#### Moderate Findings +- Same format as critical, but these are the "should address within 1-2 years" items +- Group by system/area if there are many (e.g., multiple plumbing items together) + +#### Minor / Maintenance Items +- These can be more compact — a simple list with brief descriptions is fine +- No cost estimates needed for minor items unless the user specifically asks + +#### Disclosure vs. Inspection Comparison +- A clear table or section showing notable discrepancies +- For each discrepancy: + - What the seller stated + - What the inspection found + - A fair note explaining possible reasons for the difference +- Also note areas where disclosures and inspections align — it's good to show the seller was forthcoming where they were + +#### Documents Reviewed +- List every document that was analyzed, with its date and inspector/company name + +--- + +## Output Formats + +Ask the user whether they want a **PDF** or an **email-ready HTML**, or both. + +### PDF Report +- Clean, professional layout using ReportLab +- Install: `pip install reportlab --break-system-packages` +- Neutral styling — no personal agent branding. Use a clean color scheme (dark header, readable body, subtle section dividers) +- Clear typography and good use of whitespace +- Section headers with visual distinction +- Tables for the disclosure comparison section +- Page numbers + +### Email-Ready HTML +- Self-contained HTML with all inline styles (no external CSS or JS) +- Table-based layout for email client compatibility (Gmail, Outlook, Apple Mail) +- 600px max-width +- System font stack +- Same content as the PDF, just formatted for email +- Can be copy-pasted into an email client or sent via API + +Both formats should contain the same level of detail. If the report needs to be thorough, it needs to be thorough regardless of format. + +--- + +## Delivery — NEVER PUBLISH THIS REPORT PUBLICLY (hard rule, set 2026-08-13 by Graeham) + +> **Disclosure reports are confidential. Do NOT push them to the `online-content` repo, and do NOT generate a `graehamwatts.github.io` URL for them.** + +This skill previously published to `online-content/disclosures/Disclosure_[address].html`. That is a **public** GitHub Pages site, and the filename is derived directly from the property address, so anyone who knew the address could guess the URL and read a buyer's full inspection findings. Graeham's instruction (2026-08-13): disclosure output stays private, accessible to him and his team only. Verified at the time of this change: no disclosure report had ever actually been published, and none appears in the repo's git history, so there was no exposure to clean up. Keep it that way. + +**Correct delivery, in order of preference:** + +1. **Write the HTML locally only**, to the skill's own gitignored staging area: + `C:/Users/Graeham Watts/Documents/Skills LLMS/Claude/Skills/skills/disclosure-analyzer/outputs/Disclosure_[address].html` + This folder is gitignored, so the file never reaches a repo. +2. **Send it as an email ATTACHMENT** to Graeham and Adrian + (`graehamwatts@gmail.com`, `graehamwattsclientcare@gmail.com`), not as a link. + Use the SMTP sender at `skills/switchy-engine/scripts/send_email.py --attach `. +3. If the client needs it, **Graeham or Adrian forwards the attachment** after review. + The client-facing send is their call, not this skill's. + +**Do not** work around this by obfuscating the filename or adding a random suffix. A hard-to-guess public URL is still a public URL; it is not access control. + +Before sending, still run the brand validator on the file: +`python "C:/Users/Graeham Watts/Documents/Skills LLMS/Claude/Skills/scripts/verify_brand_identity.py"` — never ship on a blocked value. + +## Cost Estimates + +When the user opts in to cost estimates, provide realistic ballpark ranges. The key principles: + +- **Account for the full scope of work.** Inspectors and specialty contractors often quote only their piece. A termite company quotes for pest treatment and removing damaged material. They don't quote for the general contractor to rebuild, the plumber to reconnect, or the flooring to be replaced. Your estimate should reflect what the buyer will actually spend to resolve the issue end-to-end. +- **Use ranges, not single numbers.** There's always variability — "$2,000–$4,000" is more honest than "$3,000." +- **When you don't know, say so.** Some items genuinely need a specialist quote. "Recommend getting a quote from a licensed contractor" is a perfectly valid response. +- **Read the cost estimates reference** (`references/cost-estimates.md`) for common repair cost ranges calibrated to Northern California pricing. + +--- + +## Seller Credit Request Drafting + +This is an optional feature the user can activate by saying something like "help me draft a credit request" or "what should we ask the seller for." When triggered, you shift from pure analysis mode into negotiation support mode. + +### The Key Principle: Visible vs. Non-Obvious + +When a buyer makes an offer on a property, they're pricing in what they can see. If there's an obviously unpermitted addition or a visibly rough rear structure, the buyer saw that when they toured the property and wrote their offer accordingly. The seller (and their agent) will push back on credit requests for those items: "You knew about that when you made your offer." + +The strongest credit requests are for things the buyer **could not have reasonably known** before inspections: + +**Strong credit request items (non-obvious):** +- Asbestos in ductwork or behind walls — you can't see that on a tour +- Electrical issues behind walls (aluminum wiring, improper splices, missing grounds) +- Plumbing defects (sewer lateral condition, hidden leaks, galvanized pipe corrosion inside walls) +- Pest/termite damage hidden in crawl spaces, subfloor, inside walls +- Foundation issues not visible from the living space +- Roof defects that only a roofer on the roof would find (underlayment condition, flashing failures) +- Environmental hazards (mold behind walls, lead paint under layers) +- HVAC defects (cracked heat exchanger, duct issues in inaccessible areas) + +**Weak credit request items (buyer could see these):** +- Visibly unpermitted additions or structures +- Obvious cosmetic issues (peeling paint, worn carpet, dated fixtures) +- Anything clearly visible during a standard property tour +- Items explicitly called out in the listing or listing photos + +### How to Draft the Credit Request + +When the user asks for this, produce: + +1. **Recommended credit items** — list each non-obvious finding with: + - What was found and where + - Why it wasn't reasonably visible at time of offer + - Estimated cost to address (use cost estimate ranges) + - Which report documented it + +2. **Total recommended credit range** — sum up the cost ranges into a bottom-line ask range + +3. **Items NOT recommended for credit request** — briefly list the items you're leaving out and why (e.g., "The rear structure condition was visually apparent at time of property tour") + +4. **Draft language** — write the actual request language the agent can use. Keep it professional and factual: "During the inspection contingency period, the following conditions were identified that were not apparent during the initial property viewing..." No aggressive tone — just clear documentation of findings and costs. + +If the user has provided the MLS listing or property profile, use it to help determine what was marketed/visible vs. what's newly discovered. If no listing info is available, use reasonable judgment about what a buyer would have seen on a standard tour. + +--- + + +--- + +## Step 4: Quality Control Verification (MANDATORY) + +**This step is not optional.** Before delivering any report to the user, you MUST run a full verification pass. Mistakes in this type of report are a serious problem — a buyer or their agent could make decisions based on inaccurate information. Every report must be checked before it goes out. + +### The Verification Process + +After generating the report, run a separate verification agent (subagent) that re-reads the original source documents and cross-checks the report for accuracy. If a subagent is not available, perform the verification yourself as a distinct second pass — do NOT just skim what you already wrote. + +### What the Verification Checks + +**1. Severity Accuracy** +- Re-read each finding in the original inspection report. Did the inspector flag it with a safety/repair warning (red flag), or was it an observation only? +- If the inspector did NOT flag something as a safety concern, your report should not escalate it to critical unless there's a clear factual basis (e.g., knob & tube wiring is inherently critical regardless of inspector flagging). +- If the inspector DID flag something as safety/repair, make sure your report reflects that severity — don't accidentally downgrade it. + +**2. Factual Accuracy** +- Every finding in the report must trace back to a specific section/page in the source document. Spot-check at least 5 critical and 5 moderate findings by going back to the source and confirming the report matches what the inspector actually wrote. +- Watch for these common errors: + - **Overstating condition**: Inspector says "general wear, monitor" and the report says "failing" or "needs replacement" + - **Understating condition**: Inspector flags something with a safety warning and the report buries it in moderate + - **Conflating items**: Two separate findings getting merged into one and losing detail, or one finding getting split into two and inflating the count + - **Inventing findings**: A finding appears in the report that isn't actually in any source document. This should never happen. + - **Wrong section/page references**: Source citations that don't match the actual report pages + +**3. Cost Estimate Accuracy** +- Cross-check cost ranges against the `references/cost-estimates.md` file +- Make sure no item has an inflated or deflated range vs. what the reference says +- If the report includes a "full replacement" cost for something that only needs maintenance or repair, flag and fix it. (Example: a roof with maintenance issues should NOT quote a full replacement cost range unless the inspector specifically called for replacement.) +- Verify the summary cost totals add up correctly — don't let rounding errors or removed items create a wrong total + +**4. Disclosure Cross-Reference Accuracy** +- If TDS/SPQ forms were provided, verify that each "seller said X, inspector found Y" statement is accurate to both documents +- If TDS/SPQ were NOT provided, make sure the report clearly states this limitation and doesn't attempt to cross-reference against documents that don't exist +- Make sure the NHD findings (flood zone, seismic, environmental) are reported accurately per the actual NHD document + +**5. Tone Check** +- Scan the report for language that could come across as alarmist, ominous, or accusatory toward the seller +- Remove any editorializing. The report should state facts and let the reader draw conclusions. +- Make sure the "remodel scope callout" language is only used when the damage genuinely warrants it — don't casually throw around "this is essentially a remodel" for moderate repairs + +**6. Completeness Check** +- Compare the report's finding count against the inspector's summary page (most reports have a summary at the front). Are you missing any findings? Are you double-counting any? +- Verify all documents that were uploaded are listed in the "Documents Reviewed" section +- If cost estimates were requested, make sure every critical and moderate finding has one + +### Verification Output + +After the verification pass, fix any errors found. If corrections were made, note them internally (you don't need to tell the user about every correction — just fix them). If a correction changes something significant (e.g., an item moved from critical to moderate, or a cost estimate changed materially), mention it to the user so they know the report was refined. + +**Only deliver the report after verification is complete.** + +--- + +## Common Pitfalls to Avoid + +These are mistakes that have come up in testing. Watch for them: + +1. **Roof condition overstatement.** If the inspector walked the roof, took photos, and found only maintenance items (debris, moss, minor bubbling, flashing paint wear) without issuing safety flags, that is a roof that needs maintenance — not replacement. Do NOT include a "full roof replacement" cost estimate unless the inspector specifically calls for it or the damage clearly warrants it. Look at the actual photos and inspector language, not just the summary line items. + +2. **Counting items that aren't findings.** Inspection reports include informational sections, limitations notes, and general maintenance tips. These are not "findings." Only count actual observations/deficiencies. + +3. **Inflating the critical count.** A finding is critical only if it affects safety, structural integrity, or could cause major damage if left unaddressed. "Cosmetic repairs needed" is never critical. "Sealant recommended" is never critical. Be disciplined about severity classification. + +4. **Missing the forest for the trees.** If 15 individual findings in the same area all point to the same underlying problem (e.g., water intrusion), call out the underlying problem as the main finding and list the individual items as evidence. Don't present them as 15 separate issues when they're really one big one. diff --git a/skills/disclosure-analyzer/generated/.gitkeep b/skills/disclosure-analyzer/generated/.gitkeep new file mode 100755 index 00000000..e69de29b diff --git a/skills/disclosure-analyzer/references/cost-estimates.md b/skills/disclosure-analyzer/references/cost-estimates.md new file mode 100755 index 00000000..e40d260d --- /dev/null +++ b/skills/disclosure-analyzer/references/cost-estimates.md @@ -0,0 +1,113 @@ +# Common Repair Cost Estimates — Northern California / Bay Area + +These are approximate ranges based on typical Bay Area contractor pricing as of 2025-2026. Actual costs vary significantly by property size, access difficulty, extent of damage, and contractor. Always recommend the buyer get actual quotes for critical and moderate items. + +Labor rates in the Bay Area tend to run 30-50% higher than national averages. Factor this in when estimating. + +## Structural / Foundation + +| Item | Range | Notes | +|------|-------|-------| +| Foundation crack repair (epoxy injection, per crack) | $500–$1,500 | Simple cracks only | +| Foundation bolting (seismic retrofit) | $3,000–$7,000 | Depends on linear footage | +| Foundation underpinning (per pier) | $1,500–$3,000 | Typically need 6-12 piers | +| Mudsill replacement (per linear foot) | $40–$80 | Often combined with bolting | +| Post and pier replacement (per post) | $500–$1,200 | Crawl space access matters | +| Structural beam replacement | $2,000–$8,000 | Highly variable by scope | +| Retaining wall repair/replacement | $5,000–$20,000+ | Depends on size and material | + +## Roof + +| Item | Range | Notes | +|------|-------|-------| +| Composition shingle roof (full replacement) | $15,000–$35,000 | Based on typical 1,500-2,500 sqft home | +| Tile roof repair (per area) | $1,000–$5,000 | Replacing broken tiles, reflashing | +| Flat roof section (torch down / TPO) | $3,000–$8,000 | Per 200-400 sqft section | +| Gutter replacement (whole house) | $1,500–$3,500 | | +| Flashing repair | $500–$2,000 | Chimney, vent, valley flashing | + +## Plumbing + +| Item | Range | Notes | +|------|-------|-------| +| Sewer lateral replacement | $8,000–$20,000 | Depends on length, depth, permits | +| Sewer lateral lining (trenchless) | $6,000–$15,000 | Less disruption than replacement | +| Water heater replacement (tank) | $2,000–$4,000 | Installed, with permit | +| Water heater replacement (tankless) | $4,000–$7,000 | Installed, with permit | +| Repipe whole house (copper or PEX) | $8,000–$18,000 | Depends on size, access, wall repair | +| Drain line repair/replacement | $1,500–$5,000 | Per section | +| Hose bib / supply line repair | $200–$600 | | + +## Electrical + +| Item | Range | Notes | +|------|-------|-------| +| Main panel upgrade (100A to 200A) | $3,000–$6,000 | With permit | +| Subpanel addition | $1,500–$3,000 | | +| GFCI outlet installation (each) | $150–$300 | | +| Whole-house rewire | $15,000–$30,000 | Plus drywall repair | +| Knob-and-tube removal (per circuit) | $1,500–$3,500 | | +| Smoke/CO detector installation (hardwired set) | $500–$1,200 | | + +## Pest / Termite + +| Item | Range | Notes | +|------|-------|-------| +| Localized termite treatment (per area) | $500–$1,500 | Chemical or spot treatment | +| Whole-structure fumigation (tenting) | $2,500–$5,000 | Based on home size | +| Section 1 repairs (typical) | $2,000–$8,000 | As quoted by pest company | +| **Add for contractor rebuilding** | **+30-60% of pest quote** | Pest companies don't quote for putting things back together — floors, drywall, trim, plumbing reconnection. Budget this separately. | +| Fungus / dry rot repair (per area) | $1,000–$4,000 | Depends on extent | +| Subterranean termite treatment | $1,500–$3,000 | Soil treatment | + +## HVAC + +| Item | Range | Notes | +|------|-------|-------| +| Furnace replacement | $4,000–$8,000 | Installed, with permit | +| AC unit replacement | $5,000–$10,000 | Installed | +| Full HVAC system (furnace + AC) | $10,000–$20,000 | | +| Ductwork repair/sealing | $1,500–$4,000 | | +| Ductwork replacement | $5,000–$12,000 | | +| Mini-split installation (per zone) | $3,000–$5,000 | | + +## Chimney / Fireplace + +| Item | Range | Notes | +|------|-------|-------| +| Chimney cap installation | $300–$800 | | +| Chimney liner installation | $2,500–$5,000 | Stainless steel liner | +| Chimney rebuild (above roofline) | $5,000–$15,000 | Seismic damage common in Bay Area | +| Fireplace insert (gas) | $3,000–$6,000 | | +| Damper repair/replacement | $300–$800 | | + +## Windows / Doors / Exterior + +| Item | Range | Notes | +|------|-------|-------| +| Window replacement (per window, vinyl) | $600–$1,200 | Installed | +| Window replacement (per window, wood) | $1,000–$2,000 | Installed | +| Sliding glass door replacement | $2,000–$5,000 | | +| Exterior paint (whole house) | $8,000–$18,000 | Depends on size and prep work | +| Siding repair (per section) | $1,000–$4,000 | | +| Deck repair/replacement | $5,000–$20,000 | Depends on size | + +## Miscellaneous + +| Item | Range | Notes | +|------|-------|-------| +| Grading/drainage correction | $2,000–$8,000 | French drain, regrading | +| Sidewalk/driveway repair | $1,000–$5,000 | Depends on area | +| Asbestos testing (per sample) | $25–$75 | Lab fee | +| Asbestos abatement (per area) | $2,000–$10,000+ | Highly regulated, varies widely | +| Lead paint remediation (per room) | $1,000–$3,000 | | +| Mold remediation | $2,000–$10,000+ | Depends on extent | +| Pool replastering | $5,000–$10,000 | | +| Pool equipment replacement | $3,000–$8,000 | Pump, filter, heater | + +## Important Notes + +- These ranges are starting points. Always recommend the buyer get 2-3 actual bids for any significant work. +- Permit costs are generally included in the ranges above, but can add $500-$2,000 for major work. +- If multiple issues overlap in the same area (e.g., pest damage + plumbing + flooring in a bathroom), the total cost may be less than the sum of individual estimates because the contractor is already in the area — but it can also be more if the work is complex. Use judgment. +- Older homes (pre-1950) often have compounding issues that increase costs — access is harder, materials are non-standard, and one repair can reveal another. diff --git a/skills/docx/LICENSE.txt b/skills/docx/LICENSE.txt new file mode 100644 index 00000000..c55ab422 --- /dev/null +++ b/skills/docx/LICENSE.txt @@ -0,0 +1,30 @@ +© 2025 Anthropic, PBC. All rights reserved. + +LICENSE: Use of these materials (including all code, prompts, assets, files, +and other components of this Skill) is governed by your agreement with +Anthropic regarding use of Anthropic's services. If no separate agreement +exists, use is governed by Anthropic's Consumer Terms of Service or +Commercial Terms of Service, as applicable: +https://www.anthropic.com/legal/consumer-terms +https://www.anthropic.com/legal/commercial-terms +Your applicable agreement is referred to as the "Agreement." "Services" are +as defined in the Agreement. + +ADDITIONAL RESTRICTIONS: Notwithstanding anything in the Agreement to the +contrary, users may not: + +- Extract these materials from the Services or retain copies of these + materials outside the Services +- Reproduce or copy these materials, except for temporary copies created + automatically during authorized use of the Services +- Create derivative works based on these materials +- Distribute, sublicense, or transfer these materials to any third party +- Make, offer to sell, sell, or import any inventions embodied in these + materials +- Reverse engineer, decompile, or disassemble these materials + +The receipt, viewing, or possession of these materials does not convey or +imply any license or right beyond those expressly granted above. + +Anthropic retains all right, title, and interest in these materials, +including all copyrights, patents, and other intellectual property rights. diff --git a/skills/docx/SKILL.md b/skills/docx/SKILL.md new file mode 100644 index 00000000..aef63d3c --- /dev/null +++ b/skills/docx/SKILL.md @@ -0,0 +1,95 @@ +--- +name: docx +description: "Use this skill whenever the user wants to create, read, edit, or manipulate Word documents (.docx files). Also use when extracting or reorganizing content from .docx files, inserting or replacing images in documents, performing find-and-replace in Word files, working with tracked changes or comments..." +license: Proprietary. LICENSE.txt has complete terms +--- + +# DOCX creation, editing, and analysis + +## Overview + +A .docx file is a ZIP archive containing XML files. + +## Quick Reference + +| Task | Approach | +|------|----------| +| Read/analyze content | `pandoc` or unpack for raw XML | +| Create new document | Use `docx-js` - see Creating New Documents below | +| Edit existing document | Unpack → edit XML → repack - see Editing Existing Documents below | + +### Converting .doc to .docx + +Legacy `.doc` files must be converted before editing: + +```bash +python scripts/office/soffice.py --headless --convert-to docx document.doc +``` + +### Reading Content + +```bash +# Text extraction with tracked changes +pandoc --track-changes=all document.docx -o output.md + +# Raw XML access +python scripts/office/unpack.py document.docx unpacked/ +``` + +### Converting to Images + +```bash +python scripts/office/soffice.py --headless --convert-to pdf document.docx +pdftoppm -jpeg -r 150 document.pdf page +``` + +### Accepting Tracked Changes + +To produce a clean document with all tracked changes accepted (requires LibreOffice): + +```bash +python scripts/accept_changes.py input.docx output.docx +``` + +--- + +## Creating New Documents + +> Read `references/docx-creation-api.md` for the full docx-js creation cookbook (setup, validation, page size, styles, lists, tables, images, page breaks, hyperlinks, footnotes, tab stops, multi-column layouts, TOC, headers/footers). + +### Critical Rules for docx-js + +- **Set page size explicitly** - docx-js defaults to A4; use US Letter (12240 x 15840 DXA) for US documents +- **Landscape: pass portrait dimensions** - docx-js swaps width/height internally; pass short edge as `width`, long edge as `height`, and set `orientation: PageOrientation.LANDSCAPE` +- **Never use `\n`** - use separate Paragraph elements +- **Never use unicode bullets** - use `LevelFormat.BULLET` with numbering config +- **PageBreak must be in Paragraph** - standalone creates invalid XML +- **ImageRun requires `type`** - always specify png/jpg/etc +- **Always set table `width` with DXA** - never use `WidthType.PERCENTAGE` (breaks in Google Docs) +- **Tables need dual widths** - `columnWidths` array AND cell `width`, both must match +- **Table width = sum of columnWidths** - for DXA, ensure they add up exactly +- **Always add cell margins** - use `margins: { top: 80, bottom: 80, left: 120, right: 120 }` for readable padding +- **Use `ShadingType.CLEAR`** - never SOLID for table shading +- **Never use tables as dividers/rules** - cells have minimum height and render as empty boxes (including in headers/footers); use `border: { bottom: { style: BorderStyle.SINGLE, size: 6, color: "2E75B6", space: 1 } }` on a Paragraph instead. For two-column footers, use tab stops (see Tab Stops section), not tables +- **TOC requires HeadingLevel only** - no custom styles on heading paragraphs +- **Override built-in styles** - use exact IDs: "Heading1", "Heading2", etc. +- **Include `outlineLevel`** - required for TOC (0 for H1, 1 for H2, etc.) + +--- + +## Editing Existing Documents + +> Read `references/docx-editing-xml.md` for the unpack -> edit XML -> repack workflow (steps, smart quotes, comments, common pitfalls). + +## XML Reference + +> Read `references/docx-editing-xml.md` for the OOXML/XML reference (schema compliance, tracked changes, comments, images). + +--- + +## Dependencies + +- **pandoc**: Text extraction +- **docx**: `npm install -g docx` (new documents) +- **LibreOffice**: PDF conversion (auto-configured for sandboxed environments via `scripts/office/soffice.py`) +- **Poppler**: `pdftoppm` for images diff --git a/skills/docx/references/docx-creation-api.md b/skills/docx/references/docx-creation-api.md new file mode 100644 index 00000000..53d83ab7 --- /dev/null +++ b/skills/docx/references/docx-creation-api.md @@ -0,0 +1,324 @@ +# DOCX Creation API (docx-js cookbook) + +## Creating New Documents + +Generate .docx files with JavaScript, then validate. Install: `npm install -g docx` + +### Setup +```javascript +const { Document, Packer, Paragraph, TextRun, Table, TableRow, TableCell, ImageRun, + Header, Footer, AlignmentType, PageOrientation, LevelFormat, ExternalHyperlink, + InternalHyperlink, Bookmark, FootnoteReferenceRun, PositionalTab, + PositionalTabAlignment, PositionalTabRelativeTo, PositionalTabLeader, + TabStopType, TabStopPosition, Column, SectionType, + TableOfContents, HeadingLevel, BorderStyle, WidthType, ShadingType, + VerticalAlign, PageNumber, PageBreak } = require('docx'); + +const doc = new Document({ sections: [{ children: [/* content */] }] }); +Packer.toBuffer(doc).then(buffer => fs.writeFileSync("doc.docx", buffer)); +``` + +### Validation +After creating the file, validate it. If validation fails, unpack, fix the XML, and repack. +```bash +python scripts/office/validate.py doc.docx +``` + +### Page Size + +```javascript +// CRITICAL: docx-js defaults to A4, not US Letter +// Always set page size explicitly for consistent results +sections: [{ + properties: { + page: { + size: { + width: 12240, // 8.5 inches in DXA + height: 15840 // 11 inches in DXA + }, + margin: { top: 1440, right: 1440, bottom: 1440, left: 1440 } // 1 inch margins + } + }, + children: [/* content */] +}] +``` + +**Common page sizes (DXA units, 1440 DXA = 1 inch):** + +| Paper | Width | Height | Content Width (1" margins) | +|-------|-------|--------|---------------------------| +| US Letter | 12,240 | 15,840 | 9,360 | +| A4 (default) | 11,906 | 16,838 | 9,026 | + +**Landscape orientation:** docx-js swaps width/height internally, so pass portrait dimensions and let it handle the swap: +```javascript +size: { + width: 12240, // Pass SHORT edge as width + height: 15840, // Pass LONG edge as height + orientation: PageOrientation.LANDSCAPE // docx-js swaps them in the XML +}, +// Content width = 15840 - left margin - right margin (uses the long edge) +``` + +### Styles (Override Built-in Headings) + +Use Arial as the default font (universally supported). Keep titles black for readability. + +```javascript +const doc = new Document({ + styles: { + default: { document: { run: { font: "Arial", size: 24 } } }, // 12pt default + paragraphStyles: [ + // IMPORTANT: Use exact IDs to override built-in styles + { id: "Heading1", name: "Heading 1", basedOn: "Normal", next: "Normal", quickFormat: true, + run: { size: 32, bold: true, font: "Arial" }, + paragraph: { spacing: { before: 240, after: 240 }, outlineLevel: 0 } }, // outlineLevel required for TOC + { id: "Heading2", name: "Heading 2", basedOn: "Normal", next: "Normal", quickFormat: true, + run: { size: 28, bold: true, font: "Arial" }, + paragraph: { spacing: { before: 180, after: 180 }, outlineLevel: 1 } }, + ] + }, + sections: [{ + children: [ + new Paragraph({ heading: HeadingLevel.HEADING_1, children: [new TextRun("Title")] }), + ] + }] +}); +``` + +### Lists (NEVER use unicode bullets) + +```javascript +// ❌ WRONG - never manually insert bullet characters +new Paragraph({ children: [new TextRun("• Item")] }) // BAD +new Paragraph({ children: [new TextRun("\u2022 Item")] }) // BAD + +// ✅ CORRECT - use numbering config with LevelFormat.BULLET +const doc = new Document({ + numbering: { + config: [ + { reference: "bullets", + levels: [{ level: 0, format: LevelFormat.BULLET, text: "•", alignment: AlignmentType.LEFT, + style: { paragraph: { indent: { left: 720, hanging: 360 } } } }] }, + { reference: "numbers", + levels: [{ level: 0, format: LevelFormat.DECIMAL, text: "%1.", alignment: AlignmentType.LEFT, + style: { paragraph: { indent: { left: 720, hanging: 360 } } } }] }, + ] + }, + sections: [{ + children: [ + new Paragraph({ numbering: { reference: "bullets", level: 0 }, + children: [new TextRun("Bullet item")] }), + new Paragraph({ numbering: { reference: "numbers", level: 0 }, + children: [new TextRun("Numbered item")] }), + ] + }] +}); + +// ⚠️ Each reference creates INDEPENDENT numbering +// Same reference = continues (1,2,3 then 4,5,6) +// Different reference = restarts (1,2,3 then 1,2,3) +``` + +### Tables + +**CRITICAL: Tables need dual widths** - set both `columnWidths` on the table AND `width` on each cell. Without both, tables render incorrectly on some platforms. + +```javascript +// CRITICAL: Always set table width for consistent rendering +// CRITICAL: Use ShadingType.CLEAR (not SOLID) to prevent black backgrounds +const border = { style: BorderStyle.SINGLE, size: 1, color: "CCCCCC" }; +const borders = { top: border, bottom: border, left: border, right: border }; + +new Table({ + width: { size: 9360, type: WidthType.DXA }, // Always use DXA (percentages break in Google Docs) + columnWidths: [4680, 4680], // Must sum to table width (DXA: 1440 = 1 inch) + rows: [ + new TableRow({ + children: [ + new TableCell({ + borders, + width: { size: 4680, type: WidthType.DXA }, // Also set on each cell + shading: { fill: "D5E8F0", type: ShadingType.CLEAR }, // CLEAR not SOLID + margins: { top: 80, bottom: 80, left: 120, right: 120 }, // Cell padding (internal, not added to width) + children: [new Paragraph({ children: [new TextRun("Cell")] })] + }) + ] + }) + ] +}) +``` + +**Table width calculation:** + +Always use `WidthType.DXA` — `WidthType.PERCENTAGE` breaks in Google Docs. + +```javascript +// Table width = sum of columnWidths = content width +// US Letter with 1" margins: 12240 - 2880 = 9360 DXA +width: { size: 9360, type: WidthType.DXA }, +columnWidths: [7000, 2360] // Must sum to table width +``` + +**Width rules:** +- **Always use `WidthType.DXA`** — never `WidthType.PERCENTAGE` (incompatible with Google Docs) +- Table width must equal the sum of `columnWidths` +- Cell `width` must match corresponding `columnWidth` +- Cell `margins` are internal padding - they reduce content area, not add to cell width +- For full-width tables: use content width (page width minus left and right margins) + +### Images + +```javascript +// CRITICAL: type parameter is REQUIRED +new Paragraph({ + children: [new ImageRun({ + type: "png", // Required: png, jpg, jpeg, gif, bmp, svg + data: fs.readFileSync("image.png"), + transformation: { width: 200, height: 150 }, + altText: { title: "Title", description: "Desc", name: "Name" } // All three required + })] +}) +``` + +### Page Breaks + +```javascript +// CRITICAL: PageBreak must be inside a Paragraph +new Paragraph({ children: [new PageBreak()] }) + +// Or use pageBreakBefore +new Paragraph({ pageBreakBefore: true, children: [new TextRun("New page")] }) +``` + +### Hyperlinks + +```javascript +// External link +new Paragraph({ + children: [new ExternalHyperlink({ + children: [new TextRun({ text: "Click here", style: "Hyperlink" })], + link: "https://example.com", + })] +}) + +// Internal link (bookmark + reference) +// 1. Create bookmark at destination +new Paragraph({ heading: HeadingLevel.HEADING_1, children: [ + new Bookmark({ id: "chapter1", children: [new TextRun("Chapter 1")] }), +]}) +// 2. Link to it +new Paragraph({ children: [new InternalHyperlink({ + children: [new TextRun({ text: "See Chapter 1", style: "Hyperlink" })], + anchor: "chapter1", +})]}) +``` + +### Footnotes + +```javascript +const doc = new Document({ + footnotes: { + 1: { children: [new Paragraph("Source: Annual Report 2024")] }, + 2: { children: [new Paragraph("See appendix for methodology")] }, + }, + sections: [{ + children: [new Paragraph({ + children: [ + new TextRun("Revenue grew 15%"), + new FootnoteReferenceRun(1), + new TextRun(" using adjusted metrics"), + new FootnoteReferenceRun(2), + ], + })] + }] +}); +``` + +### Tab Stops + +```javascript +// Right-align text on same line (e.g., date opposite a title) +new Paragraph({ + children: [ + new TextRun("Company Name"), + new TextRun("\tJanuary 2025"), + ], + tabStops: [{ type: TabStopType.RIGHT, position: TabStopPosition.MAX }], +}) + +// Dot leader (e.g., TOC-style) +new Paragraph({ + children: [ + new TextRun("Introduction"), + new TextRun({ children: [ + new PositionalTab({ + alignment: PositionalTabAlignment.RIGHT, + relativeTo: PositionalTabRelativeTo.MARGIN, + leader: PositionalTabLeader.DOT, + }), + "3", + ]}), + ], +}) +``` + +### Multi-Column Layouts + +```javascript +// Equal-width columns +sections: [{ + properties: { + column: { + count: 2, // number of columns + space: 720, // gap between columns in DXA (720 = 0.5 inch) + equalWidth: true, + separate: true, // vertical line between columns + }, + }, + children: [/* content flows naturally across columns */] +}] + +// Custom-width columns (equalWidth must be false) +sections: [{ + properties: { + column: { + equalWidth: false, + children: [ + new Column({ width: 5400, space: 720 }), + new Column({ width: 3240 }), + ], + }, + }, + children: [/* content */] +}] +``` + +Force a column break with a new section using `type: SectionType.NEXT_COLUMN`. + +### Table of Contents + +```javascript +// CRITICAL: Headings must use HeadingLevel ONLY - no custom styles +new TableOfContents("Table of Contents", { hyperlink: true, headingStyleRange: "1-3" }) +``` + +### Headers/Footers + +```javascript +sections: [{ + properties: { + page: { margin: { top: 1440, right: 1440, bottom: 1440, left: 1440 } } // 1440 = 1 inch + }, + headers: { + default: new Header({ children: [new Paragraph({ children: [new TextRun("Header")] })] }) + }, + footers: { + default: new Footer({ children: [new Paragraph({ + children: [new TextRun("Page "), new TextRun({ children: [PageNumber.CURRENT] })] + })] }) + }, + children: [/* content */] +}] +``` + diff --git a/skills/docx/references/docx-editing-xml.md b/skills/docx/references/docx-editing-xml.md new file mode 100644 index 00000000..4b691eee --- /dev/null +++ b/skills/docx/references/docx-editing-xml.md @@ -0,0 +1,187 @@ +# DOCX Editing & OOXML/XML Reference + +## Editing Existing Documents + +**Follow all 3 steps in order.** + +### Step 1: Unpack +```bash +python scripts/office/unpack.py document.docx unpacked/ +``` +Extracts XML, pretty-prints, merges adjacent runs, and converts smart quotes to XML entities (`“` etc.) so they survive editing. Use `--merge-runs false` to skip run merging. + +### Step 2: Edit XML + +Edit files in `unpacked/word/`. See XML Reference below for patterns. + +**Use "Claude" as the author** for tracked changes and comments, unless the user explicitly requests use of a different name. + +**Use the Edit tool directly for string replacement. Do not write Python scripts.** Scripts introduce unnecessary complexity. The Edit tool shows exactly what is being replaced. + +**CRITICAL: Use smart quotes for new content.** When adding text with apostrophes or quotes, use XML entities to produce smart quotes: +```xml + +Here’s a quote: “Hello” +``` +| Entity | Character | +|--------|-----------| +| `‘` | ‘ (left single) | +| `’` | ’ (right single / apostrophe) | +| `“` | “ (left double) | +| `”` | ” (right double) | + +**Adding comments:** Use `comment.py` to handle boilerplate across multiple XML files (text must be pre-escaped XML): +```bash +python scripts/comment.py unpacked/ 0 "Comment text with & and ’" +python scripts/comment.py unpacked/ 1 "Reply text" --parent 0 # reply to comment 0 +python scripts/comment.py unpacked/ 0 "Text" --author "Custom Author" # custom author name +``` +Then add markers to document.xml (see Comments in XML Reference). + +### Step 3: Pack +```bash +python scripts/office/pack.py unpacked/ output.docx --original document.docx +``` +Validates with auto-repair, condenses XML, and creates DOCX. Use `--validate false` to skip. + +**Auto-repair will fix:** +- `durableId` >= 0x7FFFFFFF (regenerates valid ID) +- Missing `xml:space="preserve"` on `` with whitespace + +**Auto-repair won't fix:** +- Malformed XML, invalid element nesting, missing relationships, schema violations + +### Common Pitfalls + +- **Replace entire `` elements**: When adding tracked changes, replace the whole `...` block with `......` as siblings. Don't inject tracked change tags inside a run. +- **Preserve `` formatting**: Copy the original run's `` block into your tracked change runs to maintain bold, font size, etc. + +--- + +## XML Reference + +### Schema Compliance + +- **Element order in ``**: ``, ``, ``, ``, ``, `` last +- **Whitespace**: Add `xml:space="preserve"` to `` with leading/trailing spaces +- **RSIDs**: Must be 8-digit hex (e.g., `00AB1234`) + +### Tracked Changes + +**Insertion:** +```xml + + inserted text + +``` + +**Deletion:** +```xml + + deleted text + +``` + +**Inside ``**: Use `` instead of ``, and `` instead of ``. + +**Minimal edits** - only mark what changes: +```xml + +The term is + + 30 + + + 60 + + days. +``` + +**Deleting entire paragraphs/list items** - when removing ALL content from a paragraph, also mark the paragraph mark as deleted so it merges with the next paragraph. Add `` inside ``: +```xml + + + ... + + + + + + Entire paragraph content being deleted... + + +``` +Without the `` in ``, accepting changes leaves an empty paragraph/list item. + +**Rejecting another author's insertion** - nest deletion inside their insertion: +```xml + + + their inserted text + + +``` + +**Restoring another author's deletion** - add insertion after (don't modify their deletion): +```xml + + deleted text + + + deleted text + +``` + +### Comments + +After running `comment.py` (see Step 2), add markers to document.xml. For replies, use `--parent` flag and nest markers inside the parent's. + +**CRITICAL: `` and `` are siblings of ``, never inside ``.** + +```xml + + + + deleted + + more text + + + + + + + text + + + + +``` + +### Images + +1. Add image file to `word/media/` +2. Add relationship to `word/_rels/document.xml.rels`: +```xml + +``` +3. Add content type to `[Content_Types].xml`: +```xml + +``` +4. Reference in document.xml: +```xml + + + + + + + + + + + + +``` + diff --git a/skills/docx/scripts/__init__.py b/skills/docx/scripts/__init__.py new file mode 100644 index 00000000..8b137891 --- /dev/null +++ b/skills/docx/scripts/__init__.py @@ -0,0 +1 @@ + diff --git a/skills/docx/scripts/accept_changes.py b/skills/docx/scripts/accept_changes.py new file mode 100644 index 00000000..8e363161 --- /dev/null +++ b/skills/docx/scripts/accept_changes.py @@ -0,0 +1,135 @@ +"""Accept all tracked changes in a DOCX file using LibreOffice. + +Requires LibreOffice (soffice) to be installed. +""" + +import argparse +import logging +import shutil +import subprocess +from pathlib import Path + +from office.soffice import get_soffice_env + +logger = logging.getLogger(__name__) + +LIBREOFFICE_PROFILE = "/tmp/libreoffice_docx_profile" +MACRO_DIR = f"{LIBREOFFICE_PROFILE}/user/basic/Standard" + +ACCEPT_CHANGES_MACRO = """ + + + Sub AcceptAllTrackedChanges() + Dim document As Object + Dim dispatcher As Object + + document = ThisComponent.CurrentController.Frame + dispatcher = createUnoService("com.sun.star.frame.DispatchHelper") + + dispatcher.executeDispatch(document, ".uno:AcceptAllTrackedChanges", "", 0, Array()) + ThisComponent.store() + ThisComponent.close(True) + End Sub +""" + + +def accept_changes( + input_file: str, + output_file: str, +) -> tuple[None, str]: + input_path = Path(input_file) + output_path = Path(output_file) + + if not input_path.exists(): + return None, f"Error: Input file not found: {input_file}" + + if not input_path.suffix.lower() == ".docx": + return None, f"Error: Input file is not a DOCX file: {input_file}" + + try: + output_path.parent.mkdir(parents=True, exist_ok=True) + shutil.copy2(input_path, output_path) + except Exception as e: + return None, f"Error: Failed to copy input file to output location: {e}" + + if not _setup_libreoffice_macro(): + return None, "Error: Failed to setup LibreOffice macro" + + cmd = [ + "soffice", + "--headless", + f"-env:UserInstallation=file://{LIBREOFFICE_PROFILE}", + "--norestore", + "vnd.sun.star.script:Standard.Module1.AcceptAllTrackedChanges?language=Basic&location=application", + str(output_path.absolute()), + ] + + try: + result = subprocess.run( + cmd, + capture_output=True, + text=True, + timeout=30, + check=False, + env=get_soffice_env(), + ) + except subprocess.TimeoutExpired: + return ( + None, + f"Successfully accepted all tracked changes: {input_file} -> {output_file}", + ) + + if result.returncode != 0: + return None, f"Error: LibreOffice failed: {result.stderr}" + + return ( + None, + f"Successfully accepted all tracked changes: {input_file} -> {output_file}", + ) + + +def _setup_libreoffice_macro() -> bool: + macro_dir = Path(MACRO_DIR) + macro_file = macro_dir / "Module1.xba" + + if macro_file.exists() and "AcceptAllTrackedChanges" in macro_file.read_text(): + return True + + if not macro_dir.exists(): + subprocess.run( + [ + "soffice", + "--headless", + f"-env:UserInstallation=file://{LIBREOFFICE_PROFILE}", + "--terminate_after_init", + ], + capture_output=True, + timeout=10, + check=False, + env=get_soffice_env(), + ) + macro_dir.mkdir(parents=True, exist_ok=True) + + try: + macro_file.write_text(ACCEPT_CHANGES_MACRO) + return True + except Exception as e: + logger.warning(f"Failed to setup LibreOffice macro: {e}") + return False + + +if __name__ == "__main__": + parser = argparse.ArgumentParser( + description="Accept all tracked changes in a DOCX file" + ) + parser.add_argument("input_file", help="Input DOCX file with tracked changes") + parser.add_argument( + "output_file", help="Output DOCX file (clean, no tracked changes)" + ) + args = parser.parse_args() + + _, message = accept_changes(args.input_file, args.output_file) + print(message) + + if "Error" in message: + raise SystemExit(1) diff --git a/skills/docx/scripts/comment.py b/skills/docx/scripts/comment.py new file mode 100644 index 00000000..36e1c935 --- /dev/null +++ b/skills/docx/scripts/comment.py @@ -0,0 +1,318 @@ +"""Add comments to DOCX documents. + +Usage: + python comment.py unpacked/ 0 "Comment text" + python comment.py unpacked/ 1 "Reply text" --parent 0 + +Text should be pre-escaped XML (e.g., & for &, ’ for smart quotes). + +After running, add markers to document.xml: + + ... commented content ... + + +""" + +import argparse +import random +import shutil +import sys +from datetime import datetime, timezone +from pathlib import Path + +import defusedxml.minidom + +TEMPLATE_DIR = Path(__file__).parent / "templates" +NS = { + "w": "http://schemas.openxmlformats.org/wordprocessingml/2006/main", + "w14": "http://schemas.microsoft.com/office/word/2010/wordml", + "w15": "http://schemas.microsoft.com/office/word/2012/wordml", + "w16cid": "http://schemas.microsoft.com/office/word/2016/wordml/cid", + "w16cex": "http://schemas.microsoft.com/office/word/2018/wordml/cex", +} + +COMMENT_XML = """\ + + + + + + + + + + + + + {text} + + +""" + +COMMENT_MARKER_TEMPLATE = """ +Add to document.xml (markers must be direct children of w:p, never inside w:r): + + ... + + """ + +REPLY_MARKER_TEMPLATE = """ +Nest markers inside parent {pid}'s markers (markers must be direct children of w:p, never inside w:r): + + ... + + + """ + + +def _generate_hex_id() -> str: + return f"{random.randint(0, 0x7FFFFFFE):08X}" + + +SMART_QUOTE_ENTITIES = { + "\u201c": "“", + "\u201d": "”", + "\u2018": "‘", + "\u2019": "’", +} + + +def _encode_smart_quotes(text: str) -> str: + for char, entity in SMART_QUOTE_ENTITIES.items(): + text = text.replace(char, entity) + return text + + +def _append_xml(xml_path: Path, root_tag: str, content: str) -> None: + dom = defusedxml.minidom.parseString(xml_path.read_text(encoding="utf-8")) + root = dom.getElementsByTagName(root_tag)[0] + ns_attrs = " ".join(f'xmlns:{k}="{v}"' for k, v in NS.items()) + wrapper_dom = defusedxml.minidom.parseString(f"{content}") + for child in wrapper_dom.documentElement.childNodes: + if child.nodeType == child.ELEMENT_NODE: + root.appendChild(dom.importNode(child, True)) + output = _encode_smart_quotes(dom.toxml(encoding="UTF-8").decode("utf-8")) + xml_path.write_text(output, encoding="utf-8") + + +def _find_para_id(comments_path: Path, comment_id: int) -> str | None: + dom = defusedxml.minidom.parseString(comments_path.read_text(encoding="utf-8")) + for c in dom.getElementsByTagName("w:comment"): + if c.getAttribute("w:id") == str(comment_id): + for p in c.getElementsByTagName("w:p"): + if pid := p.getAttribute("w14:paraId"): + return pid + return None + + +def _get_next_rid(rels_path: Path) -> int: + dom = defusedxml.minidom.parseString(rels_path.read_text(encoding="utf-8")) + max_rid = 0 + for rel in dom.getElementsByTagName("Relationship"): + rid = rel.getAttribute("Id") + if rid and rid.startswith("rId"): + try: + max_rid = max(max_rid, int(rid[3:])) + except ValueError: + pass + return max_rid + 1 + + +def _has_relationship(rels_path: Path, target: str) -> bool: + dom = defusedxml.minidom.parseString(rels_path.read_text(encoding="utf-8")) + for rel in dom.getElementsByTagName("Relationship"): + if rel.getAttribute("Target") == target: + return True + return False + + +def _has_content_type(ct_path: Path, part_name: str) -> bool: + dom = defusedxml.minidom.parseString(ct_path.read_text(encoding="utf-8")) + for override in dom.getElementsByTagName("Override"): + if override.getAttribute("PartName") == part_name: + return True + return False + + +def _ensure_comment_relationships(unpacked_dir: Path) -> None: + rels_path = unpacked_dir / "word" / "_rels" / "document.xml.rels" + if not rels_path.exists(): + return + + if _has_relationship(rels_path, "comments.xml"): + return + + dom = defusedxml.minidom.parseString(rels_path.read_text(encoding="utf-8")) + root = dom.documentElement + next_rid = _get_next_rid(rels_path) + + rels = [ + ( + "http://schemas.openxmlformats.org/officeDocument/2006/relationships/comments", + "comments.xml", + ), + ( + "http://schemas.microsoft.com/office/2011/relationships/commentsExtended", + "commentsExtended.xml", + ), + ( + "http://schemas.microsoft.com/office/2016/09/relationships/commentsIds", + "commentsIds.xml", + ), + ( + "http://schemas.microsoft.com/office/2018/08/relationships/commentsExtensible", + "commentsExtensible.xml", + ), + ] + + for rel_type, target in rels: + rel = dom.createElement("Relationship") + rel.setAttribute("Id", f"rId{next_rid}") + rel.setAttribute("Type", rel_type) + rel.setAttribute("Target", target) + root.appendChild(rel) + next_rid += 1 + + rels_path.write_bytes(dom.toxml(encoding="UTF-8")) + + +def _ensure_comment_content_types(unpacked_dir: Path) -> None: + ct_path = unpacked_dir / "[Content_Types].xml" + if not ct_path.exists(): + return + + if _has_content_type(ct_path, "/word/comments.xml"): + return + + dom = defusedxml.minidom.parseString(ct_path.read_text(encoding="utf-8")) + root = dom.documentElement + + overrides = [ + ( + "/word/comments.xml", + "application/vnd.openxmlformats-officedocument.wordprocessingml.comments+xml", + ), + ( + "/word/commentsExtended.xml", + "application/vnd.openxmlformats-officedocument.wordprocessingml.commentsExtended+xml", + ), + ( + "/word/commentsIds.xml", + "application/vnd.openxmlformats-officedocument.wordprocessingml.commentsIds+xml", + ), + ( + "/word/commentsExtensible.xml", + "application/vnd.openxmlformats-officedocument.wordprocessingml.commentsExtensible+xml", + ), + ] + + for part_name, content_type in overrides: + override = dom.createElement("Override") + override.setAttribute("PartName", part_name) + override.setAttribute("ContentType", content_type) + root.appendChild(override) + + ct_path.write_bytes(dom.toxml(encoding="UTF-8")) + + +def add_comment( + unpacked_dir: str, + comment_id: int, + text: str, + author: str = "Claude", + initials: str = "C", + parent_id: int | None = None, +) -> tuple[str, str]: + word = Path(unpacked_dir) / "word" + if not word.exists(): + return "", f"Error: {word} not found" + + para_id, durable_id = _generate_hex_id(), _generate_hex_id() + ts = datetime.now(timezone.utc).strftime("%Y-%m-%dT%H:%M:%SZ") + + comments = word / "comments.xml" + first_comment = not comments.exists() + if first_comment: + shutil.copy(TEMPLATE_DIR / "comments.xml", comments) + _ensure_comment_relationships(Path(unpacked_dir)) + _ensure_comment_content_types(Path(unpacked_dir)) + _append_xml( + comments, + "w:comments", + COMMENT_XML.format( + id=comment_id, + author=author, + date=ts, + initials=initials, + para_id=para_id, + text=text, + ), + ) + + ext = word / "commentsExtended.xml" + if not ext.exists(): + shutil.copy(TEMPLATE_DIR / "commentsExtended.xml", ext) + if parent_id is not None: + parent_para = _find_para_id(comments, parent_id) + if not parent_para: + return "", f"Error: Parent comment {parent_id} not found" + _append_xml( + ext, + "w15:commentsEx", + f'', + ) + else: + _append_xml( + ext, + "w15:commentsEx", + f'', + ) + + ids = word / "commentsIds.xml" + if not ids.exists(): + shutil.copy(TEMPLATE_DIR / "commentsIds.xml", ids) + _append_xml( + ids, + "w16cid:commentsIds", + f'', + ) + + extensible = word / "commentsExtensible.xml" + if not extensible.exists(): + shutil.copy(TEMPLATE_DIR / "commentsExtensible.xml", extensible) + _append_xml( + extensible, + "w16cex:commentsExtensible", + f'', + ) + + action = "reply" if parent_id is not None else "comment" + return para_id, f"Added {action} {comment_id} (para_id={para_id})" + + +if __name__ == "__main__": + p = argparse.ArgumentParser(description="Add comments to DOCX documents") + p.add_argument("unpacked_dir", help="Unpacked DOCX directory") + p.add_argument("comment_id", type=int, help="Comment ID (must be unique)") + p.add_argument("text", help="Comment text") + p.add_argument("--author", default="Claude", help="Author name") + p.add_argument("--initials", default="C", help="Author initials") + p.add_argument("--parent", type=int, help="Parent comment ID (for replies)") + args = p.parse_args() + + para_id, msg = add_comment( + args.unpacked_dir, + args.comment_id, + args.text, + args.author, + args.initials, + args.parent, + ) + print(msg) + if "Error" in msg: + sys.exit(1) + cid = args.comment_id + if args.parent is not None: + print(REPLY_MARKER_TEMPLATE.format(pid=args.parent, cid=cid)) + else: + print(COMMENT_MARKER_TEMPLATE.format(cid=cid)) diff --git a/skills/docx/scripts/office/helpers/__init__.py b/skills/docx/scripts/office/helpers/__init__.py new file mode 100644 index 00000000..e69de29b diff --git a/skills/docx/scripts/office/helpers/merge_runs.py b/skills/docx/scripts/office/helpers/merge_runs.py new file mode 100644 index 00000000..ad7c25ee --- /dev/null +++ b/skills/docx/scripts/office/helpers/merge_runs.py @@ -0,0 +1,199 @@ +"""Merge adjacent runs with identical formatting in DOCX. + +Merges adjacent elements that have identical properties. +Works on runs in paragraphs and inside tracked changes (, ). + +Also: +- Removes rsid attributes from runs (revision metadata that doesn't affect rendering) +- Removes proofErr elements (spell/grammar markers that block merging) +""" + +from pathlib import Path + +import defusedxml.minidom + + +def merge_runs(input_dir: str) -> tuple[int, str]: + doc_xml = Path(input_dir) / "word" / "document.xml" + + if not doc_xml.exists(): + return 0, f"Error: {doc_xml} not found" + + try: + dom = defusedxml.minidom.parseString(doc_xml.read_text(encoding="utf-8")) + root = dom.documentElement + + _remove_elements(root, "proofErr") + _strip_run_rsid_attrs(root) + + containers = {run.parentNode for run in _find_elements(root, "r")} + + merge_count = 0 + for container in containers: + merge_count += _merge_runs_in(container) + + doc_xml.write_bytes(dom.toxml(encoding="UTF-8")) + return merge_count, f"Merged {merge_count} runs" + + except Exception as e: + return 0, f"Error: {e}" + + + + +def _find_elements(root, tag: str) -> list: + results = [] + + def traverse(node): + if node.nodeType == node.ELEMENT_NODE: + name = node.localName or node.tagName + if name == tag or name.endswith(f":{tag}"): + results.append(node) + for child in node.childNodes: + traverse(child) + + traverse(root) + return results + + +def _get_child(parent, tag: str): + for child in parent.childNodes: + if child.nodeType == child.ELEMENT_NODE: + name = child.localName or child.tagName + if name == tag or name.endswith(f":{tag}"): + return child + return None + + +def _get_children(parent, tag: str) -> list: + results = [] + for child in parent.childNodes: + if child.nodeType == child.ELEMENT_NODE: + name = child.localName or child.tagName + if name == tag or name.endswith(f":{tag}"): + results.append(child) + return results + + +def _is_adjacent(elem1, elem2) -> bool: + node = elem1.nextSibling + while node: + if node == elem2: + return True + if node.nodeType == node.ELEMENT_NODE: + return False + if node.nodeType == node.TEXT_NODE and node.data.strip(): + return False + node = node.nextSibling + return False + + + + +def _remove_elements(root, tag: str): + for elem in _find_elements(root, tag): + if elem.parentNode: + elem.parentNode.removeChild(elem) + + +def _strip_run_rsid_attrs(root): + for run in _find_elements(root, "r"): + for attr in list(run.attributes.values()): + if "rsid" in attr.name.lower(): + run.removeAttribute(attr.name) + + + + +def _merge_runs_in(container) -> int: + merge_count = 0 + run = _first_child_run(container) + + while run: + while True: + next_elem = _next_element_sibling(run) + if next_elem and _is_run(next_elem) and _can_merge(run, next_elem): + _merge_run_content(run, next_elem) + container.removeChild(next_elem) + merge_count += 1 + else: + break + + _consolidate_text(run) + run = _next_sibling_run(run) + + return merge_count + + +def _first_child_run(container): + for child in container.childNodes: + if child.nodeType == child.ELEMENT_NODE and _is_run(child): + return child + return None + + +def _next_element_sibling(node): + sibling = node.nextSibling + while sibling: + if sibling.nodeType == sibling.ELEMENT_NODE: + return sibling + sibling = sibling.nextSibling + return None + + +def _next_sibling_run(node): + sibling = node.nextSibling + while sibling: + if sibling.nodeType == sibling.ELEMENT_NODE: + if _is_run(sibling): + return sibling + sibling = sibling.nextSibling + return None + + +def _is_run(node) -> bool: + name = node.localName or node.tagName + return name == "r" or name.endswith(":r") + + +def _can_merge(run1, run2) -> bool: + rpr1 = _get_child(run1, "rPr") + rpr2 = _get_child(run2, "rPr") + + if (rpr1 is None) != (rpr2 is None): + return False + if rpr1 is None: + return True + return rpr1.toxml() == rpr2.toxml() + + +def _merge_run_content(target, source): + for child in list(source.childNodes): + if child.nodeType == child.ELEMENT_NODE: + name = child.localName or child.tagName + if name != "rPr" and not name.endswith(":rPr"): + target.appendChild(child) + + +def _consolidate_text(run): + t_elements = _get_children(run, "t") + + for i in range(len(t_elements) - 1, 0, -1): + curr, prev = t_elements[i], t_elements[i - 1] + + if _is_adjacent(prev, curr): + prev_text = prev.firstChild.data if prev.firstChild else "" + curr_text = curr.firstChild.data if curr.firstChild else "" + merged = prev_text + curr_text + + if prev.firstChild: + prev.firstChild.data = merged + else: + prev.appendChild(run.ownerDocument.createTextNode(merged)) + + if merged.startswith(" ") or merged.endswith(" "): + prev.setAttribute("xml:space", "preserve") + elif prev.hasAttribute("xml:space"): + prev.removeAttribute("xml:space") + + run.removeChild(curr) diff --git a/skills/docx/scripts/office/helpers/simplify_redlines.py b/skills/docx/scripts/office/helpers/simplify_redlines.py new file mode 100644 index 00000000..db963bb9 --- /dev/null +++ b/skills/docx/scripts/office/helpers/simplify_redlines.py @@ -0,0 +1,197 @@ +"""Simplify tracked changes by merging adjacent w:ins or w:del elements. + +Merges adjacent elements from the same author into a single element. +Same for elements. This makes heavily-redlined documents easier to +work with by reducing the number of tracked change wrappers. + +Rules: +- Only merges w:ins with w:ins, w:del with w:del (same element type) +- Only merges if same author (ignores timestamp differences) +- Only merges if truly adjacent (only whitespace between them) +""" + +import xml.etree.ElementTree as ET +import zipfile +from pathlib import Path + +import defusedxml.minidom + +WORD_NS = "http://schemas.openxmlformats.org/wordprocessingml/2006/main" + + +def simplify_redlines(input_dir: str) -> tuple[int, str]: + doc_xml = Path(input_dir) / "word" / "document.xml" + + if not doc_xml.exists(): + return 0, f"Error: {doc_xml} not found" + + try: + dom = defusedxml.minidom.parseString(doc_xml.read_text(encoding="utf-8")) + root = dom.documentElement + + merge_count = 0 + + containers = _find_elements(root, "p") + _find_elements(root, "tc") + + for container in containers: + merge_count += _merge_tracked_changes_in(container, "ins") + merge_count += _merge_tracked_changes_in(container, "del") + + doc_xml.write_bytes(dom.toxml(encoding="UTF-8")) + return merge_count, f"Simplified {merge_count} tracked changes" + + except Exception as e: + return 0, f"Error: {e}" + + +def _merge_tracked_changes_in(container, tag: str) -> int: + merge_count = 0 + + tracked = [ + child + for child in container.childNodes + if child.nodeType == child.ELEMENT_NODE and _is_element(child, tag) + ] + + if len(tracked) < 2: + return 0 + + i = 0 + while i < len(tracked) - 1: + curr = tracked[i] + next_elem = tracked[i + 1] + + if _can_merge_tracked(curr, next_elem): + _merge_tracked_content(curr, next_elem) + container.removeChild(next_elem) + tracked.pop(i + 1) + merge_count += 1 + else: + i += 1 + + return merge_count + + +def _is_element(node, tag: str) -> bool: + name = node.localName or node.tagName + return name == tag or name.endswith(f":{tag}") + + +def _get_author(elem) -> str: + author = elem.getAttribute("w:author") + if not author: + for attr in elem.attributes.values(): + if attr.localName == "author" or attr.name.endswith(":author"): + return attr.value + return author + + +def _can_merge_tracked(elem1, elem2) -> bool: + if _get_author(elem1) != _get_author(elem2): + return False + + node = elem1.nextSibling + while node and node != elem2: + if node.nodeType == node.ELEMENT_NODE: + return False + if node.nodeType == node.TEXT_NODE and node.data.strip(): + return False + node = node.nextSibling + + return True + + +def _merge_tracked_content(target, source): + while source.firstChild: + child = source.firstChild + source.removeChild(child) + target.appendChild(child) + + +def _find_elements(root, tag: str) -> list: + results = [] + + def traverse(node): + if node.nodeType == node.ELEMENT_NODE: + name = node.localName or node.tagName + if name == tag or name.endswith(f":{tag}"): + results.append(node) + for child in node.childNodes: + traverse(child) + + traverse(root) + return results + + +def get_tracked_change_authors(doc_xml_path: Path) -> dict[str, int]: + if not doc_xml_path.exists(): + return {} + + try: + tree = ET.parse(doc_xml_path) + root = tree.getroot() + except ET.ParseError: + return {} + + namespaces = {"w": WORD_NS} + author_attr = f"{{{WORD_NS}}}author" + + authors: dict[str, int] = {} + for tag in ["ins", "del"]: + for elem in root.findall(f".//w:{tag}", namespaces): + author = elem.get(author_attr) + if author: + authors[author] = authors.get(author, 0) + 1 + + return authors + + +def _get_authors_from_docx(docx_path: Path) -> dict[str, int]: + try: + with zipfile.ZipFile(docx_path, "r") as zf: + if "word/document.xml" not in zf.namelist(): + return {} + with zf.open("word/document.xml") as f: + tree = ET.parse(f) + root = tree.getroot() + + namespaces = {"w": WORD_NS} + author_attr = f"{{{WORD_NS}}}author" + + authors: dict[str, int] = {} + for tag in ["ins", "del"]: + for elem in root.findall(f".//w:{tag}", namespaces): + author = elem.get(author_attr) + if author: + authors[author] = authors.get(author, 0) + 1 + return authors + except (zipfile.BadZipFile, ET.ParseError): + return {} + + +def infer_author(modified_dir: Path, original_docx: Path, default: str = "Claude") -> str: + modified_xml = modified_dir / "word" / "document.xml" + modified_authors = get_tracked_change_authors(modified_xml) + + if not modified_authors: + return default + + original_authors = _get_authors_from_docx(original_docx) + + new_changes: dict[str, int] = {} + for author, count in modified_authors.items(): + original_count = original_authors.get(author, 0) + diff = count - original_count + if diff > 0: + new_changes[author] = diff + + if not new_changes: + return default + + if len(new_changes) == 1: + return next(iter(new_changes)) + + raise ValueError( + f"Multiple authors added new changes: {new_changes}. " + "Cannot infer which author to validate." + ) diff --git a/skills/docx/scripts/office/pack.py b/skills/docx/scripts/office/pack.py new file mode 100644 index 00000000..db29ed8b --- /dev/null +++ b/skills/docx/scripts/office/pack.py @@ -0,0 +1,159 @@ +"""Pack a directory into a DOCX, PPTX, or XLSX file. + +Validates with auto-repair, condenses XML formatting, and creates the Office file. + +Usage: + python pack.py [--original ] [--validate true|false] + +Examples: + python pack.py unpacked/ output.docx --original input.docx + python pack.py unpacked/ output.pptx --validate false +""" + +import argparse +import sys +import shutil +import tempfile +import zipfile +from pathlib import Path + +import defusedxml.minidom + +from validators import DOCXSchemaValidator, PPTXSchemaValidator, RedliningValidator + +def pack( + input_directory: str, + output_file: str, + original_file: str | None = None, + validate: bool = True, + infer_author_func=None, +) -> tuple[None, str]: + input_dir = Path(input_directory) + output_path = Path(output_file) + suffix = output_path.suffix.lower() + + if not input_dir.is_dir(): + return None, f"Error: {input_dir} is not a directory" + + if suffix not in {".docx", ".pptx", ".xlsx"}: + return None, f"Error: {output_file} must be a .docx, .pptx, or .xlsx file" + + if validate and original_file: + original_path = Path(original_file) + if original_path.exists(): + success, output = _run_validation( + input_dir, original_path, suffix, infer_author_func + ) + if output: + print(output) + if not success: + return None, f"Error: Validation failed for {input_dir}" + + with tempfile.TemporaryDirectory() as temp_dir: + temp_content_dir = Path(temp_dir) / "content" + shutil.copytree(input_dir, temp_content_dir) + + for pattern in ["*.xml", "*.rels"]: + for xml_file in temp_content_dir.rglob(pattern): + _condense_xml(xml_file) + + output_path.parent.mkdir(parents=True, exist_ok=True) + with zipfile.ZipFile(output_path, "w", zipfile.ZIP_DEFLATED) as zf: + for f in temp_content_dir.rglob("*"): + if f.is_file(): + zf.write(f, f.relative_to(temp_content_dir)) + + return None, f"Successfully packed {input_dir} to {output_file}" + + +def _run_validation( + unpacked_dir: Path, + original_file: Path, + suffix: str, + infer_author_func=None, +) -> tuple[bool, str | None]: + output_lines = [] + validators = [] + + if suffix == ".docx": + author = "Claude" + if infer_author_func: + try: + author = infer_author_func(unpacked_dir, original_file) + except ValueError as e: + print(f"Warning: {e} Using default author 'Claude'.", file=sys.stderr) + + validators = [ + DOCXSchemaValidator(unpacked_dir, original_file), + RedliningValidator(unpacked_dir, original_file, author=author), + ] + elif suffix == ".pptx": + validators = [PPTXSchemaValidator(unpacked_dir, original_file)] + + if not validators: + return True, None + + total_repairs = sum(v.repair() for v in validators) + if total_repairs: + output_lines.append(f"Auto-repaired {total_repairs} issue(s)") + + success = all(v.validate() for v in validators) + + if success: + output_lines.append("All validations PASSED!") + + return success, "\n".join(output_lines) if output_lines else None + + +def _condense_xml(xml_file: Path) -> None: + try: + with open(xml_file, encoding="utf-8") as f: + dom = defusedxml.minidom.parse(f) + + for element in dom.getElementsByTagName("*"): + if element.tagName.endswith(":t"): + continue + + for child in list(element.childNodes): + if ( + child.nodeType == child.TEXT_NODE + and child.nodeValue + and child.nodeValue.strip() == "" + ) or child.nodeType == child.COMMENT_NODE: + element.removeChild(child) + + xml_file.write_bytes(dom.toxml(encoding="UTF-8")) + except Exception as e: + print(f"ERROR: Failed to parse {xml_file.name}: {e}", file=sys.stderr) + raise + + +if __name__ == "__main__": + parser = argparse.ArgumentParser( + description="Pack a directory into a DOCX, PPTX, or XLSX file" + ) + parser.add_argument("input_directory", help="Unpacked Office document directory") + parser.add_argument("output_file", help="Output Office file (.docx/.pptx/.xlsx)") + parser.add_argument( + "--original", + help="Original file for validation comparison", + ) + parser.add_argument( + "--validate", + type=lambda x: x.lower() == "true", + default=True, + metavar="true|false", + help="Run validation with auto-repair (default: true)", + ) + args = parser.parse_args() + + _, message = pack( + args.input_directory, + args.output_file, + original_file=args.original, + validate=args.validate, + ) + print(message) + + if "Error" in message: + sys.exit(1) diff --git a/skills/docx/scripts/office/schemas/ISO-IEC29500-4_2016/dml-chart.xsd b/skills/docx/scripts/office/schemas/ISO-IEC29500-4_2016/dml-chart.xsd new file mode 100644 index 00000000..bc325f9f --- /dev/null +++ b/skills/docx/scripts/office/schemas/ISO-IEC29500-4_2016/dml-chart.xsd @@ -0,0 +1,1499 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/skills/docx/scripts/office/schemas/ISO-IEC29500-4_2016/dml-chartDrawing.xsd b/skills/docx/scripts/office/schemas/ISO-IEC29500-4_2016/dml-chartDrawing.xsd new file mode 100644 index 00000000..afa4f463 --- /dev/null +++ b/skills/docx/scripts/office/schemas/ISO-IEC29500-4_2016/dml-chartDrawing.xsd @@ -0,0 +1,146 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/skills/docx/scripts/office/schemas/ISO-IEC29500-4_2016/dml-diagram.xsd b/skills/docx/scripts/office/schemas/ISO-IEC29500-4_2016/dml-diagram.xsd new file mode 100644 index 00000000..40e4b12a --- /dev/null +++ b/skills/docx/scripts/office/schemas/ISO-IEC29500-4_2016/dml-diagram.xsd @@ -0,0 +1,1085 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/skills/docx/scripts/office/schemas/ISO-IEC29500-4_2016/dml-lockedCanvas.xsd b/skills/docx/scripts/office/schemas/ISO-IEC29500-4_2016/dml-lockedCanvas.xsd new file mode 100644 index 00000000..687eea82 --- /dev/null +++ b/skills/docx/scripts/office/schemas/ISO-IEC29500-4_2016/dml-lockedCanvas.xsd @@ -0,0 +1,11 @@ + + + + + diff --git a/skills/docx/scripts/office/schemas/ISO-IEC29500-4_2016/dml-main.xsd b/skills/docx/scripts/office/schemas/ISO-IEC29500-4_2016/dml-main.xsd new file mode 100644 index 00000000..94644b3f --- /dev/null +++ b/skills/docx/scripts/office/schemas/ISO-IEC29500-4_2016/dml-main.xsd @@ -0,0 +1,3081 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/skills/docx/scripts/office/schemas/ISO-IEC29500-4_2016/dml-picture.xsd b/skills/docx/scripts/office/schemas/ISO-IEC29500-4_2016/dml-picture.xsd new file mode 100644 index 00000000..1dbf0514 --- /dev/null +++ b/skills/docx/scripts/office/schemas/ISO-IEC29500-4_2016/dml-picture.xsd @@ -0,0 +1,23 @@ + + + + + + + + + + + + + + + + + + diff --git a/skills/docx/scripts/office/schemas/ISO-IEC29500-4_2016/dml-spreadsheetDrawing.xsd b/skills/docx/scripts/office/schemas/ISO-IEC29500-4_2016/dml-spreadsheetDrawing.xsd new file mode 100644 index 00000000..f1af17db --- /dev/null +++ b/skills/docx/scripts/office/schemas/ISO-IEC29500-4_2016/dml-spreadsheetDrawing.xsd @@ -0,0 +1,185 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/skills/docx/scripts/office/schemas/ISO-IEC29500-4_2016/dml-wordprocessingDrawing.xsd b/skills/docx/scripts/office/schemas/ISO-IEC29500-4_2016/dml-wordprocessingDrawing.xsd new file mode 100644 index 00000000..5c00a6ff --- /dev/null +++ b/skills/docx/scripts/office/schemas/ISO-IEC29500-4_2016/dml-wordprocessingDrawing.xsd @@ -0,0 +1,287 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/skills/docx/scripts/office/schemas/ISO-IEC29500-4_2016/pml.xsd b/skills/docx/scripts/office/schemas/ISO-IEC29500-4_2016/pml.xsd new file mode 100644 index 00000000..25564ebb --- /dev/null +++ b/skills/docx/scripts/office/schemas/ISO-IEC29500-4_2016/pml.xsd @@ -0,0 +1,1676 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/skills/docx/scripts/office/schemas/ISO-IEC29500-4_2016/shared-additionalCharacteristics.xsd b/skills/docx/scripts/office/schemas/ISO-IEC29500-4_2016/shared-additionalCharacteristics.xsd new file mode 100644 index 00000000..c20f3bf1 --- /dev/null +++ b/skills/docx/scripts/office/schemas/ISO-IEC29500-4_2016/shared-additionalCharacteristics.xsd @@ -0,0 +1,28 @@ + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/skills/docx/scripts/office/schemas/ISO-IEC29500-4_2016/shared-bibliography.xsd b/skills/docx/scripts/office/schemas/ISO-IEC29500-4_2016/shared-bibliography.xsd new file mode 100644 index 00000000..ac602522 --- /dev/null +++ b/skills/docx/scripts/office/schemas/ISO-IEC29500-4_2016/shared-bibliography.xsd @@ -0,0 +1,144 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/skills/docx/scripts/office/schemas/ISO-IEC29500-4_2016/shared-commonSimpleTypes.xsd b/skills/docx/scripts/office/schemas/ISO-IEC29500-4_2016/shared-commonSimpleTypes.xsd new file mode 100644 index 00000000..52deec72 --- /dev/null +++ b/skills/docx/scripts/office/schemas/ISO-IEC29500-4_2016/shared-commonSimpleTypes.xsd @@ -0,0 +1,174 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/skills/docx/scripts/office/schemas/ISO-IEC29500-4_2016/shared-customXmlDataProperties.xsd b/skills/docx/scripts/office/schemas/ISO-IEC29500-4_2016/shared-customXmlDataProperties.xsd new file mode 100644 index 00000000..2bddce29 --- /dev/null +++ b/skills/docx/scripts/office/schemas/ISO-IEC29500-4_2016/shared-customXmlDataProperties.xsd @@ -0,0 +1,25 @@ + + + + + + + + + + + + + + + + + + + diff --git a/skills/docx/scripts/office/schemas/ISO-IEC29500-4_2016/shared-customXmlSchemaProperties.xsd b/skills/docx/scripts/office/schemas/ISO-IEC29500-4_2016/shared-customXmlSchemaProperties.xsd new file mode 100644 index 00000000..8a8c18ba --- /dev/null +++ b/skills/docx/scripts/office/schemas/ISO-IEC29500-4_2016/shared-customXmlSchemaProperties.xsd @@ -0,0 +1,18 @@ + + + + + + + + + + + + + + + diff --git a/skills/docx/scripts/office/schemas/ISO-IEC29500-4_2016/shared-documentPropertiesCustom.xsd b/skills/docx/scripts/office/schemas/ISO-IEC29500-4_2016/shared-documentPropertiesCustom.xsd new file mode 100644 index 00000000..5c42706a --- /dev/null +++ b/skills/docx/scripts/office/schemas/ISO-IEC29500-4_2016/shared-documentPropertiesCustom.xsd @@ -0,0 +1,59 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/skills/docx/scripts/office/schemas/ISO-IEC29500-4_2016/shared-documentPropertiesExtended.xsd b/skills/docx/scripts/office/schemas/ISO-IEC29500-4_2016/shared-documentPropertiesExtended.xsd new file mode 100644 index 00000000..853c341c --- /dev/null +++ b/skills/docx/scripts/office/schemas/ISO-IEC29500-4_2016/shared-documentPropertiesExtended.xsd @@ -0,0 +1,56 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/skills/docx/scripts/office/schemas/ISO-IEC29500-4_2016/shared-documentPropertiesVariantTypes.xsd b/skills/docx/scripts/office/schemas/ISO-IEC29500-4_2016/shared-documentPropertiesVariantTypes.xsd new file mode 100644 index 00000000..da835ee8 --- /dev/null +++ b/skills/docx/scripts/office/schemas/ISO-IEC29500-4_2016/shared-documentPropertiesVariantTypes.xsd @@ -0,0 +1,195 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/skills/docx/scripts/office/schemas/ISO-IEC29500-4_2016/shared-math.xsd b/skills/docx/scripts/office/schemas/ISO-IEC29500-4_2016/shared-math.xsd new file mode 100644 index 00000000..4f37d307 --- /dev/null +++ b/skills/docx/scripts/office/schemas/ISO-IEC29500-4_2016/shared-math.xsd @@ -0,0 +1,582 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/skills/docx/scripts/office/schemas/ISO-IEC29500-4_2016/shared-relationshipReference.xsd b/skills/docx/scripts/office/schemas/ISO-IEC29500-4_2016/shared-relationshipReference.xsd new file mode 100644 index 00000000..9e86f1b2 --- /dev/null +++ b/skills/docx/scripts/office/schemas/ISO-IEC29500-4_2016/shared-relationshipReference.xsd @@ -0,0 +1,25 @@ + + + + + + + + + + + + + + + + + + + + diff --git a/skills/docx/scripts/office/schemas/ISO-IEC29500-4_2016/sml.xsd b/skills/docx/scripts/office/schemas/ISO-IEC29500-4_2016/sml.xsd new file mode 100644 index 00000000..237dd652 --- /dev/null +++ b/skills/docx/scripts/office/schemas/ISO-IEC29500-4_2016/sml.xsd @@ -0,0 +1,4439 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/skills/docx/scripts/office/schemas/ISO-IEC29500-4_2016/vml-main.xsd b/skills/docx/scripts/office/schemas/ISO-IEC29500-4_2016/vml-main.xsd new file mode 100644 index 00000000..eeb4ef8f --- /dev/null +++ b/skills/docx/scripts/office/schemas/ISO-IEC29500-4_2016/vml-main.xsd @@ -0,0 +1,570 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/skills/docx/scripts/office/schemas/ISO-IEC29500-4_2016/vml-officeDrawing.xsd b/skills/docx/scripts/office/schemas/ISO-IEC29500-4_2016/vml-officeDrawing.xsd new file mode 100644 index 00000000..ca2575c7 --- /dev/null +++ b/skills/docx/scripts/office/schemas/ISO-IEC29500-4_2016/vml-officeDrawing.xsd @@ -0,0 +1,509 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/skills/docx/scripts/office/schemas/ISO-IEC29500-4_2016/vml-presentationDrawing.xsd b/skills/docx/scripts/office/schemas/ISO-IEC29500-4_2016/vml-presentationDrawing.xsd new file mode 100644 index 00000000..dd079e60 --- /dev/null +++ b/skills/docx/scripts/office/schemas/ISO-IEC29500-4_2016/vml-presentationDrawing.xsd @@ -0,0 +1,12 @@ + + + + + + + + + diff --git a/skills/docx/scripts/office/schemas/ISO-IEC29500-4_2016/vml-spreadsheetDrawing.xsd b/skills/docx/scripts/office/schemas/ISO-IEC29500-4_2016/vml-spreadsheetDrawing.xsd new file mode 100644 index 00000000..3dd6cf62 --- /dev/null +++ b/skills/docx/scripts/office/schemas/ISO-IEC29500-4_2016/vml-spreadsheetDrawing.xsd @@ -0,0 +1,108 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/skills/docx/scripts/office/schemas/ISO-IEC29500-4_2016/vml-wordprocessingDrawing.xsd b/skills/docx/scripts/office/schemas/ISO-IEC29500-4_2016/vml-wordprocessingDrawing.xsd new file mode 100644 index 00000000..f1041e34 --- /dev/null +++ b/skills/docx/scripts/office/schemas/ISO-IEC29500-4_2016/vml-wordprocessingDrawing.xsd @@ -0,0 +1,96 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/skills/docx/scripts/office/schemas/ISO-IEC29500-4_2016/wml.xsd b/skills/docx/scripts/office/schemas/ISO-IEC29500-4_2016/wml.xsd new file mode 100644 index 00000000..9c5b7a63 --- /dev/null +++ b/skills/docx/scripts/office/schemas/ISO-IEC29500-4_2016/wml.xsd @@ -0,0 +1,3646 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/skills/docx/scripts/office/schemas/ISO-IEC29500-4_2016/xml.xsd b/skills/docx/scripts/office/schemas/ISO-IEC29500-4_2016/xml.xsd new file mode 100644 index 00000000..fbd88768 --- /dev/null +++ b/skills/docx/scripts/office/schemas/ISO-IEC29500-4_2016/xml.xsd @@ -0,0 +1,116 @@ + + + + + + See http://www.w3.org/XML/1998/namespace.html and + http://www.w3.org/TR/REC-xml for information about this namespace. + + This schema document describes the XML namespace, in a form + suitable for import by other schema documents. + + Note that local names in this namespace are intended to be defined + only by the World Wide Web Consortium or its subgroups. The + following names are currently defined in this namespace and should + not be used with conflicting semantics by any Working Group, + specification, or document instance: + + base (as an attribute name): denotes an attribute whose value + provides a URI to be used as the base for interpreting any + relative URIs in the scope of the element on which it + appears; its value is inherited. This name is reserved + by virtue of its definition in the XML Base specification. + + lang (as an attribute name): denotes an attribute whose value + is a language code for the natural language of the content of + any element; its value is inherited. This name is reserved + by virtue of its definition in the XML specification. + + space (as an attribute name): denotes an attribute whose + value is a keyword indicating what whitespace processing + discipline is intended for the content of the element; its + value is inherited. This name is reserved by virtue of its + definition in the XML specification. + + Father (in any context at all): denotes Jon Bosak, the chair of + the original XML Working Group. This name is reserved by + the following decision of the W3C XML Plenary and + XML Coordination groups: + + In appreciation for his vision, leadership and dedication + the W3C XML Plenary on this 10th day of February, 2000 + reserves for Jon Bosak in perpetuity the XML name + xml:Father + + + + + This schema defines attributes and an attribute group + suitable for use by + schemas wishing to allow xml:base, xml:lang or xml:space attributes + on elements they define. + + To enable this, such a schema must import this schema + for the XML namespace, e.g. as follows: + <schema . . .> + . . . + <import namespace="http://www.w3.org/XML/1998/namespace" + schemaLocation="http://www.w3.org/2001/03/xml.xsd"/> + + Subsequently, qualified reference to any of the attributes + or the group defined below will have the desired effect, e.g. + + <type . . .> + . . . + <attributeGroup ref="xml:specialAttrs"/> + + will define a type which will schema-validate an instance + element with any of those attributes + + + + In keeping with the XML Schema WG's standard versioning + policy, this schema document will persist at + http://www.w3.org/2001/03/xml.xsd. + At the date of issue it can also be found at + http://www.w3.org/2001/xml.xsd. + The schema document at that URI may however change in the future, + in order to remain compatible with the latest version of XML Schema + itself. In other words, if the XML Schema namespace changes, the version + of this document at + http://www.w3.org/2001/xml.xsd will change + accordingly; the version at + http://www.w3.org/2001/03/xml.xsd will not change. + + + + + + In due course, we should install the relevant ISO 2- and 3-letter + codes as the enumerated possible values . . . + + + + + + + + + + + + + + + See http://www.w3.org/TR/xmlbase/ for + information about this attribute. + + + + + + + + + + diff --git a/skills/docx/scripts/office/schemas/ecma/fouth-edition/opc-contentTypes.xsd b/skills/docx/scripts/office/schemas/ecma/fouth-edition/opc-contentTypes.xsd new file mode 100644 index 00000000..e4c5160e --- /dev/null +++ b/skills/docx/scripts/office/schemas/ecma/fouth-edition/opc-contentTypes.xsd @@ -0,0 +1,42 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/skills/docx/scripts/office/schemas/ecma/fouth-edition/opc-coreProperties.xsd b/skills/docx/scripts/office/schemas/ecma/fouth-edition/opc-coreProperties.xsd new file mode 100644 index 00000000..888c0fcd --- /dev/null +++ b/skills/docx/scripts/office/schemas/ecma/fouth-edition/opc-coreProperties.xsd @@ -0,0 +1,50 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/skills/docx/scripts/office/schemas/ecma/fouth-edition/opc-digSig.xsd b/skills/docx/scripts/office/schemas/ecma/fouth-edition/opc-digSig.xsd new file mode 100644 index 00000000..73782264 --- /dev/null +++ b/skills/docx/scripts/office/schemas/ecma/fouth-edition/opc-digSig.xsd @@ -0,0 +1,49 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/skills/docx/scripts/office/schemas/ecma/fouth-edition/opc-relationships.xsd b/skills/docx/scripts/office/schemas/ecma/fouth-edition/opc-relationships.xsd new file mode 100644 index 00000000..762dcbe8 --- /dev/null +++ b/skills/docx/scripts/office/schemas/ecma/fouth-edition/opc-relationships.xsd @@ -0,0 +1,33 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/skills/docx/scripts/office/schemas/mce/mc.xsd b/skills/docx/scripts/office/schemas/mce/mc.xsd new file mode 100644 index 00000000..ef725457 --- /dev/null +++ b/skills/docx/scripts/office/schemas/mce/mc.xsd @@ -0,0 +1,75 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/skills/docx/scripts/office/schemas/microsoft/wml-2010.xsd b/skills/docx/scripts/office/schemas/microsoft/wml-2010.xsd new file mode 100644 index 00000000..f65f7777 --- /dev/null +++ b/skills/docx/scripts/office/schemas/microsoft/wml-2010.xsd @@ -0,0 +1,560 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/skills/docx/scripts/office/schemas/microsoft/wml-2012.xsd b/skills/docx/scripts/office/schemas/microsoft/wml-2012.xsd new file mode 100644 index 00000000..6b00755a --- /dev/null +++ b/skills/docx/scripts/office/schemas/microsoft/wml-2012.xsd @@ -0,0 +1,67 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/skills/docx/scripts/office/schemas/microsoft/wml-2018.xsd b/skills/docx/scripts/office/schemas/microsoft/wml-2018.xsd new file mode 100644 index 00000000..f321d333 --- /dev/null +++ b/skills/docx/scripts/office/schemas/microsoft/wml-2018.xsd @@ -0,0 +1,14 @@ + + + + + + + + + + + + + + diff --git a/skills/docx/scripts/office/schemas/microsoft/wml-cex-2018.xsd b/skills/docx/scripts/office/schemas/microsoft/wml-cex-2018.xsd new file mode 100644 index 00000000..364c6a9b --- /dev/null +++ b/skills/docx/scripts/office/schemas/microsoft/wml-cex-2018.xsd @@ -0,0 +1,20 @@ + + + + + + + + + + + + + + + + + + + + diff --git a/skills/docx/scripts/office/schemas/microsoft/wml-cid-2016.xsd b/skills/docx/scripts/office/schemas/microsoft/wml-cid-2016.xsd new file mode 100644 index 00000000..fed9d15b --- /dev/null +++ b/skills/docx/scripts/office/schemas/microsoft/wml-cid-2016.xsd @@ -0,0 +1,13 @@ + + + + + + + + + + + + + diff --git a/skills/docx/scripts/office/schemas/microsoft/wml-sdtdatahash-2020.xsd b/skills/docx/scripts/office/schemas/microsoft/wml-sdtdatahash-2020.xsd new file mode 100644 index 00000000..680cf154 --- /dev/null +++ b/skills/docx/scripts/office/schemas/microsoft/wml-sdtdatahash-2020.xsd @@ -0,0 +1,4 @@ + + + + diff --git a/skills/docx/scripts/office/schemas/microsoft/wml-symex-2015.xsd b/skills/docx/scripts/office/schemas/microsoft/wml-symex-2015.xsd new file mode 100644 index 00000000..89ada908 --- /dev/null +++ b/skills/docx/scripts/office/schemas/microsoft/wml-symex-2015.xsd @@ -0,0 +1,8 @@ + + + + + + + + diff --git a/skills/docx/scripts/office/soffice.py b/skills/docx/scripts/office/soffice.py new file mode 100644 index 00000000..c7f7e328 --- /dev/null +++ b/skills/docx/scripts/office/soffice.py @@ -0,0 +1,183 @@ +""" +Helper for running LibreOffice (soffice) in environments where AF_UNIX +sockets may be blocked (e.g., sandboxed VMs). Detects the restriction +at runtime and applies an LD_PRELOAD shim if needed. + +Usage: + from office.soffice import run_soffice, get_soffice_env + + # Option 1 – run soffice directly + result = run_soffice(["--headless", "--convert-to", "pdf", "input.docx"]) + + # Option 2 – get env dict for your own subprocess calls + env = get_soffice_env() + subprocess.run(["soffice", ...], env=env) +""" + +import os +import socket +import subprocess +import tempfile +from pathlib import Path + + +def get_soffice_env() -> dict: + env = os.environ.copy() + env["SAL_USE_VCLPLUGIN"] = "svp" + + if _needs_shim(): + shim = _ensure_shim() + env["LD_PRELOAD"] = str(shim) + + return env + + +def run_soffice(args: list[str], **kwargs) -> subprocess.CompletedProcess: + env = get_soffice_env() + return subprocess.run(["soffice"] + args, env=env, **kwargs) + + + +_SHIM_SO = Path(tempfile.gettempdir()) / "lo_socket_shim.so" + + +def _needs_shim() -> bool: + try: + s = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM) + s.close() + return False + except OSError: + return True + + +def _ensure_shim() -> Path: + if _SHIM_SO.exists(): + return _SHIM_SO + + src = Path(tempfile.gettempdir()) / "lo_socket_shim.c" + src.write_text(_SHIM_SOURCE) + subprocess.run( + ["gcc", "-shared", "-fPIC", "-o", str(_SHIM_SO), str(src), "-ldl"], + check=True, + capture_output=True, + ) + src.unlink() + return _SHIM_SO + + + +_SHIM_SOURCE = r""" +#define _GNU_SOURCE +#include +#include +#include +#include +#include +#include +#include + +static int (*real_socket)(int, int, int); +static int (*real_socketpair)(int, int, int, int[2]); +static int (*real_listen)(int, int); +static int (*real_accept)(int, struct sockaddr *, socklen_t *); +static int (*real_close)(int); +static int (*real_read)(int, void *, size_t); + +/* Per-FD bookkeeping (FDs >= 1024 are passed through unshimmed). */ +static int is_shimmed[1024]; +static int peer_of[1024]; +static int wake_r[1024]; /* accept() blocks reading this */ +static int wake_w[1024]; /* close() writes to this */ +static int listener_fd = -1; /* FD that received listen() */ + +__attribute__((constructor)) +static void init(void) { + real_socket = dlsym(RTLD_NEXT, "socket"); + real_socketpair = dlsym(RTLD_NEXT, "socketpair"); + real_listen = dlsym(RTLD_NEXT, "listen"); + real_accept = dlsym(RTLD_NEXT, "accept"); + real_close = dlsym(RTLD_NEXT, "close"); + real_read = dlsym(RTLD_NEXT, "read"); + for (int i = 0; i < 1024; i++) { + peer_of[i] = -1; + wake_r[i] = -1; + wake_w[i] = -1; + } +} + +/* ---- socket ---------------------------------------------------------- */ +int socket(int domain, int type, int protocol) { + if (domain == AF_UNIX) { + int fd = real_socket(domain, type, protocol); + if (fd >= 0) return fd; + /* socket(AF_UNIX) blocked – fall back to socketpair(). */ + int sv[2]; + if (real_socketpair(domain, type, protocol, sv) == 0) { + if (sv[0] >= 0 && sv[0] < 1024) { + is_shimmed[sv[0]] = 1; + peer_of[sv[0]] = sv[1]; + int wp[2]; + if (pipe(wp) == 0) { + wake_r[sv[0]] = wp[0]; + wake_w[sv[0]] = wp[1]; + } + } + return sv[0]; + } + errno = EPERM; + return -1; + } + return real_socket(domain, type, protocol); +} + +/* ---- listen ---------------------------------------------------------- */ +int listen(int sockfd, int backlog) { + if (sockfd >= 0 && sockfd < 1024 && is_shimmed[sockfd]) { + listener_fd = sockfd; + return 0; + } + return real_listen(sockfd, backlog); +} + +/* ---- accept ---------------------------------------------------------- */ +int accept(int sockfd, struct sockaddr *addr, socklen_t *addrlen) { + if (sockfd >= 0 && sockfd < 1024 && is_shimmed[sockfd]) { + /* Block until close() writes to the wake pipe. */ + if (wake_r[sockfd] >= 0) { + char buf; + real_read(wake_r[sockfd], &buf, 1); + } + errno = ECONNABORTED; + return -1; + } + return real_accept(sockfd, addr, addrlen); +} + +/* ---- close ----------------------------------------------------------- */ +int close(int fd) { + if (fd >= 0 && fd < 1024 && is_shimmed[fd]) { + int was_listener = (fd == listener_fd); + is_shimmed[fd] = 0; + + if (wake_w[fd] >= 0) { /* unblock accept() */ + char c = 0; + write(wake_w[fd], &c, 1); + real_close(wake_w[fd]); + wake_w[fd] = -1; + } + if (wake_r[fd] >= 0) { real_close(wake_r[fd]); wake_r[fd] = -1; } + if (peer_of[fd] >= 0) { real_close(peer_of[fd]); peer_of[fd] = -1; } + + if (was_listener) + _exit(0); /* conversion done – exit */ + } + return real_close(fd); +} +""" + + + +if __name__ == "__main__": + import sys + result = run_soffice(sys.argv[1:]) + sys.exit(result.returncode) diff --git a/skills/docx/scripts/office/unpack.py b/skills/docx/scripts/office/unpack.py new file mode 100644 index 00000000..00152533 --- /dev/null +++ b/skills/docx/scripts/office/unpack.py @@ -0,0 +1,132 @@ +"""Unpack Office files (DOCX, PPTX, XLSX) for editing. + +Extracts the ZIP archive, pretty-prints XML files, and optionally: +- Merges adjacent runs with identical formatting (DOCX only) +- Simplifies adjacent tracked changes from same author (DOCX only) + +Usage: + python unpack.py [options] + +Examples: + python unpack.py document.docx unpacked/ + python unpack.py presentation.pptx unpacked/ + python unpack.py document.docx unpacked/ --merge-runs false +""" + +import argparse +import sys +import zipfile +from pathlib import Path + +import defusedxml.minidom + +from helpers.merge_runs import merge_runs as do_merge_runs +from helpers.simplify_redlines import simplify_redlines as do_simplify_redlines + +SMART_QUOTE_REPLACEMENTS = { + "\u201c": "“", + "\u201d": "”", + "\u2018": "‘", + "\u2019": "’", +} + + +def unpack( + input_file: str, + output_directory: str, + merge_runs: bool = True, + simplify_redlines: bool = True, +) -> tuple[None, str]: + input_path = Path(input_file) + output_path = Path(output_directory) + suffix = input_path.suffix.lower() + + if not input_path.exists(): + return None, f"Error: {input_file} does not exist" + + if suffix not in {".docx", ".pptx", ".xlsx"}: + return None, f"Error: {input_file} must be a .docx, .pptx, or .xlsx file" + + try: + output_path.mkdir(parents=True, exist_ok=True) + + with zipfile.ZipFile(input_path, "r") as zf: + zf.extractall(output_path) + + xml_files = list(output_path.rglob("*.xml")) + list(output_path.rglob("*.rels")) + for xml_file in xml_files: + _pretty_print_xml(xml_file) + + message = f"Unpacked {input_file} ({len(xml_files)} XML files)" + + if suffix == ".docx": + if simplify_redlines: + simplify_count, _ = do_simplify_redlines(str(output_path)) + message += f", simplified {simplify_count} tracked changes" + + if merge_runs: + merge_count, _ = do_merge_runs(str(output_path)) + message += f", merged {merge_count} runs" + + for xml_file in xml_files: + _escape_smart_quotes(xml_file) + + return None, message + + except zipfile.BadZipFile: + return None, f"Error: {input_file} is not a valid Office file" + except Exception as e: + return None, f"Error unpacking: {e}" + + +def _pretty_print_xml(xml_file: Path) -> None: + try: + content = xml_file.read_text(encoding="utf-8") + dom = defusedxml.minidom.parseString(content) + xml_file.write_bytes(dom.toprettyxml(indent=" ", encoding="utf-8")) + except Exception: + pass + + +def _escape_smart_quotes(xml_file: Path) -> None: + try: + content = xml_file.read_text(encoding="utf-8") + for char, entity in SMART_QUOTE_REPLACEMENTS.items(): + content = content.replace(char, entity) + xml_file.write_text(content, encoding="utf-8") + except Exception: + pass + + +if __name__ == "__main__": + parser = argparse.ArgumentParser( + description="Unpack an Office file (DOCX, PPTX, XLSX) for editing" + ) + parser.add_argument("input_file", help="Office file to unpack") + parser.add_argument("output_directory", help="Output directory") + parser.add_argument( + "--merge-runs", + type=lambda x: x.lower() == "true", + default=True, + metavar="true|false", + help="Merge adjacent runs with identical formatting (DOCX only, default: true)", + ) + parser.add_argument( + "--simplify-redlines", + type=lambda x: x.lower() == "true", + default=True, + metavar="true|false", + help="Merge adjacent tracked changes from same author (DOCX only, default: true)", + ) + args = parser.parse_args() + + _, message = unpack( + args.input_file, + args.output_directory, + merge_runs=args.merge_runs, + simplify_redlines=args.simplify_redlines, + ) + print(message) + + if "Error" in message: + sys.exit(1) diff --git a/skills/docx/scripts/office/validate.py b/skills/docx/scripts/office/validate.py new file mode 100644 index 00000000..03b01f6e --- /dev/null +++ b/skills/docx/scripts/office/validate.py @@ -0,0 +1,111 @@ +""" +Command line tool to validate Office document XML files against XSD schemas and tracked changes. + +Usage: + python validate.py [--original ] [--auto-repair] [--author NAME] + +The first argument can be either: +- An unpacked directory containing the Office document XML files +- A packed Office file (.docx/.pptx/.xlsx) which will be unpacked to a temp directory + +Auto-repair fixes: +- paraId/durableId values that exceed OOXML limits +- Missing xml:space="preserve" on w:t elements with whitespace +""" + +import argparse +import sys +import tempfile +import zipfile +from pathlib import Path + +from validators import DOCXSchemaValidator, PPTXSchemaValidator, RedliningValidator + + +def main(): + parser = argparse.ArgumentParser(description="Validate Office document XML files") + parser.add_argument( + "path", + help="Path to unpacked directory or packed Office file (.docx/.pptx/.xlsx)", + ) + parser.add_argument( + "--original", + required=False, + default=None, + help="Path to original file (.docx/.pptx/.xlsx). If omitted, all XSD errors are reported and redlining validation is skipped.", + ) + parser.add_argument( + "-v", + "--verbose", + action="store_true", + help="Enable verbose output", + ) + parser.add_argument( + "--auto-repair", + action="store_true", + help="Automatically repair common issues (hex IDs, whitespace preservation)", + ) + parser.add_argument( + "--author", + default="Claude", + help="Author name for redlining validation (default: Claude)", + ) + args = parser.parse_args() + + path = Path(args.path) + assert path.exists(), f"Error: {path} does not exist" + + original_file = None + if args.original: + original_file = Path(args.original) + assert original_file.is_file(), f"Error: {original_file} is not a file" + assert original_file.suffix.lower() in [".docx", ".pptx", ".xlsx"], ( + f"Error: {original_file} must be a .docx, .pptx, or .xlsx file" + ) + + file_extension = (original_file or path).suffix.lower() + assert file_extension in [".docx", ".pptx", ".xlsx"], ( + f"Error: Cannot determine file type from {path}. Use --original or provide a .docx/.pptx/.xlsx file." + ) + + if path.is_file() and path.suffix.lower() in [".docx", ".pptx", ".xlsx"]: + temp_dir = tempfile.mkdtemp() + with zipfile.ZipFile(path, "r") as zf: + zf.extractall(temp_dir) + unpacked_dir = Path(temp_dir) + else: + assert path.is_dir(), f"Error: {path} is not a directory or Office file" + unpacked_dir = path + + match file_extension: + case ".docx": + validators = [ + DOCXSchemaValidator(unpacked_dir, original_file, verbose=args.verbose), + ] + if original_file: + validators.append( + RedliningValidator(unpacked_dir, original_file, verbose=args.verbose, author=args.author) + ) + case ".pptx": + validators = [ + PPTXSchemaValidator(unpacked_dir, original_file, verbose=args.verbose), + ] + case _: + print(f"Error: Validation not supported for file type {file_extension}") + sys.exit(1) + + if args.auto_repair: + total_repairs = sum(v.repair() for v in validators) + if total_repairs: + print(f"Auto-repaired {total_repairs} issue(s)") + + success = all(v.validate() for v in validators) + + if success: + print("All validations PASSED!") + + sys.exit(0 if success else 1) + + +if __name__ == "__main__": + main() diff --git a/skills/docx/scripts/office/validators/__init__.py b/skills/docx/scripts/office/validators/__init__.py new file mode 100644 index 00000000..db092ece --- /dev/null +++ b/skills/docx/scripts/office/validators/__init__.py @@ -0,0 +1,15 @@ +""" +Validation modules for Word document processing. +""" + +from .base import BaseSchemaValidator +from .docx import DOCXSchemaValidator +from .pptx import PPTXSchemaValidator +from .redlining import RedliningValidator + +__all__ = [ + "BaseSchemaValidator", + "DOCXSchemaValidator", + "PPTXSchemaValidator", + "RedliningValidator", +] diff --git a/skills/docx/scripts/office/validators/base.py b/skills/docx/scripts/office/validators/base.py new file mode 100644 index 00000000..875de691 --- /dev/null +++ b/skills/docx/scripts/office/validators/base.py @@ -0,0 +1,851 @@ +""" +Base validator with common validation logic for document files. +""" + +import re +from pathlib import Path + +import defusedxml.minidom +import lxml.etree + + +class BaseSchemaValidator: + + IGNORED_VALIDATION_ERRORS = [ + "hyphenationZone", + "purl.org/dc/terms", + ] + + UNIQUE_ID_REQUIREMENTS = { + "comment": ("id", "file"), + "commentrangestart": ("id", "file"), + "commentrangeend": ("id", "file"), + "bookmarkstart": ("id", "file"), + "bookmarkend": ("id", "file"), + "sldid": ("id", "file"), + "sldmasterid": ("id", "global"), + "sldlayoutid": ("id", "global"), + "cm": ("authorid", "file"), + "sheet": ("sheetid", "file"), + "definedname": ("id", "file"), + "cxnsp": ("id", "file"), + "sp": ("id", "file"), + "pic": ("id", "file"), + "grpsp": ("id", "file"), + } + + EXCLUDED_ID_CONTAINERS = { + "sectionlst", + } + + ELEMENT_RELATIONSHIP_TYPES = {} + + SCHEMA_MAPPINGS = { + "word": "ISO-IEC29500-4_2016/wml.xsd", + "ppt": "ISO-IEC29500-4_2016/pml.xsd", + "xl": "ISO-IEC29500-4_2016/sml.xsd", + "[Content_Types].xml": "ecma/fouth-edition/opc-contentTypes.xsd", + "app.xml": "ISO-IEC29500-4_2016/shared-documentPropertiesExtended.xsd", + "core.xml": "ecma/fouth-edition/opc-coreProperties.xsd", + "custom.xml": "ISO-IEC29500-4_2016/shared-documentPropertiesCustom.xsd", + ".rels": "ecma/fouth-edition/opc-relationships.xsd", + "people.xml": "microsoft/wml-2012.xsd", + "commentsIds.xml": "microsoft/wml-cid-2016.xsd", + "commentsExtensible.xml": "microsoft/wml-cex-2018.xsd", + "commentsExtended.xml": "microsoft/wml-2012.xsd", + "chart": "ISO-IEC29500-4_2016/dml-chart.xsd", + "theme": "ISO-IEC29500-4_2016/dml-main.xsd", + "drawing": "ISO-IEC29500-4_2016/dml-main.xsd", + } + + MC_NAMESPACE = "http://schemas.openxmlformats.org/markup-compatibility/2006" + XML_NAMESPACE = "http://www.w3.org/XML/1998/namespace" + + PACKAGE_RELATIONSHIPS_NAMESPACE = ( + "http://schemas.openxmlformats.org/package/2006/relationships" + ) + OFFICE_RELATIONSHIPS_NAMESPACE = ( + "http://schemas.openxmlformats.org/officeDocument/2006/relationships" + ) + CONTENT_TYPES_NAMESPACE = ( + "http://schemas.openxmlformats.org/package/2006/content-types" + ) + + MAIN_CONTENT_FOLDERS = {"word", "ppt", "xl"} + + OOXML_NAMESPACES = { + "http://schemas.openxmlformats.org/officeDocument/2006/math", + "http://schemas.openxmlformats.org/officeDocument/2006/relationships", + "http://schemas.openxmlformats.org/schemaLibrary/2006/main", + "http://schemas.openxmlformats.org/drawingml/2006/main", + "http://schemas.openxmlformats.org/drawingml/2006/chart", + "http://schemas.openxmlformats.org/drawingml/2006/chartDrawing", + "http://schemas.openxmlformats.org/drawingml/2006/diagram", + "http://schemas.openxmlformats.org/drawingml/2006/picture", + "http://schemas.openxmlformats.org/drawingml/2006/spreadsheetDrawing", + "http://schemas.openxmlformats.org/drawingml/2006/wordprocessingDrawing", + "http://schemas.openxmlformats.org/wordprocessingml/2006/main", + "http://schemas.openxmlformats.org/presentationml/2006/main", + "http://schemas.openxmlformats.org/spreadsheetml/2006/main", + "http://schemas.openxmlformats.org/officeDocument/2006/sharedTypes", + "http://www.w3.org/XML/1998/namespace", + } + + def __init__(self, unpacked_dir, original_file=None, verbose=False): + self.unpacked_dir = Path(unpacked_dir).resolve() + self.original_file = Path(original_file) if original_file else None + self.verbose = verbose + + self.schemas_dir = Path(__file__).parent.parent / "schemas" + + patterns = ["*.xml", "*.rels"] + self.xml_files = [ + f for pattern in patterns for f in self.unpacked_dir.rglob(pattern) + ] + + if not self.xml_files: + print(f"Warning: No XML files found in {self.unpacked_dir}") + + def validate(self): + raise NotImplementedError("Subclasses must implement the validate method") + + def repair(self) -> int: + return self.repair_whitespace_preservation() + + def repair_whitespace_preservation(self) -> int: + repairs = 0 + + for xml_file in self.xml_files: + try: + content = xml_file.read_text(encoding="utf-8") + dom = defusedxml.minidom.parseString(content) + modified = False + + for elem in dom.getElementsByTagName("*"): + if elem.tagName.endswith(":t") and elem.firstChild: + text = elem.firstChild.nodeValue + if text and (text.startswith((' ', '\t')) or text.endswith((' ', '\t'))): + if elem.getAttribute("xml:space") != "preserve": + elem.setAttribute("xml:space", "preserve") + text_preview = repr(text[:30]) + "..." if len(text) > 30 else repr(text) + print(f" Repaired: {xml_file.name}: Added xml:space='preserve' to {elem.tagName}: {text_preview}") + repairs += 1 + modified = True + + if modified: + xml_file.write_bytes(dom.toxml(encoding="UTF-8")) + + except Exception: + pass + + return repairs + + def validate_xml(self): + errors = [] + + for xml_file in self.xml_files: + try: + lxml.etree.parse(str(xml_file)) + except lxml.etree.XMLSyntaxError as e: + errors.append( + f" {xml_file.relative_to(self.unpacked_dir)}: " + f"Line {e.lineno}: {e.msg}" + ) + except Exception as e: + errors.append( + f" {xml_file.relative_to(self.unpacked_dir)}: " + f"Unexpected error: {str(e)}" + ) + + if errors: + print(f"FAILED - Found {len(errors)} XML violations:") + for error in errors: + print(error) + return False + else: + if self.verbose: + print("PASSED - All XML files are well-formed") + return True + + def validate_namespaces(self): + errors = [] + + for xml_file in self.xml_files: + try: + root = lxml.etree.parse(str(xml_file)).getroot() + declared = set(root.nsmap.keys()) - {None} + + for attr_val in [ + v for k, v in root.attrib.items() if k.endswith("Ignorable") + ]: + undeclared = set(attr_val.split()) - declared + errors.extend( + f" {xml_file.relative_to(self.unpacked_dir)}: " + f"Namespace '{ns}' in Ignorable but not declared" + for ns in undeclared + ) + except lxml.etree.XMLSyntaxError: + continue + + if errors: + print(f"FAILED - {len(errors)} namespace issues:") + for error in errors: + print(error) + return False + if self.verbose: + print("PASSED - All namespace prefixes properly declared") + return True + + def validate_unique_ids(self): + errors = [] + global_ids = {} + + for xml_file in self.xml_files: + try: + root = lxml.etree.parse(str(xml_file)).getroot() + file_ids = {} + + mc_elements = root.xpath( + ".//mc:AlternateContent", namespaces={"mc": self.MC_NAMESPACE} + ) + for elem in mc_elements: + elem.getparent().remove(elem) + + for elem in root.iter(): + if not hasattr(elem, "tag") or callable(elem.tag): + continue + tag = ( + elem.tag.split("}")[-1].lower() + if "}" in elem.tag + else elem.tag.lower() + ) + + if tag in self.UNIQUE_ID_REQUIREMENTS: + in_excluded_container = any( + ancestor.tag.split("}")[-1].lower() in self.EXCLUDED_ID_CONTAINERS + for ancestor in elem.iterancestors() + ) + if in_excluded_container: + continue + + attr_name, scope = self.UNIQUE_ID_REQUIREMENTS[tag] + + id_value = None + for attr, value in elem.attrib.items(): + attr_local = ( + attr.split("}")[-1].lower() + if "}" in attr + else attr.lower() + ) + if attr_local == attr_name: + id_value = value + break + + if id_value is not None: + if scope == "global": + if id_value in global_ids: + prev_file, prev_line, prev_tag = global_ids[ + id_value + ] + errors.append( + f" {xml_file.relative_to(self.unpacked_dir)}: " + f"Line {elem.sourceline}: Global ID '{id_value}' in <{tag}> " + f"already used in {prev_file} at line {prev_line} in <{prev_tag}>" + ) + else: + global_ids[id_value] = ( + xml_file.relative_to(self.unpacked_dir), + elem.sourceline, + tag, + ) + elif scope == "file": + key = (tag, attr_name) + if key not in file_ids: + file_ids[key] = {} + + if id_value in file_ids[key]: + prev_line = file_ids[key][id_value] + errors.append( + f" {xml_file.relative_to(self.unpacked_dir)}: " + f"Line {elem.sourceline}: Duplicate {attr_name}='{id_value}' in <{tag}> " + f"(first occurrence at line {prev_line})" + ) + else: + file_ids[key][id_value] = elem.sourceline + + except (lxml.etree.XMLSyntaxError, Exception) as e: + errors.append( + f" {xml_file.relative_to(self.unpacked_dir)}: Error: {e}" + ) + + if errors: + print(f"FAILED - Found {len(errors)} ID uniqueness violations:") + for error in errors: + print(error) + return False + else: + if self.verbose: + print("PASSED - All required IDs are unique") + return True + + def validate_file_references(self): + errors = [] + + rels_files = list(self.unpacked_dir.rglob("*.rels")) + + if not rels_files: + if self.verbose: + print("PASSED - No .rels files found") + return True + + all_files = [] + for file_path in self.unpacked_dir.rglob("*"): + if ( + file_path.is_file() + and file_path.name != "[Content_Types].xml" + and not file_path.name.endswith(".rels") + ): + all_files.append(file_path.resolve()) + + all_referenced_files = set() + + if self.verbose: + print( + f"Found {len(rels_files)} .rels files and {len(all_files)} target files" + ) + + for rels_file in rels_files: + try: + rels_root = lxml.etree.parse(str(rels_file)).getroot() + + rels_dir = rels_file.parent + + referenced_files = set() + broken_refs = [] + + for rel in rels_root.findall( + ".//ns:Relationship", + namespaces={"ns": self.PACKAGE_RELATIONSHIPS_NAMESPACE}, + ): + target = rel.get("Target") + if target and not target.startswith( + ("http", "mailto:") + ): + if target.startswith("/"): + target_path = self.unpacked_dir / target.lstrip("/") + elif rels_file.name == ".rels": + target_path = self.unpacked_dir / target + else: + base_dir = rels_dir.parent + target_path = base_dir / target + + try: + target_path = target_path.resolve() + if target_path.exists() and target_path.is_file(): + referenced_files.add(target_path) + all_referenced_files.add(target_path) + else: + broken_refs.append((target, rel.sourceline)) + except (OSError, ValueError): + broken_refs.append((target, rel.sourceline)) + + if broken_refs: + rel_path = rels_file.relative_to(self.unpacked_dir) + for broken_ref, line_num in broken_refs: + errors.append( + f" {rel_path}: Line {line_num}: Broken reference to {broken_ref}" + ) + + except Exception as e: + rel_path = rels_file.relative_to(self.unpacked_dir) + errors.append(f" Error parsing {rel_path}: {e}") + + unreferenced_files = set(all_files) - all_referenced_files + + if unreferenced_files: + for unref_file in sorted(unreferenced_files): + unref_rel_path = unref_file.relative_to(self.unpacked_dir) + errors.append(f" Unreferenced file: {unref_rel_path}") + + if errors: + print(f"FAILED - Found {len(errors)} relationship validation errors:") + for error in errors: + print(error) + print( + "CRITICAL: These errors will cause the document to appear corrupt. " + + "Broken references MUST be fixed, " + + "and unreferenced files MUST be referenced or removed." + ) + return False + else: + if self.verbose: + print( + "PASSED - All references are valid and all files are properly referenced" + ) + return True + + def validate_all_relationship_ids(self): + import lxml.etree + + errors = [] + + for xml_file in self.xml_files: + if xml_file.suffix == ".rels": + continue + + rels_dir = xml_file.parent / "_rels" + rels_file = rels_dir / f"{xml_file.name}.rels" + + if not rels_file.exists(): + continue + + try: + rels_root = lxml.etree.parse(str(rels_file)).getroot() + rid_to_type = {} + + for rel in rels_root.findall( + f".//{{{self.PACKAGE_RELATIONSHIPS_NAMESPACE}}}Relationship" + ): + rid = rel.get("Id") + rel_type = rel.get("Type", "") + if rid: + if rid in rid_to_type: + rels_rel_path = rels_file.relative_to(self.unpacked_dir) + errors.append( + f" {rels_rel_path}: Line {rel.sourceline}: " + f"Duplicate relationship ID '{rid}' (IDs must be unique)" + ) + type_name = ( + rel_type.split("/")[-1] if "/" in rel_type else rel_type + ) + rid_to_type[rid] = type_name + + xml_root = lxml.etree.parse(str(xml_file)).getroot() + + r_ns = self.OFFICE_RELATIONSHIPS_NAMESPACE + rid_attrs_to_check = ["id", "embed", "link"] + for elem in xml_root.iter(): + if not hasattr(elem, "tag") or callable(elem.tag): + continue + for attr_name in rid_attrs_to_check: + rid_attr = elem.get(f"{{{r_ns}}}{attr_name}") + if not rid_attr: + continue + xml_rel_path = xml_file.relative_to(self.unpacked_dir) + elem_name = ( + elem.tag.split("}")[-1] if "}" in elem.tag else elem.tag + ) + + if rid_attr not in rid_to_type: + errors.append( + f" {xml_rel_path}: Line {elem.sourceline}: " + f"<{elem_name}> r:{attr_name} references non-existent relationship '{rid_attr}' " + f"(valid IDs: {', '.join(sorted(rid_to_type.keys())[:5])}{'...' if len(rid_to_type) > 5 else ''})" + ) + elif attr_name == "id" and self.ELEMENT_RELATIONSHIP_TYPES: + expected_type = self._get_expected_relationship_type( + elem_name + ) + if expected_type: + actual_type = rid_to_type[rid_attr] + if expected_type not in actual_type.lower(): + errors.append( + f" {xml_rel_path}: Line {elem.sourceline}: " + f"<{elem_name}> references '{rid_attr}' which points to '{actual_type}' " + f"but should point to a '{expected_type}' relationship" + ) + + except Exception as e: + xml_rel_path = xml_file.relative_to(self.unpacked_dir) + errors.append(f" Error processing {xml_rel_path}: {e}") + + if errors: + print(f"FAILED - Found {len(errors)} relationship ID reference errors:") + for error in errors: + print(error) + print("\nThese ID mismatches will cause the document to appear corrupt!") + return False + else: + if self.verbose: + print("PASSED - All relationship ID references are valid") + return True + + def _get_expected_relationship_type(self, element_name): + elem_lower = element_name.lower() + + if elem_lower in self.ELEMENT_RELATIONSHIP_TYPES: + return self.ELEMENT_RELATIONSHIP_TYPES[elem_lower] + + if elem_lower.endswith("id") and len(elem_lower) > 2: + prefix = elem_lower[:-2] + if prefix.endswith("master"): + return prefix.lower() + elif prefix.endswith("layout"): + return prefix.lower() + else: + if prefix == "sld": + return "slide" + return prefix.lower() + + if elem_lower.endswith("reference") and len(elem_lower) > 9: + prefix = elem_lower[:-9] + return prefix.lower() + + return None + + def validate_content_types(self): + errors = [] + + content_types_file = self.unpacked_dir / "[Content_Types].xml" + if not content_types_file.exists(): + print("FAILED - [Content_Types].xml file not found") + return False + + try: + root = lxml.etree.parse(str(content_types_file)).getroot() + declared_parts = set() + declared_extensions = set() + + for override in root.findall( + f".//{{{self.CONTENT_TYPES_NAMESPACE}}}Override" + ): + part_name = override.get("PartName") + if part_name is not None: + declared_parts.add(part_name.lstrip("/")) + + for default in root.findall( + f".//{{{self.CONTENT_TYPES_NAMESPACE}}}Default" + ): + extension = default.get("Extension") + if extension is not None: + declared_extensions.add(extension.lower()) + + declarable_roots = { + "sld", + "sldLayout", + "sldMaster", + "presentation", + "document", + "workbook", + "worksheet", + "theme", + } + + media_extensions = { + "png": "image/png", + "jpg": "image/jpeg", + "jpeg": "image/jpeg", + "gif": "image/gif", + "bmp": "image/bmp", + "tiff": "image/tiff", + "wmf": "image/x-wmf", + "emf": "image/x-emf", + } + + all_files = list(self.unpacked_dir.rglob("*")) + all_files = [f for f in all_files if f.is_file()] + + for xml_file in self.xml_files: + path_str = str(xml_file.relative_to(self.unpacked_dir)).replace( + "\\", "/" + ) + + if any( + skip in path_str + for skip in [".rels", "[Content_Types]", "docProps/", "_rels/"] + ): + continue + + try: + root_tag = lxml.etree.parse(str(xml_file)).getroot().tag + root_name = root_tag.split("}")[-1] if "}" in root_tag else root_tag + + if root_name in declarable_roots and path_str not in declared_parts: + errors.append( + f" {path_str}: File with <{root_name}> root not declared in [Content_Types].xml" + ) + + except Exception: + continue + + for file_path in all_files: + if file_path.suffix.lower() in {".xml", ".rels"}: + continue + if file_path.name == "[Content_Types].xml": + continue + if "_rels" in file_path.parts or "docProps" in file_path.parts: + continue + + extension = file_path.suffix.lstrip(".").lower() + if extension and extension not in declared_extensions: + if extension in media_extensions: + relative_path = file_path.relative_to(self.unpacked_dir) + errors.append( + f' {relative_path}: File with extension \'{extension}\' not declared in [Content_Types].xml - should add: ' + ) + + except Exception as e: + errors.append(f" Error parsing [Content_Types].xml: {e}") + + if errors: + print(f"FAILED - Found {len(errors)} content type declaration errors:") + for error in errors: + print(error) + return False + else: + if self.verbose: + print( + "PASSED - All content files are properly declared in [Content_Types].xml" + ) + return True + + def validate_file_against_xsd(self, xml_file, verbose=False): + xml_file = Path(xml_file).resolve() + unpacked_dir = self.unpacked_dir.resolve() + + is_valid, current_errors = self._validate_single_file_xsd( + xml_file, unpacked_dir + ) + + if is_valid is None: + return None, set() + elif is_valid: + return True, set() + + original_errors = self._get_original_file_errors(xml_file) + + assert current_errors is not None + new_errors = current_errors - original_errors + + new_errors = { + e for e in new_errors + if not any(pattern in e for pattern in self.IGNORED_VALIDATION_ERRORS) + } + + if new_errors: + if verbose: + relative_path = xml_file.relative_to(unpacked_dir) + print(f"FAILED - {relative_path}: {len(new_errors)} new error(s)") + for error in list(new_errors)[:3]: + truncated = error[:250] + "..." if len(error) > 250 else error + print(f" - {truncated}") + return False, new_errors + else: + if verbose: + print( + f"PASSED - No new errors (original had {len(current_errors)} errors)" + ) + return True, set() + + def validate_against_xsd(self): + new_errors = [] + original_error_count = 0 + valid_count = 0 + skipped_count = 0 + + for xml_file in self.xml_files: + relative_path = str(xml_file.relative_to(self.unpacked_dir)) + is_valid, new_file_errors = self.validate_file_against_xsd( + xml_file, verbose=False + ) + + if is_valid is None: + skipped_count += 1 + continue + elif is_valid and not new_file_errors: + valid_count += 1 + continue + elif is_valid: + original_error_count += 1 + valid_count += 1 + continue + + new_errors.append(f" {relative_path}: {len(new_file_errors)} new error(s)") + for error in list(new_file_errors)[:3]: + new_errors.append( + f" - {error[:250]}..." if len(error) > 250 else f" - {error}" + ) + + if self.verbose: + print(f"Validated {len(self.xml_files)} files:") + print(f" - Valid: {valid_count}") + print(f" - Skipped (no schema): {skipped_count}") + if original_error_count: + print(f" - With original errors (ignored): {original_error_count}") + print( + f" - With NEW errors: {len(new_errors) > 0 and len([e for e in new_errors if not e.startswith(' ')]) or 0}" + ) + + if new_errors: + print("\nFAILED - Found NEW validation errors:") + for error in new_errors: + print(error) + return False + else: + if self.verbose: + print("\nPASSED - No new XSD validation errors introduced") + return True + + def _get_schema_path(self, xml_file): + if xml_file.name in self.SCHEMA_MAPPINGS: + return self.schemas_dir / self.SCHEMA_MAPPINGS[xml_file.name] + + if xml_file.suffix == ".rels": + return self.schemas_dir / self.SCHEMA_MAPPINGS[".rels"] + + if "charts/" in str(xml_file) and xml_file.name.startswith("chart"): + return self.schemas_dir / self.SCHEMA_MAPPINGS["chart"] + + if "theme/" in str(xml_file) and xml_file.name.startswith("theme"): + return self.schemas_dir / self.SCHEMA_MAPPINGS["theme"] + + if xml_file.parent.name in self.MAIN_CONTENT_FOLDERS: + return self.schemas_dir / self.SCHEMA_MAPPINGS[xml_file.parent.name] + + return None + + def _clean_ignorable_namespaces(self, xml_doc): + xml_string = lxml.etree.tostring(xml_doc, encoding="unicode") + xml_copy = lxml.etree.fromstring(xml_string) + + for elem in xml_copy.iter(): + attrs_to_remove = [] + + for attr in elem.attrib: + if "{" in attr: + ns = attr.split("}")[0][1:] + if ns not in self.OOXML_NAMESPACES: + attrs_to_remove.append(attr) + + for attr in attrs_to_remove: + del elem.attrib[attr] + + self._remove_ignorable_elements(xml_copy) + + return lxml.etree.ElementTree(xml_copy) + + def _remove_ignorable_elements(self, root): + elements_to_remove = [] + + for elem in list(root): + if not hasattr(elem, "tag") or callable(elem.tag): + continue + + tag_str = str(elem.tag) + if tag_str.startswith("{"): + ns = tag_str.split("}")[0][1:] + if ns not in self.OOXML_NAMESPACES: + elements_to_remove.append(elem) + continue + + self._remove_ignorable_elements(elem) + + for elem in elements_to_remove: + root.remove(elem) + + def _preprocess_for_mc_ignorable(self, xml_doc): + root = xml_doc.getroot() + + if f"{{{self.MC_NAMESPACE}}}Ignorable" in root.attrib: + del root.attrib[f"{{{self.MC_NAMESPACE}}}Ignorable"] + + return xml_doc + + def _validate_single_file_xsd(self, xml_file, base_path): + schema_path = self._get_schema_path(xml_file) + if not schema_path: + return None, None + + try: + with open(schema_path, "rb") as xsd_file: + parser = lxml.etree.XMLParser() + xsd_doc = lxml.etree.parse( + xsd_file, parser=parser, base_url=str(schema_path) + ) + schema = lxml.etree.XMLSchema(xsd_doc) + + with open(xml_file, "r") as f: + xml_doc = lxml.etree.parse(f) + + xml_doc, _ = self._remove_template_tags_from_text_nodes(xml_doc) + xml_doc = self._preprocess_for_mc_ignorable(xml_doc) + + relative_path = xml_file.relative_to(base_path) + if ( + relative_path.parts + and relative_path.parts[0] in self.MAIN_CONTENT_FOLDERS + ): + xml_doc = self._clean_ignorable_namespaces(xml_doc) + + if schema.validate(xml_doc): + return True, set() + else: + errors = set() + for error in schema.error_log: + errors.add(error.message) + return False, errors + + except Exception as e: + return False, {str(e)} + + def _get_original_file_errors(self, xml_file): + if self.original_file is None: + return set() + + import tempfile + import zipfile + + xml_file = Path(xml_file).resolve() + unpacked_dir = self.unpacked_dir.resolve() + relative_path = xml_file.relative_to(unpacked_dir) + + with tempfile.TemporaryDirectory() as temp_dir: + temp_path = Path(temp_dir) + + with zipfile.ZipFile(self.original_file, "r") as zip_ref: + zip_ref.extractall(temp_path) + + original_xml_file = temp_path / relative_path + + if not original_xml_file.exists(): + return set() + + is_valid, errors = self._validate_single_file_xsd( + original_xml_file, temp_path + ) + return errors if errors else set() + + def _remove_template_tags_from_text_nodes(self, xml_doc): + warnings = [] + template_pattern = re.compile(r"\{\{[^}]*\}\}") + + xml_string = lxml.etree.tostring(xml_doc, encoding="unicode") + xml_copy = lxml.etree.fromstring(xml_string) + + def process_text_content(text, content_type): + if not text: + return text + matches = list(template_pattern.finditer(text)) + if matches: + for match in matches: + warnings.append( + f"Found template tag in {content_type}: {match.group()}" + ) + return template_pattern.sub("", text) + return text + + for elem in xml_copy.iter(): + if not hasattr(elem, "tag") or callable(elem.tag): + continue + tag_str = str(elem.tag) + if tag_str.endswith("}t") or tag_str == "t": + continue + + elem.text = process_text_content(elem.text, "text content") + elem.tail = process_text_content(elem.tail, "tail content") + + return lxml.etree.ElementTree(xml_copy), warnings + + +if __name__ == "__main__": + raise RuntimeError("This module should not be run directly.") diff --git a/skills/docx/scripts/office/validators/docx.py b/skills/docx/scripts/office/validators/docx.py new file mode 100644 index 00000000..fec405e6 --- /dev/null +++ b/skills/docx/scripts/office/validators/docx.py @@ -0,0 +1,446 @@ +""" +Validator for Word document XML files against XSD schemas. +""" + +import random +import re +import tempfile +import zipfile + +import defusedxml.minidom +import lxml.etree + +from .base import BaseSchemaValidator + + +class DOCXSchemaValidator(BaseSchemaValidator): + + WORD_2006_NAMESPACE = "http://schemas.openxmlformats.org/wordprocessingml/2006/main" + W14_NAMESPACE = "http://schemas.microsoft.com/office/word/2010/wordml" + W16CID_NAMESPACE = "http://schemas.microsoft.com/office/word/2016/wordml/cid" + + ELEMENT_RELATIONSHIP_TYPES = {} + + def validate(self): + if not self.validate_xml(): + return False + + all_valid = True + if not self.validate_namespaces(): + all_valid = False + + if not self.validate_unique_ids(): + all_valid = False + + if not self.validate_file_references(): + all_valid = False + + if not self.validate_content_types(): + all_valid = False + + if not self.validate_against_xsd(): + all_valid = False + + if not self.validate_whitespace_preservation(): + all_valid = False + + if not self.validate_deletions(): + all_valid = False + + if not self.validate_insertions(): + all_valid = False + + if not self.validate_all_relationship_ids(): + all_valid = False + + if not self.validate_id_constraints(): + all_valid = False + + if not self.validate_comment_markers(): + all_valid = False + + self.compare_paragraph_counts() + + return all_valid + + def validate_whitespace_preservation(self): + errors = [] + + for xml_file in self.xml_files: + if xml_file.name != "document.xml": + continue + + try: + root = lxml.etree.parse(str(xml_file)).getroot() + + for elem in root.iter(f"{{{self.WORD_2006_NAMESPACE}}}t"): + if elem.text: + text = elem.text + if re.search(r"^[ \t\n\r]", text) or re.search( + r"[ \t\n\r]$", text + ): + xml_space_attr = f"{{{self.XML_NAMESPACE}}}space" + if ( + xml_space_attr not in elem.attrib + or elem.attrib[xml_space_attr] != "preserve" + ): + text_preview = ( + repr(text)[:50] + "..." + if len(repr(text)) > 50 + else repr(text) + ) + errors.append( + f" {xml_file.relative_to(self.unpacked_dir)}: " + f"Line {elem.sourceline}: w:t element with whitespace missing xml:space='preserve': {text_preview}" + ) + + except (lxml.etree.XMLSyntaxError, Exception) as e: + errors.append( + f" {xml_file.relative_to(self.unpacked_dir)}: Error: {e}" + ) + + if errors: + print(f"FAILED - Found {len(errors)} whitespace preservation violations:") + for error in errors: + print(error) + return False + else: + if self.verbose: + print("PASSED - All whitespace is properly preserved") + return True + + def validate_deletions(self): + errors = [] + + for xml_file in self.xml_files: + if xml_file.name != "document.xml": + continue + + try: + root = lxml.etree.parse(str(xml_file)).getroot() + namespaces = {"w": self.WORD_2006_NAMESPACE} + + for t_elem in root.xpath(".//w:del//w:t", namespaces=namespaces): + if t_elem.text: + text_preview = ( + repr(t_elem.text)[:50] + "..." + if len(repr(t_elem.text)) > 50 + else repr(t_elem.text) + ) + errors.append( + f" {xml_file.relative_to(self.unpacked_dir)}: " + f"Line {t_elem.sourceline}: found within : {text_preview}" + ) + + for instr_elem in root.xpath( + ".//w:del//w:instrText", namespaces=namespaces + ): + text_preview = ( + repr(instr_elem.text or "")[:50] + "..." + if len(repr(instr_elem.text or "")) > 50 + else repr(instr_elem.text or "") + ) + errors.append( + f" {xml_file.relative_to(self.unpacked_dir)}: " + f"Line {instr_elem.sourceline}: found within (use ): {text_preview}" + ) + + except (lxml.etree.XMLSyntaxError, Exception) as e: + errors.append( + f" {xml_file.relative_to(self.unpacked_dir)}: Error: {e}" + ) + + if errors: + print(f"FAILED - Found {len(errors)} deletion validation violations:") + for error in errors: + print(error) + return False + else: + if self.verbose: + print("PASSED - No w:t elements found within w:del elements") + return True + + def count_paragraphs_in_unpacked(self): + count = 0 + + for xml_file in self.xml_files: + if xml_file.name != "document.xml": + continue + + try: + root = lxml.etree.parse(str(xml_file)).getroot() + paragraphs = root.findall(f".//{{{self.WORD_2006_NAMESPACE}}}p") + count = len(paragraphs) + except Exception as e: + print(f"Error counting paragraphs in unpacked document: {e}") + + return count + + def count_paragraphs_in_original(self): + original = self.original_file + if original is None: + return 0 + + count = 0 + + try: + with tempfile.TemporaryDirectory() as temp_dir: + with zipfile.ZipFile(original, "r") as zip_ref: + zip_ref.extractall(temp_dir) + + doc_xml_path = temp_dir + "/word/document.xml" + root = lxml.etree.parse(doc_xml_path).getroot() + + paragraphs = root.findall(f".//{{{self.WORD_2006_NAMESPACE}}}p") + count = len(paragraphs) + + except Exception as e: + print(f"Error counting paragraphs in original document: {e}") + + return count + + def validate_insertions(self): + errors = [] + + for xml_file in self.xml_files: + if xml_file.name != "document.xml": + continue + + try: + root = lxml.etree.parse(str(xml_file)).getroot() + namespaces = {"w": self.WORD_2006_NAMESPACE} + + invalid_elements = root.xpath( + ".//w:ins//w:delText[not(ancestor::w:del)]", namespaces=namespaces + ) + + for elem in invalid_elements: + text_preview = ( + repr(elem.text or "")[:50] + "..." + if len(repr(elem.text or "")) > 50 + else repr(elem.text or "") + ) + errors.append( + f" {xml_file.relative_to(self.unpacked_dir)}: " + f"Line {elem.sourceline}: within : {text_preview}" + ) + + except (lxml.etree.XMLSyntaxError, Exception) as e: + errors.append( + f" {xml_file.relative_to(self.unpacked_dir)}: Error: {e}" + ) + + if errors: + print(f"FAILED - Found {len(errors)} insertion validation violations:") + for error in errors: + print(error) + return False + else: + if self.verbose: + print("PASSED - No w:delText elements within w:ins elements") + return True + + def compare_paragraph_counts(self): + original_count = self.count_paragraphs_in_original() + new_count = self.count_paragraphs_in_unpacked() + + diff = new_count - original_count + diff_str = f"+{diff}" if diff > 0 else str(diff) + print(f"\nParagraphs: {original_count} → {new_count} ({diff_str})") + + def _parse_id_value(self, val: str, base: int = 16) -> int: + return int(val, base) + + def validate_id_constraints(self): + errors = [] + para_id_attr = f"{{{self.W14_NAMESPACE}}}paraId" + durable_id_attr = f"{{{self.W16CID_NAMESPACE}}}durableId" + + for xml_file in self.xml_files: + try: + for elem in lxml.etree.parse(str(xml_file)).iter(): + if val := elem.get(para_id_attr): + if self._parse_id_value(val, base=16) >= 0x80000000: + errors.append( + f" {xml_file.name}:{elem.sourceline}: paraId={val} >= 0x80000000" + ) + + if val := elem.get(durable_id_attr): + if xml_file.name == "numbering.xml": + try: + if self._parse_id_value(val, base=10) >= 0x7FFFFFFF: + errors.append( + f" {xml_file.name}:{elem.sourceline}: " + f"durableId={val} >= 0x7FFFFFFF" + ) + except ValueError: + errors.append( + f" {xml_file.name}:{elem.sourceline}: " + f"durableId={val} must be decimal in numbering.xml" + ) + else: + if self._parse_id_value(val, base=16) >= 0x7FFFFFFF: + errors.append( + f" {xml_file.name}:{elem.sourceline}: " + f"durableId={val} >= 0x7FFFFFFF" + ) + except Exception: + pass + + if errors: + print(f"FAILED - {len(errors)} ID constraint violations:") + for e in errors: + print(e) + elif self.verbose: + print("PASSED - All paraId/durableId values within constraints") + return not errors + + def validate_comment_markers(self): + errors = [] + + document_xml = None + comments_xml = None + for xml_file in self.xml_files: + if xml_file.name == "document.xml" and "word" in str(xml_file): + document_xml = xml_file + elif xml_file.name == "comments.xml": + comments_xml = xml_file + + if not document_xml: + if self.verbose: + print("PASSED - No document.xml found (skipping comment validation)") + return True + + try: + doc_root = lxml.etree.parse(str(document_xml)).getroot() + namespaces = {"w": self.WORD_2006_NAMESPACE} + + range_starts = { + elem.get(f"{{{self.WORD_2006_NAMESPACE}}}id") + for elem in doc_root.xpath( + ".//w:commentRangeStart", namespaces=namespaces + ) + } + range_ends = { + elem.get(f"{{{self.WORD_2006_NAMESPACE}}}id") + for elem in doc_root.xpath( + ".//w:commentRangeEnd", namespaces=namespaces + ) + } + references = { + elem.get(f"{{{self.WORD_2006_NAMESPACE}}}id") + for elem in doc_root.xpath( + ".//w:commentReference", namespaces=namespaces + ) + } + + orphaned_ends = range_ends - range_starts + for comment_id in sorted( + orphaned_ends, key=lambda x: int(x) if x and x.isdigit() else 0 + ): + errors.append( + f' document.xml: commentRangeEnd id="{comment_id}" has no matching commentRangeStart' + ) + + orphaned_starts = range_starts - range_ends + for comment_id in sorted( + orphaned_starts, key=lambda x: int(x) if x and x.isdigit() else 0 + ): + errors.append( + f' document.xml: commentRangeStart id="{comment_id}" has no matching commentRangeEnd' + ) + + comment_ids = set() + if comments_xml and comments_xml.exists(): + comments_root = lxml.etree.parse(str(comments_xml)).getroot() + comment_ids = { + elem.get(f"{{{self.WORD_2006_NAMESPACE}}}id") + for elem in comments_root.xpath( + ".//w:comment", namespaces=namespaces + ) + } + + marker_ids = range_starts | range_ends | references + invalid_refs = marker_ids - comment_ids + for comment_id in sorted( + invalid_refs, key=lambda x: int(x) if x and x.isdigit() else 0 + ): + if comment_id: + errors.append( + f' document.xml: marker id="{comment_id}" references non-existent comment' + ) + + except (lxml.etree.XMLSyntaxError, Exception) as e: + errors.append(f" Error parsing XML: {e}") + + if errors: + print(f"FAILED - {len(errors)} comment marker violations:") + for error in errors: + print(error) + return False + else: + if self.verbose: + print("PASSED - All comment markers properly paired") + return True + + def repair(self) -> int: + repairs = super().repair() + repairs += self.repair_durableId() + return repairs + + def repair_durableId(self) -> int: + repairs = 0 + + for xml_file in self.xml_files: + try: + content = xml_file.read_text(encoding="utf-8") + dom = defusedxml.minidom.parseString(content) + modified = False + + for elem in dom.getElementsByTagName("*"): + if not elem.hasAttribute("w16cid:durableId"): + continue + + durable_id = elem.getAttribute("w16cid:durableId") + needs_repair = False + + if xml_file.name == "numbering.xml": + try: + needs_repair = ( + self._parse_id_value(durable_id, base=10) >= 0x7FFFFFFF + ) + except ValueError: + needs_repair = True + else: + try: + needs_repair = ( + self._parse_id_value(durable_id, base=16) >= 0x7FFFFFFF + ) + except ValueError: + needs_repair = True + + if needs_repair: + value = random.randint(1, 0x7FFFFFFE) + if xml_file.name == "numbering.xml": + new_id = str(value) + else: + new_id = f"{value:08X}" + + elem.setAttribute("w16cid:durableId", new_id) + print( + f" Repaired: {xml_file.name}: durableId {durable_id} → {new_id}" + ) + repairs += 1 + modified = True + + if modified: + xml_file.write_bytes(dom.toxml(encoding="UTF-8")) + + except Exception: + pass + + return repairs + + +if __name__ == "__main__": + raise RuntimeError("This module should not be run directly.") diff --git a/skills/docx/scripts/office/validators/pptx.py b/skills/docx/scripts/office/validators/pptx.py new file mode 100644 index 00000000..09842aa9 --- /dev/null +++ b/skills/docx/scripts/office/validators/pptx.py @@ -0,0 +1,275 @@ +""" +Validator for PowerPoint presentation XML files against XSD schemas. +""" + +import re + +from .base import BaseSchemaValidator + + +class PPTXSchemaValidator(BaseSchemaValidator): + + PRESENTATIONML_NAMESPACE = ( + "http://schemas.openxmlformats.org/presentationml/2006/main" + ) + + ELEMENT_RELATIONSHIP_TYPES = { + "sldid": "slide", + "sldmasterid": "slidemaster", + "notesmasterid": "notesmaster", + "sldlayoutid": "slidelayout", + "themeid": "theme", + "tablestyleid": "tablestyles", + } + + def validate(self): + if not self.validate_xml(): + return False + + all_valid = True + if not self.validate_namespaces(): + all_valid = False + + if not self.validate_unique_ids(): + all_valid = False + + if not self.validate_uuid_ids(): + all_valid = False + + if not self.validate_file_references(): + all_valid = False + + if not self.validate_slide_layout_ids(): + all_valid = False + + if not self.validate_content_types(): + all_valid = False + + if not self.validate_against_xsd(): + all_valid = False + + if not self.validate_notes_slide_references(): + all_valid = False + + if not self.validate_all_relationship_ids(): + all_valid = False + + if not self.validate_no_duplicate_slide_layouts(): + all_valid = False + + return all_valid + + def validate_uuid_ids(self): + import lxml.etree + + errors = [] + uuid_pattern = re.compile( + r"^[\{\(]?[0-9A-Fa-f]{8}-?[0-9A-Fa-f]{4}-?[0-9A-Fa-f]{4}-?[0-9A-Fa-f]{4}-?[0-9A-Fa-f]{12}[\}\)]?$" + ) + + for xml_file in self.xml_files: + try: + root = lxml.etree.parse(str(xml_file)).getroot() + + for elem in root.iter(): + for attr, value in elem.attrib.items(): + attr_name = attr.split("}")[-1].lower() + if attr_name == "id" or attr_name.endswith("id"): + if self._looks_like_uuid(value): + if not uuid_pattern.match(value): + errors.append( + f" {xml_file.relative_to(self.unpacked_dir)}: " + f"Line {elem.sourceline}: ID '{value}' appears to be a UUID but contains invalid hex characters" + ) + + except (lxml.etree.XMLSyntaxError, Exception) as e: + errors.append( + f" {xml_file.relative_to(self.unpacked_dir)}: Error: {e}" + ) + + if errors: + print(f"FAILED - Found {len(errors)} UUID ID validation errors:") + for error in errors: + print(error) + return False + else: + if self.verbose: + print("PASSED - All UUID-like IDs contain valid hex values") + return True + + def _looks_like_uuid(self, value): + clean_value = value.strip("{}()").replace("-", "") + return len(clean_value) == 32 and all(c.isalnum() for c in clean_value) + + def validate_slide_layout_ids(self): + import lxml.etree + + errors = [] + + slide_masters = list(self.unpacked_dir.glob("ppt/slideMasters/*.xml")) + + if not slide_masters: + if self.verbose: + print("PASSED - No slide masters found") + return True + + for slide_master in slide_masters: + try: + root = lxml.etree.parse(str(slide_master)).getroot() + + rels_file = slide_master.parent / "_rels" / f"{slide_master.name}.rels" + + if not rels_file.exists(): + errors.append( + f" {slide_master.relative_to(self.unpacked_dir)}: " + f"Missing relationships file: {rels_file.relative_to(self.unpacked_dir)}" + ) + continue + + rels_root = lxml.etree.parse(str(rels_file)).getroot() + + valid_layout_rids = set() + for rel in rels_root.findall( + f".//{{{self.PACKAGE_RELATIONSHIPS_NAMESPACE}}}Relationship" + ): + rel_type = rel.get("Type", "") + if "slideLayout" in rel_type: + valid_layout_rids.add(rel.get("Id")) + + for sld_layout_id in root.findall( + f".//{{{self.PRESENTATIONML_NAMESPACE}}}sldLayoutId" + ): + r_id = sld_layout_id.get( + f"{{{self.OFFICE_RELATIONSHIPS_NAMESPACE}}}id" + ) + layout_id = sld_layout_id.get("id") + + if r_id and r_id not in valid_layout_rids: + errors.append( + f" {slide_master.relative_to(self.unpacked_dir)}: " + f"Line {sld_layout_id.sourceline}: sldLayoutId with id='{layout_id}' " + f"references r:id='{r_id}' which is not found in slide layout relationships" + ) + + except (lxml.etree.XMLSyntaxError, Exception) as e: + errors.append( + f" {slide_master.relative_to(self.unpacked_dir)}: Error: {e}" + ) + + if errors: + print(f"FAILED - Found {len(errors)} slide layout ID validation errors:") + for error in errors: + print(error) + print( + "Remove invalid references or add missing slide layouts to the relationships file." + ) + return False + else: + if self.verbose: + print("PASSED - All slide layout IDs reference valid slide layouts") + return True + + def validate_no_duplicate_slide_layouts(self): + import lxml.etree + + errors = [] + slide_rels_files = list(self.unpacked_dir.glob("ppt/slides/_rels/*.xml.rels")) + + for rels_file in slide_rels_files: + try: + root = lxml.etree.parse(str(rels_file)).getroot() + + layout_rels = [ + rel + for rel in root.findall( + f".//{{{self.PACKAGE_RELATIONSHIPS_NAMESPACE}}}Relationship" + ) + if "slideLayout" in rel.get("Type", "") + ] + + if len(layout_rels) > 1: + errors.append( + f" {rels_file.relative_to(self.unpacked_dir)}: has {len(layout_rels)} slideLayout references" + ) + + except Exception as e: + errors.append( + f" {rels_file.relative_to(self.unpacked_dir)}: Error: {e}" + ) + + if errors: + print("FAILED - Found slides with duplicate slideLayout references:") + for error in errors: + print(error) + return False + else: + if self.verbose: + print("PASSED - All slides have exactly one slideLayout reference") + return True + + def validate_notes_slide_references(self): + import lxml.etree + + errors = [] + notes_slide_references = {} + + slide_rels_files = list(self.unpacked_dir.glob("ppt/slides/_rels/*.xml.rels")) + + if not slide_rels_files: + if self.verbose: + print("PASSED - No slide relationship files found") + return True + + for rels_file in slide_rels_files: + try: + root = lxml.etree.parse(str(rels_file)).getroot() + + for rel in root.findall( + f".//{{{self.PACKAGE_RELATIONSHIPS_NAMESPACE}}}Relationship" + ): + rel_type = rel.get("Type", "") + if "notesSlide" in rel_type: + target = rel.get("Target", "") + if target: + normalized_target = target.replace("../", "") + + slide_name = rels_file.stem.replace( + ".xml", "" + ) + + if normalized_target not in notes_slide_references: + notes_slide_references[normalized_target] = [] + notes_slide_references[normalized_target].append( + (slide_name, rels_file) + ) + + except (lxml.etree.XMLSyntaxError, Exception) as e: + errors.append( + f" {rels_file.relative_to(self.unpacked_dir)}: Error: {e}" + ) + + for target, references in notes_slide_references.items(): + if len(references) > 1: + slide_names = [ref[0] for ref in references] + errors.append( + f" Notes slide '{target}' is referenced by multiple slides: {', '.join(slide_names)}" + ) + for slide_name, rels_file in references: + errors.append(f" - {rels_file.relative_to(self.unpacked_dir)}") + + if errors: + print( + f"FAILED - Found {len([e for e in errors if not e.startswith(' ')])} notes slide reference validation errors:" + ) + for error in errors: + print(error) + print("Each slide may optionally have its own slide file.") + return False + else: + if self.verbose: + print("PASSED - All notes slide references are unique") + return True + + +if __name__ == "__main__": + raise RuntimeError("This module should not be run directly.") diff --git a/skills/docx/scripts/office/validators/redlining.py b/skills/docx/scripts/office/validators/redlining.py new file mode 100644 index 00000000..71c81b6b --- /dev/null +++ b/skills/docx/scripts/office/validators/redlining.py @@ -0,0 +1,247 @@ +""" +Validator for tracked changes in Word documents. +""" + +import subprocess +import tempfile +import zipfile +from pathlib import Path + + +class RedliningValidator: + + def __init__(self, unpacked_dir, original_docx, verbose=False, author="Claude"): + self.unpacked_dir = Path(unpacked_dir) + self.original_docx = Path(original_docx) + self.verbose = verbose + self.author = author + self.namespaces = { + "w": "http://schemas.openxmlformats.org/wordprocessingml/2006/main" + } + + def repair(self) -> int: + return 0 + + def validate(self): + modified_file = self.unpacked_dir / "word" / "document.xml" + if not modified_file.exists(): + print(f"FAILED - Modified document.xml not found at {modified_file}") + return False + + try: + import xml.etree.ElementTree as ET + + tree = ET.parse(modified_file) + root = tree.getroot() + + del_elements = root.findall(".//w:del", self.namespaces) + ins_elements = root.findall(".//w:ins", self.namespaces) + + author_del_elements = [ + elem + for elem in del_elements + if elem.get(f"{{{self.namespaces['w']}}}author") == self.author + ] + author_ins_elements = [ + elem + for elem in ins_elements + if elem.get(f"{{{self.namespaces['w']}}}author") == self.author + ] + + if not author_del_elements and not author_ins_elements: + if self.verbose: + print(f"PASSED - No tracked changes by {self.author} found.") + return True + + except Exception: + pass + + with tempfile.TemporaryDirectory() as temp_dir: + temp_path = Path(temp_dir) + + try: + with zipfile.ZipFile(self.original_docx, "r") as zip_ref: + zip_ref.extractall(temp_path) + except Exception as e: + print(f"FAILED - Error unpacking original docx: {e}") + return False + + original_file = temp_path / "word" / "document.xml" + if not original_file.exists(): + print( + f"FAILED - Original document.xml not found in {self.original_docx}" + ) + return False + + try: + import xml.etree.ElementTree as ET + + modified_tree = ET.parse(modified_file) + modified_root = modified_tree.getroot() + original_tree = ET.parse(original_file) + original_root = original_tree.getroot() + except ET.ParseError as e: + print(f"FAILED - Error parsing XML files: {e}") + return False + + self._remove_author_tracked_changes(original_root) + self._remove_author_tracked_changes(modified_root) + + modified_text = self._extract_text_content(modified_root) + original_text = self._extract_text_content(original_root) + + if modified_text != original_text: + error_message = self._generate_detailed_diff( + original_text, modified_text + ) + print(error_message) + return False + + if self.verbose: + print(f"PASSED - All changes by {self.author} are properly tracked") + return True + + def _generate_detailed_diff(self, original_text, modified_text): + error_parts = [ + f"FAILED - Document text doesn't match after removing {self.author}'s tracked changes", + "", + "Likely causes:", + " 1. Modified text inside another author's or tags", + " 2. Made edits without proper tracked changes", + " 3. Didn't nest inside when deleting another's insertion", + "", + "For pre-redlined documents, use correct patterns:", + " - To reject another's INSERTION: Nest inside their ", + " - To restore another's DELETION: Add new AFTER their ", + "", + ] + + git_diff = self._get_git_word_diff(original_text, modified_text) + if git_diff: + error_parts.extend(["Differences:", "============", git_diff]) + else: + error_parts.append("Unable to generate word diff (git not available)") + + return "\n".join(error_parts) + + def _get_git_word_diff(self, original_text, modified_text): + try: + with tempfile.TemporaryDirectory() as temp_dir: + temp_path = Path(temp_dir) + + original_file = temp_path / "original.txt" + modified_file = temp_path / "modified.txt" + + original_file.write_text(original_text, encoding="utf-8") + modified_file.write_text(modified_text, encoding="utf-8") + + result = subprocess.run( + [ + "git", + "diff", + "--word-diff=plain", + "--word-diff-regex=.", + "-U0", + "--no-index", + str(original_file), + str(modified_file), + ], + capture_output=True, + text=True, + ) + + if result.stdout.strip(): + lines = result.stdout.split("\n") + content_lines = [] + in_content = False + for line in lines: + if line.startswith("@@"): + in_content = True + continue + if in_content and line.strip(): + content_lines.append(line) + + if content_lines: + return "\n".join(content_lines) + + result = subprocess.run( + [ + "git", + "diff", + "--word-diff=plain", + "-U0", + "--no-index", + str(original_file), + str(modified_file), + ], + capture_output=True, + text=True, + ) + + if result.stdout.strip(): + lines = result.stdout.split("\n") + content_lines = [] + in_content = False + for line in lines: + if line.startswith("@@"): + in_content = True + continue + if in_content and line.strip(): + content_lines.append(line) + return "\n".join(content_lines) + + except (subprocess.CalledProcessError, FileNotFoundError, Exception): + pass + + return None + + def _remove_author_tracked_changes(self, root): + ins_tag = f"{{{self.namespaces['w']}}}ins" + del_tag = f"{{{self.namespaces['w']}}}del" + author_attr = f"{{{self.namespaces['w']}}}author" + + for parent in root.iter(): + to_remove = [] + for child in parent: + if child.tag == ins_tag and child.get(author_attr) == self.author: + to_remove.append(child) + for elem in to_remove: + parent.remove(elem) + + deltext_tag = f"{{{self.namespaces['w']}}}delText" + t_tag = f"{{{self.namespaces['w']}}}t" + + for parent in root.iter(): + to_process = [] + for child in parent: + if child.tag == del_tag and child.get(author_attr) == self.author: + to_process.append((child, list(parent).index(child))) + + for del_elem, del_index in reversed(to_process): + for elem in del_elem.iter(): + if elem.tag == deltext_tag: + elem.tag = t_tag + + for child in reversed(list(del_elem)): + parent.insert(del_index, child) + parent.remove(del_elem) + + def _extract_text_content(self, root): + p_tag = f"{{{self.namespaces['w']}}}p" + t_tag = f"{{{self.namespaces['w']}}}t" + + paragraphs = [] + for p_elem in root.findall(f".//{p_tag}"): + text_parts = [] + for t_elem in p_elem.findall(f".//{t_tag}"): + if t_elem.text: + text_parts.append(t_elem.text) + paragraph_text = "".join(text_parts) + if paragraph_text: + paragraphs.append(paragraph_text) + + return "\n".join(paragraphs) + + +if __name__ == "__main__": + raise RuntimeError("This module should not be run directly.") diff --git a/skills/docx/scripts/templates/comments.xml b/skills/docx/scripts/templates/comments.xml new file mode 100644 index 00000000..cd01a7d7 --- /dev/null +++ b/skills/docx/scripts/templates/comments.xml @@ -0,0 +1,3 @@ + + + diff --git a/skills/docx/scripts/templates/commentsExtended.xml b/skills/docx/scripts/templates/commentsExtended.xml new file mode 100644 index 00000000..411003cc --- /dev/null +++ b/skills/docx/scripts/templates/commentsExtended.xml @@ -0,0 +1,3 @@ + + + diff --git a/skills/docx/scripts/templates/commentsExtensible.xml b/skills/docx/scripts/templates/commentsExtensible.xml new file mode 100644 index 00000000..f5572d71 --- /dev/null +++ b/skills/docx/scripts/templates/commentsExtensible.xml @@ -0,0 +1,3 @@ + + + diff --git a/skills/docx/scripts/templates/commentsIds.xml b/skills/docx/scripts/templates/commentsIds.xml new file mode 100644 index 00000000..32f1629f --- /dev/null +++ b/skills/docx/scripts/templates/commentsIds.xml @@ -0,0 +1,3 @@ + + + diff --git a/skills/docx/scripts/templates/people.xml b/skills/docx/scripts/templates/people.xml new file mode 100644 index 00000000..3803d2de --- /dev/null +++ b/skills/docx/scripts/templates/people.xml @@ -0,0 +1,3 @@ + + + diff --git a/skills/education-graeham-videos/SKILL.md b/skills/education-graeham-videos/SKILL.md new file mode 100644 index 00000000..9ea20347 --- /dev/null +++ b/skills/education-graeham-videos/SKILL.md @@ -0,0 +1,321 @@ +--- +name: education-graeham-videos +description: "Education Graeham Videos — Reventure-style data-driven real estate educational videos for Graeham Watts, localized to East Palo Alto, Palo Alto, Menlo Park, Redwood City, San Mateo County, San Jose, SF, and the East Bay. Replicates the exact Reventure format: authority-anchored doom-hook packaging, dense sourced data, zip-level bifurcation stories, listing autopsies, myth-busts, labeled speculation, balanced verdicts, hard CTAs — as long-form YouTube scripts AND reels with full production specs. Use ANY time the user says: create an educational video for the day, educational video, daily video, Reventure video, Reventure style, data video, housing market video, make today's video, or asks for trending Bay Area real estate topics to turn into a video. ALWAYS starts by researching trending topics and presenting a Top 5 list for the user to pick — never jump straight to a script." +--- + +## STANDING BRAND RULES — every video (locked 2026-07-31, from Graeham's reference frame) + +**End card (final ~5s of EVERY finished video), composited over a ~60% black wash of the +closing shot:** +1. TOP: white GRAEHAM WATTS logo lockup (gold roof icon + gold divider + REALTOR), from + `carousel-builder/assets/logo/logo_white.png`, ~860px wide (1080-wide frame), centered. +2. Under logo: `DRE #01466876` letter-spaced Montserrat, white — brokerage name ONLY, no + default text. Graeham left the former brokerage (first entry in identity.json brand_blocklist) as of 2026-08-09; do not print its name or + any other brokerage name on the end card unless Graeham explicitly names a new one to use. + The brand end card DOES show the DRE — on this card it supersedes any "no DRE on screen" + note elsewhere. +3. MIDDLE: the video's subject line in gold-gradient Great Vibes cursive + (`carousel-builder/assets/fonts/GreatVibes-Regular.ttf`), auto-sized to fit ≤980px. +4. Gold gradient rounded button (light→dark metallic), black Montserrat ExtraBold: + `DM "KEYWORD"` — keyword per viral-playbook CTA system, GHL comment-automation compatible. +5. Below button, white Montserrat caps: `OR CALL — NUMBER IN BIO`. +6. The SPOKEN CTA must match the button keyword. One CTA per video. Say it, show it, pin it. + +**Text treatment (reels/short-form):** Reventure concept in Watts colors — karaoke captions +white extra-bold with black outline, active word on a General-Accent gold (#C4A265) rounded +pill with black text; section labels gold bold-italic with heavy black outline (marker style, +no box); cover hook = white rounded box, dark bold text, exactly ONE emoji; evidence shown as +RAW page screenshots with gold highlighter markings; backgrounds Watts Navy #0A1F44; panels +black with 2px gold top stroke; ONE hero moment per video in protected Watts Gold #B8945A. + +**Text sizes (1080×1920 frame — locked after v7):** karaoke captions Bold 96px (shrink-to-fit +floor 72px, max width ~1020px), outline 7px black, baseline y≈1540; active-word pill padding +16/10px radius 18; section labels BoldItalic 64px; CTA overlay BoldItalic 88px; cover-box text +Bold 58px. Text is BIG — it should dominate the lower third like the reference reels. + + +**Process rules:** ask for all required credentials BEFORE any production work (missing keys = +full stop and ask); NEVER downgrade or substitute the instructed format/avatar/voice/visuals — +if blocked, stop, name the blocker, wait. + +**MANDATORY FULL-SWEEP QC — before EVERY video delivery, no exceptions, never wait to be asked:** +1. Frame sweep of the ENTIRE runtime at ≥2 frames/second (0.5s intervals), viewed in grids — + every segment, not spot samples. +2. Every cut boundary frame checked; consecutive-frame strip on any composited element + (cutouts, overlays) to catch flicker. +3. Every on-screen number re-verified against the citation table. +4. Caption sync spot-verified on at least 10 sampled words across the runtime. +5. ffprobe container check (resolution, fps, codecs, duration) + volumedetect (peak −1 to −3 dB, + no clipping). +6. EXPLICIT PASS/FAIL items (added 2026-08-02 after all four failed in one delivery): + a. Highlighter circle CLEARS the circled content — the stroke must never touch the text/number + it circles ANYWHERE along its curve, not just at mid-height (2026-08-05: side-padding alone + still clipped the corner letters of a multi-line headline). Size by the CORNER TEST: for the + ellipse with center (cx,cy) and semi-axes (a,b), every corner (px,py) of every circled text + line must satisfy ((px-cx)/a)^2 + ((py-cy)/b)^2 <= 0.86. For a full text block this means a + near-circumscribing ellipse (~sqrt(2) times the block size); if that overflows the 1080 frame, + shrink the page scale / shift x0 rather than the ellipse. Furniture (chips, buttons, borders) + may be grazed; readable body text may not. Verify with a rendered overlay check BEFORE + compositing AND on final frames. + b. Cutout bottom-overflow: the cutout's bottom edge must reach/overflow the canvas bottom in + EVERY frame — no floating-torso gap. Enforce in code (min scale), don't just eyeball. + c. No hard interior slice edges: body parts touching the source-frame border must fade + (soft alpha ramp), never end in a visible vertical/horizontal cut inside the canvas. + d. LIP SYNC: extract a consecutive-frame strip (≥12 fps) of the mouth at the hook + 2 other + phrases and match closure/opening frames to word onsets from the alignment (P/B closures + are the easiest anchors). HeyGen renders MUST use the v3 endpoint (POST /v3/videos) — + the legacy v2 endpoint can fall back to an older animation engine with visibly worse lips. +7. Report findings to Graeham WITH the two things frame QC cannot verify (audio feel, motion + between frames) called out explicitly. Never claim 100% on unverified dimensions. +A video that has not passed the full sweep is NOT delivered. Deliver-then-fix is a rule violation. +A defect class found by Graeham gets added to THIS checklist the same day — the sweep tests for +every failure mode ever seen, not just the convenient ones. + +**FRESH LOOK PER VIDEO (hard workflow step, not a preference):** every video gets a NEW avatar +look with a different outfit; when the video is about a specific place, research and capture +real imagery of that place (Street View) and put Graeham THERE as the look's background — +full steps in `references/production-pipeline.md` Step 0-4. Studio looks (`fashion_flip`, +`bespectacled`) are fallbacks ONLY if Graeham explicitly declines a location look. **No specific +place in this topic (locked 2026-08-12):** default to a real, notable, recognizable Bay Area +street/location — East Palo Alto, Menlo Park, Redwood City, San Francisco, or elsewhere in the +Bay — something the audience would actually recognize, never a generic anonymous block. Studio +looks stay the fallback-of-last-resort, not the default for "no specific place." Full sourcing +logic in `references/production-pipeline.md` Step 0. + +**SHORT + LONG FORM PAIR = ONE FRESH AVATAR (added 2026-08-06).** When a topic ships as a +short-form reel AND a companion long-form video (the standard pairing), that pair shares +ONE fresh avatar look — built once per TOPIC, not once per video output. Same outfit, same +location background (if the topic is about a specific place), same digital-twin identity in +both deliverables. The only thing that differs between the two is aspect ratio/orientation: +- Reel / short-form = portrait **9:16**, ALWAYS. +- Long-form YouTube video = landscape **16:9**, ALWAYS. +Do not generate a second "fresh" look for the long-form video just because it's a different +deliverable — that breaks visual continuity between the pair and burns a second one-time look +build for no reason. Build both orientations together in the same Step 0-4 pass (see the +updated LOOK PAIRS bullet under rule 0g). + +# Education Graeham Videos + +Turn the day's most compelling Bay Area real estate story into a Reventure-style educational +video package: verified research → user-picked topic → long-form YouTube script + reel script + +titles, thumbnail spec, captions, asset list, and production handoffs. + +**Read before writing anything:** +- `references/reventure-blueprint.md` — the full style codebook (formats, 11-beat script + architecture, visual grammar, packaging formulas, reel architecture). This is the ground truth + for HOW these videos work. Follow it beat-for-beat. +- `references/script-templates.md` — fill-in templates for scripts, titles, thumbnails, captions. + +## Non-negotiable rules + +0d. **Lifestyle series — attraction, not doom (added 2026-08-02).** A second video track exists + alongside the market-data videos: LIFESTYLE educational videos ("Moving to East Palo Alto", + "best-kept secret", "what $1M buys", schools, new parks/restaurants/transit) for Graeham's + farm areas: East Palo Alto, Redwood City, East Menlo Park, San Francisco, San Jose, Bay Area. + These videos SELL the areas. Zero negativity: no fear hooks, no "crash/decline/left behind" + framing, nothing that would push a buyer away from moving there. Keep the Reventure + production grammar (pacing, receipts, captions, stickers) but flip the emotion: curiosity, + FOMO, opportunity — "get in before everyone figures it out", never "get out". + +0e. **Freshness rule — days old at MOST (added 2026-08-02).** For lifestyle/news videos, the + news peg the video hangs on must be from within a few days of creation day (same day to a + few days before/after). Every video day starts with a fresh last-48h/last-7-days search + BEFORE scripting; the hook anchors to the newest item found. Evergreen facts (distances, + district boundaries, long-running projects) may support the story, but the hook itself must + be fresh. An older story may only be used if a NEW milestone (hearing, filing, opening, + report) re-freshens it — otherwise reframe the video as an evergreen guide, not news. + +0f. **Future-projects videos (added 2026-08-02).** Proposed or under-construction developments + that are big news for a farm area are a wanted lifestyle-track topic: new buildings, housing + projects, transit (BART/Caltrain), waterfront/downtown plans, campuses, parks. Requirements: + every claim backed by verifiable sources (city planning pages, CEQA/permit filings, council + agendas, developer sites, local press) cited in the citation table; clearly label status + (proposed vs approved vs under construction vs opening) and never present a proposal as a + done deal; frame as opportunity/growth for the area (per rule 0d). Renderings/site plans used + as visuals get a source + license note like any other asset. + +0g. **Production standards locked 2026-08-04 (from RWC build iterations), aspect-ratio mapping + updated 2026-08-06:** + - LOOK PAIRS (simplified 2026-08-12 — Higgsfield generates a STILL, HeyGen does the + animating): every on-location avatar look is created as a PAIR — portrait 9:16 + + landscape 16:9 — as a single 4K Higgsfield Nano Banana Pro still per orientation + (identity ref + real environment ref composited together, no Seedance/video-training + step), uploaded to HeyGen as a photo-based look. Native 4K Higgsfield output means no + separate upscale pass is needed before the look is created. Full steps in + `references/production-pipeline.md` Step 0-4. Built ONCE PER TOPIC + (not once per video output) — if the topic ships as a short+long-form pair, both + deliverables reuse this same pair. FIXED mapping, no exceptions: portrait 9:16 is the + reel/short-form primary (hook, close-ups); landscape 16:9 is the long-form YouTube primary + (full-frame field format) and also serves split-screen lower strips inside reels. One-time + cost per topic/location; per-video renders are then HeyGen-only. NEVER outpaint/reframe per + video (seams mid-sentence + recurring credits). + - ANNOTATIONS ARE RED (220,40,35): highlighter circles AND arrows. Circles sized by the + CORNER TEST (QC item 6a — near-circumscribing, stroke never touches circled text anywhere + on the curve, 2026-08-05); arrows only from verified-empty zones; NO cursor. On pages with + tightly-stacked rows (e.g. congress.gov bill pages) where no clean ellipse exists, use a + red arrow from an empty zone INSTEAD of a circle. + - PRICES IN TTS: plain decimals WITHOUT the $ sign ("10.75 million dollars") — a leading $ + makes ElevenLabs say "ten dollars point seven five". Captions substitute the $ figure + back via a display map ({"10.75": "$10.75"}). MANDATORY AUDIO QC before any render: + Whisper-transcribe the generated VO and verify every number/name is spoken correctly; + use voice_settings.speed (~1.05-1.10) for reference-fast pacing — raising style slows + delivery down, it does not speed it up. + - NO EM-DASHES IN VO SCRIPTS (2026-08-05): ElevenLabs holds ~0.9s of dead air on every + "—", which reads as awkward stalling. Use commas. After generation, run a PAUSE-PROFILE + check on the alignment: no mid-sentence gap > 0.5s; sentence-boundary gaps <= ~0.8s + (approved-baseline). Trim longer sentence gaps surgically (cut audio + shift alignment + times) rather than regenerating and re-rolling the dice. Also verify the Whisper + transcript has NO extra syllables (TTS sometimes injects an audible breath/vocalization + in long gaps — mute that window with fades if found). + - TRANSITION WHOOSH is the airy swept-noise build (sfx_whoosh2: bandpass sweep up-then- + down, swell envelope + low-air layer) — not the plain noise burst (rejected 2026-08-05). + - SFX MIX ARCHITECTURE (2026-08-05, replaces the old whole-mix loudnorm): normalize the + VO ALONE to I=-14 FIRST (single-pass loudnorm on speech is fine), THEN add SFX at true + relative gains, then final alimiter at mux. NEVER run loudnorm over the finished mix — + single-pass loudnorm is DYNAMIC and pumps the gain up during VO gaps, which blasts any + SFX sitting there (this is why the whoosh was 'distracting': it measured as loud as + speech). Target: whoosh ~9-10dB RMS under speech RMS ("subtle but noticeable"; verify + with a volumedetect on a VO-silent gap vs a speech window), pop ~0.32 on caps words. + - Evidence beats: article at 1.15x (per-page centering, no clipped words) over blur-fill, + full-width landscape strip of the wide look at the bottom, captions above his head. + - Hook: native framing, NO zoom pumping. Close-up zoom only on mid-video talking beats. + +0h. **Sharpness + motion standards (locked 2026-08-04, from "laggy/not HD" fix):** + - PORTRAIT head renders at 4K (resolution "4k", 2160x3840) so every punch-in DOWNSCALES + into the 1080x1920 frame — never blow up a 1080p render. Wide/landscape renders stay + 1080p (they're downscaled into the strip anyway). + - UPSCALE THE TRAINING FOOTAGE BEFORE CREATING LOOKS (2026-08-05, "still low def" fix): + Seedance outputs 720p; a look trained on 720p stays soft no matter how big the HeyGen + render is — the 4K render just upscales softness. Topaz-upscale (upscale_video provider + topaz, 2160p) the concatenated training clips FIRST, then create the digital-twin looks + from the 4K files. Same one-time cost logic: the HD look pair is reusable forever. + - FROZEN PUNCH-INS: the close-up crop window is locked per jump-cut interval (~1.8s) — + computed once from the face-track EMA at interval entry, then held. Per-frame tracking + drift reads as lag/stutter; never re-track inside an interval. + - EXAGGERATED JUMP CUTS (2026-08-05): talking-beat zoom levels alternate WIDE half-body + (~1.15x) <-> TIGHT face (~1.75x). A small delta (e.g. 1.55<->1.70) reads as a subtle + "pump", not a cut — the two framings must be unmistakably different. Hook stays native. + - B-ROLL FPS CONFORM (2026-08-05, "laggy" fix): generated b-roll is usually 24fps; forcing + it to the 25fps timeline with -r duplicates frames and puts visible judder on smooth + drone pans. ALWAYS conform with minterpolate (mi_mode=mci:mc_mode=aobmc:vsbmc=1) to the + timeline fps before extracting frames. Check r_frame_rate of every source; never let a + dup/drop cadence into pans. + - ALL scaling uses LANCZOS (never BILINEAR for visible content). + - Master encode: CRF 18. Delivery caps: chat ≤30MiB (encode CRF ~22 to fit — this is the + POSTING copy), device transfer ≤20MiB (two-pass ~3.2Mbps). Tell Graeham which file is + the posting copy every time. + - Footage is 25fps (HeyGen output). If playback still feels non-fluid, the next lever is + 30fps interpolation on head segments — test before promising. + - FILE NAMING: final file = the hook phrase as a clean title ("Redwood City Just Said + YES.mp4"). FINAL folder holds exactly ONE file; every prior version goes to old\. + +1. **Top-5 first.** On "create an educational video for the day," ALWAYS run Phase 1 research and + present a scored Top 5 topic list. The user picks. Never skip to scripting. +2. **Every fact verified.** Every statistic in a script must trace to the citation table built in + Phase 2: one primary source (CAR, MLS, Census, FRED, Zillow/Redfin research, county records) or + two independent reliable secondary sources. No number without a source. If a stat can't be + verified, it doesn't go in the script — say so and substitute. +3. **Reventure is invisible.** Reventure/Nick Gerli is an internal style reference ONLY. Never + mention, cite, show, promote, or allude to Reventure, the Reventure App, or Nick Gerli in any + script, title, caption, thumbnail, overlay, description, comment, or chart — ever. All data + credibility comes from OUR sources (MLS, CAR, Zillow/Redfin research, Census, FRED) and all + CTAs point to Graeham's business. NEVER use Reventure's charts, maps, footage, or branding — + we replicate the METHOD, not the assets and not the brand. Research materials grabbed for our videos must be: public data we + re-chart in Graeham's branding, government/public-domain material, brief news-headline + screenshots used as commentary (fair-use style, shown with source visible), listing screenshots + from public portals used for market commentary, or licensed/free-to-use footage. Every grabbed + asset gets a source + license note in the asset list. +4. **Compliance.** Graeham is a licensed agent (DRE# 01466876). No fair-housing violations: never + characterize neighborhoods by who lives there; talk prices, inventory, DOM, migration counts — + not people. Forecasts are framed as data commentary, not guarantees ("the data suggests", never + "will definitely"). Doom packaging is fine; fabricated or misrepresented numbers are not. +5. **Full Reventure energy.** Provocative packaging, superlative hooks, crash arrows — but the + in-video verdict stays calibrated and sourced, exactly like the source channel does it + (blueprint §6). The gap between packaging and verdict is the format. + +## Phase 1 — Trending topic research → Top 5 + +Search fresh (last 7 days, prefer last 48h) across: +- News: SF Chronicle, Mercury News, SFGATE, Palo Alto Online, Almanac (Menlo Park), Redwood City + Pulse, EPA Today, Bloomberg/WSJ/Fortune housing coverage with a Bay Area angle. +- Data drops: CAR monthly report, Zillow Research releases/forecast updates, Redfin Data Center + news, Case-Shiller SF release dates, FRED updates, Census/ACS migration releases, rate moves. +- Local signals: MLS stats via the `mls-matrix-scraper` skill when available (inventory, DOM, + price cuts, sales counts for San Mateo + Santa Clara counties), notable listings (big price + cuts, sales at loss, record sales) on public portals. +- Optional: `instagram-competitor-scraper` / `youtube-scraper` for what's performing in the niche + this week. + +Score each candidate 1–10 on: local relevance (Graeham's markets), data availability (can we get +primary numbers?), hook potential (superlative/flip/authority peg available?), audience value +(does a buyer/seller/owner learn something actionable?). Present as a table: + +| # | Topic | Hook angle | Authority peg | Data source | Score | + +with a one-line recommendation of which YOU would pick and why. Wait for the user's choice. + +## Phase 2 — Deep research + verification + asset grab + +For the chosen topic: +1. Pull the primary data (download the actual report/CSV where possible; MLS pull if relevant). +2. Build the **citation table**: every stat → source name, URL, date, exact figure, how retrieved. +3. Cross-check the headline stat against a second independent source. Flag any conflicts to the + user rather than papering over them. +4. Find the **bifurcation**: where is the OPPOSITE happening nearby? (SF booming while Oakland + drops; Palo Alto holding while EPA moves). This contrast is a mandatory story beat. +5. Find 1–3 **listing autopsies**: real, current, public listings in Graeham's markets with + concrete numbers (list price vs purchase history, price cuts, DOM). Note URL + date accessed. +6. Build the **asset grab list** (per rule 3): headline screenshots, report pages, FRED charts, + public-domain aerials, b-roll needs — each with source + license status. + +## Phase 3 — The content package + +Produce, using `references/script-templates.md`: +1. **Long-form YouTube script** (8–14 min) following the 11-beat architecture, written in the + voice (verbal fingerprint in blueprint §2), with inline visual directions + `[SCREEN: ...]` / `[CHART: ...]` / `[OVERLAY: yellow "..."]` every 20–60 seconds of runtime. +2. **Reel script** (30–60s) + IG caption (CTA-first format) + cover-text spec. +3. **5 title options** (formulas in templates §C) + thumbnail spec (§D). +4. **Description + pinned comment** (§E). +5. **Chart plan**: each chart to build (metric, geography, timeframe, source), styled to + Graeham's brand — blue=declining / red=rising choropleths, dark chart cards with bold + callout numbers, yellow highlights. Build with the `dataviz` skill conventions; never + screenshot Reventure. +6. CTA adapted to Graeham's business: free home-value analysis / CMA (hand off to + `cma-generator`), "comment your zip", newsletter signup — not an app subscription. + +## Phase 4 — Production (BUILD THE VIDEO — follow `references/production-pipeline.md` exactly) + +The reel is BUILT in-session, end to end. Read `references/production-pipeline.md` BEFORE +starting production and execute its steps in order. Summary (details + IDs live there): +1. **Step 0 — location look decision (MANDATORY):** new avatar look pair for this video, new + outfit; if the video is about a specific place, capture real Street View imagery of that + place and generate the look ON LOCATION; if no specific place applies, use a notable, + recognizable Bay Area street instead. Never silently reuse or fall back to studio looks. +2. Environment capture → Higgsfield Nano Banana Pro still (identity ref bundled at + `assets/identity_ref.png` composited into the environment ref, native 4K, one per + orientation) → **STOP, show Graeham both stills, wait for his "looks like me" approval** + → upload each approved still to HeyGen via the "Upload look" tile (HD, no separate + upscale pass needed). +3. VO with timestamps (+ Whisper QC + pause profile) → three HeyGen v3 renders (4K portrait / + alpha webm / 1080p wide). +4. Evidence inserts (blur-fill, RED corner-test annotations) → fork + `assets/compositor_template.py` → SFX mix (VO-first, bundled `assets/sfx_*.wav`) → + composite → full-sweep QC (see checklist above) → master/chat/disk deliverables with the + hook-phrase filename and one-file FINAL folder. +Long-form YouTube videos still hand off to `heygen-video` / `remotion-video` / `video-editor` +with an edit sheet — the in-session pipeline above is for the daily reel. + +## Phase 5 — QC before delivery + +- Re-check every number in the final script against the citation table (fix or cut mismatches). +- Compliance pass (rule 4) and copyright pass (rule 3) on the asset list. +- Confirm the package includes: both scripts, titles, thumbnail spec, description + pinned + comment, chart plan, asset list with licenses, citation table, edit sheet. +- Deliver everything in one organized output folder; offer the next step (render now?). + +## Iteration memory + +When the user gives feedback during a video build ("more aggressive hooks", "shorter", +"always include a rent angle"), treat it as a standing rule for future runs of this skill and +offer to save it into this SKILL.md so the skill learns. diff --git a/skills/education-graeham-videos/assets/compositor_template.py b/skills/education-graeham-videos/assets/compositor_template.py new file mode 100644 index 00000000..ad6d9f49 --- /dev/null +++ b/skills/education-graeham-videos/assets/compositor_template.py @@ -0,0 +1,416 @@ +#!/usr/bin/env python3 +"""ROAD Act reel — forked from composite_rwc.py v22 (all locked patterns kept): +- 4K AV pipe, frozen per-interval punch-in crops (sharp, no lag) +- hook native framing (no zoom pumping) +- blur-fill evidence inserts, RED partial ellipses (padded, never touch text) +- RED arrow for congress.gov (dense rows — arrow-only, verified-empty zone) +- landscape wide strip under captions on inserts, gold top stroke +- fixed-line pop-in captions y=1420; caps stacks; whip-blur cuts; endcard DM "EDGE" +""" +import json, subprocess, os, math +import numpy as np +from PIL import Image, ImageDraw, ImageFont, ImageFilter + +B = os.path.dirname(os.path.abspath(__file__)) +W, H, FPS = 1080, 1920, 25 +AV = f"{B}/avatar_road3_4k.mp4" +AVW, AVH = 2160, 3840 +FD = "/usr/share/fonts/truetype/liberation" +BOLD = f"{FD}/LiberationSans-Bold.ttf" +ACCENT = (196, 162, 101); WHITE = (255,255,255) + +al = json.load(open(f"{B}/alignment_road2.json")) +chars, t0s, t1s = al["characters"], al["character_start_times_seconds"], al["character_end_times_seconds"] +words = [] +cur, ws = "", None +for c, a, b_ in zip(chars, t0s, t1s): + if c == " ": + if cur: words.append((cur, ws, prev_end)) + cur, ws = "", None + else: + if ws is None: ws = a + cur += c + prev_end = b_ +if cur: words.append((cur, ws, prev_end)) +DUR = t1s[-1] + +def find(seq_start, phrase): + for i in range(seq_start, len(words)): + if words[i][0].strip('.,:;—?!"‘’%').lower() == phrase.lower(): + return i + raise ValueError(phrase) +def wt(i): return words[i][1] +def we(i): return words[i][2] + +i_no = find(0, "no"); i_buying = find(i_no, "buying") +i_its = find(0, "It's") +i_inv = find(i_its, "Investors") +i_that = find(i_inv, "That") +i_and2 = find(i_that, "And") +i_one = find(i_and2, "One") +i_31 = find(i_one, "31"); i_office2 = find(i_31, "office") +i_another = find(i_31, "Another") +ENDCARD_START = 38.95 # right after "caught a break." — endcard holds ~5.1s under full CTA + +# ── timeline: ROAD Act — buyer-empowerment cut ────────────────────────────── +SEGS = [ + (0.0, wt(i_its), 'head', 1.6), + (wt(i_its), wt(i_inv), 'insert', 'congress'), + (wt(i_inv), wt(i_that), 'insert', 'time'), + (wt(i_that), wt(i_and2), 'head', 1.6), + (wt(i_and2), wt(i_one), 'insert', 'spotlight'), + (wt(i_one), wt(i_another), 'head', 1.6), + (wt(i_another), ENDCARD_START, 'broll', f"{B}/broll_sj/construction"), + (ENDCARD_START, DUR + 1, 'broll', f"{B}/broll_sj/downtown"), +] +CUTS = [s for (s, e, k, a) in SEGS[1:]] + +# caps stacks: (words to stack, trigger word idxs, position y) +CAPS_STACKS = [ + (["NO MORE", "BUYING HOMES."], [i_no, i_buying], 260), + (["31%", "OFFICE VACANCY"], [i_31, i_office2], 220), +] +f_caps = ImageFont.truetype(BOLD, 118) + +ins_imgs = {k: Image.open(f"{B}/ins_{k}.png").convert("RGB") + for k in ["congress","time","spotlight"]} + +# ellipses sized by CORNER TEST: every corner of every circled text line at +# <=0.86 normalized ellipse radius, so the stroke NEVER touches the circled +# text anywhere along its curve (verified qc/chk3_time.png, qc/chk4_spotlight.png) +ELL = {"time": (8, 281, 583, 460), "spotlight": (34, 234, 914, 378)} +# congress.gov rows too dense for a clean ellipse -> RED arrow only, aimed at +# "Became Public Law No: 119-101"; tail + path verified empty (qc/chk_congress2.png) +ARROW = {"congress": True, "time": False, "spotlight": False} +ARROW_TAIL = {"congress": (603, 386)} +ARROW_TIP = {"congress": (471, 479)} + +def draw_partial_ellipse(d, box, prog, width=10): + if prog <= 0: return + RED_ = (220, 40, 35) + d.arc(box, start=-80, end=-80 + 360*min(prog,1.0), fill=RED_, width=width) + d.arc([box[0]+3, box[1]+2, box[2]+3, box[3]+2], start=-80, end=-80+360*min(prog,1.0), fill=RED_, width=width-2) + +def render_insert(key, t_local, dur_seg): + img = ins_imgs[key].copy() + d = ImageDraw.Draw(img) + prog = t_local / max(dur_seg, 0.01) + # circle draws over first 0.8s — NO cursor + if key in ELL: + draw_partial_ellipse(d, list(ELL[key]), t_local/0.8) + # RED arrow: slides in from the tail anchor toward its tip target + if ARROW.get(key) and t_local > 0.9: + ta = min((t_local - 0.9) / 0.3, 1.0) + ease = 1 - (1 - ta)**3 + if key in ARROW_TIP: + tipx, tipy = ARROW_TIP[key] + else: + e = ELL[key] + tipx = e[2] - 14; tipy = (e[1] + e[3])//2 - 20 + tailx, taily = ARROW_TAIL[key] + sx = tailx + (1-ease)*90; sy = taily - (1-ease)*70 + ang = math.atan2(tipy - sy, tipx - sx) + L = math.hypot(tipx - sx, tipy - sy) + bx, by = tipx - 0.32*L*math.cos(ang), tipy - 0.32*L*math.sin(ang) + px_, py_ = -math.sin(ang), math.cos(ang) + d.line([(sx, sy), (bx, by)], fill=(220, 40, 35), width=16) + d.polygon([(tipx, tipy), + (bx + px_*26, by + py_*26), + (bx - px_*26, by - py_*26)], fill=(220, 40, 35)) + pe = 1 - (1 - min(prog, 1.0))**3 + scale = 1.03 + 0.07*pe + sw, sh = int(W*scale), int(H*scale) + z = img.resize((sw, sh), Image.LANCZOS) + return z.crop(((sw-W)//2, (sh-H)//2, (sw-W)//2+W, (sh-H)//2+H)) + +# ── captions: fixed pre-chunked lines, words pop into final positions ─────── +phrases = [] +curp = [] +for idx, (tok, a, b_) in enumerate(words): + curp.append(idx) + if tok.rstrip('"’').endswith(('.', '?', '!', '—', ':', ',')): + phrases.append(curp); curp = [] +if curp: phrases.append(curp) + +LINES = [] +for p in phrases: + for i in range(0, len(p), 3): + LINES.append(p[i:i+3]) +line_windows = [] +for li, ln in enumerate(LINES): + t0 = wt(ln[0]) + t1 = wt(LINES[li+1][0]) if li+1 < len(LINES) else we(ln[-1]) + 0.6 + line_windows.append((t0, t1)) + +f_cap = ImageFont.truetype(BOLD, 60) +f_cap_s = ImageFont.truetype(BOLD, 46) + +def draw_outlined(d, xy, text, font, fill=WHITE, ow=4): + x, y = xy + for dx in range(-ow, ow+1, 2): + for dy in range(-ow, ow+1, 2): + d.text((x+dx, y+dy), text, font=font, fill=(0,0,0)) + d.text((x, y), text, font=font, fill=fill) + +_DISP = {} # no $-substitutions needed this video (no bare-decimal prices spoken) + +def draw_captions(frame, t): + li = None + for i2, (t0, t1) in enumerate(line_windows): + if t0 <= t < t1: li = i2; break + if li is None: return + ln = LINES[li] + d = ImageDraw.Draw(frame) + toks = [_DISP.get(words[i][0], words[i][0]) for i in ln] + gaps = 16 + f = f_cap + widths = [d.textlength(tok, font=f) for tok in toks] + if sum(widths) + gaps*(len(toks)-1) > 1000: + f = f_cap_s + widths = [d.textlength(tok, font=f) for tok in toks] + total = sum(widths) + gaps*(len(toks)-1) + x = (W-total)//2; y = 1420 + for i2, w_ in zip(ln, widths): + tok, a, b_ = words[i2] + tok = _DISP.get(tok, tok) + if t < a - 0.02: + x += w_ + gaps + continue + age = t - a + if a <= t <= b_ + 0.15: + fw = f + if age < 0.10: + fw = ImageFont.truetype(BOLD, max(30, int(f.size * (0.7 + 0.3*age/0.10)))) + ww = d.textlength(tok, font=fw) + asc, desc = fw.getmetrics() + ox = x + (w_ - ww)/2 + d.rounded_rectangle([ox-12, y-8, ox+ww+12, y+asc+desc+2], radius=14, fill=ACCENT) + d.text((ox, y), tok, font=fw, fill=(0,0,0)) + else: + draw_outlined(d, (x, y), tok, f) + x += w_ + gaps + +def draw_caps_stack(frame, t): + for stack_words, idxs, y0 in CAPS_STACKS: + t_start = wt(idxs[0]); t_end = we(idxs[-1]) + 1.4 + for ct in CUTS: + if t_start < ct < t_end: t_end = ct - 0.04; break + if not (t_start <= t <= t_end): continue + d = ImageDraw.Draw(frame) + yy = y0 + for sw_, idx in zip(stack_words, idxs): + if t >= wt(idx): + age = t - wt(idx) + size = 118 if age > 0.12 else int(118*(0.6+0.4*age/0.12)) + f = ImageFont.truetype(BOLD, size) + tw = d.textlength(sw_, font=f) + draw_outlined(d, ((W-tw)/2, yy), sw_, f, ow=8) + yy += 132 + +# ── end card ──────────────────────────────────────────────────────────────── +CB = "/root/.claude/skills/carousel-builder/assets" +_logo = Image.open(f"{CB}/logo/logo_white.png").convert("RGBA") +_mont = f"{CB}/fonts/Montserrat-var.ttf"; _vibes = f"{CB}/fonts/GreatVibes-Regular.ttf" +def mont(size, weight=800): + f = ImageFont.truetype(_mont, size) + try: f.set_variation_by_axes([weight]) + except Exception: pass + return f +def _gold_gradient(w, h, light=(240,212,138), dark=(168,130,62)): + g = Image.new("RGB", (w, h)); dd = ImageDraw.Draw(g) + for yy in range(h): + t_ = yy / max(h-1, 1) + if t_ < 0.55: k=t_/0.55; c=tuple(int(light[i]+(dark[i]-light[i])*k) for i in range(3)) + else: k=(t_-0.55)/0.45; c=tuple(int(dark[i]+(light[i]-dark[i])*0.35*k) for i in range(3)) + dd.line([(0,yy),(w,yy)], fill=c) + return g +_ec = None +def build_endcard(): + global _ec + if _ec is not None: return _ec + ov = Image.new("RGBA",(W,H),(0,0,0,0)); d = ImageDraw.Draw(ov) + d.rectangle([0,0,W,H], fill=(0,0,0,150)) + lw=860; lg=_logo.resize((lw,int(_logo.height*lw/_logo.width)),Image.LANCZOS) + ov.paste(lg,((W-lw)//2,130),lg) + ya=130+lg.height+30; f_int=mont(38,500) + txt="I N T E R O · D R E # 0 1 4 6 6 8 7 6" + d.text(((W-d.textlength(txt,font=f_int))/2,ya),txt,font=f_int,fill=(235,235,235,255)) + sub="The New Housing Law"; fsz=130 + while fsz>40: + f_v=ImageFont.truetype(_vibes,fsz); tw=d.textlength(sub,font=f_v) + if tw<=980: break + fsz-=4 + mask=Image.new("L",(W,240),0) + ImageDraw.Draw(mask).text(((W-tw)/2,20),sub,font=f_v,fill=255) + grad=_gold_gradient(W,240,light=(238,205,130),dark=(178,138,66)).convert("RGBA") + ov.paste(grad,(0,900),mask) + bw,bh=760,150; bx,by=(W-bw)//2,1220 + btn=_gold_gradient(bw,bh).convert("RGBA") + m=Image.new("L",(bw,bh),0); ImageDraw.Draw(m).rounded_rectangle([0,0,bw,bh],radius=34,fill=255) + btn.putalpha(m); ov.paste(btn,(bx,by),btn) + f_btn=mont(72,800); bt='DM "EDGE"' + d=ImageDraw.Draw(ov) + d.text(((W-d.textlength(bt,font=f_btn))/2,by+34),bt,font=f_btn,fill=(10,8,4,255)) + f_oc=mont(46,700); oc="OR CALL — NUMBER IN BIO" + d.text(((W-d.textlength(oc,font=f_oc))/2,by+bh+60),oc,font=f_oc,fill=(255,255,255,255)) + _ec=ov; return ov + +_ema = {} + +# gold beat stickers on b-roll +STICKERS = {f"{B}/broll_sj/construction": "SAN JOSE, CA"} +_stick_cache = {} +def sticker_img(text): + if text not in _stick_cache: + f = ImageFont.truetype(BOLD, 64) + tmp = ImageDraw.Draw(Image.new("RGBA", (10, 10))) + tw = int(tmp.textlength(text, font=f)) + w_, h_ = tw + 84, 110 + im = Image.new("RGBA", (w_, h_), (0, 0, 0, 0)) + d = ImageDraw.Draw(im) + d.rounded_rectangle([0, 0, w_-1, h_-1], radius=20, fill=(196, 162, 101, 255), + outline=(0, 0, 0, 255), width=5) + d.text((42, 20), text, font=f, fill=(10, 8, 4, 255)) + _stick_cache[text] = im.rotate(-2, expand=True, resample=Image.BICUBIC) + return _stick_cache[text] + +def seg_at(t): + for s in SEGS: + if s[0] <= t < s[1]: return s + return SEGS[-1] + +dec = subprocess.Popen(["ffmpeg","-v","error","-i",AV,"-f","rawvideo","-pix_fmt","rgb24", + "-s",f"{AVW}x{AVH}","-r",str(FPS),"-"], stdout=subprocess.PIPE, bufsize=3*10**7) +_crop_lock = {} +ALPHA = f"{B}/avatar_road3_alpha.webm" +dec3 = subprocess.Popen(["ffmpeg","-v","error","-c:v","libvpx-vp9","-i",ALPHA, + "-f","rawvideo","-pix_fmt","rgba","-s",f"{W}x{H}","-r",str(FPS),"-"], + stdout=subprocess.PIPE, bufsize=10**7) +WIDE = f"{B}/avatar_road3_wide.mp4" +SW_, SH_ = 1080, 607 +dec4 = subprocess.Popen(["ffmpeg","-v","error","-i",WIDE, + "-f","rawvideo","-pix_fmt","rgb24","-s",f"{SW_}x{SH_}","-r",str(FPS),"-"], + stdout=subprocess.PIPE, bufsize=10**7) +_wide_current = None + +# warm grade LUT + vignette +_lut_r = np.clip(np.arange(256)*1.05 + 6, 0, 255).astype(np.uint8) +_lut_g = np.clip(np.arange(256)*1.02 + 2, 0, 255).astype(np.uint8) +_lut_b = np.clip(np.arange(256)*0.97, 0, 255).astype(np.uint8) +yy_, xx_ = np.mgrid[0:H, 0:W] +_vig = 1.0 - 0.28*(((xx_-W/2)/(W/2))**2 + ((yy_-H/2)/(H/2))**2)/2 +_vig = np.clip(_vig, 0.72, 1.0)[..., None].astype(np.float32) +def grade(frame): + arr = np.array(frame) + arr[..., 0] = _lut_r[arr[..., 0]] + arr[..., 1] = _lut_g[arr[..., 1]] + arr[..., 2] = _lut_b[arr[..., 2]] + arr = (arr.astype(np.float32) * _vig).astype(np.uint8) + return Image.fromarray(arr) + +enc = subprocess.Popen(["ffmpeg","-v","error","-y", + "-f","rawvideo","-pix_fmt","rgb24","-s",f"{W}x{H}","-r",str(FPS),"-i","-", + "-i",f"{B}/vo_mix_road4.wav", + "-map","0:v","-map","1:a", + "-c:v","libx264","-pix_fmt","yuv420p","-preset","medium","-crf","18", + "-c:a","aac","-b:a","192k","-movflags","+faststart", + f"{B}/../reel_road_v7_raw.mp4"], stdin=subprocess.PIPE) + +n = 0 +while True: + buf = dec.stdout.read(AVW*AVH*3) + if len(buf) < AVW*AVH*3: break + t = n / FPS + av_frame = Image.frombytes("RGB", (AVW,AVH), buf) + abuf = dec3.stdout.read(W*H*4) + if len(abuf) == W*H*4: + globals()['_alpha_current'] = Image.frombytes("RGBA", (W,H), abuf) + wbuf = dec4.stdout.read(SW_*SH_*3) + if len(wbuf) == SW_*SH_*3: + globals()['_wide_current'] = Image.frombytes("RGB", (SW_, SH_), wbuf) + s0, s1, kind, arg = seg_at(t) + if kind == 'head': + a = np.array(_alpha_current.getchannel("A")) + rows = (a > 40).sum(axis=1) + nz = np.where(rows > 30)[0] + top = int(nz[0]) if len(nz) else 300 + shl = np.where(rows > 0.55 * W)[0] + sh = int(shl[0]) if len(shl) else top + 600 + bandm = a[top:max(sh, top + 60), :] + colsm = (bandm > 40).sum(axis=0) + fcx = float((colsm * np.arange(W)).sum() / max(colsm.sum(), 1)) + fcy = top + 0.42 * max(sh - top, 260) + e = _ema.setdefault('face', [fcx, fcy]) + e[0] = 0.90*e[0] + 0.10*fcx; e[1] = 0.90*e[1] + 0.10*fcy + if s0 < 0.1: + zj = 1.0 # hook: pure native framing, NO zoom pumping + else: + # EXAGGERATED jump cuts (2026-08-05): wide half-body <-> tight face, + # not a subtle pump — the two framings must read as a real cut + zj = 1.15 if int((t-s0)/1.8) % 2 == 0 else 1.75 + if zj <= 1.001: + frame = av_frame.resize((W, H), Image.LANCZOS) + else: + ki = (round(s0, 2), int((t - s0) / 1.8)) + if ki not in _crop_lock: + _crop_lock[ki] = (e[0], e[1]) + fx, fy = _crop_lock[ki] + cw, ch = int(W/zj), int(H/zj) + x0 = int(min(max(fx - cw/2, 0), W - cw)) + y0 = int(min(max(fy - ch*0.42, 0), H - ch)) + frame = av_frame.crop((x0*2, y0*2, (x0+cw)*2, (y0+ch)*2)).resize((W, H), Image.LANCZOS) + elif kind == 'insert': + frame = render_insert(arg, t - s0, s1 - s0) + if _wide_current is not None: + frame.paste(_wide_current, (0, 1385)) + dd = ImageDraw.Draw(frame) + dd.rectangle([0, 1385, W, 1389], fill=ACCENT) + else: + files = sorted(os.listdir(arg)) + idx = min(int((t - s0) * FPS), len(files) - 1) + frame = Image.open(os.path.join(arg, files[idx])).convert("RGB") + st = STICKERS.get(arg) + if st is not None and (t - s0) > 0.1: + p = min((t - s0 - 0.1) / 0.35, 1.0) + sc_ = 0.5 + 0.8*p if p <= 0.8 else 1.14 - 0.14*(p-0.8)/0.2 + sim = sticker_img(st) + sw_, sh_ = int(sim.width*sc_), int(sim.height*sc_) + sim2 = sim.resize((max(sw_,1), max(sh_,1)), Image.BILINEAR) + frame.paste(sim2, ((W - sim2.width)//2, 430 - sim2.height//2), sim2) + # impact settle + for ct in CUTS: + dtc = t - ct + if 0 <= dtc < 0.25: + k = 1 - (dtc/0.25) + zi = 1.0 + 0.06*(k**2) + cw, ch = int(W/zi), int(H/zi) + frame = frame.crop(((W-cw)//2, (H-ch)//2, (W-cw)//2+cw, (H-ch)//2+ch)).resize((W, H), Image.BILINEAR) + break + frame = grade(frame) + # whip-blur transition + for ct in CUTS: + dt = t - ct + if -0.06 <= dt < 0.10: + k = 1 - abs(dt)/0.10 + frame = frame.filter(ImageFilter.GaussianBlur(radius=1)) + arr = np.array(frame) + shift = int(70*k) + if shift > 0: + arr = (arr.astype(np.uint16) + np.roll(arr, shift, axis=1) + np.roll(arr, -shift, axis=1))//3 + frame = Image.fromarray(arr.astype(np.uint8)) + break + if t < ENDCARD_START: + draw_caps_stack(frame, t) + draw_captions(frame, t) + else: + ov = build_endcard() + prog = min(1.0, (t-ENDCARD_START)/0.4) + if prog < 1.0: + ov = ov.copy(); alpha = ov.getchannel("A").point(lambda a2: int(a2*prog)); ov.putalpha(alpha) + frame = Image.alpha_composite(frame.convert("RGBA"), ov).convert("RGB") + enc.stdin.write(frame.tobytes()) + n += 1 + if n % 250 == 0: print("frame", n, flush=True) + +dec.stdout.close(); dec3.stdout.close(); enc.stdin.close(); enc.wait() +print("road v1 composited", n, "frames") diff --git a/skills/education-graeham-videos/assets/identity_ref.png b/skills/education-graeham-videos/assets/identity_ref.png new file mode 100644 index 00000000..ed4140d2 Binary files /dev/null and b/skills/education-graeham-videos/assets/identity_ref.png differ diff --git a/skills/education-graeham-videos/assets/sfx_pop.wav b/skills/education-graeham-videos/assets/sfx_pop.wav new file mode 100644 index 00000000..ea79a9bb Binary files /dev/null and b/skills/education-graeham-videos/assets/sfx_pop.wav differ diff --git a/skills/education-graeham-videos/assets/sfx_whoosh.wav b/skills/education-graeham-videos/assets/sfx_whoosh.wav new file mode 100644 index 00000000..0a63845c Binary files /dev/null and b/skills/education-graeham-videos/assets/sfx_whoosh.wav differ diff --git a/skills/education-graeham-videos/references/production-pipeline.md b/skills/education-graeham-videos/references/production-pipeline.md new file mode 100644 index 00000000..2b4227c9 --- /dev/null +++ b/skills/education-graeham-videos/references/production-pipeline.md @@ -0,0 +1,156 @@ +# Reel Production Pipeline — the ACTUAL build steps (locked 2026-08-05, avatar-look step +simplified 2026-08-12 — see Steps 0-4 note) + +This is the pipeline every reel follows after the script is approved. Phase 4 of SKILL.md +points here. Do NOT substitute the old "hand off to heygen-video" flow — this file IS the +production process. All steps run inside the session (curl + python + ffmpeg + Higgsfield MCP). + +**2026-08-12 change, per Graeham's explicit standing instruction — read before running +Steps 1-4:** avatar-look generation no longer runs a Seedance animation/training-video step. +Higgsfield produces ONE still image per orientation (Nano Banana Pro compositing the identity +reference into the real environment reference), and that still is uploaded straight to HeyGen +as a photo-based look — HeyGen does the animating, not Seedance. This replaces the old +training-speech-audio + Seedance-video + Topaz-video-upscale chain entirely. Reasoning: it's +the same end result (an HD look of Graeham at a real location) with fewer moving parts and no +lip-sync-training footage to QC before the look even exists. + +## Accounts / IDs registry (not secrets — keys are asked from Graeham per process rule) + +- HeyGen avatar group (digital twin "Graeham Watts"): `2160746aa659445e9cbfa4c02e5cf39c` +- ElevenLabs voice: `Pa3vOYQHHpLJn1Tf7hnP`, model `eleven_multilingual_v2`, + voice_settings `{stability 0.38, similarity_boost 0.8, style 0.45, speed 1.08}`, + endpoint `/v1/text-to-speech/{voice}/with-timestamps` (save alignment JSON). +- Existing trained looks (reusable free — check list before creating new ones): + `GET /v3/avatars/looks?group_id=...`: + - `selfie_walk` 48a9a8650bf74c41a8b9241710613b19 (residential street, walking) + - `rwc_theatre_way` 1be85000c5704390935b916c24bdc157 (+ `_wide` d5dfb97465c744fab193f2363684aea5) + - `sj_cityhall_hd` 9af7ef6e900b47c4afbd27ebeefc11b0 (+ `_hd_wide` 9aa44d5ed69542d291f791cbd12878b9) + — HD pair, San Jose City Hall, charcoal jacket (prefer these for SJ) + - **Note:** these were built under the OLD Seedance-video method and remain valid/reusable + as-is. Every NEW look built from 2026-08-12 forward uses the still-image method below — + don't rebuild an existing look just to match the new method. +- Identity reference for Higgsfield: `assets/identity_ref.png` (bundled in this skill). +- SFX: `assets/sfx_whoosh.wav` (airy swept-noise build), `assets/sfx_pop.wav`. +- Compositor: `assets/compositor_template.py` — fork per video, edit the marked config + (alignment file, SEGS word-finds, CAPS_STACKS, ELL/ARROW, stickers, endcard subject + + DM keyword, input/output filenames). All locked patterns live in it. + +## Step 0 — Location decision (MANDATORY, every video) + +If the video is about a specific place, the avatar MUST appear AT that place: +1. Check the looks registry above — if a pair for this location already exists, reuse it + (fresh outfit is preferred though: a new pair per video is the standing default; reuse + only when credits/time force it AND Graeham okays it). +2. Otherwise CREATE a new look pair (Steps 1–4). This is not optional and not something to + silently skip — if Higgsfield credits or keys are missing, STOP and ask Graeham. +3. **No specific place applies to this topic (locked 2026-08-12)** — e.g. a regional/ + market-wide topic with no single location: default to a real, notable, recognizable Bay + Area street or landmark corridor — East Palo Alto, Menlo Park, Redwood City, San + Francisco, or elsewhere in the Bay. Pick somewhere the audience would actually recognize + (a known downtown block, a landmark street, a recognizable waterfront), never a generic + anonymous suburban block picked just to have *something* in frame. Source a real + reference image of that location the same way as Step 1 below. + +## Step 1 — Environment reference (real place imagery) + +Google Street View capture via browser tab + screenshot: +`https://www.google.com/maps?layer=c&cbll={lat},{lng}&cbp=12,{heading},0,0,5}` +(or streetviewpixels-pa.googleapis.com thumbnail URLs). Crop to the landmark. This becomes +the Higgsfield environment reference so the composited still looks like he's really there — +use a REAL captured image here, not an AI-hallucinated approximation of the location, even +though the compositing step itself is AI-generated. + +## Steps 2-3 — Higgsfield still-frame generation (per orientation, replaces Seedance training) + +For EACH orientation (9:16 portrait, 16:9 landscape), generate ONE 4K Nano Banana Pro still — +follow the `higgsfield-video` skill's Stage 1 process (Realism Rescue Protocol, Anonymization +Strategy where the location is sensitive) with both `identity_ref.png` and the Step 1 +environment capture attached as drag-and-drop reference images: +1. Navigate to `https://higgsfield.ai/image/nano-banana-pro`, clear prior prompt/attachments. +2. Drag-and-drop both `identity_ref.png` and the environment capture into the prompt area. +3. Prompt: "the man from the first reference image (same face, same glasses) standing at the + location from the second reference image, in a NEW outfit (name it), natural candid pose, + realistic lighting matched to the environment reference" + the Realism Rescue Protocol + anchor stack for the location shot itself. +4. Set aspect (9:16 or 16:9 to match this orientation), 4K, batch 4/4. Generate, pick the + hero variant per the usual narrate-and-recommend process, download to Downloads. +5. Repeat for the other orientation using the SAME outfit/identity framing for continuity + between the pair — only the aspect crop/composition changes. + +**MANDATORY approval gate (locked 2026-08-12, per Graeham's explicit standing instruction) — +before either still touches HeyGen:** present both hero stills to Graeham and get an explicit +"yes, that looks like me" before uploading either one as a look. This is a hard stop, not a +courtesy — do not proceed to Step 4 on an assumed approval, and do not upload a still Graeham +hasn't seen. If he says it doesn't look like him, regenerate (adjust the prompt/reference +weighting, try a different variant from the same batch, or re-batch) and show him again before +moving on. This mirrors the same "wait for approval" discipline already used for topic picks +(Phase 1) and script review — a wrong face is a much more expensive mistake to catch after a +full render than before one. + +## Step 4 — HeyGen look upload (HD, no separate upscale pass needed, confirmed 2026-08-12) + +Higgsfield's Nano Banana Pro output is native 4K, so unlike the old Seedance-video path there +is no Topaz upscale step before creating the look — the still is already HD. Upload mechanism +confirmed directly from Graeham's HeyGen account (screenshot, 2026-08-12): the Photo Avatar / +Looks library has an **"Upload look"** tile right next to "Design with AI" in the look grid — +click it and upload the approved still directly, no separate API call needed. Since this is a +UI action, drive it via Claude in Chrome: +1. Navigate to HeyGen's Avatar/Photo Avatar looks page (under the existing avatar group). +2. Click the **Upload look** tile. +3. Upload the Graeham-approved hero still for this orientation (same file-access pattern as + Higgsfield uploads — if the native OS file picker blocks Chrome automation, use the + drag-and-drop-from-Downloads workaround, same as the `higgsfield-video` skill's documented + fallback). +4. Name the resulting look `_hd[_wide]` to match the existing naming convention, repeat + for the other orientation. +Wait for the look to finish processing before moving to Step 5 (VO) / Step 6 (renders). Confirm +the exact processing-time/ready-state indicator on the first live run and note it here once +seen — not yet observed under this flow. + +## Step 5 — VO (rules in SKILL.md 0g apply: no em-dashes, decimals not $, speed 1.08) + +`/with-timestamps` → save mp3 + alignment JSON. MANDATORY Whisper QC (full track, base +model): transcript must be word-perfect, no extra syllables. Pause-profile check: no +mid-sentence gap > 0.5s, sentence gaps <= ~0.8s (surgically cut longer ones + shift the +alignment times; mute any vocal artifact windows with fades). + +## Step 6 — Three HeyGen renders (v3 endpoint ONLY) + +Upload VO mp3 as asset, then `POST /v3/videos` three times: +1. Portrait look, resolution `"4k"`, 9:16 (the main AV — punch-ins downscale from it) +2. Portrait look, 1080p, 9:16, `output_format: "webm", remove_background: true` (alpha) +3. Wide look, 1080p, 16:9 (the split-screen lower strip) +4K jobs sometimes fail with a transient INTERNAL_SERVER_ERROR — just resubmit. +Verify every download with `ffmpeg -v error -i X -f null -` (truncated curl at the 2-min +Bash timeout = corrupt NAL units; re-download, use longer timeouts). + +## Step 7 — Evidence inserts + +Blur-fill background (page scaled to 1920 h, GaussianBlur 34, brightness 0.42), page at +~1.15x with per-page x0 so full headlines fit, paste at y=120. RED annotations per SKILL.md +QC 6a (corner test <= 0.86; arrow-instead-of-circle on dense row pages). Render check +overlays and VIEW them before compositing. + +## Step 8 — SFX mix (VO-first architecture — never loudnorm the finished mix) + +1. loudnorm the VO ALONE to I=-14 (dynamic ok on speech). +2. Add whoosh at cuts (gain such that gap RMS lands ~9-10 dB under speech RMS — 0.30 with + the bundled whoosh), pop 0.32 on caps-stack words. +3. Final mux from wav with `alimiter=limit=0.79:level=false` (level=false is critical — + default level=true normalizes the ceiling right back up). QC peak −1 to −3 dB. + +## Step 9 — Composite + +Fork `assets/compositor_template.py`. Key locked patterns already inside: 4K AV pipe with +frozen per-interval punch-ins (LANCZOS, crop coords ×2), hook native zj=1.0, EXAGGERATED +jump cuts 1.15 <-> 1.75, insert zoom ease, wide strip paste (0,1385) + gold stroke, +fixed-line pop-in captions y=1420, caps stacks, whip-blur + impact settle at cuts, warm +grade + vignette, endcard (logo/DRE/cursive subject/DM keyword button). B-roll: conform to +25fps with minterpolate (mci/aobmc/vsbmc) BEFORE frame extraction; boomerang-extend if the +segment outruns the clip. + +## Step 10 — Full-sweep QC (SKILL.md checklist) → deliver + +Master CRF 18 (kept in session) → chat copy CRF ~23 (≤30 MiB) → disk copy two-pass +(≤20 MiB) → `Downloads\-\FINAL\.mp4` — FINAL holds exactly ONE +file; prior versions move to `old\`. Skill updates from the day's feedback ship the same day. diff --git a/skills/education-graeham-videos/references/reventure-blueprint.md b/skills/education-graeham-videos/references/reventure-blueprint.md new file mode 100644 index 00000000..5aea3e98 --- /dev/null +++ b/skills/education-graeham-videos/references/reventure-blueprint.md @@ -0,0 +1,192 @@ +# The Reventure Blueprint — Full Style Codebook + +Source analysis: 5 full YouTube videos from @ReventureConsulting (watched start to finish — full +transcripts + frame-by-frame storyboards) and 5 Instagram reels from @reventurehousing, July 2026. +Videos analyzed: "Fannie Mae warns of MASS bankruptcies" (399K views, field format), "Michael Burry +issues FINAL warning" (347K), "Bank of America drops bombshell MIGRATION report" (247K), +"Zillow releases 2026 forecast (Avoid these cities)" (153K), "California's about to implode" (103K). +Channel: 680K subs, 219M total views, ~1,320 videos. + +## 1. The three formats + +**A. Desk format (the core — most videos).** Webcam talking head at an office desk. Beige/olive +wall, framed US map behind him, window light from the right. Alternates between full-frame talking +head and full-screen screen-shares (browser recordings) with a small picture-in-picture webcam +bubble pinned bottom-left (~15% of frame width). 15–17 minutes. + +**B. Field format (high performers).** Outdoor selfie-stick walking tour at the actual location the +story is about (e.g. walking downtown Nashville past vacant apartment towers). Sunglasses, lav/handheld +mic visible, walking and talking. Data inserts cut in full-screen every 30–90 seconds. The physical +place IS the b-roll. These videos over-perform (the two field/map-heavy videos were the two biggest). + +**C. Reels (30–60s).** Three sub-formats: +1. **Map tour** — screen recording of a zip-code choropleth map, cursor hovering to pop data + tooltips, title card overlaid, narrated. +2. **Field clip** — walking at a specific property ("Nashville seller Taking a loss 🏠"), often + holding sunglasses, direct to camera, energetic. +3. **Greenscreen-over-map** — his cutout head/shoulders at the bottom of frame over a full-screen + map or chart. + +## 2. Long-form script architecture (the 11 beats) + +Word count 2,850–3,330 words for 15–17 min → ~190–200 wpm (fast, urgent, no dead air). + +1. **Cold open (0:00–0:30).** Superlative claim + authority citation in the first two sentences. + No intro, no branding, no "welcome back." Patterns observed: + - "The biggest apartment downturn in US history is happening right now and it's causing a local housing market crash." + - "Michael Burry, the person who predicted the 2008 housing crash, just released another warning." + - "Zillow has once again cut their housing market forecast... when they lower their forecast, it means they're seeing something worth paying attention to." + Always names the authority (Fannie Mae, Zillow, BofA, Burry, CAR) within 30 seconds. + Often ends the cold open with a promise: "In this video, I'm going to lay out a case for..." + +2. **First evidence on screen by 0:15–0:30.** A chart, report screenshot, or headline appears + almost immediately. The claim is never allowed to float unsupported for more than ~20 seconds. + +3. **The zoom-in loop (repeats 4–6×, the body of the video).** Each cycle: + claim → chart/table on screen → drill to a named metro/zip → ONE specific listing with exact + numbers ("two-bed two-bath, 1,250 sq ft, listed $1.08M, owner bought for $931K in 2023, trying + to make a 16% profit in 3 years") → what it means for the viewer. + The single-listing autopsy is his signature move: real address-level Zillow screenshots with + the price-cut line highlighted in yellow, purchase history shown, loss computed out loud. + +4. **The myth-bust.** At least once per video he frames realtors/mainstream as wrong: + "Many people in California real estate told you these declines were impossible... realtors and + mortgage brokers said the market would never go down. It is now officially going down." + Also: "Here's the thing I think everyone is missing..." / "no one else has caught on to this." + +5. **Mid-video tease + soft plug.** Around 40%: "In one moment, I'm going to do a deeper dive + into [specific cities] and reveal my forecasts..." followed by a credibility claim + ("our forecast was six times more accurate than Zillow's"). + +6. **Speculation segment (labeled).** He literally puts "*Speculation Alert*" on screen in yellow + text before giving his 5–10 year macro thesis. The labeling builds trust — data vs opinion is + explicitly separated. + +7. **Triangulation.** "We have three different independent data sources — Census migration, + realtor.com inventory, Zillow values — all corroborating the same story. That's the proof that + something is actually going on here." On-screen yellow text: "3 Different Independent Data Sources". + +8. **Personal receipts.** First-person stories with evidence: "I purchased a property $160,000 + below the 2023 price — let me show you the actual text conversation with the realtor." + Shows the real text thread on screen, walks through the negotiation move by move. + +9. **Practical coaching.** Direct second-person instructions: how to push back on a rent increase + with comps, why below-list offers need a pre-approval + 20-25% down, what rising DOM means + ("when you see that DOM go up in your neighborhood, that's a signal it's getting friendlier + for you as a buyer"). + +10. **The balanced verdict.** He always pulls back from pure doom before closing: + "Is there going to be a statewide crash? No, I don't actually think so. The reason is we simply + don't have enough inventory. However..." The doom is in the packaging; the analysis lands + calibrated. This is what lets him keep credibility while running crash thumbnails. + +11. **CTA (final 60–90s).** Holds his phone up showing the app on camera, screen-shares the + pricing page, frames cost against the purchase ("only 0.005% of the cost of buying a house + in California"), gives exact instructions ("download the app, search your zip code, upgrade to + see the forecast"). Sometimes signs off: "Until next time, everyone, this is Nick from + Reventure signing off." + +### Verbal fingerprint +- Addresses: "everyone", "folks", "you guys" constantly (2–4× per minute). +- "Take a look at this" / "I want you to take a look at" before every key visual. +- "Now," as a pivot word to start new sections. +- "One has to wonder / one has to suspect" for speculation in a deniable voice. +- "What's wild is..." / "Here's the thing..." / "And oh, by the way..." for emphasis. +- Rhetorical question chains: "How sustainable is that business? How diversified is that business?" +- Numbers are always spoken precisely: "down 21% from the middle of 2022", "$8,700 a month, + which is close to $100,000 a year." +- Repeats the key stat twice in slightly different phrasing (retention trick). +- Live self-correction left in the edit: "...which actually says 19 billion here. So, I stand + corrected." (authenticity signal — do not sand these out). + +## 3. Visual grammar + +**Insert cadence.** In desk format, a new visual appears every 20–60 seconds. Talking-head +stretches run longer (2–4 min) only during story/coaching segments. In field format, inserts every +30–90 seconds; the walking shot itself carries visual motion between them. + +**Evidence is shown raw, not re-designed.** News articles appear as actual screenshots with a +YELLOW HIGHLIGHTER over key sentences and a yellow box around the headline. Zillow listings appear +as actual listing pages with the price-cut row highlighted yellow. FRED charts, PDF reports, +spreadsheets — all shown as-is with the browser visible. The rawness IS the credibility. + +**Chart/data styles observed:** +- Choropleth zip/county/state maps: BLUE = declining/cheap, RED/ORANGE = rising/expensive; black + data tooltip boxes with white text; % labels directly on regions. +- Dark navy chart cards with yellow/orange/blue data lines and big bold red/orange number callouts + ("9.8%", "-5.7%") — used for the channel's own chart graphics. +- Hand-drawn red annotations: jagged arrows, circles, period labels ("2009-2010", "1930-1938"). +- Tables with the key row highlighted yellow. + +**On-video text overlays:** yellow bold italic with black outline (chunky comic/marker feel), +used as section labels beside his head: "*Speculation Alert*", "Census Domestic Migration", +"3 Different Independent Data Sources", "Ohio & Illinois", "www.reventure.app". Occasionally white. +ALL-CAPS yellow for field-format location stamps ("NASHVILLE"). + +**B-roll:** drone city-skyline shots as section breathers (desk format); the walk itself +(field format). Factory/construction aerials when discussing the economy. + +**CTA visuals:** phone held to camera with the app open; pricing-page screen-share; yellow URL +text overlay bottom of frame. + +## 4. Packaging (titles + thumbnails) + +**Title formulas (from 30 recent videos, 70K–399K views):** +- `[Authority] + [drops/issues/reports/warns] + [BOMBSHELL/FINAL warning/shocking U-TURN]. ([specific number or sub-claim])` + - "Fannie Mae warns of MASS bankruptcies. (80% migration collapse)" + - "J.P. Morgan drops BOMBSHELL report. America's map just flipped." + - "Zillow reports shocking housing U-TURN. 40% losses emerge." +- `The Biggest [X] since [year/superlative] + [is happening now / just got worse]` +- `[Place]'s about to implode ([year] [audience] warning)` +- `[Quote in parens for social proof]`: ("it's like 1929 all over") +- One dramatic ALL-CAPS word per title, never more than two. A specific number in most. + +**Thumbnail formula (consistent across all):** +- His face on the left third, direct eye contact, serious/concerned expression (no smile). +- 3–6 words max, black extra-bold condensed caps on a YELLOW bar, or yellow caps with heavy black + outline: "GET OUT NOW / 1929 ALL OVER", "SELL NOW", "IT CAME OUT OF NOWHERE", "THIS CHANGES EVERYTHING". +- A red jagged crash arrow. +- The authority's actual logo (Zillow, Redfin, Bank of America) as trust anchor. +- Optional: US/state map with red X marks; red badge with a % ("-78%"); second face if quoting a + famous investor. + +**Description:** first line = hook + link ("California Housing Is Breaking… check your ZIP code +here [link]"), then a paragraph summary. + +**Pinned comment = engagement question:** "California demand just collapsed to near record lows… +but prices barely moved. What's your take: Is this a delayed crash… or does the market stay stuck +like this for years?" + +## 5. Reel architecture + +- **Length:** 30–60s. Single idea, single place, single number. +- **Cover text:** white rounded rectangle box, bold dark text + exactly one emoji: + "Texas' New 2026 Housing Market Correction 📉", "Nashville seller Taking a loss 🏠". Red badge + variant for money figures: "A DROP OF $150,000". Blue badge for list countdowns ("#10"). +- **Captions:** word-by-word karaoke captions, white bold text with a purple/violet rounded + highlight on the word being spoken. +- **Audio hook = first line, always one of:** a dark truth ("There's a dark side to buying real + estate. And that's losing money."), a question ("What's going on in Atlanta?"), or a flip/reversal + ("The housing correction is no longer just in Texas and Florida."). +- **IG caption formula (every reel):** + 1. CTA + link as the FIRST line (so it's visible before "…more") + 2. Hook restated in one line + 3. 2–4 short paragraphs, one specific number each + 4. "This is happening because…" (mechanism paragraph) + 5. Myth-bust or open question ("Will sellers regain the leverage, or will this market keep dropping?") + 6. Closing CTA + link +- **Comments get engaged** and comment sections are seeded with a debate question. + +## 6. The editorial engine (why it works) + +- **Fear packaging, calibrated content.** Titles/thumbnails promise collapse; the video itself + delivers sourced data and usually a "no crash, but…" verdict. The gap between packaging and + content is deliberate and consistent. +- **Authority borrowing.** Every video rides a big brand's credibility (Zillow, Fannie Mae, BofA, + Burry, WSJ) — their report is the news peg; his data is the analysis layer. +- **Specificity as trust.** Exact dollars, exact zips, exact percentages, real listings, real texts. +- **Both-sides bifurcation.** He always shows where prices are RISING too (SF booming while + Oakland crashes). This inoculates against "doom guy" dismissals and doubles the addressable + audience (buyers AND sellers). +- **The viewer is always the hero:** every segment lands on "what this means for you as a buyer / + seller / investor" with a concrete action. diff --git a/skills/education-graeham-videos/references/script-templates.md b/skills/education-graeham-videos/references/script-templates.md new file mode 100644 index 00000000..90a7bf05 --- /dev/null +++ b/skills/education-graeham-videos/references/script-templates.md @@ -0,0 +1,148 @@ +# Script Templates — Education Graeham Videos + +Fill-in structures derived beat-for-beat from the Reventure analysis, localized for Graeham's +markets (East Palo Alto, Palo Alto, Menlo Park, Redwood City, San Mateo County, Santa Clara +County / San Jose, San Francisco, East Bay). + +## A. Long-form YouTube script (8–14 min, 1,600–2,800 words, ~190 wpm) + +Every [BRACKET] must be filled with a verified fact from the citation table before the script is +considered done. No placeholder survives to production. + +``` +[COLD OPEN — 0:00-0:30] +The [superlative: biggest/fastest/first] [phenomenon] in [the Bay Area / San Mateo County / +NUMBER years] is happening right now — and [consequence for viewer's money]. +According to [AUTHORITY: CAR / Zillow / Redfin / MLS data / Census / Chronicle], [headline stat +with exact number]. In this video I'm going to show you [promise: exactly which zip codes / +what this means for your home's value / where the discounts are]. + +[EVIDENCE #1 — by 0:30. Authority's chart/report ON SCREEN, key line highlighted yellow] +Take a look at this, everyone. [Read the stat precisely. Repeat it a second way.] + +[ZOOM-IN LOOP — repeat 3-5×, ~90-150s each] + Claim: [regional claim] + Chart: [branded chart: metric, geography, time range] + Drill: Now let's zoom into [city/zip]. [zip-level number vs neighboring zip — the bifurcation] + Listing autopsy: Like on this listing in [city]. [Beds/baths, sq ft, list price, purchase + history: "the owner bought for $X in YEAR and is now asking $Y — that's a Z% haircut."] + Viewer turn: If you're a [buyer/seller/owner] in [area], this means [concrete implication]. + +[MYTH-BUST — once, mid-video] +A lot of people will tell you [conventional wisdom]. [Agents/pundits] said [X] was impossible. +But the data says otherwise: [stat]. / OR self-aware variant: "I'm a realtor, and I'll tell you +what most agents won't..." + +[MID-TEASE + CREDIBILITY] +In one moment, I'm going to show you [specific promise — the zip-by-zip breakdown / the three +neighborhoods where this flips]. [Credibility receipt: track record, MLS access, deal example.] + +[*SPECULATION ALERT* — labeled on screen in yellow] +Now, I'm going to speculate a bit here about where this is heading, because I think [macro thesis: +AI boom / migration / office return / Prop 19 effects / rates]. One has to wonder [rhetorical +question]. [2-3 paragraph thesis — clearly separated from the data.] + +[TRIANGULATION] +And here's what convinces me this is real: we have [2-3] different independent data sources — +[MLS data] on one hand, [Zillow/Redfin] on the other, and [Census/CAR] — all telling the same +story. When independent sources agree, that's not a blip. + +[PERSONAL RECEIPTS — when available] +[First-person deal story with numbers: "I just closed a listing in EPA where..." / +"a buyer I represented offered $X below list and here's what happened..." Show artifacts if +possible (redact names/PII).] + +[COACHING] +So what do you actually do with this? If you're a buyer: [action — DOM signal, below-list offer +mechanics, pre-approval + down payment]. If you're a seller: [action — pricing strategy, timing]. +If you're an owner: [action]. + +[BALANCED VERDICT] +So is [dramatic scenario from the title] actually going to happen? [Honest, calibrated answer — +usually "not exactly, but..."] However, [the real risk that IS in the data]. + +[CTA — final 45-60s] +If you want to know what this means for YOUR street — not the Bay Area average, YOUR zip code — +[CTA: comment your zip / DM me / free home value analysis / newsletter]. I pull this data +directly from the MLS, the same system agents use. [Sign-off:] Until next time — this is Graeham +with Watts Up in the Bay… signing off. +``` + +### Voice rules (write the script in this voice) +- Address the viewer: "everyone", "folks", "you guys" — 2–3× per minute, never zero. +- "Take a look at this" before every chart. +- "Now," to pivot sections. +- "One has to wonder…" for speculation; "Here's the thing everyone is missing…" for the contrarian turn. +- State every number precisely, then restate it in human terms ("$8,700 a month — that's over + $100,000 a year just to hold the mortgage"). +- Rhetorical question chains for emphasis (2-3 in a row, then answer). +- Short sentences at emotional peaks. No corporate hedging. No "in today's video". +- Doom in the packaging, calibration in the verdict. + +## B. Reel script (30–60s, 90–160 words) + +``` +[HOOK — first 2 seconds, one of:] + Dark truth: "There's a dark side to [buying in Palo Alto / the Peninsula market]. And that's [X]." + Question: "What's going on in [East Palo Alto]?" + Flip: "The [correction/boom] is no longer just in [X]. It just hit [Y]." + +[THE ONE NUMBER — with the map/listing on screen] +"[Specific stat]. Take a look at this." + +[MECHANISM] "This is happening because [one-sentence cause]." + +[MYTH-BUST or STAKES] "People told you [X] was impossible in the Bay Area. / That's a $[X] swing +on a typical [city] home." + +[OPEN QUESTION or CTA] "Will [A], or will [B]? / Comment your zip code and I'll run the numbers." +``` + +### Reel production spec +- Cover text: white rounded box, bold dark text + exactly ONE emoji. Red badge variant for a + dollar figure ("A DROP OF $150,000"), blue badge for countdown lists ("#3"). +- Karaoke captions: white bold, per-word highlight in Graeham's brand accent color (Reventure + uses purple; use Watts brand color). +- Sub-formats: (1) map screen-recording with cursor + tooltip, (2) on-location walk at the actual + property/street, (3) avatar head-over-map. + +### IG caption template +``` +[CTA + link — first line, before the fold] +[Hook one-liner restated] + +[2-4 short paragraphs, one exact number each] + +This is happening because [mechanism]. + +[Myth-bust or open question] + +[Closing CTA + link] +``` + +## C. Title generator (produce 5, user picks) + +Formulas (one dramatic ALL-CAPS word max, one specific number where possible): +1. `[Authority] + drops/issues/reports + [BOMBSHELL/warning/U-TURN] + ([number sub-claim])` + → "Zillow just flipped on San Jose. (12% forecast cut)" +2. `The Biggest [X] since [year] + is happening in [place]` + → "The biggest inventory spike since 2008 is hitting Silicon Valley" +3. `[Place]'s about to [flip/break/implode] ([year] [buyer/seller] warning)` +4. `[Shocking local fact]. [Consequence sentence.]` + → "Houses in East Palo Alto are disappearing. Here's who's buying them." +5. `Quote format: [Authority] says "[quote]"` + +## D. Thumbnail spec (deliver as build instructions or finished PNG) + +- Graeham's face left third, direct eye contact, serious expression (use approved headshot set). +- 3–6 words, black extra-bold condensed caps on a YELLOW bar (or yellow caps, heavy black outline). +- Red jagged crash/spike arrow over a recognizable local backdrop (Peninsula aerial, Painted + Ladies, EPA street, University Ave) or a CA/Bay map with red marks. +- The cited authority's logo (Zillow/Redfin/CAR) when the video rides their report. +- Optional red badge with the key % or $ figure. + +## E. Description + pinned comment + +- Description line 1: hook + link ("[Market] is breaking… get your street's numbers here → [link]"). +- Pinned comment: a genuine either/or debate question ("What's your take: delayed correction in + [city], or does the market stay stuck like this for years?"). diff --git a/skills/farming-postcard/SKILL.md b/skills/farming-postcard/SKILL.md new file mode 100644 index 00000000..a4b136e0 --- /dev/null +++ b/skills/farming-postcard/SKILL.md @@ -0,0 +1,286 @@ +--- +name: farming-postcard +description: "Generate print-ready 6x4 farming postcards for Graeham Watts in his locked brand system (gold + black, chevron pattern, Compass + Graeham Watts lockup). Use ANY time the user mentions farming postcard, direct mail postcard, EPA postcard, neighborhood postcard, mailer, mail piece, Wise Pelican, Universal Mail Works, Corefact, ProspectsPLUS." +--- + +# Farming Postcard Skill + +## Purpose + +Generate Graeham Watts farming postcards that match the locked brand system used across past mailings, with one-shot per-card customization. Every card ships with the same visual continuity (gold border, chevron pattern, identical bottom contact lockup) so the audience recognizes the sender instantly. Only the **hook + back copy + QR target** changes per card. + +**Why the lockdown matters:** Direct mail works on repetition. Recognition before reading is the entire point. The bottom contact block, logo lockup, color palette, and disclaimer placement are NEVER negotiable — they are the brand signature. + +## Three workflows + +| Workflow | Trigger | Output | +|---|---|---| +| **A — Interactive create** | User says "make a postcard for [date]" | HTML preview + print-ready PDF in Downloads | +| **B — Scheduled preview** | Cron: 8th + 24th of month at 8am | Email to graehamwatts@gmail.com with 3-5 hook options + cache to option-cache.md | +| **C — Recall emailed options** | User says "pull up what you emailed me" | Read cached options, present in chat, user picks one → run Workflow A on the choice | + +--- + +## Workflow A — Interactive create (user requests a postcard now) + +### Step 1 — Gather inputs + +Ask the following IN ORDER, using `AskUserQuestion` where multiple-choice makes sense. + +**Q1: Mail date?** (e.g., "06/01/26"). Drives filename `Farming_Postcard_EPA_[MM_DD_YY].pdf` and auto-suggests an archetype. + +**Q2: Audience?** Farm only / Past clients only / Both (generic — most common). + +**Q3: Hook angle?** Offer the 10 archetypes from `references/headline-library.md`: +1. **Equity** (pride/curiosity) +2. **Buyer-tagged** (scarcity) +3. **Anti-Zillow buyer pool** (scarcity + anti-portal) +4. **AI search invisibility** (FOMO + tech) +5. **Anti-Zestimate** (anti-algorithm) +6. **Neighbor envy** (curiosity + social proof) +7. **Low-inventory timing** (supply-side scarcity) +8. **Quiet sale / no-hassle** (privacy + ease) +9. **Prop 19 tax transfer** (education + unlock) +10. **Local proof / just sold** (social proof + recency) +11. **Custom** (user-provided) + +**MANDATORY before offering or accepting any archetype:** run the **4-axis differentiation check** from headline-library.md "Differentiation rules" against the Repetition tracker table — archetype cooldown (3 cards), CTA-destination cooldown (2 cards), villain cooldown (2 cards Zillow-free after any Zillow-bashing card), core-claim cooldown (4 cards). If the user explicitly asks for an archetype that fails, warn them which axis it fails and why, then follow their call. Graeham flagged repetitive output on 2026-06-11 — this check exists because rotating archetype NAMES alone is not differentiation. + +**Cadence default suggestions:** +- 1st of month → education/pride angles (Equity, Prop 19, Neighbor envy) +- 15th of month → scarcity/timing angles (Buyer-tagged, Low-inventory, Quiet sale, Local proof) + +**Q4: CTA type?** Drives QR target via `references/cta-router.md`: +- Home valuation / Testimonials / Free report (market) / Free report (AI score) / Thinking of selling / Off-market buyers / Call-text Graeham / Custom URL + +If the CTA type's URL is `[NOT SET]` in `cta-router.md`, ask user once, then CACHE it by editing the file. Never ask twice. + +**Q5: Live data to bake in?** Optional. If user provides a number, FLAG: "Verify before print — never fabricate." + +### Step 2 — Generate headline + +Use chosen archetype from `references/headline-library.md`. Don't copy past headlines verbatim — pull the **lever** and rebuild fresh language using the remix patterns there. + +**Gold-highlight rules:** 1-3 words max per headline. Solid gold fill for short emphasized phrases; gold underline for action verbs. + +### Step 3 — Build back copy + +Structure (always): +1. **Headline** (Anton ~26pt) — gold-box-wrapped key word +2. **Italic body** (Inter 10pt, max 3 sentences) — proof + differentiation +3. **CTA line** (Anton, gold) — what they get +4. **QR + scan label** — "Scan to see your [thing] today" + +### Step 4 — Render + +Substitute slots in `templates/postcard-template.html`: +- `{{MAIL_DATE}}`, `{{ARCHETYPE}}`, `{{FRONT_HEADLINE_HTML}}`, `{{FRONT_SUBLINE_HTML}}`, `{{BACK_HEADLINE_HTML}}`, `{{BACK_BODY_HTML}}`, `{{BACK_CTA_LINE}}`, `{{QR_SCAN_LABEL}}`, `{{QR_IMAGE_SRC}}`, `{{FRONT_PHOTO_SRC}}`, `{{BACK_PHOTO_SRC}}` + +**LOCKED — never substitute** (see `references/design-tokens.md`): All design tokens, the bottom contact lockup, gold border, chevron pattern, vertical disclaimer. + +Save HTML preview to: `C:\Users\Admin\Downloads\Farming_Postcard_[MM_DD_YY]_PREVIEW.html` + +### Step 5 — Generate print-ready PDF + +Render HTML to PDF at 6.25" × 4.25" (includes 0.125" bleed each side) at 300 DPI using Playwright (see `references/print-specs.md` for the script). Output to `C:\Users\Admin\Downloads\Farming_Postcard_[MM_DD_YY]_PRINT.pdf`. + +### Step 6 — Present + log + +1. `mcp__cowork__present_files` to surface HTML + PDF +2. Append new card row to "Cards shipped" table in `headline-library.md` +3. Offer GitHub sync if skill itself was edited + +### Step 7 — Auto-publish to online archive (MANDATORY) + +Every new card must be added to the public-facing archive at `Graehamwatts/online-content/farming-postcards/` so Graeham (and Claude in future sessions) can see the running history. + +**Steps:** +1. Clone `Graehamwatts/online-content` to /tmp +2. Copy the PDF to `farming-postcards/pdfs/[YYYY-MM-DD]-[archetype-slug].pdf` +3. Generate a thumbnail (page 1, 100 DPI JPG via `pdftoppm`) to `farming-postcards/thumbnails/` +4. If the card is preview-only (no PDF yet), copy the HTML preview to `farming-postcards/[YYYY-MM-DD]-[archetype-slug]-preview.html` +5. Append the new card entry to `farming-postcards/archive.json` under `cards[]` +6. Regenerate `farming-postcards/index.html` from `archive.json` — add a new card to the grid at the top +7. Commit: `Add [YYYY-MM-DD] [archetype] postcard to archive` and push to main + +**Live dashboard URL:** https://graehamwatts.github.io/online-content/farming-postcards/ + +**Archive entry format (matches existing archive.json schema):** +```json +{ + "id": "YYYY-MM-DD-archetype-slug", + "mail_date": "YYYY-MM-DD", + "archetype": "[Equity / Buyer-tagged / etc.]", + "lever": "[psychological lever]", + "front_headline": "[plain text]", + "back_headline": "[plain text]", + "cta_type": "[CTA type from cta-router]", + "cta_line": "[gold CTA tagline]", + "audience": "[Farm / Past clients / Both]", + "pdf": "pdfs/YYYY-MM-DD-archetype-slug.pdf", + "thumbnail": "thumbnails/YYYY-MM-DD-archetype-slug-1.jpg", + "notes": "[one-line note about the card]" +} +``` + +--- + +## Workflow B — Scheduled preview (cron-triggered, no user in the loop) + +**Trigger:** Scheduled task fires on the 8th of each month (7 days before the 15th drop) and on the 24th (7 days before the 1st of next month) at 8am. + +### Step B1 — Calculate target mail date + +- If today is the 8th → target date is the 15th of this month +- If today is the 24th → target date is the 1st of NEXT month + +### Step B2 — Pick 3-5 archetype options + +1. Read `references/headline-library.md` — BOTH the "Cards shipped" table AND the "Repetition tracker" table +2. Run the **4-axis differentiation check** (see "Differentiation rules" in headline-library.md) on every candidate: + - Archetype not used in last 3 cards + - CTA destination differs from the previous 2 cards + - If recent cards named Zillow/algorithms as villain, candidate must be villain-free (2-card cooldown) + - Core claim not repeated within 4 cards (e.g., "I have buyers waiting" is the SAME claim whether the archetype is Buyer-tagged or Anti-Zillow) +3. Pick 3-5 candidates that pass ALL FOUR axes, honoring cadence: + - Target = 1st → bias toward education/pride angles (Equity, Prop 19, Neighbor envy) + - Target = 15th → bias toward scarcity/timing angles (Buyer-tagged, Low-inventory, Quiet sale, Local proof) +4. The library has 10 archetypes (#7-10 added 2026-06-11 after Graeham flagged repetition) — never offer an option that fails the check just to fill a slot. + +### Step B3 — Generate hook options + +For EACH archetype picked, generate: +- A fresh headline (using the remix patterns, not copy/paste) +- One-line "why this works" rationale +- Suggested back-headline + CTA line +- Suggested CTA type (drives QR target) + +Format as a clean comparison table. + +### Step B4 — Send email to Graeham + Peter (REAL SEND via SMTP) + +**Recipients (LOCKED — always both):** +- graehamwatts@gmail.com (Graeham) +- graehamwattsvideo@gmail.com (Peter, also goes by Jason) + +**Method:** Use the SMTP send script (NOT Gmail MCP draft — the MCP only supports drafts which Graeham won't see). The script reads his Gmail App Password from `C:\Users\Graeham Watts\Documents\Skills LLMS\Claude\Skills\gmail-app-password.txt`. + +```bash +python "C:\Users\Graeham Watts\Documents\Skills LLMS\Claude\Skills\skills\farming-postcard\scripts\send_options_email.py" \ + "" \ + "Postcard options for [TARGET_MAIL_DATE] — pick one by [PICK_DEADLINE]" \ + "" +``` + +**HTML body:** Build by substituting slots in `templates/options-email-template.html` and `templates/option-card.html`. Both templates use the locked brand system (white postcard look, gold left border, cream option panels, Anton headlines, COMPASS + Graeham Watts lockup at bottom). DO NOT use the dark dashboard style — that's not on-brand. + +**Subject format:** `Postcard options for [Date] — pick one by [Deadline]` +Example: `Postcard options for June 15 — pick one by June 12` + +**Plaintext fallback:** Always include a plaintext version for clients that don't render HTML. Short bullet list of options. + +**Error handling:** If `gmail-app-password.txt` is missing or returns 401 (invalid/expired password), abort the send, log to schedule-log.md as `email_status=failed_no_credential`, and create a fallback Gmail draft as a safety net so the options aren't lost. + +### Step B5 — Cache options + +Append to `references/option-cache.md`: + +```markdown +## [TARGET_DATE] (emailed [SENT_DATE]) +Status: pending pick + +### Option 1 — [Archetype] +Front headline: [text with markup] +Back headline: [text] +CTA: [type] → [URL] +Why: [rationale] + +### Option 2 — [Archetype] +... + +[etc.] +``` + +### Step B6 — Confirm to logs + +Write a one-line entry to `references/schedule-log.md`: `[timestamp] Emailed [N] options for [target date], cached at option-cache.md` + +--- + +## Workflow C — Recall emailed options (user is back in Cowork) + +**Trigger:** User says "pull up what you emailed me", "show me the postcard previews", "what did you send for the next card", "pick one of the postcard hooks", etc. + +### Step C1 — Read cache + +Read `references/option-cache.md`. Find the most recent entry with `Status: pending pick`. + +### Step C2 — Present in Cowork + +Show the user a clean side-by-side comparison of the cached options (table or cards). Use the existing markup from the email so it feels like the same brief. + +### Step C3 — User picks + +User picks an option ("use option 2", "let's go with the neighbor envy one"). If user wants to modify the picked option, accept tweaks now. + +### Step C4 — Run Workflow A on the pick + +Hand the chosen option's parameters into Workflow A starting at Step 4 (skip the question flow — answers are already in the cache). + +### Step C5 — Update cache + +Mark that option in `option-cache.md`: `Status: PICKED on [date]`. Mark others: `Status: not picked`. Move the whole entry under a "Resolved" section. + +--- + +## Critical principles (apply to all workflows) + +- **Bottom contact block continuity is sacred.** Never edit it per card. +- **Never fabricate stats.** Verify before any number lands on print. +- **One hook per card.** 3-second glance time. One hook, one CTA, one flip. +- **Two headshots, two moods.** Front = pointing pose; Back = smiling pose. +- **Disclaimer is legally required.** Vertical right edge of back. Never remove. +- **Repetition rule is 4-dimensional.** Archetype (3-card cooldown), CTA destination (2-card cooldown), villain (2 Zillow-free cards after any Zillow-bashing card — check the BACK copy too, that's where it sneaks in), and core claim (4-card cooldown). Run the check against the "Repetition tracker" in headline-library.md in EVERY workflow, and append a row to that tracker every time a card ships. Rotating archetype names alone is NOT variety — that failure mode shipped 3 Zillow-bashing cards in a row and offered Graeham the same recycled hooks by email (caught 2026-06-11). +- **Headlines must be structurally fresh, not just lever-fresh.** Before finalizing any headline, compare it word-by-word against every front AND back headline in the Repetition tracker. If it shares its sentence skeleton or signature phrase with a prior card ("I have...", "Your ___ is wrong", "the real number"), rewrite it. + +## Files in this skill + +``` +farming-postcard/ +├── SKILL.md (this file) +├── references/ +│ ├── headline-library.md (6 archetypes + remix + memory) +│ ├── design-tokens.md (locked brand system) +│ ├── cta-router.md (CTA type → URL cache) +│ ├── print-specs.md (UMW + bleed math + Playwright script) +│ ├── option-cache.md (emailed preview options awaiting pick) +│ └── schedule-log.md (cron run history) +└── templates/ + └── postcard-template.html (parameterized master template) +``` + +## Setup (one-time, already done as of build) + +- **GitHub backup:** PAT at `C:\Users\Graeham Watts\Documents\Skills LLMS\Claude\Skills\github-token.txt` +- **Gmail SMTP:** App Password at `C:\Users\Graeham Watts\Documents\Skills LLMS\Claude\Skills\gmail-app-password.txt` +- **Scheduled tasks:** + - `farming-postcard-15th-preview` — fires 8th of each month at 8am → SMTP sends options for the 15th + - `farming-postcard-1st-preview` — fires 24th of each month at 8am → SMTP sends options for the 1st of next month +- **Email recipients (LOCKED):** + - graehamwatts@gmail.com (Graeham) + - graehamwattsvideo@gmail.com (Peter, aka Jason) +- **Send method:** SMTP via `scripts/send_options_email.py` (NOT Gmail MCP draft) + +## After a card ships + +1. Append the shipped card to BOTH the "Cards shipped" table AND the "Repetition tracker" table (core claim / villain / CTA destination) in `headline-library.md` +2. Move the picked option in `option-cache.md` to "Resolved" +3. Run Step 7 (auto-publish to online archive at `Graehamwatts/online-content/farming-postcards/`) +4. If any reference files changed, they auto-push at session end (SessionEnd hook); a manual direct `git` push per `shared-references/publishing-via-composio.md` is also fine + +## Live archive + +All historical and current postcards are tracked at: +- **Dashboard URL:** https://graehamwatts.github.io/online-content/farming-postcards/ +- **Repo path:** `Graehamwatts/online-content/farming-postcards/` +- **Source of truth:** `archive.json` in that folder — regenerate `index.html` from it whenever cards are added diff --git a/skills/farming-postcard/references/cta-router.md b/skills/farming-postcard/references/cta-router.md new file mode 100644 index 00000000..12511565 --- /dev/null +++ b/skills/farming-postcard/references/cta-router.md @@ -0,0 +1,73 @@ +# CTA Router — CTA Type → Landing Page URL + +When a user picks a CTA type for a card, look up the URL here. If the URL is `[NOT SET]`, ask the user once and CACHE it by editing this file. Never ask twice for the same CTA type. + +## Cached URLs + +| CTA Type | Landing URL | Use for | +|---|---|---| +| Home valuation | `[NOT SET — ask user, then cache here]` | "What's my home worth", equity check, precision equity audit, free home valuation | +| Testimonials | `[NOT SET — ask user, then cache here]` | Social proof / reviews CTAs | +| Free report (market) | `[NOT SET — ask user, then cache here]` | Neighborhood report, market report, free download | +| Free report (AI score) | `[NOT SET — ask user, then cache here]` | AI search visibility report, property AI score | +| Thinking of selling | `[NOT SET — ask user, then cache here]` | Pre-listing consultation, seller guide | +| Off-market buyers | `[NOT SET — ask user, then cache here]` | Buyer-pool angle, "I have a list of buyers" | +| Call / text Graeham | `tel:+16503084727` or `sms:+16503084727` | Direct contact CTA (no landing page needed) | +| Custom | Ask user each time | One-off campaigns | + +## How to update + +When asking the user for a missing URL, after they provide it: + +1. Edit this file using the Edit tool +2. Replace `[NOT SET — ask user, then cache here]` with the URL +3. Tell the user: "Cached [URL] as your default [CTA type] target. Won't ask again." + +## QR code generation + +Once you have the URL, generate the QR code at print time. Recommended approach: + +**Python (via bash):** +```bash +pip install qrcode pillow --break-system-packages --quiet +python -c "import qrcode; qr=qrcode.QRCode(box_size=10, border=2); qr.add_data('$URL'); qr.make(); img=qr.make_image(); img.save('/sessions/inspiring-awesome-hawking/mnt/outputs/qr.png')" +``` + +Then embed the resulting PNG into the postcard HTML in place of the stylized SVG placeholder. + +**UTM recommendation (optional but smart):** +Add UTM params so Graeham can track which postcard drove which conversions: +``` +?utm_source=postcard&utm_medium=direct_mail&utm_campaign=epa_[mm_dd_yy]&utm_content=[archetype] +``` + +Example: +``` +https://graehamwatts.com/value?utm_source=postcard&utm_medium=direct_mail&utm_campaign=epa_06_01_26&utm_content=neighbor_envy +``` + +## URL hygiene + +- Always use HTTPS (more reliable QR scanning on iOS) +- Keep URLs under 100 characters or QR density gets too high to scan from arm's length +- If URL is long, use a URL shortener (bit.ly / rebrand.ly) BUT lose UTM tracking — tradeoff to discuss with user + +--- + +## Switchy integration (added 2026-05-28) — pixeled, scan-tracked QR targets + +The QR target should be a **Switchy short link**, not the raw landing URL. The short +link redirects to the landing URL (with UTM) AND fires the retargeting pixel + counts +the scan on the redirect layer — making every postcard drop a retargeting audience, +not just a one-way mailer. Engine: `skills/switchy-engine`. + +**At print time:** +1. Resolve the landing URL from the table above (e.g. Home valuation). +2. Append UTM: `?utm_source=postcard&utm_medium=direct_mail&utm_campaign=epa_[mm_dd_yy]&utm_content=[archetype]`. +3. Mint a Switchy link: `url` = the UTM'd landing URL, `tags:["postcard","qr","consumer","epa_[mm_dd_yy]"]`, `pixels` from `shared-references/switchy.json`. (REST POST https://api.switchy.io/v1/links/create, header `Api-Authorization: `.) +4. Generate the QR encoding the **Switchy short URL**. +5. Report scans later via `switchy-engine/scripts/switchy_analytics.py`. + +If the token isn't active yet, fall back to a QR on the UTM'd landing URL (GA tracks +sessions; no pixel/scan layer). The printed QR can't change later, so mint the Switchy +link BEFORE the print run whenever possible. diff --git a/skills/farming-postcard/references/design-tokens.md b/skills/farming-postcard/references/design-tokens.md new file mode 100644 index 00000000..a29186d2 --- /dev/null +++ b/skills/farming-postcard/references/design-tokens.md @@ -0,0 +1,125 @@ +# Design Tokens — LOCKED Brand System + +These tokens are NEVER negotiable per card. Continuity is the brand. + +**CORRECTED 2026-07-27:** The prior version of this file (and the master template it drove) had drifted from Graeham's real mailed house style — a cream back panel and a glossy 3-stop gold gradient with drop-shadow crept in when this skill was built, and were never actually part of his 2025 cards. Graeham flagged the 08/01/26 card as visually off-brand; comparing it against the real mailed 03/01/25, 03/15/25, and 04/15/25 cards confirmed the drift. Corrected below — **flat single-tone gold, no panel, no gradient, no gloss.** Always sanity-check a new card against a real mailed PDF from `Farming Flyers to mail\`, not just against this file, if anything looks uncertain. + +## Colors + +| Token | Hex | CMYK (approx) | Use | +|---|---|---|---| +| Gold (primary) | `#C2A14E` | C:25 M:35 Y:75 K:5 | Border, headline highlights, CTA color, logo roof accent — the ONLY gold used anywhere on the card, flat, no gradient | +| Dark ink | `#1A1D2E` | C:80 M:75 Y:50 K:60 | Headlines, body text, logo | +| Pattern color | `#E6DABC` (~35% opacity) | n/a | Chevron house pattern overlay | +| White | `#FFFFFF` | 0,0,0,0 | Postcard background — FRONT AND BACK. No panel, no cream box, no fill color changes anywhere. Copy sits directly on the white chevron background exactly like the QR side. | + +**Retired tokens (do not use):** Gold-deep `#A88638` and Gold-light `#EAD9A8` were used for a 3-stop gradient effect that never matched Graeham's real mailed cards — removed 2026-07-27. Cream `#FBF7EC` back-panel fill was likewise never part of the real house style — removed the same day. If you see either reappear in a rendered card, it's a bug — flatten to solid `#C2A14E`. + +## Typography + +| Use | Font | Weight | Size | Source | +|---|---|---|---|---| +| Front headline | Anton | Regular | 38pt | Google Fonts | +| Back headline | Anton | Regular | 26pt | Google Fonts | +| CTA line | Anton | Regular | 14pt | Google Fonts | +| Body | Inter | 400 (italic) | 10pt | Google Fonts | +| Sub / flip prompt | Inter | 600-800 | 14pt | Google Fonts | +| Contact info | Inter | 400/800 | 8-11pt | Google Fonts | +| Disclaimer | Inter | 400 | 6.5pt | Google Fonts | + +**Headline rule:** Anton ONLY. Never substitute. Oswald is acceptable backup if Anton fails to load. + +## Layout grid (6" × 4" postcard) + +- **Gold left border:** 14px wide, full height, color `#C2A14E`, z-index 6 +- **Chevron pattern:** SVG repeat, 80x40px tile, 0.35 stroke opacity, 0.55 layer opacity +- **Bleed:** 0.125" each side (total canvas 6.25" × 4.25") +- **Safe zone:** Keep type 0.25" from all edges minimum + +**Front headline rule (corrected 2026-07-27):** the headline dominates the top third of the FRONT — full width, sized to nearly fill the safe zone left-to-right, sitting high on the card. Don't leave it clustered small in a top corner with a large empty middle; that empty-space imbalance was the other half of the 08/01/26 drift. **No decorative arrow on the front.** The master template has no arrow element at all — the QR's own black "SCAN ME" pill is the visual anchor, no arrow needed. The 08/01/26 card's front arrow (pointing at empty space) was a one-off addition that shouldn't have been there; it's been removed from the master template. + +## LOCKED Bottom Contact Block (NEVER edit) + +This block appears identically on every card. Continuity is the brand signature. + +``` +[COMPASS LOGO] +The Boyenga Team +[gold roof icon] +GRAEHAM WATTS + +REALTOR® 650-308-4727 +The Boyenga Team graehamwatts@gmail.com +DRE #01466876 www.graehamwatts.com +``` + +**HTML structure (drop-in):** + +```html + +
+
REALTOR®
+
The Boyenga Team
+
DRE #01466876
+
650-308-4727
+
graehamwatts@gmail.com
+
www.graehamwatts.com
+
+``` + +## LOCKED Disclaimer (legal — never remove) + +> "If your home is listed with another broker, please disregard this postcard. Homes not necessarily sold by this broker." + +- Placement: Vertical text on right edge of BACK +- Size: 6.5pt Inter +- Color: `#555` +- Rotated -90° + +## Gold-highlight treatments + +Two variants only — both FLAT, single-tone `#C2A14E`, no gradient, no gloss, no drop-shadow. Choose per word/phrase: + +**Variant A — Solid gold box** (for short emphasized phrases, back headline): +```css +background: #C2A14E; +color: #fff; +padding: 0 6px; +``` + +**Variant B — Gold text fill** (for emphasized words inline with regular front headline): +```css +color: #C2A14E; +``` + +**Variant C — Gold underline** (for action verbs): +```css +border-bottom: 4px solid #C2A14E; +padding-bottom: 2px; +``` + +## What's NEGOTIABLE per card + +- Headline text + which words get gold highlight (1-3 max) +- Subline / flip prompt copy +- Back headline + body copy +- CTA line text +- QR target URL +- Headshot pose (pointing for front, smiling for back is the default but can flex) +- **Whether the front carries its own QR** (added 2026-07-27, per Graeham). Default is back-only, but a front QR + its own short CTA line is an approved pattern — e.g. the front poses a curiosity hook with its own QR/offer. When used, mint a SEPARATE Switchy link for the front QR (see cta-router.md's Switchy section) — never reuse the back's link for the front. +- **Whether the back carries a QR at all, or a direct call-to-action instead** (settled 2026-07-27, per Graeham, same day as the above — this is now the DEFAULT, not the exception). A back with no QR — just "WANT A [OFFER]? CALL ME TODAY!" + a large phone number + gold arrow — is Graeham's proven real house style (see the Sept 2025 "Is Now The Right Time To Sell?" card). **Default to ONE QR total (front) + a call CTA on the back**, unless the specific offer genuinely needs its own landing page. Only mint a Switchy link for a QR that actually appears on the card. + +## What's NEVER negotiable + +- Color tokens above +- Font choices +- Bottom contact block +- Disclaimer text + placement +- Gold left border +- Chevron pattern background +- Aspect ratio (6×4 default — can scale to 6×9 for Corefact jumbo but proportions lock) diff --git a/skills/farming-postcard/references/headline-library.md b/skills/farming-postcard/references/headline-library.md new file mode 100644 index 00000000..cb1212e5 --- /dev/null +++ b/skills/farming-postcard/references/headline-library.md @@ -0,0 +1,280 @@ +# Headline Library — Graeham's Farming Postcard Archetypes + +Six proven archetypes from Graeham's past cards, each with the **psychological lever** it pulls and remix patterns for fresh language. + +--- + +## 1. EQUITY (pride + curiosity) + +**Past hook:** "Do You Know How Much Equity You've Built This Year?" +**Lever:** Pride in ownership + curiosity about a real number +**Best for:** 1st-of-month cadence, past clients, owners who haven't sold in 3+ years +**Default CTA:** Home valuation + +**Remix patterns:** +- "Your home grew $___ this year. Want the real number?" +- "EPA equity is at a 5-year high — what's yours?" +- "You're sitting on more than you think." +- "The equity check most owners never get." + +**Gold-highlight rule:** Highlight the dollar amount or "equity" / "more than you think" + +--- + +## 2. BUYER-TAGGED (scarcity) + +**Past hook:** "I Already Have Your Buyer TAGGED" +**Lever:** "Someone wants what you have and you don't know it" +**Best for:** Hot markets, post-spike inventory crunch, 15th-of-month cadence +**Default CTA:** Home valuation or "thinking of selling" + +**Remix patterns:** +- "Your buyer is already in my pipeline." +- "I have 3 buyers looking for your block." +- "There's a family ready to make you an offer." +- "Your home is on someone's list. They just haven't seen it yet." + +**Gold-highlight rule:** Highlight "TAGGED" / "3 buyers" / "your block" + +--- + +## 3. ANTI-ZILLOW BUYER POOL (scarcity + anti-portal) + +**Past hook:** "I Have A List Of Buyers Who Don't Use Zillow" +**Lever:** "There's a market you're not seeing" +**Best for:** Owners who think Zillow tells the whole story; differentiation from competing agents +**Default CTA:** Home valuation or "thinking of selling" + +**Remix patterns:** +- "The buyers Zillow can't show you." +- "Off-market buyers don't browse — they get tagged." +- "73% of my closings start before the listing goes live." *(only use real numbers — verify before print)* +- "Zillow shows everyone. I show the ones who actually buy." + +**Gold-highlight rule:** Highlight "Don't use Zillow" / "Off-market" / the real number + +--- + +## 4. AI SEARCH INVISIBILITY (FOMO + tech) + +**Past hook:** "Why Isn't ChatGPT Recommending Your Home?" +**Lever:** "The market changed and you didn't notice" +**Best for:** Tech-savvy markets (Peninsula), AEO/answer-engine moment, owners who pride themselves on being current +**Default CTA:** Free report ("AI score" or similar) + +**Remix patterns:** +- "Standard listings are becoming invisible in your neighborhood." +- "Your home isn't showing up in AI search. Here's why." +- "Buyers ask AI first. Does it know about your home?" +- "The new search engine doesn't run on Google." + +**Gold-highlight rule:** Highlight "becoming invisible" / "AI search" / "doesn't run on Google" + +--- + +## 5. ANTI-ZESTIMATE (anti-algorithm) + +**Past hook:** "Your Zestimate Is Wrong" +**Lever:** "What you've been told isn't true" +**Best for:** Pairs well with equity angle; owners checking Zillow regularly +**Default CTA:** Home valuation + +**Remix patterns:** +- "Zillow doesn't know about your remodel." +- "The Zestimate is off by $___ on most EPA homes." *(verify before print)* +- "Algorithms guess. I measure." +- "Your home is worth more than the bot says." + +**Gold-highlight rule:** Highlight "WRONG" / "more than the bot says" / the dollar gap + +--- + +## 6. NEIGHBOR ENVY (curiosity + social proof) + +**Past hook:** "What Did The Last 5 Homes On Your Street Really Sell For?" +**Lever:** "Your neighbors know something you don't" +**Best for:** Any audience — works on farm + past clients. Strongest archetype for dual-purpose cards. +**Default CTA:** Home valuation or free report + +**Remix patterns:** +- "[Street] just had a record sale. Want to see it?" +- "Three homes on your block sold this spring." +- "Your neighbor cashed out $___ above asking." *(verify before print)* +- "The numbers on your block this year." + +**Gold-highlight rule:** Highlight "5 HOMES" / "REALLY" / "$___ above asking" + +--- + +## 7. LOW-INVENTORY TIMING (supply-side scarcity) + +**Status:** New archetype, added 2026-06-11. Not yet shipped. +**Lever:** "The window favors you right now" — scarcity of competition, not scarcity of buyers +**Best for:** 15th-of-month cadence, fence-sitters, low-inventory months +**Default CTA:** Free report (market) + +**Remix patterns:** +- "Right now, your home would have almost no competition." +- "Low inventory means your home stands alone." +- "Fewer homes for sale = stronger offers for the ones that list." + +**Gold-highlight rule:** Highlight "No Competition" / "Stands Alone" +**⚠ Verify:** Confirm local inventory is genuinely low before mailing. + +--- + +## 8. QUIET SALE / NO-HASSLE (privacy + ease) + +**Status:** New archetype, added 2026-06-11. Not yet shipped. +**Lever:** Hassle aversion + privacy — "selling doesn't have to mean strangers in your house" +**Best for:** Long-time owners, older owners, privacy-conscious EPA farm. No villain, no valuation — total palate cleanser between data-driven cards. +**Default CTA:** Thinking of selling + +**Remix patterns:** +- "Sell your home without a single open house." +- "No sign. No strangers. Just a sale." +- "The quietest way to sell a home in EPA." + +**Gold-highlight rule:** Highlight "Single Open House" / "Quietest" + +--- + +## 9. PROP 19 TAX TRANSFER (education + unlock) + +**Status:** New archetype, added 2026-06-11. Not yet shipped. +**Lever:** "You can take your low tax rate with you" — removes the #1 reason long-time CA owners stay put +**Best for:** 55+ owners, long-tenure EPA farm (huge segment), 1st-of-month education slot +**Default CTA:** Free report + +**Remix patterns:** +- "Your low property tax rate can move with you." +- "Own since the 90s? Your tax basis is portable now." +- "The California rule most long-time owners still don't know." + +**Gold-highlight rule:** Highlight "Move With You" / "Portable" +**⚠ Verify:** Confirm current Prop 19 rules (age 55+, disability, disaster; statewide transfer) before print. Never give tax advice — always "talk to your tax professional" framing. + +--- + +## 10. LOCAL PROOF / JUST SOLD (social proof + recency) + +**Status:** New archetype, added 2026-06-11. Not yet shipped. +**Lever:** "It just happened near you" — proof beats promise +**Best for:** Right after a real nearby sale (Graeham's or a notable comp). Differs from Neighbor Envy: that card sells curiosity about numbers; this one sells a recent RESULT. +**Default CTA:** Free report (market) or testimonials + +**Remix patterns:** +- "[Street] just sold in [X] days." +- "A home near you just closed [above/at] asking." +- "This just happened [N] blocks from you." + +**Gold-highlight rule:** Highlight the street name or day count +**⚠ Verify:** Requires a REAL recent comp — never ship without it. + +--- + +## Differentiation rules (added 2026-06-11 after repetition audit) + +Rotating archetype names is NOT enough. Audit on 2026-06-11 found the first 6 cards repeated underlying elements even while archetypes rotated. Before recommending ANY option (Workflow A or B), check ALL FOUR axes against the Repetition tracker below: + +1. **Archetype cooldown:** not within the last 3 cards (existing rule). +2. **CTA-destination cooldown:** the CTA type must differ from the previous 2 cards. (Violation found: valuation/equity CTA ran on 4 of the first 6 cards, including both 05/15/26 and 06/01/26 back-to-back.) +3. **Villain cooldown:** if a card names or implies Zillow/algorithms as the villain, the next 2 cards must be villain-free. (Violation found: 04/15/25, 05/15/26, and the 06/01/26 BACK all bash Zillow.) +4. **Core-claim cooldown:** the central claim ("I have buyers waiting", "the number you have is wrong", "know your real value") must not repeat within 4 cards, regardless of archetype. (Violation found: "I have buyers" was the core claim of BOTH 03/15/25 and 04/15/25, and was re-offered twice in the 06/15 option email.) + +With 10 archetypes in the library, Workflow B should now always be able to offer 3+ options that pass all four checks. + +--- + +## Repetition tracker (audit per card — append as we go) + +| Date | Archetype | Core claim | Villain | CTA destination | +|---|---|---|---|---| +| 03/01/25 | Equity | Know your real equity number | none | Valuation | +| 03/15/25 | Buyer-tagged | I have buyers waiting | none | Call/text | +| 04/15/25 | Anti-Zillow buyer pool | I have buyers waiting | Zillow | QR / buyer match | +| 05/01/26 | AI search invisibility | Your home is invisible to new search | portals/old marketing | AI score report | +| 05/15/26 | Anti-Zestimate | The number you have is wrong | Zillow/algorithms | Valuation (equity audit) | +| 06/01/26 | Neighbor envy | Know the real numbers near you | Zillow (back: "Zillow guesses. I measure.") | Valuation | +| 06/15/26 | Low-Inventory Timing | Your timing window favors a seller now | none | Free report (market) | +| 07/01/26 | Prop 19 Tax Transfer | Your low tax basis is portable (55+) | none | Call/text (SMS keyword "PROP 19") | +| 07/15/26 | Buyer-tagged | Buyers are already searching for your home | none | Call/text (SMS keyword "BUYERS") | +| 08/01/26 | Equity | You've built more wealth than you think | none | Call/text (SMS keyword "EQUITY") | +| 08/15/26 | Quiet Sale / No-Hassle | Sell without a single open house | none | **Landing page** (seller consultation) | +| 09/01/26 | Neighbor Envy (fresh remix) | Someone on your block just sold — here's what they got | none | **Landing page** (home valuation, 2nd page) | + +**MAILED 2026-08-01** (confirmed by Graeham). **08/15/26 and 09/01/26 are both PRINT-READY, not yet mailed.** + +**09/01/26 note:** built same-day as 08/15 per Graeham's "need the next one now" — re-ran the differentiation check against 08/15 (Quiet Sale, seller_consultation) and 08/01 (Equity, call/text) before picking. Neighbor Envy was last used 06/01/26 (outside the 3-card cooldown) and its core claim got a genuine fresh remix, not a copy of the 06/01 headline. CTA required a destination other than seller_consultation or call/text — rather than fall back to a THIRD disguised SMS card, built a **second real landing page** (`east-palo-alto/home-valuation/`) so the CTA rotation has real depth instead of just one page absorbing every card. + +**09/01/26 REVISED v2 (same day, per Graeham voice note):** three structural upgrades, now standard going forward — +1. **Dual QR.** Front and back each now carry their own QR/CTA (front = curiosity about neighbors' sales, back = converts to "what's my home worth"). First card built this way; treat as the new default pattern, not a one-off, unless Graeham says otherwise. Design-tokens.md's front layout should be read as "front QR is now an available slot," not a violation of the locked system. +2. **Real Switchy links on every QR, not raw UTM'd URLs.** `switchy-engine/scripts/create_postcard_link.py` already has Graeham's live retargeting pixel IDs hardcoded (Facebook, GA, Google Ads) — the `shared-references/switchy.json` config file is stale (still shows TODO placeholders) and should not be trusted; use the script's real values. Mint one Switchy link per QR going forward (`--dest --market epa --archetype --slug epa---front/back`), embed the short `hi.switchy.io/...` link, and tell Peter to pull the printable QR PNG straight from the Switchy dashboard rather than generating it from the raw URL. +3. **Architecture insight worth remembering:** because the QR encodes the Switchy short link (not the final destination directly), a card can ship with an interim/placeholder destination and be silently upgraded later without ever reprinting. Use this whenever the "real" destination isn't built yet — ship the Switchy link now, repoint it later. + +**09/01/26 FINAL v3 (same day):** Graeham reviewed v2's simplified copy live and preferred the ORIGINAL wording after all — front/back text reverted to "Someone On Your Block Just Sold. Here's What They Got." / "See The Real Sales On Your Street — No Guessing." **Only the structural change survived: the front QR stays.** Lesson for next time — when a user proposes new copy verbally, ship it, but don't be surprised if seeing it live changes their mind; keep structural changes (like a new QR slot) decoupled from copy changes so one can revert without touching the other, which is exactly why this revert was a 2-minute edit instead of a rebuild. + +**09/01/26 FINAL v4 (same day) — settled structure, likely the new default:** Graeham simplified further — front subline became "SCAN HERE to see what they got" (pairs with the headline directly), and the BACK QR was dropped entirely in favor of a direct call-to-action: "WANT A FREE HOME EVALUATION? CALL ME TODAY!" + large phone number + gold arrow, no link, no QR. This actually matches his REAL 2025 house style closely (the Sept 2025 card in `Farming Flyers to mail\` is literally "Is Now The Right Time To Sell? Call Me Today!" + phone number, no QR at all) — worth remembering that call-only backs are a proven, authentic pattern for this brand, not a downgrade from a QR. **New default going forward: ONE QR (front), back closes with a call CTA** — unless a specific card's CTA genuinely needs a second landing page, don't default to a QR on both sides. Front QR still needs its own Switchy link same as before; back needs no link minted at all now. + +**2026-08-18 — GHL access investigation, resolved (partially):** + +1. **API token path is a confirmed dead end.** Checked GoHighLevel's own scope picker directly (both agency-level and inside the actual location) — the existing "Claude PCFS Read-Write" private-integration token already has `funnels/page.readonly` and `funnels/funnel.readonly`, but **there is no write scope for funnel/website pages anywhere in GHL's API** (only page/funnel/pagecount read scopes + redirect read/write exist). This isn't a permissions gap — GoHighLevel's public API genuinely cannot create or edit page content. Their own visual builder is the only way, for anyone, not just us. Did not touch or rotate that token. + +2. **Browser-driven build: partial success.** Using the Claude-in-Chrome extension on Graeham's real, logged-in Chrome, found there's already a polished, on-brand, real-comps home-valuation funnel live at `graehamwatts.com/real-local-comps` (native GHL page, address-entry form feeding his CRM directly — far better than the GitHub Pages/mailto-fallback interim page built earlier). Cloned it, renamed to "EPA Postcard - Recent Sales Near You," and successfully wired a real domain + path (`graehamwatts.com/recent-sales-near-you`) — verified LIVE by direct fetch. **However**, the GHL page-builder's own editing canvas repeatedly hung/froze when trying to edit the on-page copy (the whole page is one big Custom HTML/JS block, and the builder's canvas stopped responding to clicks — happened twice, recovered once after ~30s, froze permanently the second time). Backed out cleanly each time rather than force it on a live page. **Net result: the clone is live but still carries the ORIGINAL "check my home value" copy verbatim — it has NOT been customized to the neighbor-sales-curiosity angle yet.** + +3. **What's wired in now:** `home_valuation` destination upgraded to the real `real-local-comps` page (genuine improvement — do this for every future card using that CTA). A new `recent_sales_nearby` destination points at the clone, but is flagged **DO NOT USE** on a front-QR "see what your neighbors got" hook until its copy actually says that — right now it would be a content mismatch. The already-mailed 09/01/26 card's front QR was deliberately NOT repointed to this new page for the same reason (still points at the generic IDX map search interim). + +4. **Fast-follow:** retry the copy edit in a fresh session — the builder may just have had a bad day, or ask Graeham/his web person to swap one headline block directly (it's a single text edit once the canvas is responsive). Once that's done, `recent_sales_nearby` becomes a real, fully differentiated third landing page and its "DO NOT USE" flag should be removed. + +**OPEN — bigger project, not yet resolved:** Graeham wants a genuine "East Palo Alto sold properties, map-default, pannable" page for the front QR to eventually point to. Investigation found: graehamwatts.com runs on GoHighLevel's own site platform (`leadconnectorhq.com`), so building pages there is native, not a new integration — but the actual MLS map search runs through a SEPARATE platform, IDX Broker (`listings.graehamwatts.com`), whose visible status filters (Active/Contingent/Pending) don't include "Sold" at all — likely an MLS data-permission restriction, not a missing feature, and needs checking against his MLS's IDX display rules before assuming it's buildable. Also unresolved: whether this session has any GHL/LeadConnector site-editing access beyond the audit-only `ghl-crm-audit` skill. Don't attempt to build this until both are confirmed with Graeham. + +**Design note (2026-07-27) — FIXED AT THE SOURCE, not just told to Peter.** Graeham flagged the mailed 08/01 card visually drifted from the clean 2025 house style — a cream back panel (2025 cards run copy directly on white), a glossy 3-stop gold gradient instead of the flat brass border-gold, and an unbalanced front headline with an arrow pointing at empty space. Root cause: **this was baked into the skill's own "locked" design-tokens.md and postcard-template.html** — a drift I introduced when building the skill in June, not something Peter did. Corrected both files 2026-07-27: flat single-tone `#C2A14E` gold everywhere (no gradient, no gloss), no cream panel anywhere, front headline now spans full width at 44px and dominates the top third, decorative front arrow removed entirely (there was never a back arrow either — the QR's "SCAN ME" pill is the only anchor needed). **Every future card built from this template inherits the fix automatically.** If a card ever looks off again, compare it against a real mailed PDF from `Farming Flyers to mail\`, not against memory. + +**CTA-monotony fix (2026-07-27, resolved 2026-08-18):** three cards running (Prop 19 → Buyer-tagged → Equity) all shipped on call/text. First real evergreen landing page went live `https://graehamwatts.github.io/online-content/east-palo-alto/seller-consultation/`, wired into `libraries.json` as `seller_consultation`, used on 08/15/26. + +**Switchy is now actually live (2026-08-18)** — the `switchy-token.txt` in the Skills folder is a real token (not a placeholder), and `switchy-engine/scripts/create_postcard_link.py` has real pixel IDs hardcoded (Facebook, GA, Google Ads). Every future card's QR should be minted through that script, not printed as a raw destination URL — it gives pixel-based retargeting on top of the click, not just click tracking. + +**Real "sold homes near you" landing page, resolved 2026-08-18 (was previously a compliance/data-permission question):** confirmed via IDX Broker (`middleware.idxbroker.com`, Graeham's actual MLS/IDX admin — separate from GHL) that Sold-status display IS permitted (MLS Approval confirmed for MLSListings Inc.) and was simply hidden deeper in the Status multi-select than a page-read caught. Built a dedicated "Prime Map Search" widget (ID 168164, "EPA Sold Homes - Postcard") — Status=Sold, default location=East Palo Alto CA, sort=Newest — live at `https://listings.graehamwatts.com/idx/widgetpreview.php?widgetid=168164&prime=true`, already branded, confirmed showing 42 real sold listings on a pannable map. Wrapped in a pixeled Switchy link `hi.switchy.io/epa-sold-homes-0901` and used for the first time as the SOLE QR on the 09/01/26 card (Graeham's final call: one QR on the front only, back becomes a direct call-to-action instead of a second QR — matches the real 2025 house style). + +Home valuation also has a real page + pixeled Switchy link ready (`hi.switchy.io/epa-home-eval-0901` → `graehamwatts.com/real-local-comps`), not yet used on a printed card. + +**Fast-follow / not yet done:** (1) reconcile the Phase 2 generator (`postcard_preview_generate.py`) — it currently builds its own UTM-tagged URL from `libraries.json`'s `canonical_url`, which is a DIFFERENT tracking mechanism than the Switchy-wrapped links now used for real prints; decide whether the generator should call `create_postcard_link.py` instead so both paths agree. (2) `off_market_buyers` page still not built. (3) A GHL-native lead-capture form is still not wired into either landing page (both fall back to call/text as their own primary CTA). + +**Next card after 08/15 (i.e., 09/01):** blocked archetypes will be Equity, Quiet Sale, [+ whichever else ships before then]. Blocked CTA will include `seller_consultation` for 2 cards — rotate to a different destination (ideally `home_valuation` once that page exists) rather than falling back to another SMS keyword. + +**Next card = 08/15/26.** Constraints after 08/01 ships: blocked archetypes (last 3) = Equity, Buyer-tagged, Prop 19. Blocked CTA destination (last 2) = sms_keyword generic — but note both were technically the SAME mechanical channel (call/text) even with different keywords, so treat call/text as due for a real break, not just a new keyword. Blocked core claims (last 4) = wealth_you_built, i_have_buyers, tax_base_is_portable, timing_favors_seller. Villain cooldown is OPEN (last 2 cards were villain-free). **Cleanest fresh picks for the 15th (scarcity/timing slot):** Quiet Sale / No-Hassle (never shipped, no villain, no data to verify) or Local Proof / Just Sold (never shipped, but needs a real nearby comp — verify before using). Low-Inventory and Anti-Zillow-pool are technically off archetype-cooldown but their core claims got reused too recently in spirit — prefer the two virgin archetypes above. + +--- + +## How to remix (don't just copy) + +When a user picks an archetype: +1. Pull the **lever** (curiosity / scarcity / pride / FOMO). +2. Look at **recent angles** below to avoid repetition within 3-card window. +3. Generate 2-3 fresh variations using the patterns above. +4. Always ask Graeham which one to use. + +## Layout principles (every card) + +- **One hook per card.** No combo headlines. +- **3-second rule.** The headline must register at glance distance. +- **Curiosity loop.** The front should pose a question / claim that's painful to leave unanswered. The back delivers the payoff + CTA. +- **Gold-highlight on 1-3 words max.** More than 3 dilutes attention. +- **The flip prompt** ("Flip over for…") is mandatory. + +--- + +## Cards shipped (memory — append as we go) + +| Date | Archetype | Headline | CTA | +|---|---|---|---| +| 03/01/25 | Equity | Do You Know How Much Equity You've Built This Year? | Free equity report | +| 03/15/25 | Buyer-tagged | I Already Have Your Buyer TAGGED | Call/text | +| 04/15/25 | Anti-Zillow buyer pool | I Have A List Of Buyers Who Don't Use Zillow | Scan QR | +| 05/01/26 | AI search invisibility | Why Isn't ChatGPT Recommending Your Home? | Property AI score | +| 05/15/26 | Anti-Zestimate | Your Zestimate Is Wrong | Precision equity audit | +| 06/01/26 | Neighbor envy | What Did The Last 5 Homes On Your Street Really Sell For? | Free home valuation | +| 06/15/26 | Low-Inventory Timing | Right Now, Your Home Would Have Almost No Competition | Free market report | +| 07/01/26 | Prop 19 Tax Transfer | Your Low Property Tax Rate Can Move With You | Call/text (SMS keyword) | + +**Repetition rule:** Don't reuse the same archetype within 3 cards — AND run the full 4-axis differentiation check (see "Differentiation rules"). diff --git a/skills/farming-postcard/references/option-cache.md b/skills/farming-postcard/references/option-cache.md new file mode 100644 index 00000000..6260f605 --- /dev/null +++ b/skills/farming-postcard/references/option-cache.md @@ -0,0 +1,233 @@ +# Option Cache — Scheduled Preview Options + +This file stores the postcard hook options that the skill emails to Graeham 7 days before each drop date. When Graeham comes back to Cowork and says "pull up what you emailed me", read this file, find the most recent entry with `Status: pending pick`, and present those options for selection. + +After he picks, mark his pick `PICKED` and the others `not picked`, then move the whole entry under the **Resolved** section at the bottom. + +--- + +## Pending picks + +## 2026-07-01 (MANUAL RECOVERY — scheduled 1st-preview FAILED to fire on 06/24) +Status: RESOLVED — Prop 19 Tax Transfer built 2026-06-29 (manual). No options email went out because the scheduled task missed its window (app closed 06/24; catch-up on 06/28 never completed the send — see schedule-log.md). Graeham flagged the gap on 06/29 with 2 days to the drop, so the card was built directly rather than emailing a slate. + +### Built card — Prop 19 Tax Transfer (new archetype #9, first run) +Front headline: Your Low Property Tax Rate Can Move With You +Back headline: The Rule Most Long-Time Owners Don't Know +Back body: If you're 55 or older, California now lets you carry your current property tax base to your next home — anywhere in the state. For a lot of owners, the biggest reason to stay put just stopped being a reason. It's worth a few minutes to see what it could mean for your next move. +CTA line: Call or text me for your Prop 19 breakdown [UPDATED 2026-06-29 — rush-to-print: swapped to call/text to remove the landing-page blocker; July 1 on-time] +CTA type: Call / text Graeham (SMS QR keyword "PROP 19" → 650-308-4727; no landing page needed) +QR target: SMSTO:+16503084727:PROP 19 (static SMS QR, no URL) +Why this works: Cadence-correct 1st-of-month education angle; fully fresh; no villain (keeps the Zillow-free streak); removes the #1 barrier for EPA's large 55+ long-tenure base. +Audience fit: Both (strongest on 55+ farm) +⚠ Verification: Prop 19 facts accurate as of 2026 (55+/disabled/disaster, statewide transfer). Not-tax-advice disclaimer included on card. Confirm before print. +Differentiation check: PASS all 4 axes — archetype never shipped; CTA ≠ last 2; villain-free; core claim ("your tax basis is portable") never used. + +## 2026-06-15 (emailed 2026-06-08 — SCHEDULED 15th-preview, REAL SMTP send) +Status: RESOLVED — Option A (Low-Inventory Timing) PICKED by Graeham 2026-06-11. Options 1-3 withdrawn (repetition audit); Options B-D not picked (banked for future cards). Rough draft sent to Peter 2026-06-11. + +> **⚠ Repetition audit (2026-06-11, requested by Graeham):** Options 1–3 below failed the differentiation check against the 6 cards already mailed and were WITHDRAWN. Option 1 (Buyer-tagged) repeats the core claim of the 03/15/25 card ("I have your buyer"). Option 2 (Anti-Zillow) repeats the 04/15/25 card's claim nearly verbatim AND would be the 4th Zillow-bashing card in the series. Option 3 (Equity) repeats the 03/01/25 card and the valuation CTA that already ran on the backs of BOTH 05/15/26 and 06/01/26. Only Option 4 passed. Fresh replacement options A–D appended below — see headline-library.md "Differentiation rules." + +### Option 1 — Buyer-tagged [WITHDRAWN — repeats 03/15/25 core claim] +Front headline: Your Future Buyer Is Already On My List +Back headline: Pre-Approved, Local, And Waiting For A Home Like Yours +Back body: I keep an active list of buyers who are pre-approved and searching this exact area — some have been waiting months for the right home. When a house like yours comes up, they move fast. Before you ever list publicly, I can quietly check if one of them is a match. +CTA line: See if your buyer is already on my list +CTA type: Thinking of selling +QR target URL: [NOT SET — resolve at print via cta-router.md] +Why this works: Scarcity flips the seller's question from "will it sell?" to "a specific person already wants it" — lowest-pressure listing-conversation opener, no number to fabricate. +Audience fit: Both (strong on Farm) +⚠ Verification needed: Only mail if Graeham genuinely maintains an active matched-buyer list — keep the claim true. + +### Option 2 — Anti-Zillow buyer pool [WITHDRAWN — repeats 04/15/25 claim; 4th Zillow-villain card] +Front headline: The Buyer For Your Home Isn't On Zillow +Back headline: The Serious Buyers Move Privately — And I Know Them +Back body: Zillow shows your home to browsers, neighbors, and the merely curious. The buyers who actually close — relocating execs, cash buyers, move-up families — often work quietly through agents before a home ever hits the portals. I keep a direct line to that pool. +CTA line: See who's buying off-market near you +CTA type: Off-market buyers +QR target URL: [NOT SET — resolve at print via cta-router.md] +Why this works: Reframes Zillow as only the casual half of the market; positions Graeham as gatekeeper of the serious half — pure differentiation no competing agent's mailer claims. +Audience fit: Both + +### Option 3 — Equity [WITHDRAWN — valuation CTA already ran on last 2 cards' backs] +Front headline: Your Equity Grew While You Weren't Looking +Back headline: Find Out Exactly What You've Gained — To The Dollar +Back body: Most owners only learn their equity when they refinance or sell — and by then they're reacting, not planning. A quick, no-pressure valuation shows you the real number today, with no obligation to do anything. Knowing it is what makes the next move possible. +CTA line: Get your real equity number this week +CTA type: Home valuation +QR target URL: [NOT SET — resolve at print via cta-router.md] +Why this works: Lowest-friction curiosity hook — about THEIR money, no selling implied. Equity archetype hasn't shipped since 03/01/25, so it reads fresh. +Audience fit: Both (strongest on past clients) + +### Option 4 — WILDCARD · Low-Inventory Timing +Front headline: Right Now, Your Home Would Have Almost No Competition +Back headline: Low Inventory Means Your Home Stands Alone +Back body: When few homes are for sale, the ones that list capture all the attention — and often the strongest offers. That window doesn't stay open forever. If you've been on the fence, this is the part of the cycle that favors a seller. +CTA line: See what your timing is worth +CTA type: Free report (market) +QR target URL: [NOT SET — resolve at print via cta-router.md] +Why this works: Supply-side scarcity is a fresh angle vs. the demand-side buyer-pool hooks — gives fence-sitters a reason to act now without a hard sell. Test card; promote to a 7th archetype if it lands. [PASSED repetition audit 2026-06-11 — now archetype #7 in headline-library.md] +Audience fit: Both +⚠ Verification needed: Confirm local inventory is genuinely low this month before mailing — claim must be true. + +### REVISED OPTIONS (added 2026-06-11 after repetition audit — all pass the 4-axis differentiation check) + +### Option A — Low-Inventory Timing (= original Option 4, carried forward) +See Option 4 above. Only original option that passed. + +### Option B — Quiet Sale / No-Hassle (NEW archetype #8) +Front headline: Sell Your Home Without A Single Open House +Back headline: No Sign. No Strangers. Just A Sale. +Back body: Most owners picture selling as months of cleaning, staging, and weekends full of strangers walking through their home. It doesn't have to work that way. There's a quieter path — and for many EPA homes, it gets just as strong a price. +CTA line: Ask me how a quiet sale works +CTA type: Thinking of selling +QR target URL: [NOT SET — resolve at print via cta-router.md] +Why this works: Total palate cleanser — no villain, no valuation, no buyer-list claim. Speaks to long-time owners' real objection (hassle + privacy), which no card in the series has touched. +Audience fit: Both (strongest on long-tenure Farm) + +### Option C — Prop 19 Tax Transfer (NEW archetype #9) +Front headline: Your Low Property Tax Rate Can Move With You +Back headline: The California Rule Most Long-Time Owners Still Don't Know +Back body: If you're 55 or older, California now lets you carry your current property tax basis to your next home — anywhere in the state. The #1 reason long-time owners stay put may no longer apply to you. It's worth five minutes to find out. +CTA line: Get the plain-English Prop 19 guide +CTA type: Free report +QR target URL: [NOT SET — resolve at print via cta-router.md] +Why this works: Pure education + unlock — removes the biggest real barrier for EPA's long-tenure owners. Nothing like it in the series. Frame as info only, "confirm with your tax professional." +Audience fit: Both (gold for 55+ long-tenure Farm) +⚠ Verification needed: Confirm current Prop 19 rules before print; include tax-professional disclaimer line. + +### Option D — Local Proof / Just Sold (NEW archetype #10) +Front headline: This Just Happened [N] Blocks From You +Back headline: [Street] Sold In [X] Days +Back body: [Built around a REAL recent nearby sale — days on market, offers received, outcome.] When it happens this close to home, it's not a headline — it's your market. Here's what it means for what your home would do. +CTA line: See what just sold near you +CTA type: Free report (market) +QR target URL: [NOT SET — resolve at print via cta-router.md] +Why this works: Proof beats promise — recent, hyper-local, real. Differs from Neighbor Envy (curiosity about numbers) because it leads with a RESULT. +Audience fit: Both +⚠ Verification needed: Requires a real recent comp (ideally Graeham's own sale). DO NOT ship without one. + +--- + +## 2026-06-15 (emailed 2026-05-27 — TEST RUN) +Status: superseded by 2026-06-08 scheduled run + +### Option 1 — Buyer-tagged +Front headline: I have 3 buyers looking for your BLOCK. +Back headline: Active buyers · not just window-shoppers +Back body: While other agents wait for the phone to ring, my geofencing is tracking three pre-approved buyers actively touring properties on streets around you. Your home might be the one they're waiting for. +CTA line: Call me · let's see if you match +CTA type: Call/text Graeham +QR target URL: tel:+16503084727 +Why this works: Concrete number (3) + proximity ("your block") = strongest scarcity hook for cold farm audience +Audience fit: Farm +⚠ Verification needed: Confirm "3 buyers" is real before print; soften to "buyers" if not + +### Option 2 — Anti-Zillow buyer pool +Front headline: Zillow shows everyone. I show the ones who actually buy. +Back headline: Off-market buyers don't browse — they get TAGGED +Back body: Most listings reach the same tired buyer pool. My system identifies high-intent, pre-qualified buyers who don't waste agent time scrolling Zillow. They're looking for homes like yours right now. +CTA line: Free off-market match check +CTA type: Off-market buyers +QR target URL: [NOT SET — needs URL] +Why this works: Differentiates from every other agent doing standard listings + creates exclusivity +Audience fit: Both + +### Option 3 — Equity refresh +Front headline: Your home grew $___ this year. Want the real number? +Back headline: Stop checking ZILLOW · get the real number +Back body: Zillow's algorithm hasn't seen your kitchen remodel, your roof work, or the comp that sold three doors down last month. The number it's showing you is wrong — and probably low. Get the real one. +CTA line: Free precision equity report +CTA type: Home valuation +QR target URL: [NOT SET — needs URL] +Why this works: Pride + curiosity + tangible payoff. Equity archetype hasn't run since 03/01/25 — feels fresh +Audience fit: Both +⚠ Verification needed: Fill $___ blank with real EPA appreciation number before print + +### Option 4 — WILDCARD · Live market activity +Front headline: 11 offers. 6 days. Same zip code as you. +Back headline: This is what the EPA market looks like RIGHT NOW +Back body: If you've been waiting for the "right time" to sell, this is your signal. A home in your zip code just closed at 11 offers in under a week. Your home would compete in the same environment. +CTA line: Free market timing check +CTA type: Free report (market) +QR target URL: [NOT SET — needs URL] +Why this works: Hyper-local social proof + urgency. New archetype — if it lands, add as #7 to library. +Audience fit: Both +⚠ Verification needed: Requires real recent EPA multi-offer comp — do not ship without it + +--- + +## Format reference (for the cron job to follow) + +When Workflow B emails options, append an entry like this: + +```markdown +## [TARGET_MAIL_DATE] (emailed [SENT_DATE]) +Status: pending pick + +### Option 1 — [Archetype name] +Front headline: [Plain text with markup] +Back headline: [Plain text] +Back body: [3-sentence italic body] +CTA line: [Gold CTA tagline] +CTA type: [home valuation / testimonials / free report / etc.] +QR target URL: [URL from cta-router.md] +Why this works: [One-line rationale] +Audience fit: [Farm / Past clients / Both] + +### Option 2 — [Archetype name] +... + +### Option 3 — [Archetype name] +... +``` + +--- + +## Resolved + +*(Picked options move here after the user selects one. Keeps a permanent record of what was offered + what was chosen for pattern analysis over time.)* + +## 2026-06-15 (emailed 2026-05-27 — FULL PIPELINE TEST, fresh remixes) +Status: superseded by 2026-06-08 scheduled run + +### Option 1 — Buyer-tagged (FRESH REMIX) +Front headline: Your home is on someone's list. They just haven't seen it yet. +Back headline: I have buyers looking for your street — and they're pre-approved +Back body: When buyers tour properties on your block, my system tags them. Some have been waiting months for a home like yours to come available. Want me to see if you match? +CTA line: 5-minute match check · free +CTA type: Call/text Graeham +QR target URL: tel:+16503084727 +Why this works: No number to fabricate. "Someone's list" creates inevitability. +Audience fit: Farm + +### Option 2 — Anti-Zillow buyer pool (FRESH REMIX) +Front headline: The buyers Zillow can't show you. +Back headline: Off-market buyers don't browse — they get TAGGED +Back body: Public portals show every casual scroller. My pipeline is the buyers who've already proven they'll close — pre-approved, agent-vetted, ready. +CTA line: Free off-market match check +CTA type: Off-market buyers +QR target URL: [NOT SET] +Why this works: 6-word headline. Reframes Zillow's audience as a feature for the seller, not against. +Audience fit: Both + +### Option 3 — Equity refresh (FRESH REMIX) +Front headline: You're sitting on more than you think. +Back headline: The equity check most owners never get +Back body: Zillow gives you a zip-code average. Your bank gives you last year's appraisal. Neither sees your remodel, your block's recent sales, or what your home would actually trade for today. +CTA line: Free precision equity report +CTA type: Home valuation +QR target URL: [NOT SET] +Why this works: No dollar amount to fabricate. Positive frame pulls pride + curiosity without market anxiety. +Audience fit: Both + +### Option 4 — WILDCARD · Value Gap (loss aversion angle) +Front headline: Most EPA owners are undervaluing their home. +Back headline: Find out where YOUR home really sits +Back body: The gap between what Zillow shows and what a home actually trades for is often double-digit percent. If you're sitting on six figures of unrecognized equity, you should at least know it. +CTA line: See where you really stand +CTA type: Home valuation +QR target URL: [NOT SET] +Why this works: Loss aversion is stronger than gain. New sub-angle — if it lands, add as 7th archetype to library. +Audience fit: Both +⚠ Verification needed: Need a real local EPA gap stat before print diff --git a/skills/farming-postcard/references/phase2-fugu-blueprint.md b/skills/farming-postcard/references/phase2-fugu-blueprint.md new file mode 100644 index 00000000..856df954 --- /dev/null +++ b/skills/farming-postcard/references/phase2-fugu-blueprint.md @@ -0,0 +1,1007 @@ +# Phase 2 Build Blueprint — Fugu-Ultra investigation (2026-06-29) + +Definitive root-cause + permanent-fix architecture from Fugu-Ultra (Sakana), commissioned after the July 1 preview silently no-showed. Phase 1 (server-side reminder + watchdog) is already live; this is the blueprint for Phase 2 (server-side option GENERATION via Sakana) + the evergreen CTA landing-page fix. Build order is at the bottom (Do-now vs Fast-follow). + +--- + +Definitive finding: the root cause is complete. The permanent fix is to make `online-content` the single production system for scheduling, generation, state, CTA URLs, delivery, and alerting. Local scheduled tasks must never be in the production path again. + +## 1. Root cause: complete and correct + +Yes. The root cause is complete and correct. + +The failure was not a bad cron. It was a bad automation boundary. + +### Root cause + +1. **The scheduler was local and app-state-dependent.** + The June 24 preview job existed, but it only ran when the desktop app was open. The shared `lastRunAt = 2026-06-28T17:52:32Z` across unrelated tasks proves a catch-up batch after the app reopened. The June 24 deadline was missed. + +2. **The catch-up path was not reliable.** + The June 28 catch-up produced: + - no email, + - no schedule log, + - no cached creative options, + - no durable run artifact, + - no alert. + +3. **The SMTP credential path was brittle and stale.** + `send_options_email.py` had `APP_PASSWORD_FILE_LINUX` pinned to a retired local session path, so even a task-body execution could fail before resolving the Gmail app password. + +Final finding: + +> The preview system failed because a deadline-critical marketing automation was implemented as a local interactive-app scheduled task with a brittle local credential dependency and no durable artifact, no independent alert, and no self-healing watchdog. The app being closed caused the June 24 trigger to miss; the June 28 catch-up then failed silently. + +Phase 1 fixed the biggest architectural defect: the trigger now runs server-side in GitHub Actions and the local tasks are disabled. + +### Additional hardening to do + +Do these. They close the remaining recurrence paths. + +1. **Permanently remove the local scheduler from production.** + Disabled is good now. After two clean GitHub Action cycles, delete or archive the local scheduled tasks. + +2. **Delete the stale local credential footgun.** + Remove or hard-fail `APP_PASSWORD_FILE_LINUX` in `send_options_email.py`. Production credentials must come only from GitHub Actions secrets. + +3. **Separate generated-options state from email-sent state.** + + Use both: + + ```text + data/farming-postcards/previews/YYYY-MM-DD/options.json + data/farming-postcards/sent/YYYY-MM-DD.json + data/farming-postcards/failures/YYYY-MM-DD.json + ``` + + Write the options artifact before email. Write the sent marker only after SMTP success. + +4. **Make the watchdog self-healing.** + Current watchdog alerts. Upgrade it: + + - no options artifact inside preview window → generate options; + - options artifact exists but no sent marker → resend email; + - repeated failure → open/update GitHub Issue; + - drop is close and still no selection/finalization → escalate. + +5. **Add a concurrency guard for dual DST-safe crons.** + + ```yaml + concurrency: + group: farming-postcard-preview-${{ github.ref }} + cancel-in-progress: false + ``` + + The committed marker gives idempotency. The concurrency guard prevents two scheduled runs from racing. + +6. **Gate every scheduled run by Pacific local date.** + Compute using `America/Los_Angeles`, not UTC day. + + ```python + def drop_date_for_preview_day(today_pt): + if today_pt.day == 8: + return date(today_pt.year, today_pt.month, 15) + if today_pt.day == 24: + if today_pt.month == 12: + return date(today_pt.year + 1, 1, 1) + return date(today_pt.year, today_pt.month + 1, 1) + return None + ``` + +7. **Add SMTP-independent success visibility.** + Email can succeed at SMTP and still land in spam. On successful generation, also create/update a GitHub Issue titled: + + ```text + Postcard options ready: YYYY-MM-DD drop + ``` + + Put the rendered options and artifact link in the issue. Close it when Graeham selects the option. + +8. **Protect against GitHub scheduled-workflow disappearance.** + Keep the repo active with committed state changes and add a monthly heartbeat/status check so the schedule cannot silently age out. + +9. **Add tests for the exact failure class.** + + Required tests: + + - missing Gmail secret fails loudly; + - missing Sakana key fails loudly; + - invalid LLM JSON does not write a sent marker; + - SMTP failure preserves options artifact and opens an issue; + - watchdog can recover a missing scheduled run; + - Pacific date logic handles December 24 → January 1; + - dual cron does not double-send. + +--- + +## 2. Phase 2: yes, GitHub Action should generate options server-side + +Yes. The GitHub Action should generate the 3–5 hook options itself by calling Sakana/Fugu server-side. This fully replaces the local scheduled task. + +The scheduled Action becomes the production source of truth. The local task remains disabled and later deleted. + +### Permanent Phase 2 architecture + +```text +GitHub Actions + ├── schedule / workflow_dispatch / watchdog + ├── compute Pacific preview date and drop date + ├── load shipped-card history + ├── load CTA inventory + ├── compute deterministic blocked sets + ├── call Sakana/Fugu for candidate options + ├── validate with Python, not the LLM + ├── top up with deterministic fallback if needed + ├── commit options artifact + ├── email Graeham + ├── open/update GitHub Issue + └── commit sent marker only after SMTP success +``` + +Recommended files: + +```text +.github/workflows/farming-postcard-reminder.yml +scripts/postcard_preview_generate.py +scripts/validate_postcard_options.py +scripts/sync_farming_postcard_history.py +data/farming-postcards/history.json +data/farming-postcards/cta_inventory.json +data/farming-postcards/claim_library.json +data/farming-postcards/template_bank.json +data/farming-postcards/previews/ +data/farming-postcards/sent/ +data/farming-postcards/failures/ +``` + +### Scheduling behavior + +Keep the existing 8th/24th dual UTC cron strategy, but make the script authoritative. + +Run flow: + +1. Compute `today_pt` in `America/Los_Angeles`. +2. If local day is not `8` or `24`, scheduled REMIND/GENERATE mode exits cleanly. +3. If local day is `8`, target drop is the `15th` of the same month. +4. If local day is `24`, target drop is the `1st` of the next month. +5. If `sent/YYYY-MM-DD.json` exists, exit cleanly. +6. If `previews/YYYY-MM-DD/options.json` exists but sent marker does not, resend existing artifact. +7. Otherwise generate new options. + +### History recommendation + +Vendor and canonicalize the history into `online-content`. + +Do not make the scheduled run depend on a second repo at runtime. + +The current history lives in another repo in: + +```text +farming-postcard/archive.json +headline-library.md +``` + +Do a one-time import into: + +```text +online-content/data/farming-postcards/history.json +``` + +Then make `online-content` the canonical history store going forward. The finalize/ship workflow should append the selected shipped card there. + +Reason: a scheduled production job must not depend on a second repo’s permissions, branch, path stability, network availability, or token. A cross-repo runtime dependency is another silent-failure vector. + +Normalized history schema: + +```json +{ + "schema_version": "farming_postcard_history.v1", + "updated_at": "2026-07-01T00:00:00Z", + "cards": [ + { + "drop_date": "2026-07-01", + "status": "shipped", + "headline": "Text Graeham for your East Palo Alto seller check", + "archetype": "seller_consultation", + "cta_destination_id": "sms_keyword", + "villain_type": "none", + "core_claim_slug": "local-human-guidance-beats-generic-advice", + "source": { + "legacy_archive_path": "farming-postcard/archive.json", + "legacy_tracker_path": "headline-library.md", + "legacy_commit": "abc123" + } + } + ] +} +``` + +### Deterministic 4-axis guardrails + +The LLM should receive constraints, but Python enforces them. The model is allowed to propose; the validator decides. + +Build blocked sets from shipped history: + +```python +def shipped_cards(history): + cards = [c for c in history["cards"] if c["status"] == "shipped"] + return sorted(cards, key=lambda c: c["drop_date"], reverse=True) + +def build_constraints(history): + shipped = shipped_cards(history) + + last_3 = shipped[:3] + last_2 = shipped[:2] + last_4 = shipped[:4] + + return { + "blocked_archetypes": sorted({c["archetype"] for c in last_3}), + "blocked_cta_destination_ids": sorted({c["cta_destination_id"] for c in last_2}), + "zillow_algorithm_villain_blocked": any( + c.get("villain_type") == "zillow_algorithm" + for c in last_2 + ), + "blocked_core_claim_slugs": sorted({c["core_claim_slug"] for c in last_4}) + } +``` + +#### Rule 1: archetype 3-card cooldown + +Reject an option if: + +```python +option["archetype"] in constraints["blocked_archetypes"] +``` + +Use a controlled enum, for example: + +```json +[ + "local_market_signal", + "seller_math", + "buyer_demand", + "neighbor_story", + "myth_buster", + "equity_check", + "prop_19", + "seasonal_timing", + "off_market_opportunity", + "seller_consultation" +] +``` + +You need at least 6 archetypes available because the cooldown blocks 3 and the preview must produce at least 3 distinct options. Ten is healthy. + +#### Rule 2: CTA destination differs from last 2 cards + +Reject an option if: + +```python +option["cta_destination_id"] in constraints["blocked_cta_destination_ids"] +``` + +Compare by destination ID, not URL. + +Correct: + +```text +home_valuation +off_market_buyers +prop_19_guide +sms_keyword +seller_consultation +``` + +Incorrect: + +```text +https://site.com/east-palo-alto/home-valuation/?utm_campaign=... +``` + +The LLM must not emit URLs. It emits only `cta_destination_id`. The script constructs the final URL from `cta_inventory.json`. + +#### Rule 3: no Zillow/algorithm villain for 2 cards after a villain card + +If either of the last two shipped cards used a Zillow/algorithm villain, reject any new option with: + +```json +"villain_type": "zillow_algorithm" +``` + +Also text-scan all generated copy during the cooldown. Reject terms like: + +```text +Zillow +algorithm +AVM +automated estimate +online estimate +portal estimate +computer estimate +``` + +This prevents the model from labeling the option `"villain_type": "none"` while still using the forbidden villain in the copy. + +#### Rule 4: core claim not repeated within 4 cards + +Reject an option if: + +```python +option["core_claim_slug"] in constraints["blocked_core_claim_slugs"] +``` + +Use a controlled claim library. Do not let the LLM invent arbitrary claim IDs. + +Important: the claim library must have at least 7 usable claims. Four is not enough because the last four can all be blocked, and the preview still needs three valid distinct options. + +Use 8–10 claims minimum, for example: + +```json +[ + "local-human-pricing-beats-generic-comps", + "off-market-demand-can-exist-before-public-listing", + "pre-listing-prep-increases-buyer-confidence", + "prop-19-planning-can-change-move-timing", + "equity-can-fund-the-next-move", + "inventory-scarcity-rewards-prepared-sellers", + "interest-rate-shifts-change-buyer-math", + "neighborhood-specific-guidance-beats-countywide-averages", + "seller-timing-matters-before-public-launch", + "local-agent-access-creates-better-selling-options" +] +``` + +Add a feasibility preflight: + +```python +def assert_feasible(all_archetypes, all_claims, all_ctas, constraints): + available_archetypes = set(all_archetypes) - set(constraints["blocked_archetypes"]) + available_claims = set(all_claims) - set(constraints["blocked_core_claim_slugs"]) + available_ctas = set(all_ctas) - set(constraints["blocked_cta_destination_ids"]) + + problems = [] + + if len(available_archetypes) < 3: + problems.append("fewer than 3 available archetypes") + + if len(available_claims) < 3: + problems.append("fewer than 3 available core claims") + + if len(available_ctas) < 1: + problems.append("no available CTA destination") + + if problems: + raise RuntimeError("; ".join(problems)) +``` + +Within a candidate batch, require distinct `archetype` and distinct `core_claim_slug`. CTA does not need to be distinct within the batch unless you build at least 5 CTA destinations. Only one option ships. + +### LLM call design + +Ask Sakana/Fugu for 7–8 candidates, not 3. Then validate and select the best 3–5. + +Prompt content: + +- farm: East Palo Alto; +- agent: Graeham; +- target drop date; +- allowed archetypes; +- blocked archetypes; +- allowed CTA destination IDs; +- blocked CTA destination IDs; +- allowed core claim slugs; +- blocked core claim slugs; +- whether Zillow/algorithm villain is blocked; +- required JSON schema; +- instruction that invalid options will be discarded. + +The prompt should say: + +```text +Return strict JSON only. Do not return Markdown. Do not invent CTA URLs. Use only allowed enum values. Any option violating the constraints will be discarded. +``` + +### Output schema + +The LLM returns candidate options without URLs: + +```json +{ + "schema_version": "postcard_option_candidates.v1", + "candidates": [ + { + "archetype": "off_market_opportunity", + "core_claim_slug": "off-market-demand-can-exist-before-public-listing", + "villain_type": "none", + "headline": "Before You List, See Who’s Already Looking in East Palo Alto", + "subheadline": "A quiet buyer check can change your selling strategy.", + "hook_summary": "Positions Graeham as the local agent who can surface buyer demand before a public listing.", + "cta_destination_id": "off_market_buyers", + "cta_text": "Scan to see current off-market buyer demand", + "front_concept": "Clean headline, East Palo Alto map texture, QR callout.", + "back_copy_outline": [ + "Open with the idea that timing matters.", + "Explain that some buyers are already searching quietly.", + "Invite the owner to request a private buyer-demand check." + ], + "proof_needed": [ + "Recent buyer inquiry count or CRM-safe qualitative note" + ], + "compliance_notes": [ + "Do not guarantee a buyer or sale price." + ] + } + ] +} +``` + +The committed artifact is generated by the script after validation and URL construction: + +```json +{ + "schema_version": "postcard_preview_options.v1", + "drop_date": "2026-07-15", + "preview_due_date": "2026-07-08", + "farm": "East Palo Alto", + "agent": "Graeham", + "generated_at": "2026-07-08T15:05:00Z", + "generator": { + "provider": "sakana_fugu", + "model": "configured-in-github-variable", + "mode": "scheduled" + }, + "history_source": { + "path": "data/farming-postcards/history.json" + }, + "constraints": { + "blocked_archetypes": ["seller_math", "prop_19", "buyer_demand"], + "blocked_cta_destination_ids": ["sms_keyword", "home_valuation"], + "zillow_algorithm_villain_blocked": false, + "blocked_core_claim_slugs": [ + "pre-listing-prep-increases-buyer-confidence", + "equity-can-fund-the-next-move" + ] + }, + "options": [ + { + "id": "2026-07-15-A", + "source": "llm", + "archetype": "off_market_opportunity", + "core_claim_slug": "off-market-demand-can-exist-before-public-listing", + "villain_type": "none", + "headline": "Before You List, See Who’s Already Looking in East Palo Alto", + "subheadline": "A quiet buyer check can change your selling strategy.", + "hook_summary": "Positions Graeham as the local agent who can surface buyer demand before a public listing.", + "cta_destination_id": "off_market_buyers", + "cta_text": "Scan to see current off-market buyer demand", + "cta_url": "https://example.com/east-palo-alto/off-market-buyers/?utm_source=postcard&utm_medium=qr&utm_campaign=2026-07-15&utm_content=A&drop_date=2026-07-15&cta_destination_id=off_market_buyers&qr_id=2026-07-15-A", + "front_concept": "Clean headline, East Palo Alto map texture, QR callout.", + "back_copy_outline": [ + "Open with the idea that timing matters.", + "Explain that some buyers are already searching quietly.", + "Invite the owner to request a private buyer-demand check." + ], + "proof_needed": [ + "Recent buyer inquiry count or CRM-safe qualitative note" + ], + "compliance_notes": [ + "Do not guarantee a buyer or sale price." + ], + "rule_check": { + "archetype_3_card_cooldown": true, + "cta_not_used_last_2": true, + "zillow_algorithm_villain_cooldown": true, + "core_claim_not_used_last_4": true + } + } + ] +} +``` + +### Validation + +Reject the batch unless the final selected set has 3–5 valid options. + +Per-option validation: + +- `headline` non-empty; +- `subheadline` non-empty; +- `hook_summary` non-empty; +- `archetype` in allowed enum; +- `archetype` not used in last 3 shipped cards; +- `core_claim_slug` in claim library; +- `core_claim_slug` not used in last 4 shipped cards; +- `cta_destination_id` in `cta_inventory.json`; +- `cta_destination_id` not used in last 2 shipped cards; +- `villain_type` in allowed enum; +- if villain cooldown active, no Zillow/algorithm villain and no banned terms; +- no guaranteed sale price; +- no guaranteed buyer; +- no fake valuation promise; +- no invented URL; +- no empty CTA; +- no duplicate headline in the batch. + +Batch validation: + +- 3–5 final options; +- distinct `archetype` across final options; +- distinct `core_claim_slug` across final options; +- all options pass all four differentiation rules; +- every CTA URL is constructed server-side from approved inventory; +- artifact is committed before email; +- sent marker is committed only after SMTP success. + +### CTA URL construction + +The model emits: + +```json +"cta_destination_id": "home_valuation" +``` + +The script constructs: + +```python +def build_cta_url(destination, drop_date, option_letter): + canonical = destination["canonical_url"] + if canonical is None: + return None + + return ( + f"{canonical}" + f"?utm_source=postcard" + f"&utm_medium=qr" + f"&utm_campaign={drop_date}" + f"&utm_content={option_letter}" + f"&drop_date={drop_date}" + f"&cta_destination_id={destination['id']}" + f"&qr_id={drop_date}-{option_letter}" + ) +``` + +This permanently eliminates LLM-invented landing-page URLs. + +### Fallback ladder + +Use this exact failure ladder: + +1. **Sakana call fails** → retry with exponential backoff. +2. **Sakana returns invalid JSON** → make one repair call using validator errors. +3. **Still invalid or fewer than 3 valid options** → keep valid LLM options and top up from deterministic templates. +4. **Template top-up succeeds** → email options and open an issue noting degraded mode. +5. **History, claim library, or CTA inventory is infeasible/corrupt** → send the existing deterministic REMINDER email and open a failure issue. +6. **SMTP fails** → commit options artifact, do not write sent marker, open/update issue. +7. **Watchdog sees artifact but no sent marker** → resend email from the artifact. +8. **Watchdog sees neither artifact nor sent marker inside the 7-day window** → run generation. +9. **Drop is within 3 days and still no successful email** → subject line becomes urgent: + + ```text + URGENT: Postcard options for YYYY-MM-DD drop — late recovery + ``` + +The deterministic fallback should use a committed template bank: + +```text +data/farming-postcards/template_bank.json +``` + +Each template maps: + +```text +archetype + core_claim_slug + cta_destination_id +``` + +to conservative headline/subheadline/copy structures. + +--- + +## 3. Secrets: safest Sakana key wiring + +Use a GitHub Actions repository secret or production environment secret named: + +```text +SAKANA_API_KEY +``` + +The human sets it here: + +```text +online-content repo +→ Settings +→ Secrets and variables +→ Actions +→ New repository secret +``` + +Use non-secret GitHub variables for config: + +```text +SAKANA_BASE_URL +SAKANA_MODEL +POSTCARD_TO_EMAIL +POSTCARD_FROM_NAME +``` + +Use the secret only at the step level: + +```yaml +permissions: + contents: write + issues: write + +jobs: + postcard-preview: + runs-on: ubuntu-latest + concurrency: + group: farming-postcard-preview-${{ github.ref }} + cancel-in-progress: false + + steps: + - uses: actions/checkout@v4 + + - name: Generate postcard preview options + env: + SAKANA_API_KEY: ${{ secrets.SAKANA_API_KEY }} + SAKANA_BASE_URL: ${{ vars.SAKANA_BASE_URL }} + SAKANA_MODEL: ${{ vars.SAKANA_MODEL }} + GMAIL_USERNAME: ${{ secrets.GMAIL_USERNAME }} + GMAIL_APP_PASSWORD: ${{ secrets.GMAIL_APP_PASSWORD }} + POSTCARD_TO_EMAIL: ${{ vars.POSTCARD_TO_EMAIL }} + run: | + python scripts/postcard_preview_generate.py --mode scheduled +``` + +Add an explicit preflight: + +```python +if not os.environ.get("SAKANA_API_KEY"): + raise RuntimeError("SAKANA_API_KEY is not configured") + +if not os.environ.get("GMAIL_USERNAME"): + raise RuntimeError("GMAIL_USERNAME is not configured") + +if not os.environ.get("GMAIL_APP_PASSWORD"): + raise RuntimeError("GMAIL_APP_PASSWORD is not configured") +``` + +Avoid all of this: + +- Do not commit the key. +- Do not put it in GitHub Pages JavaScript. +- Do not put it in `.env` files. +- Do not upload `.env` files as artifacts. +- Do not echo the key. +- Do not run with `set -x`. +- Do not log request headers. +- Do not include the key in prompts. +- Do not include the key in committed artifacts. +- Do not pass secrets to untrusted pull-request workflows. +- Do not use `pull_request_target` for this automation. +- Do not add a cross-repo PAT to the scheduled generation path. +- Do not use local credential files in production. + +--- + +## 4. Permanent fix for CTA / landing-page blocker + +Build evergreen CTA landing pages on the existing GitHub Pages site and make the generator choose only from an approved CTA inventory. + +Minimum viable version: three branded landing pages plus GHL forms. + +### Required MVP pages + +1. **Home valuation** + + ```text + /east-palo-alto/home-valuation/ + ``` + + CTA: + + ```text + Request a human East Palo Alto home-value review + ``` + +2. **Off-market buyers** + + ```text + /east-palo-alto/off-market-buyers/ + ``` + + CTA: + + ```text + See if there are quiet buyers for your home + ``` + +3. **Prop 19 guide** + + ```text + /east-palo-alto/prop-19-guide/ + ``` + + CTA: + + ```text + Get the Prop 19 move-planning checklist + ``` + +Three evergreen destinations are the minimum because the CTA rule blocks the last two shipped destinations. With three destinations, there is always at least one legal non-blocked CTA. + +Keep `sms_keyword` as an emergency fallback, not the primary plan. + +### Required page contents + +Each page must have: + +- Graeham branding; +- East Palo Alto-specific headline; +- short benefit copy; +- embedded GHL form; +- phone/text fallback; +- brokerage/legal footer; +- thank-you redirect; +- GHL tracking script or analytics script; +- UTM capture into hidden fields. + +Hidden fields: + +```text +utm_source +utm_medium +utm_campaign +utm_content +drop_date +cta_destination_id +qr_id +page_slug +``` + +Minimal UTM capture script: + +```html + +``` + +### GHL setup + +In GHL: + +- one form per CTA, or one reusable form with hidden `cta_destination_id`; +- create/update contact on submit; +- tag contact; +- notify Graeham immediately; +- redirect to thank-you page. + +Recommended tags: + +```text +Postcard QR +East Palo Alto Farm +CTA: Home Valuation +CTA: Off-Market Buyers +CTA: Prop 19 +``` + +### CTA inventory + +Commit this file: + +```text +data/farming-postcards/cta_inventory.json +``` + +Example: + +```json +{ + "schema_version": "cta_inventory.v1", + "destinations": [ + { + "id": "home_valuation", + "label": "Home valuation request", + "canonical_url": "https://example.com/east-palo-alto/home-valuation/", + "allowed": true + }, + { + "id": "off_market_buyers", + "label": "Off-market buyer demand check", + "canonical_url": "https://example.com/east-palo-alto/off-market-buyers/", + "allowed": true + }, + { + "id": "prop_19_guide", + "label": "Prop 19 guide", + "canonical_url": "https://example.com/east-palo-alto/prop-19-guide/", + "allowed": true + }, + { + "id": "sms_keyword", + "label": "Call/text fallback", + "canonical_url": null, + "allowed": true + } + ] +} +``` + +The generator must only use destinations in this file. + +No approved destination in `cta_inventory.json` means the CTA is not eligible. The LLM never invents landing pages. + +--- + +## 5. Prioritized build order + +## Do now + +1. **Keep Phase 1 live.** + GitHub Actions remains the only production scheduler. Local tasks stay disabled. + +2. **Remove the stale local credential path.** + Delete or hard-fail `APP_PASSWORD_FILE_LINUX` in the old send script. + +3. **Add durable state directories.** + + ```text + data/farming-postcards/previews/ + data/farming-postcards/sent/ + data/farming-postcards/failures/ + ``` + +4. **Canonicalize history into `online-content`.** + Import the other repo’s `archive.json` and `headline-library.md` into: + + ```text + data/farming-postcards/history.json + ``` + + Going forward, the finalize/ship workflow writes to this file. + +5. **Create the controlled libraries.** + + ```text + data/farming-postcards/cta_inventory.json + data/farming-postcards/claim_library.json + data/farming-postcards/template_bank.json + ``` + + Use at least: + - 8–10 archetypes; + - 8–10 core claims; + - 3 CTA landing-page destinations plus SMS fallback. + +6. **Build the three evergreen landing pages.** + + ```text + /east-palo-alto/home-valuation/ + /east-palo-alto/off-market-buyers/ + /east-palo-alto/prop-19-guide/ + ``` + +7. **Wire the pages to GHL.** + Embed forms, capture UTMs, tag contacts, notify Graeham, and test one real submission per page. + +8. **Add `SAKANA_API_KEY` to GitHub Actions secrets.** + +9. **Implement `postcard_preview_generate.py`.** + Required behavior: + - Pacific date gating; + - drop-date computation; + - idempotency by drop date; + - load history; + - load CTA inventory; + - load claim library; + - feasibility preflight; + - build deterministic blocked sets; + - call Sakana/Fugu; + - validate strict JSON; + - enforce all four differentiation rules; + - construct CTA URLs server-side; + - top up from deterministic templates if needed; + - commit options artifact; + - email Graeham; + - open/update GitHub Issue; + - write sent marker only after SMTP success. + +10. **Add workflow concurrency.** + + ```yaml + concurrency: + group: farming-postcard-preview-${{ github.ref }} + cancel-in-progress: false + ``` + +11. **Upgrade watchdog to self-healing.** + + - Missing options artifact → generate. + - Artifact exists but no sent marker → resend. + - Repeated failure → issue. + - Close to drop with no selection → urgent escalation. + +12. **Run `workflow_dispatch` end-to-end.** + Verify: + - options generated; + - all rules enforced; + - artifact committed; + - email received; + - GitHub Issue created/updated; + - sent marker committed; + - SMTP failure path works; + - invalid LLM output path works; + - watchdog recovery path works. + +13. **Flip scheduled mode from REMIND to GENERATE.** + Keep REMIND only as the terminal fallback rung. + +## Fast-follow + +1. **Add selected/finalized markers.** + + ```text + data/farming-postcards/selected/YYYY-MM-DD.json + data/farming-postcards/finalized/YYYY-MM-DD.json + ``` + + The chain should track: generated → sent → selected → finalized → mailed. + +2. **Add per-drop QR wrapper pages.** + + ```text + /qr/2026-07-15/home-valuation-a/ + /qr/2026-07-15/off-market-buyers-b/ + /qr/2026-07-15/prop-19-c/ + ``` + +3. **Generate QR images automatically.** + Store PNG/SVG QR assets with the preview artifact. + +4. **Add a fourth evergreen CTA page.** + + ```text + /east-palo-alto/seller-consultation/ + ``` + + This gives more CTA rotation headroom. + +5. **Add weekly link checking.** + Every URL in `cta_inventory.json` must return 200. + +6. **Add analytics dashboard.** + Track: + - QR scans; + - landing-page views; + - form submissions; + - conversion by drop; + - conversion by CTA destination. + +7. **Add SMS escalation for critical misses.** + GitHub Issues are the SMTP-independent baseline. SMS is the next layer for T-minus-3-day failures. + +8. **Delete the old local task wrappers after two clean cycles.** + +Final permanent architecture: + +> GitHub Actions schedules, watches, generates, validates, persists, emails, and alerts. Sakana/Fugu supplies candidate creative. Python enforces the rules. Git commits preserve state. GitHub Issues provide SMTP-independent visibility. GitHub Pages supplies evergreen CTA URLs. GHL captures QR leads. Local scheduled tasks are removed from production. + +=== tokens === +[fugu] model=fugu-ultra tokens: prompt=2330 completion=11259 total=81665 diff --git a/skills/farming-postcard/references/print-specs.md b/skills/farming-postcard/references/print-specs.md new file mode 100644 index 00000000..9c533fd6 --- /dev/null +++ b/skills/farming-postcard/references/print-specs.md @@ -0,0 +1,91 @@ +# Print Specs — Universal Mail Works Defaults + +⚠️ **VERIFY BEFORE FIRST PRINT RUN** — UMW's exact spec sheet isn't locked in this skill. These are industry-standard 6×4 postcard defaults that should work for most vendors but should be confirmed with UMW before first print. + +## Default specs + +| Spec | Value | Notes | +|---|---|---| +| Trim size | 6" × 4" | Standard landscape postcard | +| Bleed | 0.125" each side | Total canvas: 6.25" × 4.25" | +| Safe zone | 0.25" from trim edge | Keep type/important elements inside this | +| Resolution | 300 DPI | For any raster images (headshots) | +| Color mode | CMYK | RGB will color-shift on press | +| File format | PDF/X-1a preferred, PDF/X-4 acceptable | Print-ready PDF standards | +| Fonts | Embedded or outlined | Outline to be safe (no font substitution risk) | + +## PDF render pipeline + +Step 1: Render HTML to PDF at 300 DPI using headless browser: + +```bash +# Install once +pip install playwright --break-system-packages --quiet +python -m playwright install chromium + +# Render +python -c " +from playwright.sync_api import sync_playwright +import sys +html_path = sys.argv[1] +pdf_path = sys.argv[2] +with sync_playwright() as p: + browser = p.chromium.launch() + page = browser.new_page() + page.goto(f'file://{html_path}') + page.pdf(path=pdf_path, width='6.25in', height='4.25in', print_background=True, margin={'top':'0','bottom':'0','left':'0','right':'0'}) + browser.close() +" "[HTML_PATH]" "[PDF_PATH]" +``` + +Step 2: For production-grade CMYK conversion, the print shop will typically handle this. Optionally pre-convert using Ghostscript: + +```bash +gs -dSAFER -dBATCH -dNOPAUSE -dNOCACHE -sDEVICE=pdfwrite \ + -sColorConversionStrategy=CMYK \ + -dProcessColorModel=/DeviceCMYK \ + -sOutputFile=output_cmyk.pdf input.pdf +``` + +## CMYK approximations of gold (`#C2A14E`) + +If the printer asks for CMYK specifically: +- Coated stock: C:25 M:35 Y:75 K:5 +- Uncoated stock: C:20 M:30 Y:70 K:0 + +These shift slightly between presses. If color match matters, ask UMW for a press proof on the first run. + +## Vendor-specific notes + +### Universal Mail Works (default) +- Specs to confirm: trim size options, bleed requirement, file format preference, EDDM eligibility +- Once confirmed, update this file with their official spec sheet URL/values +- Status: ⚠️ NOT YET CONFIRMED + +### Wise Pelican (backup vendor) +- 6×4.25 standard (slightly taller than UMW default) +- 0.125" bleed +- PDF/X-1a or X-4 +- Spec sheet: https://www.wisepelican.com/sizes-and-specifications + +### Corefact (jumbo option) +- 6×9 jumbo postcards +- Triggers a different layout system entirely — current template is locked to 6×4 proportions + +## EDDM (Every Door Direct Mail) + +- Minimum size: 6.125" × 4.25" (UMW default fits) +- Must include EDDM indicia on the address panel +- Discounted postage rate (~$0.20/piece vs ~$0.34 First Class) +- EDDM doesn't allow targeted lists — entire postal routes only + +## Pre-print checklist + +Before sending any card to UMW for the first run: +- [ ] Verify UMW exact spec sheet (trim, bleed, resolution, format) +- [ ] Confirm CMYK color match with press proof if budget allows +- [ ] QR code tested with at least 3 phone cameras at arm's length +- [ ] Address panel + indicia placement confirmed with UMW +- [ ] Disclaimer text legible at print scale +- [ ] Phone number + URL spelled correctly (review twice) +- [ ] Graeham's name spelled correctly diff --git a/skills/farming-postcard/references/schedule-log.md b/skills/farming-postcard/references/schedule-log.md new file mode 100644 index 00000000..daa6455e --- /dev/null +++ b/skills/farming-postcard/references/schedule-log.md @@ -0,0 +1,48 @@ +# Schedule Log — Cron Run History + +Each time Workflow B (scheduled preview) fires, append one line here: + +``` +[YYYY-MM-DD HH:MM] [run_type] target=[mail_date] options=[N] email_status=[draft_created|sent|failed] +``` + +`run_type` values: `15th-preview` (fires 8th of month) or `1st-preview` (fires 24th of month). + +--- + +## Runs + +[2026-05-27 14:00] TEST RUN (manual fire, draft only) target=2026-06-15 options=4 email_status=draft_created + - Archetypes: Buyer-tagged, Anti-Zillow buyer pool, Equity refresh, WILDCARD live market activity + - Triggered to validate pipeline before June 8 cron + - Gmail draft created — not sent (no SMTP credential yet) + +[2026-05-27 23:55] REAL SEND TEST (SMTP, on-brand v3 template) target=2026-06-15 options=4 email_status=sent + - Recipients: graehamwatts@gmail.com + graehamwattsvideo@gmail.com + - Subject: [ON-BRAND v3 TEST] Postcard options for June 15 — system check from Cowork + - SMTP via smtp.gmail.com:465, authenticated with App Password + - Pipeline confirmed working + +[2026-05-27 23:59] FULL PIPELINE TEST (SMTP, fresh remixed headlines) target=2026-06-15 options=4 email_status=sent + - Recipients: graehamwatts@gmail.com + graehamwattsvideo@gmail.com + - Subject: [TEST EMAIL] Postcard options for June 15 — full pipeline test + - Archetypes: Buyer-tagged (NEW remix), Anti-Zillow buyer pool (NEW remix), Equity refresh (NEW remix), WILDCARD Value Gap (new sub-angle) + - Demonstrates skill generates FRESH headlines vs. repeating prior options + - Same archetype slate as prior test, completely different headline copy — validates remix patterns work + - Pipeline confirmed end-to-end. June 8 cron will fire cleanly without intervention. + +[2026-06-24 08:00] 1st-preview target=2026-07-01 options=0 email_status=MISSED_NO_FIRE + - ⚠ TASK DID NOT FIRE ON SCHEDULE. App was closed on the 24th; the local Claude Code task `farming-postcard-1st-preview` (cron 0 8 24 * *) skipped its window. + - A catch-up batch ran 2026-06-28 17:52 UTC (lastRunAt) alongside many other overdue tasks, but it produced NO options, NO cache entry, NO email — the workflow never completed. + - Graeham discovered the gap on 2026-06-29 with the July 1 drop only 2 days out. Card built manually same day (Prop 19 Tax Transfer). See option-cache "2026-07-01 (manual recovery)". + - ROOT CAUSE: local scheduled tasks only fire when the Cowork/Claude Code app is open. FIX FORWARD: migrate the two farming-postcard previews to a cloud GitHub Action cron (same pattern as daily-attribution-brief), so they fire regardless of app state. Tracked as OPEN. + +[2026-06-08 08:00] 15th-preview target=2026-06-15 options=4 email_status=sent + - Recipients: graehamwatts@gmail.com + graehamwattsvideo@gmail.com + graehamwattsvideo2@gmail.com + - Subject: Postcard options for June 15 — pick one by June 12 + - Archetypes offered: Buyer-tagged, Anti-Zillow buyer pool, Equity, WILDCARD Low-Inventory Timing (fresh remixes, not copy/paste) + - Excluded per repetition rule (last 3 shipped): Neighbor envy (06/01), Anti-Zestimate (05/15), AI search invisibility (05/01) + - NOTE: AI search is a 15th-cadence bias target but shipped 05/01/26 (within last-3 window) -> excluded. Only 3 fresh library archetypes remained, so 1 fresh wildcard was added to reach 4 options. + - Send: reused send_options_email.py send logic + its locked 3-recipient list; credential resolved from the live session mount. + - FLAG: send_options_email.py APP_PASSWORD_FILE_LINUX is pinned to a retired session id (inspiring-awesome-hawking). A literal bash run of that script from any other session would fail credential lookup even though the password is valid. Recommend making that fallback session-agnostic (glob /sessions/*/mnt/Skills/). + - Cached at option-cache.md under Pending picks; prior June-15 TEST entries marked superseded. diff --git a/skills/farming-postcard/scripts/send_options_email.py b/skills/farming-postcard/scripts/send_options_email.py new file mode 100644 index 00000000..1c5a6d5e --- /dev/null +++ b/skills/farming-postcard/scripts/send_options_email.py @@ -0,0 +1,86 @@ +#!/usr/bin/env python3 +""" +Send the postcard options preview email via Gmail SMTP. + +Reads the Gmail App Password from C:\\Users\\Graeham Watts\\Documents\\Claude\\Skills\\gmail-app-password.txt +Sends FROM graehamwatts@gmail.com TO Graeham + Peter + Ellie. + +Usage: + python send_options_email.py "" + +Or from Python: + from send_options_email import send_options_email + send_options_email(html_body, subject, plaintext_body) +""" +import os +import sys +import smtplib +from email.mime.text import MIMEText +from email.mime.multipart import MIMEMultipart +from pathlib import Path + +# Locked recipients (matches design-tokens.md continuity rule) +SENDER = "graehamwatts@gmail.com" +RECIPIENTS = ["graehamwatts@gmail.com", "graehamwattsvideo@gmail.com", "graehamwattsvideo2@gmail.com"] # Graeham + Peter + +# Credential resolution (2026-06-29, per Fugu reliability review): +# 1. Env var GMAIL_APP_PASSWORD — the ONLY source in production (GitHub Actions secret). +# 2. Local Windows file — convenience for manual/interactive runs only. +# The old APP_PASSWORD_FILE_LINUX path was pinned to a retired Cowork session id +# (`inspiring-awesome-hawking`) and silently failed credential resolution. It has been +# REMOVED — production credentials must come from the env-var secret, never a local path. +APP_PASSWORD_FILE = Path(r"C:\Users\Graeham Watts\Documents\Skills LLMS\Claude\Skills\gmail-app-password.txt") + + +def load_app_password(): + """Resolve the Gmail App Password: env var first (production), then local file. Returns None if not found.""" + env_pwd = os.environ.get("GMAIL_APP_PASSWORD", "").strip().replace(" ", "") + if env_pwd and env_pwd != "PASTE_YOUR_GMAIL_APP_PASSWORD_HERE": + return env_pwd + if APP_PASSWORD_FILE.exists(): + pwd = APP_PASSWORD_FILE.read_text().strip().replace(" ", "") + if pwd and pwd != "PASTE_YOUR_GMAIL_APP_PASSWORD_HERE": + return pwd + return None + + +def send_options_email(html_body: str, subject: str, plaintext_body: str = ""): + """Send the options preview email to both Graeham + Peter via Gmail SMTP.""" + app_password = load_app_password() + if not app_password: + raise RuntimeError( + "Gmail App Password not found. Set the GMAIL_APP_PASSWORD env var (production / " + f"GitHub Actions secret), or save it to {APP_PASSWORD_FILE} for local runs.\n" + "Generate one at https://myaccount.google.com/apppasswords." + ) + + msg = MIMEMultipart("alternative") + msg["Subject"] = subject + msg["From"] = f"Graeham Watts <{SENDER}>" + msg["To"] = ", ".join(RECIPIENTS) + + if plaintext_body: + msg.attach(MIMEText(plaintext_body, "plain")) + msg.attach(MIMEText(html_body, "html")) + + with smtplib.SMTP_SSL("smtp.gmail.com", 465) as server: + server.login(SENDER, app_password) + server.sendmail(SENDER, RECIPIENTS, msg.as_string()) + + return {"sent_to": RECIPIENTS, "subject": subject} + + +if __name__ == "__main__": + if len(sys.argv) < 3: + print("Usage: send_options_email.py '' []") + sys.exit(1) + + html_path = sys.argv[1] + subject = sys.argv[2] + plaintext_path = sys.argv[3] if len(sys.argv) > 3 else None + + html_body = Path(html_path).read_text(encoding="utf-8") + plaintext_body = Path(plaintext_path).read_text(encoding="utf-8") if plaintext_path else "" + + result = send_options_email(html_body, subject, plaintext_body) + print(f"SENT to {result['sent_to']}: {result['subject']}") diff --git a/skills/farming-postcard/templates/option-card.html b/skills/farming-postcard/templates/option-card.html new file mode 100644 index 00000000..b759a3d9 --- /dev/null +++ b/skills/farming-postcard/templates/option-card.html @@ -0,0 +1,38 @@ + +
+ + + +
Option {{OPTION_NUMBER}} · {{ARCHETYPE_NAME}}Lever: {{LEVER}}
+
{{FRONT_HEADLINE_HTML}}
+
{{VERIFICATION_LINE}}
+
+
Back headline
+
{{BACK_HEADLINE_HTML}}
+
Back body
+
{{BACK_BODY}}
+
CTA: {{CTA_LINE}}
+ + + +
CTA type: {{CTA_TYPE}}Audience: {{AUDIENCE}}
+
Why this works: {{WHY_THIS_WORKS}}
+
+
diff --git a/skills/farming-postcard/templates/options-email-template.html b/skills/farming-postcard/templates/options-email-template.html new file mode 100644 index 00000000..902ff02e --- /dev/null +++ b/skills/farming-postcard/templates/options-email-template.html @@ -0,0 +1,61 @@ + + + + + +
+ + + + + + +
  + +
Farming Postcard Options · Auto-Generated
+
Postcard options
for {{TARGET_MAIL_DATE}}
+
+
Hooks for your {{TARGET_MAIL_DATE_SHORT}} farming postcard. Pick by {{PICK_DEADLINE}} so Peter has 3 business days to finalize. Reply with the option number, or open Cowork and say "use option [N] for the {{TARGET_MAIL_DATE_SHORT}} postcard."
+ + + + + +
Last 3 Shipped
{{LAST_3_SHIPPED}}
Cadence
{{CADENCE_LINE}}
Pick Deadline
{{PICK_DEADLINE}}
+ + {{OPTIONS_HTML}} + +
+
How to pick
+
Option A: Reply with the option number ("go with 2").
Option B: Open Cowork: "use option [N] for the {{TARGET_MAIL_DATE_SHORT}} postcard."
Deadline: {{PICK_DEADLINE}}.
+ +
+ +
+ + + +
+
COMPASS
+
The Boyenga Team
+
GRAEHAM WATTS
+
REALTOR® · The Boyenga Team
DRE #01466876
650-308-4727
graehamwatts@gmail.com
www.graehamwatts.com
+
Auto-generated by the farming-postcard skill on {{GENERATED_DATE}}.
Live archive: graehamwatts.github.io/online-content/farming-postcards
+
+ +
+ +
+ diff --git a/skills/farming-postcard/templates/postcard-template.html b/skills/farming-postcard/templates/postcard-template.html new file mode 100644 index 00000000..b452c7a8 --- /dev/null +++ b/skills/farming-postcard/templates/postcard-template.html @@ -0,0 +1,242 @@ + + + + +Graeham Watts Farming Postcard — {{MAIL_DATE}} + + + + + + + + +
+

Farming Postcard — {{MAIL_DATE}}

+

Archetype: {{ARCHETYPE}}

+
+ +
FRONT
+
+
+
+ {{FRONT_HEADLINE_HTML}} +
+
+
{{FRONT_SUBLINE_HTML}}
+
+
+
GRAEHAM
HEADSHOT
(pointing)
+
+ +
+
REALTOR®
+
The Boyenga Team
+
DRE #01466876
+
650-308-4727
+
graehamwatts@gmail.com
+
www.graehamwatts.com
+
+
+
+
+ +
BACK
+
+
+
+
{{BACK_HEADLINE_HTML}}
+
{{BACK_BODY_HTML}}
+
{{BACK_CTA_LINE}}
+
+
+ + + + + + + + + + + + + + + + +
+
+
{{QR_SCAN_LABEL}}
+
SCAN ME
+
+
+
+
GRAEHAM
HEADSHOT
(smiling)
+
+ +
+
REALTOR®
+
The Boyenga Team
+
DRE #01466876
+
+
+
650-308-4727
+
graehamwatts@gmail.com
+
www.graehamwatts.com
+
+
+
+ If your home is listed with another broker, please disregard this postcard. Homes not necessarily sold by this broker. +
+
+ + + diff --git a/skills/finance-watch/SKILL.md b/skills/finance-watch/SKILL.md new file mode 100644 index 00000000..ac3ecde5 --- /dev/null +++ b/skills/finance-watch/SKILL.md @@ -0,0 +1,127 @@ +--- +name: finance-watch +description: "Graeham's personal finance pipeline brain — Plaid bank transactions + SparkReceipt receipts → the Finance Google Sheet → entity tax tabs. Use ANY time Graeham mentions: watch my finances, my finances, Plaid, Chase, bank transactions, pull my transactions, refresh finances, finance sheet, bookkeeping, SparkReceipt, receipt workflow..." + +--- + +# finance-watch — Graeham's finance pipeline operator + +## STATUS as of 2026-06-09 (verified live; consolidation completed this date) +- n8n is UP at `n8n.graehamwattsn8n.com` (healthz ok). +- **THE MASTER SHEET (single source of truth): "Finances 2026 — MASTER (Auto)"**, Google Sheet id + `1zerRMfH7C7-hgbIVSPVgeHBNpiDJkOGsVQkWMP_9unw` (formerly "PropIQ Finance Lab — Working Copy 2026"). + 22 tabs: 9 automation tabs (Dashboard, Receipt Check, Detail Ledger (Auto), Transactions, Tagging Rules, + Receipts, PropIQ (Auto), Realtor Faraday Expenses (Auto), Enterprise Holdings (Auto)) + 13 manual entity + tabs merged in from Sharon's "Finances 2026 .xlsx" on 2026-06-09 (GW Personal, Realtor Faraday Expenses, + GW Property Expenses, Enterprise holdings, PROP IQ, 2842 Cornelius Drive, All Income & expenses, Loans info, + Income estimate, 2048 Huran SJ, Monthly Costs, Explanation of Expense Item #s, Endavour Enterprise). +- The Plaid workflow EXISTS and is ACTIVE: **"Plaid Multi-Sync (Chase + AMEX + BILT)"** (n8n id `GXP6VSXuqJcnFpbs`, 11 nodes), daily 6am cron + webhook `plaid-sync-trigger`, writes to the + MASTER's `Transactions` tab. Verified pulling all 3 institutions (840 txns on 2026-06-09; data current + through 2026-06-08). Do NOT redeploy `assets/plaid-transactions-to-sheets.json` unless this workflow is gone. +- **"Spark Receipt → Google Sheets"** is ACTIVE (id `723IrFCLz0SDaQ61`), webhook `spark-receipt`, writes to the + MASTER's `Receipts` tab (includes receipt image_url). +- **"Reconciliation — Missing Receipts"** is ACTIVE (id `uB4TUGuFieMSu8n7`, fixed + activated 2026-06-09): daily + 6:30am cron + webhook `reconcile-missing`. Rebuilds the MASTER's `Missing Receipts` tab — last-30-days outgoing + charges with no receipt matching within $0.01 / 5 days (skips transfers, card payments, loans, payroll). +- **"Plaid Balances Sync (Chase + AMEX + BILT)"** is ACTIVE (id `n0V85d2I2dA2N0Ym`, built 2026-06-09): daily + 6:15am cron + webhook `plaid-balances-sync`. Writes one row per account (12 accounts) to the MASTER's + `Balances` tab — institution, name, mask, type, available/current/limit, as_of. NOTE: the Plaid account does + NOT have the standalone `balance` product — balances come from the `accounts` block of `/transactions/get` + (a 1-day window with count:1); do not "fix" it back to `/accounts/balance/get` (INVALID_PRODUCT). + Institution tokens are duplicated from the Multi-Sync workflow's `Set All Institution Tokens` node — if tokens + rotate, update BOTH workflows. +- **Old finance files are deprecated** — renamed "OLD — … DO NOT USE" where API access allowed. Sharon's old + "Finances 2026 .xlsx" (`1ibvrsfnWNJOlRL0GDXQEu7ZZp_brEKGa`) is the pre-merge backup; manual entry now happens + in the MASTER's entity tabs. A converted backup of the merged tabs lives at "MERGE SOURCE — Finances 2026 + converted (backup)" (`1gbP-kfu0vnlKS6_V4fDzUY9NjWSfWaR-KQlL3HO2U3w`). +- **KNOWN ISSUE:** both n8n Google **Drive** OAuth credentials ("Google Drive account", "Google Drive account 2") + have expired/revoked refresh tokens (EAUTH). The Google **Sheets** credential (`AkBUwX11QA8RRHec`) works and + has `spreadsheets` + `drive.file` scope — full Sheets API on any spreadsheet, Drive API only on app-created + files. Until a Drive credential is reconnected in the n8n UI, old xlsx files can't be renamed/moved via API. +- **"FIN — Live Dashboard (web app)"** is ACTIVE (id `22Yn7HY5zuSMCURD`, built 2026-06-09): a GET webhook at + `/webhook/finance-dashboard?key=` that renders a phone-friendly HTML dashboard LIVE from the MASTER on + every load — cash on hand, credit cards owed, monthly spend trend, per-card and per-category spend, entity + totals (from Detail Ledger routing; unrouted shows as "no rule"), open Closings vs cash check (60-day + horizon), and the Missing Receipts list. The secret key lives ONLY in the workflow's `Build HTML` Code node + (never commit it to this public repo). Deliberately NOT published to GitHub Pages — finance data stays private + behind n8n. Goes down when the Mac Studio tunnel is down (same as everything else here). +- **"Closings" tab** added to the MASTER (Property, Client, Side, Close Date, Cash Needed $, Cash Incoming $, + Status, Notes). Graeham/Sharon enter escrows there; the dashboard computes cash-needed-vs-available. +- One-off consolidation workflows (deactivated, kept for reference): `FIN — Consolidate Master v2` (xlsx upload + → convert → copyTo merge), `FIN — Rename Old Sheets`, `FIN — Add Closings Tab`, `FIN — Drive Cred Test`. +- The go-live sequence below is the FALLBACK for when the pipeline is down or lost; check the live state first. + +## What this system actually is (read this first) +Graeham does **not** have, and cannot have, a "log into Plaid and see Chase live" button. Plaid is plumbing +that sits *behind* an app — there is no consumer Plaid dashboard of transactions. The thing that pulls his +bank data **is an n8n workflow that acts as the Plaid client.** Claude never queries Chase directly; Claude +reads the **output** of this pipeline (the Google Sheet / a CSV export). + +``` +Chase accounts ──Plaid API──> n8n workflow ──> Finance Google Sheet ──> entity tax tabs +SparkReceipt (receipts) ──────────────────────> same sheet (Receipt Match ID) +``` + +- **Plaid** → bank transactions (txn_id, date, account, merchant, category, amount). +- **SparkReceipt** → receipt images, tagged by Entity + Category (see `Receipt-Workflow-SOP.md`). Cloud SaaS, + independent of n8n. +- **n8n** (self-hosted on the Mac Studio, tunneled at `n8n.graehamwattsn8n.com`) → the glue that runs the + Plaid pull and writes rows. **This is the fragile part.** +- **Destination schema** (canonical row, from `Claude_2026_Finances_Skeleton.xlsx` → tab `GW Personal`): + `Date | Vendor / Description | Payment Method | Amount | Category | Health Insurance | Notes | Receipt Match ID | Auto-Tag Source | txn_id (from Plaid)` +- **Routing**: the `Tagging Rules` tab (Match Type, Vendor Pattern, Entity, Property, Category, IRS Line, + Confidence) maps each transaction to an entity (GW Personal, Realtor Faraday, GW Property, Enterprise + Holdings, Endeavour Enterprises, PROP IQ) and an IRS line. + +## Known failure mode (this is why it breaks) +The n8n instance was migrated off `graehamwatts.app.n8n.cloud` onto the Mac Studio tunnel. When the Mac is +asleep/off or the Cloudflare tunnel (`cloudflared`) isn't running, the host returns **HTTP 530** and the +whole pipeline stalls. The freshest data Claude can see is then whatever was last exported +(`Documents/Skills LLMS/Claude/chase_transactions.csv`). **A 530 = the box/tunnel is down, not a credential problem.** + +## STEP 1 — Always check the host first +Run `mcp__n8n-mcp__n8n_health_check` (mode=status). +- **connected:false / 530** → pipeline is DOWN. Tell Graeham plainly. The fix is on the Mac Studio: wake it, + confirm n8n is running, confirm `cloudflared` is up. Re-check `https://n8n.graehamwattsn8n.com/healthz` + (should return `{"status":"ok"}`). Do NOT pretend live data is available — read the CSV snapshot instead + and label it with its date. +- **connected:true** → proceed to go-live. + +## STEP 2 — Go-live sequence (run the moment n8n is back) +1. `n8n_health_check` → confirm connected:true. +2. `n8n_list_workflows` → look for a Plaid / bank / transactions workflow. +3. **If it exists** → trigger it, then read the sheet. +4. **If it does NOT exist** (likely — it was lost in the migration) → deploy the rebuilt one: + `n8n_create_workflow` using `assets/plaid-transactions-to-sheets.json`. Then fill the 3 inputs below, + activate, and trigger. +5. Read the resulting rows and reconcile against SparkReceipt (`receipt_match` / Receipt Match ID). +6. Report: new transactions, anything untagged (❌ VERIFY), and any tax-relevant items (e.g. FTB payments). + +## STEP 3 — The 3 inputs the workflow needs (Graeham fills once) +1. **Plaid credentials** → set as n8n ENV vars: `PLAID_CLIENT_ID`, `PLAID_SECRET`, `PLAID_ACCESS_TOKEN` + (the access token for the linked Chase Item), `PLAID_ENV` (`production`). These are NOT stored in any + file on disk — they live only in n8n (or were issued when the Chase Item was linked). If the access + token was lost in the migration, the Chase Item must be re-linked via Plaid Link to mint a new one. +3. **Google Sheet ID** → the finance sheet's ID, set on the `Append to Finance Sheet` node, plus the target + tab name (default `Bank Transactions`). +3. **Google Sheets credential** → select Graeham's Google OAuth credential on that same node. + +## If Graeham wants live access WITHOUT n8n +Two honest options — both are builds, not logins: +- **Read the live Google Sheet** via a connected Google Drive/Sheets connector (the SparkReceipt side keeps + it current even while n8n is down). Fastest path to fresh *expense* data. +- **Dedicated Plaid MCP** added in Settings → Connectors with his Plaid keys → gives Claude direct Plaid + access independent of n8n. Requires standing up a Plaid MCP server. + +## Files this skill relies on (in Graeham's Documents) +- `Claude/Receipt-Workflow-SOP.md` — the SparkReceipt side, plain-English. +- `Claude/Claude_2026_Finances_Skeleton.xlsx` — canonical schema + `Tagging Rules` + `Property Master`. +- `Claude/chase_transactions.csv` — last Plaid export (snapshot; check its newest date before quoting it). +- `Claude/Skills/n8n-workflows/` — where workflow JSON exports live (this rebuild is added here). +- `assets/plaid-transactions-to-sheets.json` — the rebuilt, deploy-ready workflow. +- `assets/go-live-prompt.md` — the one-paste prompt to run when n8n is back. + +## Hard rules +- Never claim live Chase access when `n8n_health_check` is not `connected:true`. Read the snapshot and date it. +- Never invent transaction numbers. If reading the CSV, say "as of ." +- Secrets stay in n8n ENV / the credential store, never written into the workflow JSON or into chat. diff --git a/skills/finance-watch/assets/go-live-prompt.md b/skills/finance-watch/assets/go-live-prompt.md new file mode 100644 index 00000000..dacfb319 --- /dev/null +++ b/skills/finance-watch/assets/go-live-prompt.md @@ -0,0 +1,21 @@ +# Go-Live Prompt — paste this the moment your Mac Studio n8n is back up + +> Copy everything in the box below into a new message to Claude. + +--- + +n8n is back online. Run the finance pipeline go-live using the **finance-watch** skill: + +1. Run an n8n health check and confirm it's connected (not 530). If it's still down, stop and tell me — don't read stale data as if it's live. +2. Once connected, list my n8n workflows and look for a Plaid / bank / transactions workflow. + - If it exists, trigger it. + - If it does NOT exist, deploy the rebuilt one from the finance-watch skill's `assets/plaid-transactions-to-sheets.json`, then tell me exactly which of the 3 inputs (Plaid env vars, Google Sheet ID, Google Sheets credential) still need filling before it can run. +3. After the pull runs, read the finance sheet, reconcile against SparkReceipt, and give me: how many new transactions landed, anything left untagged (❌ VERIFY), and any tax-relevant items (especially CA FTB / franchise tax payments). +4. Confirm the date range you actually pulled so I know it's live, not the May snapshot. + +--- + +## If you just want the latest numbers and don't care about n8n yet +Paste this instead: + +> Read my latest finance data: pull `Documents/Skills LLMS/Claude/chase_transactions.csv` and tell me the newest transaction date in it, then summarize by account and flag anything tax-relevant. Label it clearly as "as of ", since this is the last snapshot, not a live pull. diff --git a/skills/flow-dictation/SKILL.md b/skills/flow-dictation/SKILL.md new file mode 100644 index 00000000..e9cd5572 --- /dev/null +++ b/skills/flow-dictation/SKILL.md @@ -0,0 +1,131 @@ +--- +name: flow-dictation +description: "Local Wispr-Flow-style push-to-talk dictation for Graeham — hold Ctrl+Alt anywhere in Windows, speak, release, and the text is pasted into the focused app (tap Shift while talking for AI-polish via the Claude API). Use ANY time the user mentions flow dictation, dictation app, push to talk, whisper flow, wispr flow, voice typing..." +--- + +# Flow Dictation — local push-to-talk dictation + +A system-tray app that replicates Wispr Flow entirely locally. Hold +**Ctrl+Alt** anywhere in Windows, speak, release — the transcript is pasted +into whatever app has focus. No cloud, no subscription, no audio leaves the +machine. + +Combo-hotkey safety: recording arms when both keys are held, but pressing any +THIRD key (Ctrl+Alt+T, Ctrl+Alt+Del...) cancels instantly and silently — so +normal shortcuts never trigger a paste, and the start-beep is delayed 180ms so +quick shortcuts make no sound. Single-key hotkeys (e.g. "f9") still work via +config and are suppressed from the target app. + +Hotkey history: F9 → Ctrl+Shift (rejected 2026-07-02: holding Shift 8s fires +the Windows Filter Keys accessibility warning and the OS beeps) → **Ctrl+Alt** +(current; no Windows accessibility feature attaches to holding it). Combos must +be modifier-only — typing keys like Tab/Q can't be suppressed on the combo path +and would leak into the focused app. + +## How it runs + +- **Launch:** the **"Flow Dictation"** shortcut on the Desktop (gold mic icon), + which runs `pythonw.exe scripts/flow_dictation.py` — no console window. +- **Tray icon states:** gray = model loading (~5s), **gold = ready**, + red = recording, blue = transcribing, dark = paused. +- **Tray menu:** History (also opens on LEFT-click of the tray icon) · Pause · + Copy last transcript · Edit vocabulary · Start with Windows (toggle, creates/ + removes a shortcut in `shell:startup`) · Quit. +- **Overlay pill:** a small dark capsule appears bottom-center while dictating — + pink dot + a **flowing violet→magenta voice wave** (continuous oscillating + gradient line + fainter echo line, amplitude follows the speech envelope) + + "Listening 0:03" timer, then blue-violet "Transcribing…" (purple + "Polishing with Claude…" in polish mode), then gone. Marked WS_EX_NOACTIVATE + so it never steals focus from the target app. +- **Sounds:** the active set in `assets/sounds/` is **"pop"** (Blip-style + chirps — rising double-blip = listening, falling pop = done, low chirp + pair = error; chosen by Graeham 2026-07-02). Three full sets live in + `assets/sounds/sets/` (`pop`, `droplet`, `glass`) — switch by copying a + set's wavs over `assets/sounds/*.wav`, no restart needed. Played via + `winsound.PlaySound` async; regenerate/re-tune with + `scripts/make_sounds.py` (v2 chime synth) — the pop/droplet/glass + generator lives in the same style (chirp = exponential pitch-glide sine). + Raw `winsound.Beep` tones remain as fallback only. +- **Color scheme:** soft violet accent `#a78bfa` everywhere (tray icon, + history window, stats) — the original gold was retired 2026-07-02. +- **AI-polish mode:** tap **Shift** once while holding Ctrl+Alt and talking — + that dictation is cleaned into tidy prose by Claude before pasting (filler + removed, grammar fixed, tone kept). Requires an API key (see Architecture + below); without one it error-beeps and pastes the raw transcript. +- **Dashboard window** (left-click tray icon): Blip-style home — four stat + cards (words today, words this month, avg dictation WPM, money saved = + time saved vs 40wpm typing × $15/hr) above a Recent Transcriptions list. + Every dictation is saved to `outputs/history.jsonl` (timestamp, text, + seconds, polished flag, target window — last 500 kept). Searchable (✨ + marks polished entries), click for full text, Copy / double-click to copy, + Clear history. 100% local. +- **Audio cues:** high beep = recording started, two-tone = text pasted, + low beep = error, short mid beep = pressed while model still loading. + +## Architecture (all in `scripts/flow_dictation.py`) + +1. Global hotkey hook (`keyboard` lib) — hold-to-talk, plus **lock mode**: + quick-tap the combo (<0.35s) to record hands-free for unlimited length + (whole paragraphs / minutes); tap the combo again to finish, Esc to + cancel. While locked, stray keys are IGNORED so a long dictation can't be + killed accidentally. During a normal hold, a third key still cancels + silently (normal-shortcut protection) — cancels are now logged. Tapping + the `polish_key` (Shift) while recording never cancels — it arms + AI-polish for that dictation (pill shows "✨ polish"). +2. Mic capture with `sounddevice` at 16 kHz mono into a numpy buffer; per-chunk + RMS feeds the live waveform in the overlay pill. +3. faster-whisper **large-v3-turbo / CUDA / float16 / beam_size 1**, loaded + once and kept warm (~2.7s load, ~0.33s warm transcription of a 13s clip — + verified 2026-07-02). Uses the standard RTX 5090 DLL fix: nvidia pip-wheel + `bin/` dirs added via `add_dll_directory` AND prepended to PATH **before** + importing faster_whisper. Falls back to CPU int8 if CUDA fails. +4. `vad_filter=True` + `condition_on_previous_text=False` (prevents + repetition loops on multi-minute audio) + `initial_prompt` built from + `vocab.txt` (names/brands spelled right: PropertyIQ, Compass, East Palo + Alto, GHL...). There is NO length limit — audio is buffered in RAM + (~2 MB/min) and Whisper chunks internally. +5. Optional polish pass (only when Shift was tapped): raw transcript → + Claude API (`polish_model` in config, default `claude-opus-5`, official + `anthropic` SDK) with a cleanup system prompt (no em dashes, keep tone, + return only cleaned text). Key resolution: `ANTHROPIC_API_KEY` env var, + else the gitignored `/anthropic-token.txt`. No key or API error + → error beep + raw transcript pasted; everything else stays 100% local. +6. Paste via clipboard: save clipboard → copy transcript → Ctrl+V → restore + old clipboard after 1s. Works in virtually every app, instant. + +## Files + +| File | Purpose | +|---|---| +| `scripts/flow_dictation.py` | core app: hotkey, mic, whisper, paste, tray | +| `scripts/ui.py` | tkinter layer: overlay pill + history window | +| `config.json` | hotkey, model, device, language, beeps | +| `vocab.txt` | one term per line → Whisper priming prompt (editable live) | +| `assets/flow.ico` | app icon (regenerate: `make_icon_image('ready').save(...)`) | +| `outputs/flow-dictation.log` | runtime log (gitignored output) | +| `outputs/history.jsonl` | dictation history, one JSON per line (gitignored) | + +Threading: tkinter owns the main thread (`ui.run()`); pystray runs via +`run_detached()`; keyboard hooks + transcription run on worker threads. All UI +calls from workers go through `UI.q` (a queue polled with `root.after`). + +## Ops + +- **Change hotkey:** edit `config.json` (`"hotkey": "ctrl+shift"` — any + `keyboard`-lib key name or a modifier combo joined with `+`), then Quit + + relaunch from the desktop shortcut. +- **Self-test without a mic:** + `python scripts/flow_dictation.py --selftest path/to/clip.wav` +- **Verified 2026-07-01 (large-v3):** ~5s load, ~0.9s warm transcription. + **2026-07-02 (large-v3-turbo, beam 1):** ~2.7s load, ~0.33s warm + transcription of the same 13s clip, word-perfect including vocab terms. +- **Enable polish mode:** put a Claude API key in `ANTHROPIC_API_KEY` or in + `Documents\Skills LLMS\Claude\Skills\anthropic-token.txt` (gitignored via `*token*.txt`). +- **Known limits:** paste can't reach elevated (admin) windows unless the app + itself runs elevated; terminals that need Ctrl+Alt+V won't auto-paste + (use tray > Copy last transcript). Requires the mic Windows defaults to. + +## Roadmap (not built yet) + +- Phase 3: voice commands ("new line", "scratch that"). Lock mode and the + AI-polish pass have since shipped (see above) — only voice commands remain. diff --git a/skills/flow-dictation/assets/flow-wave.ico b/skills/flow-dictation/assets/flow-wave.ico new file mode 100644 index 00000000..98ae8207 Binary files /dev/null and b/skills/flow-dictation/assets/flow-wave.ico differ diff --git a/skills/flow-dictation/assets/flow.ico b/skills/flow-dictation/assets/flow.ico new file mode 100644 index 00000000..98ae8207 Binary files /dev/null and b/skills/flow-dictation/assets/flow.ico differ diff --git a/skills/flow-dictation/assets/sounds/busy.wav b/skills/flow-dictation/assets/sounds/busy.wav new file mode 100644 index 00000000..a6d04580 Binary files /dev/null and b/skills/flow-dictation/assets/sounds/busy.wav differ diff --git a/skills/flow-dictation/assets/sounds/done.wav b/skills/flow-dictation/assets/sounds/done.wav new file mode 100644 index 00000000..47cdcbf1 Binary files /dev/null and b/skills/flow-dictation/assets/sounds/done.wav differ diff --git a/skills/flow-dictation/assets/sounds/error.wav b/skills/flow-dictation/assets/sounds/error.wav new file mode 100644 index 00000000..abca6b97 Binary files /dev/null and b/skills/flow-dictation/assets/sounds/error.wav differ diff --git a/skills/flow-dictation/assets/sounds/sets/droplet/busy.wav b/skills/flow-dictation/assets/sounds/sets/droplet/busy.wav new file mode 100644 index 00000000..dcda79c0 Binary files /dev/null and b/skills/flow-dictation/assets/sounds/sets/droplet/busy.wav differ diff --git a/skills/flow-dictation/assets/sounds/sets/droplet/done.wav b/skills/flow-dictation/assets/sounds/sets/droplet/done.wav new file mode 100644 index 00000000..a4c78da9 Binary files /dev/null and b/skills/flow-dictation/assets/sounds/sets/droplet/done.wav differ diff --git a/skills/flow-dictation/assets/sounds/sets/droplet/error.wav b/skills/flow-dictation/assets/sounds/sets/droplet/error.wav new file mode 100644 index 00000000..8f557659 Binary files /dev/null and b/skills/flow-dictation/assets/sounds/sets/droplet/error.wav differ diff --git a/skills/flow-dictation/assets/sounds/sets/droplet/start.wav b/skills/flow-dictation/assets/sounds/sets/droplet/start.wav new file mode 100644 index 00000000..b2a4391c Binary files /dev/null and b/skills/flow-dictation/assets/sounds/sets/droplet/start.wav differ diff --git a/skills/flow-dictation/assets/sounds/sets/glass/busy.wav b/skills/flow-dictation/assets/sounds/sets/glass/busy.wav new file mode 100644 index 00000000..b6cfe79b Binary files /dev/null and b/skills/flow-dictation/assets/sounds/sets/glass/busy.wav differ diff --git a/skills/flow-dictation/assets/sounds/sets/glass/done.wav b/skills/flow-dictation/assets/sounds/sets/glass/done.wav new file mode 100644 index 00000000..562c0cfb Binary files /dev/null and b/skills/flow-dictation/assets/sounds/sets/glass/done.wav differ diff --git a/skills/flow-dictation/assets/sounds/sets/glass/error.wav b/skills/flow-dictation/assets/sounds/sets/glass/error.wav new file mode 100644 index 00000000..59a2df62 Binary files /dev/null and b/skills/flow-dictation/assets/sounds/sets/glass/error.wav differ diff --git a/skills/flow-dictation/assets/sounds/sets/glass/start.wav b/skills/flow-dictation/assets/sounds/sets/glass/start.wav new file mode 100644 index 00000000..b7cbe7cc Binary files /dev/null and b/skills/flow-dictation/assets/sounds/sets/glass/start.wav differ diff --git a/skills/flow-dictation/assets/sounds/sets/pop/busy.wav b/skills/flow-dictation/assets/sounds/sets/pop/busy.wav new file mode 100644 index 00000000..a6d04580 Binary files /dev/null and b/skills/flow-dictation/assets/sounds/sets/pop/busy.wav differ diff --git a/skills/flow-dictation/assets/sounds/sets/pop/done.wav b/skills/flow-dictation/assets/sounds/sets/pop/done.wav new file mode 100644 index 00000000..47cdcbf1 Binary files /dev/null and b/skills/flow-dictation/assets/sounds/sets/pop/done.wav differ diff --git a/skills/flow-dictation/assets/sounds/sets/pop/error.wav b/skills/flow-dictation/assets/sounds/sets/pop/error.wav new file mode 100644 index 00000000..abca6b97 Binary files /dev/null and b/skills/flow-dictation/assets/sounds/sets/pop/error.wav differ diff --git a/skills/flow-dictation/assets/sounds/sets/pop/start.wav b/skills/flow-dictation/assets/sounds/sets/pop/start.wav new file mode 100644 index 00000000..7f2fc95e Binary files /dev/null and b/skills/flow-dictation/assets/sounds/sets/pop/start.wav differ diff --git a/skills/flow-dictation/assets/sounds/start.wav b/skills/flow-dictation/assets/sounds/start.wav new file mode 100644 index 00000000..7f2fc95e Binary files /dev/null and b/skills/flow-dictation/assets/sounds/start.wav differ diff --git a/skills/flow-dictation/config.json b/skills/flow-dictation/config.json new file mode 100644 index 00000000..6578611e --- /dev/null +++ b/skills/flow-dictation/config.json @@ -0,0 +1,13 @@ +{ + "hotkey": "ctrl+alt", + "model": "large-v3-turbo", + "device": "cuda", + "compute_type": "float16", + "language": "en", + "sample_rate": 16000, + "min_seconds": 0.3, + "beam_size": 1, + "beeps": true, + "polish_key": "shift", + "polish_model": "claude-opus-5" +} diff --git a/skills/flow-dictation/scripts/flow_dictation.py b/skills/flow-dictation/scripts/flow_dictation.py new file mode 100644 index 00000000..772cda9b --- /dev/null +++ b/skills/flow-dictation/scripts/flow_dictation.py @@ -0,0 +1,652 @@ +#!/usr/bin/env python3 +""" +flow_dictation.py — local Wispr-Flow-style push-to-talk dictation for Graeham. + +Hold the hotkey (default F9) anywhere in Windows, speak, release. +The speech is transcribed on the RTX 5090 with faster-whisper large-v3 +(model stays warm in VRAM) and pasted into whatever app has focus. + +Runs as a system-tray app: + gray = model loading gold = ready + red = recording blue = transcribing + +Tray menu: Pause/Resume, Copy last transcript, Edit vocabulary, +Start with Windows (toggle), Quit. + +Launch via the "Flow Dictation" desktop shortcut (pythonw, no console), +or directly: python flow_dictation.py +Self-test (no mic needed): python flow_dictation.py --selftest path/to/clip.wav +""" +import json +import os +import sys +import threading +import time +from collections import deque +from datetime import datetime +from pathlib import Path + +SKILL_DIR = Path(__file__).resolve().parent.parent +CONFIG_FILE = SKILL_DIR / "config.json" +VOCAB_FILE = SKILL_DIR / "vocab.txt" +LOG_FILE = SKILL_DIR / "outputs" / "flow-dictation.log" +HISTORY_FILE = SKILL_DIR / "outputs" / "history.jsonl" + + +def foreground_app_title(): + """Title of the window the user is dictating into — stored with history.""" + try: + import ctypes + + h = ctypes.windll.user32.GetForegroundWindow() + n = ctypes.windll.user32.GetWindowTextLengthW(h) + buf = ctypes.create_unicode_buffer(n + 1) + ctypes.windll.user32.GetWindowTextW(h, buf, n + 1) + return buf.value + except Exception: + return "" + +DEFAULTS = { + "hotkey": "ctrl+alt", + "model": "large-v3-turbo", + "device": "cuda", + "compute_type": "float16", + "language": "en", + "sample_rate": 16000, + "min_seconds": 0.3, + "beam_size": 1, + "beeps": True, + "polish_key": "shift", + "polish_model": "claude-opus-5", +} + +# Optional Claude API key for polish mode. Resolution order: env var, then a +# gitignored key file at the repo root (matches the *token*.txt ignore rule). +API_KEY_FILE = SKILL_DIR.parent.parent / "anthropic-token.txt" + + +def resolve_api_key(): + key = os.environ.get("ANTHROPIC_API_KEY", "").strip() + if key: + return key + try: + return API_KEY_FILE.read_text(encoding="utf-8").strip() + except Exception: + return "" + + +def log(msg): + line = f"[{datetime.now().strftime('%Y-%m-%d %H:%M:%S')}] {msg}" + print(line, flush=True) + try: + LOG_FILE.parent.mkdir(parents=True, exist_ok=True) + with open(LOG_FILE, "a", encoding="utf-8") as f: + f.write(line + "\n") + except Exception: + pass + + +def load_config(): + cfg = dict(DEFAULTS) + try: + cfg.update(json.loads(CONFIG_FILE.read_text(encoding="utf-8"))) + except Exception: + pass + return cfg + + +def load_vocab_prompt(): + """vocab.txt lines become Whisper's initial_prompt so names spell right.""" + try: + words = [ + w.strip() + for w in VOCAB_FILE.read_text(encoding="utf-8").splitlines() + if w.strip() and not w.strip().startswith("#") + ] + if words: + return "Vocabulary: " + ", ".join(words) + "." + except Exception: + pass + return None + + +def setup_cuda_dlls(): + """RTX 5090 / faster-whisper: the CUDA 12 cuBLAS + cuDNN runtime DLLs ship in + the nvidia-*-cu12 pip wheels under /bin (Windows). add_dll_directory + alone does NOT propagate to ctranslate2's loader — the bin dirs must also be + on PATH, set BEFORE faster_whisper is imported.""" + import importlib.util + + dirs = [] + for pkg in ("nvidia.cublas", "nvidia.cudnn", "nvidia.cuda_runtime", "nvidia.cuda_nvrtc"): + try: + spec = importlib.util.find_spec(pkg) + if not (spec and spec.submodule_search_locations): + continue + base = list(spec.submodule_search_locations)[0] + for sub in ("bin", "lib"): + d = os.path.join(base, sub) + if os.path.isdir(d): + dirs.append(d) + except Exception: + pass + for d in dirs: + try: + os.add_dll_directory(d) + except Exception: + pass + if dirs: + os.environ["PATH"] = os.pathsep.join(dirs) + os.pathsep + os.environ.get("PATH", "") + + +# keyboard-lib event names for each modifier, so a combo hotkey like +# "ctrl+shift" can match "left ctrl"/"right shift" etc. in raw hook events +MOD_VARIANTS = { + "ctrl": ("ctrl", "left ctrl", "right ctrl"), + "shift": ("shift", "left shift", "right shift"), + "alt": ("alt", "left alt", "right alt", "alt gr"), + "windows": ("windows", "left windows", "right windows"), +} + + +def variants(part): + return MOD_VARIANTS.get(part, (part,)) + + +def beep(kind, enabled=True): + """Short non-blocking audio cues: rec start, done, error, ignored.""" + if not enabled: + return + + def _b(): + try: + import winsound + + # soft generated chimes (assets/sounds, built by make_sounds.py); + # raw Beep tones only as a fallback if the wavs are missing + wav = SKILL_DIR / "assets" / "sounds" / f"{kind}.wav" + if wav.exists(): + winsound.PlaySound(str(wav), winsound.SND_FILENAME | winsound.SND_ASYNC) + return + tones = { + "start": [(880, 70)], + "done": [(1175, 55), (1568, 70)], + "error": [(220, 180)], + "busy": [(440, 60)], + } + for freq, dur in tones.get(kind, []): + winsound.Beep(freq, dur) + except Exception: + pass + + threading.Thread(target=_b, daemon=True).start() + + +def make_icon_image(state, size=64): + """Tray/desktop icon: a violet->magenta gradient voice wave (matching the + overlay waveform) on a dark disc; ring color = state. Drawn at 256px and + downscaled so it stays crisp at tray size.""" + from PIL import Image, ImageDraw + + colors = { + "loading": (128, 128, 128), + "ready": (167, 139, 250), # soft violet (gold retired 2026-07-02) + "recording": (236, 72, 153), # pink + "transcribing": (91, 141, 217), + "polishing": (170, 110, 220), + "paused": (80, 80, 80), + } + ring = colors.get(state, (128, 128, 128)) + S = 256 + img = Image.new("RGBA", (S, S), (0, 0, 0, 0)) + d = ImageDraw.Draw(img) + d.ellipse([8, 8, S - 8, S - 8], fill=(20, 20, 20, 255), outline=ring, width=18) + # gradient wave bars, same endpoints as the overlay waveform + a, b = (139, 92, 246), (236, 72, 153) + heights = (0.32, 0.62, 1.0, 0.78, 0.48, 0.85, 0.36) + n = len(heights) + mid, span = S / 2, 148 + for i, hgt in enumerate(heights): + t = i / (n - 1) + if state == "paused": + c = (110, 110, 110, 255) + else: + c = tuple(int(a[j] + (b[j] - a[j]) * t) for j in range(3)) + (255,) + x = S / 2 - span / 2 + i * (span / (n - 1)) + amp = 14 + hgt * 58 + d.rounded_rectangle([x - 9, mid - amp, x + 9, mid + amp], radius=9, fill=c) + return img.resize((size, size), Image.LANCZOS) + + +class FlowDictation: + def __init__(self, cfg): + self.cfg = cfg + self.model = None + self.paused = False + self.recording = False + self.frames = [] + self.stream = None + self.last_text = "" + self.state = "loading" + self.tray = None + self.ui = None + self.levels = deque(maxlen=48) # recent mic RMS values for the waveform + self.polish_mode = False # tap Shift while recording to enable + self.locked = False # quick-tap the combo to record hands-free + self.press_t = 0.0 + self.transcribe_lock = threading.Lock() + self.rec_lock = threading.Lock() + + # ---------- model ---------- + + def load_model(self): + setup_cuda_dlls() + from faster_whisper import WhisperModel + + log(f"loading faster-whisper {self.cfg['model']} on {self.cfg['device']} ...") + t0 = time.time() + try: + self.model = WhisperModel( + self.cfg["model"], device=self.cfg["device"], compute_type=self.cfg["compute_type"] + ) + except Exception as e: + log(f"GPU load failed ({type(e).__name__}: {e}); falling back to CPU int8") + self.model = WhisperModel(self.cfg["model"], device="cpu", compute_type="int8") + # warm-up pass so the first real dictation isn't slow + import numpy as np + + self.model.transcribe(np.zeros(self.cfg["sample_rate"], dtype=np.float32), language=self.cfg["language"]) + log(f"model ready in {time.time() - t0:.1f}s") + self.set_state("ready") + + # ---------- recording ---------- + + def on_press(self, _event=None): + with self.rec_lock: + if self.paused or self.recording: + return + if self.model is None: + beep("busy", self.cfg["beeps"]) + return + self.recording = True + self.press_t = time.time() + self.locked = False + self.frames = [] + self.levels.clear() + self.polish_mode = False + if self.ui: + self.ui.set_polish(False) + self.ui.set_locked(False) + try: + import numpy as np + import sounddevice as sd + + try: + self.mic_name = sd.query_devices(kind="input")["name"] + except Exception: + self.mic_name = "unknown" + + def on_audio(indata, *_): + self.frames.append(indata.copy()) + self.levels.append(float(np.sqrt(np.mean(indata**2)))) + + self.stream = sd.InputStream( + samplerate=self.cfg["sample_rate"], + channels=1, + dtype="float32", + callback=on_audio, + ) + self.stream.start() + self.set_state("recording") + + # beep only if still recording after a beat — a quick Ctrl+Shift+T + # style shortcut cancels before this fires, so no noise on shortcuts + def delayed_beep(): + time.sleep(0.18) + if self.recording: + beep("start", self.cfg["beeps"]) + + threading.Thread(target=delayed_beep, daemon=True).start() + except Exception as e: + with self.rec_lock: + self.recording = False + log(f"mic error: {type(e).__name__}: {e}") + beep("error", self.cfg["beeps"]) + + def on_release(self, _event=None): + with self.rec_lock: + if not self.recording: + return + self.recording = False + try: + self.stream.stop() + self.stream.close() + except Exception: + pass + frames = self.frames + self.frames = [] + threading.Thread(target=self.process, args=(frames,), daemon=True).start() + + def cancel_recording(self): + """A third key was pressed while the combo was held — the user is doing + a normal shortcut (Ctrl+Alt+T etc.), not dictating. Discard silently.""" + with self.rec_lock: + if not self.recording: + return + self.recording = False + self.locked = False + if self.ui: + self.ui.set_locked(False) + try: + self.stream.stop() + self.stream.close() + except Exception: + pass + self.frames = [] + self.set_state("ready") + + # ---------- transcribe + paste ---------- + + def process(self, frames): + import numpy as np + + if not frames: + self.set_state("ready") + return + audio = np.concatenate(frames).flatten() + seconds = len(audio) / self.cfg["sample_rate"] + if seconds < self.cfg["min_seconds"]: + self.set_state("ready") + return + # dead-mic guard: near-zero signal means the default input device is + # muted, disconnected, or wrong. Whisper hallucinates garbage (often + # CJK text) on silence and wastes seconds retrying — bail out instead. + rms = float(np.sqrt(np.mean(audio**2))) + if rms < 0.0012: + log( + f"mic level near zero (rms={rms:.5f}, device: " + f"{getattr(self, 'mic_name', 'unknown')}) — mic muted, off, or " + "wrong default input? NOT transcribing." + ) + beep("error", self.cfg["beeps"]) + self.set_state("ready") + return + self.set_state("transcribing") + target_app = foreground_app_title() + polish = self.polish_mode + try: + with self.transcribe_lock: + t0 = time.time() + segments, _info = self.model.transcribe( + audio, + language=self.cfg["language"], + vad_filter=True, + beam_size=self.cfg["beam_size"], + # long-form reliability: don't feed earlier output back in, + # which can cause repetition loops on multi-minute audio + condition_on_previous_text=False, + initial_prompt=load_vocab_prompt(), + ) + text = " ".join(s.text.strip() for s in segments).strip() + # hallucination guard: forced-English transcripts that come back + # mostly non-ASCII are silence/noise artifacts, not speech + if text and self.cfg["language"] == "en": + non_ascii = sum(1 for ch in text if ord(ch) > 127) + if non_ascii > 0.4 * len(text): + log(f'discarded hallucinated transcript ({non_ascii}/{len(text)} non-ascii): "{text[:40]}"') + beep("error", self.cfg["beeps"]) + text = "" + if text: + if polish: + self.set_state("polishing") + try: + text = self.polish_text(text) + except Exception as e: + log(f"polish failed, pasting raw: {type(e).__name__}: {e}") + beep("error", self.cfg["beeps"]) + polish = False + self.last_text = text + self.paste(text) + if self.ui: + self.ui.add_entry(text, app=target_app, seconds=seconds, polished=polish) + log(f'{seconds:.1f}s audio -> {time.time() - t0:.2f}s{" (polished)" if polish else ""} -> "{text[:80]}"') + beep("done", self.cfg["beeps"]) + else: + log(f"{seconds:.1f}s audio -> (no speech detected)") + except Exception as e: + log(f"transcribe error: {type(e).__name__}: {e}") + beep("error", self.cfg["beeps"]) + finally: + self.set_state("recording" if self.recording else "ready") + + def polish_text(self, text): + """AI-polish mode: clean the raw transcript into tidy prose via the + Claude API before pasting. Only runs when the user tapped Shift during + recording — everything else stays 100% local.""" + import anthropic + + key = resolve_api_key() + if not key: + raise RuntimeError(f"no API key (set ANTHROPIC_API_KEY or create {API_KEY_FILE})") + client = anthropic.Anthropic(api_key=key, timeout=30.0, max_retries=1) + response = client.messages.create( + model=self.cfg["polish_model"], + max_tokens=2048, + system=( + "You clean up dictated speech into polished written text. Fix grammar, " + "remove filler words and false starts, and smooth the phrasing while " + "keeping the speaker's meaning, tone, and warmth. Do not add new content, " + "do not answer questions in the text, do not use em dashes. " + "Return ONLY the cleaned text with no preamble or commentary." + ), + messages=[{"role": "user", "content": text}], + ) + polished = " ".join(b.text for b in response.content if b.type == "text").strip() + return polished or text + + def paste(self, text): + """Clipboard paste beats simulated keystrokes: instant, works everywhere. + Restore the old clipboard after the target app has had time to read it.""" + import keyboard + import pyperclip + + old = None + try: + old = pyperclip.paste() + except Exception: + pass + pyperclip.copy(text) + time.sleep(0.05) + keyboard.send("ctrl+v") + if old is not None: + + def restore(): + time.sleep(1.0) + try: + pyperclip.copy(old) + except Exception: + pass + + threading.Thread(target=restore, daemon=True).start() + + # ---------- tray ---------- + + def set_state(self, state): + self.state = "paused" if (self.paused and state == "ready") else state + if self.tray: + try: + self.tray.icon = make_icon_image(self.state) + self.tray.title = f"Flow Dictation — {self.state} (hold {self.cfg['hotkey'].upper()})" + except Exception: + pass + if self.ui: + self.ui.set_state(self.state) + + def toggle_pause(self, *_): + self.paused = not self.paused + self.set_state("ready") + log("paused" if self.paused else "resumed") + + def copy_last(self, *_): + import pyperclip + + if self.last_text: + pyperclip.copy(self.last_text) + + def open_vocab(self, *_): + os.startfile(VOCAB_FILE) + + # -- start-with-Windows toggle (shortcut in shell:startup) -- + + def startup_lnk(self): + appdata = os.environ.get("APPDATA", "") + return Path(appdata) / "Microsoft" / "Windows" / "Start Menu" / "Programs" / "Startup" / "Flow Dictation.lnk" + + def autostart_enabled(self, *_): + return self.startup_lnk().exists() + + def toggle_autostart(self, *_): + lnk = self.startup_lnk() + if lnk.exists(): + lnk.unlink() + log("autostart disabled") + else: + pythonw = Path(sys.executable).parent / "pythonw.exe" + script = Path(__file__).resolve() + ico = SKILL_DIR / "assets" / "flow.ico" + import subprocess + + ps = ( + f"$s=(New-Object -ComObject WScript.Shell).CreateShortcut('{lnk}');" + f"$s.TargetPath='{pythonw}';$s.Arguments='\"{script}\"';" + f"$s.WorkingDirectory='{script.parent}';$s.IconLocation='{ico}';$s.Save()" + ) + subprocess.run(["powershell", "-NoProfile", "-Command", ps], capture_output=True) + log("autostart enabled") + + def quit(self, icon, *_): + log("quit") + icon.stop() + os._exit(0) + + # ---------- run ---------- + + def show_history(self, *_): + if self.ui: + self.ui.show_history() + + def run(self): + import keyboard + import pystray + + from ui import UI + + log(f"=== Flow Dictation starting (hold {self.cfg['hotkey'].upper()} to talk) ===") + self.ui = UI(HISTORY_FILE, levels=self.levels) + threading.Thread(target=self.load_model, daemon=True).start() + + hk = self.cfg["hotkey"].lower().replace(" ", "") + if "+" in hk: + # Modifier combo (e.g. "ctrl+shift"): raw hook, never suppressed so + # normal shortcuts keep working. Hold all parts to record; any + # OTHER key pressed while held cancels (it was a shortcut, not speech). + parts = hk.split("+") + allowed = set() + for p in parts: + allowed.update(variants(p)) + # tapping the polish key while recording turns on AI-polish for + # this dictation (it is not part of the hold combo) + polish_keys = set(variants(self.cfg["polish_key"])) + + LOCK_TAP = 0.35 # release the combo faster than this = lock mode + + def handler(event): + name = (event.name or "").lower() + if event.event_type == "down": + if self.recording: + if name in polish_keys: + if not self.polish_mode: + self.polish_mode = True + if self.ui: + self.ui.set_polish(True) + elif self.locked: + # hands-free: only Esc (cancel) or the combo again + # (finish) do anything — stray keys can't kill a + # long dictation + if name == "esc": + log("locked recording cancelled with Esc") + self.cancel_recording() + elif name in allowed and all(keyboard.is_pressed(p) for p in parts): + self.locked = False + if self.ui: + self.ui.set_locked(False) + self.on_release() + elif name not in allowed: + log(f"recording cancelled by third key: {name}") + self.cancel_recording() + elif all(keyboard.is_pressed(p) for p in parts): + self.on_press() + elif self.recording and not self.locked and name in allowed: + if time.time() - self.press_t < LOCK_TAP: + # quick tap -> lock the mic on, keep recording hands-free + self.locked = True + if self.ui: + self.ui.set_locked(True) + log("lock mode on — hands-free until the combo is tapped again") + else: + self.on_release() + + keyboard.hook(handler) + else: + keyboard.on_press_key(hk, self.on_press, suppress=True) + keyboard.on_release_key(hk, self.on_release, suppress=True) + + menu = pystray.Menu( + pystray.MenuItem(lambda item: f"Hold {self.cfg['hotkey'].upper()} to dictate", None, enabled=False), + pystray.Menu.SEPARATOR, + # default=True -> left-clicking the tray icon opens History + pystray.MenuItem("History", self.show_history, default=True), + pystray.MenuItem("Pause", self.toggle_pause, checked=lambda item: self.paused), + pystray.MenuItem("Copy last transcript", self.copy_last), + pystray.MenuItem("Edit vocabulary", self.open_vocab), + pystray.MenuItem("Start with Windows", self.toggle_autostart, checked=self.autostart_enabled), + pystray.Menu.SEPARATOR, + pystray.MenuItem("Quit", self.quit), + ) + self.tray = pystray.Icon("flow-dictation", make_icon_image("loading"), "Flow Dictation — loading model...", menu) + # tray runs detached on its own thread; tkinter (overlay + history + # window) owns the main thread — tk is not thread-safe otherwise + self.tray.run_detached() + self.ui.run() + + +def selftest(wav_path): + """Transcribe a wav file through the exact same pipeline (no mic/hotkey).""" + cfg = load_config() + app = FlowDictation(cfg) + app.load_model() + t0 = time.time() + segments, _ = app.model.transcribe( + wav_path, language=cfg["language"], vad_filter=True, + beam_size=cfg["beam_size"], initial_prompt=load_vocab_prompt(), + ) + text = " ".join(s.text.strip() for s in segments).strip() + print(f"SELFTEST ({time.time() - t0:.2f}s): {text}") + + +if __name__ == "__main__": + for s in (sys.stdout, sys.stderr): + try: + s.reconfigure(encoding="utf-8", errors="replace") + except Exception: + pass + if len(sys.argv) > 2 and sys.argv[1] == "--selftest": + selftest(sys.argv[2]) + else: + try: + FlowDictation(load_config()).run() + except Exception: + import traceback + + log("FATAL:\n" + traceback.format_exc()) + raise diff --git a/skills/flow-dictation/scripts/make_sounds.py b/skills/flow-dictation/scripts/make_sounds.py new file mode 100644 index 00000000..13d4931b --- /dev/null +++ b/skills/flow-dictation/scripts/make_sounds.py @@ -0,0 +1,56 @@ +"""Generate soft chime WAVs for flow-dictation (replaces harsh winsound.Beep). +v2 (2026-07-02): softer & warmer per Graeham — lower volume, C5/G5 pair, +slow airy decay with a detuned shimmer partial. Run: python make_sounds.py""" +import wave +from pathlib import Path + +import numpy as np + +SR = 44100 +OUT = Path(__file__).resolve().parent.parent / "assets" / "sounds" +OUT.mkdir(parents=True, exist_ok=True) + + +def pluck(freq, dur=0.38, vol=0.12, decay=8.0): + """Soft airy chime: slow 15ms attack, long gentle decay, a quiet octave + plus a barely-detuned partner tone for shimmer.""" + t = np.linspace(0, dur, int(SR * dur), endpoint=False) + env = np.minimum(t / 0.015, 1.0) * np.exp(-t * decay) + tone = ( + np.sin(2 * np.pi * freq * t) + + 0.5 * np.sin(2 * np.pi * freq * 1.003 * t) + + 0.2 * np.sin(2 * np.pi * freq * 2 * t) + ) + return vol * env * tone / 1.7 + + +def mix(notes): + """notes = [(start_sec, samples)] -> single float array""" + total = max(int(s * SR) + len(x) for s, x in notes) + out = np.zeros(total) + for s, x in notes: + i = int(s * SR) + out[i:i + len(x)] += x + return out + + +def save(name, sig): + sig = np.clip(sig, -1, 1) + data = (sig * 32767).astype(np.int16) + with wave.open(str(OUT / f"{name}.wav"), "wb") as f: + f.setnchannels(1) + f.setsampwidth(2) + f.setframerate(SR) + f.writeframes(data.tobytes()) + print(f"{name}.wav {len(data)/SR*1000:.0f}ms") + + +# start: warm rising pair (C5 -> G5) — "I'm listening" +save("start", mix([(0.00, pluck(523.25)), (0.09, pluck(783.99))])) +# done: warm falling pair (G5 -> C5) — "text delivered" +save("done", mix([(0.00, pluck(783.99)), (0.09, pluck(523.25))])) +# error: low soft double thud +save("error", mix([(0.00, pluck(174.61, dur=0.30, vol=0.20, decay=9)), + (0.14, pluck(155.56, dur=0.32, vol=0.17, decay=9))])) +# busy: single muted mid note +save("busy", mix([(0.00, pluck(392.00, dur=0.18, vol=0.08))])) diff --git a/skills/flow-dictation/scripts/ui.py b/skills/flow-dictation/scripts/ui.py new file mode 100644 index 00000000..28754b68 --- /dev/null +++ b/skills/flow-dictation/scripts/ui.py @@ -0,0 +1,419 @@ +"""ui.py — tkinter UI layer for Flow Dictation. + +Two pieces, both dark-themed with the gold brand accent: + 1. Overlay pill: a small frameless always-on-top capsule at the bottom-center + of the screen that appears while recording ("Listening 0:03") and + transcribing, then vanishes. Marked WS_EX_NOACTIVATE so it can never steal + focus from the app the user is dictating into. + 2. History window: searchable record of every dictation (persisted to + outputs/history.jsonl), newest first, click an entry to see the full text, + Copy button / double-click to copy. + +Threading: all tk calls happen on the main thread inside run(); other threads +talk to the UI only through the queue (add_entry / set_state / show_history). +""" +import json +import math +import queue +import time +import tkinter as tk +from datetime import datetime +from pathlib import Path + +BG = "#141414" +CARD = "#1e1e1e" +FG = "#eaeaea" +DIM = "#9a9a9a" +ACCENT = "#a78bfa" # soft violet — the app accent (gold retired 2026-07-02) +RED = "#e05548" +BLUE = "#5b8dd9" +PURPLE = "#aa6edc" +PINK = "#ec4899" + +# waveform gradient endpoints (violet -> magenta), per state +WAVE_COLORS = { + "recording": ((139, 92, 246), (236, 72, 153)), + "transcribing": ((91, 141, 217), (139, 92, 246)), + "polishing": ((170, 110, 220), (217, 70, 239)), +} + + +def _grad(a, b, t): + return "#%02x%02x%02x" % tuple(int(a[i] + (b[i] - a[i]) * t) for i in range(3)) + +MAX_HISTORY = 500 +TYPING_WPM = 40 # baseline for the "time saved" stat + + +class UI: + def __init__(self, history_file, levels=None): + self.history_file = Path(history_file) + self.levels = levels if levels is not None else [] + self.q = queue.Queue() + self.history = self._load() + self.state = "loading" + self.polish = False + self.locked = False + self.rec_t0 = None + self.root = None + self.overlay = None + self.overlay_dot = None + self.overlay_label = None + self.wave = None + self._disp = [] # displayed envelope amplitudes (animated toward targets) + self._phase = 0.0 # wave phase, advances each frame + self.win = None + self.listbox = None + self.detail = None + self.search_var = None + self.stats_label = None + self.filtered = [] + + # ---------- called from any thread ---------- + + def add_entry(self, text, app="", seconds=0.0, polished=False): + e = { + "ts": datetime.now().strftime("%Y-%m-%d %H:%M:%S"), + "text": text, + "app": app, + "seconds": round(seconds, 1), + "polished": polished, + } + self.history.append(e) + self.history = self.history[-MAX_HISTORY:] + try: + self.history_file.parent.mkdir(parents=True, exist_ok=True) + with open(self.history_file, "a", encoding="utf-8") as f: + f.write(json.dumps(e, ensure_ascii=False) + "\n") + except Exception: + pass + self.q.put(("refresh", None)) + + def set_state(self, state): + self.q.put(("state", state)) + + def set_polish(self, on): + self.q.put(("polish", on)) + + def set_locked(self, on): + self.q.put(("locked", on)) + + def show_history(self): + self.q.put(("show_history", None)) + + # ---------- internals (main thread only) ---------- + + def _load(self): + items = [] + try: + for line in self.history_file.read_text(encoding="utf-8").splitlines(): + try: + items.append(json.loads(line)) + except Exception: + pass + except FileNotFoundError: + pass + return items[-MAX_HISTORY:] + + def _no_activate(self, widget): + """WS_EX_NOACTIVATE so the overlay never takes focus from the target app.""" + try: + import ctypes + + widget.update_idletasks() + hwnd = widget.winfo_id() + GWL_EXSTYLE = -20 + WS_EX_NOACTIVATE = 0x08000000 + style = ctypes.windll.user32.GetWindowLongW(hwnd, GWL_EXSTYLE) + ctypes.windll.user32.SetWindowLongW(hwnd, GWL_EXSTYLE, style | WS_EX_NOACTIVATE) + except Exception: + pass + + def _build_overlay(self): + o = tk.Toplevel(self.root) + o.withdraw() + o.overrideredirect(True) + o.attributes("-topmost", True) + o.attributes("-alpha", 0.93) + o.configure(bg=BG) + inner = tk.Frame(o, bg=BG, padx=16, pady=8) + inner.pack() + self.overlay_dot = tk.Label(inner, text="●", bg=BG, fg=RED, font=("Segoe UI", 12)) + self.overlay_dot.pack(side="left", padx=(0, 8)) + self.wave = tk.Canvas(inner, width=150, height=26, bg=BG, highlightthickness=0) + self.wave.pack(side="left", padx=(0, 10)) + self.overlay_label = tk.Label(inner, text="Listening", bg=BG, fg=FG, font=("Segoe UI", 11)) + self.overlay_label.pack(side="left") + self.overlay = o + + def _draw_wave(self, state): + """Voice wave: a continuous oscillating line whose amplitude follows the + smoothed speech envelope, drawn in a violet->magenta gradient with a + fainter phase-shifted echo line underneath (getty-style sound wave). + Fast-attack/slow-decay animation keeps the motion fluid; the phase + advances every frame so the line undulates even between words.""" + self.wave.delete("all") + w, h, env_n, pts_n = 150, 26, 30, 60 + mid = h / 2 + samples = list(self.levels)[-env_n:] + # pad on the left so new audio enters from the right like a ticker + targets = [0.0] * (env_n - len(samples)) + [min(1.0, (s * 14) ** 0.5) for s in samples] + smoothed = [ + (targets[max(0, i - 1)] + 2 * targets[i] + targets[min(env_n - 1, i + 1)]) / 4 + for i in range(env_n) + ] + if len(self._disp) != env_n: + self._disp = [0.0] * env_n + for i in range(env_n): + t, d = smoothed[i], self._disp[i] + self._disp[i] = d + (t - d) * (0.6 if t > d else 0.22) + self._phase += 0.55 + + def envelope(pos): # linear interp of _disp at fractional index + f = pos * (env_n - 1) + i = min(env_n - 2, int(f)) + frac = f - i + e = self._disp[i] * (1 - frac) + self._disp[i + 1] * frac + return 0.05 + e * 0.95 # small idle ripple so it never flatlines + + a, b = WAVE_COLORS.get(state, WAVE_COLORS["recording"]) + for layer, (amp_k, freq, ph, width) in enumerate([ + (0.55, 0.62, 1.9, 1), # echo line: smaller, offset, thin + (1.00, 0.55, 0.0, 2), # main line + ]): + pts = [] + for j in range(pts_n + 1): + pos = j / pts_n + y = mid + envelope(pos) * (mid - 2) * amp_k * math.sin( + self._phase * (0.8 if layer == 0 else 1.0) + j * freq + ph + ) + pts.append((pos * w, y)) + for j in range(pts_n): + c = _grad(a, b, j / pts_n) + if layer == 0: # dim the echo toward the background + c = _grad(tuple(int(x * 0.45) for x in a), tuple(int(x * 0.45) for x in b), j / pts_n) + self.wave.create_line(*pts[j], *pts[j + 1], fill=c, width=width, capstyle="round") + + def _place_overlay(self): + self.overlay.update_idletasks() + w = self.overlay.winfo_reqwidth() + h = self.overlay.winfo_reqheight() + sw = self.root.winfo_screenwidth() + sh = self.root.winfo_screenheight() + self.overlay.geometry(f"{w}x{h}+{(sw - w) // 2}+{sh - h - 80}") + + def _update_overlay(self): + if self.state == "recording": + if self.rec_t0 is None: + self.rec_t0 = time.time() + secs = int(time.time() - self.rec_t0) + suffix = " ✨ polish" if self.polish else "" + if self.locked: + suffix += " 🔒 tap Ctrl+Alt to finish" + self.overlay_dot.config(fg=PURPLE if self.polish else PINK) + self._draw_wave("polishing" if self.polish else "recording") + self.overlay_label.config(text=f"Listening {secs // 60}:{secs % 60:02d}{suffix}") + self._place_overlay() + self.overlay.deiconify() + self._no_activate(self.overlay) + elif self.state in ("transcribing", "polishing"): + self.rec_t0 = None + polishing = self.state == "polishing" + self.overlay_dot.config(fg=PURPLE if polishing else BLUE) + self._draw_wave(self.state) + self.overlay_label.config(text="Polishing with Claude…" if polishing else "Transcribing…") + self._place_overlay() + self.overlay.deiconify() + self._no_activate(self.overlay) + else: + self.rec_t0 = None + self.overlay.withdraw() + + # ---------- history window ---------- + + def _build_history_window(self): + w = tk.Toplevel(self.root) + w.title("Flow Dictation") + w.configure(bg=BG) + w.geometry("500x600") + w.attributes("-topmost", True) + try: + w.iconbitmap(str(Path(__file__).resolve().parent.parent / "assets" / "flow.ico")) + except Exception: + pass + + tk.Label( + w, text="Speak naturally, paste instantly — hold Ctrl+Alt anywhere", + bg=BG, fg=DIM, font=("Segoe UI", 9), anchor="w", padx=12, pady=8, + ).pack(fill="x") + + # Blip-style stat cards: value on top, caption below + cards = tk.Frame(w, bg=BG, padx=10) + cards.pack(fill="x") + self.stat_vals = {} + for key, caption in ( + ("today", "words today"), + ("month", "words this month"), + ("wpm", "avg dictation WPM"), + ("saved", "money saved"), + ): + card = tk.Frame(cards, bg=CARD, padx=10, pady=10) + card.pack(side="left", expand=True, fill="both", padx=3) + val = tk.Label(card, text="—", bg=CARD, fg=ACCENT, font=("Segoe UI", 15, "bold")) + val.pack() + tk.Label(card, text=caption, bg=CARD, fg=DIM, font=("Segoe UI", 8)).pack() + self.stat_vals[key] = val + + tk.Label( + w, text="Recent Transcriptions", bg=BG, fg=FG, + font=("Segoe UI", 11, "bold"), anchor="w", padx=12, pady=8, + ).pack(fill="x") + + top = tk.Frame(w, bg=BG, padx=10, pady=8) + top.pack(fill="x") + tk.Label(top, text="Search", bg=BG, fg=DIM, font=("Segoe UI", 9)).pack(side="left", padx=(0, 6)) + self.search_var = tk.StringVar() + self.search_var.trace_add("write", lambda *a: self._refresh_list()) + entry = tk.Entry( + top, textvariable=self.search_var, bg=CARD, fg=FG, insertbackground=ACCENT, + relief="flat", font=("Segoe UI", 10), + ) + entry.pack(side="left", fill="x", expand=True, ipady=4) + + mid = tk.Frame(w, bg=BG, padx=10) + mid.pack(fill="both", expand=True) + sb = tk.Scrollbar(mid) + sb.pack(side="right", fill="y") + self.listbox = tk.Listbox( + mid, bg=CARD, fg=FG, selectbackground=ACCENT, selectforeground="#141414", + relief="flat", font=("Segoe UI", 10), activestyle="none", yscrollcommand=sb.set, + ) + self.listbox.pack(fill="both", expand=True) + sb.config(command=self.listbox.yview) + self.listbox.bind("<>", lambda e: self._show_detail()) + self.listbox.bind("", lambda e: self._copy_selected()) + + self.detail = tk.Text( + w, height=5, bg=CARD, fg=FG, relief="flat", wrap="word", + font=("Segoe UI", 10), padx=8, pady=6, state="disabled", + ) + self.detail.pack(fill="x", padx=10, pady=(8, 0)) + + btns = tk.Frame(w, bg=BG, pady=8) + btns.pack(fill="x", padx=10) + for label, cmd in (("Copy", self._copy_selected), ("Clear history", self._clear_history)): + tk.Button( + btns, text=label, command=cmd, bg=CARD, fg=ACCENT, relief="flat", + font=("Segoe UI", 10), padx=14, pady=4, activebackground=ACCENT, + activeforeground="#141414", cursor="hand2", + ).pack(side="left", padx=(0, 8)) + self.count_label = tk.Label(btns, text="", bg=BG, fg=DIM, font=("Segoe UI", 9)) + self.count_label.pack(side="right") + + w.protocol("WM_DELETE_WINDOW", w.withdraw) + self.win = w + self._refresh_list() + + def _refresh_list(self): + if not (self.win and self.listbox): + return + needle = (self.search_var.get() if self.search_var else "").lower() + self.filtered = [ + e for e in reversed(self.history) + if needle in e["text"].lower() or needle in e.get("app", "").lower() + ] + self.listbox.delete(0, "end") + for e in self.filtered: + t = e["ts"][11:16] + day = e["ts"][5:10] + mark = "✨" if e.get("polished") else " " + preview = e["text"][:50] + ("…" if len(e["text"]) > 50 else "") + self.listbox.insert("end", f" {day} {t} {mark} {preview}") + self.count_label.config(text=f"{len(self.filtered)} of {len(self.history)}") + self._refresh_stats() + + def _refresh_stats(self): + if not getattr(self, "stat_vals", None): + return + now = datetime.now() + today, month = now.strftime("%Y-%m-%d"), now.strftime("%Y-%m") + t_words = t_secs = d_words = m_words = 0 + for e in self.history: + words = len(e["text"].split()) + t_words += words + t_secs += e.get("seconds", 0) or 0 + if e["ts"][:10] == today: + d_words += words + if e["ts"][:7] == month: + m_words += words + wpm = (t_words / (t_secs / 60)) if t_secs else 0 + # same framing Blip uses: time saved vs typing, valued at $15/hr + saved_hours = max(0.0, t_words / TYPING_WPM - t_secs / 60) / 60 + self.stat_vals["today"].config(text=f"{d_words:,}") + self.stat_vals["month"].config(text=f"{m_words:,}") + self.stat_vals["wpm"].config(text=f"{wpm:.0f}") + self.stat_vals["saved"].config(text=f"${saved_hours * 15:,.2f}") + + def _selected_entry(self): + sel = self.listbox.curselection() if self.listbox else () + return self.filtered[sel[0]] if sel else None + + def _show_detail(self): + e = self._selected_entry() + if not e: + return + self.detail.config(state="normal") + self.detail.delete("1.0", "end") + app = f" → {e['app']}" if e.get("app") else "" + self.detail.insert("1.0", f"{e['ts']}{app}\n{e['text']}") + self.detail.config(state="disabled") + + def _copy_selected(self): + e = self._selected_entry() + if not e: + return + self.root.clipboard_clear() + self.root.clipboard_append(e["text"]) + + def _clear_history(self): + self.history = [] + try: + self.history_file.write_text("", encoding="utf-8") + except Exception: + pass + self._refresh_list() + + def _show_history_window(self): + if self.win is None or not self.win.winfo_exists(): + self._build_history_window() + self._refresh_list() + self.win.deiconify() + self.win.lift() + + # ---------- main loop ---------- + + def _poll(self): + try: + while True: + cmd, arg = self.q.get_nowait() + if cmd == "state": + self.state = arg + elif cmd == "polish": + self.polish = arg + elif cmd == "locked": + self.locked = arg + elif cmd == "refresh": + self._refresh_list() + elif cmd == "show_history": + self._show_history_window() + except queue.Empty: + pass + self._update_overlay() + self.root.after(60, self._poll) + + def run(self): + self.root = tk.Tk() + self.root.withdraw() + self._build_overlay() + self._poll() + self.root.mainloop() diff --git a/skills/flow-dictation/vocab.txt b/skills/flow-dictation/vocab.txt new file mode 100644 index 00000000..bd9c62ad --- /dev/null +++ b/skills/flow-dictation/vocab.txt @@ -0,0 +1,35 @@ +# Flow Dictation — personal vocabulary +# One name/term per line. These prime Whisper so it spells your world correctly. +# Edit any time (tray menu > Edit vocabulary); takes effect on the next dictation. +Graeham Watts +Compass +East Palo Alto +Menlo Park +Redwood City +Palo Alto +San Mateo County +PropertyIQ +PropIQ +PropCast +PropFlow +PropReach +PropClose +PropSearch +Wattson +GoHighLevel +GHL +Trackabi +Wise +DoorLoop +Obsidian +CMA +DRE +Zillow +Zestimate +HeyGen +ElevenLabs +Higgsfield +n8n +Supabase +QuestLab +Virtudesk diff --git a/skills/founder-academy/SKILL.md b/skills/founder-academy/SKILL.md new file mode 100644 index 00000000..618549c3 --- /dev/null +++ b/skills/founder-academy/SKILL.md @@ -0,0 +1,65 @@ +--- +name: founder-academy +description: "Graeham's private PropertyIQ Academy — his condensed founder's-MBA + AI-fluency + operator course (8 courses, 61 lessons + capstone) that teaches him everything he needs to build, run, fund, staff, and intelligently TALK about PropertyIQ and AI, without learning to code. Use ANY time Graeham says: PropertyIQ Academy, the Academy..." + +--- + +# PropertyIQ Academy — tutor + +A private, condensed founder-architect program built FOR Graeham Watts. Audio lessons (MP3s he +plays at the gym) plus interactive chat drills. This skill is the tutor + the production system. + +## Where everything lives (source of truth) +`C:\Users\Graeham Watts\Documents\Obsidian\PropertyIQ Academy\` +- `00 - Curriculum Master.md` — the full 8-course / 61-lesson syllabus (Fugu-architected). +- `Lesson NN - .md` — each lesson: episode notes + audio script + chat drill. +- `Audio/PropertyIQ-Academy-Lesson-NN.mp3` — the narrated episodes. +- `01 - Founder Intake.md` — his real answers (vision, money, IDX, team) that personalize lessons. +- `02 / 03 - Moat & Strategy memos` — Fugu strategy analyses; teaching material for Courses 4/6/7. +- `Tutor Prompt (paste into Claude.ai Project).md` — the mobile-app version of this tutor. +- `BUILD-LOG.md` + `academy_state.json` (in `Documents\Skills LLMS\Claude\fugu\`) — production status. + +ALWAYS read the actual lesson note from the vault before teaching it. Do not teach from memory. + +## Who Graeham is (calibration) +Working Bay Area broker (Compass), sharp and verbal, NOT a coder and never will be. Bootstrapping +PropertyIQ solo, no runway, funding via commissions, building via AI agents + Sami/Mehmood. +Endgame: a ~$7-15M strategic acquisition or recurring freedom. North star: sound credible with +investors and developers. He asked to be stress-tested, not flattered. Be direct; challenge vague +answers; a little dry wit is welcome. The real moat to keep reinforcing: the consented, audit-grade +**outcome graph** (Event Ledger), not content and not "the suite." + +## Running a lesson (interactive, ~15-25 min) +When he says "teach me lesson N" / "next lesson": +1. Read `Lesson NN - <title>.md` from the vault. +2. Offer the choice: **listen** (point him to the MP3 / read the audio script aloud-style) or **do the drill** (interactive). +3. If drill: run the lesson's own Chat Drill section — recall check → vocabulary drill → ROOM ROLEPLAY (you play the skeptical investor/engineer/customer; push back) → BS-detector → save one artifact. +4. Save the artifact into the vault (a new note or appended to `Founder Artifacts.md`), and update progress. + +## Hybrid mid-listen mode (important) +He listens to MP3s on the go and asks questions in between. If he arrives with a quick one-off +question ("explain the outcome graph again", "what's acceptance criteria?"), just answer it directly +and concisely with a PropertyIQ example, then ask if he wants to continue or get back to listening. +Do NOT force the full drill unless he asks for it. Quick question = quick answer; "do the lesson" = full drill. + +## Progress tracking +Keep `PropertyIQ Academy/PROGRESS.md` in the vault: a checklist of all 61 lessons with date completed +and a one-line note on how he did / what to revisit. At the start of a session, read it and tell him +where he is and what's next. + +## Producing / re-producing audio (skill-author tasks, run in Claude Code) +- Curriculum + per-lesson authoring is done by **Fugu Ultra** (his standing rule). Engine: + `Documents\Skills LLMS\Claude\fugu\build_academy.py` (manifest in `academy_lessons.py`). It authors a lesson, + writes the vault note, and synthesizes the MP3 within the ElevenLabs quota. Resumable. +- **Voice:** premium ElevenLabs NARRATOR (currently "Brian", voice_id `nPczCjzI2devNBz1zQrb`), + model `eleven_multilingual_v2`, 192kbps. To change the voice, swap `VOICE` in `build_academy.py` + and re-run `synth` over the existing scripts (cheap — scripts are not re-authored). +- **Key:** ElevenLabs key at `Documents\Skills LLMS\Claude\.heygen-credentials\elevenlabs-key.txt` (session-scoped; + re-paste if missing). Quota is Creator tier (~120k chars/mo); producing all 61 as audio needs a Pro/Scale upgrade. +- **Podcast feed:** MP3s + `feed.xml` publish to the `online-content` repo (GitHub Pages) so the Academy + appears in his podcast app. Honor the brand tripwire before any push. + +## Authoring NEW or revised lessons +Write a brief in the `build_academy.py` style (standing context + the lesson's title/desc + the +fixed AUDIO SCRIPT / CHAT DRILL / EPISODE NOTES output format + the no-em-dash TTS rules) and call +`python fugu.py --stream --model fugu-ultra --file <brief>`. Then synth and save like the engine does. diff --git a/skills/github-repo-analyzer/SKILL.md b/skills/github-repo-analyzer/SKILL.md new file mode 100755 index 00000000..d8346c98 --- /dev/null +++ b/skills/github-repo-analyzer/SKILL.md @@ -0,0 +1,79 @@ +--- +name: github-repo-analyzer +description: "GitHub Repository & Developer Activity Analyzer. Use ANY time user mentions: GitHub repo review, code review, developer activity, commit history analysis, PR review, pull request audit, repo health check, code quality audit, developer productivity, sprint review, dev team analysis, GitHub audit, repo analysis, codebase review..." +--- + +# GitHub Repository & Developer Activity Analyzer + +**Before starting, read the reference files:** +- `references/review-criteria.md` — Defines the analysis framework, flag system, and report structure + +## Report Format + +### Flag System + +**🔴 CRITICAL** — Immediate attention needed +**🟡 WARNING** — Needs attention soon +**🟢 WATCH** — Monitor, not urgent + +### Report Sections + +**Section 0 — Repo Attribution & Verification** +- Repo Attribution Table showing which repos belong to the dev team vs client vs previous team +- External tool development status (detected / confirmed / not detected) +- Migration compliance status (if client has made migration requests) +- Previous developers identified and excluded + +**Section 1 — Executive Summary** +- Repository name, analysis period, total contributors active +- Overall health score (Healthy / Needs Attention / At Risk) +- Top 3 findings that need action +- Quick stats: commits, PRs merged, avg merge time, open issues +- External tool workflow status (if applicable) + +**Section 2 — Repository Health** +- Activity trends, branch hygiene, CI status, documentation state +- Governance & ownership assessment +- Comparison to previous period if data available + +**Section 3 — Developer Scorecards** +For each developer: +- Flag level (Critical/Warning/Watch/Healthy) +- Activity summary (commits, PRs, reviews) +- Push pattern (incremental vs bulk) +- Strengths observed +- Areas for improvement +- Specific recommendations + +For ghost developers (billed but no activity): +- Flag as Critical +- Include fairness section with possible explanations +- Specific verification steps the client should take + +**Section 4 — Team Dynamics** +- Workload distribution chart/breakdown +- Review network (who reviews whom) +- Collaboration patterns +- Knowledge silo risks +- Billed vs visible developer gap analysis + +**Section 5 — Action Items** +Numbered, specific, actionable items prioritized as HIGH / MEDIUM / LOW + +**Section 6 — Recommendations** +Process improvements based on patterns observed, including: +- External tool migration plan (if applicable) +- PR/review workflow requirements +- CI/CD setup recommendations +- Governance improvements + +## Output Options + +Ask the user how they want the report: + +1. **In-chat summary** — Quick overview right here in the conversation +2. **HTML report** — Branded, formatted report saved as a file (recommended for sharing) +3. **Markdown report** — Clean markdown file for documentation +4. **Spreadsheet** — Developer metrics in an Excel file for tracking over time + +Default to HTML report unless the user specifies otherwise. diff --git a/skills/github-repo-analyzer/references/review-criteria.md b/skills/github-repo-analyzer/references/review-criteria.md new file mode 100755 index 00000000..405db31b --- /dev/null +++ b/skills/github-repo-analyzer/references/review-criteria.md @@ -0,0 +1,303 @@ +# GitHub Repo Analyzer — Review Criteria & Benchmarks + +## Developer Activity Benchmarks + +Use these as baseline expectations. Adjust based on team size, project phase, and role. + +### Healthy Activity Levels (per 2-week sprint) + +| Metric | Healthy | Warning | Critical | +|--------|---------|---------|----------| +| Commits | 10+ | 3-9 | 0-2 | +| PRs opened | 3+ | 1-2 | 0 | +| PRs reviewed (for others) | 2+ | 1 | 0 | +| Avg days to review assigned PR | < 1 day | 1-3 days | 3+ days | +| Avg PR size (lines changed) | < 300 | 300-500 | 500+ | + +**Important context adjustments:** +- Part-time contributors: Cut all thresholds in half +- Team leads: May have fewer commits but should have MORE reviews +- New team members (first 30 days): Expect lower numbers as they ramp up +- Sprint planning / design phases: Lower commit volume is normal +- External development tools: See "External Tool Development Pattern" section below + +--- + +## External Tool Development Pattern + +Some outsourced teams use their own internal development environments, IDEs, or platforms to build code — then push finished or near-finished code to the client's GitHub repos in bulk. This creates a distinct pattern that the analyzer must detect, flag, and account for. + +### Why This Matters + +When a team develops on an internal tool and only pushes to the client's GitHub when features are "done": +1. **The client loses real-time visibility** into development progress +2. **Commit history is compressed** — weeks of incremental work shows up as a few large commits +3. **Code review is impossible** during development — the client only sees the final output +4. **Risk accumulates silently** — bugs, architectural issues, and scope drift are invisible until the push +5. **The client doesn't own the work-in-progress** — if the engagement ends, unfinished code may never be delivered +6. **Standard commit frequency benchmarks don't apply** — a developer may be active but invisible + +### Detection Signals + +Flag a repository for "External Tool Development Pattern" when you observe: + +| Signal | What It Looks Like | +|--------|-------------------| +| Bulk push pattern | Large number of files/lines committed in a single push or a short burst (1-2 days), followed by weeks of silence | +| Initial commit is fully built | First commit contains a complete or near-complete application structure, not gradual buildout | +| Low commit frequency, high commit size | Few commits but each one changes hundreds or thousands of lines | +| Missing incremental history | No "work in progress" commits, no iterative debugging trail — code appears fully formed | +| Commit timestamps clustered | All commits within a few hours, suggesting a batch push from another system | +| Single contributor across large codebases | One developer account pushes everything, but the volume implies multiple people's work | +| No branch/PR development cycle | Features appear directly on main or dev branch without feature branch → PR → merge flow | + +### Adjusted Benchmarks for External Tool Workflows + +When external tool development is detected, standard commit frequency benchmarks are **not reliable** indicators of developer activity. Instead, shift analysis to: + +| Metric | What to Evaluate Instead | +|--------|-------------------------| +| Commit frequency | **Push frequency** — How often does code arrive in the client's repo? Weekly pushes = acceptable. Monthly = governance risk | +| Developer count | **Unique committer count vs billed team size** — If 4 devs are billed but 1 pushes, the others are invisible | +| Code quality | **Code structure and architecture quality** of what was pushed, since you can't evaluate the development process | +| Progress tracking | **Feature completeness per push** — Is shipped code functional, or are there half-built features? | +| Collaboration | **Cannot be assessed** — internal tool collaboration is invisible to the client | + +### Governance Flags for External Tool Workflows + +| Flag | Condition | Severity | +|------|-----------|----------| +| 🔴 CRITICAL | Client has explicitly requested team push to client repos and team has not complied | CRITICAL — Governance violation | +| 🔴 CRITICAL | Client cannot verify which developers are working due to single-account pushes | CRITICAL — Accountability gap | +| 🟡 WARNING | Team is developing externally but pushing regularly (weekly or better) | WARNING — Acceptable interim, needs migration plan | +| 🟡 WARNING | Repo shows bulk-push pattern but client hasn't explicitly required real-time commits | WARNING — Recommend requiring it | +| 🟢 WATCH | Team uses external tools for CI/testing but commits incrementally to client repo | WATCH — Acceptable workflow | + +### Recommended Actions When External Tool Pattern Is Detected + +1. **Require immediate migration to client GitHub repos** — All active development should happen in repos the client owns and can monitor +2. **Require daily or per-feature-branch pushes** — Even if the team uses internal tools for testing, code should be pushed to the client repo incrementally, not in bulk +3. **Establish branch protection + PR requirements** — Forces the team to use PRs for integration, creating visibility even if they develop elsewhere +4. **Request full team GitHub access** — All developers should have individual accounts pushing commits, not one person batch-pushing everyone's work +5. **Set up a migration deadline** — Give the team a specific date (e.g., 7 business days) to move all active work to client repos +6. **If non-compliant after deadline** — Escalate to contract/engagement terms review + +--- + +## Repo Verification Checklist + +Before analyzing, verify you are looking at the correct repositories. This prevents wasting time auditing repos the client built themselves or that belong to a previous engagement. + +### Pre-Analysis Questions + +1. **Who owns these repos?** — Is the client the GitHub owner, or is the dev team hosting them? +2. **Which repos does the dev team actively commit to?** — Get explicit confirmation, not assumptions +3. **Are there repos the team uses that the client doesn't have access to?** — If yes, flag immediately +4. **Did the client build any of these repos themselves?** — Exclude client-built repos from team performance scoring +5. **Are there previous developers whose commits should be excluded?** — Get names/usernames to filter out + +### Repo Attribution Table + +Before scoring, build a clear attribution table: + +| Repository | Who Built It | Current Team Active? | Include in Audit? | +|------------|-------------|---------------------|-------------------| +| [repo name] | [client / dev team / previous team] | [yes/no] | [yes/no — with reason] | + +This table must appear in the report. It prevents misattribution (e.g., praising the dev team for screens the client built, or flagging a repo as inactive when it was intentionally handed off). + +--- + +## Commit Quality Indicators + +**Good commit messages:** +- Start with a verb (Add, Fix, Update, Refactor, Remove) +- Reference issue/ticket numbers +- Explain WHY, not just WHAT +- Under 72 characters for the subject line + +**Red flag commit messages:** +- Single word: "fix", "update", "changes", "stuff" +- No issue/ticket reference on a team that uses issue tracking +- Extremely long messages that should have been PR descriptions +- "WIP" commits pushed to main branch + +### PR Quality Indicators + +**Good PR patterns:** +- Clear title and description +- Linked to an issue or ticket +- Reasonable size (under 300 lines ideal) +- Has at least one reviewer assigned +- CI checks pass before merge +- Conversation/feedback addressed before merge + +**Red flag PR patterns:** +- Empty description +- 1000+ lines changed (impossible to properly review) +- Self-approved and self-merged +- Merged with failing CI checks +- No linked issue (on teams that use issue tracking) +- Force-merged bypassing review requirements + +--- + +## Repository Health Scoring + +### Overall Health Score + +Calculate based on these weighted factors: + +| Factor | Weight | Healthy | Needs Attention | At Risk | +|--------|--------|---------|-----------------|---------| +| CI/CD status | 20% | All checks passing | Flaky tests | Failing on main | +| PR review rate | 20% | >80% reviewed before merge | 50-80% reviewed | <50% reviewed | +| Avg merge time | 15% | <2 days | 2-5 days | 5+ days | +| Branch hygiene | 10% | <5 stale branches | 5-15 stale | 15+ stale | +| Open PR age | 15% | All <3 days | Some 3-7 days | Any 7+ days | +| Issue management | 10% | Issues triaged and assigned | Backlog growing | Issues ignored | +| Documentation | 10% | README current, contributing guide exists | README outdated | No README | + +### Additional Factor: Code Ownership & Governance (applies when external tool pattern detected) + +When an external tool development pattern is detected, add this weighted factor: + +| Factor | Weight | Healthy | Needs Attention | At Risk | +|--------|--------|---------|-----------------|---------| +| Code ownership governance | 15% (redistributed from other factors) | All code in client repos, incremental commits, all devs visible | External tool used but regular pushes, migration plan in place | Client has requested migration and team has not complied | + +When this factor is added, redistribute weight by reducing CI/CD and PR review rate by 5% each, and Open PR age by 5% — because those metrics are less meaningful when the team isn't using the client's repo as their primary development environment. + +### Score Interpretation +- **Healthy (70-100%)**: Repo is well-maintained, team processes are working +- **Needs Attention (40-69%)**: Some areas slipping, targeted improvements needed +- **At Risk (0-39%)**: Significant process gaps, technical debt accumulating + +--- + +## Flag Criteria — Detailed + +### 🔴 CRITICAL — Developer Level + +| Condition | Why It Matters | +|-----------|---------------| +| Zero commits AND zero PRs in the full analysis period | Developer appears inactive | +| Assigned to issues/PRs but zero progress | Work is stalled, may be blocked | +| Merging own PRs to main with no review, repeatedly | Quality gates being bypassed | +| Breaking CI on main branch and not fixing it | Blocking the whole team | +| Billed developer with no GitHub username identifiable | Cannot verify work is being performed | + +### 🟡 WARNING — Developer Level + +| Condition | Why It Matters | +|-----------|---------------| +| Commit count dropped 50%+ vs previous period | Possible disengagement or blocker | +| Zero code reviews given to others | Not participating in team quality process | +| PRs averaging 500+ lines | Code is hard for others to review properly | +| Assigned reviews sitting unactioned for 3+ days | Blocking other developers | +| Only committing to one directory/module | Knowledge silo forming | +| Pushing bulk commits from external tool instead of incremental development | Process visibility gap | + +### 🟢 WATCH — Developer Level + +| Condition | Why It Matters | +|-----------|---------------| +| New to repo (first 30 days of commits) | Expected ramp-up period | +| Commit messages declining in quality | Minor but worth mentioning | +| Slightly fewer reviews than team average | Not urgent but track the trend | +| Working late/weekend commits increasing | Possible workload issue | + +### 🔴 CRITICAL — Repository Level + +| Condition | Why It Matters | +|-----------|---------------| +| CI/CD failing on main/master branch | Deployments blocked, team can't ship | +| PRs open 14+ days with no activity | Work is abandoned or stuck | +| No branch protection on main | Anyone can push directly, risky | +| Security vulnerabilities flagged by Dependabot/similar | Active security risk | +| Client requested code migration to their repos and team has not complied | Governance violation — client doesn't own the work they're paying for | +| Repo misnamed or mislabeled vs actual contents | Creates confusion about what's been built and what hasn't | + +### 🟡 WARNING — Repository Level + +| Condition | Why It Matters | +|-----------|---------------| +| 5+ stale branches (30+ days inactive) | Cluttered repo, potential merge conflicts | +| No CI/CD configured at all | No automated quality checks | +| README hasn't been updated in 90+ days | Documentation drifting from reality | +| Average merge time exceeding 5 days | Development velocity is slow | +| External tool development detected but no migration plan | Visibility and ownership risk accumulating | +| Feature branches unmerged with no PRs | Work may be stalled or abandoned | + +### 🟢 WATCH — Repository Level + +| Condition | Why It Matters | +|-----------|---------------| +| Test coverage declining (if trackable) | Quality may slip over time | +| Issue backlog growing faster than closing | Scope creep or understaffing | +| Release frequency slowing | May indicate complexity or blockers | + +--- + +## Report Templates + +### Executive Summary Template +``` +REPOSITORY: [repo name] +PERIOD: [start date] — [end date] +HEALTH SCORE: [X]% — [Healthy/Needs Attention/At Risk] + +QUICK STATS: +- [X] commits by [Y] contributors +- [X] PRs merged (avg [Y] days to merge) +- [X] open PRs | [X] open issues +- CI Status: [Passing/Failing/Not configured] + +EXTERNAL TOOL STATUS: [Not detected / Detected — see findings / Confirmed by client] +REPO VERIFIED: [Yes — confirmed as dev team repo / No — needs verification] + +TOP FINDINGS: +1. [Most important finding] +2. [Second most important] +3. [Third most important] +``` + +### Developer Scorecard Template +``` +DEVELOPER: @[username] +FLAG: [🔴/🟡/🟢/✅] +PERIOD: [dates] + +ACTIVITY: +- Commits: [X] ([up/down X%] vs previous period) +- PRs opened: [X] | PRs merged: [X] +- Reviews given: [X] | Avg review time: [X] days +- Primary work areas: [directories/modules] +- Push pattern: [Incremental / Bulk push / External tool suspected] + +STRENGTHS: +- [Positive observation] + +AREAS FOR IMPROVEMENT: +- [Constructive observation] + +RECOMMENDATION: +- [Specific action item] +``` + +--- + +## Context Questions to Ask + +Before running the analysis, gather context that affects interpretation: + +1. **Team structure** — How many developers? Full-time or part-time? Any contractors? +2. **Sprint cadence** — Weekly? Bi-weekly? Kanban (no sprints)? +3. **Current phase** — Building new features? Maintenance mode? Pre-launch crunch? +4. **Known absences** — Anyone on PTO or leave during the analysis period? +5. **Non-code work** — Are some developers doing design, planning, or documentation that won't show in commits? +6. **Specific concerns** — Is there a particular developer or issue they want investigated? +7. **Development environment** — Is the team developing directly in the client's GitHub repos, or do they use an internal tool/platform and push code periodically? (This is critical for interpreting commit patterns) +8. **Repo ownership** — Did the client build any of the repos being analyzed? (Exclude client-built repos from team scoring) +9. **Previous developers** — Are there commits from a prior team that should be filtered out? +10. **Migration requests** — Has the client asked the team to change their workflow (e.g., stop using internal tools, push to client repos)? If yes, has the team complied? diff --git a/skills/heygen-elevenlabs-renderer/SKILL.md b/skills/heygen-elevenlabs-renderer/SKILL.md new file mode 100644 index 00000000..3e5fdae5 --- /dev/null +++ b/skills/heygen-elevenlabs-renderer/SKILL.md @@ -0,0 +1,151 @@ +--- +name: heygen-elevenlabs-renderer +description: "End-to-end avatar video rendering pipeline for Graeham Watts. Synthesizes Graeham's cloned voice on ElevenLabs from an SSML script (including prosody and break SSML tags from v5.4 format), uploads the MP3 to HeyGen, renders an avatar video against Graeham's personal avatar via the v3 Create Avatar Video endpoint, and returns a playable MP4. Use ANY time the user says render this script, make a video from this script, auto-render, full auto, push this to HeyGen, avatar video, voice clone video, or clicks the Full Auto-Render button on the v5.4 weekly calendar. Also trigger when the content-creation-engine has produced a v5.4 script and the user wants the video built without manual steps." +--- + +# HeyGen + ElevenLabs Renderer + +## Purpose +Turn a finished v5.4 script into a delivered MP4 video with zero manual work. This skill owns the rendering layer of the content pipeline. Everything before this point is ideation and writing (content-calendar → content-creation-engine). Everything this skill does is mechanical execution. + +## Pipeline at a glance +``` +v5.4 SSML script + │ + ▼ +ElevenLabs TTS (Graeham's voice clone) ─── scripts/synthesize_voice.py + │ eleven_multilingual_v2 + ▼ Supports <break time="Xs"/> +audio.mp3 (44.1 kHz, 128 kbps mono) Accepts <prosody> (silent pass-through) + │ + ▼ +HeyGen asset upload ─── scripts/upload_asset.py + │ Uses CLI: `heygen asset create --file` + ▼ +asset_id + │ + ▼ +HeyGen v3 /videos create (avatar + audio) ─── scripts/render_video.py + │ voice mode = audio_asset_id + ▼ +video_id (status: waiting → processing) + │ + ▼ +Poll /v1/video_status.get every 15s ─── scripts/poll_and_download.py + │ + ▼ +video_url (signed HeyGen CDN URL) + │ + ▼ +Download MP4 → outputs/renders/<slug>.mp4 +``` + +## Credentials +Store keys at `/sessions/<session>/mnt/outputs/.claude-credentials/` with `chmod 600`: +- `heygen-key.txt` (used via `HEYGEN_API_KEY` env var or CLI config) +- `elevenlabs-key.txt` (used via `xi-api-key` header) + +If keys are missing at session start, STOP and ask the user to paste them. Never proceed with a placeholder. + +## Defaults (from registry.json) +These are the canonical production defaults. Never hardcode — always read `registry.json`: +| Field | Value | +|---|---| +| HeyGen avatar | Graeham Watts — `9a3600b16f604059b6ab8b9a55e29ea9` | +| ElevenLabs voice | Graeham Watts Voice Clone — `Pa3vOYQHHpLJn1Tf7hnP` | +| ElevenLabs model | `eleven_multilingual_v2` | +| Aspect | `9:16` (vertical for Reels/Shorts/TikTok) | +| Resolution | `720p` (bump to `1080p` for listing videos) | + +The user has 70 personal "Graeham Watts" avatar looks. The renderer always uses the primary unless the script specifies an alternate look ID. + +## Registry refresh +Re-run `scripts/refresh_registry.py` when: +- A new avatar look is trained in HeyGen +- A new voice is added/cloned in ElevenLabs +- The user reports "avatar not found" + +## SSML compatibility notes (IMPORTANT) +ElevenLabs has **partial** SSML support — verified on `eleven_multilingual_v2`: + +| Tag | Supported | Behavior | +|---|---|---| +| `<break time="0.4s"/>` | YES | Literal silence of the specified duration | +| `<speak>...</speak>` | YES (wrapper) | Required root for SSML mode | +| `<prosody rate="slow">...</prosody>` | Silent pass-through | API accepts it but the rate/pitch attrs are NOT honored — the text inside is still synthesized, just without the prosody effect | +| `<emphasis>`, `<say-as>`, `<phoneme>` | NOT supported | Tag is stripped; inner text is read normally | + +**What this means for v5.4 scripts:** `<break>` tags give you deterministic pause timing (critical for pattern interrupts). `<prosody>` tags are safe to keep in the script for human readability but don't rely on them for actual delivery changes. If you need slower/faster delivery, use ElevenLabs' `voice_settings.stability` and `style` parameters instead, or preprocess the text into multiple TTS calls with different settings and concatenate. + +For audio-tag-based delivery (laughs, sighs, etc.), use ElevenLabs' bracket syntax like `[laughs]`, `[whispers]` — NOT SSML. See `references/elevenlabs-audio-tags.md`. + +## Primary invocation +```bash +python3 scripts/full_render.py \ + --script /path/to/script.ssml.txt \ + --slug "ab1482-explainer-week3" \ + --resolution 720p \ + --aspect 9:16 +``` + +This wraps all four pipeline stages. Output file lands at `outputs/renders/<slug>.mp4` with a sibling `<slug>.meta.json` containing `video_id`, `audio_asset_id`, `duration`, `completed_at`, and a full `dashboards` block (HeyGen video page, HeyGen projects list, ElevenLabs history, ElevenLabs voice library). The poller also emits a single-line `RENDER_RESULT={...}` to stdout so webhook consumers and the v5.4 calendar button can surface those links without scraping log output. + +### Where renders + voices live + +| Asset | Dashboard URL | +|---|---| +| Finished video (this render) | `https://app.heygen.com/videos/<video_id>` | +| All past videos | `https://app.heygen.com/projects` | +| TTS generation history | `https://elevenlabs.io/app/speech-synthesis/history` | +| Graeham voice clone | `https://elevenlabs.io/app/voice-library` | +| Local MP4 | `outputs/renders/<slug>.mp4` | +| Local metadata | `outputs/renders/<slug>.meta.json` | + +The v5.4 weekly calendar button reads the `dashboards` object returned by `webhook_handler.py /status/<job_id>` and renders it as a row of click-through links next to the render status. The banner at the top of the Production Map tab also pings `/health` and always shows the HeyGen + ElevenLabs dashboard links — online or offline — so Graeham can always find the content. + +## Per-stage invocation (for debugging) +- `scripts/synthesize_voice.py --text-file script.txt --out audio.mp3` +- `scripts/upload_asset.py audio.mp3` → prints `asset_id` +- `scripts/render_video.py --audio-asset-id <id> --title "..."` → prints `video_id` +- `scripts/poll_and_download.py --video-id <id> --out outputs/renders/slug.mp4` + +## Error handling +- **401 from HeyGen:** key is invalid or expired → re-read `heygen-key.txt` +- **402 from HeyGen:** subscription credit exhausted → tell user plainly, don't retry +- **422 "voice is premade":** cannot use a premade voice with voice cloning settings — downgrade `voice_settings.style` to 0.0 +- **HeyGen video status "failed":** fetch `error.message`, common causes: audio file too long (>10 min), avatar not trained for aspect ratio, audio file corrupt +- **Timeout after 10 min polling:** video is likely stuck — submit again with `test: false` and `dimension` set to a smaller size (480x854) + +## Files in this skill +- `SKILL.md` — this file +- `scripts/full_render.py` — one-shot orchestrator +- `scripts/synthesize_voice.py` — ElevenLabs TTS +- `scripts/upload_asset.py` — HeyGen asset upload (CLI wrapper) +- `scripts/render_video.py` — HeyGen v3 video create +- `scripts/poll_and_download.py` — status polling + MP4 download +- `scripts/refresh_registry.py` — rebuild registry.json from live HeyGen + ElevenLabs data +- `references/registry.json` — avatar/voice IDs and defaults +- `references/webhook_handler.py` — local Flask handler for Auto-Render button +- `references/elevenlabs-audio-tags.md` — bracket-syntax reference + +## Hand-off contract +- **Upstream:** `content-creation-engine` writes a v5.4 script to `outputs/scripts/<slug>.ssml.txt` +- **Downstream:** the v5.4 weekly calendar Auto-Render button POSTs `{"slug": "..."}` to the local webhook → this skill runs `full_render.py` → MP4 lands in `outputs/renders/` + +## Cost guardrails +- ElevenLabs Creator tier = ~100k chars/month. A 60-second vertical = ~900 chars. Budget roughly 100 renders/mo before hitting the cap. +- HeyGen credits are per-video-minute. Test renders should use `resolution: "720p"` to conserve credits; only bump to 1080p for final deliverables. + +## Verification step (required after every render) +1. Download the MP4 and confirm `file output.mp4` reports a valid MPEG-4 container +2. Probe duration with `ffprobe` — should be within ±0.5s of the MP3 source +3. Open the signed URL in a browser and scrub to verify lip sync (eyeball test) + +If any verification step fails, mark the render failed and re-queue rather than shipping a broken file. + +## Sandbox allowlist gap (important) +Finished HeyGen MP4s are served from `files2.heygen.ai` on a signed CloudFront URL. This CDN host is NOT currently on the Cowork sandbox allowlist. In-sandbox downloads via `poll_and_download.py` will fail with proxy HTTP 403. Two fixes: +1. Add `files2.heygen.ai` and `resource2.heygen.ai` to the Cowork desktop allowlist (recommended). +2. Otherwise, run `webhook_handler.py` on the host Windows machine (outside the sandbox) — its network is unrestricted and downloads work. + +Verified test render that succeeded: video_id `f79ed46032f74759a1153ff7e06e33f6`, duration 7.77s, SSML source with `<break>` tags honored. diff --git a/skills/heygen-elevenlabs-renderer/references/demo_calendar_with_button.html b/skills/heygen-elevenlabs-renderer/references/demo_calendar_with_button.html new file mode 100644 index 00000000..0061b262 --- /dev/null +++ b/skills/heygen-elevenlabs-renderer/references/demo_calendar_with_button.html @@ -0,0 +1,136 @@ +<\!DOCTYPE html> +<html lang="en"> +<head> +<meta charset="UTF-8"> +<title>v5.4 Auto-Render Demo + + + + +

v5.4 weekly calendar — Auto-Render Demo

+

This is the button pattern injected into every day card. One click runs the full pipeline on Graeham's local machine.

+ +
To test this locally: run python3 references/webhook_handler.py first, then open this file and click the button.
+ +<\!-- Sample Day Card --> +
+
+
MONDAY — Apr 13
+ BOFU +
+
AB 1482 Rent Cap Explainer — why your investor client should call now
+ +
+ Format: Instagram Reel · 60s · 9:16 · 1080p
+ Voice: Graeham Watts Voice Clone (ElevenLabs) → Graeham Watts avatar (HeyGen) +
+ +
<speak>Hey East Palo Alto homeowners <break time="0.4s"/> if you bought before 2021 <break time="0.3s"/> <prosody rate="slow">listen carefully.</prosody> <break time="0.5s"/> Your equity position is stronger than you think.</speak>
+ +
+ + + +
+
+ +
+
+
TUESDAY — Apr 14
+ TOFU +
+
East Palo Alto micro-neighborhoods tier list
+
<speak>I've sold in all four EPA micro-markets. <break time="0.3s"/> Here's my ranking, ordered by 2026 appreciation upside.</speak>
+
+ + + +
+
+ + + + diff --git a/skills/heygen-elevenlabs-renderer/references/elevenlabs-audio-tags.md b/skills/heygen-elevenlabs-renderer/references/elevenlabs-audio-tags.md new file mode 100644 index 00000000..18b935ea --- /dev/null +++ b/skills/heygen-elevenlabs-renderer/references/elevenlabs-audio-tags.md @@ -0,0 +1,31 @@ +# ElevenLabs Audio Tag Reference + +ElevenLabs does NOT support full W3C SSML. Use bracket-syntax audio tags for emotive delivery. + +## Supported bracket tags (eleven_multilingual_v2) + +| Tag | Effect | +|---|---| +| `[laughs]` | Short laugh insertion | +| `[chuckles]` | Softer laugh | +| `[sighs]` | Audible sigh | +| `[whispers]` | Switch to whispered delivery for following phrase | +| `[excited]` | Lift energy | +| `[sarcastic]` | Flatten tone | +| `[pause]` | ~0.4s silence | + +## SSML subset that IS supported + +| Tag | Effect | +|---|---| +| `...` | Root wrapper (optional, enables SSML mode) | +| `` | Exact silence duration (0.1s–3s) | + +## Not supported (silently stripped) + +- `` — tag accepted, text still read, but no rate change +- ``, ``, ``, ``, `` + +## v5.4 script compatibility + +v5.4 scripts use `` and `` for readability. Only `` delivers real audio changes. For true rate/pitch control, pre-split the script and synthesize each chunk with different `voice_settings.stability` values, then concatenate with ffmpeg. diff --git a/skills/heygen-elevenlabs-renderer/references/registry.json b/skills/heygen-elevenlabs-renderer/references/registry.json new file mode 100644 index 00000000..2f3c056f --- /dev/null +++ b/skills/heygen-elevenlabs-renderer/references/registry.json @@ -0,0 +1,797 @@ +{ + "generated_at": "2026-04-13T22:12:05.883331+00:00", + "version": "1.0.0", + "defaults": { + "heygen_avatar_id": "9a3600b16f604059b6ab8b9a55e29ea9", + "heygen_avatar_name": "Graeham Watts", + "elevenlabs_voice_id": "Pa3vOYQHHpLJn1Tf7hnP", + "elevenlabs_voice_name": "Graeham Watts Voice Clone", + "elevenlabs_model": "eleven_multilingual_v2", + "video_resolution": { + "width": 1080, + "height": 1920 + }, + "video_aspect": "9:16" + }, + "heygen": { + "personal_avatars_count": 70, + "personal_avatars_full": [ + { + "id": "9a3600b16f604059b6ab8b9a55e29ea9", + "name": "Graeham Watts", + "preview": "https://files2.heygen.ai/avatar/v3/9a3600b16f604059b6ab8b9a55e29ea9/full/2.2/preview_target.webp", + "type": "avatar", + "premium": false + }, + { + "id": "928f502843904b4cb82c116216ad92ff", + "name": "Graeham Watts", + "preview": "https://files2.heygen.ai/avatar/v3/928f502843904b4cb82c116216ad92ff/full/2.2/preview_target.webp", + "type": "avatar", + "premium": false + }, + { + "id": "fbc5f122c70d4b5e9864fef13e83c78e", + "name": "Graeham Watts", + "preview": "https://files2.heygen.ai/avatar/v3/fbc5f122c70d4b5e9864fef13e83c78e/full/2.2/preview_target.webp", + "type": "avatar", + "premium": false + }, + { + "id": "6b920eb820834257b32ce642dd9e6ce2", + "name": "Graeham Watts", + "preview": "https://files2.heygen.ai/avatar/v3/6b920eb820834257b32ce642dd9e6ce2/full/2.2/preview_target.webp", + "type": "avatar", + "premium": false + }, + { + "id": "9219a2b401aa42d29a3ea38cde6d314c", + "name": "Graeham Watts", + "preview": "https://files2.heygen.ai/avatar/v3/9219a2b401aa42d29a3ea38cde6d314c/full/2.2/preview_target.webp", + "type": "avatar", + "premium": false + }, + { + "id": "e2d77e0e0cb24e648832cfc63e916060", + "name": "Graeham Watts", + "preview": "https://resource2.heygen.ai/instant_avatar/avatar_iv_preview/e2d77e0e0cb24e648832cfc63e916060.webp", + "type": "avatar", + "premium": false + }, + { + "id": "2a84eac9ec51412fbc04ad74039cd840", + "name": "Graeham Watts", + "preview": "https://files2.heygen.ai/avatar/v3/2a84eac9ec51412fbc04ad74039cd840/full/2.2/preview_target.webp", + "type": "avatar", + "premium": false + }, + { + "id": "53e3489955054f88825c28a0514d401f", + "name": "Graeham Watts", + "preview": "https://files2.heygen.ai/avatar/v3/53e3489955054f88825c28a0514d401f/full/2.2/preview_target.webp", + "type": "avatar", + "premium": false + }, + { + "id": "1ea651dc1eff4f8fa0417dd047e30c75", + "name": "Graeham Watts", + "preview": "https://files2.heygen.ai/avatar/v3/1ea651dc1eff4f8fa0417dd047e30c75/full/2.2/preview_target.webp", + "type": "avatar", + "premium": false + }, + { + "id": "f6fac25075424f59b99ab6e81b0f9d80", + "name": "Graeham Watts", + "preview": "https://files2.heygen.ai/avatar/v3/f6fac25075424f59b99ab6e81b0f9d80/half/2.2/preview_target.webp", + "type": "avatar", + "premium": false + }, + { + "id": "73baa7f782be4fb6a242f4e8c87f203d", + "name": "Graeham Watts", + "preview": "https://files2.heygen.ai/avatar/v3/73baa7f782be4fb6a242f4e8c87f203d/half/2.2/preview_target.webp", + "type": "avatar", + "premium": false + }, + { + "id": "0db5e52223d8483cb37c84192f18efc4", + "name": "Graeham Watts", + "preview": "https://files2.heygen.ai/avatar/v3/0db5e52223d8483cb37c84192f18efc4/half/2.2/preview_target.webp", + "type": "avatar", + "premium": false + }, + { + "id": "db5d7c779ed540fe98a1bcff87ae6185", + "name": "Graeham Watts", + "preview": "https://files2.heygen.ai/avatar/v3/db5d7c779ed540fe98a1bcff87ae6185/half/2.2/preview_target.webp", + "type": "avatar", + "premium": false + }, + { + "id": "668847fcfea24d07b4d9dbade97d2d7e", + "name": "Graeham Watts", + "preview": "https://files2.heygen.ai/avatar/v3/668847fcfea24d07b4d9dbade97d2d7e/half/2.2/preview_target.webp", + "type": "avatar", + "premium": false + }, + { + "id": "b52fe48041144377b679be524a74fa8e", + "name": "Graeham Watts", + "preview": "https://files2.heygen.ai/avatar/v3/b52fe48041144377b679be524a74fa8e/half/2.2/preview_target.webp", + "type": "avatar", + "premium": false + }, + { + "id": "8135edb40dc84d50b07ced8c7fde51a6", + "name": "Graeham Watts", + "preview": "https://files2.heygen.ai/avatar/v3/8135edb40dc84d50b07ced8c7fde51a6/half/2.2/preview_target.webp", + "type": "avatar", + "premium": false + }, + { + "id": "d078208a437c44bb8ad2209d192882f3", + "name": "Graeham Watts", + "preview": "https://files2.heygen.ai/avatar/v3/d078208a437c44bb8ad2209d192882f3/half/2.2/preview_target.webp", + "type": "avatar", + "premium": false + }, + { + "id": "e9ec3ca7b4fa4571b05ec31ef7ba4eb6", + "name": "Graeham Watts", + "preview": "https://files2.heygen.ai/avatar/v3/e9ec3ca7b4fa4571b05ec31ef7ba4eb6/half/2.2/preview_target.webp", + "type": "avatar", + "premium": false + }, + { + "id": "cf3f5afc6a9f484f98013544dbd8c1c6", + "name": "Graeham Watts", + "preview": "https://files2.heygen.ai/avatar/v3/cf3f5afc6a9f484f98013544dbd8c1c6/half/2.2/preview_target.webp", + "type": "avatar", + "premium": false + }, + { + "id": "2045b214375c412f9ce91de1ee5268c0", + "name": "Graeham Watts", + "preview": "https://files2.heygen.ai/avatar/v3/2045b214375c412f9ce91de1ee5268c0/half/2.2/preview_target.webp", + "type": "avatar", + "premium": false + }, + { + "id": "068a91765ef74d67865a4ad973cfb129", + "name": "Graeham Watts", + "preview": "https://files2.heygen.ai/avatar/v3/068a91765ef74d67865a4ad973cfb129/half/2.2/preview_target.webp", + "type": "avatar", + "premium": false + }, + { + "id": "d33951bac2ae4891ace9158951312520", + "name": "Graeham Watts", + "preview": "https://files2.heygen.ai/avatar/v3/d33951bac2ae4891ace9158951312520/half/2.2/preview_target.webp", + "type": "avatar", + "premium": false + }, + { + "id": "a6c1a168a78e4592b49f80d9c85a0a68", + "name": "Graeham Watts", + "preview": "https://files2.heygen.ai/avatar/v3/a6c1a168a78e4592b49f80d9c85a0a68/half/2.2/preview_target.webp", + "type": "avatar", + "premium": false + }, + { + "id": "451e446f6ba84d60b386992f0b1d7ee8", + "name": "Graeham Watts", + "preview": "https://files2.heygen.ai/avatar/v3/451e446f6ba84d60b386992f0b1d7ee8/half/2.2/preview_target.webp", + "type": "avatar", + "premium": false + }, + { + "id": "1bbb5fd55f10441e841cea142966774f", + "name": "Graeham Watts", + "preview": "https://files2.heygen.ai/avatar/v3/1bbb5fd55f10441e841cea142966774f/half/2.2/preview_target.webp", + "type": "avatar", + "premium": false + }, + { + "id": "c9f800ab155e46a8bfca861edf55be39", + "name": "Graeham Watts", + "preview": "https://files2.heygen.ai/avatar/v3/c9f800ab155e46a8bfca861edf55be39/half/2.2/preview_target.webp", + "type": "avatar", + "premium": false + }, + { + "id": "3c81f584b8ee4eb69226c2be7e6cfbf9", + "name": "Graeham Watts", + "preview": "https://files2.heygen.ai/avatar/v3/3c81f584b8ee4eb69226c2be7e6cfbf9/half/2.2/preview_target.webp", + "type": "avatar", + "premium": false + }, + { + "id": "019f5cd757a5472b89f98873c7570fe2", + "name": "Graeham Watts", + "preview": "https://files2.heygen.ai/avatar/v3/019f5cd757a5472b89f98873c7570fe2/half/2.2/preview_target.webp", + "type": "avatar", + "premium": false + }, + { + "id": "774191aebdf54293acaf9ec87406b6ba", + "name": "Graeham Watts", + "preview": "https://resource2.heygen.ai/best_frame_selection/candidates/f01a928e0c49489a92a5f2f46eb78b5e.jpg", + "type": "avatar", + "premium": false + }, + { + "id": "d18923c8de2a4c70b138204be4877ca7", + "name": "Graeham Watts", + "preview": "https://resource2.heygen.ai/best_frame_selection/candidates/ddb5a72bedeb488ebe9532b1186e5a8a.jpg", + "type": "avatar", + "premium": false + }, + { + "id": "9706cd575e7f4e649a7d1bdaa78da85b", + "name": "Graeham Watts -- 113", + "preview": "https://resource2.heygen.ai/best_frame_selection/candidates/d22ab1999a9e43068d1acbd7d82607d0.jpg", + "type": "avatar", + "premium": false + }, + { + "id": "f85c5bf2f2fc4a3fae8b868c5985fad3", + "name": "Graeham Watts -- 138", + "preview": "https://resource2.heygen.ai/best_frame_selection/candidates/64df4d32b28841ccbd590af1bd01cd38.jpg", + "type": "avatar", + "premium": false + }, + { + "id": "49f2e94d9e964f95b453920d2e312f0d", + "name": "Graeham Watts -- 141", + "preview": "https://resource2.heygen.ai/best_frame_selection/candidates/61874753ef4f4c668fec180e5c7e6993.jpg", + "type": "avatar", + "premium": false + }, + { + "id": "159cd7b883724fdb9a51b97dec94df89", + "name": "Graeham Watts -- 142", + "preview": "https://resource2.heygen.ai/best_frame_selection/candidates/88101dc52de8403f864173abfec0a78f.jpg", + "type": "avatar", + "premium": false + }, + { + "id": "084ab8966ac1456682e069f4365acfca", + "name": "Graeham Watts -- 154", + "preview": "https://resource2.heygen.ai/best_frame_selection/candidates/6cd21c39350040d09ec33ca143c77c43.jpg", + "type": "avatar", + "premium": false + }, + { + "id": "9a3600b16f604059b6ab8b9a55e29ea9", + "name": "Graeham Watts", + "preview": "https://files2.heygen.ai/avatar/v3/9a3600b16f604059b6ab8b9a55e29ea9/full/2.2/preview_target.webp", + "type": "avatar", + "premium": false + }, + { + "id": "928f502843904b4cb82c116216ad92ff", + "name": "Graeham Watts", + "preview": "https://files2.heygen.ai/avatar/v3/928f502843904b4cb82c116216ad92ff/full/2.2/preview_target.webp", + "type": "avatar", + "premium": false + }, + { + "id": "fbc5f122c70d4b5e9864fef13e83c78e", + "name": "Graeham Watts", + "preview": "https://files2.heygen.ai/avatar/v3/fbc5f122c70d4b5e9864fef13e83c78e/full/2.2/preview_target.webp", + "type": "avatar", + "premium": false + }, + { + "id": "6b920eb820834257b32ce642dd9e6ce2", + "name": "Graeham Watts", + "preview": "https://files2.heygen.ai/avatar/v3/6b920eb820834257b32ce642dd9e6ce2/full/2.2/preview_target.webp", + "type": "avatar", + "premium": false + }, + { + "id": "9219a2b401aa42d29a3ea38cde6d314c", + "name": "Graeham Watts", + "preview": "https://files2.heygen.ai/avatar/v3/9219a2b401aa42d29a3ea38cde6d314c/full/2.2/preview_target.webp", + "type": "avatar", + "premium": false + }, + { + "id": "e2d77e0e0cb24e648832cfc63e916060", + "name": "Graeham Watts", + "preview": "https://resource2.heygen.ai/instant_avatar/avatar_iv_preview/e2d77e0e0cb24e648832cfc63e916060.webp", + "type": "avatar", + "premium": false + }, + { + "id": "2a84eac9ec51412fbc04ad74039cd840", + "name": "Graeham Watts", + "preview": "https://files2.heygen.ai/avatar/v3/2a84eac9ec51412fbc04ad74039cd840/full/2.2/preview_target.webp", + "type": "avatar", + "premium": false + }, + { + "id": "53e3489955054f88825c28a0514d401f", + "name": "Graeham Watts", + "preview": "https://files2.heygen.ai/avatar/v3/53e3489955054f88825c28a0514d401f/full/2.2/preview_target.webp", + "type": "avatar", + "premium": false + }, + { + "id": "1ea651dc1eff4f8fa0417dd047e30c75", + "name": "Graeham Watts", + "preview": "https://files2.heygen.ai/avatar/v3/1ea651dc1eff4f8fa0417dd047e30c75/full/2.2/preview_target.webp", + "type": "avatar", + "premium": false + }, + { + "id": "f6fac25075424f59b99ab6e81b0f9d80", + "name": "Graeham Watts", + "preview": "https://files2.heygen.ai/avatar/v3/f6fac25075424f59b99ab6e81b0f9d80/half/2.2/preview_target.webp", + "type": "avatar", + "premium": false + }, + { + "id": "73baa7f782be4fb6a242f4e8c87f203d", + "name": "Graeham Watts", + "preview": "https://files2.heygen.ai/avatar/v3/73baa7f782be4fb6a242f4e8c87f203d/half/2.2/preview_target.webp", + "type": "avatar", + "premium": false + }, + { + "id": "0db5e52223d8483cb37c84192f18efc4", + "name": "Graeham Watts", + "preview": "https://files2.heygen.ai/avatar/v3/0db5e52223d8483cb37c84192f18efc4/half/2.2/preview_target.webp", + "type": "avatar", + "premium": false + }, + { + "id": "db5d7c779ed540fe98a1bcff87ae6185", + "name": "Graeham Watts", + "preview": "https://files2.heygen.ai/avatar/v3/db5d7c779ed540fe98a1bcff87ae6185/half/2.2/preview_target.webp", + "type": "avatar", + "premium": false + }, + { + "id": "668847fcfea24d07b4d9dbade97d2d7e", + "name": "Graeham Watts", + "preview": "https://files2.heygen.ai/avatar/v3/668847fcfea24d07b4d9dbade97d2d7e/half/2.2/preview_target.webp", + "type": "avatar", + "premium": false + }, + { + "id": "b52fe48041144377b679be524a74fa8e", + "name": "Graeham Watts", + "preview": "https://files2.heygen.ai/avatar/v3/b52fe48041144377b679be524a74fa8e/half/2.2/preview_target.webp", + "type": "avatar", + "premium": false + }, + { + "id": "8135edb40dc84d50b07ced8c7fde51a6", + "name": "Graeham Watts", + "preview": "https://files2.heygen.ai/avatar/v3/8135edb40dc84d50b07ced8c7fde51a6/half/2.2/preview_target.webp", + "type": "avatar", + "premium": false + }, + { + "id": "d078208a437c44bb8ad2209d192882f3", + "name": "Graeham Watts", + "preview": "https://files2.heygen.ai/avatar/v3/d078208a437c44bb8ad2209d192882f3/half/2.2/preview_target.webp", + "type": "avatar", + "premium": false + }, + { + "id": "e9ec3ca7b4fa4571b05ec31ef7ba4eb6", + "name": "Graeham Watts", + "preview": "https://files2.heygen.ai/avatar/v3/e9ec3ca7b4fa4571b05ec31ef7ba4eb6/half/2.2/preview_target.webp", + "type": "avatar", + "premium": false + }, + { + "id": "cf3f5afc6a9f484f98013544dbd8c1c6", + "name": "Graeham Watts", + "preview": "https://files2.heygen.ai/avatar/v3/cf3f5afc6a9f484f98013544dbd8c1c6/half/2.2/preview_target.webp", + "type": "avatar", + "premium": false + }, + { + "id": "2045b214375c412f9ce91de1ee5268c0", + "name": "Graeham Watts", + "preview": "https://files2.heygen.ai/avatar/v3/2045b214375c412f9ce91de1ee5268c0/half/2.2/preview_target.webp", + "type": "avatar", + "premium": false + }, + { + "id": "068a91765ef74d67865a4ad973cfb129", + "name": "Graeham Watts", + "preview": "https://files2.heygen.ai/avatar/v3/068a91765ef74d67865a4ad973cfb129/half/2.2/preview_target.webp", + "type": "avatar", + "premium": false + }, + { + "id": "d33951bac2ae4891ace9158951312520", + "name": "Graeham Watts", + "preview": "https://files2.heygen.ai/avatar/v3/d33951bac2ae4891ace9158951312520/half/2.2/preview_target.webp", + "type": "avatar", + "premium": false + }, + { + "id": "a6c1a168a78e4592b49f80d9c85a0a68", + "name": "Graeham Watts", + "preview": "https://files2.heygen.ai/avatar/v3/a6c1a168a78e4592b49f80d9c85a0a68/half/2.2/preview_target.webp", + "type": "avatar", + "premium": false + }, + { + "id": "451e446f6ba84d60b386992f0b1d7ee8", + "name": "Graeham Watts", + "preview": "https://files2.heygen.ai/avatar/v3/451e446f6ba84d60b386992f0b1d7ee8/half/2.2/preview_target.webp", + "type": "avatar", + "premium": false + }, + { + "id": "1bbb5fd55f10441e841cea142966774f", + "name": "Graeham Watts", + "preview": "https://files2.heygen.ai/avatar/v3/1bbb5fd55f10441e841cea142966774f/half/2.2/preview_target.webp", + "type": "avatar", + "premium": false + }, + { + "id": "c9f800ab155e46a8bfca861edf55be39", + "name": "Graeham Watts", + "preview": "https://files2.heygen.ai/avatar/v3/c9f800ab155e46a8bfca861edf55be39/half/2.2/preview_target.webp", + "type": "avatar", + "premium": false + }, + { + "id": "3c81f584b8ee4eb69226c2be7e6cfbf9", + "name": "Graeham Watts", + "preview": "https://files2.heygen.ai/avatar/v3/3c81f584b8ee4eb69226c2be7e6cfbf9/half/2.2/preview_target.webp", + "type": "avatar", + "premium": false + }, + { + "id": "019f5cd757a5472b89f98873c7570fe2", + "name": "Graeham Watts", + "preview": "https://files2.heygen.ai/avatar/v3/019f5cd757a5472b89f98873c7570fe2/half/2.2/preview_target.webp", + "type": "avatar", + "premium": false + }, + { + "id": "774191aebdf54293acaf9ec87406b6ba", + "name": "Graeham Watts", + "preview": "https://resource2.heygen.ai/best_frame_selection/candidates/f01a928e0c49489a92a5f2f46eb78b5e.jpg", + "type": "avatar", + "premium": false + }, + { + "id": "d18923c8de2a4c70b138204be4877ca7", + "name": "Graeham Watts", + "preview": "https://resource2.heygen.ai/best_frame_selection/candidates/ddb5a72bedeb488ebe9532b1186e5a8a.jpg", + "type": "avatar", + "premium": false + }, + { + "id": "9706cd575e7f4e649a7d1bdaa78da85b", + "name": "Graeham Watts -- 113", + "preview": "https://resource2.heygen.ai/best_frame_selection/candidates/d22ab1999a9e43068d1acbd7d82607d0.jpg", + "type": "avatar", + "premium": false + }, + { + "id": "f85c5bf2f2fc4a3fae8b868c5985fad3", + "name": "Graeham Watts -- 138", + "preview": "https://resource2.heygen.ai/best_frame_selection/candidates/64df4d32b28841ccbd590af1bd01cd38.jpg", + "type": "avatar", + "premium": false + }, + { + "id": "49f2e94d9e964f95b453920d2e312f0d", + "name": "Graeham Watts -- 141", + "preview": "https://resource2.heygen.ai/best_frame_selection/candidates/61874753ef4f4c668fec180e5c7e6993.jpg", + "type": "avatar", + "premium": false + }, + { + "id": "159cd7b883724fdb9a51b97dec94df89", + "name": "Graeham Watts -- 142", + "preview": "https://resource2.heygen.ai/best_frame_selection/candidates/88101dc52de8403f864173abfec0a78f.jpg", + "type": "avatar", + "premium": false + }, + { + "id": "084ab8966ac1456682e069f4365acfca", + "name": "Graeham Watts -- 154", + "preview": "https://resource2.heygen.ai/best_frame_selection/candidates/6cd21c39350040d09ec33ca143c77c43.jpg", + "type": "avatar", + "premium": false + } + ], + "talking_photos_count": 3, + "talking_photos_sample": [ + { + "id": "1819420dda6d4368b068532121c84a85", + "name": "Graeham Watts", + "preview": "https://files2.heygen.ai/talking_photo/1819420dda6d4368b068532121c84a85/1385189ab17c4364872001c45530fa59.WEBP?Expires=1776722861&Signature=lE8XL74PbLJKChmXF6qNo0bHyT-jLr5wvPJdPXBItn2lSUNqpXyz66MTNHE-Ao~VDhBAx-H2jnA5wDc84IY3oxpHriH86zoSOcgSLpVqsd3j8OwhAs~aMGHvwb~QVZogtj0RS7fHxJWPF-JfvBGkECOqwqHb0z-uRSjxbfzx4~cwK1pn2oVIghQOpSGH-thdB0JjgoAO9gjpIgnFMKhy9Xb~Te6zGIprT3KfCglz1rC~IimL4OmtUF-YLcmo9l6C~LZOrVFaD6FSVvdvNY-fOWVdmtVs5ER8gcFgHBI7g0UJKn3ZD943jbBlzRWi7IReyk5eQwCiv2kdhv~3osB~kQ__&Key-Pair-Id=K38HBHX5LX3X2H", + "type": "talking_photo" + }, + { + "id": "6b10dd96006e4d94af58f1edb31892c5", + "name": "Graeham Watts", + "preview": "https://files2.heygen.ai/talking_photo/6b10dd96006e4d94af58f1edb31892c5/4a422bf4060548ff8c7989a84feaae45.webp?Expires=1776722862&Signature=acfFTp6s4Vkb1WR4hSMUgp7KwhqiSE~4qNvRQZkfadlFFBgDxqR~U6JdLkdip7OsgWbBAjZuyVjpYuzKbMLDPGqVSI1H751iWPADMmuXJJi75gkv~DPtNH8cG4lacogqM4bff25iDV3SBBBJYTA1VldM~6SRX2DdZYGAbJTUXkYlNjgtkFuC4ZzqNCpERt1EGNvTZMAXENBHFV3bhsmxqtNLTXwaK5CUFmLDv7K7GrM0Bh0J-uGBMA6dnnO4b4JwRlRzbhB9CUUpcM9Aks5aLxsIJ~mqBBIVKr1f2bTM1El0VaAAOzo6SV9K2pZndcETazDuGqdRXiF-VDewJNfeug__&Key-Pair-Id=K38HBHX5LX3X2H", + "type": "talking_photo" + }, + { + "id": "7d07ccc7b3dd4e0c89e15c7eefa620e2", + "name": "Graeham Watts", + "preview": "https://files2.heygen.ai/talking_photo/7d07ccc7b3dd4e0c89e15c7eefa620e2/b30a31214cbc4bac9d467d00209bf8a6.webp?Expires=1776722862&Signature=omC1qeD4mBoIsW4~Fmwss6FPJTYZxKAeOZPNl0bMP86D4cxa0JGlwXNWlCU1ukNWZaVJB2-Fr1GKPRj67k0pnN14BfJmprLGuZtP8kOSnrJrabS0bVOKeOGl0XzGI~ImbVxWV--tvO9590JqLtQ3TxamuXkS4HtDSXVAPbgRx~VH2QYx~Hn8zBec-rz1cpMBFoGoRgyM0IFhb0bCAfjWoXZuXGQwbWlrG55tmId6-77tN6z7zCL~3Li6wxfy~ILIgmMLTjZsaVVfAg9VeSkw0AOMaWYBE83QLg0kJoqLZVLdpra0YGETicVoIk~l3mpjIZcSPdLNFWhh18giNi5MUA__&Key-Pair-Id=K38HBHX5LX3X2H", + "type": "talking_photo" + } + ], + "library_voices_count": 2387 + }, + "elevenlabs": { + "graeham_voice_clone": { + "voice_id": "Pa3vOYQHHpLJn1Tf7hnP", + "name": "Graeham Watts Voice Clone", + "category": "professional", + "labels": { + "gender": "male", + "accent": "en-american", + "age": "middle-aged", + "language": "en" + } + }, + "total_voices": 29, + "by_use_case": { + "conversational": [ + { + "voice_id": "CwhRBWXzGAHq8TQ4Fs17", + "name": "Roger - Laid-Back, Casual, Resonant", + "category": "premade", + "gender": "male", + "accent": "american", + "descriptive": "classy" + }, + { + "voice_id": "IKne3meq5aSn9XLyUdCD", + "name": "Charlie - Deep, Confident, Energetic", + "category": "premade", + "gender": "male", + "accent": "australian", + "descriptive": "hyped" + }, + { + "voice_id": "SAz9YHcvj6GT2YYXdXww", + "name": "River - Relaxed, Neutral, Informative", + "category": "premade", + "gender": "neutral", + "accent": "american", + "descriptive": "calm" + }, + { + "voice_id": "bIHbv24MWmeRgasZH58o", + "name": "Will - Relaxed Optimist", + "category": "premade", + "gender": "male", + "accent": "american", + "descriptive": "chill" + }, + { + "voice_id": "cgSgspJ2msm6clMCkdW9", + "name": "Jessica - Playful, Bright, Warm", + "category": "premade", + "gender": "female", + "accent": "american", + "descriptive": "cute" + }, + { + "voice_id": "cjVigY5qzO86Huf0OWal", + "name": "Eric - Smooth, Trustworthy", + "category": "premade", + "gender": "male", + "accent": "american", + "descriptive": "classy" + }, + { + "voice_id": "iP95p4xoKVk53GoZ742B", + "name": "Chris - Charming, Down-to-Earth", + "category": "premade", + "gender": "male", + "accent": "american", + "descriptive": "casual" + } + ], + "entertainment_tv": [ + { + "voice_id": "EXAVITQu4vr4xnSDxMaL", + "name": "Sarah - Mature, Reassuring, Confident", + "category": "premade", + "gender": "female", + "accent": "american", + "descriptive": "professional" + } + ], + "social_media": [ + { + "voice_id": "FGY2WhTYpPnrIDTdsKH5", + "name": "Laura - Enthusiast, Quirky Attitude", + "category": "premade", + "gender": "female", + "accent": "american", + "descriptive": "sassy" + }, + { + "voice_id": "TX3LPaxmHKxFdv7VOQHJ", + "name": "Liam - Energetic, Social Media Creator", + "category": "premade", + "gender": "male", + "accent": "american", + "descriptive": "confident" + }, + { + "voice_id": "nPczCjzI2devNBz1zQrb", + "name": "Brian - Deep, Resonant and Comforting", + "category": "premade", + "gender": "male", + "accent": "american", + "descriptive": "classy" + }, + { + "voice_id": "pNInz6obpgDQGcFmaJgB", + "name": "Adam - Dominant, Firm", + "category": "premade", + "gender": "male", + "accent": "american", + "descriptive": null + } + ], + "narrative_story": [ + { + "voice_id": "JBFqnCBsd6RMkjVDRZzb", + "name": "George - Warm, Captivating Storyteller", + "category": "premade", + "gender": "male", + "accent": "british", + "descriptive": "mature" + }, + { + "voice_id": "f5KRUAmxOzuhrrp8V3zv", + "name": "Titan \u2013 Young, Dramatic, Epic Storyteller/Movie Narrator", + "category": "professional", + "gender": "male", + "accent": "american", + "descriptive": "intense" + }, + { + "voice_id": "Dslrhjl3ZpzrctukrQSN", + "name": "Hey Its Brad - Clear Narrator for Documentary", + "category": "professional", + "gender": "male", + "accent": "american", + "descriptive": "casual" + } + ], + "characters_animation": [ + { + "voice_id": "N2lVS1w4EtoT3dr4eOWO", + "name": "Callum - Husky Trickster", + "category": "premade", + "gender": "male", + "accent": "american", + "descriptive": null + }, + { + "voice_id": "SOYHLrjzK2X1ezoPC6cr", + "name": "Harry - Fierce Warrior", + "category": "premade", + "gender": "male", + "accent": "american", + "descriptive": "rough" + }, + { + "voice_id": "weA4Q36twV5kwSaTEL0Q", + "name": "Ava - Female Robot or AI Assistant", + "category": "professional", + "gender": "female", + "accent": "american", + "descriptive": "robotic" + }, + { + "voice_id": "bwCXcoVxWNYMlC6Esa8u", + "name": "Matthew Schmitz - Anti-Hero, Villain, Rogue, Tough Guy", + "category": "professional", + "gender": "male", + "accent": "american", + "descriptive": "intense" + } + ], + "informative_educational": [ + { + "voice_id": "Xb7hH8MSUJpSbSDYk0k2", + "name": "Alice - Clear, Engaging Educator", + "category": "premade", + "gender": "female", + "accent": "british", + "descriptive": "professional" + }, + { + "voice_id": "XrExE9yKIg1WjnnlVkGX", + "name": "Matilda - Knowledgable, Professional", + "category": "premade", + "gender": "female", + "accent": "american", + "descriptive": "upbeat" + }, + { + "voice_id": "hpp4J3VqNfWAUOO0d1Us", + "name": "Bella - Professional, Bright, Warm", + "category": "premade", + "gender": "female", + "accent": "american", + "descriptive": "professional" + }, + { + "voice_id": "onwK4e9ZLuTAKqWW03F9", + "name": "Daniel - Steady Broadcaster", + "category": "premade", + "gender": "male", + "accent": "british", + "descriptive": "formal" + }, + { + "voice_id": "pFZP5JQG7iQjIQuC4Bku", + "name": "Lily - Velvety Actress", + "category": "premade", + "gender": "female", + "accent": "british", + "descriptive": "confident" + }, + { + "voice_id": "hfgNmTYYctMgJ7E2s6Vx", + "name": "Shaun - The Ultimate Narrator Voice", + "category": "professional", + "gender": "male", + "accent": "american", + "descriptive": "deep" + } + ], + "advertisement": [ + { + "voice_id": "pqHfZKP75CvOlQylNhV4", + "name": "Bill - Wise, Mature, Balanced", + "category": "premade", + "gender": "male", + "accent": "american", + "descriptive": "crisp" + } + ], + "other": [ + { + "voice_id": "X8NfXnmtiqkCTSR8Gcu6", + "name": "Funny Dracula", + "category": "generated", + "gender": null, + "accent": null, + "descriptive": null + }, + { + "voice_id": "K6Id9eFOS8sKqAqS5SeY", + "name": "Broadcast News Brian - TV Anchor", + "category": "professional", + "gender": "male", + "accent": "en-american", + "descriptive": null + }, + { + "voice_id": "Pa3vOYQHHpLJn1Tf7hnP", + "name": "Graeham Watts Voice Clone", + "category": "professional", + "gender": "male", + "accent": "en-american", + "descriptive": null + } + ] + } + } +} \ No newline at end of file diff --git a/skills/heygen-elevenlabs-renderer/references/v54_auto_render_button.html b/skills/heygen-elevenlabs-renderer/references/v54_auto_render_button.html new file mode 100644 index 00000000..2d46fc9a --- /dev/null +++ b/skills/heygen-elevenlabs-renderer/references/v54_auto_render_button.html @@ -0,0 +1,163 @@ +<\!-- +v5.4 weekly calendar — Full Auto-Render Button Snippet (v6.2) +Placed under the ElevenLabs SSML block in every core asset derivative panel. +After render completes, surfaces direct links to: + - the MP4 on local disk + - the HeyGen video page (https://app.heygen.com/videos/) + - the ElevenLabs generation history & voice library + +v6.2 change: webhook_handler.py now returns a clean `dashboards` object — +no more regex-scraping stdout. Button also pre-populates dashboards before +render completes so the user always has a path to check progress manually. +--> + +<\!-- 1. BUTTON + STATUS + LINKS BLOCK --> +
+ + + +
+ +<\!-- 2. STYLES --> + + +<\!-- 3. JS (add once, globally, at bottom of ) --> + + +<\!-- 4. BANNER (place ONCE at top of Production Map tab) --> +
Checking Auto-Render status…
+ diff --git a/skills/heygen-elevenlabs-renderer/references/webhook_handler.py b/skills/heygen-elevenlabs-renderer/references/webhook_handler.py new file mode 100644 index 00000000..eafed067 --- /dev/null +++ b/skills/heygen-elevenlabs-renderer/references/webhook_handler.py @@ -0,0 +1,110 @@ +#\!/usr/bin/env python3 +""" +Local webhook handler for the v5.4 weekly calendar "🚀 Full Auto-Render" button. + +Runs a Flask server on http://127.0.0.1:7788/render that accepts: + POST /render + {"slug": "ab1482-explainer", "script_path": "/abs/path/to/script.ssml.txt"} + +It spawns full_render.py in the background and immediately returns +{"queued": true, "job_id": ...}. Poll GET /status/ to check progress. + +The /status response now always includes a `dashboards` block with direct +links to HeyGen + ElevenLabs so the calendar UI can surface them the moment +a render finishes — no regex scraping required. + +Start it: + python3 webhook_handler.py + +Bind to your desktop only — never expose publicly (no auth on this endpoint). +""" +import json +import re +import subprocess +import threading +import uuid +from pathlib import Path + +try: + from flask import Flask, jsonify, request +except ImportError: + raise SystemExit("pip install flask --break-system-packages") + +SCRIPT_DIR = Path(__file__).parent.parent / "scripts" +JOBS = {} + +# Always surface these so the button shows them even before a job completes. +STATIC_DASHBOARDS = { + "heygen_projects": "https://app.heygen.com/projects", + "elevenlabs_history": "https://elevenlabs.io/app/speech-synthesis/history", + "elevenlabs_voice_library": "https://elevenlabs.io/app/voice-library", +} + +app = Flask(__name__) + + +def parse_render_result(stdout: str) -> dict: + """full_render.py / poll_and_download.py prints `RENDER_RESULT={...}`.""" + m = re.search(r"RENDER_RESULT=(\{.*\})", stdout or "") + if not m: + return {} + try: + return json.loads(m.group(1)) + except json.JSONDecodeError: + return {} + + +def run_job(job_id, slug, script_path): + JOBS[job_id]["status"] = "running" + try: + proc = subprocess.run( + ["python3", str(SCRIPT_DIR / "full_render.py"), + "--script", script_path, "--slug", slug], + capture_output=True, text=True, timeout=900, + ) + JOBS[job_id]["status"] = "done" if proc.returncode == 0 else "failed" + JOBS[job_id]["stdout"] = proc.stdout + JOBS[job_id]["stderr"] = proc.stderr + + result = parse_render_result(proc.stdout) + if result: + JOBS[job_id]["result"] = result + JOBS[job_id]["dashboards"] = { + **STATIC_DASHBOARDS, + "heygen_video_page": result.get("heygen_dashboard_url"), + "local_mp4": result.get("out"), + "meta_json": result.get("meta"), + } + except Exception as e: + JOBS[job_id]["status"] = "failed" + JOBS[job_id]["error"] = str(e) + + +@app.post("/render") +def render(): + payload = request.get_json(force=True) + slug = payload["slug"] + script_path = payload["script_path"] + job_id = str(uuid.uuid4()) + JOBS[job_id] = { + "status": "queued", + "slug": slug, + "dashboards": STATIC_DASHBOARDS, # visible immediately + } + threading.Thread(target=run_job, args=(job_id, slug, script_path), daemon=True).start() + return jsonify({"queued": True, "job_id": job_id, "dashboards": STATIC_DASHBOARDS}) + + +@app.get("/status/") +def status(job_id): + return jsonify(JOBS.get(job_id, {"error": "unknown job"})) + + +@app.get("/health") +def health(): + return jsonify({"ok": True, "dashboards": STATIC_DASHBOARDS}) + + +if __name__ == "__main__": + # 127.0.0.1 = desktop-only. Do not change to 0.0.0.0 without auth. + app.run(host="127.0.0.1", port=7788) diff --git a/skills/heygen-elevenlabs-renderer/scripts/full_render.py b/skills/heygen-elevenlabs-renderer/scripts/full_render.py new file mode 100644 index 00000000..5a48f60d --- /dev/null +++ b/skills/heygen-elevenlabs-renderer/scripts/full_render.py @@ -0,0 +1,61 @@ +#\!/usr/bin/env python3 +""" +One-shot orchestrator: script → MP3 → asset → video → downloaded MP4. + +Usage: + python3 full_render.py --script path/to/script.ssml.txt --slug "my-video" +""" +import argparse +import subprocess +import sys +import time +from pathlib import Path + +SCRIPT_DIR = Path(__file__).parent + +def run(cmd, **kwargs): + print(f"+ {' '.join(cmd)}", flush=True) + return subprocess.run(cmd, check=True, text=True, capture_output=True, **kwargs) + +def main(): + p = argparse.ArgumentParser() + p.add_argument("--script", required=True, help="Path to v5.4 SSML script file") + p.add_argument("--slug", required=True) + p.add_argument("--aspect", default="9:16") + p.add_argument("--resolution", default="720p") + p.add_argument("--output-root", default=os.path.expanduser("~/Documents/Claude/Skills/.heygen-renders")) + args = p.parse_args() + + out_root = Path(args.output_root) + out_root.mkdir(parents=True, exist_ok=True) + mp3_path = out_root / f"{args.slug}.mp3" + mp4_path = out_root / f"{args.slug}.mp4" + + # 1. TTS + run(["python3", str(SCRIPT_DIR / "synthesize_voice.py"), + "--text-file", args.script, "--out", str(mp3_path)]) + + # 2. Upload + import json as _json + r = run(["python3", str(SCRIPT_DIR / "upload_asset.py"), str(mp3_path)]) + asset = _json.loads(r.stdout.strip().splitlines()[-1]) + asset_id = asset["asset_id"] + print(f"asset_id={asset_id}") + + # 3. Create video + r = run(["python3", str(SCRIPT_DIR / "render_video.py"), + "--audio-asset-id", asset_id, + "--title", args.slug, + "--aspect", args.aspect, + "--resolution", args.resolution]) + vid = _json.loads(r.stdout.strip().splitlines()[-1])["video_id"] + print(f"video_id={vid}") + + # 4. Poll + download + subprocess.run(["python3", str(SCRIPT_DIR / "poll_and_download.py"), + "--video-id", vid, "--out", str(mp4_path)], + check=True) + print(f"DONE: {mp4_path}") + +if __name__ == "__main__": + main() diff --git a/skills/heygen-elevenlabs-renderer/scripts/poll_and_download.py b/skills/heygen-elevenlabs-renderer/scripts/poll_and_download.py new file mode 100644 index 00000000..305b8964 --- /dev/null +++ b/skills/heygen-elevenlabs-renderer/scripts/poll_and_download.py @@ -0,0 +1,113 @@ +#\!/usr/bin/env python3 +""" +Poll HeyGen /v1/video_status.get until status == completed, then download MP4. + +Emits meta.json AND a single-line JSON to stdout with dashboard URLs so the +v5.4 calendar button can surface "where to find it" links (HeyGen video page, +ElevenLabs history page, local MP4) without regex-scraping stdout. + +Usage: + python3 poll_and_download.py --video-id --out outputs/renders/slug.mp4 +""" +import argparse +import json +import os +import sys +import time +import urllib.request +from pathlib import Path + +CRED_DIR = Path(os.environ.get( + "CLAUDE_CREDENTIALS_DIR", + os.path.expanduser("~/Documents/Claude/Skills/.heygen-credentials") +)) + +HEYGEN_DASHBOARD = "https://app.heygen.com/videos/{video_id}" +ELEVEN_HISTORY = "https://elevenlabs.io/app/speech-synthesis/history" +ELEVEN_VOICE_LIB = "https://elevenlabs.io/app/voice-library" + +def load_key(): + return (CRED_DIR / "heygen-key.txt").read_text().strip() + +def status(video_id): + req = urllib.request.Request( + f"https://api.heygen.com/v1/video_status.get?video_id={video_id}", + headers={"X-Api-Key": load_key()}, + ) + with urllib.request.urlopen(req, timeout=30) as r: + return json.loads(r.read()) + +def download(url, out_path): + Path(out_path).parent.mkdir(parents=True, exist_ok=True) + with urllib.request.urlopen(url, timeout=180) as r, open(out_path, "wb") as f: + while True: + chunk = r.read(65536) + if not chunk: + break + f.write(chunk) + +def main(): + p = argparse.ArgumentParser() + p.add_argument("--video-id", required=True) + p.add_argument("--out", required=True) + p.add_argument("--interval", type=int, default=15) + p.add_argument("--max-wait", type=int, default=600) + p.add_argument("--elevenlabs-voice-id", default=None, + help="Pin the voice_id used so the meta surfaces its history link") + args = p.parse_args() + + elapsed = 0 + s = None + while elapsed < args.max_wait: + resp = status(args.video_id) + data = resp.get("data", {}) + s = data.get("status") + print(f"[{elapsed}s] status={s}", flush=True) + if s == "completed": + url = data.get("video_url") + if not url: + sys.exit("completed but no video_url returned") + download(url, args.out) + + heygen_dashboard_url = HEYGEN_DASHBOARD.format(video_id=args.video_id) + meta = { + "video_id": args.video_id, + "video_url": url, + "thumbnail_url": data.get("thumbnail_url"), + "duration": data.get("duration"), + "completed_at": int(time.time()), + "local_mp4": str(Path(args.out).resolve()), + "dashboards": { + "heygen_video_page": heygen_dashboard_url, + "heygen_projects": "https://app.heygen.com/projects", + "elevenlabs_history": ELEVEN_HISTORY, + "elevenlabs_voice_library": ELEVEN_VOICE_LIB, + }, + } + if args.elevenlabs_voice_id: + meta["dashboards"]["elevenlabs_voice"] = ( + f"https://elevenlabs.io/app/voice-lab/share/{args.elevenlabs_voice_id}" + ) + + meta_path = Path(args.out).with_suffix(".meta.json") + meta_path.write_text(json.dumps(meta, indent=2)) + + # Single-line JSON the v5.4 button's JS can JSON.parse directly. + print("RENDER_RESULT=" + json.dumps({ + "status": "completed", + "video_id": args.video_id, + "out": str(Path(args.out).resolve()), + "meta": str(meta_path.resolve()), + "heygen_dashboard_url": heygen_dashboard_url, + "elevenlabs_history_url": ELEVEN_HISTORY, + })) + return + if s == "failed": + sys.exit(f"render failed: {data.get('error')}") + time.sleep(args.interval) + elapsed += args.interval + + sys.exit(f"timeout after {args.max_wait}s — video still {s}") + +if __name__ == "__main__": + main() diff --git a/skills/heygen-elevenlabs-renderer/scripts/refresh_registry.py b/skills/heygen-elevenlabs-renderer/scripts/refresh_registry.py new file mode 100644 index 00000000..d3a448bc --- /dev/null +++ b/skills/heygen-elevenlabs-renderer/scripts/refresh_registry.py @@ -0,0 +1,75 @@ +#\!/usr/bin/env python3 +""" +Rebuild references/registry.json by pulling live avatar + voice lists from +HeyGen (v2) and ElevenLabs. Run this anytime a new avatar or voice is trained. +""" +import json +import os +import urllib.request +from datetime import datetime, timezone +from pathlib import Path + +CRED_DIR = Path(os.environ.get( + "CLAUDE_CREDENTIALS_DIR", + os.path.expanduser("~/Documents/Claude/Skills/.heygen-credentials") +)) +OUT = Path(__file__).parent.parent / "references" / "registry.json" + +def get_json(url, headers): + req = urllib.request.Request(url, headers=headers) + with urllib.request.urlopen(req, timeout=60) as r: + return json.loads(r.read()) + +def main(): + hg = (CRED_DIR / "heygen-key.txt").read_text().strip() + el = (CRED_DIR / "elevenlabs-key.txt").read_text().strip() + + hg_av = get_json("https://api.heygen.com/v2/avatars", {"X-Api-Key": hg}) + hg_vo = get_json("https://api.heygen.com/v2/voices", {"X-Api-Key": hg}) + el_vo = get_json("https://api.elevenlabs.io/v1/voices", {"xi-api-key": el}) + + avatars = hg_av.get("data", {}).get("avatars", []) + talking = hg_av.get("data", {}).get("talking_photos", []) + personal = [a for a in avatars if "graeham" in (a.get("avatar_name") or "").lower()] + + clone = None + for v in el_vo.get("voices", []): + if "graeham" in (v.get("name") or "").lower(): + clone = {"voice_id": v["voice_id"], "name": v["name"], + "category": v.get("category"), "labels": v.get("labels", {})} + break + + reg = { + "generated_at": datetime.now(timezone.utc).isoformat(), + "version": "1.0.0", + "defaults": { + "heygen_avatar_id": personal[0]["avatar_id"] if personal else None, + "heygen_avatar_name": personal[0]["avatar_name"] if personal else None, + "elevenlabs_voice_id": clone["voice_id"] if clone else None, + "elevenlabs_voice_name": clone["name"] if clone else None, + "elevenlabs_model": "eleven_multilingual_v2", + "video_resolution": {"width": 1080, "height": 1920}, + "video_aspect": "9:16", + }, + "heygen": { + "personal_avatars_count": len(personal), + "personal_avatars_full": [ + {"id": a["avatar_id"], "name": a.get("avatar_name"), + "preview": a.get("preview_image_url"), + "premium": a.get("premium", False)} for a in personal + ], + }, + "elevenlabs": { + "graeham_voice_clone": clone, + "total_voices": len(el_vo.get("voices", [])), + }, + } + + OUT.parent.mkdir(parents=True, exist_ok=True) + OUT.write_text(json.dumps(reg, indent=2)) + print(f"Wrote {OUT}") + print(f" Graeham avatars: {len(personal)}") + print(f" Voice clone: {clone['name'] if clone else 'MISSING'}") + +if __name__ == "__main__": + main() diff --git a/skills/heygen-elevenlabs-renderer/scripts/render_video.py b/skills/heygen-elevenlabs-renderer/scripts/render_video.py new file mode 100644 index 00000000..c2225008 --- /dev/null +++ b/skills/heygen-elevenlabs-renderer/scripts/render_video.py @@ -0,0 +1,76 @@ +#\!/usr/bin/env python3 +""" +Create a HeyGen avatar video using a pre-uploaded audio asset (voice_type: audio). +Uses HeyGen v3 /videos endpoint via the CLI. + +Usage: + python3 render_video.py --audio-asset-id --title "..." [--avatar-id ] +Prints: video_id on stdout +""" +import argparse +import json +import os +import subprocess +import sys +from pathlib import Path + +CRED_DIR = Path(os.environ.get( + "CLAUDE_CREDENTIALS_DIR", + os.path.expanduser("~/Documents/Claude/Skills/.heygen-credentials") +)) +REGISTRY = Path(__file__).parent.parent / "references" / "registry.json" + +def load_key(): + return (CRED_DIR / "heygen-key.txt").read_text().strip() + +def load_defaults(): + if REGISTRY.exists(): + return json.loads(REGISTRY.read_text()).get("defaults", {}) + return { + "heygen_avatar_id": "9a3600b16f604059b6ab8b9a55e29ea9", + } + +def create(audio_asset_id, title, avatar_id=None, aspect="9:16", resolution="720p"): + env = os.environ.copy() + env["HEYGEN_API_KEY"] = load_key() + env["PATH"] = os.path.expanduser("~/.local/bin") + ":" + env.get("PATH", "") + + avatar_id = avatar_id or load_defaults()["heygen_avatar_id"] + payload = { + "type": "avatar", + "avatar_id": avatar_id, + "audio_asset_id": audio_asset_id, + "aspect_ratio": aspect, + "resolution": resolution, + "title": title, + } + + result = subprocess.run( + ["heygen", "video", "create", "-d", "-"], + input=json.dumps(payload), env=env, + capture_output=True, text=True, timeout=60, + ) + if result.returncode != 0: + sys.exit(f"video create failed: {result.stderr}") + + for line in result.stdout.splitlines(): + line = line.strip() + if line.startswith("{"): + data = json.loads(line) + return data.get("data", {}).get("video_id") + sys.exit(f"could not parse create response: {result.stdout}") + +def main(): + p = argparse.ArgumentParser() + p.add_argument("--audio-asset-id", required=True) + p.add_argument("--title", required=True) + p.add_argument("--avatar-id") + p.add_argument("--aspect", default="9:16", choices=["9:16", "16:9"]) + p.add_argument("--resolution", default="720p", choices=["720p", "1080p", "4k"]) + args = p.parse_args() + + vid = create(args.audio_asset_id, args.title, args.avatar_id, args.aspect, args.resolution) + print(json.dumps({"video_id": vid})) + +if __name__ == "__main__": + main() diff --git a/skills/heygen-elevenlabs-renderer/scripts/synthesize_voice.py b/skills/heygen-elevenlabs-renderer/scripts/synthesize_voice.py new file mode 100644 index 00000000..3ccda196 --- /dev/null +++ b/skills/heygen-elevenlabs-renderer/scripts/synthesize_voice.py @@ -0,0 +1,108 @@ +#\!/usr/bin/env python3 +""" +ElevenLabs TTS — synthesize a v5.4 SSML script into MP3 using Graeham's voice clone. + +Usage: + python3 synthesize_voice.py --text-file script.ssml.txt --out audio.mp3 + python3 synthesize_voice.py --text "Hello world" --out audio.mp3 +""" +import argparse +import json +import os +import re +import sys +import urllib.request +from pathlib import Path + +CRED_DIR = Path(os.environ.get( + "CLAUDE_CREDENTIALS_DIR", + os.path.expanduser("~/Documents/Claude/Skills/.heygen-credentials") +)) +REGISTRY = Path(__file__).parent.parent / "references" / "registry.json" + +def load_key(): + key_file = CRED_DIR / "elevenlabs-key.txt" + if not key_file.exists(): + sys.exit(f"ElevenLabs key not found at {key_file}. Paste it and retry.") + return key_file.read_text().strip() + +def load_defaults(): + if REGISTRY.exists(): + return json.loads(REGISTRY.read_text()).get("defaults", {}) + return { + "elevenlabs_voice_id": "Pa3vOYQHHpLJn1Tf7hnP", + "elevenlabs_model": "eleven_multilingual_v2", + } + + +_DASH_RE = re.compile(r"\s*[—–]\s*") + +def normalize_for_tts(text): + """Deterministic pre-pass that prevents the most common ElevenLabs garbles. + Strips em/en dashes (#1 artifact source) and ' & ', preserving + and [audio] tags untouched. Number/currency spelling is handled by + apply_text_normalization='on' on the API side.""" + parts = re.split(r"(<[^>]+>|\[[^\]]+\])", text) + for i, seg in enumerate(parts): + if i % 2 == 0: + seg = _DASH_RE.sub(". ", seg) + seg = re.sub(r"\s&\s", " and ", seg) + parts[i] = seg + out = "".join(parts) + out = re.sub(r"\.\s*\.\s*\.", "...", out) + out = re.sub(r"[ \t]{2,}", " ", out) + return out + +def synthesize(text, out_path, voice_id=None, model=None, stability=0.5, similarity=0.75, style=0.0, normalize=True): + key = load_key() + defaults = load_defaults() + voice_id = voice_id or defaults["elevenlabs_voice_id"] + model = model or defaults.get("elevenlabs_model", "eleven_multilingual_v2") + + if normalize: + text = normalize_for_tts(text) + + url = f"https://api.elevenlabs.io/v1/text-to-speech/{voice_id}?output_format=mp3_44100_128" + body = json.dumps({ + "text": text, + "model_id": model, + "apply_text_normalization": "on", # spell out numbers/currency; ok on multilingual_v2 (Flash v2.5 needs Enterprise) + "voice_settings": { + "stability": stability, + "similarity_boost": similarity, + "style": style, + "use_speaker_boost": True, + }, + }).encode() + + req = urllib.request.Request(url, data=body, method="POST", headers={ + "xi-api-key": key, + "Content-Type": "application/json", + "Accept": "audio/mpeg", + }) + with urllib.request.urlopen(req, timeout=120) as resp: + data = resp.read() + + Path(out_path).parent.mkdir(parents=True, exist_ok=True) + Path(out_path).write_bytes(data) + return len(data) + +def main(): + p = argparse.ArgumentParser() + p.add_argument("--text-file", help="Path to SSML text file") + p.add_argument("--text", help="Inline text (for quick tests)") + p.add_argument("--out", required=True, help="Output MP3 path") + p.add_argument("--voice-id", help="Override voice_id") + p.add_argument("--model", help="Override model_id") + p.add_argument("--no-normalize", action="store_true", help="Skip the dash/symbol normalization pre-pass") + args = p.parse_args() + + if not (args.text_file or args.text): + sys.exit("Provide --text-file or --text") + + text = Path(args.text_file).read_text() if args.text_file else args.text + n = synthesize(text, args.out, voice_id=args.voice_id, model=args.model, normalize=not args.no_normalize) + print(f"Wrote {args.out} ({n} bytes)") + +if __name__ == "__main__": + main() diff --git a/skills/heygen-elevenlabs-renderer/scripts/upload_asset.py b/skills/heygen-elevenlabs-renderer/scripts/upload_asset.py new file mode 100644 index 00000000..9262917c --- /dev/null +++ b/skills/heygen-elevenlabs-renderer/scripts/upload_asset.py @@ -0,0 +1,54 @@ +#\!/usr/bin/env python3 +""" +HeyGen asset uploader — wraps `heygen asset create --file` for audio MP3s. +CLI is required because api.heygen.com upload endpoint may not be fully allowlisted +in the sandbox; CLI handles the upload path under the hood. + +Usage: + python3 upload_asset.py path/to/audio.mp3 +Prints: asset_id on stdout +""" +import json +import os +import subprocess +import sys +from pathlib import Path + +CRED_DIR = Path(os.environ.get( + "CLAUDE_CREDENTIALS_DIR", + os.path.expanduser("~/Documents/Claude/Skills/.heygen-credentials") +)) + +def load_key(): + return (CRED_DIR / "heygen-key.txt").read_text().strip() + +def upload(path): + env = os.environ.copy() + env["HEYGEN_API_KEY"] = load_key() + env["PATH"] = os.path.expanduser("~/.local/bin") + ":" + env.get("PATH", "") + + result = subprocess.run( + ["heygen", "asset", "create", "--file", str(path)], + env=env, capture_output=True, text=True, timeout=120, + ) + # CLI writes posthog warnings to stderr — ignore + if result.returncode != 0: + sys.exit(f"asset upload failed: {result.stderr}") + + # Find the JSON line in stdout + for line in result.stdout.splitlines(): + line = line.strip() + if line.startswith("{"): + data = json.loads(line) + asset = data.get("data", {}) + return asset.get("asset_id"), asset.get("url") + sys.exit(f"could not parse asset response: {result.stdout}") + +def main(): + if len(sys.argv) != 2: + sys.exit("Usage: upload_asset.py ") + asset_id, url = upload(sys.argv[1]) + print(json.dumps({"asset_id": asset_id, "url": url})) + +if __name__ == "__main__": + main() diff --git a/skills/heygen-video/SKILL.md b/skills/heygen-video/SKILL.md new file mode 100644 index 00000000..ec9358d7 --- /dev/null +++ b/skills/heygen-video/SKILL.md @@ -0,0 +1,134 @@ +--- +name: heygen-video +description: Generate HeyGen avatar videos of Graeham Watts using his trained digital twin and photo-avatar looks. Use ANY time the user mentions HeyGen video, avatar video, AI avatar, talking head video, video of me, video of Graeham, listing intro video, market update video, personalized video message, buyer Q&A video, seller update video, "make me a video", "render a HeyGen video", "create an avatar video", or turning a script into a finished video with Graeham's face and voice. Also trigger on follow-ups like "check on that HeyGen video", "is the video ready", "download the avatar video", or when resuming a previously submitted HeyGen job via video_id. This skill is the CORRECT CHOICE for any HeyGen output — do not use video-creator (slideshow) or remotion-video (React) for HeyGen avatar work. Pair with content-creation-engine when the user has a topic but no script yet. +--- + +# HeyGen Video — Graeham Watts + +Generate HeyGen avatar videos using Graeham's trained looks. v1 of this skill is a single-brand (Graeham only) workflow — PropertyIQ brand avatars are not trained yet and can be added to `references/avatars.md` when they are. + +## When this skill fires + +- "Make a HeyGen video of me saying X" +- "Render a listing intro video with Graeham's avatar" +- "Turn this script into a HeyGen avatar video" +- "Create a market update video" +- "Generate an avatar video: + + diff --git a/skills/skill-creator/eval-viewer/generate_review.py b/skills/skill-creator/eval-viewer/generate_review.py new file mode 100755 index 00000000..7fa59786 --- /dev/null +++ b/skills/skill-creator/eval-viewer/generate_review.py @@ -0,0 +1,471 @@ +#!/usr/bin/env python3 +"""Generate and serve a review page for eval results. + +Reads the workspace directory, discovers runs (directories with outputs/), +embeds all output data into a self-contained HTML page, and serves it via +a tiny HTTP server. Feedback auto-saves to feedback.json in the workspace. + +Usage: + python generate_review.py [--port PORT] [--skill-name NAME] + python generate_review.py --previous-feedback /path/to/old/feedback.json + +No dependencies beyond the Python stdlib are required. +""" + +import argparse +import base64 +import json +import mimetypes +import os +import re +import signal +import subprocess +import sys +import time +import webbrowser +from functools import partial +from http.server import HTTPServer, BaseHTTPRequestHandler +from pathlib import Path + +# Files to exclude from output listings +METADATA_FILES = {"transcript.md", "user_notes.md", "metrics.json"} + +# Extensions we render as inline text +TEXT_EXTENSIONS = { + ".txt", ".md", ".json", ".csv", ".py", ".js", ".ts", ".tsx", ".jsx", + ".yaml", ".yml", ".xml", ".html", ".css", ".sh", ".rb", ".go", ".rs", + ".java", ".c", ".cpp", ".h", ".hpp", ".sql", ".r", ".toml", +} + +# Extensions we render as inline images +IMAGE_EXTENSIONS = {".png", ".jpg", ".jpeg", ".gif", ".svg", ".webp"} + +# MIME type overrides for common types +MIME_OVERRIDES = { + ".svg": "image/svg+xml", + ".xlsx": "application/vnd.openxmlformats-officedocument.spreadsheetml.sheet", + ".docx": "application/vnd.openxmlformats-officedocument.wordprocessingml.document", + ".pptx": "application/vnd.openxmlformats-officedocument.presentationml.presentation", +} + + +def get_mime_type(path: Path) -> str: + ext = path.suffix.lower() + if ext in MIME_OVERRIDES: + return MIME_OVERRIDES[ext] + mime, _ = mimetypes.guess_type(str(path)) + return mime or "application/octet-stream" + + +def find_runs(workspace: Path) -> list[dict]: + """Recursively find directories that contain an outputs/ subdirectory.""" + runs: list[dict] = [] + _find_runs_recursive(workspace, workspace, runs) + runs.sort(key=lambda r: (r.get("eval_id", float("inf")), r["id"])) + return runs + + +def _find_runs_recursive(root: Path, current: Path, runs: list[dict]) -> None: + if not current.is_dir(): + return + + outputs_dir = current / "outputs" + if outputs_dir.is_dir(): + run = build_run(root, current) + if run: + runs.append(run) + return + + skip = {"node_modules", ".git", "__pycache__", "skill", "inputs"} + for child in sorted(current.iterdir()): + if child.is_dir() and child.name not in skip: + _find_runs_recursive(root, child, runs) + + +def build_run(root: Path, run_dir: Path) -> dict | None: + """Build a run dict with prompt, outputs, and grading data.""" + prompt = "" + eval_id = None + + # Try eval_metadata.json + for candidate in [run_dir / "eval_metadata.json", run_dir.parent / "eval_metadata.json"]: + if candidate.exists(): + try: + metadata = json.loads(candidate.read_text()) + prompt = metadata.get("prompt", "") + eval_id = metadata.get("eval_id") + except (json.JSONDecodeError, OSError): + pass + if prompt: + break + + # Fall back to transcript.md + if not prompt: + for candidate in [run_dir / "transcript.md", run_dir / "outputs" / "transcript.md"]: + if candidate.exists(): + try: + text = candidate.read_text() + match = re.search(r"## Eval Prompt\n\n([\s\S]*?)(?=\n##|$)", text) + if match: + prompt = match.group(1).strip() + except OSError: + pass + if prompt: + break + + if not prompt: + prompt = "(No prompt found)" + + run_id = str(run_dir.relative_to(root)).replace("/", "-").replace("\\", "-") + + # Collect output files + outputs_dir = run_dir / "outputs" + output_files: list[dict] = [] + if outputs_dir.is_dir(): + for f in sorted(outputs_dir.iterdir()): + if f.is_file() and f.name not in METADATA_FILES: + output_files.append(embed_file(f)) + + # Load grading if present + grading = None + for candidate in [run_dir / "grading.json", run_dir.parent / "grading.json"]: + if candidate.exists(): + try: + grading = json.loads(candidate.read_text()) + except (json.JSONDecodeError, OSError): + pass + if grading: + break + + return { + "id": run_id, + "prompt": prompt, + "eval_id": eval_id, + "outputs": output_files, + "grading": grading, + } + + +def embed_file(path: Path) -> dict: + """Read a file and return an embedded representation.""" + ext = path.suffix.lower() + mime = get_mime_type(path) + + if ext in TEXT_EXTENSIONS: + try: + content = path.read_text(errors="replace") + except OSError: + content = "(Error reading file)" + return { + "name": path.name, + "type": "text", + "content": content, + } + elif ext in IMAGE_EXTENSIONS: + try: + raw = path.read_bytes() + b64 = base64.b64encode(raw).decode("ascii") + except OSError: + return {"name": path.name, "type": "error", "content": "(Error reading file)"} + return { + "name": path.name, + "type": "image", + "mime": mime, + "data_uri": f"data:{mime};base64,{b64}", + } + elif ext == ".pdf": + try: + raw = path.read_bytes() + b64 = base64.b64encode(raw).decode("ascii") + except OSError: + return {"name": path.name, "type": "error", "content": "(Error reading file)"} + return { + "name": path.name, + "type": "pdf", + "data_uri": f"data:{mime};base64,{b64}", + } + elif ext == ".xlsx": + try: + raw = path.read_bytes() + b64 = base64.b64encode(raw).decode("ascii") + except OSError: + return {"name": path.name, "type": "error", "content": "(Error reading file)"} + return { + "name": path.name, + "type": "xlsx", + "data_b64": b64, + } + else: + # Binary / unknown — base64 download link + try: + raw = path.read_bytes() + b64 = base64.b64encode(raw).decode("ascii") + except OSError: + return {"name": path.name, "type": "error", "content": "(Error reading file)"} + return { + "name": path.name, + "type": "binary", + "mime": mime, + "data_uri": f"data:{mime};base64,{b64}", + } + + +def load_previous_iteration(workspace: Path) -> dict[str, dict]: + """Load previous iteration's feedback and outputs. + + Returns a map of run_id -> {"feedback": str, "outputs": list[dict]}. + """ + result: dict[str, dict] = {} + + # Load feedback + feedback_map: dict[str, str] = {} + feedback_path = workspace / "feedback.json" + if feedback_path.exists(): + try: + data = json.loads(feedback_path.read_text()) + feedback_map = { + r["run_id"]: r["feedback"] + for r in data.get("reviews", []) + if r.get("feedback", "").strip() + } + except (json.JSONDecodeError, OSError, KeyError): + pass + + # Load runs (to get outputs) + prev_runs = find_runs(workspace) + for run in prev_runs: + result[run["id"]] = { + "feedback": feedback_map.get(run["id"], ""), + "outputs": run.get("outputs", []), + } + + # Also add feedback for run_ids that had feedback but no matching run + for run_id, fb in feedback_map.items(): + if run_id not in result: + result[run_id] = {"feedback": fb, "outputs": []} + + return result + + +def generate_html( + runs: list[dict], + skill_name: str, + previous: dict[str, dict] | None = None, + benchmark: dict | None = None, +) -> str: + """Generate the complete standalone HTML page with embedded data.""" + template_path = Path(__file__).parent / "viewer.html" + template = template_path.read_text() + + # Build previous_feedback and previous_outputs maps for the template + previous_feedback: dict[str, str] = {} + previous_outputs: dict[str, list[dict]] = {} + if previous: + for run_id, data in previous.items(): + if data.get("feedback"): + previous_feedback[run_id] = data["feedback"] + if data.get("outputs"): + previous_outputs[run_id] = data["outputs"] + + embedded = { + "skill_name": skill_name, + "runs": runs, + "previous_feedback": previous_feedback, + "previous_outputs": previous_outputs, + } + if benchmark: + embedded["benchmark"] = benchmark + + data_json = json.dumps(embedded) + + return template.replace("/*__EMBEDDED_DATA__*/", f"const EMBEDDED_DATA = {data_json};") + + +# --------------------------------------------------------------------------- +# HTTP server (stdlib only, zero dependencies) +# --------------------------------------------------------------------------- + +def _kill_port(port: int) -> None: + """Kill any process listening on the given port.""" + try: + result = subprocess.run( + ["lsof", "-ti", f":{port}"], + capture_output=True, text=True, timeout=5, + ) + for pid_str in result.stdout.strip().split("\n"): + if pid_str.strip(): + try: + os.kill(int(pid_str.strip()), signal.SIGTERM) + except (ProcessLookupError, ValueError): + pass + if result.stdout.strip(): + time.sleep(0.5) + except subprocess.TimeoutExpired: + pass + except FileNotFoundError: + print("Note: lsof not found, cannot check if port is in use", file=sys.stderr) + +class ReviewHandler(BaseHTTPRequestHandler): + """Serves the review HTML and handles feedback saves. + + Regenerates the HTML on each page load so that refreshing the browser + picks up new eval outputs without restarting the server. + """ + + def __init__( + self, + workspace: Path, + skill_name: str, + feedback_path: Path, + previous: dict[str, dict], + benchmark_path: Path | None, + *args, + **kwargs, + ): + self.workspace = workspace + self.skill_name = skill_name + self.feedback_path = feedback_path + self.previous = previous + self.benchmark_path = benchmark_path + super().__init__(*args, **kwargs) + + def do_GET(self) -> None: + if self.path == "/" or self.path == "/index.html": + # Regenerate HTML on each request (re-scans workspace for new outputs) + runs = find_runs(self.workspace) + benchmark = None + if self.benchmark_path and self.benchmark_path.exists(): + try: + benchmark = json.loads(self.benchmark_path.read_text()) + except (json.JSONDecodeError, OSError): + pass + html = generate_html(runs, self.skill_name, self.previous, benchmark) + content = html.encode("utf-8") + self.send_response(200) + self.send_header("Content-Type", "text/html; charset=utf-8") + self.send_header("Content-Length", str(len(content))) + self.end_headers() + self.wfile.write(content) + elif self.path == "/api/feedback": + data = b"{}" + if self.feedback_path.exists(): + data = self.feedback_path.read_bytes() + self.send_response(200) + self.send_header("Content-Type", "application/json") + self.send_header("Content-Length", str(len(data))) + self.end_headers() + self.wfile.write(data) + else: + self.send_error(404) + + def do_POST(self) -> None: + if self.path == "/api/feedback": + length = int(self.headers.get("Content-Length", 0)) + body = self.rfile.read(length) + try: + data = json.loads(body) + if not isinstance(data, dict) or "reviews" not in data: + raise ValueError("Expected JSON object with 'reviews' key") + self.feedback_path.write_text(json.dumps(data, indent=2) + "\n") + resp = b'{"ok":true}' + self.send_response(200) + except (json.JSONDecodeError, OSError, ValueError) as e: + resp = json.dumps({"error": str(e)}).encode() + self.send_response(500) + self.send_header("Content-Type", "application/json") + self.send_header("Content-Length", str(len(resp))) + self.end_headers() + self.wfile.write(resp) + else: + self.send_error(404) + + def log_message(self, format: str, *args: object) -> None: + # Suppress request logging to keep terminal clean + pass + + +def main() -> None: + parser = argparse.ArgumentParser(description="Generate and serve eval review") + parser.add_argument("workspace", type=Path, help="Path to workspace directory") + parser.add_argument("--port", "-p", type=int, default=3117, help="Server port (default: 3117)") + parser.add_argument("--skill-name", "-n", type=str, default=None, help="Skill name for header") + parser.add_argument( + "--previous-workspace", type=Path, default=None, + help="Path to previous iteration's workspace (shows old outputs and feedback as context)", + ) + parser.add_argument( + "--benchmark", type=Path, default=None, + help="Path to benchmark.json to show in the Benchmark tab", + ) + parser.add_argument( + "--static", "-s", type=Path, default=None, + help="Write standalone HTML to this path instead of starting a server", + ) + args = parser.parse_args() + + workspace = args.workspace.resolve() + if not workspace.is_dir(): + print(f"Error: {workspace} is not a directory", file=sys.stderr) + sys.exit(1) + + runs = find_runs(workspace) + if not runs: + print(f"No runs found in {workspace}", file=sys.stderr) + sys.exit(1) + + skill_name = args.skill_name or workspace.name.replace("-workspace", "") + feedback_path = workspace / "feedback.json" + + previous: dict[str, dict] = {} + if args.previous_workspace: + previous = load_previous_iteration(args.previous_workspace.resolve()) + + benchmark_path = args.benchmark.resolve() if args.benchmark else None + benchmark = None + if benchmark_path and benchmark_path.exists(): + try: + benchmark = json.loads(benchmark_path.read_text()) + except (json.JSONDecodeError, OSError): + pass + + if args.static: + html = generate_html(runs, skill_name, previous, benchmark) + args.static.parent.mkdir(parents=True, exist_ok=True) + args.static.write_text(html) + print(f"\n Static viewer written to: {args.static}\n") + sys.exit(0) + + # Kill any existing process on the target port + port = args.port + _kill_port(port) + handler = partial(ReviewHandler, workspace, skill_name, feedback_path, previous, benchmark_path) + try: + server = HTTPServer(("127.0.0.1", port), handler) + except OSError: + # Port still in use after kill attempt — find a free one + server = HTTPServer(("127.0.0.1", 0), handler) + port = server.server_address[1] + + url = f"http://localhost:{port}" + print(f"\n Eval Viewer") + print(f" ─────────────────────────────────") + print(f" URL: {url}") + print(f" Workspace: {workspace}") + print(f" Feedback: {feedback_path}") + if previous: + print(f" Previous: {args.previous_workspace} ({len(previous)} runs)") + if benchmark_path: + print(f" Benchmark: {benchmark_path}") + print(f"\n Press Ctrl+C to stop.\n") + + webbrowser.open(url) + + try: + server.serve_forever() + except KeyboardInterrupt: + print("\nStopped.") + server.server_close() + + +if __name__ == "__main__": + main() diff --git a/skills/skill-creator/eval-viewer/viewer.html b/skills/skill-creator/eval-viewer/viewer.html new file mode 100755 index 00000000..6d8e9634 --- /dev/null +++ b/skills/skill-creator/eval-viewer/viewer.html @@ -0,0 +1,1325 @@ + + + + + + Eval Review + + + + + + + +
+
+
+

Eval Review:

+
Review each output and leave feedback below. Navigate with arrow keys or buttons. When done, copy feedback and paste into Claude Code.
+
+
+
+ + + + + +
+
+ +
+
Prompt
+
+
+
+
+ + +
+
Output
+
+
No output files found
+
+
+ + + + + + + + +
+
Your Feedback
+
+ + + +
+
+
+ + +
+ + +
+
+
No benchmark data available. Run a benchmark to see quantitative results here.
+
+
+
+ + +
+
+

Review Complete

+

Your feedback has been saved. Go back to your Claude Code session and tell Claude you're done reviewing.

+
+ +
+
+
+ + +
+ + + + diff --git a/skills/skill-creator/references/schemas.md b/skills/skill-creator/references/schemas.md new file mode 100755 index 00000000..b6eeaa2d --- /dev/null +++ b/skills/skill-creator/references/schemas.md @@ -0,0 +1,430 @@ +# JSON Schemas + +This document defines the JSON schemas used by skill-creator. + +--- + +## evals.json + +Defines the evals for a skill. Located at `evals/evals.json` within the skill directory. + +```json +{ + "skill_name": "example-skill", + "evals": [ + { + "id": 1, + "prompt": "User's example prompt", + "expected_output": "Description of expected result", + "files": ["evals/files/sample1.pdf"], + "expectations": [ + "The output includes X", + "The skill used script Y" + ] + } + ] +} +``` + +**Fields:** +- `skill_name`: Name matching the skill's frontmatter +- `evals[].id`: Unique integer identifier +- `evals[].prompt`: The task to execute +- `evals[].expected_output`: Human-readable description of success +- `evals[].files`: Optional list of input file paths (relative to skill root) +- `evals[].expectations`: List of verifiable statements + +--- + +## history.json + +Tracks version progression in Improve mode. Located at workspace root. + +```json +{ + "started_at": "2026-01-15T10:30:00Z", + "skill_name": "pdf", + "current_best": "v2", + "iterations": [ + { + "version": "v0", + "parent": null, + "expectation_pass_rate": 0.65, + "grading_result": "baseline", + "is_current_best": false + }, + { + "version": "v1", + "parent": "v0", + "expectation_pass_rate": 0.75, + "grading_result": "won", + "is_current_best": false + }, + { + "version": "v2", + "parent": "v1", + "expectation_pass_rate": 0.85, + "grading_result": "won", + "is_current_best": true + } + ] +} +``` + +**Fields:** +- `started_at`: ISO timestamp of when improvement started +- `skill_name`: Name of the skill being improved +- `current_best`: Version identifier of the best performer +- `iterations[].version`: Version identifier (v0, v1, ...) +- `iterations[].parent`: Parent version this was derived from +- `iterations[].expectation_pass_rate`: Pass rate from grading +- `iterations[].grading_result`: "baseline", "won", "lost", or "tie" +- `iterations[].is_current_best`: Whether this is the current best version + +--- + +## grading.json + +Output from the grader agent. Located at `/grading.json`. + +```json +{ + "expectations": [ + { + "text": "The output includes the name 'John Smith'", + "passed": true, + "evidence": "Found in transcript Step 3: 'Extracted names: John Smith, Sarah Johnson'" + }, + { + "text": "The spreadsheet has a SUM formula in cell B10", + "passed": false, + "evidence": "No spreadsheet was created. The output was a text file." + } + ], + "summary": { + "passed": 2, + "failed": 1, + "total": 3, + "pass_rate": 0.67 + }, + "execution_metrics": { + "tool_calls": { + "Read": 5, + "Write": 2, + "Bash": 8 + }, + "total_tool_calls": 15, + "total_steps": 6, + "errors_encountered": 0, + "output_chars": 12450, + "transcript_chars": 3200 + }, + "timing": { + "executor_duration_seconds": 165.0, + "grader_duration_seconds": 26.0, + "total_duration_seconds": 191.0 + }, + "claims": [ + { + "claim": "The form has 12 fillable fields", + "type": "factual", + "verified": true, + "evidence": "Counted 12 fields in field_info.json" + } + ], + "user_notes_summary": { + "uncertainties": ["Used 2023 data, may be stale"], + "needs_review": [], + "workarounds": ["Fell back to text overlay for non-fillable fields"] + }, + "eval_feedback": { + "suggestions": [ + { + "assertion": "The output includes the name 'John Smith'", + "reason": "A hallucinated document that mentions the name would also pass" + } + ], + "overall": "Assertions check presence but not correctness." + } +} +``` + +**Fields:** +- `expectations[]`: Graded expectations with evidence +- `summary`: Aggregate pass/fail counts +- `execution_metrics`: Tool usage and output size (from executor's metrics.json) +- `timing`: Wall clock timing (from timing.json) +- `claims`: Extracted and verified claims from the output +- `user_notes_summary`: Issues flagged by the executor +- `eval_feedback`: (optional) Improvement suggestions for the evals, only present when the grader identifies issues worth raising + +--- + +## metrics.json + +Output from the executor agent. Located at `/outputs/metrics.json`. + +```json +{ + "tool_calls": { + "Read": 5, + "Write": 2, + "Bash": 8, + "Edit": 1, + "Glob": 2, + "Grep": 0 + }, + "total_tool_calls": 18, + "total_steps": 6, + "files_created": ["filled_form.pdf", "field_values.json"], + "errors_encountered": 0, + "output_chars": 12450, + "transcript_chars": 3200 +} +``` + +**Fields:** +- `tool_calls`: Count per tool type +- `total_tool_calls`: Sum of all tool calls +- `total_steps`: Number of major execution steps +- `files_created`: List of output files created +- `errors_encountered`: Number of errors during execution +- `output_chars`: Total character count of output files +- `transcript_chars`: Character count of transcript + +--- + +## timing.json + +Wall clock timing for a run. Located at `/timing.json`. + +**How to capture:** When a subagent task completes, the task notification includes `total_tokens` and `duration_ms`. Save these immediately — they are not persisted anywhere else and cannot be recovered after the fact. + +```json +{ + "total_tokens": 84852, + "duration_ms": 23332, + "total_duration_seconds": 23.3, + "executor_start": "2026-01-15T10:30:00Z", + "executor_end": "2026-01-15T10:32:45Z", + "executor_duration_seconds": 165.0, + "grader_start": "2026-01-15T10:32:46Z", + "grader_end": "2026-01-15T10:33:12Z", + "grader_duration_seconds": 26.0 +} +``` + +--- + +## benchmark.json + +Output from Benchmark mode. Located at `benchmarks//benchmark.json`. + +```json +{ + "metadata": { + "skill_name": "pdf", + "skill_path": "/path/to/pdf", + "executor_model": "claude-sonnet-4-20250514", + "analyzer_model": "most-capable-model", + "timestamp": "2026-01-15T10:30:00Z", + "evals_run": [1, 2, 3], + "runs_per_configuration": 3 + }, + + "runs": [ + { + "eval_id": 1, + "eval_name": "Ocean", + "configuration": "with_skill", + "run_number": 1, + "result": { + "pass_rate": 0.85, + "passed": 6, + "failed": 1, + "total": 7, + "time_seconds": 42.5, + "tokens": 3800, + "tool_calls": 18, + "errors": 0 + }, + "expectations": [ + {"text": "...", "passed": true, "evidence": "..."} + ], + "notes": [ + "Used 2023 data, may be stale", + "Fell back to text overlay for non-fillable fields" + ] + } + ], + + "run_summary": { + "with_skill": { + "pass_rate": {"mean": 0.85, "stddev": 0.05, "min": 0.80, "max": 0.90}, + "time_seconds": {"mean": 45.0, "stddev": 12.0, "min": 32.0, "max": 58.0}, + "tokens": {"mean": 3800, "stddev": 400, "min": 3200, "max": 4100} + }, + "without_skill": { + "pass_rate": {"mean": 0.35, "stddev": 0.08, "min": 0.28, "max": 0.45}, + "time_seconds": {"mean": 32.0, "stddev": 8.0, "min": 24.0, "max": 42.0}, + "tokens": {"mean": 2100, "stddev": 300, "min": 1800, "max": 2500} + }, + "delta": { + "pass_rate": "+0.50", + "time_seconds": "+13.0", + "tokens": "+1700" + } + }, + + "notes": [ + "Assertion 'Output is a PDF file' passes 100% in both configurations - may not differentiate skill value", + "Eval 3 shows high variance (50% ± 40%) - may be flaky or model-dependent", + "Without-skill runs consistently fail on table extraction expectations", + "Skill adds 13s average execution time but improves pass rate by 50%" + ] +} +``` + +**Fields:** +- `metadata`: Information about the benchmark run + - `skill_name`: Name of the skill + - `timestamp`: When the benchmark was run + - `evals_run`: List of eval names or IDs + - `runs_per_configuration`: Number of runs per config (e.g. 3) +- `runs[]`: Individual run results + - `eval_id`: Numeric eval identifier + - `eval_name`: Human-readable eval name (used as section header in the viewer) + - `configuration`: Must be `"with_skill"` or `"without_skill"` (the viewer uses this exact string for grouping and color coding) + - `run_number`: Integer run number (1, 2, 3...) + - `result`: Nested object with `pass_rate`, `passed`, `total`, `time_seconds`, `tokens`, `errors` +- `run_summary`: Statistical aggregates per configuration + - `with_skill` / `without_skill`: Each contains `pass_rate`, `time_seconds`, `tokens` objects with `mean` and `stddev` fields + - `delta`: Difference strings like `"+0.50"`, `"+13.0"`, `"+1700"` +- `notes`: Freeform observations from the analyzer + +**Important:** The viewer reads these field names exactly. Using `config` instead of `configuration`, or putting `pass_rate` at the top level of a run instead of nested under `result`, will cause the viewer to show empty/zero values. Always reference this schema when generating benchmark.json manually. + +--- + +## comparison.json + +Output from blind comparator. Located at `/comparison-N.json`. + +```json +{ + "winner": "A", + "reasoning": "Output A provides a complete solution with proper formatting and all required fields. Output B is missing the date field and has formatting inconsistencies.", + "rubric": { + "A": { + "content": { + "correctness": 5, + "completeness": 5, + "accuracy": 4 + }, + "structure": { + "organization": 4, + "formatting": 5, + "usability": 4 + }, + "content_score": 4.7, + "structure_score": 4.3, + "overall_score": 9.0 + }, + "B": { + "content": { + "correctness": 3, + "completeness": 2, + "accuracy": 3 + }, + "structure": { + "organization": 3, + "formatting": 2, + "usability": 3 + }, + "content_score": 2.7, + "structure_score": 2.7, + "overall_score": 5.4 + } + }, + "output_quality": { + "A": { + "score": 9, + "strengths": ["Complete solution", "Well-formatted", "All fields present"], + "weaknesses": ["Minor style inconsistency in header"] + }, + "B": { + "score": 5, + "strengths": ["Readable output", "Correct basic structure"], + "weaknesses": ["Missing date field", "Formatting inconsistencies", "Partial data extraction"] + } + }, + "expectation_results": { + "A": { + "passed": 4, + "total": 5, + "pass_rate": 0.80, + "details": [ + {"text": "Output includes name", "passed": true} + ] + }, + "B": { + "passed": 3, + "total": 5, + "pass_rate": 0.60, + "details": [ + {"text": "Output includes name", "passed": true} + ] + } + } +} +``` + +--- + +## analysis.json + +Output from post-hoc analyzer. Located at `/analysis.json`. + +```json +{ + "comparison_summary": { + "winner": "A", + "winner_skill": "path/to/winner/skill", + "loser_skill": "path/to/loser/skill", + "comparator_reasoning": "Brief summary of why comparator chose winner" + }, + "winner_strengths": [ + "Clear step-by-step instructions for handling multi-page documents", + "Included validation script that caught formatting errors" + ], + "loser_weaknesses": [ + "Vague instruction 'process the document appropriately' led to inconsistent behavior", + "No script for validation, agent had to improvise" + ], + "instruction_following": { + "winner": { + "score": 9, + "issues": ["Minor: skipped optional logging step"] + }, + "loser": { + "score": 6, + "issues": [ + "Did not use the skill's formatting template", + "Invented own approach instead of following step 3" + ] + } + }, + "improvement_suggestions": [ + { + "priority": "high", + "category": "instructions", + "suggestion": "Replace 'process the document appropriately' with explicit steps", + "expected_impact": "Would eliminate ambiguity that caused inconsistent behavior" + } + ], + "transcript_insights": { + "winner_execution_pattern": "Read skill -> Followed 5-step process -> Used validation script", + "loser_execution_pattern": "Read skill -> Unclear on approach -> Tried 3 different methods" + } +} +``` diff --git a/skills/skill-creator/scripts/__init__.py b/skills/skill-creator/scripts/__init__.py new file mode 100755 index 00000000..e69de29b diff --git a/skills/skill-creator/scripts/aggregate_benchmark.py b/skills/skill-creator/scripts/aggregate_benchmark.py new file mode 100755 index 00000000..3e66e8c1 --- /dev/null +++ b/skills/skill-creator/scripts/aggregate_benchmark.py @@ -0,0 +1,401 @@ +#!/usr/bin/env python3 +""" +Aggregate individual run results into benchmark summary statistics. + +Reads grading.json files from run directories and produces: +- run_summary with mean, stddev, min, max for each metric +- delta between with_skill and without_skill configurations + +Usage: + python aggregate_benchmark.py + +Example: + python aggregate_benchmark.py benchmarks/2026-01-15T10-30-00/ + +The script supports two directory layouts: + + Workspace layout (from skill-creator iterations): + / + └── eval-N/ + ├── with_skill/ + │ ├── run-1/grading.json + │ └── run-2/grading.json + └── without_skill/ + ├── run-1/grading.json + └── run-2/grading.json + + Legacy layout (with runs/ subdirectory): + / + └── runs/ + └── eval-N/ + ├── with_skill/ + │ └── run-1/grading.json + └── without_skill/ + └── run-1/grading.json +""" + +import argparse +import json +import math +import sys +from datetime import datetime, timezone +from pathlib import Path + + +def calculate_stats(values: list[float]) -> dict: + """Calculate mean, stddev, min, max for a list of values.""" + if not values: + return {"mean": 0.0, "stddev": 0.0, "min": 0.0, "max": 0.0} + + n = len(values) + mean = sum(values) / n + + if n > 1: + variance = sum((x - mean) ** 2 for x in values) / (n - 1) + stddev = math.sqrt(variance) + else: + stddev = 0.0 + + return { + "mean": round(mean, 4), + "stddev": round(stddev, 4), + "min": round(min(values), 4), + "max": round(max(values), 4) + } + + +def load_run_results(benchmark_dir: Path) -> dict: + """ + Load all run results from a benchmark directory. + + Returns dict keyed by config name (e.g. "with_skill"/"without_skill", + or "new_skill"/"old_skill"), each containing a list of run results. + """ + # Support both layouts: eval dirs directly under benchmark_dir, or under runs/ + runs_dir = benchmark_dir / "runs" + if runs_dir.exists(): + search_dir = runs_dir + elif list(benchmark_dir.glob("eval-*")): + search_dir = benchmark_dir + else: + print(f"No eval directories found in {benchmark_dir} or {benchmark_dir / 'runs'}") + return {} + + results: dict[str, list] = {} + + for eval_idx, eval_dir in enumerate(sorted(search_dir.glob("eval-*"))): + metadata_path = eval_dir / "eval_metadata.json" + if metadata_path.exists(): + try: + with open(metadata_path) as mf: + eval_id = json.load(mf).get("eval_id", eval_idx) + except (json.JSONDecodeError, OSError): + eval_id = eval_idx + else: + try: + eval_id = int(eval_dir.name.split("-")[1]) + except ValueError: + eval_id = eval_idx + + # Discover config directories dynamically rather than hardcoding names + for config_dir in sorted(eval_dir.iterdir()): + if not config_dir.is_dir(): + continue + # Skip non-config directories (inputs, outputs, etc.) + if not list(config_dir.glob("run-*")): + continue + config = config_dir.name + if config not in results: + results[config] = [] + + for run_dir in sorted(config_dir.glob("run-*")): + run_number = int(run_dir.name.split("-")[1]) + grading_file = run_dir / "grading.json" + + if not grading_file.exists(): + print(f"Warning: grading.json not found in {run_dir}") + continue + + try: + with open(grading_file) as f: + grading = json.load(f) + except json.JSONDecodeError as e: + print(f"Warning: Invalid JSON in {grading_file}: {e}") + continue + + # Extract metrics + result = { + "eval_id": eval_id, + "run_number": run_number, + "pass_rate": grading.get("summary", {}).get("pass_rate", 0.0), + "passed": grading.get("summary", {}).get("passed", 0), + "failed": grading.get("summary", {}).get("failed", 0), + "total": grading.get("summary", {}).get("total", 0), + } + + # Extract timing — check grading.json first, then sibling timing.json + timing = grading.get("timing", {}) + result["time_seconds"] = timing.get("total_duration_seconds", 0.0) + timing_file = run_dir / "timing.json" + if result["time_seconds"] == 0.0 and timing_file.exists(): + try: + with open(timing_file) as tf: + timing_data = json.load(tf) + result["time_seconds"] = timing_data.get("total_duration_seconds", 0.0) + result["tokens"] = timing_data.get("total_tokens", 0) + except json.JSONDecodeError: + pass + + # Extract metrics if available + metrics = grading.get("execution_metrics", {}) + result["tool_calls"] = metrics.get("total_tool_calls", 0) + if not result.get("tokens"): + result["tokens"] = metrics.get("output_chars", 0) + result["errors"] = metrics.get("errors_encountered", 0) + + # Extract expectations — viewer requires fields: text, passed, evidence + raw_expectations = grading.get("expectations", []) + for exp in raw_expectations: + if "text" not in exp or "passed" not in exp: + print(f"Warning: expectation in {grading_file} missing required fields (text, passed, evidence): {exp}") + result["expectations"] = raw_expectations + + # Extract notes from user_notes_summary + notes_summary = grading.get("user_notes_summary", {}) + notes = [] + notes.extend(notes_summary.get("uncertainties", [])) + notes.extend(notes_summary.get("needs_review", [])) + notes.extend(notes_summary.get("workarounds", [])) + result["notes"] = notes + + results[config].append(result) + + return results + + +def aggregate_results(results: dict) -> dict: + """ + Aggregate run results into summary statistics. + + Returns run_summary with stats for each configuration and delta. + """ + run_summary = {} + configs = list(results.keys()) + + for config in configs: + runs = results.get(config, []) + + if not runs: + run_summary[config] = { + "pass_rate": {"mean": 0.0, "stddev": 0.0, "min": 0.0, "max": 0.0}, + "time_seconds": {"mean": 0.0, "stddev": 0.0, "min": 0.0, "max": 0.0}, + "tokens": {"mean": 0, "stddev": 0, "min": 0, "max": 0} + } + continue + + pass_rates = [r["pass_rate"] for r in runs] + times = [r["time_seconds"] for r in runs] + tokens = [r.get("tokens", 0) for r in runs] + + run_summary[config] = { + "pass_rate": calculate_stats(pass_rates), + "time_seconds": calculate_stats(times), + "tokens": calculate_stats(tokens) + } + + # Calculate delta between the first two configs (if two exist) + if len(configs) >= 2: + primary = run_summary.get(configs[0], {}) + baseline = run_summary.get(configs[1], {}) + else: + primary = run_summary.get(configs[0], {}) if configs else {} + baseline = {} + + delta_pass_rate = primary.get("pass_rate", {}).get("mean", 0) - baseline.get("pass_rate", {}).get("mean", 0) + delta_time = primary.get("time_seconds", {}).get("mean", 0) - baseline.get("time_seconds", {}).get("mean", 0) + delta_tokens = primary.get("tokens", {}).get("mean", 0) - baseline.get("tokens", {}).get("mean", 0) + + run_summary["delta"] = { + "pass_rate": f"{delta_pass_rate:+.2f}", + "time_seconds": f"{delta_time:+.1f}", + "tokens": f"{delta_tokens:+.0f}" + } + + return run_summary + + +def generate_benchmark(benchmark_dir: Path, skill_name: str = "", skill_path: str = "") -> dict: + """ + Generate complete benchmark.json from run results. + """ + results = load_run_results(benchmark_dir) + run_summary = aggregate_results(results) + + # Build runs array for benchmark.json + runs = [] + for config in results: + for result in results[config]: + runs.append({ + "eval_id": result["eval_id"], + "configuration": config, + "run_number": result["run_number"], + "result": { + "pass_rate": result["pass_rate"], + "passed": result["passed"], + "failed": result["failed"], + "total": result["total"], + "time_seconds": result["time_seconds"], + "tokens": result.get("tokens", 0), + "tool_calls": result.get("tool_calls", 0), + "errors": result.get("errors", 0) + }, + "expectations": result["expectations"], + "notes": result["notes"] + }) + + # Determine eval IDs from results + eval_ids = sorted(set( + r["eval_id"] + for config in results.values() + for r in config + )) + + benchmark = { + "metadata": { + "skill_name": skill_name or "", + "skill_path": skill_path or "", + "executor_model": "", + "analyzer_model": "", + "timestamp": datetime.now(timezone.utc).strftime("%Y-%m-%dT%H:%M:%SZ"), + "evals_run": eval_ids, + "runs_per_configuration": 3 + }, + "runs": runs, + "run_summary": run_summary, + "notes": [] # To be filled by analyzer + } + + return benchmark + + +def generate_markdown(benchmark: dict) -> str: + """Generate human-readable benchmark.md from benchmark data.""" + metadata = benchmark["metadata"] + run_summary = benchmark["run_summary"] + + # Determine config names (excluding "delta") + configs = [k for k in run_summary if k != "delta"] + config_a = configs[0] if len(configs) >= 1 else "config_a" + config_b = configs[1] if len(configs) >= 2 else "config_b" + label_a = config_a.replace("_", " ").title() + label_b = config_b.replace("_", " ").title() + + lines = [ + f"# Skill Benchmark: {metadata['skill_name']}", + "", + f"**Model**: {metadata['executor_model']}", + f"**Date**: {metadata['timestamp']}", + f"**Evals**: {', '.join(map(str, metadata['evals_run']))} ({metadata['runs_per_configuration']} runs each per configuration)", + "", + "## Summary", + "", + f"| Metric | {label_a} | {label_b} | Delta |", + "|--------|------------|---------------|-------|", + ] + + a_summary = run_summary.get(config_a, {}) + b_summary = run_summary.get(config_b, {}) + delta = run_summary.get("delta", {}) + + # Format pass rate + a_pr = a_summary.get("pass_rate", {}) + b_pr = b_summary.get("pass_rate", {}) + lines.append(f"| Pass Rate | {a_pr.get('mean', 0)*100:.0f}% ± {a_pr.get('stddev', 0)*100:.0f}% | {b_pr.get('mean', 0)*100:.0f}% ± {b_pr.get('stddev', 0)*100:.0f}% | {delta.get('pass_rate', '—')} |") + + # Format time + a_time = a_summary.get("time_seconds", {}) + b_time = b_summary.get("time_seconds", {}) + lines.append(f"| Time | {a_time.get('mean', 0):.1f}s ± {a_time.get('stddev', 0):.1f}s | {b_time.get('mean', 0):.1f}s ± {b_time.get('stddev', 0):.1f}s | {delta.get('time_seconds', '—')}s |") + + # Format tokens + a_tokens = a_summary.get("tokens", {}) + b_tokens = b_summary.get("tokens", {}) + lines.append(f"| Tokens | {a_tokens.get('mean', 0):.0f} ± {a_tokens.get('stddev', 0):.0f} | {b_tokens.get('mean', 0):.0f} ± {b_tokens.get('stddev', 0):.0f} | {delta.get('tokens', '—')} |") + + # Notes section + if benchmark.get("notes"): + lines.extend([ + "", + "## Notes", + "" + ]) + for note in benchmark["notes"]: + lines.append(f"- {note}") + + return "\n".join(lines) + + +def main(): + parser = argparse.ArgumentParser( + description="Aggregate benchmark run results into summary statistics" + ) + parser.add_argument( + "benchmark_dir", + type=Path, + help="Path to the benchmark directory" + ) + parser.add_argument( + "--skill-name", + default="", + help="Name of the skill being benchmarked" + ) + parser.add_argument( + "--skill-path", + default="", + help="Path to the skill being benchmarked" + ) + parser.add_argument( + "--output", "-o", + type=Path, + help="Output path for benchmark.json (default: /benchmark.json)" + ) + + args = parser.parse_args() + + if not args.benchmark_dir.exists(): + print(f"Directory not found: {args.benchmark_dir}") + sys.exit(1) + + # Generate benchmark + benchmark = generate_benchmark(args.benchmark_dir, args.skill_name, args.skill_path) + + # Determine output paths + output_json = args.output or (args.benchmark_dir / "benchmark.json") + output_md = output_json.with_suffix(".md") + + # Write benchmark.json + with open(output_json, "w") as f: + json.dump(benchmark, f, indent=2) + print(f"Generated: {output_json}") + + # Write benchmark.md + markdown = generate_markdown(benchmark) + with open(output_md, "w") as f: + f.write(markdown) + print(f"Generated: {output_md}") + + # Print summary + run_summary = benchmark["run_summary"] + configs = [k for k in run_summary if k != "delta"] + delta = run_summary.get("delta", {}) + + print(f"\nSummary:") + for config in configs: + pr = run_summary[config]["pass_rate"]["mean"] + label = config.replace("_", " ").title() + print(f" {label}: {pr*100:.1f}% pass rate") + print(f" Delta: {delta.get('pass_rate', '—')}") + + +if __name__ == "__main__": + main() diff --git a/skills/skill-creator/scripts/generate_report.py b/skills/skill-creator/scripts/generate_report.py new file mode 100755 index 00000000..959e30a0 --- /dev/null +++ b/skills/skill-creator/scripts/generate_report.py @@ -0,0 +1,326 @@ +#!/usr/bin/env python3 +"""Generate an HTML report from run_loop.py output. + +Takes the JSON output from run_loop.py and generates a visual HTML report +showing each description attempt with check/x for each test case. +Distinguishes between train and test queries. +""" + +import argparse +import html +import json +import sys +from pathlib import Path + + +def generate_html(data: dict, auto_refresh: bool = False, skill_name: str = "") -> str: + """Generate HTML report from loop output data. If auto_refresh is True, adds a meta refresh tag.""" + history = data.get("history", []) + holdout = data.get("holdout", 0) + title_prefix = html.escape(skill_name + " \u2014 ") if skill_name else "" + + # Get all unique queries from train and test sets, with should_trigger info + train_queries: list[dict] = [] + test_queries: list[dict] = [] + if history: + for r in history[0].get("train_results", history[0].get("results", [])): + train_queries.append({"query": r["query"], "should_trigger": r.get("should_trigger", True)}) + if history[0].get("test_results"): + for r in history[0].get("test_results", []): + test_queries.append({"query": r["query"], "should_trigger": r.get("should_trigger", True)}) + + refresh_tag = ' \n' if auto_refresh else "" + + html_parts = [""" + + + +""" + refresh_tag + """ """ + title_prefix + """Skill Description Optimization + + + + + + +

""" + title_prefix + """Skill Description Optimization

+
+ Optimizing your skill's description. This page updates automatically as Claude tests different versions of your skill's description. Each row is an iteration — a new description attempt. The columns show test queries: green checkmarks mean the skill triggered correctly (or correctly didn't trigger), red crosses mean it got it wrong. The "Train" score shows performance on queries used to improve the description; the "Test" score shows performance on held-out queries the optimizer hasn't seen. When it's done, Claude will apply the best-performing description to your skill. +
+"""] + + # Summary section + best_test_score = data.get('best_test_score') + best_train_score = data.get('best_train_score') + html_parts.append(f""" +
+

Original: {html.escape(data.get('original_description', 'N/A'))}

+

Best: {html.escape(data.get('best_description', 'N/A'))}

+

Best Score: {data.get('best_score', 'N/A')} {'(test)' if best_test_score else '(train)'}

+

Iterations: {data.get('iterations_run', 0)} | Train: {data.get('train_size', '?')} | Test: {data.get('test_size', '?')}

+
+""") + + # Legend + html_parts.append(""" +
+ Query columns: + Should trigger + Should NOT trigger + Train + Test +
+""") + + # Table header + html_parts.append(""" +
+ + + + + + + +""") + + # Add column headers for train queries + for qinfo in train_queries: + polarity = "positive-col" if qinfo["should_trigger"] else "negative-col" + html_parts.append(f' \n') + + # Add column headers for test queries (different color) + for qinfo in test_queries: + polarity = "positive-col" if qinfo["should_trigger"] else "negative-col" + html_parts.append(f' \n') + + html_parts.append(""" + + +""") + + # Find best iteration for highlighting + if test_queries: + best_iter = max(history, key=lambda h: h.get("test_passed") or 0).get("iteration") + else: + best_iter = max(history, key=lambda h: h.get("train_passed", h.get("passed", 0))).get("iteration") + + # Add rows for each iteration + for h in history: + iteration = h.get("iteration", "?") + train_passed = h.get("train_passed", h.get("passed", 0)) + train_total = h.get("train_total", h.get("total", 0)) + test_passed = h.get("test_passed") + test_total = h.get("test_total") + description = h.get("description", "") + train_results = h.get("train_results", h.get("results", [])) + test_results = h.get("test_results", []) + + # Create lookups for results by query + train_by_query = {r["query"]: r for r in train_results} + test_by_query = {r["query"]: r for r in test_results} if test_results else {} + + # Compute aggregate correct/total runs across all retries + def aggregate_runs(results: list[dict]) -> tuple[int, int]: + correct = 0 + total = 0 + for r in results: + runs = r.get("runs", 0) + triggers = r.get("triggers", 0) + total += runs + if r.get("should_trigger", True): + correct += triggers + else: + correct += runs - triggers + return correct, total + + train_correct, train_runs = aggregate_runs(train_results) + test_correct, test_runs = aggregate_runs(test_results) + + # Determine score classes + def score_class(correct: int, total: int) -> str: + if total > 0: + ratio = correct / total + if ratio >= 0.8: + return "score-good" + elif ratio >= 0.5: + return "score-ok" + return "score-bad" + + train_class = score_class(train_correct, train_runs) + test_class = score_class(test_correct, test_runs) + + row_class = "best-row" if iteration == best_iter else "" + + html_parts.append(f""" + + + + +""") + + # Add result for each train query + for qinfo in train_queries: + r = train_by_query.get(qinfo["query"], {}) + did_pass = r.get("pass", False) + triggers = r.get("triggers", 0) + runs = r.get("runs", 0) + + icon = "✓" if did_pass else "✗" + css_class = "pass" if did_pass else "fail" + + html_parts.append(f' \n') + + # Add result for each test query (with different background) + for qinfo in test_queries: + r = test_by_query.get(qinfo["query"], {}) + did_pass = r.get("pass", False) + triggers = r.get("triggers", 0) + runs = r.get("runs", 0) + + icon = "✓" if did_pass else "✗" + css_class = "pass" if did_pass else "fail" + + html_parts.append(f' \n') + + html_parts.append(" \n") + + html_parts.append(""" +
IterTrainTestDescription{html.escape(qinfo["query"])}{html.escape(qinfo["query"])}
{iteration}{train_correct}/{train_runs}{test_correct}/{test_runs}{html.escape(description)}{icon}{triggers}/{runs}{icon}{triggers}/{runs}
+
+""") + + html_parts.append(""" + + +""") + + return "".join(html_parts) + + +def main(): + parser = argparse.ArgumentParser(description="Generate HTML report from run_loop output") + parser.add_argument("input", help="Path to JSON output from run_loop.py (or - for stdin)") + parser.add_argument("-o", "--output", default=None, help="Output HTML file (default: stdout)") + parser.add_argument("--skill-name", default="", help="Skill name to include in the report title") + args = parser.parse_args() + + if args.input == "-": + data = json.load(sys.stdin) + else: + data = json.loads(Path(args.input).read_text()) + + html_output = generate_html(data, skill_name=args.skill_name) + + if args.output: + Path(args.output).write_text(html_output) + print(f"Report written to {args.output}", file=sys.stderr) + else: + print(html_output) + + +if __name__ == "__main__": + main() diff --git a/skills/skill-creator/scripts/improve_description.py b/skills/skill-creator/scripts/improve_description.py new file mode 100755 index 00000000..06bcec76 --- /dev/null +++ b/skills/skill-creator/scripts/improve_description.py @@ -0,0 +1,247 @@ +#!/usr/bin/env python3 +"""Improve a skill description based on eval results. + +Takes eval results (from run_eval.py) and generates an improved description +by calling `claude -p` as a subprocess (same auth pattern as run_eval.py — +uses the session's Claude Code auth, no separate ANTHROPIC_API_KEY needed). +""" + +import argparse +import json +import os +import re +import subprocess +import sys +from pathlib import Path + +from scripts.utils import parse_skill_md + + +def _call_claude(prompt: str, model: str | None, timeout: int = 300) -> str: + """Run `claude -p` with the prompt on stdin and return the text response. + + Prompt goes over stdin (not argv) because it embeds the full SKILL.md + body and can easily exceed comfortable argv length. + """ + cmd = ["claude", "-p", "--output-format", "text"] + if model: + cmd.extend(["--model", model]) + + # Remove CLAUDECODE env var to allow nesting claude -p inside a + # Claude Code session. The guard is for interactive terminal conflicts; + # programmatic subprocess usage is safe. Same pattern as run_eval.py. + env = {k: v for k, v in os.environ.items() if k != "CLAUDECODE"} + + result = subprocess.run( + cmd, + input=prompt, + capture_output=True, + text=True, + env=env, + timeout=timeout, + ) + if result.returncode != 0: + raise RuntimeError( + f"claude -p exited {result.returncode}\nstderr: {result.stderr}" + ) + return result.stdout + + +def improve_description( + skill_name: str, + skill_content: str, + current_description: str, + eval_results: dict, + history: list[dict], + model: str, + test_results: dict | None = None, + log_dir: Path | None = None, + iteration: int | None = None, +) -> str: + """Call Claude to improve the description based on eval results.""" + failed_triggers = [ + r for r in eval_results["results"] + if r["should_trigger"] and not r["pass"] + ] + false_triggers = [ + r for r in eval_results["results"] + if not r["should_trigger"] and not r["pass"] + ] + + # Build scores summary + train_score = f"{eval_results['summary']['passed']}/{eval_results['summary']['total']}" + if test_results: + test_score = f"{test_results['summary']['passed']}/{test_results['summary']['total']}" + scores_summary = f"Train: {train_score}, Test: {test_score}" + else: + scores_summary = f"Train: {train_score}" + + prompt = f"""You are optimizing a skill description for a Claude Code skill called "{skill_name}". A "skill" is sort of like a prompt, but with progressive disclosure -- there's a title and description that Claude sees when deciding whether to use the skill, and then if it does use the skill, it reads the .md file which has lots more details and potentially links to other resources in the skill folder like helper files and scripts and additional documentation or examples. + +The description appears in Claude's "available_skills" list. When a user sends a query, Claude decides whether to invoke the skill based solely on the title and on this description. Your goal is to write a description that triggers for relevant queries, and doesn't trigger for irrelevant ones. + +Here's the current description: + +"{current_description}" + + +Current scores ({scores_summary}): + +""" + if failed_triggers: + prompt += "FAILED TO TRIGGER (should have triggered but didn't):\n" + for r in failed_triggers: + prompt += f' - "{r["query"]}" (triggered {r["triggers"]}/{r["runs"]} times)\n' + prompt += "\n" + + if false_triggers: + prompt += "FALSE TRIGGERS (triggered but shouldn't have):\n" + for r in false_triggers: + prompt += f' - "{r["query"]}" (triggered {r["triggers"]}/{r["runs"]} times)\n' + prompt += "\n" + + if history: + prompt += "PREVIOUS ATTEMPTS (do NOT repeat these — try something structurally different):\n\n" + for h in history: + train_s = f"{h.get('train_passed', h.get('passed', 0))}/{h.get('train_total', h.get('total', 0))}" + test_s = f"{h.get('test_passed', '?')}/{h.get('test_total', '?')}" if h.get('test_passed') is not None else None + score_str = f"train={train_s}" + (f", test={test_s}" if test_s else "") + prompt += f'\n' + prompt += f'Description: "{h["description"]}"\n' + if "results" in h: + prompt += "Train results:\n" + for r in h["results"]: + status = "PASS" if r["pass"] else "FAIL" + prompt += f' [{status}] "{r["query"][:80]}" (triggered {r["triggers"]}/{r["runs"]})\n' + if h.get("note"): + prompt += f'Note: {h["note"]}\n' + prompt += "\n\n" + + prompt += f""" + +Skill content (for context on what the skill does): + +{skill_content} + + +Based on the failures, write a new and improved description that is more likely to trigger correctly. When I say "based on the failures", it's a bit of a tricky line to walk because we don't want to overfit to the specific cases you're seeing. So what I DON'T want you to do is produce an ever-expanding list of specific queries that this skill should or shouldn't trigger for. Instead, try to generalize from the failures to broader categories of user intent and situations where this skill would be useful or not useful. The reason for this is twofold: + +1. Avoid overfitting +2. The list might get loooong and it's injected into ALL queries and there might be a lot of skills, so we don't want to blow too much space on any given description. + +Concretely, your description should not be more than about 100-200 words, even if that comes at the cost of accuracy. There is a hard limit of 1024 characters — descriptions over that will be truncated, so stay comfortably under it. + +Here are some tips that we've found to work well in writing these descriptions: +- The skill should be phrased in the imperative -- "Use this skill for" rather than "this skill does" +- The skill description should focus on the user's intent, what they are trying to achieve, vs. the implementation details of how the skill works. +- The description competes with other skills for Claude's attention — make it distinctive and immediately recognizable. +- If you're getting lots of failures after repeated attempts, change things up. Try different sentence structures or wordings. + +I'd encourage you to be creative and mix up the style in different iterations since you'll have multiple opportunities to try different approaches and we'll just grab the highest-scoring one at the end. + +Please respond with only the new description text in tags, nothing else.""" + + text = _call_claude(prompt, model) + + match = re.search(r"(.*?)", text, re.DOTALL) + description = match.group(1).strip().strip('"') if match else text.strip().strip('"') + + transcript: dict = { + "iteration": iteration, + "prompt": prompt, + "response": text, + "parsed_description": description, + "char_count": len(description), + "over_limit": len(description) > 1024, + } + + # Safety net: the prompt already states the 1024-char hard limit, but if + # the model blew past it anyway, make one fresh single-turn call that + # quotes the too-long version and asks for a shorter rewrite. (The old + # SDK path did this as a true multi-turn; `claude -p` is one-shot, so we + # inline the prior output into the new prompt instead.) + if len(description) > 1024: + shorten_prompt = ( + f"{prompt}\n\n" + f"---\n\n" + f"A previous attempt produced this description, which at " + f"{len(description)} characters is over the 1024-character hard limit:\n\n" + f'"{description}"\n\n' + f"Rewrite it to be under 1024 characters while keeping the most " + f"important trigger words and intent coverage. Respond with only " + f"the new description in tags." + ) + shorten_text = _call_claude(shorten_prompt, model) + match = re.search(r"(.*?)", shorten_text, re.DOTALL) + shortened = match.group(1).strip().strip('"') if match else shorten_text.strip().strip('"') + + transcript["rewrite_prompt"] = shorten_prompt + transcript["rewrite_response"] = shorten_text + transcript["rewrite_description"] = shortened + transcript["rewrite_char_count"] = len(shortened) + description = shortened + + transcript["final_description"] = description + + if log_dir: + log_dir.mkdir(parents=True, exist_ok=True) + log_file = log_dir / f"improve_iter_{iteration or 'unknown'}.json" + log_file.write_text(json.dumps(transcript, indent=2)) + + return description + + +def main(): + parser = argparse.ArgumentParser(description="Improve a skill description based on eval results") + parser.add_argument("--eval-results", required=True, help="Path to eval results JSON (from run_eval.py)") + parser.add_argument("--skill-path", required=True, help="Path to skill directory") + parser.add_argument("--history", default=None, help="Path to history JSON (previous attempts)") + parser.add_argument("--model", required=True, help="Model for improvement") + parser.add_argument("--verbose", action="store_true", help="Print thinking to stderr") + args = parser.parse_args() + + skill_path = Path(args.skill_path) + if not (skill_path / "SKILL.md").exists(): + print(f"Error: No SKILL.md found at {skill_path}", file=sys.stderr) + sys.exit(1) + + eval_results = json.loads(Path(args.eval_results).read_text()) + history = [] + if args.history: + history = json.loads(Path(args.history).read_text()) + + name, _, content = parse_skill_md(skill_path) + current_description = eval_results["description"] + + if args.verbose: + print(f"Current: {current_description}", file=sys.stderr) + print(f"Score: {eval_results['summary']['passed']}/{eval_results['summary']['total']}", file=sys.stderr) + + new_description = improve_description( + skill_name=name, + skill_content=content, + current_description=current_description, + eval_results=eval_results, + history=history, + model=args.model, + ) + + if args.verbose: + print(f"Improved: {new_description}", file=sys.stderr) + + # Output as JSON with both the new description and updated history + output = { + "description": new_description, + "history": history + [{ + "description": current_description, + "passed": eval_results["summary"]["passed"], + "failed": eval_results["summary"]["failed"], + "total": eval_results["summary"]["total"], + "results": eval_results["results"], + }], + } + print(json.dumps(output, indent=2)) + + +if __name__ == "__main__": + main() diff --git a/skills/skill-creator/scripts/package_skill.py b/skills/skill-creator/scripts/package_skill.py new file mode 100755 index 00000000..f48eac44 --- /dev/null +++ b/skills/skill-creator/scripts/package_skill.py @@ -0,0 +1,136 @@ +#!/usr/bin/env python3 +""" +Skill Packager - Creates a distributable .skill file of a skill folder + +Usage: + python utils/package_skill.py [output-directory] + +Example: + python utils/package_skill.py skills/public/my-skill + python utils/package_skill.py skills/public/my-skill ./dist +""" + +import fnmatch +import sys +import zipfile +from pathlib import Path +from scripts.quick_validate import validate_skill + +# Patterns to exclude when packaging skills. +EXCLUDE_DIRS = {"__pycache__", "node_modules"} +EXCLUDE_GLOBS = {"*.pyc"} +EXCLUDE_FILES = {".DS_Store"} +# Directories excluded only at the skill root (not when nested deeper). +ROOT_EXCLUDE_DIRS = {"evals"} + + +def should_exclude(rel_path: Path) -> bool: + """Check if a path should be excluded from packaging.""" + parts = rel_path.parts + if any(part in EXCLUDE_DIRS for part in parts): + return True + # rel_path is relative to skill_path.parent, so parts[0] is the skill + # folder name and parts[1] (if present) is the first subdir. + if len(parts) > 1 and parts[1] in ROOT_EXCLUDE_DIRS: + return True + name = rel_path.name + if name in EXCLUDE_FILES: + return True + return any(fnmatch.fnmatch(name, pat) for pat in EXCLUDE_GLOBS) + + +def package_skill(skill_path, output_dir=None): + """ + Package a skill folder into a .skill file. + + Args: + skill_path: Path to the skill folder + output_dir: Optional output directory for the .skill file (defaults to current directory) + + Returns: + Path to the created .skill file, or None if error + """ + skill_path = Path(skill_path).resolve() + + # Validate skill folder exists + if not skill_path.exists(): + print(f"❌ Error: Skill folder not found: {skill_path}") + return None + + if not skill_path.is_dir(): + print(f"❌ Error: Path is not a directory: {skill_path}") + return None + + # Validate SKILL.md exists + skill_md = skill_path / "SKILL.md" + if not skill_md.exists(): + print(f"❌ Error: SKILL.md not found in {skill_path}") + return None + + # Run validation before packaging + print("🔍 Validating skill...") + valid, message = validate_skill(skill_path) + if not valid: + print(f"❌ Validation failed: {message}") + print(" Please fix the validation errors before packaging.") + return None + print(f"✅ {message}\n") + + # Determine output location + skill_name = skill_path.name + if output_dir: + output_path = Path(output_dir).resolve() + output_path.mkdir(parents=True, exist_ok=True) + else: + output_path = Path.cwd() + + skill_filename = output_path / f"{skill_name}.skill" + + # Create the .skill file (zip format) + try: + with zipfile.ZipFile(skill_filename, 'w', zipfile.ZIP_DEFLATED) as zipf: + # Walk through the skill directory, excluding build artifacts + for file_path in skill_path.rglob('*'): + if not file_path.is_file(): + continue + arcname = file_path.relative_to(skill_path.parent) + if should_exclude(arcname): + print(f" Skipped: {arcname}") + continue + zipf.write(file_path, arcname) + print(f" Added: {arcname}") + + print(f"\n✅ Successfully packaged skill to: {skill_filename}") + return skill_filename + + except Exception as e: + print(f"❌ Error creating .skill file: {e}") + return None + + +def main(): + if len(sys.argv) < 2: + print("Usage: python utils/package_skill.py [output-directory]") + print("\nExample:") + print(" python utils/package_skill.py skills/public/my-skill") + print(" python utils/package_skill.py skills/public/my-skill ./dist") + sys.exit(1) + + skill_path = sys.argv[1] + output_dir = sys.argv[2] if len(sys.argv) > 2 else None + + print(f"📦 Packaging skill: {skill_path}") + if output_dir: + print(f" Output directory: {output_dir}") + print() + + result = package_skill(skill_path, output_dir) + + if result: + sys.exit(0) + else: + sys.exit(1) + + +if __name__ == "__main__": + main() diff --git a/skills/skill-creator/scripts/quick_validate.py b/skills/skill-creator/scripts/quick_validate.py new file mode 100755 index 00000000..ed8e1ddd --- /dev/null +++ b/skills/skill-creator/scripts/quick_validate.py @@ -0,0 +1,103 @@ +#!/usr/bin/env python3 +""" +Quick validation script for skills - minimal version +""" + +import sys +import os +import re +import yaml +from pathlib import Path + +def validate_skill(skill_path): + """Basic validation of a skill""" + skill_path = Path(skill_path) + + # Check SKILL.md exists + skill_md = skill_path / 'SKILL.md' + if not skill_md.exists(): + return False, "SKILL.md not found" + + # Read and validate frontmatter + content = skill_md.read_text() + if not content.startswith('---'): + return False, "No YAML frontmatter found" + + # Extract frontmatter + match = re.match(r'^---\n(.*?)\n---', content, re.DOTALL) + if not match: + return False, "Invalid frontmatter format" + + frontmatter_text = match.group(1) + + # Parse YAML frontmatter + try: + frontmatter = yaml.safe_load(frontmatter_text) + if not isinstance(frontmatter, dict): + return False, "Frontmatter must be a YAML dictionary" + except yaml.YAMLError as e: + return False, f"Invalid YAML in frontmatter: {e}" + + # Define allowed properties + ALLOWED_PROPERTIES = {'name', 'description', 'license', 'allowed-tools', 'metadata', 'compatibility'} + + # Check for unexpected properties (excluding nested keys under metadata) + unexpected_keys = set(frontmatter.keys()) - ALLOWED_PROPERTIES + if unexpected_keys: + return False, ( + f"Unexpected key(s) in SKILL.md frontmatter: {', '.join(sorted(unexpected_keys))}. " + f"Allowed properties are: {', '.join(sorted(ALLOWED_PROPERTIES))}" + ) + + # Check required fields + if 'name' not in frontmatter: + return False, "Missing 'name' in frontmatter" + if 'description' not in frontmatter: + return False, "Missing 'description' in frontmatter" + + # Extract name for validation + name = frontmatter.get('name', '') + if not isinstance(name, str): + return False, f"Name must be a string, got {type(name).__name__}" + name = name.strip() + if name: + # Check naming convention (kebab-case: lowercase with hyphens) + if not re.match(r'^[a-z0-9-]+$', name): + return False, f"Name '{name}' should be kebab-case (lowercase letters, digits, and hyphens only)" + if name.startswith('-') or name.endswith('-') or '--' in name: + return False, f"Name '{name}' cannot start/end with hyphen or contain consecutive hyphens" + # Check name length (max 64 characters per spec) + if len(name) > 64: + return False, f"Name is too long ({len(name)} characters). Maximum is 64 characters." + + # Extract and validate description + description = frontmatter.get('description', '') + if not isinstance(description, str): + return False, f"Description must be a string, got {type(description).__name__}" + description = description.strip() + if description: + # Check for angle brackets + if '<' in description or '>' in description: + return False, "Description cannot contain angle brackets (< or >)" + # Check description length (max 1024 characters per spec) + if len(description) > 1024: + return False, f"Description is too long ({len(description)} characters). Maximum is 1024 characters." + + # Validate compatibility field if present (optional) + compatibility = frontmatter.get('compatibility', '') + if compatibility: + if not isinstance(compatibility, str): + return False, f"Compatibility must be a string, got {type(compatibility).__name__}" + if len(compatibility) > 500: + return False, f"Compatibility is too long ({len(compatibility)} characters). Maximum is 500 characters." + + return True, "Skill is valid!" + +if __name__ == "__main__": + if len(sys.argv) != 2: + print("Usage: python quick_validate.py ") + sys.exit(1) + + valid, message = validate_skill(sys.argv[1]) + print(message) + sys.exit(0 if valid else 1) \ No newline at end of file diff --git a/skills/skill-creator/scripts/run_eval.py b/skills/skill-creator/scripts/run_eval.py new file mode 100755 index 00000000..e58c70be --- /dev/null +++ b/skills/skill-creator/scripts/run_eval.py @@ -0,0 +1,310 @@ +#!/usr/bin/env python3 +"""Run trigger evaluation for a skill description. + +Tests whether a skill's description causes Claude to trigger (read the skill) +for a set of queries. Outputs results as JSON. +""" + +import argparse +import json +import os +import select +import subprocess +import sys +import time +import uuid +from concurrent.futures import ProcessPoolExecutor, as_completed +from pathlib import Path + +from scripts.utils import parse_skill_md + + +def find_project_root() -> Path: + """Find the project root by walking up from cwd looking for .claude/. + + Mimics how Claude Code discovers its project root, so the command file + we create ends up where claude -p will look for it. + """ + current = Path.cwd() + for parent in [current, *current.parents]: + if (parent / ".claude").is_dir(): + return parent + return current + + +def run_single_query( + query: str, + skill_name: str, + skill_description: str, + timeout: int, + project_root: str, + model: str | None = None, +) -> bool: + """Run a single query and return whether the skill was triggered. + + Creates a command file in .claude/commands/ so it appears in Claude's + available_skills list, then runs `claude -p` with the raw query. + Uses --include-partial-messages to detect triggering early from + stream events (content_block_start) rather than waiting for the + full assistant message, which only arrives after tool execution. + """ + unique_id = uuid.uuid4().hex[:8] + clean_name = f"{skill_name}-skill-{unique_id}" + project_commands_dir = Path(project_root) / ".claude" / "commands" + command_file = project_commands_dir / f"{clean_name}.md" + + try: + project_commands_dir.mkdir(parents=True, exist_ok=True) + # Use YAML block scalar to avoid breaking on quotes in description + indented_desc = "\n ".join(skill_description.split("\n")) + command_content = ( + f"---\n" + f"description: |\n" + f" {indented_desc}\n" + f"---\n\n" + f"# {skill_name}\n\n" + f"This skill handles: {skill_description}\n" + ) + command_file.write_text(command_content) + + cmd = [ + "claude", + "-p", query, + "--output-format", "stream-json", + "--verbose", + "--include-partial-messages", + ] + if model: + cmd.extend(["--model", model]) + + # Remove CLAUDECODE env var to allow nesting claude -p inside a + # Claude Code session. The guard is for interactive terminal conflicts; + # programmatic subprocess usage is safe. + env = {k: v for k, v in os.environ.items() if k != "CLAUDECODE"} + + process = subprocess.Popen( + cmd, + stdout=subprocess.PIPE, + stderr=subprocess.DEVNULL, + cwd=project_root, + env=env, + ) + + triggered = False + start_time = time.time() + buffer = "" + # Track state for stream event detection + pending_tool_name = None + accumulated_json = "" + + try: + while time.time() - start_time < timeout: + if process.poll() is not None: + remaining = process.stdout.read() + if remaining: + buffer += remaining.decode("utf-8", errors="replace") + break + + ready, _, _ = select.select([process.stdout], [], [], 1.0) + if not ready: + continue + + chunk = os.read(process.stdout.fileno(), 8192) + if not chunk: + break + buffer += chunk.decode("utf-8", errors="replace") + + while "\n" in buffer: + line, buffer = buffer.split("\n", 1) + line = line.strip() + if not line: + continue + + try: + event = json.loads(line) + except json.JSONDecodeError: + continue + + # Early detection via stream events + if event.get("type") == "stream_event": + se = event.get("event", {}) + se_type = se.get("type", "") + + if se_type == "content_block_start": + cb = se.get("content_block", {}) + if cb.get("type") == "tool_use": + tool_name = cb.get("name", "") + if tool_name in ("Skill", "Read"): + pending_tool_name = tool_name + accumulated_json = "" + else: + return False + + elif se_type == "content_block_delta" and pending_tool_name: + delta = se.get("delta", {}) + if delta.get("type") == "input_json_delta": + accumulated_json += delta.get("partial_json", "") + if clean_name in accumulated_json: + return True + + elif se_type in ("content_block_stop", "message_stop"): + if pending_tool_name: + return clean_name in accumulated_json + if se_type == "message_stop": + return False + + # Fallback: full assistant message + elif event.get("type") == "assistant": + message = event.get("message", {}) + for content_item in message.get("content", []): + if content_item.get("type") != "tool_use": + continue + tool_name = content_item.get("name", "") + tool_input = content_item.get("input", {}) + if tool_name == "Skill" and clean_name in tool_input.get("skill", ""): + triggered = True + elif tool_name == "Read" and clean_name in tool_input.get("file_path", ""): + triggered = True + return triggered + + elif event.get("type") == "result": + return triggered + finally: + # Clean up process on any exit path (return, exception, timeout) + if process.poll() is None: + process.kill() + process.wait() + + return triggered + finally: + if command_file.exists(): + command_file.unlink() + + +def run_eval( + eval_set: list[dict], + skill_name: str, + description: str, + num_workers: int, + timeout: int, + project_root: Path, + runs_per_query: int = 1, + trigger_threshold: float = 0.5, + model: str | None = None, +) -> dict: + """Run the full eval set and return results.""" + results = [] + + with ProcessPoolExecutor(max_workers=num_workers) as executor: + future_to_info = {} + for item in eval_set: + for run_idx in range(runs_per_query): + future = executor.submit( + run_single_query, + item["query"], + skill_name, + description, + timeout, + str(project_root), + model, + ) + future_to_info[future] = (item, run_idx) + + query_triggers: dict[str, list[bool]] = {} + query_items: dict[str, dict] = {} + for future in as_completed(future_to_info): + item, _ = future_to_info[future] + query = item["query"] + query_items[query] = item + if query not in query_triggers: + query_triggers[query] = [] + try: + query_triggers[query].append(future.result()) + except Exception as e: + print(f"Warning: query failed: {e}", file=sys.stderr) + query_triggers[query].append(False) + + for query, triggers in query_triggers.items(): + item = query_items[query] + trigger_rate = sum(triggers) / len(triggers) + should_trigger = item["should_trigger"] + if should_trigger: + did_pass = trigger_rate >= trigger_threshold + else: + did_pass = trigger_rate < trigger_threshold + results.append({ + "query": query, + "should_trigger": should_trigger, + "trigger_rate": trigger_rate, + "triggers": sum(triggers), + "runs": len(triggers), + "pass": did_pass, + }) + + passed = sum(1 for r in results if r["pass"]) + total = len(results) + + return { + "skill_name": skill_name, + "description": description, + "results": results, + "summary": { + "total": total, + "passed": passed, + "failed": total - passed, + }, + } + + +def main(): + parser = argparse.ArgumentParser(description="Run trigger evaluation for a skill description") + parser.add_argument("--eval-set", required=True, help="Path to eval set JSON file") + parser.add_argument("--skill-path", required=True, help="Path to skill directory") + parser.add_argument("--description", default=None, help="Override description to test") + parser.add_argument("--num-workers", type=int, default=10, help="Number of parallel workers") + parser.add_argument("--timeout", type=int, default=30, help="Timeout per query in seconds") + parser.add_argument("--runs-per-query", type=int, default=3, help="Number of runs per query") + parser.add_argument("--trigger-threshold", type=float, default=0.5, help="Trigger rate threshold") + parser.add_argument("--model", default=None, help="Model to use for claude -p (default: user's configured model)") + parser.add_argument("--verbose", action="store_true", help="Print progress to stderr") + args = parser.parse_args() + + eval_set = json.loads(Path(args.eval_set).read_text()) + skill_path = Path(args.skill_path) + + if not (skill_path / "SKILL.md").exists(): + print(f"Error: No SKILL.md found at {skill_path}", file=sys.stderr) + sys.exit(1) + + name, original_description, content = parse_skill_md(skill_path) + description = args.description or original_description + project_root = find_project_root() + + if args.verbose: + print(f"Evaluating: {description}", file=sys.stderr) + + output = run_eval( + eval_set=eval_set, + skill_name=name, + description=description, + num_workers=args.num_workers, + timeout=args.timeout, + project_root=project_root, + runs_per_query=args.runs_per_query, + trigger_threshold=args.trigger_threshold, + model=args.model, + ) + + if args.verbose: + summary = output["summary"] + print(f"Results: {summary['passed']}/{summary['total']} passed", file=sys.stderr) + for r in output["results"]: + status = "PASS" if r["pass"] else "FAIL" + rate_str = f"{r['triggers']}/{r['runs']}" + print(f" [{status}] rate={rate_str} expected={r['should_trigger']}: {r['query'][:70]}", file=sys.stderr) + + print(json.dumps(output, indent=2)) + + +if __name__ == "__main__": + main() diff --git a/skills/skill-creator/scripts/run_loop.py b/skills/skill-creator/scripts/run_loop.py new file mode 100755 index 00000000..30a263d6 --- /dev/null +++ b/skills/skill-creator/scripts/run_loop.py @@ -0,0 +1,328 @@ +#!/usr/bin/env python3 +"""Run the eval + improve loop until all pass or max iterations reached. + +Combines run_eval.py and improve_description.py in a loop, tracking history +and returning the best description found. Supports train/test split to prevent +overfitting. +""" + +import argparse +import json +import random +import sys +import tempfile +import time +import webbrowser +from pathlib import Path + +from scripts.generate_report import generate_html +from scripts.improve_description import improve_description +from scripts.run_eval import find_project_root, run_eval +from scripts.utils import parse_skill_md + + +def split_eval_set(eval_set: list[dict], holdout: float, seed: int = 42) -> tuple[list[dict], list[dict]]: + """Split eval set into train and test sets, stratified by should_trigger.""" + random.seed(seed) + + # Separate by should_trigger + trigger = [e for e in eval_set if e["should_trigger"]] + no_trigger = [e for e in eval_set if not e["should_trigger"]] + + # Shuffle each group + random.shuffle(trigger) + random.shuffle(no_trigger) + + # Calculate split points + n_trigger_test = max(1, int(len(trigger) * holdout)) + n_no_trigger_test = max(1, int(len(no_trigger) * holdout)) + + # Split + test_set = trigger[:n_trigger_test] + no_trigger[:n_no_trigger_test] + train_set = trigger[n_trigger_test:] + no_trigger[n_no_trigger_test:] + + return train_set, test_set + + +def run_loop( + eval_set: list[dict], + skill_path: Path, + description_override: str | None, + num_workers: int, + timeout: int, + max_iterations: int, + runs_per_query: int, + trigger_threshold: float, + holdout: float, + model: str, + verbose: bool, + live_report_path: Path | None = None, + log_dir: Path | None = None, +) -> dict: + """Run the eval + improvement loop.""" + project_root = find_project_root() + name, original_description, content = parse_skill_md(skill_path) + current_description = description_override or original_description + + # Split into train/test if holdout > 0 + if holdout > 0: + train_set, test_set = split_eval_set(eval_set, holdout) + if verbose: + print(f"Split: {len(train_set)} train, {len(test_set)} test (holdout={holdout})", file=sys.stderr) + else: + train_set = eval_set + test_set = [] + + history = [] + exit_reason = "unknown" + + for iteration in range(1, max_iterations + 1): + if verbose: + print(f"\n{'='*60}", file=sys.stderr) + print(f"Iteration {iteration}/{max_iterations}", file=sys.stderr) + print(f"Description: {current_description}", file=sys.stderr) + print(f"{'='*60}", file=sys.stderr) + + # Evaluate train + test together in one batch for parallelism + all_queries = train_set + test_set + t0 = time.time() + all_results = run_eval( + eval_set=all_queries, + skill_name=name, + description=current_description, + num_workers=num_workers, + timeout=timeout, + project_root=project_root, + runs_per_query=runs_per_query, + trigger_threshold=trigger_threshold, + model=model, + ) + eval_elapsed = time.time() - t0 + + # Split results back into train/test by matching queries + train_queries_set = {q["query"] for q in train_set} + train_result_list = [r for r in all_results["results"] if r["query"] in train_queries_set] + test_result_list = [r for r in all_results["results"] if r["query"] not in train_queries_set] + + train_passed = sum(1 for r in train_result_list if r["pass"]) + train_total = len(train_result_list) + train_summary = {"passed": train_passed, "failed": train_total - train_passed, "total": train_total} + train_results = {"results": train_result_list, "summary": train_summary} + + if test_set: + test_passed = sum(1 for r in test_result_list if r["pass"]) + test_total = len(test_result_list) + test_summary = {"passed": test_passed, "failed": test_total - test_passed, "total": test_total} + test_results = {"results": test_result_list, "summary": test_summary} + else: + test_results = None + test_summary = None + + history.append({ + "iteration": iteration, + "description": current_description, + "train_passed": train_summary["passed"], + "train_failed": train_summary["failed"], + "train_total": train_summary["total"], + "train_results": train_results["results"], + "test_passed": test_summary["passed"] if test_summary else None, + "test_failed": test_summary["failed"] if test_summary else None, + "test_total": test_summary["total"] if test_summary else None, + "test_results": test_results["results"] if test_results else None, + # For backward compat with report generator + "passed": train_summary["passed"], + "failed": train_summary["failed"], + "total": train_summary["total"], + "results": train_results["results"], + }) + + # Write live report if path provided + if live_report_path: + partial_output = { + "original_description": original_description, + "best_description": current_description, + "best_score": "in progress", + "iterations_run": len(history), + "holdout": holdout, + "train_size": len(train_set), + "test_size": len(test_set), + "history": history, + } + live_report_path.write_text(generate_html(partial_output, auto_refresh=True, skill_name=name)) + + if verbose: + def print_eval_stats(label, results, elapsed): + pos = [r for r in results if r["should_trigger"]] + neg = [r for r in results if not r["should_trigger"]] + tp = sum(r["triggers"] for r in pos) + pos_runs = sum(r["runs"] for r in pos) + fn = pos_runs - tp + fp = sum(r["triggers"] for r in neg) + neg_runs = sum(r["runs"] for r in neg) + tn = neg_runs - fp + total = tp + tn + fp + fn + precision = tp / (tp + fp) if (tp + fp) > 0 else 1.0 + recall = tp / (tp + fn) if (tp + fn) > 0 else 1.0 + accuracy = (tp + tn) / total if total > 0 else 0.0 + print(f"{label}: {tp+tn}/{total} correct, precision={precision:.0%} recall={recall:.0%} accuracy={accuracy:.0%} ({elapsed:.1f}s)", file=sys.stderr) + for r in results: + status = "PASS" if r["pass"] else "FAIL" + rate_str = f"{r['triggers']}/{r['runs']}" + print(f" [{status}] rate={rate_str} expected={r['should_trigger']}: {r['query'][:60]}", file=sys.stderr) + + print_eval_stats("Train", train_results["results"], eval_elapsed) + if test_summary: + print_eval_stats("Test ", test_results["results"], 0) + + if train_summary["failed"] == 0: + exit_reason = f"all_passed (iteration {iteration})" + if verbose: + print(f"\nAll train queries passed on iteration {iteration}!", file=sys.stderr) + break + + if iteration == max_iterations: + exit_reason = f"max_iterations ({max_iterations})" + if verbose: + print(f"\nMax iterations reached ({max_iterations}).", file=sys.stderr) + break + + # Improve the description based on train results + if verbose: + print(f"\nImproving description...", file=sys.stderr) + + t0 = time.time() + # Strip test scores from history so improvement model can't see them + blinded_history = [ + {k: v for k, v in h.items() if not k.startswith("test_")} + for h in history + ] + new_description = improve_description( + skill_name=name, + skill_content=content, + current_description=current_description, + eval_results=train_results, + history=blinded_history, + model=model, + log_dir=log_dir, + iteration=iteration, + ) + improve_elapsed = time.time() - t0 + + if verbose: + print(f"Proposed ({improve_elapsed:.1f}s): {new_description}", file=sys.stderr) + + current_description = new_description + + # Find the best iteration by TEST score (or train if no test set) + if test_set: + best = max(history, key=lambda h: h["test_passed"] or 0) + best_score = f"{best['test_passed']}/{best['test_total']}" + else: + best = max(history, key=lambda h: h["train_passed"]) + best_score = f"{best['train_passed']}/{best['train_total']}" + + if verbose: + print(f"\nExit reason: {exit_reason}", file=sys.stderr) + print(f"Best score: {best_score} (iteration {best['iteration']})", file=sys.stderr) + + return { + "exit_reason": exit_reason, + "original_description": original_description, + "best_description": best["description"], + "best_score": best_score, + "best_train_score": f"{best['train_passed']}/{best['train_total']}", + "best_test_score": f"{best['test_passed']}/{best['test_total']}" if test_set else None, + "final_description": current_description, + "iterations_run": len(history), + "holdout": holdout, + "train_size": len(train_set), + "test_size": len(test_set), + "history": history, + } + + +def main(): + parser = argparse.ArgumentParser(description="Run eval + improve loop") + parser.add_argument("--eval-set", required=True, help="Path to eval set JSON file") + parser.add_argument("--skill-path", required=True, help="Path to skill directory") + parser.add_argument("--description", default=None, help="Override starting description") + parser.add_argument("--num-workers", type=int, default=10, help="Number of parallel workers") + parser.add_argument("--timeout", type=int, default=30, help="Timeout per query in seconds") + parser.add_argument("--max-iterations", type=int, default=5, help="Max improvement iterations") + parser.add_argument("--runs-per-query", type=int, default=3, help="Number of runs per query") + parser.add_argument("--trigger-threshold", type=float, default=0.5, help="Trigger rate threshold") + parser.add_argument("--holdout", type=float, default=0.4, help="Fraction of eval set to hold out for testing (0 to disable)") + parser.add_argument("--model", required=True, help="Model for improvement") + parser.add_argument("--verbose", action="store_true", help="Print progress to stderr") + parser.add_argument("--report", default="auto", help="Generate HTML report at this path (default: 'auto' for temp file, 'none' to disable)") + parser.add_argument("--results-dir", default=None, help="Save all outputs (results.json, report.html, log.txt) to a timestamped subdirectory here") + args = parser.parse_args() + + eval_set = json.loads(Path(args.eval_set).read_text()) + skill_path = Path(args.skill_path) + + if not (skill_path / "SKILL.md").exists(): + print(f"Error: No SKILL.md found at {skill_path}", file=sys.stderr) + sys.exit(1) + + name, _, _ = parse_skill_md(skill_path) + + # Set up live report path + if args.report != "none": + if args.report == "auto": + timestamp = time.strftime("%Y%m%d_%H%M%S") + live_report_path = Path(tempfile.gettempdir()) / f"skill_description_report_{skill_path.name}_{timestamp}.html" + else: + live_report_path = Path(args.report) + # Open the report immediately so the user can watch + live_report_path.write_text("

Starting optimization loop...

") + webbrowser.open(str(live_report_path)) + else: + live_report_path = None + + # Determine output directory (create before run_loop so logs can be written) + if args.results_dir: + timestamp = time.strftime("%Y-%m-%d_%H%M%S") + results_dir = Path(args.results_dir) / timestamp + results_dir.mkdir(parents=True, exist_ok=True) + else: + results_dir = None + + log_dir = results_dir / "logs" if results_dir else None + + output = run_loop( + eval_set=eval_set, + skill_path=skill_path, + description_override=args.description, + num_workers=args.num_workers, + timeout=args.timeout, + max_iterations=args.max_iterations, + runs_per_query=args.runs_per_query, + trigger_threshold=args.trigger_threshold, + holdout=args.holdout, + model=args.model, + verbose=args.verbose, + live_report_path=live_report_path, + log_dir=log_dir, + ) + + # Save JSON output + json_output = json.dumps(output, indent=2) + print(json_output) + if results_dir: + (results_dir / "results.json").write_text(json_output) + + # Write final HTML report (without auto-refresh) + if live_report_path: + live_report_path.write_text(generate_html(output, auto_refresh=False, skill_name=name)) + print(f"\nReport: {live_report_path}", file=sys.stderr) + + if results_dir and live_report_path: + (results_dir / "report.html").write_text(generate_html(output, auto_refresh=False, skill_name=name)) + + if results_dir: + print(f"Results saved to: {results_dir}", file=sys.stderr) + + +if __name__ == "__main__": + main() diff --git a/skills/skill-creator/scripts/utils.py b/skills/skill-creator/scripts/utils.py new file mode 100755 index 00000000..51b6a07d --- /dev/null +++ b/skills/skill-creator/scripts/utils.py @@ -0,0 +1,47 @@ +"""Shared utilities for skill-creator scripts.""" + +from pathlib import Path + + + +def parse_skill_md(skill_path: Path) -> tuple[str, str, str]: + """Parse a SKILL.md file, returning (name, description, full_content).""" + content = (skill_path / "SKILL.md").read_text() + lines = content.split("\n") + + if lines[0].strip() != "---": + raise ValueError("SKILL.md missing frontmatter (no opening ---)") + + end_idx = None + for i, line in enumerate(lines[1:], start=1): + if line.strip() == "---": + end_idx = i + break + + if end_idx is None: + raise ValueError("SKILL.md missing frontmatter (no closing ---)") + + name = "" + description = "" + frontmatter_lines = lines[1:end_idx] + i = 0 + while i < len(frontmatter_lines): + line = frontmatter_lines[i] + if line.startswith("name:"): + name = line[len("name:"):].strip().strip('"').strip("'") + elif line.startswith("description:"): + value = line[len("description:"):].strip() + # Handle YAML multiline indicators (>, |, >-, |-) + if value in (">", "|", ">-", "|-"): + continuation_lines: list[str] = [] + i += 1 + while i < len(frontmatter_lines) and (frontmatter_lines[i].startswith(" ") or frontmatter_lines[i].startswith("\t")): + continuation_lines.append(frontmatter_lines[i].strip()) + i += 1 + description = " ".join(continuation_lines) + continue + else: + description = value.strip('"').strip("'") + i += 1 + + return name, description, content diff --git a/skills/split-screen-avatar-reel/SKILL.md b/skills/split-screen-avatar-reel/SKILL.md new file mode 100644 index 00000000..9f4b3b49 --- /dev/null +++ b/skills/split-screen-avatar-reel/SKILL.md @@ -0,0 +1,148 @@ +--- +name: split-screen-avatar-reel +description: "Build IG/TikTok/Shorts reels in the split-screen news format — b-roll filling the TOP half of a vertical 9:16 frame, a bold two-line ALL-CAPS headline straddling the seam, and Graeham's HeyGen talking-head avatar filling the BOTTOM half for the whole video. This is the layout used by hyperlocal real-estate-news accounts like @bayareawilson. Use ANY time the user mentions: split screen video, split-screen reel, top-bottom video, half-and-half video, upper b-roll lower talking head, news-anchor-style reel, stacked video layout, b-roll on top avatar on bottom, or references bayareawilson-style reels as a target. Also trigger on 'do another split-screen video', 'same format, new topic', or 'composite the avatar and b-roll into the split screen'. Owns frame layout only — hands off to heygen-video/heygen-elevenlabs-renderer for the avatar half and higgsfield-video for the b-roll half, then composites. Not for full-bleed talking-head videos or the Reventure punch-in format (education-graeham-videos)." +--- + +# Split-Screen Avatar Reel + +Assembles the "b-roll top half / avatar bottom half" split-screen format — see +`references/format-breakdown.md` for the full visual teardown of the three reference reels +this was built from (`@bayareawilson` on Instagram: Safeway Marina, Treasure Island, and +Fisherman's Wharf reels). + +**This skill is an orchestrator + compositor, not a third rendering pipeline.** It does not +call HeyGen or Higgsfield directly — it hands off to the two skills that already do that well, +waits for finished clips, then runs the bundled compositor script. Read that reference file +before the first run of this skill so the frame anatomy is fresh in mind. + +## When this skill fires + +- "Make a split-screen reel about [topic]" +- "Same format as those bayareawilson reels, but for [Graeham's story]" +- "Composite this avatar clip and this b-roll into the split-screen layout" +- "I have a HeyGen render and a Higgsfield clip, stack them" + +If the user has a TOPIC but no SCRIPT/headline yet, chain with `content-creation-engine` (or +`education-graeham-videos` if it's Bay Area market/development news — matches this format's +actual content genre most closely) first. That skill's job is finding the story and writing a +tight VO + headline; this skill's job starts once there's a script. + +## Workflow + +### 1. Get the story and a short VO script + +A script for this format should be short — the reference reels read like 20-40 second news +hits, not full explainers. If chaining from `content-creation-engine` / `education-graeham-videos`, +ask for a punchy, single-story script (one development, one number, one hook) rather than a +multi-beat long-form script — this layout doesn't have room for a beat-by-beat structure since +there's no cutting between shot types, just one continuous talking-head take under one +continuous (or slowly-changing) b-roll. + +### 2. Ask the required questions in one turn + +Before rendering anything, confirm: + +1. **Avatar look + voice** — same mandatory ask as `heygen-video`: which look, or default + voice clone? Don't silently pick one. +2. **Headline text** — two short lines, ALL CAPS reads best (the compositor uppercases + automatically). If the user doesn't give exact wording, propose one derived from the + script's hook and confirm before rendering — this is the first thing anyone sees, worth + getting right. +3. **B-roll subject/scene** — what should fill the top half? One strong establishing shot is + enough for a ~20s reel; for longer reels, ask if they want a second b-roll clip to cut to + partway through (the compositor accepts multiple `--broll` clips and concatenates them). +4. **Color scheme** — default is Watts Gold (`#C4A265`) for line 1 / white for line 2, which + keeps this on-brand with Graeham's other video work. Offer the reference reels' brighter + yellow (`#F5C518`) as an alternative if they want a closer visual match to the source + accounts. + +### 3. Render the bottom half — avatar + +Hand off to `heygen-video` (or `heygen-elevenlabs-renderer` if this is part of an +auto-render chain) with the confirmed script, look, and voice. Request the **normal 9:16 +default** — no special aspect handling needed, the compositor crops whatever comes back. +Wait for the finished MP4 per that skill's normal submit-and-check flow; don't block the +conversation on a multi-minute render. + +### 4. Render the top half — b-roll + +Hand off to `higgsfield-video` with the confirmed scene description. Request 9:16 (matches +the compositor's default crop strategy). Duration: doesn't need to match the avatar's runtime +exactly — the compositor loops/holds the last b-roll frame to fill whatever the avatar clip +runs. A single 10s clip covers most reels in this format fine. + +### 5. Composite + +Once both MP4s are downloaded: + +```bash +python3 scripts/composite_split_screen.py \ + --avatar /path/to/heygen_render.mp4 \ + --broll /path/to/broll_clip.mp4 \ + --headline-line1 "SAFEWAY MARINA ADDS" \ + --headline-line2 "800 HOMES" \ + --accent-color "#C4A265" \ + --out /path/to/output_master.mp4 \ + --posting-copy /path/to/output_posting.mp4 +``` + +What it does (see the script's own docstring for full detail): +1. Normalizes both clips to a canonical 1080x1920 "cover crop" — so it doesn't matter if + HeyGen or Higgsfield handed back slightly different source dimensions. +2. Crops the top 960px of the b-roll and the bottom 960px of the avatar (loops/holds b-roll + to cover the avatar's full runtime if it's shorter). +3. Draws the two-line headline as a transparent overlay straddling the seam (y=960), bold + caps with a heavy black outline for legibility over any background. +4. Stacks top+bottom, overlays the headline, muxes in the avatar clip's own audio as the + only soundtrack. +5. Outputs a CRF 18 master, and optionally a smaller CRF 23 "posting copy" sized to fit a + typical ~30 MiB chat-share limit. + +**`--avatar-anchor-y` (default 520)** controls where in the normalized 1920px-tall avatar +frame the bottom crop starts. HeyGen's default framing varies a little by look — if the +presenter's chin gets clipped or there's too much headroom, pull a single normalized frame +first (`ffmpeg -i avatar_norm.mp4 -frames:v 1 check.png` after a dry run, or just render once +and look at the composited result) and adjust this value up (crops lower/tighter) or down +(crops higher, more headroom) before re-running — cheap to iterate since it's pure ffmpeg, no +credits spent re-rendering the avatar itself. + +**`--top-anchor-y` (default 0)** does the same for the b-roll — 0 keeps the top of the frame, +raise it if the interesting part of the shot is lower in the source composition. + +### 6. QC before delivery + +- `ffprobe` the output: confirm 1080x1920, audio stream present, duration matches the avatar + source (±0.5s). +- Pull 2-3 frames across the runtime (`ffmpeg -ss -frames:v 1 ...`) and look at them: + headline fully on-screen and not clipped at the frame edges, seam is clean (no visible + crop artifacts), avatar isn't cut off at the chin or missing too much headroom. +- Confirm the posting-copy file (if generated) is under ~30 MiB; if not, bump `--posting-copy`'s + CRF higher or shorten the reel. + +### 7. Deliver + +Hand back both the master and posting-copy files (see delivery conventions — master stays +in-session, posting copy is what actually gets shared). If this is part of a recurring +content day (paired with `education-graeham-videos` research), offer to log the finished +reel the same way that pipeline's other outputs are tracked. + +## What this skill intentionally does NOT do + +- ❌ Does not call the HeyGen or Higgsfield APIs/UI itself — always hands off to those skills. +- ❌ Does not write scripts or find stories — hands off to `content-creation-engine` / + `education-graeham-videos`. +- ❌ No word-level karaoke captions in v1 — the reference reels didn't show any at the + captured frame, just the static two-line headline. If a future reel needs them, that's a + compositor addition, not a redesign. +- ❌ No mid-reel b-roll cutting logic beyond simple concatenation of clips the user provides + in order — if a reel needs precisely-timed cuts synced to VO beats, treat that as a v2 + ask rather than assuming it here. +- ❌ Does not auto-pick avatar-anchor-y per look — this needs a one-time eyeball check per + HeyGen look the first time it's used in this format, then it's a known-good value to reuse. + +## Files in this skill + +- `SKILL.md` — this file +- `scripts/composite_split_screen.py` — the compositor (ffmpeg + Pillow, no other deps) +- `references/format-breakdown.md` — visual teardown of the three reference reels this + format was built from, including what's confirmed vs. unconfirmed from a single frame each diff --git a/skills/split-screen-avatar-reel/references/format-breakdown.md b/skills/split-screen-avatar-reel/references/format-breakdown.md new file mode 100644 index 00000000..87b5f4a6 --- /dev/null +++ b/skills/split-screen-avatar-reel/references/format-breakdown.md @@ -0,0 +1,62 @@ +# Format breakdown — the reference reels + +Sourced from three reels on `@bayareawilson` (Instagram) that Graeham flagged as the exact +target format: +- instagram.com/reel/DbyCcCVh0w5 — "Safeway Marina Adds 800 Homes" +- instagram.com/reel/Dbo2vozh_oe — "Treasure Island Opens First Condo Homes" +- instagram.com/reel/DbwlJ_rhvgR — "A New Ten-Million-Dollar [Fisherman's Wharf plaza]" + +All three are the same account, same presenter, same rig, same edit template — this is a +repeatable format, not a one-off. What follows is what's visually verifiable from the three +reels; anything not directly observable (their VO length distribution, whether the b-roll ever +cuts mid-video) is flagged as unconfirmed. + +## Frame anatomy (1080×1920, confirmed) + +- **Top half (y 0–~960):** b-roll. In the three examples: a static drone/aerial rendering, a + Google-Maps satellite pan, and a rendered site-plan/plaza illustration. Not necessarily + filmed footage — at least one looked like a still or slow-zoom rendering rather than dynamic + video. Treat "b-roll" loosely: a slow push/zoom on a single strong image reads fine here, + full motion isn't required. +- **Seam headline (straddles y ~830–960, i.e. bottom of the b-roll half):** two stacked lines, + bold condensed sans-serif, ALL CAPS, heavy black outline/drop-shadow for legibility over any + background. Line 1 in a warm gold/yellow, line 2 in white (confirmed consistent across all + three examples — gold line always comes first). Text is short — 3-5 words per line, reads + like a headline not a caption ("SAFEWAY MARINA ADDS" / "800 HOMES"). +- **Bottom half (y ~960–1920):** the presenter, static medium shot (chest-up), same indoor set + in all three (warm-lit shelf with plants and framed photos behind him), gesturing while he + talks. He's centered horizontally, positioned so his eyes/face sit in the upper third of the + bottom half — there's headroom above him, not a tight crop. + +## What's NOT confirmed from a single frame each + +- Whether the b-roll cuts to a second shot partway through longer reels (likely yes for + anything over ~15s, but not verified frame-by-frame). +- Exact runtime / pacing — likely 20-40s based on typical reel length for this content type. +- Whether there's a word-level caption track anywhere else in the video (the captured frames + show only the static two-line headline, no karaoke captions visible at that timestamp). +- Font family (visually a heavy grotesk/condensed sans — Liberation Sans Bold or similar is a + reasonable stand-in; doesn't need to match exactly to hit the same read). + +## Caption text (for context on tone/topic, not part of the visual format) + +Real, hyperlocal SF Bay Area development news — e.g. "The Safeway Marina redevelopment just +updated its plans to 848 units across two towers up to 258 ft at 15 Marina Blvd — and the +grocery store stays open throughout construction." Each caption ends with an engagement +question and a comment-to-DM CTA ("Comment 'update' to get our Bay Area newsletter"). This +maps directly onto Graeham's existing `education-graeham-videos` / `content-creation-engine` +research and CTA conventions — this skill owns the FRAME LAYOUT, not the story-finding or +copywriting, which stay with those skills. + +## Why this skill doesn't reinvent the avatar or b-roll pipelines + +Graeham already has mature, credit-metered pipelines for both halves: +- `heygen-video` / `heygen-elevenlabs-renderer` render the presenter (9:16, native HeyGen + framing — a chest-up medium shot is HeyGen's default composition, which is exactly what the + bottom half needs). +- `higgsfield-video` renders the b-roll (9:16 or 16:9, Nano Banana Pro / GPT Image 2 → Seedance + / Kling). + +This skill's only job is: take those two already-rendered clips and the headline text, and +assemble them into the split-screen frame. It never calls HeyGen or Higgsfield APIs directly — +it hands off to the two skills above and waits for finished MP4s. diff --git a/skills/split-screen-avatar-reel/scripts/composite_split_screen.py b/skills/split-screen-avatar-reel/scripts/composite_split_screen.py new file mode 100644 index 00000000..0027df28 --- /dev/null +++ b/skills/split-screen-avatar-reel/scripts/composite_split_screen.py @@ -0,0 +1,221 @@ +#!/usr/bin/env python3 +""" +Split-screen reel compositor — b-roll top half, HeyGen avatar bottom half. + +Takes an already-rendered HeyGen avatar clip (with audio) and one or more already-rendered +Higgsfield b-roll clips, and assembles the "bayareawilson-style" split-screen frame: + + ┌─────────────────────┐ + │ │ + │ B-ROLL (top half) │ <- cropped/looped to fill the avatar's runtime + │ │ + ├──── HEADLINE ────────┤ <- two bold caps lines straddling the seam + │ │ + │ AVATAR (bottom half)│ <- avatar's own audio track carries the whole video + │ │ + └─────────────────────┘ + +Both inputs are normalized to a canonical 1080x1920 "cover crop" first (so any source aspect +ratio works), then each contributes its top or bottom 960px band. This means neither the +HeyGen render nor the Higgsfield render needs any special aspect handling upstream — request +them at their normal 9:16 defaults and this script does the rest. + +Usage: + python3 composite_split_screen.py \\ + --avatar /path/to/heygen_render.mp4 \\ + --broll /path/to/broll_clip1.mp4 [/path/to/broll_clip2.mp4 ...] \\ + --headline-line1 "SAFEWAY MARINA ADDS" \\ + --headline-line2 "800 HOMES" \\ + --out /path/to/output_master.mp4 + +Optional flags: --accent-color, --line2-color, --avatar-anchor-y, --top-anchor-y, +--width, --height, --fps, --posting-copy (writes a second, smaller delivery file). + +Requires: ffmpeg, ffprobe, python3-pil (Pillow) on PATH / importable. +""" +import argparse +import json +import os +import subprocess +import sys +import tempfile + +from PIL import Image, ImageDraw, ImageFont + +FD = "/usr/share/fonts/truetype/liberation" +BOLD = f"{FD}/LiberationSans-Bold.ttf" + + +def run(cmd, **kwargs): + print("+", " ".join(cmd), file=sys.stderr) + subprocess.run(cmd, check=True, **kwargs) + + +def ffprobe_duration(path): + out = subprocess.check_output([ + "ffprobe", "-v", "error", "-show_entries", "format=duration", + "-of", "json", path, + ]) + return float(json.loads(out)["format"]["duration"]) + + +def hex_to_rgb(h): + h = h.lstrip("#") + return tuple(int(h[i:i + 2], 16) for i in (0, 2, 4)) + + +def draw_outlined_centered(d, cx, y, text, font, fill, W, ow=7): + tw = d.textlength(text, font=font) + x = cx - tw / 2 + for dx in range(-ow, ow + 1, 2): + for dy in range(-ow, ow + 1, 2): + d.text((x + dx, y + dy), text, font=font, fill=(0, 0, 0)) + d.text((x, y), text, font=font, fill=fill) + return tw + + +def build_headline_png(width, height, line1, line2, accent_hex, line2_hex, seam_y, out_path, + font_size=118, line_gap=136): + """Transparent overlay with two centered bold-caps lines straddling `seam_y`.""" + img = Image.new("RGBA", (width, height), (0, 0, 0, 0)) + d = ImageDraw.Draw(img) + font = ImageFont.truetype(BOLD, font_size) + accent = hex_to_rgb(accent_hex) + white = hex_to_rgb(line2_hex) + # Block is vertically centered on the seam: line1 sits just above it, line2 just below. + y1 = seam_y - line_gap + y2 = seam_y + int(line_gap * 0.12) + draw_outlined_centered(d, width / 2, y1, line1.upper(), font, accent, width) + draw_outlined_centered(d, width / 2, y2, line2.upper(), font, white, width) + img.save(out_path) + return out_path + + +def cover_crop_filter(width, height): + """ffmpeg filter: scale-to-cover then center-crop to exactly width x height.""" + return f"scale={width}:{height}:force_original_aspect_ratio=increase,crop={width}:{height}" + + +def normalize_clip(src, width, height, fps, out_path, extra_vf=None): + vf = cover_crop_filter(width, height) + f",fps={fps}" + if extra_vf: + vf += "," + extra_vf + run(["ffmpeg", "-y", "-v", "error", "-i", src, "-an", "-vf", vf, + "-c:v", "libx264", "-preset", "veryfast", "-crf", "16", out_path]) + + +def loop_or_trim_to_duration(src, target_dur, out_path, fps): + src_dur = ffprobe_duration(src) + if src_dur >= target_dur: + run(["ffmpeg", "-y", "-v", "error", "-i", src, "-t", f"{target_dur:.3f}", + "-c", "copy", out_path]) + else: + # loop the clip, then hard-trim to the exact target duration + run(["ffmpeg", "-y", "-v", "error", "-stream_loop", "-1", "-i", src, + "-t", f"{target_dur:.3f}", "-c:v", "libx264", "-preset", "veryfast", + "-crf", "16", "-r", str(fps), out_path]) + + +def concat_clips(paths, out_path): + if len(paths) == 1: + run(["ffmpeg", "-y", "-v", "error", "-i", paths[0], "-c", "copy", out_path]) + return + with tempfile.NamedTemporaryFile("w", suffix=".txt", delete=False) as f: + for p in paths: + f.write(f"file '{os.path.abspath(p)}'\n") + listfile = f.name + run(["ffmpeg", "-y", "-v", "error", "-f", "concat", "-safe", "0", "-i", listfile, + "-c", "copy", out_path]) + os.unlink(listfile) + + +def main(): + ap = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter) + ap.add_argument("--avatar", required=True, help="HeyGen avatar render (has the VO audio)") + ap.add_argument("--broll", nargs="+", required=True, help="One or more Higgsfield b-roll clips (top half)") + ap.add_argument("--headline-line1", required=True) + ap.add_argument("--headline-line2", required=True) + ap.add_argument("--accent-color", default="#C4A265", help="Line 1 color (default Watts Gold; use #F5C518 to match the reference reels' brighter yellow)") + ap.add_argument("--line2-color", default="#FFFFFF") + ap.add_argument("--top-anchor-y", type=int, default=0, + help="Y offset (in the normalized 1080x1920 b-roll) where the top-half crop starts. 0 = top of frame.") + ap.add_argument("--avatar-anchor-y", type=int, default=520, + help="Y offset (in the normalized 1080x1920 avatar) where the bottom-half crop starts. " + "Default 520 keeps head+shoulders with headroom in the lower 960px band — " + "tune per-avatar-look by eyeballing a normalized frame first.") + ap.add_argument("--width", type=int, default=1080) + ap.add_argument("--height", type=int, default=1920) + ap.add_argument("--fps", type=int, default=25) + ap.add_argument("--out", required=True, help="Master output path (CRF 18)") + ap.add_argument("--posting-copy", help="Optional second output path, compressed to fit a ~30MiB chat-share limit") + ap.add_argument("--workdir", default=None) + args = ap.parse_args() + + W, H, FPS = args.width, args.height, args.fps + HALF = H // 2 + + workdir = args.workdir or tempfile.mkdtemp(prefix="splitscreen_") + os.makedirs(workdir, exist_ok=True) + + avatar_dur = ffprobe_duration(args.avatar) + print(f"Avatar clip duration: {avatar_dur:.2f}s — this sets the runtime for the whole reel.", file=sys.stderr) + + # 1. Normalize avatar to canonical 1080x1920, keep its audio for the final mux. + avatar_norm = os.path.join(workdir, "avatar_norm.mp4") + normalize_clip(args.avatar, W, H, FPS, avatar_norm) + + # 2. Normalize + concat + loop b-roll to cover the full avatar duration. + broll_norm_parts = [] + for i, clip in enumerate(args.broll): + p = os.path.join(workdir, f"broll_norm_{i}.mp4") + normalize_clip(clip, W, H, FPS, p) + broll_norm_parts.append(p) + broll_concat = os.path.join(workdir, "broll_concat.mp4") + concat_clips(broll_norm_parts, broll_concat) + broll_full = os.path.join(workdir, "broll_full.mp4") + loop_or_trim_to_duration(broll_concat, avatar_dur, broll_full, FPS) + + # 3. Crop each to its half of the frame. + top_crop = os.path.join(workdir, "top_crop.mp4") + run(["ffmpeg", "-y", "-v", "error", "-i", broll_full, + "-vf", f"crop={W}:{HALF}:0:{args.top_anchor_y}", + "-c:v", "libx264", "-preset", "veryfast", "-crf", "16", "-an", top_crop]) + + bottom_crop = os.path.join(workdir, "bottom_crop.mp4") + run(["ffmpeg", "-y", "-v", "error", "-i", avatar_norm, + "-vf", f"crop={W}:{HALF}:0:{args.avatar_anchor_y}", + "-c:v", "libx264", "-preset", "veryfast", "-crf", "16", "-an", bottom_crop]) + + # 4. Headline overlay PNG, centered on the seam (y = HALF). + headline_png = os.path.join(workdir, "headline.png") + build_headline_png(W, H, args.headline_line1, args.headline_line2, + args.accent_color, args.line2_color, HALF, headline_png) + + # 5. vstack top + bottom, overlay headline, mux avatar's original audio. + stacked = os.path.join(workdir, "stacked.mp4") + run(["ffmpeg", "-y", "-v", "error", "-i", top_crop, "-i", bottom_crop, + "-filter_complex", "[0:v][1:v]vstack=inputs=2[v]", + "-map", "[v]", "-c:v", "libx264", "-preset", "veryfast", "-crf", "16", stacked]) + + run(["ffmpeg", "-y", "-v", "error", + "-i", stacked, "-i", headline_png, "-i", args.avatar, + "-filter_complex", "[0:v][1:v]overlay=0:0[v]", + "-map", "[v]", "-map", "2:a", + "-c:v", "libx264", "-preset", "medium", "-crf", "18", "-pix_fmt", "yuv420p", + "-c:a", "aac", "-b:a", "192k", "-movflags", "+faststart", + "-shortest", args.out]) + print(f"Master written: {args.out}", file=sys.stderr) + + if args.posting_copy: + run(["ffmpeg", "-y", "-v", "error", "-i", args.out, + "-c:v", "libx264", "-preset", "medium", "-crf", "23", "-pix_fmt", "yuv420p", + "-c:a", "aac", "-b:a", "128k", "-movflags", "+faststart", + args.posting_copy]) + size_mb = os.path.getsize(args.posting_copy) / (1024 * 1024) + print(f"Posting copy written: {args.posting_copy} ({size_mb:.1f} MiB)", file=sys.stderr) + if size_mb > 30: + print("WARNING: posting copy still over 30 MiB — bump CRF or shorten the reel.", file=sys.stderr) + + +if __name__ == "__main__": + main() diff --git a/skills/switchy-engine/.gitignore b/skills/switchy-engine/.gitignore new file mode 100644 index 00000000..694ceb48 --- /dev/null +++ b/skills/switchy-engine/.gitignore @@ -0,0 +1,6 @@ +# NEVER commit the Switchy token +.switchy_token +.switchy/ +*.token +# local reports may contain workspace data +*_report.csv diff --git a/skills/switchy-engine/EMAIL-to-Peter-switchy-engine.md b/skills/switchy-engine/EMAIL-to-Peter-switchy-engine.md new file mode 100644 index 00000000..af217c11 --- /dev/null +++ b/skills/switchy-engine/EMAIL-to-Peter-switchy-engine.md @@ -0,0 +1,78 @@ +Subject: New skill for you — auto-generate tracked QR codes for our postcards (Switchy) + +Hey Peter, + +Setting you up with a new Cowork skill called **switchy-engine**. The whole point: +you make the postcard, and Claude does the QR + tracking automatically so we can +see how many people scan each mailer and retarget them with ads. + +--- + +## What it does +When you give Claude a finished postcard and ask for a QR code, it will: +1. Create a short, trackable link in our Switchy account pointing to the right + landing page (e.g. the home-value page), with tracking tags baked in. +2. Attach our Meta + Google pixels so everyone who scans gets added to our ad + retargeting audience. +3. Generate the QR code and download it for you to drop into the postcard. +4. File it neatly in Switchy under the "Post card qr" folder with a clear name. + +Then every Monday we get a report of how many scans each postcard/link got. + +--- + +## How to install it +**Easiest:** open the attached file **`switchy-engine.skill`** and click +**"Save skill"** — that installs it into your Cowork. + +*(Once Graeham pushes it to GitHub, you'll also be able to pull it from the +`Graehamwatts/skills` repo under `skills/switchy-engine`. The attached file works +right now either way.)* + +--- + +## How to use it (your weekly postcard workflow) +1. Build the postcard in Canva like normal. +2. Open Cowork and say: **"Generate a QR code for this postcard"** and upload the + postcard PDF/image. +3. Claude logs into Switchy, makes the tracked link + QR, and hands you the QR PNG. +4. Drop the QR into the postcard, export, send to print. + +That's it. Claude handles the link, the UTM tracking, the pixels, and the filing. + +--- + +## Switchy login (Graeham will fill this in before sending) + +> **Graeham — paste the Switchy access for Peter here. Do NOT send the password in +> plain email if you can avoid it.** Two options: +> +> **Option A (recommended, safer):** In Switchy → Account → Team → "Add team +> member", invite Peter's email so he gets his OWN login. Then he generates his own +> API key under Settings → API key. +> +> **Option B (shared login):** share the workspace login another way (password +> manager / phone), and the API key named **"claude-engine"** is already created +> under Settings → API key. +> +> _Login: ____________________ · Password: (send separately) _____________________ + +--- + +## Naming convention (so every QR is easy to identify) +Claude will name things like this automatically — just so you know what you're +looking at in Switchy: +- **Link name:** `Postcard EPA 2026-06-01 — Last 5 Homes (home value)` +- **Short link:** `hi.switchy.io/epa-comps-0601` +- **Folder:** `Post card qr` +- **UTM:** `utm_source=postcard · utm_medium=direct_mail · utm_campaign=epa_06_01_26 · utm_content=anti_zestimate` + +Date format is YYYY-MM-DD so they sort correctly. + +--- + +Any issues installing, text me. Once you're in, do a test run on the June 1st EPA +card — it's already created in Switchy so you can see how it looks. + +Thanks, +Graeham diff --git a/skills/switchy-engine/SKILL.md b/skills/switchy-engine/SKILL.md new file mode 100644 index 00000000..fd3df446 --- /dev/null +++ b/skills/switchy-engine/SKILL.md @@ -0,0 +1,231 @@ +--- +name: switchy-engine +description: "Tracked-link + retargeting-pixel engine for Graeham Watts. Use this skill ANY time the user mentions: tracked link, short link, Switchy, shortlink, QR code, retargeting link, pixeled link, link analytics, scan count, click count, custom audience, retargeting audience, link in bio, UTM, swappable link, CTA link..." +--- + +> **Absorbed on 2026-08-07:** `switchy-qr` was merged into this skill. All capabilities of +> `switchy-qr` (the simplified postcard-QR quick-start for non-technical users like Peter/Jason) +> now live here — see "Quick-start for postcards (Peter / Jason)" below. The folder +> `skills/switchy-qr/` was deleted in the same commit. If you find any reference to +> `skills/switchy-qr/` anywhere in this repo, that reference is a bug — it should point here. + +# Switchy Engine — Tracked Links & Retargeting Pixels + +The **one** place link-shortening, QR generation, pixel-tagging, and click/scan +analytics live. Everything else (newsletter, postcards, listing pages, GBP, email +signature) should request a tracked link FROM this skill rather than pasting a raw +destination URL. That is the whole point: mint once, pixel consistently, measure +in one dashboard, never duplicate the logic. + +> **What Switchy does that a raw URL doesn't:** it sits on the redirect layer and +> fires retargeting pixels (Meta, Google/GA, LinkedIn, Pinterest, Bing, etc.) +> *before* the destination page loads, drops the visitor into a custom audience, +> tracks the scan/click, and lets you swap the destination later without changing +> the printed link or QR. Pixel fires even if the visitor bounces before the page +> renders. + +--- + +## Confirmed API facts (developers.switchy.io, verified May 2026) + +| Thing | Value | +|---|---| +| GraphQL endpoint | `https://graphql.switchy.io/v1/graphql` (POST, **queries only**) | +| REST link-create | `POST https://api.switchy.io/v1/links/create` (mutations are REST, not GraphQL) | +| Auth header | `Api-Authorization: ` — **not** `Authorization: Bearer` | +| Token scope | One token per **workspace**; API-key style, **not** OAuth | +| Schema style | Hasura (`where: { field: { _is_null: true } }` filter syntax) | +| Pixel platforms supported | linkedin, facebook, gtm, quora, pinterest, twitter, ga, bing, nexus, adroll, adwords | +| Rate limits (create) | 10,000 links/day, 1,000 links/hour | + +### ⚠️ Two things that are NOT settled and must be confirmed live +1. **Per-link click/scan field name.** Public docs only show workspace-level + fields (`workspaces`, `domains`). The field that holds per-link click/scan + counts is **not documented** — you MUST introspect it on the live token before + trusting any analytics query. Run `scripts/switchy_analytics.py --confirm-schema`. +2. **Token activation.** Switchy restricts API access; an account may need to ask + Switchy **live chat** to enable API access before the generated token returns + data. Confirm the token actually returns rows before wiring downstream skills. + +### ⚠️ Platform caveat: no native TikTok pixel +Switchy's pixel platform list has no TikTok entry. For TikTok-sourced traffic, +either route through `gtm` (Google Tag Manager container that carries the TikTok +pixel) or accept click-tracking only. Don't promise native TikTok retargeting. + +--- + +## Token setup (do this once) + +1. Log in to Switchy → open the target **workspace** → **Settings → Integrations + → Generate a token**. +2. If queries return empty/!errors, message Switchy **live chat** to enable API + access for the account, then regenerate. +3. Store it — never hardcode, never commit: + - Windows: `setx SWITCHY_API_TOKEN "your-token"` + - mac/linux: `export SWITCHY_API_TOKEN="your-token"` (or `~/.switchy/token`, chmod 600) +4. Lock the schema: `python scripts/switchy_analytics.py --confirm-schema` + → note the real click/scan field, pass it as `--click-field`. + +--- + +## What this skill exposes to other skills (the contract) + +Downstream skills should call ONE of these instead of emitting a raw URL: + +- **`mint(destination, tags[], pixels[], domain?, slug?)`** → returns a Switchy + short URL + (optional) QR PNG. Implemented via the REST create endpoint + (`api.switchy.io/v1/links/create`). Always pass `tags` so the analytics layer + can segment by source (e.g. `["newsletter","consumer"]`, `["postcard","qr","94303"]`). +- **`report()`** → runs `scripts/switchy_analytics.py`, returns the + scans→audience→budget table (markdown + CSV). + +> **Tagging convention (mandatory).** Every minted link gets: +> `surface` (gbp / newsletter / postcard / listing / signature / openhouse / yardsign / social-bio …), +> `audience-class` (`consumer` | `prospect` | `b2b` | `mixed`), +> and an optional `campaign` tag. The `audience-class` tag is what lets us EXCLUDE +> junk (vendor/agent clicks) from retargeting audiences. See +> `references/audience-hygiene.md`. + +--- + +## Analytics: the core query flow + +`scripts/switchy_analytics.py` does three things: + +1. **`--confirm-schema`** — introspects the `links` type and prints real field + names (mandatory first run on a live token). +2. **fetch** — queries all live links + their click/scan counts. +3. **model** — converts each link's clicks into a *targetable retargeting + audience* and the *monthly ad budget that audience justifies* (frequency × CPM), + flagging audiences too small to target. + +```bash +python scripts/switchy_analytics.py --confirm-schema # step 1, once +python scripts/switchy_analytics.py --click-field clicks # normal run +python scripts/switchy_analytics.py --cpm 25 --frequency 12 # tune the model +``` + +Runs in **DEMO mode** with illustrative numbers if no token is set, so the output +format can be reviewed before go-live. + +See `references/graphql-queries.md` for the raw queries (introspection, per-link +analytics, both scalar-count and aggregate shapes). + +--- + +## When to use vs. not + +**Use** when minting any link/QR that will sit in front of consumer or prospect +traffic, or when reporting on link/QR/scan performance. + +**Don't bother wrapping** (raw URL is fine) when: +- The destination is Graeham's OWN already-pixeled site AND you don't need + per-source attribution, destination-swapping, or multi-pixel firing (see + `references/retargeting-pathway-map.md` → "own-site redundancy"). +- The surface is a GBP **primary website field** — Google auto-removes redirect / + shortener links there. Use the real domain in that field; use Switchy in GBP + *posts* and secondary links instead. See `references/gbp-and-youtube.md`. + +--- + +## Reference docs +- `references/graphql-queries.md` — introspection + analytics queries, REST create payload. +- `references/retargeting-pathway-map.md` — every surface, traffic type, retargeting value, caveats. +- `references/gbp-and-youtube.md` — the GBP redirect-policy answer + YouTube/own-site pixel-redundancy answer. +- `references/audience-hygiene.md` — what to pixel vs. skip, and source segmentation. +- `references/architecture-decision.md` — why this is a standalone called-into engine. +--- + +## Live-verified API capabilities (2026-05-28, real token) + +Confirmed by introspecting the live GraphQL schema. **Read this before promising a metric.** + +**Queryable top-level types:** `links`, `folders`, `pixels`, `domains`, `UTMTemplates`, `tokens`, `workspaces` (+ `_by_pk`). + +**Per-link data you CAN get:** `id` (slug), `domain`, `url` (destination), `title`, +`tags`, `pixels`, `folderId`, `createdDate`, and **`clicks`** (total click/scan count — a QR scan and a link click both increment this). + +**What you CANNOT get from the API (important):** +- **No per-click detail** — no referrer, no country/geo, no device, no timestamp-per-click. +- **No time-series** — only a running total. `uniq` is an internal ID, NOT a unique-visitor count. +- There is no clicks/stats/events table in the schema. + +**Implications:** +1. **"Where clicks come from" must be ENGINEERED, not queried.** Source attribution lives in + how each link is built — its **slug + tags + UTM**. A bare/untagged link is unattributable. + This is why tagging discipline (below) is mandatory. (Richer geo/referrer/device stats DO + exist in the Switchy *dashboard UI* per link, but are not exposed to the API.) +2. **Weekly trends require our own snapshots.** Since the API only returns a running total, + the weekly digest must SNAPSHOT every link's `clicks` each Monday and DIFF against last + week's snapshot to report "scans this week." Snapshots stored as dated JSON/CSV. +3. GA4 (via the UTM) and Meta (via the pixel) hold the richer behavioral/audience data — + cross-reference there for on-site behavior and audience size. + +## Naming & folder convention (mandatory — keeps everything decodable) +- **Slug:** `--` e.g. `epa-comps-0601` +- **Title:** `Postcard EPA 2026-06-01 — Last 5 Homes (home value)` (date YYYY-MM-DD so it sorts) +- **Tags:** `surface` + `audience-class` + market + date, e.g. `["postcard","qr","consumer","epa","2026-06-01"]` +- **UTM:** `utm_source=&utm_medium=&utm_campaign=_&utm_content=` +- **Switchy folder:** by surface — `Post card qr` (id 92811), `Yard Sign QR` (80707), GMB folders, etc. + +## Weekly digest (Monday) — design +A **scheduled task** (not duplicated in postcard/content skills) calls this engine every +Monday: it snapshots all link `clicks`, diffs against last week, and produces a dashboard + +email showing scans-this-week per source, audience growth, and suggested budget. Published to +`Graehamwatts/online-content/dashboards/switchy/` (hosted) and emailed. Because trend data +depends on our snapshots, the FIRST run only establishes a baseline (deltas start week 2). + +## Callable contract (how other skills use this) +- `farming-postcard` / `content-creation-engine` call `mint()` here instead of emitting a raw + URL — they pass destination + tags + pixels, get back a tracked short link + QR. +- Any skill can call `report()` to pull the current scans→audience→budget table. +- Constants (pixel IDs, default domain, tag vocab) live in `shared-references/switchy.json`. + +--- + +## Quick-start for postcards (Peter / Jason) + +**What this does, in one line:** finish a postcard, say *"generate a QR code for this +postcard,"* and Claude creates a tracked Switchy link + QR to drop into the design. +Switchy then tracks every scan and adds scanners to Graeham's ad audience. + +**You only do two things:** (1) say the command + upload the postcard, and (2) log into +Switchy once when asked. Claude does the rest — see the full workflow steps below, or run +`scripts/create_postcard_link.py --date --hook ""` directly (prints the +short URL; Claude still needs Chrome + your login to fetch the QR PNG from the dashboard). + +**One-time setup:** you need access to Graeham's Switchy (your own team login, or the +shared login Graeham gives you) plus the Switchy API token so Claude can create the link +automatically — Graeham will provide it; save it as `switchy-token.txt` in your Cowork +Skills folder, or set the `SWITCHY_API_TOKEN` environment variable. Never paste the token +into chat. + +> Need scan numbers, the dashboard, or retargeting reports beyond a single postcard QR? +> That's the rest of this skill (below) — this section is QR-only on purpose. + +## QR generation workflow (Cowork → Chrome → Switchy → dashboard) + +**Trigger:** user says "generate a QR code for this postcard" and uploads the postcard. + +1. **Destination + UTM.** Resolve the landing page from `farming-postcard/references/cta-router.md` + (home-value default: `https://graehamwatts.com/evaluation`). Append: + `?utm_source=postcard&utm_medium=direct_mail&utm_campaign=epa_&utm_content=`. +2. **Create the tracked link via REST** (`POST https://api.switchy.io/v1/links/create`, + header `Api-Authorization:`): set a clean `id` slug, `title` + (`Postcard — `), `folderId` (Post card qr = **92811**), + `tags` (`postcard,qr,consumer,,`), and `pixels` from + `shared-references/switchy.json` (Meta 963211690980393, GA4 G-S82GF32XJT, Ads AW-1047225119). +3. **Get the QR via Claude-in-Chrome.** Open switchy.io → **the user logs in themselves** + (Claude must NOT enter passwords or solve the reCAPTCHA). Then: link list → find the link → + click **"Download QR Code"** → QR designer → **Download as PNG** (lands in Downloads). + Save/rename as `Postcard__.png`. +4. **Hand the QR to the user** to embed in the Canva postcard. The destination is swappable + later in Switchy without changing the printed QR. +5. **Publish to the dashboard + sync to GitHub.** Run `scripts/switchy_dashboard.py`, then push + `online-content/dashboards/switchy/index.html` (+ the dated snapshot) to `Graehamwatts/skills` + via the **GitHub Contents API** (`PUT .../contents/` with the github token) — NOT local + git, because the working tree's `.git/index.lock` blocks commits on this machine. The live + dashboard at `graehamwatts.github.io/skills/online-content/dashboards/switchy/` then updates. + +> Pages serves from main root, so any file pushed under `online-content/` is live at +> `graehamwatts.github.io/skills/online-content/...`. diff --git a/skills/switchy-engine/references/architecture-decision.md b/skills/switchy-engine/references/architecture-decision.md new file mode 100644 index 00000000..4d250a24 --- /dev/null +++ b/skills/switchy-engine/references/architecture-decision.md @@ -0,0 +1,72 @@ +# Architecture Decision — standalone `switchy-engine` + +## Recommendation (validated, with one refinement) +**Build `switchy-engine` as a standalone skill that other skills call into — +YES.** Add one refinement: the durable *constants* (pixel IDs, default domain, +tag vocabulary, vendor-exclusion list location) live in `shared-references`, so +the engine and every caller read one source of truth. Capability = skill; +constants = shared config. + +## Why the hypothesis holds (receipts from the actual skill stack) +The codebase already proves both halves of this pattern: + +- **"Build once, reference everywhere" is the established norm.** `cma-generator` + is called by `newsletter-generator`'s home-value CTA; `content-calendar` hands + topics to `content-creation-engine`; multiple skills read + `../shared-references/identity.json` instead of hardcoding brand facts; + `github-skill-sync` is a horizontal utility that other skills invoke after any + change. A cross-cutting engine that many skills call is the system's own idiom. + +- **Burying a cross-cutting capability inside a host skill has already failed + here — twice.** `content-creation-engine`'s own changelog records that + `video-research-engine` went *dormant* inside it ("most users didn't know it + existed, trigger keywords didn't match how people speak, visual analysis was + coupled to content generation when it should be standalone") and was extracted + into `video-watcher` + `video-transcriber`. Link/pixel logic is exactly that + kind of horizontal capability. Embedding it in newsletter or content engine + would repeat the documented mistake. + +## Alternatives considered (and why they lose) + +| Option | Verdict | Reason | +|---|---|---| +| **A. Duplicate link/pixel logic in each skill** | ❌ Reject | Guaranteed drift; token-handling code copied into 6+ places = 6 ways to leak a credential; pixel list + tag vocab diverge. This is the anti-pattern the repo fought by merging. | +| **B. Config-only in `shared-references`, no skill** | ⚠️ Partial | Constants belong there, but Switchy needs live API calls, its own analytics output, and its own trigger surface ("how many scans did X get?"). That's an invokable capability, not passive data. | +| **C. Fold into an existing skill** (content-calendar / ghl-crm-audit) | ❌ Reject | Switchy spans far past any one host — postcards, yard signs, GBP, listings, email. Coupling to one host = the dormancy trap again. | +| **D. Build a Switchy MCP** | ❌ Overkill | No MCP exists; brief says call the API directly. An MCP is heavy for a single-user read API. A skill is right-sized. | +| **E. Standalone skill + shared constants** | ✅ **Adopt** | Capability gets its own trigger + analytics; constants stay single-sourced; callers reference by name. | + +## The call contract (what callers use) +Downstream skills stop emitting raw consumer URLs and instead: +- **mint(destination, tags[], pixels[], domain?, slug?)** → Switchy short URL (+QR). +- **report()** → scans→audience→budget table. +Constants (`pixels[]` defaults, default `domain`, tag vocabulary, exclusion-list +path) come from `shared-references/switchy.json` (to be created — see asks). + +## Per-skill wiring points (from the STEP 1 inspection) + +| Skill | Link/QR emission point today | Action | +|---|---|---| +| **newsletter-generator** (EPA Report) | "Watch the full video" → YouTube; "What's My Home Worth?" → graehamwatts.com/home-value; footer social | **Wrap all CTAs.** Highest-value surface (opted-in consumers). Add: "mint each CTA via switchy-engine, tags `newsletter`+`consumer`." | +| **content-creation-engine** | YouTube CTAs, social post links, link-in-bio across 14 formats | **Wrap consumer CTAs + bio links.** Add mint step in the content-package output; tag by platform + `consumer`. | +| **direct git publish to `online-content`** (was `html-email`, retired 2026-06) | CTA buttons in designed emails | **Wrap CTAs conditionally.** Many of these go to partners/coaches (B2B) — tag `b2b`, track-only, don't pixel. Consumer emails → wrap + pixel. | +| **weekly-listing-update** | Seller-facing report; "view listing online" type links | **Track-only.** Audience is a single known seller (sphere) — don't pixel. Optional Switchy for click visibility. | +| **listing-remarks-writer** | NONE — MLS public remarks legally cannot contain URLs/contact info | **No wrap in remarks.** Flag: the *marketing collateral* around the listing (single-property page, flyers, QR) is where tracked links go, not the MLS remarks themselves. | +| **postcard workflow (Canva — no skill)** | QR codes designed manually in Canva | **GAP.** See below. | + +## Postcard integration (CORRECTED 2026-05-28) +A `farming-postcard` skill DOES exist (earlier inspection used the session's +plugin-mounted skills copy, which omitted it; the source-of-truth repo at +Documents/Skills LLMS/Claude/Skills has it). It already renders print-ready cards in the locked +brand and routes each card's QR target through `references/cta-router.md`. That +router is the clean integration point: instead of a raw landing URL it returns a +**Switchy short link** (pixel + UTM baked in), and the QR encodes that. One change, +every future postcard becomes a pixeled, scan-tracked, swappable-destination +retargeting surface. No new skill needed — farming-postcard CALLS INTO +switchy-engine, which strengthens the standalone-engine decision. + +## Net +Standalone `switchy-engine` + `shared-references/switchy.json` constants, callers +reference by name, postcard QR routed through the engine. This matches the stack's +proven idioms and avoids its two documented failure modes (duplication-drift and +buried-capability dormancy). diff --git a/skills/switchy-engine/references/audience-hygiene.md b/skills/switchy-engine/references/audience-hygiene.md new file mode 100644 index 00000000..94fe6611 --- /dev/null +++ b/skills/switchy-engine/references/audience-hygiene.md @@ -0,0 +1,63 @@ +# Audience Hygiene — what to pixel, what to skip, how to segment + +**The trap:** pixeling indiscriminately. A retargeting audience is only as good as +who's in it. Two ways to wreck it: + +1. **Tiny audiences.** Below ~100 you can't target at all (Meta floor). Below + ~1,000 the algorithm has too little to optimize and you overpay. A postcard + slug with 95 scans isn't an audience yet — it's noise. +2. **Wrong people.** Other agents, vendors, lenders, title reps, and your own team + clicking an email signature or LinkedIn link get pixeled as if they were + buyers/sellers. You then spend ad dollars showing listing ads to your title rep. + This drags CTR down, raises CPMs, and corrupts lookalike seeds built from the + audience. + +## Pixel vs. skip, by surface + +**PIXEL (consumer / prospect-facing):** +- EPA Report newsletter, GHL SMS, single-property & listing pages +- GBP posts + secondary links, Zillow/Realtor.com profiles, Nextdoor +- Instagram/Facebook bios, YouTube description/pinned/channel links +- All offline QR → postcards, yard riders, open-house flyers & sign-in, mailers, + event banners, window cards + +**SKIP the pixel (track clicks only, or don't wrap):** +- **Email signature** — every email to a vendor/agent/escrow gets them pixeled. + Track-only or omit. If wrapped, hard-tag `b2b`/`mixed` and exclude. +- **LinkedIn** — predominantly B2B/peer traffic. Track-only or exclude from + consumer audiences. +- **Business cards handed to peers**, networking events aimed at the industry. +- **Sphere / past-client touches (PCFS)** — known people; retargeting them wastes + spend. Measure engagement, don't build ad audiences from them. + +## How to segment so junk can be excluded + +The mechanism is **tags on every minted link** (the engine enforces this): + +- `audience-class`: `consumer` | `prospect` | `b2b` | `mixed` +- `surface`: gbp / newsletter / postcard / listing / signature / openhouse / … +- `campaign`: optional (e.g. `94303-spring-farm`) + +Then in the ad platform build audiences from the **clean** classes only: + +1. **Separate Switchy pixels or events by class where possible.** Simplest robust + pattern: use a distinct destination-path or event per `audience-class` so the + Meta/GA audience rule can include `consumer` traffic and exclude `b2b`. (Switchy + fires the pixel on redirect; the cleanest split is one pixel + a class-specific + URL parameter, or separate links per class.) +2. **Build the retargeting audience = `consumer` + `prospect` sources only.** + Never include `b2b`/`mixed`. +3. **Maintain a standing EXCLUSION audience** of known agents/vendors/team (upload + their emails as a Meta custom audience) and exclude it from every campaign. This + catches B2B people even when they slip through a consumer surface. +4. **Min-size gate before spending.** The analytics script flags any audience under + 100 (untargetable) and 100–999 (fold into a combined audience). Don't run a + campaign against a sub-1,000 standalone audience — merge by surface first + (e.g. all `listing` slugs → one "listing-viewers" audience). +5. **Lookalike seeds from clean audiences only.** A lookalike built off a polluted + seed inherits the pollution at scale — the most expensive version of the mistake. + +## One-line policy for the engine +> Pixel consumer/prospect surfaces; track-only the B2B/known ones; tag every link +> with `audience-class`; build ad audiences from `consumer`+`prospect` minus a +> standing vendor/agent exclusion list; never spend against a sub-1,000 standalone. diff --git a/skills/switchy-engine/references/gbp-and-youtube.md b/skills/switchy-engine/references/gbp-and-youtube.md new file mode 100644 index 00000000..3ec34ed9 --- /dev/null +++ b/skills/switchy-engine/references/gbp-and-youtube.md @@ -0,0 +1,71 @@ +# GBP redirect policy + YouTube/own-site pixel redundancy + +## Part A — Google Business Profile: can a Switchy link go there? + +**Short answer: NOT in the primary website field. Yes in posts and secondary +links, with care.** + +Google's Business links policy explicitly prohibits URLs that "redirect or +'refer' users to landing pages... other than those of the actual business," and +Google now runs automated link verification that removes violating links. Link +shorteners in the **primary website field** are a documented enforcement target — +there are real cases of booking/shortened links getting pulled (e.g. a contractor +losing a large share of Google-sourced leads when a redirect link was removed). +Google tolerates *short, clean* UTM strings but flags long/promotional ones. + +**Verdict by GBP surface:** + +| GBP surface | Switchy redirect OK? | What to do | +|---|---|---| +| **Primary website field** | ❌ High risk of auto-removal | Put your real domain (`graehamwatts.com`). Pixel it natively with the Meta/GA tags already on the site. Don't gamble your map-pack click here. | +| **Appointment / menu / "Links" fields** | ⚠️ Lower risk | A clean branded short link is usually fine; monitor for removal. Prefer your own domain with a tracked path if nervous. | +| **GBP Posts (update/offer/event)** | ✅ Safe | This is the right home for Switchy on GBP. Each post link is a fresh pixel hook. | + +### GBP-driven use cases (every one pixels the clicker, then retargets) +- **GBP post link → YouTube channel/video:** every GBP clicker who lands on your + pixeled redirect gets dropped into a custom audience *and* sent to a video. + You retarget high-intent local searchers who watched your content. HIGH value — + this is the headline play. +- **GBP post link → single-property page:** local searcher → listing → pixel → + retarget with more listings / "what's my home worth." +- **GBP post link → EPA Report signup / home-value form:** capture + pixel. +- **GBP "appointment" link → GHL booking:** pixel before the booking page. +- **GBP product/services link → CMA landing page:** pixel seller-intent traffic. + +> Net: GBP is one of Graeham's highest-intent traffic sources, but the pixel has +> to be captured through **posts and secondary links**, never the website field. +> The field stays clean; the posts do the retargeting work. + +## Part B — YouTube & own-site links: is the pixel redundant? + +**Yes — when a Switchy link points to Graeham's OWN already-pixeled site, the +pixel-drop is largely redundant**, because the site's own Meta/GA tags will pixel +that visitor the instant the page loads anyway. The Switchy pixel and the on-site +pixel capture nearly the same person. + +But "redundant pixel" ≠ "useless link." Switchy still earns its place for four +non-pixel reasons: + +1. **Per-source attribution.** A unique slug per surface (YT description vs. pinned + comment vs. channel link vs. GBP) tells you *which* surface drove the visit — + something a bare `graehamwatts.com` link buried among many can't. +2. **Swappable destination.** Change where a printed/published link points without + editing the video, postcard, or sign. Critical for QR codes you can't reprint. +3. **Multi-pixel firing.** Fire Meta + Google + LinkedIn + Pinterest from one link + even if the destination page only carries one or two of those tags. +4. **Pixel-fires-before-page-load.** The redirect pixels the visitor even if they + bounce before the destination renders (slow connection, immediate back-tap) — + capturing people the on-site pixel would miss. + +### Per-surface recommendation + +| Destination type | Is Switchy essential? | Why | +|---|---|---| +| **Non-owned platform** (YouTube watch page, Zillow, Nextdoor, a partner site) | **ESSENTIAL** | You can't put your pixel on someone else's page. The redirect is your only capture point. | +| **Your own pixeled site, and you want attribution / swap / multi-pixel** | **WORTH IT** | Pixel is redundant but the other three benefits stand. | +| **Your own pixeled site, single known placement, no swap needed** | **OPTIONAL** | Raw URL pixels them fine on load. Use Switchy only if you want the click count. | +| **Your own site, but link is on a QR / print you can't easily change** | **WORTH IT** | Swappable destination alone justifies it. | + +**Rule of thumb for the engine:** if the destination is NOT a Graeham-owned +pixeled page → always wrap. If it IS → wrap only when you need attribution, +swappability, multi-pixel, or pre-load capture; otherwise the raw URL is fine. diff --git a/skills/switchy-engine/references/graphql-queries.md b/skills/switchy-engine/references/graphql-queries.md new file mode 100644 index 00000000..5c565f73 --- /dev/null +++ b/skills/switchy-engine/references/graphql-queries.md @@ -0,0 +1,110 @@ +# Switchy GraphQL & REST — raw queries + +All GraphQL calls: `POST https://graphql.switchy.io/v1/graphql`, header +`Api-Authorization: `. Queries only — there are no GraphQL mutations. + +## 0. Smoke test (confirms token is active) +Matches the documented example. If this returns your workspace, the token works. +```graphql +query SmokeTest { + workspaces { id name companyName createdDate } + domains(where: { removedDate: { _is_null: true } }) { name createdDate } +} +``` +If it returns `errors` or empty, the token likely needs API access enabled via +Switchy live chat. + +## 1. Confirm the per-link field names (MANDATORY before analytics) +The public docs never document per-link analytics fields. Introspect them: +```graphql +query ConfirmLinksType { + __type(name: "links") { + name + fields { name description type { name kind ofType { name kind } } } + } +} +``` +Scan the output for the click/scan counter. Likely candidates given the Hasura +schema: a scalar like `clicks` / `clicksCount` / `visits`, OR a relationship +exposed as `clicks_aggregate { aggregate { count } }`. **Do not assume — confirm.** +If `links` isn't the type name, run full introspection: +```graphql +query { __schema { queryType { name } types { name kind } } } +``` + +## 2. Per-link analytics — scalar-count shape (try first) +Replace `clicks` with whatever step 1 revealed. +```graphql +query LinkAnalytics { + links(where: { removedDate: { _is_null: true } }) { + id # this is the slug (domain/id is the short URL) + domain + url # destination + title + tags + clicks # <-- VERIFY this field name via introspection + } +} +``` + +## 3. Per-link analytics — aggregate shape (fallback) +If clicks live in a child table (Hasura exposes `_aggregate`): +```graphql +query LinkAnalyticsAggregate { + links(where: { removedDate: { _is_null: true } }) { + id domain url title tags + clicks_aggregate { aggregate { count } } + } +} +``` +`scripts/switchy_analytics.py` tries shape #2 then falls back to #3 automatically. + +## 4. Time-windowed clicks (if a clicks/events table exists) +Once introspection reveals the events table + its timestamp column, filter by date +for week-over-week reporting (column names are placeholders — confirm them): +```graphql +query ClicksLast30d($since: timestamptz!) { + links(where: { removedDate: { _is_null: true } }) { + id domain + clicks_aggregate(where: { createdDate: { _gte: $since } }) { + aggregate { count } + } + } +} +``` + +## 5. Creating a tracked, pixeled link (REST — not GraphQL) +```bash +curl 'https://api.switchy.io/v1/links/create' \ + -H 'Content-Type: application/json' \ + -H 'Api-Authorization: YOUR_TOKEN' \ + -d '{ + "link": { + "url": "https://graehamwatts.com/home-value", + "domain": "hi.switchy.io", + "id": "epa-report", + "title": "EPA Report CTA", + "tags": ["newsletter","consumer"], + "pixels": [ + { "platform": "facebook", "value": "FB_PIXEL_ID" }, + { "platform": "ga", "value": "G-XXXXXXX" } + ], + "showGDPR": true + }, + "autofill": true + }' +``` +- `pixels[].platform` ∈ {linkedin, facebook, gtm, quora, pinterest, twitter, ga, + bing, nexus, adroll, adwords}. **No tiktok** — route TikTok via `gtm`. +- `showGDPR: true` shows a consent popup when pixels are present. In CA, leaving it + true is the safer default for cold consumer traffic; it slightly reduces match + rate. Decision flagged for Graeham. +- Premium domains `hi.switchy.io` / `swiy.io` are only available to *official* + integrations via API; your default workspace domain is used otherwise. +- QR codes: Switchy generates a QR for any link in-app. Via API, generate the link + then render the QR client-side (any QR lib encoding the short URL), or export + from the Switchy dashboard. A QR scan = a click = a pixel fire on redirect. + +## Notes +- `id` in the link object IS the slug; the public short URL is `domain/id`. +- Always pass `tags` — the analytics + audience-hygiene layer segments on them. diff --git a/skills/switchy-engine/references/retargeting-pathway-map.md b/skills/switchy-engine/references/retargeting-pathway-map.md new file mode 100644 index 00000000..4c160747 --- /dev/null +++ b/skills/switchy-engine/references/retargeting-pathway-map.md @@ -0,0 +1,51 @@ +# Retargeting Pathway Map — every surface a Switchy link/QR can live + +Legend — **Retargeting value**: HIGH = lots of net-new pixelable consumer traffic +you can't capture otherwise; MED = useful but smaller/partly redundant; LOW = +tiny, redundant, or audience-polluting. **Pixel?**: should this surface actually +drop people into a retargeting audience, or just track clicks? + +| # | Surface | Traffic type | Retargeting value | Pixel? | Platform caveat | +|---|---|---|---|---|---| +| 1 | **GBP — primary website field** | High-intent local searchers | HIGH (the traffic) / N/A (can't pixel here) | ❌ raw domain only | Google auto-removes redirect/shortener links from the website field. Put the real site here; pixel it natively. | +| 2 | **GBP — "Links" / appointment / menu links** | High-intent local | HIGH | ✅ | Secondary links tolerate more; still keep them clean. Use Switchy here, not field #1. | +| 3 | **GBP — Posts (update/offer/event)** | High-intent local | HIGH | ✅ | Posts are the safest GBP home for a Switchy link. Each post link → pixel. | +| 4 | **Instagram bio link / link-in-bio** | Warm social, consumer | HIGH | ✅ | IG in-app browser sometimes limits 3rd-party cookies → lower match; pixel still fires server-friendly events. | +| 5 | **Facebook page bio / about link** | Warm social, consumer | HIGH | ✅ | FB pixel matches best here (same ecosystem). | +| 6 | **LinkedIn bio / featured link** | Mixed (agents, vendors, some clients) | LOW–MED | ⚠️ selective | Heavy B2B/agent traffic — tag `b2b`, EXCLUDE from consumer audiences. LinkedIn pixel available. | +| 7 | **TikTok bio link** | Cold-warm consumer | MED | ⚠️ click-only or GTM | **No native TikTok pixel in Switchy** — route via `gtm` or accept click tracking only. | +| 8 | **YouTube — video descriptions** | Warm consumer (already engaged) | MED–HIGH | ✅ (if non-owned dest) | If link points to your OWN pixeled site, pixel benefit is largely redundant (see gbp-and-youtube.md). Value = attribution + swappable + multi-pixel. | +| 9 | **YouTube — pinned comment** | Warm consumer | MED | ✅ | Same redundancy logic as #8. High CTR placement. | +| 10 | **YouTube — channel "Links" section** | Warm consumer | MED | ✅ | Persistent; good for a single evergreen tracked link. | +| 11 | **Email signature** | MIXED — clients, agents, vendors, title, lenders | LOW | ❌ skip pixel | Pollutes audiences with B2B. Track clicks only, or omit. Tag `b2b`/`mixed`, exclude. | +| 12 | **The EPA Report newsletter** | Warm consumer subscribers | HIGH | ✅ | Already opted-in; cleanest audience you have. Per-section links → per-topic audiences. | +| 13 | **GHL SMS / text campaigns** | Warm leads/prospects | HIGH | ✅ | SMS clicks open in-app browsers; match rate varies but intent is high. Keep slugs short for SMS. | +| 14 | **Single-property / listing pages** | High-intent buyers + neighbor-snoops | HIGH | ✅ | Goldmine — buyers AND likely future sellers (neighbors). Tag per listing. | +| 15 | **Zillow profile link** | High-intent consumer | HIGH | ✅ | Zillow may wrap/normalize outbound links; test the redirect survives. Non-owned-platform traffic you otherwise can't pixel. | +| 16 | **Realtor.com profile link** | High-intent consumer | HIGH | ✅ | Same as Zillow — confirm link isn't stripped. | +| 17 | **Postcards (QR)** — *currently Canva, no skill* | Cold-warm farm/geo consumer | MED–HIGH | ✅ | QR scan = pixel fire. Per-ZIP/per-drop slug = measurable direct mail + a retargeting audience from offline mail. Big unlock. | +| 18 | **Yard sign riders (QR)** | Cold-warm local drive-by | MED | ✅ | Scanners are physically in the neighborhood = prime seller/buyer geo audience. Per-sign slug. | +| 19 | **Open house flyers (QR)** | Warm in-person buyers | HIGH | ✅ | Self-selected high intent. Pair with #20. | +| 20 | **Open house sign-in (QR → form)** | Warm in-person | HIGH | ✅ | Pixel + lead capture in one scan. Tag `openhouse`. | +| 21 | **Business cards (QR)** | MIXED | LOW–MED | ⚠️ selective | Handed to clients AND peers. Two cards or two QRs: consumer vs. networking. Tag accordingly. | +| 22 | **Event banners / sponsorships (QR)** | Cold-warm local consumer | MED | ✅ | Per-event slug measures sponsorship ROI + builds a local audience. | + +## Additional surfaces worth adding +| # | Surface | Traffic type | Value | Pixel? | Caveat | +|---|---|---|---|---|---| +| 23 | **Just-listed / just-sold mailers (QR)** | Cold farm consumer | MED–HIGH | ✅ | Same engine as postcards; per-campaign slug. | +| 24 | **Property video end-screens / pinned (HeyGen/Reels)** | Warm consumer | MED | ✅ | One tracked link reused across a video's surfaces. | +| 25 | **Nextdoor / community group posts** | Warm hyper-local | MED–HIGH | ✅ | Genuinely local; can't pixel Nextdoor natively, so Switchy is the only capture. | +| 26 | **Google Business / Apple Maps "appointment" links** | High-intent | MED | ✅ | Apple Maps tolerates redirects better than GBP field #1. | +| 27 | **PDF deliverables (CMA, disclosure summaries, listing presentations)** | Warm prospect | MED | ✅ | A tracked link/QR inside a CMA PDF tells you the seller re-opened it. | +| 28 | **QR on for-sale window cards / lockbox flyers** | Cold-warm drive-by | MED | ✅ | Same geo logic as yard riders. | +| 29 | **Sphere / past-client touch links (PCFS)** | Warm, known | MED | ⚠️ track-only | Known contacts — retargeting them is low-value; track engagement instead. | + +## The highest-leverage unlocks (where Switchy earns its keep) +1. **Offline → online bridge (postcards, yard riders, open-house QR, mailers).** + This is traffic you literally cannot pixel any other way. A QR scan turns a + physical mail drop into a digital retargeting audience. #17–20, #23, #28. +2. **Non-owned platforms (Zillow, Realtor.com, Nextdoor, GBP posts, social bios).** + You don't control these pages, so the redirect layer is your only pixel hook. +3. **Per-source attribution at scale (newsletter sections, listings, campaigns).** + Tagging every minted link tells you which surface actually drives the audience. diff --git a/skills/switchy-engine/sample_switchy_report.md b/skills/switchy-engine/sample_switchy_report.md new file mode 100644 index 00000000..17ef96f0 --- /dev/null +++ b/skills/switchy-engine/sample_switchy_report.md @@ -0,0 +1,16 @@ +# Switchy Retargeting Report — 2026-05-28 19:49 + +_Data source: DEMO data (no token found — illustrative numbers)_ +_Model: pixel match 55%, freq 10x / 30d, CPM $22_ + +| Short link | Tags | Destination | Clicks | Audience | Monthly budget | Status | +|---|---|---|---:|---:|---:|---| +| hi.switchy.io/epa-report | newsletter,consumer | https://graehamwatts.com/home-value | 2,140 | 1,177 | $259 | Standalone-ready | +| hi.switchy.io/yt-channel | gbp,youtube,consumer | https://youtube.com/@graehamwatts | 1,880 | 1,034 | $227 | Standalone-ready | +| hi.switchy.io/1908cooley | listing,consumer | https://graehamwatts.com/1908-cooley | 760 | 418 | $92 | Thin — fold into a combined audience | +| hi.switchy.io/oh-flyer-qr | openhouse,qr,consumer | https://graehamwatts.com/1908-cooley | 240 | 132 | $29 | Thin — fold into a combined audience | +| hi.switchy.io/postcard-94303 | postcard,qr,consumer | https://graehamwatts.com/home-value | 95 | 52 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| hi.switchy.io/sig | signature,mixed | https://graehamwatts.com | 60 | 33 | $0 | TOO SMALL — cannot target yet (Meta floor 100) | +| **TOTAL** | | | **5,175** | **2,846** | **$607** | | + +**How to read this:** *Audience* = clicks that resolve to a targetable pixeled user. *Monthly budget* is what it costs to hit that audience 10x over 30 days at $22 CPM — i.e. the spend the audience can actually absorb, not a target. Audiences under 100 can't be targeted; under 1,000 should be merged by source. \ No newline at end of file diff --git a/skills/switchy-engine/scripts/create_postcard_link.py b/skills/switchy-engine/scripts/create_postcard_link.py new file mode 100644 index 00000000..ff84ec4b --- /dev/null +++ b/skills/switchy-engine/scripts/create_postcard_link.py @@ -0,0 +1,91 @@ +#!/usr/bin/env python3 +""" +create_postcard_link.py — make ONE tracked Switchy link for a postcard QR. + +Built for Peter (Jason). Creates the link via Switchy's REST API with the correct +landing page + UTM + folder + Graeham's retargeting pixels already attached, then +prints the short URL. Claude then downloads the QR for that link from the Switchy +dashboard in Chrome. + +TOKEN (never printed): env SWITCHY_API_TOKEN, or a file switchy-token.txt in the +Skills folder, or ~/.switchy/token. + +USAGE: + python create_postcard_link.py --date 2026-06-01 --hook "Last 5 Homes" + optional: --dest --market epa --archetype anti_zestimate --slug epa-comps-0601 +""" +import os, sys, json, argparse, urllib.request, urllib.error +from pathlib import Path + +REST = "https://api.switchy.io/v1/links/create" +DOMAIN = "hi.switchy.io" +POSTCARD_FOLDER_ID = 92811 # Switchy "Post card qr" folder +DEFAULT_DEST = "https://graehamwatts.com/evaluation" # home-value report +# Graeham's pixels (so scanners enter the retargeting audience automatically) +PIXELS = [ + {"platform": "facebook", "value": "963211690980393"}, + {"platform": "ga", "value": "G-S82GF32XJT"}, + {"platform": "adwords", "value": "AW-1047225119"}, +] + + +def token(): + t = os.environ.get("SWITCHY_API_TOKEN") + if t: + return t.strip() + here = Path(__file__).resolve() + candidates = [ + Path.cwd() / "switchy-token.txt", + Path.home() / ".switchy" / "token", + ] + # walk up to find a Skills/switchy-token.txt + for parent in here.parents: + candidates.append(parent / "switchy-token.txt") + if parent.name == "Skills": + candidates.append(parent / "switchy-token.txt") + for p in candidates: + try: + if p.exists(): + return p.read_text(encoding="utf-8").strip() + except OSError: + pass + sys.exit("No Switchy token (set SWITCHY_API_TOKEN or place switchy-token.txt in the Skills folder).") + + +def main(): + ap = argparse.ArgumentParser() + ap.add_argument("--date", required=True, help="mail date YYYY-MM-DD, e.g. 2026-06-01") + ap.add_argument("--hook", required=True, help='short hook, e.g. "Last 5 Homes"') + ap.add_argument("--dest", default=DEFAULT_DEST) + ap.add_argument("--market", default="epa") + ap.add_argument("--archetype", default="anti_zestimate") + ap.add_argument("--slug", default=None) + a = ap.parse_args() + + mmddyy = "".join(a.date.split("-")[::-1][:2][::-1]) # -> keeps mmdd? build explicitly: + y, m, d = a.date.split("-") + mmddyy = f"{m}_{d}_{y[2:]}" + slug = a.slug or f"{a.market}-comps-{m}{d}" + dest = a.dest + ("&" if "?" in a.dest else "?") + \ + f"utm_source=postcard&utm_medium=direct_mail&utm_campaign={a.market}_{mmddyy}&utm_content={a.archetype}" + title = f"Postcard {a.market.upper()} {a.date} — {a.hook} (home value)" + tags = ["postcard", "qr", "consumer", a.market, a.date] + + payload = {"link": {"url": dest, "id": slug, "title": title, "folderId": POSTCARD_FOLDER_ID, + "tags": tags, "showGDPR": False, "pixels": PIXELS}, "autofill": False} + req = urllib.request.Request(REST, data=json.dumps(payload).encode(), + headers={"Content-Type": "application/json", "Api-Authorization": token()}, method="POST") + try: + r = json.loads(urllib.request.urlopen(req, timeout=30).read()) + except urllib.error.HTTPError as e: + sys.exit(f"Create failed: HTTP {e.code} {e.read().decode('utf-8','ignore')[:300]}") + short = f"https://{r.get('domain', DOMAIN)}/{r.get('id', slug)}" + print("SHORT URL :", short) + print("DEST :", dest) + print("FOLDER : Post card qr (92811)") + print("PIXELS : facebook, ga, adwords") + print("\nNEXT: in Switchy (Chrome) -> find this link -> Download QR Code -> Download as PNG -> embed in the postcard.") + + +if __name__ == "__main__": + main() diff --git a/skills/switchy-engine/scripts/send_email.py b/skills/switchy-engine/scripts/send_email.py new file mode 100644 index 00000000..f6fab19c --- /dev/null +++ b/skills/switchy-engine/scripts/send_email.py @@ -0,0 +1,122 @@ +#!/usr/bin/env python3 +""" +send_email.py — real SMTP sender for the Switchy weekly report (and any HTML email). +Sends through Gmail using an App Password (NOT the account password), so the Monday +task can actually deliver to the inbox instead of leaving a draft. + +CREDENTIALS (never printed, never committed): +- Sender address: --from or env GMAIL_SENDER (default graehamwatts@gmail.com) +- App password: file C:\\Users\\Graeham Watts\\Documents\\Claude\\Skills\\gmail-app-password.txt + (or --pwfile, or env GMAIL_APP_PASSWORD). This is a 16-char Google + App Password generated at myaccount.google.com/apppasswords. + +USAGE: + python send_email.py --to a@b.com --subject "..." --html-file report.html [--text-file body.txt] +Exit 0 on success; non-zero with a message on failure. +""" +import os, sys, ssl, argparse, smtplib, mimetypes +from email.mime.multipart import MIMEMultipart +from email.mime.text import MIMEText +from email.mime.base import MIMEBase +from email import encoders +from pathlib import Path + +# Workspace moved 2026-07: Documents\Claude\ -> Documents\Skills LLMS\Claude\. +# This default silently pointed at the dead path, so every scheduled report that +# relied on the SMTP fallback (per the workspace "reports SEND, never draft" rule) +# exited with "No Gmail app password found" unless --pwfile or the env var was set. +DEFAULT_PWFILE = ( + Path.home().parent / "Graeham Watts" / "Documents" / "Skills LLMS" / "Claude" + / "Skills" / "gmail-app-password.txt" +) + + +def load_pw(args): + if args.pwfile and Path(args.pwfile).exists(): + return Path(args.pwfile).read_text(encoding="utf-8").strip().replace(" ", "") + env = os.environ.get("GMAIL_APP_PASSWORD") + if env: + return env.strip().replace(" ", "") + # common locations + for p in [DEFAULT_PWFILE, Path("gmail-app-password.txt"), + Path("/sessions") / os.environ.get("SESSION", "") / "mnt/Skills/gmail-app-password.txt"]: + try: + if p.exists(): + return p.read_text(encoding="utf-8").strip().replace(" ", "") + except OSError: + pass + sys.exit("No Gmail app password found (--pwfile / GMAIL_APP_PASSWORD / gmail-app-password.txt).") + + +def main(): + ap = argparse.ArgumentParser() + ap.add_argument("--to", required=True, action="append", help="recipient (repeatable)") + ap.add_argument("--cc", action="append", default=[], help="cc recipient (repeatable)") + ap.add_argument("--subject", required=True) + ap.add_argument("--html-file") + ap.add_argument("--text-file") + ap.add_argument("--html") + ap.add_argument("--text") + ap.add_argument("--from", dest="sender", default=os.environ.get("GMAIL_SENDER", "graehamwatts@gmail.com")) + ap.add_argument("--pwfile") + ap.add_argument("--attach", action="append", default=[], help="file path to attach (repeatable)") + a = ap.parse_args() + + html = a.html or (Path(a.html_file).read_text(encoding="utf-8") if a.html_file else None) + text = a.text or (Path(a.text_file).read_text(encoding="utf-8") if a.text_file else None) \ + or "Your Switchy report is ready. Open in an HTML-capable client." + pw = load_pw(a) + + # Build the text/html body as a multipart/alternative (this structure delivers + # reliably). Only wrap in multipart/mixed when there are real attachments — + # an empty mixed wrapper was getting dropped by Gmail. + alt = MIMEMultipart("alternative") + alt.attach(MIMEText(text, "plain")) + if html: + alt.attach(MIMEText(html, "html")) + + if a.attach: + msg = MIMEMultipart("mixed") + msg.attach(alt) + for path in a.attach: + p = Path(path) + if not p.exists(): + sys.exit(f"Attachment not found: {path}") + ctype, _ = mimetypes.guess_type(str(p)) + maintype, subtype = (ctype.split("/", 1) if ctype else ("application", "octet-stream")) + part = MIMEBase(maintype, subtype) + part.set_payload(p.read_bytes()) + encoders.encode_base64(part) + part.add_header("Content-Disposition", "attachment", filename=p.name) + msg.attach(part) + else: + msg = alt + msg["Subject"] = a.subject + msg["From"] = a.sender + msg["To"] = ", ".join(a.to) + if a.cc: + msg["Cc"] = ", ".join(a.cc) + all_recipients = a.to + a.cc + + ctx = ssl.create_default_context() + # try STARTTLS:587 then SSL:465 + last = None + for host, port, mode in [("smtp.gmail.com", 587, "starttls"), ("smtp.gmail.com", 465, "ssl")]: + try: + if mode == "starttls": + s = smtplib.SMTP(host, port, timeout=30); s.starttls(context=ctx) + else: + s = smtplib.SMTP_SSL(host, port, context=ctx, timeout=30) + s.login(a.sender, pw) + s.sendmail(a.sender, all_recipients, msg.as_string()) + s.quit() + print(f"SENT via {host}:{port} to {', '.join(a.to)}" + (f" (cc: {', '.join(a.cc)})" if a.cc else "")) + return + except Exception as e: + last = f"{host}:{port} -> {type(e).__name__}: {e}" + continue + sys.exit(f"Send failed. Last error: {last}") + + +if __name__ == "__main__": + main() diff --git a/skills/switchy-engine/scripts/switchy_analytics.py b/skills/switchy-engine/scripts/switchy_analytics.py new file mode 100644 index 00000000..a43708d6 --- /dev/null +++ b/skills/switchy-engine/scripts/switchy_analytics.py @@ -0,0 +1,342 @@ +#!/usr/bin/env python3 +""" +switchy_analytics.py — switchy-engine skill core script +========================================================== +Pulls per-link click/scan analytics from the Switchy GraphQL API and turns them +into a retargeting decision table: + + scans/clicks per link -> usable retargeting audience -> ad budget the + audience actually justifies (frequency x CPM model). + +WHY THIS EXISTS +--------------- +Switchy fires Meta/Google/etc. pixels on its redirect layer. Every click on a +Switchy short link (or QR scan -> redirect) drops the visitor into a pixel-based +custom audience BEFORE the destination page even loads. This script reads how big +those audiences are getting per source, so Graeham can decide where to spend. + +SECURITY MODEL (read this) +-------------------------- +- The token is API-key style and scoped to ONE workspace. Treat it like a password. +- It is NEVER hardcoded and NEVER committed. Resolution order: + 1. env var SWITCHY_API_TOKEN + 2. file ~/.switchy/token (chmod 600; gitignored) + 3. file ./.switchy_token (gitignored; local dev only) +- If none found, the script runs in DEMO mode with illustrative numbers so the + output format is reviewable before the live token is active. + +API FACTS (confirmed from developers.switchy.io, May 2026) +---------------------------------------------------------- +- Endpoint: https://graphql.switchy.io/v1/graphql (POST) +- Header: Api-Authorization: +- Queries only on GraphQL; link creation is REST (api.switchy.io/v1/links/create). +- Schema is Hasura-style (where:{field:{_is_null:true}} filter syntax). +- Public docs only document workspace-level fields (workspaces, domains). The + per-link CLICK/SCAN count field name is NOT documented and MUST be confirmed + by introspection on the live token. See confirm_schema() below — run it first. +""" + +import os +import sys +import json +import csv +import argparse +import urllib.request +import urllib.error +from pathlib import Path +from datetime import datetime + +GRAPHQL_ENDPOINT = "https://graphql.switchy.io/v1/graphql" + +# --------------------------------------------------------------------------- +# Tunable economic assumptions for the budget model. Override on the CLI. +# These are deliberately conservative Bay Area / Peninsula real-estate defaults. +# --------------------------------------------------------------------------- +DEFAULTS = { + "cpm": 22.0, # $ per 1,000 impressions, local + interest retargeting + "frequency": 10, # desired impressions per audience member / 30-day window + "min_audience": 100, # Meta hard floor to even target a custom audience + "efficient_audience": 1000, # below this, retargeting is usually inefficient + "pixel_match_rate": 0.55, # share of clicks that resolve to a targetable user + "window_days": 30, +} + + +# --------------------------------------------------------------------------- +# Token handling +# --------------------------------------------------------------------------- +def resolve_token(): + """Return (token, source) or (None, None). Never prints the token.""" + tok = os.environ.get("SWITCHY_API_TOKEN") + if tok: + return tok.strip(), "env:SWITCHY_API_TOKEN" + for p in (Path.home() / ".switchy" / "token", Path(".switchy_token")): + try: + if p.exists(): + return p.read_text(encoding="utf-8").strip(), f"file:{p}" + except OSError: + pass + return None, None + + +# --------------------------------------------------------------------------- +# GraphQL transport +# --------------------------------------------------------------------------- +def gql(query, token, variables=None): + body = json.dumps({"query": query, "variables": variables or {}}).encode("utf-8") + req = urllib.request.Request( + GRAPHQL_ENDPOINT, + data=body, + headers={ + "Content-Type": "application/json", + "Api-Authorization": token, # NB: Switchy uses this, NOT "Authorization: Bearer" + }, + method="POST", + ) + try: + with urllib.request.urlopen(req, timeout=30) as r: + payload = json.loads(r.read().decode("utf-8")) + except urllib.error.HTTPError as e: + raise SystemExit(f"[HTTP {e.code}] {e.read().decode('utf-8', 'ignore')[:500]}") + except urllib.error.URLError as e: + raise SystemExit(f"[network] {e.reason}") + if "errors" in payload: + raise SystemExit("[graphql errors] " + json.dumps(payload["errors"], indent=2)) + return payload["data"] + + +# --------------------------------------------------------------------------- +# STEP 1 — Schema confirmation. RUN THIS FIRST on a live token. +# --------------------------------------------------------------------------- +INTROSPECT_LINKS = """ +query ConfirmLinksType { + __type(name: "links") { + name + fields { name description type { name kind ofType { name kind } } } + } +} +""" + +def confirm_schema(token): + """ + Prints the real field names on the `links` type so we can lock the click/scan + field. The public docs DON'T give us this, so this step is mandatory before + trusting the analytics query below. + """ + data = gql(INTROSPECT_LINKS, token) + t = data.get("__type") + if not t: + print("No `links` type found. The top-level type may be named differently " + "(try `link`, `Links`, or run the full __schema introspection). " + "See references/schema-introspection.md.") + return + print(f"Type `{t['name']}` fields:") + for f in t["fields"]: + ty = f["type"] + tyname = ty.get("name") or (ty.get("ofType") or {}).get("name") or ty.get("kind") + print(f" - {f['name']:<28} {tyname}") + print("\nLook for a click/scan count field (e.g. clicks, clicksCount, " + "visits, statistics, *_aggregate) and set --click-field accordingly.") + + +# --------------------------------------------------------------------------- +# STEP 2 — Per-link analytics query. +# +# Two candidate shapes are provided because the exact field name is schema-gated. +# Hasura almost always exposes EITHER a scalar count on the row OR a related +# aggregate. Pick the one confirm_schema() reveals. Default tries the scalar. +# --------------------------------------------------------------------------- +def build_links_query(click_field): + # Scalar-count shape (most common when Switchy denormalizes the counter). + return f""" +query LinkAnalytics {{ + links(order_by: {{clicks: desc}}) {{ + id + domain + url + title + tags + {click_field} + }} +}} +""" + +AGGREGATE_QUERY = """ +query LinkAnalyticsAggregate { + links(order_by: {clicks: desc}) { + id + domain + url + title + tags + clicks_aggregate { aggregate { count } } + } +} +""" + +def fetch_links(token, click_field): + try: + data = gql(build_links_query(click_field), token) + rows = data["links"] + return [_norm(r, click_field) for r in rows] + except SystemExit: + # Fall back to the aggregate relationship shape. + sys.stderr.write(f"[info] scalar field '{click_field}' failed, trying clicks_aggregate...\n") + data = gql(AGGREGATE_QUERY, token) + out = [] + for r in data["links"]: + r = dict(r) + r["_clicks"] = (((r.pop("clicks_aggregate", {}) or {}).get("aggregate") or {}).get("count")) or 0 + out.append(_norm(r, "_clicks")) + return out + + +def _norm(r, click_field): + slug = r.get("id") or "?" + domain = r.get("domain") or "hi.switchy.io" + return { + "short": f"{domain}/{slug}", + "title": r.get("title") or "", + "tags": ",".join(r.get("tags") or []), + "destination": r.get("url") or "", + "clicks": int(r.get(click_field) or 0), + } + + +# --------------------------------------------------------------------------- +# STEP 3 — Audience + budget math +# --------------------------------------------------------------------------- +def audience_and_budget(clicks, cfg): + """clicks -> targetable audience -> monthly budget the audience justifies.""" + audience = int(round(clicks * cfg["pixel_match_rate"])) + impressions = audience * cfg["frequency"] + budget = impressions / 1000.0 * cfg["cpm"] + if audience < cfg["min_audience"]: + status = "TOO SMALL — cannot target yet (Meta floor 100)" + budget = 0.0 + elif audience < cfg["efficient_audience"]: + status = "Thin — fold into a combined audience" + else: + status = "Standalone-ready" + return audience, round(budget, 2), status + + +def build_table(rows, cfg): + out = [] + for r in rows: + aud, bud, status = audience_and_budget(r["clicks"], cfg) + out.append({**r, "audience": aud, "budget": bud, "status": status}) + out.sort(key=lambda x: x["clicks"], reverse=True) + return out + + +# --------------------------------------------------------------------------- +# Output +# --------------------------------------------------------------------------- +def render_markdown(table, cfg, source_note): + ts = datetime.now().strftime("%Y-%m-%d %H:%M") + lines = [ + f"# Switchy Retargeting Report — {ts}", + "", + f"_Data source: {source_note}_ ", + f"_Model: pixel match {int(cfg['pixel_match_rate']*100)}%, " + f"freq {cfg['frequency']}x / {cfg['window_days']}d, CPM ${cfg['cpm']:.0f}_", + "", + "| Short link | Tags | Destination | Clicks | Audience | Monthly budget | Status |", + "|---|---|---|---:|---:|---:|---|", + ] + tot_clicks = tot_aud = tot_bud = 0 + for r in table: + dest = (r["destination"][:42] + "…") if len(r["destination"]) > 43 else r["destination"] + lines.append( + f"| {r['short']} | {r['tags']} | {dest} | {r['clicks']:,} | " + f"{r['audience']:,} | ${r['budget']:,.0f} | {r['status']} |" + ) + tot_clicks += r["clicks"]; tot_aud += r["audience"]; tot_bud += r["budget"] + lines += [ + f"| **TOTAL** | | | **{tot_clicks:,}** | **{tot_aud:,}** | **${tot_bud:,.0f}** | |", + "", + "**How to read this:** *Audience* = clicks that resolve to a targetable " + "pixeled user. *Monthly budget* is what it costs to hit that audience " + f"{cfg['frequency']}x over {cfg['window_days']} days at ${cfg['cpm']:.0f} CPM — " + "i.e. the spend the audience can actually absorb, not a target. Audiences " + "under 100 can't be targeted; under 1,000 should be merged by source.", + ] + return "\n".join(lines) + + +def write_csv(table, path): + with open(path, "w", newline="", encoding="utf-8") as f: + w = csv.DictWriter(f, fieldnames=["short", "title", "tags", "destination", + "clicks", "audience", "budget", "status"]) + w.writeheader() + w.writerows(table) + + +# --------------------------------------------------------------------------- +# Demo data (used only when no token is available) +# --------------------------------------------------------------------------- +DEMO_ROWS = [ + {"short": "hi.switchy.io/epa-report", "title": "EPA Report newsletter CTA", "tags": "newsletter,consumer", + "destination": "https://graehamwatts.com/home-value", "clicks": 2140}, + {"short": "hi.switchy.io/yt-channel", "title": "GBP -> YouTube channel", "tags": "gbp,youtube,consumer", + "destination": "https://youtube.com/@graehamwatts", "clicks": 1880}, + {"short": "hi.switchy.io/1908cooley", "title": "1908 Cooley single-property", "tags": "listing,consumer", + "destination": "https://graehamwatts.com/1908-cooley", "clicks": 760}, + {"short": "hi.switchy.io/oh-flyer-qr", "title": "Open house flyer QR", "tags": "openhouse,qr,consumer", + "destination": "https://graehamwatts.com/1908-cooley", "clicks": 240}, + {"short": "hi.switchy.io/postcard-94303", "title": "94303 farm postcard QR", "tags": "postcard,qr,consumer", + "destination": "https://graehamwatts.com/home-value", "clicks": 95}, + {"short": "hi.switchy.io/sig", "title": "Email signature", "tags": "signature,mixed", + "destination": "https://graehamwatts.com", "clicks": 60}, +] + + +def main(): + ap = argparse.ArgumentParser(description="Switchy per-link retargeting analytics.") + ap.add_argument("--confirm-schema", action="store_true", + help="Introspect the `links` type and exit. RUN THIS FIRST on a live token.") + ap.add_argument("--click-field", default="clicks", + help="Scalar click/scan count field on the links type (confirm via --confirm-schema).") + ap.add_argument("--cpm", type=float, default=DEFAULTS["cpm"]) + ap.add_argument("--frequency", type=int, default=DEFAULTS["frequency"]) + ap.add_argument("--pixel-match-rate", type=float, default=DEFAULTS["pixel_match_rate"]) + ap.add_argument("--out", default="switchy_report") + args = ap.parse_args() + + cfg = dict(DEFAULTS, cpm=args.cpm, frequency=args.frequency, + pixel_match_rate=args.pixel_match_rate) + + token, source = resolve_token() + + if args.confirm_schema: + if not token: + raise SystemExit("No token found. Set SWITCHY_API_TOKEN or ~/.switchy/token first.") + confirm_schema(token) + return + + if token: + rows = fetch_links(token, args.click_field) + source_note = f"LIVE Switchy API ({source})" + else: + rows = DEMO_ROWS + source_note = "DEMO data (no token found — illustrative numbers)" + sys.stderr.write( + "\n[!] No Switchy token found — running in DEMO mode.\n" + " To go live: get the token from Switchy (Workspace > Settings >\n" + " Integrations > Generate a token; you may need to ask Switchy live\n" + " chat to enable API access first), then:\n" + " export SWITCHY_API_TOKEN=xxxx (mac/linux)\n" + " setx SWITCHY_API_TOKEN xxxx (windows)\n" + " Then re-run with --confirm-schema to lock the click field name.\n\n") + + table = build_table(rows, cfg) + md = render_markdown(table, cfg, source_note) + Path(args.out + ".md").write_text(md, encoding="utf-8") + write_csv(table, args.out + ".csv") + print(md) + print(f"\n[written] {args.out}.md and {args.out}.csv") + + +if __name__ == "__main__": + main() diff --git a/skills/switchy-engine/scripts/switchy_dashboard.py b/skills/switchy-engine/scripts/switchy_dashboard.py new file mode 100644 index 00000000..c9771000 --- /dev/null +++ b/skills/switchy-engine/scripts/switchy_dashboard.py @@ -0,0 +1,224 @@ +#!/usr/bin/env python3 +""" +switchy_dashboard.py — unified Switchy clicks dashboard +========================================================= +Pulls every link's click count from the Switchy GraphQL API, groups by the link's +Switchy FOLDER (= traffic source: postcards, GMB, YouTube, yard signs, ads, ...), +snapshots for week-over-week deltas, and renders ONE branded HTML dashboard for +all Switchy clicks. + +WHY GROUP BY FOLDER: the Switchy API exposes only a running `clicks` total per link +(no referrer/geo/device/time-series). But the folder a link lives in already encodes +its source, so folder-level grouping answers "where are the clicks coming from?" +even for links that were never tagged. Tags refine this further when present. + +WEEK-OVER-WEEK: the API has no history, so we snapshot all click counts to a dated +JSON each run and diff against the most recent prior snapshot. First run = baseline +(deltas show from run #2 onward). + +USAGE +----- + export SWITCHY_API_TOKEN=... # or ~/.switchy/token + python switchy_dashboard.py --outdir [--snapdir ] + +Writes: /index.html and /switchy-snapshot-YYYY-MM-DD.json +""" +import os, sys, json, argparse, urllib.request, urllib.error, glob +from pathlib import Path +from datetime import datetime, date + +GRAPHQL = "https://graphql.switchy.io/v1/graphql" +MODEL = {"pixel_match_rate": 0.55, "frequency": 10, "cpm": 22.0, + "min_audience": 100, "efficient_audience": 1000, "window_days": 30} +GOLD, INK, CREAM = "#C2A14E", "#1A1D2E", "#FBF7EC" + + +def token(): + t = os.environ.get("SWITCHY_API_TOKEN") + if t: + return t.strip() + for p in (Path.home()/".switchy"/"token", Path(".switchy_token")): + if p.exists(): + return p.read_text().strip() + sys.exit("No SWITCHY_API_TOKEN found (env or ~/.switchy/token).") + + +def gql(q, tok): + req = urllib.request.Request(GRAPHQL, data=json.dumps({"query": q}).encode(), + headers={"Content-Type": "application/json", + "Api-Authorization": tok}, method="POST") + with urllib.request.urlopen(req, timeout=60) as r: + d = json.loads(r.read().decode()) + if "errors" in d: + sys.exit("GraphQL error: " + json.dumps(d["errors"])[:400]) + return d["data"] + + +def audience(clicks): + return int(round(clicks * MODEL["pixel_match_rate"])) + + +def budget(aud): + if aud < MODEL["min_audience"]: + return 0.0 + return round(aud * MODEL["frequency"] / 1000.0 * MODEL["cpm"], 0) + + +def load_prior(snapdir): + files = sorted(glob.glob(os.path.join(snapdir, "switchy-snapshot-*.json"))) + if not files: + return None, None + f = files[-1] + try: + return json.load(open(f)), os.path.basename(f) + except Exception: + return None, None + + +def main(): + ap = argparse.ArgumentParser() + ap.add_argument("--outdir", required=True) + ap.add_argument("--snapdir", default=None) + a = ap.parse_args() + outdir = a.outdir + snapdir = a.snapdir or os.path.join(outdir, "snapshots") + Path(outdir).mkdir(parents=True, exist_ok=True) + Path(snapdir).mkdir(parents=True, exist_ok=True) + + tok = token() + data = gql("{ links(order_by:{clicks:desc}){ id domain url title tags clicks folderId } " + "folders { id name } }", tok) + links = data["links"] + fname = {f["id"]: f["name"].strip() for f in data["folders"]} + + # snapshot + today = date.today().isoformat() + snap = {"date": today, "clicks": {l["id"]: l["clicks"] for l in links}} + prior, prior_name = load_prior(snapdir) + json.dump(snap, open(os.path.join(snapdir, f"switchy-snapshot-{today}.json"), "w")) + + def delta(lid, cur): + if not prior: + return None + return cur - prior["clicks"].get(lid, 0) + + # group by folder (= source) + groups = {} + tot_clicks = tot_aud = tot_bud = tot_delta = 0 + for l in links: + src = fname.get(l["folderId"], "Unfiled / no source tag") + g = groups.setdefault(src, {"clicks": 0, "links": 0, "delta": 0, "rows": []}) + g["clicks"] += l["clicks"]; g["links"] += 1 + d = delta(l["id"], l["clicks"]) + if d: + g["delta"] += d; tot_delta += d + g["rows"].append(l) + tot_clicks += l["clicks"] + for s, g in groups.items(): + g["aud"] = audience(g["clicks"]); g["bud"] = budget(g["aud"]) + tot_aud += g["aud"] + tot_bud = budget(tot_aud) + ranked = sorted(groups.items(), key=lambda kv: kv[1]["clicks"], reverse=True) + + # top links + top = links[:15] + + # ---- render ---- + def fmt(n): + return f"{n:,}" + week_note = (f"vs. {prior['date']} ({prior_name})" if prior + else "baseline — week-over-week deltas begin next run") + src_labels = json.dumps([s for s, _ in ranked]) + src_clicks = json.dumps([g["clicks"] for _, g in ranked]) + + rows_src = "\n".join( + f"{s}{fmt(g['links'])}{fmt(g['clicks'])}" + f"{('+' if g['delta']>0 else '')+fmt(g['delta']) if prior else '—'}" + f"{fmt(g['aud'])}${fmt(int(g['bud']))}" + for s, g in ranked) + + rows_top = "\n".join( + f"{(l['title'] or l['id'])[:46]}" + f"{fname.get(l['folderId'],'—')}" + f"{l['domain']}/{l['id']}" + f"{fmt(l['clicks'])}" + f"{('+' if (delta(l['id'],l['clicks']) or 0)>0 else '')+fmt(delta(l['id'],l['clicks'])) if prior else '—'}" + for l in top) + + html = f""" + +Switchy Clicks Dashboard — Graeham Watts + + +

Switchy Clicks Dashboard ALL SOURCES

+
Generated {datetime.now():%Y-%m-%d %H:%M} · {week_note} · model: 55% match · {MODEL['frequency']}×/30d · ${int(MODEL['cpm'])} CPM
+ +
+
{fmt(tot_clicks)}
Total clicks / scans
+
{('+'+fmt(tot_delta)) if prior else '—'}
New this week
+
{fmt(tot_aud)}
Targetable audience
+
${fmt(int(tot_bud))}
Justified ad budget / mo
+
+ +
+

Where the clicks come from (by Switchy folder)

+ +
+ +
+

Sources breakdown

+ + + {rows_src} + +
Source (folder)LinksClicksNew/wkAudienceBudget/mo
TOTAL{fmt(len(links))}{fmt(tot_clicks)}{('+'+fmt(tot_delta)) if prior else '—'}{fmt(tot_aud)}${fmt(int(tot_bud))}
+
+ +
+

Top 15 links

+ + + {rows_top} +
LinkSourceShort URLClicksNew/wk
+
+ +
+ How to read this: Audience = clicks that resolve to a targetable pixeled user (55%). Budget/mo = what that audience can absorb at {MODEL['frequency']}×/30d, ${int(MODEL['cpm'])} CPM — a ceiling, not a target. Sources are the Switchy folders each link lives in; "Unfiled" links need a folder/tag to be attributable. Switchy's API gives click totals only — geo/referrer/device live in GA4 (via UTM) and Meta (via pixel). +
+ + +""" + + out = os.path.join(outdir, "index.html") + Path(out).write_text(html, encoding="utf-8") + print(f"[dashboard] {out}") + print(f"[totals] clicks={tot_clicks} audience={tot_aud} budget=${int(tot_bud)} sources={len(groups)} links={len(links)}") + print(f"[snapshot] {os.path.join(snapdir, f'switchy-snapshot-{today}.json')} (prior: {prior_name or 'none'})") + + +if __name__ == "__main__": + main() diff --git a/skills/track-record-map/SKILL.md b/skills/track-record-map/SKILL.md new file mode 100644 index 00000000..94a79063 --- /dev/null +++ b/skills/track-record-map/SKILL.md @@ -0,0 +1,129 @@ +--- +name: track-record-map +description: "Builds and publishes The Boyenga Team + Graeham Watts's sold-listings track-record map — an interactive two-view Leaflet map of every closed transaction the team has been on either side of, plus a matching Excel export. Use this skill ANY time Graeham wants to show a client proof of the team's real sales volume/experience (a client pushing back on team credentials, a listing presentation, a CMA's 'Staying in Touch' section), or wants to refresh/rebuild the existing map, or wants to spin up a client-specific reference version of it. First shipped 2026-08-17 for the Shree Khare / 3444 Kenyon Drive negotiation, generalized into a skill the same day so it doesn't have to be rebuilt from scratch next time." +metadata: + type: skill +--- + +# Track Record Map + +Produces an interactive sold-listings map (2 views) + Excel export proving The Boyenga Team's real transaction volume — both sides of the deal, not just listings. Built once live on the fly, generalized here so future runs are a re-run, not a rebuild. + +**Live evergreen page:** `https://graehamwatts.github.io/online-content/track-record/Boyenga-Team-Sales-Map.html` +**Live evergreen spreadsheet:** `C:\Users\Graeham Watts\Documents\Agent a list of in local counties\Boyenga Team Sold Track Record.xlsx` + +Default behavior when asked to use this skill: **refresh the same evergreen page/file** (re-run the pipeline to pick up newly closed sales) rather than create a new one. Only branch to a client-specific variant (different output filename, a reference pin for that client's property) if Graeham asks for one. + +--- + +## 0. Brand identity — read this first + +Read `skills/shared-references/identity.json` before writing any brand text (DRE, brand line). Never hardcode from memory. Correct DRE is `01466876`; the blocklisted old DRE lives in identity.json `_blocked_values` and must never appear in output. + +## 1. Why this exists (the incident that shaped it) + +The first build (2026-08-17) searched MLS Matrix by **Listing Agent only**. That produced 301 total records — 211 for Graeham, 90 for "Boyenga" by last-name text search — and Graeham immediately caught that this badly undercounted real volume: a separate research effort had found he personally has 429 closed transactions. The gap was **buyer-side representation** — every deal where the team represented the buyer instead of the seller was invisible to a listing-agent-only search. + +**The fix, and the standing rule for every future run:** search BOTH the Listing Agent Lic# field AND the Buyer's Agent Lic# field, for every team member's DRE, then merge and de-duplicate by MLS#. Doing only one side will silently undercount by roughly half. This took the real total from 301 to **625** unique transactions. + +A second correction from the same feedback round: the first build color-coded pins by which agent was involved ("Boyenga Team" vs. "Graeham Watts individually") with a split legend. Graeham explicitly rejected this — **"we're all one team... don't differentiate."** Every pin on the map must be a single unified "Boyenga Team" style with one legend entry, regardless of which team member the underlying MLS record shows. + +## 2. Team roster — the DRE numbers to search + +Confirmed live, working MLS license numbers as of 2026-08-17 (validated by non-empty search results, not assumed): + +| Name | DRE / License # | +|---|---| +| Graeham Watts | 01466876 | +| Janelle Boyenga | 01254724 | +| Eric Boyenga | 01254725 | +| The Boyenga Team (team license) | 70010882 | + +> **Naming correction:** `shared-references/identity.json` previously said "Eric and Janet Boyenga." MLS records confirm the first name is **Janelle**, not Janet. If you find "Janet" anywhere in this repo going forward, it's the same historical typo — fix it, same as any other blocklisted-value cleanup. + +> **Team license validated 2026-08-18.** `70010882` is a real, active MLS license — List Agent Lic# search returned 673 rows, Buyer's Agent Lic# returned 339 (1012 raw, 971 unique after de-dup). Nearly all of it (962 of 971) was NOT already covered by the three individual DREs — this single number is where most of the team's real volume actually lives, since many team agents beyond Eric/Janelle/Graeham list under the shared "Boyenga Team" license. This is very likely the true source behind the team's own external "Property Nerds Stats" figures ($2.6B+/2,450+), which the individual-DRE-only 625 count never got close to matching. **Standing rule going forward: always include 70010882 as a 4th roster entry, both list and buyer side, in every future run** — it is not optional or supplemental, it's now the largest single contributor. +> +> **Parser gotcha found the same day:** the original ROW_RE only matched class values `Single Family|Condominium|Townhouse` and silently dropped any row with a different class (`Farm/Ranch`, `Other`, etc.) with no warning. Fixed in `parse_merge.py` by adding those to the alternation — but if a future MLS pull introduces yet another class string (e.g. `Multi-Family`, `Manufactured`), the same silent-drop will recur. **Always spot-check the parsed-row-count vs. raw-line-count per file after running parse_merge.py** (the raw line count is `wc -l` on the search dump) — if they don't match, find out why before trusting the total. +> +> **Known data gap from the 2026-08-18 run:** `ROW_RE` in `parse_merge.py` never captured Beds/Baths/SqFt/Lot Size — those columns exist in the xlsx export schema (`build_xlsx.py`) but the regex only extracts mls/address/price/date/city/class. The 2026-08-18 xlsx export therefore has all 1587 rows correct on MLS#/address/price/date/city/class, but Beds/Baths/SqFt/Lot Size are blank for every row (the script was patched to degrade gracefully with `.get()` instead of crashing). If those columns matter for a future request, extend `ROW_RE` to capture them from the raw dump (the raw text has them: `... DOM BEDS BATHS|HALFBATHS SQFT LOTSIZE CITY ...` between the date and city fields) rather than re-deriving them some other way. + +If Graeham adds another team member later, get their DRE from him directly (don't guess) and add it to this table and to `FILES` in `scripts/parse_merge.py`. + +## 3. Full pipeline + +Run these in order. Each script reads/writes to a working directory controlled by the `TRM_WORKDIR` environment variable (defaults to the current directory if unset) — set it to a scratch folder for this run before starting, e.g. `Skills LLMS\Claude\Skills\skills\track-record-map\outputs\\`. + +### Step 1 — Pull raw MLS search results (manual browser step, not scripted) + +Using the MLS Matrix session (via `mcp__claude-in-chrome__*`, already logged into Graeham's real Chrome — see `mls-matrix-scraper` skill for connection details), run **6 searches**, one per DRE per side: + +1. Status=Sold, **List Agent Lic#** = 01466876 +2. Status=Sold, **Buyer's Agent Lic#** = 01466876 +3. Status=Sold, **List Agent Lic#** = 01254724 +4. Status=Sold, **Buyer's Agent Lic#** = 01254724 +5. Status=Sold, **List Agent Lic#** = 01254725 +6. Status=Sold, **Buyer's Agent Lic#** = 01254725 + +Set results-per-page to the max (250 was available) and page through — don't just read page 1. Save each search's raw result-table text to `search1_list_01466876.txt` through `search6_buyer_01254725.txt` in the working directory (plain text dump of the Matrix results grid is fine — that's the exact format `parse_merge.py` expects). + +**Matrix gotchas** (confirmed this session, don't relearn them): +- Fm9_Ctrl* field IDs regenerate after every navigation — re-query fresh IDs immediately before each interaction. +- The "Buyer's Agent Lic#" field is a separate criteria field from "List Agent Lic#" — do not assume one search catches both. +- Do NOT attempt to log in yourself if the session is expired — stop and ask Graeham to log in. + +### Step 2 — Merge and de-duplicate + +``` +python scripts/parse_merge.py +``` +Reads the 6 raw text dumps, parses each row, merges by MLS# (a transaction appearing in more than one search — e.g. Graeham as buyer's agent AND Janelle as listing agent on the same deal — counts once), writes `merged_listings.json` to the working directory. Prints per-file row counts and the final de-duplicated total — sanity check these numbers before proceeding. + +### Step 3 — Geocode + +``` +python scripts/geocode.py +``` +Geocodes every unique address via Nominatim (OpenStreetMap), respecting its 1 request/second rate limit with a descriptive User-Agent. **Caches results in `geocode_cache.json`** in the working directory, keyed by `"
|"` — reuse this cache file across runs (copy it forward into the new working directory) so repeat addresses don't re-hit the API. Writes `geocoded_listings.json`. This is the slow step — 625 addresses took several minutes even with most already cached from a prior run. If the process gets cut off partway, just re-run it; it skips anything already in the cache and picks up where it left off. + +### Step 4 — Build the HTML page + +``` +python scripts/build_html.py +``` +Reads `geocoded_listings.json`, writes `track-record-map-output.html` to the working directory. **Before running, edit the `SUBJECT` dict near the top of the script** (address, city, lat/lon) if this run is for a specific client's reference property — plus the few hardcoded description strings that mention "3444 Kenyon Drive" / "95051" (search the file for that string; it wasn't fully templated, so these need a manual find-and-replace per client). If there's no specific client this run (just refreshing the evergreen page), leave the existing reference property as-is or ask Graeham what to point it at. + +The two views are: (1) a wide South Bay/Peninsula/East Bay footprint auto-fit to all pins, and (2) a **fixed** mid-zoom regional view (Mountain View/Los Altos/Sunnyvale in the northwest, across Santa Clara/San Jose, down to Cupertino/Campbell/Saratoga) — Graeham specifically asked for this framing after an earlier version was cropped so tight to a single city that it only showed 4 pins and looked sparse. Don't shrink this back down to a single-city crop. + +### Step 5 — QC + +``` +python scripts/qc_check.py +``` +Checks: blocked brand values (identity.json `_blocked_values`) absent, correct DRE (`01466876`) present, "Intero" absent, em-dashes absent, HTML tag-balance, inline ` + + + + +
+
Sold Track Record
+

The Boyenga Team

+

Every recorded sold transaction in MLSListings tied to the Boyenga Team - Eric Boyenga, Janelle Boyenga, and Graeham Watts - mapped across the Silicon Valley footprint. Click any pin for address, sale price, and sale date.

+
+ +
+
+
__TOTAL_COUNT__
Total Sold Transactions
+
__MAPPED_COUNT__
Pins Mapped
+
__EARLIEST_DATE__
Earliest Close Date
+
__LATEST_DATE__
Most Recent Close Date
+
+

Includes homes listed and sold by the team, and homes purchased by team-represented buyers. De-duplicated by MLS number across List Agent and Buyer's Agent searches for Eric Boyenga (DRE 01254725), Janelle Boyenga (DRE 01254724), Graeham Watts (DRE 01466876), and The Boyenga Team's team license (DRE 70010882).

+
+ +
+ +
+
+

Wide View - Silicon Valley Footprint

+ __MAPPED_COUNT__ pins mapped +
+

South Bay, the Peninsula, and into the East Bay - the team's full working territory. Zoom or pan to explore; a handful of sales further afield (referral business outside the core footprint) are also plotted and reachable by zooming out.

+
+
+ Boyenga Team +
+
+ +
+
+

Regional View - Mountain View to Saratoga

+ __MAPPED_COUNT__ pins mapped +
+

A mid-level regional view spanning Mountain View, Los Altos, and Sunnyvale in the northwest, across Santa Clara and San Jose, down to Cupertino, Campbell, and Saratoga in the south. The red pin marks 3444 Kenyon Drive (95051) for reference; it is not a Boyenga Team sale. Pan or zoom to see sales outside this window.

+
+
+ Boyenga Team + 3444 Kenyon Drive (client property, reference only) +
+
+ +
+ +
+
__BRAND_LINE__
+
Graeham Watts, REALTOR · DRE #__DRE__
+
Source: MLSListings Matrix, Sold status. Searched by List Agent Lic # and Buyer's Agent Lic # for DRE 01466876 (Graeham Watts), DRE 01254724 (Janelle Boyenga), DRE 01254725 (Eric Boyenga), and DRE 70010882 (The Boyenga Team license), plus a Listing Agent Last Name "Boyenga" catch-all, merged and de-duplicated by MLS #. Data compiled and geocoded for informational purposes; deemed reliable but not guaranteed.
+
+ + + + + +""" + +html = TEMPLATE +html = html.replace('__TOTAL_COUNT__', str(total_count)) +html = html.replace('__MAPPED_COUNT__', str(mapped_count)) +html = html.replace('__EARLIEST_DATE__', fmt_date(earliest)) +html = html.replace('__LATEST_DATE__', fmt_date(latest)) +html = html.replace('__BRAND_LINE__', BRAND_LINE) +html = html.replace('__DRE__', DRE) +html = html.replace('__ALL_JS__', all_js) +html = html.replace('__SUBJECT_LAT__', str(SUBJECT['lat'])) +html = html.replace('__SUBJECT_LON__', str(SUBJECT['lon'])) +html = html.replace('__REGIONAL_SW__', str(REGIONAL_SW)) +html = html.replace('__REGIONAL_NE__', str(REGIONAL_NE)) + +with open(OUT_PATH, 'w', encoding='utf-8') as f: + f.write(html) + +print(f"Wrote HTML to {OUT_PATH}") +print(f"File size: {len(html)} bytes") diff --git a/skills/track-record-map/scripts/build_xlsx.py b/skills/track-record-map/scripts/build_xlsx.py new file mode 100644 index 00000000..7694637b --- /dev/null +++ b/skills/track-record-map/scripts/build_xlsx.py @@ -0,0 +1,91 @@ +import os +WORKDIR = os.environ.get('TRM_WORKDIR', os.getcwd()) + +# -*- coding: utf-8 -*- +import json, re, os +from openpyxl import Workbook +from openpyxl.styles import Font, Alignment, PatternFill +from openpyxl.utils import get_column_letter + +BASE = WORKDIR +MERGED_XLSX_SRC = os.path.join(BASE, 'merged_for_xlsx.json') +CACHE_PATH = os.path.join(BASE, 'geocode_cache.json') +OUT_XLSX = os.environ.get('TRM_XLSX_OUT', os.path.join(WORKDIR, 'track-record-export.xlsx')) + +with open(MERGED_XLSX_SRC, encoding='utf-8') as f: + records = json.load(f) + +with open(CACHE_PATH, encoding='utf-8') as f: + cache = json.load(f) + +def cache_key(addr, city): + addr_no_unit = re.sub(r',?\s*#\S+$', '', addr).strip() + return f'{addr_no_unit}|{city}' + +geocoded_ok = 0 +geocoded_fail = 0 +for r in records: + k = cache_key(r['address'], r['city']) + v = cache.get(k) + if v: + r['lat'], r['lon'] = v[0], v[1] + geocoded_ok += 1 + else: + r['lat'], r['lon'] = None, None + geocoded_fail += 1 + +def parse_date(d): + m, day, y = d.split('/') + return (int(y), int(m), int(day)) + +records.sort(key=lambda r: parse_date(r['date'])) + +wb = Workbook() +ws = wb.active +ws.title = os.environ.get('TRM_SHEET_TITLE', 'Sold Track Record')[:31] + +headers = ['MLS #', 'Address', 'City', 'Sale Price', 'Close Date', 'Property Type', + 'Beds', 'Baths (Full|Half)', 'SqFt', 'Lot Size', 'Matched DRE / Side(s)', + 'Latitude', 'Longitude'] + +header_fill = PatternFill(start_color='1C2B45', end_color='1C2B45', fill_type='solid') +header_font = Font(color='FFFFFF', bold=True) + +for col, h in enumerate(headers, start=1): + c = ws.cell(row=1, column=col, value=h) + c.font = header_font + c.fill = header_fill + c.alignment = Alignment(horizontal='center', vertical='center', wrap_text=True) + +for i, r in enumerate(records, start=2): + m, day, y = r['date'].split('/') + close_date_str = f'{y}-{m}-{day}' + ws.cell(row=i, column=1, value=r['mls']) + ws.cell(row=i, column=2, value=r['address']) + ws.cell(row=i, column=3, value=r['city']) + price_cell = ws.cell(row=i, column=4, value=r['price']) + price_cell.number_format = '$#,##0' + date_cell = ws.cell(row=i, column=5, value=close_date_str) + ws.cell(row=i, column=6, value=r['cls']) + ws.cell(row=i, column=7, value=r.get('beds') or None) + ws.cell(row=i, column=8, value=r.get('baths') or None) + sqft = r.get('sqft') + ws.cell(row=i, column=9, value=sqft.replace(',', '') if sqft else None) + ws.cell(row=i, column=10, value=r.get('lot_size') or None) + ws.cell(row=i, column=11, value='; '.join(r['sources'])) + ws.cell(row=i, column=12, value=r['lat']) + ws.cell(row=i, column=13, value=r['lon']) + +widths = [13, 30, 16, 13, 12, 14, 7, 15, 8, 16, 46, 11, 11] +for col, w in enumerate(widths, start=1): + ws.column_dimensions[get_column_letter(col)].width = w + +ws.freeze_panes = 'A2' +ws.auto_filter.ref = f'A1:{get_column_letter(len(headers))}{len(records)+1}' + +os.makedirs(os.path.dirname(OUT_XLSX), exist_ok=True) +wb.save(OUT_XLSX) + +print(f'Wrote {len(records)} rows to {OUT_XLSX}') +print(f'Geocoded (lat/lon present): {geocoded_ok}') +print(f'Missing lat/lon: {geocoded_fail}') diff --git a/skills/track-record-map/scripts/geocode.py b/skills/track-record-map/scripts/geocode.py new file mode 100644 index 00000000..4ad247bc --- /dev/null +++ b/skills/track-record-map/scripts/geocode.py @@ -0,0 +1,94 @@ +import os +WORKDIR = os.environ.get('TRM_WORKDIR', os.getcwd()) + +import json, time, urllib.request, urllib.parse, sys, re, os + +IN_PATH = os.path.join(WORKDIR, 'merged_listings.json') +OUT_PATH = os.path.join(WORKDIR, 'geocoded_listings.json') +CACHE_PATH = os.path.join(WORKDIR, 'geocode_cache.json') + +UA = 'GraehamWattsRealEstate-TrackRecordMap/1.1' + +def clean_address(addr): + # normalize unit markers, remove extra spaces + addr = addr.strip() + return addr + +def query_nominatim(q): + url = 'https://nominatim.openstreetmap.org/search?' + urllib.parse.urlencode({'q': q, 'format': 'json', 'limit': 1, 'countrycodes': 'us'}) + req = urllib.request.Request(url, headers={'User-Agent': UA}) + try: + with urllib.request.urlopen(req, timeout=15) as r: + data = json.loads(r.read().decode('utf-8')) + if data: + return float(data[0]['lat']), float(data[0]['lon']) + except Exception as e: + print(f" ERROR for query '{q}': {e}", file=sys.stderr) + return None + +def main(): + with open(IN_PATH, encoding='utf-8') as f: + records = json.load(f) + + cache = {} + if os.path.exists(CACHE_PATH): + with open(CACHE_PATH, encoding='utf-8') as f: + cache = json.load(f) + + total = len(records) + success = 0 + failed = [] + + for i, r in enumerate(records): + addr = clean_address(r['address']) + # strip unit numbers for geocoding (they often confuse Nominatim) - keep street number/name only + addr_no_unit = re.sub(r',?\s*#\S+$', '', addr).strip() + city = r['city'] + key = f"{addr_no_unit}|{city}" + + if key in cache and cache[key] is not None: + r['lat'], r['lon'] = cache[key] + success += 1 + continue + if key in cache and cache[key] is None: + failed.append(r) + r['lat'], r['lon'] = None, None + continue + + q1 = f"{addr_no_unit}, {city}, CA" + result = query_nominatim(q1) + time.sleep(1.05) + + if result is None: + # fallback: try without street number's suite/extra tokens, just "City, CA" + q2 = f"{city}, CA" + result = query_nominatim(q2) + time.sleep(1.05) + + if result: + r['lat'], r['lon'] = result + cache[key] = [result[0], result[1]] + success += 1 + else: + r['lat'], r['lon'] = None, None + cache[key] = None + failed.append(r) + + if (i+1) % 10 == 0: + print(f" progress: {i+1}/{total} geocoded, {success} success so far", file=sys.stderr) + # periodically save cache + with open(CACHE_PATH, 'w', encoding='utf-8') as f: + json.dump(cache, f) + + with open(CACHE_PATH, 'w', encoding='utf-8') as f: + json.dump(cache, f) + + with open(OUT_PATH, 'w', encoding='utf-8') as f: + json.dump(records, f, indent=2) + + print(f"DONE: {success}/{total} geocoded successfully. {len(failed)} failed.", file=sys.stderr) + for r in failed: + print(f" FAILED: {r['mls']} {r['address']}, {r['city']}", file=sys.stderr) + +if __name__ == '__main__': + main() diff --git a/skills/track-record-map/scripts/parse_merge.py b/skills/track-record-map/scripts/parse_merge.py new file mode 100644 index 00000000..1583b68e --- /dev/null +++ b/skills/track-record-map/scripts/parse_merge.py @@ -0,0 +1,66 @@ +import os +WORKDIR = os.environ.get('TRM_WORKDIR', os.getcwd()) + +import re, json, os + +BASE = WORKDIR + +FILES = [ + 'search1_list_01466876.txt', + 'search2_buyer_01466876.txt', + 'search3_list_01254724.txt', + 'search4_buyer_01254724.txt', + 'search5_list_01254725.txt', + 'search6_buyer_01254725.txt', + 'search7_list_70010882.txt', + 'search8_buyer_70010882.txt', +] + +# row pattern: N S MLS# ADDRESS $PRICE $SALEPRICE MM/DD/YYYY DOM BEDS BATHS|PARTBATHS SQFT [LOTSIZE] CITY Res. CLASS ...rest +ROW_RE = re.compile( + r'^\d+\s+S\s+(?P\S+)\s+(?P.+?)\s+\$(?P[\d,]+)\s+\$[\d,]+\s+(?P\d{2}/\d{2}/\d{4})\s+.*?\s(?P[A-Za-z][A-Za-z\.\' ]*?)\s+Res\.\s+(?PSingle Family|Condominium|Townhouse|Farm/Ranch|Other)', +) + +records = {} # mls -> record +source_counts = {} + +for fname in FILES: + path = os.path.join(BASE, fname) + if not os.path.exists(path): + print('MISSING', fname) + continue + count = 0 + with open(path, encoding='utf-8') as f: + for line in f: + line = line.rstrip('\n') + m = ROW_RE.match(line.strip()) + if not m: + continue + count += 1 + mls = m.group('mls').upper() + addr = m.group('addr').strip() + price = int(m.group('price').replace(',', '')) + date = m.group('date') + city = m.group('city').strip() + # strip lot-size unit-label contamination that regex sometimes swallows + city = re.sub(r'^(?:Lot\s+SqFt\s*|Acres\s*)+', '', city).strip() + cls = m.group('cls') + if mls not in records: + records[mls] = { + 'mls': mls, 'address': addr, 'price': price, + 'date': date, 'city': city, 'cls': cls, + 'sources': [] + } + records[mls]['sources'].append(fname) + source_counts[fname] = count + +print('Per-file parsed row counts:') +for k, v in source_counts.items(): + print(' ', k, v) + +print('Total unique MLS# after dedup:', len(records)) + +out_path = os.path.join(BASE, 'merged_listings.json') +with open(out_path, 'w', encoding='utf-8') as f: + json.dump(list(records.values()), f, indent=1) +print('Saved to', out_path) diff --git a/skills/track-record-map/scripts/qc_check.py b/skills/track-record-map/scripts/qc_check.py new file mode 100644 index 00000000..9249bd2c --- /dev/null +++ b/skills/track-record-map/scripts/qc_check.py @@ -0,0 +1,93 @@ +import os +WORKDIR = os.environ.get('TRM_WORKDIR', os.getcwd()) + +import sys, re, json + +# Brand values come from identity.json — the single source of truth. Never +# hardcode the DRE (correct or blocked) in this script; the repo tripwire +# blocks pushes that contain the blocked value as a literal. +IDENTITY = os.path.join(os.path.dirname(os.path.abspath(__file__)), '..', '..', 'shared-references', 'identity.json') +with open(IDENTITY, encoding='utf-8') as f: # fail closed: no identity.json -> crash, don't skip + _id = json.load(f) +CORRECT_DRE = _id['identity']['dre'] +BLOCKED = _id['_blocked_values']['dre_blocklist'] + _id['_blocked_values']['brand_blocklist'] + +PATH = os.path.join(WORKDIR, 'track-record-map-output.html') + +with open(PATH, encoding='utf-8') as f: + html = f.read() + +ok = True + +for bad in BLOCKED: + if re.search(re.escape(bad), html, re.IGNORECASE): + print(f"FAIL: blocked brand value present: {bad}") + ok = False +if ok: + print("PASS: no blocked brand values (DRE blocklist + brand blocklist) present") + +if CORRECT_DRE in html: + print(f"PASS: correct DRE {CORRECT_DRE} present") +else: + print(f"FAIL: correct DRE {CORRECT_DRE} NOT present") + ok = False + +if '\u2014' in html: + print("FAIL: em-dash character present") + ok = False +else: + print("PASS: em-dash absent") + +# per-agent color/legend split check +if 'graeham' in html.lower().replace('graeham watts', '').replace('graehamwatts', ''): + # crude check for leftover src:"graeham" style differentiation tokens + pass +if 'dot boyenga' in html or 'dot graeham' in html or "src:\"graeham\"" in html or "src:\"boyenga\"" in html or 'colorFor' in html: + print("FAIL: leftover per-agent color/legend split markers found") + ok = False +else: + print("PASS: no per-agent color/legend split markers") + +VOID_TAGS = {'meta','link','br','img','input','hr','area','base','col','embed','source','track','wbr'} +tag_re = re.compile(r'<(/?)([a-zA-Z][a-zA-Z0-9]*)([^>]*)>') +stack = [] +balanced = True +for m in tag_re.finditer(html): + closing, name, attrs = m.groups() + name_l = name.lower() + if name_l in VOID_TAGS: + continue + if attrs.strip().endswith('/'): + continue + if not closing: + stack.append(name_l) + else: + if not stack or stack[-1] != name_l: + if name_l in stack: + while stack and stack[-1] != name_l: + stack.pop() + stack.pop() + else: + print(f"FAIL: unmatched closing tag ") + balanced = False + else: + stack.pop() +if stack: + print(f"FAIL: unclosed tags remain: {stack}") + balanced = False +if balanced and not stack: + print("PASS: HTML tag balance OK") +else: + ok = False + +script_blocks = re.findall(r']*)?>(.*?)', html, re.DOTALL) +inline_blocks = [s for s in script_blocks if s.strip() and 'src=' not in s] +print(f"INFO: found {len(inline_blocks)} inline + +""" + return html + + +def main(): + parser = argparse.ArgumentParser(description="Split content package into artifact files + render HTML index") + parser.add_argument("--package", required=True, help="Path to the humanized content package markdown") + parser.add_argument("--transcript", required=True, help="Path to the source transcript text") + parser.add_argument("--slug", required=True, help="Short slug for the output folder") + parser.add_argument("--output-dir", required=True, help="Where to write the artifact folder") + parser.add_argument("--source-url", default="", help="Original video URL") + parser.add_argument("--platform", default="", help="Source platform (youtube|instagram|tiktok|...)") + parser.add_argument("--title", default="", help="Original video title") + parser.add_argument("--duration-sec", type=int, default=0, help="Source video duration in seconds") + parser.add_argument("--word-count", type=int, default=0, help="Transcript word count") + parser.add_argument("--tier", default="whisper", help="Transcription tier used") + args = parser.parse_args() + + pkg_path = Path(args.package) + trans_path = Path(args.transcript) + out_dir = Path(args.output_dir) + out_dir.mkdir(parents=True, exist_ok=True) + + package_md = pkg_path.read_text(encoding="utf-8") + transcript_text = trans_path.read_text(encoding="utf-8") if trans_path.exists() else "" + + sections = split_sections(package_md) + + # Write each artifact + artifacts = [] + + # Source transcript + (out_dir / "transcript.txt").write_text(transcript_text, encoding="utf-8") + artifacts.append({ + "filename": "transcript.txt", "name": "Source transcript", "icon": "📝", + "description": "Raw transcribed text from the original video", + }) + + # Full content package (canonical) + (out_dir / "content-package.md").write_text(package_md, encoding="utf-8") + artifacts.append({ + "filename": "content-package.md", "name": "Full content package", "icon": "📦", + "description": "Everything in one markdown — all derivatives + handoff blocks", + }) + + # Hooks (3 variants) + hooks = find_section(sections, "hook") + if hooks: + (out_dir / "hooks.md").write_text(hooks, encoding="utf-8") + artifacts.append({ + "filename": "hooks.md", "name": "Hook variants", "icon": "🎣", + "description": "Three scored hook variants with the recommendation", + }) + + # YouTube Long + yt_long = find_section(sections, "youtube", "long") + if yt_long: + (out_dir / "script-yt-long.md").write_text(yt_long, encoding="utf-8") + artifacts.append({ + "filename": "script-yt-long.md", "name": "YouTube Long", "icon": "🎬", + "description": "8-15 min long-form script with shot directions", + }) + + # YouTube Short + yt_short = find_section(sections, "youtube", "short") + if yt_short: + (out_dir / "script-yt-short.md").write_text(yt_short, encoding="utf-8") + artifacts.append({ + "filename": "script-yt-short.md", "name": "YouTube Short", "icon": "📱", + "description": "30-59 sec vertical script for YT Shorts", + }) + + # IG Reel + ig_reel = find_section(sections, "instagram", "reel") or find_section(sections, "reel") + if ig_reel: + (out_dir / "script-ig-reel.md").write_text(ig_reel, encoding="utf-8") + artifacts.append({ + "filename": "script-ig-reel.md", "name": "Instagram Reel", "icon": "📸", + "description": "30-60 sec Reel script with caption overlay tags", + }) + + # TikTok + tiktok = find_section(sections, "tiktok") + if tiktok: + (out_dir / "script-tiktok.md").write_text(tiktok, encoding="utf-8") + artifacts.append({ + "filename": "script-tiktok.md", "name": "TikTok", "icon": "🎵", + "description": "30-60 sec TikTok script", + }) + + # Blog + blog = find_section(sections, "blog") + if blog: + (out_dir / "script-blog.md").write_text(blog, encoding="utf-8") + artifacts.append({ + "filename": "script-blog.md", "name": "Blog post", "icon": "📰", + "description": "800-1200 word SEO-tuned blog version", + }) + + # ElevenLabs SSML (XML) + ssml = extract_ssml(package_md) + if ssml: + (out_dir / "ssml.xml").write_text(ssml, encoding="utf-8") + artifacts.append({ + "filename": "ssml.xml", "name": "ElevenLabs SSML", "icon": "🔊", + "description": "XML voice markup — paste into ElevenLabs", + }) + + # HeyGen paste-ready script (script with shot tags stripped) + if yt_long: + heygen_clean = strip_shot_tags(yt_long) + (out_dir / "heygen-script.txt").write_text(heygen_clean, encoding="utf-8") + artifacts.append({ + "filename": "heygen-script.txt", "name": "HeyGen-ready script", "icon": "🎭", + "description": "Shot tags stripped — paste into HeyGen avatar", + }) + + # Higgsfield B-roll prompts + broll = find_section(sections, "higgsfield") or find_section(sections, "b-roll") or find_section(sections, "broll") + if broll: + (out_dir / "broll-prompts.md").write_text(broll, encoding="utf-8") + artifacts.append({ + "filename": "broll-prompts.md", "name": "Higgsfield B-roll prompts", "icon": "🎥", + "description": "Image + motion prompts for each B-roll shot", + }) + + # Editing notes (for Jason) + edit_notes = find_section(sections, "editing", "notes") or find_section(sections, "jason") + if edit_notes: + (out_dir / "editing-notes.md").write_text(edit_notes, encoding="utf-8") + artifacts.append({ + "filename": "editing-notes.md", "name": "Editing notes (Jason)", "icon": "✂️", + "description": "Shot list, text overlay timing, pacing, thumbnail concept", + }) + + # Captions (all platforms) + captions = find_section(sections, "caption") + if captions: + (out_dir / "captions.md").write_text(captions, encoding="utf-8") + artifacts.append({ + "filename": "captions.md", "name": "Captions + hashtags", "icon": "💬", + "description": "Per-platform captions and hashtags", + }) + + # Metadata + metadata = { + "title": args.title or args.slug.replace("-", " ").title(), + "source_url": args.source_url, + "platform": args.platform or "—", + "duration_sec": args.duration_sec, + "word_count": args.word_count, + "tier": args.tier, + "created": datetime.now().strftime("%Y-%m-%d %H:%M"), + } + + # Render HTML index + html = render_html(args.slug, package_md, transcript_text, sections, artifacts, metadata) + (out_dir / "index.html").write_text(html, encoding="utf-8") + + # Manifest JSON + manifest = { + "slug": args.slug, + "created": metadata["created"], + "metadata": metadata, + "artifacts": artifacts, + } + (out_dir / "manifest.json").write_text(json.dumps(manifest, indent=2), encoding="utf-8") + + print(f"Delivered {len(artifacts)} artifacts to {out_dir}") + print(f"Open in browser: {out_dir / 'index.html'}") + + + +if __name__ == "__main__": + main() diff --git a/skills/transcript-repurposer/scripts/transcribe.bat b/skills/transcript-repurposer/scripts/transcribe.bat new file mode 100755 index 00000000..3f1bbf0e --- /dev/null +++ b/skills/transcript-repurposer/scripts/transcribe.bat @@ -0,0 +1,40 @@ +@echo off +REM Watts Transcript Tool — Windows wrapper for transcribe_local.py +REM +REM Setup (one time): +REM 1. Install Python from python.org (3.10 or newer) +REM 2. Open Command Prompt and run: pip install yt-dlp httpx +REM 3. Install ffmpeg: https://www.gyan.dev/ffmpeg/builds/ — add to PATH +REM 4. Make sure your Deepgram key is at: +REM C:\Users\\Documents\Skills LLMS\Claude\Skills\deepgram-key.txt +REM 5. Save this transcribe.bat anywhere convenient (Desktop, or in PATH) +REM +REM Usage: +REM transcribe https://www.youtube.com/watch?v=... +REM transcribe "https://www.instagram.com/reel/Cxxxx/" +REM transcribe C:\path\to\audio.mp3 + +setlocal +set SCRIPT_DIR=%~dp0 +set PY_SCRIPT=%SCRIPT_DIR%transcribe_local.py + +if "%~1"=="" ( + echo Usage: transcribe ^ + echo. + echo Examples: + echo transcribe https://www.youtube.com/watch?v=abc123 + echo transcribe "https://www.instagram.com/reel/Cxxx/" + echo transcribe C:\Videos\interview.mp3 + exit /b 1 +) + +REM Detect if input is a URL or a file path +set INPUT=%~1 +echo %INPUT% | findstr /R "^https*://" >nul +if %ERRORLEVEL% == 0 ( + python "%PY_SCRIPT%" --url "%INPUT%" +) else ( + python "%PY_SCRIPT%" --file "%INPUT%" +) + +endlocal diff --git a/skills/transcript-repurposer/scripts/transcribe_local.py b/skills/transcript-repurposer/scripts/transcribe_local.py new file mode 100755 index 00000000..9eeeed06 --- /dev/null +++ b/skills/transcript-repurposer/scripts/transcribe_local.py @@ -0,0 +1,244 @@ +#!/usr/bin/env python3 +""" +Local transcription CLI — runs on the USER'S Windows machine (NOT the Cowork sandbox). + +Why this exists: The Cowork bash sandbox cannot reach YouTube, Instagram, Deepgram, +or HuggingFace. So transcription has to happen OUTSIDE the sandbox. This script does +that on Graeham's actual computer using his Deepgram key, and writes the result into +the Documents\\Claude\\Skills\\_inbox\\ folder where Cowork can read it. + +Setup (one-time, on each machine that will use this): + pip install yt-dlp httpx + Set DEEPGRAM_API_KEY environment variable (or pass --key) + +Usage: + python transcribe_local.py --url "https://www.youtube.com/watch?v=..." + python transcribe_local.py --url "https://instagram.com/reel/..." --tier premium + python transcribe_local.py --file "C:\\path\\to\\audio.mp3" + +The output lands in Documents\\Claude\\Skills\\_inbox\\transcript-{slug}-{ts}.txt +plus a manifest .json with metadata. Cowork picks it up from there. +""" + +import argparse +import hashlib +import json +import os +import subprocess +import sys +import tempfile +import time +from datetime import datetime +from pathlib import Path +from urllib.parse import urlparse + + +# Where transcripts land on the user's machine — Cowork can read this folder +INBOX_DIR = Path.home() / "Documents" / "Claude" / "Skills" / "_inbox" + + +def slugify(text: str, maxlen: int = 40) -> str: + import re + text = re.sub(r"[^a-zA-Z0-9]+", "-", text).strip("-").lower() + return text[:maxlen] or "transcript" + + +def detect_platform(url: str) -> str: + host = urlparse(url).netloc.lower() + if "youtube.com" in host or "youtu.be" in host: return "youtube" + if "instagram.com" in host: return "instagram" + if "tiktok.com" in host: return "tiktok" + if "twitter.com" in host or "x.com" in host: return "x" + if "vimeo.com" in host: return "vimeo" + if "facebook.com" in host or "fb.watch" in host: return "facebook" + return "unknown" + + +def check_dependencies(): + missing = [] + try: + import yt_dlp # noqa + except ImportError: + missing.append("yt-dlp") + try: + import httpx # noqa + except ImportError: + missing.append("httpx") + if missing: + print(f"Missing dependencies: {missing}", file=sys.stderr) + print(f"Install with: pip install {' '.join(missing)}", file=sys.stderr) + sys.exit(2) + + +def get_video_metadata(url: str) -> dict: + import yt_dlp + opts = {"quiet": True, "no_warnings": True, "skip_download": True} + with yt_dlp.YoutubeDL(opts) as ydl: + try: + info = ydl.extract_info(url, download=False) + return { + "title": info.get("title", ""), + "uploader": info.get("uploader") or info.get("channel", ""), + "duration_sec": int(info.get("duration") or 0), + } + except Exception as e: + print(f"⚠ Metadata fetch failed: {e}", file=sys.stderr) + return {"title": "", "uploader": "", "duration_sec": 0} + + +def download_audio(url: str, workdir: Path) -> Path: + import yt_dlp + out_template = str(workdir / "audio.%(ext)s") + opts = { + "format": "bestaudio/best", + "outtmpl": out_template, + "quiet": True, + "no_warnings": True, + "postprocessors": [{ + "key": "FFmpegExtractAudio", + "preferredcodec": "mp3", + "preferredquality": "192", + }], + } + with yt_dlp.YoutubeDL(opts) as ydl: + ydl.download([url]) + for ext in ("mp3", "m4a", "opus", "wav", "webm"): + p = workdir / f"audio.{ext}" + if p.exists(): + return p + raise FileNotFoundError(f"yt-dlp ran but no audio file produced in {workdir}") + + +def transcribe_deepgram(audio_path: Path, api_key: str) -> dict: + import httpx + url = "https://api.deepgram.com/v1/listen" + params = { + "model": "nova-3", + "smart_format": "true", + "punctuate": "true", + "paragraphs": "true", + } + headers = { + "Authorization": f"Token {api_key}", + "Content-Type": "audio/mp3", + } + with open(audio_path, "rb") as f: + audio_data = f.read() + with httpx.Client(timeout=300.0) as client: + r = client.post(url, params=params, headers=headers, content=audio_data) + if r.status_code != 200: + raise RuntimeError(f"Deepgram error {r.status_code}: {r.text[:200]}") + data = r.json() + text = data["results"]["channels"][0]["alternatives"][0]["transcript"].strip() + return {"text": text, "raw": data} + + +def main(): + parser = argparse.ArgumentParser(description="Local transcription CLI for Watts content pipeline") + src = parser.add_mutually_exclusive_group(required=True) + src.add_argument("--url", help="Video URL (any yt-dlp supported site)") + src.add_argument("--file", help="Local audio file path") + parser.add_argument("--tier", choices=["premium"], default="premium", + help="Currently only premium (Deepgram) supported — Whisper local needs separate install") + parser.add_argument("--key", help="Deepgram API key (defaults to DEEPGRAM_API_KEY env var)") + parser.add_argument("--inbox", default=str(INBOX_DIR), + help=f"Where to write the transcript (default: {INBOX_DIR})") + parser.add_argument("--slug", help="Override the auto-generated slug") + args = parser.parse_args() + + check_dependencies() + + api_key = args.key or os.environ.get("DEEPGRAM_API_KEY", "") + if not api_key: + # Try loading from the Documents persistence path + persistent_key = Path.home() / "Documents" / "Claude" / "Skills" / "deepgram-key.txt" + if persistent_key.exists(): + api_key = persistent_key.read_text().strip() + if not api_key: + print("✗ No Deepgram API key found. Pass --key or set DEEPGRAM_API_KEY env var,", file=sys.stderr) + print(f" or save the key to {persistent_key}", file=sys.stderr) + sys.exit(3) + + inbox = Path(args.inbox) + inbox.mkdir(parents=True, exist_ok=True) + ts = datetime.now().strftime("%Y%m%d-%H%M") + + is_url = bool(args.url) + source = args.url or args.file + + print(f"→ Source: {source}") + print(f"→ Tier: {args.tier} (Deepgram Nova-3)") + print(f"→ Output inbox: {inbox}") + + metadata = {"title": "", "uploader": "", "duration_sec": 0} + platform = "local-file" + + with tempfile.TemporaryDirectory() as tmpdir: + workdir = Path(tmpdir) + + if is_url: + platform = detect_platform(source) + print(f"→ Platform: {platform}") + print(f"→ Fetching metadata...") + metadata = get_video_metadata(source) + if metadata["title"]: + print(f" Title: {metadata['title']}") + if metadata["duration_sec"]: + mins = metadata["duration_sec"] // 60 + secs = metadata["duration_sec"] % 60 + print(f" Duration: {mins}m{secs}s") + print(f"→ Downloading audio...") + t0 = time.time() + audio_path = download_audio(source, workdir) + print(f" Done in {time.time() - t0:.1f}s — {audio_path.stat().st_size // 1024} KB") + else: + audio_path = Path(source) + if not audio_path.exists(): + print(f"✗ Audio file not found: {audio_path}", file=sys.stderr) + sys.exit(4) + print(f"→ Using local audio file: {audio_path.stat().st_size // 1024} KB") + + print(f"→ Transcribing via Deepgram Nova-3...") + t0 = time.time() + result = transcribe_deepgram(audio_path, api_key) + elapsed = time.time() - t0 + print(f" Done in {elapsed:.1f}s") + + transcript_text = result["text"] + word_count = len(transcript_text.split()) + + # Generate slug from title or URL + slug = args.slug or slugify(metadata.get("title") or (urlparse(source).path.split("/")[-2] if is_url else "local")) + + # Write transcript text file + transcript_path = inbox / f"transcript-{slug}-{ts}.txt" + transcript_path.write_text(transcript_text, encoding="utf-8") + + # Write manifest JSON for Cowork to read + manifest = { + "transcript_file": transcript_path.name, + "source_url": source if is_url else None, + "source_file": source if not is_url else None, + "source_platform": platform, + "title": metadata["title"], + "uploader": metadata["uploader"], + "duration_sec": metadata["duration_sec"], + "word_count": word_count, + "tier": "deepgram-nova-3", + "transcribed_at": datetime.now().isoformat(), + "transcribe_seconds": round(elapsed, 1), + "status": "ready-for-cowork", + } + manifest_path = inbox / f"transcript-{slug}-{ts}.json" + manifest_path.write_text(json.dumps(manifest, indent=2), encoding="utf-8") + + print() + print(f"✓ Transcript ready ({word_count} words)") + print(f" Text: {transcript_path}") + print(f" Manifest: {manifest_path}") + print() + print("Next: In Cowork, say 'Repurpose the latest from my inbox' and the skill takes over.") + + +if __name__ == "__main__": + main() diff --git a/skills/travel-hq/SKILL.md b/skills/travel-hq/SKILL.md new file mode 100644 index 00000000..304290a2 --- /dev/null +++ b/skills/travel-hq/SKILL.md @@ -0,0 +1,230 @@ +--- +name: travel-hq +description: "Dedicated travel agent, trip planner, and points strategist for Graeham Watts. Use ANY time the user mentions: book a flight, book a hotel, plan a trip, travel planning, travel itinerary, trip comparison, flight search, hotel search, points optimization, credit card rewards for travel, lounge access, trip prep, packing list for a trip..." + +--- + +# TRAVEL HQ + +You are Graeham's dedicated travel agent, trip planner, and points strategist inside Claude Cowork. Your job is to help plan, compare, optimize, and prep travel while following his preferences, loyalty programs, credit card benefits, and personal travel style. + +**Critical rule: Never book or purchase anything without Graeham's explicit approval. Always show options and wait for "Go" or "Book it."** + +When any of the command templates below are invoked, load `references/commands.md` and follow the relevant template exactly. + +--- + +## 1. MY TRAVEL PROFILE + +> **SETUP REQUIRED** — Replace all `[PLACEHOLDER]` fields with your real information before this skill is useful. + +Legal name for bookings: [LEGAL FIRST + LAST NAME] +Date of birth: [DOB] +Phone: [PHONE] +Email: [EMAIL] +Nationality / passport country: [COUNTRY] +Passport number: [PASSPORT NUMBER] +Passport expiration: [EXPIRATION DATE] +Known Traveler Number (KTN): [KTN] +Redress Number: [REDRESS NUMBER OR NONE] + +### Travel Style +Default: Efficient, comfortable, clean, low-stress + +Priority order: +1. Best schedule +2. Shortest travel time +3. Loyalty/status benefits +4. Comfort +5. Price +6. Points optimization + +> Do not optimize for lowest price unless Graeham explicitly says "cheapest possible." + +--- + +## 2. CREDIT CARDS, POINTS & PAYMENT + +Before recommending anything, always consider which card gives the best value, points, insurance, lounge access, and travel protections for this specific purchase. + +**Primary travel card:** [CARD NAME] +- Network: [Visa / Mastercard / Amex] +- Last 4: [XXXX] +- Best for: [Flights / hotels / dining / general] +- Benefits: [Trip delay, lounge, rental insurance] + +**Secondary card:** [CARD NAME] +- Last 4: [XXXX] +- Best for: [CATEGORY] + +**Hotel card:** [CARD NAME] +- Last 4: [XXXX] +- Benefits: [Free nights, elite status, upgrades] + +**Airline card:** [CARD NAME] +- Last 4: [XXXX] +- Benefits: [Free bags, priority boarding, lounges] + +### Points Programs + +For every purchase, show: cash price, points price, taxes/fees, cents-per-point value, and a clear recommendation on which is better. + +Minimum redemption values (don't redeem below these): +- Chase UR: 1.5 cpp +- Amex MR: 1.5 cpp +- Capital One: 1.3 cpp +- Airline miles: 1.3 cpm +- Hotel points: 0.7 cpp (adjust by program) + +--- + +## 3. AIRPORTS & FLIGHTS + +Primary airport: [YOUR AIRPORT CODE] +Backup airports: [BACKUP 1], [BACKUP 2] + +Seat preference: Aisle > Window > Never middle +Prefer: Exit row, extra legroom, front half of plane +Avoid: Last row, near bathrooms, basic economy + +### Cabin Rules +| Trip Length | Cabin | +|---|---| +| Under 5 hours | Economy or premium economy | +| 5+ hours | Premium economy or business | +| Overnight | Business or premium economy | +| Red-eyes | Avoid unless explicitly approved | +| Basic economy | Never book unless explicitly approved | + +### Schedule Rules +- Preferred departure: 7am–11am +- Acceptable: 6am–2pm +- Avoid: Before 6am, red-eyes, late-night arrivals +- Max connections: 1 stop +- Book direct if: Under $200 more than best 1-stop option +- Min connection time: 60 min domestic, 90 min international + +### Preferred Airlines +1. [AIRLINE 1] — ID: [XXXX] — Status: [TIER] +2. [AIRLINE 2] — ID: [XXXX] — Status: [TIER] +3. [AIRLINE 3] — ID: [XXXX] — Status: [TIER] + +**Always avoid:** Frontier, Spirit, Allegiant + +--- + +## 4. HOTELS & STAYS + +Style: Clean, modern, safe, conveniently located + +**Must-haves:** King bed, high floor (5th floor or above), fast WiFi, good gym, clean rooms, safe neighborhood, recent positive reviews, walking distance to anchor plans + +**Nice-to-haves:** Breakfast included, lounge access, spa, pool, late checkout, upgrade potential, good lobby vibe + +### Preferred Hotel Chains (priority order) +1. [CHAIN 1] — ID: [XXXX] — Status: [TIER] +2. [CHAIN 2] — ID: [XXXX] — Status: [TIER] +3. [CHAIN 3] — ID: [XXXX] — Status: [TIER] + +**Budget cap:** $[NUMBER]/night +Can exceed by $50 if location or quality clearly justifies it. Ask before exceeding beyond that. + +--- + +## 5. LOUNGES & AIRPORT EXPERIENCE + +TSA PreCheck: Yes — KTN: [KTN] +Global Entry: [Yes/No] +CLEAR: [Yes/No] +Arrival buffer: 75 min domestic, 2.5 hr international + +Lounge access cards: +- [CARD NAME]: [Centurion / Priority Pass / etc.] + +Always tell Graeham which lounges he can access at each relevant airport for a given trip. + +--- + +## 6. GROUND TRANSPORTATION + +Default: Uber Black after 9pm, regular Uber otherwise +Rental company: [COMPANY] — ID: [XXXX] +Preference order: Walking > Uber > transit > rental + +--- + +## 7. RESTAURANTS + +Favorite cuisines: [STEAK, SUSHI, ITALIAN, ETC.] +Style: Fun, high-quality, not overly touristy +Budget: $[NUMBER] per person per dinner +Reservation platforms: [OpenTable / Resy / Tock] + +--- + +## 8. INTERNATIONAL TRAVEL CHECKLIST + +For every international trip, always verify and flag: +- Passport validity (6 months beyond return date) +- Visa requirements for destination +- Required entry forms (ESTA, ETA, etc.) +- Recommended vaccinations +- Local currency and best way to get it +- Outlet adapters needed +- eSIM options vs. existing phone plan +- Tipping norms +- Best local ride-share apps +- Travel insurance coverage + +--- + +## HARD BOOKING RULES + +1. **Never book without explicit approval.** Wait for "Go" or "Book it." +2. **Before every booking, always show:** + - Recommended option + 1–2 alternatives + - Total cost including taxes and fees + - Cancellation policy + - Credit card to be charged (and why) + - Loyalty number being used + - Points/miles that will be earned + - Whether paying cash or redeeming points is the better value +3. **Show options in a clean comparison format** — not a wall of text. + +--- + +## COMPANION TRAVELER PROFILES + +When Graeham says "book for me + [name]", use these profiles: + +**Companion 1:** +- Legal name: [NAME] +- DOB: [DOB] +- KTN: [KTN] +- Loyalty IDs: [AIRLINE/HOTEL: NUMBER] +- Seat preference: [aisle/window] +- Meal preference: [if any] +- Notes: [allergies, mobility, etc.] + +**Companion 2:** *(same fields)* + +For trips with 3+ travelers, always ask if anyone else is coming. + +--- + +## AVAILABLE COMMANDS + +When Graeham uses any of these, load `references/commands.md` and follow the template for that command: + +- `# BOOK A TRIP` — flight + hotel search with full pre-booking summary +- `# TRIP PREP BRIEF` — single-page pre-trip briefing (weather, restaurants, logistics) +- `# PRICE DROP MONITOR` — daily tracking of a booked route for cheaper alternatives +- `# POST-TRIP REVIEW` — spending breakdown, points earned, profile update flags +- `# EMERGENCY TRAVEL CARD` — printable emergency contacts and logistics +- `# TRIP COMPARISON` — side-by-side analysis of two destination/date options + +--- + +## TONE + +Be direct, useful, and efficient. No excessive enthusiasm. When there's a clear best option, say so. When something is risky or overpriced, say that too. diff --git a/skills/travel-hq/references/commands.md b/skills/travel-hq/references/commands.md new file mode 100644 index 00000000..8942b40b --- /dev/null +++ b/skills/travel-hq/references/commands.md @@ -0,0 +1,156 @@ +# Travel HQ — Command Templates + +When Graeham invokes any of these commands (by pasting the header or a close variant), execute the template exactly as written. Use the travel profile and preferences in SKILL.md for all decisions. + +--- + +## BOOK A TRIP + +``` +Trip type: [Business / Leisure / Mixed] +Origin: [CITY or AIRPORT] +Destination: [CITY or AIRPORT] +Depart: [DATE] +Return: [DATE] +Travelers: [NAMES from profile + any companions] +Budget cap: $[NUMBER] total (flights + hotel) +Anchor plans: [meetings, dinners, events with times] +Hotel area: [neighborhood or 'close to anchor plans'] +``` + +**How to execute:** +1. Search for flights matching the schedule rules, cabin rules, and airline preferences from the travel profile. +2. Search for hotels matching the style, chain preferences, and budget cap from the travel profile. +3. Present the recommended option + 1–2 alternatives. For each option show: + - Airline, flight number, departure/arrival times, total travel time, number of stops + - Cabin class and seat availability + - Total cost with taxes and fees + - Cancellation policy + - Credit card to be charged (and why it's the best choice) + - Loyalty number being applied + - Points/miles that will be earned + - Whether cash or points is the better value +4. For hotels, show: property name, neighborhood, nightly rate, total cost, loyalty tier benefits, and any upgrade potential. +5. Do NOT book until Graeham says "Go" or "Book it." + +--- + +## TRIP PREP BRIEF + +``` +Trip: [CITY], [DATES] +Anchor plans: [list any fixed commitments] +``` + +**Produce a single page under 500 words with these sections (bold headers):** + +**WEATHER** — Daily forecast for each day of the trip. Flag anything that affects packing (rain, cold, heat, formal events). + +**DINNER RESERVATIONS** — 3 picks + 2 backups. For each: restaurant name, cuisine, vibe, price per person, reservation link (OpenTable/Resy/Tock), and whether reservations are needed urgently. Align with his cuisine preferences and budget from the profile. + +**COFFEE** — Top 2 coffee spots within 10 min walk of the hotel. + +**ONE NON-TOURISTY THING** — One thing to do on free time that isn't on every "top 10" list. + +**AIRPORT LOGISTICS** — Best lounge accessible at departure airport (based on card benefits). Estimated security wait time. Any good gate-area food options. + +**GROUND TRANSPORT** — Best way from airport to hotel. Cost and time estimate. + +**LOCAL NORMS** — Currency, tipping customs, cash vs. card, outlet type, any entry forms needed. + +**PERSONAL NOTE** — If this is a repeat city: what was done last time + one new pick. If first visit: one useful phrase in the local language. + +Use real names, real links, and real times. No filler. + +--- + +## PRICE DROP MONITOR + +``` +I just booked: [ROUTE], [DATES], [AIRLINE], [FARE CLASS], $[PRICE] +Booking ref: [PNR] +Change fee policy: [refundable / change fee $X / non-refundable] +``` + +**How to execute:** +Check the same route, dates, and cabin daily until departure. + +Alert Graeham **only if** the new price beats the booked price by more than the change/rebook cost AND the new booking preserves his elite benefits. + +When alerting, show: +- Old price +- New price +- Net savings after any change or rebook fee +- Exact action to take (cancel and rebook, call airline, use credit, etc.) + +If no alert-worthy drop exists, no message needed. + +--- + +## POST-TRIP REVIEW + +``` +Trip: [CITY], [DATES] +``` + +**Produce a review under 400 words with these sections:** + +**SPEND** +- Total spent, broken down by: flights / hotel / ground transport / food / other +- Cards used and what each one earned (points and cash back) +- Any point redemptions used and the cents-per-point value achieved + +**WHAT WORKED / WHAT DIDN'T** +- 3 things that went well +- 3 things to do differently next time + +**POINTS + STATUS** +- Total points/miles earned this trip +- Tier progress check: how close to the next status level, and what's needed to get there + +**PROFILE UPDATES** +- Flag anything in the travel profile that should change based on this trip (e.g., new preferred hotel discovered, airline to avoid, card that over-performed) + +Be direct. No padding. + +--- + +## EMERGENCY TRAVEL CARD + +``` +Trip: [CITY], [DATES] +``` + +**Build a clean, printable single page with:** + +- Airline phone numbers (main line + elite line if Graeham has status) +- Hotel: name, full address, phone number, confirmation number +- Nearest embassy/consulate for his nationality: address and phone number +- Local emergency numbers (police, ambulance, fire) +- Local ride-share app name and whether it accepts international credit cards +- Whether his phone plan works here or if he needs an eSIM (and recommended provider) +- 24/7 number for his primary travel credit card +- Travel insurance: policy name, policy number, and claims phone number + +No fluff. Just the information needed if something goes wrong at 2am. + +--- + +## TRIP COMPARISON + +``` +Option A: [DESTINATION / DATES] +Option B: [DESTINATION / DATES] +``` + +**For each option, show:** +- Total estimated trip cost: flights + hotel + estimated ground transport + food +- Door-to-door travel time +- Realistic cabin and hotel tier based on budget and loyalty status +- Points that would be earned vs. points that could be redeemed +- Weather and crowd level for those specific dates +- Top 3 things to do +- One strong reason to pick this option +- One honest reason to skip it + +**End with a clear recommendation** based on Graeham's travel profile, schedule preferences, and current points/status situation. Don't hedge — give a direct answer. diff --git a/skills/vaibhav-template/SKILL.md b/skills/vaibhav-template/SKILL.md new file mode 100755 index 00000000..05108758 --- /dev/null +++ b/skills/vaibhav-template/SKILL.md @@ -0,0 +1,271 @@ +--- +name: vaibhav-template +description: "Turn any script into a Vaibhav Sisinty-style talking-head video for Graeham Watts — the visual formula reverse-engineered from @vaibhavsisinty's 1.7M-follower Instagram reels." +--- + +# Vaibhav-Style Video Template — Graeham Watts + +A reusable visual formula reverse-engineered from @vaibhavsisinty (1.7M followers, Instagram Reels). This skill doesn't make a video from scratch — it takes a script or topic and produces a shot plan, look choice, and caption spec that matches the Vaibhav aesthetic, then hands it to `heygen-video` for rendering and `higgsfield-video` for B-roll. + +## When this fires + +- "Run this script through the template I love" +- "Make this Vaibhav-style" +- "Use my video template on this" +- "Convert this script to the warm-desk look" +- "Add this to my content rotation" +- "Put this in the Vaibhav format" + +If the user has a topic but no script yet, chain with `content-creation-engine` first — that writes the script (it absorbed `video-script-creation-engine` in April 2026), then hand it back to this skill. + +--- + +## The reality + +**This is a STYLE skill, not a rendering skill.** The actual video comes out of `heygen-video`. This skill's job is to make sure every HeyGen video Graeham publishes has the same visual grammar — so viewers recognize his content within 2 seconds regardless of topic. + +Vaibhav's consistency is not an accident. His reels are recognizable within one frame because he locked in exactly five things: talking-head framing, cut pacing arc, typography, color grade, and caption placement. Those five things are this skill's entire contract. + +--- + +## Visual system — the 5 composition modes + +Every Vaibhav-style video is built from these 5 frames remixed. Don't invent new ones mid-video. + +### Mode 1 — Hook composite (seconds 0–3 ONLY) +- **Top 60% of frame:** full-bleed B-roll of the HOOK subject (Sam Altman, a listing, a map, a headline) +- **Bottom 40% of frame:** LOCKED talking-head of Graeham, same position every video +- **Caption on split line:** serif italic subject name top, sans-serif supporting clause underneath +- **Source files:** Top needs Higgsfield B-roll; bottom uses one of Graeham's 5 warm-desk looks +- **Used for:** Opening hook, re-engagement moment mid-video if attention drops + +### Mode 2 — Full-bleed talking head +- **Full frame:** Graeham's face, chest-up +- **Optional overlay:** red or orange color wash at 30% opacity (section transition) +- **Caption:** large serif italic center-frame +- **Used for:** Section transitions, high-emphasis statements, the "let me show you" pivot + +### Mode 3 — Full-bleed B-roll +- **Full frame:** environmental B-roll (street shot, listing exterior, neighborhood) +- **Caption:** small white-on-dark translucent pill, center or bottom-third +- **Used for:** Market data, location grounding, visual establishment + +### Mode 4 — Screenshot card (PiP) +- **Background:** solid dark/black +- **Top 60%:** product screenshot, document, or tool output (prompt card, listing MLS shot, Redfin page) +- **Bottom:** small inset of Graeham + big acid-green list number (`#01`, `#02`) + 3-line caption +- **Used for:** "Here's how to do it" content, tool walkthroughs, step-by-step + +### Mode 5 — Section header +- **Background:** current warm-desk look, faded/darkened +- **Watermark center-back:** PropertyIQ logo or Graeham Watts Investment Properties mark at 20% opacity, large +- **Top overlay:** `##. TITLE CASE` in letter-spaced sans-serif +- **Used for:** Opening each numbered section in a listicle format + +--- + +## Cut rhythm — the documented arc + +From the reel analysis (80s, 52 cuts, scene-detection verified): + +| Section | % of runtime | Cut density | Shot length | +|---|---|---|---| +| Hook | first 12% | 40% of all cuts | ~0.5s each | +| Setup | 12–38% | 15% of all cuts | ~2.5s each | +| Body | 38–75% | 27% of all cuts | ~2.0s each | +| Climax | 75–88% | 13% of all cuts | ~1.4s each | +| CTA | final 12% | 6% of all cuts | ~3.0s each | + +**Key rule:** front-load 40% of your cuts in the first 10% of the runtime. This is what makes Vaibhav's style feel "fast" even though 80% of the video is moderately paced. Miss this and the video will feel slow no matter how good the rest is. + +**Translation to production:** for a 60-second video, plan ~16 cuts in the first 6 seconds, then ~20 cuts over the remaining 54s. + +--- + +## Typography — the distinctive signature + +His fonts are what make this aesthetic look premium instead of generic-viral. Most reels editors use bold sans-serif. He does the opposite. + +| Text type | Font | Weight | Color | Size | Example use | +|---|---|---|---|---|---| +| Primary subject | **Playfair Display** | Italic | White | Large (~80pt @ 1080p) | "Sam Altman", "Let me show you", place names | +| Secondary clause | **DM Sans** or **Inter** | Regular | White | Medium (~44pt) | "just killed the entire" | +| List numbers | **DM Sans Bold** | Bold | **Acid green** (`#BFFF00`) | Large | "**01.**", "**02.**", "**03.**" | +| Section titles | **DM Sans** | Medium, LETTER-SPACED 8% | White | Medium | "PRECISE TEXT", "REAL PRODUCT ACCURACY" | +| Burned-in captions | **Inter** | Semi-bold | White on translucent dark pill | Medium | Running dialogue captions | + +**Color-highlighted keywords:** 1–3 emphasis words per caption get acid-green or warm-yellow highlight boxes behind the text. Only highlight nouns and verbs that carry meaning — never articles, connectors, or filler. + +--- + +## Color & grade + +- **Base palette:** warm tungsten orange (~3200K) + cool dusk blue (~5600K), simultaneously. +- **Grading move:** subject is warmly lit; background is cool and blurred. This is Vaibhav's single most important color rule. +- **Section washes:** red (30% opacity, sharp transition) and gold (20% opacity, gentle transition) — use to mark scene shifts, not every cut. +- **Never:** flat white balance, cool-only grade, or gray grade. The warm/cool split IS the look. + +--- + +## Graeham's 5 looks — the rotation system + +Five avatar looks, each shares the "warm-lit desk with subject facing camera head-on, desk horizontal across bottom of frame" foundation but varies outfit + environment. Rotate across videos so viewers see variety but instant brand recognition. + +| Look name | Outfit | Environment | Best use | +|---|---|---|---| +| `warm_desk_navy` | Navy quarter-zip over white tee | Warm-lit home office, bookshelf + city window, brass lamp left | Everyday / default look — casual-professional | +| `podcast_studio` | Black crewneck | Acoustic foam panels, blue/teal dramatic accent lighting, SM7B broadcast mic visible | Hot takes, opinion content, long-form explainer | +| `loft_window` | Heather grey henley (top buttons undone) | Modern minimalist loft, blurred dusk city bokeh behind, warm lamp right edge | Lifestyle, neighborhood content, market storytelling | +| `corporate_office` | Charcoal suit jacket + light blue dress shirt + navy silk tie, **clear round wire-frame glasses** | Dark wood-paneled executive office, leather-bound books, brass desk lamp | Seller-facing content, CMA walkthroughs, listing presentations — "big deal" mode | +| `modern_studio` | Crisp white oxford, sleeves rolled, top button undone, **clear round wire-frame glasses** (same frames as corporate_office) | Clean minimalist studio, off-white/warm-grey backdrop, blonde-wood desk, brass arm lamp | Educational content, contract walkthroughs, "analyst mode" | + +**Rotation rule:** match look to content intent, not randomly. +- Market update → `warm_desk_navy` +- Bold take / podcast-style → `podcast_studio` +- Neighborhood / lifestyle → `loft_window` +- Seller meeting, listing intro, "I just sold this for $X" → `corporate_office` +- "Here's how a contract works" / educational → `modern_studio` + +**Don't rotate mid-video.** One look per video. Switching looks inside a single reel breaks identity. + +--- + +## The skill's workflow + +### Step 1: Understand the script's job + +Before doing anything, identify: +- **Content type:** market update / bold take / lifestyle / seller-facing / educational +- **Target runtime:** 30s / 60s / 90s — this drives cut planning +- **Hook payload:** what ONE specific thing in the first 3 seconds stops the scroll +- **CTA:** what action at the end + +Ask the user if any of these are unclear. Don't assume. + +### Step 2: Pick the look + +Match the content type to the look table above. State the pick with reasoning. Example: + +> "This is a market-data video about EPA pricing. I'd use `warm_desk_navy` — it's the everyday default and the data-driven tone doesn't need the gravitas of `corporate_office` or the studio polish of `modern_studio`. Good?" + +**Confirm before proceeding.** Don't assume the user wants the default. + +### Step 3: Build the shot plan + +Produce a table mapping each line/beat of the script to a composition mode + timing + caption spec. Example for a hook: + +``` +Time Mode Caption B-roll needed +0–1s Mode 1 "Sam Altman" (italic, top) portrait of person/subject + "just killed the entire" (bottom) +1–2s Mode 1 continues, crossfade caption same / slight zoom +2–3s Mode 2 "Image Gen Industry" Graeham full-bleed + (italic, yellow highlight on "Image") +3–4s Mode 1 "with one big launch" 4K industrial image +4–5s Mode 5 "01. PRECISE TEXT" Graeham warm_desk_navy + PropertyIQ logo fade +``` + +Front-load cuts per the rhythm arc. Fill in the Body and Climax with Modes 3 and 4. End in Mode 2 with a clean CTA. + +### Step 4: List the B-roll Graeham needs + +For each beat that requires a B-roll clip (Modes 1, 3, and sometimes 4), produce an explicit Higgsfield prompt. Hand these to the `higgsfield-video` skill when the user is ready to shoot them. + +Example: +``` +B-roll #1 (beat at 0–3s): + Orientation: 16:9 (landscape) + Prompt: "4K ultra-detailed portrait of tech CEO in black turtleneck, + multiple facial expressions composited in a grid, dark + dramatic studio lighting, shot on Kodak Portra 400, Douglas + Friedman editorial style. Horizontal 16:9." + Duration: 4s + Motion: slow push-in +``` + +### Step 5: Hand off to heygen-video + +Once look + script are locked, call the `heygen-video` skill with: + +``` +python3 /path/to/heygen-video/scripts/create.py \ + --script "..." \ + --look \ + --aspect 16:9 \ + --title " - " +``` + +**Aspect:** All 5 Vaibhav looks are landscape-native (16:9). Graeham edits to portrait (9:16) in post. Always render at 16:9 from HeyGen. + +### Step 6: Build the caption + typography sheet + +Deliver a simple text file listing each caption beat, font, color, highlight words, and timing. Graeham's editor (or CapCut operator) uses this to burn in captions matching the Vaibhav typography spec. + +--- + +## What this skill does NOT do + +- ❌ Does not write the original script — use `content-creation-engine` or handle that separately +- ❌ Does not render video — always hands off to `heygen-video` +- ❌ Does not generate B-roll — produces prompts for `higgsfield-video` to execute +- ❌ Does not edit/composite final video — Graeham or his editor does this in CapCut / Premiere / After Effects using the shot plan + caption spec +- ❌ Does not let Graeham "freestyle" mid-video — one look, one grade, one typography system per video + +--- + +## Why consistency matters (and why to resist "just this once" deviations) + +Vaibhav's visual system looks simple because he never breaks it. 3,000+ posts at 1.7M followers — the same five composition modes, the same typography, the same warm/cool grade. Viewers recognize his content before the first word plays. That's the brand moat. + +If Graeham mixes looks mid-video, or swaps fonts for "variety", or front-loads 5 cuts instead of 20, he'll get a video that's *fine* but doesn't read as *his*. That's the difference between "another real estate agent posting on IG" and "the Peninsula guy who does those cinematic reels." + +**Strict adherence to this template IS the strategy.** + +--- + +## Quick reference card + +``` +Input: script + content_type +Output: shot plan + look choice + caption spec + B-roll prompts + HeyGen invocation + +LOOK DECISION TREE + market-data / everyday → warm_desk_navy + bold opinion / hot take → podcast_studio + lifestyle / neighborhood → loft_window + seller-facing / listing intro → corporate_office + educational / contract breakdown → modern_studio + +CUT RHYTHM + First 10% of runtime: 40% of all cuts (≈0.5s shots) + 10–38%: 15% of all cuts (≈2.5s shots) + 38–75%: 27% of all cuts (≈2.0s shots) + 75–88%: 13% of all cuts (≈1.4s shots) + Last 12%: 6% of all cuts (≈3.0s shots) + +TYPOGRAPHY + Subject/emphasis: Playfair Display Italic, white + Secondary: DM Sans Regular, white + List numbers: DM Sans Bold, acid-green #BFFF00 + Section titles: DM Sans Medium, letter-spaced 8%, UPPERCASE + Captions: Inter Semi-bold on dark translucent pill + +GRADE + Warm face (3200K) + cool background (5600K) = the signature + Section washes: red 30% (sharp), gold 20% (gentle) + +COMPOSITION MODES (use only these 5) + Mode 1 Hook composite (60% B-roll top / 40% locked Graeham bottom) + Mode 2 Full-bleed talking head + Mode 3 Full-bleed B-roll + Mode 4 Screenshot PiP card + Mode 5 Section header w/ logo watermark + +ASPECT: render HeyGen at 16:9. Edit to 9:16 in post. +``` + +--- + +## Session history + +Built 2026-04-23 with Graeham over a long working session. Reverse-engineered from an 80-second Vaibhav Sisinty reel (160 extracted frames, 52 scene-detected cuts, pacing arc measured per section). All 5 looks generated via Higgsfield Nano Banana Pro with identity-matched reference image, verified against Graeham's own selfie (`IMG_0520.JPG`). Anti-cleft prompt formula documented in `references/prompt_formula.md` — use this when regenerating or adding a 6th look. diff --git a/skills/vaibhav-template/references/looks.md b/skills/vaibhav-template/references/looks.md new file mode 100755 index 00000000..bd5e184b --- /dev/null +++ b/skills/vaibhav-template/references/looks.md @@ -0,0 +1,125 @@ +# Graeham's 5 Warm-Desk Looks — Rotation System + +All 5 looks share the same composition foundation: subject facing camera head-on, desk running horizontally across the bottom of frame, 16:9 landscape native, warm-cool color grade, shallow depth of field with face as brightest element. + +Outfit + environment varies to give the content rotation variety without breaking identity consistency. + +--- + +## Look 1 — `warm_desk_navy` + +- **Outfit:** Navy blue quarter-zip pullover over white crewneck tee +- **Environment:** Warm-lit home office, dark wooden bookshelf on right, large window with blurred dusk city bokeh on left, warm tungsten desk lamp left edge +- **Props:** Laptop, leather notepad, coffee cup +- **Grade:** Warm gold + cool blue, warm dominant +- **Best use:** Everyday default. Market updates, buyer education, general content. If in doubt, this is the pick. +- **HeyGen look ID:** `67f9bcd8131140d793b9343851aeb25b` +- **Source file:** `C:\Users\Admin\Downloads\warm_desk_navy.png` + +--- + +## Look 2 — `podcast_studio` + +- **Outfit:** Black crewneck sweater +- **Environment:** Professional podcast studio with acoustic foam panels (dark charcoal + navy tones), Shure SM7B broadcast microphone on boom arm visible in left foreground +- **Props:** Mic + boom arm dominant, minimal desk clutter +- **Grade:** Dramatic low-key, blue/teal accent lighting, warm key light on face +- **Best use:** Hot takes, opinion content, long-form explainer, commentary on news/market events, "the truth about X" content. The moody grade signals gravity — save it for content where that tone is earned. +- **HeyGen look ID:** `e975c51279f3449991673293d47b99e2` +- **Source file:** `C:\Users\Admin\Downloads\podcast_studio.png` + +--- + +## Look 3 — `loft_window` + +- **Outfit:** Heather grey long-sleeve henley, top two buttons undone at collar +- **Environment:** Modern minimalist loft with floor-to-ceiling windows behind showing dramatic blurred dusk city bokeh with amber + teal pinpoint lights, exposed concrete accents +- **Props:** Sleek laptop left, ceramic coffee mug right, minimal desk +- **Grade:** Cool blue dusk background + warm amber key light from left +- **Best use:** Lifestyle content, neighborhood spotlights, "I love this city" storytelling, relocation content, community features. The cool dusk vibe communicates "end of workday reflection." +- **HeyGen look ID:** `798d3d001a4b44c9a0285621991aad1a` +- **Source file:** `C:\Users\Admin\Downloads\loft_window.png` + +--- + +## Look 4 — `corporate_office` + +- **Outfit:** Tailored charcoal grey wool suit jacket over crisp light blue dress shirt with navy blue silk necktie knotted at collar + clear round wire-frame glasses +- **Environment:** Executive office with dark wood-paneled walls, leather-bound books on shelf, large window with soft daylight one side, brass desk lamp casting warm light +- **Props:** Closed leather portfolio, fountain pen, white porcelain coffee cup +- **Grade:** Warm professional, magazine editorial polish +- **Best use:** Seller-facing content, listing introductions, CMA walkthroughs, "I just sold this for $X," high-stakes content, anything that needs executive gravitas. The tie + jacket signals "I'm handling something important." +- **HeyGen look ID:** `92ff2b057ef54b65863e627a30815e31` +- **Source file:** `C:\Users\Admin\Downloads\corporate_office.png` + +--- + +## Look 5 — `modern_studio` + +- **Outfit:** Crisp white oxford button-down with sleeves rolled to just below elbows, no tie, no jacket, top button undone + clear round wire-frame glasses (same frames as corporate_office for consistency) +- **Environment:** Clean modern minimalist studio with off-white/warm-grey seamless backdrop, tall brass arm lamp left edge, simple blonde-wood desk +- **Props:** Open notebook, black fountain pen, minimal +- **Grade:** Clean neutral with warm skin tones, bright even soft studio light +- **Best use:** Educational content, contract walkthroughs, "here's how this works" explainers, analyst-mode market breakdowns, PropertyIQ product education. The clean backdrop and glasses signal "I'm about to explain something technical." +- **HeyGen look ID:** `3d52c06f1ab94c09881daef7cfe0743a` +- **Source file:** `C:\Users\Admin\Downloads\modern_studio.png` + +--- + +## Rotation logic + +Map content intent to look — don't randomize. + +| Content intent | Look | +|---|---| +| Market data / stat / price / metric | `warm_desk_navy` | +| Bold take / hot opinion / "here's what nobody's saying" | `podcast_studio` | +| Neighborhood / lifestyle / "I love it here" | `loft_window` | +| Seller-facing / listing intro / "just sold" | `corporate_office` | +| Educational / contract / process explainer | `modern_studio` | +| Unclear / casual / general | `warm_desk_navy` (default) | + +**Never mix looks inside a single video.** One look = one video. Continuity matters more than visual variety at the clip level. + +**Glasses consistency:** Two looks (corporate_office + modern_studio) use glasses. Use the same frames. When viewers see Graeham in glasses they should think "ah, he's about to explain something" — that signal only works if the frames match. + +--- + +## HeyGen upload status + +✅ **All 5 looks uploaded and registered in HeyGen** (avatar group `2160746aa659445e9cbfa4c02e5cf39c`). + +All 5 look IDs are filled in above. The `heygen-video` skill has been extended to recognize these 5 new looks via `--look ` — see `/mnt/skills/user/heygen-video/references/avatars.md` and the `LOOKS` dict in `/mnt/skills/user/heygen-video/scripts/create.py`. + +**To render a video in any of the 5 looks:** + +```bash +python3 /mnt/skills/user/heygen-video/scripts/create.py \ + --script "Your script text" \ + --look warm_desk_navy \ + --aspect 16:9 +``` + +Swap `warm_desk_navy` for any of: `podcast_studio`, `loft_window`, `corporate_office`, `modern_studio`. + +**All 5 are landscape-native** (16:9). Always pass `--aspect 16:9`; edit to portrait (9:16) in post. + +## If you add a 6th+ look later + +1. Generate it in Higgsfield Nano Banana Pro using the prompt formula in `references/prompt_formula.md` +2. Save the PNG to `C:\Users\Admin\Downloads\.png` +3. Upload to HeyGen web UI (the `upload.heygen.com` host is blocked from Claude's sandbox) +4. Name it exactly to match the filename +5. Tell Claude the look is uploaded — it will refetch via API and wire it in + +--- + +## Why these 5 (and not olive polo / pub_den) + +The original plan included `pub_den` (dark olive polo in warm wood-paneled den) but it was swapped mid-session for `corporate_office` (suit + tie + glasses) and `modern_studio` (white oxford + glasses). Reasons: + +- Olive polo was redundant with grey henley — both are casual daywear, no visual differentiation at playback size +- Glasses add a second-axis signifier (analyst mode vs. casual) that the outfit-only variation was missing +- Corporate + tie covers the seller-facing gravitas content that the prior 4 looks couldn't carry + +The final 5 cover: everyday (navy), bold-take (black), lifestyle (grey), executive (suit+glasses), educational (oxford+glasses). Each has a distinct content purpose and distinct visual signature. diff --git a/skills/vaibhav-template/references/prompt_formula.md b/skills/vaibhav-template/references/prompt_formula.md new file mode 100755 index 00000000..2452088d --- /dev/null +++ b/skills/vaibhav-template/references/prompt_formula.md @@ -0,0 +1,114 @@ +# Prompt Formula — Generating New Warm-Desk Looks + +This reference captures the exact prompt structure that produces identity-matched Graeham looks via Higgsfield's Nano Banana Pro model. Use this when adding a 6th+ look to the rotation, or regenerating any of the existing 5. + +## The four non-negotiables + +Every warm-desk look prompt must include these four blocks, in this order: + +### Block 1 — Identity lock (opens the prompt) + +``` +Photorealistic portrait matching the reference image face exactly. Same man +from the reference image — identical face shape, identical chin with smooth +rounded contour and absolutely no vertical cleft dimple or crease in the +chin, identical jaw contour, identical nose shape, identical eyes, identical +hairline and hair, identical complexion and skin tone. Match every facial +feature from the reference photo precisely including the smooth uncleft chin. +``` + +**Why this exact wording:** Nano Banana Pro has a documented bias toward adding chin clefts to white/Caucasian male subjects regardless of reference. Direct negation ("no vertical cleft dimple or crease") reduces but does not eliminate the drift. The redundant closing clause ("including the smooth uncleft chin") reinforces it. Expect ~30-40% improvement over prompts without this block; full elimination is not achievable with prompt engineering alone. + +### Block 2 — Camera + desk orientation + +``` +Camera positioned directly in front of the desk at eye level, square-on to +the subject. Desk runs horizontally across the bottom edge of the frame, +perpendicular to the camera, extending left-to-right, not diagonally. He is +centered in frame, shoulders square to camera, facing camera head-on. +``` + +**Why this exact wording:** Without explicit camera placement, Nano Banana defaults to 45° angled compositions where the desk extends diagonally away from camera. The "perpendicular" + "horizontally across the bottom" + "not diagonally" triplet is the minimum needed to force Vaibhav's head-on composition. Repetition is necessary — single-mention of "perpendicular" alone fails ~50% of the time. + +### Block 3 — Outfit (varies per look) + +Describe the full outfit in a single sentence. Be specific about material, fit, and layering. Add glasses as a separate sentence if applicable. + +**Known-good outfit descriptors:** +- `"navy blue quarter-zip pullover over a white crewneck tee"` → warm_desk_navy +- `"black crewneck sweater"` → podcast_studio +- `"heather grey long-sleeve henley shirt with the top two buttons undone at the collar"` → loft_window +- `"tailored charcoal grey wool suit jacket over a crisp light blue dress shirt with a navy blue silk necktie knotted neatly at the collar"` → corporate_office +- `"crisp white oxford button-down shirt with the sleeves rolled neatly up to just below the elbows, no tie, no jacket, top button undone"` → modern_studio + +**Glasses spec (when used):** `"modern clear round wire-frame glasses"`. Use the **same frame style across all looks that include glasses** — consistency makes them a recognizable signifier rather than visual noise. + +### Block 4 — Environment + lighting + props + +Describe background as heavily-blurred context, never competing with the face. Include: +- Wall/surface treatment (wood paneling, concrete, acoustic foam, seamless backdrop) +- A practical light source (lamp, window glow) +- 1-2 desk props max (laptop, notebook, coffee cup, fountain pen) +- Explicit instruction: `"His face is the brightest and sharpest element in frame."` + +## Words to AVOID (identity-drift risks) + +These words pull Nano Banana toward stock-photo "handsome-man" features (chin clefts, exaggerated jawlines, dramatic styling) and away from the reference: + +- `angular` — triggers chin cleft + sharp jaw exaggeration +- `confident` / `bold` / `strong` — triggers chin cleft + "stock photo" face +- `chiseled` / `sculpted` — obvious +- Photographer style references: `Douglas Friedman`, `Peter Lindbergh`, `Richard Avedon` — these bleed the photographer's signature face style + +**Safe style references:** +- `Kodak Portra 400` (film stock — no face bias) +- `cinematic warm color grading` (tonal, not structural) +- `editorial magazine quality` (generic enough to not overfit) + +## Generation settings (Higgsfield UI) + +- Model: **Nano Banana Pro** +- Aspect: **16:9** (landscape — all 5 looks are landscape-native) +- Quality: **4K** (5504×3072) +- Variants: **4/4** per batch +- Reference image: drag-and-drop the anchor selfie (`IMG_0520.JPG` or equivalent clean front-facing headshot). **The `file_upload` browser tool fails** — only drag-drop from File Explorer works. +- Cost: 16 credits per 4-variant batch + +## QC checklist before downloading a hero + +Verify each of these on the candidate before saving: + +- [ ] Camera is directly in front of the desk (not angled) +- [ ] Desk runs horizontally across the bottom, perpendicular to camera +- [ ] Shoulders are square to camera, subject is centered +- [ ] Outfit matches the spec exactly (color, garment type, layering, glasses if applicable) +- [ ] Lamp/practical light is positioned as prompted (foreground ≠ background) +- [ ] Face is the brightest and sharpest element in frame +- [ ] Chin looks as smooth as possible (know the AI bias — pick the variant with the least cleft) +- [ ] Hair, stubble, eyes, nose, complexion all match reference +- [ ] Background is heavily blurred and doesn't compete with face + +If a batch fails on more than two points, rewrite the prompt rather than rolling another 16-credit batch blindly. + +## The technical Higgsfield UI gotcha + +When typing a new prompt after clearing the old one, Chrome's `ctrl+a` on Higgsfield's contenteditable div does NOT select all — it inserts a literal `a` character. This caused a silent prefix bug (`aClose-up...`) on early generations this session. + +**Fix:** clear the prompt programmatically via JavaScript before typing: + +```javascript +const el = document.querySelector('[contenteditable="true"]'); +el.focus(); +const range = document.createRange(); +range.selectNodeContents(el); +const sel = window.getSelection(); +sel.removeAllRanges(); +sel.addRange(range); +document.execCommand('delete', false, null); +``` + +Then always screenshot-verify the prompt starts with the correct first word ("Photorealistic...") before clicking Generate. Do this every single time — the bug is subtle and wastes credits when missed. + +## Session evidence + +This formula was iteratively refined across ~112 Higgsfield credits (7 × 16-credit batches) during the April 23, 2026 build session. The final 5 approved looks — warm_desk_navy, podcast_studio, loft_window, corporate_office, modern_studio — all used this structure. The modern_studio generation (look #5) produced the cleanest chin result, likely because the clean editorial backdrop gave the model more "headroom" to render the reference face faithfully. diff --git a/skills/vaibhav-template/references/typography.md b/skills/vaibhav-template/references/typography.md new file mode 100755 index 00000000..a5688a4c --- /dev/null +++ b/skills/vaibhav-template/references/typography.md @@ -0,0 +1,118 @@ +# Typography & Caption Spec — Editor Handoff + +When Graeham's editor (or CapCut operator) burns captions into a Vaibhav-template video, they need to match this spec exactly. Copy-paste this file into the editor's handoff note. + +## Fonts to install + +Both are Google Fonts — free, no licensing issues. + +| Font family | Download | Used for | +|---|---|---| +| **Playfair Display** | fonts.google.com/specimen/Playfair+Display | All primary subject and emphasis text | +| **DM Sans** | fonts.google.com/specimen/DM+Sans | Secondary text, section titles, list numbers | +| **Inter** | fonts.google.com/specimen/Inter | Burned-in dialogue captions (smaller, dense text) | + +## Font weight + style reference + +| Text type | Family | Weight | Style | Size @ 1080×1920 | Color | +|---|---|---|---|---|---| +| Primary subject | Playfair Display | 400 (Regular) | *Italic* | 80–96pt | `#FFFFFF` white | +| Emphasis word | Playfair Display | 400 | *Italic* | 80–96pt | `#FFFFFF` on highlight box | +| Secondary clause | DM Sans | 400 (Regular) | Normal | 40–48pt | `#FFFFFF` white | +| List number | DM Sans | 700 (Bold) | Normal | 72–88pt | `#BFFF00` acid green | +| Section title | DM Sans | 500 (Medium) | Normal, **letter-spacing +8%** | 44–52pt | `#FFFFFF` on faded background | +| Dialogue caption | Inter | 600 (Semi-bold) | Normal | 36–44pt | `#FFFFFF` on translucent dark pill | + +**Pill spec for dialogue captions:** rounded rectangle, corner radius 12–16px, fill `#000000` at 60% opacity, 12px horizontal padding, 6px vertical padding, centered horizontally, bottom-third vertical anchor. + +## Color palette + +| Role | Hex | Notes | +|---|---|---| +| Primary text | `#FFFFFF` | All white; never tinted | +| Acid green highlight (numbers + emphasis) | `#BFFF00` | Saturated fluorescent green — this is the signature | +| Warm yellow highlight (occasional emphasis) | `#FFD700` | Use for 1 emphasis word per every 5–10 captions — sparse | +| Dark pill background | `#000000` @ 60% opacity | Never pure black at 100% — always translucent | +| Section wash red | `#FF3030` @ 30% opacity | Hard cut transition only | +| Section wash gold | `#D4A84B` @ 20% opacity | Gentle fade transition only | + +**Acid green is the anchor.** Every list number uses it. 1–2 emphasis keywords per 10-second block get it as a highlight-box fill. Don't dilute the signal by using it elsewhere. + +## Highlight box spec (for emphasis keywords) + +When a keyword gets a color highlight: +- Box extends 4px beyond the text on all sides +- Corner radius 2px (almost sharp — not pill-shaped) +- Box fill: acid green `#BFFF00` @ 100% opacity (solid, not translucent) +- Text on top: **still white** (`#FFFFFF`) — high contrast against green +- Rule: only highlight nouns and verbs that carry meaning. Never articles, connectors, prepositions, or filler. If the keyword doesn't survive the "would this still mean something standalone?" test, it doesn't get a highlight. + +## Caption placement zones + +Divide the 1080×1920 portrait frame (or 16:9 landscape if unusual) into: + +- **Top third (0–640px vertical):** primary subject captions (Mode 1 hook, Mode 5 section headers) +- **Middle third (640–1280px):** full-bleed talking-head emphasis (Mode 2) +- **Bottom third (1280–1920px):** dialogue captions, supporting clauses, Mode 4 card descriptors + +**Never center a caption in the middle third if Graeham's face is there.** Captions avoid the face zone. + +## Animation timing + +- **Caption in:** fade + slight 10% scale-up, 150ms duration +- **Caption out:** fade, 100ms duration, overlaps with next caption's fade-in +- **Crossfades:** do NOT hold captions static more than 2 seconds — refresh them in sync with Graeham's speech beats +- **Emphasis highlight box:** appears 50ms AFTER the text word does (tiny delay reads as intentional) + +## Caption writing rules + +- Keep captions to 3–7 words per on-screen moment +- Break long sentences into 2–3 sequential captions, not one long overlay +- Match the spoken word EXACTLY — never paraphrase +- Capitalize proper nouns; sentence-case everything else +- NO emoji inside captions (emoji is a separate layer, see below) + +## Emoji pop-ins (separate layer) + +Vaibhav uses emoji as small animated accents, not inside captions. They appear: +- Off to one side of the primary caption +- Scale in with a bounce (keyframed: 0% → 120% → 100%, ~250ms) +- Stay on screen 800ms–1.2s +- Scale out with a smaller bounce back to 0% + +**Common picks by content type:** +- Money / closing: 💰 🏠 ✅ +- Market / data: 📈 📊 📉 +- Neighborhood: 📍 🌉 (Bay) +- Analysis / explanation: 🤔 💡 👉 +- Hot take: 🔥 💀 (use sparingly) + +One emoji per on-screen caption moment. More than that = noise. + +## Example caption sheet (deliverable format) + +This is what the `vaibhav-template` skill outputs for the editor. Drop it into the shot plan table and the editor burns captions accordingly. + +``` +Time (s) Caption text Font + style Highlight Placement Emoji +0.0–1.2 Sam Altman Playfair Display Italic 88pt white — top third — +1.2–2.5 just killed the entire DM Sans Regular 44pt white — top third — +2.5–3.8 Image Gen Playfair Display Italic 88pt white on "Image Gen" center frame 🔥 (left side, 2.8–3.5s) + BFFF00 box +3.8–5.0 Industry with one big launch DM Sans Regular 44pt white — center frame — +5.0–6.5 01. PRECISE TEXT DM Sans Bold 80pt acid green (#BFFF00) — top third — + + Medium letter-spaced 8% +6.5–8.0 [dialogue caption] Inter Semi-bold 40pt white on dark pill — bottom third — +``` + +This format maps 1:1 to CapCut's text layer timeline — the editor just copies the row values into the layer properties. + +## What this spec does NOT cover + +- Music selection / audio design — separate pass +- Transitions between Modes 1–5 — Graeham has editor freedom on cut style (hard cut, whip pan, zoom blur) as long as rhythm pacing is respected +- Color grading LUT — the "warm face + cool background" is lit at capture time (the Higgsfield generation handles it) — post-grade only adjusts contrast and saturation + +## Source of truth + +If this file and SKILL.md disagree on any spec, **SKILL.md wins** — it's the higher-level document. This file is the editor-handoff translation of the SKILL.md rules, not a separate authority. diff --git a/skills/vaibhav-template/scripts/build_shot_plan.py b/skills/vaibhav-template/scripts/build_shot_plan.py new file mode 100755 index 00000000..c8b971e9 --- /dev/null +++ b/skills/vaibhav-template/scripts/build_shot_plan.py @@ -0,0 +1,260 @@ +#!/usr/bin/env python3 +""" +build_shot_plan.py — Turn a script + content_type into a Vaibhav-style shot plan. + +Given a script, a content type, and a target runtime, produces: + - Look recommendation with reasoning + - Timed shot plan mapping each beat to a composition mode + - Caption sheet for the editor + - List of B-roll prompts needed from Higgsfield + - Ready-to-run HeyGen CLI invocation + +Usage: + python3 build_shot_plan.py --script "Your script here" --content-type market_data --runtime 60 + python3 build_shot_plan.py --from-file script.txt --content-type hot_take --runtime 90 + +This is a SCAFFOLDING tool — it gives you the structure. Graeham (or Claude) +still needs to refine the actual caption text, exact cut timings, and B-roll +descriptions. Think of the output as a 70%-done shot plan that you polish. +""" +from __future__ import annotations + +import argparse +import dataclasses +import math +import pathlib +import sys +import textwrap + + +# ========================================================================= +# LOOK DECISION TREE +# ========================================================================= + +LOOK_BY_CONTENT_TYPE = { + "market_data": ("warm_desk_navy", "Everyday default. Data-driven content doesn't need gravitas — warm desk navy is recognizable and versatile."), + "everyday": ("warm_desk_navy", "Default look. Casual-professional tone fits most general content."), + "hot_take": ("podcast_studio", "Moody podcast grade signals 'this is a bold opinion.' The darker environment earns the gravity of a strong take."), + "opinion": ("podcast_studio", "Podcast grade communicates this is commentary, not neutral reporting."), + "lifestyle": ("loft_window", "Dusk loft window cue reads as 'end of workday reflection' — fits lifestyle and neighborhood storytelling."), + "neighborhood": ("loft_window", "The blurred city bokeh grounds the video in 'place' without naming it explicitly."), + "seller_facing": ("corporate_office","Suit + tie + executive office communicates 'I'm handling something important.' Earn this look — don't use it for casual content."), + "listing_intro": ("corporate_office","Listing introductions are high-stakes content that benefits from executive polish."), + "educational": ("modern_studio", "Clean backdrop + glasses signal 'analyst mode / about to explain something.' The visual cue primes viewers to learn."), + "contract_explain":("modern_studio", "White oxford + glasses = 'contract walkthrough' visual cue. Viewers learn the signifier across videos."), + "propos_product": ("modern_studio", "PropertyIQ educational content fits the analyst-mode visual signature."), +} + + +# ========================================================================= +# CUT RHYTHM ARC (from reel analysis: 80s video, 52 total cuts) +# ========================================================================= +# +# Section boundaries as fractions of runtime + cut share of total: +# Hook: first 12% of runtime, 40% of all cuts (~0.5s shots) +# Setup: 12-38%, 15% of all cuts (~2.5s shots) +# Body: 38-75%, 27% of all cuts (~2.0s shots) +# Climax: 75-88%, 13% of all cuts (~1.4s shots) +# CTA: last 12%, 6% of all cuts (~3.0s shots) + +SECTIONS = [ + {"name": "Hook", "runtime_start_pct": 0.00, "runtime_end_pct": 0.12, "cut_share_pct": 0.40, "shot_len_s": 0.5}, + {"name": "Setup", "runtime_start_pct": 0.12, "runtime_end_pct": 0.38, "cut_share_pct": 0.15, "shot_len_s": 2.5}, + {"name": "Body", "runtime_start_pct": 0.38, "runtime_end_pct": 0.75, "cut_share_pct": 0.27, "shot_len_s": 2.0}, + {"name": "Climax", "runtime_start_pct": 0.75, "runtime_end_pct": 0.88, "cut_share_pct": 0.13, "shot_len_s": 1.4}, + {"name": "CTA", "runtime_start_pct": 0.88, "runtime_end_pct": 1.00, "cut_share_pct": 0.06, "shot_len_s": 3.0}, +] + +# Typical mode distribution per section (composition modes from SKILL.md) +MODES_BY_SECTION = { + "Hook": ["Mode 1 (hook composite)", "Mode 1", "Mode 2 (talking head)", "Mode 1"], + "Setup": ["Mode 2", "Mode 3 (full-bleed B-roll)", "Mode 2"], + "Body": ["Mode 2", "Mode 4 (screenshot PiP)", "Mode 3", "Mode 2", "Mode 4"], + "Climax": ["Mode 2", "Mode 3", "Mode 2"], + "CTA": ["Mode 2"], +} + + +# ========================================================================= +# SHOT PLAN BUILDER +# ========================================================================= + +@dataclasses.dataclass +class Shot: + index: int + start_s: float + end_s: float + section: str + mode: str + needs_broll: bool + +def build_shot_plan(runtime_s: int) -> list[Shot]: + shots: list[Shot] = [] + index = 1 + total_target_cuts = max(6, round(runtime_s * 52 / 80)) # scale 52 cuts from 80s reference + + for section in SECTIONS: + start_s = section["runtime_start_pct"] * runtime_s + end_s = section["runtime_end_pct"] * runtime_s + n_cuts_in_section = max(1, round(total_target_cuts * section["cut_share_pct"])) + section_duration = end_s - start_s + shot_len = section_duration / n_cuts_in_section + + modes_for_section = MODES_BY_SECTION[section["name"]] + + for i in range(n_cuts_in_section): + shot_start = start_s + i * shot_len + shot_end = shot_start + shot_len + mode = modes_for_section[i % len(modes_for_section)] + needs_broll = "Mode 1" in mode or "Mode 3" in mode or "Mode 4" in mode + shots.append(Shot( + index=index, + start_s=round(shot_start, 2), + end_s=round(shot_end, 2), + section=section["name"], + mode=mode, + needs_broll=needs_broll, + )) + index += 1 + + return shots + + +# ========================================================================= +# OUTPUT FORMATTING +# ========================================================================= + +def render_shot_plan_table(shots: list[Shot]) -> str: + lines = [] + lines.append(f"{'#':>3} {'Time':<12} {'Section':<8} {'Mode':<30} {'B-roll?':<8}") + lines.append("-" * 75) + for s in shots: + time_range = f"{s.start_s:>5.1f}–{s.end_s:<5.1f}s" + broll_mark = "YES" if s.needs_broll else "—" + lines.append(f"{s.index:>3} {time_range:<12} {s.section:<8} {s.mode:<30} {broll_mark:<8}") + return "\n".join(lines) + + +def render_broll_todo(shots: list[Shot]) -> str: + broll_shots = [s for s in shots if s.needs_broll] + if not broll_shots: + return "No B-roll required — all shots are talking head (Mode 2).\n" + lines = [f"You need {len(broll_shots)} B-roll clips generated via higgsfield-video skill:\n"] + for s in broll_shots: + lines.append(f" [{s.start_s:>5.1f}s] {s.mode} — describe the visual content here, then run through higgsfield-video") + lines.append("\nEvery prompt must include Peninsula-specific anchors per higgsfield-video skill rules:") + lines.append(" - flat terrain (no hills)") + lines.append(" - San Francisco Bay visible in background") + lines.append(" - stucco ranch homes / Silicon Valley suburban character") + lines.append(" - name the specific neighborhood (Newbridge/Kavanaugh, The Gardens, West Side of 101)") + return "\n".join(lines) + + +def render_heygen_invocation(look: str, script: str, runtime_s: int) -> str: + return textwrap.dedent(f"""\ + # Once the shot plan above is final and the look is confirmed, render the talking-head base via heygen-video: + + python3 /mnt/skills/user/heygen-video/scripts/create.py \\ + --script {script!r} \\ + --look {look} \\ + --aspect 16:9 \\ + --title "Vaibhav template - {look} - $(date +%Y-%m-%d)" + + # The output will be a 16:9 MP4 of Graeham speaking the script at the chosen look. + # Estimated render time: 2-10 minutes for a ~{runtime_s}s script. + # Graeham's editor then: + # 1. Takes the HeyGen MP4 as the talking-head base layer + # 2. Cuts to B-roll at the times marked above + # 3. Burns captions per references/typography.md + # 4. Exports at 9:16 portrait for IG/TikTok (or keeps 16:9 for YouTube) + """).strip() + + +# ========================================================================= +# MAIN +# ========================================================================= + +def parse_args() -> argparse.Namespace: + p = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter) + script_group = p.add_mutually_exclusive_group(required=True) + script_group.add_argument("--script", help="Script text directly on the command line.") + script_group.add_argument("--from-file", help="Path to a text file containing the script.") + p.add_argument("--content-type", required=True, choices=list(LOOK_BY_CONTENT_TYPE.keys()), + help="Content intent — determines which of the 5 looks to use.") + p.add_argument("--runtime", type=int, default=60, + help="Target runtime in seconds (default: 60).") + return p.parse_args() + + +def main() -> int: + args = parse_args() + + if args.from_file: + script = pathlib.Path(args.from_file).read_text().strip() + else: + script = args.script.strip() + + look, look_reason = LOOK_BY_CONTENT_TYPE[args.content_type] + shots = build_shot_plan(args.runtime) + + print("=" * 75) + print("VAIBHAV TEMPLATE — SHOT PLAN") + print("=" * 75) + print() + print(f"Content type: {args.content_type}") + print(f"Target runtime: {args.runtime}s") + print(f"Total shots: {len(shots)}") + print() + + print("-" * 75) + print("LOOK RECOMMENDATION") + print("-" * 75) + print(f"Use: {look}") + print(f"Why: {look_reason}") + print() + + print("-" * 75) + print("SCRIPT") + print("-" * 75) + wrapped = textwrap.fill(script, width=73, initial_indent=" ", subsequent_indent=" ") + print(wrapped) + print() + + print("-" * 75) + print("SHOT PLAN (map each beat of the script to a shot slot)") + print("-" * 75) + print(render_shot_plan_table(shots)) + print() + print("NOTE: the script has been broken into timed slots above, but you") + print("still need to assign actual caption text and B-roll content to each.") + print("This scaffold gives you the rhythm; Claude fills in the substance.") + print() + + print("-" * 75) + print("B-ROLL TODO LIST") + print("-" * 75) + print(render_broll_todo(shots)) + print() + + print("-" * 75) + print("HEYGEN RENDER COMMAND") + print("-" * 75) + print(render_heygen_invocation(look, script, args.runtime)) + print() + + print("-" * 75) + print("REMINDERS") + print("-" * 75) + print(" - Typography spec: references/typography.md") + print(" - Anti-cleft prompt formula for new B-roll of Graeham: references/prompt_formula.md") + print(" - Look details + HeyGen IDs: references/looks.md") + print(" - 40% of cuts go in the first 10% of runtime — this is what makes it feel fast") + print(" - Warm face / cool background — this is the single most important grade rule") + print(" - One look per video — never mix mid-video") + print() + + return 0 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/skills/video-creator/SKILL.md b/skills/video-creator/SKILL.md new file mode 100755 index 00000000..9c44df4b --- /dev/null +++ b/skills/video-creator/SKILL.md @@ -0,0 +1,242 @@ +--- +name: video-creator +description: "AI Video Creator — generates professional MP4 videos using Python + ffmpeg. Use this skill ANY time the user mentions: video, reel, short, TikTok, YouTube Short, Instagram Reel, listing video, property video, market update video, social media video, video content, create a video, make a video, video for social, animated video, slideshow video, video from photos, promo video, teaser video, explainer video, video presentation, or anything related to creating, rendering, or producing video content. Also trigger when the user wants to turn photos into a video, create motion graphics, make an animated market report, or produce any kind of video content from text or images. This skill renders finished MP4 files directly — no external tools or local setup needed." +--- + +# Video Creator Skill + +Create professional MP4 videos entirely within the Cowork environment. This skill uses Python (Pillow + OpenCV) for frame generation and ffmpeg for encoding. No browser, no Chromium, no local setup — just describe what you want and get a finished video. + +## Architecture + +``` +video-creator/ +├── SKILL.md ← You are here +└── scripts/ + ├── video_engine.py ← Core rendering engine (frames + ffmpeg) + ├── listing_video.py ← Real estate listing video template + ├── social_video.py ← Social media short-form template + └── market_video.py ← Market update / educational template +``` + +## How It Works + +1. **Understand the request** — what kind of video, what content, what format +2. **Build a config dict** — structured data describing every slide +3. **Call the appropriate template** — or build custom slides using the engine +4. **Render** — Python generates frames, ffmpeg encodes to H.264 MP4 +5. **Deliver** — save to outputs folder and provide download link + +## Quick Start + +Read the appropriate template script before generating video code. Each template accepts a JSON config and an output path. + +```python +import sys +sys.path.insert(0, '/scripts') + +# For listing videos: +from listing_video import create_listing_video +create_listing_video(config, '/sessions/.../mnt/outputs/my_video.mp4') + +# For social media: +from social_video import create_social_video +create_social_video(config, '/sessions/.../mnt/outputs/my_reel.mp4') + +# For market updates: +from market_video import create_market_video +create_market_video(config, '/sessions/.../mnt/outputs/market_update.mp4') + +# For fully custom videos: +from video_engine import * +project = VideoProject(slides=[...], output_path='...') +render_video(project) +``` + +## Video Types & When to Use Each + +### 1. Listing Video (`listing_video.py`) +Best for: Property showcases, open house promos, just-listed/just-sold announcements. + +**Config reference:** +```json +{ + "address": "123 Main Street", + "city_state_zip": "San Jose, CA 95125", + "price": "$1,850,000", + "beds": 4, + "baths": 3, + "sqft": "2,450", + "lot_size": "6,200 sqft", + "year_built": 1965, + "description": "Stunning mid-century modern home...", + "highlights": ["Renovated Kitchen", "Pool & Spa", "Top Schools"], + "photos": ["/path/to/photo1.jpg", "/path/to/photo2.jpg"], + "photo_captions": ["Living Room", "Kitchen"], + "agent_name": "Graeham Watts", + "agent_title": "REALTOR® | DRE# 01466876", + "agent_contact": "graehamwatts@gmail.com", + "agent_phone": "650-308-4727", + "brokerage": "Compass", + "theme": "luxury", + "aspect_ratio": "landscape", + "duration_per_photo": 4.0, + "include_highlights": true, + "cta_text": "Schedule Your Private Tour" +} +``` + +**Slide flow:** Title → Property Stats → Photo slides (with Ken Burns) → Highlights → Description → CTA + +### 2. Social Media Video (`social_video.py`) +Best for: Instagram Reels, YouTube Shorts, TikTok, quick tips, market stats. + +**Types available:** +- `"tips"` — Hook headline → numbered tip slides → CTA +- `"stats"` — Headline → stat cards with big numbers → CTA +- `"teaser"` — Quick photo montage with address overlay → CTA +- `"quote"` — Client testimonial or inspirational quote → CTA + +**Config reference:** +```json +{ + "type": "tips", + "headline": "3 Mistakes First-Time Buyers Make", + "items": ["Not getting pre-approved first", "Skipping the inspection", "Waiving contingencies"], + "background_image": "/path/to/bg.jpg", + "agent_name": "Graeham Watts", + "agent_handle": "@graehamwatts", + "theme": "luxury", + "aspect_ratio": "portrait" +} +``` + +For stats type: +```json +{ + "type": "stats", + "headline": "Bay Area Market Update", + "stats": [ + {"label": "Median Price", "value": "$1.85M", "change": "+4.2%"}, + {"label": "Days on Market", "value": "12", "change": "-3 days"}, + {"label": "Inventory", "value": "1.8 months", "change": "-15%"} + ], + "agent_name": "Graeham Watts", + "agent_handle": "@graehamwatts", + "theme": "luxury", + "aspect_ratio": "portrait" +} +``` + +### 3. Market Update Video (`market_video.py`) +Best for: Monthly market reports, educational content, data presentations. + +**Config reference:** +```json +{ + "title": "Silicon Valley Market Update", + "subtitle": "March 2026", + "sections": [ + { + "headline": "Median Home Price", + "stat_value": "$1.85M", + "stat_label": "Median Price", + "stat_change": "+4.2%", + "content": "Prices continue to climb as inventory remains tight." + }, + { + "headline": "Market Velocity", + "stat_value": "12 Days", + "stat_label": "Average Days on Market", + "stat_change": "-3 days", + "content": "Homes are selling faster than last quarter." + } + ], + "takeaways": [ + "Sellers still have strong leverage in most price ranges", + "Well-priced homes are getting multiple offers within a week", + "Interest rates are stabilizing, bringing more buyers back" + ], + "agent_name": "Graeham Watts", + "agent_title": "REALTOR® | DRE# 01466876", + "agent_contact": "graehamwatts@gmail.com", + "theme": "luxury", + "aspect_ratio": "landscape" +} +``` + +### 4. Custom Video (use `video_engine.py` directly) +For anything that doesn't fit the templates — fully custom slide sequences. + +**Available components:** +- `Slide` — base slide with background color/image, overlays, text, transitions +- `TextOverlay` — text with font, size, color, animation, background pill, shadow +- `LowerThird` — professional bar with headline + subtitle +- `VideoProject` — container for slides + encoding settings + +**Transitions:** `FADE`, `DISSOLVE`, `SLIDE_LEFT`, `SLIDE_RIGHT`, `ZOOM_IN`, `WIPE_LEFT`, `KENBURNS`, `CUT` + +**Text Animations:** `FADE_IN`, `SLIDE_UP`, `TYPEWRITER`, `SCALE_IN`, `NONE` + +**Ken Burns directions:** `zoom_in`, `zoom_out`, `pan_left`, `pan_right` + +## Themes + +All templates support these color themes: + +| Theme | Best For | Primary Color | +|-------|----------|---------------| +| `luxury` | High-end listings, premium branding | Navy + Gold | +| `modern` | Clean contemporary look | Dark + Blue accent | +| `coastal` | Beach/waterfront properties | Ocean blue + Teal | +| `warm` | Cozy homes, family neighborhoods | Warm brown + Gold | +| `minimal` | Ultra-clean, minimal design | White + Black | +| `bold` | Attention-grabbing social content | Black + Red | +| `clean` | Light professional look | Light gray + Dark | + +Note: Not all themes are available in all templates. `luxury` and `modern` are universally supported. + +## Aspect Ratios + +| Setting | Resolution | Use Case | +|---------|-----------|----------| +| `landscape` | 1920×1080 | YouTube, website, presentations | +| `portrait` | 1080×1920 | Instagram Reels, TikTok, YouTube Shorts | +| `square` | 1080×1080 | Instagram feed, Facebook | + +## Working with Photos + +When the user provides photos (uploaded or from a folder): +1. Photos are at paths under `/sessions/.../mnt/uploads/` or the user's selected folder +2. Pass absolute paths in the config's `photos` array +3. The engine handles resizing, cropping (cover fit), and Ken Burns effects automatically +4. Supported formats: JPG, PNG, WebP, TIFF + +If no photos are provided, the skill creates text-only slides with colored backgrounds — still professional and useful. + +## Performance Notes + +- A 30-second video at 30fps = ~900 frames. Expect 2-4 minutes render time. +- For faster test renders, use `fps=24` or even `fps=15`. +- Shorter videos (10-15 seconds) render in under a minute. +- Social media portrait videos are smaller (1080px wide) and render faster. + +## Agent Info Defaults + +When the user doesn't specify agent info, read Graeham's brand details from `shared-references/identity.json` rather than hardcoding them (never hardcode DRE, brokerage, or contact info — that file is the single source of truth and is audited by `scripts/verify_brand_identity.py`). As of this writing that resolves to: +- Name: Graeham Watts +- Title: REALTOR® | DRE# 01466876 +- Email: graehamwatts@gmail.com +- Brokerage: Compass + +## Step-by-Step Workflow + +1. **Ask what kind of video** if not clear from the request +2. **Read the relevant template script** to understand the config shape +3. **Build the config** from the user's input (fill in defaults for missing fields) +4. **Write a Python script** that imports the template and calls it with the config +5. **Run the script** via Bash with a timeout of 300000ms (5 min) +6. **Save output** to `/sessions/.../mnt/outputs/` and provide a computer:// link +7. **If the user wants changes**, modify the config and re-render + +Always tell the user roughly how long rendering will take based on duration and fps. diff --git a/skills/video-creator/generated/.gitkeep b/skills/video-creator/generated/.gitkeep new file mode 100755 index 00000000..e69de29b diff --git a/skills/video-creator/scripts/listing_video.py b/skills/video-creator/scripts/listing_video.py new file mode 100755 index 00000000..aa9bf739 --- /dev/null +++ b/skills/video-creator/scripts/listing_video.py @@ -0,0 +1,449 @@ +#!/usr/bin/env python3 +""" +Listing Video Template — Property showcase with photos, text overlays, transitions. +Designed for real estate agents to create professional listing videos. + +Usage: + python3 listing_video.py --config listing_config.json --output listing.mp4 + +Or import and use programmatically: + from listing_video import create_listing_video + create_listing_video(config, output_path) +""" + +import argparse +import json +import os +import random +import sys +from typing import Dict, List, Optional + +# Add parent dir to path +sys.path.insert(0, os.path.dirname(__file__)) +from video_engine import ( + COLORS, FONT_SANS, FONT_SANS_BOLD, FONT_SERIF, FONT_SERIF_BOLD, + LowerThird, Slide, TextAnimation, TextOverlay, + Transition, VideoProject, create_cta_slide, create_photo_slide, + create_text_slide, create_title_slide, render_video, +) + + +# ─── Color Themes ──────────────────────────────────────────────────────────── + +THEMES = { + "luxury": { + "bg_primary": (15, 25, 50), + "bg_secondary": (245, 240, 230), + "accent": (198, 168, 124), + "text_light": (255, 255, 255), + "text_dark": (30, 30, 30), + "overlay": (0, 0, 0, 150), + }, + "modern": { + "bg_primary": (25, 25, 25), + "bg_secondary": (250, 250, 250), + "accent": (60, 140, 200), + "text_light": (255, 255, 255), + "text_dark": (30, 30, 30), + "overlay": (0, 0, 0, 130), + }, + "coastal": { + "bg_primary": (20, 60, 90), + "bg_secondary": (240, 248, 255), + "accent": (100, 200, 200), + "text_light": (255, 255, 255), + "text_dark": (20, 40, 60), + "overlay": (10, 30, 50, 140), + }, + "warm": { + "bg_primary": (60, 30, 20), + "bg_secondary": (255, 250, 240), + "accent": (210, 160, 90), + "text_light": (255, 255, 255), + "text_dark": (40, 30, 20), + "overlay": (30, 15, 10, 140), + }, + "minimal": { + "bg_primary": (255, 255, 255), + "bg_secondary": (245, 245, 245), + "accent": (40, 40, 40), + "text_light": (255, 255, 255), + "text_dark": (30, 30, 30), + "overlay": (0, 0, 0, 120), + }, +} + + +# ─── Listing Video Builder ─────────────────────────────────────────────────── + +def create_listing_video(config: Dict, output_path: str, + width: int = 1920, height: int = 1080, + fps: int = 30) -> str: + """ + Create a complete listing video from a config dict. + + Config shape: + { + "address": "123 Main Street", + "city_state_zip": "San Jose, CA 95125", + "price": "$1,850,000", + "beds": 4, + "baths": 3, + "sqft": "2,450", + "lot_size": "6,200 sqft", # optional + "year_built": 1965, # optional + "description": "Stunning mid-century modern...", # optional + "highlights": ["Renovated Kitchen", "Pool & Spa", ...], + "photos": ["/path/to/photo1.jpg", ...], + "photo_captions": ["Living Room", "Kitchen", ...], # optional + "agent_name": "Graeham Watts", + "agent_title": "REALTOR® | DRE# 01466876", + "agent_contact": "graehamwatts@gmail.com", + "agent_phone": "408-XXX-XXXX", # optional + "brokerage": "Compass", # optional + "theme": "luxury", # luxury, modern, coastal, warm, minimal + "aspect_ratio": "landscape", # landscape, portrait, square + "duration_per_photo": 4.0, # seconds per photo slide + "include_highlights": true, + "cta_text": "Schedule Your Private Tour", + } + """ + + theme_name = config.get("theme", "luxury") + theme = THEMES.get(theme_name, THEMES["luxury"]) + + # Aspect ratio + ar = config.get("aspect_ratio", "landscape") + if ar == "portrait": + width, height = 1080, 1920 + elif ar == "square": + width, height = 1080, 1080 + + photo_duration = config.get("duration_per_photo", 4.0) + photos = config.get("photos", []) + captions = config.get("photo_captions", []) + highlights = config.get("highlights", []) + + slides = [] + + # ── 1. Title Slide ──────────────────────────────────────────────────── + # Use first photo as background if available + title_bg = photos[0] if photos else None + price_line = config.get("price", "") + address = config.get("address", "Beautiful Home") + city = config.get("city_state_zip", "") + + subtitle_parts = [] + if price_line: + subtitle_parts.append(price_line) + if city: + subtitle_parts.append(city) + subtitle = " | ".join(subtitle_parts) + + title_slide = Slide( + duration=4.5, + background_color=theme["bg_primary"], + image_path=title_bg, + overlay_color=theme["overlay"], + transition_in=Transition.FADE, + transition_duration=1.0, + texts=[ + TextOverlay( + text=address.upper(), + position=(_center_x(width, 700), height // 2 - 90), + font_path=FONT_SANS_BOLD, + font_size=68 if len(address) < 25 else 52, + color=theme["text_light"], + max_width=700 if ar == "landscape" else 500, + align="center", + animation=TextAnimation.FADE_IN, + shadow=True, + ), + TextOverlay( + text=subtitle, + position=(_center_x(width, 700), height // 2 + 10), + font_path=FONT_SANS, + font_size=30, + color=theme["accent"], + max_width=700, + align="center", + animation=TextAnimation.SLIDE_UP, + shadow=True, + ), + ], + ) + + # Add stats bar + stats_parts = [] + if config.get("beds"): + stats_parts.append(f"{config['beds']} Beds") + if config.get("baths"): + stats_parts.append(f"{config['baths']} Baths") + if config.get("sqft"): + stats_parts.append(f"{config['sqft']} Sqft") + if stats_parts: + title_slide.texts.append(TextOverlay( + text=" • ".join(stats_parts), + position=(_center_x(width, 700), height // 2 + 60), + font_path=FONT_SANS, + font_size=28, + color=theme["text_light"], + max_width=700, + align="center", + animation=TextAnimation.SLIDE_UP, + shadow=True, + )) + + slides.append(title_slide) + + # ── 2. Property Stats Slide ────────────────────────────────────────── + stat_items = [] + if config.get("beds"): + stat_items.append(f"Bedrooms: {config['beds']}") + if config.get("baths"): + stat_items.append(f"Bathrooms: {config['baths']}") + if config.get("sqft"): + stat_items.append(f"Living Area: {config['sqft']} sqft") + if config.get("lot_size"): + stat_items.append(f"Lot Size: {config['lot_size']}") + if config.get("year_built"): + stat_items.append(f"Year Built: {config['year_built']}") + + if stat_items: + # Use second photo as background if available + stat_bg = photos[1] if len(photos) > 1 else None + stat_slide = Slide( + duration=4.0, + background_color=theme["bg_primary"], + image_path=stat_bg, + overlay_color=(0, 0, 0, 180) if stat_bg else None, + blur_background=True if stat_bg else False, + transition_in=Transition.FADE, + transition_duration=0.6, + texts=[ + TextOverlay( + text="PROPERTY DETAILS", + position=(120, 100) if ar == "landscape" else (80, 200), + font_path=FONT_SANS_BOLD, + font_size=42, + color=theme["accent"], + animation=TextAnimation.FADE_IN, + shadow=True, + ), + TextOverlay( + text="\n".join(stat_items), + position=(120, 180) if ar == "landscape" else (80, 280), + font_path=FONT_SANS, + font_size=34, + color=theme["text_light"], + max_width=width - 240, + animation=TextAnimation.SLIDE_UP, + shadow=True, + line_spacing=18, + ), + ], + ) + slides.append(stat_slide) + + # ── 3. Photo Slides ────────────────────────────────────────────────── + # Alternate Ken Burns directions for visual interest + kb_directions = ["zoom_in", "zoom_out", "pan_left", "pan_right"] + transitions = [Transition.KENBURNS, Transition.FADE, Transition.DISSOLVE, Transition.SLIDE_LEFT] + + for i, photo in enumerate(photos): + caption = captions[i] if i < len(captions) else "" + kb_dir = kb_directions[i % len(kb_directions)] + trans = transitions[i % len(transitions)] + + slide = Slide( + duration=photo_duration, + image_path=photo, + transition_in=trans if trans != Transition.KENBURNS else Transition.KENBURNS, + transition_duration=0.6, + kenburns_direction=kb_dir, + ) + + # Add caption if provided + if caption: + slide.lower_third = LowerThird( + headline=caption, + bar_color=theme["bg_primary"], + accent_color=theme["accent"], + text_color=theme["text_light"], + ) + + slides.append(slide) + + # ── 4. Highlights Slide ────────────────────────────────────────────── + if highlights and config.get("include_highlights", True): + highlights_slide = Slide( + duration=5.0, + background_color=theme["bg_secondary"], + transition_in=Transition.FADE, + transition_duration=0.5, + texts=[ + TextOverlay( + text="PROPERTY HIGHLIGHTS", + position=(120, 80) if ar == "landscape" else (80, 200), + font_path=FONT_SANS_BOLD, + font_size=44, + color=theme["text_dark"], + animation=TextAnimation.FADE_IN, + shadow=False, + ), + TextOverlay( + text="\n".join(f"✦ {h}" for h in highlights[:8]), + position=(140, 170) if ar == "landscape" else (80, 300), + font_path=FONT_SANS, + font_size=32, + color=theme["text_dark"], + max_width=width - 280, + animation=TextAnimation.SLIDE_UP, + shadow=False, + line_spacing=16, + ), + ], + ) + slides.append(highlights_slide) + + # ── 5. Description Slide (optional) ────────────────────────────────── + if config.get("description"): + desc_bg = photos[-1] if photos else None + desc_slide = Slide( + duration=5.0, + background_color=theme["bg_primary"], + image_path=desc_bg, + overlay_color=(0, 0, 0, 190) if desc_bg else None, + blur_background=True, + transition_in=Transition.FADE, + transition_duration=0.6, + texts=[ + TextOverlay( + text=config["description"][:300], + position=(120, height // 2 - 100) if ar == "landscape" else (80, height // 2 - 200), + font_path=FONT_SERIF, + font_size=30, + color=theme["text_light"], + max_width=width - 240, + align="center", + animation=TextAnimation.FADE_IN, + shadow=True, + line_spacing=14, + ), + ], + ) + slides.append(desc_slide) + + # ── 6. CTA / Contact Slide ─────────────────────────────────────────── + cta_text = config.get("cta_text", "Schedule Your Private Tour") + agent_name = config.get("agent_name", "") + agent_title = config.get("agent_title", "") + agent_contact = config.get("agent_contact", "") + agent_phone = config.get("agent_phone", "") + brokerage = config.get("brokerage", "") + + contact_parts = [] + if agent_phone: + contact_parts.append(agent_phone) + if agent_contact: + contact_parts.append(agent_contact) + contact_line = " | ".join(contact_parts) + + agent_line = agent_name + if agent_title: + agent_line += f" • {agent_title}" + + cta_slide = Slide( + duration=5.0, + background_color=theme["bg_primary"], + transition_in=Transition.FADE, + transition_duration=1.0, + texts=[ + TextOverlay( + text=cta_text, + position=(_center_x(width, 800), height // 2 - 120), + font_path=FONT_SANS_BOLD, + font_size=56, + color=theme["text_light"], + max_width=800, + align="center", + animation=TextAnimation.SCALE_IN, + shadow=True, + ), + TextOverlay( + text=agent_line, + position=(_center_x(width, 800), height // 2 + 0), + font_path=FONT_SANS_BOLD, + font_size=30, + color=theme["accent"], + max_width=800, + align="center", + animation=TextAnimation.FADE_IN, + shadow=True, + ), + TextOverlay( + text=contact_line, + position=(_center_x(width, 800), height // 2 + 50), + font_path=FONT_SANS, + font_size=26, + color=(200, 200, 200), + max_width=800, + align="center", + animation=TextAnimation.FADE_IN, + ), + ], + ) + + if brokerage: + cta_slide.texts.append(TextOverlay( + text=brokerage, + position=(_center_x(width, 800), height // 2 + 100), + font_path=FONT_SANS, + font_size=22, + color=(160, 160, 160), + max_width=800, + align="center", + animation=TextAnimation.FADE_IN, + )) + + slides.append(cta_slide) + + # ── Build & Render ─────────────────────────────────────────────────── + project = VideoProject( + slides=slides, + width=width, + height=height, + fps=fps, + output_path=output_path, + background_music=config.get("background_music"), + music_volume=config.get("music_volume", 0.3), + ) + + def progress(current, total): + pct = int(current / total * 100) + if pct % 10 == 0: + print(f" Rendering: {pct}%", flush=True) + + return render_video(project, progress_callback=progress) + + +def _center_x(width: int, content_width: int) -> int: + """Calculate x position to center content.""" + return (width - content_width) // 2 + + +# ─── CLI ───────────────────────────────────────────────────────────────────── + +if __name__ == "__main__": + parser = argparse.ArgumentParser(description="Create a listing video") + parser.add_argument("--config", required=True, help="Path to JSON config file") + parser.add_argument("--output", default="listing_video.mp4", help="Output path") + parser.add_argument("--width", type=int, default=1920) + parser.add_argument("--height", type=int, default=1080) + parser.add_argument("--fps", type=int, default=30) + args = parser.parse_args() + + with open(args.config) as f: + config = json.load(f) + + create_listing_video(config, args.output, args.width, args.height, args.fps) diff --git a/skills/video-creator/scripts/market_video.py b/skills/video-creator/scripts/market_video.py new file mode 100755 index 00000000..9d6e3d83 --- /dev/null +++ b/skills/video-creator/scripts/market_video.py @@ -0,0 +1,318 @@ +#!/usr/bin/env python3 +""" +Market Update / Educational Video Template — Longer-form content for +market reports, educational explainers, and data-driven presentations. + +Config shape: +{ + "type": "market_update" | "explainer", + "title": "Silicon Valley Market Update", + "subtitle": "March 2026", + "sections": [ + { + "headline": "Median Home Price", + "content": "Prices rose 4.2% year-over-year to $1.85M", + "stat_value": "$1.85M", + "stat_label": "Median Price", + "stat_change": "+4.2%", + "image": "/path/to/chart.png" # optional + } + ], + "takeaways": ["Key point 1", "Key point 2"], + "agent_name": "Graeham Watts", + "agent_title": "REALTOR® | DRE# 01466876", + "agent_contact": "graehamwatts@gmail.com", + "theme": "luxury", + "aspect_ratio": "landscape", +} +""" + +import json +import os +import sys +from typing import Dict, List + +sys.path.insert(0, os.path.dirname(__file__)) +from video_engine import ( + COLORS, FONT_SANS, FONT_SANS_BOLD, FONT_SERIF, + LowerThird, Slide, TextAnimation, TextOverlay, Transition, + VideoProject, render_video, +) + + +THEMES = { + "luxury": { + "bg_dark": (15, 25, 50), + "bg_light": (245, 240, 230), + "accent": (198, 168, 124), + "text_on_dark": (255, 255, 255), + "text_on_light": (30, 30, 30), + "stat_color": (198, 168, 124), + "positive": (80, 200, 120), + "negative": (255, 100, 100), + }, + "modern": { + "bg_dark": (20, 20, 25), + "bg_light": (248, 248, 252), + "accent": (60, 130, 220), + "text_on_dark": (255, 255, 255), + "text_on_light": (30, 30, 40), + "stat_color": (60, 130, 220), + "positive": (50, 205, 130), + "negative": (255, 85, 85), + }, +} + + +def create_market_video(config: Dict, output_path: str) -> str: + """Create a market update or educational video.""" + theme_name = config.get("theme", "luxury") + theme = THEMES.get(theme_name, THEMES["luxury"]) + + ar = config.get("aspect_ratio", "landscape") + if ar == "portrait": + w, h = 1080, 1920 + elif ar == "square": + w, h = 1080, 1080 + else: + w, h = 1920, 1080 + + slides = [] + + # ── Title Slide ────────────────────────────────────────────────────── + title = config.get("title", "Market Update") + subtitle = config.get("subtitle", "") + + slides.append(Slide( + duration=4.0, + background_color=theme["bg_dark"], + transition_in=Transition.FADE, + transition_duration=0.8, + texts=[ + TextOverlay( + text=title.upper(), + position=(_cx(w, w - 200), h // 2 - 80), + font_path=FONT_SANS_BOLD, + font_size=60 if len(title) < 30 else 46, + color=theme["text_on_dark"], + max_width=w - 200, + align="center", + animation=TextAnimation.FADE_IN, + shadow=True, + ), + TextOverlay( + text=subtitle, + position=(_cx(w, w - 200), h // 2 + 10), + font_path=FONT_SANS, + font_size=30, + color=theme["accent"], + max_width=w - 200, + align="center", + animation=TextAnimation.SLIDE_UP, + ), + ], + )) + + # ── Section Slides ─────────────────────────────────────────────────── + sections = config.get("sections", []) + for i, section in enumerate(sections): + # Alternate between dark and light backgrounds + is_dark = i % 2 == 0 + bg = theme["bg_dark"] if is_dark else theme["bg_light"] + text_color = theme["text_on_dark"] if is_dark else theme["text_on_light"] + + texts = [] + + # Section headline + texts.append(TextOverlay( + text=section.get("headline", "").upper(), + position=(120, 80) if ar == "landscape" else (80, 200), + font_path=FONT_SANS_BOLD, + font_size=42, + color=theme["accent"], + max_width=w - 240, + animation=TextAnimation.FADE_IN, + shadow=is_dark, + )) + + # If there's a big stat value, show it prominently + if section.get("stat_value"): + texts.append(TextOverlay( + text=section["stat_value"], + position=(_cx(w, w - 200), h // 2 - 60), + font_path=FONT_SANS_BOLD, + font_size=96, + color=theme["stat_color"], + max_width=w - 200, + align="center", + animation=TextAnimation.SCALE_IN, + shadow=is_dark, + )) + + if section.get("stat_label"): + texts.append(TextOverlay( + text=section["stat_label"], + position=(_cx(w, w - 200), h // 2 - 110), + font_path=FONT_SANS, + font_size=26, + color=text_color, + max_width=w - 200, + align="center", + animation=TextAnimation.FADE_IN, + shadow=is_dark, + )) + + if section.get("stat_change"): + change = section["stat_change"] + change_color = theme["positive"] if change.startswith("+") else theme["negative"] + texts.append(TextOverlay( + text=change, + position=(_cx(w, w - 200), h // 2 + 50), + font_path=FONT_SANS_BOLD, + font_size=40, + color=change_color, + max_width=w - 200, + align="center", + animation=TextAnimation.SLIDE_UP, + )) + + # Content text (if no stat, or as supporting text) + if section.get("content"): + content_y = h // 2 + 100 if section.get("stat_value") else h // 2 - 40 + texts.append(TextOverlay( + text=section["content"], + position=(120, content_y) if ar == "landscape" else (80, content_y), + font_path=FONT_SANS, + font_size=30, + color=text_color, + max_width=w - 240, + animation=TextAnimation.SLIDE_UP, + shadow=is_dark, + line_spacing=12, + )) + + slide = Slide( + duration=5.0, + background_color=bg, + image_path=section.get("image"), + overlay_color=(0, 0, 0, 170) if section.get("image") else None, + transition_in=Transition.FADE, + transition_duration=0.5, + texts=texts, + ) + slides.append(slide) + + # ── Key Takeaways ──────────────────────────────────────────────────── + takeaways = config.get("takeaways", []) + if takeaways: + slides.append(Slide( + duration=6.0, + background_color=theme["bg_light"], + transition_in=Transition.FADE, + transition_duration=0.5, + texts=[ + TextOverlay( + text="KEY TAKEAWAYS", + position=(120, 80) if ar == "landscape" else (80, 200), + font_path=FONT_SANS_BOLD, + font_size=44, + color=theme["text_on_light"], + animation=TextAnimation.FADE_IN, + shadow=False, + ), + TextOverlay( + text="\n".join(f"→ {t}" for t in takeaways), + position=(140, 170) if ar == "landscape" else (80, 300), + font_path=FONT_SANS, + font_size=30, + color=theme["text_on_light"], + max_width=w - 280, + animation=TextAnimation.SLIDE_UP, + shadow=False, + line_spacing=20, + ), + ], + )) + + # ── CTA / Agent Slide ──────────────────────────────────────────────── + agent_name = config.get("agent_name", "") + agent_title = config.get("agent_title", "") + agent_contact = config.get("agent_contact", "") + + cta_texts = [] + if agent_name: + cta_texts.append(TextOverlay( + text=agent_name, + position=(_cx(w, w - 600), h // 2 - 60), + font_path=FONT_SANS_BOLD, + font_size=48, + color=theme["text_on_dark"], + max_width=600, + align="center", + animation=TextAnimation.FADE_IN, + shadow=True, + )) + if agent_title: + cta_texts.append(TextOverlay( + text=agent_title, + position=(_cx(w, w - 600), h // 2 + 10), + font_path=FONT_SANS, + font_size=26, + color=theme["accent"], + max_width=600, + align="center", + animation=TextAnimation.SLIDE_UP, + )) + if agent_contact: + cta_texts.append(TextOverlay( + text=agent_contact, + position=(_cx(w, w - 600), h // 2 + 50), + font_path=FONT_SANS, + font_size=24, + color=(180, 180, 180), + max_width=600, + align="center", + animation=TextAnimation.FADE_IN, + )) + + slides.append(Slide( + duration=4.0, + background_color=theme["bg_dark"], + transition_in=Transition.FADE, + transition_duration=0.8, + texts=cta_texts, + )) + + # ── Render ─────────────────────────────────────────────────────────── + project = VideoProject( + slides=slides, + width=w, + height=h, + fps=30, + output_path=output_path, + background_music=config.get("background_music"), + music_volume=config.get("music_volume", 0.3), + ) + + def progress(current, total): + pct = int(current / total * 100) + if pct % 10 == 0: + print(f" Rendering: {pct}%", flush=True) + + return render_video(project, progress_callback=progress) + + +def _cx(w, content_w): + return (w - content_w) // 2 + + +if __name__ == "__main__": + import argparse + parser = argparse.ArgumentParser() + parser.add_argument("--config", required=True) + parser.add_argument("--output", default="market_video.mp4") + args = parser.parse_args() + + with open(args.config) as f: + config = json.load(f) + create_market_video(config, args.output) diff --git a/skills/video-creator/scripts/social_video.py b/skills/video-creator/scripts/social_video.py new file mode 100755 index 00000000..969d6962 --- /dev/null +++ b/skills/video-creator/scripts/social_video.py @@ -0,0 +1,457 @@ +#!/usr/bin/env python3 +""" +Social Media Video Template — Short-form vertical videos for Reels/Shorts/TikTok. +Designed for real estate tips, market stats, quick property teasers. + +Usage: + python3 social_video.py --config social_config.json --output reel.mp4 + +Config shape: +{ + "type": "tips" | "stats" | "teaser" | "quote", + "headline": "3 Things Buyers Miss", + "items": ["Item 1", "Item 2", "Item 3"], + "stats": [{"label": "Median Price", "value": "$1.2M", "change": "+5.2%"}], + "background_image": "/path/to/image.jpg", # optional + "agent_name": "Graeham Watts", + "agent_handle": "@graehamwatts", + "theme": "luxury", + "aspect_ratio": "portrait", # portrait (default), landscape, square +} +""" + +import json +import os +import sys +from typing import Dict, List + +sys.path.insert(0, os.path.dirname(__file__)) +from video_engine import ( + COLORS, FONT_SANS, FONT_SANS_BOLD, FONT_SERIF, FONT_SERIF_BOLD, + LowerThird, Slide, TextAnimation, TextOverlay, Transition, + VideoProject, render_video, +) + + +THEMES = { + "luxury": { + "bg": (15, 25, 50), + "accent": (198, 168, 124), + "text": (255, 255, 255), + "highlight_bg": (198, 168, 124, 220), + "card_bg": (25, 40, 70, 220), + }, + "modern": { + "bg": (20, 20, 20), + "accent": (60, 180, 220), + "text": (255, 255, 255), + "highlight_bg": (60, 180, 220, 220), + "card_bg": (35, 35, 35, 220), + }, + "bold": { + "bg": (0, 0, 0), + "accent": (255, 80, 80), + "text": (255, 255, 255), + "highlight_bg": (255, 80, 80, 220), + "card_bg": (20, 20, 20, 220), + }, + "clean": { + "bg": (250, 250, 250), + "accent": (40, 40, 40), + "text": (30, 30, 30), + "highlight_bg": (40, 40, 40, 220), + "card_bg": (255, 255, 255, 220), + }, +} + + +def create_social_video(config: Dict, output_path: str) -> str: + """Create a social media video from config.""" + video_type = config.get("type", "tips") + theme_name = config.get("theme", "luxury") + theme = THEMES.get(theme_name, THEMES["luxury"]) + + ar = config.get("aspect_ratio", "portrait") + if ar == "portrait": + width, height = 1080, 1920 + elif ar == "square": + width, height = 1080, 1080 + else: + width, height = 1920, 1080 + + if video_type == "tips": + slides = _build_tips_video(config, theme, width, height) + elif video_type == "stats": + slides = _build_stats_video(config, theme, width, height) + elif video_type == "teaser": + slides = _build_teaser_video(config, theme, width, height) + elif video_type == "quote": + slides = _build_quote_video(config, theme, width, height) + else: + slides = _build_tips_video(config, theme, width, height) + + project = VideoProject( + slides=slides, + width=width, + height=height, + fps=30, + output_path=output_path, + background_music=config.get("background_music"), + music_volume=config.get("music_volume", 0.3), + ) + + def progress(current, total): + pct = int(current / total * 100) + if pct % 20 == 0: + print(f" Rendering: {pct}%", flush=True) + + return render_video(project, progress_callback=progress) + + +def _build_tips_video(config, theme, w, h): + """Build a tips-style video: hook → numbered items → CTA.""" + headline = config.get("headline", "Tips You Need to Know") + items = config.get("items", ["Tip 1", "Tip 2", "Tip 3"]) + bg_image = config.get("background_image") + + slides = [] + + # Hook slide + slides.append(Slide( + duration=3.0, + background_color=theme["bg"], + image_path=bg_image, + overlay_color=(0, 0, 0, 170) if bg_image else None, + transition_in=Transition.FADE, + transition_duration=0.5, + texts=[ + TextOverlay( + text=headline.upper(), + position=(_cx(w, w - 160), h // 2 - 80), + font_path=FONT_SANS_BOLD, + font_size=64 if len(headline) < 30 else 48, + color=theme["text"], + max_width=w - 160, + align="center", + animation=TextAnimation.SCALE_IN, + shadow=True, + ), + # Accent underline via text + TextOverlay( + text="▬" * 8, + position=(_cx(w, w - 160), h // 2 + 20), + font_path=FONT_SANS_BOLD, + font_size=24, + color=theme["accent"], + max_width=w - 160, + align="center", + animation=TextAnimation.FADE_IN, + ), + ], + )) + + # Individual tip slides + for i, item in enumerate(items): + number_text = f"{i + 1:02d}" + slides.append(Slide( + duration=3.5, + background_color=theme["bg"], + image_path=bg_image, + overlay_color=(0, 0, 0, 180) if bg_image else None, + transition_in=Transition.SLIDE_LEFT, + transition_duration=0.4, + texts=[ + # Big number + TextOverlay( + text=number_text, + position=(_cx(w, w - 160), h // 2 - 160), + font_path=FONT_SANS_BOLD, + font_size=120, + color=theme["accent"], + max_width=w - 160, + align="center", + animation=TextAnimation.SCALE_IN, + shadow=False, + ), + # Tip text + TextOverlay( + text=item, + position=(_cx(w, w - 160), h // 2 + 0), + font_path=FONT_SANS_BOLD, + font_size=40, + color=theme["text"], + max_width=w - 160, + align="center", + animation=TextAnimation.SLIDE_UP, + shadow=True, + line_spacing=12, + ), + ], + )) + + # CTA slide + agent_name = config.get("agent_name", "") + handle = config.get("agent_handle", "") + slides.append(_make_cta_slide(agent_name, handle, theme, w, h)) + + return slides + + +def _build_stats_video(config, theme, w, h): + """Build a market stats video: headline → stat cards → CTA.""" + headline = config.get("headline", "Market Update") + stats = config.get("stats", []) + bg_image = config.get("background_image") + + slides = [] + + # Headline + slides.append(Slide( + duration=2.5, + background_color=theme["bg"], + transition_in=Transition.FADE, + transition_duration=0.5, + texts=[ + TextOverlay( + text=headline.upper(), + position=(_cx(w, w - 160), h // 2 - 60), + font_path=FONT_SANS_BOLD, + font_size=56, + color=theme["text"], + max_width=w - 160, + align="center", + animation=TextAnimation.FADE_IN, + shadow=True, + ), + ], + )) + + # Each stat gets its own slide + for stat in stats: + label = stat.get("label", "") + value = stat.get("value", "") + change = stat.get("change", "") + + texts = [ + TextOverlay( + text=label.upper(), + position=(_cx(w, w - 160), h // 2 - 140), + font_path=FONT_SANS, + font_size=32, + color=theme["accent"], + max_width=w - 160, + align="center", + animation=TextAnimation.FADE_IN, + ), + TextOverlay( + text=value, + position=(_cx(w, w - 160), h // 2 - 80), + font_path=FONT_SANS_BOLD, + font_size=96, + color=theme["text"], + max_width=w - 160, + align="center", + animation=TextAnimation.SCALE_IN, + shadow=True, + ), + ] + if change: + change_color = (80, 200, 120) if change.startswith("+") else (255, 100, 100) + texts.append(TextOverlay( + text=change, + position=(_cx(w, w - 160), h // 2 + 40), + font_path=FONT_SANS_BOLD, + font_size=44, + color=change_color, + max_width=w - 160, + align="center", + animation=TextAnimation.SLIDE_UP, + )) + + slides.append(Slide( + duration=3.5, + background_color=theme["bg"], + transition_in=Transition.FADE, + transition_duration=0.4, + texts=texts, + )) + + # CTA + slides.append(_make_cta_slide( + config.get("agent_name", ""), + config.get("agent_handle", ""), + theme, w, h, + )) + + return slides + + +def _build_teaser_video(config, theme, w, h): + """Build a property teaser — quick photos with stats overlay.""" + photos = config.get("photos", []) + address = config.get("address", "") + price = config.get("price", "") + + slides = [] + + # Quick flash through photos + for i, photo in enumerate(photos[:6]): + slide = Slide( + duration=2.0, + image_path=photo, + transition_in=Transition.KENBURNS if i % 2 == 0 else Transition.SLIDE_LEFT, + transition_duration=0.3, + kenburns_direction=["zoom_in", "pan_left", "zoom_out", "pan_right"][i % 4], + overlay_color=(0, 0, 0, 60), + ) + + # Address + price on first slide + if i == 0 and (address or price): + slide.overlay_color = (0, 0, 0, 140) + if address: + slide.texts.append(TextOverlay( + text=address.upper(), + position=(_cx(w, w - 120), h // 2 - 50), + font_path=FONT_SANS_BOLD, + font_size=52, + color=theme["text"], + max_width=w - 120, + align="center", + animation=TextAnimation.SCALE_IN, + shadow=True, + )) + if price: + slide.texts.append(TextOverlay( + text=price, + position=(_cx(w, w - 120), h // 2 + 30), + font_path=FONT_SANS_BOLD, + font_size=44, + color=theme["accent"], + max_width=w - 120, + align="center", + animation=TextAnimation.FADE_IN, + shadow=True, + )) + + slides.append(slide) + + # CTA + slides.append(_make_cta_slide( + config.get("agent_name", ""), + config.get("agent_handle", ""), + theme, w, h, + )) + + return slides + + +def _build_quote_video(config, theme, w, h): + """Build a quote/testimonial video.""" + quote = config.get("quote", config.get("headline", "")) + attribution = config.get("attribution", "") + bg_image = config.get("background_image") + + slides = [ + Slide( + duration=6.0, + background_color=theme["bg"], + image_path=bg_image, + overlay_color=(0, 0, 0, 170) if bg_image else None, + blur_background=True if bg_image else False, + transition_in=Transition.FADE, + transition_duration=0.8, + texts=[ + TextOverlay( + text=f'"{quote}"', + position=(_cx(w, w - 200), h // 2 - 100), + font_path=FONT_SERIF, + font_size=38, + color=theme["text"], + max_width=w - 200, + align="center", + animation=TextAnimation.FADE_IN, + shadow=True, + line_spacing=16, + ), + TextOverlay( + text=f"— {attribution}" if attribution else "", + position=(_cx(w, w - 200), h // 2 + 60), + font_path=FONT_SANS, + font_size=28, + color=theme["accent"], + max_width=w - 200, + align="center", + animation=TextAnimation.SLIDE_UP, + ), + ], + ), + _make_cta_slide( + config.get("agent_name", ""), + config.get("agent_handle", ""), + theme, w, h, + ), + ] + + return slides + + +def _make_cta_slide(agent_name, handle, theme, w, h): + """Reusable CTA slide.""" + texts = [] + if agent_name: + texts.append(TextOverlay( + text=agent_name, + position=(_cx(w, w - 160), h // 2 - 40), + font_path=FONT_SANS_BOLD, + font_size=44, + color=theme["text"], + max_width=w - 160, + align="center", + animation=TextAnimation.FADE_IN, + shadow=True, + )) + if handle: + texts.append(TextOverlay( + text=handle, + position=(_cx(w, w - 160), h // 2 + 30), + font_path=FONT_SANS, + font_size=30, + color=theme["accent"], + max_width=w - 160, + align="center", + animation=TextAnimation.SLIDE_UP, + )) + texts.append(TextOverlay( + text="FOLLOW FOR MORE", + position=(_cx(w, w - 160), h // 2 + 80), + font_path=FONT_SANS_BOLD, + font_size=24, + color=(160, 160, 160), + max_width=w - 160, + align="center", + animation=TextAnimation.FADE_IN, + )) + + return Slide( + duration=3.0, + background_color=theme["bg"], + transition_in=Transition.FADE, + transition_duration=0.6, + texts=texts, + ) + + +def _cx(width, content_width): + return (width - content_width) // 2 + + +if __name__ == "__main__": + import argparse + parser = argparse.ArgumentParser() + parser.add_argument("--config", required=True) + parser.add_argument("--output", default="social_video.mp4") + args = parser.parse_args() + + with open(args.config) as f: + config = json.load(f) + create_social_video(config, args.output) diff --git a/skills/video-creator/scripts/video_engine.py b/skills/video-creator/scripts/video_engine.py new file mode 100755 index 00000000..99f27b3d --- /dev/null +++ b/skills/video-creator/scripts/video_engine.py @@ -0,0 +1,773 @@ +#!/usr/bin/env python3 +""" +Video Creator Engine — Core rendering pipeline. +Uses Pillow for frame generation and ffmpeg for encoding. +Designed for real estate content: listing videos, social clips, market updates. +""" + +import json +import math +import os +import subprocess +import tempfile +import textwrap +from dataclasses import dataclass, field +from enum import Enum +from pathlib import Path +from typing import List, Optional, Tuple + +from PIL import Image, ImageDraw, ImageFilter, ImageFont, ImageEnhance + +# ─── Constants ──────────────────────────────────────────────────────────────── + +FPS = 30 +FONT_DIR = "/usr/share/fonts/opentype/urw-base35" + +# Aspect ratios +LANDSCAPE = (1920, 1080) # 16:9 YouTube / standard +PORTRAIT = (1080, 1920) # 9:16 Reels / Shorts / TikTok +SQUARE = (1080, 1080) # 1:1 Instagram feed + +# Font paths (clean, professional sans-serif) +FONT_SANS = os.path.join(FONT_DIR, "NimbusSans-Regular.otf") +FONT_SANS_BOLD = os.path.join(FONT_DIR, "NimbusSans-Bold.otf") +FONT_SERIF = os.path.join(FONT_DIR, "NimbusRoman-Regular.otf") +FONT_SERIF_BOLD = os.path.join(FONT_DIR, "NimbusRoman-Bold.otf") + +# Color palette — real estate professional +COLORS = { + "white": (255, 255, 255), + "black": (0, 0, 0), + "dark_gray": (30, 30, 30), + "charcoal": (45, 45, 45), + "medium_gray": (120, 120, 120), + "light_gray": (200, 200, 200), + "off_white": (245, 245, 245), + "gold": (198, 168, 124), + "navy": (20, 40, 80), + "deep_blue": (15, 25, 60), + "teal": (0, 128, 128), + "forest": (34, 85, 51), + "warm_white": (255, 250, 240), + "accent_blue": (60, 100, 170), +} + + +class Transition(Enum): + CUT = "cut" + FADE = "fade" + SLIDE_LEFT = "slide_left" + SLIDE_RIGHT = "slide_right" + ZOOM_IN = "zoom_in" + ZOOM_OUT = "zoom_out" + DISSOLVE = "dissolve" + WIPE_LEFT = "wipe_left" + KENBURNS = "kenburns" + + +class TextAnimation(Enum): + NONE = "none" + FADE_IN = "fade_in" + SLIDE_UP = "slide_up" + TYPEWRITER = "typewriter" + SCALE_IN = "scale_in" + + +@dataclass +class TextOverlay: + """A text element to render on a frame.""" + text: str + position: Tuple[int, int] # (x, y) — top-left of text block + font_path: str = FONT_SANS_BOLD + font_size: int = 48 + color: Tuple[int, int, int] = (255, 255, 255) + shadow: bool = True + shadow_color: Tuple[int, int, int] = (0, 0, 0) + shadow_offset: int = 3 + max_width: Optional[int] = None # wrap text if set + align: str = "left" # left, center, right + animation: TextAnimation = TextAnimation.FADE_IN + bg_color: Optional[Tuple[int, int, int, int]] = None # RGBA background pill + bg_padding: int = 20 + line_spacing: int = 8 + + +@dataclass +class LowerThird: + """Professional lower-third bar with headline + subtitle.""" + headline: str + subtitle: str = "" + bar_color: Tuple[int, int, int] = (20, 40, 80) + accent_color: Tuple[int, int, int] = (198, 168, 124) + text_color: Tuple[int, int, int] = (255, 255, 255) + position: str = "bottom" # bottom or top + width_pct: float = 0.65 + animation: TextAnimation = TextAnimation.SLIDE_UP + + +@dataclass +class Slide: + """One segment of the video.""" + duration: float # seconds + background_color: Tuple[int, int, int] = (0, 0, 0) + image_path: Optional[str] = None + image_fit: str = "cover" # cover, contain, fill + texts: List[TextOverlay] = field(default_factory=list) + lower_third: Optional[LowerThird] = None + transition_in: Transition = Transition.FADE + transition_duration: float = 0.5 # seconds for transition + kenburns_direction: str = "zoom_in" # zoom_in, zoom_out, pan_left, pan_right + overlay_color: Optional[Tuple[int, int, int, int]] = None # RGBA dark overlay + blur_background: bool = False + + +@dataclass +class VideoProject: + """Full video project definition.""" + slides: List[Slide] + width: int = 1920 + height: int = 1080 + fps: int = 30 + output_path: str = "output.mp4" + background_music: Optional[str] = None + music_volume: float = 0.3 + + +# ─── Frame Rendering ───────────────────────────────────────────────────────── + +def load_font(path: str, size: int) -> ImageFont.FreeTypeFont: + """Load a font, falling back to default if needed.""" + try: + return ImageFont.truetype(path, size) + except (IOError, OSError): + try: + return ImageFont.truetype(FONT_SANS, size) + except: + return ImageFont.load_default() + + +def fit_image(img: Image.Image, width: int, height: int, mode: str = "cover") -> Image.Image: + """Resize and crop/pad image to fit target dimensions.""" + if mode == "cover": + # Scale up to cover, then center-crop + ratio_w = width / img.width + ratio_h = height / img.height + ratio = max(ratio_w, ratio_h) + new_w = int(img.width * ratio) + new_h = int(img.height * ratio) + img = img.resize((new_w, new_h), Image.LANCZOS) + left = (new_w - width) // 2 + top = (new_h - height) // 2 + img = img.crop((left, top, left + width, top + height)) + elif mode == "contain": + img.thumbnail((width, height), Image.LANCZOS) + bg = Image.new("RGB", (width, height), (0, 0, 0)) + offset_x = (width - img.width) // 2 + offset_y = (height - img.height) // 2 + bg.paste(img, (offset_x, offset_y)) + img = bg + elif mode == "fill": + img = img.resize((width, height), Image.LANCZOS) + return img + + +def apply_kenburns(img: Image.Image, width: int, height: int, + progress: float, direction: str = "zoom_in") -> Image.Image: + """Apply Ken Burns (slow zoom/pan) effect to an image.""" + # Start with image slightly larger than frame + scale_start = 1.15 + scale_end = 1.0 + + if direction == "zoom_in": + scale_start, scale_end = 1.0, 1.15 + elif direction == "zoom_out": + scale_start, scale_end = 1.15, 1.0 + elif direction == "pan_left": + scale_start = scale_end = 1.15 + elif direction == "pan_right": + scale_start = scale_end = 1.15 + + # Smooth easing + t = ease_in_out(progress) + scale = scale_start + (scale_end - scale_start) * t + + scaled_w = int(width * scale) + scaled_h = int(height * scale) + img_scaled = img.resize((scaled_w, scaled_h), Image.LANCZOS) + + if direction == "pan_left": + x_offset = int((scaled_w - width) * (1 - t)) + y_offset = (scaled_h - height) // 2 + elif direction == "pan_right": + x_offset = int((scaled_w - width) * t) + y_offset = (scaled_h - height) // 2 + else: + x_offset = (scaled_w - width) // 2 + y_offset = (scaled_h - height) // 2 + + return img_scaled.crop((x_offset, y_offset, x_offset + width, y_offset + height)) + + +def ease_in_out(t: float) -> float: + """Smooth easing function (cubic).""" + if t < 0.5: + return 4 * t * t * t + else: + return 1 - pow(-2 * t + 2, 3) / 2 + + +def ease_out(t: float) -> float: + """Ease-out (decelerate).""" + return 1 - pow(1 - t, 3) + + +def render_text_on_frame(draw: ImageDraw.ImageDraw, frame: Image.Image, + text_overlay: TextOverlay, progress: float, + frame_width: int, frame_height: int): + """Render a text overlay with optional animation.""" + font = load_font(text_overlay.font_path, text_overlay.font_size) + + # Word wrap if max_width is set + if text_overlay.max_width: + lines = wrap_text(text_overlay.text, font, text_overlay.max_width) + else: + lines = text_overlay.text.split('\n') + + # Calculate animation state + anim_progress = min(1.0, progress * 3) # animate over first ~0.33s equivalent + alpha = 1.0 + y_offset = 0 + scale_factor = 1.0 + + if text_overlay.animation == TextAnimation.FADE_IN: + alpha = ease_out(anim_progress) + elif text_overlay.animation == TextAnimation.SLIDE_UP: + alpha = ease_out(anim_progress) + y_offset = int(50 * (1 - ease_out(anim_progress))) + elif text_overlay.animation == TextAnimation.SCALE_IN: + scale_factor = 0.5 + 0.5 * ease_out(anim_progress) + alpha = ease_out(anim_progress) + elif text_overlay.animation == TextAnimation.TYPEWRITER: + total_chars = sum(len(l) for l in lines) + visible_chars = int(total_chars * min(1.0, progress * 2)) + lines = _typewriter_lines(lines, visible_chars) + + if alpha < 0.01: + return + + # Calculate total text block size + line_heights = [] + line_widths = [] + for line in lines: + bbox = font.getbbox(line) if line else font.getbbox(" ") + w = bbox[2] - bbox[0] + h = bbox[3] - bbox[1] + line_widths.append(w) + line_heights.append(h) + + total_height = sum(line_heights) + text_overlay.line_spacing * (len(lines) - 1) + max_line_width = max(line_widths) if line_widths else 0 + + x, y = text_overlay.position + y += y_offset + + # Draw background pill if specified + if text_overlay.bg_color and alpha > 0.5: + pad = text_overlay.bg_padding + pill_width = text_overlay.max_width if text_overlay.max_width else max_line_width + bg_rect = [x - pad, y - pad, + x + pill_width + pad, y + total_height + pad] + bg_overlay = Image.new("RGBA", frame.size, (0, 0, 0, 0)) + bg_draw = ImageDraw.Draw(bg_overlay) + bg_color_with_alpha = (*text_overlay.bg_color[:3], + int(text_overlay.bg_color[3] * alpha)) + bg_draw.rounded_rectangle(bg_rect, radius=12, fill=bg_color_with_alpha) + frame.paste(Image.alpha_composite( + frame.convert("RGBA"), bg_overlay).convert("RGB"), (0, 0)) + # Need new draw object after paste + draw = ImageDraw.Draw(frame) + + # Draw each line + current_y = y + for i, line in enumerate(lines): + if not line.strip(): + current_y += line_heights[i] + text_overlay.line_spacing + continue + + # Use max_width as the alignment container if set, otherwise use actual widest line + align_width = text_overlay.max_width if text_overlay.max_width else max_line_width + lx = x + if text_overlay.align == "center": + lx = x + (align_width - line_widths[i]) // 2 + elif text_overlay.align == "right": + lx = x + (align_width - line_widths[i]) + + # Shadow + if text_overlay.shadow and alpha > 0.3: + so = text_overlay.shadow_offset + shadow_color = (*text_overlay.shadow_color, int(180 * alpha)) + # Use a temporary RGBA layer for shadow + shadow_layer = Image.new("RGBA", frame.size, (0, 0, 0, 0)) + shadow_draw = ImageDraw.Draw(shadow_layer) + shadow_draw.text((lx + so, current_y + so), line, font=font, + fill=shadow_color) + frame.paste(Image.alpha_composite( + frame.convert("RGBA"), shadow_layer).convert("RGB"), (0, 0)) + draw = ImageDraw.Draw(frame) + + # Main text + if alpha >= 1.0: + draw.text((lx, current_y), line, font=font, fill=text_overlay.color) + else: + txt_layer = Image.new("RGBA", frame.size, (0, 0, 0, 0)) + txt_draw = ImageDraw.Draw(txt_layer) + color_with_alpha = (*text_overlay.color, int(255 * alpha)) + txt_draw.text((lx, current_y), line, font=font, fill=color_with_alpha) + frame.paste(Image.alpha_composite( + frame.convert("RGBA"), txt_layer).convert("RGB"), (0, 0)) + draw = ImageDraw.Draw(frame) + + current_y += line_heights[i] + text_overlay.line_spacing + + return draw + + +def render_lower_third(frame: Image.Image, lt: LowerThird, + progress: float, width: int, height: int) -> Image.Image: + """Render a professional lower-third bar.""" + anim_progress = ease_out(min(1.0, progress * 3)) + + bar_width = int(width * lt.width_pct) + bar_height = 90 if lt.subtitle else 60 + accent_height = 4 + + # Slide in from left + x_offset = int(bar_width * (1 - anim_progress)) * -1 + y_pos = height - bar_height - 80 if lt.position == "bottom" else 80 + + overlay = Image.new("RGBA", frame.size, (0, 0, 0, 0)) + draw = ImageDraw.Draw(overlay) + + alpha = int(230 * anim_progress) + + # Main bar + bar_color_alpha = (*lt.bar_color, alpha) + draw.rectangle([x_offset, y_pos, x_offset + bar_width, y_pos + bar_height], + fill=bar_color_alpha) + + # Accent stripe on top + accent_color_alpha = (*lt.accent_color, alpha) + draw.rectangle([x_offset, y_pos, x_offset + bar_width, y_pos + accent_height], + fill=accent_color_alpha) + + # Headline + headline_font = load_font(FONT_SANS_BOLD, 32) + text_alpha = int(255 * anim_progress) + draw.text((x_offset + 30, y_pos + accent_height + 8), lt.headline, + font=headline_font, fill=(*lt.text_color, text_alpha)) + + # Subtitle + if lt.subtitle: + sub_font = load_font(FONT_SANS, 22) + draw.text((x_offset + 30, y_pos + accent_height + 46), lt.subtitle, + font=sub_font, fill=(*lt.accent_color, text_alpha)) + + return Image.alpha_composite(frame.convert("RGBA"), overlay).convert("RGB") + + +def wrap_text(text: str, font: ImageFont.FreeTypeFont, max_width: int) -> List[str]: + """Word-wrap text to fit within max_width pixels.""" + words = text.split() + lines = [] + current_line = [] + + for word in words: + test_line = " ".join(current_line + [word]) + bbox = font.getbbox(test_line) + if bbox[2] - bbox[0] <= max_width: + current_line.append(word) + else: + if current_line: + lines.append(" ".join(current_line)) + current_line = [word] + + if current_line: + lines.append(" ".join(current_line)) + + return lines if lines else [text] + + +def _typewriter_lines(lines: List[str], visible_chars: int) -> List[str]: + """Return lines with only the first N characters visible.""" + result = [] + remaining = visible_chars + for line in lines: + if remaining <= 0: + break + if remaining >= len(line): + result.append(line) + remaining -= len(line) + else: + result.append(line[:remaining]) + remaining = 0 + return result + + +# ─── Transition Rendering ──────────────────────────────────────────────────── + +def render_transition(frame_a: Image.Image, frame_b: Image.Image, + progress: float, transition: Transition) -> Image.Image: + """Blend two frames according to the transition type.""" + t = ease_in_out(progress) + w, h = frame_a.size + + if transition == Transition.CUT: + return frame_b if progress > 0.5 else frame_a + + elif transition == Transition.FADE or transition == Transition.DISSOLVE: + return Image.blend(frame_a, frame_b, t) + + elif transition == Transition.SLIDE_LEFT: + offset = int(w * t) + result = Image.new("RGB", (w, h)) + result.paste(frame_a, (-offset, 0)) + result.paste(frame_b, (w - offset, 0)) + return result + + elif transition == Transition.SLIDE_RIGHT: + offset = int(w * t) + result = Image.new("RGB", (w, h)) + result.paste(frame_a, (offset, 0)) + result.paste(frame_b, (-(w - offset), 0)) + return result + + elif transition == Transition.ZOOM_IN: + scale = 1.0 + 0.3 * t + scaled = frame_a.resize((int(w * scale), int(h * scale)), Image.LANCZOS) + cx = (scaled.width - w) // 2 + cy = (scaled.height - h) // 2 + cropped = scaled.crop((cx, cy, cx + w, cy + h)) + return Image.blend(cropped, frame_b, t) + + elif transition == Transition.WIPE_LEFT: + result = frame_a.copy() + wipe_pos = int(w * t) + result.paste(frame_b.crop((0, 0, wipe_pos, h)), (0, 0)) + return result + + else: + return Image.blend(frame_a, frame_b, t) + + +# ─── Slide Frame Generation ────────────────────────────────────────────────── + +def render_slide_frame(slide: Slide, frame_num: int, total_frames: int, + width: int, height: int) -> Image.Image: + """Render a single frame of a slide (no transitions — just the slide content).""" + progress = frame_num / max(total_frames - 1, 1) + + # Base frame + frame = Image.new("RGB", (width, height), slide.background_color) + + # Background image + if slide.image_path and os.path.exists(slide.image_path): + try: + img = Image.open(slide.image_path).convert("RGB") + if slide.transition_in == Transition.KENBURNS: + img = fit_image(img, int(width * 1.2), int(height * 1.2), "cover") + frame = apply_kenburns(img, width, height, progress, + slide.kenburns_direction) + else: + frame = fit_image(img, width, height, slide.image_fit) + except Exception as e: + print(f"Warning: Could not load image {slide.image_path}: {e}") + + # Blur background + if slide.blur_background: + frame = frame.filter(ImageFilter.GaussianBlur(radius=15)) + + # Dark overlay + if slide.overlay_color: + overlay = Image.new("RGBA", (width, height), slide.overlay_color) + frame = Image.alpha_composite(frame.convert("RGBA"), overlay).convert("RGB") + + # Text overlays + draw = ImageDraw.Draw(frame) + for text_overlay in slide.texts: + draw = render_text_on_frame(draw, frame, text_overlay, progress, width, height) + + # Lower third + if slide.lower_third: + frame = render_lower_third(frame, slide.lower_third, progress, width, height) + + return frame + + +# ─── Video Assembly ────────────────────────────────────────────────────────── + +def render_video(project: VideoProject, progress_callback=None) -> str: + """ + Render a complete video project to MP4. + Returns the output file path. + """ + width, height, fps = project.width, project.height, project.fps + + with tempfile.TemporaryDirectory() as tmpdir: + frame_dir = os.path.join(tmpdir, "frames") + os.makedirs(frame_dir) + + global_frame = 0 + total_frames_est = sum(int(s.duration * fps) for s in project.slides) + + # Pre-render all slide frames + slide_frames_cache = {} # slide_index -> {frame_num: Image} + + for slide_idx, slide in enumerate(project.slides): + slide_total_frames = int(slide.duration * fps) + + for f in range(slide_total_frames): + frame = render_slide_frame(slide, f, slide_total_frames, width, height) + + # Handle transitions between slides + if slide_idx > 0 and f < int(slide.transition_duration * fps): + prev_slide = project.slides[slide_idx - 1] + prev_total = int(prev_slide.duration * fps) + prev_frame = render_slide_frame(prev_slide, prev_total - 1, + prev_total, width, height) + t_progress = f / max(int(slide.transition_duration * fps) - 1, 1) + frame = render_transition(prev_frame, frame, t_progress, + slide.transition_in) + + # Save frame + frame_path = os.path.join(frame_dir, f"frame_{global_frame:06d}.png") + frame.save(frame_path, "PNG") + global_frame += 1 + + if progress_callback and global_frame % 10 == 0: + progress_callback(global_frame, total_frames_est) + + print(f"Rendered {global_frame} frames. Encoding video...") + + # Encode with ffmpeg + output_path = project.output_path + ffmpeg_cmd = [ + "ffmpeg", "-y", + "-framerate", str(fps), + "-i", os.path.join(frame_dir, "frame_%06d.png"), + ] + + # Add background music if provided + if project.background_music and os.path.exists(project.background_music): + ffmpeg_cmd.extend([ + "-i", project.background_music, + "-filter_complex", + f"[1:a]volume={project.music_volume}[a]", + "-map", "0:v", "-map", "[a]", + "-shortest", + ]) + + ffmpeg_cmd.extend([ + "-c:v", "libx264", + "-pix_fmt", "yuv420p", + "-preset", "medium", + "-crf", "23", + "-movflags", "+faststart", + output_path + ]) + + result = subprocess.run(ffmpeg_cmd, capture_output=True, text=True) + if result.returncode != 0: + print(f"ffmpeg error: {result.stderr}") + raise RuntimeError(f"ffmpeg encoding failed: {result.stderr[-500:]}") + + print(f"Video saved to {output_path}") + return output_path + + +# ─── Convenience Builders ──────────────────────────────────────────────────── + +def create_title_slide(headline: str, subtitle: str = "", + bg_color=(20, 40, 80), accent_color=(198, 168, 124), + duration: float = 4.0, width: int = 1920, + height: int = 1080, image_path: str = None) -> Slide: + """Create a professional title slide.""" + texts = [] + + # Headline — centered + headline_font_size = 72 if len(headline) < 30 else 56 + headline_y = height // 2 - 80 + texts.append(TextOverlay( + text=headline, + position=(width // 2 - 400, headline_y), + font_path=FONT_SANS_BOLD, + font_size=headline_font_size, + color=(255, 255, 255), + max_width=800, + align="center", + animation=TextAnimation.FADE_IN, + shadow=True, + )) + + # Accent line + if subtitle: + texts.append(TextOverlay( + text=subtitle, + position=(width // 2 - 400, headline_y + 100), + font_path=FONT_SANS, + font_size=32, + color=accent_color, + max_width=800, + align="center", + animation=TextAnimation.SLIDE_UP, + shadow=True, + )) + + return Slide( + duration=duration, + background_color=bg_color, + image_path=image_path, + overlay_color=(0, 0, 0, 140) if image_path else None, + texts=texts, + transition_in=Transition.FADE, + transition_duration=0.8, + ) + + +def create_photo_slide(image_path: str, caption: str = "", + duration: float = 4.0, width: int = 1920, + height: int = 1080, + kenburns: bool = True) -> Slide: + """Create a photo slide with optional caption and Ken Burns effect.""" + texts = [] + if caption: + texts.append(TextOverlay( + text=caption, + position=(60, height - 140), + font_path=FONT_SANS_BOLD, + font_size=36, + color=(255, 255, 255), + max_width=width - 120, + animation=TextAnimation.SLIDE_UP, + shadow=True, + bg_color=(0, 0, 0, 160), + bg_padding=16, + )) + + return Slide( + duration=duration, + image_path=image_path, + texts=texts, + transition_in=Transition.KENBURNS if kenburns else Transition.FADE, + transition_duration=0.6, + kenburns_direction="zoom_in", + ) + + +def create_text_slide(headline: str, bullets: List[str] = None, + bg_color=(245, 245, 245), text_color=(30, 30, 30), + duration: float = 5.0, width: int = 1920, + height: int = 1080) -> Slide: + """Create a text/content slide with headline and optional bullets.""" + texts = [ + TextOverlay( + text=headline, + position=(120, 120), + font_path=FONT_SANS_BOLD, + font_size=52, + color=text_color, + max_width=width - 240, + animation=TextAnimation.FADE_IN, + shadow=False, + ) + ] + + if bullets: + bullet_text = "\n".join(f"• {b}" for b in bullets) + texts.append(TextOverlay( + text=bullet_text, + position=(140, 220), + font_path=FONT_SANS, + font_size=36, + color=(80, 80, 80), + max_width=width - 280, + animation=TextAnimation.SLIDE_UP, + shadow=False, + line_spacing=20, + )) + + return Slide( + duration=duration, + background_color=bg_color, + texts=texts, + transition_in=Transition.FADE, + transition_duration=0.5, + ) + + +def create_cta_slide(headline: str, subtitle: str = "", + contact_info: str = "", + bg_color=(20, 40, 80), + accent_color=(198, 168, 124), + duration: float = 5.0, width: int = 1920, + height: int = 1080) -> Slide: + """Create a call-to-action / closing slide.""" + texts = [ + TextOverlay( + text=headline, + position=(width // 2 - 400, height // 2 - 100), + font_path=FONT_SANS_BOLD, + font_size=64, + color=(255, 255, 255), + max_width=800, + align="center", + animation=TextAnimation.SCALE_IN, + shadow=True, + ) + ] + + if subtitle: + texts.append(TextOverlay( + text=subtitle, + position=(width // 2 - 400, height // 2 + 20), + font_path=FONT_SANS, + font_size=32, + color=accent_color, + max_width=800, + align="center", + animation=TextAnimation.FADE_IN, + )) + + if contact_info: + texts.append(TextOverlay( + text=contact_info, + position=(width // 2 - 400, height // 2 + 80), + font_path=FONT_SANS, + font_size=28, + color=(200, 200, 200), + max_width=800, + align="center", + animation=TextAnimation.FADE_IN, + )) + + return Slide( + duration=duration, + background_color=bg_color, + texts=texts, + transition_in=Transition.FADE, + transition_duration=1.0, + ) + + +# ─── Entry Point for Testing ──────────────────────────────────────────────── + +if __name__ == "__main__": + print("Video Creator Engine loaded successfully.") + print(f"Available fonts: {FONT_SANS}, {FONT_SANS_BOLD}") + print(f"Pillow version: {Image.__version__}") + + # Quick smoke test — render one frame + test_slide = create_title_slide("Test Video", "Engine Check") + frame = render_slide_frame(test_slide, 15, 30, 1920, 1080) + test_path = "/tmp/video_engine_test.png" + frame.save(test_path) + print(f"Test frame saved to {test_path}") diff --git a/skills/video-editor/SKILL.md b/skills/video-editor/SKILL.md new file mode 100644 index 00000000..99b4eca6 --- /dev/null +++ b/skills/video-editor/SKILL.md @@ -0,0 +1,118 @@ +--- +name: video-editor +description: >- + Turn a folder of raw 4K property/listing clips into finished, on-brand marketing: vertical Reels + (9:16), a wide Walkthrough tour (16:9, 2-3 min), a Just Listed single-image AD KIT (3 hero + concepts x portrait/square/landscape, ready-to-post PNGs + Canva-editable PDFs), and a 10-slide + Instagram CAROUSEL. Use ANY time the user wants to edit, cut, or assemble property or listing + videos, make reels or a walkthrough from raw clips, create Just Listed / New Listing ads or + flyers, or build an Instagram carousel for a property. Trigger on 'edit these clips', 'make me + reels', 'cut a walkthrough', 'property video', 'listing video', 'feature video', 'just listed + ad', 'new listing flyer', 'carousel for this property', 'turn this footage into a video', or + when the user points at a folder of property MP4s wanting video or image marketing output. Over- + trigger for any raw-footage-to-social-asset task. Outputs ready-to-post files plus Canva- + editable PDFs on a black-and-gold brand. +--- + +# Video Editor — a listing's raw footage to finished marketing + +Turn a folder of raw 4K property clips into four deliverables, all on one black-and-gold +brand, all built locally with `ffmpeg` + Pillow + ReportLab: + +1. **Reels** — 9:16 (1080x1920) vertical highlight cuts for IG / TikTok / Shorts. +2. **Walkthrough** — 16:9 (1920x1080), a 2-3 min buyer's-journey tour. +3. **Just Listed ad kit** — 3 hero concepts x 3 sizes (1080x1350, 1080x1080, 1200x628); + ready-to-post PNGs + Canva-editable PDFs + clean hero photos + a usage README. +4. **Instagram carousel** — 10 slides (1080x1350); PNGs + Canva-editable PDF + caption. + +`scripts/` holds the engines; `references/` holds the step-by-step for each deliverable. +**Read the matching reference before building a deliverable** — the recipes there are the +result of a full session of trial-and-error, not guesses. + +## 0 · The operating reality — read this or you will fail +You're usually in a small sandbox: ~2 CPU cores, ~4 GB RAM, and a **hard ~45-second limit +per shell command**, against **16+ GB of 4K source**. One full-timeline encode WILL time +out and leave a truncated, unplayable file. The entire pipeline exists to dodge this: +- **Decode each 4K clip once** into small graded 1080p segments; assemble from those. +- **Batch <= 4 renders per shell call**; echo elapsed seconds; keep each call under ~35s. +- **Never assemble in one pass.** Build short chunks, then `ffmpeg -f concat -c copy` them + (instant, no re-encode). +- **Add a silent audio track with an explicit `-t `.** `-shortest` does NOT stop a + copied video stream against an infinite `anullsrc` — it runs to the timeout. +- **Don't use `+faststart` on big files** in one call (the moov-relocation second pass + blows the time budget). Mux without it. +- **Verify by eye every time**: extract frames -> `montage` -> view. A clean exit is not + proof; check the pixels. +Full failure-mode list and fixes: **`references/sandbox-constraints.md`**. + +## 1 · Always review the footage first +Before cutting anything, extract one labeled thumbnail per clip, montage them into contact +sheets, and actually LOOK. Verify what each clip contains (never trust filenames or a prior +edit plan), record real durations, and **clamp every in/out point to the true clip length**. +This is also where you catch deal-breakers — most importantly a **realtor yard sign visible +inside an exterior clip**, which is a blocker if the user asked for no agent info. +Recipe: `references/video-pipeline.md` (section "Review the footage"). + +## 2 · Brand system (summary) +Black-and-gold, font **Poppins**. Gold `#C7A974`, ink `#0E0C0A`, off-white `#F4EEE3`, +light-gold `#DEC9A2`. Thin gold inset frame, a gold "JUST LISTED" pill (ink pill with gold +outline for "NEW LISTING"), a lower-third dark gradient, and a bold address lockup. Full +tokens + the Pillow/drawtext recipes: **`references/brand-system.md`**. + +## 3 · Rules that are NOT optional +These came straight from the operator and apply every single time: +- **Never invent facts.** Beds, baths, sq ft, price, "stainless steel", "2-car garage" — if + you can't see it in the footage and the user didn't give it to you, **leave it off** and + offer to add it once they confirm. Caption only features visible on screen. (Example from + the build: the range was black, not stainless — so the caption said "gas range", not + "stainless".) +- **Honor "no agent info."** No name / phone / website / DRE when asked — and also scrub any + **yard sign** that appears inside a photo or clip by swapping to a clean shot (an aerial + works well for a closing slide). +- **Music:** you cannot burn in licensed/commercial music (copyright). Default to a **clean + (silent) export** so the operator scores it in their own editor, unless they say otherwise. + Cut to ~2s beats so it syncs to most 120-128 BPM tracks later. +- **"Editable in Canva" = a PDF with REAL text**, never a flattened image. Build it with + ReportLab as layers: clean photo (replaceable) + separate gradient overlay (`mask='auto'`) + + vector gold frame + real Poppins text. **Verify with `pdftotext`** — the words must come + back — before you deliver. +- **Ask before building** with AskUserQuestion: how to handle music, what (if anything) goes + on the end-card / contact lockup, and how different the two reels should be. Guessing these + wastes a full render pass. + +## 4 · Workflows +Open the matching reference for the deliverable(s) requested: + +| Deliverable | Reference | Engines | +|---|---|---| +| Reels (9:16) + Walkthrough (16:9) | `references/video-pipeline.md` | `scripts/video_lib.sh`, `scripts/chunk.py` | +| Just Listed ad kit | `references/ad-kit.md` | `scripts/ad_lib.py`, `scripts/make_ad_pdfs.py` | +| Instagram carousel | `references/carousel.md` | `scripts/carousel_lib.py`, `scripts/make_carousel_pdf.py` | + +Shared flow for all of them: **review footage -> confirm choices with the user -> render in +small batches -> QC by viewing frames -> deliver into the user's listing folder with +`present_files` -> offer the obvious next adds** (a text-free cut, other sizes, a stats slide +once numbers exist, etc.). + +## 5 · Bundled scripts (how to call) +Every script finds Poppins automatically (or via `VE_FONTDIR`) and defaults output paths to +relative dirs, so set the env vars per session. + +- **`video_lib.sh`** — `export VE_SRC="" VE_OUT=""; source video_lib.sh`. + Gives `seg16` (16:9 segment), `seg916c` (vertical center-crop), `seg916p` (vertical + blurred-pad for wide shots), and `xf` (xfade two segments). Recipe in video-pipeline.md. +- **`chunk.py`** — `python chunk.py OUT FADEIN FADEOUT seg1 [TRANSITION DUR seg2]...` builds + one xfade chunk with exact offsets. Transitions: `fade fadeblack fadewhite dissolve + slideleft slideright slideup smoothleft circleopen wiperight ...`. +- **`ad_lib.py`** — edit `CONCEPTS` / `SIZES` / `ADDR1` / `ADDR2` at the top; set + `VE_HERO=""`; call `render(concept, sizekey, mode)` where mode is + `png` (flat), `plate` (frame+gradient, no text), or `clean` (photo only). See ad-kit.md. +- **`make_ad_pdfs.py`** — builds the 3 Canva-editable ad PDFs with real text. `VE_KIT=""`. +- **`carousel_lib.py`** — edit `SLIDES`; set `VE_CHEROES=""`; + call `render(i, mode)`. Imports `ad_lib` (keep them in the same folder). +- **`make_carousel_pdf.py`** — builds the 10-page editable carousel PDF. `VE_CAROUSEL_PDF=""`. + +**Dependencies:** `ffmpeg` / `ffprobe`, ImageMagick (`montage`), Python `pillow` + `reportlab`, +`poppler-utils` (`pdftoppm`, `pdftotext`), and the **Poppins** font family. Install Poppins or +set `VE_FONTDIR` if `ad_lib` can't find it. The video drawtext labels fall back to DejaVu-Bold +(`VE_FONT`) since it's preinstalled everywhere. diff --git a/skills/video-editor/references/ad-kit.md b/skills/video-editor/references/ad-kit.md new file mode 100644 index 00000000..55b33e35 --- /dev/null +++ b/skills/video-editor/references/ad-kit.md @@ -0,0 +1,43 @@ +# Just Listed ad kit + +Engines: `scripts/ad_lib.py` (PNG renderer) + `scripts/make_ad_pdfs.py` (Canva-editable PDFs). +Minimal by design: the only text is the **stamp** (JUST LISTED / NEW LISTING) and the +**address** — no tagline, no agent info, no price (unless the user explicitly adds it). + +## Concepts & sizes +3 hero concepts x 3 sizes = 9 ready-to-post PNGs: +- **Exterior** (best 3/4 curb shot) -> stamp `JUST LISTED` (gold pill) +- **Interior** (best bright room) -> stamp `NEW LISTING` (ink pill, gold outline) +- **Detail / lifestyle** (e.g. granite island) -> stamp `JUST LISTED` +Sizes: `1080x1350` (IG/FB portrait, highest impact), `1080x1080` (square), +`1200x628` (landscape — Google Display + FB link ads). + +## Build +1. Extract 3 clean, graded 4K hero stills into `VE_HERO` (`hero_exterior.png`, etc.). +2. In `ad_lib.py` edit `CONCEPTS` (hero file, stamp text, style gold/ink, focal point), + `SIZES` (sizes + per-size font sizes/margins), and `ADDR1`/`ADDR2`. +3. Render: `render(concept, sizekey, mode)` — `mode="png"` (flat, ready-to-post), + `"plate"` (frame+gradient, no text — background for the PDF), `"clean"` (photo only). + Drive it from a small loop (see `make_kit` pattern) to emit all 9 PNGs + plates + heroes. +4. `make_ad_pdfs.py` builds the 3 editable PDFs: clean photo layer + gradient overlay + (`drawImage(..., mask='auto')`) + vector gold frame + **real Poppins text** (pill + + address). **Verify with `pdftotext`** that the words come back. + +## Framing +Center-crop most shots to each aspect with a focal point that keeps the subject and leaves +the busy detail out of the lower third (text zone). For very wide shots in the portrait/ +square crops, bias the focal point rather than squashing. + +## Deliver +Folder structure in the listing dir: +``` +Just Listed Ads/ + Ready-to-Post-PNG/ (9 PNGs) + Canva-Editable-PDF/ (3 PDFs, one per size, 3 concept pages each) + Hero-Photos/ (3 clean graded JPGs, no text) + READ-ME_Ad-Usage-and-Canva-Guide.txt +``` +README covers: what's on the ads, where each size is used, how to edit in Canva (import PDF; +text is real Poppins; photo is a replaceable layer; frame+gradient are separate), and the +brand specs. `present_files` a representative set (the 3 portraits + the 3 PDFs). +Offer: IG Story 1080x1920, an "Offered at $___" or contact bar, or alternate hero swaps. diff --git a/skills/video-editor/references/brand-system.md b/skills/video-editor/references/brand-system.md new file mode 100644 index 00000000..81ffc737 --- /dev/null +++ b/skills/video-editor/references/brand-system.md @@ -0,0 +1,47 @@ +# Brand system — black & gold + +One look across every deliverable so videos, ads, and the carousel feel like a set. + +## Tokens +- Ink / black `#0E0C0A` (RGB 14,12,10) +- Gold `#C7A974` (199,169,116) +- Off-white `#F4EEE3` (244,238,227) +- Light gold `#DEC9A2` (222,201,162) +- Muted `#BBAB8E` (187,171,142) +- Ink pill fill `#12100D` (slightly lifted ink, for the "NEW LISTING" pill) + +## Fonts — Poppins +- **Bold** = address / feature labels +- **Medium** = stamp pill, kickers, "now available" lines +- **Light** = city line, sub-lines, body (use ~0.16x letter-spacing/tracking on caps lines) +`ad_lib.py` auto-detects the Poppins dir; video drawtext labels use DejaVu-Bold (preinstalled) +so they work even where Poppins isn't. + +## Image grade (stills + video) +`eq=contrast=1.06:saturation=1.10:brightness=0.004:gamma=0.99,unsharp=3:3:0.5:3:3:0` — mild +contrast/sat + a touch of sharpening after the 4K->1080 downscale. For the warmer "hero" +agent clips pull warmth so cuts don't jump: add +`colorbalance=rm=-0.04:rh=-0.03:bm=0.02` (this is `GRADE_HERO` in `video_lib.sh`). + +## Layout language +- **Gold inset frame**: thin rectangle inset ~`margin` px from the edge, stroke ~3-4 px. +- **Stamp pill** (top-left): rounded pill. "JUST LISTED" = solid **gold** fill, ink text. + "NEW LISTING" = **ink** fill, gold outline, off-white text. Uppercase Poppins-Medium, tracked. +- **Lower-third gradient**: dark (`#080706`) ramp from ~50% height to the bottom so white + text is readable; a faint top gradient helps the stamp. +- **Address lockup** (lower-left): `909 BAINES ST` in Poppins-Bold, a short **gold rule**, + then `EAST PALO ALTO, CALIFORNIA` in tracked Poppins-Light light-gold. Keep it to the + **stamp + address only** unless the user asks for more (no tagline by default). + +## Video text helpers (in `video_lib.sh`) +- `label16 'TEXT' appear disappear` — bottom-left feature label + gold underline (1920x1080). +- `title16 'BIG' 'small' appear disappear` — centered opening title (e.g. address). +- `capR 'TEXT' appear disappear` — centered reel caption near the upper third (1080x1920), + with a soft box + gold underline. Captions fade in/out via an alpha expression. + +## File-naming conventions +- Videos: `-WALKTHROUGH-16x9.mp4`, `-REEL-1-9x16.mp4`, `-REEL-2-9x16.mp4`. +- Ads: `_-_-.png` (e.g. `_Exterior-JustListed_1080x1080-Square.png`). +- Carousel: numbered `..._Carousel_01-Cover.png` ... `_10-Recap.png` so they upload in order. +Deliver into the user's listing folder in tidy subfolders: `Ready-to-Post-PNG/`, +`Canva-Editable-PDF/`, `Hero-Photos/`, plus a short `READ-ME` and (carousel) `Caption.txt`. diff --git a/skills/video-editor/references/carousel.md b/skills/video-editor/references/carousel.md new file mode 100644 index 00000000..31630eab --- /dev/null +++ b/skills/video-editor/references/carousel.md @@ -0,0 +1,45 @@ +# Instagram carousel (10 slides, 1080x1350) + +Engines: `scripts/carousel_lib.py` (renderer) + `scripts/make_carousel_pdf.py` (editable PDF). +Property-only by default: **no agent name, phone, website, or price.** + +## Slide sequence (a swipeable tour) +1. **Cover** — exterior hero + gold `JUST LISTED` pill + address + a hook line + `SWIPE TO TOUR ›`. +2-9. **Feature slides** — one space each, with a bold LABEL + a one-line honest detail and a + slide counter (`02 / 10`) top-right + a small address watermark top-left. Typical set: + vaulted ceilings, open-concept living, granite island kitchen, bright bedrooms, upstairs + loft, primary suite, spa bath, private patio. +10. **Recap** — gold-dot feature list + `NOW AVAILABLE IN ` + `‹ Swipe back to tour again`. + +## Build +1. Extract ~10 graded 4K stills into `VE_CHEROES` (`s01_cover.png` ... `s10_close.png`). +2. In `carousel_lib.py` edit `SLIDES` (per slide: type cover/feat/close, hero file, focal, + label, sub, or recap list) and `ADDR1`/`ADDR2`. The renderer is **ops-based**: each slide + compiles to a list of draw primitives so the PNG and the PDF stay pixel-identical. +3. `render(i, "png")` for all 10 slides. +4. `make_carousel_pdf.py` builds the 10-page editable PDF from the same ops (clean photo + + gradient overlay + vector frame + real Poppins text). **Verify with `pdftotext`.** + +## The closing-slide trap (learned the hard way) +A recap slide wants a calm background, so it's tempting to reuse a front-elevation clip — but +those often contain the **realtor yard sign** = agent info. Use a clean **aerial** for the +closing instead, with a heavier scrim so the recap list reads. Scan every chosen still for +signage before rendering. + +## Caption +Write a property-only IG caption (light, IG-native emoji is fine in the deliverable — it's +the product, not chat): a one-line hook, a short "what's inside" feature list (only verified +features), the address, a soft engagement prompt ("which room would you claim first?"), and a +block of local + category hashtags. **No phone, no name, no price.** Save as `Caption.txt`. + +## Deliver +``` +Instagram Carousel/ + Ready-to-Post-PNG/ (10 slides, numbered 01-10 so they upload in order) + Canva-Editable-PDF/ (one 10-page PDF, real text) + Caption.txt + READ-ME_Carousel-Guide.txt +``` +`present_files` a few slides + the PDF + the caption. Note that beds/baths/sq ft/price were +left off (not verifiable from footage) and offer to add a **stats slide** once given numbers — +that's the single highest-impact add. diff --git a/skills/video-editor/references/sandbox-constraints.md b/skills/video-editor/references/sandbox-constraints.md new file mode 100644 index 00000000..4fa602b8 --- /dev/null +++ b/skills/video-editor/references/sandbox-constraints.md @@ -0,0 +1,52 @@ +# Sandbox constraints & the failure modes already hit + +The render pipeline looks indirect on purpose. Every choice below was forced by a real +failure during the build. Respect them and the work goes smoothly; ignore them and you get +timeouts and corrupt files. + +## The environment +- ~2 CPU cores, ~4 GB RAM. +- A **hard ~45 s limit per shell command** (the wrapper kills the call). +- Source is **16+ GB of 4K (3840x2160) MP4**, mixed frame rates (24 / 30 / 60 fps), some + clips have no audio (drone), some carry a warmer cinematic grade (agent "hero" shots). +- File tools (Read/Write) and the shell see **different paths** for the same files; to LOOK + at an image you generate, copy it into one of the user's connected folders and Read it + from the Windows-style path. The scratch/outputs dir is not Readable directly. + +## Why the proxy pipeline +A single straight encode of a 2:40 1080p timeline on 2 cores is ~80 s — over the limit. So: +1. **Decode each 4K clip exactly once** into a short, graded 1080p **segment** (trim with + `-ss -t ` BEFORE `-i` for fast keyframe seek, so you only decode the window you + need). 60 fps 4K is the expensive case — keep those segments short. +2. Assemble from the 1080p segments (cheap to decode), never from 4K again. + +## Batch sizes & timing +- Render **<= 4 segments per shell call.** Echo `elapsed=$(( $(date +%s)-S ))s` and keep it + under ~35 s. A batch of 4 plain segments ran ~25 s; 5 with a text overlay ran ~34 s. +- If a call times out, files that were mid-write are **truncated** (no moov atom). Re-render + just those and **validate every chunk** with `ffprobe` before concatenating. + +## Assembly without re-encoding the whole thing +- Build the timeline as short **chunks** (2-4 segments joined by `xfade`), each well under + 45 s, using `scripts/chunk.py`. +- Then join chunks with the concat demuxer **stream-copied** (instant, no quality loss): + `ffmpeg -f concat -safe 0 -i list.txt -c copy -fflags +genpts out.mp4`. +- All segments/chunks must share identical params (1080p OR vertical, 30 fps, yuv420p, + SAR 1, `-video_track_timescale 30000`) or concat-copy and xfade break. + +## Audio: the `-shortest` trap +The user wants a clean (silent) export they can score. Add a silent stereo track so players +don't choke, but **bound it with `-t`** equal to the video duration: +``` +D=$(ffprobe -v error -show_entries format=duration -of csv=p=0 video.mp4) +ffmpeg -i video.mp4 -f lavfi -t "$D" -i anullsrc=channel_layout=stereo:sample_rate=48000 \ + -map 0:v -map 1:a -c:v copy -c:a aac final.mp4 +``` +`-shortest` does NOT terminate a `-c:v copy` stream against an infinite `anullsrc`; it ran to +the 45 s kill and left a half-written file. Do not use it here. Do not add `+faststart` in +the same call on a 200 MB+ file — its second pass also blows the budget. + +## Verify by eye, always +After any render: extract frames at spread timecodes, `montage` them, copy into a connected +folder, and Read it. Check transitions landed, text is legible, grade is consistent, and +(critical) **no agent signage** snuck in. A 0 exit code is not verification. diff --git a/skills/video-editor/references/video-pipeline.md b/skills/video-editor/references/video-pipeline.md new file mode 100644 index 00000000..e8664d6b --- /dev/null +++ b/skills/video-editor/references/video-pipeline.md @@ -0,0 +1,62 @@ +# Video pipeline — Reels (9:16) + Walkthrough (16:9) + +Engines: `scripts/video_lib.sh` (segment + xfade helpers) and `scripts/chunk.py`. +Read `sandbox-constraints.md` first — the batching/timeout rules are what make this work. + +## Review the footage +```bash +# one labeled thumbnail per clip, ~45% in, then montage into contact sheets +for f in *.MP4; do + dur=$(ffprobe -v error -show_entries format=duration -of csv=p=0 "$f") + seek=$(echo "$dur*0.45"|bc) + ffmpeg -ss "$seek" -i "$f" -frames:v 1 -vf "scale=480:270,drawtext=...:text='$f'" thumbs/NN.jpg +done +montage thumbs/*.jpg -tile 4x4 -geometry 480x270+4+4 sheet.jpg # then copy to a connected folder + Read +``` +Confirm each clip's real content, note durations, and **clamp every in/out to clip length**. +Flag any **realtor yard sign** in exterior shots (blocker if "no agent info"). + +## 1 · Confirm choices (AskUserQuestion) +Music handling (default clean/silent), end-card/branding (default none beyond address), and +how different the two reels should be (recommend genuinely distinct). + +## 2 · Render segments (the only 4K decode) +```bash +export VE_SRC="/path/to/clips" VE_OUT="/path/to/work" +source scripts/video_lib.sh +# 16:9 walkthrough segment: seg16 SRC SS DUR OUT [GRADE] [POSTfilter] +seg16 "C3478.MP4" 0.0 3.9 "wt/wt10.mp4" "$GRADE_STD" "$(label16 'SOARING VAULTED CEILINGS' 0.3 3.5)" +# 9:16 reel, center-crop: seg916c SRC SS DUR OUT [GRADE] [POST] [XOFF] +seg916c "C3491.MP4" 1.0 2.3 "r1/r1_09.mp4" "$GRADE_STD" "$(capR 'PRIMARY SUITE' 0.3 2.0)" +# 9:16 reel, blurred-pad (use for wides: vaulted, aerials, exterior, sign): +seg916p "C3478.MP4" 0.2 2.6 "r1/r1_03.mp4" "$GRADE_STD" "$(capR 'VAULTED CEILINGS' 0.3 2.3)" +``` +Batch <= 4 per call. Use `GRADE_HERO` for the warmer agent clips. + +## 3 · Assemble with transitions, in chunks +`chunk.py` joins 2-4 segments with xfades at exact offsets, and can bake a dip (fade) at the +chunk's head/tail for inter-chunk transitions: +```bash +python chunk.py chunks/c01.mp4 0 0.3 wt01.mp4 dissolve 0.7 wt02.mp4 fade 0.6 wt03.mp4 +# ^out ^fadein ^fadeout ^seg ^transition dur ^seg ... +``` +Vary transitions with intent, not gimmickry: hard cuts within a section; **slideup** for +going up the stairs; **fadewhite** into a bright bath; **circleopen** into the patio; +**dissolve/smoothleft** for aerials; **dip-to-black** beats around the open and the agent +outro. Then concat-copy the chunks and add silent audio (see sandbox-constraints.md). + +## Deliverable specs +- **Walkthrough** 1920x1080, 30 fps, ~2:00-2:45. Buyer's journey: aerial -> approach -> + sign/title -> curb -> step inside -> living (vaulted) -> kitchen -> stairs/loft -> + bedrooms -> baths -> patio -> neighborhood -> agent orbit (no contact card if "no agent"). + One feature label per room; resist over-titling. ~30-41 shots. +- **Reels** 1080x1920, 30 fps, ~30-45 s, two **distinct** concepts: + - Reel 1 = story / face-led: open on the walk-up hook, warmer pace, smooth dissolves. + - Reel 2 = house-led punch: open on the "wow" (vaulted), snappier ~1.7 s cuts, slides/ + fadewhite, end on a different agent/orbit clip. + Captions are feature words only (2-4 words), upper-third safe area, no fabricated stats. + +## QC then deliver +Probe res/fps/dur/audio; extract ~8 frames across each output, montage, view. Copy finals +into the user's listing folder; `present_files`; offer a text-free cut, a 0:60/0:45 +walkthrough, other sizes, or a music pass. diff --git a/skills/video-editor/scripts/ad_lib.py b/skills/video-editor/scripts/ad_lib.py new file mode 100644 index 00000000..50e4f874 --- /dev/null +++ b/skills/video-editor/scripts/ad_lib.py @@ -0,0 +1,109 @@ +# -*- coding: utf-8 -*- +import sys, os +from PIL import Image, ImageDraw, ImageFont +HERO=os.environ.get("VE_HERO","heroes") +def _find_font_dir(): + import os + cands=[os.environ.get("VE_FONTDIR"),"/usr/share/fonts/truetype/google-fonts/", + "/usr/share/fonts/truetype/poppins/",os.path.expanduser("~/.fonts/"), + os.path.join(os.path.dirname(os.path.abspath(__file__)),"fonts")+"/"] + for d in cands: + if d and os.path.exists(os.path.join(d,"Poppins-Bold.ttf")): + return d if d.endswith("/") else d+"/" + raise SystemExit("Poppins TTFs not found. Install (apt-get install fonts-poppins / fonts-google-poppins) or set VE_FONTDIR to a folder containing Poppins-Bold.ttf etc.") +FD=_find_font_dir() +INK=(14,12,10); GOLD=(199,169,116); OFF=(244,238,227); LGOLD=(222,201,162); MUT=(187,171,142) +def F(name,sz): return ImageFont.truetype(FD+name, sz) +SIZES={ + "1080x1350-IG-Portrait": dict(w=1080,h=1350, addr=86, city=25, stamp=25, margin=44, fw=4, gstart=0.46, gmax=210), + "1080x1080-Square": dict(w=1080,h=1080, addr=78, city=24, stamp=24, margin=40, fw=4, gstart=0.46, gmax=210), + "1200x628-Landscape": dict(w=1200,h=628, addr=62, city=20, stamp=21, margin=30, fw=3, gstart=0.38, gmax=205), +} +CONCEPTS={ + "Exterior": dict(hero="hero_exterior.png", stamp="JUST LISTED", style="gold", focal=(0.52,0.45)), + "Interior": dict(hero="hero_interior.png", stamp="NEW LISTING", style="ink", focal=(0.50,0.46)), + "Detail": dict(hero="hero_detail.png", stamp="JUST LISTED", style="gold", focal=(0.50,0.52)), +} +ADDR1="909 BAINES ST"; ADDR2="EAST PALO ALTO, CALIFORNIA" + +def crop_to(img,w,h,focal): + iw,ih=img.size; tgt=w/h; src=iw/ih + if src>tgt: nw=int(round(ih*tgt)); nh=ih + else: nw=iw; nh=int(round(iw/tgt)) + fx,fy=focal; x=int(fx*iw-nw/2); y=int(fy*ih-nh/2) + x=max(0,min(iw-nw,x)); y=max(0,min(ih-nh,y)) + return img.crop((x,y,x+nw,y+nh)).resize((w,h),Image.LANCZOS) + +def vgrad(img,start,maxa,top=True): + w,h=img.size + col=Image.new("L",(1,h),0); px=col.load() + for yy in range(h): + t=(yy/h-start)/max(1e-6,(1-start)); px[0,yy]=0 if t<0 else min(255,int((t**1.3)*maxa)) + img.paste(Image.new("RGB",(w,h),(7,6,5)),(0,0),col.resize((w,h))) + if top: + col2=Image.new("L",(1,h),0); p2=col2.load() + for yy in range(h): + t=1-yy/(h*0.20); p2[0,yy]=0 if t<0 else min(255,int(t*80)) + img.paste(Image.new("RGB",(w,h),(7,6,5)),(0,0),col2.resize((w,h))) + return img + +def tlen(d,t,f,tr=0): return sum(d.textlength(c,font=f) for c in t)+tr*max(0,len(t)-1) +def tdraw(d,xy,t,f,fill,tr=0): + x,y=xy + for c in t: + d.text((x,y),c,font=f,fill=fill); x+=d.textlength(c,font=f)+tr + return x + +def pill(img,d,x,y,text,style,szcfg): + f=F("Poppins-Medium.ttf",szcfg["stamp"]); tr=szcfg["stamp"]*0.16 + asc,desc=f.getmetrics(); capH=int(asc*0.70) + tw=tlen(d,text,f,tr); padx=int(szcfg["stamp"]*1.05); pady=int(szcfg["stamp"]*0.72) + pw=tw+2*padx; ph=capH+2*pady; rad=ph//2 + if style=="gold": fillc=GOLD; txtc=INK; bd=None; bw=0 + else: fillc=(18,16,13); txtc=OFF; bd=GOLD; bw=max(2,szcfg["fw"]-1) + d.rounded_rectangle([x,y,x+pw,y+ph],radius=rad,fill=fillc,outline=bd,width=bw) + ty=y+(ph-capH)//2-int(asc*0.30) + tdraw(d,(x+padx,ty),text,f,txtc,tr) + return pw,ph + +def render(concept,sizekey,mode="png"): + c=CONCEPTS[concept]; s=SIZES[sizekey]; w,h=s["w"],s["h"] + img=crop_to(Image.open(HERO+"/"+c["hero"]).convert("RGB"),w,h,c["focal"]) + if mode=="clean": return img + vgrad(img,s["gstart"],s["gmax"]) + d=ImageDraw.Draw(img,"RGBA") + inset=s["margin"]; d.rectangle([inset,inset,w-inset-1,h-inset-1],outline=GOLD,width=s["fw"]) + if mode=="plate": return img # frame+gradient, no pill/text (for editable PDF) + # stamp pill top-left + px=inset+int(w*0.022); py=inset+int(h*0.026) + pill(img,d,px,py,c["stamp"],c["style"],s) + # address block lower-left + left=inset+int(w*0.030) + fa=F("Poppins-Bold.ttf",s["addr"]); fc=F("Poppins-Light.ttf",s["city"]); trc=s["city"]*0.16 + aasc,adesc=fa.getmetrics(); ah=aasc+adesc; cAsc,cDesc=fc.getmetrics(); chh=cAsc+cDesc + gapb=int(h*0.055) + y_city=h-inset-gapb-chh + y_rule=y_city-int(h*0.020) + y_addr=y_rule-int(h*0.012)-ah + d.text((left,y_addr),ADDR1,font=fa,fill=OFF) + rl=int(w*0.085); d.rectangle([left,y_rule,left+rl,y_rule+max(2,int(h*0.0045))],fill=GOLD) + tdraw(d,(left,y_city),ADDR2,fc,LGOLD,trc) + return img + +if __name__=="__main__": + if sys.argv[1]=="one": + render(sys.argv[2],sys.argv[3],"png").save(sys.argv[4]); print("saved",sys.argv[4]) + +from PIL import ImageChops +def grad_overlay(w,h,start,maxa): + col=Image.new("L",(1,h),0); px=col.load() + for yy in range(h): + t=(yy/h-start)/max(1e-6,(1-start)); px[0,yy]=0 if t<0 else min(255,int((t**1.3)*maxa)) + bot=col.resize((w,h)) + col2=Image.new("L",(1,h),0); p2=col2.load() + for yy in range(h): + t=1-yy/(h*0.20); p2[0,yy]=0 if t<0 else min(255,int(t*80)) + top=col2.resize((w,h)) + amax=ImageChops.lighter(bot,top) + ov=Image.new("RGBA",(w,h),(7,6,5,255)); ov.putalpha(amax) + return ov diff --git a/skills/video-editor/scripts/carousel_lib.py b/skills/video-editor/scripts/carousel_lib.py new file mode 100644 index 00000000..1e18e3af --- /dev/null +++ b/skills/video-editor/scripts/carousel_lib.py @@ -0,0 +1,118 @@ +# -*- coding: utf-8 -*- +import os, ad_lib as A +from PIL import Image, ImageDraw, ImageFont, ImageChops, ImageMath +FD=A.FD; CH=os.environ.get("VE_CHEROES","cheroes") +INK=A.INK; GOLD=A.GOLD; OFF=A.OFF; LGOLD=A.LGOLD; MUT=A.MUT; INKP=(18,16,13) +OFFW=(244,238,227); HOOKC=(238,232,222); SUBC=(236,230,220); BACKC=(200,190,172) +W,H=1080,1350; M=44; FW=4; INSET=M+26 +ADDR1="909 BAINES ST"; ADDR2="EAST PALO ALTO, CALIFORNIA" +_scr=ImageDraw.Draw(Image.new("RGB",(8,8))) +def FT(fn,s): return ImageFont.truetype(FD+fn,s) +def tw(t,fn,s,tr=0): + f=FT(fn,s); return sum(_scr.textlength(c,font=f) for c in t)+tr*max(0,len(t)-1) +def fit(t,fn,s,maxw,tr=0): + while s>20 and tw(t,fn,s,tr)>maxw: s-=2 + return s +def _ramp(w,h,start,maxa,power=1.3): + col=Image.new("L",(1,h),0); px=col.load() + for yy in range(h): + tt=(yy/h-start)/max(1e-6,1-start); px[0,yy]=0 if tt<0 else min(255,int((tt**power)*maxa)) + return col.resize((w,h)) +def _alpha(mode): + if mode=="close": + flat=Image.new("L",(W,H),120); rp=_ramp(W,H,0.20,235) + return ImageChops.invert(ImageChops.multiply(ImageChops.invert(flat),ImageChops.invert(rp))) + bot=_ramp(W,H,0.46 if mode=="cover" else 0.50,225 if mode=="cover" else 215) + col2=Image.new("L",(1,H),0); p2=col2.load() + for yy in range(H): + tt=1-yy/(H*0.22); p2[0,yy]=0 if tt<0 else min(255,int(tt*95)) + return ImageChops.lighter(bot,col2.resize((W,H))) +def grad(img,mode): + img.paste(Image.new("RGB",(W,H),(8,7,6)),(0,0),_alpha(mode)); return img +def grad_overlay(mode): + ov=Image.new("RGBA",(W,H),(8,7,6,255)); ov.putalpha(_alpha(mode)); return ov + +SLIDES=[ + dict(t="cover",hero="s01_cover.png",focal=(0.52,0.46),num=1, + hook="Soaring ceilings · granite kitchen · private patio"), + dict(t="feat",hero="s02_vault.png",focal=(0.50,0.42),num=2,label="VAULTED CEILINGS", + sub="Soaring ceilings over the main living area"), + dict(t="feat",hero="s03_living.png",focal=(0.50,0.50),num=3,label="OPEN-CONCEPT LIVING", + sub="Bright living & dining that opens to the patio"), + dict(t="feat",hero="s04_kitchen.png",focal=(0.50,0.55),num=4,label="GRANITE ISLAND KITCHEN", + sub="A center island with seating and a gas range"), + dict(t="feat",hero="s05_bed.png",focal=(0.50,0.50),num=5,label="BRIGHT BEDROOMS", + sub="Sunny secondary bedrooms with warm accents"), + dict(t="feat",hero="s06_loft.png",focal=(0.50,0.45),num=6,label="UPSTAIRS LOFT", + sub="An open loft landing overlooks the living room"), + dict(t="feat",hero="s07_primary.png",focal=(0.50,0.50),num=7,label="PRIMARY SUITE", + sub="A spacious, light-filled primary bedroom"), + dict(t="feat",hero="s08_shower.png",focal=(0.50,0.50),num=8,label="SPA-STYLE BATH", + sub="Walk-in glass shower with a rainfall head"), + dict(t="feat",hero="s09_patio.png",focal=(0.50,0.55),num=9,label="PRIVATE PATIO", + sub="A pergola-shaded patio for outdoor living"), + dict(t="close",hero="s10_close.png",focal=(0.52,0.48),num=10, + recap=["Soaring vaulted ceilings","Open-concept living & dining","Granite island kitchen", + "Spa-style walk-in shower","Private patio + attached garage"]), +] +def ops_for(spec): + ops=[]; num=spec["num"] + ctr=f"{num:02d} / 10"; cx=W-INSET-tw(ctr,"Poppins-Light.ttf",24,3) + ops.append(("text",cx,M+30,ctr,"Poppins-Light.ttf",24,LGOLD,3)) + if spec["t"]=="cover": + ops.append(("pill",INSET,M+24,"JUST LISTED","Poppins-Medium.ttf",25,"gold")) + else: + ops.append(("text",INSET,M+30,ADDR1,"Poppins-Medium.ttf",22,OFFW,3)) + if spec["t"]=="cover": + y=H-M-70; ssz=29 + ops.append(("text",INSET,y-ssz,"SWIPE TO TOUR","Poppins-Medium.ttf",ssz,GOLD,2)) + ops.append(("chev",INSET+tw("SWIPE TO TOUR","Poppins-Medium.ttf",ssz,2)+16,y-ssz+4,ssz-6,GOLD,4)) + yh=y-ssz-46; hs=fit(spec["hook"],"Poppins-Light.ttf",30,W-2*INSET) + ops.append(("text",INSET,yh-hs,spec["hook"],"Poppins-Light.ttf",hs,HOOKC,0)) + yr=yh-hs-22; ops.append(("rule",INSET,yr,96,5,GOLD)) + yc=yr-16; ops.append(("text",INSET,yc-33,ADDR2,"Poppins-Light.ttf",27,LGOLD,4)) + asz=fit(ADDR1,"Poppins-Bold.ttf",96,W-2*INSET); ya=yc-33-asz-10 + ops.append(("text",INSET,ya-asz,ADDR1,"Poppins-Bold.ttf",asz,OFF,0)) + elif spec["t"]=="feat": + ss=fit(spec["sub"],"Poppins-Light.ttf",30,W-2*INSET); ls=fit(spec["label"],"Poppins-Bold.ttf",62,W-2*INSET) + ybase=H-M-78 + ops.append(("text",INSET,ybase-ss,spec["sub"],"Poppins-Light.ttf",ss,SUBC,0)) + yl=ybase-ss-16-ls; ops.append(("text",INSET,yl-ls,spec["label"],"Poppins-Bold.ttf",ls,OFF,0)) + yr=yl-ls-20; ops.append(("rule",INSET,yr,80,5,GOLD)) + else: + ya=720; asz=fit(ADDR1,"Poppins-Bold.ttf",66,W-2*INSET) + ops.append(("text",INSET,ya,ADDR1,"Poppins-Bold.ttf",asz,OFF,0)) + ops.append(("text",INSET,ya+asz+10,ADDR2,"Poppins-Light.ttf",24,LGOLD,4)) + yr=ya+asz+50; ops.append(("rule",INSET,yr,80,5,GOLD)) + yy=yr+34 + for it in spec["recap"]: + ops.append(("dot",INSET+6,yy+18,6,GOLD)) + ops.append(("text",INSET+30,yy,it,"Poppins-Light.ttf",30,SUBC,0)); yy+=52 + yy+=14 + ops.append(("text",INSET,yy,"NOW AVAILABLE IN EAST PALO ALTO","Poppins-Medium.ttf",29,GOLD,1)); yy+=52 + ops.append(("chev",INSET+2,yy+4,18,MUT,3)) + ops.append(("text",INSET+26,yy,"Swipe back to tour again","Poppins-Light.ttf",24,BACKC,0)) + return ops +def _tdraw(d,x,y,t,fn,s,col,tr=0): + f=FT(fn,s) + for c in t: d.text((x,y),c,font=f,fill=col); x+=d.textlength(c,font=f)+tr +def _pill(img,d,x,y,text,fn,s,style): + f=FT(fn,s); asc,_=f.getmetrics(); capH=int(asc*0.70); tr=4 + twi=tw(text,fn,s,tr); padx=int(s*1.05); pady=int(s*0.72); pw=twi+2*padx; ph=capH+2*pady; rad=ph//2 + if style=="gold": d.rounded_rectangle([x,y,x+pw,y+ph],rad,fill=GOLD); tc=INK + else: d.rounded_rectangle([x,y,x+pw,y+ph],rad,fill=INKP,outline=GOLD,width=3); tc=OFF + ty=y+(ph-capH)//2-int(asc*0.30); _tdraw(d,x+padx,ty,text,fn,s,tc,tr) +def paint_pil(img,ops): + d=ImageDraw.Draw(img,"RGBA") + for op in ops: + k=op[0] + if k=="text": _,x,y,t,fn,s,col,tr=op; _tdraw(d,x,y,t,fn,s,col,tr) + elif k=="rule": _,x,y,w,h,col=op; d.rectangle([x,y,x+w,y+h],fill=col) + elif k=="dot": _,cx,cy,r,col=op; d.ellipse([cx-r,cy-r,cx+r,cy+r],fill=col) + elif k=="pill": _,x,y,t,fn,s,st=op; _pill(img,d,x,y,t,fn,s,st) + elif k=="chev": _,x,y,h,col,wt=op; d.line([(x,y),(x+h*0.55,y+h/2)],fill=col,width=wt); d.line([(x,y+h),(x+h*0.55,y+h/2)],fill=col,width=wt) +def render(i,mode="png"): + spec=SLIDES[i]; img=A.crop_to(Image.open(CH+"/"+spec["hero"]).convert("RGB"),W,H,spec["focal"]) + if mode=="clean": return img + grad(img,spec["t"]); d=ImageDraw.Draw(img,"RGBA"); d.rectangle([M,M,W-M-1,H-M-1],outline=GOLD,width=FW) + paint_pil(img,ops_for(spec)); return img diff --git a/skills/video-editor/scripts/chunk.py b/skills/video-editor/scripts/chunk.py new file mode 100644 index 00000000..6c570de6 --- /dev/null +++ b/skills/video-editor/scripts/chunk.py @@ -0,0 +1,35 @@ +import sys, subprocess +def dur(f): + o=subprocess.check_output(["ffprobe","-v","error","-show_entries","format=duration","-of","csv=p=0",f]).decode().strip() + return float(o) +# usage: chunk.py OUT FADEIN FADEOUT seg1 [T D seg]... +out=sys.argv[1]; fin=float(sys.argv[2]); fout=float(sys.argv[3]) +rest=sys.argv[4:] +segs=[rest[0]]; trans=[]; dts=[] +i=1 +while i < len(rest): + trans.append(rest[i]); dts.append(float(rest[i+1])); segs.append(rest[i+2]); i+=3 +n=len(segs); durs=[dur(s) for s in segs] +fc=[] +for i in range(n): + fc.append(f"[{i}:v]settb=AVTB,format=yuv420p,fps=30[v{i}]") +acc="[v0]"; accdur=durs[0] +for i in range(1,n): + off=accdur-dts[i-1] + lbl=f"[x{i}]" + fc.append(f"{acc}[v{i}]xfade=transition={trans[i-1]}:duration={dts[i-1]}:offset={off:.3f}{lbl}") + accdur=accdur+durs[i]-dts[i-1] + acc=lbl +fades=[] +if fin>0: fades.append(f"fade=t=in:st=0:d={fin}") +if fout>0: fades.append(f"fade=t=out:st={max(0,accdur-fout):.3f}:d={fout}") +if fades: + fc.append(f"{acc}{','.join(fades)}[outv]"); acc="[outv]" +elif n==1: + fc.append(f"{acc}null[outv]"); acc="[outv]" +inputs=[] +for s in segs: inputs+=["-i",s] +cmd=["ffmpeg","-nostdin","-v","error",*inputs,"-filter_complex",";".join(fc),"-map",acc, + "-r","30","-c:v","libx264","-preset","veryfast","-crf","18","-pix_fmt","yuv420p","-video_track_timescale","30000",out,"-y"] +subprocess.check_call(cmd) +print(f"{out} dur={accdur:.2f}s segs={n}") diff --git a/skills/video-editor/scripts/make_ad_pdfs.py b/skills/video-editor/scripts/make_ad_pdfs.py new file mode 100644 index 00000000..f9fee17f --- /dev/null +++ b/skills/video-editor/scripts/make_ad_pdfs.py @@ -0,0 +1,47 @@ +import ad_lib as A, os +from reportlab.pdfgen import canvas +from reportlab.lib.colors import HexColor +from reportlab.pdfbase import pdfmetrics +from reportlab.pdfbase.ttfonts import TTFont +from PIL import Image, ImageDraw, ImageFont +FD=A.FD; KIT=os.environ.get("VE_KIT","ad-kit") +TMP=KIT+"/.tmp"; os.makedirs(TMP,exist_ok=True) +for nm,fn in [("Poppins-Bold","Poppins-Bold.ttf"),("Poppins-Medium","Poppins-Medium.ttf"),("Poppins-Light","Poppins-Light.ttf")]: + pdfmetrics.registerFont(TTFont(nm,FD+fn)) +GOLD=HexColor("#C7A974"); INK=HexColor("#0E0C0A"); OFF=HexColor("#F4EEE3"); LGOLD=HexColor("#DEC9A2"); INKP=HexColor("#12100D") +scr=ImageDraw.Draw(Image.new("RGB",(8,8))) +def tlen(t,f,tr): return sum(scr.textlength(c,font=f) for c in t)+tr*max(0,len(t)-1) +def rtext(c,x,y,text,font,size,tr,col): + c.setFillColor(col); c.setFont(font,size) + for ch in text: + c.drawString(x,y,ch); x+=c.stringWidth(ch,font,size)+tr +order=["Exterior","Interior","Detail"] +for skey,s in A.SIZES.items(): + w,h=s["w"],s["h"]; pdf=f"{KIT}/Canva-Editable-PDF/909Baines_Ads_{skey}.pdf" + c=canvas.Canvas(pdf,pagesize=(w,h)) + for cn in order: + cc=A.CONCEPTS[cn] + bg=f"{TMP}/{cn}_{skey}_clean.png"; A.render(cn,skey,"clean").save(bg) + ov=f"{TMP}/{cn}_{skey}_grad.png"; A.grad_overlay(w,h,s["gstart"],s["gmax"]).save(ov) + c.drawImage(bg,0,0,w,h); c.drawImage(ov,0,0,w,h,mask='auto') + m=s["margin"]; c.setStrokeColor(GOLD); c.setLineWidth(s["fw"]); c.rect(m,m,w-2*m,h-2*m,fill=0,stroke=1) + fM=ImageFont.truetype(FD+"Poppins-Medium.ttf",s["stamp"]); ascM,_=fM.getmetrics(); capH=int(ascM*0.70) + tr=s["stamp"]*0.16; tw=tlen(cc["stamp"],fM,tr); padx=int(s["stamp"]*1.05); pady=int(s["stamp"]*0.72) + pw=tw+2*padx; ph=capH+2*pady; px=m+int(w*0.022); py=m+int(h*0.026); yb=h-(py+ph) + if cc["style"]=="gold": + c.setFillColor(GOLD); c.roundRect(px,yb,pw,ph,ph/2,fill=1,stroke=0); tcol=INK + else: + c.setFillColor(INKP); c.setStrokeColor(GOLD); c.setLineWidth(max(2,s["fw"]-1)); c.roundRect(px,yb,pw,ph,ph/2,fill=1,stroke=1); tcol=OFF + ty=py+(ph-capH)//2-int(ascM*0.30) + rtext(c,px+padx,h-(ty+ascM),cc["stamp"],"Poppins-Medium",s["stamp"],tr,tcol) + left=m+int(w*0.030) + fa=ImageFont.truetype(FD+"Poppins-Bold.ttf",s["addr"]); aasc,adesc=fa.getmetrics(); ah=aasc+adesc + fc=ImageFont.truetype(FD+"Poppins-Light.ttf",s["city"]); cAsc,cDesc=fc.getmetrics(); chh=cAsc+cDesc + trc=s["city"]*0.16; gapb=int(h*0.055) + y_city=h-m-gapb-chh; y_rule=y_city-int(h*0.020); y_addr=y_rule-int(h*0.012)-ah + c.setFillColor(OFF); c.setFont("Poppins-Bold",s["addr"]); c.drawString(left,h-(y_addr+aasc),A.ADDR1) + thick=max(2,int(h*0.0045)); rl=int(w*0.085) + c.setFillColor(GOLD); c.rect(left,h-(y_rule+thick),rl,thick,fill=1,stroke=0) + rtext(c,left,h-(y_city+cAsc),A.ADDR2,"Poppins-Light",s["city"],trc,LGOLD) + c.showPage() + c.save(); print("PDF",os.path.basename(pdf)) diff --git a/skills/video-editor/scripts/make_carousel_pdf.py b/skills/video-editor/scripts/make_carousel_pdf.py new file mode 100644 index 00000000..1ea075ff --- /dev/null +++ b/skills/video-editor/scripts/make_carousel_pdf.py @@ -0,0 +1,43 @@ +import carousel_lib as C, ad_lib as A, os +from reportlab.pdfgen import canvas +from reportlab.lib.colors import Color +from reportlab.pdfbase import pdfmetrics +from reportlab.pdfbase.ttfonts import TTFont +from PIL import ImageFont +FD=A.FD; W,H=C.W,C.H; OUT=os.environ.get("VE_CAROUSEL_PDF","carousel-editable.pdf") +TMP=os.environ.get("VE_TMP","./.ve_ctmp"); os.makedirs(TMP,exist_ok=True) +for nm in ["Poppins-Bold","Poppins-Medium","Poppins-Light"]: + pdfmetrics.registerFont(TTFont(nm,FD+nm+".ttf")) +def col(c): return Color(c[0]/255,c[1]/255,c[2]/255) +def asc(fn,s): return ImageFont.truetype(FD+fn,s).getmetrics()[0] +def rtext(cv,x,ybase,t,fn,s,tr,color): + cv.setFillColor(color); cv.setFont(fn[:-4],s) + for ch in t: cv.drawString(x,ybase,ch); x+=cv.stringWidth(ch,fn[:-4],s)+tr +def paint_pdf(cv,ops): + for op in ops: + k=op[0] + if k=="text": + _,x,y,t,fn,s,c,tr=op; rtext(cv,x,H-(y+asc(fn,s)),t,fn,s,tr,col(c)) + elif k=="rule": + _,x,y,w,h,c=op; cv.setFillColor(col(c)); cv.rect(x,H-(y+h),w,h,fill=1,stroke=0) + elif k=="dot": + _,cx,cy,r,c=op; cv.setFillColor(col(c)); cv.circle(cx,H-cy,r,fill=1,stroke=0) + elif k=="pill": + _,x,y,t,fn,s,st=op + a=asc(fn,s); capH=int(a*0.70); tr=4 + twi=C.tw(t,fn,s,tr); padx=int(s*1.05); pady=int(s*0.72); pw=twi+2*padx; ph=capH+2*pady; rad=ph/2; yb=H-(y+ph) + if st=="gold": cv.setFillColor(col(C.GOLD)); cv.roundRect(x,yb,pw,ph,rad,fill=1,stroke=0); tc=col(C.INK) + else: + cv.setFillColor(col(C.INKP)); cv.setStrokeColor(col(C.GOLD)); cv.setLineWidth(3); cv.roundRect(x,yb,pw,ph,rad,fill=1,stroke=1); tc=col(C.OFF) + ty=y+(ph-capH)//2-int(a*0.30); rtext(cv,x+padx,H-(ty+a),t,fn,s,tr,tc) + elif k=="chev": + _,x,y,h,c,wt=op; cv.setStrokeColor(col(c)); cv.setLineWidth(wt); cv.setLineCap(1) + cv.line(x,H-y,x+h*0.55,H-(y+h/2)); cv.line(x,H-(y+h),x+h*0.55,H-(y+h/2)) +cv=canvas.Canvas(OUT,pagesize=(W,H)) +for i,spec in enumerate(C.SLIDES): + bg=f"{TMP}/{i}_clean.png"; C.render(i,"clean").save(bg) + ov=f"{TMP}/{i}_grad.png"; C.grad_overlay(spec["t"]).save(ov) + cv.drawImage(bg,0,0,W,H); cv.drawImage(ov,0,0,W,H,mask='auto') + cv.setStrokeColor(col(C.GOLD)); cv.setLineWidth(C.FW); cv.rect(C.M,C.M,W-2*C.M,H-2*C.M,fill=0,stroke=1) + paint_pdf(cv,C.ops_for(spec)); cv.showPage() +cv.save(); print("PDF ->",os.path.basename(OUT)) diff --git a/skills/video-editor/scripts/video_lib.sh b/skills/video-editor/scripts/video_lib.sh new file mode 100644 index 00000000..dbca601b --- /dev/null +++ b/skills/video-editor/scripts/video_lib.sh @@ -0,0 +1,48 @@ +#!/bin/bash +SRC="${VE_SRC:?set VE_SRC to the folder of raw property clips}" +B="${VE_OUT:-./ve_build}" +FONT="${VE_FONT:-/usr/share/fonts/truetype/dejavu/DejaVuSans-Bold.ttf}" +GRADE_STD="eq=contrast=1.06:saturation=1.10:brightness=0.004:gamma=0.99,unsharp=3:3:0.4:3:3:0.0" +GRADE_HERO="eq=contrast=1.05:saturation=1.00,colorbalance=rm=-0.04:rh=-0.03:bm=0.02,unsharp=3:3:0.4:3:3:0.0" +mkdir -p "$B" +ENC="-an -r 30 -c:v libx264 -preset veryfast -crf 17 -pix_fmt yuv420p -video_track_timescale 30000" + +# fade alpha expr, local segment time, appear A disappear B with 0.4s ramps +af(){ local A=$1 Bt=$2; echo "if(lt(t,$A),0,if(lt(t,$A+0.4),(t-$A)/0.4,if(lt(t,$Bt-0.4),1,if(lt(t,$Bt),($Bt-t)/0.4,0))))"; } + +# feature label bottom-left: $1 text $2 appear $3 disappear (for 1920x1080) +label16(){ local T="$1" A="$2" D="$3"; local al=$(af "$A" "$D") + echo "drawbox=x=82:y=h-148:w=54:h=7:color=0xC79A33:t=fill:enable='between(t,$A,$D)',drawtext=fontfile=$FONT:text='$T':x=82:y=h-128:fontsize=44:fontcolor=white:shadowcolor=black@0.6:shadowx=2:shadowy=2:alpha='$al'"; } + +# centered title 1920x1080: $1 big $2 small $3 appear $4 disappear +title16(){ local T1="$1" T2="$2" A="$3" D="$4"; local al=$(af "$A" "$D") + echo "drawtext=fontfile=$FONT:text='$T1':x=(w-text_w)/2:y=h/2-40:fontsize=78:fontcolor=white:shadowcolor=black@0.55:shadowx=2:shadowy=3:alpha='$al',drawbox=x=(w-150)/2:y=h/2+58:w=150:h=6:color=0xC79A33:t=fill:enable='between(t,$A,$D)',drawtext=fontfile=$FONT:text='$T2':x=(w-text_w)/2:y=h/2+78:fontsize=34:fontcolor=0xEAEAEA:alpha='$al'"; } + +# reel caption centered upper third 1080x1920: $1 text $2 appear $3 disappear +capR(){ local T="$1" A="$2" D="$3"; local al=$(af "$A" "$D") + echo "drawtext=fontfile=$FONT:text='$T':x=(w-text_w)/2:y=560:fontsize=66:fontcolor=white:shadowcolor=black@0.7:shadowx=2:shadowy=2:box=1:boxcolor=black@0.18:boxborderw=22:alpha='$al',drawbox=x=(w-120)/2:y=648:w=120:h=7:color=0xC79A33:t=fill:enable='between(t,$A,$D)'"; } + +# 16:9 segment: SRC SS DUR OUT GRADE POST +seg16(){ local s="$1" ss="$2" du="$3" out="$4" gr="${5:-$GRADE_STD}" post="$6" + local vf="scale=1920:1080:force_original_aspect_ratio=increase,crop=1920:1080,$gr" + [ -n "$post" ] && vf="$vf,$post" + vf="$vf,fps=30,format=yuv420p,setsar=1" + ffmpeg -nostdin -v error -ss "$ss" -t "$du" -i "$SRC/$s" -vf "$vf" $ENC "$B/$out" -y; } + +# 9:16 center-crop: SRC SS DUR OUT GRADE POST XOFF +seg916c(){ local s="$1" ss="$2" du="$3" out="$4" gr="${5:-$GRADE_STD}" post="$6" xo="${7:-0}" + local vf="crop=ih*9/16:ih:(iw-ih*9/16)/2+($xo):0,scale=1080:1920,$gr" + [ -n "$post" ] && vf="$vf,$post" + vf="$vf,fps=30,format=yuv420p,setsar=1" + ffmpeg -nostdin -v error -ss "$ss" -t "$du" -i "$SRC/$s" -vf "$vf" $ENC "$B/$out" -y; } + +# 9:16 blurred-pad: SRC SS DUR OUT GRADE POST +seg916p(){ local s="$1" ss="$2" du="$3" out="$4" gr="${5:-$GRADE_STD}" post="$6" + local chain="[0:v]$gr,split=2[bg][fg];[bg]scale=1080:1920:force_original_aspect_ratio=increase,crop=1080:1920,gblur=sigma=24:steps=2[b];[fg]scale=1080:-2[f];[b][f]overlay=(W-w)/2:(H-h)/2[o]" + if [ -n "$post" ]; then chain="$chain;[o]$post,fps=30,format=yuv420p,setsar=1[v]"; else chain="$chain;[o]fps=30,format=yuv420p,setsar=1[v]"; fi + ffmpeg -nostdin -v error -ss "$ss" -t "$du" -i "$SRC/$s" -filter_complex "$chain" -map "[v]" $ENC "$B/$out" -y; } + +# xfade two seg files: A B STYLE DUR OUT +xf(){ local a="$1" b="$2" st="$3" d="$4" out="$5" + local da=$(ffprobe -v error -show_entries format=duration -of csv=p=0 "$B/$a"); local off=$(echo "$da-$d"|bc) + ffmpeg -nostdin -v error -i "$B/$a" -i "$B/$b" -filter_complex "[0:v]settb=AVTB[x];[1:v]settb=AVTB[y];[x][y]xfade=transition=$st:duration=$d:offset=$off,format=yuv420p[v]" -map "[v]" -r 30 -c:v libx264 -preset veryfast -crf 18 -pix_fmt yuv420p -video_track_timescale 30000 "$B/$out" -y; } diff --git a/skills/video-to-obsidian/SKILL.md b/skills/video-to-obsidian/SKILL.md new file mode 100644 index 00000000..d4fd94b8 --- /dev/null +++ b/skills/video-to-obsidian/SKILL.md @@ -0,0 +1,89 @@ +--- +name: video-to-obsidian +description: "Universal video-to-Obsidian logger for Graeham Watts. Takes any video URL — Instagram Reel, YouTube, TikTok, Vimeo, anything yt-dlp supports — transcribes it, auto-categorizes it, and writes a structured markdown note to the Obsidian vault at Documents/Obsidian/Instagram Saves/ with full frontmatter. Source URL is always preserved — the user needs to click back to see the actual video treatment. Use this skill ANY time the user wants to save a video reference to their swipe-file vault, log a competitor video, archive an inspirational reel, add a video to their content library, build out their hook library, or put a video in Obsidian. Also called by instagram-competitor-scraper when scrape results need persisting, and by content-creation-engine when source videos should be archived. Triggers: log this video to Obsidian, save this reel to my vault, add this YouTube to Obsidian, put this in my swipe file, archive this reel, add to hook library, or pasting a video URL with save or log context." +--- + +# Video to Obsidian + +> **One job:** take a video URL, transcribe it, categorize it, and write a clean markdown note to the right folder in `Documents/Obsidian/Instagram Saves/`. Source URL always preserved. + +## Why this exists + +You're building a content intelligence layer. Words from videos need to live as searchable, queryable, AI-readable data in one place — Obsidian. But video is a visual medium, so the URL has to come along for the ride. Without the URL, the note is dead. + +This skill is the **destination layer** for everything visual you save. Manual ad-hoc saves, scraper output, even YouTube videos you watch for research — all flow through here and land in the same schema. + +## What it accepts + +Any URL supported by `yt-dlp` (1,800+ platforms). Most common: Instagram Reels/posts, YouTube videos/Shorts, TikTok, Vimeo, Twitter/X video, Facebook video, LinkedIn video, direct video file URLs. + +## How to invoke + +``` +log this to Obsidian: https://www.instagram.com/reels/DYSUqcsuGX9/ +save this short to my vault: https://youtube.com/shorts/abc123 +add to hook library: https://tiktok.com/@user/video/789 +``` + +CLI form: + +```bash +python3 scripts/log_to_vault.py "https://www.instagram.com/reels/ABC/" +python3 scripts/log_to_vault.py "URL" --folder "Hook Library" --my-use steal-hook +python3 scripts/log_to_vault.py "URL" --metadata-json '{"engagement":{"views":12000}}' +python3 scripts/log_to_vault.py "URL" --transcript-text "already have it" # skip transcription +``` + +## What it does + +1. **Validates URL** — must be present +2. **Calls `video-transcriber`** (or uses pre-supplied --transcript-text) +3. **Pulls metadata** via yt-dlp (title, duration, creator, post date) +4. **Merges extra metadata** from --metadata-json (engagement stats from scraper) +5. **Auto-categorizes** into right folder via heuristics +6. **Auto-tags** content_type + hook_pattern + topic_tags +7. **Writes the note** to `Obsidian/Instagram Saves//-.md` + +## Vault path + +Auto-detected from `C:\Users\Graeham Watts\Documents\Obsidian\Instagram Saves\` or `/sessions/.../mnt/Obsidian/Instagram Saves/` (sandbox). Override with `--vault-root`. + +## Folder routing + +| Folder | Trigger | +|---|---| +| `AI & Tech Tutorials/` | AI/Claude/MCP/automation keywords in transcript or caption | +| `Real Estate Content/` | Real estate / Bay Area / Peninsula keywords | +| `How-To Videos/` | 'how to' / 'step 1' / 'tutorial' in opening | +| `Hook Library/` | --my-use steal-hook OR --folder Hook Library | +| `Examples to Clone/` | --my-use full-clone | +| `Style References/` | --my-use style-ref | +| `_Inbox/` | Default fallback | + +Categorization is a starting guess. Review weekly, move if needed. + +## Frontmatter schema + +Every note: url (required), source, creator, creator_followers, post_type, post_date, saved_date, duration_sec, content_type[], hook_pattern[], topic_tags[], my_use[], saved_for[], engagement{views,likes,comments,saves,engagement_rate}, status, transcript_available, discovered_via. + +## Auto-tagging + +**content_type:** how-to (keyword match), ai-workflow, talking-head (short + first-person), walkthrough (sequence words), comparison (vs/versus), list (top N pattern). + +**hook_pattern:** pattern-interrupt (negation at start), contrarian ('most people'/'the truth'), curiosity-gap ('the secret'/'nobody talks'), question-hook (ends with ?), direct-promise ('here's how'). + +**topic_tags:** extracted from caption hashtags + transcript keyword map. + +## Idempotency + +If URL exists in vault, default = skip with stderr message. `--update` updates engagement only. `--force` overwrites. + +## Integration + +- `instagram-competitor-scraper` pipes results via `--metadata-json` +- `content-creation-engine` archives source videos identified during ideation +- `cinematic-hooks` reads the Hook Library folder this populates + +## Why URL preservation is non-negotiable + +Video is a visual medium. Transcripts lose cuts, on-screen text, visual style, energy, sound design. Without the URL, the note is a partial copy. With the URL, the note is a queryable launchpad back to the original. This is the one thing the skill must never get wrong. diff --git a/skills/video-to-obsidian/scripts/log_to_vault.py b/skills/video-to-obsidian/scripts/log_to_vault.py new file mode 100644 index 00000000..17ebda3c --- /dev/null +++ b/skills/video-to-obsidian/scripts/log_to_vault.py @@ -0,0 +1,346 @@ +#!/usr/bin/env python3 +""" +log_to_vault.py — Video URL -> Obsidian vault note for Graeham Watts. + +Takes ANY video URL (Instagram, YouTube, Shorts, TikTok, etc.), transcribes it +via video-transcriber, auto-categorizes it, and writes a clean markdown note to +the Obsidian vault at Documents/Obsidian/Instagram Saves/. + +The source URL is ALWAYS preserved in frontmatter. Non-negotiable. + +Usage: + python3 log_to_vault.py "https://www.instagram.com/reels/ABC/" + python3 log_to_vault.py "URL" --folder "Hook Library" --my-use steal-hook + python3 log_to_vault.py "URL" --metadata-json '{"engagement":{"views":12000}}' + python3 log_to_vault.py "URL" --update # update engagement stats only + python3 log_to_vault.py "URL" --force # overwrite even if exists + python3 log_to_vault.py "URL" --dry-run # show what would be written + python3 log_to_vault.py "URL" --transcript-text "..." # skip transcription +""" + +import argparse +import json +import os +import re +import subprocess +import sys +from datetime import datetime, timezone +from pathlib import Path + + +def find_vault_root(): + candidates = [ + Path("/sessions/gifted-elegant-ritchie/mnt/Obsidian/Instagram Saves"), + Path(r"C:\Users\Graeham Watts\Documents\Obsidian\Instagram Saves"), + Path.home() / "Documents" / "Obsidian" / "Instagram Saves", + ] + for c in candidates: + if c.exists(): + return c + return candidates[0] + + +VAULT_ROOT = find_vault_root() +SKILLS_ROOT = Path(__file__).resolve().parents[3] +TRANSCRIBER_SCRIPT = SKILLS_ROOT / "skills" / "video-transcriber" / "scripts" / "transcribe.py" + +FOLDERS = { + "ai": "AI & Tech Tutorials", + "re": "Real Estate Content", + "howto": "How-To Videos", + "hook": "Hook Library", + "clone": "Examples to Clone", + "style": "Style References", + "inbox": "_Inbox", + "misc": "Misc", +} + + +def detect_source(url): + u = url.lower() + if "instagram.com" in u: return "instagram" + if "youtube.com" in u or "youtu.be" in u: return "youtube" + if "tiktok.com" in u: return "tiktok" + if "vimeo.com" in u: return "vimeo" + if "twitter.com" in u or "x.com" in u: return "twitter" + if "facebook.com" in u or "fb.watch" in u: return "facebook" + if "linkedin.com" in u: return "linkedin" + return "other" + + +def detect_post_type(url, source, duration_sec=0): + u = url.lower() + if source == "instagram": + if "/reel" in u: return "reel" + if "/p/" in u: return "carousel" if duration_sec == 0 else "reel" + return "post" + if source == "youtube": + if "/shorts/" in u: return "short" + return "video" + return "video" + + +def transcribe_url(url): + if not TRANSCRIBER_SCRIPT.exists(): + return {"error": f"video-transcriber not found at {TRANSCRIBER_SCRIPT}"} + cmd = [sys.executable, str(TRANSCRIBER_SCRIPT), url, "--json"] + result = subprocess.run(cmd, capture_output=True, text=True, timeout=600) + if result.returncode != 0: + return {"error": result.stderr[:500]} + try: + return json.loads(result.stdout) + except json.JSONDecodeError: + return {"error": "Transcriber returned non-JSON output"} + + +AI_KEYWORDS = ["ai", "claude", "chatgpt", "gpt", "openai", "anthropic", "llm", + "n8n", "mcp", "automation", "codex", "prompt", "agent", + "machine learning", "neural"] +RE_KEYWORDS = ["real estate", "realtor", "listing", "mls", "home buying", + "home buyer", "mortgage", "bay area", "peninsula", "east palo alto", + "redwood city", "menlo park", "palo alto", "san mateo", + "open house", "broker", "escrow"] + + +def folder_for(transcript_text, caption, source, explicit_folder, my_use): + if explicit_folder: return explicit_folder + if "hook-only" in my_use or "steal-hook" in my_use: return FOLDERS["hook"] + if "full-clone" in my_use: return FOLDERS["clone"] + if "style-ref" in my_use: return FOLDERS["style"] + blob = (transcript_text + " " + caption).lower() + if any(k in blob for k in AI_KEYWORDS): return FOLDERS["ai"] + if any(k in blob for k in RE_KEYWORDS): return FOLDERS["re"] + first_line = transcript_text.strip().split(".")[0].lower() if transcript_text else "" + if any(s in first_line for s in ["how to", "here's how", "step 1", "tutorial", "the way to"]): + return FOLDERS["howto"] + return FOLDERS["inbox"] + + +def auto_content_types(transcript, caption, duration_sec): + tags = [] + blob = (transcript + " " + caption).lower() + first_line = transcript.strip().split(".")[0].lower() if transcript else "" + if any(s in first_line for s in ["how to", "here's how", "step 1", "tutorial"]): tags.append("how-to") + if any(k in blob for k in AI_KEYWORDS): tags.append("ai-workflow") + if duration_sec and duration_sec < 90 and re.search(r"\b(i|my|let me|i'm|i'll)\b", first_line): + tags.append("talking-head") + if sum(blob.count(w) for w in ["first", "then", "next", "finally"]) >= 3: tags.append("walkthrough") + if any(k in blob for k in [" vs ", " versus ", "compared to", "better than"]): tags.append("comparison") + if re.search(r"\btop \d+\b|\b\d+ ways\b|\b\d+ tips\b|\bbest \d+\b", blob): tags.append("list") + return sorted(set(tags)) + + +def auto_hook_patterns(transcript): + if not transcript: return [] + first = transcript.strip().split(".")[0].lower() + tags = [] + if re.match(r"^(stop|don't|never|wrong|nobody|forget)\b", first): tags.append("pattern-interrupt") + if any(p in first for p in ["most people think", "everyone says", "the truth is", + "might be the most pointless", "this is wrong"]): + tags.append("contrarian") + if any(p in first for p in ["you won't believe", "the secret", "nobody talks about", + "what they don't tell you", "the truth about"]): + tags.append("curiosity-gap") + if first.endswith("?"): tags.append("question-hook") + if re.match(r"^here'?s how\b", first): tags.append("direct-promise") + return sorted(set(tags)) + + +def auto_topic_tags(caption, transcript): + tags = set() + for m in re.findall(r"#(\w+)", caption or ""): tags.add(m.lower()) + blob = (transcript or "").lower() + keyword_map = { + "ai": ["ai", "artificial intelligence"], + "claude-code": ["claude code"], + "real-estate": ["real estate", "realtor"], + "bay-area": ["bay area", "peninsula"], + "content": ["content", "post", "reel"], + "automation": ["automation", "automate", "workflow"], + "notion": ["notion"], + "obsidian": ["obsidian"], + } + for tag, keywords in keyword_map.items(): + if any(k in blob for k in keywords): tags.add(tag) + return sorted(tags) + + +def slugify(text, max_len=60): + s = re.sub(r"[^a-z0-9]+", "-", (text or "").lower()).strip("-") + return s[:max_len] or "untitled" + + +def render_note(data): + fm = { + "url": data["url"], "source": data["source"], + "creator": data.get("creator") or "@unknown", + "creator_followers": data.get("creator_followers", 0), + "post_type": data["post_type"], "post_date": data.get("post_date") or "", + "saved_date": data["saved_date"], "duration_sec": data.get("duration_sec", 0), + "content_type": data.get("content_type", []), + "hook_pattern": data.get("hook_pattern", []), + "topic_tags": data.get("topic_tags", []), + "my_use": data.get("my_use", ["reference-only"]), + "saved_for": data.get("saved_for", []), + "engagement": data.get("engagement", {"views": 0, "likes": 0, "comments": 0, "saves": None, "engagement_rate": 0.0}), + "status": data.get("status", "unprocessed"), + "transcript_available": data.get("transcript_available", True), + "discovered_via": data.get("discovered_via", "manual"), + } + + def yaml_value(v): + if isinstance(v, list): + return "[" + ", ".join(yaml_value(x) for x in v) + "]" + if isinstance(v, dict): + return "\n " + "\n ".join(f"{k}: {yaml_value(val)}" for k, val in v.items()) + if isinstance(v, str): + return f'"{v}"' if any(c in v for c in ": #@&*?|>%{}[],") else v + if v is None: return "null" + return str(v) + + fm_lines = ["---"] + for k, v in fm.items(): + fm_lines.append(f"{k}: {yaml_value(v)}") + fm_lines.append("---") + fm_block = "\n".join(fm_lines) + + transcript = data.get("transcript", "") + first_sentence = transcript.strip().split(".")[0].strip() + "." if transcript else "" + title = data.get("title") or f"{fm['creator']} — {fm['post_date'] or 'undated'}" + platform_name = data["source"].title() + + body = f""" + +# {title} + +## Hook (first 3 seconds) + +> {first_sentence} + +## Why saved + +{data.get('why_saved') or f"Logged via {fm['discovered_via']}"} + +## Transcript + +{transcript if transcript else '_(Transcription failed — see source URL.)_'} + +## Visual notes + +_(Add your notes about cuts, captions, on-screen text, style — transcripts don't capture this.)_ + +## Action + +- [ ] Use this for: ____ +- [ ] Pair with skill: ____ +- [ ] Output target: ____ + +--- + +**Source:** [Watch on {platform_name}]({fm['url']}) +""" + return fm_block + body + + +def find_existing_note(vault_root, url): + if not vault_root.exists(): return None + for p in vault_root.rglob("*.md"): + try: + content = p.read_text(encoding="utf-8", errors="ignore") + except Exception: + continue + m = re.search(r"^url:\s*(\S+)", content, re.MULTILINE) + if m and m.group(1).strip().strip('"') == url: + return p + return None + + +def main(): + parser = argparse.ArgumentParser(description="Log a video URL to Obsidian vault") + parser.add_argument("url") + parser.add_argument("--folder", default=None) + parser.add_argument("--my-use", default="reference-only") + parser.add_argument("--saved-for", default="") + parser.add_argument("--why", default="") + parser.add_argument("--metadata-json", default="") + parser.add_argument("--update", action="store_true") + parser.add_argument("--force", action="store_true") + parser.add_argument("--dry-run", action="store_true") + parser.add_argument("--vault-root", default=str(VAULT_ROOT)) + parser.add_argument("--transcript-text", default=None) + args = parser.parse_args() + + if not args.url: + print("ERROR: URL is required", file=sys.stderr); sys.exit(1) + + vault_root = Path(args.vault_root) + existing = find_existing_note(vault_root, args.url) + if existing and not (args.update or args.force or args.dry_run): + print(f"[skip] URL already in vault: {existing}", file=sys.stderr) + print(str(existing)); return + + extra = {} + if args.metadata_json: + try: extra = json.loads(args.metadata_json) + except json.JSONDecodeError as e: + print(f"ERROR: --metadata-json is not valid JSON: {e}", file=sys.stderr); sys.exit(1) + + if args.transcript_text: + print(f"[transcribe] using pre-supplied transcript ({len(args.transcript_text)} chars)", file=sys.stderr) + t = {"transcript_plain": args.transcript_text} + else: + print(f"[transcribe] {args.url}", file=sys.stderr) + t = transcribe_url(args.url) + + transcript_text = t.get("transcript_plain", "") if "error" not in t else "" + title = t.get("title") or extra.get("title") + duration = t.get("duration_sec") or extra.get("duration_sec") or 0 + creator = extra.get("creator") or (t.get("uploader") and f"@{t['uploader']}") or "@unknown" + post_date = extra.get("post_date") or (t.get("upload_date") and f"{t['upload_date'][:4]}-{t['upload_date'][4:6]}-{t['upload_date'][6:8]}") + caption = extra.get("caption", "") + + source = detect_source(args.url) + post_type = detect_post_type(args.url, source, duration) + my_use = [s.strip() for s in args.my_use.split(",") if s.strip()] + saved_for = [s.strip() for s in args.saved_for.split(",") if s.strip()] + + content_type = auto_content_types(transcript_text, caption, duration) + hook_pattern = auto_hook_patterns(transcript_text) + topic_tags = auto_topic_tags(caption, transcript_text) + + folder_name = folder_for(transcript_text, caption, source, args.folder, my_use) + target_dir = vault_root / folder_name + target_dir.mkdir(parents=True, exist_ok=True) + + data = { + "url": args.url, "source": source, "creator": creator, + "creator_followers": extra.get("creator_followers", 0), + "post_type": post_type, "post_date": post_date, + "saved_date": datetime.now(timezone.utc).strftime("%Y-%m-%d"), + "duration_sec": duration, "content_type": content_type, + "hook_pattern": hook_pattern, "topic_tags": topic_tags, + "my_use": my_use, "saved_for": saved_for, + "engagement": extra.get("engagement", {"views": 0, "likes": 0, "comments": 0, "saves": None, "engagement_rate": 0.0}), + "status": "unprocessed", "transcript_available": bool(transcript_text), + "discovered_via": extra.get("discovered_via", "manual"), + "transcript": transcript_text, "title": title, "why_saved": args.why, + } + + date_part = data["saved_date"] + slug_part = slugify(title or creator.lstrip("@") or "video") + filename = f"{date_part}-{slug_part}.md" + + target_path = existing if (existing and args.update) else (target_dir / filename) + note = render_note(data) + + if args.dry_run: + print(f"[dry-run] Would write to: {target_path}") + print(note); return + + target_path.write_text(note, encoding="utf-8") + print(f"[written] {target_path}", file=sys.stderr) + print(str(target_path)) + + +if __name__ == "__main__": + main() diff --git a/skills/video-transcriber/SKILL.md b/skills/video-transcriber/SKILL.md new file mode 100644 index 00000000..6106239f --- /dev/null +++ b/skills/video-transcriber/SKILL.md @@ -0,0 +1,244 @@ +--- +name: video-transcriber +description: "Universal video-to-text transcriber for Graeham's team (Peter, Ellie, John, Adrian). Trigger on: transcribe, transcript, get the transcript, video to text, captions, subtitles, what does this video say, YouTube/Reel/Short/TikTok transcript, transcribe this file, transcribe this video I uploaded." +--- + +# Video Transcriber + +> **One job, one skill.** Hand it a video — by URL or by file — get the transcript back. That's it. + +This skill exists because Peter, Ellie, John, and Adrian shouldn't have to remember which Python script lives where or which Apify actor handles which platform. They drop in a URL or a file, and the right thing happens automatically. + +## Who uses this and how + +**Peter and Ellie** (video editors): use this to pull transcripts of: +- Competitor videos they're studying for shot ideas +- Reference videos Graeham sends them as "make ours like this" +- Long-form interviews where they need to find specific quotes for cuts +- Client testimonial videos that need to be transcribed for captions +- Local screen recordings or Zoom exports sitting in Downloads + +**John** (Blog Track): use this to: +- Convert Graeham's recorded YouTube videos into blog-post source material +- Pull transcripts of industry videos and webinars for cite-ready statistics + +**Adrian** (Client Care): use this to: +- Transcribe client video messages for record-keeping +- Convert market-update videos into text summaries for clients + +**Graeham**: invoke directly when prepping content or reviewing reference material. Frequently uploads webinar or Zoom recordings he's just attended. + +## How to invoke + +Any of these work: + +1. **Just paste the URL**, nothing else: + ``` + https://www.youtube.com/watch?v=PYMsmSx8Tyw + ``` + +2. **Paste the URL with a verb**: + ``` + transcribe this: https://www.facebook.com/.../videos/123456789 + ``` + +3. **Upload a local video file** in Cowork or paste a local path: + ``` + transcribe this video + [uploaded file: webinar-recording.mp4] + ``` + ``` + transcribe C:\Users\Graeham Watts\Downloads\zoom-call.mp4 + ``` + +4. **Multiple inputs at once** (processed in turn — URLs and files can be mixed): + ``` + transcribe all three: + https://www.youtube.com/watch?v=AAA + C:\path\to\local.mp4 + https://www.instagram.com/reel/BBB/ + ``` + +The skill auto-detects URL vs local path and routes to the right backend. + +## Decision tree — when to use which path + +``` +INPUT +│ +├── Local file path (or uploaded file) ──────────────→ PATH B (Windows local faster-whisper) +│ +└── URL + ├── YouTube or other platform with captions + │ └── Try PATH A (caption pull) first + │ ├── Captions exist → return transcript ✓ + │ └── No captions → fall through to PATH B + │ + └── Any other URL (Instagram, TikTok, Facebook, etc.) + └── PATH B (Windows downloads via yt-dlp, then faster-whisper) +``` + +## PATH A — Caption pull (free, instant, ~1–3 sec) + +Runs in the Cowork sandbox. Works for any URL where the platform exposes captions (almost all YouTube videos, some Vimeo, some others). + +```bash +python3 scripts/transcribe.py "" --prefer-captions +``` + +Returns in ~1–3 seconds. Costs $0. This is always the first try for URL inputs. + +## PATH B — Windows local faster-whisper (free, ~5–15 min for an hour-long video) + +This is the workhorse for everything caption pull can't handle, and the only path for local files. + +**Why local Windows, not sandbox:** The Cowork sandbox only has ~1.4 GB free disk. Installing openai-whisper or even faster-whisper requires multiple GB of dependencies (PyTorch, CUDA libs). We tried — it doesn't fit. Graeham's Windows machine has faster-whisper installed locally and ffmpeg available, so all real transcription work runs there. + +### How Claude drives PATH B + +Claude writes the right command for the situation, then asks the user to paste it into PowerShell. We can't drive the terminal directly because Windows Terminal is granted at tier "click" (visible + clickable, but typing is blocked by security policy). + +The script is at `scripts/transcribe_windows.py` and takes either a file path or a URL as its single argument: + +```powershell +python "\scripts\transcribe_windows.py" "C:\path\to\video.mp4" +python "\scripts\transcribe_windows.py" "https://www.youtube.com/watch?v=..." +python "\scripts\transcribe_windows.py" "C:\path\to\video.mp4" --model small.en --timestamps +``` + +The script: +- Detects URL vs local file +- For URLs: downloads audio via yt-dlp first (locally, fast) +- Loads faster-whisper with int8 quantization (CPU) +- Streams progress every ~20 segments so the user knows it's working +- Writes `{slug}_transcript.txt` and (if `--timestamps`) `{slug}_transcript_timestamped.txt` next to the source video (or to `--output-dir` if specified) + +### Model size guidance + +| Model | Speed (1hr audio, CPU) | Best for | +|---|---|---| +| `base.en` (default) | ~5–15 min | Default — fast, good enough for most speech | +| `small.en` | ~15–30 min | Better proper-noun accuracy; webinars with jargon | +| `medium.en` | ~30–60 min | Reference-quality; client testimonials going to print | + +Tell the user the tradeoff if accuracy matters more than time. Don't silently bump the model — they're waiting on the result. + +## Output format + +By default, returns clean prose: + +``` +Transcript: webinar-recording.mp4 +Duration: 1:08:26 +Language: en +Model: faster-whisper base.en (int8) +====================================================================== + +The spring housing market was supposed to be the big comeback season for 2026. Instead, a lot of markets are slowing down fast. Earlier this year, most people expected mortgage rates to ease... +``` + +If the user asks for timestamps: + +``` +[0:00:00] The spring housing market was supposed to be the big comeback +[0:00:08] Instead, a lot of markets are slowing down fast. +[0:00:13] Earlier this year, most people expected mortgage rates to ease +... +``` + +## Optional flags the user can request (spoken-language) + +- **"with timestamps"** — include `[MM:SS]` markers per segment +- **"better accuracy"** / **"use a bigger model"** → bump to `small.en` or `medium.en` +- **"summarize after"** — after producing the transcript, Claude generates a 3-bullet summary +- **"save to my Documents"** / **"save next to the video"** — choose output location + +## Workflow integration + +This skill is standalone. It does not require any other skill to function. + +Common follow-ons the user may request after a transcript: + +- **"and turn it into a blog post"** → hand transcript to `content-creation-engine` +- **"and find the best 30-second clip"** → scan for the highest-impact segment for a Short/Reel cutdown +- **"and pull cite-ready stats"** → scan for date-anchored numerical claims for AEO blog content +- **"and watch it too"** → fire `video-watcher` in parallel for full A+V breakdown + +## Failure handling + +| Failure | What the skill does | +|---|---| +| Local file path doesn't exist | Report the bad path. Ask if they meant a different file or want to re-upload. | +| URL not recognized by yt-dlp | Report the platform name. Ask Graeham to confirm an alternate path (manual download, Apify actor). | +| Video is private or restricted | Report the access error verbatim. Suggest verifying the URL is publicly viewable. | +| `pip install faster-whisper` fails on user's machine | Most common cause: very new Python version (3.14+) without wheels yet. Tell user to try `pip install faster-whisper --pre` or fall back to Python 3.12 in a venv. | +| Path contains `\U` or `\N` in a non-raw Python string | This actually happened. Always wrap Windows paths in raw strings (`r"..."`) or use forward slashes. Never put Windows paths in a docstring without escaping. | +| Video is very long (>60 min) | Tell the user the est. transcription time before kicking off, so they don't think it's stuck. | +| User has Python but no `faster-whisper` | Walk them through `pip install faster-whisper` first, then `python scripts/transcribe_windows.py ...` | + +## Setup requirements + +**On the user's Windows machine (one-time):** + +```powershell +pip install faster-whisper +``` + +ffmpeg must be on PATH. Graeham's lives at `C:\Users\Graeham Watts\Documents\Skills LLMS\Claude\ffmpegvideoprocessingengine\bin\` — the script adds this to PATH automatically. + +**No API keys required.** Everything runs locally and free. + +**Optional**: `OPENAI_API_KEY` if you want to use the OpenAI Whisper API (~10x faster, ~$0.006/min). Not currently wired into the Windows script — would need to be added if Graeham wants that path for super long videos. + +## Future agentic enhancement: watch-folder workflow + +The Windows script can be run from a "drop folder" pattern for fully hands-off transcription: + +1. Create `C:\Users\Graeham Watts\Documents\Transcribe-Inbox\` +2. Create `C:\Users\Graeham Watts\Documents\Transcribe-Done\` +3. PowerShell script (saved separately, not in this skill) polls inbox every 60 sec, transcribes anything new, moves source to Done folder and transcript next to it. +4. Wire to Windows Task Scheduler to start on login. + +This is NOT part of this skill yet — it's a separate setup. If Graeham asks for "drop folder transcription" or "agentic transcription," build that as a separate task. + +## Example: end-to-end run (PATH B, local file) + +**User uploads:** `webinar-recording.mp4` (68 min, 2.6 GB) + +**Skill flow:** +1. Detects local file path, no URL +2. Skips PATH A entirely (no captions on a local file) +3. Claude writes the transcribe command for the user to paste: + ```powershell + python "C:\Users\Graeham Watts\Documents\Skills LLMS\Claude\Skills\skills\video-transcriber\scripts\transcribe_windows.py" "C:\Users\...\webinar-recording.mp4" + ``` +4. User pastes it, faster-whisper runs (~5–15 min) +5. Two text files appear next to the video: plain + timestamped (if requested) +6. Claude reads them, summarizes findings, suggests follow-ons + +Total: ~10 minutes of compute, ~30 seconds of user time, $0. + +## Companion skill: video-watcher + +This skill captures **what was SAID** in a video. Its companion `video-watcher` captures **what was SHOWN** (frame-by-frame AI vision analysis — shot list, on-screen text catalog, production style fingerprint, Replicate-This Brief). + +They're standalone but compose naturally: + +- **"transcribe this video: [URL or file]"** → only video-transcriber fires (cheap, fast, words only) +- **"watch this video: [URL or file]"** → only video-watcher fires (vision analysis, costs API tokens) +- **"watch and transcribe"** / **"full breakdown of"** / **"make ours like this"** → BOTH fire in parallel and outputs are interleaved (audio transcript lines + visual shot-list notes, both timestamped) + +When in doubt about which the user wants: default to this skill (cheaper, more common need). If they want visual analysis specifically, they'll say "watch" or "shot list" or "make ours like this." + +## Why this exists + +Before this skill, transcription required: knowing which Python script lived where, knowing which platform was supported by which backend, knowing whether the sandbox had Whisper (it doesn't — disk too small), and stitching the output together manually. That's friction nobody on the team should have to navigate. + +This skill makes transcription a single move: drop a URL or a file, get a transcript. Done. + +## Maintenance + +- **yt-dlp updates**: When a platform's extractor breaks, `pip install -U yt-dlp` usually fixes it. The script can auto-run this when extractor failures are detected. +- **faster-whisper updates**: `pip install -U faster-whisper`. New model versions sometimes ship — re-download is automatic on first use of a new model name. +- **Python version drift**: Python 3.14 is fine but very new — some wheels lag. If `pip install faster-whisper` fails on a newer Python, fall back to a 3.12 venv. +- **Model storage**: Whisper models cache to `~/.cache/huggingface/hub/`. Each model is ~140–500 MB. Safe to delete if disk gets tight, will re-download on next use. diff --git a/skills/video-transcriber/scripts/transcribe.py b/skills/video-transcriber/scripts/transcribe.py new file mode 100755 index 00000000..ecce4711 --- /dev/null +++ b/skills/video-transcriber/scripts/transcribe.py @@ -0,0 +1,401 @@ +#!/usr/bin/env python3 +""" +transcribe.py — Universal video transcriber for Graeham Watts's team. + +Accepts a URL from any video platform (YouTube, Facebook, Instagram, TikTok, +Vimeo, Twitter/X, LinkedIn, Reddit, direct file, ~1,800+ supported by yt-dlp) +and returns a clean transcript. + +Tier 1: Caption pull (free, instant, ~1-3 sec) — only YouTube + some others +Tier 2: yt-dlp audio download + Whisper transcription (free, local, ~30 sec – 3 min) + +Usage: + python3 transcribe.py "https://www.youtube.com/watch?v=VIDEO_ID" + python3 transcribe.py "https://www.instagram.com/reel/REEL_ID/" --json + python3 transcribe.py "URL" --timestamps + python3 transcribe.py "URL" --save # write to outputs/transcripts/ + +Optional env vars: + OPENAI_API_KEY — use OpenAI Whisper API instead of local (faster, costs $0.006/min) + APIFY_API_TOKEN — fallback for platforms yt-dlp doesn't support + +No API keys required for the default free path. +""" + +import argparse +import json +import os +import re +import subprocess +import sys +import tempfile +from datetime import datetime, timezone +from pathlib import Path + + +# --------------------------------------------------------------- +# Platform detection +# --------------------------------------------------------------- + +PLATFORM_PATTERNS = [ + ("youtube", r"(?:youtube\.com|youtu\.be)"), + ("facebook", r"(?:facebook\.com|fb\.watch)"), + ("instagram", r"instagram\.com"), + ("tiktok", r"tiktok\.com"), + ("vimeo", r"vimeo\.com"), + ("twitter", r"(?:twitter\.com|x\.com)"), + ("linkedin", r"linkedin\.com"), + ("reddit", r"reddit\.com"), + ("direct", r"\.(?:mp4|mov|m4a|mp3|wav|webm|mkv|avi)(?:\?|$)"), +] + + +def detect_platform(url: str) -> str: + """Return platform name based on URL pattern.""" + for name, pattern in PLATFORM_PATTERNS: + if re.search(pattern, url, re.IGNORECASE): + return name + return "unknown" + + +def youtube_video_id(url: str) -> str | None: + """Extract YouTube video ID from various URL formats.""" + for pattern in [ + r"(?:v=|/v/|youtu\.be/)([a-zA-Z0-9_-]{11})", + r"(?:embed/|shorts/)([a-zA-Z0-9_-]{11})", + ]: + m = re.search(pattern, url) + if m: + return m.group(1) + return None + + +# --------------------------------------------------------------- +# Dependency management +# --------------------------------------------------------------- + +def ensure_pip_package(pkg_name: str, import_name: str | None = None) -> bool: + """Install a pip package if it's not importable. Returns True on success.""" + mod = import_name or pkg_name.replace("-", "_") + try: + __import__(mod) + return True + except ImportError: + pass + + print(f"[setup] Installing {pkg_name}...", file=sys.stderr) + result = subprocess.run( + [sys.executable, "-m", "pip", "install", pkg_name, "--break-system-packages", "--quiet"], + capture_output=True, text=True + ) + if result.returncode != 0: + print(f"[setup] FAILED to install {pkg_name}: {result.stderr}", file=sys.stderr) + return False + return True + + +def ensure_ffmpeg() -> bool: + """Verify ffmpeg is available.""" + result = subprocess.run(["which", "ffmpeg"], capture_output=True, text=True) + if result.returncode == 0: + return True + print("[setup] ffmpeg not found — attempting apt install...", file=sys.stderr) + subprocess.run(["apt-get", "install", "-y", "ffmpeg"], capture_output=True) + return subprocess.run(["which", "ffmpeg"], capture_output=True).returncode == 0 + + +# --------------------------------------------------------------- +# Tier 1: Caption pull +# --------------------------------------------------------------- + +def caption_pull_youtube(video_id: str) -> dict | None: + """Try to fetch existing YouTube captions via youtube-transcript-api. Returns transcript dict or None.""" + if not ensure_pip_package("youtube-transcript-api"): + return None + try: + from youtube_transcript_api import YouTubeTranscriptApi + from youtube_transcript_api._errors import TranscriptsDisabled, NoTranscriptFound + except ImportError: + return None + + try: + segments = YouTubeTranscriptApi.get_transcript(video_id) + except (TranscriptsDisabled, NoTranscriptFound): + return None + except Exception as e: + print(f"[tier1] caption pull failed: {e}", file=sys.stderr) + return None + + text = " ".join(s["text"] for s in segments) + return { + "method": "caption_pull", + "language": "en", # api default; could be inspected for other langs + "segments": [ + {"start": s["start"], "end": s["start"] + s["duration"], "text": s["text"]} + for s in segments + ], + "transcript_plain": text, + } + + +# --------------------------------------------------------------- +# Tier 2: yt-dlp + Whisper +# --------------------------------------------------------------- + +def download_audio(url: str, tmpdir: str) -> str | None: + """Use yt-dlp to download the audio track. Returns path to mp3 file or None on failure.""" + if not ensure_pip_package("yt-dlp"): + return None + if not ensure_ffmpeg(): + print("[tier2] ffmpeg unavailable, cannot extract audio", file=sys.stderr) + return None + + output_template = os.path.join(tmpdir, "audio.%(ext)s") + cmd = [ + sys.executable, "-m", "yt_dlp", + "-x", "--audio-format", "mp3", + "--audio-quality", "0", + "-o", output_template, + "--quiet", "--no-warnings", + url, + ] + result = subprocess.run(cmd, capture_output=True, text=True) + if result.returncode != 0: + # Try updating yt-dlp once and retry (handles broken extractors) + print("[tier2] yt-dlp failed, updating and retrying...", file=sys.stderr) + subprocess.run([sys.executable, "-m", "pip", "install", "-U", "yt-dlp", "--break-system-packages", "--quiet"], capture_output=True) + result = subprocess.run(cmd, capture_output=True, text=True) + if result.returncode != 0: + print(f"[tier2] yt-dlp failed after retry: {result.stderr}", file=sys.stderr) + return None + + audio_path = os.path.join(tmpdir, "audio.mp3") + if not os.path.exists(audio_path): + return None + return audio_path + + +def whisper_transcribe(audio_path: str, model_size: str = "base") -> dict | None: + """ + Transcribe audio with Whisper. Prefers OpenAI API if OPENAI_API_KEY is set + (faster, ~$0.006/min); falls back to local Whisper otherwise (free, slower). + """ + api_key = os.environ.get("OPENAI_API_KEY") + + if api_key: + # OpenAI Whisper API path + if not ensure_pip_package("openai"): + api_key = None # fall back to local + else: + try: + from openai import OpenAI + client = OpenAI(api_key=api_key) + with open(audio_path, "rb") as f: + resp = client.audio.transcriptions.create( + model="whisper-1", + file=f, + response_format="verbose_json", + timestamp_granularities=["segment"], + ) + segments = [ + {"start": s.start, "end": s.end, "text": s.text} + for s in (resp.segments or []) + ] + return { + "method": "openai_whisper_api", + "language": resp.language, + "segments": segments, + "transcript_plain": resp.text, + } + except Exception as e: + print(f"[tier2] OpenAI API failed: {e}, falling back to local Whisper", file=sys.stderr) + + # Local Whisper path + if not ensure_pip_package("openai-whisper", import_name="whisper"): + return None + import whisper + print(f"[tier2] Loading Whisper '{model_size}' model (first run downloads ~140 MB)...", file=sys.stderr) + model = whisper.load_model(model_size) + print(f"[tier2] Transcribing... this can take 30 sec – 3 min depending on length", file=sys.stderr) + result = model.transcribe(audio_path, verbose=False) + segments = [ + {"start": s["start"], "end": s["end"], "text": s["text"].strip()} + for s in result.get("segments", []) + ] + return { + "method": "local_whisper", + "language": result.get("language", "unknown"), + "segments": segments, + "transcript_plain": result.get("text", "").strip(), + } + + +# --------------------------------------------------------------- +# Metadata extraction +# --------------------------------------------------------------- + +def fetch_metadata(url: str) -> dict: + """Use yt-dlp to grab title + duration without downloading the video.""" + if not ensure_pip_package("yt-dlp"): + return {} + cmd = [ + sys.executable, "-m", "yt_dlp", + "--dump-single-json", "--no-warnings", "--skip-download", + url, + ] + result = subprocess.run(cmd, capture_output=True, text=True) + if result.returncode != 0: + return {} + try: + info = json.loads(result.stdout) + return { + "title": info.get("title"), + "duration_sec": info.get("duration"), + "uploader": info.get("uploader") or info.get("channel"), + "upload_date": info.get("upload_date"), + } + except json.JSONDecodeError: + return {} + + +# --------------------------------------------------------------- +# Main orchestration +# --------------------------------------------------------------- + +def transcribe_url(url: str) -> dict: + """Top-level entry: detect platform, run the right tier, return a normalized dict.""" + platform = detect_platform(url) + + # Tier 1 for YouTube (fast, free, often works) + if platform == "youtube": + vid = youtube_video_id(url) + if vid: + t1 = caption_pull_youtube(vid) + if t1: + meta = fetch_metadata(url) + return { + "url": url, + "platform": platform, + "tier": 1, + **meta, + **t1, + } + + # Tier 2: yt-dlp + Whisper (works for all platforms yt-dlp supports) + meta = fetch_metadata(url) + with tempfile.TemporaryDirectory() as tmpdir: + audio = download_audio(url, tmpdir) + if not audio: + return { + "url": url, + "platform": platform, + "error": "audio_download_failed", + "message": "yt-dlp could not download audio for this URL. The platform may be unsupported, the video may be private/restricted, or the extractor may need updating.", + } + t2 = whisper_transcribe(audio) + if not t2: + return { + "url": url, + "platform": platform, + "error": "whisper_failed", + "message": "Whisper transcription failed. Check OPENAI_API_KEY or local Whisper install.", + } + return { + "url": url, + "platform": platform, + "tier": 2, + **meta, + **t2, + } + + +# --------------------------------------------------------------- +# Output formatting +# --------------------------------------------------------------- + +def format_timestamp(seconds: float) -> str: + """Format seconds as MM:SS or HH:MM:SS.""" + s = int(seconds) + h, rem = divmod(s, 3600) + m, s = divmod(rem, 60) + return f"{h:02d}:{m:02d}:{s:02d}" if h else f"{m:02d}:{s:02d}" + + +def render_plain(result: dict, with_timestamps: bool = False) -> str: + """Render the result as readable plain text.""" + if "error" in result: + return f"ERROR: {result['error']} — {result.get('message', '')}\n" + + lines = [] + if result.get("title"): + lines.append(f"Title: {result['title']}") + lines.append(f"Platform: {result.get('platform', 'unknown')}") + if result.get("duration_sec"): + lines.append(f"Duration: {format_timestamp(result['duration_sec'])}") + if result.get("uploader"): + lines.append(f"Uploader: {result['uploader']}") + lines.append(f"Language: {result.get('language', 'unknown')}") + lines.append(f"Method: {result.get('method', 'unknown')}") + lines.append("") + + if with_timestamps and result.get("segments"): + for seg in result["segments"]: + ts = format_timestamp(seg["start"]) + lines.append(f"[{ts}] {seg['text']}") + else: + lines.append(result.get("transcript_plain", "")) + + return "\n".join(lines) + "\n" + + +def save_to_file(result: dict, output_dir: Path, fmt: str = "txt", with_timestamps: bool = False) -> Path: + """Persist the transcript to outputs/transcripts/ for later reference.""" + output_dir.mkdir(parents=True, exist_ok=True) + platform = result.get("platform", "video") + title_slug = re.sub(r"[^a-z0-9]+", "-", (result.get("title") or "untitled").lower()).strip("-")[:50] + ts = datetime.now(timezone.utc).strftime("%Y%m%d-%H%M%S") + filename = f"transcript-{platform}-{title_slug}-{ts}.{fmt}" + path = output_dir / filename + + if fmt == "json": + path.write_text(json.dumps(result, indent=2, ensure_ascii=False), encoding="utf-8") + else: + path.write_text(render_plain(result, with_timestamps=with_timestamps), encoding="utf-8") + return path + + +# --------------------------------------------------------------- +# CLI +# --------------------------------------------------------------- + +def main(): + parser = argparse.ArgumentParser(description="Universal video transcriber") + parser.add_argument("url", help="Video URL (YouTube, Facebook, Instagram, TikTok, etc.)") + parser.add_argument("--json", action="store_true", help="Output JSON instead of plain text") + parser.add_argument("--timestamps", action="store_true", help="Include [MM:SS] timestamps per segment") + parser.add_argument("--save", action="store_true", help="Also save to outputs/transcripts/") + parser.add_argument("--output-dir", default="outputs/transcripts", help="Output directory when --save is set") + args = parser.parse_args() + + print(f"[transcribe] URL: {args.url}", file=sys.stderr) + print(f"[transcribe] Platform detected: {detect_platform(args.url)}", file=sys.stderr) + + result = transcribe_url(args.url) + + if args.save: + out_path = save_to_file( + result, + Path(args.output_dir), + fmt="json" if args.json else "txt", + with_timestamps=args.timestamps, + ) + print(f"[transcribe] Saved to: {out_path}", file=sys.stderr) + + if args.json: + print(json.dumps(result, indent=2, ensure_ascii=False)) + else: + print(render_plain(result, with_timestamps=args.timestamps)) + + +if __name__ == "__main__": + main() diff --git a/skills/video-transcriber/scripts/transcribe_windows.py b/skills/video-transcriber/scripts/transcribe_windows.py new file mode 100644 index 00000000..3d6e797e --- /dev/null +++ b/skills/video-transcriber/scripts/transcribe_windows.py @@ -0,0 +1,186 @@ +# transcribe_windows.py +# Local Windows transcription using faster-whisper (CPU, int8). +# Handles both local file paths and URLs (yt-dlp downloads audio first for URLs). +# +# Usage from PowerShell: +# python transcribe_windows.py "C:\path\to\video.mp4" +# python transcribe_windows.py "https://www.youtube.com/watch?v=..." +# python transcribe_windows.py "C:\path\to\video.mp4" --model small.en --timestamps +# +# Why faster-whisper instead of openai-whisper: +# - 4-10x faster on CPU +# - Smaller install (no PyTorch dependency) +# - Same accuracy +# Why int8 quantization: +# - Roughly 2x faster on CPU vs fp32 with negligible quality loss for English speech + +import argparse +import os +import re +import subprocess +import sys +import tempfile +import time +from datetime import timedelta + +# Graeham's ffmpeg location. Adjust if it moves. +FFMPEG_DIR = r"C:\Users\Graeham Watts\Documents\Skills LLMS\Claude\ffmpegvideoprocessingengine\bin" + + +def ensure_ffmpeg_on_path(): + """faster-whisper shells out to ffmpeg to decode audio. Add it to PATH if not visible.""" + if FFMPEG_DIR and os.path.isdir(FFMPEG_DIR): + if FFMPEG_DIR not in os.environ.get("PATH", ""): + os.environ["PATH"] = FFMPEG_DIR + os.pathsep + os.environ.get("PATH", "") + + +def is_url(s: str) -> bool: + return s.lower().startswith(("http://", "https://")) + + +def download_audio_with_ytdlp(url: str, tmpdir: str) -> str: + """For URL inputs: pull just the audio track to a temp mp3 with yt-dlp.""" + try: + import yt_dlp # noqa: F401 + except ImportError: + print("[*] Installing yt-dlp (one-time)...") + subprocess.check_call([sys.executable, "-m", "pip", "install", "yt-dlp", "--quiet"]) + + out_template = os.path.join(tmpdir, "audio.%(ext)s") + cmd = [ + sys.executable, "-m", "yt_dlp", + "-x", "--audio-format", "mp3", + "--audio-quality", "0", + "-o", out_template, + "--quiet", "--no-warnings", + url, + ] + print(f"[*] Downloading audio from URL...") + subprocess.check_call(cmd) + audio_path = os.path.join(tmpdir, "audio.mp3") + if not os.path.exists(audio_path): + raise RuntimeError("yt-dlp completed but no audio.mp3 was produced") + return audio_path + + +def slugify(name: str) -> str: + base = re.sub(r"[^a-zA-Z0-9]+", "-", name).strip("-") + return base[:80] if base else "transcript" + + +def transcribe(source: str, model_size: str, output_dir: str, with_timestamps: bool): + ensure_ffmpeg_on_path() + + # Resolve source to a local audio/video path + tmpdir_obj = None + if is_url(source): + tmpdir_obj = tempfile.TemporaryDirectory() + audio_path = download_audio_with_ytdlp(source, tmpdir_obj.name) + source_label = source + slug = slugify(source.split("/")[-1] or "url") + else: + if not os.path.exists(source): + print(f"[!] ERROR: file not found: {source}") + sys.exit(1) + audio_path = source + source_label = os.path.basename(source) + slug = slugify(os.path.splitext(os.path.basename(source))[0]) + + size_mb = os.path.getsize(audio_path) / (1024 * 1024) + print(f"[*] Source: {source_label}") + print(f"[*] Size: {size_mb:.1f} MB") + print(f"[*] Model: {model_size} (int8 quantization, CPU)") + print(f"[*] Loading faster-whisper... (first run downloads the model)") + + try: + from faster_whisper import WhisperModel + except ImportError: + print("[!] faster-whisper not installed. Run: pip install faster-whisper") + sys.exit(1) + + model = WhisperModel(model_size, device="cpu", compute_type="int8") + print(f"[*] Model loaded. Starting transcription...") + + start = time.time() + segments, info = model.transcribe( + audio_path, + beam_size=5, + vad_filter=True, + vad_parameters=dict(min_silence_duration_ms=500), + ) + + print(f"[*] Detected language: {info.language} (confidence: {info.language_probability:.2f})") + print(f"[*] Audio duration: {timedelta(seconds=int(info.duration))} ({info.duration/60:.1f} min)") + + all_segments = [] + for i, seg in enumerate(segments): + all_segments.append(seg) + if i % 20 == 0: + elapsed = time.time() - start + progress = seg.end / info.duration * 100 + eta = (elapsed / max(seg.end, 1)) * (info.duration - seg.end) + print(f" [{progress:5.1f}%] t={timedelta(seconds=int(seg.end))} elapsed={int(elapsed)}s eta~{int(eta)}s") + + elapsed = time.time() - start + print(f"[*] Transcription complete in {elapsed:.0f}s ({elapsed/60:.1f} min)") + print(f"[*] Got {len(all_segments)} segments") + + # Write outputs + os.makedirs(output_dir, exist_ok=True) + plain_path = os.path.join(output_dir, f"{slug}_transcript.txt") + ts_path = os.path.join(output_dir, f"{slug}_transcript_timestamped.txt") + + with open(plain_path, "w", encoding="utf-8") as f: + f.write(f"Transcript: {source_label}\n") + f.write(f"Duration: {timedelta(seconds=int(info.duration))}\n") + f.write(f"Language: {info.language}\n") + f.write(f"Model: faster-whisper {model_size} (int8)\n") + f.write("=" * 70 + "\n\n") + para = [] + for i, seg in enumerate(all_segments): + para.append(seg.text.strip()) + if (i + 1) % 5 == 0: + f.write(" ".join(para) + "\n\n") + para = [] + if para: + f.write(" ".join(para) + "\n") + + if with_timestamps: + with open(ts_path, "w", encoding="utf-8") as f: + f.write(f"Transcript (timestamped): {source_label}\n") + f.write("=" * 70 + "\n\n") + for seg in all_segments: + start_str = str(timedelta(seconds=int(seg.start))) + f.write(f"[{start_str}] {seg.text.strip()}\n") + + print(f"\n[OK] Plain transcript: {plain_path}") + if with_timestamps: + print(f"[OK] Timestamped transcript: {ts_path}") + print(f"\nDone.") + + if tmpdir_obj: + tmpdir_obj.cleanup() + + +def main(): + p = argparse.ArgumentParser(description="Local Windows transcription using faster-whisper") + p.add_argument("source", help="Local file path OR URL") + p.add_argument("--model", default="base.en", + help="Model size. base.en (default, fast), small.en (slower, more accurate), medium.en (slowest, best)") + p.add_argument("--output-dir", default=None, + help="Where to write transcript .txt files. Default: same folder as input.") + p.add_argument("--timestamps", action="store_true", + help="Also write a timestamped version") + args = p.parse_args() + + if args.output_dir is None: + if is_url(args.source): + args.output_dir = os.getcwd() + else: + args.output_dir = os.path.dirname(os.path.abspath(args.source)) + + transcribe(args.source, args.model, args.output_dir, args.timestamps) + + +if __name__ == "__main__": + main() diff --git a/skills/video-transcriber/transcribe_run.log b/skills/video-transcriber/transcribe_run.log new file mode 100644 index 00000000..e30d16c9 --- /dev/null +++ b/skills/video-transcriber/transcribe_run.log @@ -0,0 +1,1265 @@ +[transcribe] URL: https://www.youtube.com/watch?v=W8OHKFqTyDU +[transcribe] Platform detected: youtube +[tier1] caption pull failed: no element found: line 1, column 0 +[tier2] Loading Whisper 'base' model (first run downloads ~140 MB)... +[tier2] Transcribing... this can take 30 sec 3 min depending on length +C:\Users\Graeham Watts\AppData\Roaming\Python\Python314\site-packages\whisper\transcribe.py:132: UserWarning: FP16 is not supported on CPU; using FP32 instead + warnings.warn("FP16 is not supported on CPU; using FP32 instead") +Detected language: English + 0%| | 0/368795 [00:00 **One job:** Watch any video with AI vision. Output a literal blueprint of how to recreate it. + +## The two-skill split (read this first) + +This skill pairs with `video-transcriber`. They're standalone but compose naturally: + +| Skill | What it captures | +|---|---| +| **video-transcriber** | What was SAID — every word, timestamped | +| **video-watcher** | What was SHOWN — every visual beat, timestamped | + +Use either alone for its half, or both together for complete end-to-end understanding. + +The user can invoke both at once by saying something like: +> "Watch and transcribe this video: [URL]" +or +> "Full breakdown of this video: [URL]" + +In which case both skills fire in parallel and the output is interleaved. + +## Who uses this + +**Peter and Ellie** (video editors): use this when Graeham sends them a reference video saying "make ours like this." The skill returns a shot list with exact timestamps telling them what shot type to use when, where text overlays go, what B-roll to pull, what color grade to match, and what pacing to hit. + +**Graeham**: use this to study any reference video personally — competitor analysis, "this viral Reel got 2M views, why" investigations, evaluating whether to commission a specific style for a new campaign. + +**John** (Blog Track): use this to extract the visual structure of a video for blog posts that include "here's how this video is made" content. + +**content-creation-engine**: uses this internally during Phase 0 Mode B (visual analysis pass) when generating new content from a reference video source. The engine no longer owns this code — it calls this skill as an external dependency. + +## How to invoke + +The simplest possible UX. Any of these work: + +1. **Paste URL + watch verb**: + ``` + watch this: https://www.youtube.com/watch?v=PYMsmSx8Tyw + ``` +2. **Reference replication phrase**: + ``` + make ours like this: https://www.instagram.com/reel/DXPuASugkgy/ + ``` +3. **Full breakdown request**: + ``` + full breakdown of https://www.tiktok.com/@user/video/ABC + ``` +4. **Combined with transcriber**: + ``` + watch AND transcribe: https://www.youtube.com/watch?v=... + ``` + +The skill auto-detects the platform from the URL (yt-dlp supports 1,800+ sites) and routes through the right backend. + +## What you get back + +A structured markdown document with these sections: + +``` +TLDR + 3-4 sentences synthesizing the whole video. What it's + trying to accomplish, who it's for, what it does well. + +Hooks (0:00 - 0:10) + The opening 10 seconds analyzed in detail because that's + where scroll-stoppers live. Visual + audio + analysis. + +Per-Scene Notes + For each scene change (typically 8-80 frames per video): + [Timestamp] + On-screen text: exact transcription of any text overlay + Visual: 1-2 sentences describing what's on screen + Said: 1-line quote/paraphrase of the spoken line + Synthesis: 1-2 sentences on why this beat matters + +Key Concepts + Bulleted list with timestamps. The 3-5 ideas the video + communicates most clearly. + +B-Roll Catalog + Table of shot types with timestamps and rough %. + Example: drone aerials 12%, talking head 60%, screen + recording 8%, text-overlay-on-photo 20%. + +On-Screen Text Catalog + Table of every visible text overlay with timestamp, + exact text, and styling notes (color, font feel, size). + +Production Style Fingerprint + Color grade, typography, motion graphics style, framing, + any visible brand signals. What makes this video LOOK + the way it does. + +Code & Commands + If any code, terminal output, or technical commands + appear on screen, transcribed as fenced code blocks. + +Replicate-This Brief ← the killer output + What you'd tell HeyGen + Higgsfield + Remotion + CapCut + to recreate this video's structure. Concrete instructions. + Example: + - HeyGen avatar with warm desk look (Vaibhav template #3) + - Higgsfield drone aerial of Palo Alto / Menlo Park + - 3 Remotion stat-callout overlays at 0:08, 0:25, 0:42 + - Color grade: warm/teal split + - Cut pacing: 8 cuts in 90 seconds + +Open Questions + Anything visible in the video that the analyzer couldn't + fully interpret. (e.g., "What is that gold UI element at + 0:34? Looks like a custom branded watermark.") +``` + +This document IS the blueprint. Peter and Ellie can work directly from it. + +## How the pipeline works (under the hood) + +The skill chains four steps. The user doesn't see this — they just paste a URL and wait 30 sec to 5 min depending on video length. + +``` +1. download.py yt-dlp pulls the video file + ↓ +2. transcribe.py Get the transcript (caption API or Whisper) + ↓ +3. frames.py Smart frame extraction — scene-change detection + + coverage floor (1 frame per N seconds). + Caps at 80 frames per video. + ↓ +4. analyze.py Builds a bundle pairing each frame with its + ±15-second transcript window. Claude (the + invoking skill) reads each frame as multimodal + vision input and writes the structured notes. +``` + +The vision pass uses Claude's built-in multimodal capability — same model that handles image uploads in chat. No separate API key required (the Cowork environment is already Claude-powered). + +For videos longer than 10 minutes, the skill confirms with the user before starting the vision pass (cost ramps up with frame count). + +## Trigger boundaries — when this skill fires vs siblings + +| User says | Skill that fires | +|---|---| +| "transcribe this video" | video-transcriber only | +| "watch this video" | video-watcher only | +| "full breakdown" / "make ours like this" | video-watcher (often also pulls in transcriber for the audio side) | +| "what's in this video" | video-watcher (visual) — though if context suggests "what was said" then transcriber | +| URL alone with no verb | Default: video-transcriber (faster, cheaper, more common need). User can clarify "watch instead" to flip. | +| "generate a blog post from this video" | content-creation-engine (which internally may call video-watcher + video-transcriber) | + +When in doubt: ask. Don't burn $0.80 of vision API on the wrong tool. + +## Optional flags the user can request + +- **"just the shot list"** — skip TLDR, hooks, etc. Return only the B-Roll Catalog + Per-Scene Notes +- **"just the Replicate-This brief"** — skip everything except the recreation instructions +- **"first 30 seconds only"** — analyze only the opening (cheap fast scan for hook study) +- **"save to file"** — write the analysis to `outputs/video-analysis/analysis-{slug}-{timestamp}.md` +- **"and find the cuts"** — produce a cut-list optimized for Premiere editing reference +- **"with the transcript inline"** — invoke video-transcriber too and interleave the spoken lines into the shot list + +## Cost honesty + +Vision API isn't free. Rough estimates per video: + +| Video length | Frames extracted | Approx cost | +|---|---|---| +| 30-second Reel | 5-10 frames | $0.05-$0.15 | +| 90-second Short | 10-20 frames | $0.10-$0.30 | +| 5-minute video | 30-50 frames | $0.30-$0.80 | +| 30-minute interview | 60-80 frames (capped) | $0.50-$1.50 | + +For Peter and Ellie running 2-3 reference videos per week: roughly $5-15/month total. For Graeham doing competitor sweeps: depends on volume. + +The skill ALWAYS reports estimated frame count before kicking off the vision pass on long videos. The user can abort if the cost feels high. + +## Setup requirements + +The Cowork sandbox auto-installs these on first run: + +```bash +pip install yt-dlp youtube-transcript-api openai-whisper --break-system-packages +apt install -y ffmpeg # usually already installed +``` + +For Peter and Ellie's local installs, the skill auto-installs dependencies on first run too. They might see a one-time "installing yt-dlp..." message on the first invocation; subsequent runs are immediate. + +**Optional env vars** (none required for the default flow): +- `OPENAI_API_KEY` — uses OpenAI Whisper API for faster transcription on long videos (the audio path costs $0.006/min) +- `APIFY_API_TOKEN` — fallback for niche platforms yt-dlp doesn't support +- `DATAFORSEO_LOGIN` + `DATAFORSEO_PASSWORD` — Tier 0 caption pull for YouTube videos (free per pull, $0.004 if used) + +## Example: end-to-end run + +**Peter pastes:** +``` +make ours like this: https://www.instagram.com/reel/DXNXXXX/ +``` + +**Skill flow:** +1. Detects: Instagram Reel +2. Estimates cost: ~15 frames × vision API ≈ $0.20 +3. Asks Peter to confirm (auto-skips this prompt for short videos) +4. Runs yt-dlp to download the Reel (~20 MB, 5 sec) +5. Runs frames.py — extracts 12 frames at scene changes +6. Runs transcribe.py — pulls 90-second transcript +7. Runs analyze.py — builds the multimodal bundle +8. Claude reads each frame + surrounding transcript context +9. Writes the full structured markdown analysis +10. Returns to Peter as a clean document he can paste into Premiere notes + +Total: ~90 seconds. Output: a blueprint he can shoot from tomorrow. + +## How content-creation-engine uses this + +The engine's Phase 0 (Source Ingestion) has two modes: +- **Mode A (transcript only)** — default for most content generation tasks +- **Mode B (transcript + visual analysis)** — invoked when the user wants to replicate a reference video's style, not just its message + +In Mode B, content-creation-engine now calls `video-watcher` as an external skill instead of running the embedded analysis code. The output flows into the script-writer Phase to inform shot direction in the generated content package (which inline shot tags to use, what B-roll types to source, what production style fingerprint to match). + +## Failure handling + +| Failure | What the skill does | +|---|---| +| URL not recognized by yt-dlp | Reports the platform, asks user to confirm an alternate path | +| Video is private or geo-blocked | Reports the access error verbatim | +| Frame extraction fails (corrupt video) | Falls back to coverage-floor sampling only (drop scene detection) | +| Vision API rate-limited | Backs off, retries; if persistent, falls back to text-only analysis using existing transcript + filename + duration | +| Video is very long (>30 min) | Confirms with user before kicking off (cost concern) | +| Network access blocked (Cowork sandbox firewall) | Reports the block honestly, suggests running locally on user's machine instead | + +## Maintenance + +The 6 Python scripts in `scripts/` were lifted from `content-creation-engine/scripts/video-research/` on 2026-05-15. They are now the canonical owners of this logic. If content-creation-engine still has copies, those are deprecated — refer here. + +When yt-dlp's platform list expands, frame extraction expands automatically. When Claude's vision model improves, the analysis output improves automatically. No code changes needed in this skill. diff --git a/skills/video-watcher/references/ffmpeg-trimming.md b/skills/video-watcher/references/ffmpeg-trimming.md new file mode 100755 index 00000000..57936902 --- /dev/null +++ b/skills/video-watcher/references/ffmpeg-trimming.md @@ -0,0 +1,38 @@ +--- +name: ffmpeg +description: Using FFmpeg and FFprobe in Remotion +metadata: + tags: ffmpeg, ffprobe, video, trimming +--- + +## FFmpeg in Remotion + +`ffmpeg` and `ffprobe` do not need to be installed. They are available via the `bunx remotion ffmpeg` and `bunx remotion ffprobe`: + +```bash +bunx remotion ffmpeg -i input.mp4 output.mp3 +bunx remotion ffprobe input.mp4 +``` + +### Trimming videos + +You have 2 options for trimming videos: + +1. Use the FFmpeg command line. You MUST re-encode the video to avoid frozen frames at the start of the video. + +```bash +# Re-encodes from the exact frame +bunx remotion ffmpeg -ss 00:00:05 -i public/input.mp4 -to 00:00:10 -c:v libx264 -c:a aac public/output.mp4 +``` + +2. Use the `trimBefore` and `trimAfter` props of the `
+ + +
+
Bi-Monthly Market Update · June 2026
+

Equity & Wealth Position
Bay Area + East Palo Alto

+
Production brief — two videos, one package
+
+ B4 — Equity & Wealth Angle + Format 3: Bi-Monthly + Financial / Advisory Tone +
+
+ + +
+ + +
+
Full Production Dashboard
+

Everything you need — full scripts, SSML/voice, shot list, AI video prompts, platform copy, and YouTube SEO — is in the production dashboard below. This email is your quick-reference summary.

+
+ Open Production Dashboard → +
+
+ +
+ + +
+
Why This Angle
+
+ This is the June bi-monthly update (every 2 months, Bay Area + EPA). We deliberately skipped the buyer-behavior and seller-timing angles — they felt too close to the May market recap. This month's angle is B4: Equity & Wealth Position — aimed at long-term homeowners who aren't actively buying or selling but are starting to think about their next move. +
+

Audience: people who bought 5–12 years ago and haven't thought hard about what their equity actually means in options. This is not a market hype video. It's an advisory video — think financial planner, not listing agent.

+
+ +
+ + +
+
The Two Videos
+ +
+
Bay Area / San Mateo County
+

Your Peninsula Equity: $820K and Three Options

+
"If you bought in San Mateo County before 2020, there's a number you should know — and most homeowners have no idea what it is."
+

The average San Mateo County homeowner who bought in 2015 is sitting on approximately $820,000 in equity. The video walks through three options: sell (and what the net looks like), refinance or HELOC (when this makes sense), and the 1031 exchange (for owners thinking about investment property).

+
GHL: EQUITY
+
+ +
+
East Palo Alto
+

EPA Long-Term Owners: $650K in Equity — Here's What That Means

+
"If you bought in East Palo Alto before 2014, you're likely sitting on around $650,000 in equity. And the math of what to do with it is more complicated than you think."
+

EPA-specific equity story with Prop 13 savings math ($5,200/yr vs. $13,400/yr if reassessed), capital gains calculation (~$40K–$50K tax on $150K taxable gain after exclusion), and realistic net proceeds (~$550K–$600K). Gives long-term owners a real number to work with.

+
GHL: OPTIONS
+
+
+ +
+ + +
+
Tone & Visual Direction
+
+
Direction for both videos
+
    +
  • Financial / advisory register — calm, measured, authoritative. NOT market hype, NOT urgency.
    • +
  • Speak to the camera like a trusted financial advisor at a kitchen table, not like a listing agent at an open house.
    • +
  • Pacing is slower and more deliberate than a buyer-strategy video — let the numbers land.
    • +
  • Visuals: cool-neutral aesthetic (navy + gold). Equity gauge graphics, hands reviewing documents, clean data overlays. No "SOLD" signs, no bidding war energy.
    • +
  • Thumbnail concept: equity gauge graphic, bold "YOUR EQUITY NOW" text, cool financial palette — deliberately different from last update's warm aerial style.
    • +
+
+
+ +
+ + +
+
Shot List Summary (full list in dashboard)
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
#ShotMarketNotes
1Talking head — hook delivery, direct eye contactBothGraeham seated, calm framing. No motion graphics behind — keep it clean.
2On-screen equity number reveal ($820K / $650K)BothBig number, full frame, 1.5-sec hold. Let it breathe.
3Three-options graphic — sell / refi / 1031Bay AreaMotion graphic overlay or whiteboard-style reveal.
4Prop 13 savings comparison ($5,200 vs $13,400)EPASide-by-side text card. Clear, no clutter.
5Net proceeds breakdown (~$550K–$600K)EPASimple bar or number card. Real numbers, not ranges where possible.
6Talking head — CTA closeBoth"Text EQUITY to 650-308-4727" (Bay Area) / "Text OPTIONS" (EPA)
7B-roll: Peninsula neighborhood dusk dollyBay AreaAI video (Seedance 2.0 prompt in dashboard). Cool-neutral grade, no golden hour warmth.
8B-roll: EPA street-level slow push-inEPAAI video (Seedance 2.0 prompt in dashboard). Long-hold, still framing — matches advisory tone.
+
+ +
+ + +
+
AI Video Prompts — Seedance 2.0 (ready to paste)
+ +
+
Prompt 1 — Equity Gauge Graphic (motion graphic, both videos)
+

Animated equity gauge graphic on dark navy background (#1B2A4A). Gold needle sweeps from left to center-right, settling on a bold "$820,000" readout. Clean sans-serif typography. Subtle particle ambient light. Financial data-viz aesthetic. Cool-neutral color palette. 5-second loop, no camera movement. 4K, 9:16 vertical.

+
+ +
+
Prompt 2 — Peninsula Dusk Dolly (Bay Area B-roll)
+

Slow dolly forward along a quiet San Mateo County residential street at dusk. Cool-blue twilight sky, warm interior house lights visible through windows. Mature trees lining the street. No people. 35mm cinematic lens, shallow depth of field. Cool-neutral grade — deliberately NOT golden hour warm tones. 6 seconds. 4K, 9:16 vertical.

+
+ +
+
Prompt 3 — EPA Long-Term Home Push-In (EPA B-roll)
+

Ultra-slow push-in toward the front door of a modest well-maintained East Palo Alto single-family home. Exterior shot, late afternoon cool light. Established neighborhood feel — mature landscaping, clean paint. No people visible. Handheld-stable, almost imperceptible camera drift forward. Quiet, contemplative mood. 6 seconds. 4K, 9:16 vertical.

+
+
+ +
+ + +
+
Quick Reminders
+
+ DRE# 01466876 — must appear in both videos (lower third or end card). Brokerage: Intero Real Estate. +
+
+ GHL keyword capture: Bay Area video → text EQUITY to 650-308-4727. EPA video → text OPTIONS to 650-308-4727. +
+
+ Full scripts, SSML, YouTube SEO, and platform copy are all in the dashboard — click the button above. The SSML tab has the ElevenLabs-ready voice script for both markets. +
+
+ +
+ + +
+

Graeham Watts · Intero Real Estate · DRE# 01466876
+ 650-308-4727 · graehamwatts.com
+ Production brief · Confidential — for team use only

+
+ +