Develop

README.md

@@ -75,13 +75,14 @@ Select **Create extension** from the menu and follow the prompts, or skip them w
 degoog-cli create --name my-engine --type engine --out ./extensions
 ```
 
-Any flag you omit will still be asked interactively. Available types:
+Any flag you omit will still be asked interactively, unless the CLI is running headless (see below). Available types:
 
 | Type                | What it is                                                  |
 | ------------------- | ----------------------------------------------------------- |
 | `engine`            | Custom search engine                                        |
 | `transport`         | Custom HTTP fetch strategy                                  |
 | `autocomplete`      | Search suggestions provider                                 |
+| `favicon`           | Favicon provider for search results                         |
 | `theme`             | UI theme with CSS variables and optional template overrides |
 | `plugin-bang`       | Bang command (`!trigger query`)                             |
 | `plugin-slot`       | Panel injected into search results                          |
@@ -94,14 +95,43 @@ Any flag you omit will still be asked interactively. Available types:
 
 Every extension gets:
 
-- `index.ts` - entry file pre-filled with the correct contract for the type you picked
+- `index.ts` - entry file pre-filled with the correct contract for the type you picked (favicon providers get `index.js`, matching the official store)
 - `README.md` - fill this in, it shows as docs on the extension's settings page
 - `author.json` - auto-filled from your login details
 
 Themes also get `style.css` with all CSS variables pre-filled to the degoog defaults (light and dark), plus a `templates/logo.html` example showing how to override a template section.
 
+The generated theme README also covers the script-free `/nojs` page, which reuses your ordinary templates and takes per-file overrides from a `nojs/` folder by filename instead of through `theme.json`.
+
 Plugin types also have `isClientExposed` already in the template. Set it to `true` if your plugin causes the browser to fetch external URLs directly, `false` if everything goes through the server. Leaving it unset shows a warning badge in degoog settings.
 
+Engines ask whether the site puts an Anubis proof-of-work page in front of its results. Answer yes and the template sets `challenges: ["anubis"]`, so degoog solves the page before your engine reads the response. Inside a store, the package.json entry also gets `minDegoogVersion: "1.0.0"`, because degoog solves these pages from 1.0.0 onwards.
+
+Transports ask whether they drive a real browser that gets through proof-of-work pages. Answer yes and the template sets `handlesChallenges: true`, so degoog leaves those pages to the engine.
+
+Doctor checks both values. It fails on an unknown `challenges` kind or a `handlesChallenges` that isn't `true` or `false`, and warns when an engine with `challenges` has a `minDegoogVersion` below 1.0.0.
+
+## Running without prompts
+
+Every command also runs with no interactive UI, for scripts, CI, or piping. The CLI goes headless when you pass `-y` / `--yes`, when stdin or stdout is not a terminal, or when `CI` or `DEGOOG_HEADLESS=1` is set. Headless runs never prompt: they read flags, fall back to defaults, and exit `1` when a required value is missing or a check fails. Colours are dropped when output is not a terminal, or with `--no-color` / `NO_COLOR`.
+
+```sh
+degoog-cli login --yes --instance-url https://search.example.com --api-key "$DEGOOG_API_KEY" --username "Jane Dev"
+degoog-cli create --yes --name my-engine --type engine --challenges
+degoog-cli create --yes --name my-theme --type theme --init-store --include css,home-logo
+degoog-cli search --json --limit 5 "degoog extensions"
+degoog-cli doctor ./my-store --fix --plugin-type command
+```
+
+| Command  | Flags                                                                                                                                       | Headless defaults                                                                             |
+| -------- | ------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------- |
+| `create` | `--name`, `--type`, `--out`, `--init-store`, `--challenges`, `--handles-challenges`, `--include <parts\|all>`                               | `--name` and `--type` are required. Output goes to the detected store category, otherwise `.` |
+| `search` | `<query>`, `--json`, `--limit <n>`                                                                                                          | Query is required. `--json` always skips the result browser                                   |
+| `login`  | `--instance-url` (or `DEGOOG_INSTANCE_URL`), `--api-key` (or `DEGOOG_API_KEY`), `--username`, `--website`                                   | Only the values you pass are changed                                                          |
+| `doctor` | `[path]`, `--fix`, `--plugin-type <command\|slot\|interceptor\|search-result-tab>`                                                          | Path defaults to the current directory. Orphan folders are only registered with `--fix`       |
+
+Boolean flags accept `--flag=false`. Themes created headless without `--include` get `css` only. Orphan plugin folders are skipped unless `--plugin-type` is given, since doctor cannot guess their type. Run `degoog-cli --help` for the full list.
+
 ## Docker Compose
 
 If you prefer compose over the wrapper script:

package.json

@@ -1,6 +1,6 @@
 {
   "name": "degoog-cli",
-  "version": "0.6.0",
+  "version": "0.7.0",
   "description": "CLI toolset for degoog",
   "type": "module",
   "scripts": {

src/commands/create.ts

@@ -1,26 +1,29 @@
 import * as p from "@clack/prompts";
 import { join } from "node:path";
-import { ExtType, type GeneratorCtx } from "../types/index.ts";
-import { promptExtType } from "../prompts/ext-type.ts";
-import { generateEngine } from "../generators/engine.ts";
-import { generateTransport } from "../generators/transport.ts";
-import { generateAutocomplete } from "../generators/autocomplete.ts";
-import { generateTheme } from "../generators/theme.ts";
-import { generatePluginBang } from "../generators/plugin-bang.ts";
-import { generatePluginSlot } from "../generators/plugin-slot.ts";
-import { generatePluginTab } from "../generators/plugin-tab.ts";
-import { generatePluginIntercept } from "../generators/plugin-intercept.ts";
-import { generatePluginMid } from "../generators/plugin-mid.ts";
-import { generatePluginRoute } from "../generators/plugin-route.ts";
-import { loadConfig } from "../config/store.ts";
-import { argv } from "../utils/argv.ts";
+import { ExtType, type GeneratorCtx } from "../types";
+import { promptExtType } from "../prompts/ext-type";
+import { generateEngine } from "../generators/engine";
+import { generateTransport } from "../generators/transport";
+import { generateAutocomplete } from "../generators/autocomplete";
+import { generateFavicon } from "../generators/favicon";
+import { generateTheme, THEME_PARTS } from "../generators/theme";
+import { generatePluginBang } from "../generators/plugin-bang";
+import { generatePluginSlot } from "../generators/plugin-slot";
+import { generatePluginTab } from "../generators/plugin-tab";
+import { generatePluginIntercept } from "../generators/plugin-intercept";
+import { generatePluginMid } from "../generators/plugin-mid";
+import { generatePluginRoute } from "../generators/plugin-route";
+import { loadConfig } from "../config/store";
+import { argv } from "../utils/argv";
 import {
   extTypeToCategory,
   findStoreRoot,
   registerExtensionInStore,
   scaffoldStore,
-} from "../utils/store.ts";
-import { t } from "../utils/theme.ts";
+} from "../utils/store";
+import { t } from "../utils/theme";
+import { ui } from "../utils/ui";
+import { isHeadless, fail } from "../utils/headless";
 
 const SLUG_RE = /^[a-z][a-z0-9-]*$/;
 
@@ -30,6 +33,7 @@ const GENERATORS: Record<ExtType, (ctx: GeneratorCtx) => Promise<string>> = {
   [ExtType.Engine]: generateEngine,
   [ExtType.Transport]: generateTransport,
   [ExtType.Autocomplete]: generateAutocomplete,
+  [ExtType.Favicon]: generateFavicon,
   [ExtType.Theme]: generateTheme,
   [ExtType.PluginBang]: generatePluginBang,
   [ExtType.PluginSlot]: generatePluginSlot,
@@ -49,11 +53,13 @@ const resolveOutDir = async (
   let store = await findStoreRoot(process.cwd());
 
   if (store) {
-    p.log.info(t.muted(`store detected — creating in ${category}/`));
+    ui.info(t.muted(`store detected — creating in ${category}/`));
     return join(store.dir, category);
   }
 
-  const setup = await p.confirm({
+  if (isHeadless && !argv.initStore) return ".";
+
+  const setup = argv.initStore || await p.confirm({
     message: "No store detected. Set up a store in the current directory?",
     initialValue: true,
   });
@@ -63,7 +69,7 @@ const resolveOutDir = async (
     await scaffoldStore(process.cwd(), config);
     store = await findStoreRoot(process.cwd());
     if (store) {
-      p.log.info(t.muted(`store created — creating in ${category}/`));
+      ui.info(t.muted(`store created — creating in ${category}/`));
       return join(store.dir, category);
     }
   }
@@ -76,11 +82,37 @@ const resolveOutDir = async (
   return input || ".";
 };
 
+const parseThemeParts = (): string[] | undefined => {
+  if (argv.include === undefined) return undefined
+  const parts = argv.include.split(",").map((s) => s.trim()).filter(Boolean)
+  if (parts.includes("all")) return THEME_PARTS
+  const unknown = parts.filter((s) => !THEME_PARTS.includes(s))
+  if (unknown.length) {
+    fail(`unknown --include value(s): ${unknown.join(", ")}. Valid: all, ${THEME_PARTS.join(", ")}`)
+  }
+  return parts
+}
+
+const askConfirm = async (
+  preset: boolean | undefined,
+  message: string,
+): Promise<boolean | null> => {
+  if (preset !== undefined) return preset
+  if (isHeadless) return false
+  const picked = await p.confirm({ message, initialValue: false })
+  if (p.isCancel(picked)) return null
+  return picked
+}
+
 export const createCmd = async () => {
   let name: string
 
   if (argv.name && SLUG_RE.test(argv.name)) {
     name = argv.name
+  } else if (isHeadless) {
+    return fail(argv.name
+      ? `invalid --name "${argv.name}": lowercase letters, numbers and hyphens only`
+      : "--name is required when running without a terminal")
   } else {
     const input = await p.text({
       message: "Extension name",
@@ -99,26 +131,52 @@ export const createCmd = async () => {
 
   if (argv.type && VALID_TYPES.includes(argv.type)) {
     extType = argv.type as ExtType
+  } else if (isHeadless) {
+    return fail(`${argv.type ? `invalid --type "${argv.type}"` : "--type is required when running without a terminal"}. Valid: ${VALID_TYPES.join(", ")}`)
   } else {
     const picked = await promptExtType()
     if (!picked) return
     extType = picked
   }
 
+  let challenges = false
+  let handlesChallenges = false
+
+  if (extType === ExtType.Engine) {
+    const picked = await askConfirm(
+      argv.challenges,
+      "Does the site put an Anubis proof-of-work page in front of its results?",
+    )
+    if (picked === null) return
+    challenges = picked
+  }
+
+  if (extType === ExtType.Transport) {
+    const picked = await askConfirm(
+      argv.handlesChallenges,
+      "Does this transport drive a real browser that gets through proof-of-work pages?",
+    )
+    if (picked === null) return
+    handlesChallenges = picked
+  }
+
+  const themeParts = extType === ExtType.Theme ? parseThemeParts() : undefined
+
   const config = await loadConfig()
   const outDir = await resolveOutDir(extType, config)
   if (!outDir) return
 
-  const ctx: GeneratorCtx = { name, outDir, config }
+  const ctx: GeneratorCtx = { name, outDir, config, challenges, handlesChallenges }
+  if (themeParts) ctx.themeParts = themeParts
 
   const createdPath = await GENERATORS[extType](ctx)
 
   const store = await findStoreRoot(createdPath)
   if (store) {
-    await registerExtensionInStore(store.dir, name, extType)
+    await registerExtensionInStore(store.dir, name, extType, challenges)
   }
 
-  p.note(
+  ui.note(
     `That's it, you now have a sexy template for your extension. Have fun making it your own!`,
     "done",
   )

src/commands/doctor/checks.ts

@@ -1,8 +1,9 @@
 import { readFile, writeFile, readdir } from "node:fs/promises"
+import type { Dirent } from "node:fs"
 import { join, basename } from "node:path"
-import { exists } from "./detect.ts"
-import { findStoreRoot } from "../../utils/store.ts"
-import type { CheckResult, ExtensionKind, RunSummary } from "./types.ts"
+import { exists } from "./detect"
+import { findStoreRoot } from "../../utils/store"
+import { ENGINE_CHALLENGES, type CheckResult, type ExtensionKind, type RunSummary } from "./types"
 
 const readJson = async <T>(path: string): Promise<T | null> => {
   try {
@@ -182,6 +183,119 @@ const checkRouteConventions = async (dir: string, doFix: boolean): Promise<Check
   return results
 }
 
+const FAVICON_SOURCE_EXT = /\.(js|mjs|cjs|ts|html)$/
+const HAND_BUILT_FAVICON_RE = /\/api\/proxy\/favicon\?domain=/
+const FAVICON_SIGNER_RE = /\bsignFaviconUrl\b/
+
+const readPluginSources = async (dir: string, rel = ""): Promise<{ file: string; src: string }[]> => {
+  let entries: Dirent[]
+  try {
+    entries = await readdir(join(dir, rel), { withFileTypes: true })
+  } catch {
+    return []
+  }
+  const out: { file: string; src: string }[] = []
+  for (const entry of entries) {
+    const file = rel ? `${rel}/${entry.name}` : entry.name
+    if (entry.isDirectory()) {
+      if (entry.name === "node_modules" || entry.name.startsWith(".")) continue
+      out.push(...(await readPluginSources(dir, file)))
+    } else if (FAVICON_SOURCE_EXT.test(entry.name)) {
+      try {
+        out.push({ file, src: await readFile(join(dir, file), "utf-8") })
+      } catch {
+        continue
+      }
+    }
+  }
+  return out
+}
+
+export const checkFaviconUrls = async (dir: string): Promise<CheckResult[]> => {
+  const offenders = (await readPluginSources(dir))
+    .filter(({ src }) => HAND_BUILT_FAVICON_RE.test(src))
+    .map(({ file }) => file)
+  if (offenders.length === 0) {
+    return [{ label: "favicon URLs are signed by degoog", status: "pass" }]
+  }
+  return offenders.map((file) => ({
+    label: `${file} favicon URLs`,
+    status: "fail",
+    detail: "hand-built /api/proxy/favicon URL gets a 403 - sign it on the server with ctx.signFaviconUrl(url)",
+  }))
+}
+
+export const usesFaviconSigner = async (dir: string): Promise<boolean> =>
+  (await readPluginSources(dir)).some(({ src }) => FAVICON_SIGNER_RE.test(src))
+
+const ENTRY_FILES = ["index.ts", "index.js"]
+const CHALLENGES_RE = /(?<![.\w$])challenges\s*[:=]\s*(\[[^\]]*\]|[^\s,;}]+)/
+const HANDLES_CHALLENGES_RE = /(?<![.\w$])handlesChallenges\s*[:=]\s*([^\s,;}]+)/
+const STRING_LITERAL_RE = /^(["'`])([^"'`]*)\1$/
+
+const readEntrySource = async (dir: string): Promise<string | null> => {
+  for (const file of ENTRY_FILES) {
+    try {
+      return await readFile(join(dir, file), "utf-8")
+    } catch {
+      continue
+    }
+  }
+  return null
+}
+
+export const checkChallengesValue = (raw: string): CheckResult => {
+  const label = '"challenges" is valid'
+  if (!raw.startsWith("[")) {
+    return { label, status: "fail", detail: 'must be an array, e.g. ["anubis"]' }
+  }
+  const items = raw
+    .slice(1, -1)
+    .split(",")
+    .map((item) => item.trim())
+    .filter(Boolean)
+  const unknown = items.filter((item) => {
+    const kind = STRING_LITERAL_RE.exec(item)?.[2]
+    return kind === undefined || !ENGINE_CHALLENGES.includes(kind)
+  })
+  if (unknown.length > 0) {
+    return {
+      label,
+      status: "fail",
+      detail: `unknown kind ${unknown.join(", ")} - supported: ${ENGINE_CHALLENGES.map((k) => `"${k}"`).join(", ")}`,
+    }
+  }
+  return { label, status: "pass" }
+}
+
+export const checkHandlesChallengesValue = (raw: string): CheckResult => {
+  const label = '"handlesChallenges" is valid'
+  if (raw === "true" || raw === "false") return { label, status: "pass" }
+  return { label, status: "fail", detail: `must be true or false, got ${raw}` }
+}
+
+export const declaresChallenges = async (dir: string): Promise<boolean> => {
+  const src = await readEntrySource(dir)
+  return src !== null && CHALLENGES_RE.test(src)
+}
+
+const checkChallengeProps = async (
+  dir: string,
+  kind: ExtensionKind,
+): Promise<CheckResult[]> => {
+  const src = await readEntrySource(dir)
+  if (src === null) return []
+  if (kind === "engine") {
+    const raw = CHALLENGES_RE.exec(src)?.[1]
+    return raw === undefined ? [] : [checkChallengesValue(raw)]
+  }
+  if (kind === "transport") {
+    const raw = HANDLES_CHALLENGES_RE.exec(src)?.[1]
+    return raw === undefined ? [] : [checkHandlesChallengesValue(raw)]
+  }
+  return []
+}
+
 const runThemeChecks = async (dir: string, doFix: boolean): Promise<RunSummary> => {
   const results: CheckResult[] = []
   let failed = false
@@ -287,10 +401,19 @@ export const runChecks = async (
     if (authorRes.failed) failed = true
   }
 
+  if (kind === "engine" || kind === "transport") {
+    const challengeChecks = await checkChallengeProps(dir, kind)
+    results.push(...challengeChecks)
+    if (challengeChecks.some((c) => c.status === "fail")) failed = true
+  }
+
   if (kind === "plugin") {
     const routeChecks = await checkRouteConventions(dir, doFix)
     results.push(...routeChecks)
     if (routeChecks.some((c) => c.status === "fail")) failed = true
+    const faviconChecks = await checkFaviconUrls(dir)
+    results.push(...faviconChecks)
+    if (faviconChecks.some((c) => c.status === "fail")) failed = true
   }
 
   return { results, failed }