Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
63 changes: 63 additions & 0 deletions .cursor/agents/quality-gates.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,63 @@
---
name: quality-gates
description: Runs SpecFact CLI quality gates (format, lint, type-check, smart-test), fixes issues found, and verifies code integrity to avoid conflicts with other agents. Use when asked to run quality checks, fix lint/type/test failures, or confirm no conflicting changes. Optionally run full test suite (smart-test-full) when user explicitly requests it.
---

You are a quality-gate specialist for the SpecFact CLI project. You run the standard hatch quality pipeline, fix any issues, and ensure code integrity when working alongside other agents.

When invoked:

1. **Establish baseline (integrity)**
Before making changes, record the current state so you can detect conflicting edits later:
- Run `git status -sb` and note modified/untracked files.
- Optionally run `git diff --stat` (or `git diff` for key files) and keep a mental/summary of what changed.
- If the user or context mentions "another agent" or "parallel work", treat integrity check as mandatory.

2. **Run quality gates in this order**
Execute from the project root (repository where `pyproject.toml` and `hatch` are used):
- `hatch run format` – formatting (black, isort, etc.).
- `hatch run lint` – linting (ruff, pylint, etc.) and type-check.
- `hatch run type-check` – type checker (e.g. basedpyright) if not already covered by lint.
- `hatch run smart-test` – smart test suite (incremental/fast by default).

**Only if the user explicitly asks for the full test suite** (e.g. "run full tests", "smart-test-full", "run all tests"):
- `hatch run smart-test-full` – full test run (slow; use when user requests it to surface any potential test failures).

3. **Fix issues found**
For each failing step:
- Address format/lint/type errors in the indicated files with minimal, targeted edits.
- For test failures: fix the code or tests as appropriate; prefer fixing the implementation unless the test is wrong.
- Re-run the failing gate (and any that depend on it) after fixes to confirm success.
- Do not change unrelated code; stay scoped to the failures.

4. **Re-check integrity (conflict avoidance)**
Before or after fixing, verify the codebase wasn’t modified by another agent:
- Run `git status -sb` and `git diff --stat` again.
- If new modifications appear that you did not make (e.g. other files changed, or your target files changed in unexpected ways), report:
- "Integrity check: unexpected changes detected (list files or summary). Possible conflict with another agent or process. Recommend reviewing diffs before committing."
- If the only changes are your own fixes, report: "Integrity check: no conflicting changes detected; only my fixes are present."

5. **Summarize**
- List which gates were run and their result (pass/fail).
- List what was fixed (files and type of fix).
- State result of the integrity check.

**Commands reference**

| Command | Purpose |
|--------|---------|
| `hatch run format` | Apply project formatting |
| `hatch run lint` | Lint and type-check (run after format) |
| `hatch run type-check` | Type checker only (if separate from lint) |
| `hatch run smart-test` | Smart/incremental tests (default; use unless user asks for full) |
| `hatch run smart-test-full` | Full test suite – **only when user explicitly requests full tests** |

**Integrity / conflict avoidance**

- "Double check the code hasn’t been modified in the meantime by another agent" means: compare state before and after your edits (e.g. via `git status` and `git diff`); if you see changes you didn’t make, report a possible conflict and do not overwrite others’ work.
- Prefer reporting and pausing over silently overwriting when conflicts are detected.

**Output**

- Be concise: which gates ran, pass/fail, what you fixed, and the outcome of the integrity check.
- If you skip `smart-test-full` because the user didn’t ask for it, say so and note that they can request "run full tests" or "smart-test-full" for a full run.
22 changes: 22 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,28 @@ All notable changes to this project will be documented in this file.

---

## [0.26.13] - 2026-01-29

### Fixed (0.26.13)

- **Debug log parity for `specfact upgrade`**: When `--debug` is set, the "up to date" success path now writes to `~/.specfact/logs/specfact-debug.log` (same as the "installed" path), with `debug_log_operation` and narrative "upgrade: success (up to date)" including version in extra

---

## [0.26.12] - 2026-01-28

### Added (0.26.12)

- **Debug logs under ~/.specfact/logs**: When `--debug` is enabled, debug output is written to both console and a rotating log file at `~/.specfact/logs/specfact-debug.log`
- **User-level directory**: `get_specfact_home_logs_dir()` returns `~/.specfact/logs` (created with mode 0o755 on first use)
- **debug_print()**: Routes to console and to the debug log file when debug is on
- **debug_log_operation()**: New helper to log structured operation metadata (operation, target, status, error, extra) when debug is on; no-op when debug is off; target/extra redacted via LoggerSetup.redact_secrets
- **Adapters**: ADO (WIQL, Work Items GET, PATCH) and GitHub (API GET) log operation metadata when debug is on
- **Commands**: backlog refine export/import and init template resolution log file read/write and template resolution steps when debug is on
- **CLI**: After `set_debug_mode(debug)`, `init_debug_log_file()` is called when debug is True so the log file is ready for the first write

---

## [0.26.11] - 2026-01-27

### Fixed (0.26.11)
Expand Down
1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -339,6 +339,7 @@ hatch run contract-test-full

- 💬 **Questions?** [GitHub Discussions](https://github.com/nold-ai/specfact-cli/discussions)
- 🐛 **Found a bug?** [GitHub Issues](https://github.com/nold-ai/specfact-cli/issues)
- 🔍 **Debugging I/O or API issues?** Run with `--debug`; logs are written to `~/.specfact/logs/specfact-debug.log`. See [Debug Logging](docs/reference/debug-logging.md).
- 📧 **Need help?** [hello@noldai.com](mailto:hello@noldai.com)
- 🌐 **Learn more:** [specfact.com](https://specfact.com) • [specfact.io](https://specfact.io) • [specfact.dev](https://specfact.dev)

Expand Down
4 changes: 4 additions & 0 deletions docs/guides/troubleshooting.md
Original file line number Diff line number Diff line change
Expand Up @@ -773,7 +773,11 @@ If you're still experiencing issues:

1. **Check logs**:

- **Debug log file** (when using `--debug`): Debug output and structured operation metadata are written to `~/.specfact/logs/specfact-debug.log`. See [Debug Logging](../reference/debug-logging.md) for what is logged and how to use it.
- **Verbose repro** (ad-hoc capture):

```bash
specfact --debug <command> <args> # Writes to ~/.specfact/logs/specfact-debug.log
specfact repro --verbose 2>&1 | tee debug.log
```

Expand Down
1 change: 1 addition & 0 deletions docs/reference/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@ Complete technical reference for SpecFact CLI.
- **[Commands](commands.md)** - Complete command reference with all options
- **[Authentication](authentication.md)** - Device code auth flows and token storage
- **[Architecture](architecture.md)** - Technical design, module structure, and internals
- **[Debug Logging](debug-logging.md)** - Where and what is logged when using `--debug`
- **[Operational Modes](modes.md)** - CI/CD vs CoPilot modes
- **[Specmatic API](specmatic.md)** - Specmatic integration API reference (functions, classes, integration points)
- **[Telemetry](telemetry.md)** - Opt-in analytics and privacy guarantees
Expand Down
1 change: 1 addition & 0 deletions docs/reference/commands.md
Original file line number Diff line number Diff line change
Expand Up @@ -197,6 +197,7 @@ specfact [OPTIONS] COMMAND [ARGS]...
- `--help`, `-h` - Show help message and exit
- `--help-advanced`, `-ha` - Show all options including advanced configuration (progressive disclosure)
- `--no-banner` - Hide ASCII art banner (useful for CI/CD)
- `--debug` - Enable debug mode: show debug messages in the console and write them (plus structured operation metadata) to `~/.specfact/logs/specfact-debug.log`. See [Debug Logging](debug-logging.md).
- `--verbose` - Enable verbose output
- `--quiet` - Suppress non-error output
- `--mode {cicd|copilot}` - Operational mode (default: auto-detect)
Expand Down
169 changes: 169 additions & 0 deletions docs/reference/debug-logging.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,169 @@
---
layout: default
title: Debug Logging
permalink: /debug-logging/
---

<!-- markdownlint-disable-next-line MD025 -->
# Debug Logging

When you run SpecFact CLI with the global `--debug` flag, the CLI writes debug output to your **console** and to a **rotating log file** under your user directory. This helps diagnose I/O, API, and template issues without cluttering normal output.

## For Users

### Enabling Debug Mode

Pass `--debug` before any command:

```bash
specfact --debug init
specfact --debug backlog refine --adapter ado --project my-project
specfact --debug plan select
```

Debug output appears in the terminal and is also appended to a log file.

### Where Logs Are Written

| Location | Purpose |
|---------|---------|
| **Console** | Same debug messages you see in the terminal (Rich formatting). |
| **`~/.specfact/logs/specfact-debug.log`** | Rotating log file (plain text). Created on first use when `--debug` is set. Directory is created with mode `0o755` if missing. |

- **Path**: `~` is your home directory (e.g. `/home/you` on Linux, `C:\Users\you` on Windows).
- **Rotation**: The file rotates at 5 MB and keeps up to 5 backup files (`specfact-debug.log.1`, …).
- **Scope**: User-level only; not tied to a specific repo or bundle.

### What Gets Logged

When `--debug` is on, the CLI logs:

1. **Debug messages**
Any line emitted via the internal `debug_print()`: template resolution steps, path discovery, fallbacks (e.g. in `specfact init`). Each line is prefixed with a **timestamp** and **caller** (module and function) for context.

2. **Structured operation metadata**
One JSON line per operation, with:
- **operation** – Type (e.g. `api_request`, `file_read`, `template_resolution`).
- **target** – Path or URL (sensitive parts redacted).
- **status** – Result (e.g. `success`, HTTP status, `error`, `prepared`, `finished`, `failed`).
- **caller** – Module and function that logged the operation (for context).
- **error** – Optional error message on failure.
- **extra** – Optional extra fields (redacted); for API calls may include payload (sanitized), response, reason.

**Log format**: Every line in the debug log file starts with a timestamp (`YYYY-MM-DD HH:MM:SS`), then a pipe and the message or structured JSON. Narrative lines include caller (module:function) before the message. File operations log status `prepared`/`finished`/`failed`; API operations log operation, URL (redacted), payload (sanitized), response, status, error, and reason where applicable.

**Redaction**: URLs, paths, and `extra` are passed through `LoggerSetup.redact_secrets` so tokens and secrets are masked in the log file.

### What Is Logged by Component

| Component | Operations / events logged (when `--debug`) |
|----------|---------------------------------------------|
| **auth azure-devops** | Start, success (PAT or OAuth), or error; key steps (OAuth flow, device code) when `--debug` is on. |
| **init** | Template resolution: paths tried, success/failure, fallbacks (e.g. development path, package path, `importlib` fallbacks). |
| **backlog refine** | File read for import: path, success/error (e.g. `--import-from-tmp`). File write for export: path, success/error (e.g. `--export-to-tmp`). |
| **Azure DevOps adapter** | WIQL request (redacted URL, method, status); Work Items GET (redacted URL, status); Work Items PATCH (redacted URL, status); on failure, error snippet. |
| **GitHub adapter** | API request/response (redacted URL, method, status); on failure, redacted error snippet. |

### Example Log Snippets

**Plain debug line (from `debug_print`; timestamp and caller prefixed):**

```text
2025-01-28 14:30:00 | specfact_cli.commands.init:run | Debug: Trying development path: /path/to/templates
```

**Structured operation (from `debug_log_operation`; timestamp prefixed by formatter):**

```text
2025-01-28 14:30:01 | debug_log_operation {"operation": "file_read", "target": "/path/to/export.md", "status": "success", "caller": "specfact_cli.commands.backlog_commands:export_backlog"}
2025-01-28 14:30:02 | debug_log_operation {"operation": "ado_wiql", "target": "https://dev.azure.com/***/***/_apis/...", "status": "200", "caller": "specfact_cli.adapters.ado:..."}
2025-01-28 14:30:03 | debug_log_operation {"operation": "template_resolution", "target": "/usr/lib/.../templates/backlog", "status": "success", "caller": "specfact_cli.commands.init:..."}
```

### Troubleshooting With Debug Logs

1. Run the failing command with `--debug`:

```bash
specfact --debug <command> <args>
```

2. Reproduce the issue, then open `~/.specfact/logs/specfact-debug.log`.
3. Look for:
- **template_resolution** – Where `init` looked for templates and whether it succeeded.
- **file_read** / **file_write** – Paths and success/error for backlog export/import.
- **ado_wiql**, **ado_get**, **ado_patch** – ADO API calls (URLs redacted, status/error present).
- **api_request** – GitHub (or other) API calls with status and optional error.

See also [Troubleshooting](../guides/troubleshooting.md).

---

## For Developers

### Debug log standard (apply consistently)

Debug logs are **critical for anomaly analysis, unexpected errors/failures, reporting, and bug reports**. Every debug log must follow the same pattern so logs are useful like in a regular production tool—**no single-line INFO-style entries**; every significant operation must provide **full context**.

**Required pattern for every significant operation:**

| Phase | What to log | Example status / extra |
|-------|----------------|-------------------------|
| **Started / prepared** | Once when the operation begins | `status=started` or `prepared`; `target`; `extra` (e.g. flow, method, cache) |
| **Progress / attempt** | For each distinct step (if multi-step) | `status=attempt`; `extra.method`, `extra.reason` (what was tried) |
| **Outcome** | Exactly once when the operation ends | **Success**: `status=success` (or HTTP status); `extra` (method, cache, reason). **Failure**: `status=failed` or `error`; `error=<message>`; `extra.reason` |

**Minimum content:**

- **Every line**: timestamp (automatic), caller (automatic or explicit).
- **Structured lines**: `operation`, `target` (redacted), `status`; when applicable: `error`, `extra` (payload, response, reason, method, cache—sanitized).

**Apply everywhere:** Auth flows, file I/O, API calls, template resolution, and any operation that can fail or affect behavior. Reference: `auth azure-devops` (started → cache_prepared → attempt interactive_browser → success/fallback → attempt device_code → success/failed → success token_stored).

### Runtime API

- **`specfact_cli.runtime.set_debug_mode(debug: bool)`**
Turn global debug mode on or off (e.g. from the CLI callback when `--debug` is set).

- **`specfact_cli.runtime.is_debug_mode() -> bool`**
Returns whether debug mode is currently on.

- **`specfact_cli.runtime.init_debug_log_file()`**
Ensures the debug log file under `~/.specfact/logs` is created and the file handler is set up. Called by the CLI when `--debug` is True so the first write goes to the file immediately.

- **`specfact_cli.runtime.debug_print(*args, **kwargs)`**
If debug is on: prints to the configured Rich console and appends a plain-text line to `~/.specfact/logs/specfact-debug.log` (args only; no Rich markup in the file). If debug is off: no-op.

- **`specfact_cli.runtime.debug_log_operation(operation, target, status, error=None, extra=None, caller=None)`**
If debug is on: writes one JSON line to the debug log file with `operation`, `target`, `status`, optional `error`, optional `extra`, and `caller` (inferred if not provided). `target` and `extra` are redacted via `LoggerSetup.redact_secrets`. If debug is off: no-op. Follow the debug log standard: log started/prepared → attempt → success/failed with reason.

### User-Level Log Directory

- **`specfact_cli.common.logger_setup.get_specfact_home_logs_dir() -> str`**
Returns `os.path.expanduser("~/.specfact/logs")`, creating the directory with `mode=0o755` and `exist_ok=True` on first use. Use this if you need the path for the debug log or related files.

### When to Use What

- **`debug_print(...)`**
For human-oriented messages (template paths, “trying X”, “using Y”). Shown in console and written as a single plain line to the log file.

- **`debug_log_operation(...)`**
For machine-friendly operation records (API calls, file I/O, template resolution result). Always use for URLs or paths; redaction is applied to `target` and `extra`.

### Adding New Log Points

1. **Follow the debug log standard**
For each significant operation: log **started/prepared** → **attempt** (if multi-step) → **success** or **failed** with **reason/error**. Include operation, target, status, error, extra (payload/response/reason—sanitized). Never log only one line for an operation that can succeed or fail; always log outcome and reason.

2. **New adapter or command**
When `is_debug_mode()` is True, call `debug_log_operation(operation, target, status, error=..., extra=..., caller=...)` at start, at each attempt, and at outcome. Use clear operation names (e.g. `ado_wiql`, `file_read`, `template_resolution`). For file ops: prepared → finished/failed. For API ops: attempt → success/failed with payload (sanitized), response, reason.

3. **New debug messages**
Use `debug_print(...)` for narrative steps; they will appear in console and in `specfact-debug.log` as plain text. Prefer pairing with `debug_log_operation` so the log has both human-readable and machine-friendly context.

4. **Sensitive data**
Pass URLs/paths/tokens only as `target` or inside `extra`; they are redacted before being written to the log file.

### Relation to Other Logging

- **`~/.specfact/logs/`** is for the **global** `--debug` session log only (`specfact-debug.log`). It is **not** the same as bundle-specific `.specfact/projects/<bundle>/logs/` (used for other runtime/agent logs). See [Directory Structure](directory-structure.md).
2 changes: 2 additions & 0 deletions docs/reference/directory-structure.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,8 @@ All SpecFact artifacts are stored under `.specfact/` in the repository root. Thi
- **Clear separation**: Plans (versioned) vs reports (ephemeral)
- **CLI-first**: All artifacts are local, no cloud storage required

**User-level debug logs**: When you run with `--debug`, the CLI also writes a rotating debug log under your home directory: `~/.specfact/logs/specfact-debug.log`. This is separate from repo-level `.specfact/` and is used only for global debug output. See [Debug Logging](debug-logging.md).

## Canonical Structure

```bash
Expand Down
Loading