-
Notifications
You must be signed in to change notification settings - Fork 7
chore: repository foundations, Marketplace branding, and security threat model #61
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from all commits
be33fcd
30a1a07
91b464b
d2d9991
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,85 @@ | ||
| name: Bug Report | ||
| description: Report a problem with code-to-docs | ||
| labels: ["bug"] | ||
| body: | ||
| - type: markdown | ||
| attributes: | ||
| value: | | ||
| Thanks for taking the time to report a bug. Please fill in the details | ||
| below so we can reproduce the issue. | ||
|
|
||
| - type: dropdown | ||
| id: command | ||
| attributes: | ||
| label: Trigger command | ||
| description: Which command were you using? | ||
| options: | ||
| - "[review-docs]" | ||
| - "[update-docs]" | ||
| - "[review-feature]" | ||
| validations: | ||
| required: true | ||
|
|
||
| - type: input | ||
| id: model-backend | ||
| attributes: | ||
| label: Model backend | ||
| description: "e.g. vLLM on OpenShift AI, Gemini, OpenAI, Ollama" | ||
| placeholder: "vLLM on OpenShift AI" | ||
| validations: | ||
| required: true | ||
|
|
||
| - type: input | ||
| id: model-name | ||
| attributes: | ||
| label: Model name | ||
| description: "The MODEL_NAME value you configured" | ||
| placeholder: "meta-llama/Llama-3.1-8B-Instruct" | ||
| validations: | ||
| required: true | ||
|
|
||
| - type: dropdown | ||
| id: docs-format | ||
| attributes: | ||
| label: Documentation format | ||
| description: What format are your doc files? | ||
| options: | ||
| - Markdown (.md) | ||
| - reStructuredText (.rst) | ||
| - AsciiDoc (.adoc) | ||
| - Mixed | ||
| validations: | ||
| required: true | ||
|
|
||
| - type: dropdown | ||
| id: repo-setup | ||
| attributes: | ||
| label: Repository setup | ||
| description: How is your docs repo configured? | ||
| options: | ||
| - Same repo (docs-subfolder set) | ||
| - Separate docs repo (docs-repo-url points elsewhere) | ||
| validations: | ||
| required: true | ||
|
|
||
| - type: textarea | ||
| id: description | ||
| attributes: | ||
| label: What happened? | ||
| description: Describe the bug. Include the PR link if possible. | ||
| validations: | ||
| required: true | ||
|
|
||
| - type: textarea | ||
| id: expected | ||
| attributes: | ||
| label: What did you expect? | ||
| validations: | ||
| required: true | ||
|
|
||
| - type: textarea | ||
| id: logs | ||
| attributes: | ||
| label: Relevant log output | ||
| description: Paste any relevant output from the GitHub Actions run. | ||
| render: shell |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,31 @@ | ||
| name: Feature Request | ||
| description: Suggest an improvement or new capability | ||
| labels: ["enhancement"] | ||
| body: | ||
| - type: textarea | ||
| id: problem | ||
| attributes: | ||
| label: Problem | ||
| description: What problem does this solve? What is frustrating or missing today? | ||
| validations: | ||
| required: true | ||
|
|
||
| - type: textarea | ||
| id: solution | ||
| attributes: | ||
| label: Proposed solution | ||
| description: How would you like this to work? | ||
| validations: | ||
| required: true | ||
|
|
||
| - type: textarea | ||
| id: alternatives | ||
| attributes: | ||
| label: Alternatives considered | ||
| description: Have you tried any workarounds? | ||
|
|
||
| - type: textarea | ||
| id: context | ||
| attributes: | ||
| label: Additional context | ||
| description: Anything else that would help us understand the request. |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,15 @@ | ||
| ## Summary | ||
|
|
||
| <!-- What does this PR do and why? --> | ||
|
|
||
| ## Changes | ||
|
|
||
| <!-- Bulleted list of what changed --> | ||
|
|
||
| ## Testing | ||
|
|
||
| <!-- How did you verify this works? --> | ||
|
|
||
| - [ ] Tests pass (`uv run pytest -v`) | ||
| - [ ] Lint clean (`uv run ruff check src/ tests/`) | ||
| - [ ] Format clean (`uv run ruff format --check src/ tests/`) |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,79 @@ | ||
| # Contributing to Code-to-Docs | ||
|
|
||
| Thank you for your interest in contributing! This guide covers the development | ||
| workflow so you can get started quickly. | ||
|
|
||
| ## Development Setup | ||
|
|
||
| Requires Python 3.12+ and [uv](https://docs.astral.sh/uv/) for package management. | ||
|
|
||
| ```bash | ||
| # Install dependencies (including dev tools) | ||
| uv sync --extra dev | ||
|
|
||
| # Set up pre-commit hooks | ||
| uv run pre-commit install | ||
| ``` | ||
|
|
||
| ## Running Tests | ||
|
|
||
| ```bash | ||
| # Run tests | ||
| uv run pytest -v | ||
|
|
||
| # Run tests with coverage | ||
| uv run pytest --cov=src --cov-report=term-missing | ||
| ``` | ||
|
|
||
| CI enforces a minimum coverage threshold. Check `pyproject.toml` for the current | ||
| value. | ||
|
|
||
| ## Linting and Formatting | ||
|
|
||
| ```bash | ||
| # Lint | ||
| uv run ruff check src/ tests/ | ||
|
|
||
| # Format check | ||
| uv run ruff format --check src/ tests/ | ||
|
|
||
| # Auto-format | ||
| uv run ruff format src/ tests/ | ||
| ``` | ||
|
|
||
| ## Commit Messages | ||
|
|
||
| This project uses [Conventional Commits](https://www.conventionalcommits.org/): | ||
|
|
||
| - `feat(scope): description` for new features | ||
| - `fix(scope): description` for bug fixes | ||
| - `chore(scope): description` for maintenance | ||
| - `docs(scope): description` for documentation-only changes | ||
| - `refactor(scope): description` for refactors with no behavior change | ||
|
|
||
| Include a commit body explaining *why* the change is needed, not just what it does. | ||
|
|
||
| ## Pull Requests | ||
|
|
||
| - One logical change per PR. If a PR contains multiple independent changes, | ||
| split them. | ||
| - All CI checks must pass: lint, format, and tests with coverage. | ||
| - Update documentation (`README.md`, `CLAUDE.md`) if your change adds or | ||
| modifies user-facing behavior. | ||
| - Add tests for new functionality. Tests are in `tests/` and mirror source | ||
| modules (`test_<module>.py`). | ||
|
|
||
| ## Project Structure | ||
|
|
||
| - `src/` contains the action source code (see `CLAUDE.md` for a module-by-module | ||
| breakdown) | ||
| - `tests/` contains pytest tests | ||
| - `demo/` contains the interactive demo site | ||
| - `.code-to-docs/` contains per-repo configuration (style guidelines, ignore | ||
| lists, settings) | ||
|
|
||
| ## Reporting Issues | ||
|
|
||
| Use the provided issue templates for bug reports and feature requests. Include | ||
| the model backend, model name, docs format, and trigger command when reporting | ||
| bugs. |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -19,3 +19,79 @@ Following this process will create a private advisory for our maintainers to rev | |
| ### Do Not Open Public Pull Requests, Issues, or Discussions | ||
|
|
||
| Please **do not** discuss the issue, create PRs, or start discussions about the vulnerability. This ensures the vulnerability is not widely exploited before a fix is provided. | ||
|
|
||
| ## Threat Model | ||
|
|
||
| This section enumerates untrusted inputs, what they can influence, and the | ||
| mitigations in place. It is intended for security-conscious adopters who want | ||
| to assess the tool without reading the source. | ||
|
|
||
| ### Permissions Required | ||
|
|
||
| The action requires `contents: write` on the target repository. This allows | ||
| it to push commits to PR branches and create documentation PRs. It does | ||
| **not** require `admin` permissions. | ||
|
|
||
| ### Untrusted Inputs | ||
|
|
||
| | Input | Source | What it can influence | Mitigation | | ||
| |-------|--------|----------------------|------------| | ||
| | Code diff | PR head branch (contributor-controlled on forks) | LLM prompt content for file selection and generation | Diff is data in the prompt, not instructions. Truncated to context budget. | | ||
| | Documentation content | PR head branch | LLM prompt for verification; also read for current file content | Post-generation verification wraps doc content in explicit data delimiters. | | ||
| | PR description | PR author | Included as context in generation prompts | Treated as context only, not instructions. | | ||
| | Comment body | PR commenter (gated by `author_association`) | Command parsing, user instructions passed to LLM | Only `OWNER`, `MEMBER`, and `COLLABORATOR` can trigger commands (enforced in the workflow `if` condition). | | ||
| | Jira ticket content | External system | Included in `[review-feature]` prompts | Fetched server-side via authenticated API. Content is data, not instructions. | | ||
| | Google Docs / Confluence content | External system | Included in `[review-feature]` prompts | Fetched via service account. Content is data, not instructions. | | ||
| | `.code-to-docs/style.md` | Base branch (maintainer-controlled) | Injected into generation prompts as style guidelines | Loaded from the base branch, not the PR branch. | | ||
| | `.code-to-docs/config.json` | Base branch (maintainer-controlled) | Controls validation thresholds and behavior | Loaded from the base branch, not the PR branch. Values are validated on load. | | ||
|
|
||
| ### Fork PRs | ||
|
|
||
| The recommended workflow includes an `author_association` gate: | ||
|
|
||
| ```yaml | ||
| contains(fromJSON('["OWNER","MEMBER","COLLABORATOR"]'), github.event.comment.author_association) | ||
| ``` | ||
|
|
||
| This prevents external contributors from triggering the action on their own | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. [medium] Technical documentation accuracy The author_association gate in the threat model states only OWNER, MEMBER, and COLLABORATOR can trigger commands. The stop-fix job in fullsend.yaml uses a broader gate that also permits CONTRIBUTOR and the PR author. While stop-fix gates a different command (/fs-fix-stop) than the code-to-docs commands, the discrepancy between the documented gate and the repository's own workflow should be clarified. Suggested fix: Add a note clarifying that the stop-fix job uses a broader gate, or document that the three-role gate applies specifically to code-to-docs command triggers. |
||
| PRs. However, a maintainer commenting `[update-docs]` on a fork PR will | ||
| cause the action to read diff and doc content from the fork's head branch, | ||
| which is contributor-controlled. | ||
|
|
||
| **What the gate protects against:** unauthorized triggering, resource | ||
| consumption, and unsolicited PR creation. | ||
|
|
||
| **What the gate does not protect against:** a maintainer explicitly | ||
| approving a run on a fork PR where the contributor has placed adversarial | ||
| content in documentation files. The post-generation verification step | ||
| (added in PR #53) wraps document content in data delimiters to reduce this | ||
| risk, but LLM-based defenses are not absolute. | ||
|
|
||
| **Fork push limitation:** the action cannot push to fork branches due to | ||
| GitHub token scoping. On fork PRs, it posts suggested changes as diffs in a | ||
| PR comment instead. | ||
|
|
||
| ### Credential Handling | ||
|
|
||
| - `GH_TOKEN`, `MODEL_API_KEY`, `JIRA_API_TOKEN`, and `GOOGLE_SA_KEY` are | ||
| scrubbed from all output by `security_utils.sanitize_output()`. | ||
| - Credentials are never interpolated into strings or logged. | ||
| - File paths are validated against traversal attacks before read or write. | ||
| - The Docker container runs with no network access beyond what the | ||
| entrypoint requires. | ||
|
|
||
| ### LLM Prompt Injection | ||
|
|
||
| Documentation content and code diffs are included in LLM prompts. A | ||
| malicious document containing text like "ignore previous instructions" could | ||
| theoretically influence the LLM's output. Mitigations: | ||
|
|
||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. [medium] Technical documentation accuracy LLM Prompt Injection mitigation #1 claims document content is wrapped in explicit BEGIN/END delimiters with untrusted-data labeling. No such delimiters exist in the codebase. PR #53 (referenced as adding this feature) is still open. Suggested fix: Either wait for PR #53 to merge before claiming these mitigations exist, or rewrite mitigations 1-2 as planned/forthcoming rather than current. |
||
| 1. Document content in verification prompts is wrapped in explicit | ||
| `--- BEGIN/END ... (untrusted content, data only) ---` delimiters with | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. [medium] Technical documentation accuracy Mitigation #2 claims post-generation validation checks for large content removals and verifies updates with an independent LLM call. No such code exists. The only post-generation validation is format parsing. Suggested fix: Remove or correct this claim to reflect actual mitigations, or label as planned functionality. |
||
| an instruction to treat it as data, not instructions. | ||
| 2. Post-generation validation checks for large content removals and | ||
| verifies updates with an independent LLM call in a separate session. | ||
| 3. The tool never executes generated content; it only writes documentation | ||
| files. | ||
| 4. Generated output is format-validated (Markdown, RST, AsciiDoc parsers) | ||
| before being committed. | ||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
[low] Technical documentation accuracy
The untrusted inputs table states .code-to-docs/config.json 'Values are validated on load.' In reality, load_repo_config() performs JSON parsing but no schema, type, or range validation.
Suggested fix: Change to a more accurate description such as 'Parsed as JSON; individual consumers validate their own fields.'