Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
10 changes: 10 additions & 0 deletions .gitattributes
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
# Binary benchmark artifacts
*.pptx binary
*.png binary
*.ttf binary
*.ttc binary
*.mp4 binary

# Preserve licensed text and prompt-significant whitespace verbatim.
acidslide-v1/benchmark/fonts/licenses/** -whitespace
acidslide-v1/benchmark/prompts/variants/** -whitespace
50 changes: 50 additions & 0 deletions .github/ISSUE_TEMPLATE/benchmark-case.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
name: Propose a benchmark case
description: Add a slide behavior that generated PowerPoint artifacts should prove.
title: "[case]: "
labels: ["benchmark-case"]
body:
- type: markdown
attributes:
value: |
Thanks for making the corpus harder. Requirements must come from a public prompt, reference, or allowed asset — never from reverse-engineering the gold deck.
- type: textarea
id: failure
attributes:
label: What fails today?
description: Describe the generated deck behavior a screenshot or current check misses.
placeholder: A chart looks correct but is stored as a picture, so its data cannot be edited.
validations:
required: true
- type: textarea
id: requirement
attributes:
label: Proposed requirement
description: State the observable visual and/or native artifact requirement.
validations:
required: true
- type: dropdown
id: layer
attributes:
label: Primary layer
options:
- Native OOXML / package structure
- Rendered visual fidelity
- Both
validations:
required: true
- type: textarea
id: evidence
attributes:
label: Public source and evidence plan
description: Link the prompt/reference/asset source and describe a positive control plus a single-fault negative fixture.
validations:
required: true
- type: checkboxes
id: terms
attributes:
label: Contribution checks
options:
- label: I can share every attached artifact under the repository license or its stated compatible license.
required: true
- label: This proposal does not contain private decks, credentials, customer data, or proprietary fonts.
required: true
8 changes: 8 additions & 0 deletions .github/ISSUE_TEMPLATE/config.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
blank_issues_enabled: true
contact_links:
- name: Security vulnerability
url: https://github.com/aronchick/gloss/security/advisories/new
about: Report security issues privately, never in a public issue.
- name: Contribution guide
url: https://github.com/aronchick/gloss/blob/main/CONTRIBUTING.md
about: Read the evidence and fixture requirements before opening a pull request.
37 changes: 37 additions & 0 deletions .github/ISSUE_TEMPLATE/evidence-challenge.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
name: Challenge evidence or a claim
description: Question provenance, a mutation operator, a scoring assumption, or release wording.
title: "[evidence]: "
labels: ["evidence"]
body:
- type: input
id: claim
attributes:
label: Claim or evidence ID
placeholder: gloss-technical-preview-2026-07-22 or a repository path
validations:
required: true
- type: textarea
id: challenge
attributes:
label: What does the current evidence fail to establish?
description: Separate observed facts from your interpretation.
validations:
required: true
- type: textarea
id: falsify
attributes:
label: How can we test or falsify it?
description: Propose a fixture, review, independent reproduction, or wording change.
validations:
required: true
- type: dropdown
id: urgency
attributes:
label: Release impact
options:
- Blocks any public claim
- Blocks an official benchmark freeze
- Improves the technical preview
- Documentation only
validations:
required: true
46 changes: 46 additions & 0 deletions .github/ISSUE_TEMPLATE/grader-bug.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
name: Report a grader gap
description: Reproduce a false positive, false negative, crash, or nondeterministic result.
title: "[grader]: "
labels: ["grader"]
body:
- type: textarea
id: behavior
attributes:
label: Observable behavior
description: What did Gloss report, and what should it have reported?
validations:
required: true
- type: textarea
id: reproduce
attributes:
label: Minimal reproduction
description: Include exact `uv` commands, tier, platform, and a minimal safe fixture or repository path.
render: shell
validations:
required: true
- type: input
id: commit
attributes:
label: Gloss commit
placeholder: 40-character Git SHA
validations:
required: true
- type: dropdown
id: impact
attributes:
label: Result
options:
- False positive
- False negative
- Crash or incomplete verification
- Nondeterministic result
- Other
validations:
required: true
- type: checkboxes
id: safety
attributes:
label: Safety
options:
- label: This is not a security vulnerability. I will use the private process in SECURITY.md for vulnerabilities.
required: true
23 changes: 23 additions & 0 deletions .github/pull_request_template.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
## What changed

<!-- Name the observable gap and the smallest change that closes it. -->

## Evidence

<!-- Link fixtures, reports, prompt requirements, screenshots, or source records. Do not invent model results or generation provenance. -->

## Verification

```text
Exact commands run:

Results:
```

## Release truth

- [ ] Public claims in this pull request are traceable to committed evidence.
- [ ] I did not imply that Draft candidate checks or local scores are official leaderboard results.
- [ ] Grader behavior changes include focused tests or fixtures.
- [ ] I checked for private decks, credentials, customer data, and proprietary fonts.
- [ ] Documentation reflects any new command, contract, or contribution path.
127 changes: 127 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,127 @@
name: CI

on:
push:
branches: [main]
tags: ["acidslide-v*"]
pull_request:

permissions:
contents: read

jobs:
grader:
runs-on: ubuntu-22.04
defaults:
run:
working-directory: acidslide-v1/grader
steps:
- uses: actions/checkout@v4
- uses: astral-sh/setup-uv@v6
with:
version: "0.8.4"
enable-cache: true
- run: uv sync --extra dev --locked
- run: uv run ruff check .
- run: uv run ruff format --check .
- run: uv run mypy acidslide tests
- run: uv run pytest --cov=acidslide --cov-report=term-missing --cov-fail-under=85
- run: uv build
- run: uv run pip-audit
- name: Verify self-contained wheel data
run: |
uv run python -m zipfile -l dist/acidslide-1.0.0a1-py3-none-any.whl | grep 'acidslide/data/benchmark/checklist/deck.yaml'
uv run python -m zipfile -l dist/acidslide-1.0.0a1-py3-none-any.whl | grep 'acidslide/data/schemas/ecma-376/xsd-transitional/pml.xsd'

corpus:
runs-on: ubuntu-22.04
defaults:
run:
working-directory: acidslide-v1/grader
steps:
- uses: actions/checkout@v4
- uses: astral-sh/setup-uv@v6
with:
version: "0.8.4"
enable-cache: true
- run: uv sync --extra dev --locked
- run: uv run python ../benchmark/validate_corpus.py
- name: Require the complete frozen release on version tags
if: startsWith(github.ref, 'refs/tags/acidslide-v')
run: |
test "$GITHUB_REF_NAME" = "acidslide-v1.0.0"
uv run python ../benchmark/validate_corpus.py --release

launch:
runs-on: ubuntu-22.04
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: "22"
- name: Install launch media inspector
run: |
sudo apt-get update
sudo apt-get install --yes ffmpeg
- run: node launch/verify-launch.mjs

container:
runs-on: ubuntu-22.04
steps:
- uses: actions/checkout@v4
- run: docker build --platform linux/amd64 --tag acidslide-ci acidslide-v1
- name: Verify frozen runtime and font inventory
run: |
docker run --rm --platform linux/amd64 --network none acidslide-ci --version
docker run --rm --platform linux/amd64 --network none \
--entrypoint /bin/sh acidslide-ci -ec '
test "$(date -u +%Y-%m-%dT%H:%M:%SZ)" = "2025-01-01T00:00:00Z"
libreoffice --version | grep -F "LibreOffice 7.3.7.2"
pdftoppm -v 2>&1 | grep -F "22.02.0"
test "$(fc-list -f "%{file}\n" | sort -u | wc -l)" -eq 26
'
- name: Reference deck passes quarantine and ECMA-376 validation
run: |
output=$(docker run --rm --platform linux/amd64 --network none \
--volume "$PWD/acidslide-v1/benchmark/deck/gold/acidslide-v1-gold.pptx:/input.pptx:ro" \
acidslide-ci validate /input.pptx 2>&1)
grep -F "Quarantine passed" <<<"$output"
grep -F "ECMA-376 schema: valid" <<<"$output"

service:
runs-on: ubuntu-22.04
defaults:
run:
working-directory: acidslide-v1/service
steps:
- uses: actions/checkout@v4
- uses: astral-sh/setup-uv@v6
with:
version: "0.8.4"
enable-cache: true
- run: uv sync --extra dev --locked
- run: uv run ruff check .
- run: uv run ruff format --check .
- run: uv run mypy acidslide_service tests
- run: uv run pytest --cov=acidslide_service --cov-report=term-missing --cov-fail-under=85
- run: uv run pip-audit

service-images:
runs-on: ubuntu-22.04
steps:
- uses: actions/checkout@v4
- name: Build canonical grader base
run: |
docker build --platform linux/amd64 --provenance=false \
--tag acidslide/grader:1.0.0 acidslide-v1
- name: Build API, worker, and quarantine images
run: |
docker build --platform linux/amd64 --provenance=false \
--build-context schemas=acidslide-v1/schemas \
--target api --tag acidslide/service-api:1.0.0 acidslide-v1/service
docker build --platform linux/amd64 --provenance=false \
--build-context schemas=acidslide-v1/schemas \
--target worker --tag acidslide/service-worker:1.0.0 acidslide-v1/service
docker build --platform linux/amd64 --provenance=false \
--build-context schemas=acidslide-v1/schemas \
--target quarantine-job --tag acidslide/quarantine:1.0.0 acidslide-v1/service
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -21,3 +21,4 @@ htmlcov/

# Large reference docs (download separately)
ECMA/
.gstack/
Loading
Loading