QEP idea: Contributor registry, AI-use attestation, and a PR gate

[mmcky]

Status: Draft for discussion · Author: @mmcky · Created: 2026-08-14 · Proposed type: standard
This issue carries the decision only. The per-PR audit behind it, the detailed design, and the build plan are recorded internally in QuantEcon/infrastructure#5.

Summary

Adopt a Code of AI Use for QuantEcon: AI-assisted contributions are welcome, a human must submit and own them, and meaningful AI involvement must be disclosed. Back it with a public contributor registry — a whitelist of GitHub logins who have attested to the Code — and a pull-request gate, piloted on QuantEcon.py, that holds PRs from unregistered external authors with a friendly comment rather than reviewing them. The target is unattended AI crawlers that sweep public issue trackers and mass-file unsolicited PRs: attesting is a step no unattended crawler takes, while costing a genuine contributor two minutes, once. The goal is not to ban AI-assisted contributions — we use these tools ourselves — but to attach an accountable human to every PR we spend review time on.

Motivation

An audit of the 18 open pull requests on QuantEcon.py (2026-08-14) found that 11–12 involve AI authorship, and most carry no disclosure. Eight have explicit machine-readable attribution — bot accounts, Copilot co-author trailers, maintainer PRs with Claude trailers, and one exemplary disclosed-and-human-reviewed submission. Three more carry strong agent fingerprints with nothing declared, including two competing agent-written fixes for the same issue filed on the same day by different first-time authors, and one whose description presents a test run that stopped at 47% when the agent's session ended as if it were validation.

The pattern reads as issue-crawling automation rather than individual experimentation: first-time authors converging on freshly filed structured issues within days, echoing our issue text verbatim, and in two cases colliding on the same issue. The costs are already concrete — duplicated review effort, CI minutes spent on unverifiable claims, and no way to tell which PR author will still be present to answer review comments. Review attention is the binding constraint on this project, and the volume rises with agent capability.

GitHub gives us no standing fix. Interaction limits cap at six months, need manual renewal, and are all-or-nothing — they also block issues and comments from exactly the people whose bug reports we want. There is no native setting restricting who may open a pull request on a public repository.

Why does this need a QEP?

Affects more than one repository

Proposed change (optional)

Proposal

1. The Code of AI Use

AI-assisted contributions are welcome at QuantEcon; our maintainers use these tools daily and in the open. In exchange, every contributor accepts six norms:

1. A human submits. Unattended agents and automated pipelines must not open PRs or crawl our issue tracker for work. A registered human chooses the task, reviews the result, and submits it under their own account. Where a tool pipeline generates the change, the human operating it is the contributor and owns what it files.
2. Disclose. If AI tools meaningfully contributed to the code, tests, or text of a PR, say so — tick the PR-template disclosure box and leave a machine-readable line in the commit message or description (Co-Authored-By: <tool>, or AI-Model: / AI-Usage: naming what the tool did and what you verified).
3. Own it. You are the author. You have read every line, you can explain every line, and you — not an agent — respond to review. Submitting model output you have not reviewed shifts your work onto volunteer maintainers.
4. Verify before you claim. Only state that tests pass, coverage rose, or benchmarks improved if you ran them to completion. Partial runs are fine to submit, described as what they are.
5. Don't duplicate or farm. Check for an existing open PR before starting on an issue, and don't mass-submit generated PRs across repositories to build a contribution record. Issues labelled good first issue are reserved for humans learning the codebase.
6. Accept the consequences. Undisclosed AI involvement, once pointed out, is fixed by amending the PR. Repeated or willful non-disclosure leads to removal from the registry and closure of open PRs.

2. The registry and the gate

• The registry is a public list of GitHub logins who have attested to the Code. Every addition and removal is a visible, reviewable, revertible Git operation on a public file — no external service, no data beyond what is already public.
• The gate holds; it does not reject. A PR from an author who is not an org member, not registered, not a previous contributor to that repository, and not an allow-listed bot gets one friendly comment linking the Code and the registration path, plus a pending-registration label. It is released the moment the author registers, and closed after a grace window with an explicit invitation to register and return.
• Nobody with merged work hits the gate cold. Prior contributors to a covered repository are exempt automatically; the registry additionally covers contributors arriving from another QuantEcon repository.
• Registration is not CI trust. It grants review attention only; approval for workflow runs from outside collaborators stays required.
• This is an attestation mechanism, not a Turing test. Its value is that undisclosed AI use after attestation is an unambiguous, enforceable violation rather than a judgement call.

3. Enforcement

Undisclosed AI involvement in a PR draws a maintainer comment citing the Code and an undisclosed-ai label; the contributor amends the PR, and nothing else follows the first time. A repeat, or a refusal to amend, closes the PR without further review. A pattern — a third occurrence, or duplicate mass-submissions — removes the entry from the registry as a one-line revert PR, with re-registration possible after six months. Egregious abuse is an org-level block, the tool GitHub already gives us. Every step is a normal Git operation on a public file: auditable, reversible, and boring.

Non-goals

• Not a ban on AI-assisted contributions. QuantEcon maintainers use Claude and Copilot openly, including delegated agent PRs.
• Not a CLA. The attestation covers conduct and disclosure only — no copyright assignment, no license grant, and no Developer Certificate of Origin line in v1.
• Not org-wide on acceptance. QuantEcon.py is the pilot; extending the gate to QuantEcon.jl and the lecture repos is a separate decision on pilot evidence.

Alternatives considered

• Do nothing. Volume rises with agent capability, and review attention is what we are short of.
• Interaction limits. Native but temporary, manually renewed, and they silence the bug reports we want.
• Hard-close, no registry. Simplest to build, worst first impression, and no path back in for good-faith outsiders.
• A blocking status check instead of a hold (CLA-assistant UX). Keeps every PR open, so the queue fills with permanently-blocked spam — the problem we are solving.
• Org-members-only PRs. Kills the community pipeline — GSoC, students, one-off domain experts — that produces some of our best contributions.

Prior art

LLVM's AI Tool Use Policy allows AI assistance, requires labelling substantial AI content, holds the contributor fully accountable as author, and reserves good first issue tickets for human newcomers. This proposal sits at the LLVM end of the spectrum, adding attestation so the norms are accepted before the first PR rather than discovered after it. QEMU declines AI-generated contributions outright on provenance grounds — defensible, but it would cover our own workflow. Harder-line bans elsewhere (Gentoo, curl) were responses to the same review-attention problem, reached later and less gently.

Adoption

Acceptance fixes the Code of AI Use as QuantEcon policy and authorises the registry and gate. Any implementation must honour four guarantees: the registry stays a public, plain-text, auditable file, so every entry and removal is a reviewable Git operation; no prior contributor to a covered repository is ever gated cold; the gate never executes pull-request code, since it runs with repository write permissions; and a held PR is held, not rejected — one friendly comment, a clear path forward, and release the moment the author registers.

Adoption is pilot-first: QuantEcon.py only, evaluated over four weeks against zero complaints from legitimate first-time contributors, drive-by agent PRs held at the door, and registration latency under 24 hours. Disclosure norms land first and independently — the PR-template checkbox, the CONTRIBUTING.md trailer conventions, and the policy link in the structured issue templates agents already consume — and carry value even if no gate is ever enabled.

Registration mechanism, registry location, grace-window value, and rollout sequencing are implementation choices, not part of this decision; they are tracked internally in QuantEcon/infrastructure#5.

[mmcky]

Update: narrowing this to policy, and piloting GitHub's native controls first

Two things have changed the shape of this proposal since the draft above.

GitHub shipped controls that did not exist when this was written. Since February 2026 a repository can restrict pull-request creation to collaborators or turn pull requests off entirely. And since 6 August 2026 an organisation can cap how many concurrent open pull requests a user without write access may have — configured org-wide, with draft pull requests excluded from the count and a bypass list of up to 100 trusted contributors who can exceed it without being granted write access. It is a concurrency cap rather than a rate limit, so it cannot express "one a week", but it is free, native, needs no maintenance, and is completely invisible to a contributor who has one pull request open.

And the registry's value is smaller than the original draft implied. The norms bind every contributor whether or not they attest to them — like the Code of Conduct, this is project policy, not a contract that only signatories are held to. A registry therefore notifies rather than binds. Meanwhile a gate puts a bot comment and a label on a newcomer's first contribution, which is the worst possible moment to introduce friction, and it is standing infrastructure running with repository write permissions.

So the proposal splits in two.

QEP-5 becomes the AI contribution policy only. The Code of AI Use, disclosure as a machine-readable marker rather than a narrative, and a short enforcement path that starts with a rectification comment and ends — rarely — at GitHub's own organisation block. It is policy, not infrastructure: it can take effect on acceptance with nothing to build. It will arrive shortly as a draft PR against this repository.

The pilot uses GitHub's own controls. Org-level pull-request limits with the bypass list, alongside a disclosure checkbox in the pull-request template, the trailer convention in CONTRIBUTING.md, and a link to the policy from the structured issue templates that agents are already consuming. Four weeks on QuantEcon.py.

What we measure is the residue, not the reduction. A per-account cap does nothing about the pattern the audit actually found: many distinct first-time accounts filing a single pull request each. Two of the audited pull requests were competing fixes for the same issue from different first-time authors, one apiece — both would pass a cap of one untouched. So the question the pilot answers is not "did volume fall" but "what is left over after the free controls are on".

If that residue is small, QEP-5 is the whole answer and we have built nothing. If it is not, a contributor registry gets argued then, on that evidence rather than on this audit — either as an amendment to QEP-5 or as a QEP of its own covering the maintenance and procedures a registry actually needs.

The registry and gate design is not lost. It is drafted in full, along with the per-pull-request audit behind this issue, and recorded internally in QuantEcon/infrastructure#5.