fix: count a Gmail refusal we cannot read apart from a message that is gone

[yadava5]

Refs #744 — deliberately not Closes; see Scope at the bottom.

_batch_fetch_metadata._send drops any failed sub-request that is neither recognisably rate-limited nor retryable into unreadable, and the client then advances next_page_token past those ids permanently. Right for a message that is genuinely gone; wrong for a quota refusal wearing an error envelope we do not recognise.

The code already records that Gmail's envelope changed shape once — _RATE_LIMIT_REASONS carries RESOURCE_EXHAUSTED because the newer google.rpc shape reports error.status instead of error.errors[].reason. A third shape would reproduce the original defect (a scan that silently returns a fraction of the mailbox and reports success) through a narrower door.

Which of the two options the issue offers

Not "retryable-once". It collides with a shipped, deliberately argued invariant: an unrecognised 403 is more likely a revoked grant than a rate limit, and retrying burns the budget a real rate limit needs. test_an_unrecognised_refusal_is_still_not_retried pins that this change did not smuggle it in.

The distinct count, which is additive and overturns no existing test.

The predicate is not the obvious one, and that is the substance of this PR

The issue thread proposed "incremented when a failure matched neither reason set". Measured against the suite's own fixtures, that is true for three cases:

case	naive predicate	should count?
403 authError — a revoked grant	✅ true	❌ recognised, understood, permanent
404 notFound — the ordinary case	✅ true	❌ every deleted message
403 google.rpc, no readable reason	✅ true	✅ the open door

A counter that increments on every deleted message is non-zero on essentially every real page, so an envelope change would arrive as one more unit in a number already large — the opposite of loud.

The test used instead is "it named no reason we could read, on a status that does not speak for itself": _SELF_EXPLANATORY_STATUSES = {400, 404}. A 404 means the message is gone whatever its body says; a 403 is exactly where a quota refusal and a revoked grant are indistinguishable without a reason.

Scope

unrecognised is a subset of unreadable, carried on MetadataBatch, MessagePage and HistoryPage, on the four API response models, and on InboxPage web-side — plus a warning that names _RATE_LIMIT_REASONS as the thing to extend.

No UI. The issue's bar is "loud instead of quiet", which the log line and the distinct count meet. A surface for it is a separate call, and inventing one here would be scope I was not asked for.

The issue's "cheap next step" — re-run a live 200-message mine and read the per-failure log — is not done and is not doable from here (it needs the owner's mailbox). It is also largely superseded: that step existed to attribute the observed unreadable: 49, and the count now answers the same question without a manual re-run.

Verification

22 tests, up from 18. Five mutations, __pycache__ cleared between each, source restored byte-identical after every one:

mutation	result
the naive "neither reason set" predicate	2 red
drop the self-explanatory-status check	1 red
unrecognised collapses into dropped	1 red
unrecognised refusals get retried	1 red
never record — the fix becomes a no-op	2 red

The second one initially survived. _SELF_EXPLANATORY_STATUSES was a constant nothing graded: every fixture that reached it carried a reason and returned early. Added a 404 and a 400 with no parseable body, which is the case the constant exists for.

Zero new lint: 19 advisory ruff findings across the three touched files before and after. ruff format was run and then reverted — CI runs ruff check ., not the formatter, and formatting collapsed unrelated multi-line expressions across both modules, which is churn that mixes concerns into the diff.

---

Review round: three defects found, all confirmed by execution

The first commit on this branch was broken, not merely incomplete. Recording it here because two of the three share one cause.

1. Every server-side POST /gmail/sync returned 500

gmail_sync bound unrecognised only on the relay branch; the server-side branch copied unreadable off the outcome and never bound it. UnboundLocalError at the SyncResponse build, swallowed by the broad except and re-raised as a 500 with a sync failure recorded against the mailbox.

The existing suite always could see this: reverting just that one line reds 30 of the 109 test_gmail_oauth_cloud.py tests. A full-suite run on the broken commit found 31 failures from this single cause. The commit simply was never run against them before being pushed.

2. _ScanOutcome.unrecognised had no writer

Both constructions pass unreadable=incremental.unreadable / unreadable=read.unreadable. So even once bound, the sync surface would have published a permanent 0 — a field on a model that nothing populates.

Both misses share one cause worth naming. The edit was applied by exact whole-line matching against the four spellings I had already seen, and every missed site is one whose right-hand side differs. A mechanical pass that looks exhaustive because it is literal is only as wide as the examples that built it.

3. The counter missed its own target scenario

_record_unrecognised read failures, which is .clear()ed at the top of every retry round, and ran only at _send's two exits. An unrecognised refusal sharing a round with a retryable flake was wiped before anything looked at it — never deferred, never re-sent, never recorded.

That is the common presentation: an envelope change fails many sub-requests at once, and one 5xx among them is unremarkable. Every original test used a single failing id and none could see it. Now recorded once per round, which also makes the - set(results) subtraction live rather than the dead code it had been.

And the predicate was too narrow in the other direction

It treated "it named a reason" as "we recognise it" — which waves through exactly the change it watches for. The envelope already moved the quota signal once (that is why RESOURCE_EXHAUSTED is in _RATE_LIMIT_REASONS), and a renamed quota reason arrives unfamiliar rather than absent. Now three-sided: a status that speaks for itself, then a reason we have classified as permanent, then everything else. _PERMANENT_REASONS is deliberately small — erring small counts an unknown as unrecognised, which is loud, and loud is what this prefers.

New coverage, each proven able to fail

control	proof
the co-occurrence round	red before the fix, green after
five endpoint-level assertions, one per threading hop	each red when its own kwarg is deleted
the three reason sets are disjoint	red on an overlap
unrecognised <= unreadable across four page shapes	structural invariant

136 passed across the two modules. readme_facts --check green. Zero new lint.

A separate defect, filed not fixed

#863 — _error_reasons returns before it reads error.status, so a google.rpc quota envelope carrying an ErrorInfo reason is not read as a rate limit at all and never aborts the page. Measured: rate_limited=False on that shape against True on the shape the existing test uses. This counter makes that loss loud; it does not prevent it, and the prose no longer claims otherwise.

Scope — why this says Refs and not Closes

The issue's durable fix is done. Its cheap next step — re-run a live 200-message mine and read the per-failure log to identify what the original unreadable: 49 actually were — is not, and cannot be from here: it needs the owner's mailbox and the deployed app.

That step is now cheaper than it was (the count narrows the question, and the logging it depends on already shipped), but it is a real deliverable and I am not closing the issue over it. Close it after that run, or split the run into its own issue.

Also not done, stated plainly: nothing renders unrecognised. InboxWorkbench shows unreadable ("N could not be read") and reads the new field nowhere, and no sync consumer reads it either. It is log-and-wire only. That meets the issue's "loud instead of quiet" bar for an operator; it would not be honest to describe it as a user-visible surface.

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated
jobtracker-web	Ready	Preview	Sep 6, 2026 8:27am UTC