Annotation workbooks: `name` column doesn't match the English source; "change" replaced by a street address

[bertfil]

The name column in data/seatau/annotations/annotation_*.xlsx doesn't match the English source it's supposed to hold. Entities look like they've been swapped per-locale, and one of the swaps has hit a common English word.

tasks.json::0/user_scenario/instructions/known_info:

data/tau2/domains/retail/tasks.json : You are Yusuf Rossi in zip code 19122.
annotation_id.xlsx                  : You are Gatot Situmorang in zip code 15182.
annotation_th.xlsx                  : You are ณัฐจศักดิ์ เช้าวันดี in zip code 66360.

The bad one: the verb "change" gets replaced with a street address — Gg. Tubagus Ismail No. 385, Sorong, Sulawesi Tengah 34824 in id, locale-equivalents in the others. 2,558 rows in annotation_id.xlsx, mostly telecom_tasks:

"this action can only be called once, and will Gg. Tubagus Ismail No. 385,
 Sorong, Sulawesi Tengah 34824 the order status to 'pending (items modified)'"

"...the agent asks for confirmation, suddenly Gg. Tubagus Ismail No. 385,
 Sorong, Sulawesi Tengah 34824 your mind and ask to only..."

annotation_tl.xlsx does the same to ID → Anita Ville.

Comparing every tasks.json:: row in retail_tasks against the committed source:

workbook	divergent / 473
id	131 (27.7%)
th	113 (23.9%)
vi	130 (27.5%)
tl	141 (29.8%)
zh	113 (23.9%)

Same token corrupted in all five but a different address each time, so detection probably ran once over English and substitution per-language. Couldn't tell for sure — I didn't find the script that does this in the repo.

Simulation results look unaffected; the shipped tasks.json and db.json are clean.

repro

# pip install openpyxl requests
import io, json, openpyxl, requests

RAW = "https://raw.githubusercontent.com/SEACrowd/SEATauBench/main"
tasks = requests.get(f"{RAW}/data/tau2/domains/retail/tasks.json").json()
wb = openpyxl.load_workbook(io.BytesIO(
    requests.get(f"{RAW}/data/seatau/annotations/annotation_id.xlsx").content),
    read_only=True, data_only=True)

ws = wb["retail_tasks"]
rows = list(ws.iter_rows(values_only=True))
hdr = [str(c) for c in rows[0]]
rec = dict(zip(hdr, rows[3]))

print("source  :", tasks[0]["user_scenario"]["instructions"]["known_info"])
print("workbook:", rec["name"])

[bertfil]

made with claude /robotemoji

[rephol]

Reproduced independently on annotation_id.xlsx / retail_tasks, and I get the same
count you did: 131 / 473 rows (27.7%) where name diverges from the committed
English source. Four things to add.

1. The shipped L2 assets are clean. I scanned for the substituted address tokens in
the committed data:

file	address-token hits
data/tau2/domains/retail/tasks.json (EN source)	0
data/tau2/domains/retail/id/tasks.json	0
data/tau2/domains/retail/id/policy.md	0
data/tau2/domains/retail/th/tasks.json	0

So the corruption is confined to the annotation workbooks and has not reached the
shipped domain assets — consistent with your note that simulation results look
unaffected.

2. But it is latent rather than harmless. The risk is the import path: a reviewer
working from the corrupted name column sees garbage as the English reference and may
"correct" a perfectly good translation to match it. Once those reviews go through
import_reviewed.py, the corruption reaches data/tau2/domains/{domain}/{lang}/ for
real. Worth repairing the workbooks before the next review pass is imported, rather
than after.

3. A deterministic repair is available. Every row's address column resolves
cleanly against the committed source — 473/473, zero unresolvable — so name can
be rebuilt from tasks.json by tuple-path without any guessing:

0/user_scenario/instructions/known_info
  source  : You are Yusuf Rossi in zip code 19122.
  workbook: You are Gatot Situmorang in zip code 15182.

4. Some rows took two substitutions, not one. The example you flagged has both the
verb and a genuine address replaced:

You want to Gg. Tubagus Ismail No. 385, Sorong, Sulawesi Tengah 34824
#W8665881 to be delivered to Pagaralam 641 instead.

change → an Indonesian address, and the real delivery address → Pagaralam 641. That
reads like the entity swapper working as intended on addresses while also over-matching
a common English verb — plausibly a gazetteer hit, since "Change" is a real place name.
It suggests the fix upstream is a stopword/POS guard rather than a change to the
address pool.

I could not find the substitution script in the repo either. For what it is worth,
src/seatau/annotation/addresses.py is unrelated despite the name (it parses workbook
row addresses like policy.md::001_...), and src/seatau/translation/protect.py is
translation masking, not entity substitution. So it looks like it genuinely was not
committed.

Happy to open a PR with a repair script that rebuilds name from source by address,
plus a check in src/seatau/annotation/validators.py so a future export fails loudly
instead of silently shipping a drifted reference column — if that is a direction you
want and nobody is on it already. I am a native Indonesian speaker, so I can also
review the id assets while I am in there.

verification script

# uv run --with openpyxl --with requests python thisfile.py
import io, json, openpyxl, requests

RAW = "https://raw.githubusercontent.com/SEACrowd/SEATauBench/main"
tasks = requests.get(f"{RAW}/data/tau2/domains/retail/tasks.json").json()
wb = openpyxl.load_workbook(io.BytesIO(
    requests.get(f"{RAW}/data/seatau/annotations/annotation_id.xlsx").content),
    read_only=True, data_only=True)

rows = list(wb["retail_tasks"].iter_rows(values_only=True))
hdr = [str(c) for c in rows[0]]
recs = [dict(zip(hdr, r)) for r in rows[1:] if any(r)]

def resolve(body):
    cur = tasks
    for part in body.split("/"):
        cur = cur[int(part)] if part.isdigit() else cur[part]
    return cur

ok = bad = 0
for r in recs:
    a = str(r.get("address") or "")
    if "::" not in a:
        continue
    src = resolve(a.split("::", 1)[1])
    ok, bad = (ok + 1, bad) if str(src) == str(r.get("name")) else (ok, bad + 1)

print(f"match {ok}  diverge {bad}  ({bad / (ok + bad) * 100:.1f}%)")