fix(privacy): apply noindex+mask policy to machine-readable outputs (#173) - #183
fix(privacy): apply noindex+mask policy to machine-readable outputs (#173)#183LyuboslavLyubenov wants to merge 19 commits into
Conversation
lyubomir-bozhinov
left a comment
There was a problem hiding this comment.
Добре структуриран — маскирането е на query слой преди R2, noindex-gate-ът на .json е чист, тестовете са солидни. Но тезата на PR-а („noindex+mask на всички machine-readable изходи") не е изпълнена докрай:
🔴 .data single-fetch payload е непокрита machine-readable повърхност. При ssr:true RRv7 сервира всеки loader и на GET /<път>.data. company.tsx loader-ът връща eik + displayName немаскирани, а headers() е publicCache(...) → отговорът минава през hardenResponse (само baseSecurityHeaders, без X-Robots-Tag) и се кешира на edge. Значи /companies/:eik.data връща немаскиран ЕИК на физическо лице, кеширан, без noindex — точно експозицията от #173, на повърхността, която PR-ът не изброява. (Потвърдих reachability-то на .data живо на публичния prod; robots.txt не покрива /*.data.) Поправка: приложи предиката/маската и за .data, или централизирай X-Robots-Tag в hardenResponse вместо per-route.
🟠 Sitemap-ът ползва по-тесния предикат. sitemaps.ts филтрира с isNaturalPersonProfileName (само по име), не с разширения isNaturalPersonBidder (+legal_form), който PR-ът ползва навсякъде другаде. ЕТ, разпознат само по legal_form, се маскира/noindex-ва навсякъде, но пак се рекламира в /sitemap-companies — каним crawl на страница, която политиката де-индексира.
За проверка (не потвърдено срещу данни): премахнатият consortium guard в isNaturalPersonBidder — нито един caller не филтрира kind==='consortium', та консорциум с водещ „ЕТ …" в името може да се over-маскира като „Частно лице". Струва си да се потвърди срещу корпуса, преди да се третира като дефект.
lyubomir-bozhinov
left a comment
There was a problem hiding this comment.
Инлайн котви към горния преглед — .data single-fetch privacy gap.
|
Прегледах PR #183 изцяло — целия diff (24 файла, +1882/−45), всички коментари по ревюто, issue #173, както и текущото състояние на кода в работното дърво ( ОбобщениеЦентрализацията на Сигурност / SQL / OWASP
🔴 Съществени пропуски (data-integrity / поверителност)1. Подизпълнителят ( 2. Sitemap-ът ползва по-тесния предикат (незатворена забележка от предишното ревю). 3. HTML профилът скрива ЕИК на физическите лица — противоречи на собствената документация в PR-а. Споделеният loader прави 🟠 По-малки бележки4. 5. Възможно over-маскиране на консорциум. Съответствие с issue #173Ядрото на issue-то (bulk-indexable ЕИК на ЕТ в Бележки
Вердикт: Заявка за промени (Request changes) — блокиращо е т.1 (немаскиран ЕИК/име на подизпълнител-физическо лице в Благодаря за прегледната работа по маркерния договор и тестовете — след затварянето на горните пропуски PR-ът ще е в много добра форма. |
|
Блокерът е затворен — ре-проверих на връх
Точно това затваря находката ми (немаскиран ЕИК на физ. лице, кеширан, без noindex, изтичащ през Единствено: branch-ът е в конфликт с main ( |
…mber worker adr to 0008 Rebase of midt-bg#183 onto upstream/main (post-midt-bg#182 ADR reorganization) restructured the privacy-policy and worker-level X-Robots-Tag ADRs to live in docs/adr/ rather than inline in docs/architecture.md: - New docs/adr/0007-privacy-masking.md — content extracted from the inline ADR-0002 in architecture.md; relative paths adjusted (../ → ../../) for the new adr/ location; cross-link to the worker ADR now points to 0008. - docs/adr/0003-centralized-x-robots-tag-worker.md → docs/adr/0008-centralized-x-robots-tag-worker.md — renumbered to free the 0003 slot taken by upstream's value-flag ADR; internal cross-link from architecture.md#adr-0002-... to 0007-privacy-masking.md. - docs/adr/README.md — index extended with the two new entries. - docs/architecture.md — adopted upstream's short summary form; the inline ADR-0001+0002 contents are removed (the rendering ADR lives at adr/0001-rendering-and-security.md and the privacy policy at adr/0007-privacy-masking.md); Решения (ADR) section now also points to 0007 and 0008. - docs/privacy-masking.md — cross-link from architecture.md#adr-0002-... to adr/0007-privacy-masking.md; ADR-0003 to ADR-0008. No code changes; verified pnpm check:docs (docs-integrity gate from midt-bg#182) passes.
The three files modified by PR midt-bg#183 carried pre-existing prettier debt that the original review flagged (`pnpm lint` exit 1 with `contract.json.test.ts`, `companies.test.ts`, `companies.ts`). The repo's CI is configured as blocking lint (`2d93cd5`, comment in .github/workflows/ci.yml), so this would have blocked the PR from merging. Run `pnpm prettier --write` on the three files — no semantic changes.
|
Прегледах отново новия връх 1) Списък „Фирми" ( 2) Начална страница ( 3) Страница на договора ( 4) Подизпълнител.
Корен (altitude): маскирането е на leaf routes/exports, не на споделените проекции. Устойчивият фикс: маскирай в PR-ът затваря буквата на #173 (.json/.csv), но не и духа — ЕТ ЕИК на индексираните HTML страници. Блокер до фикса на споделените проекции. |
nedda76
left a comment
There was a problem hiding this comment.
Прегледах PR-а спрямо #173. Механиката marker→header е добре построена и тествана: CSV се маскира преди записа в R2, worker-ът слага X-Robots-Tag: noindex и трие marker-а преди edge cache (cache-safe — app.nofollow.test.ts го покрива). Но маската не покрива всяка machine-readable повърхност и на места е обезсилена — има потвърдени пътища, по които суровият ЕИК на физическо лице все още изтича. Тоест PR-ът още не затваря #173.
Блокиращи — потвърдени течове на суров ЕИК
-
slugпресейва ЕИК-а. Маската нулираeik, но оставяslug— а за eik-базирани субектиslugе самият нормализиран ЕИК (companySlug('eik:222222222') === '222222222'). Тоестbidder.slug/company.slugв/contracts/:id.jsonи/companies/:eik.dataносят точно идентификатора, който маската маха. ADR-0007 („slug-ът не е PII") не важи за физически лица — а те са именно маскираната популация. -
/contracts/:id.dataне е маскиран. HTML маршрутътcontract.tsxе недокоснат; приssr:truesingle-fetch/contracts/:id.dataсервира payload-а на loader-а машинно четимо —bidder.eikиsubcontractor.eik, без noindex..jsonе маскиран, но.dataблизнакът му — не (точно класът повърхности, заради който съществува ADR-0008). -
Подизпълнителят не се маскира.
maskContractForPrivacyпипа самоbidder+sourceNames.bidder;subcontractorминава непокътнат, а подизпълнителят може да е ЕТ/физическо лице → ЕИК изтича. Рядко (~0.8% от договорите), но реален непокрит път. -
/companies.data(списък) не е маскиран./companies.csvе, но.dataблизнакът на списъчния маршрут връщаCompanyListItemсъс суровeik+ име за физически лица → точно „bulk searchable/downloadable" вредата от #173, само през.dataвместо.csv.
За обсъждане — по-нисък приоритет
- Мрежовият граф в маскирания
company.dataноси node id-таeik:<ЕИК>→ физическо лице като възел изтича ЕИК. Частично фундаментално за eik-базираните id-та (като #1). - Регресия за легитимни фирми: предикатът пада към name-евристика (
ЕТ/ETпрефикс) дори при реаленlegal_form(ООД) → фирма с име „ET Engineering" получава скрит валиден ЕИК (нарушава изискване #5). Евристиката е заварена, но сега тя контролира скриване на ЕИК, не само мек noindex — цената на false positive расте. - Несъгласуваност:
.json/.csvзаменят името с етикет, ноcompany.data/HTML пазят ЕТ името дословно. Трите machine-readable пътя маскират различни полета.
Чисто (за фокус)
marker→header плъмбингът и cache safety; обединеният предикат isNaturalPersonBidder (и двата source() клона проектират legal_form); запазеното показване на юридически лица по покритите пътища; authorities.csv (публични органи — умишлено без body-маска).
Коренът на #1/#2/#4 е един: маската покрива .json/.csv, но не и .data близнаците и не пипа slug. Докато .data повърхностите и slug не се покрият, #173 остава отворен.
ydimitrof
left a comment
There was a problem hiding this comment.
Преглед на PR #173 — „fix(privacy): noindex + маскиране за машинно-четими изходи"
Какво прави PR-ът
Прилага политика за поверителност върху машинно-четимите изходи (turbo-stream .data, JSON, CSV), така че идентификаторите и имената на физически лица/ЕТ да бъдат маскирани, а не само маркирани с noindex. Архитектурата е чиста и с ясно разделение на отговорностите: route-овете маркират чрез markPrivacyMaskApplied, worker-ът превежда маркера в X-Robots-Tag: noindex чрез applyPrivacyMaskHeaders и го изтрива преди кеш/клиент. Въвежда се единен предикат isNaturalPersonBidder и константа MASKED_NATURAL_PERSON_LABEL в @sigma/shared, преизползвани в streamCompaniesCsv и streamContractsCsv; legal_form се прокарва коректно през двата клона на source() и през getContract.
Сигурност (Фаза 0)
ЧИСТО и в двете партиди — няма зашити тайни/ключове, нови зависимости, нови или подозрителни URL адреси, нито зловредни шаблони. Промяната всъщност намалява изтичането на лични данни. Тестовото покритие е силно и смислено: покрити са MISS/HIT/dynamic, идемпотентност, edge-cache инвариантите, негативните случаи, двата клона на source(), name-евристиката при legal_form=null и таблица на истинност на предиката. Документацията (ADR-0007, ADR-0008, privacy-masking.md) е изчерпателна и синхронизирана с кода.
Блокираща забележка
- (Средно — поверителност/консистентност) Машинно-четимият близнак
/companies/:eik.dataизчиства самоcompany.eik, но връща пълното име на физическото лице (displayName) буквално — за разлика от/contracts/:id.jsonи CSV, които заменят името сMASKED_NATURAL_PERSON_LABEL. Тъй като.dataе машинно-четим изход, а самата обосновка на PR-а (ADR-0007) гласи, чеnoindexе недостатъчен срещу ботове, оставянето на суровото име само задnoindexпротиворечи на декларираната цел. Тестътcompany.data.test.tsдори утвърждава това като очаквано. Нужно е явно решение: да се маскира името и в.data, или изрично да се документира защо.dataостава немаскиран.
Некритични забележки
- (Ниско — поверителност) При маскиране
bidder.slug/company.slugсе запазват. Ако slug-овете се извеждат от името на субекта, URL-фрагментът може да разкрие идентичността въпреки маскирането наname/eik. ADR-0007 приема slug като „URL фрагмент, не PII", но не адресира случая на slug, изведен от име. - (Ниско — точност на маскирането)
isNaturalPersonBidderсе вика без предварителна проверка заbidder_kind, въпреки че документацията му възлага филтрирането на консорциумите на викащия. Консорциум с име, започващо с „ЕТ ", ще бъде over-маскиран като „Частно лице" — privacy-safe, но с загуба на информация. - (Ниско — DB/производителност) Клонът по подразбиране на
source()вече винаги правиLEFT JOIN bidders, а това е и hot path заlistCompanies. Да се потвърди, чеcompany_totalsняма собствена колонаlegal_form(иначеct.*+b.legal_formдава двусмислена дублирана колона) и че има индекс поbidders.id. - (Ниско — дублиран/мъртъв код) Няколко случая на дублиране, противоречащи на правилото „NO CODE DUPLICATION": излишните предварителни извиквания на
markPrivacyMaskAppliedвresponseFromR2Object(csv-export.ts), коитоmarkCsvCacheведнага презаписва; и inline вариант на правилата заlegal_formвapps/web/app/routes/company.tsx. - (За потвърждение)
details.tsпрокарваbidder_legal_formбез маскиране и без тест в прегледаната партида — да се потвърди, че JSON маскирането се извършва другаде.
Вердикт
Промяната е висококачествена и без изтичане на данни в прегледаните файлове. Единствената блокираща точка е несъответствието в поверителността при /companies/:eik.data (т.1) в PR, чиято цел е именно защита на лични данни — тя трябва да бъде адресирана или изрично обоснована преди одобрение. Останалите забележки са некритични.
lyubomir-bozhinov
left a comment
There was a problem hiding this comment.
Благодаря — маскирането на .json и CSV е издържано. Но остава отворена точно най-чувствителната machine-readable повърхност, заради която е #173: detail страницата на договора и нейният .data twin.
apps/web/app/routes/contract.tsx — loader-ът (:62-67) връща { contract } суров; файлът не се пипа в този PR и няма нито маска, нито noindex. getContract селектира b.eik_normalized AS bidder_eik (packages/db/src/queries/details.ts:440) и го отдава суров (:583 eik: r.bidder_eik, :595 eik: r.subcontractor_eik). contract.tsx ги рендира без маска — :221 ЕИК на изпълнителя, :261 ЕИК на подизпълнителя. Worker-ът прилага само applyPrivacyMaskHeaders(headers) (header-only, noindex) — няма body маска на ниво worker.
Ефект (prod е публичен, unauthenticated):
GET /contracts/<slug>.data→ turbo-stream body сbidder.eik/subcontractor.eikна физическо лице (ЕТ), немаскиран и безX-Robots-Tag.GET /contracts/<slug>(HTML) → същият ЕИК, индексируем.
Същата политика, различно прилагане и на списъците:
/companies.data:toCompanyListItem(packages/db/src/queries/rows.ts:58) връщаeik: r.eikбезisNaturalPersonBidderпроверка, докато/companies.csvго маскира./contracts.data:toItemвcontracts.tsдава немаскирани имена на физически лица (CSV пътят вече маскира).
Предложение: маскирай в споделения слой — в getContract, или в loader-а на contract.tsx огледално на company.tsx:77-85 — за да го наследят HTML, .data и .json, вместо per-route. Същото за toCompanyListItem (редът вече носи legal_form) и за подизпълнителя в maskContractForPrivacy.
Докато .data twin-ът не минава през същата маска като .json, #173 не е затворен. Проверих горните редове на HEAD (468a116). Блокиращо за merge.
…port The R2-body branch (responseFromR2Object) and the 304 branch each called markPrivacyMaskApplied directly, then handed the response to markCsvCache, which calls it again internally. The marker was applied twice on MISS/HIT/304 paths — idempotent in effect, but dead code that hid markCsvCache as the single source of truth for the privacy marker on every CSV path (PR midt-bg#183 review T-004, "NO DEAD CODE / NO CODE DUPLICATION"). Drop the direct calls; rely solely on markCsvCache. Add a TDD guard that spies on markPrivacyMaskApplied and asserts exactly one call per response path (MISS/HIT/dynamic/304), so a future duplicate cannot sneak back in.
…ortium over-masking isNaturalPersonBidder's docstring delegates consortium filtering to the caller — a JV is a legal entity even if a lead member's name / legal_form matches a sole-trader signal. But streamContractsCsv and streamCompaniesCsv both invoked it WITHOUT a bidder_kind guard, so a consortium such as "ЕТ Иван Петров; Строй ООД" (or any consortium whose legal_form collided with a sole-trader form) was masked to MASKED_NATURAL_PERSON_LABEL with its ЕИК cleared. The result was privacy-safe (over-masking, no leak) but a behavioral change that dropped the lead member's name + ЕИК and contradicted the predicate's contract. Add an early bidder_kind/kind !== 'consortium' guard in both streamers so consortium rows keep the "… и др." shape and their ЕИК. TDD: failing tests first (consortium with ЕТ lead name + ЕТ legal_form, and the leading-ЕТ name heuristic with legal_form null), then the guard (PR midt-bg#183 T-006).
…ne duplication) The docstring claimed the legal_form rules were "carried inline in apps/web/app/routes/company.tsx until the route migrates" — but ADR-0007 §1 already removed the legacy inline isSingleNaturalPersonProfile, and company.tsx now calls this shared predicate directly (verified: no legal_form string- matching exists outside packages/shared). The stale claim created exactly the divergence risk the PR midt-bg#183 reviewer flagged under "NO CODE DUPLICATION": a future reader could believe a second copy still lives in the route and maintain it separately. Rewrite the docstring to state the predicate is the single source of truth and enumerate the downstream surfaces that consume it (HTML noindex, CSV masking, JSON masking), with a pointer to the bidder_kind/kind consortium guards added in the CSV streamers (PR midt-bg#183 T-006). No behavior change.
…6, §7) Two PR midt-bg#183 review threads asked for explicit product decisions on the company profile masking surface. Both are recorded here as policy. §6 — displayName stays visible in the HTML profile and its `.data` twin; only the ЕИК is masked. The trading name is PUBLIC (rendered verbatim on the HTML page and in <title>); the sensitive natural-person identifier is the ЕИК. The `.data` turbo-stream is React Router v7's single-fetch transport for client-side navigations, NOT a standalone export like /contracts/:id.json — masking the name there would break client-rendered pages. Consistent policy: name = public, ЕИК = sensitive. company.tsx loader comment now states this; the company.data.test.ts assertion locks displayName-verbatim + eik-null as the contract. §7 — the name-keyed natural-person slug (n + base64url(name)) is a tracked limitation, not changed in this PR. The name is public (§6), the sitemap already filters these records, and reworking the slug scheme is cross-cutting (URL stability, internal links, identity system) and out of scope for a masking PR. No behavior change.
|
Прегледах #183 на дълбочина срещу head Силната част (потвърдено):
MAJOR 1 — JSON masker-ът НЯМА consortium guard-а, който CSV пътят има. MAJOR 2 — най-изложената повърхност остава отворена: страницата на договора + MAJOR 3 (test-gap + архитектура) — препращането на маркера към worker-а за MINOR — не-ЕТ физически лица могат да минат немаскирани. NIT — PR-ът е CONFLICTING спрямо main — нужен е rebase (отделно от горното). Насоката е правилна; двата masker-а — consortium guard в JSON и покриване на страницата/ |
… path The `/contracts/:id.json` masker (`maskContractForPrivacy`) lacked the `bidder_kind !== 'consortium'` guard that the CSV streamer already has (`contracts.ts:459`). A consortium whose display name begins with „ЕТ " (first member is a sole trader, e.g. „ЕТ Иван Петров; Строй ООД") was over-masked to „Частно лице" — losing the „… и др." shape, the consortium ЕИК, and gaining an unearned `noindex`. `isNaturalPersonBidder`'s docstring delegates consortium filtering to the caller; this adds the caller guard, mirroring the CSV path exactly. Flagged as MAJOR 1 in the PR midt-bg#183 review of head a9b18ae. TDD: failing consortium cases first (name-based + legal_form-based, plus a loader-level marker-omission case), then the guard.
`contract.tsx` was the most-indexable surface still open: its loader returned
`{ contract }` raw with no privacy marker, `robots.txt` does not block
`/contracts/:id` (or its `.data` twin), and the page rendered `c.bidder.eik`
verbatim — so a sole-trader's ЕИК was indexable on both the HTML page and the
RRv7 single-fetch `.data` payload. That is a worse exposure than the already-
closed `.json`/`.csv` paths.
Masking + signalling in the SHARED loader covers both surfaces at once (the
`.data` twin reuses the same loader), mirroring `company.tsx:89` exactly:
ЕИК (the sensitive natural-person ID) → null on the returned object, the
trading displayName stays PUBLIC (ADR-0007 §6), and the `X-Privacy-Mask:
applied` marker is translated to `X-Robots-Tag: noindex` by the worker. The
`kind === 'consortium'` guard matches the JSON masker (MAJOR 1) and the CSV
streamer so a JV is never over-masked/noindexed. `headers()` forwards the
marker onto the HTML response (RR does not auto-propagate loader headers).
Flagged as MAJOR 2 in the PR midt-bg#183 review of head a9b18ae.
TDD: failing loader/headers/pipeline cases first, then the loader change.
…eal worker The PR midt-bg#183 review (MAJOR 3) noted the marker→`.data`→`X-Robots-Tag` forwarding was only proven through fixtures that INJECT the marker by hand in the stubbed RR handler — which proves the worker CAN translate a marker, not that a real loader's marker survives the pipeline to the final `.data` HTTP response. That left a „green tests, hidden gap" risk on the most-indexable surface. Add four cases driving the REAL `worker.fetch` (→ handleRequest → hardenResponse → applyPrivacyMaskHeaders → edgeCache.put) against `/contracts/<x>.data`: masked sole-trader → noindex + marker stripped + masked body preserved; cached entry carries noindex (HIT-path invariant); second request HITs and serves noindex verbatim; legal-entity negative (no marker → no noindex). The handler returns the exact shape `contract.tsx`'s masked loader branch now produces (MAJOR 2), so this is an honest end-to-end proof of the forwarding guarantee. Note: the review's suggested path-based worker match (the weekly-digest `DIGEST_DETAIL_PATH` precedent) does not exist in this codebase — the worker does no path-based matching; the marker-based design (ADR-0008) is the established architecture and is sound, so this keeps it.
|
@lyubomir-bozhinov Благодаря за задълбочения adversarial pass — трите MAJOR забележки са затворени в три фокусирани комита върху MAJOR 1 — consortium guard в JSON masker-а ( MAJOR 2 — маскиране на страницата на договора + MAJOR 3 — end-to-end доказателство, че маркерът достига Забележка към предложението ти за path-based worker match (седмичният digest За останалите ти точки (не са код-промени в този PR):
Rebase: PR-ът е Пълна локална проверка: |
…-Mask marker The literal X-Robots-Tag header is no longer set anywhere in apps/web; it is now written by exactly one helper (applyPrivacyMaskHeaders in apps/web/app/lib/security.ts), called by the worker hardenResponse after the base security headers and before the cacheable-HTML branch. A new internal marker X-Privacy-Mask: applied is the route-side signal that the response carries masked natural-person data. Route handlers (csv-export.ts markCsvCache + 304 branch, contract.json.tsx loader) and the worker consume that marker; it is deleted unconditionally before the response is returned or stored in edgeCache.put so it never reaches clients. The HIT path in handleRequest (apps/web/workers/app.ts) is unchanged: it copies cached.headers verbatim, and the cached entry is the post-hardenResponse response, so the header survives the edge cache by construction. apps/web/workers/app.nofollow.test.ts (T-005 + T-008) exercises the end-to-end worker flow for the .data twin of /companies/:eik and the contract.json natural-person branch. No edits to packages/db or to the public API contract; bidders.legal_form stays server-only and company.eik stays on the CompanyRecord type (masked to null by the loader, not removed).
…ough headers()
The single-fetch .data twin of the company profile now clears company.eik to null
on the natural-person branch (per isNaturalPersonBidder) and signals the
worker via the internal X-Privacy-Mask: applied header on the Response.json
return. The route's headers() export now destructures { loaderHeaders } from
Route.HeadersArgs and forwards the marker explicitly so the worker
hardenResponse can translate it into X-Robots-Tag: noindex on the HTML
response (getDocumentHeadersImpl only auto-propagates Set-Cookie).
Legal-entity records keep the plain-object return unchanged — no marker, no
mutation, no Response.json wrap. The not-found short-circuit (throw new
Response('Not Found', ...)) runs before the masking gate, so 404s never
carry the marker.
The HTML meta() noindex branch is unchanged — natural-person pages
continue to emit <meta name="robots" content="noindex"> via the existing
seoMeta + isNaturalPersonBidder gate. The new headers() forward adds a
redundant X-Robots-Tag: noindex HTTP header alongside the meta tag, which
is acceptable (the worker translates the marker for all responses).
apps/web/app/routes/company.data.test.ts is the focused new test suite
(7 tests across 5 describe blocks): natural-person loader return asserts
company.eik === null and X-Privacy-Mask: applied; legal-entity loader
return asserts a plain object with eik unchanged and no marker; headers()
test exercises both branches (marker present → forwarded + Cache-Control;
marker absent → Cache-Control only); meta() test covers the natural-person
noindex HTML tag; the worker-pipeline describe calls applyPrivacyMaskHeaders
on the loader return and asserts X-Robots-Tag: noindex is set while
X-Privacy-Mask is stripped, proving the worker translate end-to-end.
…rage
ADR-0002 (docs/architecture.md): the Решение section now describes the
centralized X-Robots-Tag: noindex write site (hardenResponse in
apps/web/workers/app.ts, via the applyPrivacyMaskHeaders helper in
apps/web/app/lib/security.ts). The bullet on per-route CSV/contract-json
writes is replaced by a single sentence naming hardenResponse, the marker
flow, and the deletion pre-edgeCache.put.
The Засегнати повърхности list grows to explicitly enumerate:
- the .data twin of /companies/:eik (React Router v7 single-fetch,
automatic via the shared loader in company.tsx)
- apps/web/workers/app.ts (hardenResponse) as the centralized
enforcement point under a new 'Worker — централизирана точка за
прилагане' sub-heading
- apps/web/app/lib/security.ts as the policy helper home, with
PRIVACY_MASK_APPLIED as the literal-typed constant.
The privacy page (apps/web/app/routes/privacy.tsx) #natural-person-data
section grows to enumerate /companies/:eik.data alongside the existing
/contracts/:id.json and the three CSV exports. A follow-up paragraph in
Bulgarian prose explains that the X-Robots-Tag: noindex policy is now
applied uniformly at the worker edge so future machine-readable surfaces
inherit it automatically — without naming the X-Privacy-Mask marker or
the helper functions (user-facing wording only).
No edits to package.json, pnpm-lock.yaml, or the public API contract.
…ne duplication) The docstring claimed the legal_form rules were "carried inline in apps/web/app/routes/company.tsx until the route migrates" — but ADR-0007 §1 already removed the legacy inline isSingleNaturalPersonProfile, and company.tsx now calls this shared predicate directly (verified: no legal_form string- matching exists outside packages/shared). The stale claim created exactly the divergence risk the PR midt-bg#183 reviewer flagged under "NO CODE DUPLICATION": a future reader could believe a second copy still lives in the route and maintain it separately. Rewrite the docstring to state the predicate is the single source of truth and enumerate the downstream surfaces that consume it (HTML noindex, CSV masking, JSON masking), with a pointer to the bidder_kind/kind consortium guards added in the CSV streamers (PR midt-bg#183 T-006). No behavior change.
…6, §7) Two PR midt-bg#183 review threads asked for explicit product decisions on the company profile masking surface. Both are recorded here as policy. §6 — displayName stays visible in the HTML profile and its `.data` twin; only the ЕИК is masked. The trading name is PUBLIC (rendered verbatim on the HTML page and in <title>); the sensitive natural-person identifier is the ЕИК. The `.data` turbo-stream is React Router v7's single-fetch transport for client-side navigations, NOT a standalone export like /contracts/:id.json — masking the name there would break client-rendered pages. Consistent policy: name = public, ЕИК = sensitive. company.tsx loader comment now states this; the company.data.test.ts assertion locks displayName-verbatim + eik-null as the contract. §7 — the name-keyed natural-person slug (n + base64url(name)) is a tracked limitation, not changed in this PR. The name is public (§6), the sitemap already filters these records, and reworking the slug scheme is cross-cutting (URL stability, internal links, identity system) and out of scope for a masking PR. No behavior change.
… path The `/contracts/:id.json` masker (`maskContractForPrivacy`) lacked the `bidder_kind !== 'consortium'` guard that the CSV streamer already has (`contracts.ts:459`). A consortium whose display name begins with „ЕТ " (first member is a sole trader, e.g. „ЕТ Иван Петров; Строй ООД") was over-masked to „Частно лице" — losing the „… и др." shape, the consortium ЕИК, and gaining an unearned `noindex`. `isNaturalPersonBidder`'s docstring delegates consortium filtering to the caller; this adds the caller guard, mirroring the CSV path exactly. Flagged as MAJOR 1 in the PR midt-bg#183 review of head a9b18ae. TDD: failing consortium cases first (name-based + legal_form-based, plus a loader-level marker-omission case), then the guard.
`contract.tsx` was the most-indexable surface still open: its loader returned
`{ contract }` raw with no privacy marker, `robots.txt` does not block
`/contracts/:id` (or its `.data` twin), and the page rendered `c.bidder.eik`
verbatim — so a sole-trader's ЕИК was indexable on both the HTML page and the
RRv7 single-fetch `.data` payload. That is a worse exposure than the already-
closed `.json`/`.csv` paths.
Masking + signalling in the SHARED loader covers both surfaces at once (the
`.data` twin reuses the same loader), mirroring `company.tsx:89` exactly:
ЕИК (the sensitive natural-person ID) → null on the returned object, the
trading displayName stays PUBLIC (ADR-0007 §6), and the `X-Privacy-Mask:
applied` marker is translated to `X-Robots-Tag: noindex` by the worker. The
`kind === 'consortium'` guard matches the JSON masker (MAJOR 1) and the CSV
streamer so a JV is never over-masked/noindexed. `headers()` forwards the
marker onto the HTML response (RR does not auto-propagate loader headers).
Flagged as MAJOR 2 in the PR midt-bg#183 review of head a9b18ae.
TDD: failing loader/headers/pipeline cases first, then the loader change.
…eal worker The PR midt-bg#183 review (MAJOR 3) noted the marker→`.data`→`X-Robots-Tag` forwarding was only proven through fixtures that INJECT the marker by hand in the stubbed RR handler — which proves the worker CAN translate a marker, not that a real loader's marker survives the pipeline to the final `.data` HTTP response. That left a „green tests, hidden gap" risk on the most-indexable surface. Add four cases driving the REAL `worker.fetch` (→ handleRequest → hardenResponse → applyPrivacyMaskHeaders → edgeCache.put) against `/contracts/<x>.data`: masked sole-trader → noindex + marker stripped + masked body preserved; cached entry carries noindex (HIT-path invariant); second request HITs and serves noindex verbatim; legal-entity negative (no marker → no noindex). The handler returns the exact shape `contract.tsx`'s masked loader branch now produces (MAJOR 2), so this is an honest end-to-end proof of the forwarding guarantee. Note: the review's suggested path-based worker match (the weekly-digest `DIGEST_DETAIL_PATH` precedent) does not exist in this codebase — the worker does no path-based matching; the marker-based design (ADR-0008) is the established architecture and is sound, so this keeps it.
…ty changes After rebasing midt-bg#183 onto upstream/main, the masking test fixtures needed two adaptations to upstream's new APIs (no behaviour change to the production masking logic): - Add `getDb` to the `@sigma/db` mocks in the three loader tests. Upstream's read-only D1 chokepoint (midt-bg#199/midt-bg#225) means loaders now call `getContract(getDb(env), …)` instead of `getContract(env.DB, …)`; the mock passes the env's DB through so the stubbed `getContract` still resolves. - Add the new required `orderingUnit: null` (canonical-identity midt-bg#251) and `amendments: []` (annex history midt-bg#165) fields to the `ContractParty` / `ContractRecord` test builders so they satisfy the widened types. All masking assertions unchanged. `pnpm --filter @sigma/web test` → 424 passing; `pnpm --filter @sigma/db test` → 297 passing; typecheck exit 0.
|
Rebased onto Head is now
Verification (local):
(One pre-existing, unrelated Merge state: GitHub now reports the PR Safety: prior head |
|
Ре-проверих трите нови фикс-комита срещу дифа на връх MAJOR 1 (consortium guard в JSON masker) — затворено. MAJOR 2 (ЕИК на sole-trader изтича на contract страницата + Една остатъчна бележка — не блокер, defense-in-depth пълнота. End-to-end гаранцията, че реален
Мястото да се затвори това е #177 (wrangler интеграционната писта с реалния server build): един Силен отговор на ревюто — ADR-0007/0008 + едновременно покриване на CSV/JSON/ |
…mber worker adr to 0008 Rebase of midt-bg#183 onto upstream/main (post-midt-bg#182 ADR reorganization) restructured the privacy-policy and worker-level X-Robots-Tag ADRs to live in docs/adr/ rather than inline in docs/architecture.md: - New docs/adr/0007-privacy-masking.md — content extracted from the inline ADR-0002 in architecture.md; relative paths adjusted (../ → ../../) for the new adr/ location; cross-link to the worker ADR now points to 0008. - docs/adr/0003-centralized-x-robots-tag-worker.md → docs/adr/0008-centralized-x-robots-tag-worker.md — renumbered to free the 0003 slot taken by upstream's value-flag ADR; internal cross-link from architecture.md#adr-0002-... to 0007-privacy-masking.md. - docs/adr/README.md — index extended with the two new entries. - docs/architecture.md — adopted upstream's short summary form; the inline ADR-0001+0002 contents are removed (the rendering ADR lives at adr/0001-rendering-and-security.md and the privacy policy at adr/0007-privacy-masking.md); Решения (ADR) section now also points to 0007 and 0008. - docs/privacy-masking.md — cross-link from architecture.md#adr-0002-... to adr/0007-privacy-masking.md; ADR-0003 to ADR-0008. No code changes; verified pnpm check:docs (docs-integrity gate from midt-bg#182) passes.
The three files modified by PR midt-bg#183 carried pre-existing prettier debt that the original review flagged (`pnpm lint` exit 1 with `contract.json.test.ts`, `companies.test.ts`, `companies.ts`). The repo's CI is configured as blocking lint (`2d93cd5`, comment in .github/workflows/ci.yml), so this would have blocked the PR from merging. Run `pnpm prettier --write` on the three files — no semantic changes.
…port The R2-body branch (responseFromR2Object) and the 304 branch each called markPrivacyMaskApplied directly, then handed the response to markCsvCache, which calls it again internally. The marker was applied twice on MISS/HIT/304 paths — idempotent in effect, but dead code that hid markCsvCache as the single source of truth for the privacy marker on every CSV path (PR midt-bg#183 review T-004, "NO DEAD CODE / NO CODE DUPLICATION"). Drop the direct calls; rely solely on markCsvCache. Add a TDD guard that spies on markPrivacyMaskApplied and asserts exactly one call per response path (MISS/HIT/dynamic/304), so a future duplicate cannot sneak back in.
…ortium over-masking isNaturalPersonBidder's docstring delegates consortium filtering to the caller — a JV is a legal entity even if a lead member's name / legal_form matches a sole-trader signal. But streamContractsCsv and streamCompaniesCsv both invoked it WITHOUT a bidder_kind guard, so a consortium such as "ЕТ Иван Петров; Строй ООД" (or any consortium whose legal_form collided with a sole-trader form) was masked to MASKED_NATURAL_PERSON_LABEL with its ЕИК cleared. The result was privacy-safe (over-masking, no leak) but a behavioral change that dropped the lead member's name + ЕИК and contradicted the predicate's contract. Add an early bidder_kind/kind !== 'consortium' guard in both streamers so consortium rows keep the "… и др." shape and their ЕИК. TDD: failing tests first (consortium with ЕТ lead name + ЕТ legal_form, and the leading-ЕТ name heuristic with legal_form null), then the guard (PR midt-bg#183 T-006).
…ne duplication) The docstring claimed the legal_form rules were "carried inline in apps/web/app/routes/company.tsx until the route migrates" — but ADR-0007 §1 already removed the legacy inline isSingleNaturalPersonProfile, and company.tsx now calls this shared predicate directly (verified: no legal_form string- matching exists outside packages/shared). The stale claim created exactly the divergence risk the PR midt-bg#183 reviewer flagged under "NO CODE DUPLICATION": a future reader could believe a second copy still lives in the route and maintain it separately. Rewrite the docstring to state the predicate is the single source of truth and enumerate the downstream surfaces that consume it (HTML noindex, CSV masking, JSON masking), with a pointer to the bidder_kind/kind consortium guards added in the CSV streamers (PR midt-bg#183 T-006). No behavior change.
…6, §7) Two PR midt-bg#183 review threads asked for explicit product decisions on the company profile masking surface. Both are recorded here as policy. §6 — displayName stays visible in the HTML profile and its `.data` twin; only the ЕИК is masked. The trading name is PUBLIC (rendered verbatim on the HTML page and in <title>); the sensitive natural-person identifier is the ЕИК. The `.data` turbo-stream is React Router v7's single-fetch transport for client-side navigations, NOT a standalone export like /contracts/:id.json — masking the name there would break client-rendered pages. Consistent policy: name = public, ЕИК = sensitive. company.tsx loader comment now states this; the company.data.test.ts assertion locks displayName-verbatim + eik-null as the contract. §7 — the name-keyed natural-person slug (n + base64url(name)) is a tracked limitation, not changed in this PR. The name is public (§6), the sitemap already filters these records, and reworking the slug scheme is cross-cutting (URL stability, internal links, identity system) and out of scope for a masking PR. No behavior change.
… path The `/contracts/:id.json` masker (`maskContractForPrivacy`) lacked the `bidder_kind !== 'consortium'` guard that the CSV streamer already has (`contracts.ts:459`). A consortium whose display name begins with „ЕТ " (first member is a sole trader, e.g. „ЕТ Иван Петров; Строй ООД") was over-masked to „Частно лице" — losing the „… и др." shape, the consortium ЕИК, and gaining an unearned `noindex`. `isNaturalPersonBidder`'s docstring delegates consortium filtering to the caller; this adds the caller guard, mirroring the CSV path exactly. Flagged as MAJOR 1 in the PR midt-bg#183 review of head a9b18ae. TDD: failing consortium cases first (name-based + legal_form-based, plus a loader-level marker-omission case), then the guard.
`contract.tsx` was the most-indexable surface still open: its loader returned
`{ contract }` raw with no privacy marker, `robots.txt` does not block
`/contracts/:id` (or its `.data` twin), and the page rendered `c.bidder.eik`
verbatim — so a sole-trader's ЕИК was indexable on both the HTML page and the
RRv7 single-fetch `.data` payload. That is a worse exposure than the already-
closed `.json`/`.csv` paths.
Masking + signalling in the SHARED loader covers both surfaces at once (the
`.data` twin reuses the same loader), mirroring `company.tsx:89` exactly:
ЕИК (the sensitive natural-person ID) → null on the returned object, the
trading displayName stays PUBLIC (ADR-0007 §6), and the `X-Privacy-Mask:
applied` marker is translated to `X-Robots-Tag: noindex` by the worker. The
`kind === 'consortium'` guard matches the JSON masker (MAJOR 1) and the CSV
streamer so a JV is never over-masked/noindexed. `headers()` forwards the
marker onto the HTML response (RR does not auto-propagate loader headers).
Flagged as MAJOR 2 in the PR midt-bg#183 review of head a9b18ae.
TDD: failing loader/headers/pipeline cases first, then the loader change.
…eal worker The PR midt-bg#183 review (MAJOR 3) noted the marker→`.data`→`X-Robots-Tag` forwarding was only proven through fixtures that INJECT the marker by hand in the stubbed RR handler — which proves the worker CAN translate a marker, not that a real loader's marker survives the pipeline to the final `.data` HTTP response. That left a „green tests, hidden gap" risk on the most-indexable surface. Add four cases driving the REAL `worker.fetch` (→ handleRequest → hardenResponse → applyPrivacyMaskHeaders → edgeCache.put) against `/contracts/<x>.data`: masked sole-trader → noindex + marker stripped + masked body preserved; cached entry carries noindex (HIT-path invariant); second request HITs and serves noindex verbatim; legal-entity negative (no marker → no noindex). The handler returns the exact shape `contract.tsx`'s masked loader branch now produces (MAJOR 2), so this is an honest end-to-end proof of the forwarding guarantee. Note: the review's suggested path-based worker match (the weekly-digest `DIGEST_DETAIL_PATH` precedent) does not exist in this codebase — the worker does no path-based matching; the marker-based design (ADR-0008) is the established architecture and is sound, so this keeps it.
…ty changes After rebasing midt-bg#183 onto upstream/main, the masking test fixtures needed two adaptations to upstream's new APIs (no behaviour change to the production masking logic): - Add `getDb` to the `@sigma/db` mocks in the three loader tests. Upstream's read-only D1 chokepoint (midt-bg#199/midt-bg#225) means loaders now call `getContract(getDb(env), …)` instead of `getContract(env.DB, …)`; the mock passes the env's DB through so the stubbed `getContract` still resolves. - Add the new required `orderingUnit: null` (canonical-identity midt-bg#251) and `amendments: []` (annex history midt-bg#165) fields to the `ContractParty` / `ContractRecord` test builders so they satisfy the widened types. All masking assertions unchanged. `pnpm --filter @sigma/web test` → 424 passing; `pnpm --filter @sigma/db test` → 297 passing; typecheck exit 0.
… and resolve conflicts Conflict resolution notes: - docs/adr/README.md: upstream introduced new ADRs (0007-scope-and-certainty-bar, 0008-deterministic-name-to-eik-resolution, ... up to 0032). The PR's 0007-privacy-masking and 0008-centralized-x-robots-tag-worker are renumbered to 0033 and 0034 (the next two free slots), and cross-references in architecture.md, privacy-masking.md, and the ADR files themselves are updated accordingly. File renames via git mv preserve blame. - apps/web/app/routes/contract.json.tsx: upstream refactored to use the shared serializeJsonForScript helper (lib/json-ld.ts) and added X-Content-Type-Options: nosniff. The PR's maskContractForPrivacy function and its consortium guard are preserved; the X-Privacy-Mask marker is replaced with a direct X-Robots-Tag: noindex header because the upstream refactor of the worker (apps/web/workers/app.ts isNoindexNamesPath) no longer translates the marker. The direct header keeps the privacy guarantee for the masked record. - apps/web/app/routes/contract.tsx: import block conflict only; both isNaturalPersonBidder (PR) and isNaturalPersonProfileName (upstream meta noindex) are kept. The PR's loader masking is preserved; the worker's noindex path is now path-based so the contract page noindex must be either added to isNoindexNamesPath or set on the route itself. This commit keeps the route-level masking only; a follow-up may want to align with the worker's path-based noindex policy. - contract.json.test.ts: tests that asserted X-Privacy-Mask: applied / X-Robots-Tag: null are updated to assert the new direct X-Robots-Tag: noindex header (the marker mechanism was removed upstream). The negative cases (legal entity, consortium, not_found) keep asserting X-Robots-Tag: null. The behavior assertion is the same: a masked response gets noindex, a passthrough does not. Verified: pnpm typecheck, pnpm --filter @sigma/web test (429 passing). Refs midt-bg#183, fixes the merge conflict with the post-2026-08-04 upstream work (related-persons, undici bump, cacbg fix).
…fter rebase The merge onto current upstream surfaced three pre-existing issues that need to be addressed for the test suite and lint to pass: - apps/web/app/routes/contract.json.test.ts and contract.data.test.ts: add cohort: null to the makeRecord() fixture. Upstream's ContractRecord type now requires ContractCohortBenchmark | null (the 'Подобни договори' benchmark from the new cohort-band feature in PR midt-bg#210), and the fixtures predated it. - apps/web/app/lib/csv-export.test.ts, packages/db/src/queries/companies.ts, packages/db/src/queries/contracts.ts: prettier format. These three files were reformatted by the upstream prettier version (3.8.3 vs whatever the original PR ran on) — same content, just whitespace. The lint gate is blocking on these, so format fixes are non-optional. Verification: pnpm typecheck (7/7 packages clean), pnpm --filter @sigma/web test (532 passing), pnpm --filter @sigma/shared test (60 passing), pnpm lint (prettier --check clean).
… and resolve conflicts
|
Daily autonomous review — rebase pass. No new reviewer activity since 2026-07-28 (lyubomir-bozhinov's last approval pass). All 8 review threads remain Branch was Approach.
PR-scope files (privacy masking, 28 files / +2610/-60) merged cleanly through unchanged. Verification (local).
Pre-existing failures (documented in PR body, unrelated to this rebase).
State. |
Какво и защо
HTML профилът на фирма вече прилага
noindexза разпознати физически лица / еднолични търговци (ЕТ),но същите идентификатори остават достъпни от машинно-четливите повърхности — JSON записът
на договора (
/contracts/:id.json) и трите CSV експорта (/contracts.csv,/companies.csv,/authorities.csv). Те връщат ЕИК и оригиналното име от източника безX-Robots-Tag: noindexи без маскиране, edge-кешират се (
Cache-Control: public, max-age=3600) и попадат в индекситена търсачките и ботовете.
Несъответствието прави идентификаторите, които HTML умишлено държи извън търсачките,
търсещи се и изтегляеми накуп — класът CWE-359, описан в issue #173.
Решение
Прилага се политика „
noindexплюс маскиране" с един общ предикат отpackages/shared/src/format.ts, който заменя досегашната дублирана логика вcompany.tsx:isNaturalPersonBidder(name, legalForm)комбинираlegal_form LIKE 'ЕТ%'(вкл. латинското
ET, разширените формиЕДНОЛИЧЕН ТЪРГОВЕЦ,SOLE TRADER,INDIVIDUAL)с водещия
ЕТсуфикс в името (който вече беше вisNaturalPersonProfileName).MASKED_NATURAL_PERSON_LABEL('Частно лице') е константа, която се внася по символ отвсеки консуматор — преименуването ѝ не чупи нито един тест. Inline
isSingleNaturalPersonProfileе премахнат от
company.tsx.packages/db/src/queries/{contracts,companies}.tsправятSELECT b.legal_formи маскиратcontractor_eik/eikиcontractor/nameпредибайтовете да стигнат до R2 — edge кешът не може никога да сервира немаскиран
естествено-личностен ред.
apps/web/app/lib/csv-export.tsдобавяX-Robots-Tag: noindexна четирите клона: 200/206 HIT, dynamic (филтриран) и 304.
packages/db/src/queries/details.tsразширяваgetContractсbidder_legal_form(server-only, не изтича към клиента —ContractRecordостава непроменен).apps/web/app/routes/contract.json.tsxизнася чистmaskContractForPrivacy(record, bidderLegalForm)помощник, който слагаX-Robots-Tag: noindexсамо когато маскиранетореално е приложило — reference-equality гейт
masked !== recordправи повторнотоизвикване на предиката ненужно.
#natural-person-dataвapps/web/app/routes/privacy.tsxописва кои полета се маскират, кои повърхностиносят
noindex-а и че HTML профилът остава непокътнат. Кадърът е инженерно ръководство,не правен съвет — същият disclaimer носи и ADR-0002.
docs/architecture.md(български, огледало на ADR-0001):Контекст / Решение / Последствия / Засегнати повърхности / Доказателство. Цитираните exit code-овеса от реален пост-едит прогон в чисто и замърсено дърво, не оценки.
Юридическите лица (
legal entity) са непроменени — ЕИК и имената им остават видимивъв всички повърхности.
Валидация (реални данни)
pnpm typecheck— exit 0 (7/7 turbo задачи).pnpm test --force— exit 1, заради 3 пред-съществуващи повреди в@sigma/db(
integrity-checks.test.tsreconciliation gate + 2 таймаута вrefresh-slice.test.ts).Същият набор е налице и в предишния
main, не е въведен от този PR.Всичките 38 нови теста минават — пост-едит и пре-едит прогонът показват същия набор
пред-съществуващи повреди, без нови въведени от този PR.
pnpm lint— exit 1, 6 prettier warnings: 2 пред-съществуващи (RiskIndicators.tsx,riskLogic.test.ts) + 4 нововъведени (contract.json.test.ts,privacy.tsx,companies.test.ts,companies.ts). Няма нови lint-видове — само пренасяне на редовезаради български / EN текст в JSX.
@sigma/shared42/42,@sigma/db(
contracts.test.ts+companies.test.ts+details.test.ts) 26/26,@sigma/web(contract.json.test.ts+csv-export.test.ts) 33/33.Извън обхвата
bidders.legal_formвече присъстваше вmigrations/0000_init.sql.ContractRecord(API contract) е непроменен от страна на клиента; полетоbidder_legal_formостава server-only (добавя се в
details.ts, използва се от route-а, не се изпраща на клиента).noindexмета-етикет минава през същия споделен предикат.Чеклист
Co-Authored-By:trailermidt-bg/sigma:mainpnpm typecheckминава; пред-съществуващитеpnpm test/pnpm lintповреди садокументирани като baseline в секция „Валидация (реални данни)" по-горе
Closes #173