Skip to content

Set the rule for an experiment that touches real personal data - #202

Merged
iderex merged 1 commit into
mainfrom
privacy/real-data-declared-before-the-work-starts
Aug 26, 2026
Merged

iderex merged 1 commit into
mainfrom
privacy/real-data-declared-before-the-work-starts

Conversation

@iderex

@iderex iderex commented Aug 26, 2026

Copy link
Copy Markdown
Contributor

Closes #35

What this changes

docs/privacy.md said what happens to real data if an experiment uses it and
declined to say whether an experiment may use it at all, naming that as a wider
question somebody else had to answer. The answer was given on #46 on
2026-08-24 and lived only in a tracker comment, which record 0000 refuses to
treat as recorded:

git show origin/main:docs/decisions/0000-how-decisions-are-recorded.md | sed -n '55p'
a decision that exists only in an issue is not yet recorded.

docs/decisions/0025-real-data-in-an-experiment.md writes it down. Real
personal data may be read where it belongs to the person running the experiment
and where the measurements were agreed in the record before the work started.
The host rule stands on top of both, unchanged and not restated: the data never
enters the tree, in any form, and what may be written down is the measurement.

The record supersedes 0008 for the field it adds, and 0008 gains the one line
record 0000 allows a superseded record to take. Real-Data in the record
header names what category of data, on whose host, and what will be written
down about it, or the single word none:

Real-Data: my own media library, on my own machine, and what gets written down is the scan time and the item count
Real-Data: none

It is optional, which record 0013 fixes for every field added after it, so no
record already on the default branch is turned red by this and none is edited.

internal/check/realdata.go refuses a record that declares the field and
writes nothing after the colon, under record-real-data-declaration-is-empty.
Three fixtures under testdata/cases/ carry the refusing case, its near
neighbour and the none sentinel.

docs/experiment-template.md carries the field for records written from here
on, and CONTRIBUTING.md no longer says this document is unfinished, because
that is the sentence which stops being true with this change.

This change removes no path.

What failure it prevents

An answer nobody can question afterwards. A record saying a library of a
hundred thousand items took four minutes to scan reads identically whether the
library was the author's own or an employer's, and whether the timing was the
agreed measurement or the one that came out best. Nothing in a finished answer
separates the two, and asking afterwards asks somebody to remember what they
did rather than to confirm what they wrote. The declaration puts the difference
in front of a reviewer in the commit that writes the question, while the answer
is still unknown.

The refusal prevents the narrower failure the field's own shape invites. The
line is typed while the question is being written, the value is left for the
moment the work starts, and the moment the work starts is when nobody reopens
the header. What is left then carries the appearance of a declaration and none
of the content of one, and it reads to anybody scanning the header as a record
that declared its data and was cleared.

What was run

At 7cc5840d0419ba54628488d0205eb0fa7c9c8937, the four commands
CONTRIBUTING.md names, in that order. go build, go vet and
gofmt -l cmd internal each printed nothing, which is the passing result for
all three.

go test -count=1 -v ./cmd/... ./internal/...
ok  	github.com/Flowfin/lab/cmd/contexts	0.579s
ok  	github.com/Flowfin/lab/cmd/lab	2.170s
ok  	github.com/Flowfin/lab/cmd/notices	6.476s
ok  	github.com/Flowfin/lab/cmd/pullrequest	0.608s
ok  	github.com/Flowfin/lab/internal/check	1.033s
ok  	github.com/Flowfin/lab/internal/contexts	0.624s
ok  	github.com/Flowfin/lab/internal/hardware	0.639s
ok  	github.com/Flowfin/lab/internal/invariants	1.084s
ok  	github.com/Flowfin/lab/internal/notices	0.637s
ok  	github.com/Flowfin/lab/internal/prose	0.666s
ok  	github.com/Flowfin/lab/internal/pullrequest	0.666s

What that run did not cover, printed by the package that passed:

the integration-hardware harness was not asked for and nothing in it ran.
asking costs a machine with the hardware each test names and an explicit request:
with LAB_INTEGRATION_HARDWARE=1 in the environment. its results are about that machine and are not this suite's results.

The runner over this tree:

go run ./cmd/lab check .
examined .
1 experiment directory walked, 1 record read
26 decision records read
the time this run read is 2026-08-26T18:33:43Z
0 refused

The guard proved by deleting it. With the refusing arm of refuseRealData
replaced by a silent pass, at the same commit:

go test -count=1 ./cmd/... ./internal/...
ok  	github.com/Flowfin/lab/cmd/contexts	0.667s
ok  	github.com/Flowfin/lab/cmd/lab	2.824s
ok  	github.com/Flowfin/lab/cmd/notices	8.686s
ok  	github.com/Flowfin/lab/cmd/pullrequest	0.773s
--- FAIL: TestCases (0.09s)
    --- FAIL: TestCases/a-record-declaring-real-data-with-nothing-after-the-colon (0.00s)
--- FAIL: TestAnEmptyRealDataDeclarationNamesWhatIsMissing (0.00s)
    --- FAIL: TestAnEmptyRealDataDeclarationNamesWhatIsMissing/nothing_after_the_colon (0.00s)
    --- FAIL: TestAnEmptyRealDataDeclarationNamesWhatIsMissing/a_value_that_is_only_spaces (0.00s)
FAIL	github.com/Flowfin/lab/internal/check	1.319s
ok  	github.com/Flowfin/lab/internal/contexts	0.669s
ok  	github.com/Flowfin/lab/internal/hardware	1.158s
ok  	github.com/Flowfin/lab/internal/invariants	1.708s
ok  	github.com/Flowfin/lab/internal/notices	0.842s
ok  	github.com/Flowfin/lab/internal/prose	0.859s
ok  	github.com/Flowfin/lab/internal/pullrequest	0.955s

One package reddens, at the case fixture and at the two message subtests, and
every other package stays green. The arm was restored before the commit above
was made, and the working tree was compared against it rather than trusted.

What this does not do

The refusal reads one shape and the rule is much larger than it. This is the
fourth leg of the done-when met on a stated reading rather than on an unstated
one: a declaration that is present and empty says nothing about what will be
written down, and nothing about the other two parts either, so the property is
named for the shape a checkout can see rather than for the sentence in record
0025 it serves. A reviewer who reads that leg as asking for a refusal which
judges the words after the colon should send this back, because no reading of a
checkout makes that judgement.

An experiment that touches real personal data and declares nothing at all is
refused by nothing, and cannot be: record 0013 makes an absent field legal for
every field added after it, and the runner cannot tell a record that omitted the
field from one written before the field existed. That is the same hole record
0015 accepted for Needs-Hardware and it is accepted here for the same price.

Whose data it was, whether the measurement was the agreed one, and whether the
declaration was written before the work started are outside every reading of a
checkout. The last of those is a fact about commits, and the runner reads a
checkout rather than a history.

The rule that real data never enters the tree still has no mechanism and can
have none. docs/privacy.md said so before this change and says so after it,
and nothing here narrows that sentence: a record declaring a category and
carrying a sample of it passes every rule in this package.

This change was read by nobody but me. This board has no second reader tonight,
so the deletion proof above stands in place of one rather than beside it.

The diff is 473 lines added and 18 removed, over the inherited 400-line cap. It
is one topic and it is not re-plannable into pieces that mean anything alone:
the rule, the field it is declared in, the refusal over that field and the two
documents that told a reader the question was open all have to arrive together
or the tree contradicts itself between them. Most of the addition is the
decision record and the prose that record obliges.

The board's privacy document said what happens to real data if an experiment
uses it and declined to say whether an experiment may use it at all, naming
that as a question somebody else had to answer. The answer was given on #46 on
2026-08-24 and lived only in a tracker comment, which record 0000 refuses to
treat as recorded. This writes it down and gives it the one field a machine can
read.

Record 0025 carries the decision and supersedes 0008 for the field it adds.
Real personal data may be read where it belongs to the person running the
experiment and where the measurements were agreed in the record before the work
started. The host rule stands on top of both, unchanged: the data never enters
the tree and only the measurement is written down.

`Real-Data` in the record header names what category of data, on whose host and
what will be written down about it, or the single word `none`. It is optional,
which record 0013 fixes for every field added after it, so no record already on
the default branch is turned red by this.

What it prevents is an answer nobody can question afterwards. A record saying a
library of a hundred thousand items took four minutes to scan reads identically
whether the library was the author's own or an employer's, and whether the
timing was the agreed measurement or the one that came out best. The
declaration puts the difference in front of a reviewer while the answer is
still unknown, which is the only moment asking is cheap.

`record-real-data-declaration-is-empty` refuses a record that declares the
field and writes nothing after the colon. That is the shape the field invites:
typed while the question is written, left to be filled in when the work starts,
and never reopened. I proved it bites by replacing the refusing arm with a
silent pass: `go test -count=1 ./cmd/... ./internal/...` then reddens
`internal/check` alone, at the case fixture and at the two message subtests,
and every other package stays green.

What nothing refuses is written at the check, in record 0025 and in
docs/privacy.md rather than left to be inferred from a green run. An experiment
that touches real data and declares nothing at all is not refused and cannot
be, whose data it was and whether the measurement was agreed in advance are
claims no reading of a checkout tests, and the rule that real data never enters
the tree still has no mechanism.

The means is Go and Markdown, unchanged from the rest of this tree: the
refusal is one more arm on the record walk the runner already performs, and the
rule is prose because the part a machine can read is one field.

Closes #35

Signed-off-by: Nils Lehnen <30603423+iderex@users.noreply.github.com>
@iderex iderex added documentation Improvements or additions to documentation security labels Aug 26, 2026
@iderex iderex self-assigned this Aug 26, 2026
@iderex
iderex merged commit 311f2cb into main Aug 26, 2026
25 checks passed
@iderex
iderex deleted the privacy/real-data-declared-before-the-work-starts branch August 26, 2026 18:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation security

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Set the rule for an experiment that touches real data

1 participant