This tool exists on a premise nobody has measured: that what people say about their work and what their repositories show are different, and that the gap matters.
The study. Take public repositories where a contributor also has a public written claim about their role, a README credit, a talk description, a project page, a conference bio. Measure both. Report the distribution of the difference.
This is delicate and the design carries most of the work.
- Nobody is named, and this is the constraint the design turns on. The output is a distribution, never a list of people who overstated. A study that shames individuals is not one this project will host, and a pull request that produces one will be closed. Aggregate only, and say so in the design before collecting anything.
- The claim side has to be coded consistently, and "led", "built", "co-authored" and "maintained" do not have agreed meanings. That coding scheme is the real contribution here and should be published with the data.
- Divergence is not dishonesty. Someone who wrote a design that another person typed has a low commit share and a true claim. The study measures a gap, and the write-up must refuse the inference that a gap is a lie, in its own words, prominently.
- The design goes in the repository before the run, the way the divergence study in the sibling repository did, so the conclusion cannot drift with the data.
Done when
- The design, the sampling frame and the coding scheme are committed and reviewed before any collection.
- The result is a distribution with its limits stated, including how the sample was drawn and who it excludes.
- The null result is publishable and is stated as such if it comes: if stated claims and git history agree closely, that is a finding, and it partially undermines this tool's premise, which is exactly why it is worth running.
Out of scope. Anything using private repositories, anything scraping a platform against its terms, and any output naming a person.
If this is too large, the design document alone is a complete and welcome contribution. Open a pull request with the design and stop there.
This one is a study, not a feature. The output is a result and a write-up, and it is fine if the result is boring.
How studies work in these repositories. The design goes in the repository before the data is collected, including what a null result would mean, so the conclusion cannot drift with the numbers. Every figure comes from a run somebody else can repeat, with the command and the version committed beside it. A null or negative result is published in the same place a positive one would go, including when it undercuts the tool. Nothing invented: a number is measured, cited, or it does not appear.
Taking this on: comment and it is yours. A pull request containing only the design document is a complete contribution and often the most useful one, so do not feel you have to finish the whole thing to start.
This tool exists on a premise nobody has measured: that what people say about their work and what their repositories show are different, and that the gap matters.
The study. Take public repositories where a contributor also has a public written claim about their role, a README credit, a talk description, a project page, a conference bio. Measure both. Report the distribution of the difference.
This is delicate and the design carries most of the work.
Done when
Out of scope. Anything using private repositories, anything scraping a platform against its terms, and any output naming a person.
If this is too large, the design document alone is a complete and welcome contribution. Open a pull request with the design and stop there.
This one is a study, not a feature. The output is a result and a write-up, and it is fine if the result is boring.
How studies work in these repositories. The design goes in the repository before the data is collected, including what a null result would mean, so the conclusion cannot drift with the numbers. Every figure comes from a run somebody else can repeat, with the command and the version committed beside it. A null or negative result is published in the same place a positive one would go, including when it undercuts the tool. Nothing invented: a number is measured, cited, or it does not appear.
Taking this on: comment and it is yours. A pull request containing only the design document is a complete contribution and often the most useful one, so do not feel you have to finish the whole thing to start.