The last time someone outside this
project moved it forward, the contribution was a full measurement: a schema-valid
energy.json from someone else's GPU. Yesterday a reproducer moved it further than
that, in the opposite direction — his script never ran. He submitted no data and
no code. In one day his dead script, two footnotes and a single sentence of confusion shipped
two releases and closed the last live path to a bug we thought was already dead.
He was hired to reproduce the pipeline — the first paid attempt, on a clean Ubuntu 22.04 sandbox, no GPU yet. The security model held up end-to-end: the site stub downloaded the wrapper from the pinned release, the SHA-256 check passed. Then bash read the first line and died:
set: pipefail: invalid option name
Anyone who has debugged Windows-authored shell scripts knows the invisible character at the
end of that line. Every *.sh we had ever published — the wrapper, the
quickstart.sh asset, the site stub itself — carried CRLF endings, and Linux bash
reads pipefail\r as an option name. The checksum had passed because the checksum was
computed against the CRLF bytes: integrity and correctness are different properties,
and a self-consistent pair of wrong files sails through the first one.
Fixed at every layer the same day: all repo scripts converted to LF, a CI guard that now fails
on any CRLF shell script, a new pinned release (run-v2;
run-v1 stays as the tombstone), the r4 asset re-published, and — found while re-publishing — a
latent error where the schema line in SHA256SUMS.all had never matched the tag.
All of it is in the changelog.
He came back with a v2 report verifying the fix item by item — ten checks, all green:
| check | result |
|---|---|
| stub line endings, and that it pins run-v2's checksum | pass |
run-v2 hash matches the stub's hardcoded value, bash -n, --dry-run plan | pass |
r4 quickstart.sh: LF, checksum, bash -n | pass |
SHA256SUMS.all all three entries, recomputed from the tag | pass |
.gitattributes rule and the CI CRLF guard present | pass |
A verification report is exactly what a solo project can never produce for itself — the person who reported the bug independently confirming it is dead. That alone would have made his day worthwhile. But the report also contained this, filed under "two small suggestions":
I checked the live r4 asset and the repo: both say r4. My first diagnosis was a cached download on his side. Wrong. He had a working download path — so somewhere, something was still handing out an r3. Following that instead of dismissing it found the real one: the README's quick start still pointed at the r3 release — and r3's assets are the original CRLF versions. Anyone who read the README from the top and copy-pasted the download commands would have re-created his morning bug, exactly, on a machine that had nothing to do with any cache.
The checklist that failed here is worth writing down, because "we fixed the bug" usually leaves this exact hole: a bug that lives in artifacts gets fixed in the artifacts — and every document that points at those artifacts has to be checked too, one by one. Release asset, script header, README, site pages. We checked three of four.
His two actual suggestions were both accepted, and shipped in 4b34e28 together with the README fix:
quickstart.sh and the README now state where
sha256sum -c SHA256SUMS.all must run — the root of a clone of the tag, not the
release download directory, because its entries are repo-relative paths. He hit this one
himself in his first verification pass, and documented it as a usage note rather than a bug,
which is precisely the right diagnosis;This is the part worth saying out loud, because it decides what a solo project can attract.
He has, so far, contributed: a failed run, a footnote about where a checksum command must be executed, and one sentence of confusion that was wrong about the cause and right about the symptom. No code. No measurement. Nothing that would count as a contribution under the default definition — the one that counts commits and datasets.
Under that definition, the project's two most valuable external inputs would be invisible: this, and a laptop measurement from August. The first never produced a number; the second produced one number from one run. What they share is that someone outside the project did the thing and reported what actually happened — including the parts that looked like their own mistakes. A dead script is information. A confusing cache-looking download is information. Most users who hit either bounce, and the maintainer never learns the funnel is broken.
For a maintainer, the operational lesson is just as short: treat every "it didn't work" report as a free audit, and treat every odd detail in it as a lead until proven otherwise. I initially mis-filed his r3 remark as a cache problem on his side. The cost of checking it anyway was five minutes. The cost of not checking it would have been every README-following user for the lifetime of the r3 link.
His GPU reproduction is queued: energy.json generation, schema validation, the
overlay, and a comparison against dataset v1.1.0. When it lands it will be a real contribution
under any definition.
But it did not have to land to count. If you run the one-liner and it dies — that is the report. Open an issue with what you typed and what came back, especially the parts you think are your fault. The repo's CONTRIBUTING lists five ways to help. This note is evidence for a sixth: make the project break in front of you, and tell someone.