Verify

Don't trust us.

Four steps, about a minute, no faith required.

Anyone selling data can claim it's accurate. The claim is worth nothing on its own, which is why every file we publish is built so that you can catch us if we lie — including lying later, quietly, about a file we published months ago.

In plain words

How it works

When we finish a dataset, we take a fingerprint of it — a short string of characters that changes completely if even one byte of the file changes. Then we write that fingerprint into the Bitcoin blockchain.

Bitcoin is very good at exactly one thing: making it effectively impossible to go back and change what was written. So the fingerprint we published becomes a fixed point. Later, when you download the file, you take your own fingerprint of it and compare. If they match, the file in your hands is the file we sealed — and the blockchain proves we sealed it before you ever asked for it, so we couldn't have tailored it to you.

None of this requires trusting us, our website, or our server. It requires trusting Bitcoin, which you can check yourself.

The four steps

Checking a dataset

1. Recompute the manifest

Every dataset ships a manifest.json listing row counts, per-column hashes, and the exact SQL behind every derived table. verify.py reads the data file and rebuilds all of it from scratch:

python verify.py --data fleet.parquet --manifest manifest.json

If a single row was added, dropped, or altered after sealing, the recomputed column hashes stop matching and this fails.

2. Recompute the root

The manifest reduces to a single root hash. Critically, verify.py does not read the root we shipped and nod at it — it rebuilds the root from the manifest contents and then compares. A tampered or truncated manifest fails here even if someone remembered to update the root file to match.

root recomputed  8423db24633146b8…
root claimed     8423db24633146b8…   OK

3. Check the Bitcoin timestamp

Alongside the root is an OpenTimestamps receipt: a proof that this exact root was committed to the Bitcoin blockchain at a specific block. Verify it with the standard open-source client, against your own node if you run one:

ots verify root.hash.ots

This is the step that makes backdating impossible. We cannot produce a Bitcoin block from last month to cover a number we changed today.

4. Re-derive the findings

The manifest carries the exact query behind every published figure. Run it against the data you downloaded and you should land on the same number we printed. If you don't, we're wrong and we would like to know.

Applied to ourselves

The controls that must fail

A check that always passes is decoration. Our own automated verifier runs a set of tests designed to fail — and it fails the whole run if any of them quietly starts passing. Among them:

A deliberately corrupted manifest must break root recomputation.
A day we never sealed must have no proof to find.
Known-answer cases must land on the known answer — Satoshi's block-1 coins asleep, block 9 awoken at height 170.

We learned this the hard way. An earlier version of our verifier "confirmed" the Bitcoin anchor by reading the source code of the program that creates anchors — which passed happily for weeks after the anchoring had actually stopped. It was checking that we intended to anchor, not that we had.

The rule that came out of it now governs everything here: never verify a thing by inspecting whatever was supposed to make it. Find the artefact that only exists if the work truly ran, check how old it is, and recompute anything that describes itself.

Current status

The proof layer described here runs today against our internal research record and its nightly Bitcoin anchor. The published, downloadable edition of verify.py ships with the first dataset release.