Every vendor deck in this industry now contains the word "experts," and most contain the word "verified." The words are free. The operational reality behind them varies enormously — and buyers, for the most part, have no standard way to tell the difference before the data arrives.
This is the checklist we think every buyer should run on every human-data vendor. It is the audit we are prepared to pass ourselves; publishing it is the fastest way to make that claim testable. None of the questions require trusting the vendor's marketing. Each one has answers that should satisfy you and answers that should worry you.
The ten questions
- 1. Pick any unit of data you delivered last month. Can you name the person who produced it? — The whole audit in one question. If the answer involves batch-level attribution or anonymous IDs with no person behind them, provenance does not exist.
- 2. How was that person's identity established? — Listen for the direction of trust. Self-uploaded documents and sign-up selfies are claims. Checks against authoritative records are facts.
- 3. What exactly was verified beyond identity — employer, role, seniority? — 'Verified' often means 'confirmed a government ID.' An ID check tells you nothing about whether the radiologist is a radiologist.
- 4. Against what source was each fact verified? — For every property, ask what the confirming source was and whether the contributor controlled it. A license registry is one answer; a PDF the contributor emailed in is a different answer.
- 5. How do you detect account sharing and subcontracted work after onboarding? — Verification at sign-up is void if a different person does the tasks. The vendor should have an answer that operates during the work, not before it.
- 6. Is provenance attached per unit, per batch, or per project? — Per unit means any row can be walked back to a verified person. Anything coarser means it cannot.
- 7. When a contributor is later found to be misrepresented, what happens to their historical data? — The honest answer involves tracing and disclosure to affected buyers. If the vendor cannot trace it, question six was answered untruthfully.
- 8. What share of applicants fail verification, and on which property? — Vendors doing real checks have real failure rates and know their shape. A vendor with no rejection story is running a funnel, not a verification.
- 9. Who grades the graders, and are reviewers held to the same verification standard? — QA reviewers shape the data as much as producers. A verified workforce reviewed by an unverified one is half a system.
- 10. Will you put the verification standard in the contract? — The final question converts marketing into obligation. A vendor confident in its process will warrant it; a vendor that resists has told you what the process is worth.
How to read the answers
No vendor should fail this audit on ambition — some engagements genuinely do not need licensed specialists, and a vendor that says so plainly is being honest with you. What should end the conversation is a gap between the claim and the mechanics: "verified experts" on the website and batch-level anonymity in the answers.
Ask to walk one unit of data back to a human being. Everything else is commentary.
A practical note on running this: do it before the pilot, not after. Provenance cannot be retrofitted — a vendor either built attribution into the pipeline or did not, and no amount of goodwill during a pilot changes the architecture underneath it.
Where we stand on our own checklist
Rainbow verifies identity, employer, role, and seniority at source, against systems of record, before an expert's first task — and attaches that provenance to every unit of work we deliver, with the QA trail behind it. We publish what is verified and to what standard; the mechanics stay ours. Question ten is answered in our engagement terms, where it belongs.
If you run this checklist on us and find a gap, we want to hear about it — that is what the standard is for.
Expert human data, verified at source.