The Verification Log

Every claim has a source.

We use AI research tools the same way our clients do, and then we do the part almost nobody does: we take every claim they produce and try to kill it at the source. What follows are claims that looked completely credible, carried real citations, and failed anyway. None of them reached a client, a brief, or a page on this site. They died at the gate, which is what the gate is for.

How A Claim Earns Its Place

Four steps, run on every pass.

An AI research tool will answer a question about a regulated industry in a few minutes, with citations, in language that reads like a consultant's report. That output is a starting point. It is not a source, and the citations attached to it are not evidence that anyone checked.

  1. 1 · The pass runs

    The tool answers a question about an industry we work in and returns a report with citations.

  2. 2 · Every material claim is extracted

    Each one goes to an independent checker. The checkers work separately and are not told what the report concluded.

  3. 3 · Each checker goes to the primary source only

    Regulation text, the standards body, the government database, the court record, the agency release. A trade article citing a study is a lead. The study is the source.

  4. 4 · Each claim comes back one of four ways

    Confirmed with the exact wording and a link. Corrected with what the source actually says. Unverifiable after a genuine attempt. Or dead. Only the first two are allowed into anything a client reads.

The Log

Eight ways a research claim goes wrong.

The failures are not random, and they are not obvious. Every one below passed a careful read by someone who knew the subject. Each was caught only by opening the source. A single pass on one industry produced fifteen claims that did not survive.

Pattern 01 The citation that does not exist

A real company, credited with research it never published

A report attributed a specific labor figure, eight to sixteen hours to compile a first article inspection report, to a named company. That company is real and its website resolves. It is a software consultancy with no aerospace practice, no quality content, and no page anywhere making that claim or anything like it. The attribution was constructed.

Why it survives a careful read: the company exists, the link works, and the number is plausible to anyone who has assembled one of those packages.

Pattern 02 Marketing dressed as research

One company's own advertisement, cited as an industry study

A figure of eleven hours per shipment on compliance paperwork was credited to a named source and described as drawn from time and motion studies. The company exists and the number exists. It is a single shop's before and after claim about its own software, published on its own product page. There is no study, and no second source anywhere reports anything comparable.

Why it survives a careful read: the report's own source table rated it a primary survey, and rated it strong.

Pattern 03 The right number about the wrong people

A real statistic, quietly reassigned to a different population

A widely repeated figure had 46 percent of accounting firms exposing client data to public AI tools. The underlying research is real and the number is real. It describes United States employees across all industries, not accounting firms. The population was swapped somewhere in the retelling, and every article downstream inherited the error intact.

Why it survives a careful read: the source is a name everyone trusts, and it is quoted accurately. Only the subject changed.

Pattern 04 The number the source never published

A vendor's arithmetic, attributed to a federal agency

An enterprise AI adoption rate was credited to a federal statistical agency. That agency publishes the underlying survey, and we read it in the agency's own data files. The figure appears in none of them. It was computed by a software vendor averaging the agency's noisy readings over an unstated window, then reported onward as though the agency had said it. A second figure in the same report turned out to be one hundred minus the first, presented as a measured result.

Why it survives a careful read: the chain runs agency to vendor to trade press, and by the time it reaches you only the first name is still attached.

Pattern 05 The requirement the rule does not contain

A standard described as demanding something it never mentions

A report stated that a widely held quality standard requires records to be kept for ten to forty years. The standard sets no retention period at all. It requires retention to be defined according to customer, statutory and regulatory requirements. The long horizons are real, and they come from customer flow-down documents, one of which we found published and quoted directly. Separately, a defense acquisition clause was described as requiring traceability back to the melting furnace. The clause requires the metal to be melted or produced in a qualifying country and contains no traceability language at all. Traceability is how the industry evidences compliance. It is not what the rule says.

Why this one matters most: a buyer's contract will quote the rule. If your understanding of it came from a summary, you are negotiating against a document you have not read.

Pattern 06 The citation that was never right

Deleted requirements, filed under a section number the rewrite had just created

A report set out what a manufacturing record has to contain and attached the requirement to a numbered section of a federal regulation. That regulation had been rewritten months earlier and most of its sections deleted outright, including the one those requirements actually came from. The number the report cited is real and it is current. It is also new: it did not exist under the old rule, and it governs something else entirely, being complaint files, servicing records and device identifiers. There is no version of that regulation, past or present, in which the citation was correct. Checking that the section exists confirms it. Reading it is what fails it.

Why it survives a careful read: a section number looks like the most checkable thing in a document, so it gets checked least. Confirming that the number resolves feels like verification, and it is the one step that proves nothing.

Pattern 07 The term of art nobody uses

A real principle, given an official-sounding name that appears in no source

A report named one of the governing principles of a federal regulation, capitalized it, and used it throughout as though it were settled industry vocabulary. The principle itself is real. It is written out in plain prose in the rule's own scope section, and it says what the report said it says. The name is the invention. It appears nowhere in the final rule, nowhere on the agency's page about that rule, and nowhere in the agency's own answers to frequently asked questions. Nothing in any source calls it anything at all.

Why this one is expensive: a wrong number gets corrected in a meeting. Invented vocabulary, used in front of somebody who has worked under that rule for years, is the tell that you learned the subject from a summary. It costs you the room rather than the point.

Pattern 08 The outcome rounded up

Enforcement stories drift toward severity with every retelling

In one case a defendant received supervised release and the report implied prison. In another, executives were described as convicted when their charges had in fact been dismissed under deferred prosecution agreements. In a third, a large loss figure attributed by a federal agency, and formally disputed by the company it was attributed to, was presented as the finding of a court.

Why it survives a careful read: the cases are real, the names are real, and the documents are public. Almost nobody opens them.

Companies are not named on this page unless the entity is a government body or a court record. The subject here is the claim and what the source actually says, not any individual vendor. Research passes are run with a commercial AI deep research tool, the same class of tool now sitting inside most companies.

The Point

This is the same check, pointed at your workflow.

Every failure above was caught by going to the source. Not by a smarter model, not by a better prompt, and not by asking the tool whether it was sure. A system that reports its own success is not a check. That is the whole method, and an assessment runs it inside your business instead of inside a research report: find the places where something generated is trusted without a check against a source of truth that already exists, and say which of them would hurt.

Start an Assessment

Fixed scope, $2,500. You keep the document either way.

The same discipline, pointed at production systems. Read the receipts