Open data

Dataset: what four AI models say when the answer is not public

Every response behind the write-up. Published so the finding can be checked rather than taken on trust, which is the same standard the piece argues for.

Responses

Companies

Models

1 Sept 2026

Collected

Named a manufacturer, for companies that have never disclosed one

Model A7 of 10

Model B0 of 10

Model C0 of 10

Model D0 of 10

All 40 manufacturer questions, one cell each

Scroll horizontally to inspect all response columns →

Grid of forty responses. Rows are the four models, columns are the ten companies. A filled cell means the model named a contract manufacturer for a company that has never disclosed one. Model A has seven filled cells; Models B, C and D have none.
Model	01	02	03	04	05	06	07	08	09	10
Model A
Model B
Model C
Model D

named a manufacturer said it did not knowcolumns are the ten companies, in dataset order

Download the JSON Read the write-up

Methods and limits

This is a small exploratory test collected on 1 September 2026, not a general model benchmark. Ten companies were asked about in two questions each, across four models, for 80 responses. The manufacturer comparison above uses the 40 responses to question U1.

The published write-up reports a single API, default settings, no system prompt and temperature zero. Three models answered without retrieval; one had web search. The download records the exact question, response, retrieval status and collection timestamp for each row.

The public release is not sufficient to independently reproduce the original model runs. Model identities and exact versions are withheld, and a complete API configuration is not published. Readers can inspect and re-analyse the released responses, but cannot recreate the same model and retrieval conditions from this release alone.

The reported counts describe this sample and these prompts. They do not estimate a model’s general error rate or compare consumer chat products. Retrieval differs across models, so the result does not isolate the effect of the model from the effect of web access. A later run may produce different answers.

What is in it, and what is held back

Included

The exact prompt sent, verbatim
The full response, unedited apart from the redaction below
Company name, and whether it is well or thinly covered
Whether the model had web search
Collection timestamp

Held back, and why

Model names. Reported as Model A to D. A named list turns the argument into one about versions and settings instead of behaviour.
Manufacturer names. Replaced with stable labels such as [CDMO-9]. These are real firms being associated with contracts they may not hold, on no evidence, and republishing that would repeat the error the study is about. The labels are consistent, so the pattern that matters is still visible: one label appears for three unrelated companies.

Free to reuse with attribution (CC BY 4.0). If you re-run the analysis and reach a different answer, that is a useful outcome and worth telling me about.