Checking AI answers: how to spot fabricated content

A chatbot replies within seconds, fluently, politely and with a confidence some experts would struggle to muster. That is precisely the problem. With ChatGPT, Gemini, Copilot and the rest, false statements sound just as convincing as correct ones, and some of them are pure invention. Anyone wanting to use an AI answer for something important, such as a question about rent, health insurance or a medication, therefore needs a few fixed checks. They take a few minutes and no specialist knowledge.

Why chatbots make things up

Invented statements from an AI are called hallucinations. The Federal Office for Information Security (BSI) describes it soberly: language models can output false information by chance or generate it in the first place. The reason lies in how they work. A language model calculates which word is most likely to come next. It does not check whether the statement is true. The consumer advice centre adds that the training data is not checked for accuracy and that a chatbot cannot always tell whether a piece of information is reliable.

In 2025, researchers at OpenAI, the company behind ChatGPT, identified a further reason. In training and in the usual benchmarks, guessing is rewarded while admitting ignorance is penalised. It works like a multiple-choice test in which a blank answer scores zero and a guess might score one. So the models learn to answer confidently rather than say "I don't know".

How often answers are wrong

How often this happens was shown in 2025 by a study involving 22 public service broadcasters from 18 countries, including ARD and ZDF, coordinated by the European Broadcasting Union and the BBC. The editorial teams assessed more than 3,000 answers from ChatGPT, Copilot, Gemini and Perplexity to questions about current news. The result: 45 per cent of the answers had at least one serious problem. In 31 per cent the sourcing was wrong, because evidence was missing, wrongly attributed, or the supposed source did not support the claim at all. 20 per cent contained factual errors, including invented details and outdated information.

The examples from the study are instructive. One assistant still named Olaf Scholz as Chancellor; another listed Pope Francis as the serving pope in May 2025, although he had died in April. Several systems supplied links that led nowhere. That even professionals fall for this is shown by several German courts that have reprimanded lawyers for invented case citations from AI.

Warning signs in the answer

Some patterns point to invented content even before any search is needed. The first is very precise information on a niche question: an exact date, a percentage to one decimal place, a statutory provision, a case number. The more specialised the question and the more precise the answer, the more likely the model has guessed. The second warning sign is references that look impeccable: author, year, title, sometimes a link. University libraries explicitly warn their students that such references are often invented and that every single one must be checked.

The third warning sign is time. According to the BSI, every model has a knowledge cut-off at a particular date. It only knows what happened afterwards if it looks it up online; otherwise it guesses. The fourth warning sign is purpose. The BSI cites the example of a language model asked to draft a private sales contract and making mistakes because it was not built for that. And the fifth is the complete absence of doubt. A chatbot that has an assured answer to every question straight away is behaving exactly as its training rewards.

Checking AI answers: four steps

The first step is to ask for the source and open it. A link that does not work, or a page that does not contain the claim, exposes the answer immediately. That was precisely the most common error in the broadcasters' study, at 31 per cent. The second step applies to quotations, studies and titles: enter them in a search engine in quotation marks. Not a single hit almost always means the source does not exist.

The third step is a second, independent source: the website of a public authority, the consumer advice centre, the health insurer, or the text of the law itself. For news, the consumer advice centre also recommends fact-checking sites. The fourth step is the cross-check: ask the same question rephrased, or put it to a second chatbot. If the answers differ, at least one is wrong, and you do not know which.

Checklist for everyday use

• Importance: If money, health, contracts or deadlines are involved, always check. For a cake recipe tip, common sense will do.
• Source: Is there a link, and does the page really say that?
• Existence: Does a search engine find the cited title, judgment or study?
• Currency: Does the question concern something that may have changed recently, such as laws, prices or who holds an office?
• Second opinion: Does a public authority, a consumer advice centre or a fact-check say the same thing?
• Cross-check: Does the answer stay the same if the question is put differently?

For important questions, adding a request at the end of your prompt to state any uncertainties openly tends to produce more honest answers. That is no substitute for checking, but it makes it easier, because the model is then more likely to say where it is guessing. And anyone who, after checking, finds that half of it was right and the other half invented has simply repeated the broadcasters' study on a small scale.