AI Hallucination Detection
New Cost-Effective Tool Detects 'Hallucinations' in Visual Language AI Models
Anonymous
A new tool can now detect when visual AI models are 'hallucinating' — that is, when they generate false information about what they see in an image. The method is fast and cheap to run, according to Agencia SINC, making it practical for real-world use in medicine, engineering, and science.
The technique was developed by researchers at the University of Oxford, led by Dr. Sebastian Farquhar. It works by checking whether a model's different internal 'drafts' of an answer mean the same thing. If they don't, the model is likely hallucinating. El Periódico reports the tool adds less than 5–10% extra computing time — far cheaper than older methods that could double processing costs.
What 'Hallucination' Means in Visual AI
Vision-Language Models (VLMs) are AI systems that look at images and describe what they see. They are used in medical imaging, autonomous vehicles, and assistive tech for blind users. But they have a serious flaw: they sometimes describe things that are not there. This is called object hallucination.
A striking example: shown a medical X-ray, a model might correctly spot a lung but then invent a fracture that does not exist. This happens because training data often links certain visual features together. Dr. Farquhar put it plainly — current models "struggle because they are designed to be helpful and creative, but in science and medicine, we need them to be honest about their uncertainty," according to La Opinión de Zamora.
How the Tool Catches Lies Without a Fact-Checker
Older hallucination detectors used a second AI to check the first one. That doubled the computing cost. Other methods required thousands of hours of human labeling. The new tool takes a different approach — it uses a concept called Semantic Entropy.
Semantic Entropy measures how much the model's answers vary in meaning across multiple internal drafts. If one draft says "a bone is cracked" and another says "there is a shadow," the meanings clash — a clear sign the model is guessing. If both drafts say the same thing in different words, the model is confident and likely correct. In tests on standard visual question-answering datasets, the tool reached an accuracy score (AUROC) of 0.79 to 0.88, compared to a 0.62 baseline for older methods, according to Levante-EMV.
Real-World Impact in Surgery, Engineering, and Safety
The tool flagged 82% of incorrect visual interpretations before they reached the user in engineering tests. For doctors, it could act as a "check engine light" on AI diagnostic reports — a warning that the model's output needs a second look. For blind users relying on AI to read medication labels or spot obstacles, the stakes are even higher.
Engineers could use the tool as a hallucination filter when scanning blueprints. The Alan Turing Institute, which supported the research, called it "a major step toward reliable AI in engineering." Analysts at LNE also note the low cost could help smaller European labs — not just Big Tech companies — add safety layers to their visual AI products.
Critics Say Detecting Lies Is Not the Same as Fixing Them
Not everyone is celebrating. Some researchers, including those at the AI Now Institute, argue the tool solves only half the problem. Knowing a model is hallucinating does not explain why it is doing so. The 'black box' problem — the inability to see inside an AI's reasoning — remains unsolved.
Still, regulators may move quickly to adopt the tool. The EU AI Act, which sets rules for high-risk AI, could use hallucination probability scores as a standard reliability metric, according to El Periódico de Extremadura. For now, the research marks a clear shift: from AI systems that confidently make things up, to systems that can at least signal when they are not sure.
Sentiment
NeutralNEGATIVENEUTRALPOSITIVE
Publishers
9
Articles
0
Reach
9
Topics
A
AnonymousAuthor
