Hallucination Detection
Methods and systems for automatically identifying when an AI model has generated false or unsupported information. Detection can compare outputs against source documents or use consistency checks.
Why It Matters
Hallucination detection is critical for deploying AI in high-stakes domains. Without it, users cannot distinguish confident truth from confident fiction.
Example
A system that cross-references an LLM's medical response against a trusted medical database, flagging any claims not supported by the source material.
Think of it like...
Like a fact-checker at a newspaper who reviews articles against source material before publication — they catch errors before they reach the audience.
Related Terms
Hallucination
When an AI model generates information that sounds plausible and confident but is factually incorrect, fabricated, or not grounded in its training data or provided context. The model essentially 'makes things up'.
Grounding
The practice of connecting AI model outputs to verifiable sources of information, ensuring responses are based on factual data rather than the model's potentially unreliable internal knowledge.
Evaluation
The systematic process of measuring an AI model's performance, safety, and reliability using various metrics, benchmarks, and testing methodologies.
Guardrails
Safety mechanisms and constraints built into AI systems to prevent harmful, inappropriate, or off-topic outputs. Guardrails can operate at the prompt, model, or output level.