Retrieval-augmented generation (RAG) pairs a retrieval component with a generative model so answers can be grounded in documents. That combination is powerful, but it can also produce confident-sounding claims not supported by the retrieved sources — hallucinations. This guide gives a practical checklist to reduce hallucinations in RAG, plus reproducible tests and short examples you can run with minimal engineering overhead.
Why hallucinations happen in RAG
RAG hallucinations usually come from a broken chain between retrieval and generation. Common failure modes:
- Irrelevant or low-quality retrievals: the generator has bad or missing evidence.
- Poor prompt conditioning: the model is asked to synthesize beyond the available facts.
- Ambiguous or out-of-domain queries: retrieval returns documents that look relevant but don’t answer the question.
- Aggregation errors: the model mixes facts from different sources incorrectly (false synthesis).
- No provenance or weak citations: outputs have no way to be verified, so consumers accept hallucinations.
Prevention checklist — concrete measures
1. Query engineering and canonicalization
Before retrieving, normalize the user query: expand acronyms where appropriate, correct spelling of key entities, and detect question type (fact, timeline, procedure). For example, convert “ETA iOS 18” into “estimated release date iOS 18” if you’re handling product research.
- Detect entity mentions and prefer noun phrases for retrieval.
- Rewrite vague queries with clarifying questions if the system can prompt the user.
2. Source filtering and metadata constraints
Restrict retrieval by trusted sources and metadata when possible: document type, authoritativeness tags, or date ranges. Apply filters before vector similarity scoring to avoid noisy matches.
3. Retrieval tuning and reranking
Combine dense-vector similarity with a lightweight lexical reranker. Use a secondary re-rank step that uses cosine similarity, BM25 score, or a small cross-encoder to promote passages that actually contain the answer.
// Pseudocode: rerank top-k results
top_k = vector_search(query_vector, k=50)
reranked = cross_encoder_rerank(query, top_k)
selected = reranked[:5]
4. Prompting and explicit evidence requests
Design prompts that force the model to cite sources and to say when it doesn’t know. Use templates like:
Instruction: Answer briefly and cite the source(s) by ID. If unsupported, reply "Insufficient evidence." Query: {user_query} Context: {passages_with_ids}Require the model to emit a confidence marker and list of source IDs alongside any factual claims.
5. Confidence scoring and thresholding
Calculate a composite confidence score that combines retrieval similarity, reranker score, and model-level indicators (e.g., generation-likelihood proxies). Reject or fall back to a non-committal response when confidence is below threshold.
- Example composite score = 0.5 * mean(similarity_scores) + 0.3 * reranker_score + 0.2 * model_confidence
- Define thresholds for responses: answer, partial answer with caveat, ask for clarification.
6. Citation and provenance format
Make citations machine-readable: include source IDs, offsets, and a short verbatim snippet. That makes it possible to quickly verify or present the original passage to users.
7. Human review paths and escalation
For high-risk use cases, route low-confidence or high-impact answers to a human reviewer. Log the full retrieval stack for audits and corrections.
Reproducible tests you can run
Set up a small, versioned test suite to measure hallucination rates before and after changes. Keep the datasets and evaluations in source control so results are repeatable.
1. Known-answer (gold standard) test
Use a set of questions with clear, verifiable answers extracted from your corpus. Measure whether the RAG answer matches the gold answer and whether cited sources contain the answer.
- Create 200 question-answer pairs drawn from documents in your index.
- Run RAG and record: match (yes/no), cited source ID, and position of answer in source.
- Metric: supported-answer rate and supported-claim precision.
2. Retrieval quality and contamination test
Measure retrieval precision: for a sample of queries, annotate whether the top-N passages actually contain the answer. This isolates retrieval from generation.
3. Hallucination injection (adversarial) test
Introduce decoy documents that are lexically similar but factually incorrect and see if the model synthesizes incorrect claims. This reveals over-reliance on surface similarity.
4. Provenance consistency test
Check whether every factual claim in responses is tied to at least one cited source and whether that claim appears in the source without contradiction.
5. Confidence calibration test
Compare computed confidence scores against observed accuracy on your test set. Use calibration plots and adjust thresholds so confidence maps to empirical correctness.
No-code and short code examples
No-code checklist for business users
- Select trusted document collections and tag them with metadata (source, date, category).
- Configure your RAG tool to apply metadata filters for sensitive queries.
- Enable “show source” in the user UI and require a source list for factual answers.
- Set a fallback message for low-confidence responses like “I couldn’t find a reliable source for that.”
Minimal pseudo-code: selecting passages and scoring
// 1. canonicalize query
q = normalize(user_query)
// 2. retrieve vectors with pre-filter
candidates = vector_search(q, filter=trusted=true, k=50)
// 3. rerank and pick top passages
scores = cross_encoder(q, candidates)
top_passages = pick_top(scores, n=5)
// 4. ask generator with passages and require citations
response = generator(prompt_with(top_passages))
// 5. compute composite confidence and decide
if composite_confidence(response) < threshold:
return fallback_message
else:
return response
Team onboarding checklist to measure improvement
- Define the scope: which user flows are in-scope and what counts as a hallucination.
- Create or adopt a test suite: known-answer set, adversarial examples, and retrieval annotations.
- Establish baseline metrics: supported-answer precision, retrieval precision@5, and false-claim rate.
- Decide thresholds for acceptable performance and SLAs for human review turnaround.
- Instrument logging: store query, retrieved IDs, reranker scores, model output, and final decision for each request.
- Run automated tests on every model or retrieval change; gate deployments on non-regression tests.
- Schedule periodic audits and refresh the gold dataset to reflect evolving content.
Limitations and practical trade-offs
No approach eliminates hallucinations entirely. Strengthening retrieval and being conservative with confidence thresholds often increases "I don’t know" responses and may reduce utility in exploratory tasks. Cross-encoder rerankers and stricter filters cost latency and compute. Balance risk tolerance against user experience: high-risk answers should err on the side of caution, and low-risk exploratory workflows can favor recall.
Conclusion
Reducing hallucinations in RAG is a systems problem: better queries, smarter retrieval, explicit provenance, and calibrated confidence together reduce false claims. Implement a small, repeatable test suite and baseline metrics, then iterate with data-driven changes. The checklist and tests above provide a practical roadmap teams can adopt quickly to improve trustworthiness without sacrificing the core benefits of RAG.
FAQ
What exactly counts as a hallucination in RAG?
A hallucination is when the generator asserts a fact that isn’t supported by the retrieved documents or creates details that cannot be traced to provenance. In RAG, focus on whether claims are grounded in the cited passages.
How do I choose a confidence threshold?
Choose thresholds by running a calibration test: compare composite confidence scores to actual correctness on a validation set. Pick thresholds that meet your acceptable precision for the use case, and revisit as content changes.
Will adding more context always reduce hallucinations?
Not necessarily. Extra context helps if it’s relevant and high-quality. Irrelevant or noisy context can confuse the model and increase hallucination risk. Use filtering and reranking to keep context focused.
How often should we run these tests?
Run the full test suite on any model or retrieval change, and schedule lighter daily or weekly checks for production monitoring. Re-run adversarial and gold tests whenever you update your document index or add new content sources.