A practical rubric for checking documentation answers
Read the guide →Guide preview
Check correctness, source support, version and useful limits separately; fluent prose cannot compensate for a material error.
Check correctness, citations, useful refusals and held-out cases separately.
Check correctness, citations, useful refusals and held-out cases separately.
New to the topic? Begin with the first guide. Otherwise, go straight to the question you need to answer.
Check correctness, source support, version and useful limits separately; fluent prose cannot compensate for a material error.
Reserve a distinct set of questions so repeated tuning does not become the only evidence of improvement.
Open the cited passage and compare the claim; a plausible URL is not proof that the answer is grounded.
A good documentation assistant should recognize missing evidence and provide a useful route without inventing an answer.
Ask a short clarifying question when a missing detail changes the answer; do not force a clarification when the source already supports a safe response.
Check the source version and refresh path before changing the model or adding more instructions.
Check source presence, discoverability and question wording separately before broadening the corpus.
Classify failures by their consequences for the reader, not by how awkward the sentence sounds.
Compare the disputed claim with the expected source before averaging reviewers’ scores.
Keep a frozen baseline and rerun the cases affected by a change, plus important boundary checks.