Transparency

Fleet RAG answer quality

Independent RAGAS evaluation with a disjoint GPT-5 judge (the answers are written by Claude Opus, so the judge is a different model family). Faithfulness is scored against the verbatim context the model actually retrieved. Each row is a RAG; the gauges show the mean score, with a tick at the pass threshold.

Pilot — 9 of 9 corpora evaluated so far. Rolling out across the fleet.

How each score is measured

Faithfulness

the AI's answer —checked vs— the retrieved sources

Every claim in the answer is grounded in a real retrieved source — no invented facts.

Answer relevancy

the AI's answer —checked vs— your question

The answer actually addresses what was asked, without padding or drift.

Context precision

the retrieved sources —checked vs— the reference answer

The right passages were retrieved and ranked first, not buried under noise.

Context recall

the reference answer —checked vs— the retrieved sources

All the evidence needed to answer was actually retrieved — nothing missed.

RAG	Faithfulness pass ≥ 0.75	Answer relevancy pass ≥ 0.70	Context precision pass ≥ 0.60	Context recall pass ≥ 0.60	N
Extraintestinal (EIM)					10
IBD-PSC					10
IBD-Unclassified					10
Crohn's — Large Bowel					12
Pouchology					11
Crohn's — Ileocolic					10
Ulcerative Colitis					10
Crohn's — Small Bowel					11
Perianal Crohn's					11

All runs: GPT-5 judge · Claude Opus answer generator. Each gauge fills to the score; the tick marks the pass threshold. Green = at/above, red = below.

For healthcare providers. AI-generated summaries may contain errors — verify against primary sources and clinical judgement. Not medical advice.