Judgement-led · Step 3 of 12

Reasoning

A team lead argues that because 90% of AI-assisted code reviews caught real bugs last quarter, the tool should be trusted on all production PRs. Which is the strongest counter-argument?

The 90% figure only reflects bugs caught, not bugs missed; without the base rate of bugs actually shipped, the false-negative rate is unknown.
The 90% figure came from last quarter's codebase; the tool's error profile will shift as your team adopts new frameworks and refactors modules the tool was not trained against.
A single quarter is too small a sample to draw statistical conclusions.
AI tools should not be trusted on production PRs regardless of performance data.

Illustrative item — the demo shows one representative question per section, not the full battery. In this persona, this answer anchors at L5 for COGNITIVE_PRESCREEN.

The real assessment runs a timed 10-item verbal critical-reasoning battery.