Pattern
Stratified sampling for human review
Choose which outputs a human reviews by stratifying the stream — oversampling low-confidence, rare-category, and high-stakes outputs — rather than sampling at random, so a fixed reviewer budget catches the most errors and the most consequential ones.
Classification — Sampling the output stream for review by risk stratum instead of at random.
Full entry
The complete treatment — problem, how it works, when to use it, sources, and examples — lives in the volume below.
Browse the full pattern index or the catalog.