Accuracy Tolerance Principle¶
The Question¶
When evaluating whether a workflow can use probabilistic AI (LLMs, agentic patterns) vs. requiring deterministic processing, ask: what happens when the output is wrong 2-5% of the time?
If humans review the output before acting on it → probabilistic is acceptable. If the output feeds directly into downstream systems without human review → deterministic processing is required.
Decision Framework¶
| Workflow | Accuracy Requirement | Tolerates AI? | Reasoning |
|---|---|---|---|
| Document text extraction | ~95% acceptable | Yes | OCR errors exist regardless; lawyers review extracted text before relying on it |
| Transcript summarization | ~95% acceptable | Yes | Summaries are presented alongside source material; humans verify before citing |
| Search indexing | Fuzzy by nature | Yes | Relevance ranking is inherently probabilistic; precision/recall tradeoffs are expected |
| Document classification | ~95% acceptable | Yes | Misclassified documents surface during review; bulk reclassification is available |
| PR code review | Advisory | Yes | Findings are suggestions, not automated actions; developer decides what to act on |
| Bates stamping | 100% required | No | Sequential numbering with zero gaps or duplicates; legally binding page identification |
| Billing / metering | 100% required | No | Financial accuracy required; errors create legal and contractual liability |
| Case database operations | 100% required | No | Data integrity is foundational; corrupted case data is unrecoverable |
| Event routing | 100% required | No | Misrouted events cause data loss or duplicate processing; must be deterministic |
| Migration / ETL | 100% required | No | Data transformation must be exact and reproducible; rollback depends on it |
The Pattern¶
Workflows split into two categories along a clear boundary:
Probabilistic-tolerant (AI-suitable): - Output is presented to humans for review before action - Errors are detectable and correctable after the fact - The alternative (no processing) is worse than imperfect processing - Volume makes manual processing impractical
Deterministic-required (traditional processing only): - Output feeds directly into downstream systems or legal records - Errors compound silently through the pipeline - Correctness is contractual or regulatory requirement - The system cannot self-detect when output is wrong
Applying This to New Features¶
When proposing a new feature that might use AI:
- Identify the output consumer — Is it a human reviewing results, or a downstream system consuming data programmatically?
- Assess error impact — If the output is wrong 2-5% of the time, does someone notice before harm occurs? Or does it silently propagate?
- Check for correction mechanisms — Can errors be found and fixed after the fact? Is there a feedback loop?
- Consider the alternative — Is the current manual process actually 100% accurate? If humans already make errors at similar rates, AI with human review may be a net improvement.
If the answer to #1 is "human" and #3 is "yes" → AI is appropriate. If the answer to #1 is "system" and #2 is "silently propagates" → deterministic only.
Relationship to Other Principles¶
- Guardrails pattern (
patterns/guardrails.md): Gate 2 (output verification) and Gate 3 (human approval) scale based on accuracy tolerance. Deterministic workflows need lightweight output checks; probabilistic workflows need stronger verification and human review gates. - Exception hierarchy (
patterns/exception-hierarchy.md): Deterministic workflows use Recoverable/Permanent/Silent routing. Probabilistic workflows may need a confidence threshold that routes low-confidence outputs to human review rather than the DLQ. - Checkpoint pipeline (
patterns/checkpoint-pipeline.md): Plan-and-Execute orchestration is deterministic by design. Probabilistic AI features should use the Hierarchical pattern (parallel agents + aggregation) instead.
Ask questions about Nextpoint architecture, patterns, rules, or any module. Powered by Claude Opus 4.6.