Fact Extraction Runbook

Operational runbook for diagnosing and resolving issues with AI fact extraction, evergreen generation, challenge content, and the seed pipeline.

Note: Content is pending — this is a stub created from the runbook template.

Overview

This runbook covers the fact extraction pipeline: AI-driven fact extraction from story clusters, evergreen fact generation, challenge content generation, and seed pipeline explosion.

Owner: fact-engineer

Escalation Path: architect-steward → on-call engineer

Quick Reference

MetricHealthyWarningCritical
Extraction queue depth< 2020-100> 100
AI cost (daily)< $3$3-5> $5
Extraction success rate> 90%70-90%< 70%

Diagnostic Decision Tree

Are facts being extracted?
├── No → Check worker-facts health endpoint
│   ├── Worker down → Restart worker
│   └── Worker up → Check EXTRACT_FACTS queue
│       ├── Queue empty → Check upstream clustering
│       └── Queue has messages → Check AI provider status
└── Yes → Check quality
    ├── Low notability scores → Review extraction prompts
    ├── Schema validation failures → Check fact_record_schemas
    └── High AI costs → Check model tier routing

Common Issues

To be documented as issues are encountered.