#Quality Prompts

Prompts for challenge content quality: CQ rule enforcement, drift detection, defect repair, voice compliance, and soak testing.

#Prompts

#	Prompt	Purpose
01	Run Drift Detection	Run all drift detection tests
02	Audit Fact Quality	Audit fact quality and coverage
03	Full CI Validation	Comprehensive CI check
04	CQ-002 Compliance Patch	Fix challenge content CQ-002 violations
05	Run Soak Test	End-to-end pipeline health verification
06	Check Eko Invariants	Scan for core system invariant violations

#FAQ

#Styles and Formats

What are the challenge styles and which ones are pre-generated vs runtime?

• Six pre-generated styles: multiple_choice, direct_question, fill_the_gap, statement_blank, reverse_lookup, free_text.
• One runtime-only style: progressive_image_reveal (runtime image processing, CC-003).
• Pre-generated styles are defined as PRE_GENERATED_STYLES in packages/ai/src/challenge-content-rules.ts (data sourced from packages/ai/src/config/challenge-banned-patterns.ts).
• Pre-generated content is stored in the fact_challenge_content table; runtime styles are never persisted.

What are the five fields in every challenge content row?

• setup_text -- Backstory sharing context (2-4 sentences with specific details, min 50 chars).
• challenge_text -- Invitation to answer (must contain "you" or "your" per CQ-002, min 30 chars).
• reveal_correct -- Celebration when the user knows it (1-3 sentences, teaches an extra detail, min 30 chars).
• reveal_wrong -- Kind teaching when they don't (1-3 sentences, includes the correct answer, min 30 chars).
• correct_answer -- Rich narrative for animated streaming display (3-6 sentences, min 100 chars, the storytelling payoff).
• Plus style_data (JSONB, style-specific data like MC options) and difficulty (integer 1-5).
• Validation: packages/ai/src/challenge-content-rules.ts (validateChallengeContent()).

#Voice and Quality

What is the voice constitution and where is it defined?

• The voice constitution defines Eko's global tone: core feeling, three layers, forbidden patterns, and anti-pattern words.
• Source text: packages/ai/src/config/challenge-voice.ts (exports CHALLENGE_VOICE_CONSTITUTION).
• Runtime consumer: CHALLENGE_VOICE_CONSTITUTION constant in packages/ai/src/challenge-content-rules.ts.

What are the CQ rules (CQ-001 through CQ-013) and where are they enforced?

• CQ-001: setup_text must be 2-4 sentences with at least one specific detail (name, date, number, place).
• CQ-002: challenge_text must contain second-person address ("you"/"your"). Automated patching via patchCq002().
• CQ-003 through CQ-004: Reveal fields must celebrate or teach, never use binary labels like "Correct!" or "Wrong!".
• CQ-005: multiple_choice must have exactly 4 options with exactly 1 correct; no "N/A" or "All of the above" filler.
• CQ-006: Banned pattern scan across all text fields (see next question).
• CQ-007: free_text prompts must be open-ended, not recall-based.
• CQ-008: correct_answer must be 3-6 sentences (100+ chars), narrative storytelling payoff.
• CQ-009 through CQ-013: Per-style style_data shape requirements (fill_the_gap, direct_question, statement_blank, reverse_lookup, free_text).
• Enforced in: packages/ai/src/challenge-content-rules.ts (validateChallengeContent()).
• Full spec: docs/rules/challenge-content.md.

What are banned patterns and how do I add a new one?

• Regex patterns that trigger CQ-006 violations in generated content (e.g., \bTrivia\b, \bQuiz\b, \bEasy one\b).
• Source: packages/ai/src/config/challenge-banned-patterns.ts.
• To add a new pattern: edit the TypeScript file directly, adding a new entry with pattern and flags.
• The flags field accepts regex flags: i for case-insensitive, blank for default.
• Runtime: patterns are loaded as BANNED_PATTERNS (RegExp array) in packages/ai/src/challenge-content-rules.ts.

#Difficulty

How do difficulty levels work (1-5) and what does each level mean?

• Five tiers: 1=Easy (direct recall), 2=Medium (contextual reasoning), 3=Hard (connecting multiple pieces), 4=Expert (cross-domain reasoning), 5=Master (expert synthesis).
• Source: packages/ai/src/config/challenge-difficulty.ts.
• Each level includes AI prompt guidance describing the expected cognitive demand and style-specific instructions.
• Loaded as DIFFICULTY_LEVELS in packages/ai/src/challenge-content-rules.ts.

How is difficulty distributed during generation?

• Default: balanced spread across all 5 levels (difficulty=0 means "generate all levels").
• Can be overridden via the --difficulty N flag on scripts/seed/generate-challenge-content.ts.
• Seeding controls in docs/projects/seeding/SEED.md define the challenge_difficulty setting for batch runs.
• Per-fact difficulty is stored in fact_challenge_content.difficulty (integer 1-5).
• Distribution is per-style per-fact: each fact can have content at multiple difficulty levels for each style.

#Drift Detection

What are drift coordinators and how do they validate generated content?

• Five pluggable validators that check AI output against rules at generation time (all free heuristic checks, no LLM cost by default):
  • Voice drift: packages/ai/src/drift/voice-drift.ts -- voice constitution compliance.
  • Structure drift: packages/ai/src/drift/structure-drift.ts -- field structure and length rules.
  • Schema drift: packages/ai/src/drift/schema-drift.ts -- schema key compliance.
  • Taxonomy drift: packages/ai/src/drift/taxonomy-drift.ts -- topic-specific vocabulary and domain rules.
  • Difficulty drift: packages/ai/src/drift/difficulty-drift.ts -- difficulty level alignment.
• Orchestrator: packages/ai/src/drift/index.ts (runDriftCoordinators()).
• Runtime registration: registerDriftCoordinator() allows adding custom coordinators at runtime (e.g., for testing or plugins).
• Types: packages/ai/src/drift/types.ts (DriftCoordinator, DriftResult, DriftReport).

How do I run drift detection tests locally?

• Single coordinator: bunx vitest run packages/ai/src/drift/__tests__/voice-drift.test.ts.
• Another coordinator: bunx vitest run packages/ai/src/drift/__tests__/structure-drift.test.ts.
• All drift tests: bunx vitest run packages/ai/src/drift/__tests__/.
• Orchestrator tests: bunx vitest run packages/ai/src/drift/__tests__/orchestrator.test.ts.
• Tests validate that sample generated content passes (or correctly fails) drift coordinators.

What is the pass_threshold and what happens when content drifts?

• Each coordinator returns a 0.0-1.0 score; the orchestrator averages all coordinator scores into an aggregate_score.
• Content must exceed the pass_threshold (default 0.7) to pass. Configurable via DriftCoordinatorConfig.pass_threshold.
• Failed drift checks cause the content to be flagged in the DriftReport (passed: false, total_violations count).
• The blocking config option (default false) determines whether drift failures prevent content insertion.
• Config-driven: thresholds, which coordinators to run, and LLM-judge sampling rate are all adjustable without code changes via DriftCoordinatorConfig.

#Generation and Repair

How do I regenerate challenge content for existing facts?

• Audit coverage: bun scripts/seed/generate-challenge-content.ts --audit.
• Export facts missing content: bun scripts/seed/generate-challenge-content.ts --export.
• Generate content: bun scripts/seed/generate-challenge-content.ts --generate --limit 100 --styles multiple_choice,direct_question.
• Upload to DB: bun scripts/seed/generate-challenge-content.ts --upload.
• Validate post-upload: bun scripts/seed/generate-challenge-content.ts --validate.
• Add --dry-run to any phase to preview without writing. Use --difficulty N to target a specific level.
• Worker handler (for pipeline-triggered generation): apps/worker-facts/src/handlers/generate-challenge-content.ts.

How does the defect rewrite pipeline work?

• Script: bun scripts/seed/rewrite-challenge-defects.ts -- finds CQ violations and rewrites affected content.
• Scans fact_challenge_content rows for CQ-rule failures (banned patterns, missing second-person, short fields, etc.).
• Uses AI to rewrite only the failing fields, preserving correct content in the same row.
• The patchCq002 layer handles the most common defect (missing "you"/"your") without requiring AI rewrite.

#See Also

• Challenge Content Rules -- Full CC and CQ rule spec
• Fact-Challenge Anatomy -- 6 concepts, 5 layers
• Key source: packages/ai/src/challenge-content.ts, packages/ai/src/challenge-content-rules.ts