Testing Guide

This document covers unit testing, integration testing, and end-to-end verification for Eko.

Running Tests

Quick Commands

# Run all tests
bun run test

# Run tests with coverage
bun run test:coverage

# Run specific package tests
bun run test --filter=@eko/shared

# Run single test file
bun test packages/shared/src/url-normalization.test.ts

# Watch mode
bun test --watch

Test Types

Type	Location	Purpose
Unit	`*.test.ts` beside source	Test isolated functions
Integration	`__tests__/*.integration.test.ts`	Test component interactions
Security	`packages/db/src/__tests__/security/`	RLS policy validation
E2E	See verification checklist below	Full workflow validation

Coverage Thresholds

Coverage is configured per package based on criticality (per TST-005):

Package	Target	Rationale
`packages/shared`	80%	Core business logic
`packages/ai`	70%	Prompt logic testable
`packages/db`	60%	Query logic, integration focus
`packages/queue`	60%	Dispatch logic
`apps/worker-*`	60%	Integration test focus
`apps/web`	40%	E2E focus
`apps/admin`	40%	E2E focus

Environment Variables for Tests

Some tests require environment variables:

# Required for RLS security tests
SUPABASE_URL=...
SUPABASE_ANON_KEY=...
SUPABASE_SERVICE_ROLE_KEY=...

# Required for AI tests (can skip with fallback)
OPENAI_API_KEY=...
ANTHROPIC_API_KEY=...

Tests gracefully skip when env vars are missing (see db security tests for example).

CI Mode

The CI pipeline runs:

bun run ci
# Which includes: lint, typecheck, test

End-to-End Verification Checklist

This section provides checklists to verify the V2 fact engine pipelines work correctly.

Prerequisites

Supabase project created and configured
Environment variables set (see .env.example)
Database migrations applied (up to 0174)
Upstash Redis configured (or local fallback)
At least one AI provider key: OPENAI_API_KEY, ANTHROPIC_API_KEY, or GOOGLE_API_KEY

1. News Ingestion Pipeline

Trigger news ingestion:

curl -X POST http://localhost:3000/api/cron/ingest-news \
  -H "Authorization: Bearer $CRON_SECRET"

Start worker-ingest:
```
bun run --cwd apps/worker-ingest dev
```
Verify news_sources records created (articles fetched from providers)

Trigger clustering:

curl -X POST http://localhost:3000/api/cron/cluster-sweep \
  -H "Authorization: Bearer $CRON_SECRET"

Verify stories records created (articles grouped by similarity)

2. Fact Extraction

Start worker-facts:
```
bun run --cwd apps/worker-facts dev
```
Verify fact_records created from clustered stories with:
- facts JSONB populated with schema-validated key-value pairs
- notability_score >= 0.6 (extraction threshold)
- status = pending (awaiting validation)
- topic_category_id assigned

3. Fact Validation

Start worker-validate:
```
bun run --cwd apps/worker-validate dev
```
Verify VALIDATE_FACT messages consumed
Verify fact_records.validation JSONB updated with phase results
Verify fact_records.status changed to validated or rejected

4. Challenge Generation

Verify fact_challenge_content records created for validated facts with:
- style (multiple_choice, true_false, fill_blank, free_text, etc.)
- difficulty (C1-C5)
- challenge_title populated
- content JSONB with question/answer data

5. Image Resolution

Verify RESOLVE_IMAGE messages processed by worker-ingest
Verify fact_records.image_url populated with stock photo URL
For facts where image would spoil the answer: verify RESOLVE_CHALLENGE_IMAGE processed

6. Evergreen Pipeline

Trigger evergreen generation:

curl -X POST http://localhost:3000/api/cron/generate-evergreen \
  -H "Authorization: Bearer $CRON_SECRET"

Verify fact_records created with source_type: 'evergreen'
Verify these flow through validation and challenge generation

7. Feed & UI

Start web app: bun run dev:web
Verify /api/feed returns fact cards (blended: recent + review-due + evergreen)
Verify card detail page shows fact data and challenge content
Verify challenge sessions can be started and completed

8. Admin Dashboard

Start admin app: bun run dev:admin
Verify dashboard shows pipeline metrics
Verify content browser displays fact records
Verify queue page shows queue depths

Local Development Commands

bun install
bun run dev              # Start all apps (web + admin)
bun run dev:web          # Web app only (port 3000)
bun run dev:admin        # Admin app only

# Workers (run independently)
bun run --cwd apps/worker-ingest dev
bun run --cwd apps/worker-facts dev
bun run --cwd apps/worker-validate dev

# Trigger crons manually
curl -X POST http://localhost:3000/api/cron/ingest-news \
  -H "Authorization: Bearer $CRON_SECRET"
curl -X POST http://localhost:3000/api/cron/cluster-sweep \
  -H "Authorization: Bearer $CRON_SECRET"
curl -X POST http://localhost:3000/api/cron/generate-evergreen \
  -H "Authorization: Bearer $CRON_SECRET"

Database Queries for Debugging

-- Recent fact records
SELECT id, title, status, notability_score, source_type, created_at
FROM fact_records ORDER BY created_at DESC LIMIT 10;

-- Recent stories and their article count
SELECT id, title, status, article_count, created_at
FROM stories ORDER BY created_at DESC LIMIT 10;

-- News sources by provider
SELECT provider, COUNT(*) as count, MAX(published_at) as latest
FROM news_sources GROUP BY provider ORDER BY latest DESC;

-- Validation status distribution
SELECT status, COUNT(*) FROM fact_records GROUP BY status;

-- AI cost tracking (today)
SELECT model, feature, SUM(total_cost) as cost, SUM(total_calls) as calls
FROM ai_cost_tracking
WHERE date = CURRENT_DATE
GROUP BY model, feature ORDER BY cost DESC;

-- Recent ingestion runs
SELECT id, trigger_type, status, facts_created, duration_ms, created_at
FROM ingestion_runs ORDER BY created_at DESC LIMIT 5;

-- Challenge content for a fact
SELECT style, difficulty, challenge_title
FROM fact_challenge_content
WHERE fact_record_id = '<id>' ORDER BY style, difficulty;

Troubleshooting

Worker not processing messages

Check Upstash credentials in .env.local
Verify queue has messages: check Upstash dashboard or worker logs
Check for errors in worker startup logs
Verify SUPABASE_SERVICE_ROLE_KEY is set (workers bypass RLS)

AI extraction failures

Check ai_cost_tracking for spend cap violations
Verify at least one AI provider key is configured
Check if fact_record_schemas has a schema for the target topic category
Look for model router errors in worker-facts logs

Validation stuck in pending

Check evidence API availability (Wikipedia, Wikidata, etc.)
Verify GOOGLE_API_KEY is set (Gemini 2.5 Flash powers Phase 3-4)
Trigger validation retry: curl -X POST http://localhost:3000/api/cron/validation-retry -H "Authorization: Bearer $CRON_SECRET"

Challenge generation errors

Verify fact_record_schemas has cardFormats defined for the topic
Check that the fact has status: 'validated'
Look for schema validation errors in worker-facts logs

RLS errors

Ensure service role key used for workers
Verify auth token passed correctly for user API requests
Check Supabase dashboard for RLS policy issues

#Testing Guide

#Running Tests

#Quick Commands

#Test Types

#Coverage Thresholds

#Environment Variables for Tests

#CI Mode

#End-to-End Verification Checklist

#Prerequisites

#1. News Ingestion Pipeline

#2. Fact Extraction

#3. Fact Validation

#4. Challenge Generation

#5. Image Resolution

#6. Evergreen Pipeline

#7. Feed & UI

#8. Admin Dashboard

#Local Development Commands

#Database Queries for Debugging

#Troubleshooting

#Worker not processing messages

#AI extraction failures

#Validation stuck in pending

#Challenge generation errors

#RLS errors