Eko v2: News Aggregation + Structured Fact Engine — Implementation Plan
For Claude: REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan task-by-task.
Goal: Pivot Eko from page change detection to a knowledge platform powered by structured fact records, template-based card variations, and news aggregation.
Architecture: New tables (topic_categories, fact_record_schemas, fact_records, stories, news_sources, card_interactions, card_bookmarks, ingestion_runs) alongside existing auth/billing tables. 3 new workers (ingest, facts, validate) replace 3 old workers (tracker, render, sms). Feed UI is public; card detail is subscription-gated behind a simplified Free/Eko+ model.
Tech Stack: Bun, TypeScript, Drizzle ORM, Supabase (Postgres + RLS), Upstash Redis, Next.js 16 App Router, Tailwind v4, shadcn/ui, Vercel AI SDK v6, Vitest
Design Doc: See resilient-giggling-mccarthy-v1 (the parent plan document) for full rationale, cost projections, and architectural decisions.
Phase Overview
| Phase | Focus | Dependency |
|---|---|---|
| Phase 1 | Database migrations (0092-0096) | None |
| Phase 2 | Package infrastructure (queue, AI, config, shared types, Drizzle schema) | Phase 1 |
| Phase 3 | Workers (ingest, facts, validate) + cron routes | Phase 2 |
| Phase 4 | UI — Feed (card grid, category filter, feed API) | Phase 2 |
| Phase 5 | UI — Detail + Subscription (paywall, trivia, Stripe) | Phase 4 |
| Phase 6 | Remove legacy (old routes, workers, DB queries, table drops) | Phase 3-5 |
| Phase 7 | Polish (auth redirects, onboarding, marketing site, admin) | Phase 6 |
Phase 1: Database Migrations
Task 1: Migration 0092 — Create Enums + topic_categories + fact_record_schemas
Files:
- Create:
supabase/migrations/0092_fact_engine_enums_and_taxonomy.sql
Step 1: Write the migration SQL
-- ============================================================================
-- MIGRATION 0092: Fact Engine Enums + Topic Taxonomy
--
-- Creates the foundational types and taxonomy tables for the fact engine.
-- Topic categories form a hierarchical tree (materialized path pattern).
-- Fact record schemas define expected JSON keys per topic type.
--
-- Changes:
-- - Create news_provider enum
-- - Create story_status enum
-- - Create fact_record_status enum
-- - Create card_format enum
-- - Create trivia_difficulty enum
-- - Create validation_strategy enum
-- - Create interaction_type enum
-- - Create topic_categories table (hierarchical taxonomy)
-- - Create fact_record_schemas table (per-topic JSON shape)
-- - Enable RLS on both tables
-- ============================================================================
-- 1. Enums
CREATE TYPE news_provider AS ENUM (
'newsapi', 'google_news', 'bing_news', 'gnews', 'manual', 'api_import'
);
CREATE TYPE story_status AS ENUM (
'clustering', 'published', 'archived', 'suppressed'
);
CREATE TYPE fact_record_status AS ENUM (
'pending_validation', 'validated', 'rejected', 'archived'
);
CREATE TYPE card_format AS ENUM (
'statement_blank', 'direct_question', 'fill_the_gap', 'multiple_choice', 'reverse_lookup'
);
CREATE TYPE trivia_difficulty AS ENUM (
'easy', 'medium', 'hard'
);
CREATE TYPE validation_strategy AS ENUM (
'authoritative_api', 'multi_source', 'curated_database', 'ai_cross_check'
);
CREATE TYPE interaction_type AS ENUM (
'viewed', 'expanded', 'answered', 'bookmarked', 'shared'
);
-- 2. Topic Categories (hierarchical taxonomy)
CREATE TABLE topic_categories (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
slug TEXT NOT NULL UNIQUE,
name TEXT NOT NULL,
parent_id UUID REFERENCES topic_categories(id),
depth INTEGER NOT NULL DEFAULT 0,
path TEXT NOT NULL,
icon TEXT,
color TEXT,
daily_quota INTEGER,
percent_target REAL,
is_active BOOLEAN NOT NULL DEFAULT true,
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
updated_at TIMESTAMPTZ NOT NULL DEFAULT now()
);
CREATE UNIQUE INDEX idx_topic_categories_path ON topic_categories(path);
CREATE INDEX idx_topic_categories_parent ON topic_categories(parent_id);
CREATE INDEX idx_topic_categories_active ON topic_categories(is_active) WHERE is_active = true;
-- 3. Fact Record Schemas (defines expected JSON structure per topic type)
CREATE TABLE fact_record_schemas (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
topic_category_id UUID NOT NULL REFERENCES topic_categories(id),
schema_name TEXT NOT NULL,
fact_keys JSONB NOT NULL,
card_formats card_format[] NOT NULL,
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
updated_at TIMESTAMPTZ NOT NULL DEFAULT now()
);
CREATE INDEX idx_fact_record_schemas_topic ON fact_record_schemas(topic_category_id);
CREATE UNIQUE INDEX idx_fact_record_schemas_name ON fact_record_schemas(topic_category_id, schema_name);
-- 4. RLS
ALTER TABLE topic_categories ENABLE ROW LEVEL SECURITY;
ALTER TABLE fact_record_schemas ENABLE ROW LEVEL SECURITY;
-- Public read for topic categories (taxonomy is public)
CREATE POLICY "topic_categories_public_read"
ON topic_categories FOR SELECT
TO authenticated, anon
USING (true);
-- Service role write for topic categories
CREATE POLICY "topic_categories_service_write"
ON topic_categories FOR ALL
TO service_role
USING (true) WITH CHECK (true);
-- Public read for schemas (needed for card rendering)
CREATE POLICY "fact_record_schemas_public_read"
ON fact_record_schemas FOR SELECT
TO authenticated, anon
USING (true);
-- Service role write for schemas
CREATE POLICY "fact_record_schemas_service_write"
ON fact_record_schemas FOR ALL
TO service_role
USING (true) WITH CHECK (true);
Step 2: Run migration index
Run: bun run migrations:index
Expected: Regenerated supabase/migrations-index.md includes 0092
Step 3: Commit
git add supabase/migrations/0092_fact_engine_enums_and_taxonomy.sql
git commit -m "feat(db): add fact engine enums + topic taxonomy (migration 0092)"
Task 2: Migration 0093 — Create fact_records table
Files:
- Create:
supabase/migrations/0093_fact_records.sql
Step 1: Write the migration SQL
-- ============================================================================
-- MIGRATION 0093: Fact Records Table
--
-- The core data asset of the fact engine. Each record is a structured JSON
-- collection of facts about a single notable event, person, or entity.
-- Card variations are computed at query time from these records.
--
-- Changes:
-- - Create fact_records table with validation, source tracking, and AI metadata
-- - Add indexes for feed queries, status filtering, and source lookups
-- - Enable RLS with public SELECT for validated records
-- ============================================================================
CREATE TABLE fact_records (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
topic_category_id UUID NOT NULL REFERENCES topic_categories(id),
schema_id UUID NOT NULL REFERENCES fact_record_schemas(id),
facts JSONB NOT NULL,
title TEXT NOT NULL,
notability_score REAL,
notability_reason TEXT,
context TEXT,
image_url TEXT,
image_source TEXT,
image_attribution TEXT,
image_resolved_at TIMESTAMPTZ,
-- Validation
status fact_record_status NOT NULL DEFAULT 'pending_validation',
validation JSONB,
-- Source tracking
source_type TEXT NOT NULL CHECK (source_type IN ('api_import', 'news_extraction', 'ai_generated', 'manual')),
source_story_id UUID, -- FK added after stories table exists (migration 0094)
external_source_id TEXT,
-- AI metadata
ai_model TEXT,
generation_cost_usd NUMERIC(10, 6),
-- Lifecycle
published_at TIMESTAMPTZ,
expires_at TIMESTAMPTZ,
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
updated_at TIMESTAMPTZ NOT NULL DEFAULT now()
);
-- Feed query: validated records by topic
CREATE INDEX idx_fact_records_topic_status
ON fact_records(topic_category_id, status);
-- Feed query: chronological published records
CREATE INDEX idx_fact_records_status_published
ON fact_records(status, published_at DESC)
WHERE status = 'validated';
-- Source story lookup
CREATE INDEX idx_fact_records_source_story
ON fact_records(source_story_id)
WHERE source_story_id IS NOT NULL;
-- External source dedup
CREATE INDEX idx_fact_records_external_source
ON fact_records(external_source_id)
WHERE external_source_id IS NOT NULL;
-- Notability filtering
CREATE INDEX idx_fact_records_notability
ON fact_records(notability_score DESC)
WHERE status = 'validated';
-- RLS
ALTER TABLE fact_records ENABLE ROW LEVEL SECURITY;
-- Public read for validated records (feed is public)
CREATE POLICY "fact_records_public_read"
ON fact_records FOR SELECT
TO authenticated, anon
USING (status = 'validated');
-- Service role full access
CREATE POLICY "fact_records_service_write"
ON fact_records FOR ALL
TO service_role
USING (true) WITH CHECK (true);
Step 2: Run migration index
Run: bun run migrations:index
Step 3: Commit
git add supabase/migrations/0093_fact_records.sql
git commit -m "feat(db): add fact_records table (migration 0093)"
Task 3: Migration 0094 — Create stories + news_sources tables
Files:
- Create:
supabase/migrations/0094_stories_and_news_sources.sql
Step 1: Write the migration SQL
-- ============================================================================
-- MIGRATION 0094: Stories + News Sources
--
-- News sources are individual articles from external APIs.
-- Stories are aggregated/deduplicated clusters of related articles.
-- Stories feed into fact extraction when source_count >= 3.
--
-- Changes:
-- - Create stories table (aggregated news clusters)
-- - Create news_sources table (individual articles)
-- - Add FK from fact_records.source_story_id → stories.id
-- - Enable RLS on both tables
-- ============================================================================
-- 1. Stories (aggregated news clusters)
CREATE TABLE stories (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
slug TEXT UNIQUE,
headline TEXT NOT NULL,
summary TEXT,
source_count INTEGER NOT NULL DEFAULT 1,
source_domains TEXT[],
first_seen_at TIMESTAMPTZ NOT NULL DEFAULT now(),
last_seen_at TIMESTAMPTZ NOT NULL DEFAULT now(),
category TEXT,
tags TEXT[],
is_validated BOOLEAN NOT NULL DEFAULT false,
status story_status NOT NULL DEFAULT 'clustering',
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
updated_at TIMESTAMPTZ NOT NULL DEFAULT now()
);
CREATE INDEX idx_stories_status_published
ON stories(status, last_seen_at DESC);
CREATE INDEX idx_stories_source_count
ON stories(source_count DESC)
WHERE status IN ('clustering', 'published');
-- 2. News Sources (individual articles from APIs)
CREATE TABLE news_sources (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
provider news_provider NOT NULL,
external_id TEXT,
source_name TEXT,
source_domain TEXT,
title TEXT NOT NULL,
description TEXT,
article_url TEXT NOT NULL,
image_url TEXT,
published_at TIMESTAMPTZ,
story_id UUID REFERENCES stories(id),
content_hash TEXT,
created_at TIMESTAMPTZ NOT NULL DEFAULT now()
);
CREATE UNIQUE INDEX idx_news_sources_provider_external
ON news_sources(provider, external_id)
WHERE external_id IS NOT NULL;
CREATE INDEX idx_news_sources_story
ON news_sources(story_id)
WHERE story_id IS NOT NULL;
CREATE INDEX idx_news_sources_published
ON news_sources(published_at DESC);
-- 3. Add FK from fact_records to stories
ALTER TABLE fact_records
ADD CONSTRAINT fk_fact_records_source_story
FOREIGN KEY (source_story_id) REFERENCES stories(id);
-- 4. RLS
ALTER TABLE stories ENABLE ROW LEVEL SECURITY;
ALTER TABLE news_sources ENABLE ROW LEVEL SECURITY;
-- Stories: public read for published
CREATE POLICY "stories_public_read"
ON stories FOR SELECT
TO authenticated, anon
USING (status = 'published');
CREATE POLICY "stories_service_write"
ON stories FOR ALL
TO service_role
USING (true) WITH CHECK (true);
-- News sources: admin/service only
CREATE POLICY "news_sources_service_only"
ON news_sources FOR ALL
TO service_role
USING (true) WITH CHECK (true);
Step 2: Run migration index
Run: bun run migrations:index
Step 3: Commit
git add supabase/migrations/0094_stories_and_news_sources.sql
git commit -m "feat(db): add stories + news_sources tables (migration 0094)"
Task 4: Migration 0095 — Create card_interactions + card_bookmarks + ingestion_runs
Files:
- Create:
supabase/migrations/0095_card_interactions_and_ingestion.sql
Step 1: Write the migration SQL
-- ============================================================================
-- MIGRATION 0095: Card Interactions, Bookmarks, and Ingestion Runs
--
-- User-facing interaction tracking for the card/trivia system.
-- Ingestion runs provide an audit log for the content pipeline.
--
-- Changes:
-- - Create card_interactions table (views, answers, bookmarks)
-- - Create card_bookmarks table (saved fact records)
-- - Create ingestion_runs table (pipeline audit log)
-- - Enable RLS with user-scoped policies
-- ============================================================================
-- 1. Card Interactions (trivia answers, views, bookmarks)
CREATE TABLE card_interactions (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
user_id UUID NOT NULL REFERENCES profiles(id) ON DELETE CASCADE,
fact_record_id UUID NOT NULL REFERENCES fact_records(id) ON DELETE CASCADE,
interaction_type interaction_type NOT NULL,
-- Trivia-specific fields
card_format card_format,
difficulty trivia_difficulty,
hidden_fact_key TEXT,
selected_answer TEXT,
correct_answer TEXT,
is_correct BOOLEAN,
time_to_answer_ms INTEGER,
-- Scoring & spaced repetition
scoring_method TEXT NOT NULL DEFAULT 'auto' CHECK (scoring_method IN ('auto', 'self_graded')),
next_review_at TIMESTAMPTZ,
review_count INTEGER NOT NULL DEFAULT 0,
streak INTEGER NOT NULL DEFAULT 0,
created_at TIMESTAMPTZ NOT NULL DEFAULT now()
);
CREATE INDEX idx_card_interactions_user_time
ON card_interactions(user_id, created_at DESC);
CREATE INDEX idx_card_interactions_fact
ON card_interactions(fact_record_id);
CREATE INDEX idx_card_interactions_review
ON card_interactions(user_id, next_review_at)
WHERE next_review_at IS NOT NULL;
CREATE INDEX idx_card_interactions_user_fact_type
ON card_interactions(user_id, fact_record_id, interaction_type);
-- 2. Card Bookmarks (saved fact records)
CREATE TABLE card_bookmarks (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
user_id UUID NOT NULL REFERENCES profiles(id) ON DELETE CASCADE,
fact_record_id UUID NOT NULL REFERENCES fact_records(id) ON DELETE CASCADE,
note TEXT,
created_at TIMESTAMPTZ NOT NULL DEFAULT now()
);
CREATE UNIQUE INDEX idx_card_bookmarks_user_fact
ON card_bookmarks(user_id, fact_record_id);
-- 3. Ingestion Runs (pipeline audit log)
CREATE TABLE ingestion_runs (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
run_type TEXT NOT NULL,
started_at TIMESTAMPTZ NOT NULL DEFAULT now(),
completed_at TIMESTAMPTZ,
status TEXT NOT NULL DEFAULT 'running' CHECK (status IN ('running', 'completed', 'failed')),
records_processed INTEGER NOT NULL DEFAULT 0,
records_created INTEGER NOT NULL DEFAULT 0,
records_failed INTEGER NOT NULL DEFAULT 0,
error_message TEXT,
metadata JSONB,
created_at TIMESTAMPTZ NOT NULL DEFAULT now()
);
CREATE INDEX idx_ingestion_runs_type_time
ON ingestion_runs(run_type, started_at DESC);
-- 4. RLS
ALTER TABLE card_interactions ENABLE ROW LEVEL SECURITY;
ALTER TABLE card_bookmarks ENABLE ROW LEVEL SECURITY;
ALTER TABLE ingestion_runs ENABLE ROW LEVEL SECURITY;
-- Card interactions: user can only see/create their own
CREATE POLICY "card_interactions_user_read"
ON card_interactions FOR SELECT
TO authenticated
USING ((select auth.uid()) = user_id);
CREATE POLICY "card_interactions_user_insert"
ON card_interactions FOR INSERT
TO authenticated
WITH CHECK ((select auth.uid()) = user_id);
CREATE POLICY "card_interactions_service_write"
ON card_interactions FOR ALL
TO service_role
USING (true) WITH CHECK (true);
-- Card bookmarks: user can only see/manage their own
CREATE POLICY "card_bookmarks_user_read"
ON card_bookmarks FOR SELECT
TO authenticated
USING ((select auth.uid()) = user_id);
CREATE POLICY "card_bookmarks_user_insert"
ON card_bookmarks FOR INSERT
TO authenticated
WITH CHECK ((select auth.uid()) = user_id);
CREATE POLICY "card_bookmarks_user_delete"
ON card_bookmarks FOR DELETE
TO authenticated
USING ((select auth.uid()) = user_id);
CREATE POLICY "card_bookmarks_service_write"
ON card_bookmarks FOR ALL
TO service_role
USING (true) WITH CHECK (true);
-- Ingestion runs: service role only
CREATE POLICY "ingestion_runs_service_only"
ON ingestion_runs FOR ALL
TO service_role
USING (true) WITH CHECK (true);
-- Admin read access for ingestion runs
CREATE POLICY "ingestion_runs_authenticated_read"
ON ingestion_runs FOR SELECT
TO authenticated
USING (true);
Step 2: Run migration index
Run: bun run migrations:index
Step 3: Commit
git add supabase/migrations/0095_card_interactions_and_ingestion.sql
git commit -m "feat(db): add card_interactions, card_bookmarks, ingestion_runs (migration 0095)"
Task 5: Migration 0096 — Update plan_definitions + seed topic taxonomy
Files:
- Create:
supabase/migrations/0096_plan_updates_and_seed_taxonomy.sql
Step 1: Write the migration SQL
-- ============================================================================
-- MIGRATION 0096: Plan Definition Updates + Seed Topic Taxonomy
--
-- Updates plan_definitions for the new Free/Eko+ model.
-- Seeds the initial topic category hierarchy and fact record schemas.
--
-- Changes:
-- - Add columns to plan_definitions: can_access_card_details, trial_days, max_bookmarks
-- - Deactivate old plans (base, pro, team)
-- - Insert new plan: plus (Eko+)
-- - Update free plan with new columns
-- - Seed topic_categories hierarchy
-- - Seed fact_record_schemas for initial topic types
-- ============================================================================
-- 1. Add new columns to plan_definitions
ALTER TABLE plan_definitions
ADD COLUMN IF NOT EXISTS can_access_card_details BOOLEAN NOT NULL DEFAULT false,
ADD COLUMN IF NOT EXISTS trial_days INTEGER NOT NULL DEFAULT 0,
ADD COLUMN IF NOT EXISTS max_bookmarks INTEGER NOT NULL DEFAULT 0;
-- 2. Deactivate old plans
UPDATE plan_definitions
SET is_active = false, updated_at = now()
WHERE plan_key IN ('base', 'pro', 'team');
-- 3. Update free plan
UPDATE plan_definitions
SET can_access_card_details = false,
trial_days = 0,
max_bookmarks = 0,
updated_at = now()
WHERE plan_key = 'free';
-- 4. Insert Eko+ plan
INSERT INTO plan_definitions (
plan_key, display_name, max_tracked_urls,
can_access_trends, can_access_lifetime_history,
allowed_cadences, sort_order, is_active,
can_access_card_details, trial_days, max_bookmarks
) VALUES (
'plus', 'Eko+', 0,
false, false,
ARRAY['daily']::TEXT[], 1, true,
true, 30, 100
) ON CONFLICT (plan_key) DO UPDATE SET
display_name = EXCLUDED.display_name,
can_access_card_details = EXCLUDED.can_access_card_details,
trial_days = EXCLUDED.trial_days,
max_bookmarks = EXCLUDED.max_bookmarks,
is_active = EXCLUDED.is_active,
updated_at = now();
-- 5. Seed topic categories (hierarchical)
-- Root categories
INSERT INTO topic_categories (slug, name, parent_id, depth, path, icon, color, daily_quota, percent_target) VALUES
('sports', 'Sports', NULL, 0, 'sports', 'trophy', '#3b82f6', 30, 25),
('history', 'History', NULL, 0, 'history', 'landmark', '#8b5cf6', 20, 20),
('culture', 'Culture', NULL, 0, 'culture', 'palette', '#ec4899', 15, 15),
('records', 'Records', NULL, 0, 'records', 'medal', '#f59e0b', 15, 15),
('geography', 'Geography', NULL, 0, 'geography', 'globe', '#10b981', 10, 10),
('science', 'Science', NULL, 0, 'science', 'flask-conical','#06b6d4', 10, 10),
('current-events', 'Current Events', NULL, 0, 'current-events', 'newspaper', '#6366f1', 10, 5);
-- Sports sub-categories
INSERT INTO topic_categories (slug, name, parent_id, depth, path, icon, color)
SELECT 'basketball', 'Basketball', id, 1, 'sports/basketball', 'circle', '#3b82f6'
FROM topic_categories WHERE slug = 'sports';
INSERT INTO topic_categories (slug, name, parent_id, depth, path, icon, color)
SELECT 'soccer', 'Soccer', id, 1, 'sports/soccer', 'circle', '#3b82f6'
FROM topic_categories WHERE slug = 'sports';
INSERT INTO topic_categories (slug, name, parent_id, depth, path, icon, color)
SELECT 'football', 'Football', id, 1, 'sports/football', 'circle', '#3b82f6'
FROM topic_categories WHERE slug = 'sports';
INSERT INTO topic_categories (slug, name, parent_id, depth, path, icon, color)
SELECT 'baseball', 'Baseball', id, 1, 'sports/baseball', 'circle', '#3b82f6'
FROM topic_categories WHERE slug = 'sports';
-- Culture sub-categories
INSERT INTO topic_categories (slug, name, parent_id, depth, path, icon, color)
SELECT 'quotes', 'Quotes', id, 1, 'culture/quotes', 'quote', '#ec4899'
FROM topic_categories WHERE slug = 'culture';
INSERT INTO topic_categories (slug, name, parent_id, depth, path, icon, color)
SELECT 'music', 'Music', id, 1, 'culture/music', 'music', '#ec4899'
FROM topic_categories WHERE slug = 'culture';
INSERT INTO topic_categories (slug, name, parent_id, depth, path, icon, color)
SELECT 'movies', 'Movies', id, 1, 'culture/movies', 'film', '#ec4899'
FROM topic_categories WHERE slug = 'culture';
-- History sub-categories
INSERT INTO topic_categories (slug, name, parent_id, depth, path, icon, color)
SELECT 'events', 'Historical Events', id, 1, 'history/events', 'scroll', '#8b5cf6'
FROM topic_categories WHERE slug = 'history';
-- Records sub-categories
INSERT INTO topic_categories (slug, name, parent_id, depth, path, icon, color)
SELECT 'human-achievement', 'Human Achievement', id, 1, 'records/human-achievement', 'award', '#f59e0b'
FROM topic_categories WHERE slug = 'records';
INSERT INTO topic_categories (slug, name, parent_id, depth, path, icon, color)
SELECT 'nature', 'Nature', id, 1, 'records/nature', 'leaf', '#f59e0b'
FROM topic_categories WHERE slug = 'records';
-- Geography sub-categories
INSERT INTO topic_categories (slug, name, parent_id, depth, path, icon, color)
SELECT 'countries', 'Countries', id, 1, 'geography/countries', 'flag', '#10b981'
FROM topic_categories WHERE slug = 'geography';
-- Science sub-categories
INSERT INTO topic_categories (slug, name, parent_id, depth, path, icon, color)
SELECT 'space', 'Space', id, 1, 'science/space', 'rocket', '#06b6d4'
FROM topic_categories WHERE slug = 'science';
-- 6. Seed fact record schemas
-- Game Result schema (for sports)
INSERT INTO fact_record_schemas (topic_category_id, schema_name, fact_keys, card_formats)
SELECT id, 'game_result',
'[
{"key": "team_home", "label": "Home Team", "type": "text", "required": true},
{"key": "team_away", "label": "Away Team", "type": "text", "required": true},
{"key": "date", "label": "Date", "type": "date", "required": true},
{"key": "location", "label": "Location", "type": "text", "required": true},
{"key": "score_home", "label": "Home Score", "type": "number", "required": true},
{"key": "score_away", "label": "Away Score", "type": "number", "required": true},
{"key": "winner", "label": "Winner", "type": "text", "required": true},
{"key": "largest_lead", "label": "Largest Lead", "type": "number", "required": false},
{"key": "top_scorer", "label": "Top Scorer", "type": "text", "required": false},
{"key": "top_scorer_points", "label": "Top Scorer Points", "type": "number", "required": false}
]'::jsonb,
ARRAY['statement_blank', 'direct_question', 'fill_the_gap', 'multiple_choice', 'reverse_lookup']::card_format[]
FROM topic_categories WHERE slug = 'sports';
-- Famous Quote schema (for culture/quotes)
INSERT INTO fact_record_schemas (topic_category_id, schema_name, fact_keys, card_formats)
SELECT id, 'famous_quote',
'[
{"key": "quote", "label": "Quote", "type": "text", "required": true},
{"key": "author", "label": "Author", "type": "text", "required": true},
{"key": "known_date", "label": "Known Date", "type": "text", "required": false},
{"key": "subject", "label": "Subject", "type": "text", "required": false},
{"key": "topic", "label": "Topic", "type": "text", "required": false}
]'::jsonb,
ARRAY['statement_blank', 'direct_question', 'multiple_choice', 'reverse_lookup']::card_format[]
FROM topic_categories WHERE slug = 'quotes';
-- Record Holder schema (for records)
INSERT INTO fact_record_schemas (topic_category_id, schema_name, fact_keys, card_formats)
SELECT id, 'record_holder',
'[
{"key": "record_type", "label": "Record Type", "type": "text", "required": true},
{"key": "record_description", "label": "Description", "type": "text", "required": true},
{"key": "record_value", "label": "Record Value", "type": "text", "required": true},
{"key": "holder", "label": "Holder", "type": "text", "required": true},
{"key": "date", "label": "Date", "type": "date", "required": false},
{"key": "location", "label": "Location", "type": "text", "required": false},
{"key": "previous_holder", "label": "Previous Holder", "type": "text", "required": false}
]'::jsonb,
ARRAY['statement_blank', 'direct_question', 'fill_the_gap', 'multiple_choice', 'reverse_lookup']::card_format[]
FROM topic_categories WHERE slug = 'records';
-- Country Facts schema (for geography/countries)
INSERT INTO fact_record_schemas (topic_category_id, schema_name, fact_keys, card_formats)
SELECT id, 'country_fact',
'[
{"key": "country", "label": "Country", "type": "text", "required": true},
{"key": "capital", "label": "Capital", "type": "text", "required": true},
{"key": "continent", "label": "Continent", "type": "text", "required": true},
{"key": "population", "label": "Population", "type": "number", "required": false},
{"key": "area_km2", "label": "Area (km²)", "type": "number", "required": false},
{"key": "language", "label": "Official Language", "type": "text", "required": false},
{"key": "currency", "label": "Currency", "type": "text", "required": false}
]'::jsonb,
ARRAY['statement_blank', 'direct_question', 'fill_the_gap', 'multiple_choice', 'reverse_lookup']::card_format[]
FROM topic_categories WHERE slug = 'countries';
-- Historical Event schema
INSERT INTO fact_record_schemas (topic_category_id, schema_name, fact_keys, card_formats)
SELECT id, 'historical_event',
'[
{"key": "event_name", "label": "Event", "type": "text", "required": true},
{"key": "date", "label": "Date", "type": "text", "required": true},
{"key": "location", "label": "Location", "type": "text", "required": true},
{"key": "key_figures", "label": "Key Figures", "type": "text", "required": false},
{"key": "outcome", "label": "Outcome", "type": "text", "required": false},
{"key": "significance", "label": "Significance", "type": "text", "required": false}
]'::jsonb,
ARRAY['statement_blank', 'direct_question', 'fill_the_gap', 'multiple_choice']::card_format[]
FROM topic_categories WHERE slug = 'events';
Step 2: Run migration index
Run: bun run migrations:index
Step 3: Commit
git add supabase/migrations/0096_plan_updates_and_seed_taxonomy.sql
git commit -m "feat(db): update plan_definitions + seed topic taxonomy (migration 0096)"
Task 6: Update Drizzle ORM schema with new tables
Files:
- Modify:
packages/db/src/drizzle/schema.ts
Step 1: Add new enums to Drizzle schema
Add after existing enum definitions:
// ============================================================================
// FACT ENGINE ENUMS
// ============================================================================
export const newsProviderEnum = pgEnum('news_provider', [
'newsapi', 'google_news', 'bing_news', 'gnews', 'manual', 'api_import',
])
export const storyStatusEnum = pgEnum('story_status', [
'clustering', 'published', 'archived', 'suppressed',
])
export const factRecordStatusEnum = pgEnum('fact_record_status', [
'pending_validation', 'validated', 'rejected', 'archived',
])
export const cardFormatEnum = pgEnum('card_format', [
'statement_blank', 'direct_question', 'fill_the_gap', 'multiple_choice', 'reverse_lookup',
])
export const triviaDifficultyEnum = pgEnum('trivia_difficulty', [
'easy', 'medium', 'hard',
])
export const validationStrategyEnum = pgEnum('validation_strategy', [
'authoritative_api', 'multi_source', 'curated_database', 'ai_cross_check',
])
export const interactionTypeEnum = pgEnum('interaction_type', [
'viewed', 'expanded', 'answered', 'bookmarked', 'shared',
])
Step 2: Add new table definitions
// ============================================================================
// FACT ENGINE TABLES
// ============================================================================
export const topicCategories = pgTable('topic_categories', {
id: uuid('id').primaryKey().defaultRandom(),
slug: text('slug').notNull().unique(),
name: text('name').notNull(),
parentId: uuid('parent_id').references(() => topicCategories.id),
depth: integer('depth').notNull().default(0),
path: text('path').notNull(),
icon: text('icon'),
color: text('color'),
dailyQuota: integer('daily_quota'),
percentTarget: real('percent_target'),
isActive: boolean('is_active').notNull().default(true),
createdAt: timestamp('created_at', { withTimezone: true }).notNull().defaultNow(),
updatedAt: timestamp('updated_at', { withTimezone: true }).notNull().defaultNow(),
})
export const factRecordSchemas = pgTable('fact_record_schemas', {
id: uuid('id').primaryKey().defaultRandom(),
topicCategoryId: uuid('topic_category_id').notNull().references(() => topicCategories.id),
schemaName: text('schema_name').notNull(),
factKeys: jsonb('fact_keys').notNull(),
cardFormats: text('card_formats').array().notNull(),
createdAt: timestamp('created_at', { withTimezone: true }).notNull().defaultNow(),
updatedAt: timestamp('updated_at', { withTimezone: true }).notNull().defaultNow(),
})
export const factRecords = pgTable('fact_records', {
id: uuid('id').primaryKey().defaultRandom(),
topicCategoryId: uuid('topic_category_id').notNull().references(() => topicCategories.id),
schemaId: uuid('schema_id').notNull().references(() => factRecordSchemas.id),
facts: jsonb('facts').notNull(),
title: text('title').notNull(),
notabilityScore: real('notability_score'),
notabilityReason: text('notability_reason'),
context: text('context'),
imageUrl: text('image_url'),
imageSource: text('image_source'),
imageAttribution: text('image_attribution'),
imageResolvedAt: timestamp('image_resolved_at', { withTimezone: true }),
status: factRecordStatusEnum('status').notNull().default('pending_validation'),
validation: jsonb('validation'),
sourceType: text('source_type').notNull(),
sourceStoryId: uuid('source_story_id').references(() => stories.id),
externalSourceId: text('external_source_id'),
aiModel: text('ai_model'),
generationCostUsd: numeric('generation_cost_usd', { precision: 10, scale: 6 }),
publishedAt: timestamp('published_at', { withTimezone: true }),
expiresAt: timestamp('expires_at', { withTimezone: true }),
createdAt: timestamp('created_at', { withTimezone: true }).notNull().defaultNow(),
updatedAt: timestamp('updated_at', { withTimezone: true }).notNull().defaultNow(),
})
export const stories = pgTable('stories', {
id: uuid('id').primaryKey().defaultRandom(),
slug: text('slug').unique(),
headline: text('headline').notNull(),
summary: text('summary'),
sourceCount: integer('source_count').notNull().default(1),
sourceDomains: text('source_domains').array(),
firstSeenAt: timestamp('first_seen_at', { withTimezone: true }).notNull().defaultNow(),
lastSeenAt: timestamp('last_seen_at', { withTimezone: true }).notNull().defaultNow(),
category: text('category'),
tags: text('tags').array(),
isValidated: boolean('is_validated').notNull().default(false),
status: storyStatusEnum('status').notNull().default('clustering'),
createdAt: timestamp('created_at', { withTimezone: true }).notNull().defaultNow(),
updatedAt: timestamp('updated_at', { withTimezone: true }).notNull().defaultNow(),
})
export const newsSources = pgTable('news_sources', {
id: uuid('id').primaryKey().defaultRandom(),
provider: newsProviderEnum('provider').notNull(),
externalId: text('external_id'),
sourceName: text('source_name'),
sourceDomain: text('source_domain'),
title: text('title').notNull(),
description: text('description'),
articleUrl: text('article_url').notNull(),
imageUrl: text('image_url'),
publishedAt: timestamp('published_at', { withTimezone: true }),
storyId: uuid('story_id').references(() => stories.id),
contentHash: text('content_hash'),
createdAt: timestamp('created_at', { withTimezone: true }).notNull().defaultNow(),
})
export const cardInteractions = pgTable('card_interactions', {
id: uuid('id').primaryKey().defaultRandom(),
userId: uuid('user_id').notNull().references(() => profiles.id, { onDelete: 'cascade' }),
factRecordId: uuid('fact_record_id').notNull().references(() => factRecords.id, { onDelete: 'cascade' }),
interactionType: interactionTypeEnum('interaction_type').notNull(),
cardFormat: cardFormatEnum('card_format'),
difficulty: triviaDifficultyEnum('difficulty'),
hiddenFactKey: text('hidden_fact_key'),
selectedAnswer: text('selected_answer'),
correctAnswer: text('correct_answer'),
isCorrect: boolean('is_correct'),
timeToAnswerMs: integer('time_to_answer_ms'),
scoringMethod: text('scoring_method').notNull().default('auto'),
nextReviewAt: timestamp('next_review_at', { withTimezone: true }),
reviewCount: integer('review_count').notNull().default(0),
streak: integer('streak').notNull().default(0),
createdAt: timestamp('created_at', { withTimezone: true }).notNull().defaultNow(),
})
export const cardBookmarks = pgTable('card_bookmarks', {
id: uuid('id').primaryKey().defaultRandom(),
userId: uuid('user_id').notNull().references(() => profiles.id, { onDelete: 'cascade' }),
factRecordId: uuid('fact_record_id').notNull().references(() => factRecords.id, { onDelete: 'cascade' }),
note: text('note'),
createdAt: timestamp('created_at', { withTimezone: true }).notNull().defaultNow(),
})
export const ingestionRuns = pgTable('ingestion_runs', {
id: uuid('id').primaryKey().defaultRandom(),
runType: text('run_type').notNull(),
startedAt: timestamp('started_at', { withTimezone: true }).notNull().defaultNow(),
completedAt: timestamp('completed_at', { withTimezone: true }),
status: text('status').notNull().default('running'),
recordsProcessed: integer('records_processed').notNull().default(0),
recordsCreated: integer('records_created').notNull().default(0),
recordsFailed: integer('records_failed').notNull().default(0),
errorMessage: text('error_message'),
metadata: jsonb('metadata'),
createdAt: timestamp('created_at', { withTimezone: true }).notNull().defaultNow(),
})
Step 3: Add relations
// ============================================================================
// FACT ENGINE RELATIONS
// ============================================================================
export const topicCategoriesRelations = relations(topicCategories, ({ one, many }) => ({
parent: one(topicCategories, {
fields: [topicCategories.parentId],
references: [topicCategories.id],
}),
children: many(topicCategories),
schemas: many(factRecordSchemas),
factRecords: many(factRecords),
}))
export const factRecordSchemasRelations = relations(factRecordSchemas, ({ one, many }) => ({
topicCategory: one(topicCategories, {
fields: [factRecordSchemas.topicCategoryId],
references: [topicCategories.id],
}),
factRecords: many(factRecords),
}))
export const factRecordsRelations = relations(factRecords, ({ one, many }) => ({
topicCategory: one(topicCategories, {
fields: [factRecords.topicCategoryId],
references: [topicCategories.id],
}),
schema: one(factRecordSchemas, {
fields: [factRecords.schemaId],
references: [factRecordSchemas.id],
}),
sourceStory: one(stories, {
fields: [factRecords.sourceStoryId],
references: [stories.id],
}),
interactions: many(cardInteractions),
bookmarks: many(cardBookmarks),
}))
export const storiesRelations = relations(stories, ({ many }) => ({
newsSources: many(newsSources),
factRecords: many(factRecords),
}))
export const newsSourcesRelations = relations(newsSources, ({ one }) => ({
story: one(stories, {
fields: [newsSources.storyId],
references: [stories.id],
}),
}))
export const cardInteractionsRelations = relations(cardInteractions, ({ one }) => ({
user: one(profiles, {
fields: [cardInteractions.userId],
references: [profiles.id],
}),
factRecord: one(factRecords, {
fields: [cardInteractions.factRecordId],
references: [factRecords.id],
}),
}))
export const cardBookmarksRelations = relations(cardBookmarks, ({ one }) => ({
user: one(profiles, {
fields: [cardBookmarks.userId],
references: [profiles.id],
}),
factRecord: one(factRecords, {
fields: [cardBookmarks.factRecordId],
references: [factRecords.id],
}),
}))
Step 4: Update plan_definitions table with new columns
Add to the existing planDefinitions table definition:
// Add these columns to the existing planDefinitions table
canAccessCardDetails: boolean('can_access_card_details').notNull().default(false),
trialDays: integer('trial_days').notNull().default(0),
maxBookmarks: integer('max_bookmarks').notNull().default(0),
Step 5: Run typecheck
Run: bun run typecheck
Expected: PASS
Step 6: Commit
git add packages/db/src/drizzle/schema.ts
git commit -m "feat(db): add Drizzle schema definitions for fact engine tables"
Task 7: Regenerate TypeScript types from Supabase
Step 1: Run types generation
Run: bun run db:types
Expected: Regenerated type file includes new tables
Step 2: Run typecheck
Run: bun run typecheck
Expected: PASS
Step 3: Commit if types file changed
git add packages/db/
git commit -m "chore(db): regenerate TypeScript types for fact engine schema"
Task 8: Update changelog
Files:
- Modify:
docs/changelog/unreleased.md
Step 1: Add Phase 1 changelog entry
Add under today's date:
## February 13, 2026
### Features
- **Database**: Add fact engine schema — topic_categories, fact_record_schemas, fact_records, stories, news_sources, card_interactions, card_bookmarks, ingestion_runs tables (migrations 0092-0096)
- **Database**: Add 7 new enums for fact engine (news_provider, story_status, fact_record_status, card_format, trivia_difficulty, validation_strategy, interaction_type)
- **Database**: Update plan_definitions for Free/Eko+ model (can_access_card_details, trial_days, max_bookmarks)
- **Database**: Seed initial topic taxonomy (7 root categories, 13 sub-categories) and 5 fact record schemas
Step 2: Commit
git add docs/changelog/unreleased.md
git commit -m "docs: add changelog entries for Phase 1 database migrations"
Phase 2: Package Infrastructure (outline — detail when Phase 1 complete)
Task 9: Add new queue message types to packages/shared/src/schemas.ts
- Add Zod schemas:
INGEST_NEWS,CLUSTER_STORIES,EXTRACT_FACTS,IMPORT_FACTS,VALIDATE_FACT,GENERATE_EVERGREEN - Add
FactRecord,CardVariation,FactDetailtypes - Keep old schemas temporarily (removed in Phase 6)
Task 10: Add new queue names + constructors to packages/queue/src/index.ts
- Add 6 new queue constants +
resolveQueueNamecases - Add message constructors for each queue type
Task 11: Add fact engine AI functions to packages/ai/src/
- Add
selectModelForTask(task: FactEngineTask)to model-router - Add prompt functions:
extractFactsFromStory(),scoreNotability(),validateFact(),generateEvergreenFacts() - Keep old AI functions temporarily
Task 12: Add new env vars to packages/config/src/index.ts
NEWS_API_KEY,GOOGLE_NEWS_API_KEY,NEWS_INGESTION_INTERVAL_MINUTESFACT_EXTRACTION_BATCH_SIZE,VALIDATION_MIN_SOURCES,NOTABILITY_THRESHOLDEVERGREEN_DAILY_QUOTA,EVERGREEN_ENABLEDSTRIPE_PRICE_PLUS_MONTHLY,STRIPE_PRICE_PLUS_ANNUAL
Task 13: Add DB query functions to packages/db/
- Feed queries:
getPublishedFacts(),getFactsByTopic(),getFactDetail() - Interaction queries:
recordInteraction(),getUserInteractions(),getReviewDueFacts() - Bookmark queries:
addBookmark(),removeBookmark(),getUserBookmarks() - Ingestion queries:
createIngestionRun(),updateIngestionRun()
Task 14: Update Stripe package for new pricing
- Change
PlanKeytype:'free' | 'plus'(keep'base' | 'pro' | 'team'for backwards compat temporarily) - Add
STRIPE_PRICE_PLUS_MONTHLY,STRIPE_PRICE_PLUS_ANNUALto price map - Update checkout session to support trial_from_plan
Phase 3: Workers (outline)
Task 15: Create worker-ingest app
- New app in
apps/worker-ingest/ - Consumes:
INGEST_NEWS,CLUSTER_STORIES,RESOLVE_IMAGEqueues - News API client (NewsAPI.org), clustering via TF-IDF cosine similarity
- Image resolution pipeline (Wikipedia → TheSportsDB → Unsplash → AI)
Task 16: Create worker-facts app
- New app in
apps/worker-facts/ - Consumes:
EXTRACT_FACTS,IMPORT_FACTS,GENERATE_EVERGREENqueues - AI-powered fact extraction from stories, bulk import from structured APIs
- Evergreen fact generation on cron
Task 17: Create worker-validate app
- New app in
apps/worker-validate/ - Consumes:
VALIDATE_FACTqueue - Tiered validation: authoritative API → multi-source → AI cross-check
- Status transitions: pending_validation → validated | rejected
Task 18: Add cron routes to web app
/api/cron/ingest-news(*/15 * * * *)/api/cron/cluster-sweep(0 * * * *)/api/cron/import-facts(0 4 * * *)/api/cron/generate-evergreen(0 3 * * *)/api/cron/validation-retry(0 */4 * * *)/api/cron/archive-content(0 2 * * *)/api/cron/topic-quotas(0 6 * * *)- Update
vercel.jsonwith new cron schedules
Phase 4: UI — Feed (outline)
Task 19: Create feed layout and route
apps/web/app/feed/layout.tsx— lightweight headerapps/web/app/feed/page.tsx— card grid (Server Component)apps/web/app/feed/loading.tsx— skeleton grid
Task 20: Build card grid components
card-grid.tsx— CSS Grid container (responsive 1-4 columns)card-feed.tsx— Client component with infinite scrollfact-card.tsx— Unified card shell with topic badge, title, difficultycard-skeleton.tsx— Loading statecategory-filter.tsx— Horizontal filter chips
Task 21: Build feed API route
apps/web/app/api/feed/route.ts— GET: paginated card feed (public, no auth)- Cursor-based pagination, topic filtering, notability ordering
Task 22: Update root redirect
apps/web/app/page.tsx— redirect to /feed instead of /dashboard
Phase 5: UI — Detail + Subscription (outline)
Task 23: Build card detail page
apps/web/app/card/[slug]/page.tsx— Server Component with auth checkfact-detail.tsx— Full context (gated)quiz-interface.tsx— Multiple choice (RadioGroup, auto-scored)recall-interface.tsx— Open recall ("Reveal Answer" → self-graded)paywall-gate.tsx— Subscription overlay with CTAsource-list.tsx— Validation sources with confidence badge
Task 24: Build card detail API
apps/web/app/api/cards/[slug]/route.ts— GET: card detail (auth + subscription check)apps/web/app/api/cards/[slug]/interact/route.ts— POST: record interaction
Task 25: Update subscription flow
- Update Stripe checkout for new
plusplan - Trial flow: create subscription with
status='trialing'andtrial_end - Update success/cancel redirect URLs to
/feed
Task 26: Build subscribe page
apps/web/app/subscribe/page.tsx— Pricing + trial CTA- Two-tier comparison: Free vs Eko+
/subscribe/success/page.tsx— Post-checkout confirmation/subscribe/canceled/page.tsx— Checkout canceled
Phase 6: Remove Legacy (outline)
Task 27: Delete old web app routes
- Remove:
/dashboard/**,/library/**,/favorites/**,/domains/**,/url/**,/notifications/**,/saved/** - Remove:
/api/urls/**,/api/domains/**,/api/library/**,/api/favorites/**,/api/saved-items/**,/api/discover/**,/api/v1/**,/api/enqueue-due-urls/**
Task 28: Remove old workers
- Delete
apps/worker-tracker/ - Delete
apps/worker-render/(unless needed for card images) - Delete
apps/worker-sms/
Task 29: Remove old package code
- Remove old AI functions from
packages/ai/ - Remove old queue messages from
packages/shared/andpackages/queue/ - Remove old config env vars from
packages/config/ - Remove old DB queries from
packages/db/
Task 30: Drop old tables (migrations 0097-0099)
- Migration 0097: Drop page tracking tables (pages, page_observations, page_change_events, etc.)
- Migration 0098: Drop brand/domain tables (domains, brands, brand_sources, etc.)
- Migration 0099: Drop misc legacy tables (screen_avatars, personas, use_cases, etc.)
Phase 7: Polish (outline)
Task 31: Update auth redirects
- Login → /feed, Signup → /feed, Reset password → /feed
- OAuth callback → /feed
Task 32: Simplify account pages
- Keep:
/account,/account/billing,/account/settings - Remove:
/account/notifications(simplify to email preferences only)
Task 33: Update marketing site (apps/public/)
- Rewrite copy for fact engine positioning
- Update pricing page for Free/Eko+ model
- Update features page
Task 34: Update admin app (apps/admin/)
- Replace stub pages with fact engine management
- Card management, story viewer, ingestion monitoring
- Trivia review queue
Verification Checklist
After each phase, run:
bun run typecheck # TypeScript compilation
bun run lint # Biome linting
bun run test # Vitest tests
bun run migrations:check # Migration index current (after Phase 1)
After Phase 6 (legacy removal):
- Verify no imports reference deleted modules
- Verify old API routes return 404
- Verify
bun run buildsucceeds for all apps
Execution Notes
- Phase 1 is self-contained — migrations can be applied without any code changes
- Phase 2 + 3 can partially overlap — queue types needed before workers, but AI functions and DB queries can be developed alongside
- Phase 4 + 5 are UI-heavy — consider parallel frontend agents
- Phase 6 is destructive — create a snapshot/tag before starting
- Phase 7 is incremental — can be done over multiple sessions
Recommended execution: Subagent-Driven for Phases 1-2, Team-based for Phases 3-5, Sequential for Phase 6-7.