February 2026 Changes

February 28, 2026

Features

  • DB: Full category taxonomy expansion — merge Business + Finance into unified root (migration 0140); add 3 new root categories: Health & Medicine, Language & Linguistics, Space & Astronomy with full hierarchies and domain-specific schemas (migration 0141); expand 6 previously-empty roots (Architecture, Auto, Current Events, Entertainment, Technology, Weather & Climate) and 14 thin roots with new depth-1 subcategories, depth-2 children, and sport-specific extensions across 7 migrations (0140-0148); add depth-3 (4th level) categories with ~2 children per depth-2 node; total active categories grow from ~276 to ~1,200+
  • Code: Update getActiveTopicCategories() to support childDepth >= 3 with 3-level nested Drizzle with relation loading; update category-mapper.ts PATH_CATEGORY_MAP with new root keyword patterns and aliases (health-medicine, language-linguistics, space-astronomy); update richness tier classification for new roots; add 11 new and expand 17 existing CategorySpec definitions in categories.ts for seed pipeline coverage
  • Fact Pipeline: Extend category system to 3 levels (depth 0→1→2) — getFactsByTopic() aggregates descendants at any depth (not just root); getActiveTopicCategories() gains childDepth option for nested relation loading; new getDescendantCategoriesForParent() returns tree structure for workers; AI prompts updated with formatSubcategoryHierarchy() to instruct "deepest matching level" classification; workers flatten tree for slug resolution at any depth; feed UI renders 3rd badge row for sub-subcategories with progressive drill-down
  • Fact Pipeline: Activate ~95 subcategories for pipeline routing and feed UI — AI extraction and evergreen generation now classify facts into subcategories via prompt injection; workers resolve subcategory slugs to IDs with parent-chain schema fallback; getFactsByTopic() aggregates descendant facts for root categories; feed UI shows drill-down subcategory badges when a root category is active
  • DB: Fix schema quality issues preventing seed pipeline from producing fact records (migration 0137) — add root-level schemas for 5 gap categories (history, culture, science, geography, space) unlocking ~1,798 previously unprocessable seeds; remove TV duplicate general_fact schema; upgrade insects-invertebrates from general_fact to animal_fact with insect-specific keys; delete 5,311 orphaned seed entries on inactive "Things" category; remove all remaining general_fact schemas from 4 stale inactive categories
  • DB: Add getSubcategoriesForParent() query and getDescendantIds() helper using materialized path LIKE queries; expand getPublishedFacts() with topicCategoryIds array filter; add includeChildren option to getActiveTopicCategories() using Drizzle with relation; expand getExistingTitlesForTopic() to span full category tree for dedup; add parent-chain fallback to getSchemaForTopic() (max 3 levels)
  • DB: Add domain-specific schemas for records/nature, records/human-achievement, and current-events (migration 0138) — both depth-1 records subcategories now have explicit 9-key record_holder schemas (7 base + 2 extensions) matching the convention used by all depth-2 grandchildren; nature gains ecosystem_or_phenomenon and measurement_scale; human-achievement gains achievement_category and significance; current-events gets a new 9-key news_event schema (event, category, date, location, key_figures, source, impact, status, preceding_event) for news ingestion pipeline; all active categories now have schemas
  • DB: Seed 144 depth-2 subcategories across all 65 remaining depth-1 parents (migration 0139) — completes the 3-level taxonomy with 2-3 grandchildren per parent; trigger auto-inherits parent schemas and challenge formats; 18 high-value categories gain custom schema extensions (e.g. dive_depth_meters for marine mammals, scoville_rating for spices, prize_pool for esports); total depth-2 categories now 165
  • DB: Add depth-3 (4th level) categories Part 1 — new roots and empty roots (migration 0147) — 138 depth-3 subcategories across 8 root categories (Health & Medicine, Language & Linguistics, Space & Astronomy, Architecture, Auto, Technology, Weather & Climate, Entertainment); each depth-2 parent gets exactly 2 depth-3 children; 16 high-value nodes gain custom schema extensions (2-4 keys each) for viral-diseases, historical-pandemics, large-language-models, gothic-cathedrals, teams-constructors, atlantic-hurricanes, and more; trigger auto-inherits parent schemas; total depth-3 now 138
  • DB: Add depth-3 categories Part 2 — expanded roots (migration 0148) — 488 depth-3 subcategories across 25 existing root categories; total taxonomy now 1,104 active categories (33 roots, 152 d1, 313 d2, 606 d3)
  • Seed Pipeline: Generate curated seed entries across all 44 category specs — 12,375+ new entities across all root and expanded subcategories using AI entity generation; all 33 active root categories now have seed coverage; total seed_entry_queue grows to 49,469 entries across 197 subcategories
  • Docs: Update SEED.md control prompt — fix topic examples to match actual CATEGORIES spec (science 5 subcats, history 5 subcats with correct counts); add 3-level category hierarchy documentation (31→80→165); add 7 missing scripts to Available Scripts table with verified flags; add enrichment orchestrator section (8 sources); update model eligibility to reflect actual results (gemini-3-flash-preview and gemini-2.5-flash pass, gpt-5-mini does not); add model verification step and entity materialization to execution checklist; add cost estimates for new operations; deprecate improve-titles.ts

February 27, 2026

Features

  • AI Pipeline: Add deterministic notability bypass for KG+Wikidata-confirmed entities — when both KG resultScore >= 100 and Wikidata sitelinkCount >= 20, override AI-generated notability with deterministic 0.85 score; wired into extract-facts, explode-entry, and generate-evergreen handlers; emits facts.notability_bypass metric for monitoring
  • AI: Add enrichment orchestrator (packages/ai/src/enrichment.ts) — single entry point that resolves context from multiple API clients in parallel via Promise.allSettled(), with topic-based routing (sports→TheSportsDB, music→MusicBrainz, geography→Nominatim, books→Open Library) and universal clients (KG, Wikidata, Wikipedia); supports skip option for pipelines that already have specific sources; 13 unit tests
  • AI: Wire multi-source enrichment into all 4 AI pipelines — news extraction (extract-facts.ts) upgraded from KG-only to full orchestrator, seed explosion (explode-entry.ts) and evergreen generation (generate-evergreen.ts) gain enrichment context injection, validation evidence (evidence.ts) gains Wikidata cross-reference plus domain-specific client lookups (TheSportsDB, MusicBrainz, Open Library) gated on topic path
  • AI: Add 7 free API client modules for fact engine enrichment — Wikidata REST (structured claims, 24hr cache), Wikipedia Summaries (narrative context, 1hr cache), TheSportsDB (team/player/event data, 1hr cache), MusicBrainz (artist taxonomy with 1 req/s rate limiter, 24hr cache), GDELT 2.0 (news ingestion source + event context, 15min cache), Nominatim (geographic disambiguation with 1 req/s rate limiter, 24hr cache), Open Library (book/author validation, 24hr cache); all follow kg-client.ts architecture pattern with graceful degradation, bounded in-memory cache, metrics, and @hook annotations for future pipeline integration
  • AI: Refactor Wikipedia lookup from inline evidence.ts code into shared wikipedia-client.ts — validation pipeline imports from centralized client instead of duplicating API calls
  • AI: Refactor TheSportsDB from inline resolve-image.ts code into shared sportsdb-client.ts — image resolution cascade imports from centralized client with expanded team/player/event endpoints
  • Scripts: Add taxonomy schema coverage audit tool — developer-time CLI (scripts/taxonomy/audit-schema-coverage.ts) that samples entities from DB, queries KG + Wikidata + Wikipedia + domain APIs, compares structured data against current fact_keys and vocabulary, surfaces schema gaps with 6 analysis checks (field coverage, type alignment, vocabulary gaps, entity type distribution, domain-specific coverage, sitelink notability)
  • Scripts: Add recursive tree walker to taxonomy audit tool — --depth=N flag (0-3) walks taxonomy tree from any node, audits each node individually with per-node entity sampling (no child pooling), detects ownership gaps (subcategories missing vocabulary/voice entries in TAXONOMY_RULES_DATA/TAXONOMY_VOICES_DATA), generates prioritized action plan (high/medium/low) with JSON + Markdown output; any active topic_categories slug is now valid (not just root slugs); deferred batch Wikidata property label resolution across all nodes; backward compatible at --depth=0

February 26, 2026

Features

  • Challenge Images: Add RESOLVE_CHALLENGE_IMAGE pipeline for anti-spoiler image resolution — challenges where the fact's entity image would reveal the answer (reverse_lookup always, fill_the_gap/statement_blank conditionally) get context-aware images via Wikimedia Commons, Unsplash, and Pexels cascade; adds image_url, image_source, image_attribution, image_status columns to fact_challenge_content (migration 0136); extracts shared tryUnsplash/tryPexels into image-sources.ts; registers 4th queue consumer in worker-ingest
  • AI: Add Google Knowledge Graph integration for entity validation and enrichment — shared KG client in packages/ai with in-memory cache (1hr TTL, 5K max); seeding pipeline filters non-notable entities before AI explosion (~20-30% cost savings); news extraction enriches AI prompts with KG entity context for better grounding; entirely optional (pipelines degrade gracefully without API key)

February 25, 2026

Features

  • AI: Promote gemini-3-flash-preview to default model across all tiers — replaces gemini-2.5-flash in DEFAULT_TIER_CONFIG, test pipeline, and news testing defaults; T2 canary results: 100% quality, 94/100 signoff, $0.0121/fact (cheapest), 4.2x faster than gpt-5-mini, only model to pass eligibility across all dimensions
  • AI: Prime gemini-3-flash-preview as dedicated signoff reviewer with adapter-driven customization — add signoff_review to AdaptableTask union, add default: return {} to all adapters for zero-touch future task additions, wire adapter prefix/suffix/temperature/providerOptions into both test pipeline signoff phases
  • AI: Gemini 2.5 Flash adapter v4 — cross-field consistency audit, thinking budget 1024→2048 for fact tasks, checklist relocated to prefix for primacy bias, failure examples for correct_answer and setup_text, banned generic reveal openings with content-specific requirement, domain-match instructions for individual sport schema fields, banned word enforcement, calibrated signoff rubric with factual-accuracy-first token efficiency scoring; T2 canary (28 facts): validation 100, evidence 100, challenges 100, schema 94, voice 92, style 91, token_efficiency 92
  • AI: Gemini 2.5 Flash v5 architectural fixes for 97+ eligibility — add individual-sports DB subcategory with semantically correct schema (athlete/event/result instead of team fields), deterministic patchPassiveVoice() post-generation rewriter for common passive→active voice patterns, batch-level patchRevealDuplication() to eliminate repetitive reveal openings across challenges, simplify adapter DOMAIN-MATCH workaround, upgrade passive voice drift severity info→warning
  • DB: Upgrade ~50 remaining subcategory schemas from generic general_fact (4 fields) to domain-specific schemas inheriting parent fields + 1-2 extras (migration 0135); add trg_inherit_parent_schema auto-inheritance trigger so future subcategory inserts automatically copy parent schema and challenge_format_topics

Bug Fixes

  • AI: Relax structural date validation to accept natural language dates ("January 28, 2017") instead of requiring ISO 8601 only — users prefer human-readable date formats in fact values
  • AI: Replace patchRevealDuplication() (exact-dedup, ~1% coverage) with patchGenericReveals() (semantic pattern matching, ~75% coverage) - fixes subject extraction "The" bug, double em-dash concatenation artifact, and matches generic reveal opener families ("You absolutely nailed it", "That's exactly right", "That's the exact", etc.); expand Gemini adapter banned reveal openings from 9 to 19 patterns with failure examples
  • AI: Gemini 2.5 Flash v5 gap closure — expand passive voice patcher from 20→36 patterns; expand generic reveal patterns from 8→21 semantic families; add per-entity dedup tracking to prevent rewrite template repetition across same-subject challenges; expand rewrite pools from 6→16 templates each; add patchTextbookRegister() for non-passive encyclopedic phrases ("This victory cemented" → "That victory sealed"); add patchPunctuationSpacing() for missing spaces after sentence-ending punctuation; strengthen factual accuracy guardrails (qualifier precision, team sport score audit, context sentence count gate, controversial attributions, date disambiguation, null-vs-populated rule); expand banned reveal openings from 19→40+ patterns
  • AI: Tiered eligibility thresholds — split single 97% threshold into structural (97%: validation, evidence, challenges) and subjective (90%: schema, voice, style, token_efficiency) tiers; recalibrate signoff rubric with granular 93-96 scoring bands and explicit "not a violation" guidance to reduce false deductions from LLM reviewer noise; increase challenge generation thinking budget 2048→4096 for mandatory sentence-count audit; add Zod .describe() annotations to challenge output schema for better structured output compliance
  • CI: Fix frontmatter validation, docs-code-mapping binding check (add missing code paths for superset rule), and token-optimization smoke test (checklist moved from suffix to prefix in v4 adapter)

Infrastructure

  • Docs: Add anti-staleness infrastructure for docs/prompts/ — create prompts:check CI script (scripts/check-prompt-paths.ts) that validates all file paths referenced in prompt code blocks still exist on disk, add related_code frontmatter to all 67 prompt files connecting them to the docs:staleness detector, create docs-code-mapping.yml with 9 domain-level cross-cutting mappings, and wire prompts:check into the CI pipeline
  • Docs: Fix 4 broken path references in prompt files caused by v2 fact engine config consolidation (challenge-pre-generated-styles.ts, challenge-voice-constitution.ts, challenge-voices.tschallenge-voice.ts, challenge-content-rules.ts)
  • Docs: Expand related_code frontmatter to 22 high-value docs across dev/, product/, architecture/, runbooks/, rules/, specs/, and top-level — total coverage now 195 docs feeding the docs:staleness detector

February 24, 2026

Features

  • AI: Deduplicate STYLE_RULES in challenge prompt assembly — extract 5 shared technique descriptions (Breadcrumb Trail, Pivot, Reward, Discovery, Unfolding) emitted once instead of 6×, emit only per-style refinements and style_data per style; reorder prompt sections for Gemini implicit cache optimization (stable block front-loaded, variable block last); reduces STYLE_RULES section ~63% (~4,755 tokens) and extends cacheable prefix from 9.8% to 93%
  • AI: Reduce input token costs ~25-35% via prompt compression — compact JSON serialization across all AI calls, compress Gemini fact prefix from 63 to 40 lines, deduplicate BEFORE SUBMITTING checklists into shared constant, extract recency detection into recency-utils.ts
  • AI: Move challenge_title generation from fact-level to per-challenge — each challenge in fact_challenge_content now gets its own theatrical title generated in Phase 2 alongside the question/answer, eliminating answer-leak bugs where a single fact-level title would spoil what the challenge asks (e.g., "Twenty-Three Slams" for a question about Grand Slam count)
  • AI: DeepSeek free-text adapter (V4) — bypass json_object mode entirely for DeepSeek (which caused 20-50% generation failures); use generateText() with post-parse JSON extraction and Zod validation; generateObjectForProvider() routes DeepSeek through free-text path while all other providers use standard generateObject()
  • AI: Local Supabase test workflow — --commit flag on llm-fact-quality-testing.ts manages full lifecycle: start local Supabase, seed reference data, write fact_records and challenge_content to local DB during testing, reset DB after; falls back to JSONL-only when Docker unavailable

Infrastructure

  • AI: Add Mistral AI provider (mistral-large-latest, mistral-medium-latest, mistral-small-latest) — env schema, model registry with pricing, model-router instantiation via @ai-sdk/mistral; no adapter yet (uses default pass-through)
  • Config: Replace CSV→JSON config pipeline with direct TypeScript data files — 11 new as const satisfies config files replace 22 CSVs, 6 generated JSONs, and 17 sync scripts; consumers (challenge-content-rules.ts, taxonomy-content-rules.ts, model-registry.ts, generate-curated-entries.ts) import directly from TypeScript; removes config:sync, config:check, config:export commands
  • Docs: Unify prompt system — migrate 17 existing prompts + convert 39 workflow actions into 55 domain-organized prompt files across 12 domains (docs/prompts/{model-testing,seeding,quality,content,pipeline,config,taxonomy,database,environment,deploy,brand,users}/); each domain has 00-index.md with FAQ sections absorbing 10 workflow FAQ files; delete docs/workflows/ (99 files)

Performance

  • AI: Gemini thinking token budget control and cost tracking — cap thinkingBudget per task (0 for scoring/validation/moderation, 1024 for extraction/generation); fix Gemini 2.5 Flash pricing from $0.15/$0.60 to GA $0.30/$2.50; track reasoning tokens separately in ai_cost_tracking with new reasoning_tokens_total column; thread providerOptions and reasoningTokens through all 13 AI call sites; adds migration 0133
  • Worker: Cap article sources in fact extraction to 5 sources × 1,500 chars each — prevents unbounded prompt tokens when stories have 10+ news sources (~20K chars)
  • Worker: Micro-batch challenge content generation — dequeue up to 5 GENERATE_CHALLENGE_CONTENT messages and process in a single AI call, amortizing the ~5,200-token system prompt across the batch
  • AI: Cap evergreen title dedup list to 50 — pass explicit limit to getExistingTitlesForTopic and add defensive .slice(0, 50) in prompt construction to prevent growing token costs
  • Queue: Replace flat 1s nack retry delay with exponential backoff (5s → 30s → 60s cap, 15% jitter) — prevents retry storms on rate limit errors
  • AI: Sliding window on conversational challenge history — keep first exchange + last 3 exchanges (8 messages max) instead of entire conversation, saving ~4,000 tokens on long sessions
  • AI: Use actual AI SDK token counts instead of length / 4 heuristic — reads result.usage.inputTokens and result.usage.outputTokens for accurate cost tracking
  • AI: Add Google/Gemini daily budget cap ($3.00 default) — mirrors Anthropic budget cap pattern with isGoogleBudgetExhausted(), in-memory cache, and GOOGLE_DAILY_SPEND_CAP_USD env var
  • Worker: Populate super fact entry summaries with actual fact titles — query 3 sample fact titles per entry instead of generic "N facts generated" string, giving AI real signal for correlation discovery

Testing

  • AI: Run light model adapter seed test (10 facts, gemini-2.5-flash) — 90% validation pass rate, 100% CQ-002 compliance, 270/270 challenges valid, 100/100 signoff score, $0.25 total cost

Bug Fixes

  • AI: Fix DeepSeek evidence scoring by appending schema text after voice prefix in fetch wrapper — schema blob was prepended to system prompt, pushing the 1200-token voice/accuracy prefix out of position 0 where non-reasoning models give strongest attention
  • Public Site: Fix wiki build failure by switching MDXRemote to format: 'md' — prevents curly braces and angle brackets in docs from being parsed as JSX expressions

February 23, 2026

Features

  • AI: Add taxonomy quality onboarding enforcement — drift scoring penalizes unknown topic slugs (score 0.6 instead of 1.0), worker-validate flags taxonomy_rules_missing on validated facts with no rules, and new CI completeness check (bun run taxonomy:completeness-check) validates voice coverage, content rules, and vocabulary depth across all 32 taxonomy slugs

  • Testing: Add news ingestion quality testing pipeline — 6-phase script (scripts/news-testing/) with DB-free news fetcher (NewsAPI, GNews, TheNewsAPI), fact extraction via extractFactsFromStory() with modelOverride, and reuse of seed harness validate/challenge/signoff/report phases

  • AI: Add modelOverride to ExtractFactsInput in fact-engine.ts — allows pinning a specific model for testing, bypassing the tier-based model router

  • Docs: Add FAQ quick-reference series (docs/workflows/faq/) -- 10 files with 110 Q&A entries across 9 domains (ingestion, facts, validation, seeding, challenges, frontend, config, database, operations) plus navigation index with "Which FAQ do I need?" lookup table

  • AI: Fix signoff pipeline to display all challenge content fields for quality review — was only showing challengeText, now shows setupText, correctAnswer, revealCorrect, revealWrong

  • AI: Split Gemini 2.5 Flash adapter into task-specific prefixes (GEMINI_FACT_PREFIX for schema tasks, GEMINI_CHALLENGE_PREFIX for challenge generation) with BEFORE SUBMITTING checklists

  • AI: Remove score-anchoring bias from Gemini signoff guidance — reviewer now scores on observed output quality, not historical assumptions

  • AI: Lower Gemini temperatures for schema-bound tasks (fact_extraction 0.7→0.5, seed_explosion 0.7→0.5, evergreen 0.7→0.6)

  • Testing: Bump Google provider concurrency cap from 5 to 15 for faster Gemini test runs

  • AI: Extend Gemini 2.5 Flash adapter factual accuracy guardrails with v3-specific error patterns (numeric specs, artwork claims, sports timeline reconstruction, title field capture)

  • AI: Reduce evidence likely_inaccurate soft-reject penalty from -0.25 to -0.15 — graduated penalty now meaningfully distinguishes soft vs hard rejects

  • AI: Tighten cross-model escalation confidence window from 0.50-0.75 to 0.50-0.65, reducing evidence denominator volatility

  • AI: Add explicit severity calibration to cross-model prompt — defaults to info for simplifications, reserving warning for materially incorrect claims

  • AI: Strengthen Gemini generation guardrails v5 — add schema-mismatch awareness to cross-model prompt, unfinished project guards, entity attribution checks, influence precision rules, and concrete superlative failure examples

  • AI: Filter schema-mismatch warnings from evidence escalation trigger — prevents domain-terminology flags from inflating evidence denominator

  • AI: Add v6 generation guardrails — outdated statistics awareness (pre-dam/post-dam figures), score consistency cross-check (winner must have higher score), and expanded schema-mismatch filter patterns for tennis/music domain warnings

  • AI: Add recency-aware cross-model validation — facts from news articles published within 7 days get calibrated severity rules (unverifiable-due-to-recency = info, not critical) and lower pass threshold (0.35 vs 0.50), boosting news validation pass rate from 52% to 94%

  • AI: Add banned word reinforcement to Gemini challenge adapter — GEMINI_CHALLENGE_PREFIX now explicitly bans "Trivia", "Quiz", "Easy one" in reveal_correct/reveal_wrong fields

  • AI: Add recency-aware evidence validation — evidence tier reasoner now receives article publish date context, preventing hard "inaccurate" rejections for recent events it cannot verify; hard reject downgraded to soft reject (-0.15 penalty) for recent articles without DB contradictions

  • Testing: Fix report CQ-002 weakness display — floating-point equality (=== 1.0) replaced with threshold (>= 0.99); rates above 95% no longer flagged as weakness

  • Testing: Fix report verdict threshold — single signoff failure no longer blocks "Production Ready"; verdict now uses 5% failure tolerance instead of zero-tolerance

  • Testing: Add validation concurrency throttle and response contamination detection — validation phase now uses separate --validation-concurrency (default: 3) to prevent Gemini API response mixing under load; adds isResponseContaminated() entity-name sanity check with automatic retry on cross-model and evidence phases

February 22, 2026

Features

  • Config: Add CSV → JSON config pipeline — shared library (scripts/config/lib/) with CSV reader supporting @file: blob references, Zod validation with source tracking, companion CSV joiner with FK joins and dot-notation nesting, SHA-256 checksum staleness detection, and JSON writer with embedded checksums
  • Config: Add export scripts for 6 config domains — categories, models, challenge rules, taxonomy rules, seed controls, app controls; each extracts hardcoded TypeScript constants or YAML-in-markdown blocks to normalized CSV files under config/csv/
  • Config: Add sync scripts for 6 config domains — each reads CSV, validates via domain-specific Zod schemas, and writes config/generated/*.json with content checksums; supports @file: references for large text blobs and pipe-delimited arrays
  • Config: Add config:sync orchestrator and config:check CI staleness guard — orchestrator runs all 6 sync scripts in sequence, check command verifies embedded SHA-256 checksums match current CSV content
  • Config: Migrate 4 TypeScript constant files to JSON imports — generate-curated-entries.ts (905→225 lines), model-registry.ts (257→118 lines), challenge-content-rules.ts (1276→344 lines), taxonomy-content-rules.ts (2109→95 lines); all preserve identical export names and types for backward compatibility
  • Docs: Add per-domain config control workflow docs (docs/workflows/controls/) — 6 domain guides (categories, models, challenge, taxonomy, seed-controls, app-controls) with column references, editing workflows, troubleshooting tables, plus index README and cross-references from seeding-and-testing.md
  • Docs: Add evergreen ingestion product guide (docs/product/evergreen-ingestion.md) and operator workflows (docs/workflows/evergreen-ingestion-workflows.md) — covers full pipeline from cron trigger through AI generation, validation, challenge content, and feed selection; workflows include manual triggers, monitoring, troubleshooting, and config dependencies
  • Docs: Add workflow action files system (docs/workflows/actions/) — 76 task-driven action files across 11 domains (seeding, config, pipeline, content, brand, database, quality, docs, environment, admin, users) with copy-pasteable steps, YAML frontmatter (risk, AI cost, frequency), 12 domain READMEs, and master index catalog; cross-referenced from seeding-and-testing.md, evergreen-ingestion-workflows.md, and runbook.md

Infrastructure

  • CI: Add bun run config:check to CI pipeline for config staleness detection

February 21, 2026

Features

  • Scripts: Add bounded-concurrency parallelism to LLM quality testing pipeline — models run in parallel via Promise.all(), entities/facts/challenges batched with pMap() worker pool; new --concurrency N CLI flag (default: 3)
  • Scripts: Add challenge batching — send all 6 styles per difficulty in one API call instead of individual calls, reducing challenge-phase API calls by ~6x
  • Scripts: Add domain-aware schema key fallbacks — DOMAIN_SCHEMA_KEYS map provides semantically appropriate fact keys per taxonomy (sports, science, history, music, film, technology, art, geography), preventing cross-domain misapplication
  • Scripts: Add model-specific prompt reinforcement — getModelPromptReinforcement() feeds known per-model weaknesses to the signoff reviewer for targeted quality scoring
  • Scripts: Add definitive report summaries with recommendations — executive summary with winner declaration and rankings, per-model strengths/weaknesses analysis, production readiness verdicts, tiered strategy recommendations, and actionable next steps
  • Scripts: Add timestamped report archiving — reports now dual-write to both report.md (latest) and report-{timestamp}-{models}.md for historical comparison
  • UI: Switch fact-related dates to US middle-endian (MM/DD/YYYY) format with centralized formatFactDate utility in @eko/shared
  • AI: Add ModelAdapter pattern — per-model prompt customization with suffix/prefix/override modes, adapter registry with caching, and eligibility tracking via JSONL (97% threshold across 7 dimensions)
  • AI: Add model adapters for gpt-5-mini, gemini-2.5-flash, gemini-3-flash-preview, claude-haiku-4-5
  • AI: Add gemini-3-flash-preview to model registry
  • AI: Integrate adapters into fact-engine, seed-explosion, and challenge-content generation
  • Scripts: Update test pipeline to use adapter-based signoff guidance and write eligibility JSONL
  • AI: Add per-category domain vocabulary with auto-generation in taxonomy CLI — 32 categories backfilled with domain terms, expert phrases, and term preferences
  • AI: Wire taxonomy rules, voice, and vocabulary into seed explosion pipeline — previously domain-blind, now receives full taxonomy context
  • AI: Add drift coordinator system — 5 pluggable coordinators (structure, schema, voice, taxonomy, difficulty) detect semantic drift in LLM-generated challenge content via free heuristic checks; non-blocking by default with aggregate scoring and per-coordinator violation tracking
  • Docs: Add Eko Product Bible (docs/eko-product-bible.md) — end-to-end system overview covering pipeline stages, architecture, cascading dependency chains, agent ownership, CI quality gates, operations, and improvement roadmap; includes CI integrity checker (scripts/bible-integrity-check.ts) validating file paths, package names, queue types, and cross-doc links

Bug Fixes

  • AI: Fix challenge generation 100% failure on OpenAI models — replace z.record(z.string(), z.unknown()) style_data schema with z.union() of 6 concrete per-style Zod schemas, eliminating propertyNames JSON Schema property that OpenAI's structured output API rejects

Refactoring

  • AI: Remove DeepSeek provider — drop deepseek-chat/deepseek-reasoner from registry, config, model-router, validation pipeline, and test harness (~220 lines removed)
  • Validation: Switch cross-model (Phase 3) and evidence reasoner (Phase 4c) from DeepSeek to gemini-2.5-flash

February 20, 2026

Bug Fixes

  • Validation: Fix evidence validation (Phase 4) 0% pass rate — use reasoner verdict field for pass/fail gating instead of treating all key_concerns as critical blockers; only inaccurate/likely_inaccurate verdicts now block, matching Phase 3's severity-based pattern
  • Validation: Improve Wikipedia entity extraction for evidence validation — add multi-strategy entity extraction (possessives, quoted names, proper noun sequences, topic path hints) with MediaWiki search API fallback, raising lookup success from ~15% to ~85%
  • Validation: Add graduated response for likely_inaccurate reasoner verdicts — apply -0.25 confidence penalty instead of hard fail, allowing facts with strong external evidence to survive historical nuance concerns
  • Validation: Distinguish schema format errors from factual inaccuracy in reasoner — add schema_mismatch flag to prevent tennis/basketball schema mismatches from causing false rejections of correct facts
  • AI: Reduce DeepSeek-chat parse failures — enhance json_object mode schema injection with explicit formatting directives, add response-side JSON repair in deepSeekFetchWrapper (strips markdown fences, fixes stray periods after numbers, removes trailing commas, trims post-JSON commentary)

Refactoring

  • Scripts: Rename deepseek-quality-test.ts to llm-fact-quality-testing.ts — generalize from DeepSeek-specific to model-agnostic quality testing; add --models flag for arbitrary model comparison, --signoff-model for configurable quality reviewer; supports gpt-4o-mini, claude-haiku-4-5, gemini-2.5-flash, MiniMax-M2.5, and all registry models

February 19, 2026

Features

  • Validation: Add multi-phase fact validation pipeline — 4-phase validation (structural, consistency, cross-model, evidence) replaces single ai_cross_check for non-news facts with DeepSeek cross-provider verification, Wikipedia/Wikidata evidence corroboration, and full phase-by-phase audit trails
  • Config: Add DeepSeek provider integration — deepseek-chat and deepseek-reasoner models via OpenAI-compatible API for cross-provider fact validation
  • AI: Wire topic_slug through challenge content generation — handler now passes taxonomy slug to generateChallengeContent(), enabling domain-specific voice and content rules in challenge prompts
  • AI: Inject taxonomy content rules and voice into evergreen fact generation — generateEvergreenFacts() now derives topicSlug from topicPath and appends TAXONOMY_CONTENT_RULES extraction guidance and TAXONOMY_VOICE register to the system prompt
  • Scripts: Add taxonomy:add CLI onboarding tool — interactive script that AI-generates domain-specific schema keys and content rules, creates a SQL migration, and updates taxonomy-content-rules.ts in a single command
  • Schema Overhaul: Replace all generic_fact schemas with 33+ domain-specific schemas — 28 root categories upgraded from 4-field generic fallback to 7-10 field specialized schemas, 5 new subcategory schemas added (film_fact, music_fact, tv_show_fact, celebrity_fact, sports_legend), child subcategories get pre-merged schemas with parent base + child-specific additions
  • AI: Add modelOverride parameter to explodeCategoryEntry() and generateChallengeContent() — enables A/B testing of different AI models for fact and challenge generation without modifying tier config
  • Scripts: Add DeepSeek v3.2 quality testing pipeline — 5-phase CLI script (generate, validate, challenge, signoff, report) comparing DeepSeek-chat, DeepSeek-reasoner, and GPT-5-mini as fact/challenge generators with quality scorecard and cost-benefit analysis
  • Schema Overhaul: Truncate fact_records and fact_challenge_content — clean slate for repopulation with domain-specific schemas
  • AI: Add taxonomy voice layer — per-domain emotional register (TAXONOMY_VOICE) for 33+ taxonomies, injected between universal voice constitution and per-format STYLE_VOICE in challenge generation prompts
  • AI: Add taxonomy content rules — per-domain formatting and factual conventions (TAXONOMY_CONTENT_RULES) injected into both extraction and challenge generation prompts with extraction_guidance, challenge_guidance, and avoid patterns
  • AI: Refactor STYLE_VOICE from tonal personality to structural format mechanics — personality moves to TAXONOMY_VOICE, STYLE_VOICE becomes format-specific interaction mechanics (option curator, focused questioner, puzzle architect, etc.)
  • AI: Redesign STYLE_RULES with tease-and-hint architecture — setup_text follows ANCHOR → ESCALATION → WITHHOLD arc, challenge_text uses pivot technique, reveal_correct rewards with bonus detail, reveal_wrong uses discovery technique, correct_answer follows 3-6 sentence unfolding arc
  • AI: Add typed style_data validation for all 6 challenge styles — CQ-007 through CQ-011 validate fill_the_gap, direct_question, statement_blank, reverse_lookup, free_text style_data shapes

Bug Fixes

  • AI: Fix CQ-002 enforcement gap — add patchCq002() call in generateChallengeContent() runtime path, not just seed script

  • AI: Fix hardcoded cost estimate in challenge content generation — replace GPT-4o-mini pricing constants with estimateCost() for model-aware calculation

  • AI: Fix _challenge_hints typo in SUPER_FACT_RULES — corrected to challenge_hints to match schema field name

  • AI: Fix banned word "trivia" in conversational challenge prompt — replaced with "deep knowledge" to align with BANNED_PATTERNS

  • AI: Deduplicate CHALLENGE_TONE_PREFIX — consolidated from 3 copies (challenge-content-rules.ts, fact-engine.ts, seed-explosion.ts) to single export

  • AI: Add taxonomy slug warning — log when topicSlug has no TAXONOMY_VOICE entry so generic voice fallback is visible

  • AI: Remove deprecated ModelSelection type from model-router.ts

  • AI: Fix text[] type support in buildFactObjectSchema — was falling through to z.string() instead of z.array(z.string())

  • UI: Fix snake_case mismatch in quiz-interface — AI generates is_correct in JSONB but UI read isCorrect, causing correct answers to never highlight

  • Fact Engine: Materialize 76 mid-level subcategories across 33 parent categories — migration 0129 inserts subcategory rows with correct parent_id, hierarchical path, depth, schema entries, and challenge format propagation

  • Fact Engine: Add entity materialization script (materialize-entity-categories.ts) with 4-phase pipeline — --audit finds entities with 5+ facts (2,770 found), --classify uses batched AI classification with JSONL output, --insert creates leaf topic_categories, --link reassigns facts to entity categories with dry-run support

  • AI: Add entity_classification task type to fact engine model routing at default tier for cost-efficient batch classification

  • Scripts: Update generate-curated-entries.ts to resolve subcategory IDs — new seed entries get the most specific topic_category_id and suggested_topic_path via slug matching against materialized subcategories, with fallback to root category

  • AI: Add per-style voice differentiation — STYLE_VOICE constant with 6 distinct voice definitions (gallery guide, dinner companion, co-author, knowledge-checker, mystery narrator, thoughtful interviewer), injected between voice constitution and style rules in generation prompt

  • AI: Update CQ-002 fallback prefixes to align with per-style voice registers — more natural phrasing when AI fails to include second-person address

  • Scripts: Add rewrite-challenge-defects.ts — targeted field rewrite script for defective challenges (CQ-002 prefixed, telegraphic setup/answer, short reveals) with --dry-run, --limit, --partition, --concurrency flags, batched gpt-5-mini rewriting with per-style voice guidance

Documentation

  • CLAUDE.md: Remove 5 non-existent scripts, add turbo filter pattern for workers, update project structure with 7 missing entries
  • Rules Index: Replace v1 URL-tracking invariants with v2 fact-engine invariants, fix agent name (ux-page-card-designer to card-ux-designer)
  • Challenge Content Rules: Align CQ numbering with code — add CQ-009 through CQ-013 for style_data validators, fix CC-002 heading to "Five-Field Structure"
  • Rules Index: Add Challenge Content section with CC and CQ rule summaries, add fact-engineer to ownership table
  • CHALLENGE_TONE.md: Update related_code frontmatter to include all 5 consuming source files

February 18, 2026

Documentation

  • Docs: Update 6 documentation files for v2 fact engine sync — table flow diagram (+2 tables, count fix), fact anatomy (+2 schema refs), ingestion guide (alias resolution, maxDepth), news fact engine (xAI Grok default, XAI_API_KEY), seed runbook (+2 new script sections), Eko GTD templates (XS/QS tiers, Doc-Updates domain)

Features

  • Eko GTD: Add quality tiers for Ads, Business, Research domains and introduce Doc-Updates domain
  • Fact Engine: Add taxonomy coherence layer — topic_category_aliases table maps external news API slugs (business, entertainment, health, etc.) to internal topic_categories, unmapped_category_log tracks dropped categories for audit, resolveTopicCategory() provides 3-step alias fallback (exact match → provider-specific → universal), extract-facts handler updated to use alias resolution
  • Fact Engine: Reconcile taxonomy conflicts — deactivate 4 niche orphan roots (accounting, marketing, spelling-grammar, things), merge statistical-records into records, propagate challenge_format_topics to 29 expansion roots from migration 0101 that lacked format links
  • Fact Engine: Add maxDepth parameter to getActiveTopicCategories() and getActiveTopicCategoriesWithSchemas() — all cron routes and feed page now use maxDepth: 0 to prevent quota explosion when subcategories are added
  • Rules: Add CC-010 (category alias normalization) and CC-011 (unmapped category audit) rules for taxonomy coherence
  • Docs: Add taxonomy coherence GTD project (04-taxonomy-coherence.md) with 10 challenges across 3 waves
  • Fact Engine: Unify challenge content generation post-validation — move GENERATE_CHALLENGE_CONTENT enqueue from import-facts.ts to validate-fact.ts so all source types (news, evergreen, imported) get challenge content after validation, avoiding wasted AI cost on rejected facts
  • Fact Engine: Wire generationCostUsd through extraction pipeline — extract-facts.ts and generate-evergreen.ts now compute and store per-record AI cost via estimateCost()
  • Fact Engine: Wire notabilityScore, notabilityReason, context through import pipeline — ImportFactsMessageSchema expanded with optional fields, import-facts.ts and explode-entry.ts pass metadata through to insertFactRecord()
  • Scripts: Add backfill-fact-nulls.ts for backfilling NULL notability scores and enqueuing challenge content for validated facts missing it, with --dry-run, --audit, and per-source-type defaults
  • Fact Engine: Implement GNews news provider — replace stub with full API client fetching from gnews.io/api/v4 with topic/search routing, rate limiting, and StandardArticle normalization
  • Fact Engine: Add TheNewsAPI news provider — new provider client fetching from thenewsapi.com/v1 with top/all endpoints, UUID-based dedup, and StandardArticle normalization
  • Fact Engine: Implement Unsplash image fallback — replace stub with full API client searching photos by entity title, landscape orientation, with proper attribution
  • Fact Engine: Add Pexels image fallback — new cascade step after Unsplash searching Pexels photos by topic with attribution
  • Config: Add THENEWS_API_KEY, UNSPLASH_ACCESS_KEY, and PEXELS_API_KEY env vars to config schema and FactEngineConfig
  • Database: Add 'thenewsapi' to news_provider enum (migration 0123)
  • Queue: Extend INGEST_NEWS provider enum and cron dispatch to support TheNewsAPI

February 17, 2026

Features

  • Scripts: Add monthly changelog file rotation (MM-YYYY.md) — shared changelog-utils.ts utility, updated changelog-check.ts and generate-changelog.ts with dynamic paths, auto-locking of previous month, migration script to split unreleased.md into 01-2026.md (locked) and 02-2026.md (draft), updated 3 hooks and 10 documentation files
  • Skills: Add /done session summary skill for end-of-session documentation with descriptive filenames
  • Hooks: Add /done reminder to PreCompact hook — warns when no session summary exists before context compaction
  • AI: Integrate xAI Grok as new AI provider — add @ai-sdk/xai SDK, grok-4-1-fast-reasoning, grok-4-1-fast-non-reasoning, and grok-4 models to registry with full model router support
  • AI: Switch all tier defaults to grok-4-1-fast-reasoning — 4x cheaper output tokens than gpt-5-mini ($0.50 vs $2.00/M) with reasoning capability for more specific, cinematic challenge titles
  • AI: Add content_cleanup task type to fact engine for corpus-wide title/context/notability rewriting
  • AI: Upgrade all fact generation prompts to enforce rich narrative context — seed explosion, evergreen generation, news extraction, and super-fact discovery all now require Hook→Story→Connection structure (4-8 sentences), theatrical challenge titles with anti-pattern bans, and concrete good/bad examples. New facts match cleanup-pass quality from day one
  • AI: Add challenge_title and context fields to news extraction output — extractFactsFromStory now generates theatrical titles and narrative context alongside structured facts, eliminating NULL challenge_titles for news-derived records
  • Scripts: Add cleanup-content.ts — full corpus rewrite script with 5 phases (audit, export, fix, upload, validate) using local-first JSONL architecture for crash-resilience
  • Scripts: Add --recover phase to cleanup-content.ts — classifies weak outputs (notability <= 0.5) into recoverable (rich source data) vs vague (poor source data) using classifyFactData() heuristic, re-processes recoverable facts with full structured data in prompt, supports --partition N/M for parallel execution
  • Database: Add free_text to all fact_record_schemas.card_formats arrays, expand format-style linkages (statement_blank/direct_question to know_a_lot_about), link sport and records subcategories to parent format mappings
  • Worker: Add concurrent message processing to worker-facts — WORKER_CONCURRENCY env var controls parallel messages per queue type (default: 1 for backward compat). Combined with multiple worker instances, enables ~10x throughput for seed pipeline runs
  • Fact Engine: Add gpt-5-nano and gpt-5-mini to model registry and Drizzle ai_model enum, set gpt-5-nano as default tier for cost-efficient seed generation
  • Fact Engine: Add maxRetries: 0 to seed explosion and super-facts AI calls — delegates retry logic to queue-level backoff instead of Vercel AI SDK internal retries
  • Worker: Fix spinoff entries missing topic_category_id — pass parent entry's category ID through to insertSeedEntry for correct schema resolution
  • Database: Add fact_challenge_content table for pre-generated AI challenge content — migration 0121 with three-layer emotional arc fields (setup_text, challenge_text, reveal_correct, reveal_wrong), style_data JSONB, difficulty 1-5, and composite unique constraint on (fact_record_id, challenge_style, target_fact_key, difficulty)
  • AI: Add challenge content generation system — voice constitution, per-style rules, difficulty levels 1-5 with prompt guidance, super-fact awareness for cross-entity challenges, banned pattern validation, and generateChallengeContent() function using Vercel AI SDK structured output
  • AI: Add challenge content rules and validation — challenge-content-rules.ts with CHALLENGE_VOICE_CONSTITUTION, STYLE_RULES for 6 styles, DIFFICULTY_LEVELS, SUPER_FACT_RULES, and validateChallengeContent() post-generation validator
  • Scripts: Add generate-challenge-content.ts backfill script — 6-phase JSONL pipeline (audit, export, generate, upload, validate, recover) with --difficulty, --partition, --concurrency, --limit flags following cleanup-content.ts patterns
  • UI: Add pre-generated challenge content to card detail — quiz, recall, and text-input components consume AI-crafted setup/challenge/reveal text with graceful fallback to algorithmic generation when no pre-generated content exists
  • Queue: Add GENERATE_CHALLENGE_CONTENT message type — Zod schema, queue routing, constructor, worker-facts handler that generates and upserts challenge content per fact record
  • Worker: Wire IMPORT_FACTS to enqueue challenge content generation — new facts automatically get challenge content generated after import

Refactoring

  • Database: Drop 16 v1 legacy tables, v1_backup schema, 3 orphaned enums, and 2 orphaned functions in migration 0118 — page tracking (pages, user_pages, page_observations, page_change_events, page_change_summaries, section_dom_mappings, page_title_history, screenshot_captures), SMS (sms_delivery_log, sms_notification_pages, notification_delivery_log), personas/onboarding (personas, use_cases, onboarding_state), and misc (screen_avatars, type_metadata)
  • Database: Remove ~9,000 lines of dead v1 query functions, schema definitions, and exports across packages/db — gut drizzle/queries.ts, drizzle/feature-flags.ts, queries.ts, client.ts, and index.ts barrel exports
  • Database: Delete 4 orphaned security test files (1,513 lines) referencing dropped v1 tables
  • Workers: Stub worker-sms, daily-digest cron, twilio webhook, and monthly-usage-report as deprecated no-ops
  • Admin: Gut 6 admin action files referencing dropped v1 tables (audience, avatars, user URLs, changes, content, onboarding)
  • Scripts: Add seed pipeline operations scripts — improve-titles.ts for batch AI title rewriting, generate-curated-entries.ts for AI-generated notable entries across 33 topic categories, bulk-enqueue.ts for Upstash queue dispatch

February 16, 2026

Features

  • Fact Engine: Add seed fact-challenge pipeline — file parsing infrastructure (XLSX/DOCX/CSV parsers, 22 content profiles, static category mapper with 40+ path patterns), DB tables (seed_entry_queue with priority-based consumption, super_fact_links for cross-entry correlations, challenge_title on fact_records), AI explosion engine (explodeCategoryEntry for 10-100 facts per entity with theatrical titles, findSuperFacts for cross-entry correlations), worker handlers (EXPLODE_CATEGORY_ENTRY and FIND_SUPER_FACTS consumers), dynamic validation strategy selection in IMPORT_FACTS, spin-off depth control with recursive CTE, and CLI orchestrator with --parse/--explode/--explode-spinoffs/--super-facts/--stats/--all commands
  • Database: Add migrations 0115-0116 — seed_entry_queue table with priority-ordered consumption and idempotent upserts, challenge_title column on fact_records, super_fact_links junction table with CASCADE delete
  • Queue: Add EXPLODE_CATEGORY_ENTRY and FIND_SUPER_FACTS queue message types with routing, constructors, and type exports
  • AI: Add seed_explosion and super_fact_discovery task tiers to model routing system
  • Marketing: Add Challenge Tone Document (docs/marketing/CHALLENGE_TONE.md) — canonical voice spec for all AI-generated challenge prompts with 6 format examples, scoring feedback voice, dispute voice, and anti-patterns
  • Database: Drop 15 legacy tables with no active code references — migrations 0111-0113 remove page-tracking tables (url_renders, video_renders, section_content_snapshots, page_diff_settings, user_folders, user_folder_pages, page_read_states, saved_items), brand auxiliary tables (brand_category_proposals, brand_review_queue, brand_sources, seeding_pages), and misc tables (correlation_groups, correlation_group_members, app_content)
  • Database: Clean up Drizzle schema and query functions for dropped tables — remove 16 table definitions, 11 relation blocks, 13 query functions, and 5 orphaned enums from packages/db
  • Database: Replace binary isCorrect with continuous score: 0.0–1.0 scoring on card_interactions — migration 0100 adds score column, backfills from is_correct, drops is_correct
  • Database: Add ~30 new root topic categories with seed schemas — migration 0101 adds accounting, animals, architecture, art, auto, business, cooking, design, and 21 more categories with generic fact schemas
  • Database: Add free_text card format enum value — migration 0102 for free-text challenge support
  • Database: Add score_disputes table for AI-judged score disputes — migration 0103 with decision types, RLS policies, and user indexes
  • Database: Add reward_milestones and user_reward_claims tables — migration 0104 with 4 seeded milestones (100/500/1000/2000 points) granting free Eko+ days
  • AI: Add text answer moderation function for free-text challenges — checks relevance, harm, and prompt injection
  • AI: Add text answer scoring function for free-text challenges — returns 0.0–1.0 score with reasoning
  • AI: Add dispute evaluation function — AI judge reviews user arguments and decides upheld/partial/full reversal
  • API: Add /api/cards/[slug]/dispute POST endpoint for score disputes with 10/month rate limit
  • API: Add /api/user/rewards GET/POST endpoints for reward milestone status and claiming
  • API: Update /api/cards/[slug]/interact with free-text AI scoring pipeline, moderation gate, and 50/day rate limit
  • UI: Add ChallengeReveal skeleton reveal component with fade-in animation for challenge tabs
  • UI: Add TextInputChallenge free-text challenge component with AI-graded scoring
  • UI: Add DisputeDialog for score dispute submission and AI judgment display
  • UI: Add RewardsProgress milestone progress bar with claim buttons
  • UI: Update ScoringFeedback with tri-state display (correct/partial/incorrect) and dispute button
  • UI: Add Challenge tab to card detail page alongside Learn, Quiz, and Recall
  • UI: Wrap Quiz, Recall, and Challenge tabs in skeleton reveal component
  • Shared: Add XSS sanitization utilities for fact record JSONB values — iterative HTML tag stripping, event handler removal, and dangerous URI scheme filtering with recursive deep-sanitize for nested objects/arrays
  • Testing: Add comprehensive test coverage for v2 fact engine pipeline — 10 test files with ~80 new tests across 6 packages (sanitization, AI extraction/validation/scoring/evergreen, DB queries, worker handlers for ingest/cluster/resolve-image/extract-facts/generate-evergreen/validate-fact)
  • AI: Add challenge tone prefix to fact extraction and evergreen generation prompts — canonical Eko voice from CHALLENGE_TONE.md injected via CHALLENGE_TONE_PREFIX constant
  • Fact Engine: Add 30-day expiry policy for news-derived facts — news-extracted facts get expires_at = now + 30d, high-engagement facts (50+ interactions) auto-promote to enduring in archive-content cron
  • Database: Add promoteHighEngagementFacts query function and expiresAt parameter to insertFactRecord
  • Shared: Add spaced repetition scheduling module (calculateNextReview) with SM-2 variant algorithm — interval schedule [4h, 1d, 3d, 7d, 14d, 30d] based on streak
  • API: Wire spaced repetition into /api/cards/[slug]/interact — answered interactions now compute nextReviewAt, streak, and reviewCount using SM-2 scheduling
  • API: Add /api/review GET endpoint for fetching facts due for spaced repetition review
  • API: Upgrade /api/feed with blended feed algorithm — 40% recent validated, 30% review-due, 20% evergreen, 10% exploration with round-robin interleave and per-card userStatus field
  • Database: Add feed blending queries — getEvergreenFacts, getRandomFacts, getUserFactStatuses, getLatestAnswerInteraction
  • UI: Add color-coded interaction indicator badges to fact cards — yellow (in progress), orange (review due), green (mastered)
  • UI: Polish challenge card layout — feature image with hover zoom, teaser prompt from first fact, difficulty badge, fact count, structured footer with date
  • UI: Polish card detail page — hero image, verification date badge from validation metadata, back-to-feed link, integrated RewardsProgress component
  • UI: Update card detail loading skeleton to match polished layout with image placeholder and 4-tab skeleton
  • Fact Engine: Introduce Challenge Format taxonomy with 8 named formats (Big Fan Of, Know A Lot About, Repeat After Me, Good With Dates, Degrees of Separation, Used To Work There, Partial Pictures, Originators)
  • Database: Add challenge_style and challenge_format_slug enums, challenge_formats table, challenge_format_styles and challenge_format_topics junction tables — migrations 0105-0106
  • Database: Extend card_interactions with challenge_format_id and challenge_style columns — migration 0107
  • Database: Seed 8 challenge formats with style mappings and topic eligibility — migration 0108
  • Database: Add challenge_sessions table for conversational multi-turn AI challenges — migration 0109
  • Database: Add RLS policies for challenge format tables and challenge sessions — migration 0110
  • AI: Add conversational challenge turn handler with format-aware prompting for Big Fan Of, Know A Lot About, and Degrees of Separation formats
  • API: Add challenge format discovery endpoints (GET /api/challenge-formats, GET /api/challenge-formats/[slug], GET /api/challenge-formats/for-topic/[topicSlug])
  • API: Add conversational challenge session endpoints (POST /api/challenge-sessions, POST /api/challenge-sessions/[sessionId]/turn, POST /api/challenge-sessions/[sessionId]/end)
  • UI: Add Challenge Hub discovery page at /challenges with format cards and topic selection
  • UI: Add ConversationalChallenge chat interface for multi-turn AI challenges with turn tracking and scoring
  • Subscribe: Switch to 14-day trial with Stripe Checkout and CC collection
  • Public Site: Add home page with live feed and card modal CTA
  • Public Site: Add pricing page with plan comparison and annual toggle
  • Public Site: Add features page with challenge system descriptions
  • Public Site: Add about page with mission and fact engine overview
  • Public Site: Add site layout with nav, footer, and v2 metadata
  • Public Site: Add feed API rewrite proxy to avoid CORS
  • Feed: Add subscription card injection for logged-out users
  • Database: Add getPlanTrialDays query for dynamic trial duration from plan_definitions
  • Stripe: Add payment_method_collection: 'always' to checkout sessions for trial CC collection

Database

  • Migration 0114: Update trial_days from 30 to 14 for Eko+ plan

Bug Fixes

  • Fact Engine: Fix getSchemaForTopic receiving slug strings instead of UUIDs in 5 call sites across seed CLI and worker handlers — schema lookup silently returned null, causing all entries to skip
  • Fact Engine: Add topicCategoryId to getSeedEntriesForBatch query so super-fact discovery can resolve schemas correctly
  • AI: Replace z.record() with dynamic z.object() in all AI structured output schemas — OpenAI rejects propertyNames in JSON Schema, so fact schemas are now built from topic schemaKeys at runtime via new schema-utils.ts shared utility
  • AI: Remove .min(0).max(1) from all z.number() Zod schemas — Anthropic rejects minimum/maximum in JSON Schema structured output; range enforcement moved to system prompts
  • AI: Switch default model tier to gpt-4o-mini for cost-efficient bulk seeding ($71 vs $1,752 for full corpus), keep mid/high tiers on Anthropic Sonnet for accuracy-critical tasks
  • AI: Move seed_explosion and super_fact_discovery tasks from mid to default tier — structured extraction doesn't require Sonnet-class models
  • Database: Add migration 0117 extending fact_records.source_type CHECK constraint to include file_seed and ai_super_fact values for seed pipeline
  • Shared: Remove agents barrel export from @eko/shared to fix Turbopack client bundling (node:fs not supported in browser chunks)
  • Challenges: Add force-dynamic to /challenges page to prevent static prerender failure without running backend

February 15, 2026

Features

  • Reel Schemas: Add packages/reel-schemas/ shared Zod schema package for Reel video generator — VideoBrief, VideoScript, VisualAsset, AudioAsset, RenderJobPayload, RenderJobResult, PipelineJob, BrandPreset, and RenderOutput schemas with r2Key and runpodJobId cloud fields
  • R2 Client: Add packages/r2/ Cloudflare R2 client library with S3-compatible upload/download, presigned URL generation (1h default expiry), and declarative lifecycle rules for transient assets (72h) and rendered outputs (30d)
  • Reel Render Worker: Add apps/worker-reel-render/ RunPod serverless render worker — Bun HTTP server handling RunPod protocol (POST /run, /runsync), FFmpeg subprocess wrapper, Remotion render orchestration with quality presets (draft/standard/premium), R2 asset download/upload, and Dockerfile (oven/bun:1-debian + FFmpeg + Playwright Chromium)

Documentation

  • Reel: Add phase-gated build requirements doc (docs/specs/reel-requirements.md) covering accounts, tooling, assets, open decisions, and risk validation for all 6 implementation phases

  • Reel: Add Character Studio with integrated fal.ai pipeline to spec and requirements — Flux 2 Pro anchor generation, LoRA identity training, Kontext image editing, new character wizard, roster/detail UI, two new SQLite tables (characters, character_images), updated Phase 2 deliverables

  • Docs: Consolidate video-generation/ cast system into Reel spec companion files — merge 17 source files into 6 companion files in docs/specs/reel/, add Section 16 and Appendix D to spec, delete orphaned root-level directory

  • Reel Spec: Update docs/specs/reel-video-generator-spec.md with cloud render architecture — replace FFmpeg/Playwright sidecars with RunPod + R2, add r2Key fields to VisualAsset/AudioAsset/RenderVariant, add runpodJobId to PipelineJob, update Stage 5 to RunPod dispatch, add RunPod/R2 to External APIs, update Settings/CLI/directory structure, add new Section 15 (Cloud Infrastructure)

  • Archive: Move ~300 v1 documents to docs/docs_archive/ organized into 15 subdirectories (specs-phases, specs-v1, product, admin-app, web-app, runbooks-v1, assessments, plans-v1, proposals, mvp-readiness, reports, dev-v1, projects, existing-archives, misc) with path-mapping fallback document, archive READMEs, and updated references across agents, hooks, scripts, and marketing docs

February 14, 2026

Features

  • Skills: Add 14 new skills for v2 fact engine — create-worker, add-queue-message-type, add-cron-route, build-news-api-client, create-fact-extraction-prompt, configure-validation-tier, build-feed-page, build-card-detail, configure-subscription-tier, remove-legacy-routes, drop-legacy-tables, triage-ingestion-failure, audit-fact-quality, monitor-card-engagement
  • Skills: Update create-migration skill with v2 standardized header format (MIGRATION NNNN pattern) and migrations:index regeneration step
  • Skills: Update deploy skill with 3 new Fly.io worker targets (worker-ingest, worker-facts, worker-validate)
  • Agents: Add 3 new agents for v2 fact engine pipeline — ingest-engineer (news ingestion), fact-engineer (AI extraction), validation-engineer (multi-tier verification)
  • Agents: Reorient 6 existing agents for v2 architecture — architect-steward (v2 invariants), card-ux-designer (renamed from ux-page-card-designer), subscription-manager (Free/Eko+), cron-scheduler (ingestion crons), admin-operator (fact engine admin), queue-sre (7 new queue types)
  • Agents: Deprecate 5 legacy pipeline agents — tracker-engineer, render-engineer, diff-analyst, summarization-safety, use-case-note-writer
  • Agents: Restructure agent catalog README with v2 data flow diagram, updated dependency levels, and new file pattern routing table

February 13, 2026

Features

  • Cleanup: Remove all legacy page-tracking infrastructure — 7 authenticated UI routes, 9 API route trees, worker-tracker and worker-render apps, 4 AI functions, 5 queue message types, 11 config env vars (~15,900 lines deleted)
  • Auth: Update all post-login redirects from /dashboard to /feed
  • Cleanup: Remove legacy server actions (favorites, folders, saved-items)
  • UI: Add card detail pages with subscription gating, paywall overlay, and loading skeleton at /card/[slug]
  • UI: Add multiple choice trivia quiz interface with auto-scoring and interaction recording
  • UI: Add open recall practice interface with self-grading and answer reveal
  • UI: Add subscribe page with Free vs Eko+ plan comparison and 30-day free trial CTA
  • UI: Add post-checkout success and cancellation pages at /subscribe/success and /subscribe/canceled
  • API: Add /api/cards/[slug] GET endpoint for authenticated card detail fetching with subscription check
  • API: Add /api/cards/[slug]/interact POST endpoint for recording card interactions (views, answers, bookmarks)
  • Database: Add createTrialSubscription query for no-credit-card trial subscription creation
  • UI: Add public feed page with responsive card grid, category filter chips, infinite scroll, and /api/feed pagination endpoint
  • CRON: Add 7 fact engine cron routes — ingest-news (*/15 min), cluster-sweep (hourly), import-facts (daily stub), generate-evergreen (daily), validation-retry (4h), archive-content (daily), topic-quotas (daily audit)
  • Database: Add 6 cron query functions — getActiveTopicCategories, getActiveTopicCategoriesWithSchemas, getUnclusteredNewsSources, getStuckPendingValidations, archiveExpiredFactRecords, getTopicFactCountToday
  • Worker: Create worker-validate app consuming VALIDATE_FACT queue with tiered validation strategies (authoritative_api, multi_source, curated_database, ai_cross_check)
  • Database: Add updateFactRecordStatus query for fact validation status transitions with validation JSON storage
  • Worker: Create worker-facts app consuming EXTRACT_FACTS, IMPORT_FACTS, and GENERATE_EVERGREEN queues for AI fact extraction, bulk API import, and evergreen knowledge generation
  • Worker: Create worker-ingest app consuming INGEST_NEWS, CLUSTER_STORIES, and RESOLVE_IMAGE queues with concurrent consumers, TF-IDF clustering, and image resolution cascade
  • Database: Add worker-facts queries — getStoryWithSources, getSchemaForTopic, getTopicCategoryById, getTopicCategoryBySlug, insertFactRecord, getExistingTitlesForTopic, getFactRecordSchemaById
  • Database: Add ingestion pipeline queries — insertNewsSources, getNewsSourcesByIds, createStory, updateStory, linkNewsSourceToStory, findStoriesByTitleSimilarity, getFactRecordById, updateFactRecordImage
  • Database: Add fact engine schema — topic_categories, fact_record_schemas, fact_records, stories, news_sources, card_interactions, card_bookmarks, ingestion_runs tables (migrations 0092-0096)
  • Database: Add 7 new enums for fact engine (news_provider, story_status, fact_record_status, card_format, trivia_difficulty, validation_strategy, interaction_type)
  • Database: Update plan_definitions for Free/Eko+ model (can_access_card_details, trial_days, max_bookmarks)
  • Database: Seed initial topic taxonomy (7 root categories, 12 sub-categories) and 5 fact record schemas
  • Database: Add Drizzle ORM schema definitions for all fact engine tables with relations
  • Database: Add fact engine Drizzle query functions — feed retrieval, card interactions, bookmarks, spaced repetition review, and ingestion pipeline observability
  • Shared: Add 7 fact engine queue message Zod schemas (INGEST_NEWS, CLUSTER_STORIES, EXTRACT_FACTS, IMPORT_FACTS, VALIDATE_FACT, GENERATE_EVERGREEN, RESOLVE_IMAGE) and domain types
  • Queue: Add 7 fact engine queue names, constructors, and type-safe message routing
  • Config: Add fact engine env vars (news API keys, extraction settings, notability threshold) and Eko+ Stripe pricing config
  • AI: Add fact engine AI functions — fact extraction, notability scoring, fact validation, and evergreen generation with tier-based model selection
  • Stripe: Add Free/Eko+ pricing model with billing period support and 30-day trial integration

February 11, 2026

Features

  • Dev Tooling: Enforce frontmatter on markdown files at creation time via Claude Code hook and pre-commit check
  • Marketing: Add blog copy documents for blog home, single post, categories, and archive pages
  • CRON: Eliminate N+1 query in enqueue-due-urls (17,001 → 2 queries per run) with lateral-join optimization and early-exit check for redundant hourly triggers

Bug Fixes

  • AI: Fix Haiku 4.5 token consumption during worker processing — deploy DEFAULT_TIER_CONFIG fix (all tiers → gpt-4o-mini) and repair cross-provider fallback to construct alternate-provider models directly instead of re-resolving tiers that map to the same provider
  • Queue: Fix stale processing messages never being recovered — messages stuck in :processing list now auto-reclaim during idle polling
  • Worker Tracker: Fix heartbeat counter always reporting 0 messages processed — closure variable was never incremented from consumer loop
  • Recovery Script: Fix broken Drizzle query in catch-up action — update deprecated urls/urlObservations references to pages/pageObservations

Features

  • Queue: Add recoverStaleProcessing() function and flush-processing recovery action for manual and automatic orphaned message recovery

February 10, 2026

Features

  • Marketing: Add copy documents for terms, privacy, FAQ, contact, about, and pricing pages

Documentation

  • Deployment Guide: Update Supabase auth config to reflect actual state (Site URL was localhost:3000, not eko.day; redirect URLs were empty), add Stripe webhook URL migration step, add Google Cloud Console verification, renumber steps and verification checklist, mark guide as locked

February 9, 2026

Features

  • Public Site: Integrate Sentry error monitoring for apps/public — configure @sentry/nextjs with DSN, global error boundary, client/server/edge instrumentation, and example test page
  • Public Site: Add .gitignore for apps/public matching sibling app conventions
  • Config: Document SENTRY_AUTH_TOKEN in .env.example for source map uploads

February 8, 2026

Features

  • Architecture: Split public marketing pages into apps/public for subdomain deployment (eko.day for public site, app.eko.day for authenticated app)
  • Config: Add getPublicUrl() for cross-domain links between public site and app; update getAppUrl() default to https://app.eko.day

Database

  • Security: Add security fixes, RLS InitPlan correction (split across two migrations for tables a-p and p-z), and database index cleanup (migrations 0083-0086)

Bug Fixes

  • CI: Fix docs-lint SIGPIPE false failures — replace echo | grep -q with here-strings to avoid broken pipe errors under pipefail when frontmatter content is large

Features

  • AI Routing: Add per-provider DB kill switches (feature flags ai_provider_openai, ai_provider_anthropic, ai_provider_google) for instant AI provider toggling without restarts (migration 0089)
  • AI Model Tiers: Add DB-driven model tier system — change AI models via SQL UPDATE with no restart needed (migration 0090):
    • Three-tier routing: default (gpt-4o-mini), mid (claude-haiku-4-5), high (claude-opus-4-6)
    • feature_flags.ai_model_tiers JSONB config with 60s in-memory cache
    • Model registry with deprecation detection and pricing metadata for all OpenAI, Anthropic, and Google models
    • Google/Gemini config slots (env vars, valid models) — SDK integration pending
    • Centralized createLanguageModel() helper eliminates duplicated provider instantiation across 6 AI consumer files
    • cost-tracker uses model registry for pricing lookup and provider detection
  • AI Model Tiers v2: Replace monolithic JSONB feature flag with dedicated ai_model_tier_config table and ai_model Postgres enum (migration 0091):
    • DB-level model validation via enum — invalid model names rejected at the Postgres layer
    • Per-tier row updates instead of rewriting entire JSONB blob
    • Provider auto-derived from MODEL_REGISTRY in code — single source of truth
    • 16 models registered in enum across OpenAI, Anthropic, and Google

February 7, 2026

Features

  • UI Reset (Phase 0): Controlled demolition of UI layer for clean rebuild from approved designs:
    • Strip packages/ui to 13 bare primitives (Button, Input, Select, Textarea, Checkbox, Radio, Switch, Dialog, Tooltip, Tabs, Badge, Separator, Toast) + cn() utility
    • Move design tokens CSS from packages/ui/styles/ to packages/shared/styles/
    • Create packages/ui-public empty placeholder for future marketing components
    • Stub all 50 web pages and 28 admin pages to plain text labels (no UI, no Tailwind utilities)
    • Simplify layouts to pass-through wrappers (auth redirect preserved)
    • Archive storybook to apps/_archive/storybook-v1/
    • Archive design docs to docs/_archive/design-v1/
    • Delete ~300 non-primitive components, ~155 story files, ~50 orphaned client components

Database

  • Performance: Add indexes on page_change_events(page_id) and page_change_events(page_observation_id) for unindexed foreign keys (migration 0087)

Bug Fixes

  • Design System: Restore icon size and motion tokens stripped during UI redesign that are still required by challenge tests
  • Design System: Rebuild motion.css with prefers-reduced-motion support, Tailwind/Radix animation overrides, and motion-safe/reduce utilities
  • Worker: Fix async test assertions for shouldEscalateToRender after function became async
  • Worker: Add meaningful change gating to prevent false-positive change events — skip page_change_event creation and AI summary generation when content hash changes but section analysis shows 0% change with no affected sections (filters ~1,000 wasted AI calls and DB writes from timestamp/session token noise)
  • Admin: Fix stale table references in admin dashboard stats — update raw Supabase queries from urls/url_observations/url_change_events to pages/page_observations/page_change_events after migration 0077 renamed tables
  • Worker: Fix getPreviousFullSections to use vNext page_change_events table and page_change_event_id FK instead of dropped V1 url_changes table — enables inline diff before/after comparison for section snapshots
  • Worker: Change V1_WRITE_ENABLED default from true to false — V1 tables were dropped in migration 0060 but the flag still defaulted to attempting writes, causing silent failures when processing tracking messages
  • Storage: Fix thumbnail upload 400 errors — add image/webp to renders bucket allowed_mime_types (migration 0088). Bucket was created with PNG-only allowlist but thumbnail generation produces WebP files for optimal file size
  • Infra: Add missing OPENAI_API_KEY to tracker worker — all AI summary calls were falling back from GPT-4o-mini to Claude Haiku via provider fallback, resulting in ~5-7x higher AI costs
  • Worker: Fix TypeError: Cannot destructure property 'firstElementChild' crash in content normalizer — wrap document.body access in try-catch to handle linkedom throwing on malformed HTML (e.g., CAPTCHA pages, empty responses)
  • Infra: Set AI_PROVIDER=openai on tracker worker — env var was set to anthropic, overriding model router and forcing all AI calls through Haiku regardless of OPENAI_API_KEY
  • AI: Fix OpenAI structured output schema validation — add additionalProperties: false to nested inline_diffs items object and use anyOf for nullable section_heading field

Documentation

  • Architecture: Add AI routing pipeline Excalidraw diagram documenting CRON-to-DB pipeline flow, model routing decision tree, and three-layer failure analysis

February 6, 2026

Features

  • Design System: Strip UI to studs — remove all visual opinions, ready for clean rebuild:
    • Gut tokens.css to bare semantic tokens (fonts, gray palette, status colors, shadcn slots, radius, sidebar, overlay)
    • Strip theme.css to minimal Tailwind v4 color/radius mappings
    • Empty components.css (1,392 lines of data-slot CSS removed)
    • Empty motion.css (589 lines of animation utilities removed)
    • Strip spacing.css to @theme inline scale only (remove semantic utilities)
    • Simplify globals.css to token/theme/tailwind imports + minimal base layer
    • Comment out decorative exports in index.ts (~100 lines: marketing components, animated icons, premium blocks)
    • Stub 8 marketing pages (home, about, features, pricing, use-cases, blog, faq, changelog) with minimal HTML + Tailwind
    • Remove 25 storybook stories for stripped components
    • Preserve: all shadcn primitives, 21 sacred components, app shell, auth forms, sidebar, accessibility CSS
  • AI Cost Protection: Prevent silent AI provider fallback — upgrade fallback log from debug to warn, add provider_fallback flag to AI result types, emit ai.provider_fallback metric, add startup warning when OpenAI key missing, send throttled email alert to admins when fallback is active
  • AI Cost Governance: Add $5/day Anthropic spend cap with graceful degradation to GPT-4o-mini:
    • New ai_cost_tracking table (migration 0079) for daily cost aggregation by provider, model, and feature
    • Cost-tracker module with in-memory budget cache (30s TTL), pricing table for Opus 4.6/Haiku/GPT-4o-mini
    • Budget check integrated into all 5 existing AI functions (generateSummary, generateSummaryWithDiffs, generateTrackingNote, generatePageDescription, classifyAudience)
    • Gap 1 fix: Provider fallback now tracks actual model cost, not intended model
    • Gap 2 fix: Error recovery in generateSummaryWithDiffs uses heuristic fallback instead of doubling AI calls
    • Gap 7 fix: classifyAudience budget check with heuristic fallback
    • Add claude-opus-4-6 to valid Anthropic models
  • AI Cost Reporting: Add daily AI cost report cron (06:00 UTC) emailing admins with per-model and per-feature breakdown, budget utilization, and 7-day trend
  • AI Model Router: Add Claude Opus 4.6 as third model tier for top-1% complex cases:
    • Opus escalation rules: pricing page + structure change, API page + structure change + >20% change, legal/terms + structure change + >30% change, any page + >60% change + >8 sections + structure change
    • Feature-flagged (OPUS_ESCALATION_ENABLED) with daily call cap (OPUS_MAX_DAILY_CALLS, default: 20)
    • In-memory daily call counter with midnight reset, fail-safe defaults to GPT-4o-mini
  • Claude Agents: Add memory: project to 6 agents for persistent cross-session knowledge (architect-steward, db-migration-operator, diff-analyst, security-reviewer, release-manager, ci-quality-gatekeeper)
  • Claude Agents: Add disallowedTools to all 18 agents enforcing delegation graph — agents delegate downstream only, never upstream. architect-steward has no restrictions as top-level coordinator
  • Claude Hooks: Add 3 new hook scripts and project-level .claude/settings.json:
    • auto-migrations-index.sh (PostToolUse) — auto-regenerates migrations index when migration files are written
    • changelog-check.sh (SubagentStop) — reminds to update changelog when code files change
    • subagent-quality.sh (SubagentStop) — runs typecheck after subagent completes code work
    • Register all 9 hooks (6 existing + 3 new) in .claude/settings.json for team sharing
  • AI Thin-HTML Detection: Add semantic page complexity classification using Claude Haiku:
    • New classifyPageComplexity() function classifies JS-heavy SPAs that need Playwright rendering
    • Domain-level caching (migration 0080) — classifications cached per domain with configurable TTL
    • 3-tier cascade in shouldEscalateToRender(): domain cache → AI classification → heuristic fallback
    • Feature-flagged (SEMANTIC_THIN_HTML_ENABLED) with daily call cap (THIN_HTML_MAX_DAILY_CLASSIFICATIONS, default: 50)
    • Budget-aware: respects Anthropic daily spend cap, falls back to heuristic when budget exhausted
  • Agent Teams: Add experimental agent teams with 3 coordination playbook runbooks:
    • Pipeline Team (tracker → diff → summary → UI): coordinated multi-stage pipeline changes
    • Infrastructure Team (queue, observability, cron, CI): operational backbone changes
    • Schema Team (migrations, RLS, types): database and data model changes
    • Enable CLAUDE_CODE_EXPERIMENTAL_AGENT_TEAMS=1 in project settings
  • Multi-URL Change Correlation: Detect coordinated changes across pages within a domain:
    • New correlation_groups and correlation_group_members tables (migration 0081)
    • New CORRELATE_CHANGES queue message type with dedicated queue routing
    • Opus-powered generateCorrelationSummary() analyzes related changes (e.g., "pricing + changelog + API docs = major release")
    • Worker-tracker fire-and-forget detection: after change event, checks if 3+ pages from same domain changed within configurable window
    • Feature-flagged (CHANGE_CORRELATION_ENABLED) with daily cap (CORRELATION_MAX_DAILY, default: 30)
    • Budget-aware: respects Anthropic daily spend cap
  • Premium Deep Analysis: Add Opus-powered deep analysis summaries for premium plan users:
    • New can_access_deep_analysis column on plan_definitions (migration 0082), enabled for pro and team plans
    • New is_deep_analysis flag on page_change_summaries — two-tier summary architecture (standard + deep)
    • generateDeepAnalysis() Opus function provides strategic implications, competitive context, and prioritized action items
    • Worker-tracker fire-and-forget: after standard summary, checks for premium user tracking the page before generating
    • Query-time gating: hasAnyPremiumUserTrackingPage() joins user_url_library → user_subscriptions → plan_definitions
    • Feature-flagged (DEEP_ANALYSIS_ENABLED) with min change threshold and daily cap (DEEP_ANALYSIS_MAX_DAILY, default: 50)
    • Budget-aware: respects Anthropic daily spend cap

February 5, 2026

Features

  • Database: URL→Page schema rename (migrations 0076-0078):
    • Migration 0076: Rename core tables (urlspages, user_url_libraryuser_pages), 3 enums, FK columns on 10+ tables, user_can_see_page_event() function, RLS policies
    • Migration 0077: Rename 8 dependent tables (url_observationspage_observations, url_change_eventspage_change_events, etc.), ~25 RLS policies, 4 stored functions, folder_unread_counts view
    • Migration 0078: Drop url_stats view, update saved_items item_type 'url''page', update triggers
  • Database: Update Drizzle ORM schema and queries for URL→Page rename — all table definitions, column accessors, relations, and 50+ query functions updated with deprecated aliases for backward compatibility
  • Database: Update Supabase client queries for URL→Page rename — all .from() table name strings and column references updated
  • Documentation: Complete URL→Page terminology rename across ~100 documentation files in docs/reports/features/, docs/specs/, and docs/projects/building-eko-pages/

Bug Fixes

  • Database: Add UNIQUE constraint for section_dom_mappings upsert (migration 0075). Migration 0072 created a regular index on (url_observation_id, section_id) but upsertSectionDomMappingVNext() requires a UNIQUE constraint for ON CONFLICT clause. Drop regular index and create partial unique index to enable vNext DOM mapping upserts

Features

  • Database: Add comprehensive schema remediation for orphaned V1 tables (migrations 0072-0074):
    • Migration 0072: Add vNext columns (url_id, url_observation_id, url_change_event_id) to 6 orphaned tables (sms_notification_urls, sms_delivery_log, section_content_snapshots, url_diff_settings, section_dom_mappings, page_title_history) with FK constraints to vNext tables
    • Migration 0073: Backfill vNext columns from v1_backup schema using canonical_url matching
    • Migration 0074: Add referential integrity triggers for polymorphic saved_items table (validation on insert/update, cascade delete from urls and use_cases)
  • Worker: Fix RENDER_URL database records - Add missing createScreenshotCapture call in render worker so screenshots uploaded to storage also get database records (resolves Issue #1 from schema audit)
  • Worker: Re-enable DOM mapping storage - Add upsertSectionDomMappingVNext function using vNext IDs and re-enable storeDomMapping in capture-screenshots.ts (resolves Issue #7 from schema audit)
  • Tooling: Update render smoke test (e2e-render-smoke.ts) from V1 tables to vNext (urls, screenshot_captures)

Bug Fixes

  • Database: Align url_change_summaries schema with V1 summaries (migration 0071). Add missing columns from migrations 0012 (next_steps) and 0043 (admin_override_what_changed, admin_override_why_it_matters, override_at, override_by) that were added to V1 but missed vNext table, resolving Drizzle schema drift
  • Scripts: Add backfill-summaries.ts script to regenerate AI summaries for orphaned change events (change events that exist without corresponding summaries due to API failures)
  • Database: Update Drizzle schema with vNext columns and relations for all migrated tables

February 4, 2026

Bug Fixes

  • Database: Fix broken favorites system - Migrate 11 favorites query functions from dropped V1 tables (favorite_folders, user_favorites) to unified vNext tables (user_folders with folder_type='favorite', user_folder_urls with user_library_id). Update actions and API routes to use user_library_id instead of tracked_url_id
  • Web App: Fix broken URL detail pages - Migrate /url/[id] and /dashboard/urls/[id] pages from dropped V1 tables (tracked_urls, url_checks, url_changes, summaries) to vNext tables (urls, user_url_library, url_observations, url_change_events). Update API route /api/urls/[id] for vNext CRUD operations

Features

  • AI Cost Optimization: Implement hybrid optimization strategy reducing AI costs by ~91% ($712 → ~$65/month):
    • Add configurable CRON batch size (CRON_BATCH_SIZE env var) for URL throughput control
    • Add trivial change filter - skip AI for changes below 3% threshold
    • Add section-level filtering - only include sections with >5% change in AI prompts
    • Add model router - routes 95% of calls to GPT-4o-mini, escalates complex cases to Claude Haiku 4.5
    • Enhance HTML normalization - strip dynamic attributes, cookie banners, social widgets, and normalize timestamps to reduce false positives
  • Tooling: Add migrations index system - Auto-generated index of all Supabase migrations with phase categorization, bun run migrations:index generator script, and CI validation via bun run migrations:check

Bug Fixes

  • Worker: Fix V1 code path errors after Phase 6 table removal. Replace getTrackedUrlById calls with drizzleQueries.getGlobalUrlById, update property mappings (snake_case → camelCase), remove V1 notification code (requires vNext user_url_library lookup), and clean up unused imports. Notifications temporarily disabled pending vNext implementation
  • Database: Expand updateUrlEkoNote method signature to accept 'search' generation method alongside existing types
  • Scripts: Data alignment Phase 10 - Add populate-domains-from-urls.ts script to extract unique canonical_domain values from urls table and create corresponding domains entries with auto-linking to brands via database trigger
  • Scripts: Data alignment Phase 12 - Add assign-brand-categories.ts script to assign categories to uncategorized brands using PDL industry mapping, legacy category_path fallback, and consumer-services default
  • Scripts: Data alignment Phase 13 - Add backfill-audience.ts script to backfill audience values (work/personal/hybrid) for URLs, domains, and brands using page_type classification and B2B/B2C category heuristics
  • Database: Add brand category query functions to Drizzle queries: getBrandCategories, getBrandPrimaryCategory, getBrandsInCategory, assignBrandCategory, removeBrandCategory, getL1CategoriesWithCounts
  • Database: Migration 0067 - Update domains.source CHECK constraint to use system_seeded instead of brand_seeded, aligning with Drizzle schema
  • Database: Migration 0068 - Fix urls.brand_id FK to reference domains.id instead of brands.id, establishing correct data hierarchy (brands → domains → urls)
  • Database: Migration 0069 - Normalize urls.canonical_domain by stripping www. prefix to match domains table normalization

Data Alignment Results

  • Phase 10 Complete: 2,891 domains created from URLs, 17,273 URLs linked to domains (100%)
  • Phase 11 Complete: 2,596 orphan domains linked to auto-created brands (100% domain→brand linkage)
  • Phase 12 Complete: 2,933 brands assigned categories (100% coverage) - Fixed pagination issue in assign-brand-categories.ts
  • Phase 13 Complete: 504 URL audience values updated based on page_type classification

February 3, 2026

Features

  • Database: Phase 3 link domains to brands - Add brand_id FK column to domains table, linking domain registry to curated brand library. Auto-backfill existing domains. Add trigger for automatic brand linking on domain insert/update. Add Drizzle queries for domain-brand management (linkDomainToBrand, findBrandForDomain, autoLinkOrCreateBrandForDomain, approveDomainWithBrandLink)
  • Admin: Phase 4 domain verification UI - Add admin pages for domain review (/domains/pending, /domains/rejected). Admins can approve, reject (with reason), or re-approve domains. Domain verification stats shown on dashboard
  • Queue: V1 → vNext migration Phase 6 - Update all queue message schemas to vNext-only. RenderUrlMessage, CaptureScreenshotsMessage, and SendSmsMessage now use url_id, url_observation_id, and url_change_event_id instead of V1 fields (tracked_url_id, url_check_id, url_change_id)
  • Worker: V1 → vNext migration Phase 6 - Update tracker, render, and SMS workers to produce and consume vNext-only queue messages
  • Database: V1 → vNext migration Phase 6 - Remove V1 tables and code paths. Create V1 table backups (v1_backup schema), drop FK constraints from deprecated tables, remove V1 columns from dual-path tables (screenshot_captures, notification_delivery_log). Update Drizzle schema and queries to use vNext only. Workers and web app routes migrated to vNext-only reads. Add DROP TABLE migration (0060) for final V1 table removal after stability verification
  • Scripts: Add domain expansion tooling for Volume 2 URL seeding - CLI argument support for validate-domains.ts and generate-url-patterns.ts, domain extraction script for batch processing
  • Reference Data: Add expanded domain list with ~1,900 domains across 100 sectors (50 B2B Work + 50 B2C Personal audiences) for system URL seeding
  • Scripts: Add pipeline diagnostic script (diagnose-pipeline.ts) for investigating URL processing stalls - checks queue health, database state, and provides actionable recommendations
  • Database: Add brand categorization system with 3-level taxonomy (Industry > Sector > Niche). New tables: brand_categories, brand_category_assignments, brand_category_proposals. Includes 18 L1 industries and 54 L2 sectors
  • Scripts: Add brand category migration tooling (pdl-category-mapper.ts, migrate-brand-categories.ts, seed-brand-categories.ts) for PDL-first classification with AI fallback support
  • Worker: V1 → vNext migration Phase 5.2.1 - SMS worker vNext support. Add url_change_event_id to SMS queue message schema, pass vNext FK from tracker worker, migrate logSmsDelivery to write to unified notification_delivery_log table
  • Database: V1 → vNext migration Phase 5.2.2 - Add vNext daily digest queries (getPendingDigestNotificationsVNext, batchCreateNotificationDeliveriesVNext, batchMarkNotificationsSentVNext) that read from url_change_events and write to notification_delivery_log
  • Web App: V1 → vNext migration Phase 5.2.2 - Daily digest cron now supports VNEXT_READ_ENABLED flag for full vNext read/write path
  • Database: V1 → vNext migration Phase 5.3 - Add vNext FK columns (url_id, url_observation_id, url_change_event_id) to screenshot_captures table, make tracked_url_id nullable
  • Queue: V1 → vNext migration Phase 5.3 - Update CaptureScreenshotsMessageSchema with optional V1 IDs and new vNext ID fields for screenshot capture queue messages
  • Worker: V1 → vNext migration Phase 5.3 - Tracker worker now fully supports V1_WRITE_ENABLED=false. V1 writes conditional, vNext observations created first, notifications use vNext IDs when V1 disabled
  • Worker: V1 → vNext migration Phase 5.3 - Render worker supports both V1 and vNext IDs for screenshot capture, uses fallback logic for storage paths
  • Shared: Add www. stripping to URL canonicalization — canonicalizeUrl() now normalizes www.example.comexample.com for consistent URL deduplication
  • Scripts: Add seed-urls-from-markdown.ts for seeding URLs from validated markdown reference documents with page type inference from URL patterns
  • Database: Phase 1 rename brand_sitesdomains - Rename table, enum, FK columns, indexes, RLS policies, and triggers. Update Drizzle schema and queries. Rename UI routes /brands//domains/ and API routes /api/brands//api/domains/. Add deprecated function aliases for backward compatibility
  • Database: Phase 2 domain verification workflow - Add domain_verification_status enum (pending/approved/rejected), verification columns to domains table (verification_status, submitted_by, verified_by, verified_at, rejection_reason, rejected_by, rejected_at). RLS policies enforce visibility: approved domains visible to all, pending visible only to submitter. Add Drizzle query functions for admin review workflow
  • Web App: Phase 5 URL validation integration - Add domain verification checks to URL add flow (POST /api/urls) and validation endpoint (POST /api/urls/validate). URLs from rejected domains are blocked with user-friendly error messages. New domains from user submissions start as pending and require admin approval. Add getRejectedDomainInfo query for rejection details
  • Database: Phase 5 domain rejection cascade - Add rejectDomainWithCascade function that deactivates all user subscriptions to URLs from rejected domains, returning tracking credits. Add createDomainRejectionNotifications to notify affected users via in-app notifications
  • Admin: Phase 5 rejection UI enhancement - Admin domain rejection now triggers full cascade (subscription deactivation + user notifications). Toast shows count of affected users
  • Database: Data alignment Phase 9.1 - Rename folder_urlsuser_folder_urls table with updated indexes, constraints, RLS policies, and database function references (migration 0064)
  • Database: Data alignment Phase 9.2 - Rename brand_urlsseeding_urls table to clarify its role as curated seeding catalog, with updated indexes, constraints, triggers, and RLS policies (migration 0065)
  • Database: Data alignment Phase 9.3 - Add audience field to vNext urls table with index, backfill from linked domains, and helper function get_url_effective_audience() (migration 0066)
  • Database: Update Drizzle schema exports: folderUrlsuserFolderUrls, brandUrlsseedingUrls, add audience field to urls table
  • Scripts: Add enrich-urls-from-seeding-urls.ts for backfilling URLs with page_type, brand_id, audience, and canonical_domain from seeding catalog
  • Scripts: Add audit-data-alignment.ts comprehensive audit framework for ongoing data quality monitoring across urls, brands, domains, and seeding_urls tables
  • Scripts: Update all seeding scripts to use new seeding_urls table name (seed-urls-vnext.ts, audit-seeded-urls.ts, seed-brand-urls.ts, seed-brand-library-pdl.ts, batch-db.ts)
  • Database: Update queries.ts to use user_folder_urls table name

Documentation

  • Planning: Add Phase 6 V1 cleanup plan with comprehensive artifact inventory (~25 files, 12-17 hours estimated)
  • Config: Document VNEXT_READ_ENABLED flag in .env.example with migration sequence

Bug Fixes

  • Web App: Fix uncaught promise rejection in enqueue-due-urls cron route — add await to returned promises in try block to ensure error handling catches async rejections

February 2, 2026

Features

  • Database: V1 → vNext migration Phase 1 - Fix Drizzle schema drift (observedAtcheckedAt in url_observations), add vNext query adapters (getDueUrlsVNext, getTrackedUrlsVNext, etc.)
  • Database: V1 → vNext migration Phase 2 - Migrate web app cron job to support vNext reads with VNEXT_READ_ENABLED feature flag
  • Database: V1 → vNext migration Phase 4 - Migrate admin dashboard stats from V1 tables (url_checks, url_changes) to vNext tables (url_observations, url_change_events)
  • Queue: Add optional url_id field to TrackUrlMessage for vNext worker support
  • Database: Consolidate brand tables - rename brand_library_* to simpler names (brands, brand_urls, brand_sources, brand_review_queue)
  • Database: Consolidate folder tables - merge favorite_folders + user_favorites into unified user_folders with folder_type discriminator
  • Database: Consolidate metadata tables - merge page_type_metadata + brand_type_metadata into unified type_metadata with category discriminator
  • Database: Consolidate notification delivery tables - merge notification_deliveries + sms_delivery_log into unified notification_delivery_log with channel discriminator
  • Database: Add migration to align url_source enum with Drizzle schema (brand_seededsystem_seeded) and seed 1,372 pattern URLs
  • Database: Add single-product to page_type enum and fix brand_url_type naming convention mismatch
  • Scripts: Add pagination to URL seeding and audit scripts to handle large datasets
  • Scripts: Add URL seeding migration scripts (audit-seeded-urls.ts, seed-urls-vnext.ts) to enforce pattern-only seeding
  • Shared: Add isRootDomain() and assertNotRootDomain() URL validation utilities
  • Invariants: Add "No TLD seeding" invariant to CLAUDE.md - system-seeded URLs must be patterns, never root domains
  • Database: Add use_case_audience enum and columns for work/personal/hybrid content filtering
  • Shared: Add UseCaseAudience schema, types, and getEffectiveAudience helper
  • AI Package: Add audience classification module with AI and heuristic fallback
  • Web App: Add AudienceTabs component for filtering Discover content by work/personal context
  • Web App: Add audience tabs to Discover Use Cases page with persona count display
  • Admin Dashboard: Add audience management page for brands, page types, personas, and use cases
  • Shared: Add brand type validation schema with soft enum pattern for Discover feature categories
  • Scripts: Add PDL domain enrichment script for business summaries, HQ location, and company metadata
  • Scripts: Add PDL domain discovery script for work audience seeding with URL pattern generation
  • Worker: Phase 2e - Make vNext the PRIMARY write path in tracking worker. vNext writes (url_observations, url_change_events, url_change_summaries) are now unconditional. V1 writes are guarded by V1_WRITE_ENABLED flag (default: true) for downstream compatibility
  • Database: Phase 3a - Add content_hash and render_time_ms columns to screenshot_captures table (migrated from url_renders)
  • Worker: Phase 3a - Migrate render worker from url_renders to screenshot_captures table for visual capture consolidation
  • Worker: V1 → vNext migration Phase 3 - Tracker worker supports dual message format (tracked_url_id + url_id). When url_id present, uses vNext queries (hasCheckedTodayVNext, getLatestObservationForUrl, updateNextCheckTimeVNext) and skips canonical_url resolution
  • Database: V1 → vNext migration Phase 5 - Add url_change_event_id FK to notification_delivery_log table (migration 0055) for vNext notification linking
  • Database: Update createNotificationDelivery to write to unified notification_delivery_log table with vNext FK support
  • Web App: V1 → vNext migration Phase 2.1 - Migrate URL API endpoints (/api/urls, /api/urls/validate) to read from vNext tables (urls, user_url_library) when VNEXT_READ_ENABLED=true
  • Web App: V1 → vNext migration Phase 2.2 - Migrate dashboard pages to read from vNext tables with new query helpers (getRecentChangesWithSummariesVNext, getUrlStatsVNext, getChangeEventByIdV1Compat)
  • Web App: V1 → vNext migration Phase 2.1 (complete) - Migrate brands API and dashboard pages with getTrackedUrlsForBrandVNext query helper

Documentation

  • Product Docs: Comprehensive refresh of page-type-definitions.md with database schema section, icon mappings, and updated metadata

February 1, 2026

Features

  • Docs Lint: Add active, locked, archived as valid status values for plan governance
  • Scripts: Add frontmatter maintenance script with audit, auto-fix, and interactive modes
  • Scripts: Expand persona system from 6 to 34 personas (18 work, 13 personal, 3 hybrid)
  • Scripts: Add shared persona definitions module at scripts/lib/personas.ts
  • Scripts: Update generate-use-cases.ts with new CLI options (--persona-type, --list-personas)
  • Scripts: Update seed-use-cases-migration.ts to use shared persona definitions

Bug Fixes

  • Scripts: Fix UI registry export updating last_updated date on every run regardless of content changes
  • UI Components: Fix 17 lint warnings (a11y semantic elements, array index keys, explicit any)

CI/CD

  • CI: Add frontmatter validation to CI pipeline

Generated on 2026-02-17