February 2026 Changes
February 28, 2026
Features
- DB: Full category taxonomy expansion — merge Business + Finance into unified root (migration 0140); add 3 new root categories: Health & Medicine, Language & Linguistics, Space & Astronomy with full hierarchies and domain-specific schemas (migration 0141); expand 6 previously-empty roots (Architecture, Auto, Current Events, Entertainment, Technology, Weather & Climate) and 14 thin roots with new depth-1 subcategories, depth-2 children, and sport-specific extensions across 7 migrations (0140-0148); add depth-3 (4th level) categories with ~2 children per depth-2 node; total active categories grow from ~276 to ~1,200+
- Code: Update
getActiveTopicCategories()to supportchildDepth >= 3with 3-level nested Drizzlewithrelation loading; updatecategory-mapper.tsPATH_CATEGORY_MAP with new root keyword patterns and aliases (health-medicine, language-linguistics, space-astronomy); update richness tier classification for new roots; add 11 new and expand 17 existing CategorySpec definitions incategories.tsfor seed pipeline coverage - Fact Pipeline: Extend category system to 3 levels (depth 0→1→2) —
getFactsByTopic()aggregates descendants at any depth (not just root);getActiveTopicCategories()gainschildDepthoption for nested relation loading; newgetDescendantCategoriesForParent()returns tree structure for workers; AI prompts updated withformatSubcategoryHierarchy()to instruct "deepest matching level" classification; workers flatten tree for slug resolution at any depth; feed UI renders 3rd badge row for sub-subcategories with progressive drill-down - Fact Pipeline: Activate ~95 subcategories for pipeline routing and feed UI — AI extraction and evergreen generation now classify facts into subcategories via prompt injection; workers resolve subcategory slugs to IDs with parent-chain schema fallback;
getFactsByTopic()aggregates descendant facts for root categories; feed UI shows drill-down subcategory badges when a root category is active - DB: Fix schema quality issues preventing seed pipeline from producing fact records (migration 0137) — add root-level schemas for 5 gap categories (history, culture, science, geography, space) unlocking ~1,798 previously unprocessable seeds; remove TV duplicate
general_factschema; upgrade insects-invertebrates fromgeneral_facttoanimal_factwith insect-specific keys; delete 5,311 orphaned seed entries on inactive "Things" category; remove all remaininggeneral_factschemas from 4 stale inactive categories - DB: Add
getSubcategoriesForParent()query andgetDescendantIds()helper using materialized path LIKE queries; expandgetPublishedFacts()withtopicCategoryIdsarray filter; addincludeChildrenoption togetActiveTopicCategories()using Drizzlewithrelation; expandgetExistingTitlesForTopic()to span full category tree for dedup; add parent-chain fallback togetSchemaForTopic()(max 3 levels) - DB: Add domain-specific schemas for records/nature, records/human-achievement, and current-events (migration 0138) — both depth-1 records subcategories now have explicit 9-key record_holder schemas (7 base + 2 extensions) matching the convention used by all depth-2 grandchildren; nature gains ecosystem_or_phenomenon and measurement_scale; human-achievement gains achievement_category and significance; current-events gets a new 9-key news_event schema (event, category, date, location, key_figures, source, impact, status, preceding_event) for news ingestion pipeline; all active categories now have schemas
- DB: Seed 144 depth-2 subcategories across all 65 remaining depth-1 parents (migration 0139) — completes the 3-level taxonomy with 2-3 grandchildren per parent; trigger auto-inherits parent schemas and challenge formats; 18 high-value categories gain custom schema extensions (e.g. dive_depth_meters for marine mammals, scoville_rating for spices, prize_pool for esports); total depth-2 categories now 165
- DB: Add depth-3 (4th level) categories Part 1 — new roots and empty roots (migration 0147) — 138 depth-3 subcategories across 8 root categories (Health & Medicine, Language & Linguistics, Space & Astronomy, Architecture, Auto, Technology, Weather & Climate, Entertainment); each depth-2 parent gets exactly 2 depth-3 children; 16 high-value nodes gain custom schema extensions (2-4 keys each) for viral-diseases, historical-pandemics, large-language-models, gothic-cathedrals, teams-constructors, atlantic-hurricanes, and more; trigger auto-inherits parent schemas; total depth-3 now 138
- DB: Add depth-3 categories Part 2 — expanded roots (migration 0148) — 488 depth-3 subcategories across 25 existing root categories; total taxonomy now 1,104 active categories (33 roots, 152 d1, 313 d2, 606 d3)
- Seed Pipeline: Generate curated seed entries across all 44 category specs — 12,375+ new entities across all root and expanded subcategories using AI entity generation; all 33 active root categories now have seed coverage; total seed_entry_queue grows to 49,469 entries across 197 subcategories
- Docs: Update SEED.md control prompt — fix topic examples to match actual
CATEGORIESspec (science 5 subcats, history 5 subcats with correct counts); add 3-level category hierarchy documentation (31→80→165); add 7 missing scripts to Available Scripts table with verified flags; add enrichment orchestrator section (8 sources); update model eligibility to reflect actual results (gemini-3-flash-preview and gemini-2.5-flash pass, gpt-5-mini does not); add model verification step and entity materialization to execution checklist; add cost estimates for new operations; deprecate improve-titles.ts
February 27, 2026
Features
- AI Pipeline: Add deterministic notability bypass for KG+Wikidata-confirmed entities — when both KG resultScore >= 100 and Wikidata sitelinkCount >= 20, override AI-generated notability with deterministic 0.85 score; wired into extract-facts, explode-entry, and generate-evergreen handlers; emits
facts.notability_bypassmetric for monitoring - AI: Add enrichment orchestrator (
packages/ai/src/enrichment.ts) — single entry point that resolves context from multiple API clients in parallel viaPromise.allSettled(), with topic-based routing (sports→TheSportsDB, music→MusicBrainz, geography→Nominatim, books→Open Library) and universal clients (KG, Wikidata, Wikipedia); supportsskipoption for pipelines that already have specific sources; 13 unit tests - AI: Wire multi-source enrichment into all 4 AI pipelines — news extraction (
extract-facts.ts) upgraded from KG-only to full orchestrator, seed explosion (explode-entry.ts) and evergreen generation (generate-evergreen.ts) gain enrichment context injection, validation evidence (evidence.ts) gains Wikidata cross-reference plus domain-specific client lookups (TheSportsDB, MusicBrainz, Open Library) gated on topic path - AI: Add 7 free API client modules for fact engine enrichment — Wikidata REST (structured claims, 24hr cache), Wikipedia Summaries (narrative context, 1hr cache), TheSportsDB (team/player/event data, 1hr cache), MusicBrainz (artist taxonomy with 1 req/s rate limiter, 24hr cache), GDELT 2.0 (news ingestion source + event context, 15min cache), Nominatim (geographic disambiguation with 1 req/s rate limiter, 24hr cache), Open Library (book/author validation, 24hr cache); all follow kg-client.ts architecture pattern with graceful degradation, bounded in-memory cache, metrics, and @hook annotations for future pipeline integration
- AI: Refactor Wikipedia lookup from inline evidence.ts code into shared wikipedia-client.ts — validation pipeline imports from centralized client instead of duplicating API calls
- AI: Refactor TheSportsDB from inline resolve-image.ts code into shared sportsdb-client.ts — image resolution cascade imports from centralized client with expanded team/player/event endpoints
- Scripts: Add taxonomy schema coverage audit tool — developer-time CLI (
scripts/taxonomy/audit-schema-coverage.ts) that samples entities from DB, queries KG + Wikidata + Wikipedia + domain APIs, compares structured data against current fact_keys and vocabulary, surfaces schema gaps with 6 analysis checks (field coverage, type alignment, vocabulary gaps, entity type distribution, domain-specific coverage, sitelink notability) - Scripts: Add recursive tree walker to taxonomy audit tool —
--depth=Nflag (0-3) walks taxonomy tree from any node, audits each node individually with per-node entity sampling (no child pooling), detects ownership gaps (subcategories missing vocabulary/voice entries in TAXONOMY_RULES_DATA/TAXONOMY_VOICES_DATA), generates prioritized action plan (high/medium/low) with JSON + Markdown output; any active topic_categories slug is now valid (not just root slugs); deferred batch Wikidata property label resolution across all nodes; backward compatible at--depth=0
February 26, 2026
Features
- Challenge Images: Add RESOLVE_CHALLENGE_IMAGE pipeline for anti-spoiler image resolution — challenges where the fact's entity image would reveal the answer (reverse_lookup always, fill_the_gap/statement_blank conditionally) get context-aware images via Wikimedia Commons, Unsplash, and Pexels cascade; adds image_url, image_source, image_attribution, image_status columns to fact_challenge_content (migration 0136); extracts shared tryUnsplash/tryPexels into image-sources.ts; registers 4th queue consumer in worker-ingest
- AI: Add Google Knowledge Graph integration for entity validation and enrichment — shared KG client in packages/ai with in-memory cache (1hr TTL, 5K max); seeding pipeline filters non-notable entities before AI explosion (~20-30% cost savings); news extraction enriches AI prompts with KG entity context for better grounding; entirely optional (pipelines degrade gracefully without API key)
February 25, 2026
Features
- AI: Promote gemini-3-flash-preview to default model across all tiers — replaces gemini-2.5-flash in
DEFAULT_TIER_CONFIG, test pipeline, and news testing defaults; T2 canary results: 100% quality, 94/100 signoff, $0.0121/fact (cheapest), 4.2x faster than gpt-5-mini, only model to pass eligibility across all dimensions - AI: Prime gemini-3-flash-preview as dedicated signoff reviewer with adapter-driven customization — add
signoff_reviewtoAdaptableTaskunion, adddefault: return {}to all adapters for zero-touch future task additions, wire adapter prefix/suffix/temperature/providerOptions into both test pipeline signoff phases - AI: Gemini 2.5 Flash adapter v4 — cross-field consistency audit, thinking budget 1024→2048 for fact tasks, checklist relocated to prefix for primacy bias, failure examples for correct_answer and setup_text, banned generic reveal openings with content-specific requirement, domain-match instructions for individual sport schema fields, banned word enforcement, calibrated signoff rubric with factual-accuracy-first token efficiency scoring; T2 canary (28 facts): validation 100, evidence 100, challenges 100, schema 94, voice 92, style 91, token_efficiency 92
- AI: Gemini 2.5 Flash v5 architectural fixes for 97+ eligibility — add
individual-sportsDB subcategory with semantically correct schema (athlete/event/result instead of team fields), deterministicpatchPassiveVoice()post-generation rewriter for common passive→active voice patterns, batch-levelpatchRevealDuplication()to eliminate repetitive reveal openings across challenges, simplify adapter DOMAIN-MATCH workaround, upgrade passive voice drift severity info→warning - DB: Upgrade ~50 remaining subcategory schemas from generic
general_fact(4 fields) to domain-specific schemas inheriting parent fields + 1-2 extras (migration 0135); addtrg_inherit_parent_schemaauto-inheritance trigger so future subcategory inserts automatically copy parent schema and challenge_format_topics
Bug Fixes
- AI: Relax structural date validation to accept natural language dates ("January 28, 2017") instead of requiring ISO 8601 only — users prefer human-readable date formats in fact values
- AI: Replace
patchRevealDuplication()(exact-dedup, ~1% coverage) withpatchGenericReveals()(semantic pattern matching, ~75% coverage) - fixes subject extraction "The" bug, double em-dash concatenation artifact, and matches generic reveal opener families ("You absolutely nailed it", "That's exactly right", "That's the exact", etc.); expand Gemini adapter banned reveal openings from 9 to 19 patterns with failure examples - AI: Gemini 2.5 Flash v5 gap closure — expand passive voice patcher from 20→36 patterns; expand generic reveal patterns from 8→21 semantic families; add per-entity dedup tracking to prevent rewrite template repetition across same-subject challenges; expand rewrite pools from 6→16 templates each; add
patchTextbookRegister()for non-passive encyclopedic phrases ("This victory cemented" → "That victory sealed"); addpatchPunctuationSpacing()for missing spaces after sentence-ending punctuation; strengthen factual accuracy guardrails (qualifier precision, team sport score audit, context sentence count gate, controversial attributions, date disambiguation, null-vs-populated rule); expand banned reveal openings from 19→40+ patterns - AI: Tiered eligibility thresholds — split single 97% threshold into structural (97%: validation, evidence, challenges) and subjective (90%: schema, voice, style, token_efficiency) tiers; recalibrate signoff rubric with granular 93-96 scoring bands and explicit "not a violation" guidance to reduce false deductions from LLM reviewer noise; increase challenge generation thinking budget 2048→4096 for mandatory sentence-count audit; add Zod
.describe()annotations to challenge output schema for better structured output compliance - CI: Fix frontmatter validation, docs-code-mapping binding check (add missing code paths for superset rule), and token-optimization smoke test (checklist moved from suffix to prefix in v4 adapter)
Infrastructure
- Docs: Add anti-staleness infrastructure for
docs/prompts/— createprompts:checkCI script (scripts/check-prompt-paths.ts) that validates all file paths referenced in prompt code blocks still exist on disk, addrelated_codefrontmatter to all 67 prompt files connecting them to thedocs:stalenessdetector, createdocs-code-mapping.ymlwith 9 domain-level cross-cutting mappings, and wireprompts:checkinto the CI pipeline - Docs: Fix 4 broken path references in prompt files caused by v2 fact engine config consolidation (
challenge-pre-generated-styles.ts,challenge-voice-constitution.ts,challenge-voices.ts→challenge-voice.ts,challenge-content-rules.ts) - Docs: Expand
related_codefrontmatter to 22 high-value docs acrossdev/,product/,architecture/,runbooks/,rules/,specs/, and top-level — total coverage now 195 docs feeding thedocs:stalenessdetector
February 24, 2026
Features
- AI: Deduplicate STYLE_RULES in challenge prompt assembly — extract 5 shared technique descriptions (Breadcrumb Trail, Pivot, Reward, Discovery, Unfolding) emitted once instead of 6×, emit only per-style refinements and style_data per style; reorder prompt sections for Gemini implicit cache optimization (stable block front-loaded, variable block last); reduces STYLE_RULES section ~63% (~4,755 tokens) and extends cacheable prefix from 9.8% to 93%
- AI: Reduce input token costs ~25-35% via prompt compression — compact JSON serialization across all AI calls, compress Gemini fact prefix from 63 to 40 lines, deduplicate BEFORE SUBMITTING checklists into shared constant, extract recency detection into
recency-utils.ts - AI: Move challenge_title generation from fact-level to per-challenge — each challenge in
fact_challenge_contentnow gets its own theatrical title generated in Phase 2 alongside the question/answer, eliminating answer-leak bugs where a single fact-level title would spoil what the challenge asks (e.g., "Twenty-Three Slams" for a question about Grand Slam count) - AI: DeepSeek free-text adapter (V4) — bypass
json_objectmode entirely for DeepSeek (which caused 20-50% generation failures); usegenerateText()with post-parse JSON extraction and Zod validation;generateObjectForProvider()routes DeepSeek through free-text path while all other providers use standardgenerateObject() - AI: Local Supabase test workflow —
--commitflag onllm-fact-quality-testing.tsmanages full lifecycle: start local Supabase, seed reference data, write fact_records and challenge_content to local DB during testing, reset DB after; falls back to JSONL-only when Docker unavailable
Infrastructure
- AI: Add Mistral AI provider (mistral-large-latest, mistral-medium-latest, mistral-small-latest) — env schema, model registry with pricing, model-router instantiation via
@ai-sdk/mistral; no adapter yet (uses default pass-through) - Config: Replace CSV→JSON config pipeline with direct TypeScript data files — 11 new
as const satisfiesconfig files replace 22 CSVs, 6 generated JSONs, and 17 sync scripts; consumers (challenge-content-rules.ts,taxonomy-content-rules.ts,model-registry.ts,generate-curated-entries.ts) import directly from TypeScript; removesconfig:sync,config:check,config:exportcommands - Docs: Unify prompt system — migrate 17 existing prompts + convert 39 workflow actions into 55 domain-organized prompt files across 12 domains (
docs/prompts/{model-testing,seeding,quality,content,pipeline,config,taxonomy,database,environment,deploy,brand,users}/); each domain has00-index.mdwith FAQ sections absorbing 10 workflow FAQ files; deletedocs/workflows/(99 files)
Performance
- AI: Gemini thinking token budget control and cost tracking — cap
thinkingBudgetper task (0 for scoring/validation/moderation, 1024 for extraction/generation); fix Gemini 2.5 Flash pricing from $0.15/$0.60 to GA $0.30/$2.50; track reasoning tokens separately inai_cost_trackingwith newreasoning_tokens_totalcolumn; threadproviderOptionsandreasoningTokensthrough all 13 AI call sites; adds migration 0133 - Worker: Cap article sources in fact extraction to 5 sources × 1,500 chars each — prevents unbounded prompt tokens when stories have 10+ news sources (~20K chars)
- Worker: Micro-batch challenge content generation — dequeue up to 5 GENERATE_CHALLENGE_CONTENT messages and process in a single AI call, amortizing the ~5,200-token system prompt across the batch
- AI: Cap evergreen title dedup list to 50 — pass explicit limit to
getExistingTitlesForTopicand add defensive.slice(0, 50)in prompt construction to prevent growing token costs - Queue: Replace flat 1s nack retry delay with exponential backoff (5s → 30s → 60s cap, 15% jitter) — prevents retry storms on rate limit errors
- AI: Sliding window on conversational challenge history — keep first exchange + last 3 exchanges (8 messages max) instead of entire conversation, saving ~4,000 tokens on long sessions
- AI: Use actual AI SDK token counts instead of
length / 4heuristic — readsresult.usage.inputTokensandresult.usage.outputTokensfor accurate cost tracking - AI: Add Google/Gemini daily budget cap ($3.00 default) — mirrors Anthropic budget cap pattern with
isGoogleBudgetExhausted(), in-memory cache, andGOOGLE_DAILY_SPEND_CAP_USDenv var - Worker: Populate super fact entry summaries with actual fact titles — query 3 sample fact titles per entry instead of generic "N facts generated" string, giving AI real signal for correlation discovery
Testing
- AI: Run light model adapter seed test (10 facts, gemini-2.5-flash) — 90% validation pass rate, 100% CQ-002 compliance, 270/270 challenges valid, 100/100 signoff score, $0.25 total cost
Bug Fixes
- AI: Fix DeepSeek evidence scoring by appending schema text after voice prefix in fetch wrapper — schema blob was prepended to system prompt, pushing the 1200-token voice/accuracy prefix out of position 0 where non-reasoning models give strongest attention
- Public Site: Fix wiki build failure by switching MDXRemote to
format: 'md'— prevents curly braces and angle brackets in docs from being parsed as JSX expressions
February 23, 2026
Features
-
AI: Add taxonomy quality onboarding enforcement — drift scoring penalizes unknown topic slugs (score 0.6 instead of 1.0), worker-validate flags
taxonomy_rules_missingon validated facts with no rules, and new CI completeness check (bun run taxonomy:completeness-check) validates voice coverage, content rules, and vocabulary depth across all 32 taxonomy slugs -
Testing: Add news ingestion quality testing pipeline — 6-phase script (
scripts/news-testing/) with DB-free news fetcher (NewsAPI, GNews, TheNewsAPI), fact extraction viaextractFactsFromStory()withmodelOverride, and reuse of seed harness validate/challenge/signoff/report phases -
AI: Add
modelOverridetoExtractFactsInputin fact-engine.ts — allows pinning a specific model for testing, bypassing the tier-based model router -
Docs: Add FAQ quick-reference series (
docs/workflows/faq/) -- 10 files with 110 Q&A entries across 9 domains (ingestion, facts, validation, seeding, challenges, frontend, config, database, operations) plus navigation index with "Which FAQ do I need?" lookup table -
AI: Fix signoff pipeline to display all challenge content fields for quality review — was only showing
challengeText, now showssetupText,correctAnswer,revealCorrect,revealWrong -
AI: Split Gemini 2.5 Flash adapter into task-specific prefixes (
GEMINI_FACT_PREFIXfor schema tasks,GEMINI_CHALLENGE_PREFIXfor challenge generation) with BEFORE SUBMITTING checklists -
AI: Remove score-anchoring bias from Gemini signoff guidance — reviewer now scores on observed output quality, not historical assumptions
-
AI: Lower Gemini temperatures for schema-bound tasks (fact_extraction 0.7→0.5, seed_explosion 0.7→0.5, evergreen 0.7→0.6)
-
Testing: Bump Google provider concurrency cap from 5 to 15 for faster Gemini test runs
-
AI: Extend Gemini 2.5 Flash adapter factual accuracy guardrails with v3-specific error patterns (numeric specs, artwork claims, sports timeline reconstruction, title field capture)
-
AI: Reduce evidence
likely_inaccuratesoft-reject penalty from -0.25 to -0.15 — graduated penalty now meaningfully distinguishes soft vs hard rejects -
AI: Tighten cross-model escalation confidence window from 0.50-0.75 to 0.50-0.65, reducing evidence denominator volatility
-
AI: Add explicit severity calibration to cross-model prompt — defaults to
infofor simplifications, reservingwarningfor materially incorrect claims -
AI: Strengthen Gemini generation guardrails v5 — add schema-mismatch awareness to cross-model prompt, unfinished project guards, entity attribution checks, influence precision rules, and concrete superlative failure examples
-
AI: Filter schema-mismatch warnings from evidence escalation trigger — prevents domain-terminology flags from inflating evidence denominator
-
AI: Add v6 generation guardrails — outdated statistics awareness (pre-dam/post-dam figures), score consistency cross-check (winner must have higher score), and expanded schema-mismatch filter patterns for tennis/music domain warnings
-
AI: Add recency-aware cross-model validation — facts from news articles published within 7 days get calibrated severity rules (unverifiable-due-to-recency =
info, notcritical) and lower pass threshold (0.35 vs 0.50), boosting news validation pass rate from 52% to 94% -
AI: Add banned word reinforcement to Gemini challenge adapter —
GEMINI_CHALLENGE_PREFIXnow explicitly bans "Trivia", "Quiz", "Easy one" in reveal_correct/reveal_wrong fields -
AI: Add recency-aware evidence validation — evidence tier reasoner now receives article publish date context, preventing hard "inaccurate" rejections for recent events it cannot verify; hard reject downgraded to soft reject (-0.15 penalty) for recent articles without DB contradictions
-
Testing: Fix report CQ-002 weakness display — floating-point equality (
=== 1.0) replaced with threshold (>= 0.99); rates above 95% no longer flagged as weakness -
Testing: Fix report verdict threshold — single signoff failure no longer blocks "Production Ready"; verdict now uses 5% failure tolerance instead of zero-tolerance
-
Testing: Add validation concurrency throttle and response contamination detection — validation phase now uses separate
--validation-concurrency(default: 3) to prevent Gemini API response mixing under load; addsisResponseContaminated()entity-name sanity check with automatic retry on cross-model and evidence phases
February 22, 2026
Features
- Config: Add CSV → JSON config pipeline — shared library (
scripts/config/lib/) with CSV reader supporting@file:blob references, Zod validation with source tracking, companion CSV joiner with FK joins and dot-notation nesting, SHA-256 checksum staleness detection, and JSON writer with embedded checksums - Config: Add export scripts for 6 config domains — categories, models, challenge rules, taxonomy rules, seed controls, app controls; each extracts hardcoded TypeScript constants or YAML-in-markdown blocks to normalized CSV files under
config/csv/ - Config: Add sync scripts for 6 config domains — each reads CSV, validates via domain-specific Zod schemas, and writes
config/generated/*.jsonwith content checksums; supports@file:references for large text blobs and pipe-delimited arrays - Config: Add
config:syncorchestrator andconfig:checkCI staleness guard — orchestrator runs all 6 sync scripts in sequence, check command verifies embedded SHA-256 checksums match current CSV content - Config: Migrate 4 TypeScript constant files to JSON imports —
generate-curated-entries.ts(905→225 lines),model-registry.ts(257→118 lines),challenge-content-rules.ts(1276→344 lines),taxonomy-content-rules.ts(2109→95 lines); all preserve identical export names and types for backward compatibility - Docs: Add per-domain config control workflow docs (
docs/workflows/controls/) — 6 domain guides (categories, models, challenge, taxonomy, seed-controls, app-controls) with column references, editing workflows, troubleshooting tables, plus index README and cross-references from seeding-and-testing.md - Docs: Add evergreen ingestion product guide (
docs/product/evergreen-ingestion.md) and operator workflows (docs/workflows/evergreen-ingestion-workflows.md) — covers full pipeline from cron trigger through AI generation, validation, challenge content, and feed selection; workflows include manual triggers, monitoring, troubleshooting, and config dependencies - Docs: Add workflow action files system (
docs/workflows/actions/) — 76 task-driven action files across 11 domains (seeding, config, pipeline, content, brand, database, quality, docs, environment, admin, users) with copy-pasteable steps, YAML frontmatter (risk, AI cost, frequency), 12 domain READMEs, and master index catalog; cross-referenced from seeding-and-testing.md, evergreen-ingestion-workflows.md, and runbook.md
Infrastructure
- CI: Add
bun run config:checkto CI pipeline for config staleness detection
February 21, 2026
Features
- Scripts: Add bounded-concurrency parallelism to LLM quality testing pipeline — models run in parallel via
Promise.all(), entities/facts/challenges batched withpMap()worker pool; new--concurrency NCLI flag (default: 3) - Scripts: Add challenge batching — send all 6 styles per difficulty in one API call instead of individual calls, reducing challenge-phase API calls by ~6x
- Scripts: Add domain-aware schema key fallbacks —
DOMAIN_SCHEMA_KEYSmap provides semantically appropriate fact keys per taxonomy (sports, science, history, music, film, technology, art, geography), preventing cross-domain misapplication - Scripts: Add model-specific prompt reinforcement —
getModelPromptReinforcement()feeds known per-model weaknesses to the signoff reviewer for targeted quality scoring - Scripts: Add definitive report summaries with recommendations — executive summary with winner declaration and rankings, per-model strengths/weaknesses analysis, production readiness verdicts, tiered strategy recommendations, and actionable next steps
- Scripts: Add timestamped report archiving — reports now dual-write to both
report.md(latest) andreport-{timestamp}-{models}.mdfor historical comparison - UI: Switch fact-related dates to US middle-endian (MM/DD/YYYY) format with centralized
formatFactDateutility in@eko/shared - AI: Add ModelAdapter pattern — per-model prompt customization with suffix/prefix/override modes, adapter registry with caching, and eligibility tracking via JSONL (97% threshold across 7 dimensions)
- AI: Add model adapters for gpt-5-mini, gemini-2.5-flash, gemini-3-flash-preview, claude-haiku-4-5
- AI: Add gemini-3-flash-preview to model registry
- AI: Integrate adapters into fact-engine, seed-explosion, and challenge-content generation
- Scripts: Update test pipeline to use adapter-based signoff guidance and write eligibility JSONL
- AI: Add per-category domain vocabulary with auto-generation in taxonomy CLI — 32 categories backfilled with domain terms, expert phrases, and term preferences
- AI: Wire taxonomy rules, voice, and vocabulary into seed explosion pipeline — previously domain-blind, now receives full taxonomy context
- AI: Add drift coordinator system — 5 pluggable coordinators (structure, schema, voice, taxonomy, difficulty) detect semantic drift in LLM-generated challenge content via free heuristic checks; non-blocking by default with aggregate scoring and per-coordinator violation tracking
- Docs: Add Eko Product Bible (
docs/eko-product-bible.md) — end-to-end system overview covering pipeline stages, architecture, cascading dependency chains, agent ownership, CI quality gates, operations, and improvement roadmap; includes CI integrity checker (scripts/bible-integrity-check.ts) validating file paths, package names, queue types, and cross-doc links
Bug Fixes
- AI: Fix challenge generation 100% failure on OpenAI models — replace
z.record(z.string(), z.unknown())style_dataschema withz.union()of 6 concrete per-style Zod schemas, eliminatingpropertyNamesJSON Schema property that OpenAI's structured output API rejects
Refactoring
- AI: Remove DeepSeek provider — drop deepseek-chat/deepseek-reasoner from registry, config, model-router, validation pipeline, and test harness (~220 lines removed)
- Validation: Switch cross-model (Phase 3) and evidence reasoner (Phase 4c) from DeepSeek to gemini-2.5-flash
February 20, 2026
Bug Fixes
- Validation: Fix evidence validation (Phase 4) 0% pass rate — use reasoner verdict field for pass/fail gating instead of treating all
key_concernsas critical blockers; onlyinaccurate/likely_inaccurateverdicts now block, matching Phase 3's severity-based pattern - Validation: Improve Wikipedia entity extraction for evidence validation — add multi-strategy entity extraction (possessives, quoted names, proper noun sequences, topic path hints) with MediaWiki search API fallback, raising lookup success from ~15% to ~85%
- Validation: Add graduated response for
likely_inaccuratereasoner verdicts — apply -0.25 confidence penalty instead of hard fail, allowing facts with strong external evidence to survive historical nuance concerns - Validation: Distinguish schema format errors from factual inaccuracy in reasoner — add
schema_mismatchflag to prevent tennis/basketball schema mismatches from causing false rejections of correct facts - AI: Reduce DeepSeek-chat parse failures — enhance
json_objectmode schema injection with explicit formatting directives, add response-side JSON repair indeepSeekFetchWrapper(strips markdown fences, fixes stray periods after numbers, removes trailing commas, trims post-JSON commentary)
Refactoring
- Scripts: Rename
deepseek-quality-test.tstollm-fact-quality-testing.ts— generalize from DeepSeek-specific to model-agnostic quality testing; add--modelsflag for arbitrary model comparison,--signoff-modelfor configurable quality reviewer; supports gpt-4o-mini, claude-haiku-4-5, gemini-2.5-flash, MiniMax-M2.5, and all registry models
February 19, 2026
Features
- Validation: Add multi-phase fact validation pipeline — 4-phase validation (structural, consistency, cross-model, evidence) replaces single ai_cross_check for non-news facts with DeepSeek cross-provider verification, Wikipedia/Wikidata evidence corroboration, and full phase-by-phase audit trails
- Config: Add DeepSeek provider integration — deepseek-chat and deepseek-reasoner models via OpenAI-compatible API for cross-provider fact validation
- AI: Wire
topic_slugthrough challenge content generation — handler now passes taxonomy slug togenerateChallengeContent(), enabling domain-specific voice and content rules in challenge prompts - AI: Inject taxonomy content rules and voice into evergreen fact generation —
generateEvergreenFacts()now derivestopicSlugfromtopicPathand appendsTAXONOMY_CONTENT_RULESextraction guidance andTAXONOMY_VOICEregister to the system prompt - Scripts: Add
taxonomy:addCLI onboarding tool — interactive script that AI-generates domain-specific schema keys and content rules, creates a SQL migration, and updatestaxonomy-content-rules.tsin a single command - Schema Overhaul: Replace all generic_fact schemas with 33+ domain-specific schemas — 28 root categories upgraded from 4-field generic fallback to 7-10 field specialized schemas, 5 new subcategory schemas added (film_fact, music_fact, tv_show_fact, celebrity_fact, sports_legend), child subcategories get pre-merged schemas with parent base + child-specific additions
- AI: Add
modelOverrideparameter toexplodeCategoryEntry()andgenerateChallengeContent()— enables A/B testing of different AI models for fact and challenge generation without modifying tier config - Scripts: Add DeepSeek v3.2 quality testing pipeline — 5-phase CLI script (generate, validate, challenge, signoff, report) comparing DeepSeek-chat, DeepSeek-reasoner, and GPT-5-mini as fact/challenge generators with quality scorecard and cost-benefit analysis
- Schema Overhaul: Truncate fact_records and fact_challenge_content — clean slate for repopulation with domain-specific schemas
- AI: Add taxonomy voice layer — per-domain emotional register (TAXONOMY_VOICE) for 33+ taxonomies, injected between universal voice constitution and per-format STYLE_VOICE in challenge generation prompts
- AI: Add taxonomy content rules — per-domain formatting and factual conventions (TAXONOMY_CONTENT_RULES) injected into both extraction and challenge generation prompts with extraction_guidance, challenge_guidance, and avoid patterns
- AI: Refactor STYLE_VOICE from tonal personality to structural format mechanics — personality moves to TAXONOMY_VOICE, STYLE_VOICE becomes format-specific interaction mechanics (option curator, focused questioner, puzzle architect, etc.)
- AI: Redesign STYLE_RULES with tease-and-hint architecture — setup_text follows ANCHOR → ESCALATION → WITHHOLD arc, challenge_text uses pivot technique, reveal_correct rewards with bonus detail, reveal_wrong uses discovery technique, correct_answer follows 3-6 sentence unfolding arc
- AI: Add typed style_data validation for all 6 challenge styles — CQ-007 through CQ-011 validate fill_the_gap, direct_question, statement_blank, reverse_lookup, free_text style_data shapes
Bug Fixes
-
AI: Fix CQ-002 enforcement gap — add
patchCq002()call ingenerateChallengeContent()runtime path, not just seed script -
AI: Fix hardcoded cost estimate in challenge content generation — replace GPT-4o-mini pricing constants with
estimateCost()for model-aware calculation -
AI: Fix
_challenge_hintstypo in SUPER_FACT_RULES — corrected tochallenge_hintsto match schema field name -
AI: Fix banned word "trivia" in conversational challenge prompt — replaced with "deep knowledge" to align with BANNED_PATTERNS
-
AI: Deduplicate
CHALLENGE_TONE_PREFIX— consolidated from 3 copies (challenge-content-rules.ts, fact-engine.ts, seed-explosion.ts) to single export -
AI: Add taxonomy slug warning — log when
topicSlughas no TAXONOMY_VOICE entry so generic voice fallback is visible -
AI: Remove deprecated
ModelSelectiontype from model-router.ts -
AI: Fix text[] type support in buildFactObjectSchema — was falling through to z.string() instead of z.array(z.string())
-
UI: Fix snake_case mismatch in quiz-interface — AI generates
is_correctin JSONB but UI readisCorrect, causing correct answers to never highlight -
Fact Engine: Materialize 76 mid-level subcategories across 33 parent categories — migration 0129 inserts subcategory rows with correct parent_id, hierarchical path, depth, schema entries, and challenge format propagation
-
Fact Engine: Add entity materialization script (
materialize-entity-categories.ts) with 4-phase pipeline —--auditfinds entities with 5+ facts (2,770 found),--classifyuses batched AI classification with JSONL output,--insertcreates leaf topic_categories,--linkreassigns facts to entity categories with dry-run support -
AI: Add
entity_classificationtask type to fact engine model routing at default tier for cost-efficient batch classification -
Scripts: Update
generate-curated-entries.tsto resolve subcategory IDs — new seed entries get the most specifictopic_category_idandsuggested_topic_pathvia slug matching against materialized subcategories, with fallback to root category -
AI: Add per-style voice differentiation —
STYLE_VOICEconstant with 6 distinct voice definitions (gallery guide, dinner companion, co-author, knowledge-checker, mystery narrator, thoughtful interviewer), injected between voice constitution and style rules in generation prompt -
AI: Update CQ-002 fallback prefixes to align with per-style voice registers — more natural phrasing when AI fails to include second-person address
-
Scripts: Add
rewrite-challenge-defects.ts— targeted field rewrite script for defective challenges (CQ-002 prefixed, telegraphic setup/answer, short reveals) with--dry-run,--limit,--partition,--concurrencyflags, batched gpt-5-mini rewriting with per-style voice guidance
Documentation
- CLAUDE.md: Remove 5 non-existent scripts, add turbo filter pattern for workers, update project structure with 7 missing entries
- Rules Index: Replace v1 URL-tracking invariants with v2 fact-engine invariants, fix agent name (ux-page-card-designer to card-ux-designer)
- Challenge Content Rules: Align CQ numbering with code — add CQ-009 through CQ-013 for style_data validators, fix CC-002 heading to "Five-Field Structure"
- Rules Index: Add Challenge Content section with CC and CQ rule summaries, add fact-engineer to ownership table
- CHALLENGE_TONE.md: Update related_code frontmatter to include all 5 consuming source files
February 18, 2026
Documentation
- Docs: Update 6 documentation files for v2 fact engine sync — table flow diagram (+2 tables, count fix), fact anatomy (+2 schema refs), ingestion guide (alias resolution, maxDepth), news fact engine (xAI Grok default, XAI_API_KEY), seed runbook (+2 new script sections), Eko GTD templates (XS/QS tiers, Doc-Updates domain)
Features
- Eko GTD: Add quality tiers for Ads, Business, Research domains and introduce Doc-Updates domain
- Fact Engine: Add taxonomy coherence layer —
topic_category_aliasestable maps external news API slugs (business, entertainment, health, etc.) to internal topic_categories,unmapped_category_logtracks dropped categories for audit,resolveTopicCategory()provides 3-step alias fallback (exact match → provider-specific → universal),extract-factshandler updated to use alias resolution - Fact Engine: Reconcile taxonomy conflicts — deactivate 4 niche orphan roots (accounting, marketing, spelling-grammar, things), merge statistical-records into records, propagate challenge_format_topics to 29 expansion roots from migration 0101 that lacked format links
- Fact Engine: Add
maxDepthparameter togetActiveTopicCategories()andgetActiveTopicCategoriesWithSchemas()— all cron routes and feed page now usemaxDepth: 0to prevent quota explosion when subcategories are added - Rules: Add CC-010 (category alias normalization) and CC-011 (unmapped category audit) rules for taxonomy coherence
- Docs: Add taxonomy coherence GTD project (04-taxonomy-coherence.md) with 10 challenges across 3 waves
- Fact Engine: Unify challenge content generation post-validation — move
GENERATE_CHALLENGE_CONTENTenqueue fromimport-facts.tstovalidate-fact.tsso all source types (news, evergreen, imported) get challenge content after validation, avoiding wasted AI cost on rejected facts - Fact Engine: Wire
generationCostUsdthrough extraction pipeline —extract-facts.tsandgenerate-evergreen.tsnow compute and store per-record AI cost viaestimateCost() - Fact Engine: Wire
notabilityScore,notabilityReason,contextthrough import pipeline —ImportFactsMessageSchemaexpanded with optional fields,import-facts.tsandexplode-entry.tspass metadata through toinsertFactRecord() - Scripts: Add
backfill-fact-nulls.tsfor backfilling NULL notability scores and enqueuing challenge content for validated facts missing it, with--dry-run,--audit, and per-source-type defaults - Fact Engine: Implement GNews news provider — replace stub with full API client fetching from gnews.io/api/v4 with topic/search routing, rate limiting, and StandardArticle normalization
- Fact Engine: Add TheNewsAPI news provider — new provider client fetching from thenewsapi.com/v1 with top/all endpoints, UUID-based dedup, and StandardArticle normalization
- Fact Engine: Implement Unsplash image fallback — replace stub with full API client searching photos by entity title, landscape orientation, with proper attribution
- Fact Engine: Add Pexels image fallback — new cascade step after Unsplash searching Pexels photos by topic with attribution
- Config: Add THENEWS_API_KEY, UNSPLASH_ACCESS_KEY, and PEXELS_API_KEY env vars to config schema and FactEngineConfig
- Database: Add 'thenewsapi' to news_provider enum (migration 0123)
- Queue: Extend INGEST_NEWS provider enum and cron dispatch to support TheNewsAPI
February 17, 2026
Features
- Scripts: Add monthly changelog file rotation (
MM-YYYY.md) — sharedchangelog-utils.tsutility, updatedchangelog-check.tsandgenerate-changelog.tswith dynamic paths, auto-locking of previous month, migration script to splitunreleased.mdinto01-2026.md(locked) and02-2026.md(draft), updated 3 hooks and 10 documentation files - Skills: Add
/donesession summary skill for end-of-session documentation with descriptive filenames - Hooks: Add
/donereminder to PreCompact hook — warns when no session summary exists before context compaction - AI: Integrate xAI Grok as new AI provider — add
@ai-sdk/xaiSDK,grok-4-1-fast-reasoning,grok-4-1-fast-non-reasoning, andgrok-4models to registry with full model router support - AI: Switch all tier defaults to
grok-4-1-fast-reasoning— 4x cheaper output tokens than gpt-5-mini ($0.50 vs $2.00/M) with reasoning capability for more specific, cinematic challenge titles - AI: Add
content_cleanuptask type to fact engine for corpus-wide title/context/notability rewriting - AI: Upgrade all fact generation prompts to enforce rich narrative context — seed explosion, evergreen generation, news extraction, and super-fact discovery all now require Hook→Story→Connection structure (4-8 sentences), theatrical challenge titles with anti-pattern bans, and concrete good/bad examples. New facts match cleanup-pass quality from day one
- AI: Add
challenge_titleandcontextfields to news extraction output —extractFactsFromStorynow generates theatrical titles and narrative context alongside structured facts, eliminating NULL challenge_titles for news-derived records - Scripts: Add
cleanup-content.ts— full corpus rewrite script with 5 phases (audit, export, fix, upload, validate) using local-first JSONL architecture for crash-resilience - Scripts: Add
--recoverphase to cleanup-content.ts — classifies weak outputs (notability <= 0.5) into recoverable (rich source data) vs vague (poor source data) usingclassifyFactData()heuristic, re-processes recoverable facts with full structured data in prompt, supports--partition N/Mfor parallel execution - Database: Add
free_textto allfact_record_schemas.card_formatsarrays, expand format-style linkages (statement_blank/direct_question to know_a_lot_about), link sport and records subcategories to parent format mappings - Worker: Add concurrent message processing to worker-facts —
WORKER_CONCURRENCYenv var controls parallel messages per queue type (default: 1 for backward compat). Combined with multiple worker instances, enables ~10x throughput for seed pipeline runs - Fact Engine: Add gpt-5-nano and gpt-5-mini to model registry and Drizzle ai_model enum, set gpt-5-nano as default tier for cost-efficient seed generation
- Fact Engine: Add
maxRetries: 0to seed explosion and super-facts AI calls — delegates retry logic to queue-level backoff instead of Vercel AI SDK internal retries - Worker: Fix spinoff entries missing topic_category_id — pass parent entry's category ID through to insertSeedEntry for correct schema resolution
- Database: Add
fact_challenge_contenttable for pre-generated AI challenge content — migration 0121 with three-layer emotional arc fields (setup_text, challenge_text, reveal_correct, reveal_wrong), style_data JSONB, difficulty 1-5, and composite unique constraint on (fact_record_id, challenge_style, target_fact_key, difficulty) - AI: Add challenge content generation system — voice constitution, per-style rules, difficulty levels 1-5 with prompt guidance, super-fact awareness for cross-entity challenges, banned pattern validation, and
generateChallengeContent()function using Vercel AI SDK structured output - AI: Add challenge content rules and validation —
challenge-content-rules.tswithCHALLENGE_VOICE_CONSTITUTION,STYLE_RULESfor 6 styles,DIFFICULTY_LEVELS,SUPER_FACT_RULES, andvalidateChallengeContent()post-generation validator - Scripts: Add
generate-challenge-content.tsbackfill script — 6-phase JSONL pipeline (audit, export, generate, upload, validate, recover) with --difficulty, --partition, --concurrency, --limit flags following cleanup-content.ts patterns - UI: Add pre-generated challenge content to card detail — quiz, recall, and text-input components consume AI-crafted setup/challenge/reveal text with graceful fallback to algorithmic generation when no pre-generated content exists
- Queue: Add
GENERATE_CHALLENGE_CONTENTmessage type — Zod schema, queue routing, constructor, worker-facts handler that generates and upserts challenge content per fact record - Worker: Wire IMPORT_FACTS to enqueue challenge content generation — new facts automatically get challenge content generated after import
Refactoring
- Database: Drop 16 v1 legacy tables, v1_backup schema, 3 orphaned enums, and 2 orphaned functions in migration 0118 — page tracking (pages, user_pages, page_observations, page_change_events, page_change_summaries, section_dom_mappings, page_title_history, screenshot_captures), SMS (sms_delivery_log, sms_notification_pages, notification_delivery_log), personas/onboarding (personas, use_cases, onboarding_state), and misc (screen_avatars, type_metadata)
- Database: Remove ~9,000 lines of dead v1 query functions, schema definitions, and exports across packages/db — gut drizzle/queries.ts, drizzle/feature-flags.ts, queries.ts, client.ts, and index.ts barrel exports
- Database: Delete 4 orphaned security test files (1,513 lines) referencing dropped v1 tables
- Workers: Stub worker-sms, daily-digest cron, twilio webhook, and monthly-usage-report as deprecated no-ops
- Admin: Gut 6 admin action files referencing dropped v1 tables (audience, avatars, user URLs, changes, content, onboarding)
- Scripts: Add seed pipeline operations scripts —
improve-titles.tsfor batch AI title rewriting,generate-curated-entries.tsfor AI-generated notable entries across 33 topic categories,bulk-enqueue.tsfor Upstash queue dispatch
February 16, 2026
Features
- Fact Engine: Add seed fact-challenge pipeline — file parsing infrastructure (XLSX/DOCX/CSV parsers, 22 content profiles, static category mapper with 40+ path patterns), DB tables (seed_entry_queue with priority-based consumption, super_fact_links for cross-entry correlations, challenge_title on fact_records), AI explosion engine (explodeCategoryEntry for 10-100 facts per entity with theatrical titles, findSuperFacts for cross-entry correlations), worker handlers (EXPLODE_CATEGORY_ENTRY and FIND_SUPER_FACTS consumers), dynamic validation strategy selection in IMPORT_FACTS, spin-off depth control with recursive CTE, and CLI orchestrator with --parse/--explode/--explode-spinoffs/--super-facts/--stats/--all commands
- Database: Add migrations 0115-0116 — seed_entry_queue table with priority-ordered consumption and idempotent upserts, challenge_title column on fact_records, super_fact_links junction table with CASCADE delete
- Queue: Add EXPLODE_CATEGORY_ENTRY and FIND_SUPER_FACTS queue message types with routing, constructors, and type exports
- AI: Add seed_explosion and super_fact_discovery task tiers to model routing system
- Marketing: Add Challenge Tone Document (
docs/marketing/CHALLENGE_TONE.md) — canonical voice spec for all AI-generated challenge prompts with 6 format examples, scoring feedback voice, dispute voice, and anti-patterns - Database: Drop 15 legacy tables with no active code references — migrations 0111-0113 remove page-tracking tables (url_renders, video_renders, section_content_snapshots, page_diff_settings, user_folders, user_folder_pages, page_read_states, saved_items), brand auxiliary tables (brand_category_proposals, brand_review_queue, brand_sources, seeding_pages), and misc tables (correlation_groups, correlation_group_members, app_content)
- Database: Clean up Drizzle schema and query functions for dropped tables — remove 16 table definitions, 11 relation blocks, 13 query functions, and 5 orphaned enums from packages/db
- Database: Replace binary
isCorrectwith continuousscore: 0.0–1.0scoring oncard_interactions— migration 0100 adds score column, backfills from is_correct, drops is_correct - Database: Add ~30 new root topic categories with seed schemas — migration 0101 adds accounting, animals, architecture, art, auto, business, cooking, design, and 21 more categories with generic fact schemas
- Database: Add
free_textcard format enum value — migration 0102 for free-text challenge support - Database: Add
score_disputestable for AI-judged score disputes — migration 0103 with decision types, RLS policies, and user indexes - Database: Add
reward_milestonesanduser_reward_claimstables — migration 0104 with 4 seeded milestones (100/500/1000/2000 points) granting free Eko+ days - AI: Add text answer moderation function for free-text challenges — checks relevance, harm, and prompt injection
- AI: Add text answer scoring function for free-text challenges — returns 0.0–1.0 score with reasoning
- AI: Add dispute evaluation function — AI judge reviews user arguments and decides upheld/partial/full reversal
- API: Add
/api/cards/[slug]/disputePOST endpoint for score disputes with 10/month rate limit - API: Add
/api/user/rewardsGET/POST endpoints for reward milestone status and claiming - API: Update
/api/cards/[slug]/interactwith free-text AI scoring pipeline, moderation gate, and 50/day rate limit - UI: Add
ChallengeRevealskeleton reveal component with fade-in animation for challenge tabs - UI: Add
TextInputChallengefree-text challenge component with AI-graded scoring - UI: Add
DisputeDialogfor score dispute submission and AI judgment display - UI: Add
RewardsProgressmilestone progress bar with claim buttons - UI: Update
ScoringFeedbackwith tri-state display (correct/partial/incorrect) and dispute button - UI: Add Challenge tab to card detail page alongside Learn, Quiz, and Recall
- UI: Wrap Quiz, Recall, and Challenge tabs in skeleton reveal component
- Shared: Add XSS sanitization utilities for fact record JSONB values — iterative HTML tag stripping, event handler removal, and dangerous URI scheme filtering with recursive deep-sanitize for nested objects/arrays
- Testing: Add comprehensive test coverage for v2 fact engine pipeline — 10 test files with ~80 new tests across 6 packages (sanitization, AI extraction/validation/scoring/evergreen, DB queries, worker handlers for ingest/cluster/resolve-image/extract-facts/generate-evergreen/validate-fact)
- AI: Add challenge tone prefix to fact extraction and evergreen generation prompts — canonical Eko voice from
CHALLENGE_TONE.mdinjected viaCHALLENGE_TONE_PREFIXconstant - Fact Engine: Add 30-day expiry policy for news-derived facts — news-extracted facts get
expires_at = now + 30d, high-engagement facts (50+ interactions) auto-promote to enduring in archive-content cron - Database: Add
promoteHighEngagementFactsquery function andexpiresAtparameter toinsertFactRecord - Shared: Add spaced repetition scheduling module (
calculateNextReview) with SM-2 variant algorithm — interval schedule [4h, 1d, 3d, 7d, 14d, 30d] based on streak - API: Wire spaced repetition into
/api/cards/[slug]/interact— answered interactions now computenextReviewAt,streak, andreviewCountusing SM-2 scheduling - API: Add
/api/reviewGET endpoint for fetching facts due for spaced repetition review - API: Upgrade
/api/feedwith blended feed algorithm — 40% recent validated, 30% review-due, 20% evergreen, 10% exploration with round-robin interleave and per-carduserStatusfield - Database: Add feed blending queries —
getEvergreenFacts,getRandomFacts,getUserFactStatuses,getLatestAnswerInteraction - UI: Add color-coded interaction indicator badges to fact cards — yellow (in progress), orange (review due), green (mastered)
- UI: Polish challenge card layout — feature image with hover zoom, teaser prompt from first fact, difficulty badge, fact count, structured footer with date
- UI: Polish card detail page — hero image, verification date badge from validation metadata, back-to-feed link, integrated RewardsProgress component
- UI: Update card detail loading skeleton to match polished layout with image placeholder and 4-tab skeleton
- Fact Engine: Introduce Challenge Format taxonomy with 8 named formats (Big Fan Of, Know A Lot About, Repeat After Me, Good With Dates, Degrees of Separation, Used To Work There, Partial Pictures, Originators)
- Database: Add
challenge_styleandchallenge_format_slugenums,challenge_formatstable,challenge_format_stylesandchallenge_format_topicsjunction tables — migrations 0105-0106 - Database: Extend
card_interactionswithchallenge_format_idandchallenge_stylecolumns — migration 0107 - Database: Seed 8 challenge formats with style mappings and topic eligibility — migration 0108
- Database: Add
challenge_sessionstable for conversational multi-turn AI challenges — migration 0109 - Database: Add RLS policies for challenge format tables and challenge sessions — migration 0110
- AI: Add conversational challenge turn handler with format-aware prompting for Big Fan Of, Know A Lot About, and Degrees of Separation formats
- API: Add challenge format discovery endpoints (
GET /api/challenge-formats,GET /api/challenge-formats/[slug],GET /api/challenge-formats/for-topic/[topicSlug]) - API: Add conversational challenge session endpoints (
POST /api/challenge-sessions,POST /api/challenge-sessions/[sessionId]/turn,POST /api/challenge-sessions/[sessionId]/end) - UI: Add Challenge Hub discovery page at
/challengeswith format cards and topic selection - UI: Add ConversationalChallenge chat interface for multi-turn AI challenges with turn tracking and scoring
- Subscribe: Switch to 14-day trial with Stripe Checkout and CC collection
- Public Site: Add home page with live feed and card modal CTA
- Public Site: Add pricing page with plan comparison and annual toggle
- Public Site: Add features page with challenge system descriptions
- Public Site: Add about page with mission and fact engine overview
- Public Site: Add site layout with nav, footer, and v2 metadata
- Public Site: Add feed API rewrite proxy to avoid CORS
- Feed: Add subscription card injection for logged-out users
- Database: Add
getPlanTrialDaysquery for dynamic trial duration fromplan_definitions - Stripe: Add
payment_method_collection: 'always'to checkout sessions for trial CC collection
Database
- Migration 0114: Update trial_days from 30 to 14 for Eko+ plan
Bug Fixes
- Fact Engine: Fix
getSchemaForTopicreceiving slug strings instead of UUIDs in 5 call sites across seed CLI and worker handlers — schema lookup silently returned null, causing all entries to skip - Fact Engine: Add
topicCategoryIdtogetSeedEntriesForBatchquery so super-fact discovery can resolve schemas correctly - AI: Replace
z.record()with dynamicz.object()in all AI structured output schemas — OpenAI rejectspropertyNamesin JSON Schema, so fact schemas are now built from topicschemaKeysat runtime via newschema-utils.tsshared utility - AI: Remove
.min(0).max(1)from allz.number()Zod schemas — Anthropic rejectsminimum/maximumin JSON Schema structured output; range enforcement moved to system prompts - AI: Switch default model tier to
gpt-4o-minifor cost-efficient bulk seeding ($71 vs $1,752 for full corpus), keep mid/high tiers on Anthropic Sonnet for accuracy-critical tasks - AI: Move
seed_explosionandsuper_fact_discoverytasks frommidtodefaulttier — structured extraction doesn't require Sonnet-class models - Database: Add migration 0117 extending
fact_records.source_typeCHECK constraint to includefile_seedandai_super_factvalues for seed pipeline - Shared: Remove agents barrel export from
@eko/sharedto fix Turbopack client bundling (node:fsnot supported in browser chunks) - Challenges: Add
force-dynamicto/challengespage to prevent static prerender failure without running backend
February 15, 2026
Features
- Reel Schemas: Add
packages/reel-schemas/shared Zod schema package for Reel video generator — VideoBrief, VideoScript, VisualAsset, AudioAsset, RenderJobPayload, RenderJobResult, PipelineJob, BrandPreset, and RenderOutput schemas with r2Key and runpodJobId cloud fields - R2 Client: Add
packages/r2/Cloudflare R2 client library with S3-compatible upload/download, presigned URL generation (1h default expiry), and declarative lifecycle rules for transient assets (72h) and rendered outputs (30d) - Reel Render Worker: Add
apps/worker-reel-render/RunPod serverless render worker — Bun HTTP server handling RunPod protocol (POST /run, /runsync), FFmpeg subprocess wrapper, Remotion render orchestration with quality presets (draft/standard/premium), R2 asset download/upload, and Dockerfile (oven/bun:1-debian + FFmpeg + Playwright Chromium)
Documentation
-
Reel: Add phase-gated build requirements doc (
docs/specs/reel-requirements.md) covering accounts, tooling, assets, open decisions, and risk validation for all 6 implementation phases -
Reel: Add Character Studio with integrated fal.ai pipeline to spec and requirements — Flux 2 Pro anchor generation, LoRA identity training, Kontext image editing, new character wizard, roster/detail UI, two new SQLite tables (
characters,character_images), updated Phase 2 deliverables -
Docs: Consolidate
video-generation/cast system into Reel spec companion files — merge 17 source files into 6 companion files indocs/specs/reel/, add Section 16 and Appendix D to spec, delete orphaned root-level directory -
Reel Spec: Update
docs/specs/reel-video-generator-spec.mdwith cloud render architecture — replace FFmpeg/Playwright sidecars with RunPod + R2, add r2Key fields to VisualAsset/AudioAsset/RenderVariant, add runpodJobId to PipelineJob, update Stage 5 to RunPod dispatch, add RunPod/R2 to External APIs, update Settings/CLI/directory structure, add new Section 15 (Cloud Infrastructure) -
Archive: Move ~300 v1 documents to
docs/docs_archive/organized into 15 subdirectories (specs-phases, specs-v1, product, admin-app, web-app, runbooks-v1, assessments, plans-v1, proposals, mvp-readiness, reports, dev-v1, projects, existing-archives, misc) with path-mapping fallback document, archive READMEs, and updated references across agents, hooks, scripts, and marketing docs
February 14, 2026
Features
- Skills: Add 14 new skills for v2 fact engine — create-worker, add-queue-message-type, add-cron-route, build-news-api-client, create-fact-extraction-prompt, configure-validation-tier, build-feed-page, build-card-detail, configure-subscription-tier, remove-legacy-routes, drop-legacy-tables, triage-ingestion-failure, audit-fact-quality, monitor-card-engagement
- Skills: Update create-migration skill with v2 standardized header format (MIGRATION NNNN pattern) and migrations:index regeneration step
- Skills: Update deploy skill with 3 new Fly.io worker targets (worker-ingest, worker-facts, worker-validate)
- Agents: Add 3 new agents for v2 fact engine pipeline — ingest-engineer (news ingestion), fact-engineer (AI extraction), validation-engineer (multi-tier verification)
- Agents: Reorient 6 existing agents for v2 architecture — architect-steward (v2 invariants), card-ux-designer (renamed from ux-page-card-designer), subscription-manager (Free/Eko+), cron-scheduler (ingestion crons), admin-operator (fact engine admin), queue-sre (7 new queue types)
- Agents: Deprecate 5 legacy pipeline agents — tracker-engineer, render-engineer, diff-analyst, summarization-safety, use-case-note-writer
- Agents: Restructure agent catalog README with v2 data flow diagram, updated dependency levels, and new file pattern routing table
February 13, 2026
Features
- Cleanup: Remove all legacy page-tracking infrastructure — 7 authenticated UI routes, 9 API route trees, worker-tracker and worker-render apps, 4 AI functions, 5 queue message types, 11 config env vars (~15,900 lines deleted)
- Auth: Update all post-login redirects from
/dashboardto/feed - Cleanup: Remove legacy server actions (favorites, folders, saved-items)
- UI: Add card detail pages with subscription gating, paywall overlay, and loading skeleton at
/card/[slug] - UI: Add multiple choice trivia quiz interface with auto-scoring and interaction recording
- UI: Add open recall practice interface with self-grading and answer reveal
- UI: Add subscribe page with Free vs Eko+ plan comparison and 30-day free trial CTA
- UI: Add post-checkout success and cancellation pages at
/subscribe/successand/subscribe/canceled - API: Add
/api/cards/[slug]GET endpoint for authenticated card detail fetching with subscription check - API: Add
/api/cards/[slug]/interactPOST endpoint for recording card interactions (views, answers, bookmarks) - Database: Add
createTrialSubscriptionquery for no-credit-card trial subscription creation - UI: Add public feed page with responsive card grid, category filter chips, infinite scroll, and
/api/feedpagination endpoint - CRON: Add 7 fact engine cron routes — ingest-news (*/15 min), cluster-sweep (hourly), import-facts (daily stub), generate-evergreen (daily), validation-retry (4h), archive-content (daily), topic-quotas (daily audit)
- Database: Add 6 cron query functions — getActiveTopicCategories, getActiveTopicCategoriesWithSchemas, getUnclusteredNewsSources, getStuckPendingValidations, archiveExpiredFactRecords, getTopicFactCountToday
- Worker: Create worker-validate app consuming VALIDATE_FACT queue with tiered validation strategies (authoritative_api, multi_source, curated_database, ai_cross_check)
- Database: Add updateFactRecordStatus query for fact validation status transitions with validation JSON storage
- Worker: Create worker-facts app consuming EXTRACT_FACTS, IMPORT_FACTS, and GENERATE_EVERGREEN queues for AI fact extraction, bulk API import, and evergreen knowledge generation
- Worker: Create worker-ingest app consuming INGEST_NEWS, CLUSTER_STORIES, and RESOLVE_IMAGE queues with concurrent consumers, TF-IDF clustering, and image resolution cascade
- Database: Add worker-facts queries — getStoryWithSources, getSchemaForTopic, getTopicCategoryById, getTopicCategoryBySlug, insertFactRecord, getExistingTitlesForTopic, getFactRecordSchemaById
- Database: Add ingestion pipeline queries — insertNewsSources, getNewsSourcesByIds, createStory, updateStory, linkNewsSourceToStory, findStoriesByTitleSimilarity, getFactRecordById, updateFactRecordImage
- Database: Add fact engine schema — topic_categories, fact_record_schemas, fact_records, stories, news_sources, card_interactions, card_bookmarks, ingestion_runs tables (migrations 0092-0096)
- Database: Add 7 new enums for fact engine (news_provider, story_status, fact_record_status, card_format, trivia_difficulty, validation_strategy, interaction_type)
- Database: Update plan_definitions for Free/Eko+ model (can_access_card_details, trial_days, max_bookmarks)
- Database: Seed initial topic taxonomy (7 root categories, 12 sub-categories) and 5 fact record schemas
- Database: Add Drizzle ORM schema definitions for all fact engine tables with relations
- Database: Add fact engine Drizzle query functions — feed retrieval, card interactions, bookmarks, spaced repetition review, and ingestion pipeline observability
- Shared: Add 7 fact engine queue message Zod schemas (INGEST_NEWS, CLUSTER_STORIES, EXTRACT_FACTS, IMPORT_FACTS, VALIDATE_FACT, GENERATE_EVERGREEN, RESOLVE_IMAGE) and domain types
- Queue: Add 7 fact engine queue names, constructors, and type-safe message routing
- Config: Add fact engine env vars (news API keys, extraction settings, notability threshold) and Eko+ Stripe pricing config
- AI: Add fact engine AI functions — fact extraction, notability scoring, fact validation, and evergreen generation with tier-based model selection
- Stripe: Add Free/Eko+ pricing model with billing period support and 30-day trial integration
February 11, 2026
Features
- Dev Tooling: Enforce frontmatter on markdown files at creation time via Claude Code hook and pre-commit check
- Marketing: Add blog copy documents for blog home, single post, categories, and archive pages
- CRON: Eliminate N+1 query in enqueue-due-urls (17,001 → 2 queries per run) with lateral-join optimization and early-exit check for redundant hourly triggers
Bug Fixes
- AI: Fix Haiku 4.5 token consumption during worker processing — deploy
DEFAULT_TIER_CONFIGfix (all tiers →gpt-4o-mini) and repair cross-provider fallback to construct alternate-provider models directly instead of re-resolving tiers that map to the same provider - Queue: Fix stale processing messages never being recovered — messages stuck in
:processinglist now auto-reclaim during idle polling - Worker Tracker: Fix heartbeat counter always reporting 0 messages processed — closure variable was never incremented from consumer loop
- Recovery Script: Fix broken Drizzle query in catch-up action — update deprecated
urls/urlObservationsreferences topages/pageObservations
Features
- Queue: Add
recoverStaleProcessing()function andflush-processingrecovery action for manual and automatic orphaned message recovery
February 10, 2026
Features
- Marketing: Add copy documents for terms, privacy, FAQ, contact, about, and pricing pages
Documentation
- Deployment Guide: Update Supabase auth config to reflect actual state (Site URL was
localhost:3000, noteko.day; redirect URLs were empty), add Stripe webhook URL migration step, add Google Cloud Console verification, renumber steps and verification checklist, mark guide as locked
February 9, 2026
Features
- Public Site: Integrate Sentry error monitoring for
apps/public— configure@sentry/nextjswith DSN, global error boundary, client/server/edge instrumentation, and example test page - Public Site: Add
.gitignoreforapps/publicmatching sibling app conventions - Config: Document
SENTRY_AUTH_TOKENin.env.examplefor source map uploads
February 8, 2026
Features
- Architecture: Split public marketing pages into
apps/publicfor subdomain deployment (eko.dayfor public site,app.eko.dayfor authenticated app) - Config: Add
getPublicUrl()for cross-domain links between public site and app; updategetAppUrl()default tohttps://app.eko.day
Database
- Security: Add security fixes, RLS InitPlan correction (split across two migrations for tables a-p and p-z), and database index cleanup (migrations 0083-0086)
Bug Fixes
- CI: Fix docs-lint SIGPIPE false failures — replace
echo | grep -qwith here-strings to avoid broken pipe errors underpipefailwhen frontmatter content is large
Features
- AI Routing: Add per-provider DB kill switches (feature flags
ai_provider_openai,ai_provider_anthropic,ai_provider_google) for instant AI provider toggling without restarts (migration 0089) - AI Model Tiers: Add DB-driven model tier system — change AI models via SQL UPDATE with no restart needed (migration 0090):
- Three-tier routing: default (gpt-4o-mini), mid (claude-haiku-4-5), high (claude-opus-4-6)
feature_flags.ai_model_tiersJSONB config with 60s in-memory cache- Model registry with deprecation detection and pricing metadata for all OpenAI, Anthropic, and Google models
- Google/Gemini config slots (env vars, valid models) — SDK integration pending
- Centralized
createLanguageModel()helper eliminates duplicated provider instantiation across 6 AI consumer files cost-trackeruses model registry for pricing lookup and provider detection
- AI Model Tiers v2: Replace monolithic JSONB feature flag with dedicated
ai_model_tier_configtable andai_modelPostgres enum (migration 0091):- DB-level model validation via enum — invalid model names rejected at the Postgres layer
- Per-tier row updates instead of rewriting entire JSONB blob
- Provider auto-derived from
MODEL_REGISTRYin code — single source of truth - 16 models registered in enum across OpenAI, Anthropic, and Google
February 7, 2026
Features
- UI Reset (Phase 0): Controlled demolition of UI layer for clean rebuild from approved designs:
- Strip
packages/uito 13 bare primitives (Button, Input, Select, Textarea, Checkbox, Radio, Switch, Dialog, Tooltip, Tabs, Badge, Separator, Toast) +cn()utility - Move design tokens CSS from
packages/ui/styles/topackages/shared/styles/ - Create
packages/ui-publicempty placeholder for future marketing components - Stub all 50 web pages and 28 admin pages to plain text labels (no UI, no Tailwind utilities)
- Simplify layouts to pass-through wrappers (auth redirect preserved)
- Archive storybook to
apps/_archive/storybook-v1/ - Archive design docs to
docs/_archive/design-v1/ - Delete ~300 non-primitive components, ~155 story files, ~50 orphaned client components
- Strip
Database
- Performance: Add indexes on
page_change_events(page_id)andpage_change_events(page_observation_id)for unindexed foreign keys (migration 0087)
Bug Fixes
- Design System: Restore icon size and motion tokens stripped during UI redesign that are still required by challenge tests
- Design System: Rebuild motion.css with prefers-reduced-motion support, Tailwind/Radix animation overrides, and motion-safe/reduce utilities
- Worker: Fix async test assertions for
shouldEscalateToRenderafter function became async - Worker: Add meaningful change gating to prevent false-positive change events — skip
page_change_eventcreation and AI summary generation when content hash changes but section analysis shows 0% change with no affected sections (filters ~1,000 wasted AI calls and DB writes from timestamp/session token noise) - Admin: Fix stale table references in admin dashboard stats — update raw Supabase queries from
urls/url_observations/url_change_eventstopages/page_observations/page_change_eventsafter migration 0077 renamed tables - Worker: Fix
getPreviousFullSectionsto use vNextpage_change_eventstable andpage_change_event_idFK instead of dropped V1url_changestable — enables inline diff before/after comparison for section snapshots - Worker: Change
V1_WRITE_ENABLEDdefault fromtruetofalse— V1 tables were dropped in migration 0060 but the flag still defaulted to attempting writes, causing silent failures when processing tracking messages - Storage: Fix thumbnail upload 400 errors — add
image/webpto renders bucketallowed_mime_types(migration 0088). Bucket was created with PNG-only allowlist but thumbnail generation produces WebP files for optimal file size - Infra: Add missing
OPENAI_API_KEYto tracker worker — all AI summary calls were falling back from GPT-4o-mini to Claude Haiku via provider fallback, resulting in ~5-7x higher AI costs - Worker: Fix
TypeError: Cannot destructure property 'firstElementChild'crash in content normalizer — wrapdocument.bodyaccess in try-catch to handle linkedom throwing on malformed HTML (e.g., CAPTCHA pages, empty responses) - Infra: Set
AI_PROVIDER=openaion tracker worker — env var was set toanthropic, overriding model router and forcing all AI calls through Haiku regardless ofOPENAI_API_KEY - AI: Fix OpenAI structured output schema validation — add
additionalProperties: falseto nestedinline_diffsitems object and useanyOffor nullablesection_headingfield
Documentation
- Architecture: Add AI routing pipeline Excalidraw diagram documenting CRON-to-DB pipeline flow, model routing decision tree, and three-layer failure analysis
February 6, 2026
Features
- Design System: Strip UI to studs — remove all visual opinions, ready for clean rebuild:
- Gut tokens.css to bare semantic tokens (fonts, gray palette, status colors, shadcn slots, radius, sidebar, overlay)
- Strip theme.css to minimal Tailwind v4 color/radius mappings
- Empty components.css (1,392 lines of data-slot CSS removed)
- Empty motion.css (589 lines of animation utilities removed)
- Strip spacing.css to @theme inline scale only (remove semantic utilities)
- Simplify globals.css to token/theme/tailwind imports + minimal base layer
- Comment out decorative exports in index.ts (~100 lines: marketing components, animated icons, premium blocks)
- Stub 8 marketing pages (home, about, features, pricing, use-cases, blog, faq, changelog) with minimal HTML + Tailwind
- Remove 25 storybook stories for stripped components
- Preserve: all shadcn primitives, 21 sacred components, app shell, auth forms, sidebar, accessibility CSS
- AI Cost Protection: Prevent silent AI provider fallback — upgrade fallback log from
debugtowarn, addprovider_fallbackflag to AI result types, emitai.provider_fallbackmetric, add startup warning when OpenAI key missing, send throttled email alert to admins when fallback is active - AI Cost Governance: Add $5/day Anthropic spend cap with graceful degradation to GPT-4o-mini:
- New
ai_cost_trackingtable (migration 0079) for daily cost aggregation by provider, model, and feature - Cost-tracker module with in-memory budget cache (30s TTL), pricing table for Opus 4.6/Haiku/GPT-4o-mini
- Budget check integrated into all 5 existing AI functions (
generateSummary,generateSummaryWithDiffs,generateTrackingNote,generatePageDescription,classifyAudience) - Gap 1 fix: Provider fallback now tracks actual model cost, not intended model
- Gap 2 fix: Error recovery in
generateSummaryWithDiffsuses heuristic fallback instead of doubling AI calls - Gap 7 fix:
classifyAudiencebudget check with heuristic fallback - Add
claude-opus-4-6to valid Anthropic models
- New
- AI Cost Reporting: Add daily AI cost report cron (06:00 UTC) emailing admins with per-model and per-feature breakdown, budget utilization, and 7-day trend
- AI Model Router: Add Claude Opus 4.6 as third model tier for top-1% complex cases:
- Opus escalation rules: pricing page + structure change, API page + structure change + >20% change, legal/terms + structure change + >30% change, any page + >60% change + >8 sections + structure change
- Feature-flagged (
OPUS_ESCALATION_ENABLED) with daily call cap (OPUS_MAX_DAILY_CALLS, default: 20) - In-memory daily call counter with midnight reset, fail-safe defaults to GPT-4o-mini
- Claude Agents: Add
memory: projectto 6 agents for persistent cross-session knowledge (architect-steward, db-migration-operator, diff-analyst, security-reviewer, release-manager, ci-quality-gatekeeper) - Claude Agents: Add
disallowedToolsto all 18 agents enforcing delegation graph — agents delegate downstream only, never upstream. architect-steward has no restrictions as top-level coordinator - Claude Hooks: Add 3 new hook scripts and project-level
.claude/settings.json:auto-migrations-index.sh(PostToolUse) — auto-regenerates migrations index when migration files are writtenchangelog-check.sh(SubagentStop) — reminds to update changelog when code files changesubagent-quality.sh(SubagentStop) — runs typecheck after subagent completes code work- Register all 9 hooks (6 existing + 3 new) in
.claude/settings.jsonfor team sharing
- AI Thin-HTML Detection: Add semantic page complexity classification using Claude Haiku:
- New
classifyPageComplexity()function classifies JS-heavy SPAs that need Playwright rendering - Domain-level caching (migration 0080) — classifications cached per domain with configurable TTL
- 3-tier cascade in
shouldEscalateToRender(): domain cache → AI classification → heuristic fallback - Feature-flagged (
SEMANTIC_THIN_HTML_ENABLED) with daily call cap (THIN_HTML_MAX_DAILY_CLASSIFICATIONS, default: 50) - Budget-aware: respects Anthropic daily spend cap, falls back to heuristic when budget exhausted
- New
- Agent Teams: Add experimental agent teams with 3 coordination playbook runbooks:
- Pipeline Team (tracker → diff → summary → UI): coordinated multi-stage pipeline changes
- Infrastructure Team (queue, observability, cron, CI): operational backbone changes
- Schema Team (migrations, RLS, types): database and data model changes
- Enable
CLAUDE_CODE_EXPERIMENTAL_AGENT_TEAMS=1in project settings
- Multi-URL Change Correlation: Detect coordinated changes across pages within a domain:
- New
correlation_groupsandcorrelation_group_memberstables (migration 0081) - New
CORRELATE_CHANGESqueue message type with dedicated queue routing - Opus-powered
generateCorrelationSummary()analyzes related changes (e.g., "pricing + changelog + API docs = major release") - Worker-tracker fire-and-forget detection: after change event, checks if 3+ pages from same domain changed within configurable window
- Feature-flagged (
CHANGE_CORRELATION_ENABLED) with daily cap (CORRELATION_MAX_DAILY, default: 30) - Budget-aware: respects Anthropic daily spend cap
- New
- Premium Deep Analysis: Add Opus-powered deep analysis summaries for premium plan users:
- New
can_access_deep_analysiscolumn onplan_definitions(migration 0082), enabled for pro and team plans - New
is_deep_analysisflag onpage_change_summaries— two-tier summary architecture (standard + deep) generateDeepAnalysis()Opus function provides strategic implications, competitive context, and prioritized action items- Worker-tracker fire-and-forget: after standard summary, checks for premium user tracking the page before generating
- Query-time gating:
hasAnyPremiumUserTrackingPage()joins user_url_library → user_subscriptions → plan_definitions - Feature-flagged (
DEEP_ANALYSIS_ENABLED) with min change threshold and daily cap (DEEP_ANALYSIS_MAX_DAILY, default: 50) - Budget-aware: respects Anthropic daily spend cap
- New
February 5, 2026
Features
- Database: URL→Page schema rename (migrations 0076-0078):
- Migration 0076: Rename core tables (
urls→pages,user_url_library→user_pages), 3 enums, FK columns on 10+ tables,user_can_see_page_event()function, RLS policies - Migration 0077: Rename 8 dependent tables (
url_observations→page_observations,url_change_events→page_change_events, etc.), ~25 RLS policies, 4 stored functions,folder_unread_countsview - Migration 0078: Drop
url_statsview, updatesaved_itemsitem_type'url'→'page', update triggers
- Migration 0076: Rename core tables (
- Database: Update Drizzle ORM schema and queries for URL→Page rename — all table definitions, column accessors, relations, and 50+ query functions updated with deprecated aliases for backward compatibility
- Database: Update Supabase client queries for URL→Page rename — all
.from()table name strings and column references updated - Documentation: Complete URL→Page terminology rename across ~100 documentation files in
docs/reports/features/,docs/specs/, anddocs/projects/building-eko-pages/
Bug Fixes
- Database: Add UNIQUE constraint for
section_dom_mappingsupsert (migration 0075). Migration 0072 created a regular index on(url_observation_id, section_id)butupsertSectionDomMappingVNext()requires a UNIQUE constraint forON CONFLICTclause. Drop regular index and create partial unique index to enable vNext DOM mapping upserts
Features
- Database: Add comprehensive schema remediation for orphaned V1 tables (migrations 0072-0074):
- Migration 0072: Add vNext columns (
url_id,url_observation_id,url_change_event_id) to 6 orphaned tables (sms_notification_urls,sms_delivery_log,section_content_snapshots,url_diff_settings,section_dom_mappings,page_title_history) with FK constraints to vNext tables - Migration 0073: Backfill vNext columns from
v1_backupschema using canonical_url matching - Migration 0074: Add referential integrity triggers for polymorphic
saved_itemstable (validation on insert/update, cascade delete fromurlsanduse_cases)
- Migration 0072: Add vNext columns (
- Worker: Fix RENDER_URL database records - Add missing
createScreenshotCapturecall in render worker so screenshots uploaded to storage also get database records (resolves Issue #1 from schema audit) - Worker: Re-enable DOM mapping storage - Add
upsertSectionDomMappingVNextfunction using vNext IDs and re-enablestoreDomMappingin capture-screenshots.ts (resolves Issue #7 from schema audit) - Tooling: Update render smoke test (
e2e-render-smoke.ts) from V1 tables to vNext (urls,screenshot_captures)
Bug Fixes
- Database: Align
url_change_summariesschema with V1summaries(migration 0071). Add missing columns from migrations 0012 (next_steps) and 0043 (admin_override_what_changed,admin_override_why_it_matters,override_at,override_by) that were added to V1 but missed vNext table, resolving Drizzle schema drift - Scripts: Add
backfill-summaries.tsscript to regenerate AI summaries for orphaned change events (change events that exist without corresponding summaries due to API failures) - Database: Update Drizzle schema with vNext columns and relations for all migrated tables
February 4, 2026
Bug Fixes
- Database: Fix broken favorites system - Migrate 11 favorites query functions from dropped V1 tables (
favorite_folders,user_favorites) to unified vNext tables (user_folderswithfolder_type='favorite',user_folder_urlswithuser_library_id). Update actions and API routes to useuser_library_idinstead oftracked_url_id - Web App: Fix broken URL detail pages - Migrate
/url/[id]and/dashboard/urls/[id]pages from dropped V1 tables (tracked_urls,url_checks,url_changes,summaries) to vNext tables (urls,user_url_library,url_observations,url_change_events). Update API route/api/urls/[id]for vNext CRUD operations
Features
- AI Cost Optimization: Implement hybrid optimization strategy reducing AI costs by ~91% ($712 → ~$65/month):
- Add configurable CRON batch size (
CRON_BATCH_SIZEenv var) for URL throughput control - Add trivial change filter - skip AI for changes below 3% threshold
- Add section-level filtering - only include sections with >5% change in AI prompts
- Add model router - routes 95% of calls to GPT-4o-mini, escalates complex cases to Claude Haiku 4.5
- Enhance HTML normalization - strip dynamic attributes, cookie banners, social widgets, and normalize timestamps to reduce false positives
- Add configurable CRON batch size (
- Tooling: Add migrations index system - Auto-generated index of all Supabase migrations with phase categorization,
bun run migrations:indexgenerator script, and CI validation viabun run migrations:check
Bug Fixes
- Worker: Fix V1 code path errors after Phase 6 table removal. Replace
getTrackedUrlByIdcalls withdrizzleQueries.getGlobalUrlById, update property mappings (snake_case → camelCase), remove V1 notification code (requires vNext user_url_library lookup), and clean up unused imports. Notifications temporarily disabled pending vNext implementation - Database: Expand
updateUrlEkoNotemethod signature to accept 'search' generation method alongside existing types - Scripts: Data alignment Phase 10 - Add
populate-domains-from-urls.tsscript to extract unique canonical_domain values from urls table and create corresponding domains entries with auto-linking to brands via database trigger - Scripts: Data alignment Phase 12 - Add
assign-brand-categories.tsscript to assign categories to uncategorized brands using PDL industry mapping, legacy category_path fallback, and consumer-services default - Scripts: Data alignment Phase 13 - Add
backfill-audience.tsscript to backfill audience values (work/personal/hybrid) for URLs, domains, and brands using page_type classification and B2B/B2C category heuristics - Database: Add brand category query functions to Drizzle queries:
getBrandCategories,getBrandPrimaryCategory,getBrandsInCategory,assignBrandCategory,removeBrandCategory,getL1CategoriesWithCounts - Database: Migration 0067 - Update
domains.sourceCHECK constraint to usesystem_seededinstead ofbrand_seeded, aligning with Drizzle schema - Database: Migration 0068 - Fix
urls.brand_idFK to referencedomains.idinstead ofbrands.id, establishing correct data hierarchy (brands → domains → urls) - Database: Migration 0069 - Normalize
urls.canonical_domainby strippingwww.prefix to match domains table normalization
Data Alignment Results
- Phase 10 Complete: 2,891 domains created from URLs, 17,273 URLs linked to domains (100%)
- Phase 11 Complete: 2,596 orphan domains linked to auto-created brands (100% domain→brand linkage)
- Phase 12 Complete: 2,933 brands assigned categories (100% coverage) - Fixed pagination issue in assign-brand-categories.ts
- Phase 13 Complete: 504 URL audience values updated based on page_type classification
February 3, 2026
Features
- Database: Phase 3 link domains to brands - Add
brand_idFK column todomainstable, linking domain registry to curated brand library. Auto-backfill existing domains. Add trigger for automatic brand linking on domain insert/update. Add Drizzle queries for domain-brand management (linkDomainToBrand,findBrandForDomain,autoLinkOrCreateBrandForDomain,approveDomainWithBrandLink) - Admin: Phase 4 domain verification UI - Add admin pages for domain review (
/domains/pending,/domains/rejected). Admins can approve, reject (with reason), or re-approve domains. Domain verification stats shown on dashboard - Queue: V1 → vNext migration Phase 6 - Update all queue message schemas to vNext-only.
RenderUrlMessage,CaptureScreenshotsMessage, andSendSmsMessagenow useurl_id,url_observation_id, andurl_change_event_idinstead of V1 fields (tracked_url_id,url_check_id,url_change_id) - Worker: V1 → vNext migration Phase 6 - Update tracker, render, and SMS workers to produce and consume vNext-only queue messages
- Database: V1 → vNext migration Phase 6 - Remove V1 tables and code paths. Create V1 table backups (
v1_backupschema), drop FK constraints from deprecated tables, remove V1 columns from dual-path tables (screenshot_captures,notification_delivery_log). Update Drizzle schema and queries to use vNext only. Workers and web app routes migrated to vNext-only reads. Add DROP TABLE migration (0060) for final V1 table removal after stability verification - Scripts: Add domain expansion tooling for Volume 2 URL seeding - CLI argument support for
validate-domains.tsandgenerate-url-patterns.ts, domain extraction script for batch processing - Reference Data: Add expanded domain list with ~1,900 domains across 100 sectors (50 B2B Work + 50 B2C Personal audiences) for system URL seeding
- Scripts: Add pipeline diagnostic script (
diagnose-pipeline.ts) for investigating URL processing stalls - checks queue health, database state, and provides actionable recommendations - Database: Add brand categorization system with 3-level taxonomy (Industry > Sector > Niche). New tables:
brand_categories,brand_category_assignments,brand_category_proposals. Includes 18 L1 industries and 54 L2 sectors - Scripts: Add brand category migration tooling (
pdl-category-mapper.ts,migrate-brand-categories.ts,seed-brand-categories.ts) for PDL-first classification with AI fallback support - Worker: V1 → vNext migration Phase 5.2.1 - SMS worker vNext support. Add
url_change_event_idto SMS queue message schema, pass vNext FK from tracker worker, migratelogSmsDeliveryto write to unifiednotification_delivery_logtable - Database: V1 → vNext migration Phase 5.2.2 - Add vNext daily digest queries (
getPendingDigestNotificationsVNext,batchCreateNotificationDeliveriesVNext,batchMarkNotificationsSentVNext) that read fromurl_change_eventsand write tonotification_delivery_log - Web App: V1 → vNext migration Phase 5.2.2 - Daily digest cron now supports
VNEXT_READ_ENABLEDflag for full vNext read/write path - Database: V1 → vNext migration Phase 5.3 - Add vNext FK columns (
url_id,url_observation_id,url_change_event_id) toscreenshot_capturestable, maketracked_url_idnullable - Queue: V1 → vNext migration Phase 5.3 - Update
CaptureScreenshotsMessageSchemawith optional V1 IDs and new vNext ID fields for screenshot capture queue messages - Worker: V1 → vNext migration Phase 5.3 - Tracker worker now fully supports
V1_WRITE_ENABLED=false. V1 writes conditional, vNext observations created first, notifications use vNext IDs when V1 disabled - Worker: V1 → vNext migration Phase 5.3 - Render worker supports both V1 and vNext IDs for screenshot capture, uses fallback logic for storage paths
- Shared: Add
www.stripping to URL canonicalization —canonicalizeUrl()now normalizeswww.example.com→example.comfor consistent URL deduplication - Scripts: Add
seed-urls-from-markdown.tsfor seeding URLs from validated markdown reference documents with page type inference from URL patterns - Database: Phase 1 rename
brand_sites→domains- Rename table, enum, FK columns, indexes, RLS policies, and triggers. Update Drizzle schema and queries. Rename UI routes/brands/→/domains/and API routes/api/brands/→/api/domains/. Add deprecated function aliases for backward compatibility - Database: Phase 2 domain verification workflow - Add
domain_verification_statusenum (pending/approved/rejected), verification columns to domains table (verification_status,submitted_by,verified_by,verified_at,rejection_reason,rejected_by,rejected_at). RLS policies enforce visibility: approved domains visible to all, pending visible only to submitter. Add Drizzle query functions for admin review workflow - Web App: Phase 5 URL validation integration - Add domain verification checks to URL add flow (
POST /api/urls) and validation endpoint (POST /api/urls/validate). URLs from rejected domains are blocked with user-friendly error messages. New domains from user submissions start aspendingand require admin approval. AddgetRejectedDomainInfoquery for rejection details - Database: Phase 5 domain rejection cascade - Add
rejectDomainWithCascadefunction that deactivates all user subscriptions to URLs from rejected domains, returning tracking credits. AddcreateDomainRejectionNotificationsto notify affected users via in-app notifications - Admin: Phase 5 rejection UI enhancement - Admin domain rejection now triggers full cascade (subscription deactivation + user notifications). Toast shows count of affected users
- Database: Data alignment Phase 9.1 - Rename
folder_urls→user_folder_urlstable with updated indexes, constraints, RLS policies, and database function references (migration 0064) - Database: Data alignment Phase 9.2 - Rename
brand_urls→seeding_urlstable to clarify its role as curated seeding catalog, with updated indexes, constraints, triggers, and RLS policies (migration 0065) - Database: Data alignment Phase 9.3 - Add
audiencefield to vNexturlstable with index, backfill from linked domains, and helper functionget_url_effective_audience()(migration 0066) - Database: Update Drizzle schema exports:
folderUrls→userFolderUrls,brandUrls→seedingUrls, addaudiencefield tourlstable - Scripts: Add
enrich-urls-from-seeding-urls.tsfor backfilling URLs with page_type, brand_id, audience, and canonical_domain from seeding catalog - Scripts: Add
audit-data-alignment.tscomprehensive audit framework for ongoing data quality monitoring across urls, brands, domains, and seeding_urls tables - Scripts: Update all seeding scripts to use new
seeding_urlstable name (seed-urls-vnext.ts,audit-seeded-urls.ts,seed-brand-urls.ts,seed-brand-library-pdl.ts,batch-db.ts) - Database: Update
queries.tsto useuser_folder_urlstable name
Documentation
- Planning: Add Phase 6 V1 cleanup plan with comprehensive artifact inventory (~25 files, 12-17 hours estimated)
- Config: Document
VNEXT_READ_ENABLEDflag in .env.example with migration sequence
Bug Fixes
- Web App: Fix uncaught promise rejection in enqueue-due-urls cron route — add
awaitto returned promises in try block to ensure error handling catches async rejections
February 2, 2026
Features
- Database: V1 → vNext migration Phase 1 - Fix Drizzle schema drift (
observedAt→checkedAtinurl_observations), add vNext query adapters (getDueUrlsVNext,getTrackedUrlsVNext, etc.) - Database: V1 → vNext migration Phase 2 - Migrate web app cron job to support vNext reads with
VNEXT_READ_ENABLEDfeature flag - Database: V1 → vNext migration Phase 4 - Migrate admin dashboard stats from V1 tables (
url_checks,url_changes) to vNext tables (url_observations,url_change_events) - Queue: Add optional
url_idfield to TrackUrlMessage for vNext worker support - Database: Consolidate brand tables - rename
brand_library_*to simpler names (brands,brand_urls,brand_sources,brand_review_queue) - Database: Consolidate folder tables - merge
favorite_folders+user_favoritesinto unifieduser_folderswith folder_type discriminator - Database: Consolidate metadata tables - merge
page_type_metadata+brand_type_metadatainto unifiedtype_metadatawith category discriminator - Database: Consolidate notification delivery tables - merge
notification_deliveries+sms_delivery_loginto unifiednotification_delivery_logwith channel discriminator - Database: Add migration to align
url_sourceenum with Drizzle schema (brand_seeded→system_seeded) and seed 1,372 pattern URLs - Database: Add
single-producttopage_typeenum and fixbrand_url_typenaming convention mismatch - Scripts: Add pagination to URL seeding and audit scripts to handle large datasets
- Scripts: Add URL seeding migration scripts (
audit-seeded-urls.ts,seed-urls-vnext.ts) to enforce pattern-only seeding - Shared: Add
isRootDomain()andassertNotRootDomain()URL validation utilities - Invariants: Add "No TLD seeding" invariant to CLAUDE.md - system-seeded URLs must be patterns, never root domains
- Database: Add use_case_audience enum and columns for work/personal/hybrid content filtering
- Shared: Add UseCaseAudience schema, types, and getEffectiveAudience helper
- AI Package: Add audience classification module with AI and heuristic fallback
- Web App: Add AudienceTabs component for filtering Discover content by work/personal context
- Web App: Add audience tabs to Discover Use Cases page with persona count display
- Admin Dashboard: Add audience management page for brands, page types, personas, and use cases
- Shared: Add brand type validation schema with soft enum pattern for Discover feature categories
- Scripts: Add PDL domain enrichment script for business summaries, HQ location, and company metadata
- Scripts: Add PDL domain discovery script for work audience seeding with URL pattern generation
- Worker: Phase 2e - Make vNext the PRIMARY write path in tracking worker. vNext writes (
url_observations,url_change_events,url_change_summaries) are now unconditional. V1 writes are guarded byV1_WRITE_ENABLEDflag (default: true) for downstream compatibility - Database: Phase 3a - Add
content_hashandrender_time_mscolumns toscreenshot_capturestable (migrated fromurl_renders) - Worker: Phase 3a - Migrate render worker from
url_renderstoscreenshot_capturestable for visual capture consolidation - Worker: V1 → vNext migration Phase 3 - Tracker worker supports dual message format (
tracked_url_id+url_id). Whenurl_idpresent, uses vNext queries (hasCheckedTodayVNext,getLatestObservationForUrl,updateNextCheckTimeVNext) and skips canonical_url resolution - Database: V1 → vNext migration Phase 5 - Add
url_change_event_idFK tonotification_delivery_logtable (migration 0055) for vNext notification linking - Database: Update
createNotificationDeliveryto write to unifiednotification_delivery_logtable with vNext FK support - Web App: V1 → vNext migration Phase 2.1 - Migrate URL API endpoints (
/api/urls,/api/urls/validate) to read from vNext tables (urls,user_url_library) whenVNEXT_READ_ENABLED=true - Web App: V1 → vNext migration Phase 2.2 - Migrate dashboard pages to read from vNext tables with new query helpers (
getRecentChangesWithSummariesVNext,getUrlStatsVNext,getChangeEventByIdV1Compat) - Web App: V1 → vNext migration Phase 2.1 (complete) - Migrate brands API and dashboard pages with
getTrackedUrlsForBrandVNextquery helper
Documentation
- Product Docs: Comprehensive refresh of page-type-definitions.md with database schema section, icon mappings, and updated metadata
February 1, 2026
Features
- Docs Lint: Add
active,locked,archivedas valid status values for plan governance - Scripts: Add frontmatter maintenance script with audit, auto-fix, and interactive modes
- Scripts: Expand persona system from 6 to 34 personas (18 work, 13 personal, 3 hybrid)
- Scripts: Add shared persona definitions module at
scripts/lib/personas.ts - Scripts: Update
generate-use-cases.tswith new CLI options (--persona-type,--list-personas) - Scripts: Update
seed-use-cases-migration.tsto use shared persona definitions
Bug Fixes
- Scripts: Fix UI registry export updating last_updated date on every run regardless of content changes
- UI Components: Fix 17 lint warnings (a11y semantic elements, array index keys, explicit any)
CI/CD
- CI: Add frontmatter validation to CI pipeline
Generated on 2026-02-17