2. Trigger News Ingestion

Purpose: Manually trigger the news ingestion pipeline to fetch fresh articles from configured news sources and enqueue them for processing.

Prerequisites:

  • Web app running locally (bun run dev:web)
  • Ingestion worker running (see Start Workers)
  • CRON_SECRET, NEWS_API_KEY, GOOGLE_NEWS_API_KEY, and THENEWS_API_KEY set in .env.local

Cost / Duration: Negligible AI cost | ~2 minutes

Prompt

Trigger a manual news ingestion run. This fetches articles from all configured
news APIs and enqueues INGEST_NEWS messages for the ingestion worker.

First, make sure the web app is running, then hit the ingestion cron endpoint:

```bash
curl -X POST http://localhost:3000/api/cron/ingest-news \
  -H "Authorization: Bearer $CRON_SECRET"
```

Alternative approach using the queue recovery script:
```bash
bun run scripts/queue-recovery.ts --action=trigger-cron
```

After triggering, run the pipeline diagnostic to confirm messages were enqueued:
```bash
bun scripts/diagnose-pipeline.ts
```

The INGEST_NEWS queue should show new messages. After worker processing,
new source articles will appear in the database.

If you get a 401 Unauthorized, check that CRON_SECRET in `.env.local` matches
the app config. If no articles are ingested, check that the news API keys
(NEWS_API_KEY, GOOGLE_NEWS_API_KEY, THENEWS_API_KEY) are valid and not rate-limited.

Verification

  • Cron endpoint returns success (HTTP 200)
  • INGEST_NEWS queue shows new messages in diagnostic output
  • New source articles appear in the database after worker processing
  • No errors in the ingestion worker terminal

Back to index