feat: M2 (part 2) — RSS poller, backfill, enrichment, scheduler

- Free per-channel RSS reader for quota-less fresh-video detection
- Recent-first backfill (configurable: 100 videos / 1 year) plus resumable deep
  backfill from the uploads playlist
- Enrichment via videos.list: duration, view/like counts, category, topics,
  language, Shorts heuristic and livestream/premiere classification
- Reusable sync runners + APScheduler jobs (rss / enrich / backfill), all
  quota-aware with a reserve so backfill never starves fresh enrichment
- Manual triggers: POST /api/sync/{rss,backfill,enrich}
- Exact insert counting via RETURNING with in-batch de-duplication
This commit is contained in:
npeter83 2026-06-11 01:36:41 +02:00
parent 24b6e0026d
commit cff1d23071
10 changed files with 569 additions and 4 deletions

View file

@ -9,8 +9,12 @@ Each user signs in with their own Google account (invite-only) and sees only the
subscriptions. All the expensive data (channels, videos, metadata) is fetched from YouTube
once and stored locally, so filtering/searching/sorting are instant and don't burn API quota.
> Status: early development. Milestone **M1** (foundation) is in place: docker-compose stack,
> FastAPI backend, Google OAuth login with an email invite-list, and encrypted token storage.
> Status: early development.
> - **M1** (foundation): docker-compose stack, FastAPI backend, Google OAuth login with an
> email invite-list, encrypted token storage.
> - **M2** (ingest core): subscription import, free RSS detection, recent-first + deep backfill
> from the uploads playlist, enrichment (duration/stats/category, Shorts & livestream
> classification), a shared daily quota guard, and a background scheduler.
## Requirements