Commit graph

4 commits

Author SHA1 Message Date
npeter83
75d122de75 fix(backfill): auto-complete full history when stored >= video_count
A channel whose stored uploads already meet or exceed YouTube's advertised
video_count holds its whole history, but backfill_done could stay false forever
when the deep cursor never reached the end (e.g. a small channel that was never
deep-requested, so the demand-driven deep job never ran). Such channels nagged
as 'needs full history' despite having every video. Add reconcile_full_history()
(idempotent, no quota) and run it at the end of each recent/deep backfill cycle
so backfill_done self-heals.
2026-06-14 19:01:04 +02:00
npeter83
18aa3726fb fix(backfill): don't skip the tail of the recent/deep boundary page
When recent backfill stopped partway through a page (age cutoff or count cap),
it stored the *next* page token as the deep-backfill cursor, so the older items
remaining on that same page were collected by neither pass — a channel could
report backfill_done while silently missing a band of videos right around the
365-day cutoff. Now resume deep from the page we stopped on (re-fetched; inserts
are idempotent). Verified: Pánczél went 200 -> 209 stored (= full uploads
playlist) after re-backfill.
2026-06-12 00:15:20 +02:00
npeter83
8c245e986f fix: address reader-UI feedback (shorts, search, tags, login, UX)
- Shorts: confirm via youtube.com/shorts/<id> probe (SOCS cookie bypasses the
  consent redirect) instead of a 60s heuristic; concurrent probing, shorts_probed
  flag, scheduled refinement (migration 0005)
- Search: match title + channel name only (descriptions caused noisy results)
- Faceted tag filtering: AND across categories (language AND topic narrows),
  OR within a category; any/all toggle applies to topics
- Language detection: majority vote over individual titles (fixes misdetections
  like multipoleguy -> English; drops bogus Polish/Romanian)
- Login: drop forced consent so returning sign-in is quick (select_account)
- Feed cards: clickable channel name (opens channel), persistent saved badge,
  undo toast on hide, Hidden view to restore; tag chips show counts in tooltip
2026-06-11 03:07:49 +02:00
npeter83
cff1d23071 feat: M2 (part 2) — RSS poller, backfill, enrichment, scheduler
- Free per-channel RSS reader for quota-less fresh-video detection
- Recent-first backfill (configurable: 100 videos / 1 year) plus resumable deep
  backfill from the uploads playlist
- Enrichment via videos.list: duration, view/like counts, category, topics,
  language, Shorts heuristic and livestream/premiere classification
- Reusable sync runners + APScheduler jobs (rss / enrich / backfill), all
  quota-aware with a reserve so backfill never starves fresh enrichment
- Manual triggers: POST /api/sync/{rss,backfill,enrich}
- Exact insert counting via RETURNING with in-batch de-duplication
2026-06-11 01:36:41 +02:00