Commit graph

5 commits

Author SHA1 Message Date
npeter83
b394540701 fix(sync): re-enrich live videos so ended streams gain duration
enrich_pending only touches enriched_at IS NULL, so a video first seen
while live was stamped live + duration-null and never revisited — staying
'live' with no duration forever after the broadcast ended. Add refresh_live
(run after each enrich pass) that re-fetches anything still live/upcoming,
plus just-ended was_live videos that haven't got their duration yet, until
they settle. Cheap: videos.list is 1 unit per 50 ids.
2026-06-16 10:06:58 +02:00
npeter83
64c6e44686 fix(backfill): auto-complete full history when stored >= video_count
A channel whose stored uploads already meet or exceed YouTube's advertised
video_count holds its whole history, but backfill_done could stay false forever
when the deep cursor never reached the end (e.g. a small channel that was never
deep-requested, so the demand-driven deep job never ran). Such channels nagged
as 'needs full history' despite having every video. Add reconcile_full_history()
(idempotent, no quota) and run it at the end of each recent/deep backfill cycle
so backfill_done self-heals.
2026-06-14 19:01:04 +02:00
npeter83
a5fc1e6d0b fix(backfill): don't skip the tail of the recent/deep boundary page
When recent backfill stopped partway through a page (age cutoff or count cap),
it stored the *next* page token as the deep-backfill cursor, so the older items
remaining on that same page were collected by neither pass — a channel could
report backfill_done while silently missing a band of videos right around the
365-day cutoff. Now resume deep from the page we stopped on (re-fetched; inserts
are idempotent). Verified: Pánczél went 200 -> 209 stored (= full uploads
playlist) after re-backfill.
2026-06-12 00:15:20 +02:00
npeter83
8c245e986f fix: address reader-UI feedback (shorts, search, tags, login, UX)
- Shorts: confirm via youtube.com/shorts/<id> probe (SOCS cookie bypasses the
  consent redirect) instead of a 60s heuristic; concurrent probing, shorts_probed
  flag, scheduled refinement (migration 0005)
- Search: match title + channel name only (descriptions caused noisy results)
- Faceted tag filtering: AND across categories (language AND topic narrows),
  OR within a category; any/all toggle applies to topics
- Language detection: majority vote over individual titles (fixes misdetections
  like multipoleguy -> English; drops bogus Polish/Romanian)
- Login: drop forced consent so returning sign-in is quick (select_account)
- Feed cards: clickable channel name (opens channel), persistent saved badge,
  undo toast on hide, Hidden view to restore; tag chips show counts in tooltip
2026-06-11 03:07:49 +02:00
npeter83
cff1d23071 feat: M2 (part 2) — RSS poller, backfill, enrichment, scheduler
- Free per-channel RSS reader for quota-less fresh-video detection
- Recent-first backfill (configurable: 100 videos / 1 year) plus resumable deep
  backfill from the uploads playlist
- Enrichment via videos.list: duration, view/like counts, category, topics,
  language, Shorts heuristic and livestream/premiere classification
- Reusable sync runners + APScheduler jobs (rss / enrich / backfill), all
  quota-aware with a reserve so backfill never starves fresh enrichment
- Manual triggers: POST /api/sync/{rss,backfill,enrich}
- Exact insert counting via RETURNING with in-batch de-duplication
2026-06-11 01:36:41 +02:00