feat(search): zero-quota InnerTube scrape source + search_source toggle

Live YouTube search can now use YouTube's internal InnerTube endpoint instead
of search.list, materialising results through the same enrich/provenance path
at zero API quota (search.list costs 100 units/page; scrape costs nothing, only
the cheap shared videos.list enrich is charged).

- youtube/search_scrape.py: InnerTube search returning the same page shape as
  YouTubeClient.search_videos (items + continuation cursor); SOCS consent cookie,
  videoRenderer walk (channels/playlists/Shorts shelves naturally excluded),
  type:video filter, cached InnerTube key/version with constant fallbacks.
- routes/search.py: admin-selectable source (search_source, default scrape).
  Scrape path skips the 100-unit budget pre-check and logs a zero-cost search
  event so the per-user daily cap still counts it; api path unchanged. Response
  carries the active source.
- sysconfig/config: new search_source key (scrape|api).
- quota.log_action: record a zero-cost action event for per-user rate limits.
This commit is contained in:
npeter83 2026-06-29 22:29:54 +02:00
parent 17c17c1d3c
commit 396e09189b
5 changed files with 228 additions and 12 deletions

View file

@ -71,9 +71,13 @@ class Settings(BaseSettings):
youtube_api_proxy: str = ""
# Daily quota budget (units). Default leaves headroom under the 10,000/day free limit.
quota_daily_budget: int = 9000
# Per-user cap on live YouTube searches per day. search.list costs 100 units each, so the
# shared budget only affords ~80-90/day total — this stops one user draining it. Admin-tunable.
# Per-user cap on live YouTube searches per day. With the official API (search_source="api")
# each search.list costs 100 units, so the shared budget only affords ~80-90/day total; with
# the zero-quota scrape source it's a pure anti-abuse rate limit. Admin-tunable.
search_daily_limit_per_user: int = 70
# Source for live YouTube search: "scrape" (InnerTube, no API quota) or "api" (search.list,
# 100 units/page). Scrape is the default; flip to "api" if scraping is blocked/breaks.
search_source: str = "scrape"
# Recent-first backfill: how far back to fetch on the first pass per channel.
backfill_recent_max_videos: int = 100
backfill_recent_max_days: int = 365