The web player clients need two things beyond the PO token: (1) a JavaScript runtime for
YouTube's 'n' signature challenge — install Deno (v2.9.1) + yt-dlp[default] (bundles yt-dlp-ejs
solver scripts); auto-detected on PATH. (2) an audio source — the web clients only expose
video-only DASH, so add the 'android' client (also PO-token-backed) for audio/muxed formats.
Player clients: web_safari,web,android (dropped tv — it falsely reports normal videos as DRM).
End-to-end verified in the worker: extraction (POT + Deno n-challenge) + download completes,
no bot/DRM/format errors. Full YouTube stack now: force-ipv4 + bgutil POT sidecar + Deno.
YouTube increasingly demands a Proof-of-Origin token (or shows 'confirm you're not a bot').
Instead of manual cookies, run the bgutil POT provider as a sidecar that mints tokens on demand:
- compose: bgutil-pot service (brainicism/bgutil-ytdlp-pot-provider:1.3.1, port 4416, init)
- requirements: bgutil-ytdlp-pot-provider==1.3.1 plugin (version pinned to the image)
- config: DOWNLOAD_POT_BASE_URL (http://bgutil-pot:4416) + DOWNLOAD_PLAYER_CLIENTS
(web_safari,web,tv — POT-capable; the default android_vr client never requests a token)
- formats.build_ydl_opts wires extractor_args youtube:player_client + youtubepot-bgutilhttp:base_url
Verified: the sidecar mints a valid PO token (BotGuard challenge solved) and yt-dlp fetches it
from the provider. NOTE: end-to-end download couldn't be confirmed here because the test IP got
hard-flagged by YouTube from heavy testing (~16 GB + dozens of extractions) — a hard abuse block
needs cooldown/cookies, which POT doesn't lift; POT bypasses the normal soft bot-check.
Add email+password auth alongside Google: argon2id hashing; users gains
password_hash/email_verified/is_active and google_sub becomes nullable (migration
0022); a single-use, hashed auth_tokens table for email verification + password
reset. Registration creates a pending (inactive, unverified) account + an access
request; sign-in needs verified email AND admin approval. Anti-enumeration: uniform
register/login/reset responses (a correct-password owner still gets a specific
pending reason). allow_registration flag (DB-tunable). Admin users-list + role
endpoint (guards: not self, not demo, not the last admin); approving access (or a
manual whitelist add) now activates the matching pending account. current_user
rejects deactivated accounts. Verified end-to-end via curl + DB.
- requirements: cryptography >=46.0.7 (was pinned <46, which excluded the fix for
the CVEs pip-audit flagged in our Fernet/crypto library). pip-audit now clean.
- Dockerfile: upgrade pip before installing deps (patches installer-level CVEs).
- auth: /auth/upgrade now defaults to the least-privileged read scope; only an
explicit access=write requests the write scope.
- Tag and ChannelTag models + migration 0003 (partial unique indexes split
system vs per-user tag names)
- Offline language detection (py3langid) constrained to a curated language set,
with the channel's declared default language as a strong prior
- Topic tags mapped from YouTube topicDetails + dominant video category; the
generic "Lifestyle" catch-all is intentionally dropped
- System (auto) tags are regenerated idempotently and never touch user tags;
orphaned system tags are cleaned up
- GET /api/tags and admin POST /api/tags/recompute; scheduled autotag job
- Free per-channel RSS reader for quota-less fresh-video detection
- Recent-first backfill (configurable: 100 videos / 1 year) plus resumable deep
backfill from the uploads playlist
- Enrichment via videos.list: duration, view/like counts, category, topics,
language, Shorts heuristic and livestream/premiere classification
- Reusable sync runners + APScheduler jobs (rss / enrich / backfill), all
quota-aware with a reserve so backfill never starves fresh enrichment
- Manual triggers: POST /api/sync/{rss,backfill,enrich}
- Exact insert counting via RETURNING with in-batch de-duplication
- docker-compose with Postgres 16 + slim Python API image
- FastAPI app with session middleware, health endpoint, static login page
- Google OAuth (Authlib) with email invite-list whitelist; admin role support
- User + OAuthToken models; refresh tokens encrypted at rest (Fernet)
- Alembic migrations, run automatically on container startup
- Postgres backup/restore scripts for portability between machines
