siftlode/backend/requirements.txt

fastapi>=0.115,<1.0
uvicorn[standard]>=0.30,<1.0
sqlalchemy>=2.0,<2.1
psycopg[binary]>=3.2,<4.0
alembic>=1.13,<2.0
pydantic>=2.7,<3.0
pydantic-settings>=2.3,<3.0
authlib>=1.3,<2.0
httpx>=0.27,<1.0
feedparser>=6.0,<7.0
apscheduler>=3.10,<4.0
py3langid>=0.3,<1.0
itsdangerous>=2.1,<3.0
cryptography>=46.0.7,<47
argon2-cffi>=23.1,<26
# Download center: yt-dlp does the extraction/download; ffmpeg (OS-level in the Dockerfile)
# does the merge + postprocessing. The [default] extra pulls yt-dlp-ejs (the challenge-solver
# scripts Deno runs for YouTube's n-signature). yt-dlp is release-often (YouTube changes), so
# keep the floor loose and bump it when extraction breaks.
yt-dlp[default]>=2025.5.22
# PO Token provider plugin: talks to the bgutil sidecar (brainicism/bgutil-ytdlp-pot-provider)
# to mint YouTube Proof-of-Origin tokens on demand, bypassing "confirm you're not a bot" and
# unlocking high-quality formats. Pin to match the sidecar image tag (versions must align).
bgutil-ytdlp-pot-provider==1.3.1
feat: M1 foundation — compose stack, FastAPI, Google OAuth, encrypted tokens - docker-compose with Postgres 16 + slim Python API image - FastAPI app with session middleware, health endpoint, static login page - Google OAuth (Authlib) with email invite-list whitelist; admin role support - User + OAuthToken models; refresh tokens encrypted at rest (Fernet) - Alembic migrations, run automatically on container startup - Postgres backup/restore scripts for portability between machines 2026-06-11 01:01:37 +02:00			`fastapi>=0.115,<1.0`
			`uvicorn[standard]>=0.30,<1.0`
			`sqlalchemy>=2.0,<2.1`
			`psycopg[binary]>=3.2,<4.0`
			`alembic>=1.13,<2.0`
			`pydantic>=2.7,<3.0`
			`pydantic-settings>=2.3,<3.0`
			`authlib>=1.3,<2.0`
			`httpx>=0.27,<1.0`
feat: M2 (part 2) — RSS poller, backfill, enrichment, scheduler - Free per-channel RSS reader for quota-less fresh-video detection - Recent-first backfill (configurable: 100 videos / 1 year) plus resumable deep backfill from the uploads playlist - Enrichment via videos.list: duration, view/like counts, category, topics, language, Shorts heuristic and livestream/premiere classification - Reusable sync runners + APScheduler jobs (rss / enrich / backfill), all quota-aware with a reserve so backfill never starves fresh enrichment - Manual triggers: POST /api/sync/{rss,backfill,enrich} - Exact insert counting via RETURNING with in-batch de-duplication 2026-06-11 01:36:41 +02:00			`feedparser>=6.0,<7.0`
			`apscheduler>=3.10,<4.0`
feat: M3 — automatic channel tagging (language + topic system tags) - Tag and ChannelTag models + migration 0003 (partial unique indexes split system vs per-user tag names) - Offline language detection (py3langid) constrained to a curated language set, with the channel's declared default language as a strong prior - Topic tags mapped from YouTube topicDetails + dominant video category; the generic "Lifestyle" catch-all is intentionally dropped - System (auto) tags are regenerated idempotently and never touch user tags; orphaned system tags are cleaned up - GET /api/tags and admin POST /api/tags/recompute; scheduled autotag job 2026-06-11 01:57:19 +02:00			`py3langid>=0.3,<1.0`
feat: M1 foundation — compose stack, FastAPI, Google OAuth, encrypted tokens - docker-compose with Postgres 16 + slim Python API image - FastAPI app with session middleware, health endpoint, static login page - Google OAuth (Authlib) with email invite-list whitelist; admin role support - User + OAuthToken models; refresh tokens encrypted at rest (Fernet) - Alembic migrations, run automatically on container startup - Postgres backup/restore scripts for portability between machines 2026-06-11 01:01:37 +02:00			`itsdangerous>=2.1,<3.0`
fix(security): patch cryptography CVEs, upgrade pip at build, harden /auth/upgrade - requirements: cryptography >=46.0.7 (was pinned <46, which excluded the fix for the CVEs pip-audit flagged in our Fernet/crypto library). pip-audit now clean. - Dockerfile: upgrade pip before installing deps (patches installer-level CVEs). - auth: /auth/upgrade now defaults to the least-privileged read scope; only an explicit access=write requests the write scope. 2026-06-14 05:59:34 +02:00			`cryptography>=46.0.7,<47`
feat(auth): email+password registration with two-gate activation (5a backend) Add email+password auth alongside Google: argon2id hashing; users gains password_hash/email_verified/is_active and google_sub becomes nullable (migration 0022); a single-use, hashed auth_tokens table for email verification + password reset. Registration creates a pending (inactive, unverified) account + an access request; sign-in needs verified email AND admin approval. Anti-enumeration: uniform register/login/reset responses (a correct-password owner still gets a specific pending reason). allow_registration flag (DB-tunable). Admin users-list + role endpoint (guards: not self, not demo, not the last admin); approving access (or a manual whitelist add) now activates the matching pending account. current_user rejects deactivated accounts. Verified end-to-end via curl + DB. 2026-06-19 14:03:11 +02:00			`argon2-cffi>=23.1,<26`
feat(downloads): Deno JS runtime + web/android clients to complete the POT setup The web player clients need two things beyond the PO token: (1) a JavaScript runtime for YouTube's 'n' signature challenge — install Deno (v2.9.1) + yt-dlp[default] (bundles yt-dlp-ejs solver scripts); auto-detected on PATH. (2) an audio source — the web clients only expose video-only DASH, so add the 'android' client (also PO-token-backed) for audio/muxed formats. Player clients: web_safari,web,android (dropped tv — it falsely reports normal videos as DRM). End-to-end verified in the worker: extraction (POT + Deno n-challenge) + download completes, no bot/DRM/format errors. Full YouTube stack now: force-ipv4 + bgutil POT sidecar + Deno. 2026-07-03 22:46:32 +02:00			`# Download center: yt-dlp does the extraction/download; ffmpeg (OS-level in the Dockerfile)`
			`# does the merge + postprocessing. The [default] extra pulls yt-dlp-ejs (the challenge-solver`
			`# scripts Deno runs for YouTube's n-signature). yt-dlp is release-often (YouTube changes), so`
			`# keep the floor loose and bump it when extraction breaks.`
			`yt-dlp[default]>=2025.5.22`
feat(downloads): automatic PO-token provider (bgutil sidecar) to beat YouTube bot-detection YouTube increasingly demands a Proof-of-Origin token (or shows 'confirm you're not a bot'). Instead of manual cookies, run the bgutil POT provider as a sidecar that mints tokens on demand: - compose: bgutil-pot service (brainicism/bgutil-ytdlp-pot-provider:1.3.1, port 4416, init) - requirements: bgutil-ytdlp-pot-provider==1.3.1 plugin (version pinned to the image) - config: DOWNLOAD_POT_BASE_URL (http://bgutil-pot:4416) + DOWNLOAD_PLAYER_CLIENTS (web_safari,web,tv — POT-capable; the default android_vr client never requests a token) - formats.build_ydl_opts wires extractor_args youtube:player_client + youtubepot-bgutilhttp:base_url Verified: the sidecar mints a valid PO token (BotGuard challenge solved) and yt-dlp fetches it from the provider. NOTE: end-to-end download couldn't be confirmed here because the test IP got hard-flagged by YouTube from heavy testing (~16 GB + dozens of extractions) — a hard abuse block needs cooldown/cookies, which POT doesn't lift; POT bypasses the normal soft bot-check. 2026-07-03 04:40:10 +02:00			`# PO Token provider plugin: talks to the bgutil sidecar (brainicism/bgutil-ytdlp-pot-provider)`
			`# to mint YouTube Proof-of-Origin tokens on demand, bypassing "confirm you're not a bot" and`
			`# unlocking high-quality formats. Pin to match the sidecar image tag (versions must align).`
			`bgutil-ytdlp-pot-provider==1.3.1`