siftlode/backend/app
npeter83 03a1865aa8 feat(downloads): automatic PO-token provider (bgutil sidecar) to beat YouTube bot-detection
YouTube increasingly demands a Proof-of-Origin token (or shows 'confirm you're not a bot').
Instead of manual cookies, run the bgutil POT provider as a sidecar that mints tokens on demand:
- compose: bgutil-pot service (brainicism/bgutil-ytdlp-pot-provider:1.3.1, port 4416, init)
- requirements: bgutil-ytdlp-pot-provider==1.3.1 plugin (version pinned to the image)
- config: DOWNLOAD_POT_BASE_URL (http://bgutil-pot:4416) + DOWNLOAD_PLAYER_CLIENTS
  (web_safari,web,tv — POT-capable; the default android_vr client never requests a token)
- formats.build_ydl_opts wires extractor_args youtube:player_client + youtubepot-bgutilhttp:base_url

Verified: the sidecar mints a valid PO token (BotGuard challenge solved) and yt-dlp fetches it
from the provider. NOTE: end-to-end download couldn't be confirmed here because the test IP got
hard-flagged by YouTube from heavy testing (~16 GB + dozens of extractions) — a hard abuse block
needs cooldown/cookies, which POT doesn't lift; POT bypasses the normal soft bot-check.
2026-07-03 04:40:10 +02:00
..
downloads feat(downloads): automatic PO-token provider (bgutil sidecar) to beat YouTube bot-detection 2026-07-03 04:40:10 +02:00
routes fix(downloads): force IPv4 (fixes container 'Errno -5' DNS failures) + retry resets errored asset 2026-07-03 03:47:13 +02:00
static chore: rebrand Subfeed -> Siftlode 2026-06-14 04:40:22 +02:00
sync fix(titles): normalize titles at stub insert too (RSS/backfill/live-search) 2026-07-03 03:19:00 +02:00
youtube chore: prepare repo for public release — scrub internal infra, rewrite README 2026-07-01 12:46:50 +02:00
__init__.py feat: M1 foundation — compose stack, FastAPI, Google OAuth, encrypted tokens 2026-06-11 01:01:37 +02:00
auth.py fix(downloads): file download honours the per-tab account (?account=) 2026-07-03 02:08:10 +02:00
config.py feat(downloads): automatic PO-token provider (bgutil sidecar) to beat YouTube bot-detection 2026-07-03 04:40:10 +02:00
db.py feat: M1 foundation — compose stack, FastAPI, Google OAuth, encrypted tokens 2026-06-11 01:01:37 +02:00
email.py chore: rename remaining subfeed references to siftlode 2026-06-21 06:53:12 +02:00
main.py feat(downloads): M4 — REST API (enqueue, manage, file-serve, sharing, admin) 2026-07-03 00:34:08 +02:00
models.py feat(titles): normalize video titles for display (feed, search, downloads) 2026-07-03 03:00:08 +02:00
notifications.py feat(notifications): durable per-user inbox (P1) + maintenance schema 2026-06-18 03:20:17 +02:00
progress.py feat(scheduler): admin run-now/run-all triggers + live progress + completion notices 2026-06-18 04:01:10 +02:00
quota.py feat(channels): channel-explore backend — about metadata, ephemeral provenance, cleanup 2026-06-30 02:52:44 +02:00
ratelimit.py feat(demo): demo-account schema + reusable rate limiter 2026-06-16 09:17:14 +02:00
realtime.py feat(messages): end-to-end encrypted real-time direct messaging backend 2026-06-25 22:05:35 +02:00
scheduler.py feat(downloads): M3 — per-user quota + retention GC 2026-07-03 00:26:40 +02:00
security.py refactor(auth): centralize token hashing, email validation & admin gating 2026-06-26 03:15:36 +02:00
state.py refactor(auth): centralize token hashing, email validation & admin gating 2026-06-26 03:15:36 +02:00
sysconfig.py feat(downloads): M3 — per-user quota + retention GC 2026-07-03 00:26:40 +02:00
titles.py feat(titles): normalize video titles for display (feed, search, downloads) 2026-07-03 03:00:08 +02:00
utils.py refactor(auth): centralize token hashing, email validation & admin gating 2026-06-26 03:15:36 +02:00
worker.py feat(downloads): automatic PO-token provider (bgutil sidecar) to beat YouTube bot-detection 2026-07-03 04:40:10 +02:00