Commit graph

9 commits

Author SHA1 Message Date
npeter83
03a1865aa8 feat(downloads): automatic PO-token provider (bgutil sidecar) to beat YouTube bot-detection
YouTube increasingly demands a Proof-of-Origin token (or shows 'confirm you're not a bot').
Instead of manual cookies, run the bgutil POT provider as a sidecar that mints tokens on demand:
- compose: bgutil-pot service (brainicism/bgutil-ytdlp-pot-provider:1.3.1, port 4416, init)
- requirements: bgutil-ytdlp-pot-provider==1.3.1 plugin (version pinned to the image)
- config: DOWNLOAD_POT_BASE_URL (http://bgutil-pot:4416) + DOWNLOAD_PLAYER_CLIENTS
  (web_safari,web,tv — POT-capable; the default android_vr client never requests a token)
- formats.build_ydl_opts wires extractor_args youtube:player_client + youtubepot-bgutilhttp:base_url

Verified: the sidecar mints a valid PO token (BotGuard challenge solved) and yt-dlp fetches it
from the provider. NOTE: end-to-end download couldn't be confirmed here because the test IP got
hard-flagged by YouTube from heavy testing (~16 GB + dozens of extractions) — a hard abuse block
needs cooldown/cookies, which POT doesn't lift; POT bypasses the normal soft bot-check.
2026-07-03 04:40:10 +02:00
npeter83
317750e491 feat(downloads): M1 — schema, config, worker container, deps
- models: MediaAsset (shared file cache, unique on source+format_sig), DownloadJob
  (per-user request), DownloadProfile (presets), DownloadShare (ACL), DownloadQuota
- migrations 0036 (5 tables) + 0037 (6 builtin presets)
- sysconfig 'downloads' group (per-user defaults, retention/GC, layout, worker concurrency)
- config: DOWNLOAD_ROOT / WORKER_ENABLED env + download defaults
- deps: yt-dlp in requirements, ffmpeg in Dockerfile runtime stage
- worker: dedicated container (python -m app.worker), thin idle entry (loop lands in M2)
- localdev compose: worker service + downloads bind-mount; gitignore downloads/
2026-07-02 23:57:40 +02:00
npeter83
0d44d3a34a chore: prepare repo for public release — scrub internal infra, rewrite README
- Remove owner-specific / legacy deploy files (home/prod/server compose, deploy/).
  The home compose stays as a local untracked file for the maintainer's own deploy.
- Genericise infra-specific code comments (egress-proxy examples) to neutral wording.
- Replace the hardcoded contact email on the legal pages with the instance operator's
  configured admin email, served via the public /auth/config and shown with a neutral
  fallback — so each self-hosted instance shows its own contact.
- Rewrite README for the current app + a copy-paste self-hosting quick start (prebuilt
  image + first-run wizard) with a build-from-source alternative; tidy .env.example.
2026-07-01 12:46:50 +02:00
npeter83
1ec89c8fbe chore: rename remaining subfeed references to siftlode
Replace the leftover 'subfeed' name across logger names + log_config,
frontend localStorage keys, Postgres user/db/volume defaults in the
compose files, .env.example, config.py, backup/restore scripts and the
README. Pure rename; no behavioural change. localStorage keys move from
subfeed.* to siftlode.* (one-time UI-state reset is acceptable).
2026-06-21 06:53:12 +02:00
npeter83
b71d07c098 chore(dev): self-contained local dev stack (own Postgres + scheduler)
localdev no longer points at the shared database; it now runs its own
local Postgres and its own scheduler, fully decoupled. Removes the shared-DB
migration coupling (no more redeploys after migrations) and lets the
background scheduler actually run during local dev.
2026-06-16 15:21:43 +02:00
npeter83
a93ab30fb2 feat(version): /api/version + build-time version/commit stamping
Add a VERSION file (0.1.0) and inject APP_VERSION/GIT_SHA/BUILD_DATE as Docker
build-args (both stages; Vite inlines them into the SPA). New public GET
/api/version returns app_version, git_sha, build_date and the Alembic head as the
database revision. deploy.sh and the localdev build pass the args.
2026-06-15 00:06:57 +02:00
npeter83
167ebc74d8 chore(repo): remove internal deployment plan, sanitize infra references
Drop docs/deployment-plan.md (internal planning doc, kept out of the repo) and
replace the personal lab DB IP and host names in the env/compose/deploy docs with
generic placeholders, so the repo is safe to make public later.
2026-06-14 23:20:02 +02:00
npeter83
65c75d8f05 fix(deploy): apparmor:unconfined for nested Docker
The docker-default AppArmor profile can't be loaded from inside the container; match the
lab convention (finance/arr stacks) of running services unconfined.
2026-06-11 16:42:44 +02:00
npeter83
fd1ac29704 feat(deploy): shared database + scheduler split for the server 24/7 backfill
Add a server compose (full stack, Postgres on the LAN, scheduler on, host-visible
pgdata bind mount) and a localdev compose (webapp only, no DB, scheduler off) that
points at the shared database. Document the shared-database topology, the
exactly-one-scheduler rule, and YOUTUBE_API_KEY for unattended backfill that does
not depend on a 7-day OAuth refresh token.
2026-06-11 16:35:00 +02:00