- B1 (sticky errored asset): get_or_create_asset now resets a reused status=='error'
asset back to 'pending' (+clears the error), so a fresh enqueue/edit of a once-failed
(source,format) pair actually re-downloads instead of the worker short-circuiting the
new job with the stale error. Errored assets carry no expires_at, so without this the
pair was permanently poisoned for all users until someone hit resume.
- B2 (ref_count leak): _release_asset counts 'error' as a holding state, so deleting an
errored job decrements the ref_count that enqueue always incremented (the worker never
decrements on failure). Errored rows are deliberately NOT deleted here — a concurrent
B1 reuse could otherwise be lost-updated + FK-nulled; the row is fileless and harmless.
- B3: admin storage dashboard reads sysconfig.get_int(db,'download_total_max_bytes')
(the admin-editable DB value GC enforces) instead of the raw env default.
- B4: the single-trim branch of normalize_edit_spec guards its float() coercion like the
crop/segments branches — a malformed trim now yields a 400, not an unhandled 500.
- B5: formats.normalize guards int(max_height) → falls back to "best" instead of 500.
Reviewed (race in an earlier B2 draft caught + fixed); localdev boots, B4/B5 unit-verified.
- C1: remove downloads/formats.py target_ext() — defined but never called (the
worker derives the real extension from the produced file's suffix).
- C2: the download-root containment+existence guard was copy-pasted 6× across the
file-serving endpoints (routes/downloads.py ×3, routes/public.py ×3). Extract
storage.safe_abs_path(root, rel) -> Path|None so this security-sensitive check
lives in one place; behavior identical (same containment test + messages). The
extraction also made `pathlib.Path` unused in both route modules (removed).
mp4 downloads used yt-dlp's default best video+audio, which on YouTube is
VP9+Opus at 1080p. Chromium plays that, but iOS/Safari WebKit (which every
iOS browser is forced to use) decodes only H.264/H.265+AAC inside mp4 — so a
shared /watch link showed a broken player on iPhone/iPad while the download
still worked. Rank vcodec:h264 + acodec:aac above resolution for mp4 output
(unless a custom profile pins a vcodec), keeping VP9/AV1 as a graceful
fallback. Bump the format signature (v2) so an identical spec gets a fresh
cache identity instead of hitting a stale VP9 asset.
YouTube increasingly demands a Proof-of-Origin token (or shows 'confirm you're not a bot').
Instead of manual cookies, run the bgutil POT provider as a sidecar that mints tokens on demand:
- compose: bgutil-pot service (brainicism/bgutil-ytdlp-pot-provider:1.3.1, port 4416, init)
- requirements: bgutil-ytdlp-pot-provider==1.3.1 plugin (version pinned to the image)
- config: DOWNLOAD_POT_BASE_URL (http://bgutil-pot:4416) + DOWNLOAD_PLAYER_CLIENTS
(web_safari,web,tv — POT-capable; the default android_vr client never requests a token)
- formats.build_ydl_opts wires extractor_args youtube:player_client + youtubepot-bgutilhttp:base_url
Verified: the sidecar mints a valid PO token (BotGuard challenge solved) and yt-dlp fetches it
from the provider. NOTE: end-to-end download couldn't be confirmed here because the test IP got
hard-flagged by YouTube from heavy testing (~16 GB + dozens of extractions) — a hard abuse block
needs cooldown/cookies, which POT doesn't lift; POT bypasses the normal soft bot-check.
Embedding (thumbnail/chapters) rewrote the whole file — a 10 GB video spent minutes making a
10 GB temp copy — while the Plex .nfo + poster sidecars already carry that metadata. So:
- builtin presets no longer embed (migration 0040 re-seeds them; formats._DEFAULTS embed off);
only the tiny FFmpegThumbnailsConvertor (for poster.jpg) remains, so downloads go
video -> audio -> merge -> done with no full-file rewrite
- default preset is now 1080p (Best stays available, second in the list) so a long video
doesn't silently pull ~10 GB and blow the quota
Users who want self-contained files can enable embedding in a custom profile.
Root cause of the failing downloads: containers usually have no working IPv6 route, but
googlevideo CDN hosts advertise AAAA records, so the downloader tried IPv6 and failed with
'[Errno -5] No address associated with hostname'. Small clips happened to use IPv4; long videos
(more CDN requests / the ffmpeg path) hit IPv6 and died.
- yt-dlp source_address=0.0.0.0 forces IPv4 for every connection (== --force-ipv4). Verified: the
full video+audio download completes cleanly.
- Also: retry (resume) now resets the shared asset from 'error' to 'pending', so it actually
re-downloads — previously the requeued job instantly re-inherited the asset's stale error
because the worker short-circuits a job whose asset already errored.
- Dropped the localdev DNS override (wrong hypothesis; the issue was IPv6, not the resolver).
- Kept yt-dlp retries/fragment_retries/socket_timeout for transient blips.
Verified end-to-end in the worker: two previously-failing long videos now download to done.
The progress bar looked broken (95% -> 5% -> recount) because yt-dlp downloads the video and
audio streams separately (each 0->100%) then merges — with no indication of which step is
running. Now:
- worker labels the phase from the stream codecs (video / audio) and via a postprocessor hook
(merging / processing), so the user sees what's happening
- DownloadCenter shows the phase as the status and renders an indeterminate pulse (no bogus %)
during merge/processing; phase i18n en/hu/de
- yt-dlp retries (3) + fragment_retries (5) so transient network blips self-heal instead of
failing the job (seen once as a DNS 'No address associated with hostname' error)
Verified: phase transitions queued -> video -> processing -> done.
The .gitignore 'downloads/' pattern for the DOWNLOAD_ROOT bind mount also matched the
backend/app/downloads source package, so formats.py/storage.py/service.py never got
committed with M2. Anchor the ignore to '/downloads/' (repo root only) and add the package.
storage.sanitize() now handles arbitrarily messy titles (emoji, ZWJ, fullwidth, clickbait
punctuation): NFKC-normalize, drop emoji/symbol/control unicode categories, collapse
repeated punctuation, underscore-join words -> space-free paths. Accents (HU/DE) preserved,
no ASCII folding. Plex layout uses _-_ separators + Season_{year}.
