Commit graph

6 commits

Author SHA1 Message Date
npeter83
7a6f351e64 fix(downloads): repair 5 confirmed backend bugs (B1-B5)
- B1 (sticky errored asset): get_or_create_asset now resets a reused status=='error'
  asset back to 'pending' (+clears the error), so a fresh enqueue/edit of a once-failed
  (source,format) pair actually re-downloads instead of the worker short-circuiting the
  new job with the stale error. Errored assets carry no expires_at, so without this the
  pair was permanently poisoned for all users until someone hit resume.
- B2 (ref_count leak): _release_asset counts 'error' as a holding state, so deleting an
  errored job decrements the ref_count that enqueue always incremented (the worker never
  decrements on failure). Errored rows are deliberately NOT deleted here — a concurrent
  B1 reuse could otherwise be lost-updated + FK-nulled; the row is fileless and harmless.
- B3: admin storage dashboard reads sysconfig.get_int(db,'download_total_max_bytes')
  (the admin-editable DB value GC enforces) instead of the raw env default.
- B4: the single-trim branch of normalize_edit_spec guards its float() coercion like the
  crop/segments branches — a malformed trim now yields a 400, not an unhandled 500.
- B5: formats.normalize guards int(max_height) → falls back to "best" instead of 500.

Reviewed (race in an earlier B2 draft caught + fixed); localdev boots, B4/B5 unit-verified.
2026-07-11 16:15:12 +02:00
npeter83
33fad8911b fix(downloads): clip long titles to fit display_name (VARCHAR 255)
A download whose source title exceeded 255 chars (e.g. a recipe baked into a
Facebook video's title) failed with a StringDataRightTruncation: the worker
auto-fills an empty display_name from the title, which overflows the 255-char
column. Clip the title to 255 at every point it flows into display_name (worker
completion paths + enqueue); the full title is untouched on the asset. Pre-existing
latent bug, surfaced by an unusually long title.

Release v0.31.1.
2026-07-07 21:40:15 +02:00
npeter83
f0fc542260 feat(downloads): editor backend — trim/crop derivatives (phase 2)
Per-user trim/crop clips derived from a finished download via ffmpeg, riding the same
MediaAsset/DownloadJob/worker/quota/GC machinery (asset source_kind='edit', per-user cache key):
- migration 0041: download_jobs job_kind/source_job_id/source_asset_id/edit_spec
- app/downloads/edit.py: spec normalize+sig, ffmpeg trim/crop cmd (fast stream-copy vs frame-
  accurate re-encode, per-edit 'accurate' toggle), progress+cooperative-cancel runner, filmstrip
- service.enqueue_edit; worker _process_edit branch; storage.edit_rel_path (.edits/{user} tree)
- routes: POST /edit, GET /{id}/storyboard(.jpg)
2026-07-04 01:01:46 +02:00
npeter83
c6ceaea899 fix(downloads): show real title + thumbnail while queued/downloading
A queued/downloading row previously showed the bare video id and no thumbnail because asset
metadata was only filled when the download completed. Now:
- service.populate_from_catalog fills title/uploader/thumbnail/date/duration at enqueue from
  our own Video/Channel catalog (0 network cost) — feed downloads look right instantly
- worker fills the same from yt-dlp's info_dict on the first progress event (covers ad-hoc
  URLs / catalog misses), best-effort, never fails a download

Verified: a paused feed download now shows its real title, channel, thumbnail and duration.
2026-07-03 02:23:21 +02:00
npeter83
7148335c09 feat(downloads): M3 — per-user quota + retention GC
- downloads/quota.py: per-user limits (DownloadQuota row or sysconfig defaults; unlimited
  bypass), footprint = distinct ready-asset bytes the user holds, check_enqueue (max_jobs +
  max_bytes hard at enqueue), at_concurrency_limit (worker-side max_concurrent), usage snapshot
- downloads/gc.py: run_download_gc — pre-expiry warning (once, gc_notified flag), TTL deletion
  (files + row; FK SET NULL orphans holders' jobs -> 'expired'), LRU eviction over a total-cache
  cap; notifies holders on each event
- migration 0038: media_assets.gc_notified
- config/sysconfig: download_total_max_bytes (0=unlimited) in the downloads group
- service.enqueue enforces quota; worker claim skips users at their concurrency limit
- scheduler: download_gc job registered (default 360 min, admin-tunable)

Verified on localdev: footprint accounting, max_jobs block, pre-expiry warning (notifies all
holders), TTL deletion (asset+files gone, dirs pruned, jobs orphaned, holders notified).
2026-07-03 00:26:40 +02:00
npeter83
7f9847c2c2 fix(downloads): add the downloads package (was wrongly gitignored) + robust sanitizer
The .gitignore 'downloads/' pattern for the DOWNLOAD_ROOT bind mount also matched the
backend/app/downloads source package, so formats.py/storage.py/service.py never got
committed with M2. Anchor the ignore to '/downloads/' (repo root only) and add the package.

storage.sanitize() now handles arbitrarily messy titles (emoji, ZWJ, fullwidth, clickbait
punctuation): NFKC-normalize, drop emoji/symbol/control unicode categories, collapse
repeated punctuation, underscore-join words -> space-free paths. Accents (HU/DE) preserved,
no ASCII folding. Plex layout uses _-_ separators + Season_{year}.
2026-07-03 00:19:22 +02:00