siftlode

peter/siftlode

Fork 0

Commit graph

Author	SHA1	Message	Date
npeter83	c6ceaea899	fix(downloads): show real title + thumbnail while queued/downloading A queued/downloading row previously showed the bare video id and no thumbnail because asset metadata was only filled when the download completed. Now: - service.populate_from_catalog fills title/uploader/thumbnail/date/duration at enqueue from our own Video/Channel catalog (0 network cost) — feed downloads look right instantly - worker fills the same from yt-dlp's info_dict on the first progress event (covers ad-hoc URLs / catalog misses), best-effort, never fails a download Verified: a paused feed download now shows its real title, channel, thumbnail and duration.	2026-07-03 02:23:21 +02:00
npeter83	7148335c09	feat(downloads): M3 — per-user quota + retention GC - downloads/quota.py: per-user limits (DownloadQuota row or sysconfig defaults; unlimited bypass), footprint = distinct ready-asset bytes the user holds, check_enqueue (max_jobs + max_bytes hard at enqueue), at_concurrency_limit (worker-side max_concurrent), usage snapshot - downloads/gc.py: run_download_gc — pre-expiry warning (once, gc_notified flag), TTL deletion (files + row; FK SET NULL orphans holders' jobs -> 'expired'), LRU eviction over a total-cache cap; notifies holders on each event - migration 0038: media_assets.gc_notified - config/sysconfig: download_total_max_bytes (0=unlimited) in the downloads group - service.enqueue enforces quota; worker claim skips users at their concurrency limit - scheduler: download_gc job registered (default 360 min, admin-tunable) Verified on localdev: footprint accounting, max_jobs block, pre-expiry warning (notifies all holders), TTL deletion (asset+files gone, dirs pruned, jobs orphaned, holders notified).	2026-07-03 00:26:40 +02:00
npeter83	7f9847c2c2	fix(downloads): add the downloads package (was wrongly gitignored) + robust sanitizer The .gitignore 'downloads/' pattern for the DOWNLOAD_ROOT bind mount also matched the backend/app/downloads source package, so formats.py/storage.py/service.py never got committed with M2. Anchor the ignore to '/downloads/' (repo root only) and add the package. storage.sanitize() now handles arbitrarily messy titles (emoji, ZWJ, fullwidth, clickbait punctuation): NFKC-normalize, drop emoji/symbol/control unicode categories, collapse repeated punctuation, underscore-join words -> space-free paths. Accents (HU/DE) preserved, no ASCII folding. Plex layout uses _-_ separators + Season_{year}.	2026-07-03 00:19:22 +02:00

Author

SHA1

Message

Date

npeter83

c6ceaea899

fix(downloads): show real title + thumbnail while queued/downloading

A queued/downloading row previously showed the bare video id and no thumbnail because asset
metadata was only filled when the download completed. Now:
- service.populate_from_catalog fills title/uploader/thumbnail/date/duration at enqueue from
  our own Video/Channel catalog (0 network cost) — feed downloads look right instantly
- worker fills the same from yt-dlp's info_dict on the first progress event (covers ad-hoc
  URLs / catalog misses), best-effort, never fails a download

Verified: a paused feed download now shows its real title, channel, thumbnail and duration.

2026-07-03 02:23:21 +02:00

npeter83

7148335c09

feat(downloads): M3 — per-user quota + retention GC

- downloads/quota.py: per-user limits (DownloadQuota row or sysconfig defaults; unlimited
  bypass), footprint = distinct ready-asset bytes the user holds, check_enqueue (max_jobs +
  max_bytes hard at enqueue), at_concurrency_limit (worker-side max_concurrent), usage snapshot
- downloads/gc.py: run_download_gc — pre-expiry warning (once, gc_notified flag), TTL deletion
  (files + row; FK SET NULL orphans holders' jobs -> 'expired'), LRU eviction over a total-cache
  cap; notifies holders on each event
- migration 0038: media_assets.gc_notified
- config/sysconfig: download_total_max_bytes (0=unlimited) in the downloads group
- service.enqueue enforces quota; worker claim skips users at their concurrency limit
- scheduler: download_gc job registered (default 360 min, admin-tunable)

Verified on localdev: footprint accounting, max_jobs block, pre-expiry warning (notifies all
holders), TTL deletion (asset+files gone, dirs pruned, jobs orphaned, holders notified).

2026-07-03 00:26:40 +02:00

npeter83

7f9847c2c2

fix(downloads): add the downloads package (was wrongly gitignored) + robust sanitizer

The .gitignore 'downloads/' pattern for the DOWNLOAD_ROOT bind mount also matched the
backend/app/downloads source package, so formats.py/storage.py/service.py never got
committed with M2. Anchor the ignore to '/downloads/' (repo root only) and add the package.

storage.sanitize() now handles arbitrarily messy titles (emoji, ZWJ, fullwidth, clickbait
punctuation): NFKC-normalize, drop emoji/symbol/control unicode categories, collapse
repeated punctuation, underscore-join words -> space-free paths. Accents (HU/DE) preserved,
no ASCII folding. Plex layout uses _-_ separators + Season_{year}.

2026-07-03 00:19:22 +02:00

3 commits