The web player clients need two things beyond the PO token: (1) a JavaScript runtime for
YouTube's 'n' signature challenge — install Deno (v2.9.1) + yt-dlp[default] (bundles yt-dlp-ejs
solver scripts); auto-detected on PATH. (2) an audio source — the web clients only expose
video-only DASH, so add the 'android' client (also PO-token-backed) for audio/muxed formats.
Player clients: web_safari,web,android (dropped tv — it falsely reports normal videos as DRM).
End-to-end verified in the worker: extraction (POT + Deno n-challenge) + download completes,
no bot/DRM/format errors. Full YouTube stack now: force-ipv4 + bgutil POT sidecar + Deno.
YouTube increasingly demands a Proof-of-Origin token (or shows 'confirm you're not a bot').
Instead of manual cookies, run the bgutil POT provider as a sidecar that mints tokens on demand:
- compose: bgutil-pot service (brainicism/bgutil-ytdlp-pot-provider:1.3.1, port 4416, init)
- requirements: bgutil-ytdlp-pot-provider==1.3.1 plugin (version pinned to the image)
- config: DOWNLOAD_POT_BASE_URL (http://bgutil-pot:4416) + DOWNLOAD_PLAYER_CLIENTS
(web_safari,web,tv — POT-capable; the default android_vr client never requests a token)
- formats.build_ydl_opts wires extractor_args youtube:player_client + youtubepot-bgutilhttp:base_url
Verified: the sidecar mints a valid PO token (BotGuard challenge solved) and yt-dlp fetches it
from the provider. NOTE: end-to-end download couldn't be confirmed here because the test IP got
hard-flagged by YouTube from heavy testing (~16 GB + dozens of extractions) — a hard abuse block
needs cooldown/cookies, which POT doesn't lift; POT bypasses the normal soft bot-check.
Embedding (thumbnail/chapters) rewrote the whole file — a 10 GB video spent minutes making a
10 GB temp copy — while the Plex .nfo + poster sidecars already carry that metadata. So:
- builtin presets no longer embed (migration 0040 re-seeds them; formats._DEFAULTS embed off);
only the tiny FFmpegThumbnailsConvertor (for poster.jpg) remains, so downloads go
video -> audio -> merge -> done with no full-file rewrite
- default preset is now 1080p (Best stays available, second in the list) so a long video
doesn't silently pull ~10 GB and blow the quota
Users who want self-contained files can enable embedding in a custom profile.
ffmpeg post-steps have no byte-progress, so instead of a silent 'Processing' the row now names
the current step (Merging / Extracting audio / Embedding thumbnail / Removing sponsors / Writing
metadata) with an indeterminate pulse. Byte-progress phases (video/audio) keep the % bar.
i18n en/hu/de.
Root cause of the failing downloads: containers usually have no working IPv6 route, but
googlevideo CDN hosts advertise AAAA records, so the downloader tried IPv6 and failed with
'[Errno -5] No address associated with hostname'. Small clips happened to use IPv4; long videos
(more CDN requests / the ffmpeg path) hit IPv6 and died.
- yt-dlp source_address=0.0.0.0 forces IPv4 for every connection (== --force-ipv4). Verified: the
full video+audio download completes cleanly.
- Also: retry (resume) now resets the shared asset from 'error' to 'pending', so it actually
re-downloads — previously the requeued job instantly re-inherited the asset's stale error
because the worker short-circuits a job whose asset already errored.
- Dropped the localdev DNS override (wrong hypothesis; the issue was IPv6, not the resolver).
- Kept yt-dlp retries/fragment_retries/socket_timeout for transient blips.
Verified end-to-end in the worker: two previously-failing long videos now download to done.
_insert_stubs previously stored the raw title, so a brand-new video showed its unnormalized
title until enrichment caught up. Normalize (and stash original_title) at insert as well, so
every video-title write path is covered: stub insert, enrichment, live search, and the download
worker. Every newly imported feed video is normalized from the first insert.
Noisy YouTube titles are cleaned for display + storage, raw kept in videos.original_title:
- app/titles.normalize_title: strip emoji/symbols (keep accents), remove trailing SEO hashtag
clusters (keep numeric #3 episode markers), context-aware de-shout (mostly-ALL-CAPS titles ->
Title Case with an acronym whitelist + function-word lowercasing; otherwise only long all-caps
words), collapse repeated punctuation
- applied at enrichment (sync/videos.py) and in the download worker (ad-hoc yt-dlp titles);
catalog downloads inherit the normalized title automatically
- migration 0039: add original_title, preserve raw, rewrite title (generated search_vector
regenerates); reversible via original_title
Backfill on localdev: 122115/273417 titles normalized in ~2 min. Verified in the feed + on
real messy samples (emoji/de-shout/hashtags), accents + acronyms (PS5/AI/USA/PC) preserved.
The progress bar looked broken (95% -> 5% -> recount) because yt-dlp downloads the video and
audio streams separately (each 0->100%) then merges — with no indication of which step is
running. Now:
- worker labels the phase from the stream codecs (video / audio) and via a postprocessor hook
(merging / processing), so the user sees what's happening
- DownloadCenter shows the phase as the status and renders an indeterminate pulse (no bogus %)
during merge/processing; phase i18n en/hu/de
- yt-dlp retries (3) + fragment_retries (5) so transient network blips self-heal instead of
failing the job (seen once as a DNS 'No address associated with hostname' error)
Verified: phase transitions queued -> video -> processing -> done.
A queued/downloading row previously showed the bare video id and no thumbnail because asset
metadata was only filled when the download completed. Now:
- service.populate_from_catalog fills title/uploader/thumbnail/date/duration at enqueue from
our own Video/Channel catalog (0 network cost) — feed downloads look right instantly
- worker fills the same from yt-dlp's info_dict on the first progress event (covers ad-hoc
URLs / catalog misses), best-effort, never fails a download
Verified: a paused feed download now shows its real title, channel, thumbnail and duration.
Two UAT findings:
1. The device download filename (Content-Disposition) kept emoji/symbols from the video title.
Add storage.display_filename (drops emoji/symbol/control unicode, keeps spaces + accents)
and use it for the download name — readable and clean ("…alapján!.mp4", no emoji).
2. Deleting/canceling a download removed the job but the shared MediaAsset (and its file) lingered
as cache, so 'Ready files' stayed inflated and disk wasn't freed. Rework: _release_asset drops
the hold and, once no job holds the asset, deletes the file + row immediately (the cache only
needs to span overlapping holders). Also fixes cancel never decrementing (it flipped status to
'canceled' before releasing, tripping the holding-state guard).
Verified: filename emoji-stripped; enqueue→delete removes the asset row + file from disk.
A plain <a> file download can't send the X-Siftlode-Account header, so current_user resolved
it to the session-default account — 404 'Unknown download' when the tab acts as a non-default
wallet account that owns the file. resolved_user_id now also honours a ?account= query param
(the same wallet-gated selection the WebSocket already uses), and downloadFileUrl appends the
active account id. Verified: default account -> 404, ?account=<owner> -> 206 with the right
Content-Disposition.
The .gitignore 'downloads/' pattern for the DOWNLOAD_ROOT bind mount also matched the
backend/app/downloads source package, so formats.py/storage.py/service.py never got
committed with M2. Anchor the ignore to '/downloads/' (repo root only) and add the package.
storage.sanitize() now handles arbitrarily messy titles (emoji, ZWJ, fullwidth, clickbait
punctuation): NFKC-normalize, drop emoji/symbol/control unicode categories, collapse
repeated punctuation, underscore-join words -> space-free paths. Accents (HU/DE) preserved,
no ASCII folding. Plex layout uses _-_ separators + Season_{year}.
Add an accent-tinted 'Free and open source' band between the app preview and the feature grid
— a short line about reading the code / self-hosting / filing an issue, linking to the public
repo. The compact footer link stays. Trilingual.
Multi-account-in-one-browser (esp. with per-tab accounts) leaked one account's client state
into another via shared localStorage keys. Scope every account-specific key by the tab's active
account (accountKey/readAccount/writeAccount/useAccountPersistedState helpers), so nothing bleeds
across accounts or tabs:
- Real leaks: selected playlist, client notification history + settings, onboarding-dismissed.
- UI position: feed page, channel-manager filter/view + tables, playlist sort, Settings/Stats/
Users/Config tabs.
- Previously DB-adopted caches (theme, hints, performance mode, sidebar layout, nav/filter
collapse) — now the cache is per-account too, so there's no flash of the other account's value
on login.
Kept intentionally global: siftlode.lang (needed pre-login on the Welcome page; the DB pref still
scopes it per-account after sign-in) and siftlode.seenVersion (a per-browser 'new version' banner).
E2EE private keys (IndexedDB) and the chat-dock key were already per-user.
Feed filters and the default-saved-view mirror lived under shared localStorage keys, so one
account's view (incl. its starred default) leaked into another account signed into the same
browser — visible with per-tab accounts as a fresh account showing the previous account's
'N active' filters instead of a clean default.
Key both by the tab's active account (siftlode.filters.<id> / siftlode.defaultViewFilters.<id>);
load a tab's filters from its own account on login/switch/add, falling back to defaults when the
account isn't known yet. A share link's filters still win and are persisted to the account. The
old shared keys are simply no longer read (a one-time reset to defaults on the first load after
this change; migrating them would risk re-leaking across accounts).
A tab that never explicitly picked an account rode the session's default, so when ANOTHER
tab changed that default (adding or switching an account), this tab silently swapped identity
on its next refetch — the account chip updated while the feed/saved-views lagged until reload.
- Pin every tab to whatever account it first loaded as (write the sessionStorage override on
load when none is set), so its requests always carry its own account header and cross-tab
default changes can't reach it.
- 'Add account' now clears this tab's pin before the Google redirect, so on return the tab
adopts the freshly-added account (and pins that) — the current tab switches to it while other
tabs keep their own, matching the Gmail add-account model.
Two tabs in one browser can now run two different signed-in accounts at once.
- The signed session cookie stays the browser's account WALLET (account_ids). Which account a
given tab acts as is a per-tab choice held in sessionStorage and sent per-request via the
X-Siftlode-Account header; current_user honours it only for an account already in the wallet,
without mutating the cookie's default account. Switching accounts sets the header + reloads
THIS tab only, instead of the old cookie-wide switch that changed every tab.
- WebSocket can't send headers, so the per-tab account rides in the ?account= query param
(validated against the wallet).
- Logout is per-tab aware: it signs the requesting tab's active account out of the wallet
(promoting a new default only if the removed one was the default), and the tab drops its
override. A stale per-tab header account 401s just that tab instead of clearing the session.
- Serve index.html with Cache-Control: no-cache so a deploy's new hashed bundle is picked up
immediately instead of the browser running a heuristically-cached stale index.html.
- Filter header: show the active-filter count as a compact number pill instead of
'{n} active' text, and keep the action buttons on one line (shrink-0 + nowrap) so
nothing truncates or wraps in a narrow / zoomed-in sidebar (e.g. 125% browser zoom).
- Sync status: anchor the 'all synced' state with a small check icon so it no longer
reads as orphaned text at the top of the nav rail.
Restructure the app shell into three top-level columns:
- The per-user sync status (video counts + live sync state) moves from the top bar to a
compact block at the top of the left nav rail (icon-only with a tooltip when collapsed).
- The feed's Mine/Library scope toggle moves to the top of the filter sidebar.
- The filter sidebar becomes a full-height sibling column with its own collapse control
(a thin rail carrying the active-filter count), mirroring the nav rail. The top bar is
now just the feed search / page title.
- Both panels' collapsed state is persisted to the user's preferences (server-side, so it
follows the account across devices), seeded from a localStorage cache to avoid a flash.
Default: both panels open.
The icon-only rail switcher now shows the active language code as a small corner badge,
so the current language reads at a glance without opening the menu.
- A clearer 'Admin' section header (thicker centred rule when collapsed) above the
admin-only modules (Scheduler/Configuration/Users), replacing the faint hairline.
- A small role chip (admin/user/demo) next to the account name — an avatar-corner dot
when the rail is collapsed.
useBackToClose eagerly pushed a history entry on mount and called history.back()
on unmount. During a modal->modal handoff (e.g. About -> Release Notes) the two ran
interleaved in one React commit, so the entering modal's popstate listener mistook the
leaving modal's back() for a genuine user Back and closed itself instantly. It also left
the history pointer behind the surviving entry, so a later browser Back walked off the app.
Replace the eager per-mount push/pop with a single shared popstate handler plus a
coalesced microtask that reconciles history depth to the live overlay count once per tick.
A handoff's -1/+1 nets to zero, so the new modal simply reuses the old entry -- no churn,
no flash, and Back closes the modal in-app.
- Remove owner-specific / legacy deploy files (home/prod/server compose, deploy/).
The home compose stays as a local untracked file for the maintainer's own deploy.
- Genericise infra-specific code comments (egress-proxy examples) to neutral wording.
- Replace the hardcoded contact email on the legal pages with the instance operator's
configured admin email, served via the public /auth/config and shown with a neutral
fallback — so each self-hosted instance shows its own contact.
- Rewrite README for the current app + a copy-paste self-hosting quick start (prebuilt
image + first-run wizard) with a build-from-source alternative; tidy .env.example.
Tighten the hero + feature cards to punchy one-liners and surface current
capabilities: swap the standalone channel-manager card for live YouTube
search (folding tagging into the readable card). Regenerate the three
preview images (feed, channels, playlists) from a rich real account instead
of the sparse demo, so the landing shows what a full instance looks like.
EN/HU/DE.
Moves the Tags widget up (after Upload date) in the default sidebar layout.
Only affects fresh layouts; existing users keep their persisted arrangement.
The "Filters N active" badge and Clear all only tracked the sidebar facets
(tags/show/sort/content/date/channel), so the toolbar-level Source and the
Mine/Library scope neither bumped the count nor reset — inconsistent with
filtersToParams (which treats them as filters, so Save view/Share already
captured them).
- activeCount now includes Library scope and a non-default Source; "relevance"
(auto-applied while searching, no manual option) is excluded from the sort
count so typing a query no longer inflates the badge.
- Clear all resets scope back to "my" (Mine) and drops Source to organic,
alongside the facets. The search q is intentionally preserved.
- The header search box gets a clear (X) button when it holds a value, so the
search has its own reset (clearing it reverts the relevance sort to newest).
A new "Saved views" filter-sidebar widget (SavedViewsWidget): save the
current filters under a name, apply one with a click (active-highlighted
when it matches), rename, delete (confirm), drag-reorder, share by link
(reuses shareUrl), and star one as the default. The default view's filters
are mirrored to localStorage so loadInitialFilters applies it synchronously
on load/F5 (an api/url share link still wins; no default → last-session
filters as before). Hidden for the demo account. EN/HU/DE.
Per-user named snapshots of the feed's FeedFilters. New saved_views table
(migration 0035, unique per user+name, position, is_default) + SavedView
model, and a /api/saved-views router (list/create/patch/delete/reorder)
gated behind require_human so the shared demo account can't pollute it.
Setting is_default clears the flag on the user's other views (one default).
