Correcting the earlier scope-union-only fix. The real damage from a logout→login isn't
just the stored `scope` field — verified via a live token refresh that the stored refresh
token itself had been REPLACED with a base-only one: a plain sign-in requests only
BASE_SCOPES and Google handed back a fresh base-only access+refresh token, which
_store_token adopted verbatim, destroying the read/write grant (the old refresh token
stays valid on Google's side, so overwriting it is what loses access).
_store_token now detects when an exchange would NARROW our YouTube scopes and, in that
case, keeps the WHOLE existing grant (refresh token, access token, scopes) untouched.
Broader-or-equal exchanges (first grant, read→write upgrade) adopt + union as before.
Unit-verified across base-relogin / upgrade / new-user / base-user cases.
Two user-reported signin bugs:
1) A plain re-login (or logout→login) wiped the user's YouTube grant: login requests
only BASE_SCOPES and Google's returned `scope` can list just those, but _store_token
wrote it verbatim — dropping a previously-granted youtube.readonly/youtube scope, so
can_read flipped to false and the feed demanded a reconnect. The underlying grant
(refresh token) survives such a login, so UNION the scopes instead of narrowing; they
only shrink on full disconnect (purge deletes the token row).
2) Admin 'new access request' email: (a) it named the wrong menu ('Settings → Account'
instead of Users → Access requests); (b) the requester address was invisible so the
'reply reaches them' note looked wrong (Reply-To is in fact set to the requester) —
now spelled out + a deep-link straight to the approve view (?admin=access-requests,
handled in App); (c) it emailed only the static env ADMIN_EMAILS — now notifies the
ACTUAL admins (active role=admin users) unioned with env, so a UI-promoted admin (who
CAN approve — the approve UI is role-gated) is notified too.
register + password-reset-request move their lookup/create/token work off the response
path (background task, own session) so response timing no longer reveals whether an
email is registered. messages_ws shares resolved_user_id (now HTTPConnection-typed)
instead of re-implementing the per-tab wallet-gated resolution. Behavior-preserving;
reviewed (no security regression) + verified.
resolved_user_id already accesses ws.session unguarded just above (SessionMiddleware
covers the WS scope), so the '"session" in ws.scope' fallback was dead code — read
ws.session directly for consistency (review follow-up).
Loose ends to finish the auth security round:
- register + password-reset-request had an enumeration TIMING oracle: an already-
registered email skipped the create path (hash + row writes + email scheduling) and
responded measurably faster than a new one. Move the whole lookup+create (register)
and lookup+token+email (reset) into a background task with its own DB session, so the
endpoint returns in the same time for any valid email regardless of whether it exists.
Verified: existing vs new now ~equal (was 34ms vs 82ms on register); accounts/tokens
still created off-path.
- messages_ws re-implemented resolved_user_id's per-tab wallet-gated account resolution.
Generalize resolved_user_id to take any HTTPConnection (Request OR WebSocket) and call
it from the WS — one shared, wallet-gated resolution. Behavior-identical (unit-checked).
Auth security round: SB3 (email tokens out of URL query → fragment), SA4 (server-side
session revocation + 'Log out other sessions'), SA3 (trusted-proxy X-Forwarded-For so
rate limits can't be bypassed via a forged header).
Post-review hardening (both fail-safe, not bugs):
- _client_ip normalizes IPs via ipaddress and unwraps IPv4-mapped IPv6, so a plain
TRUSTED_PROXY_IPS=10.10.0.1 also matches a peer surfaced as ::ffff:10.10.0.1 (a
Docker/IPv6 self-host footgun that would otherwise silently collapse everyone into
one rate-limit bucket).
- entrypoint.sh + _client_ip docstring: explicit warning never to add uvicorn
--proxy-headers, which would rewrite request.client from the forgeable XFF and defeat
the trust check.
_client_ip trusted the first X-Forwarded-For hop unconditionally, so anyone able to
reach the app port could forge XFF and dodge the login/register/reset/demo rate limits.
Now trust XFF ONLY when the request's socket peer is a configured reverse proxy
(settings.trusted_proxy_ips, e.g. the VPS Caddy's WireGuard peer IP), and take the
RIGHTMOST entry — the client our proxy actually saw and appended, immune to a client
pre-seeding a fake XFF. A request from any other peer (hitting the port directly) is
keyed on its real socket IP, so XFF can't be forged to bypass the limits.
New TRUSTED_PROXY_IPS env (empty default = no proxy, use the direct peer). Documented in
.env.example, docs/self-hosting.md, README. Unit-verified against spoof-through-proxy and
direct-bypass cases.
SB3: reset/verify email tokens moved from URL query strings to the fragment (never
sent to the server / proxy logs / Referer); verify GET→POST with a legacy-GET fallback.
SA4: server-side session revocation via per-user session_epoch (migration 0053) — a
signed cookie records its epoch, current_user + the messages WS reject a stale-epoch
cookie. Bumped on password reset (all sessions), password change + a new 'Log out other
sessions' action (both keep the current session). E2E-verified; UAT passed.
SA3 (trusted-proxy for X-Forwarded-For) intentionally deferred to a prod-tested follow-up.
Adversarial re-review of the session-epoch work surfaced:
- WS auth (messages_ws) skipped the epoch check, so a revoked-but-unexpired cookie
could still open the live push channel after a reset/logout-others. Now mirrors
current_user: loads the user once, rejects a stale-epoch cookie before connecting.
- set_password + logout_others re-stamped the cookie BEFORE db.commit(); a failed
commit would strand the current session at a newer epoch than the DB and wrongly
401 it. Commit first, then re-stamp.
- Welcome verify effect could double-POST the single-use token (StrictMode/remount)
and flip the banner to a false 'invalid'. Fire-once useRef guard.
Left as-is (low value, documented): the Plex image proxy authenticates without a DB
load / epoch check (poster/art fetches only); adding one would cost a DB hit per image.
Signed client-side session cookies had no server-side kill switch: logout + password
reset couldn't evict a stolen/copied cookie (valid until expiry). Add User.session_epoch
(migration 0053), record it in the cookie at every login, and reject in current_user any
cookie whose recorded epoch is behind the account's current one.
- Bump the epoch on: password reset (kills ALL sessions — a reset is a compromise response),
password change + a new 'Log out other sessions' action (both re-stamp the CURRENT cookie
so the caller stays signed in, evicting only the others).
- Per-account epoch map in the session so one account's revocation doesn't evict the other
signed-in accounts in the same browser wallet.
- Missing epoch (pre-SA4 cookie) is treated as 0, so the first bump revokes grandfathered
sessions too.
- New POST /auth/logout-others + a Settings → Account 'Active sessions' button (trilingual).
Secret email tokens now ride the URL fragment (#reset=/#verify=), never the query
(?reset=/?token=): a fragment isn't sent to the server, so the token can't leak into
proxy/access logs or a Referer header.
- Reset: link → /#reset=; the SPA reads the token from location.hash and POSTs it
(unchanged /password-reset/confirm).
- Verify: link → /#verify=; new POST /auth/verify (token in body). The legacy GET
/auth/verify?token= is kept so pre-deploy emails in flight still work until they
expire. The SPA reads the fragment token, POSTs it, shows ok/invalid.
- Welcome: read secret tokens from the fragment, status flags from the query; strip
both after capture so nothing lingers in history.
Add the missing collapse trigger to the expanded PlexSidebar (state + collapsed rail
already wired via the shared filterCollapsed); mirrors the feed Sidebar. E2E-verified.
The collapse feature was only half-present on the Plex page: App already passed the
shared filterCollapsed state + toggle to PlexSidebar, and PlexSidebar rendered the
CollapsedFilterRail when collapsed — but the EXPANDED PlexSidebar had no trigger, so
you could only collapse it by first collapsing on the feed page. Add the same
ChevronLeft 'Filters' collapse header the feed Sidebar has (reusing the existing
sidebar.* i18n keys). E2E-verified: collapse→rail→expand round-trips on the Plex page.
- CollapsedFilterRail: the byte-identical 31-line collapsed sidebar rail was in
both Sidebar (feed) and PlexSidebar → one shared component.
- VideoCard: the title + channel-button + meta block was duplicated across the
list-row and card layouts → a local `textBlock` element (only one layout branch
renders, so reusing the element is safe).
- subsColumn<T>(t): the identical 'subscribers' DataTable column in Channels +
ChannelDiscovery → a generic factory in channelColumns.tsx.
All behavior-neutral; jscpd now reports 0 clones (was 8 at Phase 0 baseline).
tsc + knip green.
The standard input style was copy-pasted verbatim into 5 components
(DownloadCenter/DownloadDialog/ProfileEditor/ShareDialog/VideoEditor) and the
text-button style into 2 (ShareDialog/VideoEditor). Export both from ui/form.tsx
and import them. Byte-identical strings → no visual change.
The settings-family panels (Settings/Config/Setup/Welcome) use a DIFFERENT input
style (bg-card/rounded-xl/focus:border-accent); unifying the two is a design call
left to the glass-consistency epic, not this DRY extract.
Callback-ref the grid's IntersectionObserver sentinel so it re-attaches on the
grid's unmount/remount cycle (drill into info/show/season → Back), instead of
observing a detached old node and freezing the feed at the loaded pages.
The unified-library grid unmounts when you open an item's info/show/season page
and remounts as a NEW element on Back. The IntersectionObserver effect keyed only
on [hasNextPage, isFetchingNextPage, fetchNextPage] — none of which change across a
pure navigation — so it never re-ran: the observer kept watching the detached old
sentinel node and fetchNextPage never fired again, leaving the feed frozen at the
already-loaded pages (e.g. 80/1227) even though more exist. Intermittent because a
coincident isFetchingNextPage toggle around the nav could re-run the effect.
Fix: observe via a callback ref (state) instead of useRef, so the effect re-runs on
every sentinel mount/unmount and always watches the live node.
Closes the Plex #9 backend backlog (frontend was the prior follow-up). Behavior-
neutral cleanup + two two-way-sync bug fixes; PW2 verified a non-bug.
fix(plex): watch-sync PW1 pagination + PW3 lost-update guard
- PW1 watch_history: the history is a GLOBAL cross-account feed read as a single
500-row page, then filtered to the owner — on a busy family server the owner's
recent views can sit past page 1 and be missed. Now paginate (desc) until the
since-cutoff, bounded by max_pages (daily reconcile is the backstop; logs if hit).
- PW3 push_state_to_plex: it flagged synced_to_plex=True unconditionally after a
push, so a local edit landing between the push and the flag-set was marked clean
and never pushed (lost update). Capture the row's freshest timestamp when the
push is scheduled (_push_watch) and only settle the flag if the row is unchanged.
- PW2 was flagged as "accountID hardcoded to 1" but is NOT a bug: on a PMS account 1
is reserved for the owner/admin and an owner link always holds the admin token —
added a clarifying comment so it isn't re-flagged.
chore(plex): backend dedup
- PC2 unified_library + facets shared a ~13-field filter Query signature + p-dict →
one LibraryFilters dataclass injected via Depends(); as_dict() feeds the builders.
- PS-C1 sync.py collection upsert dup (resync_collection ≈ _sync_collections) →
_upsert_collection().
- PS-C2 sync.py 8-field filterable-metadata block dup (_apply_item ≈ _sync_shows) →
_apply_facet_fields().
- PW-C1 _repush_dirty read duration from the mirrored PlexItem.duration_s instead of
a per-row plex.metadata() round-trip.
- PW-C2 rk→id map built inline in two places → _rk_to_id() helper.
- PP1: reset cancelledMarkerRef on item change — a cancelled intro/credits
auto-skip on one episode suppressed a same-offset marker on the next (auto-skip
silently dead for the rest of a binge).
- PP2: loadSession no longer depends on the i18n `t` (read via a ref). A
mid-playback language switch rebuilt loadSession → re-ran the detail effect →
reloaded the session from the STALE saved resume position, losing live progress.
- PP3: tracks (audio/subtitle) menu now stopPropagation on wheel, like the gear
menu — scrolling the list no longer changes volume.
- PS1: collapsed PlexSidebar filter badge clamps to "9+" (was a raw count),
matching the feed Sidebar.
- format.ts: new formatRuntime() ("1h 30m"); PlexBrowse dur() + PlexInfo
fmtDur() were byte-identical copies, now both call it.
- PlexPlayer fmt() delegates to formatDuration (keeps the NaN/negative clamp
the live media clock needs); local reimplementation gone.
- Register LS.plexPlayerPrefs; PlexPlayer uses it instead of a bare string.
- Remove the dead PlexPlayerPrefs.wasPlaying field (written on play/pause, never
read) + its two patchPrefs writes.
- PlexBrowse item page: drop the redundant onStateChange={() => q.refetch()} —
PlexInfo.setState already invalidates ["plex-item", id], so q refetches itself.
- plex.py: delete the no-op _enabled() (never wired as a Depends/called; the real
gate is sysconfig plex_enabled).
- PlexBrowse.tsx: collapse toggleEpisodeWatched into toggleWatched — they were
byte-identical (both take a PlexCard).
- Delete 6 dead plex.json keys (loadMore, playerSoon, filter.library,
playlist.up/down/remove) from en/hu/de — verified 0 refs (the live
playlist.removeShow/removeSeason keys are kept).
Behavior-neutral. tsc green, ruff clean on touched files, localdev boots, Plex renders.
stream.py enforced a hardcoded _MAX_SESSIONS = 4 and never read the
DB-overridable plex_max_transcodes ConfigSpec, so the Configuration → Plex →
"Max concurrent transcodes" knob was dead (same env-vs-DB drift class as the
Downloads/Admin fixes). _enforce_cap(cap) now takes the cap from
sysconfig.get_int(db, "plex_max_transcodes") (>=1 so playback can't be capped to
zero). Bumped the config default 1→4 so the effective default matches the old
hardcoded cap (no behavior change for non-overriders; the knob now works).
From the auth /security-review (contained fixes; architectural SA3 proxy-trust +
SA4 session-revocation deferred to the user):
- SB1: encrypt the OAuth access_token at rest (was plaintext while refresh_token
was encrypted) — a ~1h Google bearer credential. New security.decrypt_optional()
falls back to a refresh for legacy plaintext tokens; column is unbounded String.
E2E-verified: token refreshed → stored as Fernet ciphertext → 319-subscription
YouTube sync succeeded.
- SA5: password_login is no longer a timing/enumeration oracle — it always runs
argon2 (against a decoy hash for unknown/passwordless emails), so response time
can't reveal whether an account exists.
- SB2: Google login only ADOPTS+activates a pre-existing password account when
Google actually attests email_verified (defense-in-depth against takeover); and
the email sync won't overwrite with a value another account owns (avoids a 500).
- demo_login clears the wallet first (demo can't switch to / act as a real account
via the multi-account header); switch_account rejects a suspended target (which
would otherwise clear the whole session on the next request).
Re-review clean; ruff clean; localdev boots; YouTube auth path E2E-verified.
set_user_role (demote) and admin_delete_user counted ALL admins incl. suspended
ones, so with e.g. 1 active + 1 suspended admin you could demote/delete the only
ACTIVE admin — leaving a single suspended admin who can never sign in → permanent
admin lockout needing DB surgery. Now both guards mirror the (already-correct)
suspend guard: block only when the target is an active admin AND it's the last one
(`not target.is_suspended and count_admins(active_only=True) <= 1`) — which also
correctly still allows removing a SUSPENDED admin while one active admin remains.
Resolves the deferred clearDevice security gap: logout never removed the E2EE
private key from IndexedDB, so on a shared machine the conversations stayed
decryptable after sign-out (ChatDock auto-unlocks from the persisted key next
visit). Per the user's decision (clear on EVERY logout), NavSidebar.logout() now
awaits e2ee.clearDevice(me.id) before reload — guarded so an IndexedDB failure
(e.g. private mode) can never block the logout. Re-entering the message
passphrase is required on the next visit.
Resolves the Phase-1 NEEDS-DECISION #3 (clearDevice) + retires the last e2ee
unused export (knip now only flags loadDefaultViewFilters). tsc green, re-review
clean (found + fixed an idb-open throw path blocking logout).
Fix: AES-GCM nonce reuse in e2ee.setup() (+ drop redundant key_check oracle,
backward-compatible); silent 404/429 send failures now toast; scroll/render-churn
in ChatThread + Messages; chatDock LS helper; deleted dead lock(). Verified: tsc,
re-review clean, localdev boots, self-E2E green (unlock + full-thread decrypt on a
real existing key, user entered the passphrase).
- ChatThread send had no onError: a 404 (recipient gone) or 429 (rate-limited)
send failed silently (api.req shows no modal for those "caller-handled" statuses).
Added onError → toast for 404/429 (400/409/500 already raise the global dialog);
draft is kept for retry. New trilingual messages.sendFailed key.
- ChatThread scroll-to-bottom effect keyed on `plain` too, so every 20s poll's
decrypt pass yanked a scrolled-up reader back to the bottom. Key on items.length.
- ChatThread + Messages decrypt effects keyed on `items` (a fresh `?? []` array
every render) → a render storm while loading. Key on q.dataUpdatedAt.
- chatDock.ts: use LS.chatDock(userId) instead of the bare "siftlode.chatDock.*"
literal (matches the storage LS registry).
tsc green, re-review clean, localdev boots, Messages renders (locked-state) w/o console errors.
setup() encrypted BOTH the wrapped private key AND the key_check under the same
wrapKey with the SAME wrapIv — a GCM nonce-reuse bug (leaks keystream + enables
GHASH/tag forgery, which matters under E2EE's malicious-server threat model).
Fixes:
- setup() gives key_check its own independent nonce (checkIv).
- unlock() no longer decrypts key_check to verify the passphrase — that was
redundant (a wrong passphrase already fails the wrapped_private_key GCM auth
tag) AND was the second consumer of the reused nonce.
Backward-compatible: the uploaded bundle shape is unchanged, and existing users'
bundles still unlock (unlock only presence-checks key_check now; the private-key
decrypt path with wrap_iv is untouched). Also removed the unused, architecturally
ineffective lock() export (re-unlocks from IndexedDB on next mount; the meaningful
primitive is clearDevice). Re-review clean; tsc green.
Bugs:
- ConfigPanel (CB1): unchecking a Plex library from the "all" (empty) state made
that library the ONLY selected one instead of all-except-it, because the toggle
ADDED the key to an empty list. Expand "all" to the explicit section set before
toggling. E2E-verified: uncheck from all → all-except-one.
- Scheduler (SB1): "Purge discovery" bulk-deleted search videos/videos/channels
immediately with no confirm (every AdminUsers destructive action uses useConfirm).
Wrapped in a danger useConfirm dialog (+ trilingual purgeConfirmTitle key).
E2E-verified: dialog appears, Cancel aborts.
Cleanups (behavior-neutral):
- Register LS.configTab; ConfigPanel uses it instead of the bare "siftlode.configTab".
- ConfigPanel + SettingsPanel import SaveState from ui/DraftSaveBar instead of
redeclaring the union (PrefsSaveState is now an alias).
tsc green, re-review clean, localdev boots.
- Extract runner.token_users(db): the "all users with a stored refresh token"
query was spelled 3× (get_service_user, run_subscription_resync,
sync_all_playlists). get_service_user now reuses it ([0]); sync_all_playlists
imports it from runner (no cycle — runner doesn't import playlists); the now-
unused OAuthToken import is dropped there.
- scheduler.py: _job("rss_poll", lambda db: run_rss_poll(db)) → _job("rss_poll",
run_rss_poll) — the lambda was dead indirection; all sibling jobs pass the
callable directly and _job calls fn(db).
Behavior-neutral. ruff clean, localdev boots, re-review clean.
download_layout and download_retention_days are registered ConfigSpec keys
(admin-editable on the Configuration page), but worker.py read them from
settings.* (env) at download/edit finalize time — so an admin's edit was a
silent no-op (files kept the old layout; TTL stayed at the env default). Read
them via sysconfig (_download_settings), which falls back to the env default
when there's no DB override. Same class as the Downloads B3 fix.
ruff clean, localdev boots, re-review clean.
Fix: short videos were auto-marked watched almost immediately (negative finish
margin). Cleanups: extract format.formatDate, reuse goPrev/goNext for the in-bar
buttons, memoize the savedPrefs cache read. Verified: tsc, knip, localdev healthy.
BUG (YB1): the auto-watch checkpoint marked a video watched once position
crossed `duration - FINISH_MARGIN` (10s). For clips shorter than ~10s that
threshold is negative, and for ~11-20s clips it's only a few seconds in, so the
5s checkpoint tick marked short videos watched almost immediately (clearing their
resume position). Cap the margin at half the clip: Math.min(FINISH_MARGIN, dur/2).
Cleanups (behavior-neutral):
- Extract format.formatDate(iso, lang) — the day/month/year toLocaleDateString
option object was duplicated verbatim in PlayerModal (fullDate) and VideoCard.
(ChannelPage's "joined" is month/year only, so it's left as-is.)
- The in-bar prev/next buttons now reuse goPrev/goNext instead of re-implementing
the step + bounds inline (they already exist for the side arrows + keyboard).
- Memoize savedPrefs so the ['me'] cache read + spread runs once, not on every
render (it only seeds the initial autoMode/loopMode state).
tsc green, knip clean, localdev boots healthy.
Cleanup (_remove_item dedup; drop dead Check import + plName shadow) plus
review-found bug fixes: reorder position-collision under concurrent edits,
push/sync/revert YouTube 403s now route to the Connect affordance, and
removeItem rolls back on failure. Verified: ruff, tsc, localdev boots healthy.
BUGS:
- push/sync/revert (PF1/PF2/PF4) swallowed every failure into a generic warning,
so a missing-write-scope 403 gave no hint and no "Connect your YouTube account"
affordance (unlike Channels). Route all three through notifyYouTubeActionError
(no wizard handle on this screen, so message-only for the 403).
- removeItem (PF5) reset the order optimistically then awaited the delete with no
catch: a failed call left the row gone locally (and an unhandled rejection).
Now resync via refreshAll on either path (api.req surfaces the error dialog).
CLEANUP:
- Drop the dead `Check` lucide import (PC6; only AddToPlaylist uses it).
- Use the module plName(detail) helper instead of the inline watch_later-vs-name
ternary that shadowed it in push/revert (PC7).
tsc green, localdev boots healthy.
BUG (PB1): reorder_items only repositioned the items present in the payload,
leaving any omitted item (e.g. one added concurrently in another tab) at its old
position — which then collides with the freshly assigned 0..N-1 range, so
get_playlist's order-by-position returns a nondeterministic/duplicated order.
Now the omitted items get fresh trailing positions (keeping their relative
order), guaranteeing unique contiguous positions.
CLEANUP (PC4): extract _remove_item(db, pl, video_id, mark_dirty=) mirroring the
existing _add_item — remove_watch_later and remove_item duplicated the same
find-by-(playlist,video)/delete/commit block.
Behavior-neutral for the normal full-payload reorder. ruff clean, localdev boots.
BUG (FB3): the channel page's Subscribe mutation had no onError, so a read-only
user (no YouTube write scope) clicking Subscribe got a silent 403 — nothing
happened, no toast. Wire notifyYouTubeActionError (no wizard handle here, so the
message without the Connect button). Block is a local-only action and can't 403
for scope, so it's left as-is.
BUG (FB4): subscribe.onSuccess invalidated channel/feed/channels but omitted
my-status and discovered-channels (which ChannelDiscovery invalidates for the
same action), so after subscribing from a channel page the Discovery list and
the manager stats stayed stale. Added both.
tsc green, localdev boots healthy.
