Verified every deferred per-subsystem item against current source (8 parallel
agents), then resolved all of them: 20 fixed, 14 WONTFIX (recorded with rationale
in siftlode-ops/CODE-HYGIENE.md), 2 no-op (already-shipped / already-mitigated).
ruff+tsc+knip green; backend runtime-smoke-tested; /code-review (high) findings all
addressed; SB2 confirm-dialog E2E-verified. Detail in the 3 branch commits.
- channels CB3: sort discovery rows NULLs-last (title is None) to match the DB
ORDER BY exactly — coercing NULL title to "" floated untitled channels to the top.
- AdminUsers SC1: track in-flight rows in a Set, not a single id — per-row disabling
now lets the admin start concurrent row actions, and a shared id was cleared by
whichever settled first (re-enabling a still-pending row → possible double-submit).
- ChatThread CT4: reset the incoming-count ref when partnerId changes — the Messages
page reuses one ChatThread across conversation switches, so a same-count switch
could skip the open-marks-read badge refresh.
- messages MB3: push a bare {type:"unread"} (drop the now-dead count query — the
client re-fetches on the signal and ignored the number anyway).
- api.ts: extract the shared keepalive `beacon()` helper (saveProgressBeacon +
plexProgressBeacon were near-verbatim copies).
- player YB2: keepalive pagehide beacon (api.saveProgressBeacon) so the resume
position isn't lost on F5/close within 5s of the last checkpoint (mirrors Plex).
- player YB4: bound consecutive unplayable items in auto-advance so an
auto-advancing/loop=all queue can't spin over a run of dead videos.
- downloads B6 (client): localise structured quota/edit errors centrally in
api.req() via localizeDetail() + Intl.NumberFormat (fixes HU/DE + decimal sep);
+ 3-locale download error strings.
- channels FB1: focusChannelToken bump so re-clicking the same channel re-seeds
the search box; FB2: syncSubs invalidates feed/feed-count (set can change now).
- config CB2: reseed the ConfigPanel draft on a key-set dataVersion + explicit
post-save token, so a mid-edit refetch can't clobber an in-progress edit.
- config AB3 (UI): a blank allow_empty field stores "" instead of resetting.
- admin SB2/SC2: confirm + success toast on demo-whitelist remove and deny-invite
(+ trilingual strings); SC1: disable only the acted-on row (pendingId), not all;
SB3: pause the 1s scheduler countdown ticker while the tab is hidden.
- messages CT4: only invalidate conversations/unread when the incoming-message
count actually changed, not on every 20s poll; MB3 (client): react to the live
unread ping (onUnread → invalidate); MC3: extract e2ee installKey (setup/unlock);
CC2: hoist POLL_MS to messaging.ts; MB4: document the reload-scoped socket.
Verified each deferred per-subsystem item against current source, then fixed the
real ones (tradeoffs left as-is, see siftlode-ops/CODE-HYGIENE.md closeout):
- search BUG-3: don't finalise shorts_probed on un-enriched stubs (enrichment
failure/omitted rows) — restores the scheduler's enriched_at guard so a real
Short can't leak into the feed and never get reclassified.
- downloads GC: 4th pass reaps orphaned errored, fileless, unreferenced assets
(they carry no TTL, so the expiry passes never cleared them).
- downloads B6: quota/edit errors now return a structured {code,reason,limit}
the trilingual client localises, instead of hardcoded English + dot-decimal GB.
- channels BB1: reset-backfill re-arms deep sync on every subscriber (bulk update)
instead of 404ing when the admin isn't personally subscribed.
- channels BB2: enrich the stub on BOTH the normal and "already exists" desync
path (nest the insert try inside the client `with`).
- channels CB3: drop the redundant second _discovery_rows read (identity-mapped
rows are already enriched; re-sort in Python for the title tie-break).
- playlists PC1: read the live playlist once in push() and share it with plan_push
+ push_playlist (halves read-quota, closes the second TOCTOU window).
- playlists PC2/PC5: batch the cover thumbnails in one window query (was N+1);
rename combines count+duration into one aggregate + a single cover lookup.
- messages MB2: get_thread only resolves a messageable partner OR one we already
share a thread with (closes the user-enumeration oracle).
- messages MB3: push a live unread-count to the user's other tabs after mark-read.
- messages MC4: list_conversations uses DISTINCT ON + grouped COUNT instead of
loading the whole message history into memory.
- config AB3: per-spec allow_empty so smtp_user/youtube_api_proxy can be blanked
to disable them rather than snapping back to the env default.
Two user-reported signin bugs + a review-surfaced revoke fix:
- A logout→login no longer wipes a user's YouTube grant: _store_token keeps the whole
existing grant when a base-scope sign-in would narrow the stored YouTube scopes (the
old refresh token stays valid on Google's side; overwriting it was the loss).
- Admin 'new access request' email: correct menu path (Users → Access requests), the
requester address spelled out + a deep-link to the approve view, and it now notifies
the ACTUAL admins (active role=admin) unioned with env, not just the static env list.
- purge_user's Google-revoke fallback decrypts the access token (was sending ciphertext).
purge_user's revoke used `decrypt(refresh) or tok.access_token`, but access_token is
stored ENCRYPTED — so a token row without a refresh token would send ciphertext to
Google's revoke endpoint and silently fail. Decrypt it. (Pre-existing; surfaced by the
review of the OAuth-scope fix.)
Correcting the earlier scope-union-only fix. The real damage from a logout→login isn't
just the stored `scope` field — verified via a live token refresh that the stored refresh
token itself had been REPLACED with a base-only one: a plain sign-in requests only
BASE_SCOPES and Google handed back a fresh base-only access+refresh token, which
_store_token adopted verbatim, destroying the read/write grant (the old refresh token
stays valid on Google's side, so overwriting it is what loses access).
_store_token now detects when an exchange would NARROW our YouTube scopes and, in that
case, keeps the WHOLE existing grant (refresh token, access token, scopes) untouched.
Broader-or-equal exchanges (first grant, read→write upgrade) adopt + union as before.
Unit-verified across base-relogin / upgrade / new-user / base-user cases.
Two user-reported signin bugs:
1) A plain re-login (or logout→login) wiped the user's YouTube grant: login requests
only BASE_SCOPES and Google's returned `scope` can list just those, but _store_token
wrote it verbatim — dropping a previously-granted youtube.readonly/youtube scope, so
can_read flipped to false and the feed demanded a reconnect. The underlying grant
(refresh token) survives such a login, so UNION the scopes instead of narrowing; they
only shrink on full disconnect (purge deletes the token row).
2) Admin 'new access request' email: (a) it named the wrong menu ('Settings → Account'
instead of Users → Access requests); (b) the requester address was invisible so the
'reply reaches them' note looked wrong (Reply-To is in fact set to the requester) —
now spelled out + a deep-link straight to the approve view (?admin=access-requests,
handled in App); (c) it emailed only the static env ADMIN_EMAILS — now notifies the
ACTUAL admins (active role=admin users) unioned with env, so a UI-promoted admin (who
CAN approve — the approve UI is role-gated) is notified too.
register + password-reset-request move their lookup/create/token work off the response
path (background task, own session) so response timing no longer reveals whether an
email is registered. messages_ws shares resolved_user_id (now HTTPConnection-typed)
instead of re-implementing the per-tab wallet-gated resolution. Behavior-preserving;
reviewed (no security regression) + verified.
resolved_user_id already accesses ws.session unguarded just above (SessionMiddleware
covers the WS scope), so the '"session" in ws.scope' fallback was dead code — read
ws.session directly for consistency (review follow-up).
Loose ends to finish the auth security round:
- register + password-reset-request had an enumeration TIMING oracle: an already-
registered email skipped the create path (hash + row writes + email scheduling) and
responded measurably faster than a new one. Move the whole lookup+create (register)
and lookup+token+email (reset) into a background task with its own DB session, so the
endpoint returns in the same time for any valid email regardless of whether it exists.
Verified: existing vs new now ~equal (was 34ms vs 82ms on register); accounts/tokens
still created off-path.
- messages_ws re-implemented resolved_user_id's per-tab wallet-gated account resolution.
Generalize resolved_user_id to take any HTTPConnection (Request OR WebSocket) and call
it from the WS — one shared, wallet-gated resolution. Behavior-identical (unit-checked).
Auth security round: SB3 (email tokens out of URL query → fragment), SA4 (server-side
session revocation + 'Log out other sessions'), SA3 (trusted-proxy X-Forwarded-For so
rate limits can't be bypassed via a forged header).
Post-review hardening (both fail-safe, not bugs):
- _client_ip normalizes IPs via ipaddress and unwraps IPv4-mapped IPv6, so a plain
TRUSTED_PROXY_IPS=10.10.0.1 also matches a peer surfaced as ::ffff:10.10.0.1 (a
Docker/IPv6 self-host footgun that would otherwise silently collapse everyone into
one rate-limit bucket).
- entrypoint.sh + _client_ip docstring: explicit warning never to add uvicorn
--proxy-headers, which would rewrite request.client from the forgeable XFF and defeat
the trust check.
_client_ip trusted the first X-Forwarded-For hop unconditionally, so anyone able to
reach the app port could forge XFF and dodge the login/register/reset/demo rate limits.
Now trust XFF ONLY when the request's socket peer is a configured reverse proxy
(settings.trusted_proxy_ips, e.g. the VPS Caddy's WireGuard peer IP), and take the
RIGHTMOST entry — the client our proxy actually saw and appended, immune to a client
pre-seeding a fake XFF. A request from any other peer (hitting the port directly) is
keyed on its real socket IP, so XFF can't be forged to bypass the limits.
New TRUSTED_PROXY_IPS env (empty default = no proxy, use the direct peer). Documented in
.env.example, docs/self-hosting.md, README. Unit-verified against spoof-through-proxy and
direct-bypass cases.
SB3: reset/verify email tokens moved from URL query strings to the fragment (never
sent to the server / proxy logs / Referer); verify GET→POST with a legacy-GET fallback.
SA4: server-side session revocation via per-user session_epoch (migration 0053) — a
signed cookie records its epoch, current_user + the messages WS reject a stale-epoch
cookie. Bumped on password reset (all sessions), password change + a new 'Log out other
sessions' action (both keep the current session). E2E-verified; UAT passed.
SA3 (trusted-proxy for X-Forwarded-For) intentionally deferred to a prod-tested follow-up.
Adversarial re-review of the session-epoch work surfaced:
- WS auth (messages_ws) skipped the epoch check, so a revoked-but-unexpired cookie
could still open the live push channel after a reset/logout-others. Now mirrors
current_user: loads the user once, rejects a stale-epoch cookie before connecting.
- set_password + logout_others re-stamped the cookie BEFORE db.commit(); a failed
commit would strand the current session at a newer epoch than the DB and wrongly
401 it. Commit first, then re-stamp.
- Welcome verify effect could double-POST the single-use token (StrictMode/remount)
and flip the banner to a false 'invalid'. Fire-once useRef guard.
Left as-is (low value, documented): the Plex image proxy authenticates without a DB
load / epoch check (poster/art fetches only); adding one would cost a DB hit per image.
Signed client-side session cookies had no server-side kill switch: logout + password
reset couldn't evict a stolen/copied cookie (valid until expiry). Add User.session_epoch
(migration 0053), record it in the cookie at every login, and reject in current_user any
cookie whose recorded epoch is behind the account's current one.
- Bump the epoch on: password reset (kills ALL sessions — a reset is a compromise response),
password change + a new 'Log out other sessions' action (both re-stamp the CURRENT cookie
so the caller stays signed in, evicting only the others).
- Per-account epoch map in the session so one account's revocation doesn't evict the other
signed-in accounts in the same browser wallet.
- Missing epoch (pre-SA4 cookie) is treated as 0, so the first bump revokes grandfathered
sessions too.
- New POST /auth/logout-others + a Settings → Account 'Active sessions' button (trilingual).
Secret email tokens now ride the URL fragment (#reset=/#verify=), never the query
(?reset=/?token=): a fragment isn't sent to the server, so the token can't leak into
proxy/access logs or a Referer header.
- Reset: link → /#reset=; the SPA reads the token from location.hash and POSTs it
(unchanged /password-reset/confirm).
- Verify: link → /#verify=; new POST /auth/verify (token in body). The legacy GET
/auth/verify?token= is kept so pre-deploy emails in flight still work until they
expire. The SPA reads the fragment token, POSTs it, shows ok/invalid.
- Welcome: read secret tokens from the fragment, status flags from the query; strip
both after capture so nothing lingers in history.
Add the missing collapse trigger to the expanded PlexSidebar (state + collapsed rail
already wired via the shared filterCollapsed); mirrors the feed Sidebar. E2E-verified.
The collapse feature was only half-present on the Plex page: App already passed the
shared filterCollapsed state + toggle to PlexSidebar, and PlexSidebar rendered the
CollapsedFilterRail when collapsed — but the EXPANDED PlexSidebar had no trigger, so
you could only collapse it by first collapsing on the feed page. Add the same
ChevronLeft 'Filters' collapse header the feed Sidebar has (reusing the existing
sidebar.* i18n keys). E2E-verified: collapse→rail→expand round-trips on the Plex page.
- CollapsedFilterRail: the byte-identical 31-line collapsed sidebar rail was in
both Sidebar (feed) and PlexSidebar → one shared component.
- VideoCard: the title + channel-button + meta block was duplicated across the
list-row and card layouts → a local `textBlock` element (only one layout branch
renders, so reusing the element is safe).
- subsColumn<T>(t): the identical 'subscribers' DataTable column in Channels +
ChannelDiscovery → a generic factory in channelColumns.tsx.
All behavior-neutral; jscpd now reports 0 clones (was 8 at Phase 0 baseline).
tsc + knip green.
The standard input style was copy-pasted verbatim into 5 components
(DownloadCenter/DownloadDialog/ProfileEditor/ShareDialog/VideoEditor) and the
text-button style into 2 (ShareDialog/VideoEditor). Export both from ui/form.tsx
and import them. Byte-identical strings → no visual change.
The settings-family panels (Settings/Config/Setup/Welcome) use a DIFFERENT input
style (bg-card/rounded-xl/focus:border-accent); unifying the two is a design call
left to the glass-consistency epic, not this DRY extract.
Callback-ref the grid's IntersectionObserver sentinel so it re-attaches on the
grid's unmount/remount cycle (drill into info/show/season → Back), instead of
observing a detached old node and freezing the feed at the loaded pages.
The unified-library grid unmounts when you open an item's info/show/season page
and remounts as a NEW element on Back. The IntersectionObserver effect keyed only
on [hasNextPage, isFetchingNextPage, fetchNextPage] — none of which change across a
pure navigation — so it never re-ran: the observer kept watching the detached old
sentinel node and fetchNextPage never fired again, leaving the feed frozen at the
already-loaded pages (e.g. 80/1227) even though more exist. Intermittent because a
coincident isFetchingNextPage toggle around the nav could re-run the effect.
Fix: observe via a callback ref (state) instead of useRef, so the effect re-runs on
every sentinel mount/unmount and always watches the live node.
Closes the Plex #9 backend backlog (frontend was the prior follow-up). Behavior-
neutral cleanup + two two-way-sync bug fixes; PW2 verified a non-bug.
fix(plex): watch-sync PW1 pagination + PW3 lost-update guard
- PW1 watch_history: the history is a GLOBAL cross-account feed read as a single
500-row page, then filtered to the owner — on a busy family server the owner's
recent views can sit past page 1 and be missed. Now paginate (desc) until the
since-cutoff, bounded by max_pages (daily reconcile is the backstop; logs if hit).
- PW3 push_state_to_plex: it flagged synced_to_plex=True unconditionally after a
push, so a local edit landing between the push and the flag-set was marked clean
and never pushed (lost update). Capture the row's freshest timestamp when the
push is scheduled (_push_watch) and only settle the flag if the row is unchanged.
- PW2 was flagged as "accountID hardcoded to 1" but is NOT a bug: on a PMS account 1
is reserved for the owner/admin and an owner link always holds the admin token —
added a clarifying comment so it isn't re-flagged.
chore(plex): backend dedup
- PC2 unified_library + facets shared a ~13-field filter Query signature + p-dict →
one LibraryFilters dataclass injected via Depends(); as_dict() feeds the builders.
- PS-C1 sync.py collection upsert dup (resync_collection ≈ _sync_collections) →
_upsert_collection().
- PS-C2 sync.py 8-field filterable-metadata block dup (_apply_item ≈ _sync_shows) →
_apply_facet_fields().
- PW-C1 _repush_dirty read duration from the mirrored PlexItem.duration_s instead of
a per-row plex.metadata() round-trip.
- PW-C2 rk→id map built inline in two places → _rk_to_id() helper.
- PP1: reset cancelledMarkerRef on item change — a cancelled intro/credits
auto-skip on one episode suppressed a same-offset marker on the next (auto-skip
silently dead for the rest of a binge).
- PP2: loadSession no longer depends on the i18n `t` (read via a ref). A
mid-playback language switch rebuilt loadSession → re-ran the detail effect →
reloaded the session from the STALE saved resume position, losing live progress.
- PP3: tracks (audio/subtitle) menu now stopPropagation on wheel, like the gear
menu — scrolling the list no longer changes volume.
- PS1: collapsed PlexSidebar filter badge clamps to "9+" (was a raw count),
matching the feed Sidebar.
- format.ts: new formatRuntime() ("1h 30m"); PlexBrowse dur() + PlexInfo
fmtDur() were byte-identical copies, now both call it.
- PlexPlayer fmt() delegates to formatDuration (keeps the NaN/negative clamp
the live media clock needs); local reimplementation gone.
- Register LS.plexPlayerPrefs; PlexPlayer uses it instead of a bare string.
- Remove the dead PlexPlayerPrefs.wasPlaying field (written on play/pause, never
read) + its two patchPrefs writes.
- PlexBrowse item page: drop the redundant onStateChange={() => q.refetch()} —
PlexInfo.setState already invalidates ["plex-item", id], so q refetches itself.
- plex.py: delete the no-op _enabled() (never wired as a Depends/called; the real
gate is sysconfig plex_enabled).
- PlexBrowse.tsx: collapse toggleEpisodeWatched into toggleWatched — they were
byte-identical (both take a PlexCard).
- Delete 6 dead plex.json keys (loadMore, playerSoon, filter.library,
playlist.up/down/remove) from en/hu/de — verified 0 refs (the live
playlist.removeShow/removeSeason keys are kept).
Behavior-neutral. tsc green, ruff clean on touched files, localdev boots, Plex renders.
stream.py enforced a hardcoded _MAX_SESSIONS = 4 and never read the
DB-overridable plex_max_transcodes ConfigSpec, so the Configuration → Plex →
"Max concurrent transcodes" knob was dead (same env-vs-DB drift class as the
Downloads/Admin fixes). _enforce_cap(cap) now takes the cap from
sysconfig.get_int(db, "plex_max_transcodes") (>=1 so playback can't be capped to
zero). Bumped the config default 1→4 so the effective default matches the old
hardcoded cap (no behavior change for non-overriders; the knob now works).
From the auth /security-review (contained fixes; architectural SA3 proxy-trust +
SA4 session-revocation deferred to the user):
- SB1: encrypt the OAuth access_token at rest (was plaintext while refresh_token
was encrypted) — a ~1h Google bearer credential. New security.decrypt_optional()
falls back to a refresh for legacy plaintext tokens; column is unbounded String.
E2E-verified: token refreshed → stored as Fernet ciphertext → 319-subscription
YouTube sync succeeded.
- SA5: password_login is no longer a timing/enumeration oracle — it always runs
argon2 (against a decoy hash for unknown/passwordless emails), so response time
can't reveal whether an account exists.
- SB2: Google login only ADOPTS+activates a pre-existing password account when
Google actually attests email_verified (defense-in-depth against takeover); and
the email sync won't overwrite with a value another account owns (avoids a 500).
- demo_login clears the wallet first (demo can't switch to / act as a real account
via the multi-account header); switch_account rejects a suspended target (which
would otherwise clear the whole session on the next request).
Re-review clean; ruff clean; localdev boots; YouTube auth path E2E-verified.
set_user_role (demote) and admin_delete_user counted ALL admins incl. suspended
ones, so with e.g. 1 active + 1 suspended admin you could demote/delete the only
ACTIVE admin — leaving a single suspended admin who can never sign in → permanent
admin lockout needing DB surgery. Now both guards mirror the (already-correct)
suspend guard: block only when the target is an active admin AND it's the last one
(`not target.is_suspended and count_admins(active_only=True) <= 1`) — which also
correctly still allows removing a SUSPENDED admin while one active admin remains.
Resolves the deferred clearDevice security gap: logout never removed the E2EE
private key from IndexedDB, so on a shared machine the conversations stayed
decryptable after sign-out (ChatDock auto-unlocks from the persisted key next
visit). Per the user's decision (clear on EVERY logout), NavSidebar.logout() now
awaits e2ee.clearDevice(me.id) before reload — guarded so an IndexedDB failure
(e.g. private mode) can never block the logout. Re-entering the message
passphrase is required on the next visit.
Resolves the Phase-1 NEEDS-DECISION #3 (clearDevice) + retires the last e2ee
unused export (knip now only flags loadDefaultViewFilters). tsc green, re-review
clean (found + fixed an idb-open throw path blocking logout).
Fix: AES-GCM nonce reuse in e2ee.setup() (+ drop redundant key_check oracle,
backward-compatible); silent 404/429 send failures now toast; scroll/render-churn
in ChatThread + Messages; chatDock LS helper; deleted dead lock(). Verified: tsc,
re-review clean, localdev boots, self-E2E green (unlock + full-thread decrypt on a
real existing key, user entered the passphrase).
- ChatThread send had no onError: a 404 (recipient gone) or 429 (rate-limited)
send failed silently (api.req shows no modal for those "caller-handled" statuses).
Added onError → toast for 404/429 (400/409/500 already raise the global dialog);
draft is kept for retry. New trilingual messages.sendFailed key.
- ChatThread scroll-to-bottom effect keyed on `plain` too, so every 20s poll's
decrypt pass yanked a scrolled-up reader back to the bottom. Key on items.length.
- ChatThread + Messages decrypt effects keyed on `items` (a fresh `?? []` array
every render) → a render storm while loading. Key on q.dataUpdatedAt.
- chatDock.ts: use LS.chatDock(userId) instead of the bare "siftlode.chatDock.*"
literal (matches the storage LS registry).
tsc green, re-review clean, localdev boots, Messages renders (locked-state) w/o console errors.
setup() encrypted BOTH the wrapped private key AND the key_check under the same
wrapKey with the SAME wrapIv — a GCM nonce-reuse bug (leaks keystream + enables
GHASH/tag forgery, which matters under E2EE's malicious-server threat model).
Fixes:
- setup() gives key_check its own independent nonce (checkIv).
- unlock() no longer decrypts key_check to verify the passphrase — that was
redundant (a wrong passphrase already fails the wrapped_private_key GCM auth
tag) AND was the second consumer of the reused nonce.
Backward-compatible: the uploaded bundle shape is unchanged, and existing users'
bundles still unlock (unlock only presence-checks key_check now; the private-key
decrypt path with wrap_iv is untouched). Also removed the unused, architecturally
ineffective lock() export (re-unlocks from IndexedDB on next mount; the meaningful
primitive is clearDevice). Re-review clean; tsc green.
Bugs:
- ConfigPanel (CB1): unchecking a Plex library from the "all" (empty) state made
that library the ONLY selected one instead of all-except-it, because the toggle
ADDED the key to an empty list. Expand "all" to the explicit section set before
toggling. E2E-verified: uncheck from all → all-except-one.
- Scheduler (SB1): "Purge discovery" bulk-deleted search videos/videos/channels
immediately with no confirm (every AdminUsers destructive action uses useConfirm).
Wrapped in a danger useConfirm dialog (+ trilingual purgeConfirmTitle key).
E2E-verified: dialog appears, Cancel aborts.
Cleanups (behavior-neutral):
- Register LS.configTab; ConfigPanel uses it instead of the bare "siftlode.configTab".
- ConfigPanel + SettingsPanel import SaveState from ui/DraftSaveBar instead of
redeclaring the union (PrefsSaveState is now an alias).
tsc green, re-review clean, localdev boots.
- Extract runner.token_users(db): the "all users with a stored refresh token"
query was spelled 3× (get_service_user, run_subscription_resync,
sync_all_playlists). get_service_user now reuses it ([0]); sync_all_playlists
imports it from runner (no cycle — runner doesn't import playlists); the now-
unused OAuthToken import is dropped there.
- scheduler.py: _job("rss_poll", lambda db: run_rss_poll(db)) → _job("rss_poll",
run_rss_poll) — the lambda was dead indirection; all sibling jobs pass the
callable directly and _job calls fn(db).
Behavior-neutral. ruff clean, localdev boots, re-review clean.
