Backend (migration 0046_plex_people_search): a new people_text column (cast +
director names, de-duped) is folded into the generated search_vector at weight B,
so the top search box finds titles by an actor/director name and ranks them above
summary-only mentions. New /api/plex/people endpoint returns the cast/crew matching
the term (name prefix or word-start) with a film count + a headshot (pulled from one
representative film's live metadata; image bytes disk-cached).
Frontend: PlexBrowse shows matching people as virtual cards above the grid; clicking
one adds them to the actor/director filter (multi-value, from the C1 work) and clears
the search box so you land on exactly that person's films. Answers the 'why does
"drew" match Rambo?' confusion — it was matching the word in the synopsis; now names
are searchable. i18n en/hu/de.
⚠️ Prod needs a Plex re-sync after deploy to populate people_text (search_vector
regenerates automatically once the column has data).
Backend (migration 0045_plex_filter_meta): plex_items gains rating (audienceRating
~IMDb), content_rating, studio, originally_available_at, and GIN-indexed genres /
directors / cast_names — all mirrored from the cheap section listing (no per-item API
calls; they also seed a future watch-habit recommender). /browse gains genre / content-
rating / year / rating / duration / added-within / director / actor / studio filters
(@> containment, GIN) + sort by year|rating|duration|release; new /facets endpoint
returns available genres+ratings (with counts) and the year/rating/duration bounds. A
thin on-disk image cache (.plex-img-cache) serves posters/art/cast photos from local
disk after first fetch (~7-14x faster repeat loads).
Frontend: PlexSidebar grows the full filter set (facet-driven genre/age chips, rating
steps, year range inputs, duration buckets, added-within, active people/studio chips,
clear-all); filters persist per-account as one JSON blob. PlexInfo metadata (year,
genre, director, cast, studio, IMDb score) is clickable → sets the matching filter and
returns to the filtered grid (page variant only; the in-player overlay stays read-only
so a stray click can't stop playback). i18n en/hu/de.
Every download now records the clean source page URL and shows it on the
Downloads page (open in a new tab, or copy to clipboard). The worker stores
yt-dlp's canonical webpage_url on the asset (migration 0043 adds
media_assets.source_webpage_url); the serializer prefers it and falls back to a
URL derived from source_kind+source_ref, so YouTube, external YouTube links and
external URLs (e.g. Facebook reels) all get a correct reference, and queued/older
rows work before the worker fills it. Edit clips return null (a clip's source is
the user's own earlier download, not a web page). i18n en/hu/de.
Share a download by a capability URL (/watch/{token}) that anyone can play on a login-free page —
distinct from the registered-user ACL share. Per-link controls: optional expiry, allow-download
toggle (stream-only vs downloadable), optional argon2 password. Revoke = delete.
- migration 0042 + DownloadLink model; app/downloads/links.py (token, HMAC grant sign/verify)
- owner endpoints (POST/GET/PATCH/DELETE links) + /recipients for the internal user picker
- public router (no auth): watch meta / password unlock→signed grant / range-aware file serve
(inline vs attachment); unlock is rate-limited; meta verifies the file exists on disk
Verified end-to-end: 206 range, inline vs attachment, wrong-password 403, tampered-grant 403.
Noisy YouTube titles are cleaned for display + storage, raw kept in videos.original_title:
- app/titles.normalize_title: strip emoji/symbols (keep accents), remove trailing SEO hashtag
clusters (keep numeric #3 episode markers), context-aware de-shout (mostly-ALL-CAPS titles ->
Title Case with an acronym whitelist + function-word lowercasing; otherwise only long all-caps
words), collapse repeated punctuation
- applied at enrichment (sync/videos.py) and in the download worker (ad-hoc yt-dlp titles);
catalog downloads inherit the normalized title automatically
- migration 0039: add original_title, preserve raw, rewrite title (generated search_vector
regenerates); reversible via original_title
Backfill on localdev: 122115/273417 titles normalized in ~2 min. Verified in the feed + on
real messy samples (emoji/de-shout/hashtags), accents + acronyms (PS5/AI/USA/PC) preserved.
Per-user named snapshots of the feed's FeedFilters. New saved_views table
(migration 0035, unique per user+name, position, is_default) + SavedView
model, and a /api/saved-views router (list/create/patch/delete/reorder)
gated behind require_human so the shared demo account can't pollute it.
Setting is_default clears the flag on the user's other views (one default).
- Live-search results stay ephemeral: the discovery-cleanup job now also reclaims un-kept
via_search videos (no watch state / not playlisted / channel not subscribed) after a grace
period (search_grace_days), and POST /api/search/clear discards a given result set 'as if
never added' (drops the user's search-finds + deletes the now-orphaned, un-kept videos).
Admin POST /api/admin/purge-discovery runs it on demand (grace 0).
- Count selector: GET /api/search/youtube gains a 'limit' — the free scrape source pages
through continuations until that many results are gathered (no manual load-more); the API
source stays one ≤50 page (cost).
- Per-user channel blocklist (migration 0034 blocked_channels): block/unblock/list endpoints;
blocked channels' videos are dropped from live search before ingest and hidden from the feed
/ Library / explore / channel page; explore refuses a blocked channel.
- New-first ordering: results you already have (subscribed channel, watched/in-progress/saved/
playlisted) sink below genuinely-new discoveries, preserving relevance within each group.
Adds the backend for a dedicated channel page + ephemeral browsing of un-subscribed
channels:
- migration 0033: channels.{total_view_count,published_at,banner_url,external_links,
from_explore}, videos.via_explore, explored_channels (per-user, grace-clocked).
- enrich channels with part=brandingSettings (banner/links) + statistics.viewCount +
snippet.publishedAt — no extra quota.
- GET /api/channels/{id}: About detail + this user's relationship; lazy-enriches the new
About fields (published_at sentinel).
- POST /api/channels/{id}/explore (require_human, quota-reserve guarded): records the
exploration, flags the channel ephemeral unless followed, ingests one page of uploads
(via_explore) + enriches; returns next_page_token for load-more.
- feed: per-explorer gate — a via_explore video is visible only to users who explored or
subscribe to its channel, so exploration never leaks into everyone's catalog.
- subscribe = keep: clears from_explore + via_explore on the channel's videos.
- scheduler explore_cleanup job + explore_grace_days config: hard-delete explored-but-
unkept channels and their untouched ephemeral videos after a grace period.
Makes the local search behave more like YouTube's — finding videos by the
uploader's own keywords or the query that surfaced them, not only words in the
title. A DB-generated, weighted search_vector (migration 0032) replaces the
title-only FTS index:
- keywords: the creator's snippet.tags (free — already in the snippet we fetch),
stored on enrich.
- search_terms: distinct live-search queries that surfaced the video (across all
users), appended by the search route — folds YouTube's relevance into local
search (a video YT returned for a query becomes findable by it even without a
title match), the user's own idea.
- description (truncated) for broad recall on the existing catalog.
Weighted title(A) > keywords+queries(B) > description(C) so ts_rank keeps title
hits on top. A plain GIN index on the generated column guarantees index use (no
expression/param matching). Verified on localdev: recall 146->213 for one query;
7 'eurovision' hits via the document but not the title; index scan confirmed.
Two related search improvements:
1) Your own live-search results now belong to your Mine feed. A new per-user
search_finds table (migration 0030) records each video you surface via your
YouTube search (the route inserts them idempotently). The Mine feed becomes
'your non-hidden subscriptions OR your search finds', and the Source filter
now applies in Mine too: organic = subscriptions, search = your search finds,
all = both (default stays organic, so the main feed is unchanged). The shared
Library keeps using the global via_search flag.
2) Feed search ranks by relevance instead of a whole-phrase substring. A custom
unaccent_simple text-search config + GIN index (migration 0031) back a
YouTube-like fuzzy match: word-order-independent, multi-word AND, prefix on
the word being typed, accent-insensitive. A new 'relevance' sort orders by
ts_rank; the channel name still matches as a substring. The rank is scaled to
an integer so the keyset cursor pages it exactly (a raw float4 breaks paging).
_filtered_query returns the rank expr so only the feed list uses it.
Add a live YouTube search that materialises results into the shared catalog so
they render with the normal feed cards + in-app player and gain per-user state.
- YouTubeClient.search_videos(): search.list (100 units), embeddable-only, returns
flat stubs + nextPageToken; surfaces liveBroadcastContent for live filtering.
- routes/search.py GET /api/search/youtube: require_human + per-user daily cap
(search_daily_limit_per_user, default 70) + can_spend pre-check (429 on either);
drops live/upcoming, upserts channel stubs (channels.list) + video stubs, enriches
(videos.list), runs the youtube.com/shorts probe, then excludes Shorts/live and
returns feed cards in relevance order with the YouTube pageToken as the cursor.
- Provenance: videos.via_search / channels.from_search (migration 0028) flag
search-discovered rows; the feed hides them from the Library (scope=all) by default
via exclude_search_discovered, leaving the Mine feed untouched.
- quota.actions_today() counts a user's per-action events today for the cap; only the
search.list call is attributed VIDEOS_SEARCH so the counter is exactly 1 per search.
TimestampMixin (created_at) + UpdatedAtMixin (updated_at) replace the repeated
column blocks across 9 + 3 models. Models that index created_at or default it
Python-side keep their own column on purpose. Verified zero schema drift via an
identical alembic-check diff before/after.
Private user-to-user messages are end-to-end encrypted: the server only ever
stores ciphertext + iv and acts as a key directory (public keys) plus an opaque
store for each user's private key, wrapped client-side with a passphrase the
server never sees — so not even an admin can read a conversation. A separate
kind=system message (plaintext, no sender) powers a server-authored Siftlode
welcome shown on first open, reusable later for announcements.
- models: rework Message (kind, nullable sender/body, ciphertext+iv) + MessageKey;
migrations 0026 (table) + 0027 (E2EE rework).
- routes/messages.py: key directory/blob endpoints, ciphertext send, conversations
+ threads (system + user), lazy welcome, all gated by require_human.
- realtime.py: in-process WebSocket connection registry; /ws delivers sent
messages to a user's open tabs instantly (sync-callable push, single-process).
- app_state gains configured + setup_token_hash (migration 0025); existing installs (any users)
are marked configured so they never see the wizard, a fresh DB starts in setup mode.
- On first boot the app mints a one-time setup token and logs the wizard URL (only the hash is
stored); the token gates the setup steps (added in 6c).
- A middleware locks all /api + /auth routes (except /api/setup, /api/version, /healthz, static)
to 503 until configured, with a DB-free cached fast-path once setup completes.
- GET /api/setup/status tells the SPA whether to show the wizard.
- New Users page tabs (Users & roles / Access requests / Demo) and a tab-ified Configuration
page, both via a reusable Tabs component (persisted active tab).
- Admin can suspend/unsuspend (migration 0023 adds users.is_suspended) and delete accounts
from the Users & roles tab, with guards (demo / self / last admin).
- Email notifications to the affected user: suspended (reactive, on a blocked sign-in),
reinstated, role changed, and account deleted; the approval email now carries a clickable
app link. The scheduled/manual demo reset also seeds sample channel subscriptions.
- Hide the 'unverified email' chip for Google accounts (Google attests the email).
Add email+password auth alongside Google: argon2id hashing; users gains
password_hash/email_verified/is_active and google_sub becomes nullable (migration
0022); a single-use, hashed auth_tokens table for email verification + password
reset. Registration creates a pending (inactive, unverified) account + an access
request; sign-in needs verified email AND admin approval. Anti-enumeration: uniform
register/login/reset responses (a correct-password owner still gets a specific
pending reason). allow_registration flag (DB-tunable). Admin users-list + role
endpoint (guards: not self, not demo, not the last admin); approving access (or a
manual whitelist add) now activates the matching pending account. current_user
rejects deactivated accounts. Verified end-to-end via curl + DB.
Add a generic admin-editable config layer (epic 4a): system_config KV table
(migration 0021), a sysconfig registry that is the single source of truth for
DB-overridable keys (type/group/default/bounds/secret) + a DB-override-or-env
resolver, and admin endpoints (GET/PATCH/DELETE /api/admin/config + a test-email
probe). Secrets are Fernet-encrypted at rest (TOKEN_ENCRYPTION_KEY); without it
they stay env-only. First group wired end-to-end: Email/SMTP — email.py now
reads host/port/user/from/password via the resolver (own session, since email
is sent from background tasks).
Add a "Discover from playlists" tab to the Channel manager that lists
channels appearing in the user's playlists they don't subscribe to, with
a one-click Subscribe.
- GET /api/channels/discovery: local join (playlist_items -> videos ->
channels) minus the user's subscriptions and their own channel. Enriches
stub channels' metadata up front (title/thumbnail/subscriber count via the
API key) so the user can judge a channel before subscribing; videos are
not pulled (the scheduler picks those up).
- POST /api/channels/{id}/subscribe: write-scope gated, subscriptions.insert
+ local Subscription with the returned resource id.
- YouTubeClient.insert_subscription / get_my_channel_id.
- users.yt_channel_id (migration 0019) caches the user's own channel id so
discovery can exclude it.
- Frontend: ChannelDiscovery DataTable, Channels tab toggle (persisted),
api methods, trilingual strings. Subscribe ships a typed notification
payload (ChannelSubscribedMeta) for the inbox to act on.
Expose the maintenance job's rolling re-validation batch (videos re-checked per
run) as an admin control on the Scheduler dashboard, stored in app_state
(migration 0018; NULL = the env/config default). The job reads the effective
value each run via a state helper, clamped to a sane range. Trilingual.
Add a server-backed notification center that coexists with the client-side
transient bell: a per-user `notifications` table (type/title/body/data JSON/
read/dismissed), a `/api/me/notifications` CRUD API (list, unread_count, read,
read_all, dismiss, clear), and a left-nav inbox module with a live unread badge
polled via useLiveQuery. Known types render trilingual text from type+data
(English stored text is the fallback); read rows are trimmed past a soft cap.
Also adds the schema the maintenance job builds on: videos.list?part=status
columns (embeddable/privacy_status/upload_status) and the validation lifecycle
columns (last_checked_at, unavailable_since, unavailable_reason).
Page-id validation is centralized in one PAGES source of truth (isPage) so the
new page survives reload without a second allowlist to keep in sync.
Add scheduler_settings (per-job interval override, migration 0015) and a
PATCH /api/admin/scheduler/jobs/{id} that persists the override and live-
reschedules the running APScheduler job. Intervals load from the DB (env
defaults as fallback); the snapshot reports the live trigger interval.
Add the shared demo-account plumbing: users.is_demo marks the single
shared demo user, demo_whitelist holds the admin-curated emails that may
enter it without Google sign-in, and a small in-process RateLimiter
(generic groundwork) for throttling the demo-login endpoint per IP.
Store a SHA-256 of each playlist's name + ordered video ids as last synced from /
pushed to YouTube (migration 0013, backfilled for already-clean linked playlists).
'dirty' is now recomputed on every edit by comparing the current fingerprint to that
baseline instead of being stickily set true. So manually restoring the original order
(or undo back to it) clears dirty on its own — the Reset to YouTube button disappears
and no YouTube read is needed. Baseline is set on read-sync, repull (reset), and a
clean push; a missing baseline counts as dirty (unknown YT state). Verified the
migration's fingerprint matches the app's runtime fingerprint exactly.
Add per-user local playlists: Playlist + PlaylistItem models (with forward-looking
kind/source/yt_playlist_id/dirty columns for the later YouTube-sync phases) and
migration 0011. New /api/playlists routes: list (with optional contains=<video_id>
membership flags for the add-to-playlist popover), create, get detail (items
serialized via the feed serializer, with per-user watch state), rename, delete,
add/remove item, and reorder. Watch later (built-in) is protected from deletion.
Add position_seconds (+progress_updated_at) to video_states so watch progress
survives across devices and can drive a feed filter. New POST
/api/videos/{id}/progress checkpoints the player position (clearing trivially
-early and near-finished positions). Feed serialize exposes position_seconds and
a show=in_progress filter lists started-but-unfinished videos. Un-marking
'watched' now keeps a stored position instead of deleting the row.
Track who burned how much YouTube API quota. A QuotaEvent audit log (migration
0009) records every spend with the triggering user (NULL = background/system) and
an action label, set via a request/job-scoped contextvar (quota.attribute) so no
call signatures change. User-initiated work (sync subscriptions, unsubscribe,
opt-in recent backfill, manual enrich) attributes to the user; scheduler work to
System, split by action.
- backend: QuotaEvent model + migration 0009; quota.attribute() contextvar;
record_usage logs events; entry points wrapped (routes/sync, routes/channels,
scheduler); GET /api/quota/my-usage + GET /api/quota/admin
- frontend: admin-only Stats page (header nav, page=stats) with daily bars +
per-user breakdown by action and range picker; 'Your API usage' in Settings ->
Sync for every user
Verified: attribution + endpoints compute correctly; events are per-user vs System.
Move the access whitelist from the ALLOWED_EMAILS env var into a DB Invite table
(env kept as bootstrap fallback), and add a self-service request + admin approval
flow with fail-soft email.
- models: Invite(email, status pending|approved|denied, requested_at, decided_*)
- migration 0008: invites table; seed env ALLOWED_EMAILS u ADMIN_EMAILS as approved
- auth: is_allowed() (DB-first, env fallback); a denied Google login records a pending
request and bounces to /?access=requested instead of a raw 403; public POST
/auth/request-access; upsert is idempotent so repeats don't re-spam admins
- routes/admin.py (admin-only): list/approve/deny invites + manual add
- email.py: smtplib + Gmail App Password, fail-soft (skips if SMTP unset)
- /api/me exposes pending_invites; config + .env.example gain SMTP_*
- UI: Login 'Request access' form + access=requested/denied handling; Settings ->
Access requests (approve/deny + add); admin nudge toast on pending requests
Verified locally: request-access creates a pending invite and emails the admin;
seed approved npeter83; guinea-pig yt.trash2023 denied until approved.
Per-user opt-in to full-history (deep) backfill so a new user's unique
channels no longer trigger a big shared-quota burst.
- migration 0007: Subscription.deep_requested (default false); seed admins'
existing subscriptions to preserve today's global-backfill behaviour
- run_deep_backfill is now demand-driven: only channels at least one user has
requested are deep-backfilled; recent backfill stays unconditional (cheap)
- estimate_deep_backfill ETA helper (quota-bound) surfaced in /api/sync/my-status
- POST /api/sync/deep-all to opt all my channels in; PATCH channels accepts
deep_requested
- UI: per-channel Full history toggle, Backfill everything action, deep
progress + ETA in Channels header and Settings - Sync
- Header shows a live sync status (total videos + channels still backfilling, or
"paused"), polled every 30s
- Admins can pause/resume the background sync; a paused flag in app_state makes
scheduled jobs skip (migration 0006)
- GET /api/feed/count returns the number of videos matching the current filters;
shared filter builder keeps it in sync with /api/feed; shown above the feed
- /api/sync/status reports backfill progress, pending enrichment and paused state
- Shorts: confirm via youtube.com/shorts/<id> probe (SOCS cookie bypasses the
consent redirect) instead of a 60s heuristic; concurrent probing, shorts_probed
flag, scheduled refinement (migration 0005)
- Search: match title + channel name only (descriptions caused noisy results)
- Faceted tag filtering: AND across categories (language AND topic narrows),
OR within a category; any/all toggle applies to topics
- Language detection: majority vote over individual titles (fixes misdetections
like multipoleguy -> English; drops bogus Polish/Romanian)
- Login: drop forced consent so returning sign-in is quick (select_account)
- Feed cards: clickable channel name (opens channel), persistent saved badge,
undo toast on hide, Hidden view to restore; tag chips show counts in tooltip
- Tag and ChannelTag models + migration 0003 (partial unique indexes split
system vs per-user tag names)
- Offline language detection (py3langid) constrained to a curated language set,
with the channel's declared default language as a strong prior
- Topic tags mapped from YouTube topicDetails + dominant video category; the
generic "Lifestyle" catch-all is intentionally dropped
- System (auto) tags are regenerated idempotently and never touch user tags;
orphaned system tags are cleaned up
- GET /api/tags and admin POST /api/tags/recompute; scheduled autotag job
- Channel/Subscription/Video/ApiQuotaUsage models + migration 0002
- Synchronous YouTube Data API client with OAuth token refresh and per-call
quota accounting (API key preferred for public reads when configured)
- Subscription import: upsert channels + subscription links, prune unsubscribed,
fetch channel details (uploads playlist, topics, stats, handle, country)
- Central quota guard tracking units per US-Pacific day against a daily budget
- POST /api/sync/subscriptions and GET /api/sync/status endpoints
- docker-compose with Postgres 16 + slim Python API image
- FastAPI app with session middleware, health endpoint, static login page
- Google OAuth (Authlib) with email invite-list whitelist; admin role support
- User + OAuthToken models; refresh tokens encrypted at rest (Fernet)
- Alembic migrations, run automatically on container startup
- Postgres backup/restore scripts for portability between machines
