TimestampMixin (created_at) + UpdatedAtMixin (updated_at) replace the repeated
column blocks across 9 + 3 models. Models that index created_at or default it
Python-side keep their own column on purpose. Verified zero schema drift via an
identical alembic-check diff before/after.
Private user-to-user messages are end-to-end encrypted: the server only ever
stores ciphertext + iv and acts as a key directory (public keys) plus an opaque
store for each user's private key, wrapped client-side with a passphrase the
server never sees — so not even an admin can read a conversation. A separate
kind=system message (plaintext, no sender) powers a server-authored Siftlode
welcome shown on first open, reusable later for announcements.
- models: rework Message (kind, nullable sender/body, ciphertext+iv) + MessageKey;
migrations 0026 (table) + 0027 (E2EE rework).
- routes/messages.py: key directory/blob endpoints, ciphertext send, conversations
+ threads (system + user), lazy welcome, all gated by require_human.
- realtime.py: in-process WebSocket connection registry; /ws delivers sent
messages to a user's open tabs instantly (sync-callable push, single-process).
- app_state gains configured + setup_token_hash (migration 0025); existing installs (any users)
are marked configured so they never see the wizard, a fresh DB starts in setup mode.
- On first boot the app mints a one-time setup token and logs the wizard URL (only the hash is
stored); the token gates the setup steps (added in 6c).
- A middleware locks all /api + /auth routes (except /api/setup, /api/version, /healthz, static)
to 503 until configured, with a DB-free cached fast-path once setup completes.
- GET /api/setup/status tells the SPA whether to show the wizard.
- New Users page tabs (Users & roles / Access requests / Demo) and a tab-ified Configuration
page, both via a reusable Tabs component (persisted active tab).
- Admin can suspend/unsuspend (migration 0023 adds users.is_suspended) and delete accounts
from the Users & roles tab, with guards (demo / self / last admin).
- Email notifications to the affected user: suspended (reactive, on a blocked sign-in),
reinstated, role changed, and account deleted; the approval email now carries a clickable
app link. The scheduled/manual demo reset also seeds sample channel subscriptions.
- Hide the 'unverified email' chip for Google accounts (Google attests the email).
Add email+password auth alongside Google: argon2id hashing; users gains
password_hash/email_verified/is_active and google_sub becomes nullable (migration
0022); a single-use, hashed auth_tokens table for email verification + password
reset. Registration creates a pending (inactive, unverified) account + an access
request; sign-in needs verified email AND admin approval. Anti-enumeration: uniform
register/login/reset responses (a correct-password owner still gets a specific
pending reason). allow_registration flag (DB-tunable). Admin users-list + role
endpoint (guards: not self, not demo, not the last admin); approving access (or a
manual whitelist add) now activates the matching pending account. current_user
rejects deactivated accounts. Verified end-to-end via curl + DB.
Add a generic admin-editable config layer (epic 4a): system_config KV table
(migration 0021), a sysconfig registry that is the single source of truth for
DB-overridable keys (type/group/default/bounds/secret) + a DB-override-or-env
resolver, and admin endpoints (GET/PATCH/DELETE /api/admin/config + a test-email
probe). Secrets are Fernet-encrypted at rest (TOKEN_ENCRYPTION_KEY); without it
they stay env-only. First group wired end-to-end: Email/SMTP — email.py now
reads host/port/user/from/password via the resolver (own session, since email
is sent from background tasks).
Add a "Discover from playlists" tab to the Channel manager that lists
channels appearing in the user's playlists they don't subscribe to, with
a one-click Subscribe.
- GET /api/channels/discovery: local join (playlist_items -> videos ->
channels) minus the user's subscriptions and their own channel. Enriches
stub channels' metadata up front (title/thumbnail/subscriber count via the
API key) so the user can judge a channel before subscribing; videos are
not pulled (the scheduler picks those up).
- POST /api/channels/{id}/subscribe: write-scope gated, subscriptions.insert
+ local Subscription with the returned resource id.
- YouTubeClient.insert_subscription / get_my_channel_id.
- users.yt_channel_id (migration 0019) caches the user's own channel id so
discovery can exclude it.
- Frontend: ChannelDiscovery DataTable, Channels tab toggle (persisted),
api methods, trilingual strings. Subscribe ships a typed notification
payload (ChannelSubscribedMeta) for the inbox to act on.
Expose the maintenance job's rolling re-validation batch (videos re-checked per
run) as an admin control on the Scheduler dashboard, stored in app_state
(migration 0018; NULL = the env/config default). The job reads the effective
value each run via a state helper, clamped to a sane range. Trilingual.
Add a server-backed notification center that coexists with the client-side
transient bell: a per-user `notifications` table (type/title/body/data JSON/
read/dismissed), a `/api/me/notifications` CRUD API (list, unread_count, read,
read_all, dismiss, clear), and a left-nav inbox module with a live unread badge
polled via useLiveQuery. Known types render trilingual text from type+data
(English stored text is the fallback); read rows are trimmed past a soft cap.
Also adds the schema the maintenance job builds on: videos.list?part=status
columns (embeddable/privacy_status/upload_status) and the validation lifecycle
columns (last_checked_at, unavailable_since, unavailable_reason).
Page-id validation is centralized in one PAGES source of truth (isPage) so the
new page survives reload without a second allowlist to keep in sync.
Add scheduler_settings (per-job interval override, migration 0015) and a
PATCH /api/admin/scheduler/jobs/{id} that persists the override and live-
reschedules the running APScheduler job. Intervals load from the DB (env
defaults as fallback); the snapshot reports the live trigger interval.
Add the shared demo-account plumbing: users.is_demo marks the single
shared demo user, demo_whitelist holds the admin-curated emails that may
enter it without Google sign-in, and a small in-process RateLimiter
(generic groundwork) for throttling the demo-login endpoint per IP.
Store a SHA-256 of each playlist's name + ordered video ids as last synced from /
pushed to YouTube (migration 0013, backfilled for already-clean linked playlists).
'dirty' is now recomputed on every edit by comparing the current fingerprint to that
baseline instead of being stickily set true. So manually restoring the original order
(or undo back to it) clears dirty on its own — the Reset to YouTube button disappears
and no YouTube read is needed. Baseline is set on read-sync, repull (reset), and a
clean push; a missing baseline counts as dirty (unknown YT state). Verified the
migration's fingerprint matches the app's runtime fingerprint exactly.
Add per-user local playlists: Playlist + PlaylistItem models (with forward-looking
kind/source/yt_playlist_id/dirty columns for the later YouTube-sync phases) and
migration 0011. New /api/playlists routes: list (with optional contains=<video_id>
membership flags for the add-to-playlist popover), create, get detail (items
serialized via the feed serializer, with per-user watch state), rename, delete,
add/remove item, and reorder. Watch later (built-in) is protected from deletion.
Add position_seconds (+progress_updated_at) to video_states so watch progress
survives across devices and can drive a feed filter. New POST
/api/videos/{id}/progress checkpoints the player position (clearing trivially
-early and near-finished positions). Feed serialize exposes position_seconds and
a show=in_progress filter lists started-but-unfinished videos. Un-marking
'watched' now keeps a stored position instead of deleting the row.
Track who burned how much YouTube API quota. A QuotaEvent audit log (migration
0009) records every spend with the triggering user (NULL = background/system) and
an action label, set via a request/job-scoped contextvar (quota.attribute) so no
call signatures change. User-initiated work (sync subscriptions, unsubscribe,
opt-in recent backfill, manual enrich) attributes to the user; scheduler work to
System, split by action.
- backend: QuotaEvent model + migration 0009; quota.attribute() contextvar;
record_usage logs events; entry points wrapped (routes/sync, routes/channels,
scheduler); GET /api/quota/my-usage + GET /api/quota/admin
- frontend: admin-only Stats page (header nav, page=stats) with daily bars +
per-user breakdown by action and range picker; 'Your API usage' in Settings ->
Sync for every user
Verified: attribution + endpoints compute correctly; events are per-user vs System.
Move the access whitelist from the ALLOWED_EMAILS env var into a DB Invite table
(env kept as bootstrap fallback), and add a self-service request + admin approval
flow with fail-soft email.
- models: Invite(email, status pending|approved|denied, requested_at, decided_*)
- migration 0008: invites table; seed env ALLOWED_EMAILS u ADMIN_EMAILS as approved
- auth: is_allowed() (DB-first, env fallback); a denied Google login records a pending
request and bounces to /?access=requested instead of a raw 403; public POST
/auth/request-access; upsert is idempotent so repeats don't re-spam admins
- routes/admin.py (admin-only): list/approve/deny invites + manual add
- email.py: smtplib + Gmail App Password, fail-soft (skips if SMTP unset)
- /api/me exposes pending_invites; config + .env.example gain SMTP_*
- UI: Login 'Request access' form + access=requested/denied handling; Settings ->
Access requests (approve/deny + add); admin nudge toast on pending requests
Verified locally: request-access creates a pending invite and emails the admin;
seed approved npeter83; guinea-pig yt.trash2023 denied until approved.
Per-user opt-in to full-history (deep) backfill so a new user's unique
channels no longer trigger a big shared-quota burst.
- migration 0007: Subscription.deep_requested (default false); seed admins'
existing subscriptions to preserve today's global-backfill behaviour
- run_deep_backfill is now demand-driven: only channels at least one user has
requested are deep-backfilled; recent backfill stays unconditional (cheap)
- estimate_deep_backfill ETA helper (quota-bound) surfaced in /api/sync/my-status
- POST /api/sync/deep-all to opt all my channels in; PATCH channels accepts
deep_requested
- UI: per-channel Full history toggle, Backfill everything action, deep
progress + ETA in Channels header and Settings - Sync
- Header shows a live sync status (total videos + channels still backfilling, or
"paused"), polled every 30s
- Admins can pause/resume the background sync; a paused flag in app_state makes
scheduled jobs skip (migration 0006)
- GET /api/feed/count returns the number of videos matching the current filters;
shared filter builder keeps it in sync with /api/feed; shown above the feed
- /api/sync/status reports backfill progress, pending enrichment and paused state
- Shorts: confirm via youtube.com/shorts/<id> probe (SOCS cookie bypasses the
consent redirect) instead of a 60s heuristic; concurrent probing, shorts_probed
flag, scheduled refinement (migration 0005)
- Search: match title + channel name only (descriptions caused noisy results)
- Faceted tag filtering: AND across categories (language AND topic narrows),
OR within a category; any/all toggle applies to topics
- Language detection: majority vote over individual titles (fixes misdetections
like multipoleguy -> English; drops bogus Polish/Romanian)
- Login: drop forced consent so returning sign-in is quick (select_account)
- Feed cards: clickable channel name (opens channel), persistent saved badge,
undo toast on hide, Hidden view to restore; tag chips show counts in tooltip
- Tag and ChannelTag models + migration 0003 (partial unique indexes split
system vs per-user tag names)
- Offline language detection (py3langid) constrained to a curated language set,
with the channel's declared default language as a strong prior
- Topic tags mapped from YouTube topicDetails + dominant video category; the
generic "Lifestyle" catch-all is intentionally dropped
- System (auto) tags are regenerated idempotently and never touch user tags;
orphaned system tags are cleaned up
- GET /api/tags and admin POST /api/tags/recompute; scheduled autotag job
- Channel/Subscription/Video/ApiQuotaUsage models + migration 0002
- Synchronous YouTube Data API client with OAuth token refresh and per-call
quota accounting (API key preferred for public reads when configured)
- Subscription import: upsert channels + subscription links, prune unsubscribed,
fetch channel details (uploads playlist, topics, stats, handle, country)
- Central quota guard tracking units per US-Pacific day against a daily budget
- POST /api/sync/subscriptions and GET /api/sync/status endpoints
- docker-compose with Postgres 16 + slim Python API image
- FastAPI app with session middleware, health endpoint, static login page
- Google OAuth (Authlib) with email invite-list whitelist; admin role support
- User + OAuthToken models; refresh tokens encrypted at rest (Fernet)
- Alembic migrations, run automatically on container startup
- Postgres backup/restore scripts for portability between machines
