Commit graph

95 commits

Author SHA1 Message Date
npeter83
a88254c841 feat(plex): P0 backend foundations — catalog model, config, client
Optional Plex module groundwork (design locked 2026-07-05): plays the LOCAL
physical file; Plex is metadata + optional watch-sync only.

- models + migration 0044: plex_libraries/plex_shows/plex_seasons/plex_items
  (playable leaf, ffprobe playability class, markers, weighted FTS search_vector)
  + plex_states (per-user watch state, mirrors video_states)
- sysconfig 'plex' group + config.py env defaults (server url/token[secret]/
  path-map/libraries/sync-interval/max-transcodes)
- app/plex/client.py (PlexClient httpx: server info, sections, items, metadata
  +markers, image) + app/plex/paths.py (Plex→local path map + file resolve)
- routes/plex.py admin POST /api/plex/test (verify connection + list sections)
2026-07-05 01:35:08 +02:00
npeter83
42d465d760 feat(downloads): store + show a canonical "downloaded from" source URL
Every download now records the clean source page URL and shows it on the
Downloads page (open in a new tab, or copy to clipboard). The worker stores
yt-dlp's canonical webpage_url on the asset (migration 0043 adds
media_assets.source_webpage_url); the serializer prefers it and falls back to a
URL derived from source_kind+source_ref, so YouTube, external YouTube links and
external URLs (e.g. Facebook reels) all get a correct reference, and queued/older
rows work before the worker fills it. Edit clips return null (a clip's source is
the user's own earlier download, not a web page). i18n en/hu/de.
2026-07-04 21:25:37 +02:00
npeter83
603cc9854c fix(auth): /api/me answers 200 when logged out (no console error)
The logged-out landing probed /api/me, which 401'd, and the browser logs every
non-2xx fetch as a console error regardless of how the app handles it. The
bootstrap probe now returns 200 with {authenticated:false} via a new
optional_current_user dependency; api.me() maps that to null. The render gate
treats no-data as 'signed out' -> the landing, and a thrown error as a real
network/5xx failure. Other protected endpoints still 401, so mid-session expiry
is still caught by the global handler. Lighthouse (dev): Best Practices 96->100.
2026-07-04 18:17:24 +02:00
npeter83
1e3fad9a8f feat(downloads): edit + remove actions on shared-with-me items (no re-share)
A shared-with-me item only had a download button. Add two actions (Share is intentionally NOT
offered — no chain re-sharing of someone else's file):
- Edit: the editor now accepts an accessible (owned OR shared) source, so editing a shared video
  produces the editor's OWN clip in their library (counts against their quota, fully theirs
  including share); the source file is only read. Route uses _accessible_job.
- Remove from my list: DELETE /api/downloads/shared/{job_id} deletes only the recipient's share
  grant — the owner's job and physical file are untouched (per-user dismissal).
i18n en/hu/de. Verified in a real browser (edit a shared 70-min video → own 2:31 clip; remove
confirm shows 'won't delete the owner's file').
2026-07-04 05:21:50 +02:00
npeter83
d672583830 feat(downloads): public share-by-link + watch page backend
Share a download by a capability URL (/watch/{token}) that anyone can play on a login-free page —
distinct from the registered-user ACL share. Per-link controls: optional expiry, allow-download
toggle (stream-only vs downloadable), optional argon2 password. Revoke = delete.
- migration 0042 + DownloadLink model; app/downloads/links.py (token, HMAC grant sign/verify)
- owner endpoints (POST/GET/PATCH/DELETE links) + /recipients for the internal user picker
- public router (no auth): watch meta / password unlock→signed grant / range-aware file serve
  (inline vs attachment); unlock is rate-limited; meta verifies the file exists on disk
Verified end-to-end: 206 range, inline vs attachment, wrong-password 403, tampered-grant 403.
2026-07-04 04:19:29 +02:00
npeter83
f0fc542260 feat(downloads): editor backend — trim/crop derivatives (phase 2)
Per-user trim/crop clips derived from a finished download via ffmpeg, riding the same
MediaAsset/DownloadJob/worker/quota/GC machinery (asset source_kind='edit', per-user cache key):
- migration 0041: download_jobs job_kind/source_job_id/source_asset_id/edit_spec
- app/downloads/edit.py: spec normalize+sig, ffmpeg trim/crop cmd (fast stream-copy vs frame-
  accurate re-encode, per-edit 'accurate' toggle), progress+cooperative-cancel runner, filmstrip
- service.enqueue_edit; worker _process_edit branch; storage.edit_rel_path (.edits/{user} tree)
- routes: POST /edit, GET /{id}/storyboard(.jpg)
2026-07-04 01:01:46 +02:00
npeter83
4b15d55092 fix(downloads): force IPv4 (fixes container 'Errno -5' DNS failures) + retry resets errored asset
Root cause of the failing downloads: containers usually have no working IPv6 route, but
googlevideo CDN hosts advertise AAAA records, so the downloader tried IPv6 and failed with
'[Errno -5] No address associated with hostname'. Small clips happened to use IPv4; long videos
(more CDN requests / the ffmpeg path) hit IPv6 and died.
- yt-dlp source_address=0.0.0.0 forces IPv4 for every connection (== --force-ipv4). Verified: the
  full video+audio download completes cleanly.
- Also: retry (resume) now resets the shared asset from 'error' to 'pending', so it actually
  re-downloads — previously the requeued job instantly re-inherited the asset's stale error
  because the worker short-circuits a job whose asset already errored.
- Dropped the localdev DNS override (wrong hypothesis; the issue was IPv6, not the resolver).
- Kept yt-dlp retries/fragment_retries/socket_timeout for transient blips.

Verified end-to-end in the worker: two previously-failing long videos now download to done.
2026-07-03 03:47:13 +02:00
npeter83
f61ca96b6c fix(downloads): strip emoji from device filename + free disk when a download is deleted
Two UAT findings:
1. The device download filename (Content-Disposition) kept emoji/symbols from the video title.
   Add storage.display_filename (drops emoji/symbol/control unicode, keeps spaces + accents)
   and use it for the download name — readable and clean ("…alapján!.mp4", no emoji).
2. Deleting/canceling a download removed the job but the shared MediaAsset (and its file) lingered
   as cache, so 'Ready files' stayed inflated and disk wasn't freed. Rework: _release_asset drops
   the hold and, once no job holds the asset, deletes the file + row immediately (the cache only
   needs to span overlapping holders). Also fixes cancel never decrementing (it flipped status to
   'canceled' before releasing, tripping the holding-state guard).

Verified: filename emoji-stripped; enqueue→delete removes the asset row + file from disk.
2026-07-03 02:16:36 +02:00
npeter83
7d1ed24fea feat(downloads): M5 — Download Center frontend
- api.ts: download types + methods (profiles, enqueue, list/manage, file url, share,
  usage, index, admin storage/quota)
- format.ts: formatBytes / formatSpeed
- i18n: en/hu/de downloads.json (trilingual)
- Downloads nav module (Download icon, active-count badge, hidden for demo) placed after
  Messages; urlState page + App render + Header title
- DownloadButton: self-contained per-card/player affordance (demo-hidden, reflects the
  per-user download index: downloaded / in-queue), opens the preset dialog
- DownloadDialog: preset picker + optional display name -> enqueue + 'View' toast
- ProfileEditor: manage built-in + custom formats (full spec form incl. SponsorBlock)
- DownloadCenter: Queue / Library / Shared / (admin) System tabs, live progress via
  useLiveQuery, usage bar, pause/resume/cancel/delete, save-to-device, rename, share,
  admin storage dashboard + per-user quota editor
- wired DownloadButton into VideoCard + PlayerModal
- lib/nav.ts: decoupled navigator so the download toast can jump to the page

tsc + vite build clean. Verified in a headless authed browser: page + tabs render, Library
shows a real download with usage bar + actions, no console errors.
2026-07-03 01:52:58 +02:00
npeter83
51158ad9cf feat(downloads): M4 — REST API (enqueue, manage, file-serve, sharing, admin)
routes/downloads.py (require_human; admin_router adds admin_user):
- profiles: list (builtins + own), create/update/delete custom
- enqueue: resolve_source (bare id / watch / youtu.be / shorts URLs -> youtube id; else
  raw URL for the generic extractor), profile_id or inline spec or builtin fallback; maps
  quota.QuotaExceeded -> 422 speaking message
- list / usage / shared-with-me; rename (display name only); pause/resume/cancel/delete
  (ref_count bookkeeping)
- GET /{id}/file: ownership-or-share check, path-traversal guard, range-aware FileResponse
  with the user's custom display name (Content-Disposition), bumps last_access
- share/unshare by email + a download_shared notification
- admin: all-jobs, storage dashboard (totals + per-user footprint), per-user quota GET/PUT/DELETE
- cast SUM(bigint) footprints to int for clean numeric JSON
- wired both routers in main.py

Verified via TestClient: full enqueue->download->206 range fetch->share->access-control
(cross-user delete 404); unauth = 401 on user + admin routes; all 15 paths registered.
2026-07-03 00:34:08 +02:00
npeter83
5cd807ec51 feat(auth): per-tab account — different account per browser tab
Two tabs in one browser can now run two different signed-in accounts at once.

- The signed session cookie stays the browser's account WALLET (account_ids). Which account a
  given tab acts as is a per-tab choice held in sessionStorage and sent per-request via the
  X-Siftlode-Account header; current_user honours it only for an account already in the wallet,
  without mutating the cookie's default account. Switching accounts sets the header + reloads
  THIS tab only, instead of the old cookie-wide switch that changed every tab.
- WebSocket can't send headers, so the per-tab account rides in the ?account= query param
  (validated against the wallet).
- Logout is per-tab aware: it signs the requesting tab's active account out of the wallet
  (promoting a new default only if the removed one was the default), and the tab drops its
  override. A stale per-tab header account 401s just that tab instead of clearing the session.
- Serve index.html with Cache-Control: no-cache so a deploy's new hashed bundle is picked up
  immediately instead of the browser running a heuristically-cached stale index.html.
2026-07-01 23:43:35 +02:00
npeter83
3056734231 feat(views): saved-views backend — table, model, CRUD + reorder API
Per-user named snapshots of the feed's FeedFilters. New saved_views table
(migration 0035, unique per user+name, position, is_default) + SavedView
model, and a /api/saved-views router (list/create/patch/delete/reorder)
gated behind require_human so the shared demo account can't pollute it.
Setting is_default clears the flag on the user's other views (one default).
2026-07-01 03:17:36 +02:00
npeter83
0c18845c34 feat(search): ephemeral results UX, count selector, channel blocklist (frontend)
- Live-search view: a results-count selector (20/40/60/100) replaces manual load-more (the free
  scrape source pages until that many are gathered); an 'these results are temporary' banner with
  a 'Clear now' button that discards them 'as if never added' (api.clearSearch) and returns to
  the feed.
- Channel blocklist: a Block/Unblock toggle + 'Blocked' badge on the channel page (blocked
  channels don't auto-explore and their videos are hidden), and a 'Blocked channels' section in
  the Channel manager with one-click unblock. ChannelDetail.blocked from the backend.
- Admin: a 'Purge discovery' button on the Scheduler page (immediate un-kept search/explore
  cleanup). EN/HU/DE throughout.
2026-07-01 01:00:32 +02:00
npeter83
f27f31b6db feat(search): ephemeral results, count selector, blocklist, new-first ordering (backend)
- Live-search results stay ephemeral: the discovery-cleanup job now also reclaims un-kept
  via_search videos (no watch state / not playlisted / channel not subscribed) after a grace
  period (search_grace_days), and POST /api/search/clear discards a given result set 'as if
  never added' (drops the user's search-finds + deletes the now-orphaned, un-kept videos).
  Admin POST /api/admin/purge-discovery runs it on demand (grace 0).
- Count selector: GET /api/search/youtube gains a 'limit' — the free scrape source pages
  through continuations until that many results are gathered (no manual load-more); the API
  source stays one ≤50 page (cost).
- Per-user channel blocklist (migration 0034 blocked_channels): block/unblock/list endpoints;
  blocked channels' videos are dropped from live search before ingest and hidden from the feed
  / Library / explore / channel page; explore refuses a blocked channel.
- New-first ordering: results you already have (subscribed channel, watched/in-progress/saved/
  playlisted) sink below genuinely-new discoveries, preserving relevance within each group.
2026-07-01 00:42:32 +02:00
npeter83
2b5b6ac965 feat(channels): search box + clickable tag chips; fix(feed): contextual Your-tags
- Channels manager: prominent channel-name search box and the 'Your tags' chips are now
  clickable to filter the table by tag (replacing the hidden per-column DataTable popovers).
  Both filter client-side over the status-filtered list; a focus-channel intent seeds the
  search, the reset intent clears both.
- Feed 'Your tags' sidebar: count user tags in the facet endpoint (the 'other' category) and
  make the widget contextual like language/topic — counts reflect the current filter and
  zero-count chips hide (e.g. a source=search view with no tagged channels shows 'no matching
  tags' instead of the full static list). EN/HU/DE searchPlaceholder.
2026-06-30 22:30:35 +02:00
npeter83
bc4c362423 feat(channels): channel-explore backend — about metadata, ephemeral provenance, cleanup
Adds the backend for a dedicated channel page + ephemeral browsing of un-subscribed
channels:
- migration 0033: channels.{total_view_count,published_at,banner_url,external_links,
  from_explore}, videos.via_explore, explored_channels (per-user, grace-clocked).
- enrich channels with part=brandingSettings (banner/links) + statistics.viewCount +
  snippet.publishedAt — no extra quota.
- GET /api/channels/{id}: About detail + this user's relationship; lazy-enriches the new
  About fields (published_at sentinel).
- POST /api/channels/{id}/explore (require_human, quota-reserve guarded): records the
  exploration, flags the channel ephemeral unless followed, ingests one page of uploads
  (via_explore) + enriches; returns next_page_token for load-more.
- feed: per-explorer gate — a via_explore video is visible only to users who explored or
  subscribe to its channel, so exploration never leaks into everyone's catalog.
- subscribe = keep: clears from_explore + via_explore on the channel's videos.
- scheduler explore_cleanup job + explore_grace_days config: hard-delete explored-but-
  unkept channels and their untouched ephemeral videos after a grace period.
2026-06-30 02:52:44 +02:00
npeter83
4395afc210 feat(feed): broaden search to a weighted document (title + keywords + queries + description)
Makes the local search behave more like YouTube's — finding videos by the
uploader's own keywords or the query that surfaced them, not only words in the
title. A DB-generated, weighted search_vector (migration 0032) replaces the
title-only FTS index:
- keywords: the creator's snippet.tags (free — already in the snippet we fetch),
  stored on enrich.
- search_terms: distinct live-search queries that surfaced the video (across all
  users), appended by the search route — folds YouTube's relevance into local
  search (a video YT returned for a query becomes findable by it even without a
  title match), the user's own idea.
- description (truncated) for broad recall on the existing catalog.
Weighted title(A) > keywords+queries(B) > description(C) so ts_rank keeps title
hits on top. A plain GIN index on the generated column guarantees index use (no
expression/param matching). Verified on localdev: recall 146->213 for one query;
7 'eurovision' hits via the document but not the title; index scan confirmed.
2026-06-30 02:00:38 +02:00
npeter83
6cef826ecc feat(feed): per-user search finds in Mine scope + full-text relevance search
Two related search improvements:

1) Your own live-search results now belong to your Mine feed. A new per-user
   search_finds table (migration 0030) records each video you surface via your
   YouTube search (the route inserts them idempotently). The Mine feed becomes
   'your non-hidden subscriptions OR your search finds', and the Source filter
   now applies in Mine too: organic = subscriptions, search = your search finds,
   all = both (default stays organic, so the main feed is unchanged). The shared
   Library keeps using the global via_search flag.

2) Feed search ranks by relevance instead of a whole-phrase substring. A custom
   unaccent_simple text-search config + GIN index (migration 0031) back a
   YouTube-like fuzzy match: word-order-independent, multi-word AND, prefix on
   the word being typed, accent-insensitive. A new 'relevance' sort orders by
   ts_rank; the channel name still matches as a substring. The rank is scaled to
   an integer so the keyset cursor pages it exactly (a raw float4 breaks paging).
   _filtered_query returns the rank expr so only the feed list uses it.
2026-06-30 00:39:09 +02:00
npeter83
396e09189b feat(search): zero-quota InnerTube scrape source + search_source toggle
Live YouTube search can now use YouTube's internal InnerTube endpoint instead
of search.list, materialising results through the same enrich/provenance path
at zero API quota (search.list costs 100 units/page; scrape costs nothing, only
the cheap shared videos.list enrich is charged).

- youtube/search_scrape.py: InnerTube search returning the same page shape as
  YouTubeClient.search_videos (items + continuation cursor); SOCS consent cookie,
  videoRenderer walk (channels/playlists/Shorts shelves naturally excluded),
  type:video filter, cached InnerTube key/version with constant fallbacks.
- routes/search.py: admin-selectable source (search_source, default scrape).
  Scrape path skips the 100-unit budget pre-check and logs a zero-cost search
  event so the per-user daily cap still counts it; api path unchanged. Response
  carries the active source.
- sysconfig/config: new search_source key (scrape|api).
- quota.log_action: record a zero-cost action event for per-user rate limits.
2026-06-29 22:29:54 +02:00
npeter83
a7a72c4c7e feat(feed): accent-insensitive search (unaccent)
Feed text search used plain ILIKE, which is case- but not diacritic-insensitive, so 'tiesto'
missed the many titles spelled 'Tiësto' — a search that ingested ~45 results showed only ~12.
Enable the postgres unaccent extension (migration 0029) and wrap both sides of the title/
channel match in unaccent(), so 'tiesto' now matches 'Tiësto'. Applies to feed, count and
facets alike.
2026-06-29 02:30:37 +02:00
npeter83
8b19faaed1 feat(search): Library provenance filter (organic / all / search-only)
Replace the binary 'show search-discovered' toggle with a 3-way Source selector in the
Library toolbar, so users can also see ONLY search-discovered videos — not just hide or
mix them. Backend: feed param library_source = organic (default, hides via_search) | all
(both) | search (only via_search), applied in scope=all. Strings in HU/EN/DE.
2026-06-29 02:11:53 +02:00
npeter83
9b1bdb6b42 feat(search): live YouTube search backend
Add a live YouTube search that materialises results into the shared catalog so
they render with the normal feed cards + in-app player and gain per-user state.

- YouTubeClient.search_videos(): search.list (100 units), embeddable-only, returns
  flat stubs + nextPageToken; surfaces liveBroadcastContent for live filtering.
- routes/search.py GET /api/search/youtube: require_human + per-user daily cap
  (search_daily_limit_per_user, default 70) + can_spend pre-check (429 on either);
  drops live/upcoming, upserts channel stubs (channels.list) + video stubs, enriches
  (videos.list), runs the youtube.com/shorts probe, then excludes Shorts/live and
  returns feed cards in relevance order with the YouTube pageToken as the cursor.
- Provenance: videos.via_search / channels.from_search (migration 0028) flag
  search-discovered rows; the feed hides them from the Library (scope=all) by default
  via exclude_search_discovered, leaving the Mine feed untouched.
- quota.actions_today() counts a user's per-action events today for the cap; only the
  search.list call is attributed VIDEOS_SEARCH so the counter is exactly 1 per search.
2026-06-29 02:01:31 +02:00
npeter83
022505ff18 refactor(playlists,feed): share feed-row projection & playlist helpers
- feed_columns() is the single feed-row projection, consumed by both /feed and the
  playlist-detail query (was copy-pasted, drift risk on every new column).
- _replace_items_from_youtube() dedups the YouTube playlist-mirror block shared by
  sync_user_playlists and repull_playlist.
- _next_playlist_position/_next_item_position/_add_item replace the append-position
  idiom (x4) and the exists-then-insert blocks (x2).
2026-06-26 03:15:53 +02:00
npeter83
5b81e22677 refactor(backend): quota.measured helper, consistent admin gating, messageable predicate
- quota.measured() context manager folds the before/after units diff that the
  manual sync routes each re-spelled (also makes quota_remaining_today consistent).
- sync pause/resume now use Depends(admin_user) instead of inline role checks.
- is_messageable_user() unifies the 'real, active, non-suspended human' rule that
  was encoded three ways (WS auth, send recipient, and the SQL _messageable()).
2026-06-26 03:15:53 +02:00
npeter83
63701f5344 refactor(auth): centralize token hashing, email validation & admin gating
- app/utils.py: shared valid_email() + now_utc() (one email regex, was duplicated
  in auth.py and admin.py with a looser "@"-in check elsewhere).
- security.hash_token(): one SHA-256 token hasher (was duplicated in auth.py + state.py).
- auth.admin_user dependency + count_admins() helper, replacing the inline role
  checks and the last-admin count query repeated across admin.py/me.py/tags.py.
2026-06-26 03:15:36 +02:00
npeter83
8392037e5f feat(messages): auto-open/flash dock on incoming + persist dock across reload
The live-message push now carries both parties, so an incoming message opens
that conversation's dock window if it's closed, or flashes it once if it's
already open (expanded or minimised, without disturbing the minimised state) —
only for messages from someone else, never your own echo.

Dock state (open windows + minimised state) is persisted per user, so a reload
restores exactly what was open, minimised, or closed.
2026-06-25 22:46:54 +02:00
npeter83
002a79949b feat(messages): end-to-end encrypted real-time direct messaging backend
Private user-to-user messages are end-to-end encrypted: the server only ever
stores ciphertext + iv and acts as a key directory (public keys) plus an opaque
store for each user's private key, wrapped client-side with a passphrase the
server never sees — so not even an admin can read a conversation. A separate
kind=system message (plaintext, no sender) powers a server-authored Siftlode
welcome shown on first open, reusable later for announcements.

- models: rework Message (kind, nullable sender/body, ciphertext+iv) + MessageKey;
  migrations 0026 (table) + 0027 (E2EE rework).
- routes/messages.py: key directory/blob endpoints, ciphertext send, conversations
  + threads (system + user), lazy welcome, all gated by require_human.
- realtime.py: in-process WebSocket connection registry; /ws delivers sent
  messages to a user's open tabs instantly (sync-callable push, single-process).
2026-06-25 22:05:35 +02:00
npeter83
424b19f3b6 feat(feed): keyset/cursor pagination replacing OFFSET/LIMIT
Replace deep-OFFSET paging (which scans-and-discards skipped rows on the
233k-row catalog and shifts/duplicates rows as the scheduler ingests mid-
scroll) with stable keyset pagination over an opaque cursor.

A single sort registry drives both the ORDER BY and a generic, NULL-aware
keyset WHERE so they can never drift, with Video.id as the unique final
tiebreaker. Covers every sort: newest/oldest, views, duration, title,
subscribers (joined), priority (coalesced multi-key) and the seeded
shuffle. The response now returns next_cursor instead of offset.
2026-06-25 19:54:40 +02:00
npeter83
1ec89c8fbe chore: rename remaining subfeed references to siftlode
Replace the leftover 'subfeed' name across logger names + log_config,
frontend localStorage keys, Postgres user/db/volume defaults in the
compose files, .env.example, config.py, backup/restore scripts and the
README. Pure rename; no behavioural change. localStorage keys move from
subfeed.* to siftlode.* (one-time UI-state reset is acceptable).
2026-06-21 06:53:12 +02:00
npeter83
c2a388e4ea fix(setup): hide YouTube/Google affordances when Google sign-in isn't configured
- /api/me exposes instance-wide google_enabled. The onboarding nudge no longer auto-opens, and
  Settings hides the YouTube-access section + the 'Connect Google' option, when the instance has
  no Google OAuth configured (e.g. an email+password-only self-host). A linked account still shows
  its 'Connected' status.
2026-06-21 02:06:37 +02:00
npeter83
2689173208 feat(setup): install-wizard step endpoints (epic 6c, API)
- POST /api/setup/admin (create the first admin: active+verified+admin), /config (persist any
  registry-backed settings — Google creds, SMTP, quota…), /test-email, and /finish (mark
  configured + invalidate the token). All behind require_setup (valid token AND not configured).
- GET /api/setup/info reports secrets_manageable so the wizard hides the Google/SMTP secret steps
  on a box without TOKEN_ENCRYPTION_KEY.
2026-06-21 00:40:54 +02:00
npeter83
283c4c9a1e feat(setup): first-boot detection + setup-mode gating + one-time token (epic 6a)
- app_state gains configured + setup_token_hash (migration 0025); existing installs (any users)
  are marked configured so they never see the wizard, a fresh DB starts in setup mode.
- On first boot the app mints a one-time setup token and logs the wizard URL (only the hash is
  stored); the token gates the setup steps (added in 6c).
- A middleware locks all /api + /auth routes (except /api/setup, /api/version, /healthz, static)
  to 503 until configured, with a DB-free cached fast-path once setup completes.
- GET /api/setup/status tells the SPA whether to show the wizard.
2026-06-21 00:38:47 +02:00
npeter83
3f9c395b17 feat(admin): user management — roles, suspend, delete + lifecycle emails
- New Users page tabs (Users & roles / Access requests / Demo) and a tab-ified Configuration
  page, both via a reusable Tabs component (persisted active tab).
- Admin can suspend/unsuspend (migration 0023 adds users.is_suspended) and delete accounts
  from the Users & roles tab, with guards (demo / self / last admin).
- Email notifications to the affected user: suspended (reactive, on a blocked sign-in),
  reinstated, role changed, and account deleted; the approval email now carries a clickable
  app link. The scheduled/manual demo reset also seeds sample channel subscriptions.
- Hide the 'unverified email' chip for Google accounts (Google attests the email).
2026-06-19 19:52:12 +02:00
npeter83
8c727dd99e feat(auth): account lifecycle — Google linking, passwords, suspension & deletion plumbing
- Link a Google account to a password account, and adopt the Google identity onto a matching
  email account instead of 500ing on a duplicate; set or change a password from Settings.
- Expose has_google/has_password on /api/me for the Sign-in methods UI.
- Mark Google logins email-verified (backfill existing rows, migration 0024); stop a routine
  login from clobbering an admin-assigned role (env ADMIN_EMAILS stays the bootstrap admin).
- Suspension login-gates (password + Google callback + current_user) with a rate-limited
  'suspended' notice; shared purge_user (cascade delete + access-request cleanup + Google-grant
  revoke) behind self- and admin-deletion; single app_base source for user-facing email links.
2026-06-19 19:52:02 +02:00
npeter83
84b55c0567 feat(account): GDPR self-service account + data deletion (5d)
Add DELETE /api/me/account: permanently erases the signed-in account and all its
personal data — OAuth tokens, subscriptions, tags, video states, playlists,
notifications all cascade on the users row (FK ON DELETE CASCADE); quota-audit
events are kept but anonymised (SET NULL); the email's invite/whitelist row is
removed too. Guards: the shared demo account can't be deleted, and the last
remaining admin can't delete itself. Frontend: a Danger zone in Settings -> Account
(non-demo) with a danger-confirm; on success the session clears and the app lands
on the welcome page. EN/HU/DE. Verified via curl: self-delete + session clear +
demo 403.
2026-06-19 15:46:49 +02:00
npeter83
571f337fdf feat(demo): scheduled demo-account reset (5c)
A security audit of every mutating endpoint found no isolation gaps — all routes
scope by current_user, admin routes are gated, and the demo account is sandboxed
in its own user_id space (can't reach others' data, escalate, or hit admin
routes). The remaining demo concern is communal pollution of its own shared state,
so add an automatic reset: a new 'demo_reset' scheduler job (admin-tunable
interval, default 12h) reuses the manual reset logic, and no-ops without ever
creating a demo account if none exists. Verified: a triggered run wiped the demo's
video states and re-seeded its sample playlists.
2026-06-19 15:25:13 +02:00
npeter83
7efd4f4867 feat(auth): email+password registration with two-gate activation (5a backend)
Add email+password auth alongside Google: argon2id hashing; users gains
password_hash/email_verified/is_active and google_sub becomes nullable (migration
0022); a single-use, hashed auth_tokens table for email verification + password
reset. Registration creates a pending (inactive, unverified) account + an access
request; sign-in needs verified email AND admin approval. Anti-enumeration: uniform
register/login/reset responses (a correct-password owner still gets a specific
pending reason). allow_registration flag (DB-tunable). Admin users-list + role
endpoint (guards: not self, not demo, not the last admin); approving access (or a
manual whitelist add) now activates the matching pending account. current_user
rejects deactivated accounts. Verified end-to-end via curl + DB.
2026-06-19 14:03:11 +02:00
npeter83
e6bcf5ba1e feat(config): move operational params to DB (quota/backfill/shorts/batch)
Register 8 more keys in the sysconfig registry (quota daily budget + backfill
reserve, recent-backfill window, shorts-probe params, enrich/autotag batch sizes)
and route their reads through the DB-override-or-env resolver at every call site
(quota.py, sync/runner.py, sync/videos.py, sync/autotag.py, sync/maintenance.py,
routes/scheduler.py). All read sites already had a db session in scope. Reads are
read-through (no cache), matching app.state; admin can tune them live, no redeploy.
2026-06-19 13:07:45 +02:00
npeter83
48cb6a5dbd feat(config): DB-backed system_config infrastructure + SMTP group
Add a generic admin-editable config layer (epic 4a): system_config KV table
(migration 0021), a sysconfig registry that is the single source of truth for
DB-overridable keys (type/group/default/bounds/secret) + a DB-override-or-env
resolver, and admin endpoints (GET/PATCH/DELETE /api/admin/config + a test-email
probe). Secrets are Fernet-encrypted at rest (TOKEN_ENCRYPTION_KEY); without it
they stay env-only. First group wired end-to-end: Email/SMTP — email.py now
reads host/port/user/from/password via the resolver (own session, since email
is sent from background tasks).
2026-06-19 12:22:36 +02:00
npeter83
b23f805533 refactor(quota): canonical <entity>_<action> taxonomy for quota events
The quota-attribution action keys were inconsistent (mixed verbs, ad-hoc names,
English-only labels). Introduce a QuotaAction constants holder (one source of
truth), rename every attribution call site to it, and add migration 0020 to
rename historical quota_events.action values. The display label now resolves
from i18n (quotaActions.<key>, EN/HU/DE) instead of a hard-coded English map.
Scheduler job ids and progress phase labels are a separate namespace and are
left untouched.
2026-06-19 11:48:11 +02:00
npeter83
fa86e3d282 fix(api): surface YouTube action failures as 422, not 502
Same fix as the subscribe reconcile, applied to the remaining endpoints that
wrapped a YouTubeError in a 502: video lookup (feed) and playlist refresh,
push-plan, sync, and delete. A 502 is treated by the client as a transient
"connection lost" blip, so a real, explainable YouTube failure showed the wrong
message. 422 carries the clear detail to the error dialog instead.
2026-06-19 04:27:22 +02:00
npeter83
9c7beec443 fix(channels): reconcile an already-followed channel instead of erroring
Subscribing to a channel the user already follows on YouTube (a local/YouTube
desync — e.g. a stale import dropped the local row, so it resurfaced in the
discovery list) made YouTube return 400 "subscription already exists", which
the endpoint reported as a 502 — and the client shows a 502 as a transient
"connection lost" blip, not the real reason. Treat the duplicate as success:
record the subscription locally (the desired end state already holds; the next
resync fills in the resource id). Surface any other YouTube error as 422 with a
clear message, on both subscribe and unsubscribe, so it isn't mistaken for a
connection drop.
2026-06-19 04:16:23 +02:00
npeter83
f69e34e3a6 fix(header): show sync activity only while a job is actually running
The header's sync indicator spun "fetching history" whenever any channel
still lacked full history — i.e. perpetually, even when the scheduler was
idle between its periodic runs. It now spins only while a channel-sync job
(backfill or RSS poll) is actually running; otherwise pending deep-history
work shows as the calm, static "N without full history" link, and recent
work queued for the next run shows a static "N queued". This makes the
header coherent with the Scheduler dashboard's live state.

Backend exposes running_job_ids() from the scheduler activity and a derived
sync_active flag on /api/sync/my-status.
2026-06-19 02:57:45 +02:00
npeter83
fb6f0c5dcb feat(channels): discover & subscribe to channels from playlists
Add a "Discover from playlists" tab to the Channel manager that lists
channels appearing in the user's playlists they don't subscribe to, with
a one-click Subscribe.

- GET /api/channels/discovery: local join (playlist_items -> videos ->
  channels) minus the user's subscriptions and their own channel. Enriches
  stub channels' metadata up front (title/thumbnail/subscriber count via the
  API key) so the user can judge a channel before subscribing; videos are
  not pulled (the scheduler picks those up).
- POST /api/channels/{id}/subscribe: write-scope gated, subscriptions.insert
  + local Subscription with the returned resource id.
- YouTubeClient.insert_subscription / get_my_channel_id.
- users.yt_channel_id (migration 0019) caches the user's own channel id so
  discovery can exclude it.
- Frontend: ChannelDiscovery DataTable, Channels tab toggle (persisted),
  api methods, trilingual strings. Subscribe ships a typed notification
  payload (ChannelSubscribedMeta) for the inbox to act on.
2026-06-19 02:16:42 +02:00
npeter83
dd83718304 feat(scheduler): admin-tunable maintenance re-validation batch size
Expose the maintenance job's rolling re-validation batch (videos re-checked per
run) as an admin control on the Scheduler dashboard, stored in app_state
(migration 0018; NULL = the env/config default). The job reads the effective
value each run via a state helper, clamped to a sane range. Trilingual.
2026-06-18 04:37:08 +02:00
npeter83
ed4194a8d3 feat(scheduler): admin run-now/run-all triggers + live progress + completion notices
Add per-job "Run now" buttons and a "Start all now" button to the admin Scheduler
dashboard (admin-gated endpoints). Triggers run the job in a background thread
independent of its interval, refusing a concurrent run (409). While running, the
long jobs (maintenance, enrich, backfill, shorts) report live progress through a
decoupled contextvar sink, shown as a progress bar on the job row via the existing
4s poll. A manually-triggered run posts a completion notification to the triggering
admin's inbox (scheduled runs stay silent to avoid spam); the inbox renders the
"scheduler" type trilingually from type+data. While here, give the maintenance job
its missing dashboard label/description in all three languages.
2026-06-18 04:01:10 +02:00
npeter83
c4aecf9f77 feat(maintenance): scheduled job to retire unplayable videos
Add a daily maintenance/validation job that detects videos which can't be played
anywhere and retires them safely. Two phases: re-check already-flagged videos
(recover if available again, else hard-delete once the grace period elapses,
cascading to states/playlist items), and a rolling re-validation of the
least-recently-checked currently-available videos that flags newly-unplayable
ones (hidden from the feed immediately via unavailable_since).

Detection is ~free: a video missing from the videos.list response is
deleted-or-private; an `upcoming` premiere >2 days past its scheduled start that
never went live is abandoned. A still-live broadcast is kept (legit 24/7 stream).
Enrichment now also fetches part=status to populate the status columns. Grace is
7 days for removed videos, none for abandoned. Before deleting, affected users
get one batched notification (never per-video). Interval is admin-tunable via the
Scheduler dashboard; batch size and grace are config. Quota-attributed to the
system and bounded by the same backfill reserve as the other jobs.
2026-06-18 03:20:28 +02:00
npeter83
b9a3a9012d feat(notifications): durable per-user inbox (P1) + maintenance schema
Add a server-backed notification center that coexists with the client-side
transient bell: a per-user `notifications` table (type/title/body/data JSON/
read/dismissed), a `/api/me/notifications` CRUD API (list, unread_count, read,
read_all, dismiss, clear), and a left-nav inbox module with a live unread badge
polled via useLiveQuery. Known types render trilingual text from type+data
(English stored text is the fallback); read rows are trimmed past a soft cap.

Also adds the schema the maintenance job builds on: videos.list?part=status
columns (embeddable/privacy_status/upload_status) and the validation lifecycle
columns (last_checked_at, unavailable_since, unavailable_reason).

Page-id validation is centralized in one PAGES source of truth (isPage) so the
new page survives reload without a second allowlist to keep in sync.
2026-06-18 03:20:17 +02:00
npeter83
1e530d23a6 feat(tags): tag UX overhaul + per-card video reset
- Channel-row tags show only what's attached; a '+' opens a per-channel picker. Clicking a tag
  filters the feed by it; a 'Your tags' sidebar widget makes that filter visible/clearable.
- Tag manager dialog (add/rename/delete; delete only confirms when the tag is in use), reachable
  from the Channel manager and the feed sidebar; hovering a tag's count lists its channels, each a
  link that focuses it in the Channel manager (DataTable gains an external filter input).
- Video cards get a reset action that clears all watch state (incl. an in-progress position).
- api.req() now raises the error dialog on 5xx and 400/409/422 with the server's reason.
2026-06-18 01:17:31 +02:00
npeter83
b55a944e7c fix(ux): modal error dialog for server-refused actions + ESC closes only the topmost
- Duplicate-tag create/rename hit the uq_tags_user_name constraint and 500'd; now caught and
  returned as a clear 409 ('You already have a tag named …').
- New global error dialog (errorDialog store + ErrorDialog modal) for definitive server refusals;
  the api layer wiring + mount land with the rest of the round.
- Modal now keeps a stack so ESC dismisses only the top modal — an error over the tag editor
  closes the error and leaves the editor open.
2026-06-18 01:17:31 +02:00