Commit graph

102 commits

Author SHA1 Message Date
npeter83
0c76730797 merge: friendlier onboarding emails + deliverability headers 2026-06-12 02:05:18 +02:00
npeter83
8135527d75 fix(email): warmer onboarding copy + Reply-To/Date headers
Personalize the access-request and approval emails, add a real Date header and a
Reply-To (approval -> admin, new-request -> requester) so they read as a
conversation rather than a no-reply blast — small nudges for deliverability;
sender reputation still does most of the work.
2026-06-12 02:05:18 +02:00
npeter83
a92112cefb merge: M5c — onboarding (DB invites, request-access, admin approval, email) 2026-06-12 01:43:14 +02:00
npeter83
2add173760 feat(m5c): onboarding — DB invites, request-access, admin approval, email
Move the access whitelist from the ALLOWED_EMAILS env var into a DB Invite table
(env kept as bootstrap fallback), and add a self-service request + admin approval
flow with fail-soft email.

- models: Invite(email, status pending|approved|denied, requested_at, decided_*)
- migration 0008: invites table; seed env ALLOWED_EMAILS u ADMIN_EMAILS as approved
- auth: is_allowed() (DB-first, env fallback); a denied Google login records a pending
  request and bounces to /?access=requested instead of a raw 403; public POST
  /auth/request-access; upsert is idempotent so repeats don't re-spam admins
- routes/admin.py (admin-only): list/approve/deny invites + manual add
- email.py: smtplib + Gmail App Password, fail-soft (skips if SMTP unset)
- /api/me exposes pending_invites; config + .env.example gain SMTP_*
- UI: Login 'Request access' form + access=requested/denied handling; Settings ->
  Access requests (approve/deny + add); admin nudge toast on pending requests

Verified locally: request-access creates a pending invite and emails the admin;
seed approved npeter83; guinea-pig yt.trash2023 denied until approved.
2026-06-12 01:43:07 +02:00
npeter83
5754f47f20 merge: fix recent/deep boundary-page video loss 2026-06-12 00:15:20 +02:00
npeter83
18aa3726fb fix(backfill): don't skip the tail of the recent/deep boundary page
When recent backfill stopped partway through a page (age cutoff or count cap),
it stored the *next* page token as the deep-backfill cursor, so the older items
remaining on that same page were collected by neither pass — a channel could
report backfill_done while silently missing a band of videos right around the
365-day cutoff. Now resume deep from the page we stopped on (re-fetched; inserts
are idempotent). Verified: Pánczél went 200 -> 209 stored (= full uploads
playlist) after re-backfill.
2026-06-12 00:15:20 +02:00
npeter83
69e9c60ca1 merge: immediate recent backfill on full-history opt-in 2026-06-12 00:00:50 +02:00
npeter83
15007250db feat(channels): kick recent backfill immediately on 'full history' opt-in
When a user requests full history for a channel whose recent uploads aren't
fetched yet, run a one-channel recent backfill synchronously in the request so
the feed populates at once instead of waiting for the scheduler. Deep paging
still follows on the scheduler (recent-then-deep). The deep-toggle mutation now
also refreshes my-status and the feed.
2026-06-12 00:00:50 +02:00
npeter83
c8a50472f3 merge: M5b — optional YouTube write scope (incremental OAuth) 2026-06-11 23:27:17 +02:00
npeter83
43c05ea14b feat(m5b): optional YouTube write scope via incremental OAuth
Default login now requests read-only (youtube.readonly); write (unsubscribe,
later playlist export) is an explicit opt-in.

- auth.py: split READ_SCOPES / WRITE_SCOPES; new GET /auth/upgrade (incremental
  consent, prompt=consent); has_write_scope() helper
- /api/me exposes can_write
- youtube/client.py: delete_subscription (50 units, OAuth-only)
- DELETE /api/channels/{id}/subscription, gated on write scope (403 otherwise)
- UI: Settings - Account 'Playlist editing & YouTube export' enable button;
  per-channel 'Unsubscribe on YouTube' (with confirm) shown only when can_write

Browser-facing; develop/test locally until the public HTTPS login lands. Needs a
one-time Console step: add youtube.readonly to the OAuth consent screen scopes.
2026-06-11 23:27:11 +02:00
npeter83
f4b8a721fb merge: M5d — demand-driven deep backfill + per-user ETA 2026-06-11 23:07:15 +02:00
npeter83
beb961c021 feat(m5d): demand-driven deep backfill + per-user ETA
Per-user opt-in to full-history (deep) backfill so a new user's unique
channels no longer trigger a big shared-quota burst.

- migration 0007: Subscription.deep_requested (default false); seed admins'
  existing subscriptions to preserve today's global-backfill behaviour
- run_deep_backfill is now demand-driven: only channels at least one user has
  requested are deep-backfilled; recent backfill stays unconditional (cheap)
- estimate_deep_backfill ETA helper (quota-bound) surfaced in /api/sync/my-status
- POST /api/sync/deep-all to opt all my channels in; PATCH channels accepts
  deep_requested
- UI: per-channel Full history toggle, Backfill everything action, deep
  progress + ETA in Channels header and Settings - Sync
2026-06-11 23:07:09 +02:00
npeter83
bbcfd46ea9 merge: M5a — channel manager, settings panel, per-user sync, liquid-glass UI
Channel manager (priority/hide/user-tags, per-channel sync state, view-in-feed) with
a 'Channel priority' feed sort; a tabbed slide-in Settings panel (appearance, 6b
notification settings, per-user sync status + actions, account); per-user sync status
endpoint; an app-wide toggleable hint/tooltip system (portal-rendered); and a
theme-aware liquid-glass design system applied across panels, popovers, toasts and
cards (78% frosted, perf-mode opt-out).
2026-06-11 22:52:23 +02:00
npeter83
0a6befd578 style(ui): glass opacity 82% -> 78% 2026-06-11 22:49:04 +02:00
npeter83
94a077d699 fix(ui): restore glass translucency (82%) — opaque value killed the effect
backdrop-filter blur does work here; the previous 94% opacity made .glass nearly
solid so the blurred backdrop never showed. Dial back to 82% with a strong blur so
overlays read as frosted glass again, while staying less see-through than the old
72% that made the account menu look too transparent.
2026-06-11 22:45:14 +02:00
npeter83
f429889250 fix(ui): opacity-based frosted glass; drop full-page settings blur
Evidence (sharp video behind overlays) shows backdrop-filter is a no-op here, most
likely because browser hardware acceleration is off — so the glass relied on a blur
that never rendered. Lean on opacity instead (94% surface) so overlay menus/panels
read as frosted glass regardless of GPU; keep the blur as a bonus when available.
Revert the settings backdrop to a plain dim scrim (no whole-page blur) and make the
header a solid bar (no backdrop-filter ancestor).
2026-06-11 22:39:52 +02:00
npeter83
4a35e8b42a fix(ui): stronger frosted glass + frosted settings backdrop; declutter header
- Increase .glass blur (32px) so overlay menus (account) read as proper frosted glass.
- Blur the settings modal backdrop (overlay backdrop-blur) so the panel reliably
  frosts the content behind it instead of showing a sharp video through.
- Remove the now-redundant header buttons (dark/light, color scheme, grid/list) —
  they live in Settings → Appearance. Header keeps search, sync, bell, account.
2026-06-11 22:16:07 +02:00
npeter83
b7d448b403 perf(ui): drop backdrop-filter from bulk glass cards
Video cards / channel rows / sidebar widgets render in bulk over a solid background
where backdrop blur adds little but is GPU-expensive and triggers forced reflow
(browser perf 'Violation' logs). Keep translucency + depth; reserve blur for the few
.glass overlay surfaces (menus, panels, toasts, tooltips).
2026-06-11 22:04:01 +02:00
npeter83
3f4d5ed9d7 fix(dev): vite proxy to 127.0.0.1 + throttle error notifications
The dev proxy targeted 'localhost', which Node can resolve to IPv6 ::1 that the
Docker publish doesn't answer after a container recreate — every /api call failed,
spamming 'Network error'. Pin the proxy to 127.0.0.1. Also collapse bursts of
connection/5xx failures into one notification per 30s so a brief restart no longer
floods the notification center.
2026-06-11 22:00:57 +02:00
npeter83
bc9b9a854c fix(ui): portal tooltips, opaque glass, settings rail+stable height, notif test, channel help
- Tooltip: render in a portal with fixed positioning + edge-flip so hints are never
  clipped by overflow/stacking ancestors (fixes mispositioned/hidden bubbles app-wide).
- Glass: raise opacity so overlay menus/panels stay readable over content.
- SettingsPanel: vertical tab rail (no wrapping/jumping), content grid-stacked so the
  panel sizes to the tallest tab (stable height) and floats to its content height.
- Notifications: the test toast is now a normal auto-dismissing toast (with countdown
  bar) that also plays the sound via a new force-sound flag.
- Channel manager: explain priority/tags/hide and what 'Sync subscriptions' does;
  add a 'Channel priority' feed sort so priority is actually meaningful.
2026-06-11 21:30:25 +02:00
npeter83
002579e5e5 feat(ui): liquid-glass design system, settings polish, hints, notif fixes
- Add a theme-aware glass surface system (.glass/.glass-card + ambient backdrop,
  performance-mode opt-out) and apply it across panels, popovers, toasts, cards,
  sidebar widgets, channel rows, video cards and login.
- SettingsPanel: slide in/out animation, glass styling, wrapping pill tabs (no
  horizontal scrollbar) with a prominent active state.
- Notifications: auto-dismiss can be switched off (stays until closed); the test
  notification now also triggers the alert sound; resume a suspended AudioContext.
- Add an app-wide, toggleable hint/tooltip system (lib/hints + Tooltip) and wire
  hints across the settings and channel-manager surfaces; persisted per account.
2026-06-11 21:08:35 +02:00
npeter83
a8822d3935 feat(m5a): channel manager, tabbed settings panel, per-user sync status
Backend: /api/channels (list + PATCH priority/hidden + attach/detach user tags),
user-tag CRUD on /api/tags, /api/sync/my-status (per-user channel sync counts).
Frontend: feed|channels page nav (URL-synced) from the account menu; a slide-in
tabbed Settings panel (Appearance, Notifications=6b sound+duration, Sync status +
actions + admin pause/resume, Account); a channel manager with priority, hide,
per-channel user tags, sync badges and 'view in feed'. Notifications now honor the
configurable sound + auto-dismiss settings.
2026-06-11 20:45:48 +02:00
npeter83
b8a8605797 merge: M4 polish — sidebar widgets, URL state, notification center, account menu
Account hover-popup, upload-date chips, top clear-filters bar, filters reflected in
the URL, a customizable (collapsible/reorderable/toggleable) sidebar, and a client-side
notification center with leveled toasts, countdown bars and in-app hide/unhide actions.
2026-06-11 19:59:26 +02:00
npeter83
a419ac2943 fix(ui): notification UX — outside-click close, toast countdown, hide entry actions
- NotificationCenter closes on outside click (document listener, not an overlay
  that the blurred header trapped) and no longer needs a second bell click.
- Toasts show a level-colored countdown bar and auto-dismiss faster (6s default).
- Hidden-video notifications carry structured meta so the center offers an in-app
  'Find in feed' (jump to that channel's hidden videos) and a one-click 'Unhide',
  working even after a reload when the live Undo callback is gone.
2026-06-11 19:46:58 +02:00
npeter83
a105e5c184 feat(ui): notification center with leveled toasts and error capture
- Replace the toast store with a notification store: levels (info/success/
  warning/error/fatal), requiresInteraction, and a persisted history.
- Move toasts to the top-right, styled per level, with manual dismiss.
- Add a bell in the header opening a Notification Center (history, unread badge,
  'needs attention' vs info, clear all).
- Capture network failures and 5xx responses (api layer) and render crashes
  (ErrorBoundary) as notifications.
- Sound + server-sourced events + per-account settings remain for 6b.
2026-06-11 19:26:34 +02:00
npeter83
ae0cd89e20 feat(ui): customizable sidebar — collapsible, reorderable, toggleable widgets
Each filter group is now a card with a collapse chevron. An Edit mode (pencil)
reveals drag handles (@dnd-kit) to reorder and eye toggles to show/hide widgets,
plus Reset to defaults. Layout (order/collapsed/hidden) persists to localStorage
and the server preferences blob, adopted on login.
2026-06-11 19:21:48 +02:00
npeter83
98025d8fab feat(ui): reflect filters/sort/search in the URL
Serialize the active filters to a compact, readable query string (only non-default
values) and parse them back on load, with URL taking precedence over localStorage.
Uses history.replaceState so it never spams browser history. A pasted link now
reproduces the exact view.
2026-06-11 19:17:08 +02:00
npeter83
3e1823b08b feat(ui): account hover-popup, upload-date chips, top clear-filters bar
- Header: avatar opens an account popover (hover + click) with identity, admin
  badge and Sign out; removes the standalone logout icon.
- Sidebar: relative upload-date chips (24h/1w/1m/6m/1y) backed by max_age_days,
  with a Custom toggle revealing the from/to range; mutually exclusive.
- Sidebar: 'Filters · N active · Clear all' moved to the top; drop bottom button.
2026-06-11 17:39:24 +02:00
npeter83
3afb284d46 chore(frontend): add package-lock.json for reproducible installs 2026-06-11 17:39:24 +02:00
npeter83
6120e6b0f8 merge: shared-database deployment (24/7 backfill)
shared database + scheduler in the server; local dev points at it with the
scheduler off. Adds docker-compose.server.yml / docker-compose.localdev.yml, the
shared-database docs, and the nested Docker apparmor fix.
2026-06-11 16:59:09 +02:00
npeter83
65c75d8f05 fix(deploy): apparmor:unconfined for nested Docker
The docker-default AppArmor profile can't be loaded from inside the container; match the
lab convention (finance/arr stacks) of running services unconfined.
2026-06-11 16:42:44 +02:00
npeter83
fd1ac29704 feat(deploy): shared database + scheduler split for the server 24/7 backfill
Add a server compose (full stack, Postgres on the LAN, scheduler on, host-visible
pgdata bind mount) and a localdev compose (webapp only, no DB, scheduler off) that
points at the shared database. Document the shared-database topology, the
exactly-one-scheduler rule, and YOUTUBE_API_KEY for unattended backfill that does
not depend on a 7-day OAuth refresh token.
2026-06-11 16:35:00 +02:00
npeter83
93e1d7afc2 merge: M4 reader UI 2026-06-11 04:27:31 +02:00
npeter83
21662534bc docs: mark M4 (reader UI) complete in README 2026-06-11 04:27:31 +02:00
npeter83
f6c5488566 chore: structured timestamped logging across the backend
- uvicorn --log-config (log_config.json): timestamped formatters for uvicorn
  access/error and the app's own "subfeed" loggers (ms precision)
- Log key activity: startup/shutdown, login/denied, token refresh, YouTube API
  errors, subscription imports, sync pause/resume
- Background sync jobs no longer swallow errors silently — failures in RSS poll,
  backfill, enrichment, autotag and resync are logged with tracebacks
2026-06-11 04:26:18 +02:00
npeter83
ecaf516429 chore: surface scheduler logs in container output (INFO for subfeed loggers) 2026-06-11 04:19:54 +02:00
npeter83
dc73b43b71 feat: sync status indicator, admin pause/resume, filtered video count
- Header shows a live sync status (total videos + channels still backfilling, or
  "paused"), polled every 30s
- Admins can pause/resume the background sync; a paused flag in app_state makes
  scheduled jobs skip (migration 0006)
- GET /api/feed/count returns the number of videos matching the current filters;
  shared filter builder keeps it in sync with /api/feed; shown above the feed
- /api/sync/status reports backfill progress, pending enrichment and paused state
2026-06-11 04:15:25 +02:00
npeter83
f73cbdb490 feat: unhide action, date-range filter, more sort options
- Hidden view shows an Unhide action (eye icon) instead of Hide
- Upload-date filter: From/To date range (inclusive); feed shows only videos
  published in that window (backend published_after / published_before)
- New sort options: Name (A-Z) and Channel subscribers, alongside date/views/
  duration/shuffle
- Native controls follow the theme via color-scheme (dark date picker)
2026-06-11 04:00:17 +02:00
npeter83
ecbecbb9f4 feat: feedback round 3 — content-type toggles, channel filter, card polish
- Content type is now three independent toggles (Normal / Shorts / Live·Upcoming);
  the feed is the union of enabled types (backend show_normal + include_shorts +
  include_live)
- Per-channel filter: a button on each card scopes the feed to that channel, shown
  as a removable chip in the sidebar
- Hide now refetches the feed once the change is persisted, so a hidden video shows
  up in the Hidden view immediately (no refresh needed); optimistic updates respect
  the current view
- Grid cards are now distinct panels (card background, border, shadow) that lift on
  hover; clickable channel name in list view too
2026-06-11 03:47:51 +02:00
npeter83
e07a37622d fix: feedback round 2 — language, subscriptions, feed scope, UX
- Language detection: classify one cleaned+concatenated blob (strip emoji,
  @mentions, #tags, numbers, punctuation); fixes caps/emoji-heavy channels
  (e.g. Nessaj -> Hungarian, no more bogus Chinese/Korean)
- Feed now joins the user's subscriptions, so unsubscribing on YouTube removes a
  channel from the feed; periodic subscription re-sync job picks up changes
- Watched/Saved/Hidden views ignore the Shorts/live default-hiding so the full
  set is visible (fixes hidden videos missing from the Hidden view)
- Persist feed filters + search across reloads (localStorage)
- 3D polish: cards lift with shadow on hover; chips/buttons get depth and a
  press effect; undo toast lasts longer
2026-06-11 03:28:45 +02:00
npeter83
8c245e986f fix: address reader-UI feedback (shorts, search, tags, login, UX)
- Shorts: confirm via youtube.com/shorts/<id> probe (SOCS cookie bypasses the
  consent redirect) instead of a 60s heuristic; concurrent probing, shorts_probed
  flag, scheduled refinement (migration 0005)
- Search: match title + channel name only (descriptions caused noisy results)
- Faceted tag filtering: AND across categories (language AND topic narrows),
  OR within a category; any/all toggle applies to topics
- Language detection: majority vote over individual titles (fixes misdetections
  like multipoleguy -> English; drops bogus Polish/Romanian)
- Login: drop forced consent so returning sign-in is quick (select_account)
- Feed cards: clickable channel name (opens channel), persistent saved badge,
  undo toast on hide, Hidden view to restore; tag chips show counts in tooltip
2026-06-11 03:07:49 +02:00
npeter83
e56502789f feat: M4 (part 2) — React reader UI with theming
- Vite + React + TS + Tailwind SPA served by FastAPI (multi-stage Docker build)
- Four color schemes (Midnight default, Forest, Slate, YouTube) x dark/light,
  adjustable text size; persisted to user preferences and localStorage
- Header search, grid/list toggle, theme menu; sidebar filters (show state,
  sort, include Shorts/live, language + topic tag chips with any/all matching)
- Infinite-scroll feed of video cards; click opens youtube.com in a new tab;
  per-video watched / saved / hidden actions with optimistic updates
- SPA fallback routing; login screen for unauthenticated users
2026-06-11 02:19:47 +02:00
npeter83
9a377b7e7e feat: M4 (part 1) — feed API, watch state, user preferences
- GET /api/feed: filter by tags (and/or), channel, duration, age, search; sort
  newest/oldest/views/duration/shuffle; default hides Shorts, live/upcoming and
  watched (was_live VODs stay visible); offset paging with has_more
- POST /api/videos/{id}/state for watched/saved/hidden/new
- User.preferences JSON + GET /api/me and PUT /api/me/preferences
- Migration 0004 (preferences column + video_states table)
2026-06-11 02:11:02 +02:00
npeter83
96f9c5a797 merge: M3 auto-tagging 2026-06-11 01:57:19 +02:00
npeter83
68dad91e8a feat: M3 — automatic channel tagging (language + topic system tags)
- Tag and ChannelTag models + migration 0003 (partial unique indexes split
  system vs per-user tag names)
- Offline language detection (py3langid) constrained to a curated language set,
  with the channel's declared default language as a strong prior
- Topic tags mapped from YouTube topicDetails + dominant video category; the
  generic "Lifestyle" catch-all is intentionally dropped
- System (auto) tags are regenerated idempotently and never touch user tags;
  orphaned system tags are cleaned up
- GET /api/tags and admin POST /api/tags/recompute; scheduled autotag job
2026-06-11 01:57:19 +02:00
npeter83
64b18b3cc1 merge: M2 ingest core 2026-06-11 01:36:41 +02:00
npeter83
cff1d23071 feat: M2 (part 2) — RSS poller, backfill, enrichment, scheduler
- Free per-channel RSS reader for quota-less fresh-video detection
- Recent-first backfill (configurable: 100 videos / 1 year) plus resumable deep
  backfill from the uploads playlist
- Enrichment via videos.list: duration, view/like counts, category, topics,
  language, Shorts heuristic and livestream/premiere classification
- Reusable sync runners + APScheduler jobs (rss / enrich / backfill), all
  quota-aware with a reserve so backfill never starves fresh enrichment
- Manual triggers: POST /api/sync/{rss,backfill,enrich}
- Exact insert counting via RETURNING with in-batch de-duplication
2026-06-11 01:36:41 +02:00
npeter83
24b6e0026d feat: M2 (part 1) — subscription import, YouTube client, quota guard
- Channel/Subscription/Video/ApiQuotaUsage models + migration 0002
- Synchronous YouTube Data API client with OAuth token refresh and per-call
  quota accounting (API key preferred for public reads when configured)
- Subscription import: upsert channels + subscription links, prune unsubscribed,
  fetch channel details (uploads playlist, topics, stats, handle, country)
- Central quota guard tracking units per US-Pacific day against a daily budget
- POST /api/sync/subscriptions and GET /api/sync/status endpoints
2026-06-11 01:22:07 +02:00
npeter83
3a774cf7d6 fix: request offline access on OAuth login so Google returns a refresh token 2026-06-11 01:14:06 +02:00
npeter83
10ef62bbad fix: default app UI language to English (login page) 2026-06-11 01:03:36 +02:00