peter/siftlode
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

feat(deploy): central Postgres + scheduler split for Proxmox 24/7 backfill

Browse source 
Add a server compose (full stack, Postgres on the LAN, scheduler on, host-visible
pgdata bind mount) and a localdev compose (webapp only, no DB, scheduler off) that
points at the central DB. Document the single-shared-database topology, the
exactly-one-scheduler rule, and YOUTUBE_API_KEY for unattended backfill that does
not depend on a 7-day OAuth refresh token.
This commit is contained in:
npeter83 2026-06-11 16:35:00 +02:00
parent 93e1d7afc2
commit 79e328af8d
4 changed files with 136 additions and 1 deletions
Download patch file Download diff file Expand all files Collapse all files

29
README.md
View file

@ -61,7 +61,7 @@ once and stored locally, so filtering/searching/sorting are instant and don't bu
## Backup & moving to another machine
All data lives in the `pgdata` Postgres volume — moving to another host (e.g. a Proxmox Linux
All data lives in the Postgres database — moving to another host (e.g. a Proxmox Linux
server) does **not** require re-fetching from YouTube. Copy your `.env` (keep the same
`TOKEN_ENCRYPTION_KEY` and Google client so stored tokens stay valid), then:
@ -70,6 +70,33 @@ server) does **not** require re-fetching from YouTube. Copy your `.env` (keep th
./scripts/restore.sh backups/<file> # on the new host after `docker compose up`
```
## Central server + local dev (single shared database)
For always-on operation the recommended topology is a single **central Postgres** running on a
24/7 host (e.g. a Proxmox LXC) that also runs the background scheduler, with every other instance
(your local dev machine, a future VPS) pointing at that same database. One source of truth, no
sync.
- **Server** (`docker-compose.server.yml`): full stack, Postgres published on the LAN, scheduler
on, DB files in a host-visible `./pgdata` bind mount. In the LXC, from the project root:
```sh
echo 'COMPOSE_FILE=docker-compose.server.yml' >> .env # so backup/restore target this stack
docker compose up -d --build
./scripts/restore.sh backups/<file> # migrate existing data in
```
Set `YOUTUBE_API_KEY` in `.env` so unattended backfill/enrichment use the API key and don't
depend on a refresh token (which expires after 7 days while the OAuth screen is in *Testing*).
- **Local dev** (`docker-compose.localdev.yml`): webapp only, no local DB, no scheduler. In `.env`
set `DATABASE_URL` to the central DB (e.g. `…@your-db-host:5432/subfeed`), then
`docker compose -f docker-compose.localdev.yml up --build` and browse http://localhost:8080.
**Exactly one instance may run the scheduler.** The server keeps `SCHEDULER_ENABLED=true`; every
other instance must be `false` (the compose files enforce this) to avoid double quota burn and
write races on the shared DB.
## Tech
FastAPI + PostgreSQL backend, React + Vite frontend (added in a later milestone), packaged with