Each of the 4 backend processes has its own independent rotation state — "View now" / "Now displaying" silently target the wrong process on any LAN port #1

New issue

Closed

opened 2026-07-28 21:00:32 +00:00 by paulloth1 · 1 comment

paulloth1 commented

2026-07-28 21:00:32 +00:00

(Migrated from github.com)

Problem

backend/rotation.py keeps rotation state (STATE = RotationState(), current slide id, forced-slide requests, "now displaying" info) as an in-process, module-level singleton, and backend/app.py's lifespan() starts its own asyncio.create_task(rotation_loop()) per process.

We run four separate uvicorn processes (see README architecture table): backend.service (127.0.0.1:8081, kiosk-internal), backend-http.service (0.0.0.0:80), backend-tls.service (0.0.0.0:8080), backend-tls-443.service (0.0.0.0:443). Each is a fully independent OS process/Python interpreter, so each one has its own separate RotationState instance and its own independent rotation loop, all reading the same SQLite DB on the same schedule but never sharing in-memory state with each other.

The kiosk display itself only ever polls http://127.0.0.1:8081/api/slide/current — i.e. only backend.service's rotation loop actually drives what's on the TV.

Impact

The admin UI is only reachable on ports 80/443/8080 (8081 is loopback-only) — i.e. every real LAN user's browser talks to a different process than the one driving the display.
Clicking "View now" from the LAN-accessible admin UI calls STATE.request_forced() on that process's STATE — which has no effect on the actual display, since the display polls a completely different process's state.
The "Now displaying" label, the highlighted table row, and the "time left" countdown in the admin UI reflect that process's own independent rotation cycle, not necessarily what's actually on screen. Since all four processes start around the same time and use the same interval, they usually stay roughly in phase — but drift (and any forced push) breaks that immediately and silently.
Side effect: outbound polling (Mastodon/Matrix/train is_available()/render() HTTP calls) happens 4x redundantly, once per process, every rotation cycle.

Suggested fix

Move rotation state out of process memory and into something shared across all four processes — e.g. back it with the SQLite DB (already shared), or run a single rotation-owning process and have the other three become thin proxies to it (e.g. over a local socket/HTTP call to the 8081 instance), or collapse to a single process serving multiple ports via an ASGI server capable of multi-bind (if feasible) instead of 4 separate processes.

## Problem `backend/rotation.py` keeps rotation state (`STATE = RotationState()`, current slide id, forced-slide requests, "now displaying" info) as an **in-process, module-level singleton**, and `backend/app.py`'s `lifespan()` starts its own `asyncio.create_task(rotation_loop())` per process. We run **four separate `uvicorn` processes** (see README architecture table): `backend.service` (127.0.0.1:8081, kiosk-internal), `backend-http.service` (0.0.0.0:80), `backend-tls.service` (0.0.0.0:8080), `backend-tls-443.service` (0.0.0.0:443). Each is a fully independent OS process/Python interpreter, so each one has **its own separate `RotationState` instance and its own independent rotation loop**, all reading the same SQLite DB on the same schedule but never sharing in-memory state with each other. The kiosk display itself only ever polls `http://127.0.0.1:8081/api/slide/current` — i.e. only `backend.service`'s rotation loop actually drives what's on the TV. ## Impact - The admin UI is only reachable on ports 80/443/8080 (8081 is loopback-only) — i.e. **every real LAN user's browser talks to a different process than the one driving the display.** - Clicking **"View now"** from the LAN-accessible admin UI calls `STATE.request_forced()` on that process's `STATE` — which has no effect on the actual display, since the display polls a completely different process's state. - The **"Now displaying" label**, the highlighted table row, and the "time left" countdown in the admin UI reflect that process's own independent rotation cycle, not necessarily what's actually on screen. Since all four processes start around the same time and use the same interval, they usually stay roughly in phase — but drift (and any forced push) breaks that immediately and silently. - Side effect: outbound polling (Mastodon/Matrix/train `is_available()`/`render()` HTTP calls) happens **4x redundantly**, once per process, every rotation cycle. ## Suggested fix Move rotation state out of process memory and into something shared across all four processes — e.g. back it with the SQLite DB (already shared), or run a single rotation-owning process and have the other three become thin proxies to it (e.g. over a local socket/HTTP call to the 8081 instance), or collapse to a single process serving multiple ports via an ASGI server capable of multi-bind (if feasible) instead of 4 separate processes.

paulloth1 commented

2026-07-28 21:19:28 +00:00

(Migrated from github.com)

Fixed in 8835778: only the kiosk-internal process (127.0.0.1:8081, env SIGNAGE_ROTATION_OWNER=1) now runs the rotation loop; the other 3 processes forward /api/slide/current reads and /admin/slides/{id}/view-now writes to it over loopback HTTP. Verified live: a view-now POST sent to port 80 is now reflected identically on port 8081. Also took the opportunity to move the admin UI from /admin to / and the kiosk display from / to /display, per discussion.

Fixed in 8835778: only the kiosk-internal process (127.0.0.1:8081, env SIGNAGE_ROTATION_OWNER=1) now runs the rotation loop; the other 3 processes forward /api/slide/current reads and /admin/slides/{id}/view-now writes to it over loopback HTTP. Verified live: a view-now POST sent to port 80 is now reflected identically on port 8081. Also took the opportunity to move the admin UI from /admin to / and the kiosk display from / to /display, per discussion.