/proxy's regex-based HTML rewriting is fragile and applies site-specific hacks to every proxied page #5

New issue

Closed

opened 2026-07-28 21:01:27 +00:00 by paulloth1 · 1 comment

paulloth1 commented

2026-07-28 21:01:27 +00:00

(Migrated from github.com)

Problem

backend/app.py's /proxy route rewrites fetched HTML using several hand-rolled regexes applied unconditionally to every proxied page, regardless of hostname:

SCRIPT_TAG_RE / HEAVY_SCRIPT_HOSTS — strips <script src="..."> tags from opencmp.net/cdntrf.com (fine, scoped by host).
THEME_LINK_RE — force-rewrites any <link id="theme"> tag's light.min.css to dark.min.css. This is a dbf.finalrewind.org-specific convention (their theme-switch mechanism), but nothing scopes it to that host — if some other proxied site happens to use id="theme" with a light.min.css-style href, this will silently point it at a stylesheet path that doesn't exist on that site, breaking its styling.
INLINE_SCRIPT_RE — strips any inline <script> (no src) whose body contains the string "prefers-color-scheme", again on every proxied site, to stop finalrewind.org's theme-detection script from re-asserting light mode. A legitimate, unrelated inline script on some other proxied site that happens to mention that CSS media feature would also get silently deleted.

More generally, matching/removing tags with regex against arbitrary third-party HTML is inherently fragile (nested quotes, minified script bodies containing the literal string </script> inside a JS string literal, attribute ordering, etc. can all produce wrong matches).

Suggested fix

Scope the theme-forcing rewrites (THEME_LINK_RE, INLINE_SCRIPT_RE) to urlparse(url).hostname == "dbf.finalrewind.org" (or a small per-host rewrite-rules table) instead of applying them globally.
Consider replacing the regex approach with a real HTML parser (e.g. lxml.html or BeautifulSoup) for the tag stripping/rewriting, which handles malformed/edge-case markup correctly and makes host-scoped rules much easier to express and extend.

## Problem `backend/app.py`'s `/proxy` route rewrites fetched HTML using several hand-rolled regexes applied unconditionally to **every** proxied page, regardless of hostname: - `SCRIPT_TAG_RE` / `HEAVY_SCRIPT_HOSTS` — strips `<script src="...">` tags from `opencmp.net`/`cdntrf.com` (fine, scoped by host). - `THEME_LINK_RE` — force-rewrites **any** `<link id="theme">` tag's `light.min.css` to `dark.min.css`. This is a `dbf.finalrewind.org`-specific convention (their theme-switch mechanism), but nothing scopes it to that host — if some other proxied site happens to use `id="theme"` with a `light.min.css`-style href, this will silently point it at a stylesheet path that doesn't exist on that site, breaking its styling. - `INLINE_SCRIPT_RE` — strips **any** inline `<script>` (no `src`) whose body contains the string `"prefers-color-scheme"`, again on every proxied site, to stop `finalrewind.org`'s theme-detection script from re-asserting light mode. A legitimate, unrelated inline script on some other proxied site that happens to mention that CSS media feature would also get silently deleted. More generally, matching/removing tags with regex against arbitrary third-party HTML is inherently fragile (nested quotes, minified script bodies containing the literal string `</script>` inside a JS string literal, attribute ordering, etc. can all produce wrong matches). ## Suggested fix - Scope the theme-forcing rewrites (`THEME_LINK_RE`, `INLINE_SCRIPT_RE`) to `urlparse(url).hostname == "dbf.finalrewind.org"` (or a small per-host rewrite-rules table) instead of applying them globally. - Consider replacing the regex approach with a real HTML parser (e.g. `lxml.html` or `BeautifulSoup`) for the tag stripping/rewriting, which handles malformed/edge-case markup correctly and makes host-scoped rules much easier to express and extend.

paulloth1 commented

2026-07-28 21:22:56 +00:00

(Migrated from github.com)

Fixed in a841b02 (merged to main at 103cdf2): THEME_LINK_RE/INLINE_SCRIPT_RE now only apply when urlparse(url).hostname == "dbf.finalrewind.org". HEAVY_SCRIPT_HOSTS stripping, the tag injection, and COOKIE_BANNER_CSS remain global since those are legitimately host-agnostic.

Fixed in a841b02 (merged to main at 103cdf2): THEME_LINK_RE/INLINE_SCRIPT_RE now only apply when `urlparse(url).hostname == "dbf.finalrewind.org"`. HEAVY_SCRIPT_HOSTS stripping, the <base> tag injection, and COOKIE_BANNER_CSS remain global since those are legitimately host-agnostic.