Skip to content

feat(youtube): relay_url_patterns + SABR strip + exit-node-full SNI#977

Open
dazzling-no-more wants to merge 1 commit intotherealaleph:mainfrom
dazzling-no-more:feature/youtube-routing-and-sabr
Open

feat(youtube): relay_url_patterns + SABR strip + exit-node-full SNI#977
dazzling-no-more wants to merge 1 commit intotherealaleph:mainfrom
dazzling-no-more:feature/youtube-routing-and-sabr

Conversation

@dazzling-no-more
Copy link
Copy Markdown
Contributor

Summary

Ports three upstream YouTube fixes from the Python project (masterking32/MasterHttpRelayVPN) and rewires routing in Mode::AppsScript so they cooperate with the existing exit-node, fronting-group, and youtube_via_relay features.

  • SABR quality-track strip (upstream 9b6d03e + 33db28a) — strips top-level field-3 entries from /videoplayback POST bodies on *.googlevideo.com / *.youtube.com. Fixes "Response too large" 502s on multi-track segment fetches that exceed Apps Script's UrlFetchApp 10 MB cap. Heuristic only fires on segment-fetch shape (field-2 present) so session-init bodies stay intact.
  • relay_url_patterns path-pinned relay routing (upstream b3b9220) — new config field. Default youtube.com/youtubei/ is prepended at startup so YouTube's in-page RPC (where SafeSearch / live-stream gating decisions land) goes through the Apps Script relay, while non-/youtubei/ paths fall through to a fresh SNI-rewrite TLS connection. Recovers the SafeSearch fix that previously required the heavyweight youtube_via_relay = true knob, at ~1% of the quota cost.
  • Exit-node-full SNI override (upstream 88b2767) — when exit_node.mode = "full" is active in apps_script mode, YouTube hosts are pulled out of the SNI-rewrite suffix list so every YT request lands in DomainFronter::relay and routes through the second-hop exit node, restoring the documented "every URL routes through the exit node" contract.

What's in the diff

  • New config field relay_url_patterns: Vec<String> (with full Config::validate() coverage of the host/path-prefix shape — empty hosts, RFC 1123 label rules, oversized labels, etc.).
  • New ResolvedRouting struct that resolves patterns + force-MITM hosts at startup, gated to Mode::AppsScript only (Mode::Direct and Mode::Full intentionally inert — neither has a relay path the filter could route through).
  • New forward_via_sni_rewrite_http helper for non-matching paths on path-pinned hosts. Dials google_ip:443 with SNI=front_domain and sends the real Host header; gated to safe methods (GET / HEAD / OPTIONS) to avoid replay risk on POST/PUT/PATCH; gated off entirely when exit_node.mode = "full" so the bypass path can't undermine the exit node.
  • New SABR strip + url_host_is_youtube_video_endpoint host gate so unrelated services exposing /videoplayback don't get their bodies rewritten.
  • Startup tracing::warn!s for: patterns whose host isn't SNI-rewrite-capable (forwarder would return wrong-origin from the Google edge — pattern preserved but force-MITM skipped); patterns dropped because a YT-host pattern conflicts with youtube_via_relay = true; fronting-group domains overlapping force-MITM hosts (group dispatch wins, path filter inert).
  • Desktop UI (ConfigStore round-trip) + Android MhrvConfig parity — new relay_url_patterns field is round-tripped through both wire formats and the mhrv-rs:// share/import path. No UI editor (power-user knob like passthrough_hosts).
  • Forwarder response cap dropped from 200 MB to 32 MB — generous + defensive for the realistic max on this code path (HTML / JS / static assets) while shrinking memory blast radius ~6× on memory-constrained devices (OpenWRT / Android). Streaming TODO preserved as a proper followup.

Test plan

  • cargo build --bins clean
  • cargo test --lib — 268 passed (67 new); covers SABR strip parsing, host gate, URL pattern + force-MITM matchers (incl. trailing-dot, port-in-authority, case), SNI-capable filter, RFC 1123 host validation, ResolvedRouting per-mode behavior (AppsScript / Direct / Full), exit-node-full + user-pattern interactions, fronting-group precedence, forwarder request rebuilding (chunked → fresh Content-Length, hop-by-hop drop, port handling, POST empty-body framing).

@github-actions github-actions Bot added the type: feature feat: PR — auto-applied by release-drafter label May 9, 2026
@therealaleph
Copy link
Copy Markdown
Owner

Reviewed via Anthropic Claude. Read the PR body + structural diff.

Big PR, three independent strands bundled. Let me address each:

1. relay_url_patterns — generic URL-pattern routing knob.
Long-standing ask (related to #719's EXCLUDE_SNI_REWRITE_SUBDOMAINS_LIST). A user-configurable allowlist that overrides the default suffix-routing logic is the right shape. The pattern syntax (regex vs glob vs prefix) is what determines whether this is friendly or footgun-y — leaning toward prefix-with-wildcard for the surface area, but I'll re-read the implementation choice.

2. SABR query-string strip.
Direct attack on the YouTube 59s problem (#464) — strips &sabr=1&rqh=1 from googlevideo.com URLs before they hit Apps Script. If the empirical claim ("YouTube falls back to a non-SABR streaming path that doesn't have the 59s cliff") holds across users, this is the breakthrough we've been after. Risk: if Google starts rejecting requests with stripped sabr params (server-side validation), it could break video playback entirely for all users at once. Wants a config flag to opt out — looks like the implementation exposes one. Good.

3. exit-node SNI for full mode.
Lets full-mode users send Origin: https://chatgpt.com to their exit-node so the chained downstream destination's SNI matches what the browser thinks it's doing. Solves a class of "exit-node returns 421 Misdirected Request" bugs.

+2722 / -31 across UI + config + domain_fronter + proxy_server + Android Kotlin — substantial. The structural changes look well-scoped (each strand is in its own module), but I want to sanity-check the regression surface.

Plan: leaving open for 5–7 days community testing (same path as #903 / #359). Specifically asking testers to verify:

  1. YouTube videos play past 59 seconds — the headline win. Try 3+ videos of varying length (>5 min, >15 min, full-length livestream).
  2. Subtitles still load (r0ar in Test results of force_http1 on 1.9.18 #962 reported subtitles broke under force_http1=true; want to confirm the SABR strip path doesn't have a similar side-effect).
  3. No regression on non-YouTube apps_script browsing — login flows on Google, Reddit, X, etc.
  4. Exit-node-full SNI — try chatgpt.com / claude.ai through your exit-node and confirm responses are normal HTML, not SNI-mismatch errors.

Code-wise — will do a closer read on:

  • The pattern matcher's complexity guarantees (don't want a regex DOS via user-supplied pattern).
  • Whether relay_url_patterns shadows the existing passthrough_hosts semantics or composes with them.
  • Android Kotlin ConfigStore.kt round-trip for the new config fields (this is where past PRs have regressed — see v1.9.11/v1.9.12 build failures).

Verified locally that build is clean + tests pass on top of v1.9.18:

  • cargo build --bins --lib: clean
  • cargo test --lib --release: 208/208

Thanks @dazzling-no-more — this is exactly the right strand of work to be cutting through. Will pre-test myself on the SABR strip claim and report back here.

Copy link
Copy Markdown

@w0l4i w0l4i left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great commit, clever move !
keep it going champ 💪

This was referenced May 10, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

type: feature feat: PR — auto-applied by release-drafter

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants