mirror of https://github.com/zoriya/v10.git synced 2026-08-16 02:45:09 +00:00

Files

T

00aa6247b8 feat(spf): multi-cdn support (#1668 )

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

2026-06-11 10:09:16 -07:00

16 KiB

Raw Blame History

status, date, definition

status	date	definition
draft	2026-06-05	sketched

Multi-CDN failover

CDN selection and failover for HLS sources that publish the same content on more than one host (e.g. Mux Video's ?redundant_streams=true). The redundant variants parse as ordinary candidate tracks — one per (rendition × CDN) — so the work is selecting which CDN to use and keeping the whole presentation on it, modeled inside the track-switching rule model rather than as URL rewriting at fetch time.

Two sub-features, mapping onto the two rule kinds in that model:

Sticky CDN pick (implemented) — a session-level behavior picks a CDN (the manifest-head host) and holds it; a shared scope rule (preferActiveCdn) narrows every track type's candidates to that CDN, so video / audio / text resolve from one host. This is the active-pathway scope the track-switching model lists for multi-cdn-failover.
Failover (deferred) — when requests to the active CDN fail too often within a window, a constraint removes that CDN's tracks from the candidate set during cooldown and the session behavior rotates the active CDN. This is the failed-CDN constraint the model lists, and it consumes network-resilience's per-host circuit-breaker — its hard prerequisite.

A Media-src feature for sources that genuinely require failover, with a Player feature surface at the failover tier (customer-customizable CDN policy). Notion epic #9 classifies as "Media-src? / Player?" — the ambiguity reflects that split.

Status

Composition: sub-feature 1 (sticky CDN pick) is implemented in createSimpleHlsEngine and createHlsAudioOnlyEngine. The resolveCdnPriority behavior owns the cdnPriority signal (the manifest-ordered CDN list, most-preferred first); the preferActiveCdn scope rule (shared by the video + audio chains in track-switching) narrows candidates to the highest-priority CDN with surviving tracks. Failover (sub-feature 2) is not implemented — no constraints pass, no per-CDN failure tracking, no rotation.
Definition depth: sketched — sub-feature 1 has a populated implementation surface + verification; sub-feature 2 stays at the scope-and-constraints level pending its prerequisite.
Hard prerequisite (failover only): network-resilience. The failed-CDN constraint consumes the foundation's per-host circuit-breaker / retry-exhaustion state. Sub-feature 1 has no such dependency — it's pure selection over the already-parsed candidate set.
Governing model: track-switching-model.md — multi-CDN is the canonical constraint + scope feature there. The active-CDN scope is a soft filter in the rule chain; the failed-CDN constraint is a hard filter in the (not-yet-built) constraints pre-pass.

How redundant streams are modeled

There is no alternateUris field and no fetch-time URL rotation. A redundant-streams manifest lists each rendition once per CDN (duplicate #EXT-X-STREAM-INF / #EXT-X-MEDIA entries on different hosts), and the existing parser already emits one Track per entry with a unique id and its own absolute url. So the candidate set for each type already contains one variant per CDN. CDN identity is derived from each track URL's origin (getCdnId); the set of CDNs is published as a single per-presentation ordered signal (cdnPriority, most-preferred first — mirroring HLS content steering's PATHWAY-PRIORITY). The active CDN is not stored: the scope derives it as the highest-priority cdnPriority entry that still has tracks after the constraints pass. Selecting a CDN-tagged track id means resolveTrack / segment loading fetch from that CDN with no further plumbing.

This list shape is what makes failover fall out cleanly: the failed-CDN constraint (sub-feature 2) prunes a cooled-down CDN's tracks, so "first-with-survivors" moves to the next CDN automatically and returns to the primary when it recovers — no reactive rewrite of an "active" value. Content steering, likewise, just reorders cdnPriority (pathway priority as a sort key).

This is why sub-feature 1 needed no parser change and no new data shape — only a selection behavior and a scope rule over existing tracks.

Phases of complexity

Phase	Sub-feature	Kind	What	State
Sticky CDN pick	1	scope	`resolveCdnPriority` publishes the manifest-ordered CDN list (`cdnPriority`); `preferActiveCdn` narrows every type's candidates to the highest-priority CDN with surviving tracks, falling through when nothing matches. Shared list → all types on one CDN	Implemented
Failed-CDN constraint	2	constraint	A constraint in the track-switching constraints pre-pass removes a cooled-down CDN's tracks from the candidate set. The scope then picks the next `cdnPriority` entry automatically. Requires the generic constraints phase (track-switching-model "Phase 2") to be built first	Deferred
Per-CDN failure tracking	2	—	Count per-CDN fetch failures within a window; mark a CDN unhealthy / in cooldown. Consumes `network-resilience`'s circuit-breaker	Deferred (prereq)
CDN priority override / steering	2	scope	Reorder `cdnPriority` to bias the pick (region-preferred, weighted, or content-steering's pathway priority). No reactive "active" rewrite needed — the order is the policy	Deferred
Customer CDN-id derivation	2	config	Pluggable `getCdnId` for non-origin identity. The origin-based default is built in; a config seam is anticipated	Deferred

What's in scope vs out of scope

In scope:

Sticky per-presentation CDN selection (sub-feature 1, done)
Failover via a track-switching constraint + active-CDN rotation (sub-feature 2)
Per-CDN health derived from network-resilience's circuit-breaker
Customer-configurable CDN-id derivation / rotation policy

Out of scope (separate cluster G sister features):

network-resilience (foundation, prerequisite for failover) — retry + backoff + circuit-breaker. Multi-CDN consumes; doesn't reimplement.
content-steering — HLS content-steering protocol (server-advertised, dynamically-updated host pool). Different mechanism than static redundant streams, but the same active-pathway scope shape: content-steering picks the active pathway dynamically; the scope reflecting it is the one implemented here. Designed-with-in-mind: cdnPriority is a reorderable list a steering behavior writes (pathway priority as a sort key), and the scope honors the order unchanged.

Out of scope (different architectural layer):

Adapter-layer customer-facing UI ("Switch CDN" buttons). Consumer policy is expressed via the failover tier's config seam.
CDN-side load balancer / origin-shield infrastructure. Service-side.
DRM key-server failover — lives under drm-support.

Likely cross-cutting impact

Per-presentation, not per-rendition (resolved). A single cdnPriority list governs all track types — the per-presentation coherence requirement. The track-switching model's "cross-type consistency is a composition convention" applies: both the video and audio chains reference the same preferActiveCdn definition reading the same list, so they agree on the CDN even if their per-type track arrays differ. (The doc's earlier per-rendition lean is superseded.)
cdnPriority writer composition. resolveCdnPriority is the sole writer today (publishes the manifest order). Failover needs no second writer — the failed-CDN constraint prunes tracks and the scope re-derives the active CDN. Content-steering would reorder cdnPriority (still a single owning behavior; the list reflects one upstream priority).
Active CDN is derived, not stored. The scope computes "highest priority with surviving tracks," so failover and recovery need no extra state: pruning moves the pick to the next CDN, un-pruning returns it to the primary. This is why the array beats a single reactive activeCdn value — the failed-set information is applied once (in the constraint), not duplicated into an active-value rewrite.
Constraints phase is a shared prerequisite. The failed-CDN constraint can't land until the generic constraints pre-pass (applyConstraints + the constraints config field + empty-playable-set terminal state) is built into setupTrackSwitching. The seam exists (candidateSet computed); the machinery does not. capability-probing shares this prerequisite.
Live + multi-CDN. During live playback the reload loop re-resolves the presentation; resolveCdnPriority re-publishes only when the CDN set changes (idempotent for a stable manifest). Cross-feature with live-stream-support (not yet implemented).
Priority state across source changes. cdnPriority tears down with the source via the resolved/unresolved cascade (per source-replacement); per-host circuit-breaker state in network-resilience may outlive a source.

Open questions

Constraints-phase shape. How applyConstraints and the empty-playable-set terminal state are modeled — owned by the track-switching constraints work, consumed here. (See track-switching-model.md → Fitting the model to the track-switching behavior.)
Failure-window policy. Threshold count / window length / cooldown duration for marking a CDN unhealthy. Empirical; lives with network-resilience's circuit-breaker.
CDN-id derivation configurability. Origin-based getCdnId is the default; whether/where to expose a consumer override (config field vs. rule config view) is deferred until a non-origin case appears.
Composition with content-steering. Static redundant CDNs + dynamic steered host pool: does steering replace, intersect, or reprioritize the candidate CDNs? Open until content-steering lands; the reorderable cdnPriority list keeps it tractable (steering reorders it).

Resolved during sub-feature 1 implementation

Signal shape → a per-presentation ordered cdnPriority list (active = first-with-survivors, derived), not a single stored activeCdn value and not per-rendition. The list makes failover a pure constraint and composes with content-steering as a reorder. Matches the cross-type coherence requirement (one shared list).
CDN identity → URL origin (getCdnId); configurable derivation deferred.
Manifest syntax / parser → no change. Redundant variants already parse as separate per-CDN tracks; no alternateUris field needed.
Architecture → constraint + scope in the track-switching model, not active-URI rotation in resolveTrack.

Implementation surface

packages/spf/src/media/utils/cdn.ts — getCdnId(url) (origin-based CDN identity) and getOrderedCdnIds(presentation) (distinct CDNs in manifest order; head = primary).
packages/spf/src/playback/behaviors/resolve-cdn-priority.ts — resolveCdnPriority behavior + ResolveCdnPriorityState. Machine reactor on presentation-unresolved ↔ presentation-resolved; owns the cdnPriority signal; publishes getOrderedCdnIds(presentation) (skipping the write when the CDN set is unchanged), clears on exit.
packages/spf/src/playback/behaviors/track-switching.ts — preferActiveCdn scope rule (soft filter on cdnPriority: narrow to the highest-priority CDN with surviving tracks), added to both variants' chains: [filterByUserSelection, preferActiveCdn, rankByBandwidth]. SwitchableTrack gains url (the rule's input).
packages/spf/src/playback/engines/hls/engine.ts + engine-audio-only.ts — resolveCdnPriority composed after resolvePresentation; cdnPriority?: string[] added to both engine state interfaces.

State signal: cdnPriority?: string[] (CDN origins, most-preferred first), owned by resolveCdnPriority, read optionally by preferActiveCdn.

Verification

Sub-feature 1:

media/utils/tests/cdn.test.ts — getCdnId (origin extraction; same/different host; scheme+port; unparseable fallback); getOrderedCdnIds (distinct CDNs in order; dedupe; single-CDN; unresolved → []).
playback/behaviors/tests/resolve-cdn-priority.test.ts — publishes the manifest-ordered CDN list; single-CDN source; skips the write when a resolved swap keeps the same CDNs; updates when the order changes; clears on src unload + on destroy; re-publishes after a src reset.
playback/behaviors/tests/track-switching.test.ts (preferActiveCdn block) — narrows to the highest-priority CDN overriding manifest track order; keeps the pick on the primary; falls through to the next CDN when the first has no survivors; falls through to all when none match; no-op when cdnPriority absent; re-picks reactively when the order changes (steering/failover seam); same scope applied to the audio chain (cross-type coherence).
playback/engines/hls/tests/engine.test.ts — integration: a redundant-stream presentation yields cdnPriority = manifest order and a video selection on the primary; reordering cdnPriority re-narrows and the selection follows.

Out of scope / deferred:

End-to-end + sandbox verification against a real Mux ?redundant_streams=true source (needs a fixture).
All of sub-feature 2 (failover): blocked on the constraints phase + network-resilience.

track-switching-model.md (governing model) — multi-CDN is its canonical constraint + scope feature; the active-CDN scope is implemented against the rule chain it specifies.
network-resilience (hard prerequisite for failover) — per-host circuit-breaker the failed-CDN constraint consumes.
content-steering — dynamic host-pool sibling; shares the active-pathway scope shape (cdnPriority as a reorderable reflected list — pathway priority as a sort key).
capability-probing — shares the not-yet-built constraints pre-pass with the failed-CDN constraint.
live-stream-support (not implemented) — reload-loop re-resolution; cdnPriority is republished only when the CDN set changes.
source-replacement — cdnPriority tears down via the resolved/unresolved cascade on source change.
video-abr / audio-abr — ABR ranks within the CDN-narrowed set; the scope runs before the ranker.

16 KiB Raw Blame History Unescape Escape