v10/internal/design/spf/features/multi-cdn-failover.md

---
status: draft
date: 2026-06-05
definition: sketched
---

# Multi-CDN failover

CDN selection and failover for HLS sources that publish the same content
on more than one host (e.g. Mux Video's `?redundant_streams=true`). The
redundant variants parse as ordinary candidate tracks — one per
(rendition × CDN) — so the work is **selecting which CDN to use and
keeping the whole presentation on it**, modeled inside the
[track-switching rule model](../track-switching-model.md) rather than as
URL rewriting at fetch time.

Two sub-features, mapping onto the two rule kinds in that model:

1. **Sticky CDN pick** *(implemented)* — a session-level behavior picks a
   CDN (the manifest-head host) and holds it; a shared **scope** rule
   (`preferActiveCdn`) narrows every track type's candidates to that CDN,
   so video / audio / text resolve from one host. This is the
   *active-pathway scope* the track-switching model lists for
   multi-cdn-failover.
2. **Failover** *(deferred)* — when requests to the active CDN fail too
   often within a window, a **constraint** removes that CDN's tracks from
   the candidate set during cooldown and the session behavior rotates the
   active CDN. This is the *failed-CDN constraint* the model lists, and it
   consumes [network-resilience](./network-resilience.md)'s per-host
   circuit-breaker — its hard prerequisite.

A **Media-src feature** for sources that genuinely require failover, with
a **Player feature** surface at the failover tier (customer-customizable
CDN policy). Notion epic #9 classifies as "Media-src? / Player?" — the
ambiguity reflects that split.

## Status

- **Composition:** sub-feature 1 (sticky CDN pick) is implemented in
  `createSimpleHlsEngine` and `createHlsAudioOnlyEngine`. The
  `resolveCdnPriority` behavior owns the `cdnPriority` signal (the
  manifest-ordered CDN list, most-preferred first); the `preferActiveCdn`
  scope rule (shared by the video + audio chains in `track-switching`)
  narrows candidates to the highest-priority CDN with surviving tracks.
  Failover (sub-feature 2) is not implemented — no constraints pass, no
  per-CDN failure tracking, no rotation.
- **Definition depth:** sketched — sub-feature 1 has a populated
  implementation surface + verification; sub-feature 2 stays at the
  scope-and-constraints level pending its prerequisite.
- **Hard prerequisite (failover only):**
  [network-resilience](./network-resilience.md). The failed-CDN
  constraint consumes the foundation's per-host circuit-breaker /
  retry-exhaustion state. Sub-feature 1 has no such dependency — it's
  pure selection over the already-parsed candidate set.
- **Governing model:** [track-switching-model.md](../track-switching-model.md)
  — multi-CDN is the canonical *constraint + scope* feature there. The
  active-CDN scope is a soft filter in the rule chain; the failed-CDN
  constraint is a hard filter in the (not-yet-built) constraints pre-pass.

## How redundant streams are modeled

There is no `alternateUris` field and no fetch-time URL rotation. A
redundant-streams manifest lists each rendition once per CDN (duplicate
`#EXT-X-STREAM-INF` / `#EXT-X-MEDIA` entries on different hosts), and the
existing parser already emits one `Track` per entry with a unique id and
its own absolute `url`. So the candidate set for each type *already*
contains one variant per CDN. CDN identity is derived from each track
URL's origin (`getCdnId`); the set of CDNs is published as a single
per-presentation ordered signal (`cdnPriority`, most-preferred first —
mirroring HLS content steering's `PATHWAY-PRIORITY`). The *active* CDN is
not stored: the scope derives it as the highest-priority `cdnPriority`
entry that still has tracks after the constraints pass. Selecting a
CDN-tagged track id means `resolveTrack` / segment loading fetch from that
CDN with no further plumbing.

This list shape is what makes failover fall out cleanly: the failed-CDN
constraint (sub-feature 2) prunes a cooled-down CDN's tracks, so
"first-with-survivors" moves to the next CDN automatically and returns to
the primary when it recovers — no reactive rewrite of an "active" value.
Content steering, likewise, just reorders `cdnPriority` (pathway priority
as a sort key).

This is why sub-feature 1 needed **no parser change** and no new data
shape — only a selection behavior and a scope rule over existing tracks.

## Phases of complexity

| Phase | Sub-feature | Kind | What | State |
|---|---|---|---|---|
| Sticky CDN pick | 1 | scope | `resolveCdnPriority` publishes the manifest-ordered CDN list (`cdnPriority`); `preferActiveCdn` narrows every type's candidates to the highest-priority CDN with surviving tracks, falling through when nothing matches. Shared list → all types on one CDN | **Implemented** |
| Failed-CDN constraint | 2 | constraint | A constraint in the track-switching constraints pre-pass removes a cooled-down CDN's tracks from the candidate set. The scope then picks the next `cdnPriority` entry automatically. Requires the generic constraints phase (track-switching-model "Phase 2") to be built first | Deferred |
| Per-CDN failure tracking | 2 | — | Count per-CDN fetch failures within a window; mark a CDN unhealthy / in cooldown. Consumes `network-resilience`'s circuit-breaker | Deferred (prereq) |
| CDN priority override / steering | 2 | scope | Reorder `cdnPriority` to bias the pick (region-preferred, weighted, or content-steering's pathway priority). No reactive "active" rewrite needed — the order *is* the policy | Deferred |
| Customer CDN-id derivation | 2 | config | Pluggable `getCdnId` for non-origin identity. The origin-based default is built in; a config seam is anticipated | Deferred |

## What's in scope vs out of scope

**In scope:**
- Sticky per-presentation CDN selection (sub-feature 1, done)
- Failover via a track-switching constraint + active-CDN rotation
  (sub-feature 2)
- Per-CDN health derived from `network-resilience`'s circuit-breaker
- Customer-configurable CDN-id derivation / rotation policy

**Out of scope (separate cluster G sister features):**
- **[network-resilience](./network-resilience.md)** *(foundation,
  prerequisite for failover)* — retry + backoff + circuit-breaker.
  Multi-CDN consumes; doesn't reimplement.
- **[content-steering](./content-steering.md)** — HLS content-steering
  protocol (server-advertised, dynamically-updated host pool). Different
  mechanism than static redundant streams, but the *same* active-pathway
  scope shape: content-steering picks the active pathway dynamically; the
  scope reflecting it is the one implemented here. Designed-with-in-mind:
  `cdnPriority` is a reorderable list a steering behavior writes (pathway
  priority as a sort key), and the scope honors the order unchanged.

**Out of scope (different architectural layer):**
- Adapter-layer customer-facing UI ("Switch CDN" buttons). Consumer
  policy is expressed via the failover tier's config seam.
- CDN-side load balancer / origin-shield infrastructure. Service-side.
- DRM key-server failover — lives under [drm-support](./drm-support.md).

## Likely cross-cutting impact

- **Per-presentation, not per-rendition (resolved).** A single
  `cdnPriority` list governs all track types — the per-presentation
  coherence requirement. The track-switching model's
  "cross-type consistency is a composition convention" applies: both the
  video and audio chains reference the *same* `preferActiveCdn` definition
  reading the *same* list, so they agree on the CDN even if their per-type
  track arrays differ. (The doc's earlier per-rendition lean is superseded.)
- **`cdnPriority` writer composition.** `resolveCdnPriority` is the sole
  writer today (publishes the manifest order). Failover needs no second
  writer — the failed-CDN constraint prunes tracks and the scope re-derives
  the active CDN. Content-steering would *reorder* `cdnPriority` (still a
  single owning behavior; the list reflects one upstream priority).
- **Active CDN is derived, not stored.** The scope computes "highest
  priority with surviving tracks," so failover and recovery need no extra
  state: pruning moves the pick to the next CDN, un-pruning returns it to
  the primary. This is why the array beats a single reactive `activeCdn`
  value — the failed-set information is applied once (in the constraint),
  not duplicated into an active-value rewrite.
- **Constraints phase is a shared prerequisite.** The failed-CDN
  constraint can't land until the generic constraints pre-pass
  (`applyConstraints` + the `constraints` config field + empty-playable-set
  terminal state) is built into `setupTrackSwitching`. The seam exists
  (`candidateSet` computed); the machinery does not. capability-probing
  shares this prerequisite.
- **Live + multi-CDN.** During live playback the reload loop re-resolves
  the presentation; `resolveCdnPriority` re-publishes only when the CDN set
  changes (idempotent for a stable manifest). Cross-feature with
  [live-stream-support](./live-stream-support.md) (not yet implemented).
- **Priority state across source changes.** `cdnPriority` tears down with
  the source via the resolved/unresolved cascade (per
  [source-replacement](./source-replacement.md)); per-host circuit-breaker
  state in `network-resilience` may outlive a source.

## Open questions

- **Constraints-phase shape.** How `applyConstraints` and the
  empty-playable-set terminal state are modeled — owned by the
  track-switching constraints work, consumed here. (See
  [track-switching-model.md](../track-switching-model.md) → *Fitting the
  model to the track-switching behavior*.)
- **Failure-window policy.** Threshold count / window length / cooldown
  duration for marking a CDN unhealthy. Empirical; lives with
  `network-resilience`'s circuit-breaker.
- **CDN-id derivation configurability.** Origin-based `getCdnId` is the
  default; whether/where to expose a consumer override (config field vs.
  rule config view) is deferred until a non-origin case appears.
- **Composition with content-steering.** Static redundant CDNs +
  dynamic steered host pool: does steering replace, intersect, or
  reprioritize the candidate CDNs? Open until content-steering lands; the
  reorderable `cdnPriority` list keeps it tractable (steering reorders it).

### Resolved during sub-feature 1 implementation

- **Signal shape** → a per-presentation ordered `cdnPriority` list (active
  = first-with-survivors, derived), not a single stored `activeCdn` value
  and not per-rendition. The list makes failover a pure constraint and
  composes with content-steering as a reorder. Matches the cross-type
  coherence requirement (one shared list).
- **CDN identity** → URL origin (`getCdnId`); configurable derivation
  deferred.
- **Manifest syntax / parser** → no change. Redundant variants already
  parse as separate per-CDN tracks; no `alternateUris` field needed.
- **Architecture** → constraint + scope in the track-switching model, not
  active-URI rotation in `resolveTrack`.

## Implementation surface

- **`packages/spf/src/media/utils/cdn.ts`** — `getCdnId(url)` (origin-based
  CDN identity) and `getOrderedCdnIds(presentation)` (distinct CDNs in
  manifest order; head = primary).
- **`packages/spf/src/playback/behaviors/resolve-cdn-priority.ts`** —
  `resolveCdnPriority` behavior + `ResolveCdnPriorityState`. Machine reactor
  on `presentation-unresolved` ↔ `presentation-resolved`; owns the
  `cdnPriority` signal; publishes `getOrderedCdnIds(presentation)` (skipping
  the write when the CDN set is unchanged), clears on exit.
- **`packages/spf/src/playback/behaviors/track-switching.ts`** —
  `preferActiveCdn` scope rule (soft filter on `cdnPriority`: narrow to the
  highest-priority CDN with surviving tracks), added to both variants'
  chains: `[filterByUserSelection, preferActiveCdn, rankByBandwidth]`.
  `SwitchableTrack` gains `url` (the rule's input).
- **`packages/spf/src/playback/engines/hls/engine.ts` +
  `engine-audio-only.ts`** — `resolveCdnPriority` composed after
  `resolvePresentation`; `cdnPriority?: string[]` added to both engine state
  interfaces.

State signal: `cdnPriority?: string[]` (CDN origins, most-preferred first),
owned by `resolveCdnPriority`, read optionally by `preferActiveCdn`.

## Verification

Sub-feature 1:

- `media/utils/tests/cdn.test.ts` — `getCdnId` (origin extraction;
  same/different host; scheme+port; unparseable fallback);
  `getOrderedCdnIds` (distinct CDNs in order; dedupe; single-CDN;
  unresolved → `[]`).
- `playback/behaviors/tests/resolve-cdn-priority.test.ts` — publishes the
  manifest-ordered CDN list; single-CDN source; skips the write when a
  resolved swap keeps the same CDNs; updates when the order changes; clears
  on src unload + on destroy; re-publishes after a src reset.
- `playback/behaviors/tests/track-switching.test.ts` (`preferActiveCdn`
  block) — narrows to the highest-priority CDN overriding manifest track
  order; keeps the pick on the primary; falls through to the next CDN when
  the first has no survivors; falls through to all when none match; no-op
  when `cdnPriority` absent; re-picks reactively when the order changes
  (steering/failover seam); same scope applied to the audio chain
  (cross-type coherence).
- `playback/engines/hls/tests/engine.test.ts` — integration: a
  redundant-stream presentation yields `cdnPriority` = manifest order and a
  video selection on the primary; reordering `cdnPriority` re-narrows and
  the selection follows.

Out of scope / deferred:

- End-to-end + sandbox verification against a real Mux
  `?redundant_streams=true` source (needs a fixture).
- All of sub-feature 2 (failover): blocked on the constraints phase +
  `network-resilience`.

## Related features

- **[track-switching-model.md](../track-switching-model.md)** *(governing
  model)* — multi-CDN is its canonical constraint + scope feature; the
  active-CDN scope is implemented against the rule chain it specifies.
- **[network-resilience](./network-resilience.md)** *(hard prerequisite
  for failover)* — per-host circuit-breaker the failed-CDN constraint
  consumes.
- **[content-steering](./content-steering.md)** — dynamic host-pool
  sibling; shares the active-pathway scope shape (`cdnPriority` as a
  reorderable reflected list — pathway priority as a sort key).
- **[capability-probing](./capability-probing.md)** — shares the
  not-yet-built constraints pre-pass with the failed-CDN constraint.
- **[live-stream-support](./live-stream-support.md)** *(not implemented)*
  — reload-loop re-resolution; `cdnPriority` is republished only when the
  CDN set changes.
- **[source-replacement](./source-replacement.md)** — `cdnPriority` tears
  down via the resolved/unresolved cascade on source change.
- **[video-abr](./video-abr.md)** / **[audio-abr](./audio-abr.md)** — ABR
  ranks within the CDN-narrowed set; the scope runs before the ranker.

## See also

- [track-switching-model.md](../track-switching-model.md) — the rule
  model (constraints → soft filters → ranker) this feature composes into
- [clusters.md § Selection resilience](./clusters.md#selection-resilience)
  — cluster G; this feature is the selection-side resilience sister to
  `network-resilience`'s response-error handling
- [clusters.md § Selection / filtering across clusters](./clusters.md#selection--filtering-across-clusters)
  — cluster G's role: alternate-CDN selection within the chosen track
- [network-resilience.md](./network-resilience.md) — circuit-breaker
  foundation the failover tier consumes
- [SPF Epics Working Doc](https://www.notion.so/35f97a7f89d08123a13fecab1ca1cac4)
  — source material; epic #9 (Multi-CDN Failover)
- [Mux Video — `?redundant_streams=true`](https://www.mux.com/docs/guides/play-back-on-multiple-cdns)
  — Mux Video convention for redundant-CDN sources