mirror of https://github.com/zoriya/v10.git synced 2026-08-15 02:14:06 +00:00

Files

T

c9d6ccecdd docs(spf): seed feature registry, clusters reference, and document-feature skill (#1581 )

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

2026-05-21 12:07:33 -07:00

17 KiB

Raw Blame History

status, date, definition

status	date	definition
draft	2026-05-20	technical

5.1 surround selection

Capability-aware audio rendition selection on the channel-count axis: filter presentation.audioTracks to channel-count / codec combinations the browser supports (with implicit stereo fallback), plus customer-driven channel-count preference (config), plus engine- inferred preference based on the downstream playback environment (runtime), plus optional mid-stream codec switching via SourceBuffer.changeType() between channel/codec configurations.

The audio sibling of hevc-variant-selection: both are codec-change variant selection consumers of capability-probing. HEVC operates on the video codec axis (AVC ↔ HEVC); this feature operates on the audio channel-count axis (stereo ↔ 5.1 surround, with codecs typically differing between the two — AAC stereo vs AC-3 / EAC-3 5.1). The parallel sibling structure is intentional; the channel-count axis has one additional Tier 2 phase (downstream-environment-aware preference) that doesn't have a HEVC analog.

A Media-src feature in the framing from clusters.md § Feature classification axes at Tier 1; a Player feature at Tier 2 (customer-driven overrides

engine-inferred preferences layered on top).

Status

Composition: not implemented in createSimpleHlsEngine. Today selectAudioTrack runs the default picker (pickFirstTrackId) over all audio renditions without channel-count filtering. The parser surfaces channels on audio renditions (parse-multivariant.ts) but the value isn't consumed for selection or filtering — see audio-playback.md's "Channels exposure vs use" open question.
Definition depth: technical — scope and constraints articulated; no implementation. Source material: SPF Epics Working Doc — epic #23 5.1 Surround Sound (cluster C+D, Media-src, eng M, validation M, "Audio rendition capability gating").
Hard prerequisite: capability-probing. Tier 1 filter applies that feature's "Multivariant CODECS-attribute filtering" mechanism to the audio channel/codec combination; Tier 2 mid-stream changeType() consumes that feature's "Cross-codec transition (changeType()) probing" phase (which explicitly cites AAC stereo ↔ AC-3 5.1 as a transition pair).

Phases of complexity

Tier 1 / Tier 2 framing per the Notion epics doc (5.1 row; sibling hevc-variant-selection and multi-language-audio precedents).

Phase	Tier	What	Notes
Capability-aware initial selection	Tier 1	Filter `presentation.audioTracks` to channel-count / codec combinations the browser supports per capability-probing's verdict. 5.1 variants (typically AC-3 / EAC-3 / AAC 5.1) retained if the codec+channel combination is supported; excluded otherwise. Stereo fallback emerges from the filter: 5.1 variants removed → stereo variants remain in the candidate set → `selectAudioTrack` picks among the remainder	Mechanism lives in capability-probing.md's "Multivariant CODECS-attribute filtering" phase. This feature is the applied use case for the channel-count axis. Same codec/channel combination for the session — no `changeType()`, no setup re-entry. The `channels` field already surfaced by the parser is the input axis
Customer channel-count preference (config)	Tier 2	Config-driven biases on top of the Tier 1 filter: `audioChannelPreference: 'stereo'` excludes 5.1 even when supported (the force-stereo override called out in the Notion epic); `audioChannelPreference: 'surround'` biases toward 5.1 when available	Policy mechanism per clusters.md § Feature classification axes. Constraint slot read by the selection filter step; same shape as HEVC's customer override. Customer-policy-driven; same codec/channel combination for the session
Downstream-environment-aware channel preference (runtime)	Tier 2	When both 5.1 and stereo codec/channel combinations are supported, infer a channel preference from the playback environment — e.g., `AudioContext.destination.maxChannelCount === 2` suggests the effective playback is stereo (5.1 would be downsampled by the browser anyway, wasting bandwidth). Engine-inferred, not stated by config	Middle pattern per clusters.md § Feature classification axes — a new state-producing behavior monitors the audio context and writes a preference slot; the selection filter step reads it. Parallel shape to viewport-driven cap in rendition-selection-caps.md. No HEVC analog (video codec selection has no equivalent downstream-environment signal). Composes with the Tier 2 config-driven override above: explicit config wins over engine-inferred preference if both set
Mid-stream codec change via `changeType()`	Tier 2	Allow channel-count / codec switching between AAC-stereo and AC-3 / EAC-3 5.1 (or other channel-count-distinct codec pairs) mid-stream when the customer/environment preference changes, or when audio ABR (if introduced) crosses a channel-count boundary. Buffer-side `SourceBuffer.changeType(newMimeCodec)` call before appending segments of the new codec	Depends on capability-probing.md's "Cross-codec transition probing" phase landing — capability-probing answers "can the browser changeType from AAC to AC-3?"; this feature owns the actual call site for the audio buffer specifically. Mirror of HEVC's mid-stream phase; significantly more complex than the prior phases (codec re-init, buffer-state coordination, MSE quirks). Browser support for audio cross-codec changeType is fragile and pair-specific

What's in scope vs out of scope

In scope:

All four phases above for channel-count-distinct audio variant selection (typically stereo ↔ 5.1, but also stereo ↔ 7.1 or similar higher-channel-count combinations)
Audio-side channel-count preference state slot + filter step on presentation.audioTracks
AudioContext.destination.maxChannelCount (or equivalent) detection hook for the runtime-inferred preference phase
changeType() call site for cross-codec audio transitions in the buffer-setup neighborhood
Customer-facing config surface (audioChannelPreference)
Stereo fallback semantics (inherent in the capability filter — 5.1 excluded ⇒ stereo remains)

Out of scope (separate Media-src candidate features):

hevc-variant-selection — parallel sibling on the video codec axis. Same structural shape; the downstream-environment phase is 5.1-specific.
audio-abr — audio bandwidth-driven quality switching. May eventually cross channel-count boundaries (a low- bandwidth 5.1 variant vs a higher-bandwidth stereo variant), which would consume this feature's changeType() phase.
[multi-language-audio] — Tier 2 mid-stream language switching. Different axis (language) but consumes the same changeType() primitive for codec-distinct language variants.

Out of scope (different architectural layer):

Adapter-layer customer-facing API surfaces (e.g., a hypothetical Mux Video element force-stereo or prefer-channels attribute). The SPF feature owns the state slot + filter + detection hook; consumer-facing config attribute names live above-engine.
Above-engine "is 5.1 playing" UI affordances. Consume the resolved audio track's channels value via existing audio-track surfaces; not SPF concerns.
AudioContext lifecycle management outside the engine. The runtime- detection phase needs an AudioContext or equivalent capability query; whether the engine creates its own short-lived context for probing or consumes one provided by the adapter is open (see Open questions).

Likely cross-cutting impact

Things this feature probably forces decisions on, not just additions:

Filter-then-select shape. Same open question as hevc-variant-selection.md and rendition-selection-caps.md: does the selection picker take the pre-filtered candidate set (option a — keeps the picker pure, matches the precedent), or extend its signature to take preference slots and filter internally (option b)? Cross-cuts with the parallel features; whichever lands first sets the audio-side shape.
Channel-preference slot granularity. Single slot like audioChannelPreference: 'stereo' | 'surround' | undefined, or numeric (audioMaxChannels: 2 | 6 | 8) for finer control across 5.1 / 7.1 / Atmos / etc.? Numeric is more flexible; symbolic is simpler. Open question shared with HEVC's analog slot-shape question.
Override-vs-runtime-detection precedence. When the Tier 2 config-driven slot AND the Tier 2 runtime-detected slot both express a preference, which wins? Default: config wins (explicit customer intent overrides engine inference). Open whether to expose a "merge-priority" config field for finer control.
Pipeline ordering with capability-filter and codec-preference. When all relevant features land, the audio-side filter pipeline becomes: capability filter (physics) → channel-count preference (config + runtime) → audio-side selection. Mirror of the video-side pipeline established in hevc-variant-selection.md / rendition-selection-caps.md.
MSE codec-change boundary. Tier 1 and Tier 2 customer-override and runtime-detection-only do not require changeType() — the codec/channel combination is fixed for the session. Tier 2 mid- stream changeType() is the exception; it crosses into mse-mms-pipeline.md's buffer-setup neighborhood. The MSE codec-change check fires only for the mid- stream phase.
AudioContext lifecycle. The runtime-detection phase needs to query the audio output environment. Options: (a) the engine creates a short-lived AudioContext for the channel-count query and immediately closes it (low cost on most browsers, but not free); (b) the engine reads from an AudioContext provided by the adapter / consumer; (c) the engine uses a different probing API entirely. Decision affects whether this feature has a DOM dependency for detection or stays consumer-driven.
Environment changes mid-stream. The downstream playback environment can change (user plugs in headphones, switches output device). Detecting and reacting to this change crosses the runtime- detection phase into the mid-stream-changeType phase. The two are composable but the interaction shape — does the runtime-detection signal trigger an automatic mid-stream switch? — is open.

Open questions

Channel-preference slot granularity — symbolic vs numeric. Per the cross-cutting note: audioChannelPreference: 'stereo' | 'surround' vs audioMaxChannels: number. Customer use cases drive the call.
AudioContext lifecycle for runtime detection. Short-lived engine-managed context vs adapter-provided vs alternative probing API. Affects feature shape (one behavior owns DOM detection vs one slot the adapter writes) and SSR / non-browser contexts.
Override-vs-detection precedence merge policy. Config-driven override vs runtime-detected preference: default "config wins," but expose a merge-priority config field? Open until use cases surface.
Environment-change mid-stream reaction. When the runtime- detected preference changes mid-stream (user plugs in headphones), does the engine automatically trigger a changeType() switch, or does the preference change only affect future selection? The bandwidth-savings argument suggests automatic switch; UX continuity may prefer stay-current.
Mid-stream changeType() trigger policy. Only when capability requires (e.g., manifest reload introduces new codec), or also preference-driven (config or detection change), or also bandwidth- driven if audio-ABR crosses channel-count boundaries? Three trigger sources; policy may differ per source.
Buffer-state coordination during audio changeType(). Flush vs continue-append vs other coordination, dependent on browser pair behavior (AAC ↔ AC-3 specifically). Likely lives in this feature but interacts with buffer-management's planner.
Audio-side codec-priority comparator (if audio-ABR lands). Equal-bandwidth-different-codec/channel tie-breaking at audio-ABR upgrade time. May be [multi-signal-abr] follow-on work rather than this feature.
Fallback-when-empty chain. If capability-filter narrows to zero AND customer-override further excludes the residue, what happens? Same chain as HEVC; refer to capability-probing.md's broader fallback-when-empty open question.
Multivariant CHANNELS parsing. parse-multivariant.ts hard-codes channels: 2 as default (lines 225, 255); actual channel info would need to flow from the media playlist or from EXT-X-MEDIA:CHANNELS=... parsing if the multivariant carries it. Whether the multivariant or only the media playlist carries channel info varies by encoder; worth verifying with test fixtures.

capability-probing (hard prerequisite) — provides the multivariant-CODECS filter (Tier 1)
- the changeType() probe (Tier 2 mid-stream, explicitly citing AAC stereo ↔ AC-3 5.1 as a transition pair). This feature is scoped explicitly as a consumer in capability-probing.md's Out of scope ("Same shape as HEVC") and Related features.
hevc-variant-selection — parallel sibling on the video codec axis. The codec-preference slot pattern, customer-override phase, and mid-stream changeType() phase mirror across the two. The downstream-environment-aware preference phase is 5.1-specific (no video analog).
audio-playback — baseline this feature extends. Today's selectAudioTrack is channel-count-unaware; this feature wires channels into the selection logic.
audio-abr — audio quality switching. May eventually cross channel-count boundaries, consuming this feature's changeType() phase. Algorithm change for audio-ABR is its own feature; this feature provides the cross-codec primitive when audio-ABR needs it.
[multi-language-audio] — Tier 2 mid-stream language switching. Different selection axis (language) but consumes the same changeType() primitive for codec-distinct language variants. Worth harmonizing the audio-side changeType() call site shape.
mse-mms-pipeline — changeType() call site lives in the audio buffer-setup neighborhood (Tier 2 mid- stream phase only). Same-codec phases don't touch MSE setup.
rendition-selection-caps — sibling pattern: the middle-pattern mechanism (state-producing behavior monitors signal, selection consumer respects it) is the same shape used by the viewport-driven cap. The audio context detection is the analog of the resize observer.
[unsupported-case-error-mapping] (candidate) — when the filter pipeline narrows the audio candidate set to zero, the error surfaces through capability-probing's error primitive.

17 KiB Raw Blame History