mirror of
https://github.com/zoriya/v10.git
synced 2026-08-16 02:45:09 +00:00
feat(spf): HLS engine composition walkthrough + doc-driven cleanups (#1512)
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
Christian Pillsbury
co-authored by
Claude Opus 4.7
Claude Opus 4.7
parent
17d44a5d32
commit
0cfd3bb395
@@ -0,0 +1,303 @@
|
||||
# SPF — Stream Processing Framework
|
||||
|
||||
SPF is a general-purpose framework for building streaming playback engines. It provides a small set of composable primitives — reactive state, actors, reactors, and tasks — that let you assemble a playback engine from independent, decoupled features.
|
||||
|
||||
SPF keeps its opinions as localized as possible. The framework doesn't know about HLS, DASH, or any specific protocol. It provides the composition model; you provide the features.
|
||||
|
||||
---
|
||||
|
||||
## The Engine
|
||||
|
||||
A playback engine is a composition of **features** — functions that each handle one concern (manifest resolution, segment loading, ABR, etc.). The engine wires them together with shared reactive state and runs them.
|
||||
|
||||
```ts
|
||||
import { createPlaybackEngine } from '@videojs/spf';
|
||||
|
||||
const engine = createPlaybackEngine([
|
||||
resolvePresentation,
|
||||
selectVideoTrack,
|
||||
loadVideoSegments,
|
||||
endOfStream,
|
||||
]);
|
||||
```
|
||||
|
||||
`createPlaybackEngine` is generic. It:
|
||||
|
||||
1. Creates shared **state** and **owners** signals
|
||||
2. Passes them (along with **config**) to each feature
|
||||
3. Returns the engine interface: `{ state, owners, destroy() }`
|
||||
|
||||
```ts
|
||||
function createPlaybackEngine(
|
||||
features: Feature[],
|
||||
options?: {
|
||||
config?: C;
|
||||
initialState?: S;
|
||||
initialOwners?: O;
|
||||
}
|
||||
): PlaybackEngine;
|
||||
```
|
||||
|
||||
All options are optional. The simplest engine is just `createPlaybackEngine([myFeature])`.
|
||||
|
||||
### State, Owners, and Config
|
||||
|
||||
An engine has three shared channels:
|
||||
|
||||
| Channel | What it holds | Lifecycle |
|
||||
|---------|--------------|-----------|
|
||||
| **state** | Application state (selected tracks, bandwidth estimates, current time, etc.) | Reactive signal — read and written throughout the engine's lifetime |
|
||||
| **owners** | Platform objects (media elements, SourceBuffers, actors) | Reactive signal — populated by features as resources are created |
|
||||
| **config** | Static configuration (initial bandwidth, preferred languages, etc.) | Passed once at creation — not reactive |
|
||||
|
||||
None of these have a fixed shape. Their types are determined by the features in the composition — each feature declares what state, owners, and config fields it needs, and the engine's types are the intersection of those requirements.
|
||||
|
||||
### Features
|
||||
|
||||
A feature is a function that receives `{ state, owners, config }` and optionally returns a cleanup handle:
|
||||
|
||||
```ts
|
||||
type Feature = (deps: { state, owners, config }) => void | (() => void) | { destroy(): void };
|
||||
```
|
||||
|
||||
Features are independent. They don't know about each other — they communicate through shared state. A feature might:
|
||||
|
||||
- Observe state changes and react (a **reactor**)
|
||||
- Own a stateful worker that processes messages (an **actor**)
|
||||
- Run a one-time side effect on setup
|
||||
- Do nothing but return a cleanup function
|
||||
|
||||
The engine doesn't care which. It calls the feature, collects any cleanup, and moves on.
|
||||
|
||||
---
|
||||
|
||||
## An HLS Engine
|
||||
|
||||
An HLS playback engine is one specific composition of features. There's nothing special about it from SPF's perspective — it's just a list of features and some config.
|
||||
|
||||
```ts
|
||||
import { createPlaybackEngine } from '@videojs/spf';
|
||||
|
||||
function createSimpleHlsEngine(config = {}) {
|
||||
return createPlaybackEngine(
|
||||
[
|
||||
// Preload and playback tracking
|
||||
syncPreloadAttribute,
|
||||
trackPlaybackInitiated,
|
||||
|
||||
// Manifest resolution
|
||||
resolvePresentation,
|
||||
|
||||
// Track selection (reads config for initial preferences)
|
||||
selectVideoTrackFromConfig,
|
||||
selectAudioTrackFromConfig,
|
||||
selectTextTrackFromConfig,
|
||||
|
||||
// Resolve selected tracks (fetch media playlists)
|
||||
resolveVideoTrack,
|
||||
resolveAudioTrack,
|
||||
resolveTextTrack,
|
||||
|
||||
// Presentation duration
|
||||
calculatePresentationDuration,
|
||||
|
||||
// MSE setup
|
||||
setupMediaSource,
|
||||
updateDuration,
|
||||
setupSourceBuffers,
|
||||
|
||||
// Playback tracking and ABR
|
||||
trackCurrentTime,
|
||||
switchQualityFromConfig,
|
||||
|
||||
// Segment loading
|
||||
loadVideoSegments,
|
||||
loadAudioSegments,
|
||||
|
||||
// End of stream coordination
|
||||
endOfStream,
|
||||
|
||||
// Text tracks
|
||||
syncTextTracks,
|
||||
loadTextTrackCues,
|
||||
],
|
||||
{ config, initialState: { bandwidthState: initialBandwidthState() } }
|
||||
);
|
||||
}
|
||||
```
|
||||
|
||||
Reading top to bottom, the engine tells a story: resolve a manifest, pick tracks, set up MSE, load segments, coordinate end-of-stream. Each line is a feature. To build a different engine — fewer features, different features, different protocol — you change the list.
|
||||
|
||||
### Config threading
|
||||
|
||||
Features that need configuration read it from `deps.config`. The engine passes config to every feature; each one destructures what it needs:
|
||||
|
||||
```ts
|
||||
// A feature that reads config
|
||||
const selectVideoTrackFromConfig = ({ config, ...deps }) =>
|
||||
selectVideoTrack(deps, {
|
||||
type: 'video',
|
||||
...(config.initialBandwidth !== undefined && { initialBandwidth: config.initialBandwidth }),
|
||||
});
|
||||
|
||||
// A feature that doesn't need config — just ignores it
|
||||
function resolvePresentation({ state }) {
|
||||
// ...
|
||||
}
|
||||
```
|
||||
|
||||
### Media-type wrappers
|
||||
|
||||
Some features are parameterized by media type (video, audio, text). Rather than passing `{ type: 'video' }` as inline config, we define thin wrappers that close over the type:
|
||||
|
||||
```ts
|
||||
const loadVideoSegments = (deps) => loadSegments(deps, { type: 'video' });
|
||||
const loadAudioSegments = (deps) => loadSegments(deps, { type: 'audio' });
|
||||
```
|
||||
|
||||
This keeps the engine composition readable — a flat list of named features.
|
||||
|
||||
---
|
||||
|
||||
## The Three Layers
|
||||
|
||||
Features are built from three layers of primitives. Data flows in one direction:
|
||||
|
||||
```
|
||||
Reactive State (signals)
|
||||
│ observed by
|
||||
▼
|
||||
Reactors (thin subscribers)
|
||||
│ send messages to
|
||||
▼
|
||||
Actors (stateful workers)
|
||||
│ execute
|
||||
▼
|
||||
Tasks (async work units)
|
||||
```
|
||||
|
||||
### Reactive State — Signals
|
||||
|
||||
All shared state in SPF is built on [TC39 Signals](https://github.com/tc39/proposal-signals). A signal is a reactive value: it always has a current reading, and computations that read it are automatically notified when it changes.
|
||||
|
||||
```ts
|
||||
import { signal, computed, effect } from '@videojs/spf';
|
||||
|
||||
const count = signal(0);
|
||||
const doubled = computed(() => count.get() * 2);
|
||||
|
||||
effect(() => {
|
||||
console.log(doubled.get()); // re-runs when count changes
|
||||
});
|
||||
|
||||
count.set(5); // logs: 10
|
||||
```
|
||||
|
||||
SPF chose signals over observables because all SPF state is **state-over-time** — current track, buffer contents, bandwidth estimate. These are naturally modeled as "a value that changes" rather than "a stream of events." Signals give you automatic dependency tracking for derived state, synchronous reads, and a lower barrier to entry than observable pipelines.
|
||||
|
||||
### Reactors
|
||||
|
||||
A reactor observes reactive state and responds to changes. It has its own finite state machine and can run per-state effects. Reactors don't receive messages — they're driven entirely by signal observation.
|
||||
|
||||
```ts
|
||||
const reactor = createMachineReactor({
|
||||
initial: 'preconditions-unmet',
|
||||
monitor: () => canResolve(state.get()) ? 'resolving' : 'preconditions-unmet',
|
||||
states: {
|
||||
'preconditions-unmet': {},
|
||||
resolving: {
|
||||
entry: () => {
|
||||
// Run once on state entry (automatically untracked)
|
||||
const ac = new AbortController();
|
||||
fetchAndParse(state, ac.signal);
|
||||
return ac; // cleanup on state exit
|
||||
},
|
||||
},
|
||||
},
|
||||
});
|
||||
```
|
||||
|
||||
Key concepts:
|
||||
- **`monitor`** — derives the target state from signals. The framework drives the transition.
|
||||
- **`entry`** — runs once on state entry, automatically untracked. Return a cleanup.
|
||||
- **`effects`** — re-run when tracked signals change. For reactive sync.
|
||||
|
||||
### Actors
|
||||
|
||||
An actor is a long-lived stateful worker that processes messages serially. It owns a reactive snapshot (status + context) and uses tasks and runners to execute work.
|
||||
|
||||
```ts
|
||||
const actor = createMachineActor({
|
||||
runner: () => new SerialRunner(),
|
||||
initial: 'idle',
|
||||
context: { segments: [], initTrackId: undefined },
|
||||
states: {
|
||||
idle: {
|
||||
on: {
|
||||
'append-segment': (msg, { transition, runner, setContext }) => {
|
||||
transition('updating');
|
||||
const task = makeAppendTask(msg);
|
||||
runner.schedule(task).then(setContext);
|
||||
},
|
||||
},
|
||||
},
|
||||
updating: {
|
||||
onSettled: 'idle', // auto-return when runner settles
|
||||
},
|
||||
},
|
||||
});
|
||||
```
|
||||
|
||||
Key concepts:
|
||||
- **`send(message)`** — the only way to communicate with an actor
|
||||
- **`snapshot`** — reactive read-only state, observable by reactors and other consumers
|
||||
- **`runner`** — schedules and executes tasks (serial, concurrent, etc.)
|
||||
- **`onSettled`** — automatic transition when all tasks complete
|
||||
|
||||
### How they connect
|
||||
|
||||
A reactor observes state, decides something needs to happen, and sends a message to an actor. The actor does the work and updates its snapshot. Other reactors (or the engine) observe the actor's snapshot and react.
|
||||
|
||||
```
|
||||
state changes → reactor observes → actor.send({ type: 'load', track })
|
||||
↓
|
||||
actor executes tasks
|
||||
↓
|
||||
actor.snapshot updates
|
||||
↓
|
||||
other reactors observe snapshot
|
||||
```
|
||||
|
||||
This separation means:
|
||||
- **Actors don't know about external state.** They receive messages and produce state changes.
|
||||
- **Reactors don't do work.** They observe and coordinate.
|
||||
- **State is the single source of truth.** Features communicate through it, not through each other.
|
||||
|
||||
---
|
||||
|
||||
## Tasks
|
||||
|
||||
A task is an ephemeral unit of async work with lifecycle tracking. Unlike a Promise, a task:
|
||||
|
||||
- Starts in a **pending** state before it runs — it can be inspected, queued, or aborted before any work begins
|
||||
- Can be **aborted** from outside at any point
|
||||
- Exposes its **status synchronously** (`pending`, `running`, `done`, `error`)
|
||||
- Carries a typed **value** and **error**, readable once settled
|
||||
|
||||
Tasks are the unit of work inside actors. They're not exposed externally — actors plan and execute tasks, and the task's status may or may not surface in the actor's snapshot.
|
||||
|
||||
**Task runners** control scheduling:
|
||||
- **SerialRunner** — one task at a time (used for SourceBuffer operations, which are inherently serial)
|
||||
- **ConcurrentRunner** — parallel with deduplication by ID
|
||||
|
||||
---
|
||||
|
||||
## Open Questions
|
||||
|
||||
> These are areas where the design direction is clear but implementation details are still being worked through.
|
||||
|
||||
- **`presentation?: any`** — The engine-level state type uses `any` for the presentation field because it transitions from unresolved (`{ url }`) to resolved (`Presentation`) at runtime. Individual features narrow the type themselves. Signal invariance prevents using a union type here. A cleaner pattern is needed.
|
||||
- **Feature state interfaces and Signal invariance** — More broadly, how should features declare their state requirements given that `Signal<T>` is invariant? The current pattern (features use generic constraints, engine state uses `any` where types conflict) works but has rough edges.
|
||||
- **Task `aborted` as a distinct terminal state** — Currently abort lands in `error`. Should be first-class: `pending → running → done | error | aborted`.
|
||||
- **Effect scheduling semantics** — Effects are deferred via `queueMicrotask`. The exact behavior under compound state changes (multiple signal writes in one turn) is behavioral, not formally specified.
|
||||
- **Reactor lifecycle ownership** — Currently the engine explicitly creates and destroys everything. With signals, reactors could self-scope to a reactive context and auto-dispose.
|
||||
@@ -0,0 +1,386 @@
|
||||
# The HLS Engine
|
||||
|
||||
A reference walkthrough of how SPF composes a playback engine.
|
||||
|
||||
The HLS engine in `@videojs/spf/hls` is one specific composition — manifest resolution, track selection, MSE setup, segment loading, end-of-stream — but the patterns it uses are how any SPF playback engine is built. This doc walks the composition stage-by-stage and calls out the patterns that recur.
|
||||
|
||||
It assumes familiarity with HLS, MSE, and adaptive streaming. If you want to learn what a "switching set" is, look elsewhere. If you want to learn how SPF lets you turn those concepts into a composable, declarative engine, you're in the right place.
|
||||
|
||||
For SPF primitives (signals, reactors, tasks, actors), see [fundamentals.md](./fundamentals.md).
|
||||
|
||||
---
|
||||
|
||||
## The engine at a glance
|
||||
|
||||
`createSimpleHlsEngine` is a thin wrapper around `createComposition`. The full composition, lifted from `src/playback/engines/hls/engine.ts`:
|
||||
|
||||
```ts
|
||||
export function createSimpleHlsEngine(
|
||||
config: SimpleHlsEngineConfig = {}
|
||||
): Composition<SimpleHlsEngineState, SimpleHlsEngineOwners> {
|
||||
return createComposition(
|
||||
[
|
||||
syncPreloadAttribute,
|
||||
trackPlaybackInitiated,
|
||||
resolvePresentation,
|
||||
|
||||
// Track selection (reads config for initial preferences)
|
||||
selectVideoTrack,
|
||||
selectAudioTrack,
|
||||
selectTextTrack,
|
||||
|
||||
// Resolve selected tracks (fetch media playlists)
|
||||
resolveVideoTrack,
|
||||
resolveAudioTrack,
|
||||
resolveTextTrack,
|
||||
|
||||
// Presentation duration
|
||||
calculatePresentationDuration,
|
||||
|
||||
// MSE setup
|
||||
setupMediaSource,
|
||||
updateDuration,
|
||||
setupSourceBuffers,
|
||||
|
||||
// Playback tracking
|
||||
trackCurrentTime,
|
||||
switchQuality,
|
||||
|
||||
// Segment loading
|
||||
loadVideoSegments,
|
||||
loadAudioSegments,
|
||||
|
||||
// End of stream coordination
|
||||
endOfStream,
|
||||
|
||||
// Text tracks
|
||||
syncTextTracks,
|
||||
setupTextTrackActors,
|
||||
loadTextTrackCues,
|
||||
],
|
||||
{ config, initialState, initialOwners }
|
||||
);
|
||||
}
|
||||
```
|
||||
|
||||
Read top to bottom, the engine tells a story: resolve a manifest, pick tracks, set up MSE, load segments, coordinate end-of-stream, render text tracks. Each line is a behavior — a small, focused unit of logic that owns one job. To build a different engine — fewer behaviors, different protocol, different platform — you change the list.
|
||||
|
||||
Three things are doing the work here:
|
||||
|
||||
1. **`createComposition`** — the SPF primitive that wires the behaviors together. It owns the lifecycle and gives each behavior access to two shared reactive channels — `state` and `owners` — plus the engine's static `config`.
|
||||
2. **The behaviors** — independent functions, each declaring its slice of the state and owners shapes and its job.
|
||||
3. **The engine wrapper** — a few small helpers that adapt behaviors to the engine's specific needs (closing over media types, threading config). Nothing structural; just glue.
|
||||
|
||||
The rest of this doc walks each stage.
|
||||
|
||||
---
|
||||
|
||||
## State, owners, and config
|
||||
|
||||
Every SPF composition is parameterized by three shapes:
|
||||
|
||||
```ts
|
||||
export interface SimpleHlsEngineState {
|
||||
/**
|
||||
* The presentation being played. A caller writes `{ url }`;
|
||||
* `resolvePresentation` parses the manifest and populates the rest.
|
||||
*/
|
||||
presentation?: MaybeResolvedPresentation;
|
||||
preload?: 'auto' | 'metadata' | 'none';
|
||||
selectedVideoTrackId?: string;
|
||||
selectedAudioTrackId?: string;
|
||||
selectedTextTrackId?: string;
|
||||
bandwidthState?: BandwidthState;
|
||||
abrDisabled?: boolean;
|
||||
currentTime?: number;
|
||||
playbackInitiated?: boolean;
|
||||
mediaSourceReadyState?: MediaSource['readyState'];
|
||||
}
|
||||
|
||||
export interface SimpleHlsEngineOwners {
|
||||
mediaElement?: HTMLMediaElement;
|
||||
mediaSource?: MediaSource;
|
||||
videoBuffer?: SourceBuffer;
|
||||
audioBuffer?: SourceBuffer;
|
||||
videoBufferActor?: SourceBufferActor;
|
||||
audioBufferActor?: SourceBufferActor;
|
||||
textTracksActor?: TextTracksActor;
|
||||
segmentLoaderActor?: TextTrackSegmentLoaderActor;
|
||||
}
|
||||
|
||||
export interface SimpleHlsEngineConfig {
|
||||
initialBandwidth?: number;
|
||||
preferredAudioLanguage?: string;
|
||||
preferredSubtitleLanguage?: string;
|
||||
includeForcedTracks?: boolean;
|
||||
enableDefaultTrack?: boolean;
|
||||
}
|
||||
```
|
||||
|
||||
The split:
|
||||
|
||||
- **State** holds reactive playback data — the manifest, selected track ids, current time, bandwidth estimate. It flows through the composition over time. Each field is a slot in the state signal that any behavior can read or write.
|
||||
- **Owners** holds **resources** — values with identity and imperative interfaces, not just data. The `<video>` element, the `MediaSource`, source buffers, the actors managing text-track lifecycles. If you'd pass the thing around by reference and call methods on it, it belongs in owners. Behaviors observe and act on resources directly; the signal makes their lifecycle (appearance, replacement, removal) reactive.
|
||||
- **Config** holds static creation-time options — thresholds, language preferences, feature flags. It doesn't change during the engine's lifetime. Behaviors read it directly, usually with a fallback.
|
||||
|
||||
The shapes are unions of what every behavior in the composition needs. Adding a new behavior that requires a new state field means adding it to the engine's state interface. Removing a behavior is the inverse.
|
||||
|
||||
Each behavior receives `{ state, owners, config }` and reads or writes only the slots it cares about. Nothing else. A behavior that only reads `mediaElement` doesn't know about `bandwidthState` and can't accidentally interact with it. (The HLS engine's source aliases this argument shape as `Deps` for brevity in the wrapper helpers — it's a local convenience, not framework vocabulary.)
|
||||
|
||||
---
|
||||
|
||||
## Stage 1 — Initial state and manifest resolution
|
||||
|
||||
The first three entries are the lead-in:
|
||||
|
||||
```ts
|
||||
syncPreloadAttribute,
|
||||
trackPlaybackInitiated,
|
||||
resolvePresentation,
|
||||
```
|
||||
|
||||
**`syncPreloadAttribute`** mirrors the `<video preload>` attribute into `state.preload`. This makes the preload mode reactive — anything downstream that wants to react to preload changes (for example, "should we eagerly fetch the manifest?") can subscribe to a signal instead of polling the DOM.
|
||||
|
||||
**`trackPlaybackInitiated`** sets `state.playbackInitiated` to `true` once the user has tried to play (the element is no longer paused). It's a small reactor that watches the media element's `play`/`pause` events. Why it matters: behaviors that should only run after the user interacts (or after autoplay fires) can gate on `state.playbackInitiated`.
|
||||
|
||||
**`resolvePresentation`** is the first behavior that does real network work. It watches `state.presentation` and, when it sees an unresolved value (`{ url }` with no `id`), fetches the multivariant playlist, parses it, and writes the resolved `Presentation` back to the same slot. The lifecycle lives in one slot: a caller writes `{ url }`, the resolver replaces it with a fully populated `Presentation`. Behaviors that only need the URL read `presentation.url`; behaviors that need resolved fields use `isResolvedPresentation` (or check for `selectionSets`) to narrow.
|
||||
|
||||
A pattern shows up here that recurs throughout: **behaviors gate themselves on preconditions and write their results back to state.** They don't take inputs as function arguments. They read from a known signal slot, do their work, and write to a known signal slot. The composition is wired through state, not through call ordering.
|
||||
|
||||
---
|
||||
|
||||
## Stage 2 — Track selection
|
||||
|
||||
Once the manifest is resolved, the engine picks one track per type:
|
||||
|
||||
```ts
|
||||
selectVideoTrack,
|
||||
selectAudioTrack,
|
||||
selectTextTrack,
|
||||
```
|
||||
|
||||
These are *wrappers* defined in `engine.ts` that share their names with — and close over — the imported behaviors from `playback/behaviors/select-tracks.ts`. The imported behaviors accept their own configuration parameter (initial bandwidth for video, preferred language for audio, and so on); the wrapper closes over the engine's config and threads the relevant fields into the behavior:
|
||||
|
||||
```ts
|
||||
import { selectVideoTrack as _selectVideoTrack } from '../../behaviors/select-tracks';
|
||||
|
||||
const selectVideoTrack = ({ config, ...deps }: Deps) =>
|
||||
_selectVideoTrack(deps, {
|
||||
type: 'video',
|
||||
...(config.initialBandwidth !== undefined && { initialBandwidth: config.initialBandwidth }),
|
||||
});
|
||||
```
|
||||
|
||||
The wrapper takes the natural name; the imported behavior gets a leading-underscore alias for local use. The composition list reads as a flat list of behaviors at the right level of abstraction — the wrappers vanish into anonymity, which is exactly right since they're just adapter glue.
|
||||
|
||||
This is the **wrapper pattern** that recurs throughout the engine. Two kinds, both visible in the composition list:
|
||||
|
||||
- **Media-type wrappers** (`loadVideoSegments`, `resolveVideoTrack`, …) close over a fixed `type: 'video' | 'audio' | 'text'` value. Their underlying behavior takes the type as config; the wrapper makes it concrete.
|
||||
- **Config-aware wrappers** (`selectVideoTrack`, `switchQuality`, the engine-local versions) close over the engine's config and pass relevant fields to the underlying behavior's own config parameter.
|
||||
|
||||
The wrappers exist because the underlying behaviors are **engine-agnostic** — `_selectVideoTrack` doesn't know about `SimpleHlsEngineConfig` or that `bandwidthState` lives on engine state. It accepts a `VideoSelectionConfig` from its caller. The engine wrapper is the thin layer that says "for this engine's config, the initial bandwidth comes from `config.initialBandwidth`."
|
||||
|
||||
The behaviors themselves are split across two locations on purpose:
|
||||
|
||||
- **Pure logic** lives in `media/primitives/select-tracks.ts` — `pickVideoTrack`, `pickAudioTrack`, `pickTextTrack`, `canSelectTrack`, `shouldSelectTrack`. No signals, no effects. Just functions that take a `Presentation` and a config and return an id.
|
||||
- **Orchestrations** live in `playback/behaviors/select-tracks.ts` — `selectVideoTrack`, `selectAudioTrack`, `selectTextTrack`. These wrap the pure logic in `effect()`, gate on preconditions, and write the chosen id to `state.selected{Video,Audio,Text}TrackId`.
|
||||
|
||||
The split keeps the SPF-free CML-style helpers reusable outside SPF, while the SPF-integrated behaviors stay thin and easy to swap.
|
||||
|
||||
---
|
||||
|
||||
## Stage 3 — Track resolution
|
||||
|
||||
A selected track is just an id at first — its segment list still has to come from a *media playlist*. The next three behaviors fetch and parse those:
|
||||
|
||||
```ts
|
||||
resolveVideoTrack,
|
||||
resolveAudioTrack,
|
||||
resolveTextTrack,
|
||||
```
|
||||
|
||||
These are media-type wrappers around the same `resolveTrack` behavior:
|
||||
|
||||
```ts
|
||||
const resolveVideoTrack = (deps: Deps) => resolveTrack(deps, { type: 'video' as const });
|
||||
```
|
||||
|
||||
`resolveTrack` watches `state.presentation` and the matching `selectedXTrackId` for its type. When both are set, it finds the partially-resolved track, fetches its media playlist, parses it into segments, and writes the now-resolved track back into `state.presentation`. (The `Presentation` type allows tracks to hold either a partially-resolved or fully-resolved shape — the URL is enough to find a track; the segments are what loadSegments needs.)
|
||||
|
||||
Three behaviors, one shared `resolveTrack`. Each closes over its `type` so the composition list stays flat. From outside, you'd hardly know they share an implementation.
|
||||
|
||||
Now `state.presentation` is fully populated for the selected tracks. Everything downstream — duration, MSE, segment loading, end-of-stream — reads from there.
|
||||
|
||||
---
|
||||
|
||||
## Stage 4 — Presentation duration
|
||||
|
||||
```ts
|
||||
calculatePresentationDuration,
|
||||
```
|
||||
|
||||
`calculatePresentationDuration` reads the resolved tracks and computes the presentation's total duration (max of selected track durations). It writes the result back to `state.presentation.duration`. This is a small bookkeeping behavior — the duration is derived from data already in state — but breaking it out keeps the derivation reactive: any downstream behavior that needs to know the duration just reads `state.presentation.duration`, and re-runs when it changes.
|
||||
|
||||
It's separated from `resolveTrack` because the duration depends on *which* tracks are selected (the engine could change selection later, in theory, with a different duration). Keeping the derivation in its own behavior means every track-selection change automatically re-derives.
|
||||
|
||||
---
|
||||
|
||||
## Stage 5 — MSE setup
|
||||
|
||||
The next three behaviors stand up the MSE pipeline:
|
||||
|
||||
```ts
|
||||
setupMediaSource,
|
||||
updateDuration,
|
||||
setupSourceBuffers,
|
||||
```
|
||||
|
||||
This is where the engine first touches the media element directly. Up until now, behaviors have been operating on plain data in `state` — manifests, URLs, ids, durations. MSE is the bridge: a `MediaSource` attaches to the `<video>` element via `srcObject` (or an object URL), and `SourceBuffer`s under it accept appended segments.
|
||||
|
||||
**`setupMediaSource`** waits for two preconditions: a `mediaElement` in owners and a `presentation.url` in state. When both arrive, it creates a `MediaSource`, attaches it to the element, and writes both back to owners (`mediaSource`) and state (`mediaSourceReadyState`). The DOM event for "MediaSource is open" is bridged onto `state.mediaSourceReadyState` via the `onMediaSourceReadyStateChange` callback primitive — once that flips to `'open'`, the `mediaSource` is published to owners so downstream behaviors can use it.
|
||||
|
||||
The split between owners and state is deliberate. The MediaSource itself is a resource — you call `addSourceBuffer()` on it, you set its `duration` — so it lives in owners. Its readyState is data — a string that other behaviors gate decisions on — so it lives in state. The DOM events that drive readyState changes get bridged into the SPF signal graph by the small primitive in `media/dom/mse/`, keeping `setupMediaSource` clean.
|
||||
|
||||
**`updateDuration`** waits for the resolved presentation duration (from stage 4) and a MediaSource that's open with idle source buffers, then writes `mediaSource.duration = presentation.duration`. The order matters: setting duration while a SourceBuffer has `updating === true` throws `InvalidStateError`, so the behavior waits for any in-flight appends to settle before writing.
|
||||
|
||||
This is the first place the engine has real coordination concerns: timing among multiple resources. The behavior expresses it declaratively — `effect()` re-runs when any input signal changes, and the gate function checks every precondition. There's no manual sequencing, no callbacks-on-callbacks. Each precondition becomes a signal read; the framework figures out when to fire.
|
||||
|
||||
**`setupSourceBuffers`** does the same lifecycle dance as `setupMediaSource`, but per-track: when a video or audio track is resolved and the MediaSource is open, it calls `addSourceBuffer()` with the track's mime/codec, wraps the resulting `SourceBuffer` in a `SourceBufferActor`, and publishes both the raw buffer and the actor onto owners.
|
||||
|
||||
The actor wrapping is worth pausing on. A raw `SourceBuffer` is imperative: you call `appendBuffer(data)`, you wait for the `updateend` event, you handle errors. Multiple appends can collide. An actor — a state machine that owns the buffer — gives every consumer a single point of contact. They send a message ("append this segment"); the actor serializes the work, exposes its current state via a snapshot signal, and behaves predictably even when several behaviors want to write at once.
|
||||
|
||||
So owners ends up with both `videoBuffer` (the raw `SourceBuffer`, used by `endOfStream` to read `buffered` ranges) and `videoBufferActor` (the wrapper, used by `loadSegments` to send append messages). Same lifetime, two roles.
|
||||
|
||||
---
|
||||
|
||||
## Stage 6 — Playback tracking and ABR
|
||||
|
||||
Once buffers are live and segments can be appended, the engine starts watching playback:
|
||||
|
||||
```ts
|
||||
trackCurrentTime,
|
||||
switchQuality,
|
||||
```
|
||||
|
||||
**`trackCurrentTime`** mirrors the media element's `currentTime` onto `state.currentTime`. Same shape as `syncPreloadAttribute` from stage 1: the DOM event becomes a signal write, and downstream behaviors gate on the reactive value rather than polling the element. `loadSegments` reads it to know how far ahead to fetch; `endOfStream` reads it to know whether the user has reached the end.
|
||||
|
||||
**`switchQuality`** is the ABR loop. It watches `state.bandwidthState` (a running estimate, written by `loadSegments` after each successful segment fetch) and `state.selectedVideoTrackId`. When the estimate moves enough to justify a switch, it writes a different `selectedVideoTrackId`. That triggers `resolveVideoTrack` → `setupSourceBuffers` (if mime/codec changes) → `loadSegments` to start fetching from the new variant. (Like the track-selection wrappers, this is the engine's local version that closes over engine config; the underlying behavior is imported as `_switchQuality`.)
|
||||
|
||||
Two patterns worth pausing on:
|
||||
|
||||
- **State is the bus.** The bandwidth estimator doesn't push to the quality switcher. The quality switcher doesn't pull from the loader. Both read `state.bandwidthState` and respond. Adding a third behavior that needs the estimate (a buffer-health probe, a telemetry stream) is the same: another reader, no rewiring.
|
||||
- **Selection cascades.** Changing `selectedVideoTrackId` doesn't tell anything to "re-resolve, re-buffer, re-load." It just changes the value. The behaviors downstream were already reading it; they'll re-run because `effect()` tracked their reads. Re-resolution and re-buffering happen because the framework noticed.
|
||||
|
||||
The engine has no orchestrator. Each behavior is a small reactor plus a cleanup. The composition is the dataflow.
|
||||
|
||||
---
|
||||
|
||||
## Stage 7 — Segment loading
|
||||
|
||||
```ts
|
||||
loadVideoSegments,
|
||||
loadAudioSegments,
|
||||
```
|
||||
|
||||
Two media-type wrappers around `loadSegments`. This is the busiest behavior — the one that actually fetches segments, samples bandwidth, and pushes data into the source buffers.
|
||||
|
||||
`loadSegments` watches:
|
||||
- `state.presentation` — for the resolved track of its type
|
||||
- `state.selectedVideoTrackId` (or audio) — to know which track's segments to load
|
||||
- `state.currentTime` — to know how far ahead to fetch
|
||||
- `state.bandwidthState` — to keep a running estimate after each fetch
|
||||
- `owners.videoBufferActor` (or audio) — to send append messages
|
||||
- `owners.mediaSource` — to know it's safe to operate on the buffer
|
||||
|
||||
When the preconditions are met, the behavior:
|
||||
|
||||
1. **Plans** which segments to fetch using a forward-buffer policy (look ahead from currentTime by some target buffer length, find segments whose ranges aren't already buffered). The planner is a pure function from `media/buffer/forward-buffer.ts` — given the segments, the buffered ranges, and currentTime, return the list of segments to fetch.
|
||||
2. **Fetches and appends** each segment via the network layer, sampling timing to update `state.bandwidthState`. The `SourceBufferActor` receives `append` messages and serializes them; multiple in-flight calls don't collide.
|
||||
3. **Records** segment metadata (id, byte size, timestamp) into the actor's context so `endOfStream` later knows the last segment is loaded.
|
||||
|
||||
The pure planner from `media/buffer/` is the same kind of split we saw in track selection: framework-free logic that takes data and returns data, lifted out of the orchestration. The behavior wraps it in `effect()`, gates on preconditions, and pushes the results to the actor.
|
||||
|
||||
A subtle bit: `loadSegments` doesn't await the actor's append messages serially in JS — it sends them and lets the actor own ordering. The actor's `SerialRunner` ensures `appendBuffer` calls happen one at a time on the underlying `SourceBuffer`, which is what the MSE spec requires. The behavior just sends.
|
||||
|
||||
---
|
||||
|
||||
## Stage 8 — End of stream
|
||||
|
||||
```ts
|
||||
endOfStream,
|
||||
```
|
||||
|
||||
A single behavior, but one that has to coordinate across the rest of the pipeline. Calling `MediaSource.endOfStream()` tells the browser the stream is done — without it, playback stalls at the end of the buffered range waiting for data that will never arrive. Calling it too early (or while a SourceBuffer is updating) causes errors that crash the demuxer.
|
||||
|
||||
`endOfStream` reads:
|
||||
- `state.presentation` and `state.selected{Video,Audio}TrackId` — to know which tracks' last segments to wait for
|
||||
- `state.mediaSourceReadyState` — must be `'open'` (not `'ended'` or `'closed'`)
|
||||
- `owners.mediaElement` — to check `readyState >= HAVE_METADATA` (a precondition Chrome enforces)
|
||||
- `owners.{video,audio}Buffer` — must exist for selected tracks
|
||||
- `owners.{video,audio}BufferActor` — to know if any are still updating
|
||||
- `owners.{video,audio}BufferActor.snapshot.context.segments` — to verify the last segment of each selected track has been appended
|
||||
|
||||
Every one of those reads happens inside an `effect()`. When *any* changes, the gate function reruns. When all preconditions line up, the behavior calls `mediaSource.endOfStream()`.
|
||||
|
||||
Two things this behavior makes visible about SPF compositions:
|
||||
|
||||
- **Coordination is just reading.** No callbacks, no publish/subscribe, no event bus. The behavior reads from state and owners; the reactive graph re-runs it whenever any read changes. A composition with N behaviors and one shared state signal has N inputs to coordinate, not N² connections.
|
||||
- **Actor snapshots are signals too.** `owners.videoBufferActor?.snapshot.get()` is a regular signal read inside the effect. When the actor transitions from `'updating'` to `'idle'`, the snapshot signal fires, the effect re-runs, and `endOfStream` re-evaluates whether the gate is now passable. Actor state and engine state are the same kind of channel from a behavior's point of view.
|
||||
|
||||
The behavior also handles the seek-back case: if `appendBuffer` re-opens an `'ended'` MediaSource (per the MSE spec, a fresh append on an ended buffer transitions readyState back to `'open'`), `endOfStream` re-evaluates and may call `endOfStream()` again once the new last segment lands.
|
||||
|
||||
---
|
||||
|
||||
## Stage 9 — Text tracks
|
||||
|
||||
```ts
|
||||
syncTextTracks,
|
||||
setupTextTrackActors,
|
||||
loadTextTrackCues,
|
||||
```
|
||||
|
||||
Text tracks are an interesting wrinkle. They don't go through MSE — VTT cues land directly on the `<track>` elements of the media element. The shape of this stage is therefore different from the MSE pipeline above: there are no source buffers, no append serialization, no end-of-stream gate. But the SPF patterns are the same.
|
||||
|
||||
**`syncTextTracks`** mirrors the resolved text-track list onto `<track>` elements under the media element. When a text track is selected (`state.selectedTextTrackId`), the matching `<track>` element gets `mode = 'showing'`; others go to `'disabled'`. This is reactive: changing the selection in state re-runs the effect, which flips the modes.
|
||||
|
||||
**`setupTextTrackActors`** is the actor-provider for text tracks, parallel to `setupSourceBuffers` for video/audio. It creates two actors — `TextTracksActor` (owns the cue list per track) and `TextTrackSegmentLoaderActor` (orchestrates per-segment fetches via the VTT resolver) — and publishes them onto owners.
|
||||
|
||||
In the engine, this is a wrapper that closes over the DOM-bound VTT resolver (`resolveVttSegment`):
|
||||
|
||||
```ts
|
||||
import { setupTextTrackActors as _setupTextTrackActors } from '../../behaviors/dom/setup-text-track-actors';
|
||||
|
||||
const setupTextTrackActors = ({ owners }: Deps) =>
|
||||
_setupTextTrackActors({ owners, config: { resolveTextTrackSegment: resolveVttSegment } });
|
||||
```
|
||||
|
||||
The underlying `_setupTextTrackActors` (in `playback/behaviors/dom/`) is parser-agnostic — it accepts any `resolveTextTrackSegment` function. The DOM-specific binding (using a hidden `<video>` + `<track>` element to leverage the browser's VTT parser) happens at the engine layer. A different engine could supply a different parser without changing the behavior.
|
||||
|
||||
**`loadTextTrackCues`** is the orchestrator. It watches the resolved text track and the actors, and dispatches `load` messages to the segment loader actor whenever new segments need fetching. The actor handles per-segment fetching (and abort on track switch); the orchestrator decides *when* to ask.
|
||||
|
||||
This is the same pattern as MSE: actor-as-resource (`setupTextTrackActors` puts it on owners), orchestrator-as-behavior (`loadTextTrackCues` decides when to send messages). Different platform, same shape.
|
||||
|
||||
---
|
||||
|
||||
## Two ways to call `createComposition`
|
||||
|
||||
A note on the engine's call to `createComposition` itself.
|
||||
|
||||
The HLS engine uses the *explicit* form — passing `<S, O, C>` type arguments to fix the engine's shape directly:
|
||||
|
||||
```ts
|
||||
return createComposition<SimpleHlsEngineState, SimpleHlsEngineOwners, SimpleHlsEngineConfig>(
|
||||
[...behaviors],
|
||||
{ config, initialState, initialOwners }
|
||||
);
|
||||
```
|
||||
|
||||
The minimal/inferred form lets TypeScript intersect each behavior's deps to compute the engine's shape:
|
||||
|
||||
```ts
|
||||
const engine = createComposition([myBehavior]);
|
||||
```
|
||||
|
||||
For engines that aggregate many wrapper-style behaviors (`(deps: Deps) => behavior(deps, {...})`) all sharing the same `Behavior<S, O, C>` type, TypeScript's distributive intersection inference can drop fields — e.g. for the HLS engine, `bandwidthState` would silently disappear from the inferred state, and `initialState: { bandwidthState: ... }` would be flagged as an unknown property. The explicit form sidesteps the inference and uses the engine's declared shapes directly.
|
||||
|
||||
The inferred form remains the right call for small or single-behavior compositions where the per-feature state slices are the source of truth. For aggregating engines, prefer explicit.
|
||||
Reference in New Issue
Block a user