v10/internal/design/spf/presentation-modeling.md

status, date

status	date
draft	2026-05-20

Presentation Modeling

The format-neutral data shape, lifecycle, and parser interface that SPF engines use to model streaming presentations. Today the HLS engine is the only consumer; the contract is intentionally protocol-agnostic so DASH, MoQ, and short-form-specific engines can plug into the same modeling layer with their own parsers.

This document is an architectural deep-dive in the same shape as text-track-architecture.md — it's the canonical reference for the Presentation type, the parsePresentation contract, the per-track resolution pattern, and what new format support would entail. Feature docs in features/ reference this doc rather than duplicate its contents.

Audience:

• Engine contributors writing new behaviors that consume state.presentation — for the data shape and type guards
• Format-extension contributors (DASH / MoQ / short-form) — for the parser interface and what's pluggable vs. hardcoded
• Debuggers working through presentation-related issues — for the lifecycle picture

For the user-observable engine behaviors built on this layer, see the relevant feature docs (source-replacement, audio-playback, subtitles, video-abr, mse-mms-pipeline, buffer-management).

Subsumed candidates: This doc covers the architectural concerns that the candidates hls-multivariant-parsing, media-playlist- resolution, and hls-vod-presentation-modeling previously named — each tracked the HLS-specific parsing + lifecycle work this doc captures format-neutrally. They are not separately-scoped features; implementation work on those areas extends the parser-interface section here. The candidate list (project_spf_feature_candidates.md in agent memory) annotates them as tabled / subsumed.

---

Architecture Overview

   { url }                                 Presentation
   (unresolved)                            (resolved + enriched)
       │                                          ▲
       │                                          │
       ▼                                          │
  ┌─────────────────────────────────────────────────┐
  │   resolvePresentation                            │  parses multivariant manifest
  │   gates on preload + loadActivated               │  via config.parsePresentation
  │   4-state FSM: preconditions-unmet → idle →      │  (HLS today; pluggable for
  │                resolving → resolved              │  other formats)
  └─────────────────────────────────────────────────┘
       │
       ▼  presentation = { id, selectionSets, ... } (resolved, tracks partially-resolved)
  ┌────────────────────────────────────────────────────┐
  │   resolveVideoTrack / resolveAudioTrack /          │  fetches per-track media
  │   resolveTextTrack                                  │  playlist via direct import
  │   2-state FSM; ConcurrentRunner per-track          │  of HLS parseMediaPlaylist
  │   (multiple tracks resolve in parallel)            │  ← not pluggable today
  └────────────────────────────────────────────────────┘
       │
       ▼  tracks gain segments + initialization (resolved)
  ┌────────────────────────────────────────────────┐
  │   calculatePresentationDuration                 │  via config.resolveDuration
  │   variant-agnostic; resolver decides            │  (VoD: track durations;
  │   "when is duration derivable"                  │   Live: Infinity)
  └────────────────────────────────────────────────┘
       │
       ▼  presentation.duration (final)
   consumers: mse-mms-pipeline, buffer-management,
              audio-playback, subtitles, video-abr, ...

The state.presentation slot is a pipeline-pattern multi-writer: multiple behaviors patch different fields onto the same value over time. See features/source-replacement.md for the full multi-writer characterization and the resolved/unresolved cleanup cascade every consumer rides.

---

The CMAF-HAM data model

The data shape is based on CMAF-HAM (Common Media Application Format — Hypothetical Application Model). The framing is explicit in the code:

Protocol-agnostic representation of streaming media content. See https://github.com/AcademySoftwareFoundation/common-media-library

All types live in packages/spf/src/media/types/index.ts.

Base types

Type	Adds	Role
Ham	id: string	All HAM objects carry a string id
AddressableObject	url: string, optional byteRange	URL + optional byte range; range requests are out of scope today but the shape is reserved
TimeSpan	startTime: number, duration: number	Timed range; tracks have startTime: 0 (multi-period support is future)
FrameRate	frameRateNumerator, optional frameRateDenominator	Rational fps (30 vs 29.97)
MediaElementLike	preload: string	Platform-agnostic media element minimum surface; HTMLMediaElement satisfies

Track hierarchy

Track = Ham & AddressableObject & TimeSpan & {
  type: TrackType;            // 'video' | 'audio' | 'text'
  codecs?: string[];          // Optional per HLS spec
  mimeType: string;
  language?: string;
  bandwidth: number;
  initialization?: AddressableObject;  // Init segment (CMAF)
  segments: Segment[];        // Loaded once media playlist resolves
}

VideoTrack = Track & {
  type: 'video';
  initialization: required;   // Always present once resolved
  codecs: required;
  width?: number; height?: number;
  frameRate?: FrameRate;
  audioGroupId?: string;
}

AudioTrack = Track & {
  type: 'audio';
  groupId: string;
  name: string;
  sampleRate: number;
  channels: number;
  default?: boolean;
  autoselect?: boolean;
}

TextTrack = Track & {
  type: 'text';
  groupId: string;
  label: string;
  kind: 'subtitles' | 'captions';
  default?: boolean;
  autoselect?: boolean;
  forced?: boolean;
}

Partial resolution

After multivariant parsing, tracks are partially resolved — metadata populated, but no segments / initialization until the per-track media playlist is fetched. Expressed as a generic:

type PartiallyResolved<T extends Track> = Omit<T, 'segments' | 'initialization' | keyof TimeSpan> & {
  segments?: never;
  duration?: never;
  startTime?: never;
  initialization?: never;
};

Type aliases: PartiallyResolvedVideoTrack, PartiallyResolvedAudioTrack, PartiallyResolvedTextTrack, and the union PartiallyResolvedTrack.

Sets

SwitchingSet  = Ham & { type, tracks: (Partial | Resolved)[] }  — group of tracks switchable seamlessly
SelectionSet  = Ham & { type, switchingSets: SwitchingSet[] }   — top-level grouping by type

Each set is discriminated by track type — VideoSwitchingSet, AudioSwitchingSet, TextSwitchingSet, and the same pattern for selection sets.

Presentation

Presentation = Ham & AddressableObject & Partial<TimeSpan> & {
  selectionSets: SelectionSet[];
}

MaybeResolvedPresentation = AddressableObject & Partial<Omit<Presentation, keyof AddressableObject>>

MaybeResolvedPresentation is the in-state shape: url is always present (the adapter or external code writes it as the seed); id, selectionSets, and duration populate as the pipeline progresses.

Segment

Segment = Ham & AddressableObject & TimeSpan

Plus the constant SEGMENT_TIME_EPSILON = 0.0001 — floating-point tolerance for matching segments by startTime. Used by both the source-buffer dedup and the segment-loader quality-aware filter; single source of truth.

Type guards

isResolvedTrack(track)               // narrows to ResolvedTrack
isResolvedPresentation(presentation) // narrows to Presentation
                                     // requires both id AND selectionSets present
hasPresentationDuration(presentation) // narrows to include required duration

isResolvedPresentation is the load-bearing predicate for the cleanup cascade — every setup behavior in the engine gates on it. See features/source-replacement.md for the cleanup contract this predicate anchors.

Intermediate parser output

interface MediaPlaylistInfo {
  version: number;
  targetDuration: number;
  playlistType: 'VOD' | 'EVENT' | undefined;
  initSegment: AddressableObject | null;
  segments: Segment[];
  duration: number;
  endList: boolean;
}

Internal representation used by the media-playlist parser before being assembled into the full Track structure. Not part of the public API, but format implementations may need analogous intermediate types.

---

The presentation lifecycle

Brief — the full multi-writer pipeline and cleanup cascade live in features/source-replacement.md.

The slot transitions through four phases:

Phase	Slot value	Driver
Unresolved	{ url } (no id, no selectionSets)	Adapter / external write
Resolving	{ url } still (in-flight fetch)	resolvePresentation's 'resolving' entry
Resolved	{ id, url, selectionSets, ... } — tracks partially-resolved	resolvePresentation writes via state.presentation.set(parsed)
Enriched	Same shape, with segments + duration patched	resolveXTrack family + calculatePresentationDuration patch via update(state.presentation, ...)

Source change (URL replacement) routes the slot back through unresolved, which routes the FSM back through 'resolving'. All downstream behaviors gating on isResolvedPresentation exit their positive state and tear down via reactor state-exit.

---

The parser interface

Three pluggability layers exist; only two are actually pluggable today.

config.parsePresentation — multivariant parser (pluggable)

type ParsePresentation = (
  text: string,
  presentation: MaybeResolvedPresentation
) => Presentation;

Format-neutral entry point. The composing engine wires in its format's parser at composition setup. HLS engine wires parseMultivariantPlaylist from media/hls/.

The contract:

• Input: raw manifest body text + the in-state presentation (url always set; consumers may carry through context)
• Output: a fully resolved Presentation with id and selectionSets. Tracks within the selection sets are partially-resolved (no segments / initialization yet)
• Errors: the resolver throws on parse failure; resolvePresentation's entry catches and logs via console.error (with a TODO to route to a state-error slot — see Open questions)

parseMediaPlaylist — per-track playlist parser (not pluggable today)

This is the load-bearing coupling for future format support. packages/spf/src/playback/behaviors/resolve-track.ts imports parseMediaPlaylist directly:

import { parseMediaPlaylist } from '../../media/hls/parse-media-playlist';

There's no config.parseMediaPlaylist config hook today. Adding DASH / MoQ / short-form support requires either:

1. Adding a config.parseMediaPlaylist hook to mirror config.parsePresentation, or
2. Introducing format-specific resolveXTrack behavior variants (e.g., resolveDashVideoTrack) composed instead of the HLS ones.

Choice (1) is simpler structurally; choice (2) is more flexible if per-track resolution differs substantially across formats. The doc flags this; the call goes to whoever lands the second format.

config.resolveDuration — duration resolver (pluggable)

type PresentationDurationResolver = (
  state: PresentationDurationState
) => number | undefined;

Variant-agnostic. The resolver decides "when is duration derivable" for the current variant — calculatePresentationDuration stays format-/composition-agnostic.

• VoD wiring: HLS engine wires getResolvedSelectedTrackDuration — "first resolved selected track's duration, video preferred, audio fallback." Audio-only falls out naturally.
• Live wiring: future live engines would return Number.POSITIVE_INFINITY once the presentation is established as live. MSE-spec value for mediaSource.duration under live; updateMediaSourceDuration propagates it through.

---

Per-track resolution

packages/spf/src/playback/behaviors/resolve-track.ts exports three behaviors — resolveVideoTrack, resolveAudioTrack, resolveTextTrack — that share a setupTrackResolution<K> helper. Each variant supplies its own findTrackToResolve resolver via per-type config.

FSM shape

presentation-unresolved ⟷ presentation-resolved

Two states. Entry into 'presentation-resolved' returns () => runner.abortAll() for state-exit cleanup — source-change cancellation expressed structurally through the state machine.

Concurrent execution

Uses ConcurrentRunner, not SerialRunner. Tracks resolve in parallel; the variants are independent. (There's a @todo in the code about pulling the runner choice into a per-use-case factory.)

The effect body

effects: [
  () => {
    const presentation = peek(state.presentation);
    const trackId = state[selectedKey].get();
    if (!presentation || !trackId) return;

    const track = findTrackToResolve(presentation, trackId);
    if (!track || isResolvedTrack(track)) return;

    runner.schedule(
      new Task(
        async (signal) => {
          const response = await fetchResolvable(track, { signal });
          const text = await getResponseText(response);
          const mediaTrack = parseMediaPlaylist(text, track);
          update(state.presentation, (current) =>
            isResolvedPresentation(current)
              ? updateTrackInPresentation(current, mediaTrack)
              : current
          );
        },
        { id: track.id }
      )
    );
  },
],

Worth noting:

• peek(state.presentation) (untracked) — internal updates (segments added by sibling tasks) don't re-fire the effect. The reactor's state transitions handle the source-change cascade; intra-resolved changes aren't load-bearing for re-firing.
• state[selectedKey].get() (tracked) — selection changes do re-fire the effect; new selection → schedule new fetch.
• Task { id: track.id } — task dedup keyed on track id. Same-track selection re-fires don't double-schedule.
• The commit-time isResolvedPresentation(current) check in the update() callback covers the pathological resolved→resolved- without-unresolved transition — e.g., a direct overwrite of state.presentation from one resolved value to another. State-exit on resolving → unresolved fires runner.abortAll() before the URL change settles, and per the Fetch spec the signal abort cancels in-flight body reads. So by commit time, the resolution we're writing matches the live presentation.

Pipeline-pattern writeback

update(state.presentation, ...) patches the resolved track into the current presentation. The other writers on the slot are:

• Adapter / external (initial unresolved { url } seed)
• resolvePresentation (resolved Presentation with id + selectionSets)
• calculatePresentationDuration (patches duration)

Each writer reads current and writes new with their field added — they never overwrite a field someone else owns. See features/source-replacement.md for the full characterization.

---

Duration resolution

packages/spf/src/playback/behaviors/calculate-presentation-duration.ts

Behavior shape

Pure effect() — not a reactor. Re-runs whenever the resolver's declared inputs change. Today the resolver reads (presentation, selectedVideoTrackId, selectedAudioTrackId).

Resolver contract

type PresentationDurationResolver =
  (state: PresentationDurationState) => number | undefined;

The behavior calls the resolver and writes whatever it returns as long as it's a positive number, including Infinity. undefined, NaN, and <= 0 are skipped — the resolver may return undefined while duration is still indeterminate; subsequent tracked-slot changes re-run the effect until the resolver commits a value.

Fires-at-most-once-per-presentation

The effect early-returns if presentation.duration !== undefined — an already-set duration is never overwritten. The next reset arrives structurally when a new (unresolved) presentation replaces the current one.

VoD default

The HLS engine wires getResolvedSelectedTrackDuration from media/utils/track-selection.ts:

• Pick the first resolved selected track (video preferred, audio fallback)
• Return its duration
• Audio-only falls out naturally (no video selected → audio is first resolved)

Why a resolver hook instead of variant-specific behaviors

calculatePresentationDuration itself doesn't know about VoD vs Live; the resolver does. Live engines compose the same behavior with a different config.resolveDuration that returns Infinity once the presentation is established as live. This avoids forking the behavior on variant axis.

---

What new format support would entail

Concrete TODO list for an engine contributor adding DASH / MoQ / short-form:

1. Write a parsePresentation implementation for the new format. Output a Presentation with selectionSets containing partially-resolved tracks. Reuse the CMAF-HAM types — the data model is format-neutral; the parser maps format-specific concepts onto it.

2. Address the parseMediaPlaylist coupling. Pick one:

   • Add a config.parseMediaPlaylist hook to mirror config.parsePresentation. Simplest; minimal change to the resolution path.
   • Introduce format-specific resolve-track behavior variants. More flexible if per-track resolution differs substantially (e.g., DASH segment indexes via sidx vs HLS media playlists).

3. Decide on resolveDuration wiring. Most VoD-like formats can reuse getResolvedSelectedTrackDuration — it only depends on resolved tracks having duration set, which is format-neutral.

4. Compose an engine variant — e.g., createSimpleDashEngine — that wires the new parsers into createComposition. Reuse the existing behavior set; swap only the parser configs.

The bracketed candidates in clusters.md Presentation modeling cluster ([hls-multivariant-parsing], [media-playlist-resolution], [hls-vod-presentation-modeling]) are placeholders that, in practice, collapse into "presentation-modeling (architecture) + per-format parser implementations." When the second format lands, drop the brackets and file format-specific architectural docs as siblings to this one (dash-manifest-parsing.md, etc.).

---

Open questions

• parseMediaPlaylist pluggability — when does this become urgent? Today's direct import is a clear coupling. Two distinct forcing functions: (a) the first format-extension PR will surface the cost for non-HLS formats, and (b) HLS-only extensions like features/ll-hls-support.md grow the parsed-track output schema (server-control flags, parts, preload hints, skip metadata) — even staying HLS-only, the parser-output shape doesn't stay frozen. Worth keeping the option open in design discussions.
• The PartiallyResolved<T> pattern — fully format-neutral? The shape is defined generically (Omit<T, 'segments' | 'initialization' | keyof TimeSpan>), but the semantic (multivariant playlist surfaces metadata; media playlist surfaces segments) is HLS-shaped. Whether DASH's manifest-only-no-mediaplaylist or MoQ's streaming model fits this two-phase resolution naturally is an open question for the second format.
• resolve-track.ts @todos. The runner choice (Concurrent vs Serial vs replace-previous) is currently hardcoded; per-use-case factory mentioned as future work. Same for the task-creation path (createResolveTrackTask mentioned). When per-track resolution needs richer policies (retry, backoff, multi-CDN failover), these hook points become load-bearing.
• Error surface. resolvePresentation and resolveXTrack both log to console.error on fetch / parse failure. A state-error slot doesn't exist yet; consumers can't observe "this source failed to load." Cross-references the same open question in features/source-replacement.md.
• Multi-period support. Track.startTime is documented as "always 0 (for future multi-period support)." When multi-period arrives, the Presentation shape needs an axis for period grouping and the lifecycle changes (period transitions inside a single source).

---

See also

• features/source-replacement.md — the multi-writer state.presentation pipeline + the isResolvedPresentation cleanup contract every consumer rides. Required reading for the lifecycle picture.
• features/capability-probing.md — post-parse consumer of the data shape this doc defines. Filters presentation.selectionSets to drop browser-unsupported renditions before selection runs. Parser stays format-neutral.
• features/clusters.md § Presentation modeling — the cluster description for this layer
• features/audio-playback.md, features/subtitles.md, features/video-abr.md — per-type consumers of resolved tracks; ground their feature surface in attributes this layer surfaces
• features/mse-mms-pipeline.md, features/buffer-management.md — setup behaviors and segment loaders that consume the resolved presentation
• text-track-architecture.md — peer architectural doc (text-track-specific deep-dive)
• conventions/signals.md — pipeline-pattern multi-writer slot convention (state.presentation is the canonical worked example)
• conventions/config.md — when a tunable becomes config vs hardcoded (the parseMediaPlaylist coupling is a case study for the opposite direction: hardcoded, but should be config)
• packages/spf/src/media/types/index.ts — canonical type definitions
• common-media-library upstream — the CMAF-HAM model this layer is based on