mirror of
https://github.com/zoriya/v10.git
synced 2026-08-15 18:34:22 +00:00
docs(spf): add live-presentation-modeling design doc
Companion to presentation-modeling.md for streams that change over time (live / DVR / event). Spine is a category decomposition that keeps data separated by change-rate and consumer: content snapshot (snapshot/merge segment identity via media-sequence + URL/byteRange), streamType, completeness-as-duration, refetch policy as its own category, the PROGRAM-DATE-TIME sync anchor, and the derived consumer surface. Captures the debated decisions and open questions; scopes to sliding-window live (LL-HLS / DVR / discontinuity / timestampOffset deferred). Adds the reciprocal companion link in presentation-modeling.md. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Christian Pillsbury
co-authored by
Claude Opus 4.8
Claude Opus 4.8
parent
a74b8c0993
commit
820f6fe45c
@@ -0,0 +1,376 @@
|
||||
---
|
||||
status: draft
|
||||
date: 2026-06-12
|
||||
---
|
||||
|
||||
# Live Presentation Modeling
|
||||
|
||||
The data-model extension for streaming presentations that **change over
|
||||
time** — live, and the related DVR / event-stream variants. Companion
|
||||
to [presentation-modeling.md](./presentation-modeling.md), which models
|
||||
a presentation as a *one-shot parse* (the VOD assumption baked into the
|
||||
engine today); this doc adds the categories the model needs once a media
|
||||
playlist is re-fetched repeatedly and its content slides forward.
|
||||
|
||||
It defines **what** the model captures for live and **how** that's
|
||||
shaped — protocol-agnostically, capturing change over time, without
|
||||
conflating categories that change at different rates. It is a model /
|
||||
"why" doc, not an implementation plan: the parser refactor and the
|
||||
reload-loop behavior that *consume* this model are
|
||||
[live-stream-support](./features/live-stream-support.md) +
|
||||
`.claude/plans/` territory.
|
||||
|
||||
**Scope.** The foundation is **sliding-window live** (segments roll off
|
||||
the front; standard HLS live). Explicitly downstream, called out where
|
||||
they touch the model but not designed here:
|
||||
[dvr-event-stream-support](./features/dvr-event-stream-support.md)
|
||||
(growing window),
|
||||
[ll-hls-support](./features/ll-hls-support.md) (partial segments / delta
|
||||
playlists), `EXT-X-DISCONTINUITY` handling, and the A/V-sync-via-
|
||||
`timestampOffset` decision ([non-zero-pts-support](./features/non-zero-pts-support.md)).
|
||||
|
||||
**Audience:**
|
||||
- **Engine contributors** building the reload loop / parser refactor —
|
||||
for the categories and where each lives.
|
||||
- **Format-extension contributors** — the live concepts here are
|
||||
protocol-neutral (DASH `@type=dynamic` etc. map onto the same homes).
|
||||
- **Debuggers** working live timeline / sync issues.
|
||||
|
||||
---
|
||||
|
||||
## Problem
|
||||
|
||||
The model in [presentation-modeling.md](./presentation-modeling.md)
|
||||
assumes **one-shot resolution**: `resolvePresentation` parses the
|
||||
manifest once, each track's media playlist is fetched once, and
|
||||
`Track.segments` is a permanent list. For live that's wrong on every
|
||||
axis:
|
||||
|
||||
- The media playlist is **re-fetched** on a cadence; segments **append**
|
||||
at the live edge and **roll off** the front.
|
||||
- Duration is **open-ended** until the stream ends.
|
||||
- Separate audio / video media playlists must stay **time-aligned**
|
||||
across their independent reloads.
|
||||
|
||||
And the current types drop everything live needs. `parseMediaPlaylist`
|
||||
recognizes but discards `EXT-X-TARGETDURATION`, `EXT-X-PLAYLIST-TYPE`,
|
||||
and `EXT-X-ENDLIST` (see `packages/spf/src/media/hls/parse-media-playlist.ts`),
|
||||
never reads `EXT-X-MEDIA-SEQUENCE` or `EXT-X-PROGRAM-DATE-TIME`, and the
|
||||
`MediaPlaylistInfo` type that *names* the playlist-level fields is
|
||||
**orphaned** — nothing produces it (the parser merges straight into a
|
||||
`Track`). Two snapshot-local artifacts actively break on the first
|
||||
reload: `segment.id` is `segment-${index}` (position within *this*
|
||||
parse) and `startTime` accumulates from `0` each parse — so after
|
||||
roll-off, the same media segment gets a new id and a new start time.
|
||||
|
||||
So before any reload behavior exists, we have to decide *what* to model
|
||||
and *how* — and the central discipline is **not conflating categories**:
|
||||
"when should this playlist be re-fetched" is not the same kind of thing
|
||||
as "what segments does this playlist contain at time *t*."
|
||||
|
||||
---
|
||||
|
||||
## The category decomposition
|
||||
|
||||
The move: data that today is lumped onto `Track` or dropped actually
|
||||
falls into distinct categories that **change at different rates** and
|
||||
**serve different consumers**. Separating them *is* the design.
|
||||
|
||||
| Cat | Concept | Protocol-neutral home | Changes… |
|
||||
|---|---|---|---|
|
||||
| **[1] Content snapshot** | the windowed segment list at time *t* | per-track | every reload (append + roll-off) |
|
||||
| **[2a] Stream nature** | `streamType: live \| on-demand` | `Presentation` | never (stable for the source) |
|
||||
| **[2b] Completeness** | will it keep changing? → duration finite vs `Infinity` | presentation duration | once (terminal transition) |
|
||||
| **[3] Refetch policy** | suggested reload cadence | *separate from [1]* | ~stable |
|
||||
| **[4] Sync anchor** | wall-clock anchor for a shared timeline | per-segment → presentation-level | per discontinuity region |
|
||||
| **[5] Derived surface** | live edge, seekable window, DVR window | computed, not stored | continuously |
|
||||
|
||||
The change-rate column is itself the argument for the split: the segment
|
||||
list churns every reload, the nature is fixed, the policy is ~constant,
|
||||
the anchor moves only at discontinuities. Entangling them — as today's
|
||||
`Track` does, and as CMAF-HAM's internal `Manifest` does (see
|
||||
[Prior art](#prior-art)) — drags the stable facts around with the
|
||||
churning snapshot.
|
||||
|
||||
### [1] Content snapshot — a windowed, mergeable segment list
|
||||
|
||||
A resolved track's content is a **snapshot at time *t***, not a
|
||||
permanent list. Each reload yields a new snapshot that must be
|
||||
**merged** into the retained one. Three identifiers do three distinct
|
||||
jobs and compose:
|
||||
|
||||
- **media-sequence — the merge arithmetic (primary).** `EXT-X-MEDIA-
|
||||
SEQUENCE` is the sequence number of the first segment; it is monotonic
|
||||
and immutable per segment (advances as the window slides). So the
|
||||
merge is pure arithmetic:
|
||||
`offset = next.firstMediaSequence − prev.firstMediaSequence` is how
|
||||
many rolled off the front, `prev[offset]` aligns with `next[0]`, and
|
||||
anything past prev's tail is new. No scanning for the overlap — and it
|
||||
distinguishes the edges informatively: `offset ≥ prev.length` means a
|
||||
full window turnover (everything new), which it *knows* rather than
|
||||
inferring from "zero matches."
|
||||
- **URL + byteRange — the equality check / fallback.** Per the HLS spec,
|
||||
a server may only append to the end and remove from the front; a
|
||||
segment's URI is stable while it remains in the playlist. So
|
||||
`prev[offset].url === next[0].url` (with byteRange — byterange streams
|
||||
share a URL) **validates** the arithmetic alignment, and is the
|
||||
fallback when a non-conformant server resets sequence numbers. Our
|
||||
`Segment` already carries both `url` and `byteRange`.
|
||||
- **`segment.id` — the stable model handle.** Kept as a modeled field,
|
||||
separate from the cross-snapshot identity question.
|
||||
|
||||
**`startTime` stability.** media-sequence gives identity, ordering, and
|
||||
new-vs-old exactly — but *not* absolute `startTime` across a full
|
||||
turnover, because rolled-off durations are gone and per-segment
|
||||
durations vary (counting segments ≠ counting time). The common path is
|
||||
covered: the spec's removal rule (a live playlist must retain ≥ 3×
|
||||
target duration) **guarantees consecutive reloads overlap**, so we carry
|
||||
`startTime` forward from the matched overlap and accumulate `EXTINF` for
|
||||
the new tail — exact. Only the no-overlap recovery case (long
|
||||
background / stall) needs the [4] PDT anchor, or accepts an estimate.
|
||||
|
||||
**Naive → optimized.** Naive: re-parse + offset-splice every reload.
|
||||
Optimizations later: skip the merge when `mediaSequence` is unchanged
|
||||
(this is literally the spec's "wait one-half target duration" backoff),
|
||||
then avoid re-allocating the carried-over portion. Delta playlists
|
||||
(`EXT-X-SKIP`) stay deferred with LL-HLS.
|
||||
|
||||
**Known limitation: URL recycling.** A server that recycles segment
|
||||
filenames (a ring buffer faking live by looping fixed files) violates
|
||||
URL uniqueness across the stream lifetime. media-sequence is immune;
|
||||
URL-only identity is not. We accept this as a known limitation because
|
||||
it is **self-consistently out of scope**: a conformant recycling stream
|
||||
inherently repeats encoded timestamps and so requires `EXT-X-
|
||||
DISCONTINUITY` at the loop point — which we've deferred anyway. Real
|
||||
live (Mux, or ffmpeg with monotonic timestamps + unique URLs) is
|
||||
unaffected.
|
||||
|
||||
### [2a] Stream nature — `streamType`
|
||||
|
||||
A new `streamType: 'live' | 'on-demand'` on `Presentation`: the
|
||||
semantic, consumer-facing nature of the source, stable for its life.
|
||||
**Derived from `EXT-X-PLAYLIST-TYPE` alone** — `VOD → on-demand`;
|
||||
everything else (`EVENT`, or the tag absent) `→ live`. Crucially, this
|
||||
axis is **orthogonal to completeness ([2b])**: `EXT-X-ENDLIST` never feeds
|
||||
it. That's the payoff — a *live stream that has ended* (finite duration,
|
||||
no refetch) stays distinguishable from an *on-demand asset* (also finite,
|
||||
no refetch) by `streamType`, instead of collapsing into "they look the
|
||||
same." (Note this is the axis DASH models only implicitly — DASH's
|
||||
`@type=dynamic/static` is really [2b], not [2a].)
|
||||
|
||||
### [2b] Completeness — expressed as duration finiteness
|
||||
|
||||
Whether the presentation will keep changing — and therefore whether
|
||||
refetch is needed and when `endOfStream` becomes reachable — is
|
||||
**expressed as duration finiteness**, not a separate flag: ongoing live
|
||||
→ `duration = Infinity`; complete (`EXT-X-ENDLIST` observed) → finite.
|
||||
This reuses the existing pluggable `config.resolveDuration` hook (see
|
||||
[mse-mms-pipeline](./features/mse-mms-pipeline.md)) — no new model field.
|
||||
The equivalence holds for **both** sliding-window *and* event/growing
|
||||
streams (both are `Infinity` while live → finite on termination); the
|
||||
sliding-vs-event difference lives entirely in [5] (`seekableStart`), not
|
||||
here.
|
||||
|
||||
### [3] Refetch policy — separate from content
|
||||
|
||||
`EXT-X-TARGETDURATION` drives the suggested reload cadence (reload ≈ 1×
|
||||
target; ½× on an unchanged playlist). This is the category the
|
||||
decomposition most insists on isolating: it is a **delivery / scheduling**
|
||||
concern, *not* part of the time-*t* content snapshot, even though HLS
|
||||
happens to deliver both in the same playlist text. Its model placement
|
||||
(per-track vs presentation-level) is an [open question](#open-questions).
|
||||
|
||||
### [4] Sync anchor — `PROGRAM-DATE-TIME`
|
||||
|
||||
`EXT-X-PROGRAM-DATE-TIME` is captured per-segment in the snapshot. It is
|
||||
the **wall-clock anchor common across separate renditions** — the spec's
|
||||
designated mechanism for keeping demuxed audio and video aligned. The
|
||||
model's job here is to *capture* PDT; the presentation-level anchor
|
||||
derivation and *how it drives per-track `timestampOffset`* is the
|
||||
deferred A/V-sync decision (see [Open questions](#open-questions) and
|
||||
[non-zero-pts-support](./features/non-zero-pts-support.md)). PDT alone
|
||||
suffices as a sync anchor under the no-mid-stream-discontinuity
|
||||
assumption this foundation makes; `EXT-X-DISCONTINUITY(-SEQUENCE)` is
|
||||
deferred (it is not live-exclusive — ad-stitched VOD has it too — and is
|
||||
exactly *when* `timestampOffset` must be recomputed).
|
||||
|
||||
### [5] Derived surface — computed, never stored
|
||||
|
||||
The consumer-facing live vocabulary —
|
||||
[media-ui-extensions](#prior-art) `streamType`, `liveEdgeStart`,
|
||||
`seekable` (with `end()` constrained to the seekable live edge),
|
||||
`targetLiveWindow`, and `duration = Infinity` — is **entirely
|
||||
derivable** from [1]–[4] (e.g. `liveEdgeStart ≈ seekable.end() − 3×
|
||||
targetDuration`). The model's job is to capture [1]–[4] cleanly enough to
|
||||
*produce* these, not to store them. This matches
|
||||
[live-stream-support](./features/live-stream-support.md)'s "live edge is
|
||||
a derived signal, not a state slot." The one place sliding vs event
|
||||
diverges is the `seekableStart` derivation (sliding advances; event/DVR
|
||||
pinned at the first retained segment).
|
||||
|
||||
---
|
||||
|
||||
## Type-shape implications
|
||||
|
||||
What this means for `packages/spf/src/media/types/index.ts` and the
|
||||
parser — at the "what / why" level; the parser refactor and merge
|
||||
behavior are the implementation efforts that consume this.
|
||||
|
||||
- **`Segment`** — keep `id`; add optional `programDateTime` ([4]). Identity
|
||||
via `url` + `byteRange` already present.
|
||||
- **Per-snapshot media-playlist representation** — the live fields
|
||||
(`mediaSequence`, `targetDuration`) belong to *the playlist at time t*,
|
||||
which is what the orphaned `MediaPlaylistInfo` was reaching for. The
|
||||
parser refactor's central question is whether to **resurrect
|
||||
`MediaPlaylistInfo` as the real parser output** (a faithful per-fetch
|
||||
representation, assembled/merged into `Track` separately) or keep
|
||||
merging into `Track` and hang the fields there. The snapshot/merge
|
||||
model argues for a faithful per-fetch representation. This is the seam
|
||||
where this doc meets the parser refactor.
|
||||
- **`Presentation`** — add `streamType` ([2a]). Completeness ([2b]) rides the
|
||||
existing `duration` field. Refetch policy ([3]) placement TBD.
|
||||
|
||||
---
|
||||
|
||||
## Prior art
|
||||
|
||||
- **HLS spec (RFC 8216bis).** The invariants this model leans on:
|
||||
media-sequence monotonic + immutable per segment; `EXTINF` immutable
|
||||
per segment; the ≥ 3×-target-duration retention rule (which guarantees
|
||||
reload overlap); reload cadence (≈ 1× target, ½× unchanged); PDT as the
|
||||
per-segment cross-rendition wall-clock anchor; `PLAYLIST-TYPE` /
|
||||
`ENDLIST` as the two orthogonal nature/completeness signals.
|
||||
- **CMAF-HAM** (`common-media-library`) — *cautionary.* Its mappers model
|
||||
the content snapshot only and **drop** media-sequence (hardcodes 0),
|
||||
endlist, playlist-type, and PDT. Where it gestures at policy (the
|
||||
internal, unexported `Manifest` with `type: dynamic` /
|
||||
`minimumUpdatePeriod` / `timeShiftBufferDepth`) it **lumps refetch
|
||||
policy onto the same container as content** — the exact [1]/[3] conflation
|
||||
we're avoiding. It confirms the homes we need but offers no clean one
|
||||
to copy.
|
||||
- **media-ui-extensions** — the [5] consumer vocabulary: `streamType`
|
||||
(`live` / `on-demand`), `liveEdgeStart`, constrained `seekable`,
|
||||
`targetLiveWindow` (`0` / finite / `Infinity` — non-DVR / sliding-DVR /
|
||||
full-DVR), `duration = Infinity` for live.
|
||||
|
||||
---
|
||||
|
||||
## Key decisions
|
||||
|
||||
Documented because they were debated; alternatives weighed.
|
||||
|
||||
### Segment identity: media-sequence arithmetic, URL+byteRange as check
|
||||
|
||||
**Decision:** Merge snapshots by media-sequence arithmetic; use
|
||||
URL+byteRange to validate the alignment and as the fallback when
|
||||
sequence numbers can't be trusted; keep `segment.id` as a separate model
|
||||
handle.
|
||||
|
||||
**Alternatives:**
|
||||
- *URL+byteRange only* — works for the steady-state overlap and needs no
|
||||
sequence field, but requires *scanning* to find the overlap and can't
|
||||
distinguish a full turnover from corruption (both look like "zero
|
||||
matches").
|
||||
- *Minimize fields (drop media-sequence)* — optimizes field count at the
|
||||
cost of update-logic complexity.
|
||||
|
||||
**Rationale:** Optimizing update-logic simplicity beats minimizing
|
||||
fields. Retaining media-sequence makes the merge pure arithmetic and
|
||||
handles the turnover edge *more* informatively, with URL+byteRange a
|
||||
cheap correctness guard layered on top.
|
||||
|
||||
### `streamType` orthogonal to completeness
|
||||
|
||||
**Decision:** Derive `streamType` from `PLAYLIST-TYPE` alone; express
|
||||
completeness as duration finiteness; never let `ENDLIST` touch
|
||||
`streamType`.
|
||||
|
||||
**Alternatives:**
|
||||
- *Fold "ended live looks like VOD" into on-demand* — i.e. let
|
||||
endlist-at-first-fetch mean on-demand. Rejected: it leaks [2b] into [2a],
|
||||
the very conflation we're avoiding, and erases the live-vs-was-live
|
||||
distinction.
|
||||
|
||||
**Rationale:** Keeps the two axes independent and the live-ended state
|
||||
expressible. Accepted cost: an untagged true-VOD (no `PLAYLIST-TYPE:VOD`)
|
||||
is labeled `live` — behaviorally invisible (refetch gates on
|
||||
duration/endlist, not `streamType`), only the label differs. Matters
|
||||
more for VJS-as-general-player than for well-tagged sources; `unknown`
|
||||
stays available as a hedge if a real consumer is bitten.
|
||||
|
||||
### Discontinuities, LL-HLS, and the `timestampOffset` strategy are out of scope
|
||||
|
||||
**Decision:** The foundation assumes continuous timestamps and no
|
||||
mid-stream discontinuities; PDT is captured but its consumption into
|
||||
`timestampOffset` is deferred.
|
||||
|
||||
**Rationale:** Discontinuity handling is not live-exclusive and is its
|
||||
own concern; LL-HLS is a separate XL feature; and the A/V-sync question
|
||||
(how PDT drives per-track `timestampOffset`) is the genuinely hard
|
||||
downstream decision that this model should *enable* rather than
|
||||
prematurely fix.
|
||||
|
||||
---
|
||||
|
||||
## Open questions
|
||||
|
||||
- **[3] Refetch-policy placement — per-track vs presentation-level.** In
|
||||
HLS each media playlist (= each `Track`) carries its own
|
||||
`TARGETDURATION`; in DASH, `minimumUpdatePeriod` is MPD-level
|
||||
(presentation). Per-track bakes an HLS assumption; presentation-level
|
||||
fights HLS's per-playlist reality. Likely answer: a presentation-level
|
||||
concept fed by per-track source data — but decide deliberately. Ties
|
||||
to live-stream-support's per-type reload-coordination question, and to
|
||||
the observation that a not-yet-ended track *must* keep being refetched
|
||||
to stay current, so timeline reconciliation across tracks can't be
|
||||
fully independent even if fetch scheduling is.
|
||||
- **[4] How captured PDT feeds `timestampOffset` — the A/V-sync decision.**
|
||||
The primary concern is keeping demuxed audio and video synchronized
|
||||
when each track's `timestampOffset` is set independently; a shared
|
||||
presentation-level anchor (PDT) rather than per-track zeroing is the
|
||||
likely shape. Deliberately downstream of this model work; couples back
|
||||
to discontinuity handling when that lands.
|
||||
- **Parser output shape — resurrect `MediaPlaylistInfo` or hang fields on
|
||||
`Track`?** The snapshot/merge model wants a faithful per-fetch
|
||||
representation; today's parser merges into `Track`. This is the parser
|
||||
refactor's load-bearing call.
|
||||
- **Turnover `startTime` recovery.** When no overlap exists (long
|
||||
background/stall), recover absolute `startTime` from PDT, or accept a
|
||||
target-duration-based estimate?
|
||||
- **`streamType` first-fetch ambiguity.** Accept untagged-VOD → `live`,
|
||||
or introduce `unknown`?
|
||||
- **DVR / event windowing.** `dvr-event-stream-support` reintroduces a
|
||||
"growing but not complete" notion (finite-but-increasing duration)
|
||||
that duration-finiteness alone can't express, plus the variant-specific
|
||||
`seekableStart` producer. Out of scope here; flagged so the foundation
|
||||
doesn't assume sliding-window everywhere.
|
||||
|
||||
---
|
||||
|
||||
## See also
|
||||
|
||||
- [presentation-modeling.md](./presentation-modeling.md) — companion;
|
||||
the static/VOD snapshot, the `Presentation` types, and the parser
|
||||
interface this doc extends. Required reading.
|
||||
- [features/live-stream-support.md](./features/live-stream-support.md) —
|
||||
the engine behaviors (reload loop, sliding window, termination) built
|
||||
on this model; the implementation target.
|
||||
- [features/dvr-event-stream-support.md](./features/dvr-event-stream-support.md),
|
||||
[features/ll-hls-support.md](./features/ll-hls-support.md) — downstream
|
||||
variants that extend [1]/[5] and the reload loop.
|
||||
- [features/mse-mms-pipeline.md](./features/mse-mms-pipeline.md) —
|
||||
`config.resolveDuration → Infinity` is the [2b] surface; `endOfStream`
|
||||
becomes reachable once completeness commits.
|
||||
- [features/non-zero-pts-support.md](./features/non-zero-pts-support.md) —
|
||||
the [4] `timestampOffset` / A/V-sync concern.
|
||||
- [features/clusters.md § Manifest reload loop](./features/clusters.md#manifest-reload-loop)
|
||||
— the cluster this work anchors.
|
||||
- `packages/spf/src/media/types/index.ts`,
|
||||
`packages/spf/src/media/hls/parse-media-playlist.ts` — the types and
|
||||
parser the refactor touches.
|
||||
- [RFC 8216bis](https://datatracker.ietf.org/doc/html/draft-pantos-hls-rfc8216bis),
|
||||
[common-media-library](https://github.com/AcademySoftwareFoundation/common-media-library)
|
||||
(CMAF-HAM), media-ui-extensions proposals `0010-stream-type`,
|
||||
`0007-live-edge`, `0000-target-live-window`.
|
||||
</content>
|
||||
</invoke>
|
||||
@@ -559,6 +559,11 @@ file format-specific architectural docs as siblings to this one
|
||||
presentation
|
||||
- [text-track-architecture.md](./text-track-architecture.md) — peer
|
||||
architectural doc (text-track-specific deep-dive)
|
||||
- [live-presentation-modeling.md](./live-presentation-modeling.md) —
|
||||
companion that extends this model for presentations that *change over
|
||||
time* (live / DVR / event): the snapshot/merge segment model,
|
||||
`streamType`, refetch policy as a distinct category, and the PDT sync
|
||||
anchor. This doc covers the one-shot / VOD snapshot
|
||||
- [conventions/signals.md](./conventions/signals.md) — pipeline-pattern
|
||||
multi-writer slot convention (`state.presentation` is the canonical
|
||||
worked example)
|
||||
|
||||
Reference in New Issue
Block a user