feat(spf): recover end-of-stream stall on skewed A/V (spike)

Make a complete VOD reach native `ended` (and loop) reliably even when audio
and video tracks end a few ms apart, via two narrow changes in the behaviors
that already own end-of-playback:

1. `end-of-stream` reactor — add `LAST_SEGMENT_REACHED_SLACK` (0.5s) to the
   "playhead reached the last segment" gate. A tiny final segment (e.g. Apple's
   ~44ms last segment) starts at the buffered end and Chrome freezes the
   playhead ~50-70ms short of it, so a strict `currentTime >= lastSegStart`
   never opens -> `endOfStream()` never fires -> MediaSource stays `'open'` ->
   frozen. The slack fires EOS just before the freeze.

2. `recover-end-stall` (new DOM behavior) — on `waiting`, if the MediaSource is
   `'ended'`, duration is finite, and the playhead is within
   `endStallNudgeWindow` (0.2s, config) of the reachable buffered end, nudge
   `currentTime = duration` to force native `ended`. Event-driven, no poll
   (`waiting` fires at ~0ms). Inert for live and clean-ending streams.

Composed in both engines. ADR:
internal/decisions/end-of-stream-av-skew-recovery.md

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Christian Pillsbury
2026-07-13 14:37:46 -07:00
co-authored by Claude Opus 4.8
parent 6d26e083a5
commit c3e137cf39
6 changed files with 308 additions and 6 deletions
@@ -0,0 +1,102 @@
---
status: decided
date: 2026-07-13
---
# End-of-Stream Recovery for Skewed / Tiny-Final-Segment A/V
## Decision
Make a complete VOD reach native `ended` (and therefore loop) reliably, even when the
audio and video tracks don't end at exactly the same time, via **two narrow changes** in
the two behaviors that already own the end of playback:
1. **`end-of-stream` reactor** — add slack to the "playhead reached the last segment"
gate: fire `endOfStream()` once `currentTime >= lastSegStart LAST_SEGMENT_REACHED_SLACK`
(0.5 s) rather than `>= lastSegStart` exactly.
2. **`recover-end-stall` behavior** (new, DOM) — on the `waiting` event, if the
MediaSource is `'ended'`, the stream is finite, and the playhead is within
`endStallNudgeWindow` (default 0.2 s) of the reachable buffered end, set
`currentTime = duration` to force native `ended`.
Both are event-driven; neither polls. `recover-end-stall` reads only `mediaElement` +
`mediaSource`. The window/slack are config-tunable.
## Context
Chrome hangs at the end of the Apple `bipbop_adv_example_hevc` VOD (a source with a ~44 ms
A/V PTS skew): the playhead freezes a few frames short of the end and native `ended` never
fires, so playback stalls and loop never re-triggers. Investigation (measured in the
`spf-non-zero-pts` sandbox) found **two** compounding causes:
- **A tiny final segment deadlocks the EOS reactor.** Apple's last video segment is ~44 ms
and starts right at the buffered end (`lastSegStart ≈ 600.0`, buffered end ≈ `600.044`).
Chrome paces `currentTime` off the audio clock and freezes the playhead ~5070 ms short of
the buffered end — i.e. *below* `lastSegStart`. The reactor's `currentTime >= lastSegStart`
gate never opens → `endOfStream()` is never called → the MediaSource stays `'open'` → the
browser keeps the playhead frozen waiting for data/EOS that never comes. A seek past
`lastSegStart` (e.g. to `duration`) breaks the deadlock.
- **Even once `endOfStream()` fires, the playhead still freezes short of `duration`.** With
MS `'ended'`, Chrome still stops ~5070 ms short (the audio clock can't advance past the
shorter track's end), so `currentTime` never reaches `duration` and `ended` never fires.
Nudging `currentTime = duration` fires it immediately.
Empirical findings that shaped the design:
- The freeze gap (playhead-stop → reachable buffered end) is stably ~5070 ms (measured 52,
58, 62, 71 ms), with ~20 ms non-deterministic jitter in the exact stop position. The
reachable end (`buffered.end(last)` = `min(video, audio)`, the audio/pacing track) is
stable; `duration` (= `max`, the video end) varies run-to-run with ABR rendition.
- `waiting` fires at the freeze with ~0 ms latency (measured 3.2 ms vs a rAF sampler), so a
poll would only add latency — no reason to poll for this.
- A MediaSource reaches `'ended'` **only** via `endOfStream()` (MSE spec); the browser never
does it spontaneously. The manual nudge appeared to "end without EOS" only because the
seek re-triggered our own reactor, which then called `endOfStream()`.
- hls.js handles the same class of stall in its `GapController`: it does **not** trim buffers;
it detects the near-end stall (MS `'ended'` + within 1 s of the edge) and, in a player-layer
event, declares ended. We adapt this to SPF's native-`ended` shape by nudging to `duration`.
## Alternatives Considered
- **Trim the longer track's tail to align A/V ends, and set `duration` to the min.**
Prototyped and rejected: trimming *video* by PTS can orphan B-frames (removing frames a
displayable frame depends on), and it doesn't even fix the stall (the freeze persists), and
the mismatch is frame-granular so exact alignment is impossible. It also needs a per-cycle
trim target + re-entry bookkeeping. Adds risk for no benefit once the nudge is in place.
- **Poll for the stall (hls.js's `GapController` 100 ms tick).** Unnecessary here: `waiting`
fires at the freeze with ~0 latency; a poll adds its interval + a stall threshold
(hls.js waits up to `detectStallWithCurrentTimeMs = 1250 ms`).
- **hls.js's "within ~1 s of `duration`" window.** Looser than needed. We gate on proximity to
the reachable buffered end (`buffered.end(last)`), which is tied to real buffered content and
~5× tighter, sized just above the measured freeze gap.
- **A single fix in one behavior.** Neither alone suffices: without the reactor slack the MS
never reaches `'ended'`; without the nudge the playhead never reaches `duration`.
## Rationale
- **Root + residual, in their owners.** The reactor slack fixes the *root* (EOS never firing);
`recover-end-stall` handles the *residual* audio-clock freeze. Each change lives in the
behavior whose concern it is (MediaSource finalization vs. playhead recovery), mirroring
hls.js's separation.
- **Ordering removes the race.** With slack, `endOfStream()` fires as `currentTime` crosses
`lastSegStart slack`*before* the freeze — so by the time `waiting` fires the MS is
already `'ended'` and `recover-end-stall`'s gate is satisfied.
- **Grounded constants.** `LAST_SEGMENT_REACHED_SLACK` (0.5 s) and `endStallNudgeWindow`
(0.2 s) both comfortably exceed the measured ~5071 ms freeze gap. Too-tight risks *missing*
the stall (a permanent hang — worse than the bug), so both bias generous and are tunable.
- **Inert when not needed.** `recover-end-stall` no-ops for live (MS never `'ended'` while
growing) and for streams that end cleanly (no `waiting`); the reactor slack only shifts EOS
slightly earlier near the true end (harmless — the last segment is already appended).
## Scope
General EOS robustness for any skewed / short-final-segment A/V — not specific to the
non-zero-PTS relocation work it was discovered alongside. Composed in `engine.ts` and
`engine-audio-only.ts`.
## Follow-ups
- The nudge is a single jump to `duration`; if a stream ever freezes *further* short than
`endStallNudgeWindow`, add a bounded retry rather than widening the default blindly.
- `recover-end-stall` currently keys off `waiting`; if a pathological stream froze without a
`waiting`, a one-shot re-check on the MS→`'ended'` transition is the fallback.
@@ -61,11 +61,12 @@
*
* # currentTime gate
*
* `currentTime` must have reached at least one active track's last
* segment startTime. Prevents `'eos-ready'` entry when a back-buffer
* `remove()` / `appendBuffer()` briefly re-opens the MediaSource while
* the user is mid-stream. HLS rendition time-alignment means any active
* track works as the reference.
* `currentTime` must have reached (within {@link LAST_SEGMENT_REACHED_SLACK})
* at least one active track's last segment startTime. Prevents `'eos-ready'`
* entry when a back-buffer `remove()` / `appendBuffer()` briefly re-opens the
* MediaSource while the user is mid-stream. HLS rendition time-alignment means
* any active track works as the reference. The slack absorbs the near-end
* playhead freeze (see the constant) so a tiny final segment doesn't deadlock.
*
* # MS readyState — local subscription
*
@@ -119,6 +120,18 @@ export interface EndOfStreamContext {
type EndOfStreamFsmState = 'preconditions-unmet' | 'eos-ready';
/**
* Slack (seconds) on the "playhead has reached the last segment" gate. A tiny final
* segment (e.g. Apple's ~44ms last segment) starts right at the buffered end, and the
* browser freezes the playhead ~5070ms short of that end (its render horizon), so a
* strict `currentTime >= lastSegStart` would never open — deadlocking `endOfStream`
* (the MediaSource stays `'open'`, so the browser keeps the playhead frozen waiting for
* data/EOS that never comes). This slack lets a playhead stalled just short of the final
* segment still finalize. Firing slightly early is harmless: the last segment is already
* appended (the gate above), so no more data is expected.
*/
const LAST_SEGMENT_REACHED_SLACK = 0.5;
function deriveState(
presentation: MaybeResolvedPresentation | undefined,
mediaSource: MediaSource | undefined,
@@ -164,7 +177,8 @@ function deriveState(
}
}
if (lastSegStart !== undefined && (currentTime ?? 0) < lastSegStart) {
// Slack absorbs the near-end playhead freeze so a tiny final segment doesn't deadlock.
if (lastSegStart !== undefined && (currentTime ?? 0) < lastSegStart - LAST_SEGMENT_REACHED_SLACK) {
return 'preconditions-unmet';
}
@@ -0,0 +1,121 @@
/**
* Recover the end-of-stream stall that Chrome exhibits on skewed A/V. After
* `endOfStream`, when the audio and video tracks end a few ms apart (e.g. a source
* with an A/V PTS skew), Chrome's audio-clock-paced playback freezes the playhead
* ~5070ms short of the reachable buffered end and never fires `ended` so playback
* hangs at the very end and loop never re-triggers. This behavior watches for the
* `waiting` event that fires at that freeze and, when the MediaSource is `ended` and
* the playhead sits at the reachable buffered end, nudges `currentTime` to `duration`
* to force the native `ended`.
*
* **Event-driven, no poll.** `waiting` fires at the instant the playhead stalls
* (measured ~0ms latency), so there's nothing to gain from polling and polling would
* add its interval + a stall threshold before reacting.
*
* **Proximity to the *intersection* buffered end** is the discriminator.
* `mediaElement.buffered.end(last)` is already `min(videoEnd, audioEnd)` the furthest
* point playback can reach and once `endOfStream` is signalled that's the true content
* end. Requiring the playhead within `endStallNudgeWindow` of it distinguishes the real
* end-of-stream freeze from a mid-stream buffer-hole stall (which sits far from the
* buffered end), so we never skip content. The window must exceed the freeze gap; too
* small would miss the stall (a permanent hang), so the default is generous relative to
* the measured gap and is config-tunable for empirical tuning.
*
* Inert where it shouldn't act: live (the MediaSource never reaches `ended` while the
* window grows; `duration` is `Infinity`) and streams that end cleanly (no `waiting`).
*
* See `internal/decisions/end-of-stream-av-skew-recovery.md`.
*/
import { listen } from '@videojs/utils/dom';
import { defineBehavior } from '../../../core/composition/create-composition';
import { effect } from '../../../core/signals/effect';
import type { ReadonlySignal } from '../../../core/signals/primitives';
export interface RecoverEndStallContext {
mediaElement?: HTMLMediaElement | undefined;
mediaSource?: MediaSource | undefined;
}
export interface RecoverEndStallConfig {
/**
* How close (seconds) the playhead must be to the reachable buffered end for a
* `waiting` to count as the end-of-stream freeze. Must exceed the audio-clock freeze
* gap (~5070ms measured on Chrome); too small risks missing the stall (a permanent
* hang), so keep a margin. Default {@link DEFAULT_END_STALL_NUDGE_WINDOW}.
*/
endStallNudgeWindow?: number;
}
/** ~2.8× the measured max freeze gap (71ms) — tight, but with headroom against a miss. */
export const DEFAULT_END_STALL_NUDGE_WINDOW = 0.2;
/**
* Whether a `waiting` should be forced to `ended`: the MediaSource is `ended`, the
* stream is finite (not live), playback is active (not paused/seeking/already-ended),
* and the playhead sits within `nudgeWindow` of the reachable buffered end (so it's the
* true end, not a mid-stream buffer hole). Pure the behavior supplies the live values.
*/
export function shouldForceEnded(
input: {
msEnded: boolean;
durationFinite: boolean;
paused: boolean;
seeking: boolean;
ended: boolean;
currentTime: number;
bufferedEnd: number | undefined;
},
nudgeWindow: number
): boolean {
const { msEnded, durationFinite, paused, seeking, ended, currentTime, bufferedEnd } = input;
if (!msEnded || !durationFinite || paused || seeking || ended || bufferedEnd === undefined) {
return false;
}
const gap = bufferedEnd - currentTime;
return gap >= 0 && gap < nudgeWindow;
}
function recoverEndStallSetup({
context,
config,
}: {
context: {
mediaElement: ReadonlySignal<RecoverEndStallContext['mediaElement']>;
mediaSource: ReadonlySignal<RecoverEndStallContext['mediaSource']>;
};
config?: RecoverEndStallConfig;
}): () => void {
const nudgeWindow = config?.endStallNudgeWindow ?? DEFAULT_END_STALL_NUDGE_WINDOW;
return effect(() => {
const mediaElement = context.mediaElement.get();
if (!mediaElement) return;
const onWaiting = () => {
const { buffered } = mediaElement;
const forceEnded = shouldForceEnded(
{
msEnded: context.mediaSource.get()?.readyState === 'ended',
durationFinite: Number.isFinite(mediaElement.duration),
paused: mediaElement.paused,
seeking: mediaElement.seeking,
ended: mediaElement.ended,
currentTime: mediaElement.currentTime,
bufferedEnd: buffered.length > 0 ? buffered.end(buffered.length - 1) : undefined,
},
nudgeWindow
);
// Nudge to `duration` → native `ended` (the seeking/ended guards above prevent a
// re-fire while the nudge-seek is in flight, so no latch is needed).
if (forceEnded) mediaElement.currentTime = mediaElement.duration;
};
return listen(mediaElement, 'waiting', onWaiting);
});
}
export const recoverEndStall = defineBehavior({
stateKeys: [] as const,
contextKeys: ['mediaElement', 'mediaSource'] as const,
setup: recoverEndStallSetup,
});
@@ -0,0 +1,50 @@
import { describe, expect, it } from 'vitest';
import { shouldForceEnded } from '../recover-end-stall';
// At the end-of-stream freeze: MediaSource ended, finite duration, actively playing,
// and the playhead a few frames short of the reachable (intersection) buffered end.
const atEndStall = {
msEnded: true,
durationFinite: true,
paused: false,
seeking: false,
ended: false,
currentTime: 599.95,
bufferedEnd: 600.0,
} as const;
const WINDOW = 0.2;
describe('shouldForceEnded', () => {
it('fires at the end-of-stream freeze (playhead within the window of the buffered end)', () => {
expect(shouldForceEnded(atEndStall, WINDOW)).toBe(true); // gap 0.05 < 0.2
});
it('does not fire mid-content (playhead far from the buffered end)', () => {
expect(shouldForceEnded({ ...atEndStall, currentTime: 300, bufferedEnd: 600 }, WINDOW)).toBe(false);
});
it('does not fire until endOfStream is signalled', () => {
expect(shouldForceEnded({ ...atEndStall, msEnded: false }, WINDOW)).toBe(false);
});
it('does not fire for live (non-finite duration)', () => {
expect(shouldForceEnded({ ...atEndStall, durationFinite: false }, WINDOW)).toBe(false);
});
it('does not fire while paused, seeking, or already ended', () => {
expect(shouldForceEnded({ ...atEndStall, paused: true }, WINDOW)).toBe(false);
expect(shouldForceEnded({ ...atEndStall, seeking: true }, WINDOW)).toBe(false);
expect(shouldForceEnded({ ...atEndStall, ended: true }, WINDOW)).toBe(false);
});
it('does not fire with no buffered ranges', () => {
expect(shouldForceEnded({ ...atEndStall, bufferedEnd: undefined }, WINDOW)).toBe(false);
});
it('respects the configured window', () => {
const gap015 = { ...atEndStall, currentTime: 599.85, bufferedEnd: 600.0 }; // gap 0.15
expect(shouldForceEnded(gap015, 0.2)).toBe(true);
expect(shouldForceEnded(gap015, 0.1)).toBe(false);
});
});
@@ -21,6 +21,7 @@ import {
import { deriveCdnPriority } from '../../behaviors/derive-cdn-priority';
import { endOfStream } from '../../behaviors/dom/end-of-stream';
import { loadAudioSegments } from '../../behaviors/dom/load-segments';
import { recoverEndStall } from '../../behaviors/dom/recover-end-stall';
import { relocationPipelinesFor } from '../../behaviors/dom/relocation-steps';
import { setupAudioBufferActors } from '../../behaviors/dom/setup-buffer-actors';
import { setupMediaSource } from '../../behaviors/dom/setup-mediasource';
@@ -250,6 +251,9 @@ export function createHlsAudioOnlyEngine(
// `mediaSource.sourceBuffers` aggregately — composes unchanged with
// only audio in scope.
endOfStream,
// Force native `ended` if Chrome freezes the playhead short of the buffered end
// after `endOfStream`. Inert for a clean-ending single-track source.
recoverEndStall,
// Adapter signal callback.
shareSignals,
@@ -43,6 +43,7 @@ import {
import { deriveCdnPriority } from '../../behaviors/derive-cdn-priority';
import { endOfStream } from '../../behaviors/dom/end-of-stream';
import { loadAudioSegments, loadTextTrackSegments, loadVideoSegments } from '../../behaviors/dom/load-segments';
import { recoverEndStall } from '../../behaviors/dom/recover-end-stall';
import { relocatingTextPipelines, relocationPipelinesFor } from '../../behaviors/dom/relocation-steps';
import { seekToLiveEdge } from '../../behaviors/dom/seek-to-live-edge';
import { setupAudioBufferActors, setupVideoBufferActors } from '../../behaviors/dom/setup-buffer-actors';
@@ -297,6 +298,13 @@ export interface SimpleHlsEngineConfig extends ShareSignalsConfig<SimpleHlsEngin
* `internal/design/spf/presentation-timeline-model.md`.
*/
deriveStartMediaTime?: DeriveStartMediaTime;
/**
* Proximity window (seconds) for the `recoverEndStall` behavior how close the
* playhead must be to the reachable buffered end for a `waiting` to be treated as the
* end-of-stream freeze and nudged to `ended`. Defaults to `0.2`. See
* `behaviors/dom/recover-end-stall`.
*/
endStallNudgeWindow?: number;
}
// ============================================================================
@@ -473,6 +481,9 @@ export function createSimpleHlsEngine(
// End of stream coordination
endOfStream,
// Force native `ended` when Chrome freezes the playhead a few frames short of a
// skewed-A/V end after `endOfStream` (audio-clock stall). Inert otherwise.
recoverEndStall,
// Text tracks
syncTextTracks,