mirror of
https://github.com/zoriya/v10.git
synced 2026-08-09 07:37:48 +00:00
feat(spf): recover end-of-stream stall on skewed A/V (spike)
Make a complete VOD reach native `ended` (and loop) reliably even when audio and video tracks end a few ms apart, via two narrow changes in the behaviors that already own end-of-playback: 1. `end-of-stream` reactor — add `LAST_SEGMENT_REACHED_SLACK` (0.5s) to the "playhead reached the last segment" gate. A tiny final segment (e.g. Apple's ~44ms last segment) starts at the buffered end and Chrome freezes the playhead ~50-70ms short of it, so a strict `currentTime >= lastSegStart` never opens -> `endOfStream()` never fires -> MediaSource stays `'open'` -> frozen. The slack fires EOS just before the freeze. 2. `recover-end-stall` (new DOM behavior) — on `waiting`, if the MediaSource is `'ended'`, duration is finite, and the playhead is within `endStallNudgeWindow` (0.2s, config) of the reachable buffered end, nudge `currentTime = duration` to force native `ended`. Event-driven, no poll (`waiting` fires at ~0ms). Inert for live and clean-ending streams. Composed in both engines. ADR: internal/decisions/end-of-stream-av-skew-recovery.md Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Opus 4.8
parent
6d26e083a5
commit
c3e137cf39
@@ -0,0 +1,102 @@
|
||||
---
|
||||
status: decided
|
||||
date: 2026-07-13
|
||||
---
|
||||
|
||||
# End-of-Stream Recovery for Skewed / Tiny-Final-Segment A/V
|
||||
|
||||
## Decision
|
||||
|
||||
Make a complete VOD reach native `ended` (and therefore loop) reliably, even when the
|
||||
audio and video tracks don't end at exactly the same time, via **two narrow changes** in
|
||||
the two behaviors that already own the end of playback:
|
||||
|
||||
1. **`end-of-stream` reactor** — add slack to the "playhead reached the last segment"
|
||||
gate: fire `endOfStream()` once `currentTime >= lastSegStart − LAST_SEGMENT_REACHED_SLACK`
|
||||
(0.5 s) rather than `>= lastSegStart` exactly.
|
||||
2. **`recover-end-stall` behavior** (new, DOM) — on the `waiting` event, if the
|
||||
MediaSource is `'ended'`, the stream is finite, and the playhead is within
|
||||
`endStallNudgeWindow` (default 0.2 s) of the reachable buffered end, set
|
||||
`currentTime = duration` to force native `ended`.
|
||||
|
||||
Both are event-driven; neither polls. `recover-end-stall` reads only `mediaElement` +
|
||||
`mediaSource`. The window/slack are config-tunable.
|
||||
|
||||
## Context
|
||||
|
||||
Chrome hangs at the end of the Apple `bipbop_adv_example_hevc` VOD (a source with a ~44 ms
|
||||
A/V PTS skew): the playhead freezes a few frames short of the end and native `ended` never
|
||||
fires, so playback stalls and loop never re-triggers. Investigation (measured in the
|
||||
`spf-non-zero-pts` sandbox) found **two** compounding causes:
|
||||
|
||||
- **A tiny final segment deadlocks the EOS reactor.** Apple's last video segment is ~44 ms
|
||||
and starts right at the buffered end (`lastSegStart ≈ 600.0`, buffered end ≈ `600.044`).
|
||||
Chrome paces `currentTime` off the audio clock and freezes the playhead ~50–70 ms short of
|
||||
the buffered end — i.e. *below* `lastSegStart`. The reactor's `currentTime >= lastSegStart`
|
||||
gate never opens → `endOfStream()` is never called → the MediaSource stays `'open'` → the
|
||||
browser keeps the playhead frozen waiting for data/EOS that never comes. A seek past
|
||||
`lastSegStart` (e.g. to `duration`) breaks the deadlock.
|
||||
- **Even once `endOfStream()` fires, the playhead still freezes short of `duration`.** With
|
||||
MS `'ended'`, Chrome still stops ~50–70 ms short (the audio clock can't advance past the
|
||||
shorter track's end), so `currentTime` never reaches `duration` and `ended` never fires.
|
||||
Nudging `currentTime = duration` fires it immediately.
|
||||
|
||||
Empirical findings that shaped the design:
|
||||
|
||||
- The freeze gap (playhead-stop → reachable buffered end) is stably ~50–70 ms (measured 52,
|
||||
58, 62, 71 ms), with ~20 ms non-deterministic jitter in the exact stop position. The
|
||||
reachable end (`buffered.end(last)` = `min(video, audio)`, the audio/pacing track) is
|
||||
stable; `duration` (= `max`, the video end) varies run-to-run with ABR rendition.
|
||||
- `waiting` fires at the freeze with ~0 ms latency (measured −3.2 ms vs a rAF sampler), so a
|
||||
poll would only add latency — no reason to poll for this.
|
||||
- A MediaSource reaches `'ended'` **only** via `endOfStream()` (MSE spec); the browser never
|
||||
does it spontaneously. The manual nudge appeared to "end without EOS" only because the
|
||||
seek re-triggered our own reactor, which then called `endOfStream()`.
|
||||
- hls.js handles the same class of stall in its `GapController`: it does **not** trim buffers;
|
||||
it detects the near-end stall (MS `'ended'` + within 1 s of the edge) and, in a player-layer
|
||||
event, declares ended. We adapt this to SPF's native-`ended` shape by nudging to `duration`.
|
||||
|
||||
## Alternatives Considered
|
||||
|
||||
- **Trim the longer track's tail to align A/V ends, and set `duration` to the min.**
|
||||
Prototyped and rejected: trimming *video* by PTS can orphan B-frames (removing frames a
|
||||
displayable frame depends on), and it doesn't even fix the stall (the freeze persists), and
|
||||
the mismatch is frame-granular so exact alignment is impossible. It also needs a per-cycle
|
||||
trim target + re-entry bookkeeping. Adds risk for no benefit once the nudge is in place.
|
||||
- **Poll for the stall (hls.js's `GapController` 100 ms tick).** Unnecessary here: `waiting`
|
||||
fires at the freeze with ~0 latency; a poll adds its interval + a stall threshold
|
||||
(hls.js waits up to `detectStallWithCurrentTimeMs = 1250 ms`).
|
||||
- **hls.js's "within ~1 s of `duration`" window.** Looser than needed. We gate on proximity to
|
||||
the reachable buffered end (`buffered.end(last)`), which is tied to real buffered content and
|
||||
~5× tighter, sized just above the measured freeze gap.
|
||||
- **A single fix in one behavior.** Neither alone suffices: without the reactor slack the MS
|
||||
never reaches `'ended'`; without the nudge the playhead never reaches `duration`.
|
||||
|
||||
## Rationale
|
||||
|
||||
- **Root + residual, in their owners.** The reactor slack fixes the *root* (EOS never firing);
|
||||
`recover-end-stall` handles the *residual* audio-clock freeze. Each change lives in the
|
||||
behavior whose concern it is (MediaSource finalization vs. playhead recovery), mirroring
|
||||
hls.js's separation.
|
||||
- **Ordering removes the race.** With slack, `endOfStream()` fires as `currentTime` crosses
|
||||
`lastSegStart − slack` — *before* the freeze — so by the time `waiting` fires the MS is
|
||||
already `'ended'` and `recover-end-stall`'s gate is satisfied.
|
||||
- **Grounded constants.** `LAST_SEGMENT_REACHED_SLACK` (0.5 s) and `endStallNudgeWindow`
|
||||
(0.2 s) both comfortably exceed the measured ~50–71 ms freeze gap. Too-tight risks *missing*
|
||||
the stall (a permanent hang — worse than the bug), so both bias generous and are tunable.
|
||||
- **Inert when not needed.** `recover-end-stall` no-ops for live (MS never `'ended'` while
|
||||
growing) and for streams that end cleanly (no `waiting`); the reactor slack only shifts EOS
|
||||
slightly earlier near the true end (harmless — the last segment is already appended).
|
||||
|
||||
## Scope
|
||||
|
||||
General EOS robustness for any skewed / short-final-segment A/V — not specific to the
|
||||
non-zero-PTS relocation work it was discovered alongside. Composed in `engine.ts` and
|
||||
`engine-audio-only.ts`.
|
||||
|
||||
## Follow-ups
|
||||
|
||||
- The nudge is a single jump to `duration`; if a stream ever freezes *further* short than
|
||||
`endStallNudgeWindow`, add a bounded retry rather than widening the default blindly.
|
||||
- `recover-end-stall` currently keys off `waiting`; if a pathological stream froze without a
|
||||
`waiting`, a one-shot re-check on the MS→`'ended'` transition is the fallback.
|
||||
@@ -61,11 +61,12 @@
|
||||
*
|
||||
* # currentTime gate
|
||||
*
|
||||
* `currentTime` must have reached at least one active track's last
|
||||
* segment startTime. Prevents `'eos-ready'` entry when a back-buffer
|
||||
* `remove()` / `appendBuffer()` briefly re-opens the MediaSource while
|
||||
* the user is mid-stream. HLS rendition time-alignment means any active
|
||||
* track works as the reference.
|
||||
* `currentTime` must have reached (within {@link LAST_SEGMENT_REACHED_SLACK})
|
||||
* at least one active track's last segment startTime. Prevents `'eos-ready'`
|
||||
* entry when a back-buffer `remove()` / `appendBuffer()` briefly re-opens the
|
||||
* MediaSource while the user is mid-stream. HLS rendition time-alignment means
|
||||
* any active track works as the reference. The slack absorbs the near-end
|
||||
* playhead freeze (see the constant) so a tiny final segment doesn't deadlock.
|
||||
*
|
||||
* # MS readyState — local subscription
|
||||
*
|
||||
@@ -119,6 +120,18 @@ export interface EndOfStreamContext {
|
||||
|
||||
type EndOfStreamFsmState = 'preconditions-unmet' | 'eos-ready';
|
||||
|
||||
/**
|
||||
* Slack (seconds) on the "playhead has reached the last segment" gate. A tiny final
|
||||
* segment (e.g. Apple's ~44ms last segment) starts right at the buffered end, and the
|
||||
* browser freezes the playhead ~50–70ms short of that end (its render horizon), so a
|
||||
* strict `currentTime >= lastSegStart` would never open — deadlocking `endOfStream`
|
||||
* (the MediaSource stays `'open'`, so the browser keeps the playhead frozen waiting for
|
||||
* data/EOS that never comes). This slack lets a playhead stalled just short of the final
|
||||
* segment still finalize. Firing slightly early is harmless: the last segment is already
|
||||
* appended (the gate above), so no more data is expected.
|
||||
*/
|
||||
const LAST_SEGMENT_REACHED_SLACK = 0.5;
|
||||
|
||||
function deriveState(
|
||||
presentation: MaybeResolvedPresentation | undefined,
|
||||
mediaSource: MediaSource | undefined,
|
||||
@@ -164,7 +177,8 @@ function deriveState(
|
||||
}
|
||||
}
|
||||
|
||||
if (lastSegStart !== undefined && (currentTime ?? 0) < lastSegStart) {
|
||||
// Slack absorbs the near-end playhead freeze so a tiny final segment doesn't deadlock.
|
||||
if (lastSegStart !== undefined && (currentTime ?? 0) < lastSegStart - LAST_SEGMENT_REACHED_SLACK) {
|
||||
return 'preconditions-unmet';
|
||||
}
|
||||
|
||||
|
||||
@@ -0,0 +1,121 @@
|
||||
/**
|
||||
* Recover the end-of-stream stall that Chrome exhibits on skewed A/V. After
|
||||
* `endOfStream`, when the audio and video tracks end a few ms apart (e.g. a source
|
||||
* with an A/V PTS skew), Chrome's audio-clock-paced playback freezes the playhead
|
||||
* ~50–70ms short of the reachable buffered end and never fires `ended` — so playback
|
||||
* hangs at the very end and loop never re-triggers. This behavior watches for the
|
||||
* `waiting` event that fires at that freeze and, when the MediaSource is `ended` and
|
||||
* the playhead sits at the reachable buffered end, nudges `currentTime` to `duration`
|
||||
* to force the native `ended`.
|
||||
*
|
||||
* **Event-driven, no poll.** `waiting` fires at the instant the playhead stalls
|
||||
* (measured ~0ms latency), so there's nothing to gain from polling — and polling would
|
||||
* add its interval + a stall threshold before reacting.
|
||||
*
|
||||
* **Proximity to the *intersection* buffered end** is the discriminator.
|
||||
* `mediaElement.buffered.end(last)` is already `min(videoEnd, audioEnd)` — the furthest
|
||||
* point playback can reach — and once `endOfStream` is signalled that's the true content
|
||||
* end. Requiring the playhead within `endStallNudgeWindow` of it distinguishes the real
|
||||
* end-of-stream freeze from a mid-stream buffer-hole stall (which sits far from the
|
||||
* buffered end), so we never skip content. The window must exceed the freeze gap; too
|
||||
* small would miss the stall (a permanent hang), so the default is generous relative to
|
||||
* the measured gap and is config-tunable for empirical tuning.
|
||||
*
|
||||
* Inert where it shouldn't act: live (the MediaSource never reaches `ended` while the
|
||||
* window grows; `duration` is `Infinity`) and streams that end cleanly (no `waiting`).
|
||||
*
|
||||
* See `internal/decisions/end-of-stream-av-skew-recovery.md`.
|
||||
*/
|
||||
import { listen } from '@videojs/utils/dom';
|
||||
import { defineBehavior } from '../../../core/composition/create-composition';
|
||||
import { effect } from '../../../core/signals/effect';
|
||||
import type { ReadonlySignal } from '../../../core/signals/primitives';
|
||||
|
||||
export interface RecoverEndStallContext {
|
||||
mediaElement?: HTMLMediaElement | undefined;
|
||||
mediaSource?: MediaSource | undefined;
|
||||
}
|
||||
|
||||
export interface RecoverEndStallConfig {
|
||||
/**
|
||||
* How close (seconds) the playhead must be to the reachable buffered end for a
|
||||
* `waiting` to count as the end-of-stream freeze. Must exceed the audio-clock freeze
|
||||
* gap (~50–70ms measured on Chrome); too small risks missing the stall (a permanent
|
||||
* hang), so keep a margin. Default {@link DEFAULT_END_STALL_NUDGE_WINDOW}.
|
||||
*/
|
||||
endStallNudgeWindow?: number;
|
||||
}
|
||||
|
||||
/** ~2.8× the measured max freeze gap (71ms) — tight, but with headroom against a miss. */
|
||||
export const DEFAULT_END_STALL_NUDGE_WINDOW = 0.2;
|
||||
|
||||
/**
|
||||
* Whether a `waiting` should be forced to `ended`: the MediaSource is `ended`, the
|
||||
* stream is finite (not live), playback is active (not paused/seeking/already-ended),
|
||||
* and the playhead sits within `nudgeWindow` of the reachable buffered end (so it's the
|
||||
* true end, not a mid-stream buffer hole). Pure — the behavior supplies the live values.
|
||||
*/
|
||||
export function shouldForceEnded(
|
||||
input: {
|
||||
msEnded: boolean;
|
||||
durationFinite: boolean;
|
||||
paused: boolean;
|
||||
seeking: boolean;
|
||||
ended: boolean;
|
||||
currentTime: number;
|
||||
bufferedEnd: number | undefined;
|
||||
},
|
||||
nudgeWindow: number
|
||||
): boolean {
|
||||
const { msEnded, durationFinite, paused, seeking, ended, currentTime, bufferedEnd } = input;
|
||||
if (!msEnded || !durationFinite || paused || seeking || ended || bufferedEnd === undefined) {
|
||||
return false;
|
||||
}
|
||||
const gap = bufferedEnd - currentTime;
|
||||
return gap >= 0 && gap < nudgeWindow;
|
||||
}
|
||||
|
||||
function recoverEndStallSetup({
|
||||
context,
|
||||
config,
|
||||
}: {
|
||||
context: {
|
||||
mediaElement: ReadonlySignal<RecoverEndStallContext['mediaElement']>;
|
||||
mediaSource: ReadonlySignal<RecoverEndStallContext['mediaSource']>;
|
||||
};
|
||||
config?: RecoverEndStallConfig;
|
||||
}): () => void {
|
||||
const nudgeWindow = config?.endStallNudgeWindow ?? DEFAULT_END_STALL_NUDGE_WINDOW;
|
||||
|
||||
return effect(() => {
|
||||
const mediaElement = context.mediaElement.get();
|
||||
if (!mediaElement) return;
|
||||
|
||||
const onWaiting = () => {
|
||||
const { buffered } = mediaElement;
|
||||
const forceEnded = shouldForceEnded(
|
||||
{
|
||||
msEnded: context.mediaSource.get()?.readyState === 'ended',
|
||||
durationFinite: Number.isFinite(mediaElement.duration),
|
||||
paused: mediaElement.paused,
|
||||
seeking: mediaElement.seeking,
|
||||
ended: mediaElement.ended,
|
||||
currentTime: mediaElement.currentTime,
|
||||
bufferedEnd: buffered.length > 0 ? buffered.end(buffered.length - 1) : undefined,
|
||||
},
|
||||
nudgeWindow
|
||||
);
|
||||
// Nudge to `duration` → native `ended` (the seeking/ended guards above prevent a
|
||||
// re-fire while the nudge-seek is in flight, so no latch is needed).
|
||||
if (forceEnded) mediaElement.currentTime = mediaElement.duration;
|
||||
};
|
||||
|
||||
return listen(mediaElement, 'waiting', onWaiting);
|
||||
});
|
||||
}
|
||||
|
||||
export const recoverEndStall = defineBehavior({
|
||||
stateKeys: [] as const,
|
||||
contextKeys: ['mediaElement', 'mediaSource'] as const,
|
||||
setup: recoverEndStallSetup,
|
||||
});
|
||||
@@ -0,0 +1,50 @@
|
||||
import { describe, expect, it } from 'vitest';
|
||||
import { shouldForceEnded } from '../recover-end-stall';
|
||||
|
||||
// At the end-of-stream freeze: MediaSource ended, finite duration, actively playing,
|
||||
// and the playhead a few frames short of the reachable (intersection) buffered end.
|
||||
const atEndStall = {
|
||||
msEnded: true,
|
||||
durationFinite: true,
|
||||
paused: false,
|
||||
seeking: false,
|
||||
ended: false,
|
||||
currentTime: 599.95,
|
||||
bufferedEnd: 600.0,
|
||||
} as const;
|
||||
|
||||
const WINDOW = 0.2;
|
||||
|
||||
describe('shouldForceEnded', () => {
|
||||
it('fires at the end-of-stream freeze (playhead within the window of the buffered end)', () => {
|
||||
expect(shouldForceEnded(atEndStall, WINDOW)).toBe(true); // gap 0.05 < 0.2
|
||||
});
|
||||
|
||||
it('does not fire mid-content (playhead far from the buffered end)', () => {
|
||||
expect(shouldForceEnded({ ...atEndStall, currentTime: 300, bufferedEnd: 600 }, WINDOW)).toBe(false);
|
||||
});
|
||||
|
||||
it('does not fire until endOfStream is signalled', () => {
|
||||
expect(shouldForceEnded({ ...atEndStall, msEnded: false }, WINDOW)).toBe(false);
|
||||
});
|
||||
|
||||
it('does not fire for live (non-finite duration)', () => {
|
||||
expect(shouldForceEnded({ ...atEndStall, durationFinite: false }, WINDOW)).toBe(false);
|
||||
});
|
||||
|
||||
it('does not fire while paused, seeking, or already ended', () => {
|
||||
expect(shouldForceEnded({ ...atEndStall, paused: true }, WINDOW)).toBe(false);
|
||||
expect(shouldForceEnded({ ...atEndStall, seeking: true }, WINDOW)).toBe(false);
|
||||
expect(shouldForceEnded({ ...atEndStall, ended: true }, WINDOW)).toBe(false);
|
||||
});
|
||||
|
||||
it('does not fire with no buffered ranges', () => {
|
||||
expect(shouldForceEnded({ ...atEndStall, bufferedEnd: undefined }, WINDOW)).toBe(false);
|
||||
});
|
||||
|
||||
it('respects the configured window', () => {
|
||||
const gap015 = { ...atEndStall, currentTime: 599.85, bufferedEnd: 600.0 }; // gap 0.15
|
||||
expect(shouldForceEnded(gap015, 0.2)).toBe(true);
|
||||
expect(shouldForceEnded(gap015, 0.1)).toBe(false);
|
||||
});
|
||||
});
|
||||
@@ -21,6 +21,7 @@ import {
|
||||
import { deriveCdnPriority } from '../../behaviors/derive-cdn-priority';
|
||||
import { endOfStream } from '../../behaviors/dom/end-of-stream';
|
||||
import { loadAudioSegments } from '../../behaviors/dom/load-segments';
|
||||
import { recoverEndStall } from '../../behaviors/dom/recover-end-stall';
|
||||
import { relocationPipelinesFor } from '../../behaviors/dom/relocation-steps';
|
||||
import { setupAudioBufferActors } from '../../behaviors/dom/setup-buffer-actors';
|
||||
import { setupMediaSource } from '../../behaviors/dom/setup-mediasource';
|
||||
@@ -250,6 +251,9 @@ export function createHlsAudioOnlyEngine(
|
||||
// `mediaSource.sourceBuffers` aggregately — composes unchanged with
|
||||
// only audio in scope.
|
||||
endOfStream,
|
||||
// Force native `ended` if Chrome freezes the playhead short of the buffered end
|
||||
// after `endOfStream`. Inert for a clean-ending single-track source.
|
||||
recoverEndStall,
|
||||
|
||||
// Adapter signal callback.
|
||||
shareSignals,
|
||||
|
||||
@@ -43,6 +43,7 @@ import {
|
||||
import { deriveCdnPriority } from '../../behaviors/derive-cdn-priority';
|
||||
import { endOfStream } from '../../behaviors/dom/end-of-stream';
|
||||
import { loadAudioSegments, loadTextTrackSegments, loadVideoSegments } from '../../behaviors/dom/load-segments';
|
||||
import { recoverEndStall } from '../../behaviors/dom/recover-end-stall';
|
||||
import { relocatingTextPipelines, relocationPipelinesFor } from '../../behaviors/dom/relocation-steps';
|
||||
import { seekToLiveEdge } from '../../behaviors/dom/seek-to-live-edge';
|
||||
import { setupAudioBufferActors, setupVideoBufferActors } from '../../behaviors/dom/setup-buffer-actors';
|
||||
@@ -297,6 +298,13 @@ export interface SimpleHlsEngineConfig extends ShareSignalsConfig<SimpleHlsEngin
|
||||
* `internal/design/spf/presentation-timeline-model.md`.
|
||||
*/
|
||||
deriveStartMediaTime?: DeriveStartMediaTime;
|
||||
/**
|
||||
* Proximity window (seconds) for the `recoverEndStall` behavior — how close the
|
||||
* playhead must be to the reachable buffered end for a `waiting` to be treated as the
|
||||
* end-of-stream freeze and nudged to `ended`. Defaults to `0.2`. See
|
||||
* `behaviors/dom/recover-end-stall`.
|
||||
*/
|
||||
endStallNudgeWindow?: number;
|
||||
}
|
||||
|
||||
// ============================================================================
|
||||
@@ -473,6 +481,9 @@ export function createSimpleHlsEngine(
|
||||
|
||||
// End of stream coordination
|
||||
endOfStream,
|
||||
// Force native `ended` when Chrome freezes the playhead a few frames short of a
|
||||
// skewed-A/V end after `endOfStream` (audio-clock stall). Inert otherwise.
|
||||
recoverEndStall,
|
||||
|
||||
// Text tracks
|
||||
syncTextTracks,
|
||||
|
||||
Reference in New Issue
Block a user