diff --git a/internal/design/README.md b/internal/design/README.md index f8d958c5..c0c8222e 100644 --- a/internal/design/README.md +++ b/internal/design/README.md @@ -60,7 +60,7 @@ Use a different structure for a living reference or registry, but keep frontmatt ## Maintenance - Link current source and tests; do not copy APIs, schemas, or file inventories. -- When implementation lands, update status and remove speculative mechanics. +- When implementation lands, collapse the record to durable rationale, constraints, consequences, and source pointers; remove speculative mechanics and current-behavior inventories. - When a record becomes wrong, update it, mark it superseded with a successor, or delete it if no rationale remains. - Keep implemented records only when their constraints, alternatives, or tradeoffs still help future changes. diff --git a/internal/design/i18n/architecture.md b/internal/design/i18n/architecture.md index 0ff8253f..d16a923c 100644 --- a/internal/design/i18n/architecture.md +++ b/internal/design/i18n/architecture.md @@ -3,1020 +3,34 @@ status: implemented date: 2026-03-25 --- -# Internationalization (i18n) +# Internationalization -Opaque-key translation system with a global registry and ES module locale files for CDN, typed hooks for React. +This record preserves the rationale behind Video.js 10 internationalization. The package source, exports, tests, and generated locale modules define the current API and supported locales. ## Problem -All `aria-label`, `aria-valuetext`, tooltip labels, and time unit strings in Video.js 10 are hardcoded English. `PlayButtonCore.getLabel()` returns `'play'`. `formatDuration` returns `'2 minutes, 30 seconds'`. There is no mechanism to supply translated strings without overriding each component's `label` prop individually, and no single language switch for skins. - -Requirements: - -- Single entry point — one API replaces all English defaults regardless of layer (HTML, React, CDN) -- Stable keys — renaming an English label must not require updating every locale file -- Typed keys — TypeScript autocomplete for all keys -- Decoupled — i18n providers are not embedded in skins; any skin works with any provider -- Works without skins — standalone VJS components (buttons, sliders, time elements used outside of ``) can consume a provider directly -- CDN-compatible — ES module locale files self-register with no bundler and no global namespace pollution -- No side-effect locale imports — bundlers don't accidentally include all languages -- Framework-agnostic core — the registry and translator live in `@videojs/core/i18n`, re-exported to consumers as `@videojs/html/i18n` and `@videojs/react/i18n` -- Locale-aware — native `Intl` APIs handle duration, number, and plural formatting - -## API - -### Registering translations - -The single entry point for both HTML and React consumers: - -```ts -// HTML -import { registerI18n } from '@videojs/html/i18n'; -import es from '@videojs/html/i18n/locales/es'; - -// React -import { registerI18n } from '@videojs/react'; -import es from '@videojs/react/i18n/locales/es'; - -registerI18n('es', es); -``` - -`registerI18n` merges (does not replace) — multiple calls for the same language are additive. English defaults (`en.ts`) are pre-registered as the base layer on module import. - -### HTML - -`` reads from the nearest ancestor `lang` attribute, matching the inheritance model of the native HTML `lang` attribute. Setting `lang` on `` — or any ancestor — is enough: - -```html - - - - - - - - - - - -``` - -An explicit `lang` attribute on the provider overrides the inherited value — useful when a single page hosts players in different languages: - -```html - - - - - - - - - -``` - -The element reads from the registry for whichever `lang` is active. No changes to `` or any other skin element. - -**CDN** - -Locale files are self-registering ES modules. They import `registerI18n` from the same CDN module URL, sharing the same registry instance — no global namespace pollution: - -```html - - - - - - - - - - - - -``` - -```js -// es.js — self-registering, no window touch -import { registerI18n } from 'https://cdn.jsdelivr.net/npm/@videojs/html/cdn/i18n.js'; -registerI18n('es', { play: 'Reproducir', pause: 'Pausa', /* … */ }); -``` - -**Custom registration** - -```ts -// Override individual keys -registerI18n('es', { play: 'Comenzar' }); - -// Or override a built-in pack entirely -import myEs from './my-es'; -registerI18n('es', myEs); -``` - -### React - -`I18nProvider` is a standalone component. Consumers add it explicitly inside the player's `Provider` — the skin itself has no i18n awareness. When `locale` is omitted, it reads from the nearest ancestor `lang` attribute (typically ``): - -```tsx -import { I18nProvider } from '@videojs/react'; -import { VideoSkin, Video } from '@videojs/react/video'; - -// locale inherited from - - - - - - -``` - -Pass `locale` explicitly to override inheritance, or `translations` to bypass the registry entirely: - -```tsx -import es from '@videojs/react/i18n/locales/es'; - - - - - - - -``` - -**SSR / no flash-of-English** — pass `translations` directly on first render: - -```tsx -// Server component -const { default: translations } = await import(`@videojs/react/i18n/locales/${locale}`); - - - - - - - -``` - -**Dynamic switching** - -Changing locale loads the shipped pack automatically via `loadLocale` (or pass `translations` / call `registerI18n` for zero-flash SSR): - -```tsx -const [locale, setLocale] = useState('es'); - - - - - - - -``` - -For zero-flash switching, pre-import the locale and pass `translations` directly: - -```tsx -const [{ locale, translations }, setLocale] = useState({ locale: 'en', translations: undefined }); - -async function switchTo(next: string) { - const { default: translations } = await import(`@videojs/react/i18n/locales/${next}`); - setLocale({ locale: next, translations }); -} -``` - -**With next-intl** - -```tsx -import { getLocale } from 'next-intl/server'; - -const locale = await getLocale(); -const { default: translations } = await import(`@videojs/react/i18n/locales/${locale}`); - - - - - - - -``` - -### Translation keys - -Keys are opaque camelCase identifiers. The English string is the value in `en.ts` — not the key. All keys are optional; missing keys fall back to the English default. - -| Key | English default | Params | Used by | -| --- | --------------- | ------ | ------- | -| `play` | `'Play'` | — | `PlayButtonCore` | -| `pause` | `'Pause'` | — | `PlayButtonCore` | -| `replay` | `'Replay'` | — | `PlayButtonCore` | -| `mute` | `'Mute'` | — | `MuteButtonCore` | -| `unmute` | `'Unmute'` | — | `MuteButtonCore` | -| `seekForward` | `'Seek forward {seconds} seconds'` | `{seconds}` | `SeekButtonCore` | -| `seekBackward` | `'Seek backward {seconds} seconds'` | `{seconds}` | `SeekButtonCore` | -| `enterFullscreen` | `'Enter fullscreen'` | — | `FullscreenButtonCore` | -| `exitFullscreen` | `'Exit fullscreen'` | — | `FullscreenButtonCore` | -| `enableCaptions` | `'Enable captions'` | — | `CaptionsButtonCore` | -| `disableCaptions` | `'Disable captions'` | — | `CaptionsButtonCore` | -| `enterPictureInPicture` | `'Enter picture-in-picture'` | — | `PipButtonCore` | -| `exitPictureInPicture` | `'Exit picture-in-picture'` | — | `PipButtonCore` | -| `playingLive` | `'Playing live'` | — | `LiveButtonCore` | -| `seekToLiveEdge` | `'Seek to live edge'` | — | `LiveButtonCore` | -| `liveBadge` | `'Live'` | — | Live badge / time display | -| `startCasting` | `'Start casting'` | — | `CastButtonCore` | -| `stopCasting` | `'Stop casting'` | — | `CastButtonCore` | -| `connectingCast` | `'Connecting'` | — | `CastButtonCore` | -| `seek` | `'Seek'` | — | `TimeSliderCore` (aria-label) | -| `volume` | `'Volume'` | — | `VolumeSliderCore` (aria-label) | -| `timeCurrent` | `'Current time'` | — | `TimeCore` | -| `timeDuration` | `'Duration'` | — | `TimeCore` | -| `timeRemaining` | `'Remaining'` | — | `TimeCore` | -| `remainingTimeSuffix` | `'remaining'` | — | `formatDuration` (negative time suffix) | -| `playbackRateAria` | `'Playback rate {rate}'` | `{rate}` | `PlaybackRateButtonCore`, playback-rate menu | -| `timeSliderValueTextRange` | `'{current} of {duration}'` | `{current}`, `{duration}` | `TimeSliderCore` (aria-valuetext) | -| `volumeSliderValueTextMuted` | `'{percent}, muted'` | `{percent}` | `VolumeSliderCore` (aria-valuetext when muted) | -| `indicatorMuted` | `'Muted'` | — | Input feedback (status / announcer) | -| `indicatorVolume` | `'Volume'` | — | Input feedback (status indicator label) | -| `indicatorVolumeWithValue` | `'Volume {value}'` | `{value}` | Input feedback (status announcer) | -| `indicatorCaptionsOn` | `'Captions on'` | — | Input feedback | -| `indicatorCaptionsOff` | `'Captions off'` | — | Input feedback | -| `indicatorPaused` | `'Paused'` | — | Input feedback | -| `indicatorPlaying` | `'Playing'` | — | Input feedback | -| `indicatorFullscreen` | `'Enter fullscreen'` | — | Input feedback | -| `indicatorExitFullscreen` | `'Exit fullscreen'` | — | Input feedback | -| `indicatorPictureInPicture` | `'Picture in picture'` | — | Input feedback | -| `indicatorExitPictureInPicture` | `'Exit picture in picture'` | — | Input feedback | -| `mediaErrorAborted` | `'You aborted the media playback'` | — | Error dialog | -| `mediaErrorNetwork` | `'A network error caused…'` | — | Error dialog | -| `mediaErrorDecode` | `'A media error caused playback…'` | — | Error dialog | -| `mediaErrorSrcNotSupported` | `'An unsupported error occurred…'` | — | Error dialog | -| `mediaErrorEncrypted` | `'The media is encrypted…'` | — | Error dialog | -| `mediaErrorCustom` | `''` | — | Error dialog (custom errors use literal message) | -| `errorDialogTitle` | `'Something went wrong.'` | — | Error dialog | -| `errorDialogDismiss` | `'OK'` | — | Error dialog | -| `mediaErrorFallback` | `'An error occurred. Please try again.'` | — | Error dialog | - -> `timeSliderValueTextRange` params are already-formatted time phrases from `Intl.DurationFormat`, not raw numbers. - -> `Intl.DurationFormat` handles duration unit labels; `Intl.NumberFormat` handles percent formatting. Only `remainingTimeSuffix` and `volumeSliderValueTextMuted` need translation keys for suffixes `Intl` cannot express. - -> Full key list and param contracts: `packages/core/src/core/i18n/types.ts` (`TranslationParams`). - -## Architecture - -### Layers - -``` -@videojs/core/i18n - Translations · Translator · createTranslator - registerI18n · getI18nTranslations · onI18nRegistryChange - Registry (singleton Map) · en.ts (base layer, always present) - │ - ├── @videojs/react/i18n - │ createI18n() → { I18nContext, I18nProvider, useTranslator, useLocale } - │ Consumer wraps skin: - │ - └── @videojs/html/i18n - createI18n() → { context, I18nController, ProviderMixin, TextMixin } - = ProviderMixin(ReactiveElement) - Play = TextMixin(ReactiveElement) - MediaButtonElement uses I18nController → t(core.getLabel(state)) - -@videojs/utils/i18n pluralize(count, forms, locale?) -@videojs/utils/time formatDuration(seconds, options?) -``` - -### Core types - -Authoritative definitions: `packages/core/src/core/i18n/types.ts` and `locales.ts`. - -- **`Locale`** — `(typeof LOCALES)[number] | (string & {})` for any BCP 47 tag. -- **`TranslationParams`** — per-key param contract (`never` = no params). -- **`Translations`** — optional overlay; parametric values must contain required `{placeholder}` substrings (`Contains<>` helper). -- **`Translator`** — typed `(key, params?) => string` from `createTranslator`. - -The `Contains` helper makes locale files compile-error when they forget a `{param}` placeholder. `Translator`'s signature catches two classes of mistake at call sites: - -```ts -t('play'); // ✓ no params -t('play', { foo: 1 }); // ✗ TS error — no params accepted -t('seekForward'); // ✗ TS error — missing { seconds } -t('seekForward', { seconds: 10 }); // ✓ -t('seekForward', { second: 10 }); // ✗ TS error — typo, wrong key -``` - -### Registry - -```ts -// @videojs/core/i18n/registry.ts -const registry = new Map>(); -const subscribers = new Set<() => void>(); - -// en.ts is the base layer — pre-registered at module init -import en from './locales/en'; -registry.set('en', en); - -export function registerI18n(locale: string, translations: Partial): void { - const existing = registry.get(locale) ?? {}; - registry.set(locale, { ...existing, ...translations }); - subscribers.forEach(fn => fn()); -} - -export function getI18nTranslations(locale: string): Partial { - const en = registry.get('en') ?? {}; - // Walk up the BCP 47 subtag chain: zh-Hant-HK → zh-Hant → zh → en - const parts = locale.split('-'); - for (let i = parts.length; i > 0; i--) { - const tag = parts.slice(0, i).join('-'); - const pack = registry.get(tag); - if (pack) return { ...en, ...pack }; - } - return { ...en }; -} - -export function onI18nRegistryChange(callback: () => void): () => void { - subscribers.add(callback); - return () => subscribers.delete(callback); -} -``` - -`getI18nTranslations` always returns at least the English defaults (from `en.ts`). No key is ever missing — the per-key fallback is `registry[locale][key] → registry[parent-subtag][key] → … → en[key] → key`. See the [BCP 47 fallback](#bcp-47-fallback) section for the full subtag-truncation rules. - -### `createTranslator` - -```ts -export function createTranslator( - translations: Partial = {}, - locale?: Locale -): Translator { - // Overload signatures are type-only; implementation uses the loose callable form. - function t(key: keyof Translations, params?: Record): string { - let value = (translations[key] ?? key) as string; - if (params) { - for (const [k, v] of Object.entries(params)) { - value = value.replace(`{${k}}`, String(v)); - } - } - return value; - } - t.locale = locale; - return t as Translator; -} -``` - -### Locale files - -Locale files are TypeScript source files. They use `satisfies Partial` so typos in key names are caught at build time — a misspelled key is a compile error, not a silent runtime miss: - -```ts -// locales/en.ts -import type { Translations } from '../types'; - -export default { - play: 'Play', - pause: 'Pause', - replay: 'Replay', - mute: 'Mute', - unmute: 'Unmute', - seek: 'Seek', - volume: 'Volume', - muted: 'muted', - enterFullscreen: 'Enter fullscreen', - exitFullscreen: 'Exit fullscreen', - enableCaptions: 'Enable captions', - disableCaptions: 'Disable captions', - enterPictureInPicture: 'Enter picture-in-picture', - exitPictureInPicture: 'Exit picture-in-picture', - currentTime: 'Current time', - duration: 'Duration', - remaining: 'Remaining', - seekForward: 'Seek forward {seconds} seconds', - seekBackward: 'Seek backward {seconds} seconds', - playbackRate: 'Playback rate {rate}', - timePosition: '{current} of {duration}', -} satisfies Partial; -``` - -```ts -// locales/es.ts — partial locale, falls back to en.ts for missing keys -import type { Translations } from '../types'; - -export default { - // Play controls - play: 'Reproducir', - pause: 'Pausa', - replay: 'Repetir', - // Mute - mute: 'Silenciar', - unmute: 'Activar sonido', - // … -} satisfies Partial; -``` - -The `satisfies` constraint checks key names and value types without widening the type — `keyof typeof es` remains the narrow set of keys present, not `keyof Translations`. Comments above groups of keys are useful context for translators. - -For CDN, locale files are self-registering ES modules. The build emits a separate CDN entry point alongside the bundler-importable output: - -```ts -// locales/es.cdn.ts — CDN entry, self-registers -import { registerI18n } from '../registry'; -import translations from './es'; - -registerI18n('es', translations); -``` - -```js -// Built output: es.js (CDN) -import { registerI18n } from 'https://cdn.jsdelivr.net/npm/@videojs/html/cdn/i18n.js'; -registerI18n('es', { play: 'Reproducir', pause: 'Pausa', /* … */ }); -``` - -### React: `createI18n` - -A factory that closes over a React context so `useTranslator()` returns `Translator` with no explicit type annotation at call sites. Called once in `@videojs/react/i18n`; the result is exported directly. - -When `locale` is omitted, `I18nProvider` reads `document.documentElement.lang` and subscribes to changes via `MutationObserver`: - -```tsx -export function createI18n() { - const I18nContext = createContext(null); - - function I18nProvider({ locale: explicitLocale, translations, children }: { - locale?: Locale; - translations?: Partial; - children: ReactNode; - }) { - // Subscribe to ambient DOM lang attribute when no explicit prop is given - const ambientLocale = useSyncExternalStore( - subscribeToDocumentLang, - () => document.documentElement.lang || undefined, - () => undefined, // SSR: no ambient locale - ); - const locale = explicitLocale ?? ambientLocale; - - // Lazy-load built-in overlays, then browser fallback when no pack exists. - useEffect(() => { - const seq = ++lazySeqRef.current; - const locale = resolvedLocale; - void (async () => { - const { merged, loadedTags } = await mergeLocaleOverlays(locale, loadLocale, localeLookupChain); - if (seq !== lazySeqRef.current) return; - if (shouldAttemptBrowserTranslation(locale, loadedTags, merged)) { - const browser = await getBrowserTranslations(locale); - if (Object.keys(browser).length) registerI18n(locale, browser); - } - if (seq !== lazySeqRef.current) return; - setLazyLayer(merged); - })(); - }, [resolvedLocale]); - - // Priority: registry (incl. browser registerI18n) < lazy built-in < consumer translations - const translations = useMemo( - () => ({ ...getI18nTranslations(resolvedLocale), ...lazyLayer, ...translationsProp }), - [resolvedLocale, lazyLayer, translationsProp, registryEpoch] - ); - const translator = useMemo(() => createTranslator(translations, resolvedLocale), [translations, resolvedLocale]); - - return {children}; - } - - function useTranslator(): Translator { - return useContext(I18nContext) ?? createTranslator(getI18nTranslations('en')); - } - - function useLocale(): Locale | undefined { - return useTranslator().locale; - } - - return { I18nContext, I18nProvider, useTranslator, useLocale }; -} - -/** Shared MutationObserver subscription for document.documentElement.lang changes. */ -function subscribeToDocumentLang(onChange: () => void): () => void { - const observer = new MutationObserver(onChange); - observer.observe(document.documentElement, { attributes: true, attributeFilter: ['lang'] }); - return () => observer.disconnect(); -} -``` - -### HTML: `createI18n` factory - -Parallel to React's `createI18n`, `createI18n` bundles the context, controller, provider element, and text element into one call. `@videojs/html/i18n` invokes it once and re-exports the result. Consumer-facing API is a single side-effect import that registers `` / ``: - -```ts -// @videojs/html/i18n/create-i18n.ts - -export function createI18n() { - const context = createContext(Symbol('@videojs/i18n')); - - /** Controller for any element that consumes translated strings (e.g. button labels). */ - class I18nController implements ReactiveController { /* subscribes to context */ } - - /** Mixin: adds lang attribute + DOM lang resolution + ContextProvider for i18nContext. */ - const ProviderMixin = >(base: Base) => - class extends base { /* see "Provider mixin" below */ }; - - /** Mixin: captures source text + subscribes to i18nContext + writes textContent. */ - const TextMixin = >(base: Base) => - class extends base { /* see "Text mixin" below */ }; - - return { context, I18nController, ProviderMixin, TextMixin }; -} -``` - -Mirrors `createPlayer` — the factory returns building blocks (context, controller, mixins). Define files compose them onto concrete bases and register the custom elements: - -```ts -// @videojs/html/src/define/media-i18n.ts -import { createI18n } from '@videojs/html/i18n'; - -const { ProviderMixin } = createI18n(); -export class MediaI18nProviderElement extends ProviderMixin(ReactiveElement) {} -safeDefine('media-i18n', MediaI18nProviderElement); -``` - -```ts -// @videojs/html/src/define/media-text.ts -import { createI18n } from '@videojs/html/i18n'; - -const { TextMixin } = createI18n(); -export class MediaTextElement extends TextMixin(ReactiveElement) {} -safeDefine('media-text', MediaTextElement); -``` - -### HTML: Provider mixin - -Reads from the registry, provides via `i18nContext`. Resolves `lang` from the element's own attribute if set, otherwise from the nearest ancestor's `lang` attribute (mirroring native HTML inheritance). Subscribes to both registry changes and DOM `lang` attribute mutations so live elements re-render when either changes: - -```ts -const ProviderMixin = >(base: Base) => - class extends base { - static override properties = { ...base.properties, lang: { type: String, reflect: true } }; - - /** Explicit override. When unset, resolves from DOM ancestor chain. */ - lang: Locale | undefined = undefined; - - readonly #provider = new ContextProvider(this, { - context, - initialValue: createTranslator(getI18nTranslations('en')), - }); - - #unsubscribeRegistry: (() => void) | null = null; - #langObserver: MutationObserver | null = null; - - /** Resolve effective locale: explicit lang attribute → nearest ancestor[lang] → documentElement.lang → undefined. */ - get #effectiveLocale(): Locale | undefined { - if (this.lang) return this.lang; - const ancestor = this.parentElement?.closest('[lang]'); - return (ancestor?.getAttribute('lang') || document.documentElement.lang) || undefined; - } - - override connectedCallback(): void { - super.connectedCallback(); - this.#unsubscribeRegistry = onI18nRegistryChange(() => this.#refresh()); - // Observe document-wide lang attribute changes so ambient updates re-render descendants. - this.#langObserver = new MutationObserver(() => this.#refresh()); - this.#langObserver.observe(document.documentElement, { - subtree: true, attributes: true, attributeFilter: ['lang'], - }); - this.#refresh(); - } - - override disconnectedCallback(): void { - super.disconnectedCallback(); - this.#unsubscribeRegistry?.(); - this.#langObserver?.disconnect(); - this.#unsubscribeRegistry = this.#langObserver = null; - } - - protected override updated(changed: PropertyValues): void { - super.updated(changed); - if (changed.has('lang')) this.#refresh(); - } - - async #resetLazyAndLoad(): Promise { - const localeSnapshot = resolveProviderLocale(this); - this.#lazyResetStartedForLocale = localeSnapshot; - this.#lazySeq += 1; - const seq = this.#lazySeq; - this.#lazyLayer = {}; - void (async () => { - const { merged, loadedTags } = await mergeLocaleOverlays(localeSnapshot, loadLocale, localeLookupChain); - if (seq !== this.#lazySeq) return; - if (shouldAttemptBrowserTranslation(localeSnapshot, loadedTags, merged)) { - const browser = await getBrowserTranslations(localeSnapshot); - if (Object.keys(browser).length) registerI18n(localeSnapshot, browser); - } - if (seq !== this.#lazySeq) return; - this.#lazyLayer = merged; - this.requestUpdate(); - })(); - } - - #publish(): void { - const locale = resolveProviderLocale(this); - const translations = { ...getI18nTranslations(locale), ...this.#lazyLayer }; - this.#i18nProvider.setValue({ translator: createTranslator(translations, locale), locale }); - } - }; -``` - -### HTML: Text mixin - -Renders a translated string for its initial text content. Subscribes to `i18nContext`: - -```ts -const TextMixin = >(base: Base) => - class extends base { - readonly #i18n = new I18nController(this); - #text: string | undefined; - - override connectedCallback(): void { - this.#text ??= this.textContent?.trim() ?? ''; - super.connectedCallback(); - } - - protected override update(changed: PropertyValues): void { - super.update(changed); - this.textContent = this.#text ? this.#i18n.value(this.#text) : ''; - } - }; -``` - -### HTML: Built-in skin tooltips (trigger sync) - -Built-in skins use **empty** `` elements linked to controls via `commandfor`. Tooltip text is not authored in the skin template — `TooltipElement` copies the trigger button's translated label at runtime (`getLabel()` / `getResolvedLabel()` via `I18nController`), keeping tooltip copy and `aria-label` in sync. - -```html - - - -``` - -Changing text inside `` in a built-in skin has no effect. To customize tooltip and `aria-label`, set `label` on the control (e.g. ``) or override the component's core label key. - -### HTML: `` (ejected / custom skins only) - -`` renders translated copy for static shadow DOM strings **not** owned by a component's `getLabel()` — standalone help text, dialog bodies, or tooltips in **ejected** skins without a linked trigger. Built-in button labels and built-in tooltips do not use ``. - -```html - - - Seek - -``` - -### HTML: `I18nController` and `MediaButtonElement` - -`I18nController` is returned from `createI18n()` alongside the mixins, closing over the same `context` symbol so it reads the value set by any `ProviderMixin`-backed element. Added to `MediaButtonElement` and `MediaUIElement` for auto-forwarded labels: - -```ts -// Returned from createI18n() — closes over the factory's context -class I18nController implements ReactiveController { - readonly #consumer: ContextConsumer; - - constructor(host: ReactiveElement) { - this.#consumer = new ContextConsumer(host, { context, subscribe: true }); - host.addController(this); - } - - get value(): Translator { - return this.#consumer.value ?? createTranslator(getI18nTranslations('en')); - } - - hostConnected(): void {} - hostDisconnected(): void {} -} -``` - -```ts -// MediaButtonElement.update() -readonly #i18n = new I18nController(this); - -protected override update(changed: PropertyValues): void { - super.update(changed); - const media = this.mediaState.value; - if (!media) return; - - this.core.setMedia(media); - const state = this.core.getState(); - const t = this.#i18n.value; - - const key = this.core.getLabel(state); // opaque key, e.g. 'play' - const params = this.core.getLabelParams?.(state); - this.setAttribute('aria-label', t(key, params)); - - applyElementProps(this, this.core.getAttrs(state)); - applyStateDataAttrs(this, state, this.stateAttrMap); -} -``` - -When no `` ancestor is present, `I18nController.value` falls back to a translator seeded from the English registry — English strings are rendered by default. - -### Locale resolution - -The provider resolves its active locale from the first source that yields a value: - -``` -1. Explicit locale prop (React) / lang attribute (HTML) on the provider -2. Nearest ancestor element with a [lang] attribute (HTML only; document.documentElement.lang in React) -3. undefined → English defaults -``` - -`` walks the DOM with `this.parentElement?.closest('[lang]')` and falls back to `document.documentElement.lang`. Both the explicit attribute and ambient sources are tracked — a `MutationObserver` watches `document.documentElement`'s subtree for `lang` attribute changes, so any update to `` (or any intermediate wrapper's `lang`) re-renders descendants automatically. - -React's `I18nProvider` uses `useSyncExternalStore` over `document.documentElement.lang` for the ambient value. SSR returns `undefined` from the server snapshot; on hydration the client reads the rendered `` attribute, keeping server and client in sync. - -This matches the inheritance semantics of the native HTML `lang` attribute — set it once at the page level and every player picks it up without extra wiring. - -### BCP 47 fallback - -Locale values are [BCP 47 language tags](https://datatracker.ietf.org/doc/html/rfc5646). The registry lookup in `getI18nTranslations(locale)` applies a left-truncation fallback — progressively dropping the rightmost subtag until a registered entry is found: - -``` -es-419-u-nu-latn → es-419 → es → en -zh-Hant-HK → zh-Hant → zh → en -pt-BR → pt-BR → pt → en -en-GB-scotland → en-GB-scotland → en-GB → en -sr-Latn → sr-Latn → sr → en -``` - -The algorithm treats each hyphen as a subtag boundary and truncates from the right. Script subtags (`Hant`, `Latn`), region subtags (`419`, `HK`, `GB`), and extension subtags (`u-nu-latn`) are all handled uniformly — no special cases. - -**What it does *not* do.** Sibling fallback is not implemented: a request for `es-AR` will not fall back to `es-MX` even if only `es-MX` is registered. The chain is strictly up the parent hierarchy. Consumers who need sibling negotiation should register the base language (`es`) or the specific variants they want to cover. - -`` applies the same fallback when lazy-loading built-in packs: it tries `pt-BR.ts`, then `pt.ts`, before giving up. Only full subtag matches succeed — `en-GB-scotland` requires a literal `en-GB-scotland.ts` file; otherwise it falls through to `en-GB` and finally `en`. - -The Browser Translation API path uses the native `Intl.Locale` minimization where available for cross-referencing user preferences against the registered pack list. - -### Browser Translation API - -The [Translator API](https://developer.chrome.com/docs/ai/translator-api) (WICG draft, Chrome 138+) provides on-device text translation. Video.js uses it as a background fallback for locales with no registered or lazy-loaded pack. Implementation lives in `@videojs/core/i18n` (`getBrowserTranslations`, `shouldAttemptBrowserTranslation`). - -Only activates when `Translator.availability()` returns `'available'` — model already present, no network cost. `'downloadable'` / `'downloading'` / `'unavailable'` are silently skipped in production providers. - -**Sandbox exception.** The dev sandbox may call `getBrowserTranslations` with `downloadIfNeeded: true` for browser-only locale tags (Chrome Translator API languages without a shipped pack). This is for demo coverage only — not production behavior. - -Providers call `shouldAttemptBrowserTranslation(locale, loadedTags, merged)` after `mergeLocaleOverlays` — skip when locale is English, when lazy built-ins loaded, or when any non-`en` tag in `localeLookupChain(locale)` is already registered (covers `es-MX` → `es`). - -Since keys are opaque, the browser API translates the *English values* from `en.ts`, then maps the results back to keys. Results are registered via `registerI18n` (triggering `onI18nRegistryChange`) and cached per target language at module level. - -**Priority merge:** - -``` -English defaults (en.ts — always present) - ↑ -Browser API (registerI18n after async translate; pre-installed model only) - ↑ -Registry pack (registerI18n / CDN locale modules) - ↑ -Lazy built-in (`loadLocale` overlay from `@videojs/core/i18n`) - ↑ -Consumer prop (React translations — always wins) -``` - -### `loadLocale` - -`loadLocale(tag)` in `@videojs/core/i18n` lazy-imports shipped locale packs by exact BCP 47 tag. It skips tags already present in the registry (including `en`) so explicit `registerI18n` overrides are preserved. Default `createI18n()` providers call it via `mergeLocaleOverlays` when the locale changes — `` or `` loads Spanish without a prior `registerI18n` call. Override with `createI18n({ loadLocale })` in tests or custom apps. - -Explicit `registerI18n`, CDN locale modules, and React `translations` still merge on top and remain the preferred path for SSR (zero flash) and CDN. - -### Intl API integration - -**`Intl.DurationFormat`** — drives `formatDuration`. Handles unit labels, pluralization, and locale-specific ordering automatically (baseline: Chrome 122+, Firefox 127+, Safari 18+). The remaining-time suffix uses `remainingTimeSuffix` via `formatOptions.translate` since `Intl.DurationFormat` has no concept of remaining time. - -**`Intl.NumberFormat`** with `style: 'percent'` — formats volume values. No translation key needed. - -**`Intl.PluralRules`** — powers the `pluralize` utility for skin authors who need locale-correct plurals in custom components: - -```ts -import { pluralize } from '@videojs/utils/i18n'; - -const label = `${count} ${pluralize(count, { one: t('minute'), other: t('minutes') }, locale)}`; -``` - -### SSR & hydration safety - -`Intl` APIs produce locale-specific output. If the active locale differs between server and client, hydration mismatches occur. The fix: always derive the locale from the same source on both sides (URL routing, cookie, or `Accept-Language` header passed explicitly). - -``` -HTTP request - → read locale from cookie / URL / Accept-Language - → set on the server-rendered document - → load translations - → server render: … (locale inherited from ) - → client hydrate: … ← identical = no mismatch -``` - -Passing `locale` explicitly to `I18nProvider` also works but is only required when overriding the document locale for a subtree. The ambient-locale pattern makes single-locale pages zero-config. - -### CDN build strategy - -The CDN bundle is an ES module. Locale files are separate self-registering ES modules — they import `registerI18n` from the same CDN URL, sharing the singleton registry instance with no global namespace touch: - -```html - - -``` - -```js -// https://cdn.jsdelivr.net/npm/@videojs/html/cdn/locales/es.js -import { registerI18n } from 'https://cdn.jsdelivr.net/npm/@videojs/html/cdn/i18n.js'; -registerI18n('es', { play: 'Reproducir', pause: 'Pausa', /* … */ }); -``` - -`` reads from the registry regardless of how data arrived — same element, same attribute, whether the translations came from an inline `registerI18n` call or a CDN locale module. +Video.js needed one framework-neutral translation system for HTML, React, and CDN consumers without coupling language state to a skin or making English copy part of the translation key contract. ## Decisions -### Opaque camelCase keys, not English strings +- Translation keys are stable camelCase identifiers. English strings remain values, so copy changes do not invalidate every locale. +- A global core registry loads each locale once per page. Explicit registration keeps locale modules tree-shakeable; self-registering modules support CDN use. +- Providers are independent of skins and can scope one or many players. Without an explicit locale, they inherit the nearest DOM language declaration rather than browser preference. +- React and HTML integrations are created by `createI18n()` factories so contexts can be isolated and their providers, hooks, and controllers remain paired. +- A control's translated label also supplies built-in tooltip text. `` exists only for standalone copy in custom or ejected templates. +- Interpolation uses lightweight `{param}` replacement. Native `Intl` APIs own locale-aware number, duration, percentage, and plural formatting. +- Browser translation is opportunistic only when its model is already installed; it must never trigger a large implicit download. -**Decision.** Keys are camelCase identifiers (`play`, `seekForward`). The English string is the value in `en.ts`. +## Consequences -**Alternatives considered.** +Consumers can switch languages for an entire provider tree while skins remain presentation-only. The registry and core translator stay framework-neutral, and applications opt into only the locale modules they need. SSR callers that must avoid an English first render provide translations synchronously. -- *English as key* — `t('Play')` falls back to `'Play'` automatically. Self-documenting. Renaming `'Play'` to `'Start'` in English breaks all other locale files. Established by VJS v7/8 and [Media Chrome](https://www.media-chrome.org/docs/en/internationalization/adding-language-support). -- *Numeric keys* — stable, compact, completely opaque. No useful fallback. -- *Namespaced paths* — `t('controls.play')`. Common in i18next; adds indirection. +This intentionally does not define locale negotiation, right-to-left layout, caption-language selection, ICU message syntax, CAT-tool integration, or a consumer-supplied translator adapter. -**Rationale.** Stable keys decouple English UX copy from locale file versioning. Adding a new language or updating an English label are independent operations. The English string is still the default — it lives in `en.ts`, which is always pre-registered as the base layer. Consistent with i18next, FormatJS, iOS `NSLocalizedString`, and Android `strings.xml`. +## Current sources of truth -### Global registry, not lazy-loading on skin element - -**Decision.** A singleton `Map` in `@videojs/core/i18n`. `registerI18n(locale, translations)` is the imperative entry point. `` reads from it. - -**Alternatives considered.** - -- *JSON attribute on skin element* — ``. Couples translations to the skin element; JSON attributes are fragile and lose type safety. -- *Side-effect locale imports* — `import '@videojs/core/i18n/locales/es'` auto-registers. Media Chrome uses this pattern (`import 'media-chrome/lang/es.js'`). Simple for CDN, but bundlers may include unwanted locales, and there's no explicit registration call for overrides. -- *Lazy-load only inside provider* — each provider independently loads its pack. Multiple players for the same locale each trigger a separate network request. - -**Rationale.** Module-level singleton means each locale is loaded once per page regardless of how many players are mounted. For CDN, locale files are self-registering ES modules — they import `registerI18n` from the same module URL, sharing the registry instance with no global namespace touch. For bundlers, consumers may call `registerI18n` explicitly or rely on default provider `loadLocale` lazy imports. Pass React `translations` for SSR zero-flash. - -### Standalone providers — skins are i18n-unaware - -**Decision.** Neither HTML skins nor React skins own the i18n provider. Consumers add `` or `` explicitly and place them wherever they need. - -**Alternatives considered.** - -- *`I18nMixin` on the skin element* — skin element owns `translations` and `lang` attributes. Simpler consumer API but couples skins to i18n. Adding `lang` to `` creates an attribute collision with the native HTML `lang` attribute. -- *`locale`/`translations` props on ``* — skin internally wraps itself with provider. Zero friction, but skin must know about i18n; harder to share a provider across multiple skins. - -**Rationale.** Skins are presentation layers. i18n is a cross-cutting concern. Decoupling follows the same principle as `` for the store — skins consume context, providers supply it, and neither owns the other. Consumers can wrap multiple skins in one provider, or give each its own. The HTML `lang` attribute conflict disappears since `` is a distinct element. - -### Provider inherits locale from the DOM - -**Decision.** When no explicit `locale` prop (React) / `lang` attribute (HTML) is set on the provider, it resolves the locale from the nearest ancestor `[lang]` attribute (HTML) or `document.documentElement.lang` (React). A `MutationObserver` watches for changes so updates to `` propagate automatically. - -**Alternatives considered.** - -- *Explicit locale always required on the provider* — previous revision behavior. Ergonomic friction: every app must pass the locale into the provider even when `` already declares the page language. -- *Skin reads DOM `lang` directly, no provider* — would eliminate the provider for simple cases, but the provider is needed anyway for context distribution, registry subscription, and inline `translations`. -- *Read `navigator.language`* — reflects the user's browser preference, not the page's declared language. Can be wrong (e.g., a Spanish-language page viewed by a French user). - -**Rationale.** The HTML `lang` attribute is the standard mechanism for declaring content language — search engines, screen readers, and spell-checkers already read it. Making the i18n provider honor the same attribute means zero extra config for the common case: set `` once and every player on the page uses Spanish. The provider remains explicit and required; only its locale input becomes optional. React uses `locale` as the prop name (aligning with `Intl` and the rest of the TypeScript surface) while HTML uses the native `lang` attribute. - -### `createI18n()` factory, not direct exports - -**Decision.** A factory that closes over a React context. Returns `{ I18nContext, I18nProvider, useTranslator, useLocale }`. Called once in `@videojs/react/i18n` and re-exported. - -**Alternatives considered.** - -- *Direct module-level exports* — `I18nProvider` and `useTranslator` exported without a factory. Works but commits to a single context instance, preventing independent provider trees. -- *`useTranslator()` with call-site generic* — verbose; every component needs an explicit type argument. - -**Rationale.** The factory pattern allows consumers to create independent provider trees if needed (e.g., two players with different languages and fully isolated contexts). It also keeps the provider and hooks as a cohesive unit — impossible to import `useTranslator` without the matching context. - -### Built-in tooltips: trigger sync, not `` - -**Decision.** Shipped video/audio/live skins use empty `` + `commandfor`; `TooltipElement` syncs translated text from the linked control's label. - -**Alternatives considered.** - -- *`Play` inside each tooltip* — duplicates the control's translation key; two sources of truth for the same string. -- *`` self-translating* — couples tooltip semantics to i18n; translation key becomes a tooltip API concern. -- *Hardcoded English in skin templates* — original v9 pattern; breaks locale switching. - -**Rationale.** Control cores already expose opaque label keys; `MediaButtonElement` and `TooltipElement` both resolve through `I18nController`. One key → one translated string for `aria-label` and tooltip. - -### `` for ejected skin template strings - -**Decision.** A `…` element renders translated text inside shadow DOM templates where no component owns the phrase. - -**Alternatives considered.** - -- *Skin re-renders full template on locale change* — not viable; shadow DOM templates are static HTML cloned once on element creation. - -**Rationale.** `` is the minimal reactive primitive for ad-hoc translated copy in custom/ejected skins. It subscribes to `i18nContext` independently and updates only `textContent` — no parent re-render needed. - -### `{param}` interpolation, not ICU message format - -**Decision.** Simple `{key}` replacement, as in `t('seekForward', { seconds: 10 })`. - -**Rationale.** The player has a small, known set of interpolated strings; none require plural rules at the interpolation site. ICU format requires a runtime library (~20 KB). `{key}` replacement adds zero runtime weight and is established in VJS v8/9. - -### Native `Intl` APIs for locale-aware formatting - -**Decision.** `Intl.DurationFormat` for time phrases, `Intl.NumberFormat` with `style: 'percent'` for volume, and `Intl.PluralRules` via `pluralize` for plural selection. - -**Rationale.** `Intl.DurationFormat` became baseline in 2024 and handles unit labels, pluralization, and locale-specific ordering — zero translation keys. `Intl.NumberFormat` handles percent symbols and digit forms. Only `remainingTimeSuffix` and `volumeSliderValueTextMuted` need translation keys because `Intl` has no concept of those suffixes. - -### Browser Translation API — pre-installed model only - -**Decision.** Only activate when `Translator.availability()` returns `'available'`. Skip silently for all other states. - -**Rationale.** `'downloadable'` triggers a ~100 MB background download without consumer awareness. `'available'` means the model is already on-device — zero cost, free bonus. The feature is not reliable enough to be the primary translation mechanism; built-in locale packs are. - -## Prior Art - -| System | Key type | Distribution | Registry? | -| --- | --- | --- | --- | -| VJS v7/8 | English string | Built-in JSON + consumer | No — set via `videojs.addLanguage` | -| i18next | Opaque string | Consumer JSON | Yes — `i18next.addResourceBundle` | -| FormatJS / react-intl | Opaque string | Consumer JSON | No — prop-based | -| GNU gettext | English string | `.po` files | No | -| Android `strings.xml` | Opaque XML ID | Resource files | No | -| iOS `NSLocalizedString` | Opaque string | `.strings` files | No | -| Media Chrome | English string | Side-effect `import "media-chrome/lang/es.js"` | Yes — auto-registered on import; `lang` attribute on `` | - -VJS v8's `addLanguage` is the closest precedent. This design replaces English-as-key (VJS v7/8) with opaque keys (consistent with i18next/Android/iOS) while keeping the imperative registration API. - -## Edge Cases - -**Multiple players, different languages.** Each `` or `` scopes its locale independently. The registry holds all registered locales simultaneously — providers read from it without interfering with each other. An explicit `locale` prop (React) or `lang` attribute (HTML) on the provider overrides any ambient ``. - -**`registerI18n` called after element mounts.** `` subscribes to `onI18nRegistryChange`. Calling `registerI18n` after mount triggers a re-render of all descendant elements. - -**`` changed at runtime.** Both HTML and React providers observe `document.documentElement` via `MutationObserver`. Dynamic locale switching works without remounting — update `document.documentElement.lang` and all providers without an explicit override re-render. - -**SSR flash-of-English.** React `I18nProvider` starts with `builtIn: {}` — first render is English unless `translations` is passed. Import the locale module directly and pass as `translations` for zero flash (see SSR section above). - -**`en.ts` bundle footprint.** `en.ts` is imported as a module-level side effect of `@videojs/core/i18n`. Consumers who never use i18n should not import this module — they will not pay the cost if they use only `@videojs/core/ui` or similar. - -**When to use ``.** Prefer control `label` + trigger-synced tooltips for anything tied to a button. Use `` only for standalone copy in ejected skins (help text, custom dialog bodies) where no component exposes `getLabel()`. - -## Descoped - -- **Right-to-left layout** — RTL support (mirroring, bidirectional text) is a separate concern from string translation and is not covered here. -- **Plural forms in translation strings** — e.g., `{count, plural, one {# result} other {# results}}`. ICU format is not supported. Skin authors who need locale-correct plurals use the `pluralize` utility from `@videojs/utils/i18n`. -- **Locale negotiation** — determining which locale to use from `Accept-Language`, cookies, or user preference is left to the consumer. Video.js accepts an explicit locale value. -- **Caption auto-selection by locale** — matching a `textTrack` against the active locale and toggling it on is a separate concern (application vs. user vs. stream-default language preferences all overlap here). Can be revisited as its own feature. -- **Translation memory / CAT tools** — locale files are TypeScript source; a compile step to JSON for CAT tooling compatibility is not provided. -- **Translator perf optimizations** — `createTranslator` re-parses `{param}` templates on every call via `String.prototype.replace`. Future work, not required for v1: - - *Run-time* — parse each template once on first use, cache tokens keyed by source string, reuse thereafter (similar to i18next's format cache). - - *Build-time* — AOT-compile locale entries into functions (`seekForward: ({ seconds }) => \`Seek forward ${seconds} seconds\``); inline static keys with dead-code elimination for the active locale (similar to`babel-plugin-formatjs` / `lingui`). -- **Custom translator implementations** — a consumer-supplied `translate(key) → string` override (e.g. to delegate to `i18next` or FormatJS). Deferred follow-up; use `registerI18n` or React `translations` prop today. - -## File Structure - -``` -packages/ -├── core/src/core/i18n/ -│ ├── types.ts ← Locale, Translations, Translator -│ ├── translator.ts ← createTranslator -│ ├── browser-translation.ts ← getBrowserTranslations, shouldAttemptBrowserTranslation -│ ├── load-locale.ts ← loadLocale (codegen lazy import map) -│ ├── registry.ts ← registerI18n, getI18nTranslations, onI18nRegistryChange -│ ├── index.ts ← re-exports -│ └── locales/ -│ ├── en.ts ← English defaults (pre-registered at module init) -│ ├── ar.ts … zh.ts ← built-in locale packs (default export each) -│ ├── all.ts ← aggregated map `{ all, localeTags }` (generated; loads every pack) -│ └── *.cdn.ts ← CDN self-registering entry points (one per locale) -│ -├── react/src/i18n/ -│ ├── create-i18n.tsx ← createI18n -│ ├── locales/ ← generated re-exports of core/locales/*.ts -│ └── index.ts ← public entry: registerI18n, I18nProvider, useTranslator, useLocale, Translations, Translator, Locale -│ -└── html/src/i18n/ - ├── create-i18n.ts ← createI18n → { context, I18nController, ProviderMixin, TextMixin } - ├── locales/ ← generated re-exports of core/locales/*.ts - ├── define/ - │ ├── media-i18n.ts ← ProviderMixin(ReactiveElement) + customElements.define - │ └── media-text.ts ← TextMixin(ReactiveElement) + customElements.define - └── index.ts ← public entry: registerI18n, Translations, Translator, Locale (side-effect registers elements) - -packages/utils/src/i18n/ -└── pluralize.ts - -packages/utils/src/time/ -└── format.ts ← formatDuration, TimeFormatOptions, TimeTranslate -``` - -**Adding a locale:** author `packages/core/src/core/i18n/locales/{tag}.ts`, append the tag to `locales.ts`, then run `pnpm -F @videojs/core generate:locales` (also runs on `@videojs/core` prebuild). That regenerates `locales/all.ts` and html/react re-export stubs. - -### Modified files - -| File | Change | -| ---- | ------ | -| `packages/core/src/core/ui/seek-button/seek-button-core.ts` | `getLabel` returns `'seekForward'`/`'seekBackward'`; add `getLabelParams` | -| `packages/core/src/core/ui/playback-rate-button/playback-rate-button-core.ts` | Same | -| `packages/utils/src/time/format.ts` | Add `TimeTranslate`, `TimeFormatOptions`, optional `translate` param | -| `packages/html/src/ui/media-button-element.ts` | Add `I18nController`; apply `t()` in `update()` | -| `packages/html/src/ui/tooltip/tooltip-element.ts` | Sync tooltip text from trigger `getResolvedLabel()` | -| `packages/core/src/core/i18n/browser-translation.ts` | Browser Translation API fallback + `shouldAttemptBrowserTranslation` | -| `packages/react/src/i18n/index.ts` | Export `{ I18nProvider, useTranslator, useLocale }` from `createI18n()` | +- Core registry, resolution, formatting, locale loading, and tests: `packages/core/src/core/i18n/` +- HTML provider, controller, text integration, and tests: `packages/html/src/i18n/` +- React provider, hooks, and tests: `packages/react/src/i18n/` +- Generated locale inputs and output task: `packages/core/src/core/i18n/locales.ts` and `packages/core/scripts/generate-i18n-locales.ts` +- Public package behavior: package exports, README files, and generated API reference diff --git a/internal/design/spf/actor-reactor-factories.md b/internal/design/spf/actor-reactor-factories.md index 402a80d5..99292223 100644 --- a/internal/design/spf/actor-reactor-factories.md +++ b/internal/design/spf/actor-reactor-factories.md @@ -3,669 +3,34 @@ status: implemented date: 2026-04-03 --- -# Actor and Reactor Factories +# Actor and reactor factories -Design for `createMachineActor` and `createMachineReactor` — the declarative factory functions that replace -bespoke Actor classes and function-based Reactors in SPF. +This record explains why SPF uses declarative machine factories. The implementations and their tests define the current types, handlers, scheduling, and cleanup behavior. -Motivated by the text track architecture spike (videojs/v10#1158), which produced the first -`createMachineActor` / `createMachineReactor`-based implementations in SPF and surfaced the need for -shared, principled primitives. See [text-track-architecture.md](text-track-architecture.md) -for the reference implementation and spike assessment. +## Problem ---- +Early SPF actors and reactors repeated state-signal, lifecycle, runner, and destruction mechanics. A shared abstraction needed to remove that boilerplate without forcing all work into one class hierarchy or erasing the distinction between message-driven actors and signal-driven reactors. -## Decision +## Decisions -Actors and Reactors are defined via a **declarative definition object** passed to a factory -function. The factory constructs the live instance — managing the state signal, runner -lifecycle, and `'destroyed'` terminal state. Consumers define behavior; the framework handles -mechanics. +- Use factory functions with definition objects, not base classes. Definitions describe behavior; factories own snapshots, transitions, lifecycle, and cleanup. +- Keep `createMachineActor` and `createMachineReactor` separate. Their inputs and work models differ enough that a unified optional-property shape would weaken type guarantees. +- Add the terminal `destroyed` state implicitly and enforce it in the framework. Domain definitions cannot omit or redefine destruction semantics. +- Give an actor runner actor-lifetime scope. The factory constructs and destroys it, while per-state handlers decide what work to schedule. +- Register reactor monitors before state effects. The factory guarantees that its monitors resolve transitions before active-state effects run, even though general signal-effect ordering is not a platform guarantee. +- Put message handlers and settling transitions on individual states so valid state/message combinations remain visible in the definition. +- Distinguish one-time, automatically untracked `entry` work from reactive `effects` that rerun with their dependencies. -Two separate factories: +## Consequences -```typescript -const actor = createMachineActor(actorDefinition); -const reactor = createMachineReactor(reactorDefinition); -``` +The factories make state machines inspectable and consistent while allowing lightweight callback or transition actors when a full machine is unnecessary. Async work remains explicit and owned by the actor rather than hidden in generic invoked services. -Both return instances that implement `SignalActor` and expose `snapshot` and `destroy()`. +A state-scoped runner remains a possible extension if actor-lifetime cancellation proves too coarse. It should be added only from demonstrated implementation pressure. -A third factory, `createTransitionActor`, handles actors with reactive context but no -FSM. Lightweight callback actors implement the `CallbackActor` interface directly. +## Current sources of truth -### Actor and Reactor Types - -| Factory | States | Reactive? | Runner | Use when | -|---|---|---|---|---| -| `createMachineActor` | User-defined FSM | Yes (`snapshot`) | Optional | Per-state message dispatch, `onSettled`, async work | -| `createTransitionActor` | `active` / `destroyed` | Yes (`snapshot`) | No | Reactive context via reducer, no FSM needed | -| `CallbackActor` (manual) | None | No | Manual | Fire-and-forget messages, minimal overhead | -| `createMachineReactor` | User-defined FSM | Yes (`snapshot`) | No | Signal-driven transitions, per-state effects | - -**Actors** (message-driven): -- **`MessageActor`** — returned by `createMachineActor`. Has finite states, per-state - handlers, optional runner, and reactive `snapshot` with `value` + `context`. - Used by: `SourceBufferActor`, `SegmentLoaderActor`. -- **`TransitionActor`** — returned by `createTransitionActor`. Pure reducer model: - `(context, message) => context`. No finite states — `snapshot.value` is always - `'active' | 'destroyed'`. Reactive context for downstream consumers. - Used by: `TextTracksActor`. -- **`CallbackActor`** — manual implementation. `send()` + `destroy()`, no snapshot. - Used when the actor needs no reactive state and the overhead of a factory isn't - warranted. Used by: `TextTrackSegmentLoaderActor`. - -**Reactors** (signal-driven): -- **`Reactor`** — returned by `createMachineReactor`. Has finite states, `monitor` for - state derivation, and `entry`/`effects` per-state effects. No `send()` — driven - entirely by signal observation. - Used by: `syncTextTracks`, `loadTextTrackCues`, `resolvePresentation`, `trackPlaybackInitiated`. - ---- - -## Actor Definition - -### Shape - -```typescript -type ActorDefinition< - UserState extends string, - Context extends object, - Message extends { type: string }, - RunnerFactory extends (() => RunnerLike) | undefined = undefined, -> = { - runner?: RunnerFactory; // factory — called once at createMachineActor() time - initial: UserState; - context: Context; - states: Partial, - ctx: HandlerContext - ) => void; - }; - }>>; -}; - -// runner is present and typed as the exact runner only when runner: is declared. -// When omitted, runner is absent from the type entirely (not undefined). -type HandlerContext = { - transition: (to: UserState) => void; - context: Context; // snapshot at dispatch time — stale after any setContext call - getContext: () => Context; // live untracked read — always current - setContext: (next: Context) => void; -} & (RunnerFactory extends () => infer R ? { runner: R } : object); -``` - -### Example — `SourceBufferActor` - -Serializes SourceBuffer operations. Shows `onSettled` for auto-return, an `onMessage` -helper to deduplicate handlers, `batch` for atomic multi-message dispatch, and `cancel` -in the work state. Tasks return the next context — `getContext` threading ensures each -task reads the context committed by the previous task. - -```typescript -const onMessage = (msg: IndividualSourceBufferMessage, { transition, setContext, getContext, runner }: Ctx): void => { - transition('updating'); - const task = messageToTask(msg, { getContext, sourceBuffer, setContext }); - runner.schedule(task).then(setContext, handleError); -}; - -return createMachineActor SerialRunner>({ - runner: () => new SerialRunner(), - initial: 'idle', - context: { segments: [], bufferedRanges: [], initTrackId: undefined }, - states: { - idle: { - on: { - 'append-init': onMessage, - 'append-segment': onMessage, - remove: onMessage, - batch: (msg, { transition, setContext, getContext, runner }) => { - if (msg.messages.length === 0) return; - transition('updating'); - msg.messages.forEach((m) => { - const task = messageToTask(m, { getContext, sourceBuffer, setContext }); - runner.schedule(task).then(setContext, handleError); - }); - }, - }, - }, - updating: { - onSettled: 'idle', - on: { - cancel: (_, { runner }) => { runner.abortAll(); }, - }, - }, - }, -}); -``` - -### Example — `SegmentLoaderActor` - -Plans and executes segment fetches. Shows context threading (`inFlightInitTrackId`, -`inFlightSegmentId`), continue/preempt decision in the `loading` handler, and -`abortPending()` vs `abortAll()` for fine-grained runner control. - -```typescript -return createMachineActor SerialRunner>({ - runner: () => new SerialRunner(), - initial: 'idle', - context: { inFlightInitTrackId: null, inFlightSegmentId: null }, - states: { - idle: { - on: { - load: (msg, ctx) => { - const allTasks = planTasks(msg); - if (allTasks.length === 0) return; - ctx.transition('loading'); - scheduleAll(allTasks, ctx); - }, - }, - }, - loading: { - onSettled: 'idle', - on: { - load: (msg, ctx) => { - const { context, runner } = ctx; - const allTasks = planTasks(msg); - const inFlightStillNeeded = /* check context against new plan */; - - if (inFlightStillNeeded) { - runner.abortPending(); // continue in-flight - scheduleAll(excludeInFlight(allTasks), ctx); // schedule remainder - } else { - runner.abortAll(); // preempt everything - sourceBufferActor.send({ type: 'cancel' }); - scheduleAll(allTasks, ctx); - } - }, - }, - }, - }, -}); -``` - ---- - -## Reactor Definition - -### Shape - -```typescript -type ReactorDefinition = { - initial: State; - /** - * Cross-cutting monitor — returns the target state. The framework compares - * to the current state and drives the transition. Registered before per-state - * effects — see the monitor-before-state ordering guarantee below. - * - * Accepts a single function or an array of functions. - */ - monitor?: ReactorDeriveFn | ReactorDeriveFn[]; - /** - * Per-state definitions. States with no effects use `{}`. - */ - states: Record; -}; - -/** Returns the target state. Framework drives the transition. */ -type ReactorDeriveFn = () => State; - -type ReactorStateDefinition = { - /** - * Entry effects — run once on state entry, automatically untracked. - * No untrack() needed inside the fn body. Return a cleanup function or - * AbortController to run on state exit. - */ - entry?: ReactorEffectFn | ReactorEffectFn[]; - /** - * Reactive effects — re-run whenever a tracked signal changes while - * this state is active. Return a cleanup to run before each re-run - * and on state exit. - */ - effects?: ReactorEffectFn | ReactorEffectFn[]; -}; - -type ReactorEffectFn = () => (() => void) | { abort(): void } | void; -``` - -### Example — `syncTextTracks` - -Two states (`preconditions-unmet` ↔ `set-up`), one `monitor`, and one `entry` + one -`effects` effect in `set-up` with independent tracking and cleanup. - -```typescript -const reactor = createMachineReactor<'preconditions-unmet' | 'set-up'>({ - initial: 'preconditions-unmet', - // monitor returns the target state; framework drives the transition. - monitor: () => preconditionsMetSignal.get() ? 'set-up' : 'preconditions-unmet', - states: { - 'preconditions-unmet': {}, // no effects — monitor handles exit - - 'set-up': { - // entry: automatically untracked — runs once on state entry. - // Reading mediaElement and modelTextTracks here does NOT create dependencies. - entry: () => { - const el = mediaElementSignal.get() as HTMLMediaElement; - const tracks = modelTextTracksSignal.get() as PartiallyResolvedTextTrack[]; - tracks.forEach(t => el.appendChild(createTrackElement(t))); - return () => { - el.querySelectorAll('track[data-src-track]').forEach(t => t.remove()); - update(state, { selectedTextTrackId: undefined }); - }; - }, - - // effects: re-runs when selectedId changes. el is read with untrack() - // since element changes go through the monitor (preconditions-unmet path). - effects: () => { - const el = untrack(() => mediaElementSignal.get() as HTMLMediaElement); - const selectedId = selectedIdSignal.get(); // tracked — re-run on change - syncModes(el.textTracks, selectedId); - const unlisten = listen(el.textTracks, 'change', onChange); - return () => unlisten(); - }, - }, - }, -}); -``` - -### Example — `loadTextTrackCues` - -Four states with actor lifecycle managed across states, the `deriveState` pattern for -complex multi-condition transitions, and `untrack()` for non-reactive owner reads. - -```typescript -// Hoist computeds outside the reactor — computed() inside an effect body -// creates a new Computed node on every re-run with no memoization. -const derivedStateSignal = computed(() => deriveState(state.get(), owners.get())); -const currentTimeSignal = computed(() => state.get().currentTime ?? 0); -const selectedTrackSignal = computed(() => findSelectedTrack(state.get())); - -const reactor = createMachineReactor({ - initial: 'preconditions-unmet', - monitor: () => derivedStateSignal.get(), - states: { - 'preconditions-unmet': { - // entry: defensive actor reset on state entry (no-op if already undefined). - // Handles all paths back from active states. - entry: () => { teardownActors(owners); }, - }, - - 'setting-up': { - entry: () => { - teardownActors(owners); // defensive — same as preconditions-unmet - const mediaElement = owners.get().mediaElement as HTMLMediaElement; - const textTracksActor = createTextTracksActor(mediaElement); - const segmentLoaderActor = createTextTrackSegmentLoaderActor(textTracksActor); - update(owners, { textTracksActor, segmentLoaderActor }); - // No return — deriveState drives the onward transition automatically. - }, - }, - - pending: {}, // neutral waiting state — no effects - - 'monitoring-for-loads': { - // effects: re-runs whenever currentTime or selectedTrack changes. - // owners is read with untrack() — actor presence is guaranteed by - // deriveState when in this state; actor snapshot changes must not - // re-trigger this effect. - effects: () => { - const currentTime = currentTimeSignal.get(); // tracked - const track = selectedTrackSignal.get()!; // tracked - const { segmentLoaderActor } = untrack(() => owners.get()); - segmentLoaderActor!.send({ type: 'load', track, currentTime }); - }, - }, - }, -}); -``` - ---- - -## Key Design Decisions - -### Factory functions, not base classes - -**Decision:** `createMachineActor(def)` and `createMachineReactor(def)` rather than `extends BaseActor` / -`extends Reactor`. - -**Alternatives considered:** -- **Base class + subclass** — `class TextTracksActor extends BaseActor<...>`. Familiar OO pattern, - explicit contract. But inheritance couples the consumer to the framework's class hierarchy, - limits composition, and makes the definition implicit (spread across the constructor body). -- **Interface only** — each Actor/Reactor implements `SignalActor` directly. No boilerplate - reduction; every implementation reimplements the same snapshot/signal/destroy mechanics. - -**Rationale:** A definition object is pure data — inspectable, serializable, testable in isolation -without instantiation. The factory owns all mechanics (snapshot signal, runner lifecycle, -`'destroyed'` guard); the definition owns behavior. Aligns with the XState model and keeps the -door open for a future definition-vs-implementation separation (see below). - ---- - -### Separate `createMachineActor` and `createMachineReactor` - -**Decision:** Two distinct factories with distinct definition shapes. - -**Alternatives considered:** -- **Unified `createMachine`** — one factory for both, distinguishing by definition shape (Actors - have `on`/`runner`; Reactors have effect arrays). XState does this. - -**Rationale:** Actors and Reactors have genuinely different input shapes and internal mechanics. -A unified factory would produce a definition type with optional properties for both cases, -losing type-level guarantees (e.g., a Reactor definition shouldn't have `runner` or `on`). -The shared core — state signal, `'destroyed'` terminal, `destroy()` — is thin enough to -extract as an internal `createMachineCore` without a unified public API. XState unifies because -its actors ARE the reactive graph; in SPF, the separation between reactive observation (Reactor) -and message dispatch (Actor) is intentional and worth preserving in the API surface. - ---- - -### `'destroyed'` is implicit and always enforced - -**Decision:** User-defined state types never include `'destroyed'`. The framework always adds it -as the terminal state. `destroy()` on any Actor or Reactor always transitions to `'destroyed'` -and calls exit cleanup for the currently active state. - -```typescript -// User defines: -type LoaderUserState = 'idle' | 'loading'; -// Framework produces: -type LoaderState = 'idle' | 'loading' | 'destroyed'; -``` - -**Rationale:** `'destroyed'` is universal — every Actor and Reactor has it. Making it implicit -ensures it can't be accidentally omitted or given a custom behavior that breaks framework -guarantees (e.g., `send()` being a no-op in the destroyed state). Users only define their -domain-meaningful states. - ---- - -### Runner as a factory function, actor-lifetime scope - -**Decision:** `runner: () => new SerialRunner()` — a factory function called once when -`createMachineActor()` is called. The runner lives for the actor's full lifetime and is destroyed -when the actor is destroyed. - -**Alternatives considered:** -- **Magic strings** (`runner: 'serial'`) — requires a string-to-class registry and introduces an - extra import layer. Deferred to a possible future XState-style definition-vs-implementation - split. -- **Constructor reference** (`runner: SerialRunner`) — `new def.runner()`. Slightly less explicit - than a factory; doesn't compose as naturally when construction needs configuration. -- **State-lifetime runners** — runner created on state entry, destroyed on state exit. Naturally - eliminates the generation-token problem (`onSettled` always refers to the fresh chain), and - aligns with XState's `invoke` model where async work is tied to the state that started it. - Not adopted as the default because `TextTrackSegmentLoaderActor` intentionally persists runner - state across idle/loading cycles. But this is worth revisiting per-actor — see - [Open Questions](#state-scoped-runner). - -**Rationale:** Actor-lifetime scope matches the current pattern and is the most flexible default. -A factory function (`() => new X(options)`) handles configured runners without changing the -framework. The generation-token problem (`onSettled` must refer to the latest chain, not a -stale one) is handled by the framework internally rather than by runner scope. - ---- - -### `monitor`-before-state ordering guarantee - -**Decision:** `monitor` effects are registered before per-state effects in `createMachineReactor`. -This ordering is **load-bearing**: per-state effects can rely on invariants established by -`monitor` having already run. - -**How it works:** The effect scheduler drains pending computeds into an insertion-ordered -`Set` before executing them. Because `monitor` effects are registered first, they are -guaranteed to execute before per-state effects in every flush. - -**What this enables:** When a `monitor` fn returns a new state, `createMachineReactor` calls -`transition()` immediately and updates the snapshot signal. By the time per-state effects run, -the reactor is already in the new state — so a per-state effect gated on -`snapshot.value !== state` correctly no-ops without needing to re-check conditions that the -`monitor` just resolved. - -**Important caveat:** This guarantee is specific to `createMachineReactor`'s registration order. -It is not a formal guarantee of the TC39 Signals proposal — it depends on the polyfill's -`Watcher` implementation preserving insertion order in `getPending()`. Do not assume this -ordering holds outside of `createMachineReactor`. See [signals.md § Effect Execution Order](signals.md) -for the general principle. - ---- - -### Per-state `on` handlers - -**Decision:** Message handlers are declared per state. The same message type can appear in -multiple states with different behavior. - -```typescript -states: { - idle: { on: { load: (msg, ctx) => { /* plan + schedule; transition → loading */ } } }, - loading: { on: { load: (msg, ctx) => { /* abort + replan; stay loading */ } } } -} -``` - -**Alternatives considered:** -- **Top-level `on`** with internal state guard — one handler per message type, branches on - `context.state` internally. More compact for simple cases, but hides state-dependent - behavior in imperative branches rather than making it explicit in the definition. - -**Rationale:** Matches XState's model. State-scoped handlers make valid message/state combinations -explicit and inspectable from the definition alone — no need to trace imperative branches. - ---- - -### `onSettled` at the state level - -**Decision:** Each state can declare `onSettled: 'targetState'`. When the actor's runner settles -(all scheduled tasks have completed) while the actor is in that state, the framework automatically -transitions to `targetState`. - -**Rationale:** The framework owns the generation-token logic — re-subscribing to -`runner.whenSettled()` each time the handler returns so that `abortAll()` + reschedule -correctly supersedes the previous settled callback. Both `SourceBufferActor` and -`SegmentLoaderActor` use `onSettled: 'idle'` to auto-return from their work states. - ---- - -### `entry` vs `effects` per-state effects - -Per-state effects fall into two distinct categories, each with its own key in the state definition: - -- **`entry`** — run once on state entry, **automatically untracked**. No `untrack()` needed inside - the fn body. Use for one-time setup: creating DOM elements, reading `owners`, starting a fetch. - Return a cleanup function or `AbortController` to run on state exit (or re-entry if the effect - runs again). -- **`effects`** — intentionally re-run when a tracked signal changes while the state is active. - Use for effects that must stay in sync with reactive data: mode sync, message dispatch. - -Signals that should not trigger re-runs in a `effects` effect must be wrapped with `untrack()`. -Signal reads inside `entry` are automatically untracked — the fn body runs inside `untrack()`. - -**Inline computed anti-pattern:** `computed()` inside an effect body creates a new `Computed` -node on every re-run with no memoization. `Computed`s that gate effect re-runs must be hoisted -*outside* the effect body (typically at the factory function scope, before `createMachineReactor()`). - ---- - -## XState Comparison - -### Definition vs. Implementation - -The current design uses a single definition object that contains both structure (states, runner -type, initial state) and behavior (handler functions). XState v5 separates these: - -```typescript -// Definition — pure structure, no runtime dependencies -const def = setup({ actors: { fetcher: fetchActor } }).createMachine({ ... }); - -// Implementation — runtime wiring -const actor = createMachineActor(def, { input: { ... } }); -``` - -This separation enables serialization, visualization, and testing the definition without -instantiation. SPF's current factory approach is compatible with this future direction: -`runner: () => new SerialRunner()` today becomes a named reference resolved against a provided -implementation map later. The migration path is additive — no existing definitions need to change. - -#### Handler context API - -The second argument to Actor message handlers is: -```typescript -{ - transition: (to: UserState) => void; - context: Context; // snapshot at dispatch time — stale after any setContext call - getContext: () => Context; // live untracked read — always current - setContext: (next: Context) => void; -} - & (RunnerFactory extends () => infer R ? { runner: R } : {}) -``` - -`runner` is present and typed as the exact runner instance *only* when the definition -declares a `runner` factory. When no runner is declared, `runner` is absent from the type -entirely (not `undefined` — it simply doesn't exist). This is enforced at the type level via -conditional intersection. - -`context` vs `getContext`: use `context` for synchronous logic that runs in the handler body -itself (dispatch time). Use `getContext` when passing it to tasks scheduled on the runner — -async tasks execute after the handler returns, by which point `context` may be stale (e.g. a -previous task in a batch has already called `setContext`). Passing `getContext` ensures each -task reads the context committed by the task before it, making `workingCtx` threading -unnecessary for sequential operations. - ---- - -### Async Work Model: Where Does Work "Belong"? - -This is the most significant behavioral divergence from XState, with real tradeoffs in both -directions. - -#### The SPF pattern - -In SPF, when an actor like `SourceBufferActor` receives an `append-init` message while `idle`, -the `idle` handler does three things: transitions to `updating`, schedules the work on the -runner, and registers callbacks to update context and settle back to `idle` via `onSettled`. - -```typescript -idle: { - on: { - 'append-init': (msg, { transition, setContext, runner }) => { - transition('updating'); // 1. route - const task = makeTask(msg); - runner.schedule(task).then(setContext); // 2. start work - // 3. onSettled: 'idle' in updating handles the return - } - } -}, -updating: { onSettled: 'idle' } -``` - -The work starts in the `idle` handler and finishes in `updating` via `onSettled`. Two things -happen in separate microtasks: `setContext` (from the task's `.then()`), then `transition('idle')` -(from `onSettled`). Observers see two emissions: `{ value: 'updating', context: newCtx }` followed -by `{ value: 'idle', context: newCtx }`. - -#### The XState pattern - -In XState, `idle` *only routes* — the work belongs to the state that is doing it: - -```typescript -idle: { - on: { 'append-init': { target: 'updating' } } // just routing -}, -updating: { - invoke: { - src: 'executeMessage', - input: ({ event }) => event, // the triggering event travels with the transition - onDone: { - target: 'idle', - actions: assign(({ event }) => event.output) // context + state update, atomically - } - } -} -``` - -`updating` invokes the work on entry, using the event that caused the transition as input. -When the work completes, `onDone` updates context and transitions state in one atomic step — -one emission: `{ value: 'idle', context: newCtx }`. - -#### Consequences - -**Atomicity.** XState's `onDone` updates context and state together; SPF does it in two -microtasks. Currently harmless — all consumers wait for `idle` before reading context — but -it's load-bearing discipline rather than a model guarantee. - -**Lifecycle scoping.** In XState, when the machine leaves `updating` for any reason (a -`cancel` event, `destroy()`, etc.), the invoked service is cancelled automatically. In SPF, the -runner outlives any particular state. Cancellation is handled explicitly — via -`runner.abortAll()` / `runner.abortPending()` in message handlers and a first-class `cancel` -message on `SourceBufferActor`. `SegmentLoaderActor`'s `loading.on.load` handler encodes the -preempt/continue decision explicitly rather than through automatic state-exit cleanup. - -**Partial / streaming updates.** SPF calls `setContext` — a closure callback that writes -context directly from inside the task, bypassing the event system. XState's equivalent is the -invoked service sending intermediate events back to the machine -(`sendBack({ type: 'CHUNK', data })`), which trigger context-updating transitions while the -machine stays in `updating`. More ceremony, but each intermediate state is a proper -event-driven transition — observable, testable, guarded. - -**State graph scalability.** With two states the differences are manageable. If the actor grew -to handle `errored`, `draining`, or `quota-exceeded` states, the XState model scales cleanly — -each state owns its behavior, and leaving any state cancels its work. The SPF model requires -increasingly careful manual management as the graph grows. - -#### Tradeoffs of adopting the XState model - -The XState approach is not strictly better. The costs: - -- **The runner doesn't go away.** `SerialRunner`'s serial queuing and abort semantics don't - exist in XState's `invoke` primitive. The runner would move inside the invoked service rather - than being eliminated. The benefit is lifecycle scoping, not simplification. -- **The dispatch table is the same either way.** Whether messages are routed in the `idle` - handler or via `input: ({ event }) => event` in `updating`'s invoke, the `messageToTask` - dispatch exists in both models. Location changes, not complexity. -- **Partial updates as events adds ceremony for a narrow case.** `setContext` fires once - per streaming segment when the first chunk lands. Modeling it as machine events means the - `updating` state handles task-internal events alongside external messages, with every chunk - going through the full dispatch loop. Heavy machinery for one operation type. -- **TypeScript complexity.** With multiple message types all targeting `updating`, the invoke - `input` type is a union and the service must discriminate on `event.type`. - -#### Middle ground: state-scoped runner - -The most targeted improvement would be making the runner *state-scoped* — created on entry to -`updating`, destroyed on exit — without adopting the full `invoke` model. This was explored -and deferred; see [Open Questions](#state-scoped-runner). - ---- - -## Open Questions - -### State-scoped runner {#state-scoped-runner} - -The current actor-lifetime runner means work scheduled in `updating` completes regardless of -subsequent state transitions. For `SourceBufferActor`, this is intentional (a physical -SourceBuffer write must be reflected in the model even if a signal fires mid-operation). For -other actors, it's accidental — there's no mechanism to say "if the actor leaves this state, -abandon in-flight work." - -A state-scoped runner would close this gap, but the investigation concluded it requires more -than convention: - -- **Scheduling happens in `idle`, not `updating`.** `idle` handlers transition to `updating` - and then schedule tasks — in the same function body, on the same runner reference. For the - runner to be state-scoped, either `transition()` must return the new state's runner (magic, - rejected), or task inputs must travel through context so that an `onEnter` hook on `updating` - can drain them and do the scheduling there. -- **`onEnter` is a real API addition.** The "entry hook drains context" model is essentially - a lightweight `invoke` — it requires `onEnter` in `ActorStateDefinition`, a per-state - runner factory, and the framework to wire them up on state entry and exit. That's a meaningful - framework change, not a convention. -- **Context as side channel is awkward.** Task inputs (message payloads) traveling through - reactive actor context leaks internal scheduling details into the public snapshot. - -**Decision:** Keep actor-lifetime runners for now. The generation-token problem is already -handled by the framework. The cancellation gap (work outliving its state) is real but not -currently exploited — no actor today has a non-settle path out of its work state. Revisit if -a new actor needs explicit cancellation on state exit, or if the framework grows `onEnter` -support for other reasons (at which point state-scoped runners become straightforward). +- Actor factories and tests: `packages/spf/src/core/actors/` +- Reactor factory and tests: `packages/spf/src/core/reactors/` +- Signal scheduling: `packages/spf/src/core/signals/` +- Usage patterns: `packages/spf/src/playback/actors/` and `packages/spf/src/playback/behaviors/` +- Related rationale: [Signals](signals.md) and [Text track architecture](text-track-architecture.md) diff --git a/internal/design/spf/features/audio-playback.md b/internal/design/spf/features/audio-playback.md index a0b6dabd..86b7d6ee 100644 --- a/internal/design/spf/features/audio-playback.md +++ b/internal/design/spf/features/audio-playback.md @@ -6,245 +6,24 @@ definition: sketched # Audio playback -Today's single-rendition audio playback in the HLS engine. The engine -parses audio renditions from the multivariant playlist, picks one via -the default picker at source load, fetches its media playlist, sets up -an audio `SourceBuffer`, and loads segments. The selected rendition is -locked for the lifetime of the source — mid-stream switching belongs to -[`multi-language-audio`](./multi-language-audio.md) (coarse, not yet -implemented). +Single-rendition audio playback is the audio-specific layer of the HLS engine. Shared MediaSource, buffering, preload, and replacement behavior is recorded by their owning features and implemented in source. -Symmetric with `video-abr` (video) and `subtitles` (text) in the -per-track-type capability set. Most of the implementation surface is -*owned architecturally* by other features (`mse-mms-pipeline`, -`buffer-management`, `source-replacement`); this doc captures what's -audio-specific. +## Implemented decision -## Status +The engine parses audio renditions, resolves one rendition when a source loads, creates the audio buffer path, and loads its segments. Audio is modeled symmetrically with video and text so per-track behavior can compose without a media-type-specific engine architecture. -- **Composition:** `createSimpleHlsEngine` (HLS VoD) -- **Definition depth:** sketched — capability surface documented; - language-aware selection exists in `media/primitives/` but isn't - wired (see *What's not implemented*) +Selection is intentionally stable for the current source. Mid-stream language or rendition switching belongs to [multi-language audio](multi-language-audio.md), and bandwidth-driven selection belongs to [audio ABR](audio-abr.md). -## Phases of complexity +## Boundaries -Capability slices around today's audio playback contract. +- Browser MediaSource owns audio/video synchronization. +- Buffer policy is owned by [buffer management](buffer-management.md). +- MediaSource lifecycle is owned by [the MSE/MMS pipeline](mse-mms-pipeline.md). +- Cleanup across a new source is owned by [source replacement](source-replacement.md). +- Surround and audio-only optimization are independent follow-ups. -| Phase | What | Notes | -|---|---|---| -| Audio rendition recognition | `parseMultivariantPlaylist` surfaces audio renditions from `#EXT-X-MEDIA:TYPE=AUDIO` lines with `language`, `name`, `default`, `autoselect`, `channels`, `codecs`, `uri` | Engine consumes at most one today; parsing handles multiple — the foundation `multi-language-audio` builds on | -| Default audio rendition selection | `selectAudioTrack` runs the default picker once on `'presentation-resolved'` entry. Default picker today is `pickFirstTrackId` (first track in the audio selection set) — language-unaware | Config-overridable via `SelectAudioTrackConfig.picker`. The language-aware `pickAudioTrack` exists in `media/primitives/` but isn't wired — see *What's not implemented* | -| Audio media playlist resolution | `resolveAudioTrack` (sibling of resolveVideoTrack / resolveTextTrack, shared `setupTrackResolution` helper) fetches the selected rendition's media playlist on entry; aborts on source un-resolve via state-bound `AbortController` | Same shape as video / text resolution | -| Audio SourceBuffer + actor setup | `setupAudioBufferActors` creates the audio `SourceBuffer` + `SourceBufferActor` + `SegmentLoaderActor`. Uses plain `fetchStream` (no bandwidth sampling — there's no audio ABR yet) | Owned architecturally by `mse-mms-pipeline`; the Firefox `mozHasAudio` cross-type invariant lives in that feature's documentation | -| Audio segment loading | `loadAudioSegments` dispatches the per-type 4-state segment-load FSM (`'preconditions-unmet' / 'dormant' / 'metadata-only' / 'full-range'`) consuming `(preload, loadActivated)`. Same gate behavior as video | Owned architecturally by `buffer-management` | -| Source-change clearance | On source un-resolve, `selectedAudioTrackId` clears, audio actors tear down, in-flight playlist fetch aborts. New source's audio rendition picked fresh on the next resolve | Owned architecturally by `source-replacement` | +## Current sources of truth -## What's not implemented - -- **Multi-rendition recognition + programmatic selection + mid-stream - switching** — covered by - [`multi-language-audio`](./multi-language-audio.md) (partial, sketched — - Tier 1 + most of Tier 2 implemented). Default selection now uses - `pickAudioTrack`'s three-tier picker (`preferredAudioLanguage` → - `DEFAULT=YES` → first-track), so `preferredAudioLanguage` config - takes effect. Tier 2 programmatic selection via - `userAudioTrackSelection` filter and same-codec mid-stream switching - with next-segment-boundary flush also implemented. Persistence and - A/V sync policy refinements deferred. -- **Audio ABR** — covered by [audio-abr](./audio-abr.md). - `setupAudioBufferActors` uses plain `fetchStream`; the - bandwidth-sampling `createTrackedFetch` isn't wired into the audio - fetch path. The sampling primitive exists (used by video); audio - needs a parallel `switchAudioQuality` behavior to consume it. -- **Channels-aware selection (5.1 / surround)** — covered by - [5.1-surround-selection](./5.1-surround-selection.md). Audio tracks - carry a `channels` field surfaced by the parser, but it's not yet - used for selection or capability filtering. -- **Audio-only composition optimizations** — the engine tolerates - audio-only sources today (basic coverage via `engine.test.ts` - "handles audio-only stream"), but the default engine doesn't - *explicitly* compose for audio-only contexts (no shorter buffer - targets, no subtracted video-related work). The explicit variant - ships via the - [audio-only-mode-override](../use-cases/audio-only-mode-override.md) - use case (Phase 1 landed); detect-from-parser routing in the default - engine to supplant the tolerance is future work tracked there. -- **A/V sync handling** — delegated entirely to the browser's MSE - pipeline. The engine doesn't intervene on audio/video sync. - -## Implementation surface - -**Composition:** `packages/spf/src/playback/engines/hls/engine.ts` — -audio behaviors composed alongside video and text: - -```ts -// Track selection (reads config for initial preferences). -selectAudioTrack, - -// Resolve selected tracks (fetch media playlists) -resolveVideoTrack, -resolveAudioTrack, -resolveTextTrack, - -// ... - -// MSE setup -// ... -setupAudioBufferActors, - -// Segment loading -// ... -loadAudioSegments, -``` - -**Behaviors:** - -| Behavior | File | Responsibility | -|---|---|---| -| `selectAudioTrack` | `packages/spf/src/playback/behaviors/select-tracks.ts` | Default audio rendition selection on source load. Lifecycle-only; mutually exclusive with `switchAudioTrack` | -| `switchAudioTrack` | `packages/spf/src/playback/behaviors/track-switching.ts` | Audio variant of the shared constraint/rule pipeline. **Owned architecturally by [`multi-language-audio`](./multi-language-audio.md)** — composed in both HLS engine variants today | -| `resolveAudioTrack` | `packages/spf/src/playback/behaviors/resolve-track.ts` | Fetches the selected audio media playlist | -| `setupAudioBufferActors` | `packages/spf/src/playback/behaviors/dom/setup-buffer-actors.ts` | Audio SourceBuffer + actor setup. **Owned architecturally by `mse-mms-pipeline`** | -| `loadAudioSegments` | `packages/spf/src/playback/behaviors/dom/load-segments.ts` | Audio segment loading dispatcher. **Owned architecturally by `buffer-management`** | - -**Helpers:** - -| Helper | File | Status | -|---|---|---| -| `pickAudioTrack(presentation, config)` | `packages/spf/src/media/primitives/select-tracks.ts` | **Default picker today** — language-aware three-tier (`preferredAudioLanguage` → `DEFAULT=YES` → first-track). Wired by [`multi-language-audio`](./multi-language-audio.md) Tier 1 | -| `pickFirstTrackId(presentation, 'audio')` | `packages/spf/src/media/primitives/select-tracks.ts` | Simple first-track fallback (still available for callers overriding via `SelectAudioTrackConfig.picker`) | - -**State slots:** - -- `selectedAudioTrackId` — single-writer. Owner depends on which audio- - selection behavior is composed: `selectAudioTrack` (lifecycle-only) or - `switchAudioTrack` (filter-reactive + mid-stream flush; the variant - composed in `createSimpleHlsEngine` and `createHlsAudioOnlyEngine` - today). Becomes `switchAudioQuality`'s responsibility when - [audio-abr](./audio-abr.md) lands (extends `switchAudioTrack`). -- `userAudioTrackSelection` — added by [`multi-language-audio`](./multi-language-audio.md). - Consumer-driven `Partial` filter narrowing the audio - candidate set before `switchAudioTrack`'s picker runs. -- Reads `presentation` (audio renditions surface in - `presentation.selectionSets`) - -**Manifest parsing:** `parseMultivariantPlaylist` -(`packages/spf/src/media/hls/parse-multivariant.ts`) extracts audio -renditions from `#EXT-X-MEDIA:TYPE=AUDIO` lines. - -## Config surface - -```ts -{ - preferredAudioLanguage?: string; // Exposed but inert with the - // default picker; see What's - // not implemented -} -``` - -Plus the behavior-level override: - -```ts -// SelectAudioTrackConfig (consumed via composition config) -{ - picker?: TrackPicker; // Override default - // pickFirstTrackId -} -``` - -Consumers wanting language-aware selection today must override -`picker` — e.g., to use `pickAudioTrack` from -`@videojs/spf` primitives. - -## Verification - -- **Unit tests:** - - `packages/spf/src/playback/engines/hls/tests/engine.test.ts` → - "orchestrates complete pipeline" — asserts `selectedAudioTrackId` - defined, audio track resolved, audio buffer actor present - - `packages/spf/src/playback/engines/hls/tests/engine.test.ts` → - "handles audio-only stream" — basic audio-only coverage - - `packages/spf/src/playback/behaviors/dom/tests/setup-buffer-actors.test.ts` - — per-type audio setup - - **Coverage gap:** no dedicated `select-tracks.test.ts` coverage - specifically asserting audio picker behavior (the default - `pickFirstTrackId` path is exercised through the orchestration - tests; the `pickAudioTrack` primitive has its own unit tests in - `media/primitives/tests/`) -- **Sandbox:** - - `apps/sandbox/src/spf-segment-loading/` — exercises audio - playback as part of HLS playback - -## Open questions - -- **Audio-only composition guarantees.** The engine *tolerates* - audio-only sources today, but is it *designed* for them? The - [audio-only-mode-override](../use-cases/audio-only-mode-override.md) - use case (Phase 1 landed) supplies the explicit variant; whether - the default engine routes to it for audio-only sources via - detect-from-parser is future work tracked there. -- **Channels exposure vs use.** The parser surfaces `channels` on - audio tracks but no selection logic consumes it. Stays inert until - [5.1-surround-selection](./5.1-surround-selection.md) wires it. - -## Related features - -- **[multi-language-audio](./multi-language-audio.md)** *(partial, sketched)* — - the extension covering multi-rendition surfacing (free via parser), - language-aware default selection (wires `pickAudioTrack`), programmatic - selection via `userAudioTrackSelection` filter, and same-codec mid-stream - switching via next-segment-boundary flush. Today's `audio-playback` - is the baseline it builds on. -- **subtitles** — parallel structure (default picker + per-type - segment loading; same `setupTrackResolution` helper). Text-track - selection is user opt-in by default; audio's default selection - always picks something (or nothing if no audio renditions). -- **video-abr** — video equivalent. Video has ABR overlay; audio - doesn't (yet — see [audio-abr](./audio-abr.md)). -- **mse-mms-pipeline** — owns audio SourceBuffer setup - (`setupAudioBufferActors`). The Firefox `mozHasAudio` cross-type - invariant lives in that feature's documentation. -- **buffer-management** — owns audio segment loading - (`loadAudioSegments`). Same 4-state FSM as video / text. -- **source-replacement** — audio actors and `selectedAudioTrackId` - tear down via the resolved/unresolved cascade. -- **preload-modes** — gates audio loading via the same FSM that - gates video and text. -- **[audio-abr](./audio-abr.md)** — bandwidth sampling + quality - switching for audio. Parallel sibling of video-abr on the audio - axis; consumes audio-playback's single-rendition baseline + (when - it lands) multi-language-audio's rendition-group machinery. -- **[5.1-surround-selection](./5.1-surround-selection.md)** — - channels-aware codec-change selection. Consumer of capability- - probing on the audio channel-count axis. Wires the `channels` field - surfaced by the parser into selection logic. -- **[audio-only-mode-override](../use-cases/audio-only-mode-override.md)** *(use case; Phase 1 landed)* — - engine variant optimized for audio-only delivery; covers both - truly-audio-only sources and mixed sources delivered as audio-only. -- **capability-probing** *(candidate)* — narrows the audio candidate - set selection runs over; unsupported codecs filtered upstream of - the `selectAudioTrack` picker. - -## Use cases that compose this feature - -- **[`audio-only-mode-override`](../use-cases/audio-only-mode-override.md)** - *(partial — Phase 1 landed)* — Phase 1 baseline constituent. The - audio-only delivery variant composes this feature's rendition - selection, media playlist resolution, and segment loading as-is via - `createHlsAudioOnlyEngine`. - -## See also - -- [presentation-modeling.md](../presentation-modeling.md) — - architectural deep-dive on the format-neutral data shape and parser - interface that surfaces audio renditions (the `parsePresentation` - contract this feature's recognition phase relies on) -- [multi-language-audio.md](./multi-language-audio.md) — future - extension on top of this feature -- [clusters.md § Track & variant registry](./clusters.md#track--variant-registry) -- [conventions/behaviors.md](../conventions/behaviors.md) — per-type - specialization pattern (`setupTrackResolution`) -- [packages/spf/docs/hls-engine.md](../../../../packages/spf/docs/hls-engine.md) - — engine composition walkthrough (Stage 2: track selection; Stage 3: - track resolution) +- HLS engine composition and adapter tests: `packages/spf/src/playback/engines/hls/` +- Buffer and segment actors: `packages/spf/src/playback/actors/dom/` +- Track resolution and selection: `packages/spf/src/playback/behaviors/` diff --git a/internal/design/spf/features/buffer-management.md b/internal/design/spf/features/buffer-management.md index 5a787da1..858b990c 100644 --- a/internal/design/spf/features/buffer-management.md +++ b/internal/design/spf/features/buffer-management.md @@ -6,292 +6,25 @@ definition: sketched # Buffer management -The engine's segment-level buffer policy: deciding which segments to -load, which to evict, how to handle the buffered ranges during seeks -and quality switches. Covers the per-type segment-loader actors that -plan and execute fetches, the forward-buffer / back-buffer policy -primitives, and the dispatcher behaviors that turn gate state + -currentTime + selected tracks into `load` messages. +Buffer management is the segment-level policy between loading gates and MediaSource operations. -Sits between `preload-modes` (produces the gate state this feature -consumes) and `mse-mms-pipeline` (owns the `SourceBufferActor` this -feature's operations land on). +## Implemented decisions -This doc captures the **capability surface**: what works, what doesn't, -which behaviors / actors / primitives implement it, and how -seek + playback-rate behavior emerges from the planner's design. +- Plan a bounded amount of media ahead of current time and evict old back-buffer data. +- Deduplicate work by media type, segment, and selected quality while still permitting gap repair and seek-back recovery. +- Continue useful in-flight work and preempt work that no longer serves the current plan. +- Stream media chunks to the buffer actor while treating initialization data atomically. +- Give each media type a shared loading gate derived from preload and source state. +- Keep planning policy separate from the SourceBuffer actor that serializes browser operations. -## Status +These choices make seeks, quality changes, and playback-rate changes inputs to the same planner rather than special loading pipelines. -- **Composition:** `createSimpleHlsEngine` (HLS VoD) -- **Definition depth:** sketched — capability surface and implementation - footprint documented; richer forward-buffer / back-buffer policies and - the codec-change extension (`changeType()`) are tracked as candidates +## Deferred scope -## Phases of complexity +Rate- or network-aware buffer targets, learned quota policy, loop anticipation, cross-codec `changeType()`, and a public buffer-health model require separate evidence and design. -What's implemented today, organized as capability slices around the -engine's buffer policy. Phases are non-orthogonal in places — seek -handling depends on forward-flush; quality-aware planning depends on -the in-flight tracking — but each row is a slice that earns its place. +## Current sources of truth -| Phase | What | Notes | -|---|---|---| -| Forward-buffer planning | Engine maintains ~30 seconds ahead of the playhead via `getSegmentsToLoad`. Re-plans on segment-boundary crossings (debounced via `segmentStartForTime` — within-segment `currentTime` ticks don't trigger re-planning). Configurable via `forwardBuffer.bufferDuration` | Default 30 s; threaded through both dispatcher (range calc) and actor (planner) | -| Back-buffer eviction | Engine evicts older buffered ranges, keeping `keepSegments` behind the playhead. Bounded memory regardless of session length | Default `keepSegments: 2`; v/a actor only — text-track loader doesn't evict | -| Quality-aware buffer planning | Skip re-loading a segment if equal-or-higher-quality content is already buffered at the same time slot. Compares `trackBandwidth` on actor-tracked segments | Preserves buffered high-quality content during ABR downgrade; replans on upgrade | -| Seek handling | Non-contiguous buffer planning (gap-filling for the `[currentTime, currentTime + bufferDuration)` window) + forward-flush after seek-back. The forward-flush prevents unbounded scattered buffer that can cause `QuotaExceededError` on long-form content | Works because the planner is currentTime-driven and operates over actor-tracked buffered segments rather than `SourceBuffer.buffered` ranges directly | -| In-flight continue / preempt | New `load` message during `'loading'` — if the in-flight segment / init is still needed by the new plan (same track + same id), continue (`abortPending` + schedule remainder); otherwise preempt (`abortAll` + cancel SourceBuffer if needed + replan) | Driven by `inFlightInitTrackId` + `inFlightSegmentId` in actor context; the SourceBuffer cancel is conditional to avoid clobbering an init append that's still needed | -| Streaming append for media segments | Media segment body stream passed to `SourceBufferActor` as it arrives — chunks appended in flight. Init segments accumulate fully before append (`minChunkSize: Infinity`) | Affects time-to-first-frame for the first segment; init must land complete before media-segment chunks can be parsed | -| Per-type segment loading | Video / audio / text dispatcher variants share `setupSegmentLoading` helper + (`preload`, `loadActivated`) → FSM mapping. V/A share `SegmentLoaderActor`; text uses `TextTrackSegmentLoaderActor` (no MSE buffer, no codec, no append distinction) | Cross-type constraint: all three variants honor the same gate FSM, so preload behavior is uniform across rendition types | -| Playback rate | `playbackRate ≠ 1` works (the engine doesn't break) but the forward buffer is not rate-aware — 30 s of media at 2× rate is 15 s of wall-time runway | Today: basic. Consumers can manually override `forwardBuffer.bufferDuration` for higher rates; engine-side adaptation is open work — see *Smarter forward-buffer sizing* below | - -## What's not implemented - -Today's policy choices are simple-by-design; richer policies are tracked -here as candidate phases. The list will grow as motivating use cases -surface. - -- **Smarter forward-buffer sizing** — fixed `bufferDuration` regardless - of context. Three known motivations for a richer policy: - - **Playback-rate-aware** — at 2× rate, 30 s of media is 15 s of - wall-time runway; at 0.5× rate, 60 s (wasteful). - - **`canplaythrough`-equivalent** — bigger buffer when network - throughput is high enough to "playthrough to end without rebuffer." - Mirrors HTMLMediaElement's `canplaythrough` event semantics. - - **Network-conditions / seek-density-aware** — longer runway on slow - networks; shorter runway during heavy seeking. - Each variant is a candidate phase row when implemented. - -- **Smarter back-buffer eviction** — fixed `keepSegments` proactive - policy today. Alternative shapes to track: - - **Quota-learning eviction** — record observed `QuotaExceededError` - byte-thresholds and use them as a per-device ceiling for both - eviction *and* forward-buffer fetching. Reference point: - [mux-background-video's `evictBuffer`](https://github.com/muxinc/mux-background-video/blob/main/src/engines/hls-mini/mediasource.ts#L173-L183) - catches the error and evicts past `currentTime - 5 s`, but doesn't - remember the threshold for next time. A smarter variant would track - total buffered bytes; if the error fires at 300 MB but didn't at - 299 MB, future planning keeps the buffer below ~300 MB proactively - — both for back-buffer eviction and to constrain forward-buffer - fetching at long `bufferDuration` / high bitrate. - - **Time-based / size-based** — "keep N seconds" or "keep ≤ M MB" - rather than segment count. - -- **Loop-around buffer fetching for `loop=true`** — when the media - element has `loop=true`, playback wraps from near-end back to 0. The - planner today sees the wrap as a seek-back (gap-fill works) but - doesn't *anticipate* it. A loop-aware policy would pre-fetch the - beginning while still playing the end. - -- **`changeType()` codec-change buffer transitions** — the planner's - Case 1 (removes) deliberately does *not* fire on track switch - ("appending new content overwrites existing buffer ranges, and the - actor's time-aligned deduplication keeps the segment model accurate"). - Same-codec switches work; cross-codec switches don't. - -- **Buffer health observability** — no surfaced state for "buffered - ahead" / "back buffer used." Consumers compute from - `SourceBuffer.buffered` + `state.currentTime` themselves. - -## Implementation surface - -**Composition:** `packages/spf/src/playback/engines/hls/engine.ts` — -dispatchers composed after MSE setup, after `trackCurrentTime` and -`switchVideoQuality`: - -```ts -trackCurrentTime, -switchVideoQuality, -loadVideoSegments, -loadAudioSegments, -// ... -syncTextTracks, -setupTextTrackActors, -loadTextTrackSegments, -``` - -The per-type segment-loader **actors** are owned by the MSE setup -behaviors (`setupVideoBufferActors` / `setupAudioBufferActors` / -`setupTextTrackActors`) — they're constructed there and published on -context for these dispatchers to read. - -**Behaviors:** - -| Behavior | File | Responsibility | -|---|---|---| -| `loadVideoSegments` / `loadAudioSegments` / `loadTextTrackSegments` | `packages/spf/src/playback/behaviors/dom/load-segments.ts` | Per-type dispatcher; shares `setupSegmentLoading` helper. 4-state FSM (`preconditions-unmet` / `dormant` / `metadata-only` / `full-range`) consumes `(preload, loadActivated)` from `preload-modes`. Sends `load` messages to the variant's loader actor | - -**Actors:** - -| Actor | File | Role | -|---|---|---| -| `SegmentLoaderActor` (v/a) | `packages/spf/src/playback/actors/dom/segment-loader.ts` | 3-state (`idle` / `loading` / `destroyed`). Planner produces ordered `LoadTask` list (Case 1 removes / Case 2 init / Case 3 segments). In-flight continue/preempt via `inFlightInitTrackId` + `inFlightSegmentId`. Owns the actual fetch + append-to-SourceBufferActor sequence | -| `TextTrackSegmentLoaderActor` | `packages/spf/src/playback/actors/text-track-segment-loader.ts` | Text-track variant satisfying the same `SegmentLoaderLike` contract for the dispatcher. Different actor shape (no SourceBuffer; appends to `TextTracksActor` cue cache instead) | - -**Policy primitives (DOM-free):** `packages/spf/src/media/buffer/` - -| Module | Exports | Role | -|---|---|---| -| `forward-buffer.ts` | `getSegmentsToLoad`, `calculateForwardFlushPoint`, `segmentStartForTime`, `ForwardBufferConfig` | Forward buffer planning + segment-boundary debounce primitive | -| `back-buffer.ts` | `calculateBackBufferFlushPoint`, `BackBufferConfig` | Back buffer eviction policy | - -**State slots — reads only.** This feature consumes engine state and -emits actor messages; it doesn't write state. - -- Reads: `presentation`, `preload`, `currentTime`, `loadActivated`, - `selectedVideoTrackId` (or audio / text per variant) - -**Context slots — reads only.** - -- Reads: `videoSegmentLoaderActor` / `audioSegmentLoaderActor` / - `textTrackSegmentLoaderActor` (each variant reads only its own type's - loader-actor slot). The actors themselves are owned and written by - `setupVideoBufferActors` / `setupAudioBufferActors` / - `setupTextTrackActors` (under `mse-mms-pipeline` and `subtitles`). - -**Downstream:** `SegmentLoaderActor` sends `append-init` / -`append-segment` / `remove` / `cancel` messages to the -`SourceBufferActor` (`mse-mms-pipeline`). Awaits the actor's `'idle'` -snapshot between operations rather than awaiting `send()` directly. - -## Config surface - -```ts -{ - forwardBuffer?: Partial; - // ForwardBufferConfig: { bufferDuration: number } // default 30 (seconds) - backBuffer?: Partial; - // BackBufferConfig: { keepSegments: number } // default 2 -} -``` - -- `forwardBuffer.bufferDuration` is threaded into the dispatcher - behaviors (for `range` calculation) and into the v/a + text segment- - loader actors (for planner forward-flush + load planning). -- `backBuffer.keepSegments` is threaded into the v/a `SegmentLoaderActor` - only. The text-track loader doesn't perform back-buffer eviction; - cue caches are bounded per-track-lifetime instead. - -## Verification - -- **Unit tests (behaviors):** - - `packages/spf/src/playback/behaviors/dom/tests/load-segments.test.ts` — - dispatcher FSM transitions, range calculation, segment-boundary - debounce, per-type wiring - - `packages/spf/src/playback/behaviors/dom/tests/load-segments-track-switch.test.ts` — - in-flight continue/preempt during track switches (ABR + audio - rendition); covers the v/a `SegmentLoaderActor` through the - dispatcher - - `packages/spf/src/playback/behaviors/dom/tests/track-current-time.test.ts` — - `currentTime` mirroring (precondition input) -- **Unit tests (policy primitives):** - - `packages/spf/src/media/buffer/tests/forward-buffer.test.ts` — - `getSegmentsToLoad`, `calculateForwardFlushPoint`, - `segmentStartForTime` algorithm coverage - - `packages/spf/src/media/buffer/tests/back-buffer.test.ts` — - `calculateBackBufferFlushPoint` algorithm coverage -- **Coverage gap:** no direct `packages/spf/src/playback/actors/dom/tests/segment-loader.test.ts` - for the v/a `SegmentLoaderActor`. The actor's planner, continue/ - preempt logic, and SourceBuffer-cancel decisions are exercised through - the behavior-level tests. Direct actor-level tests would isolate - regressions from dispatcher changes. -- **Sandbox:** - - `apps/sandbox/src/spf-segment-loading/` — main SPF demo; exercises - forward-buffer, back-buffer, seek, and ABR-driven quality switches - end-to-end - -## Open questions - -- **Quality-aware filter granularity.** The filter is "≥ bandwidth" — - it preserves any same-bandwidth or higher-bandwidth content. Should - it also consider codec / rendition identity (e.g., HEVC and H.264 at - the same bandwidth might both be "≥" by bps but represent different - rendition choices)? Today's assumption: bandwidth is the canonical - quality comparator. -- **`LoadTask` / `Task` naming collision.** The actor file's own - `@todo` flags: "LoadTask risks confusion with the Task class used for - SourceBufferActor scheduling. These are closer to operation - descriptors or messages than tasks in that sense." Rename pending; - also affects `planTasks` / `scheduleAll`. -- **Direct actor-level tests.** Coverage gap noted above. When does the - cost of behavior-level-only testing exceed the cost of writing actor - tests? - -## Related features - -- **preload-modes** — produces the gate state this feature consumes. - The 4-state load FSM maps directly to `(preload, loadActivated)`. -- **source-replacement** — segment-loader actors and in-flight fetches - tear down via the same resolved/unresolved cascade that drives source - replacement. New buffer-management behaviors that gate on resolved - presentation must honor the cleanup contract. -- **mse-mms-pipeline** — owns the `SourceBufferActor` this feature's - operations land on, plus the loader-actor lifecycle - (`setupVideoBufferActors` etc.). The fetch path - (`createTrackedFetch` for video) is wired in there. -- **video-abr** — quality-aware buffer planning preserves buffered - high-quality content during ABR downgrade; bandwidth sampling lands - in this feature's fetch path (`fetchBytes = createTrackedFetch`). -- **subtitles** — `loadTextTrackSegments` is the text-track variant of - this feature's dispatcher; `TextTrackSegmentLoaderActor` is the - text-side counterpart to `SegmentLoaderActor`. -- **multi-language-audio** *(coarse)* — Tier 2 audio-flush-on-switch - will use the `remove` message primitive surfaced through this - feature's actor. -- **Audio SourceBuffer flush orchestration** — not a separately- - scoped feature. The primitives (`SourceBufferActor.remove` message - + `flushBuffer` helper) live in - [mse-mms-pipeline](./mse-mms-pipeline.md) and surface through this - feature's actor; orchestration on `selectedAudioTrackId` change is - part of [multi-language-audio](./multi-language-audio.md)'s Tier 2 - mid-stream-switching phase. -- **[ll-hls-support](./ll-hls-support.md)** *(candidate)* — extends the - forward-buffer planner with partial-segment-head tracking past the - last complete segment. Planner extension shape is an open question - in that feature's doc. -- **[dvr-event-stream-support](./dvr-event-stream-support.md)** - *(candidate)* — puts pressure on the back-buffer eviction policy - (default `keepSegments: 2` evicts history before user can back- - seek to it). Variant-specific policy vs configurable threshold is - an open question shared with this doc's "Smarter back-buffer - eviction" section. -- **5.1-surround-selection** / **hevc-variant-selection** - *(candidates)* — cross-codec switches need a `changeType()` - extension beyond today's same-codec planning. - -## Use cases that compose this feature - -- **[`audio-only-mode-override`](../use-cases/audio-only-mode-override.md)** - *(partial — Phase 1 landed)* — Phase 1 baseline constituent (used - as-is via `createHlsAudioOnlyEngine`); Phase 3 surfaces - alternative-default-config candidates (shorter - `forwardBuffer.bufferDuration` for audio-only) and a Path-B - candidate (audio-only-tuned buffer-management behavior). -- **[`video-only-mode-override`](../use-cases/video-only-mode-override.md)** - *(coarse)* — Phase 1 baseline constituent (used as-is). Loop- - friendly buffer fetching is the peer `[background- - video]` use case's concern, not this one's; video-only-mode- - override's Phase 3 may surface other defaults but not - loop-around. - -## See also - -- [presentation-modeling.md](../presentation-modeling.md) — architectural - deep-dive on the format-neutral data shape and per-track resolution; - this feature consumes resolved per-track segments surfaced by that - layer -- [text-track-architecture.md](../text-track-architecture.md) — peer - architectural deep-dive (text-track-loader internals; same SPF - shape as v/a) -- [packages/spf/docs/hls-engine.md](../../../../packages/spf/docs/hls-engine.md) - — engine composition walkthrough (Stage 7: segment loading covers - this feature's dispatcher + actor surface) -- [conventions/behaviors.md](../conventions/behaviors.md) — per-type - specialization details (`setupSegmentLoading` is a canonical - instance) -- [conventions/actors.md](../conventions/actors.md) — actor + tasks - pattern (`SegmentLoaderActor` shape) -- [conventions/signals.md](../conventions/signals.md) — read-only - state-slot conventions (this feature consumes the engine's state - surface without writing to it) +- Segment planner and loading behavior: `packages/spf/src/playback/behaviors/dom/load-segments.ts` +- Segment and SourceBuffer actors plus tests: `packages/spf/src/playback/actors/dom/` +- Loading gates: `packages/spf/src/playback/behaviors/sync-preload.ts` and `packages/spf/src/playback/behaviors/dom/track-load-triggers.ts` diff --git a/internal/design/spf/features/engine-adapter-integration.md b/internal/design/spf/features/engine-adapter-integration.md index 6f19611a..9be46c32 100644 --- a/internal/design/spf/features/engine-adapter-integration.md +++ b/internal/design/spf/features/engine-adapter-integration.md @@ -6,219 +6,22 @@ definition: sketched # Engine-adapter integration -The engine's external-driving contract: `shareSignals` exposes the -composition's writable + readonly signal refs to a consumer callback at -setup time, and `SimpleHlsMediaMixin` is the canonical adapter that -maps a WHATWG HTMLMediaElement-shaped API onto those refs. The -*audience* for this feature is adapter authors and contributors who -need to drive the engine from outside — not end users, who see the -adapter's API only through whatever wraps it (e.g., -`packages/core`'s `SimpleHlsMedia` class). +Adapters drive an SPF engine through shared signal references rather than reaching into its behavior graph. -The feature ships as a *pair*: the framework-level `shareSignals` -mechanism + the canonical mixin. New adapter shapes (React hooks, RN -bridges, etc.) would compose on top of `shareSignals` independently of -the mixin. +## Implemented decisions -## Status +- `shareSignals` publishes writable and readonly references after composition setup, creating a small framework-level adapter boundary. +- The HLS DOM adapter maps an HTMLMediaElement-shaped contract onto those references; other platforms may build different adapters on the same boundary. +- Engine lifetime is independent of media-element attachment. Attach/detach does not destroy the engine, and assigning another source recycles it. +- Programmatic play activates loading before delegating to native playback so preload gating cannot deadlock the request. +- The adapter owns platform semantics; behaviors remain reusable and unaware of wrapper classes. -- **Composition:** `createSimpleHlsEngine` (HLS VoD); `shareSignals` - composed last so other behaviors' setups have run by the time the - callback fires -- **Definition depth:** sketched — capability surface and the - adapter-rationale open question both documented +## Deferred scope -## Phases of complexity +Curated notifications, a public error-state contract, multiple simultaneous engines, and non-HTML adapters are separate API decisions. -| Phase | What | Notes | -|---|---|---| -| Writable signal refs via `onSignalsReady` | `shareSignals` captures `Signal` / `ReadonlySignal` refs into a consumer-supplied callback at setup time. Generic over composition shape (`makeShareSignals()`) | Per-slot read/write intent is expressed at the use site (callers type captured refs as `Signal` or `ReadonlySignal`). Composed last in the engine so initial state writes are visible to the consumer | -| Mixin adapter pattern | `SimpleHlsMediaMixin` is the canonical consumer: function-of-base-class structure (mix into any base), captures refs once in `onSignalsReady`, exposes a WHATWG HTMLMediaElement-shaped API mapping each setter/method to engine writes | Downstream use: `class SimpleHlsMedia extends SimpleHlsMediaMixin(HTMLVideoElementHost) {}` in `packages/core/src/dom/media/simple-hls/` | -| Media element binding | `attach(el)` writes `context.mediaElement`; `detach()` clears it. **Engine persists across attach/detach cycles** — only `src` reassignment or explicit `destroy()` tears it down | Re-attach to a different element is supported. The engine is the durable state holder; `mediaElement` is a context slot | -| Source assignment via in-place recycling | Adapter's `set src` overwrites `state.presentation` on its single recycled engine (`{ url }`, or `undefined` for empty src). Media element + engine-wide preload persist; no engine recreation, no signal re-capture | Drives the engine's in-place source-replacement cascade — see [source-replacement.md](./source-replacement.md). (The adapter previously destroyed + recreated the engine per assignment.) | -| Preload reflection | `set preload(value)` writes W3C values to `state.preload`; clearing (`preload = ''`) doesn't patch the current engine but is re-applied on the next src change. Pre-attach src + preload combinations are supported | Extended preload values flow through state but don't reach the DOM (per [`preload-modes`](./preload-modes.md)'s sticky-extended-values semantics) | -| Programmatic `play()` with retry | `play()` writes `state.loadActivated = true` (co-writer with `trackLoadTriggers`'s DOM listener path) before invoking native play. **Defensive retry:** if native play rejects with "no supported sources" while src is pending, wait for `loadstart` (MSE attaches blob URL) and retry once | The retry handles MSE pipeline timing — adapter doesn't know exactly when MSE setup attaches the blob URL. Listener canceled on src change | +## Current sources of truth -## What's not implemented - -- **Reactive change-notification surface** — `onSignalsReady` fires - once at setup. Consumers wanting to react to state changes from - outside the engine must keep refs and subscribe via SPF primitives - (`effect()`, signal `subscribe()`). The adapter doesn't expose - curated `onPlay` / `onSrcChange` / `onError` callbacks. -- **Multiple engine instances per adapter** — one engine per adapter - instance. No built-in pattern for multi-engine scenarios - (picture-in-picture with two streams, A/B testing). -- **Non-HTMLMediaElement adapter shapes** — React-friendly hooks, - React Native bridges, etc. would compose on top of `shareSignals` - independently. Today the canonical adapter is HTMLMediaElement- - shaped via the mixin. No bracketed candidate features tracked yet; - add when concrete need surfaces. -- **Curated state / error introspection** — consumers can read - `signals.state.*.get()` directly, but there's no adapter-level - "current playback state" / "current error" shape that doesn't - require knowing the engine's signal map. - -## Implementation surface - -**Composition:** `packages/spf/src/playback/engines/hls/engine.ts` — -`shareSignals` is the last behavior in the composition. Instantiated -once at module load: - -```ts -const shareSignals = makeShareSignals(); - -// ... - -return createComposition( - [ - // ... all other behaviors ... - shareSignals, - ], - { config, initialState } -); -``` - -**Behavior factory:** - -| Export | File | Role | -|---|---|---| -| `makeShareSignals()` | `packages/spf/src/core/composition/share-signals.ts` | Generic behavior factory. Returns a `Behavior, ContextSignals, ShareSignalsConfig>` whose setup invokes `config.onSignalsReady?.({ state, context })` | -| `ShareSignalsConfig` | same | Config interface carrying the `onSignalsReady` callback | - -**Canonical adapter:** - -| Export | File | Role | -|---|---|---| -| `SimpleHlsMediaMixin` | `packages/spf/src/playback/engines/hls/adapter.ts` | Function-of-base-class mixin. Captures refs in `onSignalsReady`, exposes WHATWG HTMLMediaElement-shaped API | -| `SimpleHlsMediaElement` | same | Standalone subclass: `SimpleHlsMediaMixin(class {})`. Bare-bones reference instance | -| `SimpleHlsMediaProps` / `SimpleHlsMediaAPI` | same | The adapter's public-facing shape | - -**Adapter ↔ engine state/context map:** - -| Adapter call | Engine write | -|---|---| -| `attach(el)` | `context.mediaElement.set(el)` | -| `detach()` | `context.mediaElement.set(undefined)` | -| `destroy()` | `engine.destroy()` | -| `set src(value)` | `state.presentation.set({ url: value })` on the recycled engine (`undefined` for empty src) | -| `set preload(value)` | `state.preload.set(value)` (W3C values only; pre-empties stay engine-local) | -| `play()` | `state.loadActivated.set(true)` → native `play()` with `loadstart` retry on "no supported sources" | - -**Downstream consumer:** `packages/core/src/dom/media/simple-hls/index.ts`: - -```ts -export class SimpleHlsMedia extends SimpleHlsMediaMixin(HTMLVideoElementHost) {} -``` - -This is the canonical end consumer — used wherever the HTML player -expects an HTMLMediaElement-shaped object backed by SPF. - -## Config surface - -```ts -// ShareSignalsConfig -{ - onSignalsReady?: (signals: { - state: StateSignals; - context: ContextSignals; - }) => void; -} -``` - -The HLS engine config (`SimpleHlsEngineConfig`) extends -`ShareSignalsConfig`, so -`onSignalsReady` is part of the engine's config surface. - -`SimpleHlsMediaMixin`'s constructor takes optional `config` and threads -it through to every engine instance (including the ones created on -each `set src`). - -## Verification - -- **Unit tests:** - - `packages/spf/src/playback/engines/hls/tests/adapter.test.ts` — - extensive coverage: src assignment / re-assignment / clear, - engine recreation on src change, mediaElement preservation across - src changes, play retry on `loadstart`, preload propagation, - attach/detach lifecycle - - `packages/spf/src/playback/engines/hls/tests/engine.test.ts` - → "allows patching state and owners from outside" — direct - engine-level write surface (bypasses the mixin) - - `packages/spf/src/core/composition/tests/share-signals.test.ts` - — the behavior itself -- **Downstream usage:** - - `packages/core/src/dom/media/simple-hls/index.ts` — - `SimpleHlsMedia` consumer -- **Walkthrough:** - - `packages/spf/docs/hls-engine.md` Stage 10 — high-level coverage - of the pattern - -## Open questions - -- **Destroy-recreate vs in-place source replacement.** Resolved: the - canonical adapter now recycles a single engine and overwrites - `state.presentation` in place on every `src` change, driving the same - cascade validated by - [`source-replacement`](./source-replacement.md)'s test. This unifies - per-source teardown on one path and lets adapter-side projections - wire once at construction rather than re-wiring on every src change. - It also makes source-change behavior stable enough to build the - media-tracks mixin integration on top of. -- **Callback timing semantics.** `shareSignals`'s JSDoc explicitly - notes the callback fires while other behaviors are still in setup; - reads inside the callback may yield only initial-seed values. The - documented use is "capture refs, use later." Is read-at-setup-time - ever a supported case, or always discouraged? -- **Mixin base-class genericity.** `SimpleHlsMediaMixin>` - accepts any base; today's only documented consumer is - `HTMLVideoElementHost`. Other bases are structurally allowed but - not exercised — if usage broadens, the contract may need - tightening. - -## Related features - -- **preload-modes** — adapter `set preload(value)` and `play()` are - external writers on `state.preload` and `state.loadActivated` - respectively. The adapter's preload-clearing semantics (clear - `#preload` but don't patch the current engine) interact with - `preload-modes`'s sticky-extended-values rule. -- **source-replacement** — adapter `set src` is the canonical - user-facing entry into source replacement. The adapter's - destroy-recreate path bypasses the in-place reactor cascade — see - [`source-replacement.md`](./source-replacement.md) for the in-place - contract and the same open question. -- **mse-mms-pipeline** — adapter `attach(el)` binds the element MS - attaches to. The engine handles the rest of the MS lifecycle via - the resolved/unresolved cascade. -- **audio-playback** / **subtitles** / **video-abr** / - **buffer-management** — all driven by engine state the adapter - writes through. The adapter doesn't expose these features' surfaces - directly; consumers read engine state via the captured signal refs. - -## Use cases that compose this feature - -- **[`audio-only-mode-override`](../use-cases/audio-only-mode-override.md)** - *(partial — Phase 1 landed)* — Phase 1 baseline constituent with an - alternative adapter shape. The variant ships an independent - `SimpleHlsAudioOnlyMediaElement` adapter (via - `SimpleHlsAudioOnlyMediaMixin`) parallel to `SimpleHlsMediaElement`; - the `shareSignals` mechanism + mixin pattern compose unchanged. The - consumer-facing API matches the WHATWG `HTMLMediaElement` surface. -- **[`video-only-mode-override`](../use-cases/video-only-mode-override.md)** - *(coarse)* — Phase 1 baseline constituent on the inverse axis. - Ships an independent `SimpleVideoOnlyHlsMediaElement`-style - adapter parallel to `SimpleHlsMediaElement`. Same `shareSignals` - pattern; consumer-facing API differs from both default and - audio-only-mode-override. - -## See also - -- [clusters.md § Engine lifecycle](./clusters.md#engine-lifecycle) -- [packages/spf/docs/hls-engine.md § Stage 10](../../../../packages/spf/docs/hls-engine.md) - — `shareSignals` and the adapter pattern walkthrough -- [conventions/signals.md](../conventions/signals.md) — per-slot - `Signal` / `ReadonlySignal` intent (relevant for how consumers - type captured refs at the use site) -- `packages/core/src/dom/media/simple-hls/index.ts` — canonical - downstream consumer +- Signal-sharing primitive and tests: `packages/spf/src/core/composition/share-signals.ts` and `packages/spf/src/core/composition/tests/share-signals.test.ts` +- HLS adapters and tests: `packages/spf/src/playback/engines/hls/` +- Public wrapper behavior: `packages/core/src/dom/media/simple-hls/` diff --git a/internal/design/spf/features/mse-mms-pipeline.md b/internal/design/spf/features/mse-mms-pipeline.md index d8c01f37..ae03f2fc 100644 --- a/internal/design/spf/features/mse-mms-pipeline.md +++ b/internal/design/spf/features/mse-mms-pipeline.md @@ -4,257 +4,24 @@ date: 2026-05-20 definition: sketched --- -# MSE / MMS pipeline +# MSE/MMS pipeline -The lifecycle that makes a `MediaSource` a valid driver of the -`HTMLMediaElement`: create + attach the MediaSource, set up per-type -SourceBuffers wrapped in actors, propagate presentation duration, and -coordinate `endOfStream()` once playback reaches the appended tail. Spans -both standard `MediaSource` and Safari's `ManagedMediaSource`. +This feature owns the MediaSource lifecycle boundary: attaching a source, constructing per-type buffers, reflecting duration, and ending the stream. Segment policy is intentionally separate. -This doc captures the **capability surface**: what works, what doesn't, -which behaviors / actors / helpers implement it, and how it relates to -other features. Segment loading orchestration and buffer flushing are -sibling features — this one ends at the lifecycle boundary, not at "data -appears in the buffer." +## Implemented decisions -## Status +- Support standard `MediaSource` and Safari `ManagedMediaSource` behind the same setup behavior. +- Serialize operations through one SourceBuffer actor per media type. +- Add video before audio and create both buffers in the same pending turn when required by Firefox audio detection behavior. +- Treat duration writes as idempotent and coordinate `endOfStream()` only after active buffers reach their appended tails. +- Keep MediaSource setup and teardown tied to resolved-presentation lifecycle so source replacement cleans up the entire browser pipeline. -- **Composition:** `createSimpleHlsEngine` (HLS VoD) -- **Definition depth:** sketched — capability surface and implementation - footprint documented; `MediaSourceActor` factoring is an open - follow-up tracked under sibling candidates. Audio SourceBuffer flush - orchestration is part of [multi-language-audio](./multi-language-audio.md)'s - Tier 2 mid-stream-switching phase, not a separately-scoped feature +## Deferred scope -## Phases of complexity +Audio switching flushes, cross-codec `changeType()`, live seekable-range management, a dedicated MediaSource actor, and consumer control over managed-source preference are separate features. -What's implemented today, organized as platform / capability slices. Each -row is a slice that could in principle stand alone; in practice they -share the same four behaviors and one actor. +## Current sources of truth -| Phase | What | Notes | -|---|---|---| -| MediaSource attach / detach lifecycle | Create `MediaSource`, attach to element, await `'open'`, publish on `context.mediaSource`; detach + clear on source reset | `setupMediaSource` rides `resolvePresentation`'s resolved/unresolved transitions for source resets — direct URL replacement is structural, not a special case | -| Per-type SourceBuffer + actor setup | One `SourceBuffer` per type (CMAF A+V), each wrapped in `SourceBufferActor` + `SegmentLoaderActor`; gates only on that type's selection + codecs | `setupVideoBufferActors` / `setupAudioBufferActors` share `setupBufferActors`. The behaviors are decoupled — no cross-type coupling in `stateKeys` | -| Firefox `mozHasAudio` cross-type invariant | Both `addSourceBuffer` calls land in the same `runPending` iteration so the video buffer exists by the time any append begins | Preserves Firefox's permanent-`mozHasAudio=false` guard. Carried by SPF effect coalescing + composition order (video registered before audio in the engine). Sandbox repro at `apps/sandbox/src/firefox-mse-repro/` | -| ManagedMediaSource (Safari) | `preferManaged: true` → MMS via `srcObject` + `disableRemotePlayback`; standard MSE via `createObjectURL` otherwise | Same lifecycle shape, different attach surface. `preferManaged` is hardcoded `true` today (no config knob) | -| Initial `mediaSource.duration` write | Write `presentation.duration` to `mediaSource.duration` exactly once per MediaSource, gated on MS open + sourceBuffers idle + clamp ≥ `getMaxBufferedEnd` | `updateMediaSourceDuration`; idempotent — leaves any non-NaN value alone. `Infinity` supported for live | -| End-of-stream coordination | Call `mediaSource.endOfStream()` once every active actor's currently-loading track has its last segment appended + the playhead reaches that segment; re-arm on each `open → ended → open` cycle | `endOfStream`; sets final duration from `getMaxBufferedEnd` first to keep the value deterministic against CMAF timestamp drift | - -## What's not implemented - -- **Mid-stream same-codec buffer flush orchestration** — `SourceBufferActor` - accepts `remove` messages and `flushBuffer` exists in `media/dom/mse/`, - but no SPF behavior drives flushing on language switch or other - mid-stream cleanup. Belongs to [multi-language-audio](./multi-language-audio.md)'s - Tier 2 mid-stream-switching phase, which orchestrates flush on top - of this feature's `remove`-message + `flushBuffer` primitives. -- **`changeType()` codec-change switching** — cross-tick mid-stream track - switch after appends begin is "out of scope for this behavior" per - `setup-buffer-actors.ts`. Routes to `[5.1-surround-selection]` and - `[hevc-variant-selection]` (where the codec change motivates the - buffer-recreation or `changeType` path). -- **`MediaSourceActor` abstraction** — `endOfStream` and - `updateMediaSourceDuration` both subscribe to readyState via - `onMediaSourceReadyStateChange` + a behavior-local signal; both wait - for buffers idle. A `MediaSourceActor` whose snapshot exposes - `readyState` + accepts `duration-write` / `end-of-stream` / - `add-source-buffer` messages would coalesce these call sites. Hinted at - in both behaviors' JSDoc. -- **`preferManaged` opt-out** — hardcoded `true`. No engine config to - force standard MSE on Safari (for testing parity or debugging). -- **Continuous live-duration sync** — `Infinity` is supported on the - initial write, but `updateMediaSourceDuration`'s "exactly once" - contract doesn't re-sync if `presentation.duration` drifts mid-source. - Likely fine for live (one continuous `Infinity`), but - [live-stream-support](./live-stream-support.md) may surface - counterexamples. -- **`setLiveSeekableRange` / `clearLiveSeekableRange`** — neither is - called today. Live streams under `duration === Infinity` have an - empty `HTMLMediaElement.seekable` without the explicit setter. - Belongs to [live-stream-support](./live-stream-support.md)'s Live - edge tracking + Terminated state transition phases; lives at this - feature's MSE boundary. - -## Implementation surface - -**Composition:** `packages/spf/src/playback/engines/hls/engine.ts` — MSE -behaviors composed between presentation duration calculation and segment -loading. Video buffer setup registered before audio for the Firefox -invariant. - -**Behaviors:** - -| Behavior | File | Responsibility | -|---|---|---| -| `setupMediaSource` | `packages/spf/src/playback/behaviors/dom/setup-mediasource.ts` | Create + attach MediaSource, await `'open'`, publish; detach + clear on source reset | -| `setupVideoBufferActors` | `packages/spf/src/playback/behaviors/dom/setup-buffer-actors.ts` | Per-type video buffer + actor setup; sole writer of `bandwidthState` via `createTrackedFetch` | -| `setupAudioBufferActors` | `packages/spf/src/playback/behaviors/dom/setup-buffer-actors.ts` | Per-type audio buffer + actor setup; uses plain `fetchStream` (no bandwidth sampling today) | -| `updateMediaSourceDuration` | `packages/spf/src/playback/behaviors/dom/update-mediasource-duration.ts` | Write `mediaSource.duration = presentation.duration` once per MS, gated on MS open + buffers idle + spec clamp | -| `endOfStream` | `packages/spf/src/playback/behaviors/dom/end-of-stream.ts` | Drive each `open → ended` transition once last segments + playhead align; re-arm on cycles | - -**Actor:** - -| Actor | File | Role | -|---|---|---| -| `SourceBufferActor` | `packages/spf/src/playback/actors/dom/source-buffer.ts` | Serializes `append-init` / `append-segment` / `remove` / `batch` / `cancel` via `SerialRunner` on a single `SourceBuffer`; snapshot exposes `'idle'` / `'updating'` for downstream gating (canonical idle gate for `endOfStream`) | - -**DOM-bound helpers:** `packages/spf/src/media/dom/mse/` - -| Module | Role | -|---|---| -| `mediasource-setup.ts` | `createMediaSource({ preferManaged })`, `attachMediaSource` (MMS `srcObject` + `disableRemotePlayback` / MSE `createObjectURL` branch), `createSourceBuffer`, `buildMimeCodec`, `isCodecSupported`, `onMediaSourceReadyStateChange`, `waitForMediaSourceOpen`, `supportsMediaSource`, `supportsManagedMediaSource` | -| `append-segment.ts` | `appendSegment` — ArrayBuffer + streaming append primitive (used by `SourceBufferActor`'s append tasks) | -| `buffer-flusher.ts` | `flushBuffer` — range removal primitive (used by `SourceBufferActor`'s `remove` task; [multi-language-audio](./multi-language-audio.md)'s Tier 2 mid-stream-switching phase consumes via the actor message, not directly) | -| `duration.ts` | `shouldUpdateDuration`, `waitForSourceBuffersReady`, `getMaxBufferedEnd` (spec-clamp helper) | -| `end-of-stream.ts` | `isLastSegmentAppended` predicate | -| `mediasource.d.ts` | `ManagedMediaSource` global type augmentation (Safari-only API, not in standard DOM lib) | - -**State slots — reads only.** Every MSE behavior has read-only state -signatures. Engine state flows in; DOM mutations flow out. - -- Reads: `presentation` (all four behaviors), `currentTime` (`endOfStream`), - `selectedVideoTrackId` (`setupVideoBufferActors`), `selectedAudioTrackId` - (`setupAudioBufferActors`) - -**Context slots:** - -- `context.mediaSource` — sole writer `setupMediaSource`. Readers: - `setupVideoBufferActors`, `setupAudioBufferActors`, - `updateMediaSourceDuration`, `endOfStream`, `loadVideoSegments`, - `loadAudioSegments`. -- `context.videoBufferActor` + `context.videoSegmentLoaderActor` — sole - writer `setupVideoBufferActors`. Readers: `loadVideoSegments`, - `endOfStream`. -- `context.audioBufferActor` + `context.audioSegmentLoaderActor` — sole - writer `setupAudioBufferActors`. Readers: `loadAudioSegments`, - `endOfStream`. - -**DOM-property multi-writer.** `mediaSource.duration` is written by both -`updateMediaSourceDuration` (initial, once-per-MS while NaN) and -`endOfStream` (final, from `getMaxBufferedEnd` before EOS). Decision -domains are non-overlapping — `updateMediaSourceDuration`'s idempotency -on non-NaN keeps it out of the EOS path. Not a state-signal multi-writer; -the convention from `conventions/signals.md` doesn't apply directly. - -## Config surface - -This feature has essentially no engine-level config surface today — -behaviors read `presentation` for codecs + duration and operate on -context-published resources, with no tuning knobs of their own. -`preferManaged: true` is hardcoded in `setupMediaSource`. Related -engine config (`forwardBuffer`, `backBuffer`, `bandwidth`, `quality`) -flows through `setupVideoBufferActors`'s SegmentLoaderActor construction -but belongs to `buffer-management` (`forwardBuffer` / `backBuffer`) and -`video-abr` (`bandwidth` / `quality`) respectively. - -## Verification - -- **Unit tests:** - - `packages/spf/src/playback/behaviors/dom/tests/setup-mediasource.test.ts` - - `packages/spf/src/playback/behaviors/dom/tests/setup-buffer-actors.test.ts` - - `packages/spf/src/playback/behaviors/dom/tests/update-mediasource-duration.test.ts` - - `packages/spf/src/playback/behaviors/dom/tests/end-of-stream.test.ts` - - `packages/spf/src/playback/actors/dom/tests/source-buffer.test.ts` - - `packages/spf/src/media/dom/mse/tests/*` — helper-level coverage - (MS/MMS detection, attach branch, duration helpers, EOS predicate) -- **Sandbox:** - - `apps/sandbox/src/spf-segment-loading/` — main SPF MSE pipeline - demo; exercises full lifecycle end-to-end - - `apps/sandbox/src/firefox-mse-repro/` — Firefox `mozHasAudio` - invariant repro; load-bearing for verifying composition order + - `runPending` semantics survive future refactors - - `apps/sandbox/src/simple-hls-html/` / `simple-hls-react/` — engine - integration through the adapter layer - -## Open questions - -- **`MediaSourceActor`?** Both `endOfStream` and - `updateMediaSourceDuration` carry behavior-local `msIsOpen` mirrors and - buffers-idle waits. An actor that owns the MediaSource (snapshot for - `readyState`; messages for `duration-write` / `end-of-stream` / - `add-source-buffer`) would coalesce three call sites and prepare for a - future loop-mode (auto-fetch earlier segments mid-`ended`) where MS and - SourceBuffer coordination grows. Hinted at in both behaviors' JSDoc. -- **`preferManaged` as config.** Should there be an engine-level opt-out - for testing standard MSE on Safari, or remain hardcoded? -- **Two-fire `endOfStream()` on mid-end ABR switches.** Accepted today - as the price of dropping `selectedTrackId` dependence in - `endOfStream`. Worth flagging if it ever surfaces downstream issues - (e.g., spurious `ended` events on the element between the two fires). - -## Related features - -- **preload-modes** — gates this feature indirectly. `setupMediaSource` - rides `resolvePresentation`'s resolved/unresolved transitions, which - only flip to resolved once the preload gate (`preload !== 'none'` or - `loadActivated`) is open. -- **capability-probing** *(candidate)* — owns the upstream codec - filtering that would prevent `createSourceBuffer`'s late-failure - throw from firing in practice. Today's `isCodecSupported` helper is - the seed primitive; capability-probing wraps it into a uniform - surface and adds multivariant-level filtering before selection. -- **source-replacement** — the resolved/unresolved lifecycle - `setupMediaSource` rides is the canonical mechanism for in-place - source replacement. Detach-on-state-exit is what makes URL changes - work without recreating the engine. -- **subtitles** — text tracks share the per-type segment-loading FSM - but do **not** touch MSE (no SourceBuffer for text); cleanly separated - by `media/dom/mse/` not appearing in the text path. -- **video-abr** — `setupVideoBufferActors` is the sample producer - (`createTrackedFetch` writes `bandwidthState`); ABR consumes. Sampling - lives here, selection lives there. -- **multi-language-audio** — its Tier 2 "audio SourceBuffer flush on - switch" orchestrates flush on top of this feature's `remove`-message - + `flushBuffer` primitives. The orchestration belongs in - multi-language-audio (not a separately-scoped buffer-flushing - feature). -- **buffer-management** — sibling feature for the per-type load-FSM - and segment planning that runs *on top of* the buffers + actors this - feature stands up. Sends `append-init` / `append-segment` / `remove` - / `cancel` messages to the `SourceBufferActor` documented here. -- **5.1-surround-selection** *(not yet documented, candidate)* — - cross-codec switching via `changeType()`; out of scope for this - feature's same-codec lifecycle. -- **hevc-variant-selection** *(not yet documented, candidate)* — - same pattern as 5.1 but for video codec swap. -- **live-stream-support** — `Infinity` duration + the EOS picture - differ; the "exactly once" duration contract is the spot to revisit. - Also the home for the `setLiveSeekableRange` / `clearLiveSeekableRange` - DOM-exposure surface (see *What's not implemented* above). -- **drm-support** *(not yet documented, candidate, issue #1411)* — - key-system readiness would gate MSE setup + append per `clusters.md`. - -## Use cases that compose this feature - -- **[`audio-only-mode-override`](../use-cases/audio-only-mode-override.md)** - *(partial — Phase 1 landed)* — Phase 1 baseline constituent. Used - as-is — `MediaSource` + `endOfStream` gate compose unchanged across - variants per the uniform-across-tracks discipline (`endOfStream` - reads `mediaSource.sourceBuffers` aggregately). Verified end-to-end - in `engine-audio-only.test.ts` against both audio-only and mixed-AV - manifests. -- **[`video-only-mode-override`](../use-cases/video-only-mode-override.md)** - *(coarse)* — Phase 1 baseline constituent. Used as-is; the - Firefox `mozHasAudio` cross-type invariant documented here is - more pointedly relevant — the variant must produce - `mozHasAudio=false` cleanly under subtractive-audio composition. - Phase 1 includes empirical verification. - -## See also - -- [presentation-modeling.md](../presentation-modeling.md) — architectural - deep-dive on the format-neutral data shape and per-track resolution - layer; setup behaviors here gate on `isResolvedPresentation` from - that layer -- [text-track-architecture.md](../text-track-architecture.md) — peer - architectural deep-dive (different domain, same SPF shape) -- [packages/spf/docs/hls-engine.md](../../../../packages/spf/docs/hls-engine.md) - — full engine composition walkthrough (Stage 5: MSE setup; Stage 8: - end-of-stream) -- [conventions/behaviors.md](../conventions/behaviors.md) — per-type - specialization details -- [conventions/signals.md](../conventions/signals.md) — multi-writer - slot conventions (relevant for `mediaSource.duration` as a non-signal - multi-writer footnote) +- MediaSource setup, duration, EOS behavior, and tests: `packages/spf/src/playback/behaviors/dom/` +- SourceBuffer actor and tests: `packages/spf/src/playback/actors/dom/source-buffer.ts` and `packages/spf/src/playback/actors/dom/tests/source-buffer.test.ts` +- HLS engine composition: `packages/spf/src/playback/engines/hls/` diff --git a/internal/design/spf/features/preload-modes.md b/internal/design/spf/features/preload-modes.md index 7b5bc5b7..12cbb408 100644 --- a/internal/design/spf/features/preload-modes.md +++ b/internal/design/spf/features/preload-modes.md @@ -6,198 +6,22 @@ definition: sketched # Preload modes -The engine's loading-semantics implementation: how `preload="none|metadata|auto"` -plus user / programmatic activation determine when the engine fetches the -manifest, sets up MSE, and starts segment loading. Together, the -`syncPreload` and `trackLoadTriggers` behaviors model loading behavior -similar to native `HTMLMediaElement` playback — `state.preload` is the -mode, `state.loadActivated` is the override that fires on -`play` / `seeking` (or programmatic intent), and downstream gates read -both. +SPF models native-like preload semantics with two values: the requested preload mode and whether user or programmatic intent has activated loading. -This doc captures the **capability surface**: what works, what doesn't, -which behaviors / slots implement it, and how downstream features gate -on it. +## Implemented decisions -## Status +- `none` blocks source work until activation, `metadata` allows manifest/MediaSource/initialization work, and `auto` permits segment loading. +- Play and seeking activate loading. Activation is sticky for the current source and resets when the source changes. +- Keep DOM preload synchronization separate from activation tracking so adapter input and engine intent cannot overwrite each other accidentally. +- Default to metadata-like behavior and preserve unknown extended values in state for forward compatibility without reflecting them as invalid DOM values. +- Downstream behaviors consume derived gates rather than reinterpreting preload independently. -- **Composition:** `createSimpleHlsEngine` (HLS VoD) -- **Definition depth:** sketched — capability surface and implementation - footprint documented; the extended-preload-value mechanism is a - forward-compatibility hook with no shipped consumer yet +## Deferred scope -## Phases of complexity +More granular modes, deactivation after activation, and additional activation triggers require their own use cases. -What's implemented today, organized as capability slices around the -engine's loading-semantics contract. +## Current sources of truth -| Phase | What | Notes | -|---|---|---| -| W3C preload mode honoring | Engine respects `preload="none" \| "metadata" \| "auto"` via the `isBlockingPreload` predicate that downstream gates consume (`resolvePresentation`, the per-type segment-loading FSM). `'none'` is the strictest gate — blocks manifest fetch and all segment loading until activation. `'metadata'` resolves manifests + sets up MSE + fetches init segments only. `'auto'` runs the full pipeline | `isBlockingPreload` in `media/utils/preload.ts` is the core predicate | -| Bidirectional DOM ↔ state sync | `state.preload` and `mediaElement.preload` stay synchronized for W3C values. Effects registered read-before-write so a freshly mounted `