# VidStudio SDK: full documentation in one file This file holds the developer guide for @vidstudio/engine and @vidstudio/editor, both package readmes, and the public TypeScript declarations. Each section starts with a ruler and the path of the file it was taken from. The guide is also at https://vidstudio.app/developers and the API reference at https://vidstudio.app/developers/reference/. ======================================================================== # docs/developers/index.md ======================================================================== Title: Getting started Description: Install the editor, mount it in React or on a plain element, and serve the two folders it needs. # Getting started `@vidstudio/engine` is a video editing engine for the browser. It keeps a project (tracks, clips, effects, captions) and plays, renders and exports it on the user's device. `@vidstudio/editor` is the user interface for that engine: one React component, or one mount function for pages without React. Three names are used on every page. "The editor" is the component. "The store" is where projects live. "The backend" is the engine object behind one project. ## Install ``` npm install @vidstudio/editor ``` The engine comes with the editor. `react` and `react-dom` 18 or later are peer dependencies. Install `@vidstudio/engine` alone when you drive the backend without the user interface; see [Driving the engine](./engine.md). ## Mount in React Create the store once, create or pick a project, and give the editor a parent with a height. ```tsx import { useEffect, useState } from "react"; import { VidstudioEditor, createStore } from "@vidstudio/editor"; import "@vidstudio/editor/editor.css"; type Store = Awaited>; // Defined once: a new options object rebuilds the editor. const OPTIONS = { namespace: "my-app" }; export function Editor() { const [opened, setOpened] = useState<{ store: Store; projectId: string } | null>(null); useEffect(() => { let cancelled = false; (async () => { const store = await createStore({ namespace: "my-app" }); const project = await store.projects.create("My project", { frame_rate: 30 }); if (!cancelled) setOpened({ store, projectId: project.id }); })(); return () => { cancelled = true; }; }, []); if (!opened) return

Opening

; return (
); } ``` The editor fills its parent. Keep the `options` object stable, or the editor rebuilds on every render. ## Mount without React `mountEditor` renders the editor into any element with a height. ```ts import { createStore, mountEditor, lightTheme } from "@vidstudio/editor"; import "@vidstudio/editor/editor.css"; const store = await createStore({ namespace: "my-app" }); const project = await store.projects.create("My project", { frame_rate: 30 }); const editor = mountEditor(document.getElementById("editor")!, { store, projectId: project.id }); editor.update({ theme: lightTheme }); // Later, when the page is done with it: editor.unmount(); ``` `update` replaces only the props you pass. `unmount` disposes the backend and empties the element. ## Next.js The engine touches browser globals when it loads, so load it on the client only. No bundler setting is needed, with webpack or with Turbopack. ```tsx "use client"; import { useEffect, useRef } from "react"; import type { MountedEditor } from "@vidstudio/editor"; import "@vidstudio/editor/editor.css"; export function Editor() { const element = useRef(null); useEffect(() => { let editor: MountedEditor | null = null; let cancelled = false; (async () => { const { createStore, mountEditor } = await import("@vidstudio/editor"); const store = await createStore({ namespace: "my-app" }); const project = await store.projects.create("My project", { frame_rate: 30 }); if (!cancelled) editor = mountEditor(element.current!, { store, projectId: project.id }); })(); return () => { cancelled = true; editor?.unmount(); }; }, []); return
; } ``` ## Vite Set the worker format to ES modules. The engine's workers load their code in chunks, and Vite's default worker format cannot emit chunks. ```ts import { defineConfig } from "vite"; export default defineConfig({ worker: { format: "es" }, }); ``` The dev server needs no `optimizeDeps` setting. The workers load from `node_modules/@vidstudio/engine/dist/workers/`, checked with version 0.1.0 on Vite 5.4.21. ## Two folders to serve | Folder | What it holds | Command | |---|---|---| | `/fonts/` | The twelve text fonts | `cp -r node_modules/@vidstudio/engine/assets/fonts public/fonts` | | `/models/` | The speech models for captions, about 450 MB, plus 19 MB for background removal | `npx vidstudio-fetch-models public/models` | Serve them elsewhere and pass the addresses, each ending in a slash: `options={{ assets: { fonts: "https://cdn.example.com/fonts/", models: "https://cdn.example.com/models/" } }}`. With `features={{ captions: false }}` no model is ever downloaded. Details, sizes and flags are in [Media, fonts and models](./media.md). ## Cross-origin isolation is optional The editor runs without the headers `Cross-Origin-Opener-Policy: same-origin` and `Cross-Origin-Embedder-Policy: require-corp`. With them, captions transcribe on several threads, about twice as fast. Import, preview and export are the same either way. The headers also restrict which files your page can load from other origins; see [Media, fonts and models](./media.md). ## Licence Without a licence key the editor works in full on `localhost`, with a small "vidstudio.app" mark on the preview and on every exported frame. Any other public domain requires a licence key for that domain, which also removes the mark; see [Limits and support](./limits-and-support.md). ## Next pages - [Projects and storage](./projects-and-storage.md): the store, namespaces, where data lives, quotas, two editors on one page. - [Media, fonts and models](./media.md): what imports, proxies, web addresses, fonts and models. - [Save and load](./save-and-load.md): saving a project with its media to your server and loading it back. - [Theme, features and events](./theming-and-events.md): theme keys, feature flags, shortcuts, events. - [Driving the engine](./engine.md): the backend without the user interface. - [Limits and support](./limits-and-support.md): browsers, sizes, the licence, help. - [Troubleshooting](./troubleshooting.md): symptoms and fixes. - [The reference](/developers/reference/): every type and method. ======================================================================== # docs/developers/projects-and-storage.md ======================================================================== Title: Projects and storage Description: Where the editor keeps projects and media in the browser, how namespaces separate editors, and what happens when storage is full. # Projects and storage The store keeps projects in the browser. The database (IndexedDB) holds the rows: projects, tracks, clips, effects, keyframes, transitions. File storage (the origin private file system, OPFS) holds the media files. Nothing leaves the browser unless your code sends it; see [Save and load](./save-and-load.md). ## Create the store `createStore` opens the namespace's database and returns the store. ```ts import { createStore } from "@vidstudio/engine"; const store = await createStore({ namespace: "my-app" }); const project = await store.projects.create("My project", { frame_rate: 30 }); ``` `create` takes a name and optional `frame_rate`, `canvas_width` and `canvas_height`. Leave the size out and the project takes it from the first video. ## Namespaces A namespace names one editor's database, file folder and saved preferences. `storageNames(namespace)` returns the first two. | Namespace | Database | File folder | Preference keys (local storage) | |---|---|---|---| | `"vidstudio"` (the default) | `video-editor` | the root of file storage | `ve-new-project-defaults`, `vidstudio_safe_zone_guide`, `vidstudio_tour_seen` | | `"my-app"` | `my-app-video-editor` | `my-app-files` | `my-app:new-project-defaults`, `my-app:safe-zone-guide`, `my-app:tour-seen` | The default keeps the names vidstudio.app has always used. Pick your own namespace. Then your data never mixes with another editor's data on the same origin. ## Where data lives | What | Where | Path inside the file folder | |---|---|---| | Project rows | the database | | | Imported media | file storage | `projects//media/_.bin` | | Proxies (lighter copies for the preview) | file storage | `projects//proxy/_proxy.mp4` | | Exports | file storage | `exports//` | | Viewer preferences | local storage | | | Downloaded speech models | the browser's cache storage | | A file in file storage is a handle on disk, not bytes in memory. The engine reads only the parts it needs. A one gigabyte recording costs a few megabytes to open. ## Quota The browser sets the quota for the origin. Ask `navigator.storage.estimate()` for the usage and the quota. When a write is refused, the backend emits a `storage-full` event and the call rejects. | Call | Outcome when storage is full | |---|---| | `importVideo`, `importAudio`, `importImage`, `importAsset`, `importFromUrl` | Rejects with the code `import-failed`. Event `storage-full` with operation `"import"`. | | `loadDocument` | Rejects. The files written so far are deleted. The project stays as it was. Event `storage-full` with `"import"`. | | `exportProject` | Rejects. Events `export-failed` and `storage-full` with `"export"`. The export dialog tells the user the device ran out of storage. | | `removeTrack` | Rejects. Event `storage-full` with `"delete_track"`. | After its first write, the store sweeps once. It deletes files that no project row points at and that are older than thirty minutes. It also deletes export files older than six hours. Free space in your code with `store.deleteProject` and `backend.discardExportFile`. ## List and delete projects `store.projects.getAll()` returns every project in the namespace, in no fixed order. ```ts import { createStore } from "@vidstudio/engine"; const store = await createStore({ namespace: "my-app" }); const projects = (await store.projects.getAll()).sort((a, b) => b.updated_at - a.updated_at); for (const project of projects) console.log(project.id, project.name, new Date(project.updated_at)); if (projects.length > 0) { const removed = await store.deleteProject(projects[0].id); console.log(`deleted ${removed.assetCount} media files, ${removed.bytesFreed} bytes`); } ``` Use `store.deleteProject`, not `store.projects.delete`. The first removes the rows, the media folder and the export folder. The second removes only the project row. ## Two editors on one page Give each editor its own store and its own namespace. Pass the namespace to `options` too, so the saved preferences are separate. ```tsx import { useEffect, useState } from "react"; import { VidstudioEditor, createStore } from "@vidstudio/editor"; import "@vidstudio/editor/editor.css"; type Opened = { store: Awaited>; projectId: string }; const LEFT = { namespace: "left" }; const RIGHT = { namespace: "right" }; async function open(namespace: string): Promise { const store = await createStore({ namespace }); const project = await store.projects.create("Project", { frame_rate: 30 }); return { store, projectId: project.id }; } export function TwoEditors() { const [left, setLeft] = useState(null); const [right, setRight] = useState(null); useEffect(() => { open("left").then(setLeft); open("right").then(setRight); }, []); if (!left || !right) return null; return (
); } ``` Each editor keeps its own media, undo history and keyboard focus. Fonts load once per page and both editors share them. ## Your own storage and monitoring `createStore` takes `services`. The backend inherits them from the store. | Service | Contract | Default | |---|---|---| | `files` | `StorageProvider`: `write`, `read`, `delete`, `createWritable`, `commit`, `clearDirectory`, `list`, `deleteDirectoryTree`, and optional `describe` | file storage under the namespace's folder | | `monitoring` | `MonitoringAdapter`: `init`, `captureException`, `captureMessage`, `withScope`, `addBreadcrumb`, `setTag`, `setContext`, `setExtra`, `setUser` | nothing is reported | This adapter sends every error the engine catches to the console. ```ts import { createStore, type MonitoringAdapter } from "@vidstudio/engine"; const monitoring: MonitoringAdapter = { init: () => {}, captureException: (error, hint) => { console.error("[editor]", error, hint?.extra); return undefined; }, captureMessage: (message, level) => { console.log(`[editor] ${level ?? "info"}: ${message}`); return undefined; }, withScope: (callback) => callback({ setTag() {}, setContext() {}, setExtra() {}, setLevel() {}, setUser() {} }), addBreadcrumb: () => {}, setTag: () => {}, setContext: () => {}, setExtra: () => {}, setUser: () => {}, }; const store = await createStore({ namespace: "my-app", services: { monitoring } }); ``` `MemoryFileStorage` is a second provider that keeps files in memory, for tests. A worker cannot open a provider without `describe`. The engine then makes no proxies and runs exports on the page thread. ======================================================================== # docs/developers/media.md ======================================================================== Title: Media, fonts and models Description: Which files import, when proxies are made, how media from a web address works, and which fonts and models to serve. # Media, fonts and models ## What imports The engine reads the first sixteen bytes of a file to find its container. The file name does not matter. Decoding uses the browser's WebCodecs, so the codec must be one the browser decodes. | Kind | Containers | Codecs | Read by | |---|---|---|---| | Video | MP4, MOV, M4V | H.264, H.265, VP9, AV1 | mp4box; mediabunny when mp4box fails | | Video | WebM, MKV | VP8, VP9, AV1, H.264, where the browser decodes them | mediabunny | | Sound of a video, or an M4A | MP4, MOV, M4A | AAC, MP3, FLAC, Opus, AC-3, where the browser decodes them | mp4box and WebCodecs | | Audio files | MP3, WAV, Ogg, FLAC, WebM, MKV | what the browser decodes | mediabunny and WebCodecs, then the browser's `decodeAudioData` | | Images | PNG, JPEG, WebP, GIF, BMP, SVG | none | the browser's image decoder | Things to know: - H.264 decodes everywhere. H.265 depends on the browser and the hardware. Firefox refuses it, and the error names Chrome or Edge. - A phone video with a rotation tag plays upright. The rotation is applied when drawing, never baked into the file. - A fragmented MP4 (written with `moof` boxes) can import with fewer frames than it holds. - A sound file is decoded whole into memory, about ten megabytes per minute of stereo. - A GIF imports as a still picture, its first frame. An image clip lasts five seconds unless you pass `durationSec`. The editor's file pickers accept fewer types than the engine does. The video picker lists MP4 and WebM. The audio picker lists any audio type. The image picker lists PNG, JPEG, WebP, SVG, GIF and BMP. The backend methods `importVideo`, `importAudio` and `importImage` accept any file the engine can read. ## Proxies A proxy is a lighter copy of a video, made for the preview. The engine makes one in the background after an import and when a project opens. | Rule | Value | |---|---| | Made when the video is wider than | 1280 pixels | | Or taller than | 720 pixels | | Or when the keyframes are further apart than | 2 seconds on average | | Never on | iPhone, iPad, Android | Clips read the original until the proxy is done, then switch to it. Export and transcription always read the original. Progress arrives as the `proxy-progress` event, from 0 to 1, then `null` when the proxy is finished, failed or stopped. When a proxy fails, the original keeps playing. Nothing else changes. ## Media from a web address `importFromUrl` downloads a file into file storage as a stream. It then adds the file to the project's media, like a picked file. ```ts import type { EditorBackend } from "@vidstudio/engine"; export async function addFromServer(backend: EditorBackend): Promise { const asset = await backend.importFromUrl("https://media.example.com/clips/intro.mp4"); await backend.addTrackFromAsset(asset.id, "video"); const sound = await backend.importFromUrl("https://media.example.com/clips/intro.mp4", { type: "audio" }); await backend.addTrackFromAsset(sound.id, "audio"); } ``` `type: "audio"` keeps only the sound of a video. `type: "video"` on a file without video rejects with the code `type-mismatch`. A failed download rejects with `import-failed`. The file takes its name from the last part of the address. A name without an extension gets one from the content type. The download sends no cookies to another origin. An address on your own origin needs nothing. An address on another origin needs headers from that server. | Your page | Headers the file server must send | |---|---| | Without cross-origin isolation | `Access-Control-Allow-Origin: https://your-site.example` (or `*`) | | With cross-origin isolation (the two headers from [Getting started](./index.md)) | The header above, plus `Cross-Origin-Resource-Policy: cross-origin` | The same rules apply to a web address returned by a resolver in [Save and load](./save-and-load.md). ## Fonts Text clips use twelve fonts, one regular weight each, as `.woff2` files under `node_modules/@vidstudio/engine/assets/fonts`. All twelve are under the SIL Open Font License, included as `OFL.txt`. | Category | Fonts | |---|---| | Sans serif | Inter (the default), Roboto, Montserrat, Open Sans, Poppins | | Serif | Playfair Display, Lora | | Display | Bebas Neue, Oswald | | Handwriting | Dancing Script, Pacifico | | Monospace | JetBrains Mono | The editor loads a font the first time a clip uses it, from `/fonts/` by default. Copy the folder there, or serve it elsewhere and set `options.assets.fonts` to that address, ending in a slash. A font that fails to load logs a warning, and the text draws in the browser's fallback font. ``` cp -r node_modules/@vidstudio/engine/assets/fonts public/fonts ``` ## Models Captions come from Whisper speech models that run on the device. The editor picks the model by device. Both are English only: the `.en` models accept no other language. | Device | Model | Size | |---|---|---| | Desktop | `onnx-community/whisper-base.en` | 294 MB | | iPhone, iPad, Android | `onnx-community/whisper-tiny.en` | 154 MB | `vidstudio-fetch-models` puts the files into a folder you serve. It downloads from Hugging Face at pinned revisions and skips files already present. It writes the Apache 2.0 licence and a notice file beside them. ``` npx vidstudio-fetch-models public/models ``` | Flag | Effect | |---|---| | `--models base.en` | Only the models named, comma separated | | `--models all` | Every model in the manifest, including three word-timestamped variants the editor does not use | | `--from ` | Copy from an existing models folder instead of downloading | | `--no-whisper` | Skip the speech models | | `--no-mediapipe` | Skip the background removal files | The engine loads the models from `/models/` by default. Serve the folder elsewhere and set `options.assets.models`, ending in a slash. With a full web address, the files are fetched on first use and cached in the browser's cache storage. With `features={{ captions: false }}` the editor never downloads a model. Transcription also loads the onnxruntime runtime files from `cdn.jsdelivr.net` on first use. A page that is offline, or whose content security policy blocks that address, cannot transcribe. ## Background removal files `vidstudio-fetch-models` also writes `mediapipe/wasm/` and `mediapipe/selfie_segmenter.tflite`, about 19 MB together, under the models folder. The editor's effect list has no background removal today. The engine draws the effect only for a loaded document that already carries an effect of type `background-removal`. Such an export runs on the page thread instead of the worker. Pass `--no-mediapipe` if you never load such a document. ======================================================================== # docs/developers/save-and-load.md ======================================================================== Title: Save and load Description: Save a project as a document plus a list of media handles, load it back with a resolver, and keep the files on your server. # Save and load A document is the project as plain JSON (JavaScript Object Notation): `version`, `project`, `mediaAssets`, `tracks`, `clips`, `effects`, `keyframes` and `transitions`. The media files are not in it. `saveDocument` returns an array next to the document; this page calls it "the media list". One entry of that list is "a media handle". ## Save `saveDocument` returns the document and the media list, one media handle per stored file. ```ts import type { EditorBackend } from "@vidstudio/engine"; export async function inspect(backend: EditorBackend): Promise { const { document, media } = await backend.saveDocument(); console.log(document.project.name, document.clips.length, "clips"); for (const item of media) console.log(item.assetId, item.name, item.type, item.bytes, item.hash); } ``` | Field | Meaning | |---|---| | `assetId` | The asset's `id` inside the document | | `name` | The file name the user imported | | `type` | `"video"`, `"audio"` or `"image"` | | `bytes` | The file size | | `hash` | `":"`: the size plus a SHA-256 of the first and last 64 KB | | `read()` | A `File` handle to the stored file; no bytes are read until you upload or stream it | `hash` is cheap on purpose. It is enough to skip an upload your server already has. It is not a checksum of the whole file. Two files of the same size that differ only in the middle share a hash. Compute your own checksum on the server when that matters. An asset marked missing (see below) has no media handle. ## Load `loadDocument(document, resolve)` replaces the project's content with the document. It calls `resolve(asset)` once per asset in `document.mediaAssets`. Return a `File`, or a web address the engine downloads. Either one is written to file storage as a stream, and proxies are made again in the background. The document's name and settings replace the project's. Every id is minted again. Undo and redo are cleared. A web address on another origin needs the headers in [Media, fonts and models](./media.md). ## Missing media When `resolve` throws, or the download fails, the asset loads with `missing: true`. Its clips show a flat grey frame. `exportProject` rejects with the code `missing-media` while a clip still uses it. The editor opens the project and shows a warning with the count. To repair it, load the document again with a resolver that finds the file, or remove the clips. Any other failure, such as an unknown effect type, a wrong `version`, or a full storage, rejects. The files written so far are deleted, and the project stays exactly as it was. ## Document versions `version` is 1 today. `migrateDocument(document)` brings an older document up to the current format and returns it; `loadDocument` calls it for you. A document with another version is refused. ## The built-in zip Without handlers, the editor's "Save project" writes one zip and "Open project" reads one. `writeProjectZip(document, media)` returns a `Blob`. The zip is stored uncompressed. It points at the media files instead of copying them, and reads each file once for its checksum. Files and archives past 4 GB use ZIP64. `readProjectZip(blob)` returns `{ document, resolve }`; each media file comes back as a slice of the zip. A zip re-packed with compression is refused. ```ts import { readProjectZip, writeProjectZip } from "@vidstudio/editor"; import type { EditorBackend } from "@vidstudio/engine"; export async function roundTrip(backend: EditorBackend): Promise { const { document, media } = await backend.saveDocument(); const zip = await writeProjectZip(document, media); const opened = await readProjectZip(zip); await backend.loadDocument(opened.document, opened.resolve); return zip; } ``` ## The editor's handlers `onSaveDocument(document, media)` replaces the zip download. `onOpenDocument()` replaces the zip picker and returns `{ document, resolve }`, or `null` when the user cancelled. An error thrown by either becomes an error toast. `features={{ projectFiles: false }}` hides both menu entries. ## Save to a server This example uploads each file once, keyed by its hash, then stores the document with the asset-to-hash map. Save it as `upload-project.ts`. ```ts import type { MediaManifest, ProjectDocument } from "@vidstudio/engine"; async function serverHas(hash: string): Promise { const response = await fetch(`/api/media/${encodeURIComponent(hash)}`, { method: "HEAD" }); return response.ok; } export async function uploadProject(key: string, document: ProjectDocument, media: MediaManifest): Promise { for (const item of media) { if (await serverHas(item.hash)) continue; // read() is a handle; fetch streams the file from disk. const response = await fetch(`/api/media/${encodeURIComponent(item.hash)}`, { method: "PUT", body: await item.read() }); if (!response.ok) throw new Error(`Uploading ${item.name} failed: ${response.status}`); } const hashes = Object.fromEntries(media.map((item) => [item.assetId, item.hash])); const response = await fetch(`/api/projects/${encodeURIComponent(key)}`, { method: "PUT", headers: { "content-type": "application/json" }, body: JSON.stringify({ document, hashes }), }); if (!response.ok) throw new Error(`Saving the project failed: ${response.status}`); } ``` The server keeps `{ document, hashes }` per project and one file per hash. ## Load from a server This example fetches the document and resolves each asset to its file's web address. Save it as `fetch-project.ts`. ```ts import type { MediaResolver, ProjectDocument } from "@vidstudio/engine"; export async function fetchProject(key: string): Promise<{ document: ProjectDocument; resolve: MediaResolver }> { const response = await fetch(`/api/projects/${encodeURIComponent(key)}`); if (!response.ok) throw new Error(`Loading the project failed: ${response.status}`); const saved = (await response.json()) as { document: ProjectDocument; hashes: Record }; const resolve: MediaResolver = (asset) => { const hash = saved.hashes[asset.id]; // Throwing marks this one asset missing; the rest of the project still loads. if (!hash) throw new Error(`${asset.filename} was never uploaded`); return `/api/media/${encodeURIComponent(hash)}`; }; return { document: saved.document, resolve }; } ``` Wire both into the editor, or call them around the backend. ```tsx import { VidstudioEditor, createStore } from "@vidstudio/editor"; import "@vidstudio/editor/editor.css"; import { uploadProject } from "./upload-project"; import { fetchProject } from "./fetch-project"; type Store = Awaited>; export function ServerBackedEditor({ store, projectId }: { store: Store; projectId: string }) { return (
uploadProject(projectId, document, media)} onOpenDocument={() => fetchProject(projectId)} />
); } ``` With the backend alone: `const saved = await backend.saveDocument(); await uploadProject(key, saved.document, saved.media);` to save, and `const opened = await fetchProject(key); await backend.loadDocument(opened.document, opened.resolve);` to load. ======================================================================== # docs/developers/theming-and-events.md ======================================================================== Title: Theme, features and events Description: Every theme key, feature flag, shortcut mode and event of the editor, with the payload of each event. # Theme, features and events The editor is the `VidstudioEditor` component from `@vidstudio/editor`. Three props shape it: `theme`, `features` and `shortcuts`. One prop, `onEvent`, reports what happens inside it. `mountEditor` takes the same props. ## The theme A theme is an object of type `EditorTheme`. Each key sets one custom property on the editor root element. The package exports two full themes: `darkTheme` (the default) and `lightTheme`. Pass a partial theme to change some keys only. The other keys keep their default. ```tsx import { VidstudioEditor, lightTheme } from "@vidstudio/editor"; import type { VidstudioEditorProps } from "@vidstudio/editor"; export function BrandedEditor(props: Pick) { return (
); } ``` The host page is not changed. The variables live on the editor root, not on `:root`. Four keys need an explanation. `textOnAccent` colours text and icons on an accent button. `accentWarning` also colours keyframes. `mediaBg`, `mediaText` and `mediaTextMuted` sit behind the video and over it. They stay dark and light in both themes. | Key | darkTheme | lightTheme | |---|---|---| | `bgPrimary` | `#0a0a0b` | `#ffffff` | | `bgSecondary` | `#111113` | `#f6f6f7` | | `bgTertiary` | `#18181b` | `#eeeef0` | | `bgCard` | `#141416` | `#fbfbfc` | | `bgCardHover` | `#1c1c1f` | `#f1f1f3` | | `bgElevated` | `#202024` | `#ffffff` | | `textPrimary` | `#ffffff` | `#18181b` | | `textSecondary` | `#a1a1aa` | `#52525b` | | `textTertiary` | `#71717a` | `#71717a` | | `textMuted` | `#52525b` | `#a1a1aa` | | `textOnAccent` | `#ffffff` | `#ffffff` | | `accentPrimary` | `#ff6363` | `#e5484d` | | `accentPrimaryHover` | `#ff7a7a` | `#d63c41` | | `accentSecondary` | `#3b82f6` | `#2563eb` | | `accentSuccess` | `#22c55e` | `#16a34a` | | `accentWarning` | `#f59e0b` | `#d97706` | | `accentWarningHover` | `#fbbf24` | `#b45309` | | `accentError` | `#ef4444` | `#dc2626` | | `borderPrimary` | `#27272a` | `#e4e4e7` | | `borderSecondary` | `#3f3f46` | `#d4d4d8` | | `borderHover` | `#52525b` | `#a1a1aa` | | `errorSurface` | `#3a1a1a` | `#fef2f2` | | `errorBorder` | `#662222` | `#fecaca` | | `errorText` | `#ff8888` | `#b91c1c` | | `warningSurface` | `#3a2e1a` | `#fffbeb` | | `warningBorder` | `#665522` | `#fde68a` | | `warningText` | `#f0b45a` | `#b45309` | | `mediaBg` | `#000000` | `#000000` | | `mediaText` | `#ffffff` | `#ffffff` | | `mediaTextMuted` | `#cccccc` | `#cccccc` | | `timelineBg` | `#111111` | `#fafafa` | | `timelineTrackBorder` | `#2a2a2a` | `#e4e4e7` | | `timelineRulerText` | `#888888` | `#71717a` | | `timelinePlayhead` | `#e8524a` | `#e5484d` | | `timelineWaveform` | `#f4845f` | `#ea580c` | | `timelineSnap` | `#4fc3f7` | `#0284c7` | | `radiusSm` | `6px` | `6px` | | `radiusMd` | `8px` | `8px` | | `fontFamily` | `-apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, 'Inter', 'Helvetica Neue', Arial, sans-serif` | same as dark | ## Feature flags The `features` prop is a partial object of type `EditorFeatures`. Every flag defaults to true. The full set is exported as `allFeatures`. This mounts an editor with captions and the tour off. ```tsx import { VidstudioEditor } from "@vidstudio/editor"; import type { VidstudioEditorProps } from "@vidstudio/editor"; export function PlainEditor(props: Pick) { return (
); } ``` | Flag | What disappears when it is off | |---|---| | `sourceMonitor` | The Source and Program tabs over the preview, and the Source Monitor panel. | | `tour` | The tour button and the tour that opens on the first visit. | | `safeZones` | The guides picker over the preview, and the guide lines it draws. | | `captions` | The Transcribe control in the effect panel. No speech model is ever downloaded. | | `transitions` | The transition handles where two clips meet on a video or audio track. | | `keyframes` | The keyframe markers on timeline clips, and the keyframe controls in the effect panel. | | `export` | The Export button and the export dialog. | | `projectSettings` | The canvas size picker and the frame rate picker. | | `textTracks` | The "Add text track" button in the timeline toolbar. | | `imageImport` | The image import button in the timeline toolbar. | | `audioImport` | The audio import button in the timeline toolbar. | | `shortcutsHelp` | The shortcuts button, and the key that opens the shortcut list. | | `projectFiles` | The "Save project" and "Open project" entries of the project menu. | ## Shortcuts The `shortcuts` prop has two values. | Value | Where keys are heard | |---|---| | `"scoped"` (default) | On the editor root only. A key reaches the editor only while focus is inside it. Inputs on the host page are never affected. | | `"global"` | On the window. Every key on the page reaches the editor. Use it for a page that holds nothing but the editor. | ## Events `onEvent` receives two kinds of event. An engine event comes from the backend; its type is `EditorEvent` from `@vidstudio/engine`. A user interface event comes from the editor's own components; its type is `EditorUiEvent` from `@vidstudio/editor`. Every event has a `kind` field. The other fields are the payload. | Kind | Source | Payload | When it fires | |---|---|---|---| | `timeline-changed` | engine | none | A track, clip or transition changed. | | `effects-changed` | engine | `clipId: string \| null` | An effect, keyframe or geometry changed on one clip. `null` means re-read every clip. | | `project-changed` | engine | none | The name, canvas size, frame rate, work area or loop setting changed. | | `player-state` | engine | `state: "idle" \| "loading" \| "ready" \| "playing" \| "paused" \| "seeking"` | The player changed state. | | `time` | engine | `seconds: number` | The playhead moved. | | `duration` | engine | `seconds: number` | The timeline length changed. | | `load-progress` | engine | `fraction: number` | The project is loading into the player, 0 to 1. | | `proxy-progress` | engine | `assetId: string`, `fraction: number \| null` | A preview proxy is being made, 0 to 1. `null` means it finished, failed or stopped. | | `audio-loading` | engine | `count: number` | How many audio tracks are still decoding their sound. | | `export-progress` | engine | `progress: { phase, currentFrame, totalFrames, percent }` | The export advanced. `phase` is `"preparing"`, `"video"`, `"audio"` or `"finalizing"`. | | `warning` | engine | `message: string` | The engine has something to show the user. | | `error` | engine | `message: string`, `error: Error` | An import or playback failure. | | `preview-played` | engine | none | `play()` started the preview. Sent on every play. | | `media-imported` | engine | `mediaType: "video" \| "audio" \| "both"`, `clipDurationSec: number`, `addedTo: "new_track" \| "existing_track"` | An asset was placed on the timeline with `addTrackFromAsset` or `addClipFromAsset`. A picture with its linked sound is one event, `"both"`. | | `export-started` | engine | `settings: ExportSettings` | An export began. | | `export-finished` | engine | `settings: ExportSettings`, `result: ExportResult`, `durationMs: number` | An export produced a file. | | `export-failed` | engine | `settings: ExportSettings`, `error: Error`, `phase: string \| null`, `durationMs: number` | An export failed for any reason but a cancel. `phase` is the last progress phase reached. | | `storage-full` | engine | `operation: "import" \| "export" \| "delete_track"`, `error: Error` | Browser storage refused a write because it is full. | | `export-downloaded` | user interface | `settings: ExportSettings` | The user pressed Download in the export dialog, or `onExport` received the file. | | `export-released` | user interface | `reason: "dialog_closed" \| "page_left"`, `outputBytes: number` | The finished export file was deleted from browser storage. | `ExportSettings` and `ExportResult` are described in [Driving the engine](./engine.md). This component counts exports. ```tsx import { useState } from "react"; import { VidstudioEditor } from "@vidstudio/editor"; import type { VidstudioEditorProps } from "@vidstudio/editor"; export function CountingEditor(props: Pick) { const [finished, setFinished] = useState(0); const [downloaded, setDownloaded] = useState(0); return (

{finished} exports finished, {downloaded} downloaded

{ if (event.kind === "export-finished") setFinished((n) => n + 1); if (event.kind === "export-downloaded") setDownloaded((n) => n + 1); }} />
); } ``` A new `onEvent` function does not rebuild the editor. A new `options` object does, so keep `options` stable. ======================================================================== # docs/developers/engine.md ======================================================================== Title: Driving the engine Description: Use the backend from code without the user interface, in the browser or headless in Node, and handle its errors. # Driving the engine The backend is the class `EditorBackendImpl` from `@vidstudio/engine`. It implements the `EditorBackend` interface. The editor's user interface calls nothing else. Everything the editor does, your code can do too. The full method list is in the [reference](/developers/reference/). ## Open a backend Build a backend with a store, a project id and a canvas. Then call `open()`. ```ts import { createStore, EditorBackendImpl } from "@vidstudio/engine"; export async function openBackend(canvas: HTMLCanvasElement): Promise { const store = await createStore({ namespace: "my-app" }); const project = await store.projects.create("My project", { frame_rate: 30 }); const backend = new EditorBackendImpl({ store, projectId: project.id, previewCanvas: canvas, options: { namespace: "my-app" }, }); await backend.open(); return backend; } ``` The store is described in [Projects and storage](./projects-and-storage.md). `open()` checks the licence key once, seeds the effect catalog and loads the project into the player. Call `dispose()` when you are done with the backend. ## The command queue Every edit is one method on the backend. Edits run one after another on a queue. Two edits never interleave. An edit that changes the project records one undo step; `undo()` and `redo()` walk the last fifty steps. ## The methods, by group Times are seconds. Each method that changes the project returns a promise and ends by emitting an event (see [Theme, features and events](./theming-and-events.md)). | Group | Methods | |---|---| | Lifecycle | `open`, `dispose`, `subscribe`, `isHeadless`, `getLicenceState` | | Reading | `getProject`, `getTimeline`, `getClip`, `findClipAt`, `getClipEffects`, `getEffectCatalog`, `getMediaAssets`, `getTransitions`, `snapshot` | | Project settings | `renameProject`, `setCanvasSize`, `setFrameRate`, `setWorkArea`, `setLoopPlayback` | | Import | `importVideo`, `importAudio`, `importImage`, `importAsset`, `importFromUrl`, `addColorLayer`, `readAssetFile`, `readAssetPreviewFile` | | Tracks | `addTrackFromAsset`, `addClipFromAsset`, `removeTrack`, `setTrackStartTime`, `setTrackVolume`, `reorderTracks`, `setTrackHidden`, `renameTrack` | | Clips | `splitClipAt`, `deleteClip`, `deleteClips`, `trimClip`, `moveClip`, `placeClips`, `linkClips`, `unlinkClip`, `updateClip`, `copyClip`, `pasteClip`, `duplicateClip`, `hasClipboard` | | Text | `addTextTrack`, `addTextTrackWithClips`, `restyleTextClips`, `transcribeClipToTextTrack` | | Effects and keyframes | `addEffect`, `removeEffect`, `reorderEffects`, `setEffectParams`, `setEffectWindow`, `setKeyframe`, `moveKeyframe`, `setKeyframeEasing`, `deleteKeyframe`, `activateKeyframing`, `deactivateKeyframing`, `easeAllKeyframes`, `easeKeyframeLane`, `setClipGeometry`, `previewClipGeometry`, `adoptLegacyOpacity` | | Transitions | `setTransition`, `removeTransition` | | Undo | `undo`, `redo`, `canUndo`, `canRedo` | | Playback | `play`, `pause`, `seek`, `nextFrame`, `previousFrame`, `setPlaybackRate`, `getCurrentTime`, `getDuration`, `getPlayerState` | | Render a frame | `renderFrameAt` | | Export | `exportProject`, `cancelExport`, `readExportFile`, `discardExportFile` | | Documents | `saveDocument`, `loadDocument` | Transition types are `"crossfade"`, `"wipe-left"`, `"wipe-right"`, `"push-left"` and `"push-right"`. Effect ids come from `getEffectCatalog()`; `"brightness"`, `"contrast"`, `"grayscale"` and `"opacity"` are four of them. `addTextTrack()` adds a text track with one clip at the playhead. The clip reads "Text" and lasts five seconds. Change it with `updateClip(clipId, { text_content: "..." })`. Saving and loading projects are described in [Save and load](./save-and-load.md). ## Headless mode Leave out `previewCanvas` and the backend runs headless. No player is built, so nothing from the browser is needed. Every edit then runs in Node in milliseconds. The engine's own unit tests work this way. This builds a headless backend in Node with in-memory storage. ```ts import { EditorBackendImpl, MemoryFileStorage, MemoryStorageAdapter, ModelStore } from "@vidstudio/engine"; export async function headlessBackend(): Promise { const store = new ModelStore(new MemoryStorageAdapter(), { files: new MemoryFileStorage() }); const project = await store.projects.create("Test", { frame_rate: 30 }); const backend = new EditorBackendImpl({ store, projectId: project.id }); await backend.open(); return backend; } ``` | Works headless | Throws `engine-required` headless | |---|---| | Every read, `snapshot`, every project setting | `importVideo`, `importAudio`, `importImage`, `addColorLayer` | | Every clip, text, effect, keyframe, transition and layer edit | `addTextTrack`, `addTextTrackWithClips`, `addClipFromAsset`, `addTrackFromAsset` | | `undo`, `redo` | `removeTrack`, `setTrackStartTime`, `setTrackVolume` | | `importAsset`, `importFromUrl`, `readAssetFile`, `readAssetPreviewFile` | Every playback method | | `saveDocument`, `loadDocument` | `renderFrameAt`, `exportProject`, `transcribeClipToTextTrack` | | `getLicenceState` (always `"invalid"` in Node, where there is no page) | | In Node, use `importAsset` with a `File` and a probe, or `loadDocument` with a resolver, to get media into a project. Edit it, then `saveDocument()` and load the document in a browser to render or export. ## Codecs and export settings `detectCodecs` asks the browser which encoders it has. It tries H.264 profiles from High 5.2 down to Baseline 3.0 and falls back to VP9. For sound it tries AAC-LC and falls back to Opus. `fallback` is true when either fell back. | `ExportSettings` field | Meaning | Usual value | |---|---|---| | `width`, `height` | Output size in pixels. H.264 needs even numbers. | 1920, 1080 | | `frameRate` | Frames per second | 30 | | `videoBitrate` | Bits per second | 8000000 | | `videoCodec` | The WebCodecs codec string | `codecs.video.webCodec` | | `muxerVideoCodec` | `"avc"` or `"vp9"` | `codecs.video.muxerCodec` | | `audioBitrate` | Bits per second | 128000 | | `audioSampleRate` | Samples per second | 48000 | | `audioChannels` | Channel count | 2 | | `audioCodec` | The WebCodecs codec string | `codecs.audio.webCodec` | | `muxerAudioCodec` | `"aac"` or `"opus"` | `codecs.audio.muxerCodec` | `exportProject` resolves to an `ExportResult` with three fields. `opfsPath` is where the MP4 sits in browser storage. `size` is its length in bytes. `skippedAudioClips` lists the audio clips that failed to decode and were rendered as silence. Read the file with `readExportFile(opfsPath)`. Delete it with `discardExportFile(opfsPath)` once the user has it, or it stays in storage. ## Errors A refusal is an `EditorBackendError`. Its `code` field tells you why. Check the code, not the message. ```ts import { EditorBackendError } from "@vidstudio/engine"; export function explain(err: unknown): string { if (err instanceof EditorBackendError && err.code === "invalid-split") return "Move the cut further inside the clip."; if (err instanceof EditorBackendError) return `The backend refused: ${err.code}`; return "Something else failed"; } ``` | Code | When it happens | |---|---| | `engine-required` | An engine method was called on a headless backend, or before `open()` finished. | | `clip-not-found` | No clip has that id. | | `track-not-found` | No track has that id, or the track belongs to another project. | | `asset-not-found` | No media asset has that id. | | `effect-not-found` | No effect has that id, or the effect is not on that clip. | | `keyframe-not-found` | No keyframe has that id. | | `transition-not-found` | No transition has that id. | | `transform-not-found` | The effect id is not in the catalog. | | `invalid-argument` | A value is out of range: a time outside the clip, an unknown easing, a transition that does not fit, a clip placed twice, a name over 60 characters, a clip that is not linked. | | `invalid-split` | The cut is closer than 0.05 s to a clip edge. | | `type-mismatch` | A clip and a track of different kinds, a volume on a video track, a hidden audio track, two clips of the same kind linked. | | `clip-overlap` | The clip would overlap another clip on its new track. | | `nothing-to-export` | The timeline is empty. | | `export-cancelled` | `cancelExport()` was called while an export ran. | | `import-failed` | The file or the web address could not be decoded, downloaded or stored. | | `missing-media` | A clip plays an asset whose file never arrived at load time. | | `not-implemented` | Reserved; nothing throws it today. | A failure that is not a refusal, such as a decoder crash, is a plain `Error`. ## A complete example This opens a backend, imports a video, adds a title, exports the timeline and returns the MP4. ```ts import { createStore, detectCodecs, EditorBackendImpl } from "@vidstudio/engine"; import type { ExportSettings } from "@vidstudio/engine"; export async function exportWithTitle(video: File): Promise { const store = await createStore({ namespace: "my-app" }); const project = await store.projects.create("Titled", { frame_rate: 30 }); const canvas = document.createElement("canvas"); const backend = new EditorBackendImpl({ store, projectId: project.id, previewCanvas: canvas, options: { namespace: "my-app" }, }); await backend.open(); await backend.setCanvasSize({ width: 1280, height: 720 }); await backend.importVideo(video); const textTrackId = await backend.addTextTrack(); const timeline = await backend.getTimeline(); const textTrack = timeline.tracks.find((track) => track.id === textTrackId); if (!textTrack) throw new Error("text track not found"); await backend.updateClip(textTrack.clips[0].id, { text_content: "Hello" }); const codecs = await detectCodecs({ width: 1280, height: 720, audioSampleRate: 48000, audioChannels: 2 }); const settings: ExportSettings = { width: 1280, height: 720, frameRate: 30, videoBitrate: 8_000_000, videoCodec: codecs.video.webCodec, muxerVideoCodec: codecs.video.muxerCodec, audioBitrate: 128_000, audioSampleRate: 48000, audioChannels: 2, audioCodec: codecs.audio.webCodec, muxerAudioCodec: codecs.audio.muxerCodec, }; const result = await backend.exportProject(settings); const file = await backend.readExportFile(result.opfsPath); console.log(`${result.size} bytes, ${result.skippedAudioClips.length} audio clips skipped`); await backend.dispose(); return file; } ``` The canvas never joins the page. It only gives the player somewhere to draw. On localhost the export carries the mark; see [Limits and support](./limits-and-support.md). ======================================================================== # docs/developers/limits-and-support.md ======================================================================== Title: Limits and support Description: Browsers, phones, export sizes, what runs on the device, the network, the licence, versions and how to get help. # Limits and support ## Browsers | Browser | State | |---|---| | Chromium (Chrome, Edge) | Tested in automation on every change: the browser test suite runs in Chromium through Playwright. | | Safari | Works where WebCodecs encoders exist (Safari 17 and later). Not automated. | | Firefox | Not automated. | Two things need a secure context (HTTPS, or localhost). File storage for media needs it. The licence check needs it too, plus Ed25519 in WebCrypto: Chrome 137, Safari 17 or Firefox 129, or later. In an older browser the key cannot be checked, so the mark is drawn. Export needs the WebCodecs `VideoEncoder` and `AudioEncoder`. A browser without them gets a message in the export dialog instead of a crash. ## Phones and tablets The editor runs on phones. Its layout changes with the width of its root element, not the window: the properties panel folds into a bar below 640 px. The engine also changes four things on a phone. It makes no preview proxies. It runs one transcode at a time. It uses the smaller `tiny.en` model for captions. It passes export frames in I420 format. Big exports can still run out of memory on a phone. The export dialog then says so and suggests a smaller size. ## Export size, codecs and container There is no fixed maximum. The limit is the browser's encoder. Before an export starts, the dialog asks the encoder whether it accepts the chosen size, bit rate and frame rate. It greys out a combination the encoder refuses. From code, `detectCodecs` does the same for the codec; a refused size fails inside `exportProject`. | Setting | Choices in the export dialog | |---|---| | Size, landscape | 720p, 1080p, 1440p, 4K (3840 px wide), always in the canvas's shape | | Size, portrait or square | 720p, 1080p, 1440p (2560 px tall), plus the canvas's own size when it is not listed | | Frame rate | 24, 25, 30, 60, plus the project's own rate | | Video bit rate | 4, 8, 16 or 32 megabits per second | | Container | MP4, always | | Video codec | H.264 (High 5.2 down to Baseline 3.0, the first the browser accepts); VP9 when the browser has no H.264 encoder | | Audio codec | AAC-LC at 128 kilobits per second; Opus when the browser has no AAC encoder | H.264 needs even pixel sizes; the dialog rounds odd values up. ## Captions Captions come from Whisper models running on the device: `base.en` on a desktop and `tiny.en` on a phone. Both are English only. There is no language option today. ## Memory and big files Import and preview read only a file's index, about 2 MB, and the frames being decoded. A 1 GB screen recording imports without loading into memory. Sound is the exception: an audio file is decoded whole, about 10 MB a minute. A project waits for every sound on its timeline before the first frame. ## The network The engine itself fetches nothing from us. It makes no licence call and sends no usage report. Three things reach the network. You control the first two addresses. | What | Where it goes | |---|---| | Fonts for text tracks | Your `options.assets.fonts` folder, default `/fonts/` | | Speech and background-removal models | Your `options.assets.models` folder, default `/models/` | | The speech runtime's WebAssembly files | `https://cdn.jsdelivr.net/npm/@huggingface/transformers@3.8.1/dist/`, the default of the transformers.js library the engine uses. Only a transcription triggers it. | A page that blocks that content delivery network loses captions and nothing else. Turning captions off (`features={{ captions: false }}`) removes the fetch entirely. ## Content Security Policy The engine runs three module workers from your own origin, compiles WebAssembly, loads fonts and decodes image clips through `blob:` addresses. This policy covers all of it with the default asset folders. | Directive | Value | |---|---| | `script-src` | `'self' 'wasm-unsafe-eval' https://cdn.jsdelivr.net` | | `worker-src` | `'self'` | | `connect-src` | `'self' https://cdn.jsdelivr.net`, plus every host you import media from | | `font-src` | `'self'` | | `img-src` | `'self' blob:` | If your fonts or models sit on another host, add that host to `connect-src` and `font-src`. For the models, add it to `script-src` too, because background removal loads a small script from the model folder. Leave out `https://cdn.jsdelivr.net` when captions are off. ## The licence The packages are free to install and to run on localhost. Without a licence key, and outside the domains a key covers, the engine draws a small "vidstudio.app" mark. The mark sits in the lower right of the preview, of `renderFrameAt` and of every exported frame. Every feature still works. Serving the editor to the public on any domain other than localhost requires a key for that domain. This holds whether or not the mark is shown. A licence key is `{ name, key }`. Pass it as the `licence` prop of the editor, or as `options.licence` of the backend. The key names one or more domains and an end date. It covers each listed domain and every subdomain of it, so a key for `example.com` covers `app.example.com` and `www.example.com`, but not `example.org`. Ports do not matter. The check runs once, in `open()`, offline, against the page's host name. It takes about a millisecond and makes no network call. `getLicenceState()` resolves to one of three states. | State | Meaning | Mark | |---|---|---| | `"valid"` | The signature checks out, the name matches, the key has not expired, the host is covered. | No | | `"localhost"` | The page is on `localhost` or `127.0.0.1` with no key covering it. | Yes | | `"invalid"` | Anything else: no key, another domain, expired, or a browser that cannot check Ed25519. | Yes, plus a notice in the editor the user can dismiss | To get a key, write to vidstudio446@gmail.com with your domains. The full terms are in the `LICENSE` file of each package. ## Versions Both packages are at version 0.1.0. `@vidstudio/editor` pins `@vidstudio/engine` to the exact same version, and both pin every dependency exactly. Pin an exact version in your own `package.json` too, and upgrade both packages together. There is no changelog yet. Read the readme of the new version before you upgrade. ## Help Write to vidstudio446@gmail.com. Include the package version, the browser and its version, and the console output. For an export problem, include the export settings and the size of the source file. ======================================================================== # docs/developers/troubleshooting.md ======================================================================== Title: Troubleshooting Description: Every symptom found while embedding the editor, with its cause and the exact fix. # Troubleshooting Each entry has the symptom, the cause and the fix. Search this page for the text in your console. ## Vite build fails with "UMD and IIFE output formats are not supported for code-splitting builds" Cause: the engine's workers load their code in chunks, and Vite's default worker format cannot emit chunks. Fix: set the worker format to ES modules in `vite.config.ts`. ```ts import { defineConfig } from "vite"; export default defineConfig({ worker: { format: "es" }, }); ``` ## Vite dev server fails to load the export or transcode worker The console shows `Transcode worker error` or `does not provide an export named 'default'`, and the export silently runs on the main thread. Cause: the dev server serves a worker from `node_modules` and resolves one of its dependencies to a CommonJS file that was not pre-bundled. Version 0.1.0 on Vite 5.4.21 does not show this; an earlier build did. Fix: name the dependency from the error message in `optimizeDeps`. ```ts import { defineConfig } from "vite"; export default defineConfig({ worker: { format: "es" }, optimizeDeps: { include: ["pixi.js"] }, }); ``` ## Fonts return 404 in Next.js The console shows a 404 for `/fonts/Inter-Regular.woff2` and `[FontRegistry] Failed to load font "Inter"`. Cause: text tracks load fonts from `/fonts/`, and nothing puts them there. Fix: copy the fonts into the public folder once. ``` cp -r node_modules/@vidstudio/engine/assets/fonts public/fonts ``` Or serve them elsewhere and pass `options={{ assets: { fonts: "https://static.example.com/fonts/" } }}`, ending in a slash. ## Models return 404 The console shows a 404 for `/models/onnx-community/whisper-base.en/config.json` when a transcription starts. Cause: the speech models are not in the package; they are fetched once by a script. Fix: run the script, or turn captions off. ``` npx vidstudio-fetch-models public/models ``` Or pass `options={{ assets: { models: "https://static.example.com/models/" } }}`, or pass `features={{ captions: false }}` so no model is ever requested. ## The mark is drawn on a licensed domain The preview and the exports carry the "vidstudio.app" mark although you passed a key. Cause: one of five things, listed in the table. | Cause | How to tell | Fix | |---|---|---| | Old browser | Chrome before 137, Safari before 17, Firefox before 129 cannot check Ed25519 | Use a newer browser | | Plain `http://` on a network address | `window.isSecureContext` is false, so WebCrypto is missing | Serve over HTTPS; only `localhost` is secure without it | | Domain mismatch | The key covers `example.com` and the page is on `example.org` | Ask for a key that lists the page's domain; subdomains and `www.` are covered already | | Name mismatch | `licence.name` differs from the name the key was issued to | Pass the exact name you were given | | Expired | The key's end date has passed | Ask for a new key | Add this function to your page and call it with your licence to see which one it is. ```ts import { checkLicence, readLicenceKey } from "@vidstudio/engine"; export async function inspectLicence(licence: { name: string; key: string }): Promise { console.log(await readLicenceKey(licence.key)); console.log(await checkLicence(licence, location.hostname)); } ``` `readLicenceKey` returns `null` when the signature or the format is wrong, otherwise the name, domains and end date. `checkLicence` returns the state for the host. ## Transcription is slow A minute of speech takes about twice as long as expected. Cause: without the cross-origin isolation headers the speech runtime runs on one thread. Fix: send both headers from your server. Then every cross-origin resource the page loads must send `Cross-Origin-Resource-Policy: cross-origin` too. ``` Cross-Origin-Opener-Policy: same-origin Cross-Origin-Embedder-Policy: require-corp ``` Without the headers everything still works; only transcription is slower. ## An import or export fails with "would exceed its storage quota" The backend emits a `storage-full` event and the operation rejects. Cause: the browser gave this origin less storage than the file needs. A private window and a Playwright context without a persistent profile get a very small quota. Fix: free space on the device, or use a normal window. Delete finished exports with `discardExportFile`. In Playwright, use `launchPersistentContext`. ## `importFromUrl` rejects with `import-failed` The address is correct and opens in a tab, but the import fails. Cause: the file is on another origin and that server does not allow your page to read it. Fix: have the server send `Access-Control-Allow-Origin` with your origin, or `*`. With the isolation headers on your page, it must also send `Cross-Origin-Resource-Policy: cross-origin`. ## Server rendering fails with "navigator is not defined" Next.js, or another server renderer, throws `ReferenceError: navigator is not defined` when a page imports `@vidstudio/editor`. Cause: the editor's rendering library reads the browser at module load time, and there is no browser on the server. Fix: load the editor only in the browser, inside an effect. ```tsx import { useEffect, useRef } from "react"; import type { MountedEditor, VidstudioEditorProps } from "@vidstudio/editor"; export function ClientEditor({ store, projectId }: Pick) { const element = useRef(null); useEffect(() => { let editor: MountedEditor | null = null; import("@vidstudio/editor").then(({ mountEditor }) => { if (element.current) editor = mountEditor(element.current, { store, projectId }); }); return () => editor?.unmount(); }, [store, projectId]); return
; } ``` In Next.js, `dynamic(() => import("./Editor"), { ssr: false })` does the same for a component file that imports the package at the top. ## Two editors on one page show each other's media Importing into one editor adds the clip to the other, or deleting in one removes from both. Cause: both editors use the same namespace, so they share one database and one file folder. Fix: give each editor its own namespace, in the store and in the options. ```ts import { createStore } from "@vidstudio/editor"; export async function twoStores() { const left = await createStore({ namespace: "left" }); const right = await createStore({ namespace: "right" }); return { left: { store: left, options: { namespace: "left" } }, right: { store: right, options: { namespace: "right" } }, }; } ``` Pass each pair as the `store` and `options` props of its editor. ## A notice sits over the timeline A bar in the lower left of the editor reads "This editor is unlicensed for this domain, so its picture and exports carry a watermark." Cause: the licence state is `"invalid"` on this host. On localhost there is no notice. Fix: pass a key that covers the domain. The user can dismiss the bar with its close button; it comes back on the next mount. ## Export rejects with `missing-media` `exportProject` rejects with the message `"" is missing. Remove its clips or open the project with the file before exporting.` Cause: when the project was loaded, the resolver threw or the download failed for that asset, so it loaded with `missing: true`. Fix: load the document again with a resolver that returns the file, or delete the clips that play it. `getMediaAssets()` lists the assets; the missing ones have `missing: true`. ## A warning about a Canvas 2D fallback in headless Chromium The console shows `PixiJS WebGL init failed` followed by `using Canvas 2D fallback`. Cause: headless Chromium has no WebGL unless it is launched with a software renderer, so the preview falls back to Canvas 2D. Fix: nothing is broken; the frames are the same. To silence it, pass `options={{ renderer: "canvas2d" }}` in automation. ======================================================================== # packages/engine/README.md ======================================================================== # @vidstudio/engine ## Install Run `npm install @vidstudio/engine`. The engine runs in the browser. Import it only on the client; with server rendering, load it with a dynamic `import()` inside an effect. ## Create a store The store keeps projects in IndexedDB and media files in the origin private file system. Give each editor on a page its own namespace. `createStore` opens the namespace's database and returns the store; then create a project. ```ts import { createStore } from "@vidstudio/engine"; const store = await createStore({ namespace: "my-app" }); const project = await store.projects.create("My project", { frame_rate: 30 }); ``` To bring your own file storage or monitoring, pass `services: { files, monitoring }`. ## Open a backend on a canvas ```ts import { EditorBackendImpl } from "@vidstudio/engine"; const backend = new EditorBackendImpl({ store, projectId: project.id, previewCanvas: canvas, options: { namespace: "my-app" }, }); await backend.open(); await backend.setCanvasSize({ width: 1280, height: 720 }); await backend.importVideo(file); const frame = await backend.renderFrameAt(0.5); ``` Without a canvas the backend runs headless: editing, snapshots, `importAsset`, `importFromUrl`, `saveDocument` and `loadDocument` work, while playback, the imports that place media on the timeline (`importVideo`, `importAudio`, `importImage`, `addTextTrack`), `renderFrameAt`, export and transcription do not. ## Save and load a project `saveDocument()` returns the project as plain JSON (`document`) and a `media` list with one entry per media file: `assetId`, `name`, `type`, `bytes`, `hash` and `read()`. `read()` gives a handle to the stored file, not its bytes, so you can upload a gigabyte without holding it in memory. `hash` is the file size plus a SHA-256 of its first and last 64 KB: enough to skip an upload you already have, not a checksum of the whole file. ```ts const { document, media } = await backend.saveDocument(); await fetch("/api/projects/1", { method: "PUT", body: JSON.stringify(document) }); for (const item of media) { if (!(await serverHas(item.hash))) await fetch(`/api/media/${item.hash}`, { method: "PUT", body: await item.read() }); } ``` `loadDocument(document, resolve)` replaces the backend's project with the document. It calls `resolve(asset)` once per media asset of the document; return the `File`, or a web address the engine downloads. Either way the file is written to the editor's file storage as a stream, and proxies are made again in the background. If `resolve` throws, or the download fails, that asset loads with `missing: true`: its clips show a flat grey frame and `exportProject` rejects with the code `missing-media`. Any other failure leaves the project exactly as it was. Undo history is cleared. ```ts await backend.loadDocument(document, (asset) => `/api/media/${hashes[asset.id]}`); ``` `migrateDocument(document)` brings an older document up to the current format; `loadDocument` calls it for you. The format is version 1. ## Media from a web address `importFromUrl(url, { type })` downloads the file into file storage as a stream and adds it to the project's media, like a file the user picked. `type: "audio"` keeps only a video's sound. A failed download rejects with the code `import-failed`. Web addresses on another origin, for `importFromUrl` and for a resolver alike, only work when that server allows your page to read them: it must send `Access-Control-Allow-Origin` with your origin (or `*`). If the page runs with the cross-origin isolation headers, the server must also send `Cross-Origin-Resource-Policy: cross-origin`. ## Fonts Text tracks load fonts from `/fonts/` by default. Copy `node_modules/@vidstudio/engine/assets/fonts` to `public/fonts`, or serve them elsewhere and set `options.assets.fonts` to that folder's address, ending in a slash. ## Models for captions and background removal Transcription loads Whisper models, and background removal loads MediaPipe files, from `/models/` by default. Run `npx vidstudio-fetch-models public/models` to put them in a folder you serve. This downloads `tiny.en` (used on phones) and `base.en` (used on desktops), about 450 MB, plus about 19 MB for MediaPipe. `--models all` fetches every model in the manifest, `--from ` copies from an existing folder instead of downloading, and `--no-mediapipe` skips background removal. If the folder is served somewhere else, set `options.assets.models` to its address, ending in a slash. ## Cross-origin isolation is optional The engine works without the `Cross-Origin-Opener-Policy: same-origin` and `Cross-Origin-Embedder-Policy: require-corp` headers. With them, transcription runs on several threads. We measured 146 seconds of speech with base.en on a ten-core laptop: about 16 seconds with the headers and about 30 seconds without. Import, preview and export work either way. ## Vite Set the worker format to ES modules. The engine's workers load their code in chunks, and Vite's default worker format cannot emit chunks: `defineConfig({ worker: { format: "es" } })`. Webpack 5 and Next.js, with webpack or Turbopack, need no setting. ## Licence and watermark Pass the licence you were issued as an option. The drop-in editor takes the same thing as its `licence` prop. ```ts const backend = new EditorBackendImpl({ store, projectId: project.id, previewCanvas: canvas, options: { licence: { name: "Acme Ltd", key: "eyJuYW1lIjoi…" } }, }); ``` **Getting a key.** Write to vidstudio446@gmail.com with your domains. A key covers a list of domains until a date. **How the check works.** `open()` checks the key once, offline, in about a millisecond. It makes no network call, now or later. The engine sends no usage reports of any kind. `getLicenceState()` resolves to the outcome: - `"valid"`: the key's signature checks out, its name equals `licence.name`, it has not expired, and the page's host name is one of its domains or a subdomain of one. A key for `example.com` covers `example.com` and `app.example.com`, but not `example.org` or `notexample.com`. Ports do not matter, and neither does `www.`, which is just a subdomain. - `"localhost"`: the page is served from `localhost` or `127.0.0.1` with no key that covers it. Development works with no key and no network. - `"invalid"`: anything else. This includes a missing key, a key for another domain, an expired key, and a browser that cannot check Ed25519 signatures. Browsers that can: Chrome 137, Safari 17 and Firefox 129, or later. The check also needs a secure context (HTTPS, or localhost). **The watermark rule.** In the `"invalid"` and `"localhost"` states, a small "vidstudio.app" mark sits in the lower right of the preview, of `renderFrameAt`, and of every exported frame. Nothing else changes: every feature works without a licence. The licence allows this state for development on localhost; serving the engine to the public on another domain requires a key for that domain. With `"invalid"`, the drop-in editor also shows a notice the user can dismiss. To try the licensed picture on your machine, use a key whose domains include `localhost`. **Engine objects built without a backend.** The player, the project exporter and the frame composer (from `@vidstudio/engine/internal`) draw the mark unless they are handed a `LicenceGrant`. Only `checkPageLicence(licence)` makes one: it checks the licence with the built-in key against the host name of the current page. Run it once and pass the grant to each object. A grant cannot be forged, and an export worker repeats the check against its own origin. **The key format.** The key is two base64url strings without padding, joined by a dot. The first is the UTF-8 JSON token `{ "name", "domains", "expires", "plan" }`. The second is the Ed25519 signature of exactly those bytes. `expires` is an ISO 8601 date or date-time; a bare date ends at midnight UTC at the start of that day. `checkLicence(licence, hostname, { publicKey, now })` and `readLicenceKey(key, publicKey)` are exported, so you can check a key outside a page. **Keys for tests.** No option replaces the public key, so a deployed page always checks against ours. The engine's own tests use `withTestPublicKey(publicKey)` or `testLicenceOptions(domains)` from `@vidstudio/engine/testing`. These work only on the engine's source; the published bundle throws. To sign keys with your own keypair, make one with `node bin/make-keypair.mjs `, which prints the raw public key in base64url. Then sign with `node bin/sign-licence.mjs --private-key --name --domains a.com,b.com --expires 2027-12-31 --plan pro`, which prints the key. Both scripts live in the repository, not in the published package. **The signing key.** `LICENCE_PUBLIC_KEY` in `src/licence.ts` is the production public key. Its private half lives outside the repository and signs every issued key, including vidstudio.app's own (`vidstudio-app/src/engine-options.ts`). ======================================================================== # packages/editor/README.md ======================================================================== # @vidstudio/editor A video editor you drop into your own site: timeline with video, audio, image and text tracks; cut, trim, move and cross-fade; effects with keyframes; captions from on-device speech recognition; export to MP4. Everything runs in the user's browser. Nothing is uploaded. Without a licence key the editor works in full on localhost and draws a small "vidstudio.app" mark on the preview and on every exported frame. Serving it on any other domain requires a key for that domain, which also removes the mark. See "Licence" below. ## Install ``` npm install @vidstudio/editor ``` `react` and `react-dom` 18 or later are peer dependencies. The engine comes with the package. ## Mount it The editor fills its parent, so give the parent a height. ```tsx import { VidstudioEditor, createStore } from "@vidstudio/editor"; import "@vidstudio/editor/editor.css"; const store = await createStore({ namespace: "my-app" }); const project = await store.projects.create("My project", { frame_rate: 30 });
``` Without React, mount it on any element: ```ts import { mountEditor, createStore } from "@vidstudio/editor"; const handle = mountEditor(document.getElementById("editor"), { store, projectId: project.id }); handle.update({ theme: lightTheme }); handle.unmount(); ``` The store keeps projects in the browser's IndexedDB and media in its private file storage. One namespace per editor on a page. ## Two folders to serve - **Fonts.** Copy `node_modules/@vidstudio/engine/assets/fonts` to `public/fonts`, or serve them elsewhere and pass `options: { assets: { fonts: "https://…/fonts/" } }`. - **Models**, only if you keep captions on. Run `npx vidstudio-fetch-models public/models` once (about 450 MB), or pass `options: { assets: { models: "https://…/models/" } }`. With `features={{ captions: false }}` nothing is downloaded. ## Props | Prop | What it does | |---|---| | `store`, `projectId` | The project to edit. Required. | | `options` | Engine options: `namespace`, `assets`, `renderer` (`"pixi"` or `"canvas2d"`), `exportInWorker`, `licence`, `services`. | | `theme` | A theme object. `darkTheme` (default) and `lightTheme` are exported; pass any subset of keys to override colours, radii and the font. | | `features` | Flags, all on by default: `sourceMonitor`, `tour`, `safeZones`, `captions`, `transitions`, `keyframes`, `export`, `projectSettings`, `projectFiles`, `textTracks`, `imageImport`, `audioImport`, `shortcutsHelp`. | | `licence` | `{ name, key }` as issued. Same as `options.licence`. | | `shortcuts` | `"scoped"` (default): keyboard shortcuts work only when the editor has focus. `"global"`: the whole page. | | `onEvent` | Every engine event: `timeline-changed`, `export-started`, `export-finished`, `media-imported`, `preview-played` and more, plus two user interface events: `export-downloaded` and `export-released`. | | `onExport` | `(result, file) => void`. Replaces the download button's action. | | `onSaveDocument`, `onOpenDocument` | Take over "Save project" and "Open project". Save gets the project as JSON plus a list of media handles; open returns a document and a resolver. Without them the editor writes and reads one zip file. | | `onClose` | Shows a Close button that calls it. | ## Bundlers Vite: set `worker: { format: "es" }` in `vite.config`. Webpack 5 and Next.js need nothing. The package needs a bundler; there is no script-tag build. ## Cross-origin isolation is optional The editor runs without the `Cross-Origin-Opener-Policy` and `Cross-Origin-Embedder-Policy` headers. With them, captions transcribe about twice as fast. ## Licence The editor checks the key once when the project opens, offline. On `localhost` no key is needed and the mark is drawn. On any other host the mark is drawn, plus a small notice the user can dismiss, until a key covering that domain is passed in; the licence requires a key for any domain you serve to the public. Keys: vidstudio446@gmail.com. The full terms are in `LICENSE`; dependencies are listed in `THIRD_PARTY_NOTICES.md`. The engine underneath is `@vidstudio/engine`; its README covers the programmatic surface, saving and loading projects, and media from web addresses. ======================================================================== # packages/engine/dist/types/src/index.d.ts ======================================================================== export type * from "./editor/backend/types"; export { EditorBackendImpl, type EditorBackendDeps } from "./editor/backend/EditorBackendImpl"; export { EditorBackendError, type EditorBackendErrorCode } from "./editor/backend/errors"; export { ModelStore } from "./models/model"; export { createStore, type CreateStoreOptions } from "./models/create-store"; export { IndexedDBAdapter } from "./models/lib/db/index-db/indexeddb-adapter"; export { MemoryStorageAdapter } from "./models/lib/db/memory/memory-storage-adapter"; export type { StorageAdapter } from "./models/plugin"; export type { EngineServices } from "./services"; export type * from "./monitoring/types"; export type * from "./models/types"; export type { DeletedProjectSummary } from "./models/model"; export type { ClipRepository } from "./models/clip/clip-repository"; export type { ClipTransformRepository } from "./models/transform/clip-transform-repository"; export type { TransformRepository } from "./models/transform/transform-repository"; export type { KeyframeRepository } from "./models/keyframe/keyframe-repository"; export type { MediaAssetRepository } from "./models/media-asset/media-asset-repository"; export type { CreateProjectOptions, ProjectRepository } from "./models/project/project-repository"; export type { TrackRepository } from "./models/track/track-repository"; export type { TransitionRepository } from "./models/transition/transition-repository"; export type { CaptionConfigRecord, CaptionConfigRepository, TranscriptRecord, TranscriptRecordRepository, TranscriptSegment, TranscriptSegmentRepository } from "./models/transcript/transcript-store"; export type { WordToken } from "./models/transcript/types"; export type { CaptionStyle } from "./processing/caption-text/caption-presets"; export type { StorageDescription, StoredEntry } from "./lib/file-storage/types"; export type { AudioInfo, VideoInfo, VideoProbeInfo } from "./editor/engine/types"; export type { CaptionClipSpec } from "./editor/captions/transcribe-clip"; export type { DetectedCodecs } from "./editor/export/codec-support"; export type { UndoManager, UndoableCommand } from "./editor/undo/undo-manager"; export { OPFSStorageProvider, type StorageProvider } from "./lib/file-storage"; export { MemoryFileStorage } from "./lib/file-storage/memory-provider"; export { probeFile } from "./editor/engine/import/demuxer"; export { seedTransformDefinitions } from "./editor/transforms/seed-transforms"; export * from "./editor/project-document"; export { detectCodecs } from "./editor/export/codec-support"; export type { ExportProgress, ExportResult, ExportSettings, SkippedAudioClipSummary } from "./editor/export/types"; export { resolveOptions, storageNames, type EngineOptions, type ResolvedEngineOptions } from "./options"; export type { DecodeTuning } from "./editor/engine/decode/decode-tuning"; export { checkLicence, checkPageLicence, readLicenceKey, LICENCE_PUBLIC_KEY, type Licence, type LicenceGrant, type LicenceState, type LicenceToken } from "./licence"; ======================================================================== # packages/engine/dist/types/src/editor/backend/types.d.ts ======================================================================== import type { Clip, ClipTransform, Keyframe, MediaAsset, MediaType, Project, TextProperties, Transform, Transition, TransitionType, WorkArea } from "../../models/types"; import type { Unsubscribe } from "../../lib/event-bus"; import type { PlayerState } from "../engine/player"; import type { MediaProbe } from "../engine/types"; import type { ResolvedTransform } from "../engine/transforms/types"; import type { GeometryValues, GeomType, Rect } from "../transforms/clip-geometry"; import type { ExportProgress, ExportResult, ExportSettings } from "../export/types"; import type { CaptionClipSpec } from "../captions/transcribe-clip"; import type { Progress as TranscriptionProgress } from "../../processing/transcription/types"; import type { MediaManifest, MediaResolver, ProjectDocument, ProjectSnapshot } from "../project-document"; import type { LicenceState } from "../../licence"; export type { Unsubscribe, PlayerState, MediaProbe, ResolvedTransform, GeometryValues, GeomType, Rect, TranscriptionProgress, LicenceState }; /** * The one door between the editor UI (or a test, or a coding agent) and the * project. Times are seconds. Every method that changes the project is async, * runs on the backend's serial queue, is undoable unless stated, and ends by * emitting an event. Reads that hit the store are async; reads of engine state * are sync. * * The design lives in design-docs/video-editor/agent-interface/backend.md. */ export interface EditorBackend { readonly projectId: string; /** * Runs the licence check, seeds effect definitions, renumbers an old project's track orders * once (see stacking-migration.ts) and loads the project into the player (if a preview is * attached). Missing proxies are made in the background afterwards. */ open(): Promise; /** Cancels queued work, drops every listener and destroys the player. Safe to call twice. */ dispose(): Promise; /** Hears every event the backend emits; returns the function that unsubscribes. */ subscribe(listener: (event: EditorEvent) => void): Unsubscribe; /** True when constructed without a preview canvas. Engine methods throw in this mode. */ isHeadless(): boolean; /** * The outcome of the licence check `open()` runs once; resolves when it has run. * "invalid" and "localhost" draw a watermark on the preview, `renderFrameAt` and exports. */ getLicenceState(): Promise; /** The project row. Throws a plain Error when the store has no project with this id. */ getProject(): Promise; /** The timeline as the panel draws it: tracks, layers, audio, transitions and the duration. */ getTimeline(): Promise; /** The clip row, or undefined when there is no such clip. */ getClip(clipId: string): Promise; /** The clip on the track playing at `timeSec` (start inclusive, end exclusive), or undefined. */ findClipAt(trackId: string, timeSec: number): Promise; /** Every effect on the clip with its definition and keyframes; throws when a definition is missing. */ getClipEffects(clipId: string): Promise; /** Every effect definition a clip can take, seeded by `open()`. */ getEffectCatalog(): Promise; /** Every media file stored in the project, placed on the timeline or not. */ getMediaAssets(): Promise; /** Every transition between two clips of this project. */ getTransitions(): Promise; /** The project with ids replaced by stable labels and numbers rounded, for comparing in tests. */ snapshot(): Promise; /** Sets the project name as given; nothing is checked. */ renameProject(name: string): Promise; /** Fixes the output size in pixels; `null` lets the canvas follow the first video again. */ setCanvasSize(size: { width: number; height: number; } | null): Promise; /** Sets the frame rate used for frame stepping and export; nothing is checked. */ setFrameRate(fps: number): Promise; /** Sets the range to play and export. A first range switches looping on; `null` clears the range and the loop. */ setWorkArea(range: WorkArea | null): Promise; /** Switches looping over the work area on or off. */ setLoopPlayback(on: boolean): Promise; /** Stores the file and adds a video track playing it, after everything else unless `startSec` says where; import-failed when it does not decode. Not undoable. */ importVideo(file: File, options?: PlaceOptions): Promise; /** Stores the file and adds an audio track playing it; `audio-loading` reports the decode. import-failed when it does not decode. Not undoable. */ importAudio(file: File, options?: PlaceOptions): Promise; /** Stores the picture and adds an image track showing it; import-failed when it does not decode. Not undoable. */ importImage(file: File, options?: PlaceOptions): Promise; /** Adds an image track holding a canvas-sized flat colour (any CSS colour) starting at `atTimeSec`. Not undoable. */ addColorLayer(color: string, atTimeSec: number): Promise; /** Adds a text track with one "Text" clip of five seconds at the playhead; change it with `updateClip` and `trimClip`. Not undoable. */ addTextTrack(): Promise; /** Adds a text track with one clip per caption, all in `style`. Not undoable. */ addTextTrackWithClips(captions: CaptionClipSpec[], style: TextProperties): Promise; /** Adds a clip of a stored asset to an existing video or audio track; type-mismatch for a track of another kind. Not undoable. */ addClipFromAsset(trackId: string, assetId: string, options: PlaceOptions): Promise; /** * A new video or audio track playing an asset that is already in the * project (the source monitor's "Add to Timeline"). Lands after everything * else unless `startSec` says where. */ addTrackFromAsset(assetId: string, type: "video" | "audio", options?: PlaceOptions): Promise; /** Removes the track and its clips. The partner of a linked clip on it is unlinked, since its clip is gone. */ removeTrack(trackId: string): Promise; /** Shifts the whole track so it starts at `startSec`; a text track moves its first clip. Not undoable. */ setTrackStartTime(trackId: string, startSec: number): Promise; /** Sets the track's volume; type-mismatch for anything but an audio track. Not undoable. */ setTrackVolume(trackId: string, volume: number): Promise; /** * Stores a media file in the project without placing it on the timeline * (the source monitor's Import). `probe` is what the caller already learned * about the file; it fills the asset row. Not undoable. Works headless. */ importAsset(file: File, probe: MediaProbe): Promise; /** The media file behind an asset (for waveforms and transcription). Works headless. */ readAssetFile(assetId: string): Promise; /** The asset's proxy when it has one, else the original: lighter to preview. Works headless. */ readAssetPreviewFile(assetId: string): Promise; /** * Videos whose proxy is being made, by asset id, each 0 to 1. Proxies are made * in the background after an import and on open; `proxy-progress` reports changes. */ getProxyProgress(): Record; /** * Audio tracks being added whose sound is still decoding (a long file takes a while). * `audio-loading` reports changes. */ getLoadingAudioCount(): number; /** * Fetches a web address into file storage as a stream (never whole into memory), * probes it and adds it like `importAsset`. `type: "audio"` keeps only a video's sound. * Cross-origin addresses must allow this page (`Access-Control-Allow-Origin`). * A failed download is import-failed. Not undoable. Works headless. */ importFromUrl(url: string, options?: { type?: "video" | "audio"; }): Promise; /** * The project as plain JSON, plus one manifest entry per stored media file. An entry's * `read()` hands back a handle to the stored file, never its bytes. */ saveDocument(): Promise<{ document: ProjectDocument; media: MediaManifest; }>; /** * Replaces this project's content with `document`. `resolve` gives each asset's file or * a web address; either is written to file storage as a stream. An asset it fails on * loads with `missing: true`: its clips show a flat placeholder and export refuses * (missing-media). Any other failure leaves the project as it was. Clears undo and redo. */ loadDocument(document: ProjectDocument, resolve: MediaResolver): Promise; /** * `frontToBack` must name every visual track of the project exactly once, * front first, which is the order the timeline shows. Writes new orders so the * picture matches. Refuses audio tracks (type-mismatch) and incomplete or * repeated lists (invalid-argument). No undo step and no event when nothing changes. */ reorderTracks(frontToBack: string[]): Promise; /** Visual tracks only (type-mismatch for audio, which keeps its mute). A hidden track is not drawn. */ setTrackHidden(trackId: string, hidden: boolean): Promise; /** `null` or an empty string clears the name back to the default. Trimmed, at most 60 characters. */ renameTrack(trackId: string, label: string | null): Promise; /** * Cuts the clip in two at `timelineSec` and returns the new second half. A linked clip * whose partner is also cut by that time is split with it, and the two new halves are * linked to each other. One undo step. invalid-split when the cut is too near an edge. */ splitClipAt(clipId: string, timelineSec: number): Promise; /** The same as `deleteClips([clipId])`. */ deleteClip(clipId: string): Promise; /** * Deletes every named clip and the linked partner of each, in one undo step. An unknown * id is clip-not-found and nothing is deleted. An empty list changes nothing. */ deleteClips(clipIds: string[]): Promise; /** Trims this one clip only; a linked partner is left alone so sound and picture can be cut apart. */ trimClip(clipId: string, window: ClipWindow): Promise; /** The same as `placeClips` with one clip kept on its own track. */ moveClip(clipId: string, newStartSec: number): Promise; /** * Puts every clip at its placement in one undo step, or refuses and changes nothing: * a track of another kind (type-mismatch), a start before zero (invalid-argument), a * clip overlapping another on its new track (clip-overlap). Judged on the final layout, * so clips moving together may swap places. A transition whose two clips no longer * touch on one track is removed in the same step. A linked clip brings its partner: a * partner that is not named is placed on its own track shifted by the same seconds, and * a pair named with two different shifts is refused (invalid-argument). */ placeClips(placements: ClipPlacement[]): Promise; /** * Marks a video clip and the audio clip that is its sound as linked. From then on the two * move, delete and split together. Refuses clips that are not one video and one audio * (type-mismatch) and a clip that is linked already (invalid-argument). */ linkClips(videoClipId: string, audioClipId: string): Promise; /** Breaks the link the clip is in. Refuses a clip that is not linked (invalid-argument). */ unlinkClip(clipId: string): Promise; /** Writes the given clip fields as one undo step named `label`; nothing but the clip's existence is checked. */ updateClip(clipId: string, patch: ClipPatch, label?: string): Promise; /** * Writes only the touched style properties onto each text clip, leaving the * rest of every clip's style alone. One undo step. Returns how many clips changed. */ restyleTextClips(clipIds: string[], changes: Partial): Promise; /** Copies the clip with its effects and keyframes to the backend's clipboard. Not undoable. */ copyClip(clipId: string): Promise; /** Pastes the clipboard clip at `atTimeSec`; null when the clipboard is empty, type-mismatch for a track of another kind, clip-overlap when it would overlap. */ pasteClip(trackId: string, atTimeSec: number): Promise; /** Pastes a copy of the clip right after it on its own track; clip-overlap when the next clip is in the way. */ duplicateClip(clipId: string): Promise; /** Whether `copyClip` has put a clip on the clipboard. */ hasClipboard(): boolean; /** Sets or replaces the transition between two clips that meet on one track, trimming a clip to make footage when needed; invalid-argument when they do not meet, have no media or the duration does not fit. */ setTransition(outgoingClipId: string, incomingClipId: string, type: TransitionType, durationSec: number): Promise; /** Removes the transition; transition-not-found when there is none. */ removeTransition(transitionId: string): Promise; /** Adds an effect from the catalog (transform-not-found otherwise). Recipes (zoom presets) open their window at `atClipLocalSec` (default 0); other effects ignore it. */ addEffect(clipId: string, transformId: string, options?: AddEffectOptions): Promise; /** Removes the effect and its keyframes; effect-not-found when there is none. */ removeEffect(clipTransformId: string): Promise; /** `orderedClipTransformIds` must name every effect on the clip exactly once. */ reorderEffects(clipId: string, orderedClipTransformIds: string[]): Promise; /** * Merges `params` over the effect's stored params (a key set to `undefined` * is removed). With `record: false` the write is a live preview: no undo * step; the next recorded write on the same effect undoes back to the params * from before the first preview. */ setEffectParams(clipTransformId: string, params: Record, options?: SetEffectParamsOptions): Promise; /** Sets the clip-local span the effect applies over; invalid-argument when it starts before 0 or ends before it starts. */ setEffectWindow(clipTransformId: string, window: { start_offset: number; end_offset: number; }): Promise; /** Adds a keyframe, or changes the one already at that time, and returns its id; invalid-argument for a non-finite value, an unknown easing or a time outside the clip. */ setKeyframe(clipTransformId: string, param: string, clipLocalSec: number, value: number, easing?: string): Promise; /** Moves the keyframe, taking the place of one already at the new time; invalid-argument for a time outside the clip. */ moveKeyframe(keyframeId: string, newClipLocalSec: number): Promise; /** Sets the keyframe's easing; invalid-argument for an unknown easing name. */ setKeyframeEasing(keyframeId: string, easing: string): Promise; /** Deletes the keyframe; keyframe-not-found when there is none. */ deleteKeyframe(keyframeId: string): Promise; /** Switches animation on for one param: its stored keyframes come back, or a first one is made at `clipLocalSec` from the current value (invalid-argument outside the clip). */ activateKeyframing(clipTransformId: string, param: string, clipLocalSec: number): Promise; /** Switches animation off for one param; its keyframes stay stored so `activateKeyframing` can bring them back. */ deactivateKeyframing(clipTransformId: string, param: string): Promise; /** Eases every keyframe lane of every effect on the clip. Returns how many changed. */ easeAllKeyframes(clipId: string): Promise; /** Eases one lane: the keyframes of one param on one effect. Returns how many changed. */ easeKeyframeLane(clipTransformId: string, param: string): Promise; /** * Old projects kept opacity on the clip row. Moves it into an Opacity row of * the Transform group so the rows are the single owner; the picture does not * change. Returns false when there was nothing to move. Not undoable. */ adoptLegacyOpacity(clipId: string): Promise; /** * Persists the Transform group rows for `changed`. A type that is animated * gets a keyframe at `atClipLocalSec`; one that is not gets a static value * (mode "auto"). "keyframe" always keyframes, "static" always freezes. */ setClipGeometry(clipId: string, geometry: GeometryValues, changed: Iterable, options?: SetClipGeometryOptions): Promise; /** Engine-only, not persisted: shows a drag in progress. Cleared by the next persisted change. */ previewClipGeometry(clipId: string, geometry: GeometryValues): void; /** Takes back the latest undo step and refreshes everything; does nothing when there is none. */ undo(): Promise; /** Replays the latest undone step; does nothing when there is none. */ redo(): Promise; /** Whether `undo()` has a step to take back. */ canUndo(): boolean; /** Whether `redo()` has a step to replay. */ canRedo(): boolean; /** Starts the preview and emits `preview-played`. */ play(): void; /** Pauses the preview. */ pause(): void; /** Moves the playhead to `timeSec`. */ seek(timeSec: number): Promise; /** Steps the playhead one frame forward at the project frame rate. */ nextFrame(): Promise; /** Steps the playhead one frame back at the project frame rate. */ previousFrame(): Promise; /** Sets the preview speed, 1 being normal. */ setPlaybackRate(rate: number): void; /** The playhead in seconds; 0 before `open()`. */ getCurrentTime(): number; /** The timeline length in seconds; 0 before `open()`. */ getDuration(): number; /** The state the player last reported; "idle" before `open()`. */ getPlayerState(): PlayerState; /** The clip's effects resolved at the playhead, for the canvas overlay. */ getResolvedTransformsAt(clipId: string): ResolvedTransform[]; /** The rectangle the clip's picture fills on the canvas before its effects, or null when its frame is not loaded. */ getClipBaseRect(clipId: string): Rect | null; /** Whether any of `types` animates on the clip, and whether a keyframe sits at `clipLocalSec`. */ getClipGeometryKeyframeInfo(clipId: string, clipLocalSec: number, types: readonly GeomType[]): GeometryKeyframeInfo; /** The size the preview draws at, or null before the renderer has one. */ getCanvasSize(): { width: number; height: number; } | null; /** Which text clip the preview outlines as selected; null for none. */ setSelectedTextClipId(clipId: string | null): void; /** The text clips showing at the playhead. */ getActiveTextClipModels(): Clip[]; /** Composes one frame at the preview size: a slow cold path for tests and thumbnails, not scrubbing. invalid-argument before 0; engine-required headless. */ renderFrameAt(timelineSec: number): Promise; /** Encodes the timeline, or `range`, to an MP4 in file storage, reporting `export-progress`; nothing-to-export when empty, missing-media when a placed file never arrived, export-cancelled after `cancelExport()`, engine-required headless. */ exportProject(settings: ExportSettings, range?: { startSec: number; endSec: number; }): Promise; /** Asks the running export to stop, so `exportProject` rejects with export-cancelled. Nothing happens when none runs. */ cancelExport(): void; /** The finished MP4 an export left behind (`ExportResult.opfsPath`), for the download button. */ readExportFile(opfsPath: string): Promise; /** * Drop a finished export's MP4 and the project's export folder from browser * storage. Call it once the user is done with the result, otherwise every * export the user downloads sits in storage for ever. Never rejects, and * never before the browser has finished streaming a download from the file. */ discardExportFile(opfsPath: string): Promise; /** Transcribes the clip's sound on the device and adds a text track of captions, returning how many (none: no track). invalid-argument for a clip without media; engine-required headless. */ transcribeClipToTextTrack(clipId: string, opts: { signal?: AbortSignal; onProgress?: (p: TranscriptionProgress) => void; }): Promise; } /** What the timeline panel draws. */ export interface TimelineView { /** Back to front by the stacking rule (`compareStacking`), audio included. */ tracks: TrackView[]; /** Visual tracks, front first: the order the layer list draws. */ layers: TrackView[]; /** Audio tracks, in their own order. */ audio: TrackView[]; /** Only transitions whose both clips are in this project. */ transitions: Transition[]; /** The latest clip end over every track; 0 for an empty project. */ durationSec: number; } export interface TrackView { id: string; type: MediaType; order: number; label: string | null; /** Not drawn in the preview or the export. Always false for audio. */ hidden: boolean; /** Span from the first clip's start to the last clip's end (both 0 when empty). */ startSec: number; durationSec: number; /** Audio tracks: the first clip's volume; every other track: 1. */ volume: number; /** Text tracks: from the first clip; every other track: "" and "Inter". */ textPreview: string; fontFamily: string; /** Sorted by start_time. */ clips: ClipView[]; } export interface ClipView { id: string; startSec: number; durationSec: number; sourceOffsetSec: number; /** Infinity for text and image clips, which have no natural length. */ assetDurationSec: number; /** Null for text clips, which have no media. */ assetId: string | null; assetName: string | null; /** The clip this one is linked with (its sound or its picture), or null. */ linkedClipId: string | null; } export interface GeometryKeyframeInfo { /** Some keyframes are switched on for the asked types. */ animated: boolean; /** One of them sits at the asked time (within half a frame). */ atPlayhead: boolean; } export interface ClipEffectView { clipTransform: ClipTransform; definition: Transform; keyframes: Keyframe[]; } export interface AddEffectOptions { /** Clip-local seconds where a recipe's window starts. */ atClipLocalSec?: number; } export interface SetEffectParamsOptions { /** Undo label; defaults to "Change effect params". */ label?: string; /** False during a slider drag: write for preview only, no undo step. Default true. */ record?: boolean; } export interface SetClipGeometryOptions { /** Clip-local seconds a keyframe lands at; default 0. */ atClipLocalSec?: number; /** Default "auto": keyframe the types that already animate, freeze the rest. */ mode?: "auto" | "keyframe" | "static"; /** Undo label; defaults to "Transform clip". */ label?: string; } export interface PlaceOptions { startSec?: number; sourceOffsetSec?: number; durationSec?: number; /** * A clip already on the timeline that the new clip is the picture or the sound of; the * two are linked. Honoured by addTrackFromAsset and addClipFromAsset, refused before * anything is added when the pair cannot be linked (type-mismatch, invalid-argument). */ linkWithClipId?: string; /** * The caller places this picture's sound next, linked to it. Changes only the * `media-imported` event: this call announces "both" and the sound's call none. */ withLinkedSound?: boolean; } /** Where a clip sits on the timeline and which part of its source it plays. */ export interface ClipWindow { start_time: number; duration: number; source_offset: number; } /** Where a clip is to sit: which track, and when it starts. */ export interface ClipPlacement { clipId: string; trackId: string; startSec: number; } export type ClipPatch = Partial>; export type EditorEvent = /** Tracks, clips or transitions changed: re-read getTimeline(). */ { kind: "timeline-changed"; } /** Effects, keyframes or geometry changed on one clip (null: unknown, re-read all). */ | { kind: "effects-changed"; clipId: string | null; } /** Name, canvas size, frame rate, work area or loop changed. */ | { kind: "project-changed"; } | { kind: "player-state"; state: PlayerState; } | { kind: "time"; seconds: number; } | { kind: "duration"; seconds: number; } | { kind: "load-progress"; fraction: number; } /** A proxy is being made (0 to 1), or null once it is finished, failed or stopped. */ | { kind: "proxy-progress"; assetId: string; fraction: number | null; } /** How many audio tracks are being added and still decoding their sound. */ | { kind: "audio-loading"; count: number; } | { kind: "export-progress"; progress: ExportProgress; } | { kind: "warning"; message: string; } | { kind: "error"; message: string; error: Error; } /** play() started the preview. Sent on every play; a host that counts sessions dedupes. */ | { kind: "preview-played"; } /** * An asset was placed on the timeline with addTrackFromAsset or addClipFromAsset. A picture * and its linked sound are one event, "both"; see `PlaceOptions.withLinkedSound`. */ | { kind: "media-imported"; mediaType: "video" | "audio" | "both"; clipDurationSec: number; addedTo: "new_track" | "existing_track"; } | { kind: "export-started"; settings: ExportSettings; } | { kind: "export-finished"; settings: ExportSettings; result: ExportResult; durationMs: number; } /** Any failure but a cancel, refusals included. `phase` is the last progress phase reached. */ | { kind: "export-failed"; settings: ExportSettings; error: Error; phase: ExportProgress["phase"] | null; durationMs: number; } /** A write was refused because browser storage is full. */ | { kind: "storage-full"; operation: "import" | "export" | "delete_track"; error: Error; }; ======================================================================== # packages/engine/dist/types/src/editor/backend/EditorBackendImpl.d.ts ======================================================================== import type { ModelStore } from "../../models/model"; import type { Clip, MediaAsset, Project, TextProperties, Transform, Transition, TransitionType, WorkArea } from "../../models/types"; import { type EngineServices } from "../../services"; import { type EngineOptions } from "../../options"; import { type LicenceState } from "../../licence"; import { type CaptionClipSpec } from "../captions/transcribe-clip"; import { UndoManager } from "../undo/undo-manager"; import { type GeometryValues, type GeomType, type Rect } from "../transforms/clip-geometry"; import type { ExportResult, ExportSettings } from "../export/types"; import { type MediaManifest, type MediaResolver, type ProjectDocument, type ProjectSnapshot } from "../project-document"; import type { AddEffectOptions, ClipEffectView, ClipPatch, ClipWindow, EditorBackend, EditorEvent, GeometryKeyframeInfo, MediaProbe, PlaceOptions, PlayerState, ResolvedTransform, SetClipGeometryOptions, SetEffectParamsOptions, TimelineView, TranscriptionProgress, Unsubscribe, ClipPlacement } from "./types"; export interface EditorBackendDeps { store: ModelStore; projectId: string; /** Leave out to run headless: no player is created and engine methods throw. */ previewCanvas?: HTMLCanvasElement; /** Tests may share one manager; production callers let the backend own it. */ undoManager?: UndoManager; /** File storage and monitoring. Each one left out is the store's. Wins over `options.services`. */ services?: EngineServices; /** What the engine would otherwise read from the page; see `EngineOptions`. */ options?: EngineOptions; } /** * The editor facade. Wiring, not logic: every edit is an existing undo * command, run on a serial queue so two edits never interleave, followed by * one engine refresh and one event. See backend.md for the design. */ export declare class EditorBackendImpl implements EditorBackend { readonly projectId: string; private readonly store; private readonly undoManager; private readonly queue; private readonly events; private readonly services; private readonly options; /** Set once by open(); everything that draws frames gets it, and without it they draw the mark. */ private licence; private readonly licenceChecked; private settleLicence; private readonly previewCanvas; /** Renders into the preview canvas; created by open(), always null while headless. */ private player; /** Stops every proxy still being made when the backend is disposed. */ private readonly proxyAbort; /** Assets whose proxy this session has made, is making, or decided against; each is looked at once. */ private readonly proxyHandled; /** Proxies being made right now, by asset id, 0 to 1. */ private readonly proxyProgress; /** Audio tracks being added right now; the timeline shows a placeholder for each. */ private loadingAudioCount; /** Mirrors the player's state, which it announces on every change. */ private playerState; /** * The last error the player handed to onError. An import that fails with * this same object has already been shown and reported by the player. */ private lastPlayerError; /** The export that is running right now, so cancelExport() can reach it. */ private exporter; /** A cancel that arrived before the exporter existed. Building one is async * (it dumps the project and may spawn a worker), so the window is real. */ private exportCancelRequested; private clipboard; /** * Params an effect had before its first preview-only write (setEffectParams * with record: false), so the recorded write at the end of the drag undoes * back to them and not to the last preview frame. */ private readonly paramsBeforePreview; private disposed; constructor(deps: EditorBackendDeps); open(): Promise; private runLicenceCheck; getLicenceState(): Promise; /** Builds the player and loads the project into it. Does nothing while headless. */ private startPlayer; dispose(): Promise; subscribe(listener: (event: EditorEvent) => void): Unsubscribe; isHeadless(): boolean; getProject(): Promise; getTimeline(): Promise; getClip(clipId: string): Promise; findClipAt(trackId: string, timeSec: number): Promise; getClipEffects(clipId: string): Promise; getEffectCatalog(): Promise; getMediaAssets(): Promise; getTransitions(): Promise; snapshot(): Promise; renameProject(name: string): Promise; setCanvasSize(size: { width: number; height: number; } | null): Promise; setFrameRate(fps: number): Promise; setWorkArea(range: WorkArea | null): Promise; setLoopPlayback(on: boolean): Promise; importVideo(file: File, options?: PlaceOptions): Promise; importAudio(file: File, options?: PlaceOptions): Promise; importImage(file: File, options?: PlaceOptions): Promise; addColorLayer(color: string, atTimeSec: number): Promise; addTextTrack(): Promise; addTextTrackWithClips(captions: CaptionClipSpec[], style: TextProperties): Promise; addClipFromAsset(trackId: string, assetId: string, options: PlaceOptions): Promise; addTrackFromAsset(assetId: string, type: "video" | "audio", options?: PlaceOptions): Promise; /** Refuses a `linkWithClipId` that could never be linked with a clip of `addingType`, before anything is added. */ private checkLinkBeforeAdding; /** * One `media-imported` per picture or sound placed alone, and one per pair: the picture * placed with `withLinkedSound` announces "both", so the sound linked to it stays quiet. */ private announcePlacement; /** Links a just-added clip with the clip the caller named. Part of the import, so not an undo step. */ private markLinkAfterAdding; importAsset(file: File, probe: MediaProbe): Promise; importFromUrl(url: string, options?: { type?: "video" | "audio"; }): Promise; /** The asset row for a file already in storage at `filePath`, filled from what the probe learned. */ private addStoredAsset; saveDocument(): Promise<{ document: ProjectDocument; media: MediaManifest; }>; loadDocument(document: ProjectDocument, resolve: MediaResolver): Promise; removeTrack(trackId: string): Promise; setTrackStartTime(trackId: string, startSec: number): Promise; setTrackVolume(trackId: string, volume: number): Promise; readAssetFile(assetId: string): Promise; readAssetPreviewFile(assetId: string): Promise; getProxyProgress(): Record; getLoadingAudioCount(): number; reorderTracks(frontToBack: string[]): Promise; setTrackHidden(trackId: string, hidden: boolean): Promise; renameTrack(trackId: string, label: string | null): Promise; splitClipAt(clipId: string, timelineSec: number): Promise; /** A SplitClipCommand with the effects, keyframes and transitions it must put back on undo. */ private splitOf; deleteClip(clipId: string): Promise; deleteClips(clipIds: string[]): Promise; /** A DeleteClipCommand with the effects, keyframes and transitions it must bring back on undo. */ private deletionOf; trimClip(clipId: string, window: ClipWindow): Promise; moveClip(clipId: string, newStartSec: number): Promise; placeClips(placements: ClipPlacement[]): Promise; /** The work of placeClips, for callers already on the queue. */ private placeClipsNow; /** Every track and clip of this project, for rules that judge the whole layout. */ private projectRows; updateClip(clipId: string, patch: ClipPatch, label?: string): Promise; restyleTextClips(clipIds: string[], changes: Partial): Promise; copyClip(clipId: string): Promise; pasteClip(trackId: string, atTimeSec: number): Promise; duplicateClip(clipId: string): Promise; hasClipboard(): boolean; linkClips(videoClipId: string, audioClipId: string): Promise; unlinkClip(clipId: string): Promise; setTransition(outgoingClipId: string, incomingClipId: string, type: TransitionType, durationSec: number): Promise; removeTransition(transitionId: string): Promise; addEffect(clipId: string, transformId: string, options?: AddEffectOptions): Promise; removeEffect(clipTransformId: string): Promise; reorderEffects(clipId: string, orderedClipTransformIds: string[]): Promise; setEffectParams(clipTransformId: string, params: Record, options?: SetEffectParamsOptions): Promise; setEffectWindow(clipTransformId: string, window: { start_offset: number; end_offset: number; }): Promise; setKeyframe(clipTransformId: string, param: string, clipLocalSec: number, value: number, easing?: string): Promise; moveKeyframe(keyframeId: string, newClipLocalSec: number): Promise; setKeyframeEasing(keyframeId: string, easing: string): Promise; deleteKeyframe(keyframeId: string): Promise; activateKeyframing(clipTransformId: string, param: string, clipLocalSec: number): Promise; deactivateKeyframing(clipTransformId: string, param: string): Promise; easeAllKeyframes(clipId: string): Promise; easeKeyframeLane(clipTransformId: string, param: string): Promise; adoptLegacyOpacity(clipId: string): Promise; setClipGeometry(clipId: string, geometry: GeometryValues, changed: Iterable, options?: SetClipGeometryOptions): Promise; previewClipGeometry(clipId: string, geometry: GeometryValues): void; undo(): Promise; redo(): Promise; canUndo(): boolean; canRedo(): boolean; play(): void; pause(): void; seek(timeSec: number): Promise; nextFrame(): Promise; previousFrame(): Promise; setPlaybackRate(rate: number): void; getCurrentTime(): number; getDuration(): number; getPlayerState(): PlayerState; getResolvedTransformsAt(clipId: string): ResolvedTransform[]; getClipBaseRect(clipId: string): Rect | null; getClipGeometryKeyframeInfo(clipId: string, clipLocalSec: number, types: readonly GeomType[]): GeometryKeyframeInfo; getCanvasSize(): { width: number; height: number; } | null; setSelectedTextClipId(clipId: string | null): void; getActiveTextClipModels(): Clip[]; /** * A cold path: every call loads the project, opens a decoder per video * asset and a renderer, composes the one frame and closes it all again, so * it costs hundreds of milliseconds. Meant for tests and thumbnails, not for * scrubbing (the player does that). Runs on the queue so the frame shows a * project no edit is changing halfway through. */ renderFrameAt(timelineSec: number): Promise; exportProject(settings: ExportSettings, range?: { startSec: number; endSec: number; }): Promise; private runExport; /** The export reads every file it plays, so a clip whose file never arrived stops it before it starts. */ private refuseMissingMedia; cancelExport(): void; readExportFile(opfsPath: string): Promise; discardExportFile(opfsPath: string): Promise; transcribeClipToTextTrack(clipId: string, opts?: { signal?: AbortSignal; onProgress?: (p: TranscriptionProgress) => void; }): Promise; /** Nothing reaches the listeners of a disposed backend. */ private emit; private noteStorageFull; /** * The player reports through callbacks fixed at construction; each one * becomes an event. After dispose nothing is forwarded, so a decoder that * is still winding down cannot reach listeners of a dead backend. */ private playerCallbacks; /** * Starts a proxy for every video that would edit more smoothly with one and has * none. Runs on open and after each import, in the background: clips read the * original until the proxy is done, then switch to it. Engine only, since only * the preview reads proxies. */ private ensureProxies; /** Makes one asset's proxy if it needs one. A failure only means the original keeps playing. */ private makeProxy; /** Counts an audio track as loading from the call, including its wait in the queue, until it lands or fails. */ private whileAudioLoads; private setLoadingAudioCount; private setProxyProgress; /** * Runs one import on the queue. A failure is carried out of the queue as a * value and thrown here, because the queue reports every thrown failure to * Sentry and the player has already reported its own (see addVideoTrack); * a failure the player never saw is shown and reported exactly once here. */ private importMedia; /** Runs one track change on the queue (engine required, not undoable), then tells listeners once. */ private changeTracks; /** * Runs one undoable edit of track rows on the queue. `work` returns the * changes; none means nothing to do, so no undo step and no event. */ private editTracks; /** Runs one undoable clip edit on the queue, then refreshes the engine and tells listeners once. */ private editClips; /** * Runs one effect, keyframe or geometry edit on the queue, then refreshes the * engine and tells listeners which clip changed. `work` reports the clip * because most edits only learn it after looking the effect or keyframe up. */ private editEffects; /** Runs one project-row change on the queue, then tells listeners once. Not undoable. */ private changeProject; /** Undo may have touched anything, so refresh everything and let the frontend re-read it all. */ private afterUndoOrRedo; private requireClip; private requireTrack; /** Like requireTrack, but a track of another project counts as not found. */ private requireProjectTrack; private requireAsset; /** * The transitions touching `clip` that would no longer sit at a cut once the * clip occupies `window`. The player and the exporter blend the partner's * footage at the clip's end regardless, so a transition whose cut is gone * plays wrongly; the edit that breaks the cut removes it in the same undo step. */ /** * The transitions on `planned` clips that would no longer join two clips touching on * one track once each planned clip sits where it is planned. A partner that is itself * planned is judged at its planned spot, so clips moving together keep their cross-fade. */ private transitionsDetachedBy; /** One undo step: the detached transitions go first, then the edit itself. */ private removingTransitions; /** The media behind a clip; text clips have none and so cannot take a transition. */ private requireFootage; private requireEffect; /** * Position is stored as a fraction of the canvas. With a player the canvas * is the truth; headless, the project's fixed size, else the clip's own * picture size (what the renderer would follow), else a 1x1 square. */ private canvasSizeFor; private requireKeyframe; /** Keyframe times are clip-local, so anything before 0 or after the clip's end can never play. */ private requireTimeInsideClip; private requireDefinition; /** Engine methods refuse to run headless; with a canvas the player exists once open() has run. */ private enginePlayer; private requirePlayer; } ======================================================================== # packages/engine/dist/types/src/editor/backend/errors.d.ts ======================================================================== /** * Errors raised by the editor backend. The `code` lets the frontend pick a * toast and lets tests assert on the reason without matching message text. */ export type EditorBackendErrorCode = "engine-required" | "clip-not-found" | "track-not-found" | "asset-not-found" | "effect-not-found" | "keyframe-not-found" | "transition-not-found" | "transform-not-found" | "invalid-argument" | "invalid-split" | "type-mismatch" | "clip-overlap" | "nothing-to-export" | "export-cancelled" | "import-failed" | "missing-media" | "not-implemented"; export declare class EditorBackendError extends Error { readonly code: EditorBackendErrorCode; constructor(code: EditorBackendErrorCode, message?: string); } ======================================================================== # packages/engine/dist/types/src/models/model.d.ts ======================================================================== import type { StorageAdapter } from "./plugin"; import { ProjectRepository } from "./project/project-repository"; import { MediaAssetRepository } from "./media-asset/media-asset-repository"; import { TrackRepository } from "./track/track-repository"; import { ClipRepository } from "./clip/clip-repository"; import { TransformRepository } from "./transform/transform-repository"; import { ClipTransformRepository } from "./transform/clip-transform-repository"; import { KeyframeRepository } from "./keyframe/keyframe-repository"; import { TransitionRepository } from "./transition/transition-repository"; import { CaptionConfigRepository, TranscriptRecordRepository, TranscriptSegmentRepository } from "./transcript/transcript-store"; import { type EngineServices } from "../services"; /** What deleting a project gave back, for the "delete as cleanup" metric. */ export interface DeletedProjectSummary { /** Asset rows that went with the project. An asset still in use is not counted. */ assetCount: number; /** * Bytes those assets held, as recorded on their rows. The project's export * output goes too, but nothing records its size, so it is not included. */ bytesFreed: number; } export declare class ModelStore { readonly projects: ProjectRepository; readonly mediaAssets: MediaAssetRepository; readonly tracks: TrackRepository; readonly clips: ClipRepository; readonly transforms: TransformRepository; readonly clipTransforms: ClipTransformRepository; readonly keyframes: KeyframeRepository; readonly transitions: TransitionRepository; readonly transcriptRecords: TranscriptRecordRepository; readonly transcriptSegments: TranscriptSegmentRepository; readonly captionConfig: CaptionConfigRepository; /** Where this store's media files live; deleting a project or asset deletes them there. */ readonly services: EngineServices; constructor(storage: StorageAdapter, services?: EngineServices); /** * Cascade delete: remove a project and EVERYTHING it ever stored — its rows * at every level, and its whole media/proxy folder plus its export folder. * Order: keyframes → clip transforms → transitions → clips → tracks → * transcripts → caption config → media assets → project → files * * Returns what went, counted here where the asset rows are already in hand so * no caller has to read them back after they are gone. */ deleteProject(projectId: string): Promise; /** * Deep-clone a project into a fresh, UNMARKED project (classic-editor surface): * copies media (rows + OPFS files), tracks, clips, clip-transforms, keyframes, * and transitions, remapping every foreign key. Orator transcript/caption rows * are intentionally NOT copied — the clone is an editor project (captions are * materialized as a text track by the caller). Returns the new project id. * * Quota-guarded and self-rolling-back: estimates free OPFS space before starting * and, if any step throws partway, tears the half-built clone down via * `deleteProject` before rethrowing, so a failure never strands a zombie project * or orphaned bytes. */ cloneProject(sourceId: string, name: string): Promise; /** * Cascade delete: remove a track, its clips, their transforms, * and the associated media assets + OPFS files. */ deleteTrack(trackId: string): Promise; /** * Cascade delete: remove a clip and all its transforms. */ deleteClip(clipId: string): Promise; /** * Delete a media asset and its OPFS file. * Does NOT delete clips that reference this asset — caller must handle that. */ deleteMediaAsset(assetId: string): Promise; /** * Reclaim auxiliary video assets that nothing references any more (zero clips * point at them). Deletes each asset's row and OPFS file(s); returns the count * reclaimed. Scoped to `auxiliary` assets so it can NEVER delete user-imported * media — only edit-generated b-roll is eligible. * * These orphans are produced by Orator's in-place video replace: the command * keeps its imported asset across undo (so a redo needn't re-copy the file), * which leaves the asset unreferenced if the edit is undone and abandoned. It * is therefore safe to call ONLY when no live undo history can still resurrect * such an asset — i.e. at project load, where the `UndoManager` starts empty. * Never call it mid-session. */ pruneUnreferencedVideoAssets(projectId: string): Promise; } ======================================================================== # packages/engine/dist/types/src/models/create-store.d.ts ======================================================================== import type { EngineServices } from "../services"; import type { ModelStore } from "./model"; export interface CreateStoreOptions { /** Names the database and the file folder; two editors on one origin need two namespaces. Default "vidstudio". */ namespace?: string; /** Default: the origin private file system under the namespace's folder, and no monitoring. */ services?: EngineServices; } /** Opens the namespace's database and returns the store a host lists, creates and deletes projects through. */ export declare function createStore(options?: CreateStoreOptions): Promise; ======================================================================== # packages/engine/dist/types/src/models/lib/db/index-db/indexeddb-adapter.d.ts ======================================================================== import type { StorageAdapter } from "../../../plugin"; export declare class IndexedDBUnavailableError extends Error { readonly originalError: unknown; constructor(originalError: unknown); } export declare class DbConnectionClosedError extends Error { constructor(); } export declare class IndexedDBAdapter implements StorageAdapter { private db; private readonly name; constructor(options?: { name?: string; }); /** Open (or create) the database. Must be called before any other method. */ open(): Promise; private createStores; private getDb; private tx; get(storeName: string, id: string): Promise; getAll(storeName: string): Promise; getAllByIndex(storeName: string, indexName: string, value: string): Promise; add(storeName: string, record: T): Promise; put(storeName: string, record: T): Promise; delete(storeName: string, id: string): Promise; deleteAllByIndex(storeName: string, indexName: string, value: string): Promise; replaceAllByIndex(storeName: string, indexName: string, value: string, records: T[]): Promise; } ======================================================================== # packages/engine/dist/types/src/models/lib/db/memory/memory-storage-adapter.d.ts ======================================================================== import type { StorageAdapter } from "../../../plugin"; /** * The StorageAdapter contract held in plain Maps. Used by Node tests and by * anything that wants a project without a browser. * * Behaves like the IndexedDB adapter: an index name is the field it reads * (`getAllByIndex("clips", "track_id", id)` filters on `row.track_id`), and * `add` rejects when the id already exists. * * Rows are structured-cloned on the way in and out, like IndexedDB does, so a * caller can never change a stored row by mutating a reference. Rows hold * only JSON-like data, never a File or Blob; keep it that way. */ export declare class MemoryStorageAdapter implements StorageAdapter { private stores; private store; get(storeName: string, id: string): Promise; getAll(storeName: string): Promise; getAllByIndex(storeName: string, indexName: string, value: string): Promise; add(storeName: string, record: T): Promise; put(storeName: string, record: T): Promise; delete(storeName: string, id: string): Promise; deleteAllByIndex(storeName: string, indexName: string, value: string): Promise; replaceAllByIndex(storeName: string, indexName: string, value: string, records: T[]): Promise; private removeMatching; } ======================================================================== # packages/engine/dist/types/src/models/plugin.d.ts ======================================================================== /** Store name constants — used by repositories and the adapter */ export declare const STORE_NAMES: { readonly projects: "projects"; readonly mediaAssets: "media_assets"; readonly tracks: "tracks"; readonly clips: "clips"; readonly transforms: "transforms"; readonly clipTransforms: "clip_transforms"; readonly keyframes: "keyframes"; readonly transitions: "transitions"; readonly orator_transcripts: "orator_transcripts"; readonly orator_transcript_segments: "orator_transcript_segments"; readonly orator_caption_config: "orator_caption_config"; }; /** * Abstract storage contract. IndexedDB is the default implementation, * but any backend (SQLite, REST API, in-memory) can implement this. */ export interface StorageAdapter { /** Get a single record by its primary key */ get(storeName: string, id: string): Promise; /** Get all records in a store */ getAll(storeName: string): Promise; /** Get all records matching a specific index value (foreign key lookup) */ getAllByIndex(storeName: string, indexName: string, value: string): Promise; /** Insert a new record. Rejects if id already exists. */ add(storeName: string, record: T): Promise; /** Replace an existing record (full replacement by primary key). */ put(storeName: string, record: T): Promise; /** Delete a record by primary key */ delete(storeName: string, id: string): Promise; /** Delete all records matching an index value (cascade helper) */ deleteAllByIndex(storeName: string, indexName: string, value: string): Promise; /** * Atomically replace every record matching an index value with a new set, * in a single transaction: delete all matches, then insert `records`. * A concurrent read either sees the entire old set or the entire new set, * never a partial mix, and a mid-write failure rolls back the whole change. */ replaceAllByIndex(storeName: string, indexName: string, value: string, records: T[]): Promise; } ======================================================================== # packages/engine/dist/types/src/services.d.ts ======================================================================== import type { StorageProvider } from "./lib/file-storage/types"; import type { MonitoringAdapter } from "./monitoring/types"; /** What a host hands the engine instead of setting module singletons. */ export interface EngineServices { /** Where media, proxies and exports are stored. Default: the origin private file system. */ files?: StorageProvider; /** Where errors and breadcrumbs go. Default: nowhere; one instance never reports into another's adapter. */ monitoring?: MonitoringAdapter; } /** Read at use, not at construction, so a test that swaps the default storage still reaches it. */ export declare function filesOf(services: EngineServices | undefined): StorageProvider; export declare function monitoringOf(services: EngineServices | undefined): MonitoringAdapter; ======================================================================== # packages/engine/dist/types/src/monitoring/types.d.ts ======================================================================== /** * Monitoring Plugin System - Type Definitions * * These types define the contract for monitoring adapters. * Any monitoring provider (Sentry, Datadog, LogRocket, etc.) * must implement the MonitoringAdapter interface. */ export type MonitoringSeverity = 'fatal' | 'error' | 'warning' | 'info' | 'debug'; export type FeedbackType = 'general' | 'bug' | 'feature'; export interface FeedbackContext { currentTool?: string; processingState?: 'idle' | 'processing' | 'completed' | 'error'; errorContext?: unknown; } export interface FeedbackOptions { type?: FeedbackType; context?: FeedbackContext; } export type BreadcrumbCategory = 'navigation' | 'ui' | 'http' | 'console' | 'ffmpeg' | 'transcription' | 'custom'; export interface Breadcrumb { category: BreadcrumbCategory; message: string; level?: MonitoringSeverity; data?: Record; timestamp?: number; } export interface MonitoringUser { id?: string; email?: string; username?: string; [key: string]: unknown; } export interface MonitoringScope { setTag(key: string, value: string): void; setContext(key: string, context: Record): void; setExtra(key: string, value: unknown): void; setLevel(level: MonitoringSeverity): void; setUser(user: MonitoringUser | null): void; } export interface MonitoringConfig { /** Data source name / endpoint URL */ dsn?: string; /** Environment identifier (development, staging, production) */ environment?: string; /** Application version/release */ release?: string; /** Enable debug logging */ debug?: boolean; /** Sample rate for performance traces (0.0 to 1.0) */ tracesSampleRate?: number; /** Hook to filter/modify events before sending */ beforeSend?: (event: unknown, hint: unknown) => unknown | null; /** Initial tags to set on all events */ initialTags?: Record; /** Initial context to set */ initialContext?: Record>; } /** * The core adapter interface - what each monitoring provider must implement. * * To add a new provider: * 1. Create a new adapter file (e.g., adapters/datadog.ts) * 2. Implement all methods of this interface * 3. Map provider-specific APIs to these common methods */ export interface MonitoringAdapter { init(config: MonitoringConfig): void; captureException(error: unknown, hint?: { extra?: Record; }): string | undefined; captureMessage(message: string, level?: MonitoringSeverity): string | undefined; withScope(callback: (scope: MonitoringScope) => void): void; addBreadcrumb(breadcrumb: Breadcrumb): void; setTag(key: string, value: string): void; setContext(key: string, context: Record): void; setExtra(key: string, value: unknown): void; setUser(user: MonitoringUser | null): void; showFeedbackWidget?(options?: FeedbackOptions): void; attachFeedbackToElement?(element: HTMLElement, options?: FeedbackOptions): () => void; } ======================================================================== # packages/engine/dist/types/src/models/types.d.ts ======================================================================== export type MediaType = "video" | "audio" | "text" | "image"; /** A start/end pair on the timeline, in seconds. Stored as set; consumers clamp to the current duration. */ export interface WorkArea { start_time: number; end_time: number; } /** Top-level container. One project = one editing session. */ export interface Project { id: string; name: string; created_at: number; updated_at: number; canvas_width?: number; canvas_height?: number; frame_rate?: number; /** The slice of the timeline being worked on; playback and export can limit to it. Missing = whole timeline. */ work_area?: WorkArea; /** Playback jumps back to the work area's start (or 0) when it reaches the end. Missing = false. */ loop_playback?: boolean; /** * Which surface owns this project. Absent on classic video-editor projects * (they predate this field and never set it); `'orator'` on projects created * by Orator. Orator only ever opens its own projects, so the editor's media * never bleeds into the Orator library and vice-versa. */ surface?: "orator"; /** * 2 = visual tracks share one stacking order and text is sorted with them. * Missing = the old rule (orders counted per type, text always on top); such a * project is renumbered once when it is opened (see stacking-migration.ts). */ stacking_version?: 2; } /** A source file imported into the project. Stores intrinsic file properties. */ export interface MediaAsset { id: string; project_id: string; type: MediaType; filename: string; file_path: string; file_size: number; duration: number; codec: string; width: number | null; height: number | null; sample_rate: number | null; channel_count: number | null; proxy_path?: string; /** * Intrinsic source orientation, in degrees clockwise, that the frames must be * rotated by to display upright (from the container's rotation matrix/tag). * This is the source's *coded* orientation — it is applied at composite time * (preview and export), never baked into the stored pixels, so the small H.264 * proxy inherits it for free and renders identically to the original. Distinct * from the keyframed user "rotation" transform, which stacks on top of this. * Absent or 0 means the source is already upright. Video only. */ rotation?: 0 | 90 | 180 | 270; /** * Internal asset introduced by an edit (e.g. the b-roll spliced in by an * in-place video replace), NOT user-imported project media. Consumers that * present the project's media library exclude these and never auto-select one * as the active transcription target. Absent/false for normal imported media. */ auxiliary?: boolean; /** * The file could not be found when a project document was loaded (`loadDocument`). * Written at load, never inferred: clips on it show a flat placeholder and export refuses. */ missing?: boolean; } /** A lane on the timeline. Tracks contain clips. */ export interface Track { id: string; project_id: string; type: MediaType; order: number; label: string | null; muted: boolean; locked: boolean; /** Visual tracks only: not drawn in the preview or the export. Missing = visible. */ hidden?: boolean; is_spine?: boolean; } /** Style and position properties for text clips. */ export interface TextProperties { fontFamily: string; fontSize: number; fontColor: string; fontAlpha: number; bold: boolean; italic: boolean; textAlign: "left" | "center" | "right"; positionX: number; positionY: number; backgroundEnabled: boolean; backgroundColor: string; backgroundPadding: number; borderEnabled: boolean; borderWidth: number; borderColor: string; shadowEnabled: boolean; shadowColor: string; shadowX: number; shadowY: number; } export declare const DEFAULT_TEXT_PROPERTIES: TextProperties; /** A placement of a media asset on a track at a specific position. */ export interface Clip { id: string; track_id: string; media_asset_id: string | null; start_time: number; duration: number; source_offset: number; volume: number | null; opacity: number | null; text_content: string | null; text_properties: TextProperties | null; /** The clip this one is linked with: a video clip's sound, or an audio clip's picture. Absent when not linked. */ linked_clip_id?: string; } /** A supported transformation that can be applied to clips. */ export interface Transform { id: string; name: string; type: string; category: string; description: string; default_params: Record; } /** Links a transform to a clip with instance-specific settings. */ export interface ClipTransform { id: string; clip_id: string; transform_id: string; order: number; start_offset: number; end_offset: number; params: Record; } /** A single keyframe — a snapshot of a numeric property value at a specific time. */ export interface Keyframe { id: string; clip_transform_id: string; param_name: string; time: number; value: number; easing: string; } /** Supported transition types between adjacent clips. */ export type TransitionType = "crossfade" | "wipe-left" | "wipe-right" | "push-left" | "push-right"; /** A transition between two adjacent clips on the same track. */ export interface Transition { id: string; outgoing_clip_id: string; incoming_clip_id: string; type: TransitionType; duration: number; easing: string; } ======================================================================== # packages/engine/dist/types/src/models/clip/clip-repository.d.ts ======================================================================== import type { StorageAdapter } from "../plugin"; import type { Clip } from "../types"; export declare class ClipRepository { private storage; constructor(storage: StorageAdapter); getById(id: string): Promise; /** Returns clips sorted by `start_time` */ /** Every clip row in the database, across all tracks. */ getAll(): Promise; getByTrackId(trackId: string): Promise; /** Find all clips that use a specific media asset */ getByMediaAssetId(mediaAssetId: string): Promise; create(data: Omit): Promise; update(id: string, changes: Partial>): Promise; /** Restore a clip with its original ID (for undo). Creates or replaces. */ restore(clip: Clip): Promise; /** Writes the partner onto the row, or takes the field off when `partnerId` is null. */ setLinkedClip(clipId: string, partnerId: string | null): Promise; delete(id: string): Promise; deleteByTrackId(trackId: string): Promise; } ======================================================================== # packages/engine/dist/types/src/models/transform/clip-transform-repository.d.ts ======================================================================== import type { StorageAdapter } from "../plugin"; import type { ClipTransform } from "../types"; export declare class ClipTransformRepository { private storage; constructor(storage: StorageAdapter); getById(id: string): Promise; /** Returns clip-transforms sorted by `order` */ /** Every clip-transform row in the database, across all clips. */ getAll(): Promise; getByClipId(clipId: string): Promise; /** Find all clips using a specific transform */ getByTransformId(transformId: string): Promise; create(data: Omit): Promise; update(id: string, changes: Partial>): Promise; /** Restore a ClipTransform with its original ID (for undo). Creates or replaces. */ restore(ct: ClipTransform): Promise; delete(id: string): Promise; deleteByClipId(clipId: string): Promise; deleteByTransformId(transformId: string): Promise; } ======================================================================== # packages/engine/dist/types/src/models/transform/transform-repository.d.ts ======================================================================== import type { StorageAdapter } from "../plugin"; import type { Transform } from "../types"; export declare class TransformRepository { private storage; constructor(storage: StorageAdapter); getById(id: string): Promise; getAll(): Promise; getByCategory(category: string): Promise; create(data: Omit): Promise; update(id: string, changes: Partial>): Promise; delete(id: string): Promise; } ======================================================================== # packages/engine/dist/types/src/models/keyframe/keyframe-repository.d.ts ======================================================================== import type { StorageAdapter } from "../plugin"; import type { Keyframe } from "../types"; export declare class KeyframeRepository { private storage; constructor(storage: StorageAdapter); getById(id: string): Promise; /** * All keyframes for a ClipTransform, sorted by time ascending. * Returns ALL param_names — caller filters by param_name if needed. */ /** Every keyframe row in the database, across all clip-transforms. */ getAll(): Promise; getByClipTransformId(clipTransformId: string): Promise; /** * Keyframes for a specific (ClipTransform, param) pair, sorted by time. * Loads all keyframes for the ClipTransform, then filters in JS. */ getByClipTransformIdAndParam(clipTransformId: string, paramName: string): Promise; create(data: Omit): Promise; /** * Update mutable fields on a keyframe (time, value, easing). * Does NOT allow changing clip_transform_id or param_name — those are structural. */ update(id: string, changes: Partial>): Promise; /** Restore a keyframe with its original ID (for undo). Creates or replaces. */ restore(kf: Keyframe): Promise; delete(id: string): Promise; /** Delete ALL keyframes belonging to a ClipTransform (cascade helper). */ deleteByClipTransformId(clipTransformId: string): Promise; /** * Upsert: set a keyframe at a specific (clipTransform, param, time). * * If a keyframe already exists at the same (clipTransformId, paramName, time) * within EPSILON (0.001s), update it. Otherwise create a new one. * This prevents duplicate keyframes at the same time from rapid slider drags. */ setKeyframe(clipTransformId: string, paramName: string, time: number, value: number, easing?: string): Promise; } ======================================================================== # packages/engine/dist/types/src/models/media-asset/media-asset-repository.d.ts ======================================================================== import type { StorageAdapter } from "../plugin"; import type { MediaAsset } from "../types"; export declare class MediaAssetRepository { private storage; constructor(storage: StorageAdapter); getById(id: string): Promise; /** Every asset row in the database, across all projects. */ getAll(): Promise; getByProjectId(projectId: string): Promise; create(data: Omit): Promise; update(id: string, changes: Partial>): Promise; delete(id: string): Promise; deleteByProjectId(projectId: string): Promise; } ======================================================================== # packages/engine/dist/types/src/models/project/project-repository.d.ts ======================================================================== import type { StorageAdapter } from "../plugin"; import type { Project } from "../types"; /** Settings a project can be created with. Every field is optional. */ export interface CreateProjectOptions { surface?: Project["surface"]; canvas_width?: number; canvas_height?: number; frame_rate?: number; } export declare class ProjectRepository { private storage; constructor(storage: StorageAdapter); getById(id: string): Promise; getAll(): Promise; create(name: string, options?: CreateProjectOptions): Promise; update(id: string, changes: Partial>): Promise; delete(id: string): Promise; } ======================================================================== # packages/engine/dist/types/src/models/track/track-repository.d.ts ======================================================================== import type { StorageAdapter } from "../plugin"; import type { Track } from "../types"; export declare class TrackRepository { private storage; constructor(storage: StorageAdapter); getById(id: string): Promise; /** Returns tracks sorted by `order` */ /** Every track row in the database, across all projects. */ getAll(): Promise; getByProjectId(projectId: string): Promise; create(data: Omit): Promise; update(id: string, changes: Partial>): Promise; delete(id: string): Promise; deleteByProjectId(projectId: string): Promise; } ======================================================================== # packages/engine/dist/types/src/models/transition/transition-repository.d.ts ======================================================================== import type { StorageAdapter } from "../plugin"; import type { Transition } from "../types"; export declare class TransitionRepository { private storage; constructor(storage: StorageAdapter); getById(id: string): Promise; getAll(): Promise; /** Get all transitions where the given clip is outgoing OR incoming. */ getByClipId(clipId: string): Promise; /** Find a transition between a specific clip pair (order-independent). */ getByClipPair(clipIdA: string, clipIdB: string): Promise; create(data: Omit): Promise; update(id: string, changes: Partial>): Promise; /** Restore a transition with its original ID (for undo). Creates or replaces. */ restore(t: Transition): Promise; delete(id: string): Promise; /** Delete all transitions referencing a clip (cascade helper for clip deletion). */ deleteByClipId(clipId: string): Promise; } ======================================================================== # packages/engine/dist/types/src/models/transcript/transcript-store.d.ts ======================================================================== import type { StorageAdapter } from "../plugin"; import type { CaptionStyle } from "../../processing/caption-text/caption-presets"; import type { TranscriptSegment as OratorTranscriptSegment } from "./types"; /** * Persistence + in-memory caching for Orator transcripts. * * The transcript is the only new entity Orator persists; everything else * (clips, tracks, undo) rides the existing video-editor model. We follow that * model's conventions exactly: two normalized IDB stores, each with its own * row and a foreign key to its parent (record → project, segment → record). * * Reads are served synchronously from in-memory caches so the * `OratorBackend.getTranscript()` contract can stay sync; IndexedDB is the * durable backing store hydrated once per project load. */ /** Per-media-asset transcript metadata. One row per media asset. */ export interface TranscriptRecord { id: string; project_id: string; source_duration_ms: number; model_used: string; created_at: number; } /** * One spoken paragraph (or word, post task 11) on the transcript, as persisted. * * This is the public `TranscriptSegment` (what `getTranscript()` returns) plus * two storage-only fields. Extending the public type keeps the two shapes from * drifting: anything the frontend sees round-trips through IDB unchanged, and * the conversion back to the public shape (task 09) is just dropping the FK and * `order`. */ export interface TranscriptSegment extends OratorTranscriptSegment { transcript_record_id: string; order: number; /** For `kind: 'word'` rows only: the `id` of the parent `kind: 'speech'` * paragraph this word belongs to. Undefined on speech/silence rows. */ parent_id?: string; } /** * Records keyed by their primary key (= media_asset_id). * `put` (never `add`) so re-transcribe overwrites cleanly. */ export declare class TranscriptRecordRepository { private storage; constructor(storage: StorageAdapter); getById(id: string): Promise; /** Every transcript row in the database, across all projects. */ getAll(): Promise; getByProjectId(projectId: string): Promise; put(record: TranscriptRecord): Promise; delete(id: string): Promise; } /** * Segments keyed by their primary key, indexed by `transcript_record_id`. * `put` (never `add`) so re-transcribe overwrites cleanly. */ export declare class TranscriptSegmentRepository { private storage; constructor(storage: StorageAdapter); getById(id: string): Promise; /** Every segment row in the database, across all transcripts. */ getAll(): Promise; /** Returns segments sorted by `order` ascending. */ getByRecordId(recordId: string): Promise; put(segment: TranscriptSegment): Promise; /** * Bulk replace: deletes all segments for the record, then inserts the new * set — in a single transaction, so a concurrent read never sees a partial * mix and a mid-replace crash leaves the old set intact. */ replaceByRecordId(recordId: string, segments: TranscriptSegment[]): Promise; delete(id: string): Promise; deleteByRecordId(recordId: string): Promise; } /** * Persisted caption config for a project. One row per project (PK = project_id), * holding the caption tool's full state: on/off, the active preset id (null once * the user customizes a style), and the resolved `CaptionStyle` used by both the * live preview and the burned export. */ export interface CaptionConfigRecord { id: string; enabled: boolean; template: string | null; style: CaptionStyle; } /** * Caption config rows keyed by their primary key (= project_id). `put` (never * `add`) so saving overwrites the single per-project row cleanly. */ export declare class CaptionConfigRepository { private storage; constructor(storage: StorageAdapter); getById(id: string): Promise; /** Every caption-config row in the database, one per Orator project. */ getAll(): Promise; put(record: CaptionConfigRecord): Promise; delete(id: string): Promise; } /** * In-memory layer over the two repositories. All caches are keyed by * `mediaAssetId` (which equals the record id). Repository writes always resolve * before the matching cache is updated, so a failed write leaves the cache * untouched and the next `load` heals it from IDB. */ export declare class TranscriptStore { private recordCache; private segmentCache; private pcmCache; private records; private segments; constructor(records: TranscriptRecordRepository, segments: TranscriptSegmentRepository); /** Hydrate both caches for a project, replacing any previously-loaded project. */ load(projectId: string): Promise; getRecord(mediaAssetId: string): TranscriptRecord | null; /** Synchronous read backing `getTranscript()`. Returns all rows — speech, * silence, and word — in stored order. Callers that face the frontend should * project through `toPublicTranscript` to hide/nest word rows. */ getSegments(mediaAssetId: string): TranscriptSegment[]; /** Word-level rows for a media asset, sorted by start. Empty when the * transcript was produced by a segment-only model (no word timestamps). */ getWords(mediaAssetId: string): TranscriptSegment[]; /** * Persist a fresh transcription: write the record + full segment set, then * swap both caches. Used after a `transcribe()` run. */ set(mediaAssetId: string, projectId: string, segments: TranscriptSegment[], meta: { sourceDurationMs: number; modelUsed: string; }): Promise; /** Single-row text edit; sibling segments are untouched and identity is preserved. */ updateSegmentText(mediaAssetId: string, segmentId: string, text: string): Promise; /** * Bulk replace just the segments (record untouched). Used by the edit * coordinator after a `DeleteTimeRangeCommand` (task 06). */ replaceSegments(mediaAssetId: string, segments: TranscriptSegment[]): Promise; getPCM(mediaAssetId: string): Float32Array | null; setPCM(mediaAssetId: string, pcm: Float32Array): void; clearPCM(mediaAssetId: string): void; } /** * Project stored rows to the public `getTranscript()` shape. The single place * this conversion happens, so the facade and the edit-coordinator can't drift. * * - Drops storage-only fields (`transcript_record_id`, `order`, `parent_id`). * - Hides `kind: 'word'` rows from the top level and nests each under its * parent paragraph's `words` array, so the UI keeps rendering paragraphs * while word timings stay available for word-accurate cuts. * * Order of the surviving (non-word) rows is preserved from the input. This is * safe because every word of a paragraph sorts within that paragraph's * [start, end] — strictly before the next paragraph's start — so removing word * rows never reorders paragraphs or silences relative to each other. */ export declare function toPublicTranscript(rows: TranscriptSegment[]): OratorTranscriptSegment[]; ======================================================================== # packages/engine/dist/types/src/models/transcript/types.d.ts ======================================================================== export type TranscriptSegment = { id: string; kind: 'speech' | 'silence' | 'word'; startMs: number; endMs: number; text: string; speakerId?: string; /** * Word-level timing for a speech paragraph, populated when a word-timestamped * model produced the transcript. Present on `kind: 'speech'` segments returned * by `getTranscript()`; absent on silences and on transcripts made by * segment-only models. Used for word-accurate cuts (snap a selection to word * boundaries). Word rows are never returned as top-level segments — they are * nested here under their parent paragraph. */ words?: WordToken[]; }; /** One timed word inside a speech paragraph. */ export type WordToken = { id: string; startMs: number; endMs: number; text: string; }; ======================================================================== # packages/engine/dist/types/src/processing/caption-text/caption-presets.d.ts ======================================================================== /** * Caption style presets — the shared source of truth for both the standalone * AutoCaptionPage gallery and Orator's caption tool (`orator/components`). * * Each preset is a complete bundle of styling that gets applied in one * click — font, size, weight, colors, background, position, karaoke * settings, line wrap. The granular controls behind "Customize" are still * available; presets are the headline UX for users who just want a vibe. * * Adding a new preset: * 1. Add an entry below. * 2. The card thumbnail uses `thumbnail` (pure CSS preview spec) — no * bundling of fonts or canvas work needed at render time. * 3. Apply values live in `apply`; consumers read them through * `resolveTemplate()` which produces a full `CaptionStyle`. * * (Relocated from `pages/AutoCaptionPage/caption-presets.ts` so Orator and the * standalone tool import one definition — see `backend/13-caption-styling.md`.) */ import type { TextProperties } from '../../models/types'; export type CaptionPosition = 'top' | 'middle' | 'bottom'; /** Spec used to render the small CSS-only preview inside each card. */ export interface PresetThumbnail { /** Sample text shown in the thumbnail. Kept short so it fits at any size. */ sample: string; /** Inline style applied to the sample text — mirrors the preset's apply * values at a scaled-down size for visual parity. */ textStyle: Record; /** Optional second `` rendered after `sample`, used to suggest the * karaoke highlight on cards whose preset enables karaoke. */ highlight?: { text: string; color: string; }; /** Card background color so each preset reads against an appropriate * surface (e.g. light grey for "Minimal", deeper grey for "TikTok"). */ cardBg: string; } export interface CaptionPreset { id: string; name: string; thumbnail: PresetThumbnail; apply: { /** TextProperties overrides merged on top of DEFAULT_BURN_IN_STYLE. */ textStyle: Partial; animationStyle: 'static' | 'karaoke'; /** Only honored when animationStyle === 'karaoke'. */ highlightColor?: string; /** 0 = no wrap. */ maxCharsPerLine: number; /** Max wrapped lines shown at once; a longer paragraph advances in groups. * 0 = show the whole paragraph (no chunking). Only takes effect alongside * `maxCharsPerLine > 0`. Optional so older presets default to 0. */ maxLinesPerCue?: number; position: CaptionPosition; }; } /** * The resolved, non-partial caption style bundle. This is what the frontend * stores in state, passes to `export({ captionStyle })`, and feeds to * `renderCaptionFrame`. Defined here so frontend and backend import one * definition (`backend/13-caption-styling.md`). */ export interface CaptionStyle { textStyle: TextProperties; animationStyle: 'static' | 'karaoke'; /** Honored only when animationStyle === 'karaoke'. */ highlightColor: string; /** 0 = no wrap. */ maxCharsPerLine: number; /** Max wrapped lines shown at once (0 = whole paragraph, no chunking). */ maxLinesPerCue: number; } export declare function positionYForPreset(p: CaptionPosition): number; export declare const CAPTION_PRESETS: CaptionPreset[]; /** Look a preset up by id. Returns undefined when the id doesn't exist * (e.g. an old persisted state that referenced a removed preset). */ export declare function getPresetById(id: string): CaptionPreset | undefined; /** * Resolve a preset id into a full `CaptionStyle`: merge the preset's * `apply.textStyle` over `DEFAULT_BURN_IN_STYLE` and fold `apply.position` * into `textStyle.positionY`. Both the frontend (template apply) and the * caption preview/export consume this, so the same id always produces the * same concrete style. * * An unknown id resolves to the default style (bottom, static) rather than * throwing — a persisted state that referenced a removed preset degrades to * a sane default instead of breaking the page. */ export declare function resolveTemplate(id: string): CaptionStyle; ======================================================================== # packages/engine/dist/types/src/lib/file-storage/types.d.ts ======================================================================== /** * StorageProvider — abstract contract for binary file storage. * * All paths returned by write() are opaque tokens. Callers store them * in MediaAsset.file_path and pass them back to read()/delete(). The * provider alone knows how to resolve them to actual storage locations. */ /** * One entry found directly inside a storage directory. * * `path` is a full path token of the same shape write() returns, so it can be * compared against a `MediaAsset.file_path` and passed back to read()/delete(). */ export interface StoredEntry { /** Name inside the directory it was listed from. */ name: string; /** Full path token from the storage root. */ path: string; kind: 'file' | 'directory'; /** Bytes the file holds. 0 for a directory. */ size: number; /** Epoch ms the file was last written. 0 for a directory, which carries no timestamp. */ lastModified: number; } /** How a worker opens the same storage for itself: names only, never the object or any bytes. */ export interface StorageDescription { kind: 'opfs'; /** Folder of the OPFS root the page's storage lives in; absent or "" is the OPFS root itself. */ root?: string; } export interface StorageProvider { /** Absent when a worker cannot reach this storage (memory storage); exports then stay on the page. */ describe?(): StorageDescription; /** Persist a File, return an opaque path token stored in MediaAsset.file_path */ write(projectId: string, assetId: string, file: File): Promise; /** Retrieve a File by its path token */ read(path: string): Promise; /** Delete a file by path token. Idempotent (no-op if missing). */ delete(path: string): Promise; /** * Open a seekable writable stream for large outputs (export MP4, proxy). * Returns writable: null if backend doesn't support streaming writes. * Callers MUST check null and fall back to buffer mode (ArrayBufferTarget). */ createWritable(...segments: string[]): Promise<{ writable: FileSystemWritableFileStream | null; path: string; }>; /** * Mark a file written via createWritable() as ready for reliable reads. * Providers that mirror to a secondary backend (e.g. Cache API) use this * to backfill from the primary write target. No-op for OPFS-only. * Best-effort — failures are logged but do not throw. */ commit(path: string): Promise; /** Delete all files in a directory/namespace. Idempotent. */ clearDirectory(...segments: string[]): Promise; /** * List the entries directly inside a directory, files and subdirectories * alike. Returns an empty array when the directory does not exist, and never * creates it. Needed to find stored files that no database row points at. */ list(...segments: string[]): Promise; /** * Delete a directory and everything under it, the directory itself included. * Idempotent. Use this (not clearDirectory) when the directory will never be * written to again, so no empty directory is left occupying storage. */ deleteDirectoryTree(...segments: string[]): Promise; } ======================================================================== # packages/engine/dist/types/src/editor/engine/types.d.ts ======================================================================== /** Metadata extracted from the MP4 container's video track */ export interface VideoInfo { codec: string; codedWidth: number; codedHeight: number; timescale: number; duration: number; durationSeconds: number; trackId: number; description: Uint8Array; nbSamples: number; /** * Intrinsic clockwise rotation (degrees) the coded frames need to display * upright, read from the container's transformation matrix/tag. Applied at * composite (see MediaAsset.rotation); never baked into the decoded frames. * 0 = upright. Flows through to `VideoProbeInfo`/`MediaProbe` via the Omit. */ rotation: 0 | 90 | 180 | 270; } /** * Lightweight metadata for one sample — stored in memory for ALL samples. * Only ~48 bytes per entry vs ~5-50 KB if we stored the actual data. * A 1-hour video at 30fps = ~108,000 entries = ~5 MB of metadata. */ export interface SampleMetadata { timestamp: number; duration: number; isKeyframe: boolean; byteOffset: number; byteLength: number; } /** A single encoded video sample with actual data, ready for VideoDecoder */ export interface EncodedSample { data: ArrayBuffer; timestamp: number; duration: number; isKeyframe: boolean; } /** * Opaque source of encoded sample bytes. Tier 1 (mp4box) implements this * over `Blob.slice` against the original file; Tier 2 (mediabunny) over * `EncodedPacketSink.getNextPacket` against a live `Input`. Consumers go * through `SampleCache` and never see which tier produced the source. */ export interface SampleSource { fetchRange(startIdx: number, endIdx: number): Promise; /** Release any resources held by the source (e.g. mediabunny's `Input`). */ dispose(): void; } /** Complete result of demuxing a file — stores only metadata, not sample data */ export interface DemuxResult { info: VideoInfo; sampleIndex: SampleMetadata[]; keyframeIndices: number[]; /** * Lazy fetcher for the encoded bytes of any [start, end) sample range. * Tier 1: backed by `file.slice`. Tier 2: backed by mediabunny's * `EncodedPacketSink`. Both produce the same `EncodedSample[]` shape. */ sampleSource: SampleSource; /** * Original file, kept for any caller that still needs it (e.g. legacy * audio decode paths reading the whole buffer). Lazy byte reads should * go through `sampleSource` instead. */ file: File; } /** A decoded frame sitting in the frame cache */ export interface CachedFrame { frame: VideoFrame; timestamp: number; } export type TrackId = string; /** A clip placed on the global timeline */ export interface ClipPlacement { trackId: TrackId; clipId: string; type: "video" | "audio" | "text" | "image"; startTime: number; endTime: number; sourceOffset: number; } /** Result of querying the timeline at a given time */ export interface ActiveClip { trackId: TrackId; clipId: string; type: "video" | "audio" | "text" | "image"; localTime: number; } /** Metadata extracted from the MP4 container's audio track */ export interface AudioInfo { codec: string; timescale: number; duration: number; durationSeconds: number; trackId: number; sampleRate: number; channelCount: number; } /** Video probe info — like VideoInfo but without codec description bytes */ export type VideoProbeInfo = Omit; /** Result of probing a media file's tracks */ export interface MediaProbe { video: VideoProbeInfo | null; audio: AudioInfo | null; durationSeconds: number; } /** Which streams to extract from a source file */ export type StreamSelection = "video" | "audio" | "both"; ======================================================================== # packages/engine/dist/types/src/editor/captions/transcribe-clip.d.ts ======================================================================== /** * transcribe-clip.ts — turn one audio clip into subtitle clips for a text track. * * Only the audio the clip actually plays is transcribed: the file's PCM is * sliced from the clip's in-point for the clip's duration, so a short clip cut * from a long recording does not wait for the whole recording. Because the * slice starts at the in-point, every segment Whisper returns is already in * clip-local seconds; placing it on the timeline is just adding the clip's * start time. */ import type { Clip } from "../../models/types"; import type { MonitoringAdapter } from "../../monitoring/types"; import type { Progress, Segment } from "../../processing/transcription"; /** One subtitle, positioned in project seconds on the timeline. */ export interface CaptionClipSpec { startTime: number; duration: number; text: string; } /** A transcription that is running for one clip; drives the panel's progress row. */ export interface ClipTranscriptionJob { clipId: string; progress: Progress | null; cancel: () => void; } /** Subtitle lines wrap at this many characters, like the auto-caption tool. */ export declare const CAPTION_MAX_CHARS_PER_LINE = 42; type ClipWindow = Pick; type ClipPlacement = Pick; /** The samples the clip plays: from its in-point in the file, for its duration. */ export declare function sliceClipAudio(pcm: Float32Array, sampleRate: number, clip: ClipWindow): Float32Array; /** * Move clip-local segments onto the timeline and clamp them to the clip, so a * subtitle can never start before the clip or run past its end. */ export declare function segmentsToCaptionClips(segments: readonly Pick[], clip: ClipPlacement, maxCharsPerLine?: number): CaptionClipSpec[]; export interface TranscribeClipOptions { signal?: AbortSignal; onProgress?: (p: Progress) => void; monitoring?: MonitoringAdapter; modelBase?: string; } /** Read the clip's audio from its file, transcribe it, and return timeline-ready subtitles. */ export declare function transcribeClip(file: File, clip: Pick, options?: TranscribeClipOptions): Promise; export {}; ======================================================================== # packages/engine/dist/types/src/editor/export/codec-support.d.ts ======================================================================== /** * Codec detection — probes WebCodecs `isConfigSupported()` to find * encoders the current platform actually supports. * * Fallback chains: * Video: H.264 Baseline → VP9 * Audio: AAC-LC → Opus */ export interface DetectedCodecs { video: { webCodec: string; muxerCodec: 'avc' | 'vp9'; }; audio: { webCodec: string; muxerCodec: 'aac' | 'opus'; }; /** True if any codec fell back from the preferred choice. */ fallback: boolean; } /** * True when the browser exposes the WebCodecs encoders the export path needs. * Some browsers (older iOS Safari, some Android webviews) ship the decoders but * not `VideoEncoder`/`AudioEncoder`, so calling `detectCodecs` throws a * `ReferenceError`. Check this first and gate the UI instead of crashing. */ export declare function isWebCodecsExportSupported(): boolean; /** * Probe the *actual* export config (real resolution/bitrate/frame rate), not the * fixed 8Mbps/30fps shape `detectVideoCodec` uses. This is the same probe * `ProjectExporter` runs right before `configure()`, so a `false` here means the * export would have failed with "This browser cannot encode…". Surfacing it up * front lets the dialog grey out unsupported combos (e.g. 4K@60 on some Android) * instead of failing after the user commits. */ export declare function probeVideoExportConfig(settings: { videoCodec: string; width: number; height: number; videoBitrate: number; frameRate: number; }): Promise; export declare function detectVideoCodec(width: number, height: number): Promise<{ webCodec: string; muxerCodec: 'avc' | 'vp9'; }>; export declare function detectAudioCodec(sampleRate: number, channels: number): Promise<{ webCodec: string; muxerCodec: 'aac' | 'opus'; }>; export declare function detectCodecs(settings: { width: number; height: number; audioSampleRate: number; audioChannels: number; }): Promise; ======================================================================== # packages/engine/dist/types/src/editor/undo/undo-manager.d.ts ======================================================================== /** * undo-manager.ts — Command-pattern undo/redo system. * * Every undoable user action becomes an UndoableCommand with execute() and undo(). * Commands are pushed onto a history stack. Cmd+Z pops and calls undo(). * Cmd+Shift+Z re-pushes and calls execute(). */ export interface UndoableCommand { readonly label: string; execute(): Promise; undo(): Promise; } export declare class UndoManager { private undoStack; private redoStack; private maxHistory; private executing; /** * Execute a command and push it onto the undo stack. * Clears the redo stack (new action invalidates redo history). */ execute(cmd: UndoableCommand): Promise; /** * Push a command that has ALREADY been executed onto the undo stack. * Used for coalesced slider drags where the mutations happen during the drag * and the command is created retroactively on mouseup with before/after values. */ push(cmd: UndoableCommand): void; undo(): Promise; redo(): Promise; canUndo(): boolean; canRedo(): boolean; clear(): void; } ======================================================================== # packages/engine/dist/types/src/lib/file-storage/index.d.ts ======================================================================== /** * File storage module — provides a pluggable StorageProvider. * * Hosts pass their storage in `EngineServices.files`. The default below is the * engine's fallback, one per JS context. The public entry does not export it; * `setStorageProvider` is for tests only. */ import type { StorageDescription, StorageProvider } from './types'; import type { MonitoringAdapter } from '../../monitoring/types'; /** * Get the file storage provider singleton for the current JS context. * Each context (main thread, each Web Worker) gets its own instance. */ export declare function getStorageProvider(): StorageProvider; /** * Override the storage provider (for testing or alternative backends). * Must be called before any storage operations. */ export declare function setStorageProvider(provider: StorageProvider): void; /** A worker's own handle on the storage the page described to it. */ export declare function openDescribedStorage(description: StorageDescription, monitoring?: MonitoringAdapter): StorageProvider; export type { StorageDescription, StorageProvider, StoredEntry } from './types'; export { OPFSStorageProvider } from './opfs-provider'; ======================================================================== # packages/engine/dist/types/src/lib/file-storage/memory-provider.d.ts ======================================================================== import type { StorageProvider, StoredEntry } from './types'; /** * The StorageProvider contract held in a Map. Used by Node tests and by * anything that wants media files without OPFS. * * Paths have the same shape as the OPFS provider's, so a test can read a * `MediaAsset.file_path` and recognise it. */ export declare class MemoryFileStorage implements StorageProvider { private files; write(projectId: string, assetId: string, file: File): Promise; read(path: string): Promise; delete(path: string): Promise; createWritable(...segments: string[]): Promise<{ writable: FileSystemWritableFileStream | null; path: string; }>; commit(path: string): Promise; clearDirectory(...segments: string[]): Promise; /** * Memory has no directory entries, so a directory exists exactly while some * file is stored under it — the same shape OPFS reports, minus empty folders. */ list(...segments: string[]): Promise; /** * Memory has no directory entries, so removing a tree is removing every path * under it — nested paths included, unlike OPFS where a subdirectory remains. */ deleteDirectoryTree(...segments: string[]): Promise; /** True when a file is stored at this path. For test assertions. */ has(path: string): boolean; /** Re-date a stored file, so a test can age it past a sweep's threshold. */ touch(path: string, lastModified: number): void; /** Every stored path. For test assertions. */ paths(): string[]; } ======================================================================== # packages/engine/dist/types/src/editor/engine/import/demuxer.d.ts ======================================================================== /** * demuxer.ts — MP4 demuxing with mp4box.js * * Reads a File in 2 MB chunks, feeds them to mp4box for container parsing, * and builds a lightweight sample index (metadata only — no encoded data). * * The actual encoded sample data is read on-demand from the File object * using Blob.slice() when the player needs it for decoding. This keeps * memory usage constant regardless of video length: * * - 10-second video: ~300 entries × 48 bytes = ~14 KB * - 1-hour video: ~108,000 entries × 48 bytes = ~5 MB * * Compare with the old approach that stored all encoded data: * - 1-hour at 5Mbps = ~340 MB in JS heap * * The keyframe index enables O(log n) binary search for seeking. */ import type { SampleMetadata, EncodedSample, DemuxResult, MediaProbe, SampleSource } from "../types"; /** * Demux an MP4 file and return a lightweight sample index + keyframe index. * * No encoded sample data is stored — only metadata (timestamps, byte offsets, * sizes, keyframe flags). Use fetchSamples() to read actual data on demand. */ export declare function demuxFile(file: File, onProgress?: (fraction: number) => void): Promise; /** * The original demux: feeds the whole file through mp4box and collects every * sample it hands out. Kept for fragmented files, whose samples live in moof * boxes spread through the file, and as the reference the index path is tested against. */ export declare function demuxFileByScan(file: File, onProgress?: (fraction: number) => void): Promise; /** * Tier-1-then-Tier-2 demux. Mirrors `VideoTrack.loadFile`'s fallback chain so * the export pipeline can demux container/codec pairs the mp4box path can't * handle (WebM, MKV, MP4-with-broken-description) without each call site * re-implementing the wrap. * * On Tier 1 failure (any error other than `BrowserCodecUnsupportedError` — * mediabunny can't fix browser-codec gaps either) we lazy-import the * mediabunny adapter and run it. The returned `DemuxResult` carries a * `sampleSource` (mediabunny-backed lazy fetcher) — heap stays bounded * regardless of source duration. */ export declare function demuxFileWithFallback(file: File, onProgress?: (fraction: number) => void, importTier2?: boolean): Promise; /** * Fetch encoded sample bytes for the range [startIdx, endIdx). Delegates to * whichever `SampleSource` the demuxer attached to the result — Tier 1 * uses `Mp4BoxSampleSource` (lazy `Blob.slice`), Tier 2 uses * `MediabunnySampleSource` (lazy `EncodedPacketSink.getNextPacket`). Both * produce the same `EncodedSample[]` shape; consumers stay tier-blind. */ export declare function fetchSamples(result: DemuxResult, startIdx: number, endIdx: number): Promise; /** * Tier 1 (mp4box) lazy sample fetcher. Holds a reference to the original * `File` and the metadata-only `sampleIndex`; reads encoded bytes on demand * via `Blob.slice`, batching adjacent samples within `GAP_THRESHOLD` (64 KB) * into a single read to amortize the per-read overhead. */ export declare class Mp4BoxSampleSource implements SampleSource { private readonly file; private readonly sampleIndex; constructor(file: File, sampleIndex: SampleMetadata[]); fetchRange(startIdx: number, endIdx: number): Promise; dispose(): void; } /** * Probe a media file to detect available streams without extracting samples. * * Dispatches based on the file's container — detected from the first 16 bytes, * not the filename — to the right backend: * - ISOBMFF (mp4/mov/m4a/m4v) → mp4box (fast happy path). * - Anything else (matroska/webm, ogg, wav, mp3, flac, …) → mediabunny, * dynamically imported so the MP4 path stays free of the cost. * - Truly unknown magic bytes → try mp4box first (some odd files have * non-standard headers), then fall back to mediabunny. * * When the Tier 2 kill-switch (`importTier2: false`) is set, * non-ISOBMFF files revert to the pre-Tier-2 stub-probe behaviour so the * rollout doc's "Tier 1 unchanged" guarantee continues to hold. */ export declare function probeFile(file: File, importTier2?: boolean): Promise; ======================================================================== # packages/engine/dist/types/src/editor/transforms/seed-transforms.d.ts ======================================================================== import type { ModelStore } from "../../models/model"; /** * Seed all transform definitions into the database. * Idempotent — upserts each definition by type:name key. * New definitions are created; existing ones are updated if * default_params or description have changed. * * Runs the legacy rename first so an old "zoom:Zoom" definition becomes * "scale:Scale" in place and the upsert below finds it instead of creating a * duplicate. * * Call this during project initialization (after ModelStore is ready). */ export declare function seedTransformDefinitions(store: ModelStore): Promise; ======================================================================== # packages/engine/dist/types/src/editor/project-document/index.d.ts ======================================================================== export type { DocumentEffect, ProjectDocument, ProjectSnapshot, SnapshotMediaAsset } from "./types"; export { dumpProject } from "./dump-project"; export { loadProject } from "./load-project"; export { snapshotProject } from "./snapshot-project"; export type { LoadProjectOptions, LoadedIds, LoadedProject } from "./load-project"; export { migrateDocument } from "./migrate"; export type { MediaManifest, MediaManifestEntry } from "./manifest"; export type { DocumentMediaAsset, MediaResolver } from "./resolve-media"; ======================================================================== # packages/engine/dist/types/src/editor/export/types.d.ts ======================================================================== export interface ExportSettings { width: number; height: number; frameRate: number; videoBitrate: number; videoCodec: string; muxerVideoCodec: 'avc' | 'hevc' | 'vp9' | 'av1'; audioBitrate: number; audioSampleRate: number; audioChannels: number; audioCodec: string; muxerAudioCodec: 'aac' | 'opus'; } export interface ExportProgress { phase: "preparing" | "video" | "audio" | "finalizing"; currentFrame: number; totalFrames: number; percent: number; } export type ExportProgressCallback = (progress: ExportProgress) => void; export interface SkippedAudioClipSummary { clipId: string; assetId: string; filename: string; reason: string; } export interface ExportResult { opfsPath: string; size: number; /** * Audio clips that failed to decode and were rendered as silence in * their timeline range. Empty when the export completed cleanly. */ skippedAudioClips: SkippedAudioClipSummary[]; } ======================================================================== # packages/engine/dist/types/src/options.d.ts ======================================================================== import { type DecodeTuning } from "./editor/engine/decode/decode-tuning"; import type { EngineServices } from "./services"; import type { Licence } from "./licence"; /** What a host tells the engine instead of the engine reading the page it is on. */ export interface EngineOptions { /** Names this instance's database, file root and saved preferences. Default "vidstudio", which keeps the names vidstudio.app has always used. */ namespace?: string; /** Folders (ending in "/") that serve the font files and the model files. Default "/fonts/" and "/models/". */ assets?: { fonts?: string; models?: string; }; /** Default "pixi" (WebGL); "canvas2d" is the fallback renderer. */ renderer?: "pixi" | "canvas2d"; /** Default true: exports run in a worker so a hidden tab cannot throttle them. */ exportInWorker?: boolean; /** Default true: the mediabunny import path for files mp4box cannot read. */ importTier2?: boolean; /** Decode experiments; default none. */ decodeTuning?: Partial; services?: EngineServices; /** * Checked once in `EditorBackendImpl.open()` by `checkPageLicence`, against the page's * host name. That read of `location.hostname` is the one place the engine reads the * page: a licence is bound to the domain the editor is served from, which only the * page knows. Without a licence that covers the host, the picture carries a watermark. */ licence?: Licence; } export interface ResolvedEngineOptions { namespace: string; assets: { fonts: string; models: string; }; renderer: "pixi" | "canvas2d"; exportInWorker: boolean; importTier2: boolean; decodeTuning: DecodeTuning; services: EngineServices; licence?: Licence; } export declare const DEFAULT_NAMESPACE = "vidstudio"; export declare function resolveOptions(options?: EngineOptions): ResolvedEngineOptions; /** * The database and the file-storage root of one namespace. The default namespace * maps to the names used before namespaces existed, so existing projects stay * where they are without a migration that would copy every stored video. */ export declare function storageNames(namespace?: string): { database: string; fileRoot: string; }; declare const LEGACY_PREFERENCE_KEYS: { readonly "new-project-defaults": "ve-new-project-defaults"; readonly "safe-zone-guide": "vidstudio_safe_zone_guide"; readonly "tour-seen": "vidstudio_tour_seen"; }; export type PreferenceName = keyof typeof LEGACY_PREFERENCE_KEYS; /** The local-storage key of a per-viewer preference; the default namespace keeps the keys saved before namespaces. */ export declare function preferenceKey(namespace: string, name: PreferenceName): string; /** What a video reader and its decoder need from the editor that owns them. */ export type DecodeOptions = Pick; /** Used where an owner passes no options: the defaults, never another editor's. */ export declare const DEFAULT_OPTIONS: ResolvedEngineOptions; export {}; ======================================================================== # packages/engine/dist/types/src/editor/engine/decode/decode-tuning.d.ts ======================================================================== /** * decode-tuning.ts — runtime knobs for testing the video decode path on a * deployed build (e.g. a Vercel preview opened on a Windows machine), no * rebuild needed. * * The host passes them as `EngineOptions.decodeTuning`. vidstudio.app reads * them from URL query params with `readDecodeTuningKnobs`, which copies them to * sessionStorage so they survive in-app navigation and reloads in the same * tab, and vanish when the tab closes. `?decodeTuning=reset` clears them. * * ?decodeCache=8 cap the editor's per-decoder frame cache * (defaults in decode-budget.ts). Tests the * hardware frame-pool theory for seek/playback. * ?exportGopCap=8 export/burn-in keeps at most N decoded frames of * a keyframe group open at once instead of the * whole group (the Android default is 8, see * decode-budget.ts). Same theory, export path. * Slower: the group is re-decoded every N frames. * ?stallTimeout=2000 stall timeout in ms (default 4000). * ?stallFallback=on|off force the software-fallback policy regardless * of platform (default: Windows only). * ?decodeHw=software build every decoder with * hardwareAcceleration: "prefer-software" from the * start — the in-app equivalent of disabling * chrome://flags/#disable-accelerated-video-decode. * ?decodeCopy=on|off copy every hardware-decoded frame off the * driver's pool (frame-copy.ts). Default: on for * Windows and Android, off elsewhere. `off` there * reproduces the original pool-exhaustion stall. * * Everything here is best-effort: with no knobs set the tuning is all-null * and behaviour is unchanged. Each editor carries its own tuning down to its * decoders. */ export interface DecodeTuning { decodeCache: number | null; exportGopCap: number | null; stallTimeoutMs: number | null; stallFallback: boolean | null; forceSoftware: boolean; detachFrames: boolean | null; } export declare const NO_DECODE_TUNING: DecodeTuning; type SessionStore = Pick; /** The knobs in a query string, merged over the ones an earlier page of this tab kept in `storage`. */ export declare function readDecodeTuningKnobs(search: string, storage?: SessionStore): DecodeTuning; export {}; ======================================================================== # packages/engine/dist/types/src/licence.d.ts ======================================================================== /** "valid": the key covers this host. "localhost": a development host with no key that covers it. "invalid": anything else. */ export type LicenceState = "valid" | "invalid" | "localhost"; /** What a host passes: the name it was issued to and the key from the dashboard. */ export interface Licence { name: string; key: string; } /** The signed part of a key. `expires` is an ISO 8601 date or date-time; a bare date means midnight UTC starting that day. */ export interface LicenceToken { name: string; domains: string[]; expires: string; plan: string; } /** Raw Ed25519 public key, base64url. The private half lives outside the repository and signs every issued key. */ export declare const LICENCE_PUBLIC_KEY = "KSuo97GoNQzw-nXuqjMZzbaQYpXHfmbHuDPmGrvtxhE"; /** * Test builds only: makes `open()` verify keys against `publicKey` instead of the built-in key. * Reached through `@vidstudio/engine/testing`; a published bundle refuses it. */ export declare function setTestLicencePublicKey(publicKey: string | null): void; /** The installed test key, which the export worker needs to repeat the check; null outside tests. */ export declare function activeTestPublicKey(): string | null; /** Proof of one check of the page's licence. Only `checkPageLicence` makes one, so a host cannot forge "valid". */ export interface LicenceGrant { readonly state: LicenceState; } /** * Checks a licence against the host name this page (or worker) was served from, with the * built-in key. This is the engine's one read of the page; see `EngineOptions.licence`. */ export declare function checkPageLicence(licence?: Licence): Promise; /** Checks a licence key against a host name the caller supplies, offline. Its result draws nothing; see `checkPageLicence`. */ export declare function checkLicence(licence: Licence | undefined, hostname: string, options?: { publicKey?: string; now?: number; }): Promise; /** Default deny: the mark is drawn unless handed a grant `checkPageLicence` made with the state "valid". */ export declare function showsWatermark(grant: LicenceGrant | null | undefined): boolean; /** The token a key carries, or null when the key is malformed or its signature does not match `publicKey`. */ export declare function readLicenceKey(key: string, publicKey?: string): Promise; ======================================================================== # packages/editor/dist/types/index.d.ts ======================================================================== import "./editor.css"; export { VidstudioEditor, type VidstudioEditorProps } from "./VidstudioEditor"; export { mountEditor, type MountedEditor } from "./mount"; export { darkTheme, lightTheme, type EditorTheme } from "./theme"; export { allFeatures, type EditorFeatures } from "./features"; export type { EditorShortcuts, EditorUiEvent, OpenDocumentHandler, SaveDocumentHandler } from "./backend/editor-host-context"; export { readProjectZip, writeProjectZip } from "./utils/project-zip"; export { FontPicker, type FontPickerProps } from "./components/TransformPanel/FontPicker"; export { createStore, type CreateStoreOptions } from "@vidstudio/engine"; ======================================================================== # packages/editor/dist/types/VidstudioEditor.d.ts ======================================================================== import type { EditorEvent, EngineOptions, ExportResult, ModelStore } from "@vidstudio/engine"; import type { EditorShortcuts, EditorUiEvent, OpenDocumentHandler, SaveDocumentHandler } from "./backend/editor-host-context"; import { type EditorFeatures } from "./features"; import { type EditorTheme } from "./theme"; import "./editor.css"; export interface VidstudioEditorProps { /** The host owns the store (see `createStore`); the editor never creates one. */ store: ModelStore; projectId: string; /** A new object rebuilds the editor, so keep it stable. */ options?: EngineOptions; theme?: Partial; features?: Partial; /** Checked once against the page's domain; without a licence that covers it the picture carries a watermark. */ licence?: { name: string; key: string; }; /** "scoped" (default): keys reach the editor only while focus is inside it. "global": any key on the page. */ shortcuts?: EditorShortcuts; /** Every backend event, plus the facts only the user interface knows (a download, a released export). */ onEvent?: (event: EditorEvent | EditorUiEvent) => void; /** Replaces the browser download. The file is deleted when the export dialog closes, so copy or upload it first. */ onExport?: (result: ExportResult, file: File) => void; /** * Replaces the "Save project" zip download: gets the document and one entry per media * file, whose `read()` is a handle to the stored file (upload it, or skip it by its hash). */ onSaveDocument?: SaveDocumentHandler; /** Replaces the "Open project" zip picker: return the document and a resolver for its media, or null. */ onOpenDocument?: OpenDocumentHandler; /** Shows a Close button; the host decides what leaving means. */ onClose?: () => void; } /** The whole editor, filling its parent: give the parent a height. */ export declare function VidstudioEditor({ store, projectId, options, theme, features, licence, shortcuts, onEvent, onExport, onSaveDocument, onOpenDocument, onClose }: VidstudioEditorProps): import("react/jsx-runtime").JSX.Element; ======================================================================== # packages/editor/dist/types/mount.d.ts ======================================================================== import { type VidstudioEditorProps } from "./VidstudioEditor"; export interface MountedEditor { /** Replaces the props; the ones left out keep their values. */ update(props: Partial): void; /** Disposes the editor and its backend, and empties the element. */ unmount(): void; } /** For hosts without React: renders the editor into `element`, which needs a height. */ export declare function mountEditor(element: HTMLElement, props: VidstudioEditorProps): MountedEditor; ======================================================================== # packages/editor/dist/types/theme.d.ts ======================================================================== import type { CSSProperties } from "react"; /** Every colour, radius and font the editor draws with. Each key is one custom property on the editor root. */ export interface EditorTheme { bgPrimary: string; bgSecondary: string; bgTertiary: string; bgCard: string; bgCardHover: string; bgElevated: string; textPrimary: string; textSecondary: string; textTertiary: string; textMuted: string; /** Text and icons on an accent-coloured button. */ textOnAccent: string; accentPrimary: string; accentPrimaryHover: string; accentSecondary: string; accentSuccess: string; /** Keyframes and warnings. */ accentWarning: string; accentWarningHover: string; accentError: string; borderPrimary: string; borderSecondary: string; borderHover: string; errorSurface: string; errorBorder: string; errorText: string; warningSurface: string; warningBorder: string; warningText: string; /** Behind the video, and text drawn over it; usually dark and light in every theme. */ mediaBg: string; mediaText: string; mediaTextMuted: string; timelineBg: string; timelineTrackBorder: string; timelineRulerText: string; timelinePlayhead: string; timelineWaveform: string; timelineSnap: string; radiusSm: string; radiusMd: string; fontFamily: string; } /** The defaults, which `editor.css` also declares on `.vs-editor`. */ export declare const darkTheme: EditorTheme; export declare const lightTheme: EditorTheme; /** Only the keys the host set become inline properties; the rest stay on the stylesheet's defaults. */ export declare function themeStyle(theme: Partial | undefined): CSSProperties; ======================================================================== # packages/editor/dist/types/features.d.ts ======================================================================== /** Parts of the editor a host can hide. Every key defaults to true. */ export interface EditorFeatures { sourceMonitor: boolean; tour: boolean; safeZones: boolean; /** The transcribe button; off means the speech model is never downloaded. */ captions: boolean; transitions: boolean; keyframes: boolean; export: boolean; /** The canvas size and frame rate pickers. */ projectSettings: boolean; textTracks: boolean; imageImport: boolean; audioImport: boolean; shortcutsHelp: boolean; /** The "Save project" and "Open project" entries of the project menu. */ projectFiles: boolean; } export declare const allFeatures: EditorFeatures; export declare const EditorFeaturesContext: import("react").Context; export declare function useEditorFeatures(): EditorFeatures; ======================================================================== # packages/editor/dist/types/backend/editor-host-context.d.ts ======================================================================== import { type ExportResult, type ExportSettings, type MonitoringAdapter, type ResolvedEngineOptions } from "@vidstudio/engine/internal"; import type { MediaManifest, MediaResolver, ProjectDocument } from "@vidstudio/engine"; /** Facts only the user interface knows, for the host to measure. What the backend knows arrives as an `EditorEvent`. */ export type EditorUiEvent = { kind: "export-downloaded"; settings: ExportSettings; } | { kind: "export-released"; reason: "dialog_closed" | "page_left"; outputBytes: number; }; /** What the components below `EditorBackendProvider` get from the page that hosts them. */ export interface EditorHost { monitoring: MonitoringAdapter; onUiEvent: (event: EditorUiEvent) => void; /** Prefixes the per-viewer preferences the components keep in local storage. */ namespace: string; /** The editor's own options, for the engine objects the components build (the Source Monitor) and the font folder. */ engineOptions: ResolvedEngineOptions; shortcuts: EditorShortcuts; /** Takes the finished file instead of the browser download. */ onExport?: (result: ExportResult, file: File) => void; /** Takes a saved project instead of the zip download. */ onSaveDocument?: SaveDocumentHandler; /** Supplies the project to open instead of the zip picker. */ onOpenDocument?: OpenDocumentHandler; } /** Gets the document and a handle to each media file; it decides where they go. */ export type SaveDocumentHandler = (document: ProjectDocument, media: MediaManifest) => void | Promise; /** Returns the project to open and how to find its media, or null when the user cancelled. */ export type OpenDocumentHandler = () => Promise<{ document: ProjectDocument; resolve: MediaResolver; } | null>; /** "scoped": keys reach the editor only while focus is inside it. "global": any key on the page, for a full-page editor. */ export type EditorShortcuts = "scoped" | "global"; export declare const EditorHostContext: import("react").Context; export declare function useEditorHost(): EditorHost; ======================================================================== # packages/editor/dist/types/utils/project-zip.d.ts ======================================================================== import type { MediaManifest, MediaResolver, ProjectDocument } from "@vidstudio/engine"; /** * One zip with the document and every media file, stored uncompressed. The zip is a * Blob made of small headers and the media files themselves, so no media byte is * copied into memory; each file is read once, a chunk at a time, for its checksum. */ export declare function writeProjectZip(document: ProjectDocument, media: MediaManifest): Promise; /** Reads a zip written by `writeProjectZip`. Each media file comes back as a slice of the zip, not a copy. */ export declare function readProjectZip(zip: Blob): Promise<{ document: ProjectDocument; resolve: MediaResolver; }>; export declare function writeStoredZip(entries: { name: string; data: Blob; }[]): Promise; /** Every stored entry of a zip by name, each a slice of `zip`. Reads only the directory and the local headers. */ export declare function readStoredZip(zip: Blob): Promise>; ======================================================================== # packages/editor/dist/types/components/TransformPanel/FontPicker.d.ts ======================================================================== import "./FontPicker.css"; export interface FontPickerProps { selectedFamily: string; onSelect: (family: string) => void; } export declare function FontPicker({ selectedFamily, onSelect }: FontPickerProps): import("react/jsx-runtime").JSX.Element;