Driving the engine

The backend is the class EditorBackendImpl from @vidstudio/engine. It implements the EditorBackend interface. The editor's user interface calls nothing else. Everything the editor does, your code can do too. The full method list is in the reference.

Open a backend

Build a backend with a store, a project id and a canvas. Then call open().

import { createStore, EditorBackendImpl } from "@vidstudio/engine";

export async function openBackend(canvas: HTMLCanvasElement): Promise<EditorBackendImpl> {
  const store = await createStore({ namespace: "my-app" });
  const project = await store.projects.create("My project", { frame_rate: 30 });
  const backend = new EditorBackendImpl({
    store,
    projectId: project.id,
    previewCanvas: canvas,
    options: { namespace: "my-app" },
  });
  await backend.open();
  return backend;
}

The store is described in Projects and storage. open() checks the licence key once, seeds the effect catalog and loads the project into the player. Call dispose() when you are done with the backend.

The command queue

Every edit is one method on the backend. Edits run one after another on a queue. Two edits never interleave. An edit that changes the project records one undo step; undo() and redo() walk the last fifty steps.

The methods, by group

Times are seconds. Each method that changes the project returns a promise and ends by emitting an event (see Theme, features and events).

Group	Methods
Lifecycle	open, dispose, subscribe, isHeadless, getLicenceState
Reading	getProject, getTimeline, getClip, findClipAt, getClipEffects, getEffectCatalog, getMediaAssets, getTransitions, snapshot
Project settings	renameProject, setCanvasSize, setFrameRate, setWorkArea, setLoopPlayback
Import	importVideo, importAudio, importImage, importAsset, importFromUrl, addColorLayer, readAssetFile, readAssetPreviewFile
Tracks	addTrackFromAsset, addClipFromAsset, removeTrack, setTrackStartTime, setTrackVolume, reorderTracks, setTrackHidden, renameTrack
Clips	splitClipAt, deleteClip, deleteClips, trimClip, moveClip, placeClips, linkClips, unlinkClip, updateClip, copyClip, pasteClip, duplicateClip, hasClipboard
Text	addTextTrack, addTextTrackWithClips, restyleTextClips, transcribeClipToTextTrack
Effects and keyframes	addEffect, removeEffect, reorderEffects, setEffectParams, setEffectWindow, setKeyframe, moveKeyframe, setKeyframeEasing, deleteKeyframe, activateKeyframing, deactivateKeyframing, easeAllKeyframes, easeKeyframeLane, setClipGeometry, previewClipGeometry, adoptLegacyOpacity
Transitions	setTransition, removeTransition
Undo	undo, redo, canUndo, canRedo
Playback	play, pause, seek, nextFrame, previousFrame, setPlaybackRate, getCurrentTime, getDuration, getPlayerState
Render a frame	renderFrameAt
Export	exportProject, cancelExport, readExportFile, discardExportFile
Documents	saveDocument, loadDocument

Transition types are "crossfade", "wipe-left", "wipe-right", "push-left" and "push-right". Effect ids come from getEffectCatalog(); "brightness", "contrast", "grayscale" and "opacity" are four of them. addTextTrack() adds a text track with one clip at the playhead. The clip reads "Text" and lasts five seconds. Change it with updateClip(clipId, { text_content: "..." }).

Saving and loading projects are described in Save and load.

Headless mode

Leave out previewCanvas and the backend runs headless. No player is built, so nothing from the browser is needed. Every edit then runs in Node in milliseconds. The engine's own unit tests work this way.

This builds a headless backend in Node with in-memory storage.

import { EditorBackendImpl, MemoryFileStorage, MemoryStorageAdapter, ModelStore } from "@vidstudio/engine";

export async function headlessBackend(): Promise<EditorBackendImpl> {
  const store = new ModelStore(new MemoryStorageAdapter(), { files: new MemoryFileStorage() });
  const project = await store.projects.create("Test", { frame_rate: 30 });
  const backend = new EditorBackendImpl({ store, projectId: project.id });
  await backend.open();
  return backend;
}

Works headless	Throws engine-required headless
Every read, snapshot, every project setting	importVideo, importAudio, importImage, addColorLayer
Every clip, text, effect, keyframe, transition and layer edit	addTextTrack, addTextTrackWithClips, addClipFromAsset, addTrackFromAsset
undo, redo	removeTrack, setTrackStartTime, setTrackVolume
importAsset, importFromUrl, readAssetFile, readAssetPreviewFile	Every playback method
saveDocument, loadDocument	renderFrameAt, exportProject, transcribeClipToTextTrack
getLicenceState (always "invalid" in Node, where there is no page)

In Node, use importAsset with a File and a probe, or loadDocument with a resolver, to get media into a project. Edit it, then saveDocument() and load the document in a browser to render or export.

Codecs and export settings

detectCodecs asks the browser which encoders it has. It tries H.264 profiles from High 5.2 down to Baseline 3.0 and falls back to VP9. For sound it tries AAC-LC and falls back to Opus. fallback is true when either fell back.

ExportSettings field	Meaning	Usual value
width, height	Output size in pixels. H.264 needs even numbers.	1920, 1080
frameRate	Frames per second	30
videoBitrate	Bits per second	8000000
videoCodec	The WebCodecs codec string	codecs.video.webCodec
muxerVideoCodec	"avc" or "vp9"	codecs.video.muxerCodec
audioBitrate	Bits per second	128000
audioSampleRate	Samples per second	48000
audioChannels	Channel count	2
audioCodec	The WebCodecs codec string	codecs.audio.webCodec
muxerAudioCodec	"aac" or "opus"	codecs.audio.muxerCodec

exportProject resolves to an ExportResult with three fields. opfsPath is where the MP4 sits in browser storage. size is its length in bytes. skippedAudioClips lists the audio clips that failed to decode and were rendered as silence. Read the file with readExportFile(opfsPath). Delete it with discardExportFile(opfsPath) once the user has it, or it stays in storage.

Errors

A refusal is an EditorBackendError. Its code field tells you why. Check the code, not the message.

import { EditorBackendError } from "@vidstudio/engine";

export function explain(err: unknown): string {
  if (err instanceof EditorBackendError && err.code === "invalid-split") return "Move the cut further inside the clip.";
  if (err instanceof EditorBackendError) return `The backend refused: ${err.code}`;
  return "Something else failed";
}

Code	When it happens
engine-required	An engine method was called on a headless backend, or before open() finished.
clip-not-found	No clip has that id.
track-not-found	No track has that id, or the track belongs to another project.
asset-not-found	No media asset has that id.
effect-not-found	No effect has that id, or the effect is not on that clip.
keyframe-not-found	No keyframe has that id.
transition-not-found	No transition has that id.
transform-not-found	The effect id is not in the catalog.
invalid-argument	A value is out of range: a time outside the clip, an unknown easing, a transition that does not fit, a clip placed twice, a name over 60 characters, a clip that is not linked.
invalid-split	The cut is closer than 0.05 s to a clip edge.
type-mismatch	A clip and a track of different kinds, a volume on a video track, a hidden audio track, two clips of the same kind linked.
clip-overlap	The clip would overlap another clip on its new track.
nothing-to-export	The timeline is empty.
export-cancelled	cancelExport() was called while an export ran.
import-failed	The file or the web address could not be decoded, downloaded or stored.
missing-media	A clip plays an asset whose file never arrived at load time.
not-implemented	Reserved; nothing throws it today.

A failure that is not a refusal, such as a decoder crash, is a plain Error.

A complete example

This opens a backend, imports a video, adds a title, exports the timeline and returns the MP4.

import { createStore, detectCodecs, EditorBackendImpl } from "@vidstudio/engine";
import type { ExportSettings } from "@vidstudio/engine";

export async function exportWithTitle(video: File): Promise<File> {
  const store = await createStore({ namespace: "my-app" });
  const project = await store.projects.create("Titled", { frame_rate: 30 });
  const canvas = document.createElement("canvas");
  const backend = new EditorBackendImpl({
    store,
    projectId: project.id,
    previewCanvas: canvas,
    options: { namespace: "my-app" },
  });
  await backend.open();
  await backend.setCanvasSize({ width: 1280, height: 720 });

  await backend.importVideo(video);
  const textTrackId = await backend.addTextTrack();
  const timeline = await backend.getTimeline();
  const textTrack = timeline.tracks.find((track) => track.id === textTrackId);
  if (!textTrack) throw new Error("text track not found");
  await backend.updateClip(textTrack.clips[0].id, { text_content: "Hello" });

  const codecs = await detectCodecs({ width: 1280, height: 720, audioSampleRate: 48000, audioChannels: 2 });
  const settings: ExportSettings = {
    width: 1280,
    height: 720,
    frameRate: 30,
    videoBitrate: 8_000_000,
    videoCodec: codecs.video.webCodec,
    muxerVideoCodec: codecs.video.muxerCodec,
    audioBitrate: 128_000,
    audioSampleRate: 48000,
    audioChannels: 2,
    audioCodec: codecs.audio.webCodec,
    muxerAudioCodec: codecs.audio.muxerCodec,
  };
  const result = await backend.exportProject(settings);
  const file = await backend.readExportFile(result.opfsPath);
  console.log(`${result.size} bytes, ${result.skippedAudioClips.length} audio clips skipped`);
  await backend.dispose();
  return file;
}

The canvas never joins the page. It only gives the player somewhere to draw. On localhost the export carries the mark; see Limits and support.