What Is WebCodecs?

WebCodecs vs MediaRecorder vs WebAssembly FFmpeg

All three can turn frames into a video file in the browser, but they work at different levels.

The approaches combine well: WebCodecs for decoding and encoding, plus a small JavaScript or WebAssembly library for reading and writing containers such as MP4 and WebM. FFmpeg builds also carry LGPL or GPL license terms.

Browser support for WebCodecs

Chromium-based browsers, including Chrome and Edge, shipped WebCodecs in version 94 in 2021. Safari 16.4 added the video encoder and decoder, with the audio interfaces following in Safari 26. Firefox enabled WebCodecs on desktop in version 130. Mobile and per-codec support vary by operating system and hardware, so treat this as a snapshot and check current compatibility tables before you ship.

WebCodecs is only available in secure contexts (HTTPS), in pages and dedicated workers. Detect the interface first, then test the exact configuration you plan to use:

const cfg = {
  codec: "avc1.42001f",
  width: 1280,
  height: 720,
  bitrate: 4_000_000,
  framerate: 30,
};

if ("VideoEncoder" in globalThis) {
  const { supported } = await
    VideoEncoder.isConfigSupported(cfg);
  console.log(supported);
}

Here avc1.42001f means H.264 Baseline profile, level 3.1.

How a browser video editor uses WebCodecs

A browser-based editor that plays and exports video typically runs a pipeline like this:

1. Demux. A container library reads the MP4 or WebM file and returns encoded chunks plus decoder settings. For H.264 in MP4, those settings include a description taken from the file's avcC box.
2. Decode. VideoDecoder turns EncodedVideoChunk objects into VideoFrame objects; AudioDecoder produces AudioData.
3. Process. Frames are drawn to a canvas or imported into WebGPU as external textures for compositing, color, and effects. A new VideoFrame can be built from the rendered canvas.
4. Encode. VideoEncoder compresses the processed frames and returns encoded chunks, along with the decoder configuration a muxer needs.
5. Mux. A muxing library writes the chunks and their timestamps into a container file. Muxing and demuxing are not part of WebCodecs.

Running the pipeline in a worker keeps the interface responsive. For background on the formats themselves, see H.264 vs H.265 vs AV1 vs ProRes.

Common pitfalls

• Codec strings. WebCodecs expects a full codec string such as avc1.42001f, vp09.00.10.08, or av01.0.04M.08, not "h264". Its profile and level must cover your resolution and frame rate, or the configuration may be rejected.
• Hardware acceleration varies. A configuration that works on one device may be unsupported, or fall back to slower software encoding, on another, and hardware encoders differ in quality. The hardwareAcceleration option is only a preference, so keep a fallback codec or resolution.
• Memory and VideoFrame.close(). Decoded frames can hold large buffers, sometimes in GPU memory. Call close() on each frame as soon as you finish with it rather than waiting for garbage collection; decoders can stall when frames are not released.
• Queues, timing, and key frames. Watch decodeQueueSize and encodeQueueSize instead of queuing a whole file, call flush() to collect the last outputs, and remember that timestamps are in microseconds. Decoding must start from a key frame.