Rendering Canvas to mp4 using NodeJS

less than 1 minute read

I recently put together a “it just works” version of WOsu and WOsu-record.

WOsu is a JavaScript library for the rhythm game osu!. Although designed to use any sort of rendering (Canvas, CSS), its working implementation primarily relies on Three.js and WebGL.

Currently the primary use of WOsu is the Web osu! Replay Viewer, which allows users to view replays straight on the web.

WOsu-record is an Electron application that uses WOsu, ffmpeg and nodejs to record replays to a video file.

Currently the idea is to use the ffmpeg image2pipe feature to write frames through stdin.

First spawn a child ffmpeg process.

var recorder = child_process.spawn("ffmpeg.exe", [
    "-y", "-f", "image2pipe",
    "-vcodec", "png", "-r", "60",
    "-i", "-", "-vcodec", "h264",
    "-r", "60", "output.mp4"
]);

Next, for each frame grab the canvas data and convert to a binary string.

var url = canvas.toDataURL();
var data = atob( url.substring(url.indexOf("base64") + 7) );

Finally, we can write the data to stdin.

recorder.stdin.write(data, "binary");

When finished, we can close the stream.

recorder.stdin.end();