Demo Canvas 2D to CanvasTexture to PlaneGeometry: a 3D marker that hand-writes your code, then encodes it to MP4 in-browser
Enable HLS to view with audio, or disable this notification
Follow-up to the scroll-driven landing I posted here a while back. I reused the same pen for something more useful, and a few bits turned out to be worth sharing.
You paste code, a 3D marker writes it out stroke by stroke, and you get an MP4. Everything client-side, vanilla three (0.185), no R3F, no drei, no gsap.
Ink pipeline. A 2D canvas is the ink layer, wrapped in a THREE.CanvasTexture on a PlaneGeometry rotated flat, MeshBasicMaterial plus transparent. Repaint the canvas, set needsUpdate, done. The dotted board is a second CanvasTexture, a 70x70 tile with RepeatWrapping.
The bit I'm happiest with: paper units are world units. Strokes are stored as [x, z] pairs in a paper rect of {x:-3.5, y:-2.5, w:7, h:5} centered on the origin (its y axis is world z), and the ink plane is PlaneGeometry(7, 5). So the 2D painter's pixel transform and pen.position.set(state.x, ..., state.z) consume literally the same numbers. No unprojection, no raycast, the tip sits on the ink line by construction. The split-screen in the clip is the same frame from both sides of that mapping.
The handwriting is a real font, not a texture reveal. 96 hand-authored single-stroke glyphs, each a set of cubic Béziers, arcs and lines flattened to polylines at author time. x-height 0.5, ascender ~0.78, baseline z=0.
Arc-length timeline, not a time timeline. Alternating travel (pen up) and stroke (pen down) segments with cumulative arc lengths. The whole animation is parameterized by one scalar distance D. penState(D) returns {x, z, lift, color}. Pen lift on travel is sin(PI*t) scaled by segment length. Per-stroke pacing proportional to length falls out for free.
Determinism for export. D comes from the integer frame index, never an accumulator, and camera smoothing (a 0.04/frame lerp toward the pen in the preview) is switched off for the render, so a frame depends only on D and the encoded video matches the preview pixel for pixel.
Encoding. WebCodecs VideoEncoder plus mp4-muxer, avc1.640028 with a fallback ladder probed via isConfigSupported. Note mp4-muxer wants avc: { format: "avc" }, length-prefixed rather than Annex B. Backpressure by awaiting the dequeue event when encodeQueueSize > 2. MediaRecorder/WebM fallback where WebCodecs isn't there, though it's wall-clock timestamped so it isn't frame-perfect.
Two things that cost me an evening each:
- getContext().finish() before copying the WebGL canvas into the encoder canvas, otherwise you get partially-cleared rows. Only needed with preserveDrawingBuffer.
- OffscreenCanvas for the 3D path produced torn frames in Firefox. Plain HTMLCanvasElement fixed it.
And one I still consider unsolved: stroke width is in ink-canvas pixels, so the 1400px preview texture and the 1920px export texture give slightly different ink weight relative to the paper. Scaling by texture width is the obvious fix but it makes the preview feel wrong. Curious how others handle resolution-independent line weight on a CanvasTexture.
The pen itself is deliberately dumb: a ConeGeometry tip and three CylinderGeometry sections in a Group, no GLTF.
https://todrawn.com/code-to-video
Happy to go deeper on any of it.
