Save the run as a movie: the scene, its caption and the sound - proof-of-concept/plate-dynamics-lab

proof-of-concept / plate-dynamics-lab

Save the run as a movie: the scene, its caption and the sound

The 3-D scene as it is oriented, composited with a title bar carrying the simulated time and a caption bar carrying the field, its colour scale and the exaggeration, recorded with the radiated sound as its audio track. The scene component now draws a frame on demand through an imperative handle, so the recorder can render at the movie's own resolution between React renders. WebM with Opus comes first because Chrome and Firefox record MP4 with no audio codec at all; MP4 is left for Safari. MediaRecorder omits the length from a live WebM, so webmDuration.ts splices a Duration element into the Info section from the measured recording time, which turns ffmpeg's "Duration: N/A" into a real one. Verified end to end in headless Chrome: a 1 s recording plays back with both streams and its declared duration, and 40 of 40 frames were drawn.

Jeremy Magland <jeremy.magland@gmail.com> committed 1 month ago commit 7540997bff33 parent 80c0b6d Browse files

9 changed files+1308−382

README.mdmodified+11−1View file

@@ -1,6 +1,6 @@

11 # Plate Dynamics Lab

22

3-An interactive lab for the dynamics of a thin metal plate held the way a plate can actually be held on a bench: hung from a thread or two, resting on foam or felt pads, on a cymbal stand, gripped in a vise, or pinched between finger and thumb. Gravity is part of the model. A hung plate swings on its thread and rocks about its hole while it rings; a plate on pads sags into them and can be knocked off them; a cantilever droops and then vibrates about its droop. You choose the plate (a rectangle or an ellipse), the material, the mount and its orientation, and strike the plate or pull it aside and let go; the app shows the plate moving in its mount, the flexural displacement and surface stress fields, the rigid-body swing and the support forces over time, a probe time series and spectrum, and the radiated sound, which you can play or save.

3+An interactive lab for the dynamics of a thin metal plate held the way a plate can actually be held on a bench: hung from a thread or two, resting on foam or felt pads, on a cymbal stand, gripped in a vise, or pinched between finger and thumb. Gravity is part of the model. A hung plate swings on its thread and rocks about its hole while it rings; a plate on pads sags into them and can be knocked off them; a cantilever droops and then vibrates about its droop. You choose the plate (a rectangle or an ellipse), the material, the mount and its orientation, and strike the plate or pull it aside and let go; the app shows the plate moving in its mount, the flexural displacement and surface stress fields, the rigid-body swing and the support forces over time, a probe time series and spectrum, and the radiated sound, which you can play or save. The scene can also be saved as a movie, at whatever orientation and colouring you have set, with that sound as its audio track.

44

55 The numerics run in [numbl](https://github.com/flatironinstitute/numbl), a MATLAB-syntax environment executing in the browser. The MATLAB source shown on screen is the source that ran: edit any file and rerun. The time loop, which is hundreds of thousands of steps for a run of a few seconds, has a WebGPU port that is on by default; the setup, the statics, the stresses, the sound and the spectra run in numbl in double precision, and the double-precision loop in numbl remains the reference and the fallback. No physics is computed in JavaScript outside that port; the host draws arrays and plays samples.

66

@@ -64,6 +64,15 @@ The scenario picker sets everything at once; every field stays editable afterwar

6464 - Cantilever in a vise: a steel strip pulled down 10 mm and released.

6565 - Disk pinched in the hand: a soft, lossy grip near the rim.

6666

67+## Saving a run

68+

69+Two exports, both written by the browser itself with no server and no bundled encoder.

70+

71+- **The sound**, as a 16-bit mono WAV at the simulation's own sample rate, peak-normalized (the Rayleigh estimate is not a calibrated level, so the absolute scale would be misleading either way).

72+- **The movie**, as WebM (VP9 and Opus) or MP4 where the browser records it, 960 pixels wide: the 3-D scene as it stands, at its present orientation, exaggeration and colouring, with a title bar carrying the simulated time and a caption bar carrying the field, its colour scale and the exaggeration factors, so the file explains itself. The sound is the audio track. `packages/components/src/video/recordMovie.ts` composites the scene and the caption onto a canvas, captures that canvas as a video track, plays the sound into an audio track, and lets `MediaRecorder` mux the two.

73+

74+The recording is real time, which is what using nothing but the platform costs: a three-second movie takes three seconds to write, the tab has to stay in front, and a machine that cannot keep up drops frames rather than slowing down (the lab says how many of the run's frames it drew). Below real time the picture and the sound are slowed together, so the sound drops in pitch as a tape would. A run whose frames would need more than about sixty a second cannot have all of them drawn at real time, and the lab says so and offers the slower speeds.

75+

6776 ## Running

6877

6978 ```bash

@@ -91,4 +100,5 @@ npx numbl run --path packages/core/src/m my_script.m

91100 - Damping is the two-parameter loss model plus the mount's dashpots, chosen, not measured; the sigma1 loss acts on the flexural part only.

92101 - The probe series are averages over each output sample; stress fields are evaluated only at the saved frames.

93102 - Radiated sound is an estimate of the waveform, not a calibrated level: Rayleigh's integral in a baffle, or a rough front-back cancellation for an unbaffled plate.

103+- The movie is recorded in real time from the browser's own encoder, so its length is the length of the run divided by the speed, its bitrate is the browser's choice, and frames are dropped when the machine falls behind. It is a recording of the animation on screen, not a re-render: the time resolution is the run's saved frames, not its time step.

94104 - The GPU loop is single precision. The rigid-flexural split and the factored operators keep it close to the double-precision loop, but a run of many seconds accumulates rounding that the tests over tens of milliseconds do not measure.

apps/lab/src/views/ResultsView.tsxmodified+66−1View file

@@ -1,5 +1,6 @@

1-import { useEffect, useMemo, useState } from "react";

1+import { useEffect, useMemo, useRef, useState } from "react";

22 import {

3+ prepareAudio,

34 rangeOf,

45 symmetricRange,

56 type Field2D,

@@ -14,6 +15,7 @@ import {

1415 FieldView,

1516 FramePlayer,

1617 LinePlot,

18+ MovieExport,

1719 Plate3DView,

1820 ValidityNotes,

1921 useFramePlayer,

@@ -21,6 +23,8 @@ import {

2123 type FieldArrow,

2224 type FieldMarker,

2325 type FieldRect,

26+ type MovieOverlay,

27+ type Plate3DHandle,

2428 } from "@plate-dyn/components";

2529

2630 type Quantity = "w" | "w_total" | "vm" | "sp" | "env_vm" | "env_sp" | "w0" | "nvm";

@@ -119,6 +123,7 @@ export function ResultsView({ result, spec, method, fieldWidth = 460 }: ResultsV

119123 const setIndex = player.setIndex;

120124 useEffect(() => setIndex(startFrame), [startFrame, setIndex]);

121125

126+ const sceneRef = useRef<Plate3DHandle>(null);

122127 const nvm = useMemo(() => inPlaneVonMises(result), [result]);

123128 const frames: FieldFrames | undefined =

124129 quantity === "w"

@@ -229,6 +234,36 @@ export function ResultsView({ result, spec, method, fieldWidth = 460 }: ResultsV

229234 "#008b8b",

230235 ];

231236 const orientation = describeOrientation(result, spec);

237+ // The movie records the scene as it stands: same orientation, same colouring,

238+ // the sound the player and the WAV export use, and a caption that says what

239+ // is drawn and by how much it is exaggerated.

240+ const audioPrep = useMemo(() => prepareAudio(result.audio.p, result.audio.fs), [result.audio]);

241+ const movieSource = useMemo(

242+ () => ({

243+ canvas: () => sceneRef.current?.canvas() ?? null,

244+ drawFrame: (k: number, size?: { width: number; height: number }) =>

245+ sceneRef.current?.drawFrame(k, size),

246+ }),

247+ [],

248+ );

249+ const movieOverlay: MovieOverlay = useMemo(

250+ () => ({

251+ title: `${describePlate(spec)}, ${MOUNT_TEXT[spec.mount.type]}`,

252+ note: `flexure ×${flexScale}, rigid motion ×${rigidScale}`,

253+ legend:

254+ frames && colorRange

255+ ? {

256+ label: q.label,

257+ unit: q.unit,

258+ min: colorRange.min,

259+ max: colorRange.max,

260+ scale: q.scale,

261+ colormap: q.symmetric ? "bwr" : "viridis",

262+ }

263+ : undefined,

264+ }),

265+ [spec, flexScale, rigidScale, frames, colorRange, q],

266+ );

232267

233268 return (

234269 <div className="results">

@@ -244,6 +279,7 @@ export function ResultsView({ result, spec, method, fieldWidth = 460 }: ResultsV

244279 <div className="pl-scene">

245280 <div className="pl-plot-title">The plate in its mount</div>

246281 <Plate3DView

282+ ref={sceneRef}

247283 result={result}

248284 spec={spec}

249285 frameIndex={player.index}

@@ -280,6 +316,16 @@ export function ResultsView({ result, spec, method, fieldWidth = 460 }: ResultsV

280316 </label>

281317 <span className="pl-muted">drag to orbit, wheel to zoom</span>

282318 </div>

319+ <MovieExport

320+ source={movieSource}

321+ frameTimes={result.frames.t}

322+ audio={audioPrep}

323+ overlay={movieOverlay}

324+ aspect={0.7}

325+ filename={`plate-${spec.mount.type}`}

326+ restoreFrame={player.index}

327+ onRecording={(on) => on && player.setPlaying(false)}

328+ />

283329 <div className="pl-small pl-muted">

284330 {orientation} The static sag and the flexural field are exaggerated by the first factor,

285331 the swing and sway by the second (1 is true scale); the colour is the quantity chosen

@@ -491,6 +537,25 @@ export function ResultsView({ result, spec, method, fieldWidth = 460 }: ResultsV

491537 );

492538 }

493539

540+const MOUNT_TEXT: Record<string, string> = {

541+ thread: "hanging from one thread",

542+ threads: "hanging from two threads",

543+ pads: "resting on pads",

544+ cymbal: "on a cymbal stand",

545+ vise: "one edge in a vise",

546+ pinched: "pinched between fingers",

547+};

548+

549+/** The stock, for the movie's title: "300 × 200 × 2 mm sheet, A36 structural steel". */

550+function describePlate(spec: ProblemSpec): string {

551+ const p = spec.plate;

552+ const mm = (x: number) => Number((x * 1000).toPrecision(3)).toString();

553+ const round = p.shape === "ellipse" && p.Lx === p.Ly;

554+ const noun = p.shape === "rectangle" ? "sheet" : round ? "disk" : "ellipse";

555+ const dims = round ? `${mm(p.Lx)} × ${mm(p.h)}` : `${mm(p.Lx)} × ${mm(p.Ly)} × ${mm(p.h)}`;

556+ return `${dims} mm ${noun}, ${spec.material.name}`;

557+}

558+

494559 function captionFor(q: Quantity): string {

495560 switch (q) {

496561 case "w":

packages/components/src/index.tsmodified+17−1View file

@@ -30,4 +30,20 @@ export { EquationsView } from "./status/EquationsView";

3030 export { RunStatus, fmtMs, fmtTime, type RunStatusProps } from "./status/RunStatus";

3131 export { ValidityNotes } from "./status/ValidityNotes";

3232 export { AudioPlayer, type AudioPlayerProps } from "./audio/AudioPlayer";

33-export { Plate3DView, type Plate3DViewProps } from "./scene/Plate3DView";

33+export { Plate3DView, type Plate3DViewProps, type Plate3DHandle } from "./scene/Plate3DView";

34+export { MovieExport, type MovieExportProps } from "./video/MovieExport";

35+export {

36+ recordMovie,

37+ saveMovie,

38+ movieSupport,

39+ movieSpan,

40+ pickMovieFormat,

41+ frameAtTime,

42+ type MovieOptions,

43+ type MovieResult,

44+ type MovieSource,

45+ type MovieOverlay,

46+ type MovieLegend,

47+ type MovieSupport,

48+} from "./video/recordMovie";

49+export { withWebmDuration, readWebmDuration } from "./video/webmDuration";

packages/components/src/scene/Plate3DView.tsxmodified+413−379View file

@@ -1,4 +1,4 @@

1-import { useEffect, useMemo, useRef, useState } from "react";

1+import { forwardRef, useEffect, useImperativeHandle, useMemo, useRef, useState } from "react";

22 import {

33 colormapTable,

44 getColormap,

@@ -120,250 +120,252 @@ function rotate(v: Vec3, k: Vec3, a: number): Vec3 {

120120 * (applied as exact rotations about the pivot, scaled by rigidScale). Drag to

121121 * orbit, wheel to zoom.

122122 */

123-export function Plate3DView(props: Plate3DViewProps) {

124- const {

125- result,

126- spec,

127- frameIndex,

128- colorFrames,

129- colorRange,

130- colormap = "bwr",

131- flexScale,

132- rigidScale,

133- width = 460,

134- height = 340,

135- } = props;

136- const canvasRef = useRef<HTMLCanvasElement>(null);

137- const glRef = useRef<{

138- gl: WebGLRenderingContext;

139- prog: WebGLProgram;

140- buffers: { pos: WebGLBuffer; nrm: WebGLBuffer; col: WebGLBuffer; idx: WebGLBuffer };

141- lines: { pos: WebGLBuffer; col: WebGLBuffer; nrm: WebGLBuffer };

142- loc: Record<string, number | WebGLUniformLocation | null>;

143- } | null>(null);

144- const [cam, setCam] = useState({ az: 0.6, el: 0.35, zoom: 1 });

145- const drag = useRef<{ x: number; y: number } | null>(null);

123+/** What the movie recorder needs of the scene: its canvas, and a frame drawn on demand. */

124+export interface Plate3DHandle {

125+ canvas: () => HTMLCanvasElement | null;

126+ /** Draw frame k; `size` overrides the backing store, for recording at a larger size */

127+ drawFrame: (k: number, size?: { width: number; height: number }) => void;

128+}

146129

147- const nx = result.grid.x.length;

148- const ny = result.grid.y.length;

149- const { Lx, Ly } = spec.plate;

150- const size = Math.max(Lx, Ly);

130+export const Plate3DView = forwardRef<Plate3DHandle, Plate3DViewProps>(

131+ function Plate3DView(props, ref) {

132+ const {

133+ result,

134+ spec,

135+ frameIndex,

136+ colorFrames,

137+ colorRange,

138+ colormap = "bwr",

139+ flexScale,

140+ rigidScale,

141+ width = 460,

142+ height = 340,

143+ } = props;

144+ const canvasRef = useRef<HTMLCanvasElement>(null);

145+ const glRef = useRef<{

146+ gl: WebGLRenderingContext;

147+ prog: WebGLProgram;

148+ buffers: { pos: WebGLBuffer; nrm: WebGLBuffer; col: WebGLBuffer; idx: WebGLBuffer };

149+ lines: { pos: WebGLBuffer; col: WebGLBuffer; nrm: WebGLBuffer };

150+ loc: Record<string, number | WebGLUniformLocation | null>;

151+ } | null>(null);

152+ const [cam, setCam] = useState({ az: 0.6, el: 0.35, zoom: 1 });

153+ const drag = useRef<{ x: number; y: number } | null>(null);

151154

152- // Lab frame: X horizontal in the plate's plane, Y up, Z the rest

153- const frame = useMemo(() => {

154- const s = result.static;

155- const g: Vec3 = [s.gx, s.gy, s.gz];

156- const gl = Math.hypot(...g);

157- const up: Vec3 = gl > 0 ? scl(g, -1 / gl) : [0, 0, 1];

158- const eh: Vec3 = [result.rigid.eh[0], result.rigid.eh[1], 0];

159- const ed: Vec3 = [result.rigid.ed[0], result.rigid.ed[1], 0];

160- let X: Vec3 = Math.abs(dot(eh, up)) < 0.99 ? eh : [1, 0, 0];

161- X = norm(sub(X, scl(up, dot(X, up))));

162- const Z = cross(X, up);

163- const toLab = (p: Vec3): Vec3 => [dot(X, p), dot(up, p), dot(Z, p)];

164- return { toLab, ed, eh, up };

165- }, [result]);

155+ const nx = result.grid.x.length;

156+ const ny = result.grid.y.length;

157+ const { Lx, Ly } = spec.plate;

158+ const size = Math.max(Lx, Ly);

166159

167- // Static geometry of the triangles: which cells are on the plate

168- const mesh = useMemo(() => {

169- const w0 = result.static.w0.data;

170- const inside = (i: number, j: number) => Number.isFinite(w0[i + j * ny]);

171- const idx: number[] = [];

172- for (let j = 0; j < nx - 1; j++)

173- for (let i = 0; i < ny - 1; i++) {

174- if (!(inside(i, j) && inside(i + 1, j) && inside(i, j + 1) && inside(i + 1, j + 1)))

175- continue;

176- const a = i + j * ny;

177- const b = i + 1 + j * ny;

178- const c = i + (j + 1) * ny;

179- const d = i + 1 + (j + 1) * ny;

180- idx.push(a, b, c, b, d, c);

181- }

182- return { index: new Uint16Array(idx), inside };

183- }, [result, nx, ny]);

160+ // Lab frame: X horizontal in the plate's plane, Y up, Z the rest

161+ const frame = useMemo(() => {

162+ const s = result.static;

163+ const g: Vec3 = [s.gx, s.gy, s.gz];

164+ const gl = Math.hypot(...g);

165+ const up: Vec3 = gl > 0 ? scl(g, -1 / gl) : [0, 0, 1];

166+ const eh: Vec3 = [result.rigid.eh[0], result.rigid.eh[1], 0];

167+ const ed: Vec3 = [result.rigid.ed[0], result.rigid.ed[1], 0];

168+ let X: Vec3 = Math.abs(dot(eh, up)) < 0.99 ? eh : [1, 0, 0];

169+ X = norm(sub(X, scl(up, dot(X, up))));

170+ const Z = cross(X, up);

171+ const toLab = (p: Vec3): Vec3 => [dot(X, p), dot(up, p), dot(Z, p)];

172+ return { toLab, ed, eh, up };

173+ }, [result]);

184174

185- useEffect(() => {

186- const canvas = canvasRef.current;

187- if (!canvas) return;

188- const gl = canvas.getContext("webgl", { antialias: true });

189- if (!gl) return;

190- const compile = (type: number, src: string) => {

191- const sh = gl.createShader(type)!;

192- gl.shaderSource(sh, src);

193- gl.compileShader(sh);

194- return sh;

195- };

196- const prog = gl.createProgram()!;

197- gl.attachShader(prog, compile(gl.VERTEX_SHADER, VS));

198- gl.attachShader(prog, compile(gl.FRAGMENT_SHADER, FS));

199- gl.linkProgram(prog);

200- const mk = () => gl.createBuffer()!;

201- glRef.current = {

202- gl,

203- prog,

204- buffers: { pos: mk(), nrm: mk(), col: mk(), idx: mk() },

205- lines: { pos: mk(), col: mk(), nrm: mk() },

206- loc: {

207- aPos: gl.getAttribLocation(prog, "aPos"),

208- aNormal: gl.getAttribLocation(prog, "aNormal"),

209- aColor: gl.getAttribLocation(prog, "aColor"),

210- uProj: gl.getUniformLocation(prog, "uProj"),

211- uView: gl.getUniformLocation(prog, "uView"),

212- uFlat: gl.getUniformLocation(prog, "uFlat"),

213- },

214- };

215- return () => {

216- glRef.current = null;

217- };

218- }, []);

175+ // Static geometry of the triangles: which cells are on the plate

176+ const mesh = useMemo(() => {

177+ const w0 = result.static.w0.data;

178+ const inside = (i: number, j: number) => Number.isFinite(w0[i + j * ny]);

179+ const idx: number[] = [];

180+ for (let j = 0; j < nx - 1; j++)

181+ for (let i = 0; i < ny - 1; i++) {

182+ if (!(inside(i, j) && inside(i + 1, j) && inside(i, j + 1) && inside(i + 1, j + 1)))

183+ continue;

184+ const a = i + j * ny;

185+ const b = i + 1 + j * ny;

186+ const c = i + (j + 1) * ny;

187+ const d = i + 1 + (j + 1) * ny;

188+ idx.push(a, b, c, b, d, c);

189+ }

190+ return { index: new Uint16Array(idx), inside };

191+ }, [result, nx, ny]);

219192

220- useEffect(() => {

221- const r = glRef.current;

222- const canvas = canvasRef.current;

223- if (!r || !canvas) return;

224- const { gl, prog, buffers, lines, loc } = r;

225- const dpr = Math.min(2, window.devicePixelRatio || 1);

226- canvas.width = Math.round(width * dpr);

227- canvas.height = Math.round(height * dpr);

228- gl.viewport(0, 0, canvas.width, canvas.height);

193+ useEffect(() => {

194+ const canvas = canvasRef.current;

195+ if (!canvas) return;

196+ const gl = canvas.getContext("webgl", { antialias: true });

197+ if (!gl) return;

198+ const compile = (type: number, src: string) => {

199+ const sh = gl.createShader(type)!;

200+ gl.shaderSource(sh, src);

201+ gl.compileShader(sh);

202+ return sh;

203+ };

204+ const prog = gl.createProgram()!;

205+ gl.attachShader(prog, compile(gl.VERTEX_SHADER, VS));

206+ gl.attachShader(prog, compile(gl.FRAGMENT_SHADER, FS));

207+ gl.linkProgram(prog);

208+ const mk = () => gl.createBuffer()!;

209+ glRef.current = {

210+ gl,

211+ prog,

212+ buffers: { pos: mk(), nrm: mk(), col: mk(), idx: mk() },

213+ lines: { pos: mk(), col: mk(), nrm: mk() },

214+ loc: {

215+ aPos: gl.getAttribLocation(prog, "aPos"),

216+ aNormal: gl.getAttribLocation(prog, "aNormal"),

217+ aColor: gl.getAttribLocation(prog, "aColor"),

218+ uProj: gl.getUniformLocation(prog, "uProj"),

219+ uView: gl.getUniformLocation(prog, "uView"),

220+ uFlat: gl.getUniformLocation(prog, "uFlat"),

221+ },

222+ };

223+ return () => {

224+ glRef.current = null;

225+ };

226+ }, []);

229227

230- // ---- node positions in the lab frame

231- const N = nx * ny;

232- const kf = Math.min(frameIndex, result.frames.t.length - 1);

233- const phi = result.frames.w;

234- const w0 = result.static.w0.data;

235- const c = result.rigid.frames.map((row) => (kf >= 0 ? row[kf] : 0));

236- const pivot: Vec3 = [result.rigid.pivot.x, result.rigid.pivot.y, 0];

237- const { ed, eh, toLab } = frame;

238- const flexOf = (n: number) => {

239- const z0 = Number.isFinite(w0[n]) ? w0[n] : 0;

240- const f = kf >= 0 ? phi.data[n + kf * N] : 0;

241- return flexScale * (z0 + (Number.isFinite(f) ? f : 0));

242- };

243- const rigidPos = (p: Vec3): Vec3 => {

244- if (result.rigid.nr === 0) return p;

245- let q = sub(p, pivot);

246- q = rotate(q, eh, -c[1] * rigidScale);

247- q = rotate(q, ed, c[2] * rigidScale);

248- q = add(q, pivot);

249- return [q[0], q[1], q[2] + c[0] * rigidScale];

250- };

251- const pos = new Float32Array(3 * N);

252- const labPos: Vec3[] = new Array(N);

253- for (let j = 0; j < nx; j++)

254- for (let i = 0; i < ny; i++) {

255- const n = i + j * ny;

256- const p: Vec3 = [result.grid.x[j], result.grid.y[i], flexOf(n)];

257- const lp = toLab(rigidPos(p));

258- labPos[n] = lp;

259- pos[3 * n] = lp[0];

260- pos[3 * n + 1] = lp[1];

261- pos[3 * n + 2] = lp[2];

262- }

263- // normals from neighbouring nodes

264- const nrm = new Float32Array(3 * N);

265- for (let j = 0; j < nx; j++)

266- for (let i = 0; i < ny; i++) {

267- const n = i + j * ny;

268- const jl = Math.max(0, j - 1);

269- const jr = Math.min(nx - 1, j + 1);

270- const il = Math.max(0, i - 1);

271- const ir = Math.min(ny - 1, i + 1);

272- const tx = sub(labPos[i + jr * ny], labPos[i + jl * ny]);

273- const ty = sub(labPos[ir + j * ny], labPos[il + j * ny]);

274- const nn = norm(cross(tx, ty));

275- nrm[3 * n] = nn[0];

276- nrm[3 * n + 1] = nn[1];

277- nrm[3 * n + 2] = nn[2];

278- }

279- // colours

280- const col = new Float32Array(3 * N);

281- const table = colormapTable(getColormap(colormap));

282- const range = colorRange ?? { min: -1, max: 1 };

283- const span = range.max - range.min || 1;

284- for (let n = 0; n < N; n++) {

285- let v = colorFrames && kf >= 0 ? colorFrames.data[n + kf * N] : NaN;

286- if (!Number.isFinite(v)) v = 0.5 * (range.min + range.max);

287- let t = (v - range.min) / span;

288- t = t < 0 ? 0 : t > 1 ? 1 : t;

289- const k = Math.round(t * 255) * 3;

290- col[3 * n] = table[k] / 255;

291- col[3 * n + 1] = table[k + 1] / 255;

292- col[3 * n + 2] = table[k + 2] / 255;

293- }

228+ /**

229+ * Draw one frame. `out` overrides the backing store, which the movie

230+ * recorder uses to render the scene larger than it appears here; the next

231+ * on-screen draw puts it back.

232+ */

233+ const draw = (kIndex: number, out?: { width: number; height: number }) => {

234+ const r = glRef.current;

235+ const canvas = canvasRef.current;

236+ if (!r || !canvas) return;

237+ const { gl, prog, buffers, lines, loc } = r;

238+ const vw = out?.width ?? width;

239+ const vh = out?.height ?? height;

240+ const dpr = out ? 1 : Math.min(2, window.devicePixelRatio || 1);

241+ canvas.width = Math.round(vw * dpr);

242+ canvas.height = Math.round(vh * dpr);

243+ gl.viewport(0, 0, canvas.width, canvas.height);

294244

295- // ---- the mount, as lines and boxes in the lab frame

296- const L: number[] = [];

297- const LC: number[] = [];

298- const line = (a: Vec3, b: Vec3, rgb: Vec3) => {

299- L.push(...a, ...b);

300- LC.push(...rgb, ...rgb);

301- };

302- const grey: Vec3 = [0.35, 0.35, 0.38];

303- const dark: Vec3 = [0.15, 0.15, 0.18];

304- const red: Vec3 = [0.82, 0.29, 0.36];

305- const box = (lo: Vec3, hi: Vec3, rgb: Vec3, toLabFn = toLab) => {

306- const cs: Vec3[] = [];

307- for (let k = 0; k < 8; k++)

308- cs.push(toLabFn([k & 1 ? hi[0] : lo[0], k & 2 ? hi[1] : lo[1], k & 4 ? hi[2] : lo[2]]));

309- const e = [

310- [0, 1],

311- [2, 3],

312- [4, 5],

313- [6, 7],

314- [0, 2],

315- [1, 3],

316- [4, 6],

317- [5, 7],

318- [0, 4],

319- [1, 5],

320- [2, 6],

321- [3, 7],

322- ];

323- for (const [a, b] of e) line(cs[a], cs[b], rgb);

324- };

325- const circle = (centre: Vec3, r: number, rgb: Vec3) => {

326- const n = 32;

327- for (let k = 0; k < n; k++) {

328- const a0 = (2 * Math.PI * k) / n;

329- const a1 = (2 * Math.PI * (k + 1)) / n;

330- line(

331- toLab([centre[0] + r * Math.cos(a0), centre[1] + r * Math.sin(a0), centre[2]]),

332- toLab([centre[0] + r * Math.cos(a1), centre[1] + r * Math.sin(a1), centre[2]]),

333- rgb,

334- );

335- }

336- };

337- const m: Mount = spec.mount;

338- const nodeAt = (x: number, y: number): number => {

339- let best = 0;

340- let bd = Infinity;

245+ // ---- node positions in the lab frame

246+ const N = nx * ny;

247+ const kf = Math.min(kIndex, result.frames.t.length - 1);

248+ const phi = result.frames.w;

249+ const w0 = result.static.w0.data;

250+ const c = result.rigid.frames.map((row) => (kf >= 0 ? row[kf] : 0));

251+ const pivot: Vec3 = [result.rigid.pivot.x, result.rigid.pivot.y, 0];

252+ const { ed, eh, toLab } = frame;

253+ const flexOf = (n: number) => {

254+ const z0 = Number.isFinite(w0[n]) ? w0[n] : 0;

255+ const f = kf >= 0 ? phi.data[n + kf * N] : 0;

256+ return flexScale * (z0 + (Number.isFinite(f) ? f : 0));

257+ };

258+ const rigidPos = (p: Vec3): Vec3 => {

259+ if (result.rigid.nr === 0) return p;

260+ let q = sub(p, pivot);

261+ q = rotate(q, eh, -c[1] * rigidScale);

262+ q = rotate(q, ed, c[2] * rigidScale);

263+ q = add(q, pivot);

264+ return [q[0], q[1], q[2] + c[0] * rigidScale];

265+ };

266+ const pos = new Float32Array(3 * N);

267+ const labPos: Vec3[] = new Array(N);

341268 for (let j = 0; j < nx; j++)

342269 for (let i = 0; i < ny; i++) {

343- const d = (result.grid.x[j] - x) ** 2 + (result.grid.y[i] - y) ** 2;

344- if (d < bd && mesh.inside(i, j)) {

345- bd = d;

346- best = i + j * ny;

347- }

270+ const n = i + j * ny;

271+ const p: Vec3 = [result.grid.x[j], result.grid.y[i], flexOf(n)];

272+ const lp = toLab(rigidPos(p));

273+ labPos[n] = lp;

274+ pos[3 * n] = lp[0];

275+ pos[3 * n + 1] = lp[1];

276+ pos[3 * n + 2] = lp[2];

348277 }

349- return best;

350- };

351- const t = 0.02 * size; // a nominal thickness for the drawn parts

352- if (m.type === "thread") {

353- const n = nodeAt(m.x, m.y);

354- const hole = labPos[n];

355- const hook = add(

356- toLab([result.grid.x[Math.floor(n / ny)], result.grid.y[n % ny], 0]),

357- scl(frame.up, m.length),

358- );

359- line(hole, hook, dark);

360- line(add(hook, [-t, 0, 0]), add(hook, [t, 0, 0]), dark);

361- } else if (m.type === "threads") {

362- for (const [x, y] of [

363- [m.x1, m.y1],

364- [m.x2, m.y2],

365- ]) {

366- const n = nodeAt(x, y);

278+ // normals from neighbouring nodes

279+ const nrm = new Float32Array(3 * N);

280+ for (let j = 0; j < nx; j++)

281+ for (let i = 0; i < ny; i++) {

282+ const n = i + j * ny;

283+ const jl = Math.max(0, j - 1);

284+ const jr = Math.min(nx - 1, j + 1);

285+ const il = Math.max(0, i - 1);

286+ const ir = Math.min(ny - 1, i + 1);

287+ const tx = sub(labPos[i + jr * ny], labPos[i + jl * ny]);

288+ const ty = sub(labPos[ir + j * ny], labPos[il + j * ny]);

289+ const nn = norm(cross(tx, ty));

290+ nrm[3 * n] = nn[0];

291+ nrm[3 * n + 1] = nn[1];

292+ nrm[3 * n + 2] = nn[2];

293+ }

294+ // colours

295+ const col = new Float32Array(3 * N);

296+ const table = colormapTable(getColormap(colormap));

297+ const range = colorRange ?? { min: -1, max: 1 };

298+ const span = range.max - range.min || 1;

299+ for (let n = 0; n < N; n++) {

300+ let v = colorFrames && kf >= 0 ? colorFrames.data[n + kf * N] : NaN;

301+ if (!Number.isFinite(v)) v = 0.5 * (range.min + range.max);

302+ let t = (v - range.min) / span;

303+ t = t < 0 ? 0 : t > 1 ? 1 : t;

304+ const k = Math.round(t * 255) * 3;

305+ col[3 * n] = table[k] / 255;

306+ col[3 * n + 1] = table[k + 1] / 255;

307+ col[3 * n + 2] = table[k + 2] / 255;

308+ }

309+

310+ // ---- the mount, as lines and boxes in the lab frame

311+ const L: number[] = [];

312+ const LC: number[] = [];

313+ const line = (a: Vec3, b: Vec3, rgb: Vec3) => {

314+ L.push(...a, ...b);

315+ LC.push(...rgb, ...rgb);

316+ };

317+ const grey: Vec3 = [0.35, 0.35, 0.38];

318+ const dark: Vec3 = [0.15, 0.15, 0.18];

319+ const red: Vec3 = [0.82, 0.29, 0.36];

320+ const box = (lo: Vec3, hi: Vec3, rgb: Vec3, toLabFn = toLab) => {

321+ const cs: Vec3[] = [];

322+ for (let k = 0; k < 8; k++)

323+ cs.push(toLabFn([k & 1 ? hi[0] : lo[0], k & 2 ? hi[1] : lo[1], k & 4 ? hi[2] : lo[2]]));

324+ const e = [

325+ [0, 1],

326+ [2, 3],

327+ [4, 5],

328+ [6, 7],

329+ [0, 2],

330+ [1, 3],

331+ [4, 6],

332+ [5, 7],

333+ [0, 4],

334+ [1, 5],

335+ [2, 6],

336+ [3, 7],

337+ ];

338+ for (const [a, b] of e) line(cs[a], cs[b], rgb);

339+ };

340+ const circle = (centre: Vec3, r: number, rgb: Vec3) => {

341+ const n = 32;

342+ for (let k = 0; k < n; k++) {

343+ const a0 = (2 * Math.PI * k) / n;

344+ const a1 = (2 * Math.PI * (k + 1)) / n;

345+ line(

346+ toLab([centre[0] + r * Math.cos(a0), centre[1] + r * Math.sin(a0), centre[2]]),

347+ toLab([centre[0] + r * Math.cos(a1), centre[1] + r * Math.sin(a1), centre[2]]),

348+ rgb,

349+ );

350+ }

351+ };

352+ const m: Mount = spec.mount;

353+ const nodeAt = (x: number, y: number): number => {

354+ let best = 0;

355+ let bd = Infinity;

356+ for (let j = 0; j < nx; j++)

357+ for (let i = 0; i < ny; i++) {

358+ const d = (result.grid.x[j] - x) ** 2 + (result.grid.y[i] - y) ** 2;

359+ if (d < bd && mesh.inside(i, j)) {

360+ bd = d;

361+ best = i + j * ny;

362+ }

363+ }

364+ return best;

365+ };

366+ const t = 0.02 * size; // a nominal thickness for the drawn parts

367+ if (m.type === "thread") {

368+ const n = nodeAt(m.x, m.y);

367369 const hole = labPos[n];

368370 const hook = add(

369371 toLab([result.grid.x[Math.floor(n / ny)], result.grid.y[n % ny], 0]),

@@ -371,152 +373,184 @@ export function Plate3DView(props: Plate3DViewProps) {

371373 );

372374 line(hole, hook, dark);

373375 line(add(hook, [-t, 0, 0]), add(hook, [t, 0, 0]), dark);

376+ } else if (m.type === "threads") {

377+ for (const [x, y] of [

378+ [m.x1, m.y1],

379+ [m.x2, m.y2],

380+ ]) {

381+ const n = nodeAt(x, y);

382+ const hole = labPos[n];

383+ const hook = add(

384+ toLab([result.grid.x[Math.floor(n / ny)], result.grid.y[n % ny], 0]),

385+ scl(frame.up, m.length),

386+ );

387+ line(hole, hook, dark);

388+ line(add(hook, [-t, 0, 0]), add(hook, [t, 0, 0]), dark);

389+ }

390+ } else if (m.type === "pads") {

391+ for (const p of m.pads)

392+ box([p.x - p.rx, p.y - p.ry, -2 * t], [p.x + p.rx, p.y + p.ry, 0], grey);

393+ } else if (m.type === "cymbal") {

394+ const cx = Lx / 2;

395+ const cy = Ly / 2;

396+ circle([cx, cy, -t], m.r_felt, grey);

397+ circle([cx, cy, t], m.r_felt, grey);

398+ circle([cx, cy, -t], m.r_rod, dark);

399+ line(toLab([cx, cy, -6 * t]), toLab([cx, cy, 3 * t]), dark);

400+ } else if (m.type === "vise") {

401+ const d = m.depth;

402+ const lo: Vec3 =

403+ m.edge === "x0"

404+ ? [-t, -t, -t]

405+ : m.edge === "x1"

406+ ? [Lx - d, -t, -t]

407+ : m.edge === "y0"

408+ ? [-t, -t, -t]

409+ : [-t, Ly - d, -t];

410+ const hi: Vec3 =

411+ m.edge === "x0"

412+ ? [d, Ly + t, t]

413+ : m.edge === "x1"

414+ ? [Lx + t, Ly + t, t]

415+ : m.edge === "y0"

416+ ? [Lx + t, d, t]

417+ : [Lx + t, Ly + t, t];

418+ box(lo, hi, dark);

419+ } else if (m.type === "pinched") {

420+ const n = nodeAt(m.x, m.y);

421+ const p = labPos[n];

422+ const nrmL = norm(

423+ cross(

424+ sub(labPos[nodeAt(m.x + Lx / 10, m.y)], p),

425+ sub(labPos[nodeAt(m.x, m.y + Ly / 10)], p),

426+ ),

427+ );

428+ line(add(p, scl(nrmL, t)), add(p, scl(nrmL, 4 * t)), dark);

429+ line(add(p, scl(nrmL, -t)), add(p, scl(nrmL, -4 * t)), dark);

430+ }

431+ // gravity arrow beside the plate

432+ {

433+ const corner = add(toLab([Lx * 1.15, Ly * 1.05, 0]), [0, 0, 0]);

434+ const tip = add(corner, scl(frame.up, -0.25 * size));

435+ line(corner, tip, red);

436+ const side = scl(cross(frame.up, [0, 0, 1]), 0.03 * size);

437+ line(tip, add(add(tip, scl(frame.up, 0.06 * size)), side), red);

438+ line(tip, sub(add(tip, scl(frame.up, 0.06 * size)), side), red);

439+ }

440+ // outline of the undeformed plate at rest, faint

441+ {

442+ const pale: Vec3 = [0.75, 0.75, 0.78];

443+ if (spec.plate.shape === "ellipse") circle([Lx / 2, Ly / 2, 0], Lx / 2, pale);

444+ else box([0, 0, 0], [Lx, Ly, 0], pale);

374445 }

375- } else if (m.type === "pads") {

376- for (const p of m.pads)

377- box([p.x - p.rx, p.y - p.ry, -2 * t], [p.x + p.rx, p.y + p.ry, 0], grey);

378- } else if (m.type === "cymbal") {

379- const cx = Lx / 2;

380- const cy = Ly / 2;

381- circle([cx, cy, -t], m.r_felt, grey);

382- circle([cx, cy, t], m.r_felt, grey);

383- circle([cx, cy, -t], m.r_rod, dark);

384- line(toLab([cx, cy, -6 * t]), toLab([cx, cy, 3 * t]), dark);

385- } else if (m.type === "vise") {

386- const d = m.depth;

387- const lo: Vec3 =

388- m.edge === "x0"

389- ? [-t, -t, -t]

390- : m.edge === "x1"

391- ? [Lx - d, -t, -t]

392- : m.edge === "y0"

393- ? [-t, -t, -t]

394- : [-t, Ly - d, -t];

395- const hi: Vec3 =

396- m.edge === "x0"

397- ? [d, Ly + t, t]

398- : m.edge === "x1"

399- ? [Lx + t, Ly + t, t]

400- : m.edge === "y0"

401- ? [Lx + t, d, t]

402- : [Lx + t, Ly + t, t];

403- box(lo, hi, dark);

404- } else if (m.type === "pinched") {

405- const n = nodeAt(m.x, m.y);

406- const p = labPos[n];

407- const nrmL = norm(

408- cross(

409- sub(labPos[nodeAt(m.x + Lx / 10, m.y)], p),

410- sub(labPos[nodeAt(m.x, m.y + Ly / 10)], p),

411- ),

412- );

413- line(add(p, scl(nrmL, t)), add(p, scl(nrmL, 4 * t)), dark);

414- line(add(p, scl(nrmL, -t)), add(p, scl(nrmL, -4 * t)), dark);

415- }

416- // gravity arrow beside the plate

417- {

418- const corner = add(toLab([Lx * 1.15, Ly * 1.05, 0]), [0, 0, 0]);

419- const tip = add(corner, scl(frame.up, -0.25 * size));

420- line(corner, tip, red);

421- const side = scl(cross(frame.up, [0, 0, 1]), 0.03 * size);

422- line(tip, add(add(tip, scl(frame.up, 0.06 * size)), side), red);

423- line(tip, sub(add(tip, scl(frame.up, 0.06 * size)), side), red);

424- }

425- // outline of the undeformed plate at rest, faint

426- {

427- const pale: Vec3 = [0.75, 0.75, 0.78];

428- if (spec.plate.shape === "ellipse") circle([Lx / 2, Ly / 2, 0], Lx / 2, pale);

429- else box([0, 0, 0], [Lx, Ly, 0], pale);

430- }

431446

432- // ---- camera

433- const centre = toLab([Lx / 2, Ly / 2, 0]);

434- const dist = (2.2 * size) / cam.zoom;

435- const eye: Vec3 = add(centre, [

436- dist * Math.cos(cam.el) * Math.sin(cam.az),

437- dist * Math.sin(cam.el),

438- dist * Math.cos(cam.el) * Math.cos(cam.az),

439- ]);

440- const view = lookAt(eye, centre, [0, 1, 0]);

441- const proj = perspective(0.6, width / height, 0.01 * size, 20 * size);

447+ // ---- camera

448+ const centre = toLab([Lx / 2, Ly / 2, 0]);

449+ const dist = (2.2 * size) / cam.zoom;

450+ const eye: Vec3 = add(centre, [

451+ dist * Math.cos(cam.el) * Math.sin(cam.az),

452+ dist * Math.sin(cam.el),

453+ dist * Math.cos(cam.el) * Math.cos(cam.az),

454+ ]);

455+ const view = lookAt(eye, centre, [0, 1, 0]);

456+ const proj = perspective(0.6, vw / vh, 0.01 * size, 20 * size);

442457

443- gl.clearColor(0.97, 0.97, 0.975, 1);

444- gl.enable(gl.DEPTH_TEST);

445- gl.clear(gl.COLOR_BUFFER_BIT | gl.DEPTH_BUFFER_BIT);

446- gl.useProgram(prog);

447- gl.uniformMatrix4fv(loc.uProj as WebGLUniformLocation, false, proj);

448- gl.uniformMatrix4fv(loc.uView as WebGLUniformLocation, false, view);

449- const attr = (buf: WebGLBuffer, data: Float32Array, a: number) => {

450- gl.bindBuffer(gl.ARRAY_BUFFER, buf);

451- gl.bufferData(gl.ARRAY_BUFFER, data, gl.DYNAMIC_DRAW);

452- gl.enableVertexAttribArray(a);

453- gl.vertexAttribPointer(a, 3, gl.FLOAT, false, 0, 0);

458+ gl.clearColor(0.97, 0.97, 0.975, 1);

459+ gl.enable(gl.DEPTH_TEST);

460+ gl.clear(gl.COLOR_BUFFER_BIT | gl.DEPTH_BUFFER_BIT);

461+ gl.useProgram(prog);

462+ gl.uniformMatrix4fv(loc.uProj as WebGLUniformLocation, false, proj);

463+ gl.uniformMatrix4fv(loc.uView as WebGLUniformLocation, false, view);

464+ const attr = (buf: WebGLBuffer, data: Float32Array, a: number) => {

465+ gl.bindBuffer(gl.ARRAY_BUFFER, buf);

466+ gl.bufferData(gl.ARRAY_BUFFER, data, gl.DYNAMIC_DRAW);

467+ gl.enableVertexAttribArray(a);

468+ gl.vertexAttribPointer(a, 3, gl.FLOAT, false, 0, 0);

469+ };

470+ gl.uniform1f(loc.uFlat as WebGLUniformLocation, 0);

471+ attr(buffers.pos, pos, loc.aPos as number);

472+ attr(buffers.nrm, nrm, loc.aNormal as number);

473+ attr(buffers.col, col, loc.aColor as number);

474+ gl.bindBuffer(gl.ELEMENT_ARRAY_BUFFER, buffers.idx);

475+ gl.bufferData(gl.ELEMENT_ARRAY_BUFFER, mesh.index, gl.STATIC_DRAW);

476+ gl.drawElements(gl.TRIANGLES, mesh.index.length, gl.UNSIGNED_SHORT, 0);

477+ // lines

478+ gl.uniform1f(loc.uFlat as WebGLUniformLocation, 1);

479+ const lp = new Float32Array(L);

480+ attr(lines.pos, lp, loc.aPos as number);

481+ attr(lines.nrm, new Float32Array(L.length), loc.aNormal as number);

482+ attr(lines.col, new Float32Array(LC), loc.aColor as number);

483+ gl.drawArrays(gl.LINES, 0, lp.length / 3);

454484 };

455- gl.uniform1f(loc.uFlat as WebGLUniformLocation, 0);

456- attr(buffers.pos, pos, loc.aPos as number);

457- attr(buffers.nrm, nrm, loc.aNormal as number);

458- attr(buffers.col, col, loc.aColor as number);

459- gl.bindBuffer(gl.ELEMENT_ARRAY_BUFFER, buffers.idx);

460- gl.bufferData(gl.ELEMENT_ARRAY_BUFFER, mesh.index, gl.STATIC_DRAW);

461- gl.drawElements(gl.TRIANGLES, mesh.index.length, gl.UNSIGNED_SHORT, 0);

462- // lines

463- gl.uniform1f(loc.uFlat as WebGLUniformLocation, 1);

464- const lp = new Float32Array(L);

465- attr(lines.pos, lp, loc.aPos as number);

466- attr(lines.nrm, new Float32Array(L.length), loc.aNormal as number);

467- attr(lines.col, new Float32Array(LC), loc.aColor as number);

468- gl.drawArrays(gl.LINES, 0, lp.length / 3);

469- }, [

470- result,

471- spec,

472- frameIndex,

473- colorFrames,

474- colorRange,

475- colormap,

476- flexScale,

477- rigidScale,

478- width,

479- height,

480- cam,

481- frame,

482- mesh,

483- nx,

484- ny,

485- Lx,

486- Ly,

487- size,

488- ]);

489485

490- return (

491- <canvas

492- ref={canvasRef}

493- style={{ width, height, display: "block", cursor: "grab", borderRadius: 4 }}

494- onPointerDown={(e) => {

495- drag.current = { x: e.clientX, y: e.clientY };

496- (e.target as HTMLElement).setPointerCapture(e.pointerId);

497- }}

498- onPointerMove={(e) => {

499- if (!drag.current) return;

500- const dx = e.clientX - drag.current.x;

501- const dy = e.clientY - drag.current.y;

502- drag.current = { x: e.clientX, y: e.clientY };

503- setCam((c) => ({

504- ...c,

505- az: c.az - dx * 0.01,

506- el: Math.max(-1.4, Math.min(1.4, c.el + dy * 0.01)),

507- }));

508- }}

509- onPointerUp={() => {

510- drag.current = null;

511- }}

512- onWheel={(e) => {

513- e.preventDefault();

514- setCam((c) => ({

515- ...c,

516- zoom: Math.max(0.3, Math.min(5, c.zoom * Math.exp(-e.deltaY * 0.001))),

517- }));

518- }}

519- aria-label="the plate in its mount"

520- />

521- );

522-}

486+ // The latest draw, so that the recorder can call it between renders; a

487+ // re-render with unchanged props leaves the effect alone, so the two never

488+ // fight over the canvas.

489+ const drawRef = useRef(draw);

490+ drawRef.current = draw;

491+ useImperativeHandle(

492+ ref,

493+ () => ({

494+ canvas: () => canvasRef.current,

495+ drawFrame: (k: number, out?: { width: number; height: number }) => drawRef.current(k, out),

496+ }),

497+ [],

498+ );

499+

500+ useEffect(() => {

501+ drawRef.current(frameIndex);

502+ }, [

503+ result,

504+ spec,

505+ frameIndex,

506+ colorFrames,

507+ colorRange,

508+ colormap,

509+ flexScale,

510+ rigidScale,

511+ width,

512+ height,

513+ cam,

514+ frame,

515+ mesh,

516+ nx,

517+ ny,

518+ Lx,

519+ Ly,

520+ size,

521+ ]);

522+

523+ return (

524+ <canvas

525+ ref={canvasRef}

526+ style={{ width, height, display: "block", cursor: "grab", borderRadius: 4 }}

527+ onPointerDown={(e) => {

528+ drag.current = { x: e.clientX, y: e.clientY };

529+ (e.target as HTMLElement).setPointerCapture(e.pointerId);

530+ }}

531+ onPointerMove={(e) => {

532+ if (!drag.current) return;

533+ const dx = e.clientX - drag.current.x;

534+ const dy = e.clientY - drag.current.y;

535+ drag.current = { x: e.clientX, y: e.clientY };

536+ setCam((c) => ({

537+ ...c,

538+ az: c.az - dx * 0.01,

539+ el: Math.max(-1.4, Math.min(1.4, c.el + dy * 0.01)),

540+ }));

541+ }}

542+ onPointerUp={() => {

543+ drag.current = null;

544+ }}

545+ onWheel={(e) => {

546+ e.preventDefault();

547+ setCam((c) => ({

548+ ...c,

549+ zoom: Math.max(0.3, Math.min(5, c.zoom * Math.exp(-e.deltaY * 0.001))),

550+ }));

551+ }}

552+ aria-label="the plate in its mount"

553+ />

554+ );

555+ },

556+);

packages/components/src/styles.cssmodified+22−0View file

@@ -415,6 +415,28 @@ button.pl-btn-danger {

415415 cursor: default;

416416 }

417417

418+/* Movie export */

419+.pl-movie {

420+ display: flex;

421+ gap: 10px;

422+ align-items: center;

423+ flex-wrap: wrap;

424+ font-size: 12px;

425+}

426+.pl-movie .pl-bar {

427+ width: 130px;

428+ height: 7px;

429+ border: 1px solid var(--pl-border);

430+ border-radius: 4px;

431+ overflow: hidden;

432+ background: #fff;

433+}

434+.pl-movie .pl-bar > div {

435+ height: 100%;

436+ background: var(--pl-accent);

437+ transition: width 0.1s linear;

438+}

439+

418440 /* 3D scene */

419441 .pl-scene {

420442 display: flex;

packages/components/src/video/MovieExport.tsxadded+160−0View file

@@ -0,0 +1,160 @@

1+import { useMemo, useRef, useState } from "react";

2+import type { AudioPrep } from "@plate-dyn/core";

3+import {

4+ movieSpan,

5+ movieSupport,

6+ recordMovie,

7+ saveMovie,

8+ type MovieOverlay,

9+ type MovieResult,

10+ type MovieSource,

11+} from "./recordMovie";

12+

13+export interface MovieExportProps {

14+ source: MovieSource;

15+ /** Simulated time of every frame [s] */

16+ frameTimes: Float64Array;

17+ audio?: AudioPrep;

18+ overlay?: MovieOverlay;

19+ /** Height of the scene as a fraction of the width */

20+ aspect?: number;

21+ width?: number;

22+ /** Base name of the downloaded file, without extension */

23+ filename?: string;

24+ /** Frame to leave on screen when the recording ends */

25+ restoreFrame?: number;

26+ /** Told when the recording starts and stops, so the caller can pause its player */

27+ onRecording?: (recording: boolean) => void;

28+}

29+

30+const SPEEDS = [

31+ { value: 1, label: "real time" },

32+ { value: 0.5, label: "half speed" },

33+ { value: 0.25, label: "quarter speed" },

34+ { value: 0.1, label: "a tenth of speed" },

35+];

36+

37+type State =

38+ | { status: "idle" }

39+ | { status: "recording"; fraction: number }

40+ | { status: "done"; result: MovieResult; speed: number }

41+ | { status: "error"; message: string };

42+

43+/**

44+ * Records the scene as it stands, at its present orientation and colouring,

45+ * with the radiated sound under it. The recording is real time, so it takes as

46+ * long as the movie lasts; below real time the sound is slowed with the

47+ * picture and drops in pitch, as a tape would.

48+ */

49+export function MovieExport(props: MovieExportProps) {

50+ const { frameTimes, audio, overlay, aspect, width, filename = "plate", onRecording } = props;

51+ const support = useMemo(() => movieSupport(), []);

52+ const [speed, setSpeed] = useState(1);

53+ const [state, setState] = useState<State>({ status: "idle" });

54+ const sourceRef = useRef(props.source);

55+ sourceRef.current = props.source;

56+ const restoreRef = useRef(props.restoreFrame ?? 0);

57+ restoreRef.current = props.restoreFrame ?? 0;

58+ const abort = useRef<AbortController | null>(null);

59+

60+ const span = useMemo(() => movieSpan(frameTimes, audio), [frameTimes, audio]);

61+ const length = span / speed;

62+ const recording = state.status === "recording";

63+ // The recorder draws on the display's own refresh, so a run whose frames

64+ // come faster than that loses some of them; a slower speed spreads them out.

65+ const fps = length > 0 ? frameTimes.length / length : 0;

66+ const crowded = fps > 60;

67+

68+ const start = async () => {

69+ if (recording || support.status !== "available") return;

70+ const controller = new AbortController();

71+ abort.current = controller;

72+ setState({ status: "recording", fraction: 0 });

73+ onRecording?.(true);

74+ try {

75+ const result = await recordMovie({

76+ source: sourceRef.current,

77+ frameTimes,

78+ audio,

79+ overlay,

80+ aspect,

81+ width,

82+ speed,

83+ restoreFrame: restoreRef.current,

84+ signal: controller.signal,

85+ onProgress: (fraction) => setState({ status: "recording", fraction }),

86+ });

87+ saveMovie(result, `${filename}-${speed === 1 ? "realtime" : `${speed}x`}`);

88+ setState({ status: "done", result, speed });

89+ } catch (e) {

90+ setState({ status: "error", message: e instanceof Error ? e.message : String(e) });

91+ } finally {

92+ abort.current = null;

93+ onRecording?.(false);

94+ }

95+ };

96+

97+ return (

98+ <div className="pl-movie">

99+ <button

100+ className="pl-btn"

101+ onClick={() => void start()}

102+ disabled={recording || support.status !== "available" || frameTimes.length < 2}

103+ title="Record the scene as it is oriented, with the sound"

104+ >

105+ Save movie

106+ </button>

107+ <label className="pl-muted">

108+ speed{" "}

109+ <select

110+ className="pl-select"

111+ style={{ width: "auto" }}

112+ value={speed}

113+ disabled={recording}

114+ onChange={(e) => setSpeed(Number(e.target.value))}

115+ >

116+ {SPEEDS.map((s) => (

117+ <option key={s.value} value={s.value}>

118+ {s.label}

119+ </option>

120+ ))}

121+ </select>

122+ </label>

123+ {recording && (

124+ <>

125+ <div className="pl-bar" aria-label="recording progress">

126+ <div style={{ width: `${Math.round(state.fraction * 100)}%` }} />

127+ </div>

128+ <button className="pl-btn pl-btn-danger" onClick={() => abort.current?.abort()}>

129+ Stop

130+ </button>

131+ <span className="pl-muted">

132+ recording in real time, {Math.max(0, length * (1 - state.fraction)).toFixed(1)} s left;

133+ keep this tab in front

134+ </span>

135+ </>

136+ )}

137+ {!recording && support.status !== "available" && (

138+ <span className="pl-muted">Not available here ({support.reason}).</span>

139+ )}

140+ {!recording && support.status === "available" && state.status !== "done" && (

141+ <span className="pl-muted">

142+ {length.toFixed(1)} s of {support.ext.toUpperCase()} with the sound, written in real time

143+ {speed === 1 ? "" : "; the sound is slowed with the picture and drops in pitch"}.

144+ {crowded

145+ ? ` The run's ${frameTimes.length} frames would need ${fps.toFixed(0)} a second here, more than the display refreshes: choose a slower speed to keep them all.`

146+ : ""}

147+ </span>

148+ )}

149+ {!recording && state.status === "done" && (

150+ <span className="pl-muted">

151+ Saved {state.result.seconds.toFixed(1)} s, {state.result.width} × {state.result.height},{" "}

152+ {(state.result.blob.size / 1e6).toFixed(1)} MB, {state.result.drawn} of{" "}

153+ {state.result.frames} frames drawn

154+ {state.result.drawn < state.result.frames ? " (the rest were dropped to keep time)" : ""}.

155+ </span>

156+ )}

157+ {state.status === "error" && <span className="pl-issue">{state.message}</span>}

158+ </div>

159+ );

160+}

packages/components/src/video/recordMovie.tsadded+332−0View file

@@ -0,0 +1,332 @@

1+/**

2+ * The scene as a movie, with the radiated sound as its audio track.

3+ *

4+ * The browser's own recorder does the encoding: the composite canvas below is

5+ * captured as a video track, the sound is played into a MediaStream audio

6+ * track, and MediaRecorder muxes the two into WebM (or MP4 where the browser

7+ * writes it). That makes the recording real time, which is the price of using

8+ * nothing but the platform: a three-second movie takes three seconds to write,

9+ * and a machine too busy to keep up drops frames rather than slowing down.

10+ * The result reports how many of the run's frames were actually drawn.

11+ *

12+ * The composite canvas is the scene, drawn at the movie's own resolution

13+ * rather than the one on screen, with a title bar carrying the simulated time

14+ * and a caption bar carrying the field, its colour scale and the exaggeration.

15+ * Everything a viewer needs to read the movie is therefore in the file.

16+ */

17+import { colormapTable, getColormap, toAudioBuffer, type AudioPrep } from "@plate-dyn/core";

18+import { withWebmDuration } from "./webmDuration";

19+

20+export interface MovieLegend {

21+ label: string;

22+ unit: string;

23+ /** Range of the colour scale, in SI; `scale` multiplies it for the labels */

24+ min: number;

25+ max: number;

26+ scale: number;

27+ colormap: string;

28+}

29+

30+export interface MovieOverlay {

31+ title: string;

32+ /** A second line: what is exaggerated, and by how much */

33+ note?: string;

34+ legend?: MovieLegend;

35+}

36+

37+/** The scene the recorder draws from (Plate3DView's imperative handle). */

38+export interface MovieSource {

39+ canvas: () => HTMLCanvasElement | null;

40+ drawFrame: (k: number, size?: { width: number; height: number }) => void;

41+}

42+

43+export interface MovieOptions {

44+ source: MovieSource;

45+ /** Simulated time of every frame [s] */

46+ frameTimes: Float64Array;

47+ /** The sound, normalized as the player and the WAV export have it */

48+ audio?: AudioPrep;

49+ /** 1 is real time; below that the movie and the sound are slowed together */

50+ speed?: number;

51+ /** Width of the movie in pixels; the height follows `aspect` plus the bars */

52+ width?: number;

53+ /** Height of the scene as a fraction of the width */

54+ aspect?: number;

55+ overlay?: MovieOverlay;

56+ /** Frame to leave on screen when the recording ends */

57+ restoreFrame?: number;

58+ onProgress?: (fraction: number) => void;

59+ signal?: AbortSignal;

60+}

61+

62+export interface MovieResult {

63+ blob: Blob;

64+ ext: string;

65+ mimeType: string;

66+ /** Length of the movie [s] */

67+ seconds: number;

68+ width: number;

69+ height: number;

70+ /** Frames the run has, and how many of them the recording drew */

71+ frames: number;

72+ drawn: number;

73+}

74+

75+/**

76+ * Containers to try, in order. WebM with Opus comes first because it is the

77+ * one Chrome and Firefox record with the sound: their MP4 recording accepts no

78+ * audio codec at all (`isTypeSupported` says no to every `video/mp4` string

79+ * that names one), and a movie without its sound is not what was asked for.

80+ * The MP4 entries are for Safari, which records MP4 with AAC and no WebM.

81+ */

82+const FORMATS = [

83+ { mimeType: "video/webm;codecs=vp9,opus", ext: "webm" },

84+ { mimeType: "video/webm;codecs=vp8,opus", ext: "webm" },

85+ { mimeType: "video/webm", ext: "webm" },

86+ { mimeType: 'video/mp4;codecs="avc1.42E01E,mp4a.40.2"', ext: "mp4" },

87+ { mimeType: "video/mp4", ext: "mp4" },

88+];

89+

90+const defaultSupported = (mime: string): boolean =>

91+ typeof MediaRecorder !== "undefined" && MediaRecorder.isTypeSupported(mime);

92+

93+/** The first container this browser can write, or null. */

94+export function pickMovieFormat(

95+ isSupported: (mime: string) => boolean = defaultSupported,

96+): { mimeType: string; ext: string } | null {

97+ return FORMATS.find((f) => isSupported(f.mimeType)) ?? null;

98+}

99+

100+export type MovieSupport =

101+ | { status: "available"; mimeType: string; ext: string }

102+ | { status: "unavailable"; reason: string };

103+

104+/** Whether this browser can record the scene at all, and in what. */

105+export function movieSupport(): MovieSupport {

106+ if (typeof MediaRecorder === "undefined")

107+ return { status: "unavailable", reason: "this browser has no MediaRecorder" };

108+ if (

109+ typeof HTMLCanvasElement === "undefined" ||

110+ typeof HTMLCanvasElement.prototype.captureStream !== "function"

111+ )

112+ return { status: "unavailable", reason: "this browser cannot capture a canvas as video" };

113+ const f = pickMovieFormat();

114+ if (!f)

115+ return { status: "unavailable", reason: "this browser writes no video format the lab can use" };

116+ return { status: "available", mimeType: f.mimeType, ext: f.ext };

117+}

118+

119+/**

120+ * The last frame whose time has arrived at simulated time t, searching forward

121+ * from `from`. Frames are monotonic, so a run that falls behind skips to the

122+ * frame it should be showing rather than replaying the ones it missed.

123+ */

124+export function frameAtTime(frameTimes: ArrayLike<number>, t: number, from = 0): number {

125+ let k = Math.max(0, Math.min(from, frameTimes.length - 1));

126+ while (k + 1 < frameTimes.length && frameTimes[k + 1] <= t) k++;

127+ return k;

128+}

129+

130+/** Simulated span of the movie [s]: the frames, or the sound if it lasts longer. */

131+export function movieSpan(frameTimes: ArrayLike<number>, audio?: AudioPrep): number {

132+ const last = frameTimes.length ? frameTimes[frameTimes.length - 1] : 0;

133+ return Math.max(last, audio?.durationS ?? 0);

134+}

135+

136+const even = (x: number): number => 2 * Math.round(x / 2);

137+

138+const TEXT = "#1c2430";

139+const MUTED = "#5b6573";

140+const RULE = "#d9dde3";

141+

142+function fmt(x: number): string {

143+ if (x === 0) return "0";

144+ const a = Math.abs(x);

145+ if (a >= 1e4 || a < 1e-2) return x.toExponential(2);

146+ return Number(x.toPrecision(3)).toString();

147+}

148+

149+/** Record the scene, resolving with the finished file. */

150+export async function recordMovie(opts: MovieOptions): Promise<MovieResult> {

151+ const { source, frameTimes, audio, overlay, onProgress, signal } = opts;

152+ const speed = opts.speed ?? 1;

153+ const support = movieSupport();

154+ if (support.status !== "available") throw new Error(support.reason);

155+ if (!frameTimes.length) throw new Error("the run kept no frames to record");

156+ if (!(speed > 0)) throw new Error("the speed must be positive");

157+

158+ const width = even(opts.width ?? 960);

159+ const sceneH = even(width * (opts.aspect ?? 0.7));

160+ const titleH = overlay ? 44 : 0;

161+ const legendH = overlay ? (overlay.legend ? 62 : 32) : 0;

162+ const height = even(sceneH + titleH + legendH);

163+

164+ const out = document.createElement("canvas");

165+ out.width = width;

166+ out.height = height;

167+ const ctx = out.getContext("2d");

168+ if (!ctx) throw new Error("this browser gave no 2-D canvas to compose the movie on");

169+

170+ const drawOverlay = (t: number) => {

171+ ctx.fillStyle = "#fff";

172+ ctx.fillRect(0, 0, width, titleH);

173+ ctx.fillRect(0, titleH + sceneH, width, legendH);

174+ ctx.strokeStyle = RULE;

175+ ctx.lineWidth = 1;

176+ ctx.beginPath();

177+ ctx.moveTo(0, titleH + 0.5);

178+ ctx.lineTo(width, titleH + 0.5);

179+ ctx.moveTo(0, titleH + sceneH + 0.5);

180+ ctx.lineTo(width, titleH + sceneH + 0.5);

181+ ctx.stroke();

182+ if (!overlay) return;

183+ ctx.textBaseline = "alphabetic";

184+ ctx.fillStyle = TEXT;

185+ ctx.font = "600 19px system-ui, -apple-system, 'Segoe UI', Roboto, sans-serif";

186+ ctx.textAlign = "left";

187+ ctx.fillText(overlay.title, 16, 29);

188+ ctx.textAlign = "right";

189+ ctx.font = "17px ui-monospace, SFMono-Regular, Menlo, Consolas, monospace";

190+ ctx.fillText(`t = ${t.toFixed(3)} s`, width - 16, 29);

191+ ctx.textAlign = "left";

192+ const base = titleH + sceneH;

193+ const lg = overlay.legend;

194+ if (lg) {

195+ ctx.font = "13px system-ui, -apple-system, 'Segoe UI', Roboto, sans-serif";

196+ ctx.fillStyle = TEXT;

197+ ctx.fillText(`${lg.label} [${lg.unit}]`, 16, base + 20);

198+ const bx = 16;

199+ const by = base + 30;

200+ const bw = 240;

201+ const bh = 10;

202+ const table = colormapTable(getColormap(lg.colormap));

203+ for (let i = 0; i < bw; i++) {

204+ const k = Math.round((i / (bw - 1)) * 255) * 3;

205+ ctx.fillStyle = `rgb(${table[k]},${table[k + 1]},${table[k + 2]})`;

206+ ctx.fillRect(bx + i, by, 1, bh);

207+ }

208+ ctx.strokeStyle = RULE;

209+ ctx.strokeRect(bx + 0.5, by + 0.5, bw - 1, bh - 1);

210+ ctx.fillStyle = MUTED;

211+ ctx.font = "12px ui-monospace, SFMono-Regular, Menlo, Consolas, monospace";

212+ ctx.fillText(fmt(lg.min * lg.scale), bx, by + bh + 14);

213+ ctx.textAlign = "right";

214+ ctx.fillText(fmt(lg.max * lg.scale), bx + bw, by + bh + 14);

215+ ctx.textAlign = "left";

216+ }

217+ if (overlay.note) {

218+ ctx.fillStyle = MUTED;

219+ ctx.font = "13px system-ui, -apple-system, 'Segoe UI', Roboto, sans-serif";

220+ ctx.textAlign = "right";

221+ ctx.fillText(overlay.note, width - 16, base + (lg ? 40 : 21));

222+ ctx.textAlign = "left";

223+ }

224+ };

225+

226+ let drawn = 0;

227+ const render = (k: number) => {

228+ source.drawFrame(k, { width, height: sceneH });

229+ const src = source.canvas();

230+ ctx.fillStyle = "#fff";

231+ ctx.fillRect(0, titleH, width, sceneH);

232+ if (src) ctx.drawImage(src, 0, titleH, width, sceneH);

233+ drawOverlay(frameTimes[k] ?? 0);

234+ drawn++;

235+ };

236+

237+ const stream = out.captureStream(0);

238+ const video = stream.getVideoTracks()[0] as CanvasCaptureMediaStreamTrack;

239+ let actx: AudioContext | undefined;

240+ let node: AudioBufferSourceNode | undefined;

241+ const chunks: Blob[] = [];

242+ let rec: MediaRecorder | undefined;

243+ try {

244+ if (audio && audio.samples.length > 1) {

245+ actx = new AudioContext();

246+ await actx.resume();

247+ const dest = actx.createMediaStreamDestination();

248+ node = actx.createBufferSource();

249+ node.buffer = toAudioBuffer(actx, audio);

250+ // below real time the sound is slowed like a tape, and drops in pitch with it

251+ node.playbackRate.value = speed;

252+ node.connect(dest);

253+ const track = dest.stream.getAudioTracks()[0];

254+ if (track) stream.addTrack(track);

255+ }

256+

257+ rec = new MediaRecorder(stream, { mimeType: support.mimeType, videoBitsPerSecond: 8e6 });

258+ rec.ondataavailable = (e) => {

259+ if (e.data.size > 0) chunks.push(e.data);

260+ };

261+ const stopped = new Promise<void>((resolve) => {

262+ rec!.onstop = () => resolve();

263+ });

264+

265+ const span = movieSpan(frameTimes, audio);

266+ const wall = span / speed;

267+ render(0);

268+ video.requestFrame();

269+ const tRec = performance.now();

270+ rec.start(250);

271+ node?.start();

272+

273+ const t0 = performance.now();

274+ let last = 0;

275+ await new Promise<void>((resolve) => {

276+ const step = () => {

277+ const elapsed = (performance.now() - t0) / 1000;

278+ const k = frameAtTime(frameTimes, elapsed * speed, last);

279+ if (k !== last) {

280+ last = k;

281+ render(k);

282+ }

283+ video.requestFrame();

284+ onProgress?.(Math.min(1, elapsed / Math.max(wall, 1e-6)));

285+ if (signal?.aborted || elapsed >= wall + 0.15) resolve();

286+ else requestAnimationFrame(step);

287+ };

288+ requestAnimationFrame(step);

289+ });

290+

291+ rec.stop();

292+ await stopped;

293+ const seconds = (performance.now() - tRec) / 1000;

294+ let blob = new Blob(chunks, { type: support.mimeType });

295+ if (blob.size === 0) throw new Error("the browser's recorder produced an empty file");

296+ if (support.ext === "webm") {

297+ // the recorder leaves the length out, since it does not know it until now

298+ const patched = withWebmDuration(new Uint8Array(await blob.arrayBuffer()), seconds);

299+ blob = new Blob([patched as BlobPart], { type: support.mimeType });

300+ }

301+ return {

302+ blob,

303+ ext: support.ext,

304+ mimeType: support.mimeType,

305+ seconds,

306+ width,

307+ height,

308+ frames: frameTimes.length,

309+ drawn,

310+ };

311+ } finally {

312+ try {

313+ node?.stop();

314+ } catch {

315+ /* already stopped */

316+ }

317+ void actx?.close();

318+ for (const t of stream.getTracks()) t.stop();

319+ // back to the size and the frame the app is showing

320+ source.drawFrame(opts.restoreFrame ?? 0);

321+ }

322+}

323+

324+/** Hand the finished movie to the browser as a download. */

325+export function saveMovie(result: MovieResult, filename: string): void {

326+ const url = URL.createObjectURL(result.blob);

327+ const a = document.createElement("a");

328+ a.href = url;

329+ a.download = `${filename}.${result.ext}`;

330+ a.click();

331+ setTimeout(() => URL.revokeObjectURL(url), 0);

332+}

packages/components/src/video/webmDuration.tsadded+196−0View file

@@ -0,0 +1,196 @@

1+/**

2+ * A duration for the WebM the browser records.

3+ *

4+ * MediaRecorder writes a live stream: the Segment is left with an unknown

5+ * size and the Info section carries no Duration, because neither is known

6+ * while the recording is still running. Browsers cope by scanning the file,

7+ * but many other players do not: ffmpeg reports "Duration: N/A", editors and

8+ * upload previews show 0:00, and seeking is unreliable. Since the lab knows

9+ * exactly how long it recorded, it can say so.

10+ *

11+ * `withWebmDuration` splices a Duration element into the Info section (or

12+ * overwrites one that is already there) and returns the patched bytes. Only

13+ * the Info section moves; everything after it is copied unchanged, and the

14+ * Segment's unknown size stays unknown, which is legal and is what every

15+ * player already handles. Anything that does not parse as WebM is returned

16+ * untouched, so a browser that writes something else loses nothing.

17+ *

18+ * The format: an EBML element is an id, then a size as a variable-length

19+ * integer whose leading zeros give its byte count, then the payload. Duration

20+ * (id 0x4489) is a big-endian float in units of the Segment's TimecodeScale

21+ * (id 0x2AD7B1, nanoseconds, one millisecond by default).

22+ */

23+

24+const ID_EBML = 0x1a45dfa3;

25+const ID_SEGMENT = 0x18538067;

26+const ID_INFO = 0x1549a966;

27+const ID_TIMECODE_SCALE = 0x2ad7b1;

28+const ID_DURATION = 0x4489;

29+

30+interface Element {

31+ id: number;

32+ /** Offset of the id, the lengths of the id and size fields, then the payload's bounds */

33+ start: number;

34+ idLen: number;

35+ sizeLen: number;

36+ body: number;

37+ end: number;

38+ /** The size field said "unknown", so the payload runs to the end of its parent */

39+ unknownSize: boolean;

40+}

41+

42+/** Length in bytes of the variable-length integer starting with this byte, or 0 if invalid. */

43+function vintLength(first: number): number {

44+ for (let i = 0; i < 8; i++) if (first & (0x80 >> i)) return i + 1;

45+ return 0;

46+}

47+

48+/** Read one element header at `pos`, or null when the buffer runs out. */

49+function readElement(b: Uint8Array, pos: number, parentEnd: number): Element | null {

50+ if (pos >= parentEnd) return null;

51+ const idLen = vintLength(b[pos]);

52+ if (idLen === 0 || pos + idLen > parentEnd) return null;

53+ let id = 0;

54+ for (let i = 0; i < idLen; i++) id = id * 256 + b[pos + i];

55+ const sizePos = pos + idLen;

56+ if (sizePos >= parentEnd) return null;

57+ const sizeLen = vintLength(b[sizePos]);

58+ if (sizeLen === 0 || sizePos + sizeLen > parentEnd) return null;

59+ let size = b[sizePos] & (0xff >> sizeLen);

60+ let allOnes = size === (0xff >> sizeLen);

61+ for (let i = 1; i < sizeLen; i++) {

62+ size = size * 256 + b[sizePos + i];

63+ allOnes = allOnes && b[sizePos + i] === 0xff;

64+ }

65+ const body = sizePos + sizeLen;

66+ const end = allOnes ? parentEnd : Math.min(body + size, parentEnd);

67+ return { id, start: pos, idLen, sizeLen, body, end, unknownSize: allOnes };

68+}

69+

70+/** The first child with this id, searching the children of `parent` only. */

71+function findChild(b: Uint8Array, parent: Element, id: number): Element | null {

72+ let pos = parent.body;

73+ for (;;) {

74+ const el = readElement(b, pos, parent.end);

75+ if (!el) return null;

76+ if (el.id === id) return el;

77+ pos = el.end;

78+ if (pos <= el.start) return null; // a zero-length element would loop forever

79+ }

80+}

81+

82+/** An unsigned integer payload, for TimecodeScale. */

83+function readUint(b: Uint8Array, el: Element): number {

84+ let v = 0;

85+ for (let i = el.body; i < el.end; i++) v = v * 256 + b[i];

86+ return v;

87+}

88+

89+/** The EBML size field for `value`, at least `minLength` bytes long. */

90+export function writeVintSize(value: number, minLength = 1): Uint8Array {

91+ let len = minLength;

92+ while (len < 8 && value >= 2 ** (7 * len) - 1) len++;

93+ const out = new Uint8Array(len);

94+ let v = value;

95+ for (let i = len - 1; i >= 0; i--) {

96+ out[i] = v % 256;

97+ v = Math.floor(v / 256);

98+ }

99+ out[0] |= 0x80 >> (len - 1);

100+ return out;

101+}

102+

103+/** The Duration element: its id, its size, and the value as a big-endian double. */

104+function durationElement(ticks: number): Uint8Array {

105+ const out = new Uint8Array(11);

106+ out[0] = 0x44;

107+ out[1] = 0x89;

108+ out[2] = 0x88; // an 8-byte payload

109+ new DataView(out.buffer).setFloat64(3, ticks, false);

110+ return out;

111+}

112+

113+/**

114+ * Return the recording with its length written into the Info section. Bytes

115+ * that are not WebM, or that carry no Info section, come back unchanged.

116+ */

117+export function withWebmDuration(bytes: Uint8Array, seconds: number): Uint8Array {

118+ if (!(seconds > 0) || bytes.length < 8) return bytes;

119+ const header = readElement(bytes, 0, bytes.length);

120+ if (!header || header.id !== ID_EBML) return bytes;

121+ let segment: Element | null = null;

122+ let pos = header.end;

123+ while (pos < bytes.length) {

124+ const el = readElement(bytes, pos, bytes.length);

125+ if (!el || el.end <= el.start) break;

126+ if (el.id === ID_SEGMENT) {

127+ segment = el;

128+ break;

129+ }

130+ pos = el.end;

131+ }

132+ if (!segment) return bytes;

133+ const info = findChild(bytes, segment, ID_INFO);

134+ if (!info || info.unknownSize) return bytes;

135+

136+ const scaleEl = findChild(bytes, info, ID_TIMECODE_SCALE);

137+ const scale = scaleEl ? readUint(bytes, scaleEl) : 1e6; // nanoseconds per tick

138+ const ticks = (seconds * 1e9) / (scale > 0 ? scale : 1e6);

139+

140+ const existing = findChild(bytes, info, ID_DURATION);

141+ if (existing && existing.end - existing.body === 8) {

142+ const out = bytes.slice();

143+ new DataView(out.buffer, out.byteOffset, out.byteLength).setFloat64(existing.body, ticks, false);

144+ return out;

145+ }

146+ if (existing) return bytes; // a 4-byte float: leave a file we did not write alone

147+

148+ // Splice the element in, and grow the Info size field to match

149+ const add = durationElement(ticks);

150+ const payload = info.end - info.body;

151+ const idBytes = bytes.subarray(info.start, info.start + info.idLen);

152+ const newSize = writeVintSize(payload + add.length, info.sizeLen);

153+ const out = new Uint8Array(

154+ info.start + idBytes.length + newSize.length + payload + add.length + (bytes.length - info.end),

155+ );

156+ let o = 0;

157+ out.set(bytes.subarray(0, info.start), o);

158+ o += info.start;

159+ out.set(idBytes, o);

160+ o += idBytes.length;

161+ out.set(newSize, o);

162+ o += newSize.length;

163+ out.set(bytes.subarray(info.body, info.end), o);

164+ o += payload;

165+ out.set(add, o);

166+ o += add.length;

167+ out.set(bytes.subarray(info.end), o);

168+ return out;

169+}

170+

171+/** The duration an EBML file declares [s], or null when it declares none. */

172+export function readWebmDuration(bytes: Uint8Array): number | null {

173+ const header = readElement(bytes, 0, bytes.length);

174+ if (!header || header.id !== ID_EBML) return null;

175+ let pos = header.end;

176+ let segment: Element | null = null;

177+ while (pos < bytes.length) {

178+ const el = readElement(bytes, pos, bytes.length);

179+ if (!el || el.end <= el.start) break;

180+ if (el.id === ID_SEGMENT) {

181+ segment = el;

182+ break;

183+ }

184+ pos = el.end;

185+ }

186+ if (!segment) return null;

187+ const info = findChild(bytes, segment, ID_INFO);

188+ if (!info) return null;

189+ const dur = findChild(bytes, info, ID_DURATION);

190+ if (!dur) return null;

191+ const scaleEl = findChild(bytes, info, ID_TIMECODE_SCALE);

192+ const scale = scaleEl ? readUint(bytes, scaleEl) : 1e6;

193+ const view = new DataView(bytes.buffer, bytes.byteOffset, bytes.byteLength);

194+ const ticks = dur.end - dur.body === 4 ? view.getFloat32(dur.body, false) : view.getFloat64(dur.body, false);

195+ return (ticks * scale) / 1e9;

196+}

packages/components/test/movie.test.tsadded+91−0View file

@@ -0,0 +1,91 @@

1+import { describe, expect, it } from "vitest";

2+import { frameAtTime, movieSpan, pickMovieFormat } from "../src/video/recordMovie";

3+import { readWebmDuration, withWebmDuration } from "../src/video/webmDuration";

4+import type { AudioPrep } from "@plate-dyn/core";

5+

6+const prep = (durationS: number): AudioPrep => ({

7+ samples: new Float32Array(2),

8+ fs: 44100,

9+ peakPa: 1,

10+ gain: 1,

11+ durationS,

12+});

13+

14+describe("the movie recorder's bookkeeping", () => {

15+ it("prefers WebM with sound, and falls back to what the browser writes", () => {

16+ // WebM with Opus first: it is the one that records with the sound

17+ expect(pickMovieFormat(() => true)?.mimeType).toBe("video/webm;codecs=vp9,opus");

18+ expect(pickMovieFormat((m) => m === "video/webm")?.ext).toBe("webm");

19+ expect(pickMovieFormat((m) => m.startsWith("video/mp4"))?.ext).toBe("mp4");

20+ expect(pickMovieFormat(() => false)).toBe(null);

21+ });

22+

23+ it("shows the frame whose time has arrived, skipping the ones it fell behind", () => {

24+ const t = Float64Array.from([0, 0.1, 0.2, 0.3, 0.4]);

25+ expect(frameAtTime(t, 0)).toBe(0);

26+ expect(frameAtTime(t, 0.05)).toBe(0);

27+ expect(frameAtTime(t, 0.1)).toBe(1);

28+ expect(frameAtTime(t, 0.25)).toBe(2);

29+ expect(frameAtTime(t, 99)).toBe(4);

30+ // a recording that fell behind jumps forward rather than replaying

31+ expect(frameAtTime(t, 0.35, 1)).toBe(3);

32+ // and never goes back, since the search starts where it left off

33+ expect(frameAtTime(t, 0.0, 3)).toBe(3);

34+ });

35+

36+ it("runs as long as the frames or the sound, whichever lasts longer", () => {

37+ const t = Float64Array.from([0, 0.5, 1.0]);

38+ expect(movieSpan(t)).toBe(1.0);

39+ expect(movieSpan(t, prep(2.5))).toBe(2.5);

40+ expect(movieSpan(t, prep(0.4))).toBe(1.0);

41+ expect(movieSpan(new Float64Array(0), prep(3))).toBe(3);

42+ });

43+});

44+

45+/** A minimal WebM: the header, a Segment of unknown size, an Info with a TimecodeScale, a Cluster. */

46+function fakeWebm(): Uint8Array {

47+ const header = [0x1a, 0x45, 0xdf, 0xa3, 0x84, 1, 2, 3, 4];

48+ const segment = [0x18, 0x53, 0x80, 0x67, 0x01, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff];

49+ const info = [0x15, 0x49, 0xa9, 0x66, 0x87, 0x2a, 0xd7, 0xb1, 0x83, 0x0f, 0x42, 0x40];

50+ const cluster = [0x1f, 0x43, 0xb6, 0x75, 0x84, 9, 8, 7, 6];

51+ return Uint8Array.from([...header, ...segment, ...info, ...cluster]);

52+}

53+

54+describe("the length written into a recorded WebM", () => {

55+ it("writes a duration the file did not have, and keeps everything after it", () => {

56+ const before = fakeWebm();

57+ expect(readWebmDuration(before)).toBe(null);

58+ const after = withWebmDuration(before, 2.5);

59+ expect(after.length).toBe(before.length + 11);

60+ expect(readWebmDuration(after)).toBeCloseTo(2.5, 9);

61+ // the Cluster that followed the Info section is byte for byte where it was

62+ expect(Array.from(after.subarray(after.length - 9))).toEqual([

63+ 0x1f, 0x43, 0xb6, 0x75, 0x84, 9, 8, 7, 6,

64+ ]);

65+ // and the Info section now declares the longer payload: its size byte sits

66+ // after the 9-byte header, the 12-byte Segment header and the 4-byte id

67+ expect(after[9 + 12 + 4]).toBe(0x87 + 11);

68+ });

69+

70+ it("overwrites a duration that is already there", () => {

71+ const once = withWebmDuration(fakeWebm(), 2.5);

72+ const twice = withWebmDuration(once, 7.25);

73+ expect(twice.length).toBe(once.length);

74+ expect(readWebmDuration(twice)).toBeCloseTo(7.25, 9);

75+ });

76+

77+ it("writes the length in the units the timecode scale sets", () => {

78+ // the fixture's scale is 0x0F4240 ns, a millisecond, so a second is a thousand ticks

79+ const patched = withWebmDuration(fakeWebm(), 1);

80+ const view = new DataView(patched.buffer, patched.byteOffset, patched.byteLength);

81+ expect(view.getFloat64(patched.length - 9 - 8, false)).toBeCloseTo(1000, 9);

82+ expect(readWebmDuration(patched)).toBeCloseTo(1, 9);

83+ });

84+

85+ it("leaves anything that is not WebM alone", () => {

86+ const notWebm = Uint8Array.from([0, 0, 0, 0x20, 0x66, 0x74, 0x79, 0x70, 1, 2, 3, 4]);

87+ expect(withWebmDuration(notWebm, 3)).toBe(notWebm);

88+ expect(readWebmDuration(notWebm)).toBe(null);

89+ expect(withWebmDuration(fakeWebm(), 0)).toEqual(fakeWebm());

90+ });

91+});