Add a Benchmark button that measures the solver and the GPU's clock ramp - concept-collection/turing-sphere-2

concept-collection / turing-sphere-2

Add a Benchmark button that measures the solver and the GPU's clock ramp

Diagnosing "the terminal is faster than the browser" needed the browser to make the same measurement the terminal does, which until now meant opening test.html and reading a soak line. The button does it in place: rendering paused, batches submitted continuously for two seconds, nothing read back, no animation pacing. It also reports the ramp — the first third of the run against the last. GPUs downclock when idle and an animation-paced loop leaves them idle most of every 16 ms frame, so a large ramp says the steady-state rate is limited by clocks rather than by the work, which is otherwise very hard to distinguish from a slower GPU stack. Buffer robustness is ruled out while here. WebGPU clamps every array access for safety, which looked like it could cost real time in the transform kernels' inner loops; measured against Dawn's disable_robustness toggle it makes no difference at all, 0.59 ms/step either way. README records that alongside the encoding result so neither becomes folklore, and notes that other GPU work invalidates a comparison made across two separate invocations.

Jeremy Magland <jmagland@flatironinstitute.org> committed 20 days ago commit 3221abd82e48 parent 17db8f1 Browse files

3 changed files+80−4

README.mdmodified+22−4View file

@@ -241,13 +241,21 @@ browser quietly falling back to a software adapter is a common cause of "the

241241 browser is much slower", and then the ratio compares different hardware and means

242242 nothing.

243243

244-By hand, four numbers, in increasing order of what they include:

244+Or press **Benchmark** in the app: it pauses rendering and runs batches

245+continuously for two seconds, reporting the same measurement the terminal makes,

246+plus the **ramp** — the first third of the run against the last. GPUs downclock

247+when idle and an animation-paced loop leaves them idle most of every frame, so a

248+large ramp means the steady-state number is limited by clocks rather than by the

249+work.

250+

251+By hand, five numbers, in increasing order of what they include:

245252

246253 | number | includes |

247254 |---|---|

248255 | `npm run bench -- --lmax 63` | desktop solver: batched steps, one sync per batch, in-process Dawn |

249-| `test.html?soak=2000&lmax=63` → `solver` | browser solver: same batching, no rendering at all |

250-| the app's `solver` | browser solver, measured in a periodic batch of 32 |

256+| the app's **Benchmark** button | browser solver, sustained, no rendering, no pacing |

257+| `test.html?soak=2000&lmax=63` → `solver` | the same, without the page around it |

258+| the app's `solver` | browser solver, one batch of 32 every two seconds |

251259 | the app's `ms/frame` | four steps **plus** a readback per species, colormapping and the vertex upload |

252260

253261 If the soak matches the benchmark, the solver is fine in the browser and

@@ -271,7 +279,17 @@ remaining suspects are:

271279 - **clocks.** An animation-paced loop leaves the GPU idle for most of each 16 ms

272280 frame, so it may never leave its low-power state, while the benchmark hammers it

273281 continuously and boosts. On a thermally managed laptop this alone can be worth a

274- factor of two, and it is not something the code can fix.

282+ factor of two, and it is not something the code can fix. The **Benchmark**

283+ button's ramp figure measures it directly.

284+- **anything else using the GPU.** Another process competing for it changes

285+ whichever run overlaps it, which makes a comparison across two separate

286+ invocations meaningless. `compare-perf.mjs` runs both sides back to back in one

287+ invocation partly for this reason.

288+- **not buffer robustness**, another plausible suspect: WebGPU clamps every array

289+ access for safety, which could cost real time in the transform kernels' inner

290+ loops. Measured with Dawn's `disable_robustness` toggle

291+ (`DAWN_FLAGS='enable-dawn-features=disable_robustness' npm run bench`), it makes

292+ no difference here at all — 0.59 ms/step either way.

275293 - **which browser.** WebGPU implementations differ substantially in maturity;

276294 Chrome and Safari are not interchangeable for this.

277295

index.htmlmodified+2−0View file

@@ -184,12 +184,14 @@

184184 <select id="colormap"></select>

185185 </label>

186186 <button id="runpause" class="primary">Run</button>

187+ <button id="benchmark">Benchmark</button>

187188 <button id="reseed">Re-seed</button>

188189 <button id="resetview">Reset view</button>

189190 </div>

190191 <div class="controls" id="params"></div>

191192 <div id="panels"></div>

192193 <p class="stats" id="stats"></p>

194+ <p class="stats" id="benchresult"></p>

193195 <div class="editor">

194196 <div class="editor-head">

195197 <span id="editor-title">the solver, in MATLAB</span>

src/main.tsmodified+56−0View file

@@ -28,11 +28,13 @@ const elModel = $<HTMLSelectElement>('model');

2828 const elLmax = $<HTMLSelectElement>('lmax');

2929 const elColormap = $<HTMLSelectElement>('colormap');

3030 const elRunPause = $<HTMLButtonElement>('runpause');

31+const elBenchmark = $<HTMLButtonElement>('benchmark');

3132 const elReseed = $<HTMLButtonElement>('reseed');

3233 const elResetView = $<HTMLButtonElement>('resetview');

3334 const elParams = $('params');

3435 const elPanels = $('panels');

3536 const elStats = $('stats');

37+const elBenchResult = $('benchresult');

3638 const elCmd = $('cmd');

3739 const elCopyCmd = $<HTMLButtonElement>('copycmd');

3840 const elBlurb = $('blurb');

@@ -188,6 +190,7 @@ function setRunning(next: boolean): void {

188190 }

189191

190192 elRunPause.addEventListener('click', () => setRunning(!running));

193+elBenchmark.addEventListener('click', () => void benchmark());

191194 elReseed.addEventListener('click', () => {

192195 seed = (Math.random() * 2 ** 31) >>> 0;

193196 setRunning(false);

@@ -456,6 +459,59 @@ async function pump(): Promise<void> {

456459 }

457460 }

458461

462+/**

463+ * Sustained solver benchmark, in the page.

464+ *

465+ * The same measurement `npm run bench` makes: batches of steps submitted

466+ * together, waited for, never read back, with no rendering and no animation

467+ * pacing in between. That makes it directly comparable to the terminal number,

468+ * which is the only way to tell a genuinely slower browser GPU stack apart from

469+ * the costs the app adds on top.

470+ *

471+ * It also reports the ramp — the first third of the run against the last. GPUs

472+ * downclock when idle, and an animation-paced loop leaves them idle most of every

473+ * frame, so a large ramp means the app's steady-state number is limited by clocks

474+ * rather than by the work.

475+ *

476+ * These are ordinary steps: the simulation advances by them.

477+ */

478+async function benchmark(): Promise<void> {

479+ if (!session) return;

480+ setRunning(false);

481+ const BATCH = 32;

482+ const DURATION_MS = 2000;

483+ elBenchResult.textContent = 'benchmarking…';

484+ await nextFrame();

485+

486+ const gen = generation;

487+ const perStep: number[] = [];

488+ const t0 = performance.now();

489+ while (performance.now() - t0 < DURATION_MS) {

490+ const b0 = performance.now();

491+ session.step(BATCH);

492+ await session.sync();

493+ if (gen !== generation) return;

494+ perStep.push((performance.now() - b0) / BATCH);

495+ }

496+

497+ const mean = (xs: number[]): number => xs.reduce((a, b) => a + b, 0) / xs.length;

498+ const all = mean(perStep);

499+ const best = Math.min(...perStep);

500+ const third = Math.max(1, Math.floor(perStep.length / 3));

501+ const first = mean(perStep.slice(0, third));

502+ const last = mean(perStep.slice(-third));

503+ const steps = perStep.length * BATCH;

504+

505+ elBenchResult.innerHTML =

506+ `sustained solver: <b>${all.toFixed(2)} ms/step</b> ` +

507+ `(${(1000 / all).toFixed(0)} steps/s) · best ${best.toFixed(2)} · ` +

508+ `ramp ${(first / last).toFixed(2)}× (${first.toFixed(2)} → ${last.toFixed(2)}) · ` +

509+ `${steps} steps in batches of ${BATCH} · ` +

510+ `compare with <code>npm run bench -- --lmax ${session.cfg.lmax}</code>`;

511+ await draw();

512+ updateStats();

513+}

514+

459515 // ---------------------------------------------------------------- boot

460516 async function boot(): Promise<void> {

461517 elModel.value = presets[0].key;