Fix the GPU suite in CI: keep it short, and split the metric tolerance - concept-collection/turing-surface

concept-collection / turing-surface

Fix the GPU suite in CI: keep it short, and split the metric tolerance

The headless runner died at exactly 180 s with "Runtime.callFunctionOn timed out": that is puppeteer's default protocolTimeout, which bounds the CDP call waitForFunction polls inside, so its own 600 s timeout could never take effect. Set it explicitly, and pass a copy of the flag array — puppeteer splices --enable-features out of the array it is given, which made the failure message report fewer flags than were actually used. The suite had also grown past what belongs in CI: the niter x geometry sweep was 353 s of its 386 s. It is compilation-bound, not compute-bound — no pipeline cache across sessions, and niter 8 unrolls to 445 kernels — so it costs ~3 s on desktop Dawn and minutes on software WebGPU. It now runs by default under test:node and only on request in the browser (--sweep). Lowering SWEEP_LMAX was not an option: at lmax 31 or 15 the known-divergent peanut cases all stay finite. The sphere inverse-metric check failed for real, at 8.8e-3 against a 2e-3 tolerance. It is conditioning, not a wrong formula: det and g_pp are both O(sin^2 theta), so the transforms' absolute fp32 error is amplified like 1/sin^2 theta toward the poles. Checked in two bands now, each set a few times the looser of Dawn and SwiftShader.

Jeremy Magland <jmagland@flatironinstitute.org> committed 14 days ago commit 3b2f40c8cde1 parent cc86692 Browse files

3 changed files+80−9

scripts/test-gpu.mjsmodified+22−3View file

@@ -3,7 +3,13 @@

33 * Chrome (falling back to the SwiftShader software WebGPU adapter when no

44 * hardware GPU is available), and reports the suite results.

55 *

6- * Run after `vite build`: node scripts/test-gpu.mjs

6+ * Run after `vite build`: node scripts/test-gpu.mjs [--sweep]

7+ *

8+ * --sweep adds the niter x geometry sweep, which the page leaves out by

9+ * default because it is far too slow here to belong in CI: every session it

10+ * builds recompiles its whole unrolled step (445 kernels at niter 8), and

11+ * software WebGPU compiles those at about a second each. See

12+ * test/geometryChecks.ts.

713 */

814 import { createServer } from 'node:http';

915 import { readFile } from 'node:fs/promises';

@@ -40,14 +46,27 @@ const flagSets = [

4046 ['--headless=new', '--no-sandbox', '--enable-unsafe-webgpu', '--use-webgpu-adapter=swiftshader', '--enable-unsafe-swiftshader'],

4147 ];

4248

49+const query = process.argv.includes('--sweep') ? '?sweep=1' : '';

50+

4351 let final = null;

4452 for (const flags of flagSets) {

45- const browser = await puppeteer.launch({ executablePath: CHROME, args: flags });

53+ const browser = await puppeteer.launch({

54+ executablePath: CHROME,

55+ // A copy: puppeteer splices --enable-features out of the array it is given

56+ // and re-adds it merged with its own, which would drop it from the

57+ // diagnostic below and make a failure look like it ran with fewer flags.

58+ args: [...flags],

59+ // Puppeteer's default is 180 s, and it bounds the CDP call that

60+ // waitForFunction polls inside — so without this the wait below silently

61+ // caps at 3 minutes no matter what timeout it is given, and a suite that

62+ // runs longer fails as 'Runtime.callFunctionOn timed out' with no results.

63+ protocolTimeout: 900_000,

64+ });

4665 try {

4766 const page = await browser.newPage();

4867 page.on('console', (msg) => console.log(` [page] ${msg.text()}`));

4968 page.on('pageerror', (err) => console.log(` [pageerror] ${err.message}`));

50- await page.goto(`http://127.0.0.1:${port}/test.html`, { waitUntil: 'load' });

69+ await page.goto(`http://127.0.0.1:${port}/test.html${query}`, { waitUntil: 'load' });

5170 const results = await page.waitForFunction(() => window.__RESULTS__, { timeout: 600_000 });

5271 final = await results.jsonValue();

5372 } catch (e) {

test/geometryChecks.tsmodified+55−5View file

@@ -67,11 +67,26 @@ async function buildGeometry(device: GPUDevice, key: string) {

6767 return { g, sht, deriv, cfg, geometry };

6868 }

6969

70+export interface GeometryCheckOptions {

71+ /**

72+ * Run the niter x geometry sweep at the end. On by default, and nearly free

73+ * on desktop Dawn (~3 s for all 20 combinations), but in a browser it

74+ * dominates the whole suite: every session recompiles its unrolled step from

75+ * scratch — there is no pipeline cache across sessions — so the sweep costs

76+ * ~6 minutes on software WebGPU against ~30 s for every other check here put

77+ * together. The browser page therefore leaves it out unless asked (?sweep=1),

78+ * which is what keeps CI short.

79+ */

80+ sweep?: boolean;

81+}

82+

7083 export async function geometryChecks(

7184 device: GPUDevice,

7285 check: Check,

7386 log: Log,

87+ opts: GeometryCheckOptions = {},

7488 ): Promise<void> {

89+ const runSweep = opts.sweep ?? true;

7590 // ---- every geometry compiles and closes ---------------------------------

7691 for (const spec of mGeometries) {

7792 const { sht, deriv, geometry } = await buildGeometry(device, spec.key);

@@ -139,7 +154,25 @@ export async function geometryChecks(

139154 // V_phi = (-sin(phi)/sin(theta), cos(phi)/sin(theta), 0). Checking these

140155 // pins the sign convention of computeMetric (src/geom/metric.ts) before

141156 // it is buried under the Laplace-Beltrami operator built on top of it.

157+ //

158+ // Split by latitude, because the accuracy available here is not uniform.

159+ // computeMetric divides by det = g_tt*g_pp - g_tp^2, and on the sphere

160+ // g_pp and det are both O(sin^2 theta) -- 1.4e-3 at the outermost Gauss

161+ // latitude of this grid. The transforms deliver X_theta/X_phi with an

162+ // *absolute* fp32 error of ~1e-6, which is a large *relative* error once

163+ // it is squared into quantities that small, so the error in both V's grows

164+ // like 1/sin^2 theta toward the poles. That is conditioning, not a wrong

165+ // formula: measured worst cases are

166+ //

167+ // sin(theta) >= 0.2 sin(theta) < 0.2 (8 of 64 latitudes)

168+ // Dawn 3.2e-4 1.7e-3

169+ // SwiftShader 1.5e-3 8.8e-3

170+ //

171+ // and a sign or formula error would be O(1) in either band, so tolerances

172+ // a few times the looser stack still catch one.

173+ const POLE_SIN = 0.2;

142174 let maxMetricErr = 0;

175+ let maxPoleErr = 0;

143176 for (let i = 0; i < sht.cfg.nlat; i++) {

144177 const ct = sht.cosTheta[i];

145178 const st = Math.sqrt(Math.max(0, 1 - ct * ct));

@@ -154,8 +187,7 @@ export async function geometryChecks(

154187 const wantVpx = -sphi / st;

155188 const wantVpy = cphi / st;

156189 const wantVpz = 0;

157- maxMetricErr = Math.max(

158- maxMetricErr,

190+ const worst = Math.max(

159191 Math.abs(geometry.Vtx[k] - wantVtx),

160192 Math.abs(geometry.Vty[k] - wantVty),

161193 Math.abs(geometry.Vtz[k] - wantVtz),

@@ -163,12 +195,20 @@ export async function geometryChecks(

163195 Math.abs(geometry.Vpy[k] - wantVpy),

164196 Math.abs(geometry.Vpz[k] - wantVpz),

165197 );

198+ if (st < POLE_SIN) maxPoleErr = Math.max(maxPoleErr, worst);

199+ else maxMetricErr = Math.max(maxMetricErr, worst);

166200 }

167201 }

168202 check(

169203 'geometry: sphere.m has the closed-form inverse metric quantities',

170- maxMetricErr < 2e-3,

171- `max |V - closed form| = ${maxMetricErr.toExponential(2)}`,

204+ maxMetricErr < 4e-3,

205+ `max |V - closed form| = ${maxMetricErr.toExponential(2)} ` +

206+ `away from the poles (sin theta >= ${POLE_SIN})`,

207+ );

208+ check(

209+ 'geometry: the polar caps stay within their conditioning',

210+ maxPoleErr < 2e-2,

211+ `max |V - closed form| = ${maxPoleErr.toExponential(2)} at sin theta < ${POLE_SIN}`,

172212 );

173213

174214 deriv.destroy();

@@ -348,7 +388,17 @@ export async function geometryChecks(

348388 // app's <select> actually offers, so a regression anywhere in that grid is

349389 // caught -- without asserting away the one combination already known to be

350390 // outside the convergence radius.

351- {

391+ //

392+ // SWEEP_LMAX cannot be lowered to make this cheaper: at lmax 31 or 15 the

393+ // peanut/2, /4 and /8 combinations below all stay finite, so a smaller grid

394+ // would quietly turn KNOWN_DIVERGENT into three failures and stop testing

395+ // the thing this sweep exists to pin down.

396+ if (!runSweep) {

397+ log(

398+ ' sweep: skipped — run `npm run test:node` (desktop Dawn, ~3 s) or ' +

399+ '`npm run test:gpu -- --sweep` for the niter x geometry sweep.',

400+ );

401+ } else {

352402 const model = mModelByKey('schnakenberg')!;

353403 const params = defaultParams(model);

354404 const SWEEP_NITER = [0, 1, 2, 4, 8];

test/test-page.tsmodified+3−1View file

@@ -215,7 +215,9 @@ async function main(): Promise<void> {

215215 await transformChecks(device, check, log);

216216 await analyticChecks(device, check, log);

217217 await modelChecks(device, check, log);

218- await geometryChecks(device, check, log);

218+ // The sweep is opt-in here (?sweep=1): it is a few seconds on desktop Dawn

219+ // but minutes in a browser, where each session recompiles its unrolled step.

220+ await geometryChecks(device, check, log, { sweep: q.has('sweep') });

219221

220222 window.__RESULTS__ = { ok: failures === 0, lines };

221223 log(failures === 0 ? 'ALL PASS' : `${failures} FAILURE(S)`);