compare-native: say when a row is measuring launches, not the GPU
On an RTX PRO 6000 at lmax 63, shtns cuda comes out at 0.021 ms/round trip
with 0.018 of that spent on the CPU queueing kernels — 86%. So that number
is roughly what it costs to submit a round trip on that host, not what the
GPU spends on one, and the real GPU time is below it and invisible to this
measurement. The WGSL row is 14% CPU on the same machine and genuinely
GPU-bound, so the 4.05x ratio between them understates the difference in
GPU work rather than reporting it.
Both shares were already printed; nothing said what they implied. Flag any
row above 50% and say how to read a ratio involving it.
1 changed file+18−0
scripts/compare-native.mjsmodified+18−0View file
@@ -421,6 +421,24 @@ if (wantJson) {
421421 );
422422 }
423423 }
424+ // A run whose CPU-side share is most of its wall time is not measuring the GPU
425+ // at all — it is measuring how long the host takes to queue the work. That is a
426+ // real cost, but it puts a floor under the number that has nothing to do with
427+ // the transform, and it means a ratio against it understates the gap in GPU
428+ // work. Say so rather than letting the headline ratio be read as compute.
429+ const LAUNCH_BOUND = 0.5;
430+ for (const r of good) {
431+ const share = (r.json.throughput.encodeMsPerStep ?? 0) / rate(r);
432+ if (share > LAUNCH_BOUND) {
433+ console.log(
434+ `\n NOTE ${r.label} spends ${(100 * share).toFixed(0)}% of its time on the CPU queueing\n` +
435+ ` work, so ${rate(r).toFixed(3)} ms is roughly what it costs to *submit* a round trip\n` +
436+ ` here, not what the GPU spends on one — the real GPU time is below that and\n` +
437+ ` this measurement cannot see it. Raise --lmax until the GPU dominates, or read\n` +
438+ ` any ratio involving this row as a lower bound on the difference in GPU work.`,
439+ );
440+ }
441+ }
424442 if (cuda && wg) {
425443 const ratio = rate(cuda) / rate(wg);
426444 console.log(