4f822e1turing-surface-cache: reaction-diffusion solutions at a chosen end time, shared through a cloud cacheJeremy Magland 1# turing-surface-cache
3Reaction-diffusion systems (Turing patterns) on curved closed surfaces,
4evaluated at a chosen end time, with the solutions shared between all visitors
5through a cloud cache.
7This is a trimmed fork of
8[turing-surface](https://github.com/concept-collection/turing-surface), which
9solves the same systems live and freely tunable. What this app changes is the
10contract: every setting is a choice from a short list, so each combination of
11choices names exactly one solution. Solutions already in the shared cache load
12by themselves as the choices are browsed; a combination nobody has computed
13shows empty surfaces, and nothing runs until the user presses **Compute
14solution**, which runs the solver locally (in the browser, on WebGPU),
15watching the pattern form and stopping at exactly the requested time. Users
16who hold an upload API key contribute their locally-computed solutions back,
17so the next visitor who asks for the same combination gets it in a second
18rather than a minute.
20## The discrete parameter space
481eeb9Add Brusselator and Allen-Cahn modelsJeremy Magland 22Three models ship, all in turing-surface's 6-transform flux form —
23Schnakenberg (the default), Brusselator, and Allen–Cahn — on three geometries
24(sphere, ellipsoid, peanut). The Algorithm-4 reference variant is deliberately
25absent: it solves the same equations as Schnakenberg and would only duplicate
26cache entries under different hashes. The choices, defined in
4f822e1turing-surface-cache: reaction-diffusion solutions at a chosen end time, shared through a cloud cacheJeremy Magland 27[`src/cache/options.ts`](src/cache/options.ts):
29| setting | choices |
30|---|---|
481eeb9Add Brusselator and Allen-Cahn modelsJeremy Magland 31| Schnakenberg | a: 0.05/**0.1**/0.15/0.2 · b: 0.7/**0.9**/1.1/1.3 · D₁: 1.6e-4/**4e-4**/1e-3 · D₂: 3.2e-3/**8e-3**/2e-2 · dt: 0.02/**0.05**/0.1 |
32| Brusselator | A: 2/**3**/4 · B: 7/**9**/11 · D₁: 1.7e-3/**3.33e-3**/6.7e-3 · D₂: 8.3e-3/**1.67e-2**/3.3e-2 · dt: 0.01/**0.02**/0.05 |
33| Allen–Cahn | ε²: 5e-4/**1e-3**/2e-3 · dt: 0.01/**0.02**/0.05 |
4f822e1turing-surface-cache: reaction-diffusion solutions at a chosen end time, shared through a cloud cacheJeremy Magland 34| geometry | sphere, **ellipsoid** (axes each 0.6/1/1.5), peanut (waist 0.4/0.6/0.8, stretch 0/0.6/1.2) |
35| seed | **1**–5 |
36| end time | **100**, 200, 400, 800, 1600 |
481eeb9Add Brusselator and Allen-Cahn modelsJeremy Magland 38The model, unlike every other choice, is compiled into the GPU session, so
39switching it pays a recompile of a second or two; everything else swaps into
40the running session. Allen–Cahn evolves one species, so it shows one panel
41where the others show two.
4f822e1turing-surface-cache: reaction-diffusion solutions at a chosen end time, shared through a cloud cacheJeremy Magland 43(Defaults in bold.) The numerical-scheme settings are fixed — lmax 63, 8
44solve iterations, seed wavelength λ = 0.5 — but are recorded in every cache
45key, so offering them as choices later invalidates nothing.
47The whole selection is mirrored into the URL fragment, every value written
48explicitly, so reloading returns to the same combination and a shared link
49opens on the same spec (and, when cached, the same solution) for whoever
50follows it. A "Reset to defaults" button puts every choice back.
52Every end time is an exact multiple of every dt, so a run to T = 800 passes
53exactly through t = 100, 200 and 400. Those intermediate states are captured
54as the run passes them and, for an uploading user, encoded and uploaded in
55the background while the run continues: one long run populates four cache
56entries, and the earlier ones are already shared before the run finishes. The same structure works in the other
57direction: the state is Markovian in the spectral coefficients, so before
58computing anything the app looks for the longest cached shorter run of the
59same spec and continues from its final state, computing only the remainder.
60Asking for T = 1600 when T = 800 is cached costs half the run, and the t = 0
61initial state travels inside every file of the chain, so a continuation
62writes files identical in kind to a from-scratch run.
64## How the cache works
66The page's whole state is one small spec object (model, parameters, geometry,
67seed, end time, scheme settings, plus the app name and a format version). Its
68canonical JSON — keys sorted at every level — is hashed with SHA-256, and the
69hash is the object name:
71```
72https://tempory.net/tmpbucket/turing-surface-cache/v1/schnakenberg/<sha256>.h5
73```
75A lookup is therefore a single GET with no index or API, and a 404 means a
76miss. The path carries the app name, format version and model in the clear so
77that future cleanup (lifecycle rules, prefix deletes) never has to open a file
78to know what it belongs to; the hash input includes the version string, so a
79format change moves every object rather than silently colliding with the old
80ones.
1a1e473Add auto-fill: work through the parameter space on an idle machineJeremy Magland 82## Filling the cache
84A cache only pays off once it holds what people ask for, and nobody wants to
85sit through the first computation of every combination. **Auto-fill the
86cache** — offered only when an upload API key is present — turns an otherwise
87idle machine into a contributor: it works through the parameter space,
88skipping whatever is already cached and computing and uploading the rest, and
89runs until stopped.
377e83eStop counting the model as a knobJeremy Magland 91Two decisions make that practical. The first is the order. About 8,200
92combinations exist (228 model-parameter sets × 36 geometries, with the seed
1a1e473Add auto-fill: work through the parameter space on an idle machineJeremy Magland 93and dt pinned), which is roughly three GPU-weeks at this repo's ~180 steps/s —
94exhaustible in principle, but only if the useful part comes first. Since a
95visitor starts at the defaults and changes one dropdown at a time, the chance
96that a combination is ever requested falls off steeply with the number of
97knobs that differ from the defaults, so the walk proceeds by that distance:
377e83eStop counting the model as a knobJeremy Magland 98every one-knob deviation before any two-knob one. The model is not counted
99among those knobs: somebody who came for Allen–Cahn starts at its defaults
100rather than at Schnakenberg's, so the three models are three origins rather
101than one origin and two deviations from it, and each is surrounded before any
102of them is explored far. One machine overnight covers every one- and two-knob
103deviation from every model's defaults, which is most of what anyone will ever
104click; the long tail can take as long as it likes.
106The second is that within a distance the order is **random**, and that is the
107entire coordination mechanism. Several idle browsers walking the same tiers in
108different orders, each skipping what it finds already cached, rarely duplicate
109each other and need no coordinator, no work queue, and no knowledge of one
110another. A skip costs one `HEAD` request, so a machine joining a
111well-filled region catches up in seconds.
113The seed and dt are pinned rather than surveyed (seed 1, dt 0.05): a seed
114picks a draw and means nothing on its own, and dt is a numerical knob rather
115than a property of the problem, so surveying either would multiply the work
116without adding a solution anyone asked for. Both are the default of every
117model, so an auto-filled entry is exactly what a visitor arriving at the
118defaults requests. Two smaller points: the walk skips the ellipsoid with all
119axes 1, since that *is* the unit sphere and the sphere geometry already
120covers it, and any run whose state goes non-finite is reported and discarded
121rather than uploaded — an unattended walk must not publish wreckage under a
122hash someone later trusts.
124Because it is meant to run unattended, the compute loop never waits on an
125animation frame and skips rendering entirely while the page is hidden, so a
126minimized window or a background tab keeps computing at full speed rather
127than being throttled to a crawl.
4f822e1turing-surface-cache: reaction-diffusion solutions at a chosen end time, shared through a cloud cacheJeremy Magland 129Uploads go through the [tmpbucket](https://github.com/scratchrealm/tmpbucket)
130Worker: the client presents the API key and a file name, receives a presigned
131R2 PUT URL, and uploads directly. Only holders of the key can write; everyone
132can read. The key is entered in the page and kept in localStorage.
c2317d0Fill the cache from the command line, without a browserJeremy Magland 134## Filling it from the command line
136A browser window is a poor place to leave a long computation, so the same walk
137runs outside one:
139```
140TURING_SURFACE_CACHE_KEY=… npx https://concept-collection.github.io/turing-surface-cache/fill.tgz
141```
143Nothing is published to the npm registry — npm installs a tarball from a URL
144as happily as from a package name, and the tarball is built and deployed
145beside the page, so the command line is always the same commit as the app.
146The page itself offers this command, ready to copy, once an upload key is
b32cc02Offer the fill command in the page, ready to copyJeremy Magland 147entered; the key is masked in what the page shows and real in what it copies,
148so that pasting it onto a fresh machine takes one step while a screenshot of
149the page still gives nothing away. The key can also be saved for later runs
150(`login` prompts for it and
c2317d0Fill the cache from the command line, without a browserJeremy Magland 151writes `~/.config/turing-surface-cache/key`), or passed as `--key`, though the
152environment is preferable: a key on the command line is visible to every user
153on the machine through `ps`, while another process's environment is not.
d3fa654Say what is wrong when node is too old, and run on node 18Jeremy Magland 155Node 18 or newer is required — the cache keys are SHA-256 through WebCrypto,
156which older node does not have as a global, and node 18 itself has it only
157under `node:crypto`, which is worth supporting since that is what several
158current distributions ship. An older node than that cannot even parse the
159bundle, and would otherwise report a syntax error pointing at a brace, so the
160published command is a small ES5 launcher that checks the version first and
161says what to do about it.
b0546a9Fix the command line's file writing, and how it gets updatedJeremy Magland 163One consequence of installing from a URL is worth knowing. npx keys its
164install directory on the whole spec string it was given, so a URL that never
165changes keeps running whatever it first installed, however often the file
166behind it has been replaced — and neither `--prefer-online` nor a changed
167version in the manifest makes any difference, since nothing remote is
168consulted once that directory exists. The command the page offers therefore
169carries the build it belongs to (`fill.tgz?v=<commit>`), which makes every
170deployment a new spec and so a fresh install. The bare URL above is right the
171first time and stale ever after; `--help` says which build is running.
c2317d0Fill the cache from the command line, without a browserJeremy Magland 173The walk, the runs and the uploads are the page's own — the same modules under
174[`src/cache/`](src/cache/), driven by console output instead of a status bar
175(see [`src/cli/fill.ts`](src/cli/fill.ts)). What differs is the WebGPU: node
176has none, so the command line brings its own, the optional `webgpu` package of
177prebuilt [Google Dawn](https://dawn.googlesource.com/dawn) binaries, installed
178under the globals the transform code expects. Dawn reaches the GPU through
179Vulkan on Linux and Windows and Metal on macOS, so a machine wanting to
180contribute needs a GPU and its driver — on a machine without one, Dawn
181either finds no adapter at all or falls back to a software rasterizer, which
182is roughly a thousand times slower and worth nothing to anybody. The command
183names its adapter on startup, reports its rate in steps per second, and says
184so plainly when either looks wrong; it does not refuse to run, since the
185judgment is the operator's.
d71391eFold the troubleshooting into the page, beside the commandJeremy Magland 187A machine can also be too old for the command in a way that has nothing to do
188with its GPU. Dawn's prebuilt binary wants glibc 2.34, which a long-lived
189Linux workstation may well not have — Rocky and RHEL 8 are on 2.28 — and the
190obvious remedy of running the command in a container turns out not to work:
191inside one the NVIDIA driver declines to bring up its Vulkan driver
192(`vk_icdNegotiateLoaderICDInterfaceVersion` returns
193`VK_ERROR_INITIALIZATION_FAILED`), while the same call on the host succeeds.
194What does work is to borrow only the userland from a container image and run
195node through its loader, on the host, leaving the GPU, `/dev` and `/proc`
196exactly as they were; the host's own `/usr/lib64` stays last on the library
197path, since the NVIDIA libraries and the Vulkan loader have to match the
7df2a28Borrow npx along with node, since the recipe needs bothJeremy Magland 198running kernel module. Note that `npx` must be borrowed along with `node`,
199being a script beside node rather than a part of it: a machine whose node is
200too old has an equally old npx, and a path carrying only the borrowed node
201still finds that one. The page carries that recipe, folded away beside the
d71391eFold the troubleshooting into the page, beside the commandJeremy Magland 202command it belongs to, along with what the other common failures mean —
203they are worth writing down where someone will meet them, since none of them
204is guessable from the error alone.
c2317d0Fill the cache from the command line, without a browserJeremy Magland 206Progress is a line per target and a rate that updates in place:
208```
209[2] schnakenberg a=0.15 b=0.9 D1=4e-4 D2=8e-3 dt=0.05 · sphere · 2 knobs from the defaults
210 computing to t = 1600 (32,000 steps)
211 t = 812.4 / 1600 51% 184 steps/s eta 1m11s uploaded 3/3
212 computed in 2m54s — uploaded 5 solutions (t = 100, 200, 400, 800, 1600)
213```
215When the output is not a terminal the same lines are written periodically
216instead of in place, so a `nohup`ed log stays readable. `--dry-run` lists the
217first targets and whether each is already cached, which is a cheap way to see
218what a machine would take on before committing it; `--limit` and `--model`
219narrow the work; and ctrl-C stops after the current run, so nothing in flight
220is lost.
e53205fShow one parameter across its range, not only at a pointJeremy Magland 222## Sweeping one parameter
224A second page, [`sweep.html`](sweep.html) (linked from the main one), shows
225how the solution changes across one parameter rather than at one point: every
226setting is fixed except a single model parameter, which runs over a list of
227values, and a knob steps the display through the range. On any selection
228change the page fetches every value's cache file at once (a sweep is a handful
229of files of about 90 KB) and synthesizes each solution onto the render mesh as
230it arrives, so that moving the knob afterwards touches neither the network nor
231the solver: it recolors the mesh from values already in memory.
232One color scale per species, computed over all loaded values and held fixed,
233covers the whole sweep, so what changes under the knob is the pattern rather
234than the palette. The URL fragment carries the selection, the swept parameter
235(`sweep=b`), its values (`values=0.7,0.9,1.1,1.3`) and the knob's position, so
236a shared link opens on the same sweep at the same place; the serialization is
237the main page's with two entries added (see
238[`src/cache/selection.ts`](src/cache/selection.ts)).
240The value list starts as the swept parameter's own choices, which is what the
241main page's dropdown offers and what the auto-fill walk surveys, but it is a
242text box, and any numbers may be typed in its place. This is the one control
243in the app that is not a choice from a list. It is nonetheless safe for the
244cache, since a typed number is parsed once and serialized in canonical
245shortest form ever after, so that it names one exact spec and one exact
246object just as a listed value does. Of course, a value the walk never
247surveyed will not already be there, so a sweep over typed values arrives
248entirely as gaps and has to be computed.
250Values nobody has computed show as gaps on the knob's track. **Compute
251missing values** runs them in the browser, one after another, each through the
252same local run as the main page, warm start and background snapshot uploads
253and divergence guard included. The command line fills a sweep on a machine
254with no browser on it:
256```
257TURING_SURFACE_CACHE_KEY=… npx …/fill.tgz sweep '<sweep page URL>'
258```
260The argument is the sweep page's own URL: its fragment already says which
261parameter sweeps, over which values, and what everything else is fixed to, so
262the copied command and the page it came from always name the same solutions,
263and pasting the same URL into a browser shows the result. The page offers this
264command ready to copy once an upload key is entered, the key masked on screen
265and real in the clipboard as before. `--dry-run` lists the sweep's values and
266whether each is cached. Note that unlike the auto-fill walk, a sweep honors
267the selection's seed and end time exactly; a run to the sweep's end time still
268uploads every shorter listed end time it passes, so filling a sweep at
269T = 1600 also fills the same sweep at every smaller T.
4f822e1turing-surface-cache: reaction-diffusion solutions at a chosen end time, shared through a cloud cacheJeremy Magland 271## The cache file
273Cache files are HDF5, written in the browser with
274[h5wasm](https://github.com/usnistgov/h5wasm) and readable from Python with
275h5py. The layout is turing-surface's reference-file layout (see
276`docs/ellipsoid-reference-spec.md` there) extended with the cache's identity
277at the root, so a cache file is *also* a valid reference file — it can be
278loaded straight into turing-surface's "Compare against uploaded data" mode:
280```
281/ attrs: app, format_version, spec_json, model, species,
282 created_utc, adapter
283├─ backend/ attrs: adapter, runtime, precision
284├─ spec/ attrs: geometry, lmax, seed, steps, niter, lam3, t_end
285│ ├─ params/ attrs: a, b, D1, D2, dt
286│ └─ geometry_params/ attrs: the geometry's params
287├─ grid/ attrs: lmax, mmax, nlat, nphi, nlm
288├─ geometry/ Gx, Gy, Gz float32[2·nlm]
289├─ initial/ U, V (spectral state at t = 0) float32[2·nlm]
290└─ final/ U, V (at the end time) float32[2·nlm]
291```
293`spec_json` is the exact string that was hashed into the object name, and the
294reader verifies it matches what was asked for. The initial state is included
295so a file fully defines its run; the coefficients are the spherical-harmonic
296convention documented in turing-surface (orthonormal + Condon-Shortley,
297m-major, [re, im] interleaved). At lmax 63 a file is about 90 KB.
299Note that the solver is deterministic given the spec only to fp32 round-off:
300different GPUs round differently, so a cached solution and a local recompute
301agree closely but not bit-for-bit. The cache stores whichever trusted user
302computed a combination first, and the file records which adapter that was.
304## Development
306```
307npm install
308npm run dev # local dev server
309npm run build # type-check + production build to dist/
c2317d0Fill the cache from the command line, without a browserJeremy Magland 310npm run build:cli # the command-line bundle, packed as dist/fill.tgz
4f822e1turing-surface-cache: reaction-diffusion solutions at a chosen end time, shared through a cloud cacheJeremy Magland 311```
c2317d0Fill the cache from the command line, without a browserJeremy Magland 313`npm run build` runs `build:cli` too, so a deployment carries both. The
314command-line bundle is an SSR vite build of
315[`src/cli/fill.ts`](src/cli/fill.ts) with everything under `src/` and numbl's
316compiler bundled in, exactly as the page's build has them; the only things
317left external are h5wasm, whose node build reads its wasm off disk, and Dawn,
318which is a native addon. [`scripts/pack-cli.mjs`](scripts/pack-cli.mjs) writes
319the published manifest, which therefore depends on neither numbl nor a
320checkout of anything.
4f822e1turing-surface-cache: reaction-diffusion solutions at a chosen end time, shared through a cloud cacheJeremy Magland 322numbl is a local `file:../../numbl` dependency, exactly as in turing-surface —
323a sibling checkout of [numbl](https://github.com/flatironinstitute/numbl) is
324required, reached through the `numbl-src` alias in
325[`vite.config.ts`](vite.config.ts). See turing-surface's README for the
326details; nothing about the arrangement changed here.
328Checks:
330- `node scripts/check-app.mjs` — end-to-end in headless Chrome (SwiftShader
331 WebGPU) with the cloud cache mocked: a miss computes locally and produces
e53205fShow one parameter across its range, not only at a pointJeremy Magland 332 the .h5 (verified with h5py), a fresh page loads that .h5 as a hit, a
333 third page asking for a longer end time warm-starts from it, and the sweep
334 page loads that .h5 as one value of a dt sweep, shows the other two as
335 gaps, and fills them with Compute missing values. The pages use the
336 `?tend=` query hook, which substitutes short test end times for the UI's
337 list. This is what CI runs.
4f822e1turing-surface-cache: reaction-diffusion solutions at a chosen end time, shared through a cloud cacheJeremy Magland 338- `node scripts/check-live.mjs [url]` — smoke-check a deployed URL against
e53205fShow one parameter across its range, not only at a pointJeremy Magland 339 the real cache, both pages.
340- `node scripts/screenshot.mjs out.png [light|dark] [tEnd] [index|sweep]` —
341 screenshot after the boot-time solve.
4f822e1turing-surface-cache: reaction-diffusion solutions at a chosen end time, shared through a cloud cacheJeremy Magland 342
343Deployed to GitHub Pages by `.github/workflows/deploy.yml` on push to `main`.
345## License
347CECILL-2.1 (inherited from SHTNS via shtns-webgpu, whose sources are
348vendored under `src/sht/`).