1# turing-surface-cache
3Reaction-diffusion systems (Turing patterns) on curved closed surfaces,
4evaluated at a chosen end time, with the solutions shared between all visitors
5through a cloud cache.
7This is a trimmed fork of
8[turing-surface](https://github.com/concept-collection/turing-surface), which
9solves the same systems live and freely tunable. What this app changes is the
10contract: every setting is a choice from a short list, so each combination of
11choices names exactly one solution. Solutions already in the shared cache load
12by themselves as the choices are browsed; a combination nobody has computed
13shows empty surfaces, and nothing runs until the user presses **Compute
14solution**, which runs the solver locally (in the browser, on WebGPU),
15watching the pattern form and stopping at exactly the requested time. Users
16who hold an upload API key contribute their locally-computed solutions back,
17so the next visitor who asks for the same combination gets it in a second
18rather than a minute.
20## The discrete parameter space
22Three models ship, all in turing-surface's 6-transform flux form —
23Schnakenberg (the default), Brusselator, and Allen–Cahn — on three geometries
24(sphere, ellipsoid, peanut). The Algorithm-4 reference variant is deliberately
25absent: it solves the same equations as Schnakenberg and would only duplicate
26cache entries under different hashes. The choices, defined in
27[`src/cache/options.ts`](src/cache/options.ts):
29| setting | choices |
30|---|---|
31| Schnakenberg | a: 0.05/**0.1**/0.15/0.2 · b: 0.7/**0.9**/1.1/1.3 · D₁: 1.6e-4/**4e-4**/1e-3 · D₂: 3.2e-3/**8e-3**/2e-2 · dt: 0.02/**0.05**/0.1 |
32| Brusselator | A: 2/**3**/4 · B: 7/**9**/11 · D₁: 1.7e-3/**3.33e-3**/6.7e-3 · D₂: 8.3e-3/**1.67e-2**/3.3e-2 · dt: 0.01/**0.02**/0.05 |
33| Allen–Cahn | ε²: 5e-4/**1e-3**/2e-3 · dt: 0.01/**0.02**/0.05 |
34| geometry | sphere, **ellipsoid** (axes each 0.6/1/1.5), peanut (waist 0.4/0.6/0.8, stretch 0/0.6/1.2) |
35| seed | **1**–5 |
36| end time | **100**, 200, 400, 800, 1600 |
38The model, unlike every other choice, is compiled into the GPU session, so
39switching it pays a recompile of a second or two; everything else swaps into
40the running session. Allen–Cahn evolves one species, so it shows one panel
41where the others show two.
43(Defaults in bold.) The numerical-scheme settings are fixed — lmax 63, 8
44solve iterations, seed wavelength λ = 0.5 — but are recorded in every cache
45key, so offering them as choices later invalidates nothing.
47The whole selection is mirrored into the URL fragment, every value written
48explicitly, so reloading returns to the same combination and a shared link
49opens on the same spec (and, when cached, the same solution) for whoever
50follows it. A "Reset to defaults" button puts every choice back.
52Every end time is an exact multiple of every dt, so a run to T = 800 passes
53exactly through t = 100, 200 and 400. Those intermediate states are captured
54as the run passes them and, for an uploading user, encoded and uploaded in
55the background while the run continues: one long run populates four cache
56entries, and the earlier ones are already shared before the run finishes. The same structure works in the other
57direction: the state is Markovian in the spectral coefficients, so before
58computing anything the app looks for the longest cached shorter run of the
59same spec and continues from its final state, computing only the remainder.
60Asking for T = 1600 when T = 800 is cached costs half the run, and the t = 0
61initial state travels inside every file of the chain, so a continuation
62writes files identical in kind to a from-scratch run.
64## How the cache works
66The page's whole state is one small spec object (model, parameters, geometry,
67seed, end time, scheme settings, plus the app name and a format version). Its
68canonical JSON — keys sorted at every level — is hashed with SHA-256, and the
69hash is the object name:
71```
72https://tempory.net/tmpbucket/turing-surface-cache/v1/schnakenberg/<sha256>.h5
73```
75A lookup is therefore a single GET with no index or API, and a 404 means a
76miss. The path carries the app name, format version and model in the clear so
77that future cleanup (lifecycle rules, prefix deletes) never has to open a file
78to know what it belongs to; the hash input includes the version string, so a
79format change moves every object rather than silently colliding with the old
80ones.
82## Filling the cache
84A cache only pays off once it holds what people ask for, and nobody wants to
85sit through the first computation of every combination. **Auto-fill the
86cache** — offered only when an upload API key is present — turns an otherwise
87idle machine into a contributor: it works through the parameter space,
88skipping whatever is already cached and computing and uploading the rest, and
89runs until stopped.
91Two decisions make that practical. The first is the order. About 8,400
92combinations exist (228 model-parameter sets × 37 geometries, with the seed
93and dt pinned), which is roughly three GPU-weeks at this repo's ~180 steps/s —
94exhaustible in principle, but only if the useful part comes first. Since a
95visitor starts at the defaults and changes one dropdown at a time, the chance
96that a combination is ever requested falls off steeply with the number of
97knobs that differ from the defaults, so the walk proceeds by that distance:
98every one-knob deviation before any two-knob one. One machine overnight covers
99every one- and two-knob deviation from every model's defaults, which is most
100of what anyone will ever click; the long tail can take as long as it likes.
102The second is that within a distance the order is **random**, and that is the
103entire coordination mechanism. Several idle browsers walking the same tiers in
104different orders, each skipping what it finds already cached, rarely duplicate
105each other and need no coordinator, no work queue, and no knowledge of one
106another. A skip costs one `HEAD` request, so a machine joining a
107well-filled region catches up in seconds.
109The seed and dt are pinned rather than surveyed (seed 1, dt 0.05): a seed
110picks a draw and means nothing on its own, and dt is a numerical knob rather
111than a property of the problem, so surveying either would multiply the work
112without adding a solution anyone asked for. Both are the default of every
113model, so an auto-filled entry is exactly what a visitor arriving at the
114defaults requests. Two smaller points: the walk skips the ellipsoid with all
115axes 1, since that *is* the unit sphere and the sphere geometry already
116covers it, and any run whose state goes non-finite is reported and discarded
117rather than uploaded — an unattended walk must not publish wreckage under a
118hash someone later trusts.
120Because it is meant to run unattended, the compute loop never waits on an
121animation frame and skips rendering entirely while the page is hidden, so a
122minimized window or a background tab keeps computing at full speed rather
123than being throttled to a crawl.
125Uploads go through the [tmpbucket](https://github.com/scratchrealm/tmpbucket)
126Worker: the client presents the API key and a file name, receives a presigned
127R2 PUT URL, and uploads directly. Only holders of the key can write; everyone
128can read. The key is entered in the page and kept in localStorage.
130## Filling it from the command line
132A browser window is a poor place to leave a long computation, so the same walk
133runs outside one:
135```
136TURING_SURFACE_CACHE_KEY=… npx https://concept-collection.github.io/turing-surface-cache/fill.tgz
137```
139Nothing is published to the npm registry — npm installs a tarball from a URL
140as happily as from a package name, and the tarball is built and deployed
141beside the page, so the command line is always the same commit as the app.
142The page itself offers this command, ready to copy, once an upload key is
143entered. The key can also be saved for later runs (`login` prompts for it and
144writes `~/.config/turing-surface-cache/key`), or passed as `--key`, though the
145environment is preferable: a key on the command line is visible to every user
146on the machine through `ps`, while another process's environment is not.
148The walk, the runs and the uploads are the page's own — the same modules under
149[`src/cache/`](src/cache/), driven by console output instead of a status bar
150(see [`src/cli/fill.ts`](src/cli/fill.ts)). What differs is the WebGPU: node
151has none, so the command line brings its own, the optional `webgpu` package of
152prebuilt [Google Dawn](https://dawn.googlesource.com/dawn) binaries, installed
153under the globals the transform code expects. Dawn reaches the GPU through
154Vulkan on Linux and Windows and Metal on macOS, so a machine wanting to
155contribute needs a GPU and its driver — on a machine without one, Dawn
156either finds no adapter at all or falls back to a software rasterizer, which
157is roughly a thousand times slower and worth nothing to anybody. The command
158names its adapter on startup, reports its rate in steps per second, and says
159so plainly when either looks wrong; it does not refuse to run, since the
160judgment is the operator's.
162Progress is a line per target and a rate that updates in place:
164```
165[2] schnakenberg a=0.15 b=0.9 D1=4e-4 D2=8e-3 dt=0.05 · sphere · 2 knobs from the defaults
166 computing to t = 1600 (32,000 steps)
167 t = 812.4 / 1600 51% 184 steps/s eta 1m11s uploaded 3/3
168 computed in 2m54s — uploaded 5 solutions (t = 100, 200, 400, 800, 1600)
169```
171When the output is not a terminal the same lines are written periodically
172instead of in place, so a `nohup`ed log stays readable. `--dry-run` lists the
173first targets and whether each is already cached, which is a cheap way to see
174what a machine would take on before committing it; `--limit` and `--model`
175narrow the work; and ctrl-C stops after the current run, so nothing in flight
176is lost.
178## The cache file
180Cache files are HDF5, written in the browser with
181[h5wasm](https://github.com/usnistgov/h5wasm) and readable from Python with
182h5py. The layout is turing-surface's reference-file layout (see
183`docs/ellipsoid-reference-spec.md` there) extended with the cache's identity
184at the root, so a cache file is *also* a valid reference file — it can be
185loaded straight into turing-surface's "Compare against uploaded data" mode:
187```
188/ attrs: app, format_version, spec_json, model, species,
189 created_utc, adapter
190├─ backend/ attrs: adapter, runtime, precision
191├─ spec/ attrs: geometry, lmax, seed, steps, niter, lam3, t_end
192│ ├─ params/ attrs: a, b, D1, D2, dt
193│ └─ geometry_params/ attrs: the geometry's params
194├─ grid/ attrs: lmax, mmax, nlat, nphi, nlm
195├─ geometry/ Gx, Gy, Gz float32[2·nlm]
196├─ initial/ U, V (spectral state at t = 0) float32[2·nlm]
197└─ final/ U, V (at the end time) float32[2·nlm]
198```
200`spec_json` is the exact string that was hashed into the object name, and the
201reader verifies it matches what was asked for. The initial state is included
202so a file fully defines its run; the coefficients are the spherical-harmonic
203convention documented in turing-surface (orthonormal + Condon-Shortley,
204m-major, [re, im] interleaved). At lmax 63 a file is about 90 KB.
206Note that the solver is deterministic given the spec only to fp32 round-off:
207different GPUs round differently, so a cached solution and a local recompute
208agree closely but not bit-for-bit. The cache stores whichever trusted user
209computed a combination first, and the file records which adapter that was.
211## Development
213```
214npm install
215npm run dev # local dev server
216npm run build # type-check + production build to dist/
217npm run build:cli # the command-line bundle, packed as dist/fill.tgz
218```
220`npm run build` runs `build:cli` too, so a deployment carries both. The
221command-line bundle is an SSR vite build of
222[`src/cli/fill.ts`](src/cli/fill.ts) with everything under `src/` and numbl's
223compiler bundled in, exactly as the page's build has them; the only things
224left external are h5wasm, whose node build reads its wasm off disk, and Dawn,
225which is a native addon. [`scripts/pack-cli.mjs`](scripts/pack-cli.mjs) writes
226the published manifest, which therefore depends on neither numbl nor a
227checkout of anything.
229numbl is a local `file:../../numbl` dependency, exactly as in turing-surface —
230a sibling checkout of [numbl](https://github.com/flatironinstitute/numbl) is
231required, reached through the `numbl-src` alias in
232[`vite.config.ts`](vite.config.ts). See turing-surface's README for the
233details; nothing about the arrangement changed here.
235Checks:
237- `node scripts/check-app.mjs` — end-to-end in headless Chrome (SwiftShader
238 WebGPU) with the cloud cache mocked: a miss computes locally and produces
239 the .h5 (verified with h5py), a fresh page loads that .h5 as a hit, and a
240 third page asking for a longer end time warm-starts from it. The pages use
241 the `?tend=` query hook, which substitutes short test end times for the
242 UI's list. This is what CI runs.
243- `node scripts/check-live.mjs [url]` — smoke-check a deployed URL against
244 the real cache.
245- `node scripts/screenshot.mjs out.png [light|dark] [tEnd]` — screenshot
246 after the boot-time solve.
248Deployed to GitHub Pages by `.github/workflows/deploy.yml` on push to `main`.
250## License
252CECILL-2.1 (inherited from SHTNS via shtns-webgpu, whose sources are
253vendored under `src/sht/`).