Blame README.md at 377e83e2f351534c34a4c33cad17e804e52a14fd - concept-collection/turing-surface-cache

concept-collection / turing-surface-cache

4f822e1 turing-surface-cache: reaction-diffusion solutions at a chosen end time, shared through a cloud cacheJeremy Magland 9 days ago1# turing-surface-cache

3Reaction-diffusion systems (Turing patterns) on curved closed surfaces,

4evaluated at a chosen end time, with the solutions shared between all visitors

5through a cloud cache.

7This is a trimmed fork of

8[turing-surface](https://github.com/concept-collection/turing-surface), which

9solves the same systems live and freely tunable. What this app changes is the

10contract: every setting is a choice from a short list, so each combination of

11choices names exactly one solution. Solutions already in the shared cache load

12by themselves as the choices are browsed; a combination nobody has computed

13shows empty surfaces, and nothing runs until the user presses **Compute

14solution**, which runs the solver locally (in the browser, on WebGPU),

15watching the pattern form and stopping at exactly the requested time. Users

16who hold an upload API key contribute their locally-computed solutions back,

17so the next visitor who asks for the same combination gets it in a second

18rather than a minute.

20## The discrete parameter space

481eeb9 Add Brusselator and Allen-Cahn modelsJeremy Magland 9 days ago22Three models ship, all in turing-surface's 6-transform flux form —

23Schnakenberg (the default), Brusselator, and Allen–Cahn — on three geometries

24(sphere, ellipsoid, peanut). The Algorithm-4 reference variant is deliberately

25absent: it solves the same equations as Schnakenberg and would only duplicate

26cache entries under different hashes. The choices, defined in

4f822e1 turing-surface-cache: reaction-diffusion solutions at a chosen end time, shared through a cloud cacheJeremy Magland 9 days ago27[`src/cache/options.ts`](src/cache/options.ts):

29| setting | choices |

30|---|---|

481eeb9 Add Brusselator and Allen-Cahn modelsJeremy Magland 9 days ago31| Schnakenberg | a: 0.05/**0.1**/0.15/0.2 · b: 0.7/**0.9**/1.1/1.3 · D₁: 1.6e-4/**4e-4**/1e-3 · D₂: 3.2e-3/**8e-3**/2e-2 · dt: 0.02/**0.05**/0.1 |

32| Brusselator | A: 2/**3**/4 · B: 7/**9**/11 · D₁: 1.7e-3/**3.33e-3**/6.7e-3 · D₂: 8.3e-3/**1.67e-2**/3.3e-2 · dt: 0.01/**0.02**/0.05 |

33| Allen–Cahn | ε²: 5e-4/**1e-3**/2e-3 · dt: 0.01/**0.02**/0.05 |

4f822e1 turing-surface-cache: reaction-diffusion solutions at a chosen end time, shared through a cloud cacheJeremy Magland 9 days ago34| geometry | sphere, **ellipsoid** (axes each 0.6/1/1.5), peanut (waist 0.4/0.6/0.8, stretch 0/0.6/1.2) |

35| seed | **1**–5 |

36| end time | **100**, 200, 400, 800, 1600 |

481eeb9 Add Brusselator and Allen-Cahn modelsJeremy Magland 9 days ago38The model, unlike every other choice, is compiled into the GPU session, so

39switching it pays a recompile of a second or two; everything else swaps into

40the running session. Allen–Cahn evolves one species, so it shows one panel

41where the others show two.

4f822e1 turing-surface-cache: reaction-diffusion solutions at a chosen end time, shared through a cloud cacheJeremy Magland 9 days ago43(Defaults in bold.) The numerical-scheme settings are fixed — lmax 63, 8

44solve iterations, seed wavelength λ = 0.5 — but are recorded in every cache

45key, so offering them as choices later invalidates nothing.

47The whole selection is mirrored into the URL fragment, every value written

48explicitly, so reloading returns to the same combination and a shared link

49opens on the same spec (and, when cached, the same solution) for whoever

50follows it. A "Reset to defaults" button puts every choice back.

52Every end time is an exact multiple of every dt, so a run to T = 800 passes

53exactly through t = 100, 200 and 400. Those intermediate states are captured

54as the run passes them and, for an uploading user, encoded and uploaded in

55the background while the run continues: one long run populates four cache

56entries, and the earlier ones are already shared before the run finishes. The same structure works in the other

57direction: the state is Markovian in the spectral coefficients, so before

58computing anything the app looks for the longest cached shorter run of the

59same spec and continues from its final state, computing only the remainder.

60Asking for T = 1600 when T = 800 is cached costs half the run, and the t = 0

61initial state travels inside every file of the chain, so a continuation

62writes files identical in kind to a from-scratch run.

64## How the cache works

66The page's whole state is one small spec object (model, parameters, geometry,

67seed, end time, scheme settings, plus the app name and a format version). Its

68canonical JSON — keys sorted at every level — is hashed with SHA-256, and the

69hash is the object name:

71```

72https://tempory.net/tmpbucket/turing-surface-cache/v1/schnakenberg/<sha256>.h5

73```

75A lookup is therefore a single GET with no index or API, and a 404 means a

76miss. The path carries the app name, format version and model in the clear so

77that future cleanup (lifecycle rules, prefix deletes) never has to open a file

78to know what it belongs to; the hash input includes the version string, so a

79format change moves every object rather than silently colliding with the old

80ones.

1a1e473 Add auto-fill: work through the parameter space on an idle machineJeremy Magland 8 days ago82## Filling the cache

84A cache only pays off once it holds what people ask for, and nobody wants to

85sit through the first computation of every combination. **Auto-fill the

86cache** — offered only when an upload API key is present — turns an otherwise

87idle machine into a contributor: it works through the parameter space,

88skipping whatever is already cached and computing and uploading the rest, and

89runs until stopped.

377e83e Stop counting the model as a knobJeremy Magland 8 days ago91Two decisions make that practical. The first is the order. About 8,200

92combinations exist (228 model-parameter sets × 36 geometries, with the seed

1a1e473 Add auto-fill: work through the parameter space on an idle machineJeremy Magland 8 days ago93and dt pinned), which is roughly three GPU-weeks at this repo's ~180 steps/s —

94exhaustible in principle, but only if the useful part comes first. Since a

95visitor starts at the defaults and changes one dropdown at a time, the chance

96that a combination is ever requested falls off steeply with the number of

97knobs that differ from the defaults, so the walk proceeds by that distance:

377e83e Stop counting the model as a knobJeremy Magland 8 days ago98every one-knob deviation before any two-knob one. The model is not counted

99among those knobs: somebody who came for Allen–Cahn starts at its defaults

100rather than at Schnakenberg's, so the three models are three origins rather

101than one origin and two deviations from it, and each is surrounded before any

102of them is explored far. One machine overnight covers every one- and two-knob

103deviation from every model's defaults, which is most of what anyone will ever

104click; the long tail can take as long as it likes.

1a1e473 Add auto-fill: work through the parameter space on an idle machineJeremy Magland 8 days ago105

106The second is that within a distance the order is **random**, and that is the

107entire coordination mechanism. Several idle browsers walking the same tiers in

108different orders, each skipping what it finds already cached, rarely duplicate

109each other and need no coordinator, no work queue, and no knowledge of one

110another. A skip costs one `HEAD` request, so a machine joining a

111well-filled region catches up in seconds.

112

113The seed and dt are pinned rather than surveyed (seed 1, dt 0.05): a seed

114picks a draw and means nothing on its own, and dt is a numerical knob rather

115than a property of the problem, so surveying either would multiply the work

116without adding a solution anyone asked for. Both are the default of every

117model, so an auto-filled entry is exactly what a visitor arriving at the

118defaults requests. Two smaller points: the walk skips the ellipsoid with all

119axes 1, since that *is* the unit sphere and the sphere geometry already

120covers it, and any run whose state goes non-finite is reported and discarded

121rather than uploaded — an unattended walk must not publish wreckage under a

122hash someone later trusts.

123

124Because it is meant to run unattended, the compute loop never waits on an

125animation frame and skips rendering entirely while the page is hidden, so a

126minimized window or a background tab keeps computing at full speed rather

127than being throttled to a crawl.

128

4f822e1 turing-surface-cache: reaction-diffusion solutions at a chosen end time, shared through a cloud cacheJeremy Magland 9 days ago129Uploads go through the [tmpbucket](https://github.com/scratchrealm/tmpbucket)

130Worker: the client presents the API key and a file name, receives a presigned

131R2 PUT URL, and uploads directly. Only holders of the key can write; everyone

132can read. The key is entered in the page and kept in localStorage.

133

c2317d0 Fill the cache from the command line, without a browserJeremy Magland 8 days ago134## Filling it from the command line

135

136A browser window is a poor place to leave a long computation, so the same walk

137runs outside one:

138

139```

140TURING_SURFACE_CACHE_KEY=… npx https://concept-collection.github.io/turing-surface-cache/fill.tgz

141```

142

143Nothing is published to the npm registry — npm installs a tarball from a URL

144as happily as from a package name, and the tarball is built and deployed

145beside the page, so the command line is always the same commit as the app.

146The page itself offers this command, ready to copy, once an upload key is

b32cc02 Offer the fill command in the page, ready to copyJeremy Magland 8 days ago147entered; the key is masked in what the page shows and real in what it copies,

148so that pasting it onto a fresh machine takes one step while a screenshot of

149the page still gives nothing away. The key can also be saved for later runs

150(`login` prompts for it and

c2317d0 Fill the cache from the command line, without a browserJeremy Magland 8 days ago151writes `~/.config/turing-surface-cache/key`), or passed as `--key`, though the

152environment is preferable: a key on the command line is visible to every user

153on the machine through `ps`, while another process's environment is not.

154

d3fa654 Say what is wrong when node is too old, and run on node 18Jeremy Magland 8 days ago155Node 18 or newer is required — the cache keys are SHA-256 through WebCrypto,

156which older node does not have as a global, and node 18 itself has it only

157under `node:crypto`, which is worth supporting since that is what several

158current distributions ship. An older node than that cannot even parse the

159bundle, and would otherwise report a syntax error pointing at a brace, so the

160published command is a small ES5 launcher that checks the version first and

161says what to do about it.

162

b0546a9 Fix the command line's file writing, and how it gets updatedJeremy Magland 8 days ago163One consequence of installing from a URL is worth knowing. npx keys its

164install directory on the whole spec string it was given, so a URL that never

165changes keeps running whatever it first installed, however often the file

166behind it has been replaced — and neither `--prefer-online` nor a changed

167version in the manifest makes any difference, since nothing remote is

168consulted once that directory exists. The command the page offers therefore

169carries the build it belongs to (`fill.tgz?v=<commit>`), which makes every

170deployment a new spec and so a fresh install. The bare URL above is right the

171first time and stale ever after; `--help` says which build is running.

172

c2317d0 Fill the cache from the command line, without a browserJeremy Magland 8 days ago173The walk, the runs and the uploads are the page's own — the same modules under

174[`src/cache/`](src/cache/), driven by console output instead of a status bar

175(see [`src/cli/fill.ts`](src/cli/fill.ts)). What differs is the WebGPU: node

176has none, so the command line brings its own, the optional `webgpu` package of

177prebuilt [Google Dawn](https://dawn.googlesource.com/dawn) binaries, installed

178under the globals the transform code expects. Dawn reaches the GPU through

179Vulkan on Linux and Windows and Metal on macOS, so a machine wanting to

180contribute needs a GPU and its driver — on a machine without one, Dawn

181either finds no adapter at all or falls back to a software rasterizer, which

182is roughly a thousand times slower and worth nothing to anybody. The command

183names its adapter on startup, reports its rate in steps per second, and says

184so plainly when either looks wrong; it does not refuse to run, since the

185judgment is the operator's.

186

d71391e Fold the troubleshooting into the page, beside the commandJeremy Magland 8 days ago187A machine can also be too old for the command in a way that has nothing to do

188with its GPU. Dawn's prebuilt binary wants glibc 2.34, which a long-lived

189Linux workstation may well not have — Rocky and RHEL 8 are on 2.28 — and the

190obvious remedy of running the command in a container turns out not to work:

191inside one the NVIDIA driver declines to bring up its Vulkan driver

192(`vk_icdNegotiateLoaderICDInterfaceVersion` returns

193`VK_ERROR_INITIALIZATION_FAILED`), while the same call on the host succeeds.

194What does work is to borrow only the userland from a container image and run

195node through its loader, on the host, leaving the GPU, `/dev` and `/proc`

196exactly as they were; the host's own `/usr/lib64` stays last on the library

197path, since the NVIDIA libraries and the Vulkan loader have to match the

198running kernel module. The page carries that recipe, folded away beside the

199command it belongs to, along with what the other common failures mean —