Replace instrawave with simulated instruments driven by manifests
The instruments in ../simulated_instruments (bass, recorder, triangle,
cymbal, maraca, woodblock) are MATLAB physical models executed by numbl;
each now carries an instrument.json manifest naming its entry function,
fixed overrides, pitch/velocity/duration mapping, preview tier, and
curated knobs. midiboot discovers every manifest at build time, renders
each distinct note once in a Web Worker (velocity quantized to eight
levels, LRU note cache), and mixes per track; playback uses the fast
tier and WAV export the full tier. Manifest knobs appear as sliders on
the active track row, stored per track. The instrawave adapter, its
GPU smoke test, and the webgpu deps are removed; npm run test:sim runs
the real models through numbl in node.
22 changed files+2494−291
README.mdmodified+7−5View file
@@ -1,6 +1,6 @@
11 # midiboot
22
3-An in-browser application for recording, importing, editing, and playing MIDI. A limitation of most browser MIDI tools is that the instrument is fixed: notes go into a General MIDI soundfont or a hard-coded synth, and there is no place to put your own rendering logic. Here we describe a small workstation built around one extension point, an instrument interface that turns note events into audio, so that the built-in synths can later be joined by custom renderers (a GPU physical model in the style of [instrawave](../instrawave), a sampler, anything) without changing the editor, the transport, or the file handling around them.
3+An in-browser application for recording, importing, editing, and playing MIDI. A limitation of most browser MIDI tools is that the instrument is fixed: notes go into a General MIDI soundfont or a hard-coded synth, and there is no place to put your own rendering logic. Here we describe a small workstation built around one extension point, an instrument interface that turns note events into audio, so that the built-in synths sit alongside custom renderers without changing the editor, the transport, or the file handling around them. The first such renderers are the physically based models of the [simulated_instruments](../../simulated_instruments) collection, executed client-side by [numbl](https://numbl.org).
44
55 ## The instrument interface
66
@@ -18,7 +18,9 @@ interface Instrument {
1818
1919 There are two paths. The offline path, `render`, receives every note of a track with times already converted to seconds and produces one mono buffer; it may be async and arbitrarily expensive, and playback, mixing, and WAV export all go through it. The live path, `noteOn`, is optional and exists for low-latency monitoring while recording and for previewing notes from the piano roll; an instrument without it still works everywhere, with a plain beep standing in for monitoring. New instruments are added with `registerInstrument` in [src/instruments/registry.ts](src/instruments/registry.ts); tracks refer to instruments by id, so a song whose instrument is missing falls back to a builtin rather than failing.
2020
21-Alongside the builtins there is a placeholder adapter for the three [instrawave](../instrawave) instruments (drum, dulcimer, chime), registered when the browser has WebGPU. Each appears at its default geometry with deliberately reduced accuracy so the first bake takes seconds rather than minutes; the bake is held in memory for the session, and every note becomes one stimulus with the pitch parameter overridden to the note's frequency and the excitation scaled by velocity, against that one shared bake. We emphasize that this is scaffolding: instrawave is itself a prototype, the adapter lives entirely in [src/instruments/instrawave/](src/instruments/instrawave/), and it will be swapped for a successor without touching the rest of the application. On a machine without WebGPU the adapter registers nothing and a song bound to these instruments falls back to a builtin.
21+Alongside the builtins are the simulated instruments: each repository in ../simulated_instruments (standup bass, recorder, triangle, cymbal, maraca, woodblock) is a physically based model written in MATLAB syntax and executed by numbl, and each declares how it may be played in an `instrument.json` manifest committed next to its sources. The manifest names the entry function (`y = entry(P)` for one note), the defaults function, fixed parameter overrides (a single-event gesture, a fixed seed, normalization off so velocity survives in pascals), how MIDI pitch reaches a parameter (directly for the bass's `root_midi`, through a nearest-entry fingering table for the recorder, or not at all for the unpitched percussion), how velocity 0..1 maps linearly onto an excitation range, optionally how note duration reaches a parameter (the recorder's `note_dur_s`; the others treat notes as onsets), fast-tier overrides (`oversample: 1`), and a curated list of knobs. Crafting a new instrument for midiboot is therefore writing a model and its manifest in a new sibling repository; midiboot discovers every manifest at build time and nothing in the application is instrument-specific. The schema's one definition, with validation, lives in [src/instruments/simulated/manifest.ts](src/instruments/simulated/manifest.ts).
22+
23+The models render in a Web Worker so a seconds-long solve never blocks the editor. Playback renders each distinct note once (velocity quantized to eight levels, duration to quarter seconds), caches it in an LRU bounded by total samples, places copies at the onsets, and sums; a repeated drum pattern therefore costs one render. The knobs a manifest declares appear as sliders on the track (the active row), stored per track and invalidating that track's render when moved. The trade-off of per-note rendering is the loss of inter-note physics: the bass's choke of a note when the next lands, and a cymbal's energy build-up across a roll, are outside this arrangement, and a model whose expensive stage does not depend on the note (the cymbal's eigensolve) repeats it per distinct velocity rather than caching it across notes.
2224
2325 The four builtin instruments (Karplus-Strong pluck, FM electric piano, a filtered square lead, and a General MIDI drum kit) are constructed from pure per-note DSP functions in [src/instruments/dsp.ts](src/instruments/dsp.ts). These are deterministic, with noise drawn from a PRNG seeded by pitch, so the node test harness checks the actual audio. The `instrumentFromNoteDsp` helper derives both paths from one such function; a custom instrument is free to ignore it and implement the interface directly.
2426
@@ -45,11 +47,11 @@ npm install
4547 npm run dev # vite dev server
4648 npm run build # typecheck and bundle
4749 npm test # node-side tests (no browser, no audio device)
48-npm run test:gpu # instrawave adapter smoke test on desktop WebGPU (Dawn)
50+npm run test:sim # renders real notes through numbl in node; slower
4951 ```
5052
51-The tests cover the MIDI writer/parser round trip, tempo map arithmetic, quantize and transpose, mixing rules, and the determinism and boundedness of the DSP.
53+The tests cover the MIDI writer/parser round trip, tempo map arithmetic, quantize and transpose, mixing rules, the determinism and boundedness of the DSP, and validation of every instrument manifest in the sibling collection. `npm run test:sim` additionally executes the actual models through numbl in node and checks the audio: finiteness, decay, determinism under the fixed seed, and that velocity and knobs genuinely change the sound.
5254
5355 ## Limitations and future work
5456
55-The mixdown is mono, matching the mono instrument interface; a stereo render path is a straightforward extension. There is no loop region, no controller or pitch bend data, and editing the tempo of a song that already has a tempo map is not supported (the bpm field edits only single-tempo songs rather than silently flattening the map). The live path plays each note as a pre-rendered buffer with a gain release, which is fine for monitoring but is not a substitute for a real streaming voice. The instrawave adapter is a placeholder: geometry, tuning, and accuracy are fixed, bakes are not persisted, and there is no live monitoring voice, so recording against an instrawave track monitors through a plain beep. Making instrument instances editable and persisting their bakes in IndexedDB are where this goes next, most likely against instrawave's successor rather than instrawave itself.
57+The mixdown is mono, matching the mono instrument interface; a stereo render path is a straightforward extension. There is no loop region, no controller or pitch bend data, and editing the tempo of a song that already has a tempo map is not supported (the bpm field edits only single-tempo songs rather than silently flattening the map). The live path plays each note as a pre-rendered buffer with a gain release, which is fine for monitoring but is not a substitute for a real streaming voice. The simulated instruments have no live monitoring voice, so recording against one monitors through a plain beep; a small pre-rendered voice bank per instrument is the natural fix. Note caches live in worker memory and are redone per page load; persisting them in IndexedDB, keyed the way the render cache already is, is a straightforward extension. Manifest knobs are the only editable parameters; the full parameter surfaces stay in each instrument's own web app, which remains the place to design a sound before curating its knobs here.
package-lock.jsonmodified+1756−64View file
This diff is 2,043 lines long and is not shown.
package.jsonmodified+3−4View file
@@ -11,12 +11,12 @@
1111 "dev": "vite",
1212 "build": "tsc --noEmit && vite build",
1313 "test": "vite-node scripts/test-node.ts",
14- "test:gpu": "vite-node scripts/test-gpu.ts"
14+ "test:sim": "vite-node scripts/test-sim.ts"
1515 },
1616 "dependencies": {
1717 "react": "^18.3.1",
1818 "react-dom": "^18.3.1",
19- "instrawave": "file:../instrawave"
19+ "numbl": "^0.4.19"
2020 },
2121 "devDependencies": {
2222 "@types/node": "^26.1.1",
@@ -25,7 +25,6 @@
2525 "@vitejs/plugin-react": "^4.3.1",
2626 "typescript": "^5.5.0",
2727 "vite": "^5.4.0",
28- "vite-node": "^6.0.0",
29- "@webgpu/types": "^0.1.44"
28+ "vite-node": "^6.0.0"
3029 }
3130 }
scripts/test-gpu.tsdeleted+0−72View file
@@ -1,72 +0,0 @@
1-/**
2- * GPU smoke test for the placeholder instrawave adapter, on desktop
3- * WebGPU (Google Dawn) through instrawave's node shim: bake the drum once
4- * at the placeholder accuracy, render two notes through the midiboot
5- * Instrument interface, and check the result is shaped like audio. Run
6- * with `npm run test:gpu`; exits 0 with a note when no GPU is available.
7- */
8-import { installWebGpu, errMsg } from 'instrawave/scripts/nodeWebGpu.ts';
9-
10-try {
11- console.log(await installWebGpu());
12-} catch (e) {
13- console.log(`skipped: ${errMsg(e)}`);
14- process.exit(0);
15-}
16-
17-const { createInstrawaveInstruments } = await import('../src/instruments/instrawave/adapter.ts');
18-
19-let failures = 0;
20-function check(cond: boolean, msg: string): void {
21- console.log(`${cond ? 'ok ' : 'FAIL'} ${msg}`);
22- if (!cond) failures++;
23-}
24-
25-const drum = createInstrawaveInstruments().find((i) => i.id === 'iw-drum')!;
26-const SR = 44100;
27-const len = Math.round(SR * 2.0);
28-const t0 = performance.now();
29-const out = (await drum.render({
30- notes: [
31- { timeSec: 0.1, durationSec: 0.25, pitch: 48, velocity: 0.9 },
32- { timeSec: 0.9, durationSec: 0.25, pitch: 55, velocity: 0.6 },
33- ],
34- sampleRate: SR,
35- lengthSamples: len,
36-})) as Float32Array;
37-console.log(`bake + render in ${((performance.now() - t0) / 1000).toFixed(1)} s`);
38-
39-check(out.length === len, `trace length ${out.length} = ${len}`);
40-let finite = true;
41-for (let i = 0; i < out.length; i++) if (!Number.isFinite(out[i])) { finite = false; break; }
42-check(finite, 'all samples finite');
43-const seg = (a: number, b: number) => {
44- let p = 0;
45- for (let i = Math.round(a * SR); i < Math.round(b * SR); i++) p = Math.max(p, Math.abs(out[i]));
46- return p;
47-};
48-// The raw trace is at mic-pressure scale (peaks around 1e-5); the app
49-// normalizes per track, so the checks here are relative to the peak.
50-const peakAll = seg(0, 2.0);
51-check(peakAll > 1e-7, `trace is not silent (peak ${peakAll.toExponential(2)})`);
52-check(seg(0, 0.09) < peakAll * 1e-3, 'silent before the first strike');
53-check(seg(0.12, 0.8) > peakAll * 0.1, 'first strike sounds');
54-check(seg(0.92, 1.8) > peakAll * 0.05, 'second strike sounds');
55-
56-// The render is deterministic against the same bake.
57-const again = (await drum.render({
58- notes: [{ timeSec: 0.1, durationSec: 0.25, pitch: 48, velocity: 0.9 }],
59- sampleRate: SR,
60- lengthSamples: SR,
61-})) as Float32Array;
62-const again2 = (await drum.render({
63- notes: [{ timeSec: 0.1, durationSec: 0.25, pitch: 48, velocity: 0.9 }],
64- sampleRate: SR,
65- lengthSamples: SR,
66-})) as Float32Array;
67-let same = again.length === again2.length;
68-if (same) for (let i = 0; i < again.length; i++) if (again[i] !== again2[i]) { same = false; break; }
69-check(same, 'repeat render is bit-identical');
70-
71-console.log(failures === 0 ? 'GPU smoke test passed' : `${failures} failure(s)`);
72-process.exit(failures > 0 ? 1 : 0);
scripts/test-node.tsmodified+67−0View file
@@ -9,6 +9,9 @@ import { quantizeNotes, transposeNotes } from '../src/midi/ops.ts';
99 import { drumNote, epianoNote, leadNote, pluckNote } from '../src/instruments/dsp.ts';
1010 import { builtinInstruments } from '../src/instruments/builtins.ts';
1111 import { audibleTracks, encodeWavMono16, mixTracks, trackRenderKey } from '../src/audio/render.ts';
12+import { bucketVelocity, mapPitch, mapVelocity, parseManifest } from '../src/instruments/simulated/manifest.ts';
13+import { readdirSync, readFileSync, existsSync } from 'node:fs';
14+import { join } from 'node:path';
1215 import type { Note, Song, Track } from '../src/types.ts';
1316
1417 let failed = 0;
@@ -249,5 +252,69 @@ test('WAV encoding writes a well-formed header', () => {
249252 assert.equal(dv.getInt16(44 + 6, true), 32767); // clipped to full scale
250253 });
251254
255+// --- simulated-instrument manifests ---
256+
257+const SIM_ROOT = join(import.meta.dirname, '../../../simulated_instruments');
258+
259+test('every instrument.json in ../simulated_instruments validates', () => {
260+ if (!existsSync(SIM_ROOT)) return; // vault layout absent; nothing to check
261+ const dirs = readdirSync(SIM_ROOT).filter((d) => existsSync(join(SIM_ROOT, d, 'instrument.json')));
262+ assert.ok(dirs.length >= 1, 'no manifests found');
263+ for (const d of dirs) {
264+ const m = parseManifest(readFileSync(join(SIM_ROOT, d, 'instrument.json'), 'utf8'));
265+ assert.equal(m.name, d, `manifest name '${m.name}' should match its directory '${d}'`);
266+ assert.ok(existsSync(join(SIM_ROOT, d, 'matlab', `${m.entry}.m`)), `${d}: entry ${m.entry}.m missing`);
267+ assert.ok(existsSync(join(SIM_ROOT, d, 'matlab', `${m.defaults}.m`)), `${d}: defaults ${m.defaults}.m missing`);
268+ }
269+});
270+
271+test('manifest pitch and velocity mapping', () => {
272+ const m = parseManifest(JSON.stringify({
273+ format: 'simulated-instrument-v1', name: 't', label: 'T', entry: 'e', defaults: 'd',
274+ fixed: {}, preview: {},
275+ pitch: { param: 'root_midi', mode: 'midi', min: 28, max: 55 },
276+ velocity: { param: 'force', min: 2, max: 12 },
277+ knobs: [],
278+ }));
279+ assert.deepEqual(mapPitch(m, 40), { key: 'root_midi', value: 40 });
280+ assert.deepEqual(mapPitch(m, 10), { key: 'root_midi', value: 28 }); // clamped
281+ assert.deepEqual(mapVelocity(m, 0), { key: 'force', value: 2 });
282+ assert.deepEqual(mapVelocity(m, 1), { key: 'force', value: 12 });
283+ const t = parseManifest(JSON.stringify({
284+ format: 'simulated-instrument-v1', name: 't2', label: 'T', entry: 'e', defaults: 'd',
285+ fixed: {}, preview: {},
286+ pitch: { param: 'note1', mode: 'table', table: [{ midi: 72, value: 1 }, { midi: 74, value: 2 }] },
287+ velocity: { param: 'v', min: 0, max: 1 },
288+ knobs: [],
289+ }));
290+ assert.deepEqual(mapPitch(t, 73), { key: 'note1', value: 1 }); // nearest, ties go low
291+ assert.deepEqual(mapPitch(t, 90), { key: 'note1', value: 2 });
292+ const un = parseManifest(JSON.stringify({
293+ format: 'simulated-instrument-v1', name: 't3', label: 'T', entry: 'e', defaults: 'd',
294+ fixed: {}, preview: {}, velocity: { param: 'v', min: 0, max: 1 }, knobs: [],
295+ }));
296+ assert.equal(mapPitch(un, 60), null);
297+});
298+
299+test('velocity bucketing quantizes into few levels', () => {
300+ assert.equal(bucketVelocity(1), 1);
301+ assert.equal(bucketVelocity(0.999), 1);
302+ assert.equal(bucketVelocity(0.51), 0.625);
303+ assert.equal(bucketVelocity(0.01), 0.125);
304+ assert.ok(bucketVelocity(0) > 0);
305+});
306+
307+test('manifest validation rejects malformed knobs', () => {
308+ assert.throws(() => parseManifest(JSON.stringify({
309+ format: 'simulated-instrument-v1', name: 'x', label: 'X', entry: 'e', defaults: 'd',
310+ fixed: {}, preview: {}, velocity: { param: 'v', min: 0, max: 1 },
311+ knobs: [{ key: 'k', label: 'K', min: 5, max: 1, default: 2 }],
312+ })));
313+ assert.throws(() => parseManifest(JSON.stringify({
314+ format: 'wrong', name: 'x', label: 'X', entry: 'e', defaults: 'd',
315+ fixed: {}, preview: {}, velocity: { param: 'v', min: 0, max: 1 }, knobs: [],
316+ })));
317+});
318+
252319 console.log(`\n${passed} passed, ${failed} failed`);
253320 process.exit(failed > 0 ? 1 : 0);
scripts/test-sim.tsadded+82−0View file
@@ -0,0 +1,82 @@
1+/**
2+ * Integration test for the simulated-instrument path: real manifests, real
3+ * MATLAB sources, real numbl execution in node (no browser, no worker).
4+ * Renders a woodblock strike and a bass pluck at the preview tier and
5+ * checks the audio is shaped right, including that velocity actually
6+ * changes the result. Run with `npm run test:sim`; slower than npm test
7+ * because it runs the physics.
8+ */
9+import { readdirSync, readFileSync, existsSync } from 'node:fs';
10+import { join } from 'node:path';
11+import { parseManifest, type SimManifest } from '../src/instruments/simulated/manifest.ts';
12+import { renderSimNote, type MFile } from '../src/instruments/simulated/mrender.ts';
13+
14+const ROOT = join(import.meta.dirname, '../../../simulated_instruments');
15+
16+function loadDef(name: string): { manifest: SimManifest; mfiles: MFile[] } {
17+ const manifest = parseManifest(readFileSync(join(ROOT, name, 'instrument.json'), 'utf8'));
18+ const mdir = join(ROOT, name, 'matlab');
19+ const mfiles = readdirSync(mdir)
20+ .filter((f) => f.endsWith('.m'))
21+ .map((f) => ({ name: f, source: readFileSync(join(mdir, f), 'utf8') }));
22+ return { manifest, mfiles };
23+}
24+
25+let failures = 0;
26+function check(cond: boolean, msg: string): void {
27+ console.log(`${cond ? 'ok ' : 'FAIL'} ${msg}`);
28+ if (!cond) failures++;
29+}
30+
31+function peak(x: Float32Array, from = 0, to = x.length): number {
32+ let p = 0;
33+ for (let i = from; i < Math.min(to, x.length); i++) p = Math.max(p, Math.abs(x[i]));
34+ return p;
35+}
36+
37+function finite(x: Float32Array): boolean {
38+ for (let i = 0; i < x.length; i++) if (!Number.isFinite(x[i])) return false;
39+ return true;
40+}
41+
42+if (!existsSync(ROOT)) {
43+ console.log(`skipped: ${ROOT} not found`);
44+ process.exit(0);
45+}
46+
47+const SR = 44100;
48+
49+{
50+ const { manifest, mfiles } = loadDef('woodblock');
51+ const t0 = performance.now();
52+ const soft = renderSimNote(manifest, mfiles, { pitch: 60, velocity: 0.25, durationSec: 0.25, sampleRate: SR, quality: 'preview', settings: {} });
53+ const hard = renderSimNote(manifest, mfiles, { pitch: 60, velocity: 1.0, durationSec: 0.25, sampleRate: SR, quality: 'preview', settings: {} });
54+ console.log(`woodblock: two strikes in ${((performance.now() - t0) / 1000).toFixed(1)} s`);
55+ check(soft.length > SR * 0.2, `woodblock trace has audio length (${soft.length})`);
56+ check(finite(soft) && finite(hard), 'woodblock samples finite');
57+ check(peak(soft) > 0, 'woodblock soft strike sounds');
58+ check(peak(hard) > peak(soft) * 1.5, `harder mallet is louder (${peak(hard).toExponential(2)} vs ${peak(soft).toExponential(2)})`);
59+ const moved = renderSimNote(manifest, mfiles, { pitch: 60, velocity: 1.0, durationSec: 0.25, sampleRate: SR, quality: 'preview', settings: { strike_position: 0.25 } });
60+ let differs = moved.length !== hard.length;
61+ for (let i = 0; i < Math.min(moved.length, hard.length) && !differs; i++) if (moved[i] !== hard[i]) differs = true;
62+ check(differs, 'strike position knob changes the sound');
63+}
64+
65+{
66+ const { manifest, mfiles } = loadDef('bass');
67+ const t0 = performance.now();
68+ const y = renderSimNote(manifest, mfiles, { pitch: 33, velocity: 0.7, durationSec: 0.5, sampleRate: SR, quality: 'preview', settings: { tail_s: 2.0 } });
69+ console.log(`bass: one pluck in ${((performance.now() - t0) / 1000).toFixed(1)} s`);
70+ check(finite(y), 'bass samples finite');
71+ check(peak(y) > 0, 'bass pluck sounds');
72+ check(Math.abs(y.length - SR * 2.0) < SR * 0.2, `bass trace spans the tail (${y.length} samples)`);
73+ check(peak(y, 0, Math.round(SR * 0.5)) > peak(y, Math.round(SR * 1.5)), 'bass pluck decays');
74+ // Determinism: fixed seed, same request, same samples.
75+ const y2 = renderSimNote(manifest, mfiles, { pitch: 33, velocity: 0.7, durationSec: 0.5, sampleRate: SR, quality: 'preview', settings: { tail_s: 2.0 } });
76+ let same = y.length === y2.length;
77+ for (let i = 0; i < y.length && same; i++) if (y[i] !== y2[i]) same = false;
78+ check(same, 'bass render is deterministic');
79+}
80+
81+console.log(failures === 0 ? 'simulated-instrument test passed' : `${failures} failure(s)`);
82+process.exit(failures > 0 ? 1 : 0);
src/audio/player.tsmodified+2−2View file
@@ -77,7 +77,7 @@ class Player {
7777 const key = trackRenderKey(song, track) + '@' + sampleRate;
7878 const cached = this.cache.get(track.id);
7979 if (cached && cached.key === key) continue;
80- const buffer = await renderTrackBuffer(song, track, getInstrument(track.instrumentId), sampleRate);
80+ const buffer = await renderTrackBuffer(song, track, getInstrument(track.instrumentId), sampleRate, 'preview');
8181 this.cache.set(track.id, { key, buffer });
8282 }
8383 }
@@ -214,7 +214,7 @@ class Player {
214214 async exportWav(song: Song): Promise<Uint8Array> {
215215 const buffers = new Map<string, Float32Array>();
216216 for (const track of song.tracks) {
217- buffers.set(track.id, await renderTrackBuffer(song, track, getInstrument(track.instrumentId), EXPORT_SAMPLE_RATE));
217+ buffers.set(track.id, await renderTrackBuffer(song, track, getInstrument(track.instrumentId), EXPORT_SAMPLE_RATE, 'full'));
218218 }
219219 const mix = mixTracks(song.tracks, buffers);
220220 return encodeWavMono16(mix, EXPORT_SAMPLE_RATE);
src/audio/render.tsmodified+3−3View file
@@ -21,10 +21,10 @@ export function noteSpecsForTrack(song: Song, track: Track): { notes: NoteSpec[]
2121 return { notes, lengthSec: lengthSec + RENDER_TAIL_SEC };
2222 }
2323
24-export async function renderTrackBuffer(song: Song, track: Track, instrument: Instrument, sampleRate: number): Promise<Float32Array> {
24+export async function renderTrackBuffer(song: Song, track: Track, instrument: Instrument, sampleRate: number, quality: 'preview' | 'full' = 'preview'): Promise<Float32Array> {
2525 const { notes, lengthSec } = noteSpecsForTrack(song, track);
2626 const lengthSamples = Math.max(1, Math.ceil(lengthSec * sampleRate));
27- const buf = await instrument.render({ notes, sampleRate, lengthSamples });
27+ const buf = await instrument.render({ notes, sampleRate, lengthSamples, quality, settings: track.settings ?? {} });
2828 // Level each track to a common peak so mixing is predictable.
2929 let peak = 0;
3030 for (let i = 0; i < buf.length; i++) peak = Math.max(peak, Math.abs(buf[i]));
@@ -71,7 +71,7 @@ export function mixTracks(tracks: Track[], buffers: Map<string, Float32Array>):
7171
7272 export function trackRenderKey(song: Song, track: Track): string {
7373 // FNV-1a over everything that affects the rendered audio.
74- const parts = [String(song.ppq), track.instrumentId, JSON.stringify(song.tempo)];
74+ const parts = [String(song.ppq), track.instrumentId, JSON.stringify(song.tempo), JSON.stringify(track.settings ?? {})];
7575 for (const n of track.notes) parts.push(`${n.tick},${n.duration},${n.pitch},${n.velocity}`);
7676 let h = 0x811c9dc5;
7777 const s = parts.join(';');
src/components/TrackList.tsxmodified+44−1View file
@@ -1,7 +1,38 @@
11 import { store, useApp } from '../state/store.ts';
2-import { listInstruments } from '../instruments/registry.ts';
2+import { getInstrument, listInstruments } from '../instruments/registry.ts';
3+import type { KnobSpec } from '../instruments/types.ts';
34 import { trackColor } from './colors.ts';
45
6+function fmtKnob(v: number): string {
7+ const a = Math.abs(v);
8+ if (a >= 1e5 || (a > 0 && a < 0.01)) return v.toExponential(1);
9+ return String(Number(v.toPrecision(3)));
10+}
11+
12+function KnobSlider({ knob, value, onChange }: { knob: KnobSpec; value: number | undefined; onChange: (v: number) => void }) {
13+ const cur = value ?? knob.default;
14+ const log = knob.scale === 'log';
15+ const pos = log ? (1000 * Math.log(cur / knob.min)) / Math.log(knob.max / knob.min) : cur;
16+ return (
17+ <label className="knob" title={`${knob.label} (default ${fmtKnob(knob.default)})`}>
18+ <span className="knoblabel">{knob.label}</span>
19+ <input
20+ type="range"
21+ min={log ? 0 : knob.min}
22+ max={log ? 1000 : knob.max}
23+ step={log ? 1 : (knob.step ?? (knob.max - knob.min) / 200)}
24+ value={pos}
25+ onChange={(e) => {
26+ const raw = Number(e.target.value);
27+ onChange(log ? knob.min * Math.pow(knob.max / knob.min, raw / 1000) : raw);
28+ }}
29+ onClick={(e) => e.stopPropagation()}
30+ />
31+ <span className="knobvalue">{fmtKnob(cur)}</span>
32+ </label>
33+ );
34+}
35+
536 export function TrackList() {
637 const app = useApp();
738 const song = app.rec?.song ?? null;
@@ -56,6 +87,18 @@ export function TrackList() {
5687 onClick={(e) => { e.stopPropagation(); store.setArmedTrack(armed ? null : t.id); }}
5788 >R</button>
5889 </div>
90+ {active && (getInstrument(t.instrumentId).knobs ?? []).length > 0 && (
91+ <div className="trackrow-knobs">
92+ {(getInstrument(t.instrumentId).knobs ?? []).map((k) => (
93+ <KnobSlider
94+ key={k.key}
95+ knob={k}
96+ value={t.settings?.[k.key]}
97+ onChange={(v) => store.setTrackSetting(t.id, k.key, v)}
98+ />
99+ ))}
100+ </div>
101+ )}
59102 <div className="trackrow-vol">
60103 <input
61104 type="range"
src/instruments/instrawave/adapter.tsdeleted+0−130View file
@@ -1,130 +0,0 @@
1-/**
2- * Placeholder instrawave adapter: the three bundled instrawave instruments
3- * (drum, dulcimer, chime) exposed through midiboot's Instrument interface,
4- * each at its default geometry with reduced accuracy so the first bake is
5- * quick rather than faithful. Instrawave is itself a prototype and this
6- * seam will be swapped for its successor; nothing outside this directory
7- * knows instrawave exists, so the swap is confined here.
8- *
9- * What the placeholder deliberately leaves out: instance editing (geometry
10- * and tuning are fixed at the file defaults), bake persistence (bakes live
11- * in memory and are redone per page load), and a live noteOn voice
12- * (monitoring falls back to the player's beep). Notes are treated as pure
13- * onsets, the way a struck drum works: each note becomes one stimulus with
14- * the pitch parameter overridden to the note's frequency (clamped to the
15- * parameter's range) and the excitation strength scaled linearly by
16- * velocity, against one shared bake.
17- */
18-import { requestDevice } from 'instrawave/src/host/device.ts';
19-import { InstrumentSession } from 'instrawave/src/host/session.ts';
20-import type { Bake } from 'instrawave/src/host/bake.ts';
21-import { parseInstrument, paramValues, type InstrumentFile, type StimulusEvent } from 'instrawave/src/format.ts';
22-import drumJson from 'instrawave/instruments/drum.instrument.json';
23-import dulcimerJson from 'instrawave/instruments/dulcimer.instrument.json';
24-import chimeJson from 'instrawave/instruments/chime.instrument.json';
25-import type { Instrument, RenderRequest } from '../types.ts';
26-import { registerInstrument } from '../registry.ts';
27-import { midiToFreq } from '../dsp.ts';
28-
29-/** Coarse grid, short room response: a bake in seconds, not minutes. */
30-const PLACEHOLDER_ACCURACY: Record<string, number> = { nx: 64, ir_seconds: 0.4 };
31-
32-type StatusFn = (msg: string) => void;
33-let onStatus: StatusFn = () => {};
34-export function setInstrawaveStatus(cb: StatusFn): void {
35- onStatus = cb;
36-}
37-
38-let devicePromise: Promise<GPUDevice> | null = null;
39-function ensureDevice(): Promise<GPUDevice> {
40- return (devicePromise ??= requestDevice());
41-}
42-
43-function resampleLinear(x: Float32Array, srFrom: number, srTo: number, outLen: number): Float32Array {
44- const out = new Float32Array(outLen);
45- if (x.length === 0) return out;
46- for (let i = 0; i < outLen; i++) {
47- const s = (i * srFrom) / srTo;
48- const k = Math.floor(s);
49- if (k >= x.length - 1) break;
50- const f = s - k;
51- out[i] = (1 - f) * x[k] + f * x[k + 1];
52- }
53- return out;
54-}
55-
56-class InstrawaveInstrument implements Instrument {
57- readonly values: Record<string, number>;
58- private readonly pitchKey: string;
59- private readonly strengthKey: string;
60- private bake: Bake | null = null;
61- private queue: Promise<unknown> = Promise.resolve();
62-
63- constructor(
64- readonly id: string,
65- readonly name: string,
66- readonly file: InstrumentFile,
67- ) {
68- this.values = paramValues(file, PLACEHOLDER_ACCURACY);
69- this.pitchKey = file.render.eventParams.includes('f0') ? 'f0' : 'f1';
70- this.strengthKey = file.render.eventParams.includes('vel') ? 'vel' : 'amp';
71- }
72-
73- render(req: RenderRequest): Promise<Float32Array> {
74- // One GPU render at a time per instrument; concurrent tracks queue.
75- const run = this.queue.then(() => this.renderNow(req));
76- this.queue = run.catch(() => undefined);
77- return run;
78- }
79-
80- private async renderNow(req: RenderRequest): Promise<Float32Array> {
81- if (req.notes.length === 0) return new Float32Array(req.lengthSamples);
82- const device = await ensureDevice();
83- const session = new InstrumentSession(device, this.file, this.values);
84- if (!this.bake) {
85- try {
86- this.bake = await session.precompute((done, total) =>
87- onStatus(`baking ${this.name}: ${done}/${total}`),
88- );
89- } finally {
90- onStatus('');
91- }
92- }
93- const spec = this.file.params.find((p) => p.key === this.pitchKey)!;
94- let clamped = 0;
95- const events: StimulusEvent[] = req.notes.map((n) => {
96- const f = midiToFreq(n.pitch);
97- const v = Math.max(spec.min, Math.min(spec.max, f));
98- if (v !== f) clamped++;
99- return { t: n.timeSec, set: { [this.pitchKey]: v, [this.strengthKey]: this.values[this.strengthKey] * n.velocity } };
100- });
101- if (clamped > 0) console.warn(`${this.name}: ${clamped} note(s) clamped to the ${spec.min}..${spec.max} Hz pitch range`);
102- const seconds = req.lengthSamples / req.sampleRate;
103- let trace: Float32Array;
104- let dt: number;
105- try {
106- const r = await session.render(this.bake, seconds, events, [], (done, total) =>
107- onStatus(`rendering ${this.name}: ${done}/${total}`),
108- );
109- trace = r.trace;
110- dt = r.dt;
111- } finally {
112- onStatus('');
113- }
114- return resampleLinear(trace, 1 / dt, req.sampleRate, req.lengthSamples);
115- }
116-}
117-
118-export function createInstrawaveInstruments(): Instrument[] {
119- const files = [drumJson, dulcimerJson, chimeJson].map((j) => parseInstrument(JSON.stringify(j)));
120- return files.map((f) => new InstrawaveInstrument(`iw-${f.name}`, `Instrawave ${f.name} (GPU)`, f));
121-}
122-
123-/** Registers the placeholder instruments when WebGPU is available;
124- * returns whether it did. Without WebGPU nothing is registered and a song
125- * bound to these ids falls back to a builtin. */
126-export function registerInstrawaveInstruments(): boolean {
127- if (typeof navigator === 'undefined' || !navigator.gpu) return false;
128- for (const ins of createInstrawaveInstruments()) registerInstrument(ins);
129- return true;
130-}
src/instruments/simulated/adapter.tsadded+88−0View file
@@ -0,0 +1,88 @@
1+/**
2+ * Simulated instruments (../simulated_instruments) behind the generic
3+ * Instrument interface: physically based MATLAB models executed by numbl
4+ * in a worker, one render per distinct note, mixed per track. What each
5+ * instrument exposes (entry point, pitch and velocity mapping, curated
6+ * knobs, preview overrides) comes from its own instrument.json manifest;
7+ * nothing here is instrument-specific. There is no live noteOn voice yet,
8+ * so monitoring while recording falls back to the player's beep.
9+ */
10+import type { Instrument, RenderRequest } from '../types.ts';
11+import { registerInstrument } from '../registry.ts';
12+import { loadCatalog } from './catalog.ts';
13+import type { SimManifest } from './manifest.ts';
14+import type { WorkerRenderMsg, WorkerResponse } from './worker.ts';
15+
16+type StatusFn = (msg: string) => void;
17+let onStatus: StatusFn = () => {};
18+export function setSimulatedStatus(cb: StatusFn): void {
19+ onStatus = cb;
20+}
21+
22+let worker: Worker | null = null;
23+let nextId = 1;
24+const pending = new Map<number, { resolve: (v: Float32Array) => void; reject: (e: Error) => void }>();
25+
26+function ensureWorker(): Worker {
27+ if (worker) return worker;
28+ worker = new Worker(new URL('./worker.ts', import.meta.url), { type: 'module' });
29+ worker.onmessage = (ev: MessageEvent<WorkerResponse>) => {
30+ const msg = ev.data;
31+ if (msg.kind === 'status') {
32+ onStatus(msg.text);
33+ return;
34+ }
35+ const p = pending.get(msg.id);
36+ if (!p) return;
37+ pending.delete(msg.id);
38+ if (msg.kind === 'result') p.resolve(msg.samples);
39+ else p.reject(new Error(msg.message));
40+ };
41+ worker.onerror = (ev) => {
42+ const err = new Error(`simulated-instrument worker failed: ${ev.message}`);
43+ for (const p of pending.values()) p.reject(err);
44+ pending.clear();
45+ };
46+ return worker;
47+}
48+
49+class SimInstrument implements Instrument {
50+ readonly id: string;
51+ readonly name: string;
52+ readonly knobs;
53+
54+ constructor(private readonly manifest: SimManifest) {
55+ this.id = `sim-${manifest.name}`;
56+ this.name = `${manifest.label} (sim)`;
57+ this.knobs = manifest.knobs;
58+ }
59+
60+ render(req: RenderRequest): Promise<Float32Array> {
61+ if (req.notes.length === 0) return Promise.resolve(new Float32Array(req.lengthSamples));
62+ const w = ensureWorker();
63+ const id = nextId++;
64+ const msg: WorkerRenderMsg = {
65+ kind: 'render',
66+ id,
67+ name: this.manifest.name,
68+ notes: req.notes.map((n) => ({ pitch: n.pitch, velocity: n.velocity, durationSec: n.durationSec, timeSec: n.timeSec })),
69+ sampleRate: req.sampleRate,
70+ lengthSamples: req.lengthSamples,
71+ quality: req.quality ?? 'preview',
72+ settings: req.settings ?? {},
73+ };
74+ return new Promise<Float32Array>((resolve, reject) => {
75+ pending.set(id, { resolve, reject });
76+ w.postMessage(msg);
77+ });
78+ }
79+}
80+
81+export function registerSimulatedInstruments(): number {
82+ let n = 0;
83+ for (const def of loadCatalog()) {
84+ registerInstrument(new SimInstrument(def.manifest));
85+ n++;
86+ }
87+ return n;
88+}
src/instruments/simulated/catalog.tsadded+51−0View file
@@ -0,0 +1,51 @@
1+import { parseManifest, type SimManifest } from './manifest.ts';
2+import type { MFile } from './mrender.ts';
3+
4+/**
5+ * Discovery: every ../simulated_instruments/<repo>/instrument.json, with
6+ * that repo's matlab/*.m sources, bundled at build time. A repo without a
7+ * manifest is simply not an instrument; a manifest that fails validation
8+ * is reported and skipped rather than taking the app down.
9+ */
10+
11+const manifestModules = import.meta.glob('../../../../../simulated_instruments/*/instrument.json', {
12+ eager: true,
13+ query: '?raw',
14+ import: 'default',
15+}) as Record<string, string>;
16+
17+const mModules = import.meta.glob('../../../../../simulated_instruments/*/matlab/*.m', {
18+ eager: true,
19+ query: '?raw',
20+ import: 'default',
21+}) as Record<string, string>;
22+
23+export interface SimInstrumentDef {
24+ manifest: SimManifest;
25+ mfiles: MFile[];
26+}
27+
28+export function loadCatalog(): SimInstrumentDef[] {
29+ const out: SimInstrumentDef[] = [];
30+ for (const [path, text] of Object.entries(manifestModules)) {
31+ const dir = path.slice(0, path.lastIndexOf('/'));
32+ let manifest: SimManifest;
33+ try {
34+ manifest = parseManifest(text);
35+ } catch (err) {
36+ console.warn(`skipping ${path}: ${err instanceof Error ? err.message : String(err)}`);
37+ continue;
38+ }
39+ const mfiles: MFile[] = Object.entries(mModules)
40+ .filter(([p]) => p.startsWith(dir + '/matlab/'))
41+ .map(([p, source]) => ({ name: p.split('/').pop()!, source }))
42+ .sort((a, b) => a.name.localeCompare(b.name));
43+ if (mfiles.length === 0) {
44+ console.warn(`skipping ${path}: no matlab/*.m sources found`);
45+ continue;
46+ }
47+ out.push({ manifest, mfiles });
48+ }
49+ out.sort((a, b) => a.manifest.name.localeCompare(b.manifest.name));
50+ return out;
51+}
src/instruments/simulated/manifest.tsadded+172−0View file
@@ -0,0 +1,172 @@
1+import type { KnobSpec } from '../types.ts';
2+
3+/**
4+ * The simulated-instrument manifest, format "simulated-instrument-v1".
5+ *
6+ * Each repo under ../simulated_instruments carries an instrument.json
7+ * describing how midiboot may play its MATLAB model:
8+ *
9+ * name / label identity and display name
10+ * entry .m function called as y = entry(P) for one note
11+ * defaults .m function returning the parameter struct P
12+ * fixed parameter overrides always applied (single-event gesture,
13+ * fixed seed, normalize off so velocity survives in pascals)
14+ * pitch optional: how MIDI pitch reaches a parameter; mode "midi"
15+ * assigns the clamped note number, mode "table" picks the
16+ * nearest entry of a {midi, value} table (fingering charts)
17+ * velocity how velocity 0..1 maps linearly onto a parameter range
18+ * duration optional: how the note's length reaches a parameter
19+ * preview parameter overrides for the fast tier (e.g. oversample 1)
20+ * knobs curated user-editable parameters, with ranges and
21+ * defaults; everything else stays at the model's defaults
22+ *
23+ * This file is the schema's one definition: parseManifest validates a
24+ * manifest and the map* helpers below are the whole pitch/velocity/duration
25+ * semantics, shared by the browser worker and the node tests.
26+ */
27+
28+export interface PitchSpec {
29+ param: string;
30+ mode: 'midi' | 'table';
31+ min?: number;
32+ max?: number;
33+ table?: { midi: number; value: number }[];
34+}
35+
36+export interface VelocitySpec {
37+ param: string;
38+ min: number;
39+ max: number;
40+}
41+
42+export interface DurationSpec {
43+ param: string;
44+ min: number;
45+ max: number;
46+}
47+
48+export interface SimManifest {
49+ format: 'simulated-instrument-v1';
50+ name: string;
51+ label: string;
52+ entry: string;
53+ defaults: string;
54+ fixed: Record<string, number>;
55+ pitch?: PitchSpec;
56+ velocity: VelocitySpec;
57+ duration?: DurationSpec;
58+ preview: Record<string, number>;
59+ knobs: KnobSpec[];
60+}
61+
62+function fail(name: string, msg: string): never {
63+ throw new Error(`instrument manifest ${name}: ${msg}`);
64+}
65+
66+export function parseManifest(text: string): SimManifest {
67+ let j: unknown;
68+ try {
69+ j = JSON.parse(text);
70+ } catch (err) {
71+ throw new Error(`instrument manifest is not valid JSON: ${err instanceof Error ? err.message : String(err)}`);
72+ }
73+ const m = j as SimManifest;
74+ const name = typeof m.name === 'string' ? m.name : '(unnamed)';
75+ if (m.format !== 'simulated-instrument-v1') fail(name, `unknown format '${String(m.format)}'`);
76+ for (const k of ['name', 'label', 'entry', 'defaults'] as const) {
77+ if (typeof m[k] !== 'string' || m[k] === '') fail(name, `missing string field '${k}'`);
78+ }
79+ if (typeof m.fixed !== 'object' || m.fixed === null) fail(name, `missing 'fixed' overrides`);
80+ if (typeof m.preview !== 'object' || m.preview === null) fail(name, `missing 'preview' overrides`);
81+ for (const [k, v] of [...Object.entries(m.fixed), ...Object.entries(m.preview)]) {
82+ if (typeof v !== 'number' || !Number.isFinite(v)) fail(name, `override '${k}' is not a finite number`);
83+ }
84+ const vel = m.velocity;
85+ if (!vel || typeof vel.param !== 'string' || typeof vel.min !== 'number' || typeof vel.max !== 'number' || !(vel.min < vel.max)) {
86+ fail(name, `'velocity' needs {param, min < max}`);
87+ }
88+ if (m.pitch) {
89+ const p = m.pitch;
90+ if (typeof p.param !== 'string') fail(name, `'pitch' needs a param`);
91+ if (p.mode === 'midi') {
92+ if (typeof p.min !== 'number' || typeof p.max !== 'number' || !(p.min < p.max)) fail(name, `pitch mode 'midi' needs min < max`);
93+ } else if (p.mode === 'table') {
94+ if (!Array.isArray(p.table) || p.table.length === 0) fail(name, `pitch mode 'table' needs a non-empty table`);
95+ for (const row of p.table) {
96+ if (typeof row.midi !== 'number' || typeof row.value !== 'number') fail(name, `pitch table rows need {midi, value}`);
97+ }
98+ } else {
99+ fail(name, `unknown pitch mode '${String(p.mode)}'`);
100+ }
101+ }
102+ if (m.duration) {
103+ const d = m.duration;
104+ if (typeof d.param !== 'string' || typeof d.min !== 'number' || typeof d.max !== 'number' || !(d.min < d.max)) {
105+ fail(name, `'duration' needs {param, min < max}`);
106+ }
107+ }
108+ if (!Array.isArray(m.knobs)) fail(name, `missing 'knobs' array`);
109+ for (const k of m.knobs) {
110+ if (typeof k.key !== 'string' || typeof k.label !== 'string') fail(name, `knobs need key and label`);
111+ if (typeof k.min !== 'number' || typeof k.max !== 'number' || typeof k.default !== 'number' || !(k.min < k.max)) {
112+ fail(name, `knob '${k.key}' needs min < max and a default`);
113+ }
114+ if (k.default < k.min || k.default > k.max) fail(name, `knob '${k.key}' default outside its range`);
115+ if (k.scale !== undefined && k.scale !== 'log') fail(name, `knob '${k.key}' has unknown scale`);
116+ if (k.scale === 'log' && k.min <= 0) fail(name, `knob '${k.key}' log scale needs min > 0`);
117+ }
118+ return m;
119+}
120+
121+/** MIDI pitch to a parameter assignment, or null for unpitched instruments. */
122+export function mapPitch(m: SimManifest, midiPitch: number): { key: string; value: number } | null {
123+ const p = m.pitch;
124+ if (!p) return null;
125+ if (p.mode === 'midi') {
126+ return { key: p.param, value: Math.max(p.min!, Math.min(p.max!, Math.round(midiPitch))) };
127+ }
128+ let best = p.table![0];
129+ for (const row of p.table!) {
130+ if (Math.abs(row.midi - midiPitch) < Math.abs(best.midi - midiPitch)) best = row;
131+ }
132+ return { key: p.param, value: best.value };
133+}
134+
135+/** Velocity 0..1 to a parameter assignment, linear across the range. */
136+export function mapVelocity(m: SimManifest, velocity01: number): { key: string; value: number } {
137+ const v = Math.max(0, Math.min(1, velocity01));
138+ return { key: m.velocity.param, value: m.velocity.min + (m.velocity.max - m.velocity.min) * v };
139+}
140+
141+/** Note duration to a parameter assignment, or null when the model ignores it. */
142+export function mapDuration(m: SimManifest, durationSec: number): { key: string; value: number } | null {
143+ const d = m.duration;
144+ if (!d) return null;
145+ return { key: d.param, value: Math.max(d.min, Math.min(d.max, durationSec)) };
146+}
147+
148+/** Velocity quantized to a few levels so identical notes share a cached
149+ * render; 8 levels is inaudible next to the models' own dynamics. */
150+export function bucketVelocity(velocity01: number, levels = 8): number {
151+ return Math.ceil(Math.max(0.001, Math.min(1, velocity01)) * levels) / levels;
152+}
153+
154+/** Duration quantized to 0.25 s steps, only meaningful with a duration spec. */
155+export function bucketDuration(durationSec: number): number {
156+ return Math.max(0.25, Math.round(durationSec / 0.25) * 0.25);
157+}
158+
159+/** Everything that identifies one cached note render. */
160+export function noteCacheKey(
161+ m: SimManifest,
162+ quality: 'preview' | 'full',
163+ sampleRate: number,
164+ settings: Record<string, number>,
165+ pitch: number,
166+ velocityBucket: number,
167+ durationBucket: number,
168+): string {
169+ const s = m.knobs.map((k) => `${k.key}=${settings[k.key] ?? k.default}`).join(',');
170+ const pv = mapPitch(m, pitch);
171+ return `${m.name}|${quality}|${sampleRate}|${s}|${pv ? pv.value : '-'}|${velocityBucket}|${m.duration ? durationBucket : '-'}`;
172+}
src/instruments/simulated/mrender.tsadded+63−0View file
@@ -0,0 +1,63 @@
1+import { executeCode, type RuntimeValue } from 'numbl';
2+import { mapDuration, mapPitch, mapVelocity, type SimManifest } from './manifest.ts';
3+
4+/**
5+ * One note through one simulated instrument: build the parameter struct
6+ * from the model's own defaults plus the manifest's overrides, call the
7+ * entry function through numbl, and hand back the mono trace. Pure with
8+ * respect to the environment (no DOM, no worker), so the browser worker
9+ * and the node tests share it.
10+ */
11+
12+export interface MFile {
13+ name: string;
14+ source: string;
15+}
16+
17+export interface SimNoteOpts {
18+ pitch: number; // MIDI note number
19+ velocity: number; // 0..1
20+ durationSec: number;
21+ sampleRate: number;
22+ quality: 'preview' | 'full';
23+ settings: Record<string, number>; // knob values; unknown keys are ignored
24+}
25+
26+interface RuntimeTensorLike {
27+ kind: 'tensor';
28+ data: Float64Array;
29+}
30+
31+export function renderSimNote(manifest: SimManifest, mfiles: MFile[], opts: SimNoteOpts): Float32Array {
32+ const assigns: string[] = [];
33+ const setP = (key: string, value: number) => {
34+ if (!Number.isFinite(value)) throw new Error(`parameter ${key} is not finite`);
35+ assigns.push(`P.${key} = ${value};`);
36+ };
37+ for (const [k, v] of Object.entries(manifest.fixed)) setP(k, v);
38+ for (const knob of manifest.knobs) {
39+ const v = opts.settings[knob.key];
40+ setP(knob.key, v === undefined ? knob.default : Math.max(knob.min, Math.min(knob.max, v)));
41+ }
42+ if (opts.quality === 'preview') for (const [k, v] of Object.entries(manifest.preview)) setP(k, v);
43+ setP('sample_rate', opts.sampleRate);
44+ const pv = mapPitch(manifest, opts.pitch);
45+ if (pv) setP(pv.key, pv.value);
46+ const vv = mapVelocity(manifest, opts.velocity);
47+ setP(vv.key, vv.value);
48+ const dv = mapDuration(manifest, opts.durationSec);
49+ if (dv) setP(dv.key, dv.value);
50+
51+ const code = [
52+ `P = ${manifest.defaults}();`,
53+ ...assigns,
54+ `y_out = ${manifest.entry}(P);`,
55+ ].join('\n');
56+
57+ const r = executeCode(code, { displayResults: false }, mfiles, 'midiboot_note.m');
58+ const y = r.variableValues.y_out as RuntimeValue as RuntimeTensorLike;
59+ if (!y || y.kind !== 'tensor') throw new Error(`${manifest.name}: entry '${manifest.entry}' did not return a tensor`);
60+ const out = new Float32Array(y.data.length);
61+ for (let i = 0; i < y.data.length; i++) out[i] = y.data[i];
62+ return out;
63+}
src/instruments/simulated/worker.tsadded+111−0View file
@@ -0,0 +1,111 @@
1+/**
2+ * The render worker: numbl runs here so a seconds-long eigensolve or jet
3+ * integration never blocks the UI. One request renders one track: each
4+ * note is rendered individually (velocity and duration quantized so
5+ * repeats hit the cache), placed at its onset, and summed; the mixed
6+ * buffer is transferred back. The per-note cache is an LRU bounded by
7+ * total samples, keyed by everything that affects one note's audio.
8+ */
9+import { loadCatalog, type SimInstrumentDef } from './catalog.ts';
10+import { bucketDuration, bucketVelocity, noteCacheKey } from './manifest.ts';
11+import { renderSimNote } from './mrender.ts';
12+
13+export interface WorkerNote {
14+ pitch: number;
15+ velocity: number; // 0..1
16+ durationSec: number;
17+ timeSec: number;
18+}
19+
20+export interface WorkerRenderMsg {
21+ kind: 'render';
22+ id: number;
23+ name: string; // manifest name
24+ notes: WorkerNote[];
25+ sampleRate: number;
26+ lengthSamples: number;
27+ quality: 'preview' | 'full';
28+ settings: Record<string, number>;
29+}
30+
31+export type WorkerResponse =
32+ | { kind: 'result'; id: number; samples: Float32Array }
33+ | { kind: 'error'; id: number; message: string }
34+ | { kind: 'status'; text: string };
35+
36+const catalog = new Map<string, SimInstrumentDef>();
37+for (const def of loadCatalog()) catalog.set(def.manifest.name, def);
38+
39+const CACHE_MAX_SAMPLES = 48e6; // ~192 MB of Float32 note renders
40+const cache = new Map<string, Float32Array>();
41+let cacheSamples = 0;
42+
43+function cacheGet(key: string): Float32Array | undefined {
44+ const v = cache.get(key);
45+ if (v) {
46+ cache.delete(key);
47+ cache.set(key, v);
48+ }
49+ return v;
50+}
51+
52+function cachePut(key: string, buf: Float32Array): void {
53+ cache.set(key, buf);
54+ cacheSamples += buf.length;
55+ while (cacheSamples > CACHE_MAX_SAMPLES && cache.size > 1) {
56+ const oldest = cache.keys().next().value as string;
57+ cacheSamples -= cache.get(oldest)!.length;
58+ cache.delete(oldest);
59+ }
60+}
61+
62+const post = (m: WorkerResponse, transfer?: Transferable[]) =>
63+ (self as unknown as Worker).postMessage(m, transfer ?? []);
64+
65+self.onmessage = (ev: MessageEvent<WorkerRenderMsg>) => {
66+ const msg = ev.data;
67+ if (msg.kind !== 'render') return;
68+ try {
69+ const def = catalog.get(msg.name);
70+ if (!def) throw new Error(`unknown simulated instrument '${msg.name}'`);
71+ const m = def.manifest;
72+ const out = new Float32Array(msg.lengthSamples);
73+ // Group notes by cache key so each distinct sound renders once.
74+ const groups = new Map<string, { velB: number; durB: number; pitch: number; times: number[] }>();
75+ for (const n of msg.notes) {
76+ const velB = bucketVelocity(n.velocity);
77+ const durB = bucketDuration(n.durationSec);
78+ const key = noteCacheKey(m, msg.quality, msg.sampleRate, msg.settings, n.pitch, velB, durB);
79+ const g = groups.get(key);
80+ if (g) g.times.push(n.timeSec);
81+ else groups.set(key, { velB, durB, pitch: n.pitch, times: [n.timeSec] });
82+ }
83+ let done = 0;
84+ for (const [key, g] of groups) {
85+ let buf = cacheGet(key);
86+ if (!buf) {
87+ post({ kind: 'status', text: `${m.label}: rendering note ${done + 1}/${groups.size} (${msg.quality})` });
88+ buf = renderSimNote(m, def.mfiles, {
89+ pitch: g.pitch,
90+ velocity: g.velB,
91+ durationSec: g.durB,
92+ sampleRate: msg.sampleRate,
93+ quality: msg.quality,
94+ settings: msg.settings,
95+ });
96+ cachePut(key, buf);
97+ }
98+ for (const t of g.times) {
99+ const off = Math.round(t * msg.sampleRate);
100+ const end = Math.min(out.length, off + buf.length);
101+ for (let i = Math.max(0, off); i < end; i++) out[i] += buf[i - off];
102+ }
103+ done++;
104+ }
105+ post({ kind: 'status', text: '' });
106+ post({ kind: 'result', id: msg.id, samples: out }, [out.buffer]);
107+ } catch (err) {
108+ post({ kind: 'status', text: '' });
109+ post({ kind: 'error', id: msg.id, message: err instanceof Error ? err.message : String(err) });
110+ }
111+};
src/instruments/types.tsmodified+19−0View file
@@ -27,6 +27,23 @@ export interface RenderRequest {
2727 notes: NoteSpec[];
2828 sampleRate: number;
2929 lengthSamples: number;
30+ /** Fast preview for playback, full quality for export; instruments
31+ * without tiers ignore it. */
32+ quality?: 'preview' | 'full';
33+ /** Values for the instrument's knobs (per track); missing keys mean the
34+ * knob's default. */
35+ settings?: Record<string, number>;
36+}
37+
38+/** A user-editable instrument parameter, declared by the instrument. */
39+export interface KnobSpec {
40+ key: string;
41+ label: string;
42+ min: number;
43+ max: number;
44+ default: number;
45+ step?: number;
46+ scale?: 'log';
3047 }
3148
3249 export interface LiveVoice {
@@ -39,4 +56,6 @@ export interface Instrument {
3956 name: string;
4057 render(req: RenderRequest): Float32Array | Promise<Float32Array>;
4158 noteOn?(ctx: BaseAudioContext, dest: AudioNode, pitch: number, velocity: number, whenSec: number): LiveVoice;
59+ /** Curated editable parameters; tracks store their values in settings. */
60+ knobs?: KnobSpec[];
4261 }
src/main.tsxmodified+3−3View file
@@ -1,10 +1,10 @@
11 import { createRoot } from 'react-dom/client';
22 import { App } from './components/App.tsx';
33 import { store } from './state/store.ts';
4-import { registerInstrawaveInstruments, setInstrawaveStatus } from './instruments/instrawave/adapter.ts';
4+import { registerSimulatedInstruments, setSimulatedStatus } from './instruments/simulated/adapter.ts';
55 import './styles.css';
66
7-registerInstrawaveInstruments();
8-setInstrawaveStatus((msg) => store.setStatus(msg));
7+registerSimulatedInstruments();
8+setSimulatedStatus((msg) => store.setStatus(msg));
99 void store.init();
1010 createRoot(document.getElementById('root')!).render(<App />);
src/state/store.tsmodified+8−0View file
@@ -276,6 +276,14 @@ class Store {
276276 }
277277 }
278278
279+ setTrackSetting(id: string, key: string, value: number): void {
280+ this.editSong((song) => {
281+ const t = song.tracks.find((x) => x.id === id);
282+ if (t) t.settings = { ...(t.settings ?? {}), [key]: value };
283+ });
284+ this.scheduleSave();
285+ }
286+
279287 setSelection(ids: string[]): void { this.set({ selection: ids }); }
280288
281289 setSnapBeats(v: number): void { this.set({ snapBeats: v }); }
src/styles.cssmodified+6−0View file
@@ -98,6 +98,12 @@ label { display: inline-flex; align-items: center; gap: 4px; white-space: nowrap
9898 .trackrow .trackname:hover, .trackrow .trackname:focus { border-color: #333a47; background: #10131a; }
9999 .trackrow select { flex: 1; min-width: 0; }
100100 .trackrow .notecount { color: #777; font-size: 11px; white-space: nowrap; }
101+.trackrow-knobs { display: flex; flex-direction: column; gap: 2px; }
102+.knob { display: flex; align-items: center; gap: 5px; font-size: 11px; color: #9aa0ab; }
103+.knob .knoblabel { flex: none; width: 88px; white-space: nowrap; overflow: hidden; text-overflow: ellipsis; }
104+.knob input[type='range'] { flex: 1; }
105+.knob .knobvalue { flex: none; width: 44px; text-align: right; font-variant-numeric: tabular-nums; }
106+
101107 .addtrack { margin-top: 2px; }
102108
103109 .pianoroll { flex: 1; min-width: 0; position: relative; }
src/types.tsmodified+2−0View file
@@ -13,6 +13,8 @@ export interface Track {
1313 id: string;
1414 name: string;
1515 instrumentId: string;
16+ /** Instrument knob values, keyed by knob key; affects rendering. */
17+ settings?: Record<string, number>;
1618 volume: number; // 0..1.5, applied at mix time
1719 mute: boolean;
1820 solo: boolean;
tsconfig.jsonmodified+1−1View file
@@ -12,7 +12,7 @@
1212 "noEmit": true,
1313 "skipLibCheck": true,
1414 "lib": ["ES2022", "DOM", "DOM.Iterable"],
15- "types": ["@webgpu/types", "vite/client", "node"]
15+ "types": ["vite/client", "node"]
1616 },
1717 "include": ["src", "scripts"]
1818 }
vite.config.tsmodified+6−6View file
@@ -3,15 +3,15 @@ import react from '@vitejs/plugin-react';
33 import { realpathSync } from 'node:fs';
44 import { resolve } from 'node:path';
55
6-// instrawave is a file: dependency on the sibling checkout (the vault
7-// layout); it is TypeScript source behind a symlink, so it must be served
8-// from its real path and kept out of prebundling.
9-const instrawave = realpathSync(resolve(import.meta.dirname, '../instrawave'));
6+// The simulated instruments live in the sibling collection (the vault
7+// layout): each repo carries an instrument.json manifest and MATLAB
8+// sources that midiboot imports at build time via import.meta.glob, so
9+// the dev server must be allowed to read them from their real path.
10+const simulated = realpathSync(resolve(import.meta.dirname, '../../simulated_instruments'));
1011
1112 export default defineConfig({
1213 base: './',
1314 plugins: [react()],
14- optimizeDeps: { exclude: ['instrawave'] },
15- server: { fs: { allow: [import.meta.dirname, instrawave] } },
15+ server: { fs: { allow: [import.meta.dirname, simulated] } },
1616 build: { target: 'es2022' },
1717 });