Render whole tracks through direct-import instrument modules
Instruments now receive the entire MIDI track and own their rendering
end to end: caching, progress reporting, and cooperative cancellation.
The simulated instruments are imported as source from the sibling
repos' instrument/ modules, replacing the numbl-instrument-v1 bundle
catalog and the runtime import UI. The builtin set grows to nine
GM-family synths, MIDI import maps GM programs onto them, and the
track list and transport show per-track render progress with cancel.
lengthSamples is a minimum, so tails that outlive the last note are
kept.
25 changed files+549−2619
README.mdmodified+12−12View file
@@ -1,6 +1,6 @@
11 # midiboot
22
3-An in-browser application for recording, importing, editing, and playing MIDI. A limitation of most browser MIDI tools is that the instrument is fixed: notes go into a General MIDI soundfont or a hard-coded synth, and there is no place to put your own rendering logic. Here we describe a small workstation built around one extension point, an instrument interface that turns note events into audio, so that the built-in synths sit alongside custom renderers without changing the editor, the transport, or the file handling around them. The first such renderers are the physically based models of the [simulated_instruments](../../simulated_instruments) collection, packaged as self-contained instrument bundles and executed client-side by [numbl](https://numbl.org).
3+An in-browser application for recording, importing, editing, and playing MIDI. A limitation of most browser MIDI tools is that the instrument is fixed: notes go into a General MIDI soundfont or a hard-coded synth, and there is no place to put your own rendering logic. Here we describe a small workstation built around one extension point, an instrument interface that turns an entire MIDI track into audio, so that the built-in synths sit alongside custom renderers without changing the editor, the transport, or the file handling around them. The first such renderers are the physically based models of the [simulated_instruments](../../simulated_instruments) collection, imported directly from their repos and executed client-side by [numbl](https://numbl.org).
44
55 ## The instrument interface
66
@@ -10,19 +10,18 @@ An instrument is anything implementing the interface in [src/instruments/types.t
1010 interface Instrument {
1111 id: string;
1212 name: string;
13- render(req: RenderRequest): Float32Array | Promise<Float32Array>;
13+ knobs?: KnobSpec[];
14+ renderTrack(req: TrackRenderRequest): Promise<TrackRenderResult>;
1415 noteOn?(ctx: BaseAudioContext, dest: AudioNode, pitch: number,
1516 velocity: number, whenSec: number): LiveVoice;
1617 }
1718 ```
1819
19-There are two paths. The offline path, `render`, receives every note of a track with times already converted to seconds and produces one mono buffer; it may be async and arbitrarily expensive, and playback, mixing, and WAV export all go through it. The live path, `noteOn`, is optional and exists for low-latency monitoring while recording and for previewing notes from the piano roll; an instrument without it still works everywhere, with a plain beep standing in for monitoring. New instruments are added with `registerInstrument` in [src/instruments/registry.ts](src/instruments/registry.ts); tracks refer to instruments by id, so a song whose instrument is missing falls back to a builtin rather than failing.
20+There are two paths. The offline path, `renderTrack`, receives every note of a track with times already converted to seconds and produces one mono buffer for the whole track, which may run past the requested length so that a ring outliving the last note (a cymbal crash, a long bass tail) is kept rather than clipped; playback, mixing, and WAV export all go through it and mix buffers of unequal lengths. The whole track arrives in one call, so an instrument owns its rendering strategy end to end: it may render notes independently and cache them, share expensive pitch-independent stages across notes, or in principle simulate the track as one continuous process, which is what makes inter-note physics (a string choked by the next note, a cymbal heating up across a roll) expressible at all. The request carries a cooperative `AbortSignal` and an `onProgress` callback, so a seconds-long physical render is visible and cancellable from the transport and the track list. The live path, `noteOn`, is optional and exists for low-latency monitoring while recording and for previewing notes from the piano roll; an instrument without it still works everywhere, with a plain beep standing in for monitoring. New instruments are added with `registerInstrument` in [src/instruments/registry.ts](src/instruments/registry.ts); tracks refer to instruments by id, so a song whose instrument is missing falls back to a builtin rather than failing.
2021
21-Alongside the builtins are the exported numbl instruments. Each repository in ../simulated_instruments (standup bass, recorder, triangle, cymbal, maraca, woodblock) is a physically based model written in MATLAB syntax, and each exports one or more instrument bundles: self-contained JSON files, format `numbl-instrument-v1`, carrying the model's sources together with driver functions, written by the instrument author in the same MATLAB syntax, that map one MIDI note onto the model (`y = render_note(note)`, with an optional `state = prepare(params)` for pitch-independent setup). The note struct carries pitch, velocity 0..1, duration in seconds, sample rate, a preview flag, and the values of the curated knobs the bundle declares; how those become model parameters is entirely the driver's business, so midiboot knows nothing about any model. The format, the driver contract, and the export build script are specified in [instrument-format](../../simulated_instruments/instrument-format), and one repository may export several bundles (the bass ships a muted variant with the damping baked in). Committed bundles under each repo's `exports/` are discovered at build time; a bundle file can also be imported at runtime from the track panel, persisted in IndexedDB, and an import whose id matches a catalog bundle shadows it, which is how a newer export is tried without rebuilding the app. The vendored format definition, with validation, lives in [src/instruments/numbl/bundle.ts](src/instruments/numbl/bundle.ts).
22+Alongside the builtins are the simulated instruments, imported directly from the sibling repos: each of [bass](../../simulated_instruments/bass) (with a muted variant), [woodblock](../../simulated_instruments/woodblock), [recorder](../../simulated_instruments/recorder), [triangle](../../simulated_instruments/triangle), [maraca](../../simulated_instruments/maraca), and [cymbal2](../../simulated_instruments/cymbal2) carries an `instrument/` module, a peer of its `matlab/` sources and its own web app, that implements the interface above. The module owns a Web Worker running the MATLAB model through numbl, so a seconds-long solve never blocks the editor, and it owns its caching: distinct sounds (pitch, velocity quantized to eight levels, duration where the model actually reads it) render once and are summed at the onsets, and where the model's stages permit it, expensive pitch-independent intermediates are shared across notes (the bass reuses one coupled eigensolve per fingered pitch across velocities and repeats, and the triangle does the same with its FEM eigensolve). The note-to-parameter mapping, curated knobs included, lives in the same module and is the instrument author's business; midiboot knows nothing about any model. The backend is equally the module's business: the cymbal's time loop runs on WebGPU when the browser exposes it (the WGSL port living beside the model), with per-chunk progress and cancellation, and falls back to the numbl interpreter otherwise; midiboot cannot tell the difference. Even so, a full-quality cymbal stroke is a real computation (tens of seconds on an integrated GPU, minutes on the CPU), which is exactly what the per-track render cache and the progress UI exist for. The same interface, through the request's app-facing optional fields (`params` for the full parameter surface, `mfiles` for live-edited sources, `wantDiagnostics` for the plot payloads), is what each instrument's own web app renders through, so there is exactly one render path per model. Cancelling is cooperative, checked between render stages, and the caches survive it; a worker that never responds is torn down after a grace period. There is no live noteOn voice for these yet, so monitoring while recording falls back to the player's beep.
2223
23-The models render in a Web Worker so a seconds-long solve never blocks the editor. Playback renders each distinct note once (velocity quantized to eight levels, duration to quarter seconds), caches it in an LRU bounded by total samples, places copies at the onsets, and sums; a repeated drum pattern therefore costs one render. The knobs a bundle declares appear as sliders on the track (the active row), stored per track and invalidating that track's render when moved. The trade-off of per-note rendering is the loss of inter-note physics: the bass's choke of a note when the next lands, and a cymbal's energy build-up across a roll, are outside this arrangement, and a model whose expensive stage does not depend on the note (the cymbal's eigensolve) repeats it per distinct velocity rather than caching it across notes.
24-
25-The four builtin instruments (Karplus-Strong pluck, FM electric piano, a filtered square lead, and a General MIDI drum kit) are constructed from pure per-note DSP functions in [src/instruments/dsp.ts](src/instruments/dsp.ts). These are deterministic, with noise drawn from a PRNG seeded by pitch, so the node test harness checks the actual audio. The `instrumentFromNoteDsp` helper derives both paths from one such function; a custom instrument is free to ignore it and implement the interface directly.
24+The builtin instruments are cheap synthetic ones, one per broad General MIDI family (an FM piano and e-piano, a drawbar organ, a Karplus-Strong pluck, a synth bass, a string pad, a saw brass section, a square lead, and a General MIDI drum kit), constructed from pure per-note DSP functions in [src/instruments/dsp.ts](src/instruments/dsp.ts). These are deterministic, with noise drawn from a PRNG seeded by pitch, so the node test harness checks the actual audio. Importing a MIDI file assigns each track the builtin nearest its GM program (channel 10 gets the drum kit), so an imported file is playable without a soundfont. The `instrumentFromNoteDsp` helper derives both paths from one per-note DSP function; a custom instrument is free to ignore it and implement the interface directly.
2625
2726 ## Recording
2827
@@ -36,11 +35,11 @@ Dragging in the ruler makes a *time selection*, shown as a band across the roll,
3635
3736 ## Playback and rendering
3837
39-Playback is offline-first: each track is rendered by its instrument into a buffer, cached under a key derived from everything that affects the audio (notes, tempo map, ppq, instrument), and the buffers are played through per-track gains. Volume, mute, and solo act at mix time and never cause a re-render, an arrangement borrowed from [instrawave-midi](../instrawave-midi). This is what makes expensive future instruments practical: a render costs what it costs, once, and editing anything else costs nothing. The trade-off is that pressing play after an edit waits for the affected track to re-render, which for the builtin synths is not noticeable. Export produces a 16-bit mono WAV at 44.1 kHz, or a standard MIDI file (format 1, with the tempo and time signature maps and with drum tracks on channel 10).
38+Playback is offline-first: each track is rendered by its instrument into a buffer, cached under a key derived from everything that affects the audio (notes, tempo map, ppq, instrument, knob settings), and the buffers are played through per-track gains. Volume, mute, and solo act at mix time and never cause a re-render, an arrangement borrowed from [instrawave-midi](../instrawave-midi). This is what makes the expensive instruments practical: a render costs what it costs, once, and editing anything else costs nothing. The trade-off is that pressing play after an edit waits for the affected track to re-render; for the builtin synths this is not noticeable, and for a simulated instrument the track row shows a progress bar with the render stage and a cancel button, with a one-line summary in the transport. Export produces a 16-bit mono WAV at 44.1 kHz rendered at each instrument's full quality tier, or a standard MIDI file (format 1, with the tempo and time signature maps and with drum tracks on channel 10).
4039
4140 ## Files and storage
4241
43-Songs autosave to IndexedDB a moment after each edit, and the Library dialog lists, opens, renames, duplicates, and deletes them. Import reads standard MIDI files, formats 0 and 1, with running status and merged tempo maps; each (track, channel) pair carrying notes becomes an editable track, channel 10 defaults to the drum kit, and track names fall back to General MIDI program names. Note that import keeps only notes, tempo, and time signature: controllers, pitch bend, and aftertouch are parsed past and dropped, and SMPTE-timed files are refused rather than misread.
42+Songs autosave to IndexedDB a moment after each edit, and the Library dialog lists, opens, renames, duplicates, and deletes them. Import reads standard MIDI files, formats 0 and 1, with running status and merged tempo maps; each (track, channel) pair carrying notes becomes an editable track, channel 10 defaults to the drum kit, other tracks get the builtin synth nearest their GM program, and track names fall back to General MIDI program names. Note that import keeps only notes, tempo, and time signature: controllers, pitch bend, and aftertouch are parsed past and dropped, and SMPTE-timed files are refused rather than misread.
4443
4544 ## Development
4645
@@ -49,11 +48,12 @@ npm install
4948 npm run dev # vite dev server
5049 npm run build # typecheck and bundle
5150 npm test # node-side tests (no browser, no audio device)
52-npm run test:sim # renders real notes through numbl in node; slower
5351 ```
5452
55-The tests cover the MIDI writer/parser round trip, tempo map arithmetic, quantize and transpose, mixing rules, the determinism and boundedness of the DSP, and validation of every instrument manifest in the sibling collection. `npm run test:sim` additionally executes the actual models through numbl in node and checks the audio: finiteness, decay, determinism under the fixed seed, and that velocity and knobs genuinely change the sound.
53+The simulated instruments are imported from the sibling repos as source, and their `numbl` dependency resolves from each module's own directory, so run `npm install` once in each `../../simulated_instruments/<repo>/instrument/` as well.
54+
55+The tests cover the MIDI writer/parser round trip, tempo map arithmetic, quantize and transpose, mixing rules, the determinism and boundedness of the DSP, the GM program mapping, and the track-render contract (note timing, settings, progress, leveling, and abort) against a fake instrument, so they run fast and do not depend on the sibling repos. The tests that execute the actual models through numbl in node live with each model, in `simulated_instruments/<repo>/instrument/` (`npm test` there); they check finiteness, decay, determinism under the fixed seed, cache hits, and that velocity and knobs genuinely change the sound.
5656
5757 ## Limitations and future work
5858
59-The mixdown is mono, matching the mono instrument interface; a stereo render path is a straightforward extension. There is no loop region, no controller or pitch bend data, and editing the tempo of a song that already has a tempo map is not supported (the bpm field edits only single-tempo songs rather than silently flattening the map). The live path plays each note as a pre-rendered buffer with a gain release, which is fine for monitoring but is not a substitute for a real streaming voice. The simulated instruments have no live monitoring voice, so recording against one monitors through a plain beep; a small pre-rendered voice bank per instrument is the natural fix. Note caches live in worker memory and are redone per page load; persisting them in IndexedDB, keyed the way the render cache already is, is a straightforward extension. Manifest knobs are the only editable parameters; the full parameter surfaces stay in each instrument's own web app, which remains the place to design a sound before curating its knobs here.
59+The mixdown is mono, matching the mono instrument interface; a stereo render path is a straightforward extension. There is no loop region, no controller or pitch bend data, and editing the tempo of a song that already has a tempo map is not supported (the bpm field edits only single-tempo songs rather than silently flattening the map). The live path plays each note as a pre-rendered buffer with a gain release, which is fine for monitoring but is not a substitute for a real streaming voice. The simulated instruments have no live monitoring voice, so recording against one monitors through a plain beep; a small pre-rendered voice bank per instrument is the natural fix. Although the interface hands each instrument the whole track, the three current models still render notes independently and sum them, so inter-note physics (the bass choking a ringing string, legato breath on the recorder, a cymbal still moving when the next stroke lands) remains future work per instrument; the interface no longer stands in its way. Rendered notes persist across page loads: each instrument keeps its finished note renders in an in-memory LRU in front of an IndexedDB store (per-instrument sample budgets, least-recently-used trimming, and a 30-day age-out), so reopening a song replays it without re-rendering; the keys carry the .m code hash, so a model update simply stops matching and the stale entries age out. Curated knobs are the only editable parameters here; the full parameter surfaces stay in each instrument's own web app, which now renders through the same instrument module and remains the place to design a sound. Every current simulated_instruments repo is converted (cymbal is superseded by cymbal2); cymbal2, whose WebGPU time loop runs inside its instrument module, shows what a conversion looks like when the model warrants a backend of its own.
package-lock.jsonmodified+44−1748View file
This diff is 2,001 lines long and is not shown.
package.jsonmodified+2−4View file
@@ -10,13 +10,11 @@
1010 "scripts": {
1111 "dev": "vite",
1212 "build": "tsc --noEmit && vite build",
13- "test": "vite-node scripts/test-node.ts",
14- "test:sim": "vite-node scripts/test-sim.ts"
13+ "test": "vite-node scripts/test-node.ts"
1514 },
1615 "dependencies": {
1716 "react": "^18.3.1",
18- "react-dom": "^18.3.1",
19- "numbl": "^0.4.19"
17+ "react-dom": "^18.3.1"
2018 },
2119 "devDependencies": {
2220 "@types/node": "^26.1.1",
scripts/test-node.tsmodified+83−87View file
@@ -6,13 +6,11 @@ import { TempoMap } from '../src/midi/time.ts';
66 import { parseMidi } from '../src/midi/parse.ts';
77 import { writeMidi } from '../src/midi/write.ts';
88 import { clearRange, deleteTimeFromEvents, deleteTimeRange, extractRange, insertTime, insertTimeIntoEvents, quantizeNotes, transposeNotes } from '../src/midi/ops.ts';
9-import { drumNote, epianoNote, leadNote, pluckNote } from '../src/instruments/dsp.ts';
9+import { brassNote, drumNote, epianoNote, leadNote, organNote, pianoNote, pluckNote, stringsNote, synthBassNote } from '../src/instruments/dsp.ts';
1010 import { builtinInstruments } from '../src/instruments/builtins.ts';
11-import { audibleTracks, encodeWavMono16, mixTracks, trackRenderKey } from '../src/audio/render.ts';
12-import { parseBundle } from '../src/instruments/numbl/bundle.ts';
13-import { bucketDuration, bucketVelocity, buildProgram, noteCacheKey } from '../src/instruments/numbl/render.ts';
14-import { readdirSync, readFileSync, existsSync } from 'node:fs';
15-import { join } from 'node:path';
11+import { gmProgramInstrument } from '../src/midi/gm.ts';
12+import { audibleTracks, encodeWavMono16, mixTracks, renderTrackBuffer, trackRenderKey } from '../src/audio/render.ts';
13+import type { Instrument, TrackRenderRequest, TrackRenderResult } from '../src/instruments/types.ts';
1614 import type { Note, Song, Track } from '../src/types.ts';
1715
1816 let failed = 0;
@@ -29,6 +27,18 @@ function test(name: string, fn: () => void) {
2927 }
3028 }
3129
30+async function testAsync(name: string, fn: () => Promise<void>) {
31+ try {
32+ await fn();
33+ passed++;
34+ console.log(`ok - ${name}`);
35+ } catch (err) {
36+ failed++;
37+ console.error(`FAIL - ${name}`);
38+ console.error(err);
39+ }
40+}
41+
3242 function close(a: number, b: number, tol = 1e-9) {
3343 assert.ok(Math.abs(a - b) <= tol, `${a} !~ ${b}`);
3444 }
@@ -124,7 +134,7 @@ test('MIDI write/parse round trip preserves notes, tempo, and drums', () => {
124134 assert.deepEqual(sortedNotes(back.tracks[1]), sortedNotes(song.tracks[1]));
125135 assert.equal(back.tracks[0].name, 'melody');
126136 assert.equal(back.tracks[1].instrumentId, 'drumkit'); // came back on channel 10
127- assert.equal(back.tracks[0].instrumentId, 'pluck');
137+ assert.equal(back.tracks[0].instrumentId, 'piano'); // GM family mapping of the default program
128138 assert.deepEqual(back.tempo, song.tempo);
129139 assert.equal(back.timeSig[0].numerator, 3);
130140 assert.equal(back.timeSig[0].denominator, 4);
@@ -226,8 +236,13 @@ test('event lists ripple with time edits and keep tick 0', () => {
226236
227237 test('DSP is deterministic and finite', () => {
228238 for (const [name, fn] of [
229- ['pluck', pluckNote],
239+ ['piano', pianoNote],
230240 ['epiano', epianoNote],
241+ ['organ', organNote],
242+ ['pluck', pluckNote],
243+ ['synthbass', synthBassNote],
244+ ['strings', stringsNote],
245+ ['brass', brassNote],
231246 ['lead', leadNote],
232247 ['drum', drumNote],
233248 ] as const) {
@@ -245,14 +260,15 @@ test('DSP is deterministic and finite', () => {
245260 }
246261 });
247262
248-test('builtin instrument render places notes at the right time', () => {
263+await testAsync('builtin instrument renderTrack places notes at the right time', async () => {
249264 const pluck = builtinInstruments.find((i) => i.id === 'pluck')!;
250265 const sr = 44100;
251- const out = pluck.render({
266+ const { audio: out } = await pluck.renderTrack({
252267 notes: [{ timeSec: 1, durationSec: 0.25, pitch: 69, velocity: 0.8 }],
253268 sampleRate: sr,
254269 lengthSamples: sr * 3,
255- }) as Float32Array;
270+ quality: 'preview',
271+ });
256272 assert.equal(out.length, sr * 3);
257273 let before = 0;
258274 for (let i = 0; i < sr - 10; i++) before = Math.max(before, Math.abs(out[i]));
@@ -309,86 +325,66 @@ test('WAV encoding writes a well-formed header', () => {
309325 assert.equal(dv.getInt16(44 + 6, true), 32767); // clipped to full scale
310326 });
311327
312-// --- exported instrument bundles ---
313-
314-const SIM_ROOT = join(import.meta.dirname, '../../../simulated_instruments');
315-
316-test('every committed bundle in ../simulated_instruments validates', () => {
317- if (!existsSync(SIM_ROOT)) return; // vault layout absent; nothing to check
318- const dirs = readdirSync(SIM_ROOT).filter((d) => existsSync(join(SIM_ROOT, d, 'exports')));
319- assert.ok(dirs.length >= 1, 'no exports found');
320- let count = 0;
321- for (const d of dirs) {
322- for (const f of readdirSync(join(SIM_ROOT, d, 'exports')).filter((f) => f.endsWith('.json'))) {
323- const b = parseBundle(readFileSync(join(SIM_ROOT, d, 'exports', f), 'utf8'));
324- assert.equal(`${b.id}.json`, f, `bundle id '${b.id}' should match its filename '${f}'`);
325- count++;
326- }
327- }
328- assert.ok(count >= 7, `expected at least 7 bundles, found ${count}`);
329-});
330-
331-const TEST_BUNDLE = {
332- format: 'numbl-instrument-v1',
333- id: 'test',
334- name: 'Test',
335- knobs: [{ key: 'k', label: 'K', min: 1, max: 5, default: 2 }],
336- files: [{ name: 'render_note.m', content: 'function y = render_note(note)\ny = [note.pitch];\nend' }],
337-};
338-
339-test('buildProgram assigns knobs clamped and omits prepare without prepare.m', () => {
340- const b = parseBundle(JSON.stringify(TEST_BUNDLE));
341- const code = buildProgram(b, [{ pitch: 60, velocity: 0.5, durationSec: 1 }], {
342- sampleRate: 22050, quality: 'preview', settings: { k: 99 },
343- });
344- assert.ok(code.includes('params.k = 5;'), 'knob clamped to its max');
345- assert.ok(code.includes('note.pitch = 60;'));
346- assert.ok(code.includes('params.preview = 1;'));
347- assert.ok(!code.includes('prepare('), 'no prepare stage without prepare.m');
348- const withPrep = parseBundle(JSON.stringify({
349- ...TEST_BUNDLE,
350- files: [...TEST_BUNDLE.files, { name: 'prepare.m', content: 'function state = prepare(params)\nstate = struct();\nend' }],
351- }));
352- const code2 = buildProgram(withPrep, [{ pitch: 60, velocity: 0.5, durationSec: 1 }], {
353- sampleRate: 44100, quality: 'full', settings: {},
354- });
355- assert.ok(code2.includes('state = prepare(params);'));
356- assert.ok(code2.includes('note.state = state;'));
357- assert.ok(code2.includes('params.k = 2;'), 'missing knob takes its default');
358- assert.ok(code2.includes('params.preview = 0;'));
359-});
360-
361-test('velocity and duration bucketing quantize into few levels', () => {
362- assert.equal(bucketVelocity(1), 1);
363- assert.equal(bucketVelocity(0.999), 1);
364- assert.equal(bucketVelocity(0.51), 0.625);
365- assert.equal(bucketVelocity(0.01), 0.125);
366- assert.ok(bucketVelocity(0) > 0);
367- assert.equal(bucketDuration(0.9), 1);
368- assert.equal(bucketDuration(0.05), 0.25);
328+// --- the track-render instrument contract ---
329+
330+await testAsync('renderTrackBuffer drives renderTrack with progress, settings, and leveling', async () => {
331+ const seen: TrackRenderRequest[] = [];
332+ const progress: number[] = [];
333+ const fake: Instrument = {
334+ id: 'fake',
335+ name: 'Fake',
336+ renderTrack(req: TrackRenderRequest): Promise<TrackRenderResult> {
337+ seen.push(req);
338+ req.onProgress?.({ frac: 0.5, stage: 'half' });
339+ const audio = new Float32Array(req.lengthSamples);
340+ audio[0] = 0.1; // quiet, so leveling must scale it up to 0.5
341+ return Promise.resolve({ audio });
342+ },
343+ };
344+ const song: Song = {
345+ ppq: 480,
346+ tempo: [{ tick: 0, usPerQuarter: 500000 }],
347+ timeSig: [{ tick: 0, numerator: 4, denominator: 4 }],
348+ tracks: [],
349+ };
350+ const track = mkTrack('a', 'fake', [mkNote(0, 480, 60, 100, 'n1')], { settings: { k: 3 } });
351+ const buf = await renderTrackBuffer(song, track, fake, 44100, 'preview', { onProgress: (p) => progress.push(p.frac) });
352+ assert.equal(seen.length, 1);
353+ assert.equal(seen[0].notes.length, 1);
354+ close(seen[0].notes[0].durationSec, 0.5, 1e-6);
355+ assert.equal(seen[0].settings?.k, 3);
356+ assert.deepEqual(progress, [0.5]);
357+ close(buf[0], 0.5, 1e-6); // leveled to the common peak
369358 });
370359
371-test('noteCacheKey separates settings, pitch, and quality', () => {
372- const b = parseBundle(JSON.stringify(TEST_BUNDLE));
373- const k1 = noteCacheKey(b, 'preview', 22050, {}, 60, 1, 0.5);
374- const k2 = noteCacheKey(b, 'preview', 22050, { k: 3 }, 60, 1, 0.5);
375- const k3 = noteCacheKey(b, 'preview', 22050, {}, 62, 1, 0.5);
376- const k4 = noteCacheKey(b, 'full', 22050, {}, 60, 1, 0.5);
377- assert.ok(k1 !== k2 && k1 !== k3 && k1 !== k4 && k2 !== k3);
360+await testAsync('an aborted signal reaches the instrument', async () => {
361+ const ctrl = new AbortController();
362+ const fake: Instrument = {
363+ id: 'fake',
364+ name: 'Fake',
365+ renderTrack(req: TrackRenderRequest): Promise<TrackRenderResult> {
366+ if (req.signal?.aborted) return Promise.reject(new DOMException('render cancelled', 'AbortError'));
367+ return Promise.resolve({ audio: new Float32Array(req.lengthSamples) });
368+ },
369+ };
370+ const song: Song = { ppq: 480, tempo: [{ tick: 0, usPerQuarter: 500000 }], timeSig: [{ tick: 0, numerator: 4, denominator: 4 }], tracks: [] };
371+ ctrl.abort();
372+ await assert.rejects(
373+ renderTrackBuffer(song, mkTrack('a', 'fake', []), fake, 44100, 'preview', { signal: ctrl.signal }),
374+ (e: unknown) => e instanceof DOMException && e.name === 'AbortError',
375+ );
378376 });
379377
380-test('bundle validation rejects malformed input', () => {
381- assert.throws(() => parseBundle(JSON.stringify({ ...TEST_BUNDLE, format: 'wrong' })));
382- assert.throws(() => parseBundle(JSON.stringify({ ...TEST_BUNDLE, id: 'Bad Id' })));
383- assert.throws(() => parseBundle(JSON.stringify({ ...TEST_BUNDLE, files: [] })));
384- assert.throws(() => parseBundle(JSON.stringify({
385- ...TEST_BUNDLE,
386- files: [{ name: 'other.m', content: 'x = 1;' }],
387- })), /render_note/);
388- assert.throws(() => parseBundle(JSON.stringify({
389- ...TEST_BUNDLE,
390- knobs: [{ key: 'k', label: 'K', min: 5, max: 1, default: 2 }],
391- })));
378+test('every builtin id used by the GM mapping exists', () => {
379+ const ids = new Set(builtinInstruments.map((i) => i.id));
380+ for (let p = 0; p < 128; p++) {
381+ assert.ok(ids.has(gmProgramInstrument(p)), `program ${p} maps to missing instrument`);
382+ }
383+ assert.equal(gmProgramInstrument(0), 'piano');
384+ assert.equal(gmProgramInstrument(33), 'synthbass');
385+ assert.equal(gmProgramInstrument(48), 'strings');
386+ assert.equal(gmProgramInstrument(56), 'brass');
387+ assert.equal(gmProgramInstrument(undefined), 'piano');
392388 });
393389
394390 console.log(`\n${passed} passed, ${failed} failed`);
scripts/test-sim.tsdeleted+0−148View file
@@ -1,148 +0,0 @@
1-/**
2- * Integration test for the exported-instrument path: real committed
3- * bundles, real numbl execution in node (no browser, no worker). Renders
4- * woodblock strikes and bass plucks at the preview tier through the same
5- * renderNoteBatch the worker uses, and checks the audio is shaped right,
6- * including that velocity actually changes the result and that the
7- * recorder's prepare stage feeds its fingering chart through. Run with
8- * `npm run test:sim`; slower than npm test because it runs the physics.
9- */
10-import { readFileSync, existsSync } from 'node:fs';
11-import { join } from 'node:path';
12-import { parseBundle, type InstrumentBundle } from '../src/instruments/numbl/bundle.ts';
13-import { renderNoteBatch } from '../src/instruments/numbl/render.ts';
14-
15-const ROOT = join(import.meta.dirname, '../../../simulated_instruments');
16-
17-function loadBundle(repo: string, id: string): InstrumentBundle {
18- return parseBundle(readFileSync(join(ROOT, repo, 'exports', `${id}.json`), 'utf8'));
19-}
20-
21-let failures = 0;
22-function check(cond: boolean, msg: string): void {
23- console.log(`${cond ? 'ok ' : 'FAIL'} ${msg}`);
24- if (!cond) failures++;
25-}
26-
27-function peak(x: Float32Array, from = 0, to = x.length): number {
28- let p = 0;
29- for (let i = from; i < Math.min(to, x.length); i++) p = Math.max(p, Math.abs(x[i]));
30- return p;
31-}
32-
33-function finite(x: Float32Array): boolean {
34- for (let i = 0; i < x.length; i++) if (!Number.isFinite(x[i])) return false;
35- return true;
36-}
37-
38-/** Goertzel magnitude at one frequency over a window into the ring. */
39-function goertzel(y: Float32Array, from: number, len: number, f: number, sr: number): number {
40- const w = (2 * Math.PI * f) / sr;
41- const c = 2 * Math.cos(w);
42- let s1 = 0, s2 = 0;
43- for (let i = from; i < from + len && i < y.length; i++) {
44- const s0 = y[i] + c * s1 - s2;
45- s2 = s1;
46- s1 = s0;
47- }
48- return Math.sqrt(s1 * s1 + s2 * s2 - c * s1 * s2);
49-}
50-
51-/** The strongest spectral peak in [lo, hi], 2 Hz grid, past the attack. */
52-function peakFreq(y: Float32Array, sr: number, lo: number, hi: number): number {
53- const from = Math.floor(y.length * 0.3);
54- const len = Math.min(8192, y.length - from);
55- let best = lo, bm = -1;
56- for (let f = lo; f <= hi; f += 2) {
57- const m = goertzel(y, from, len, f, sr);
58- if (m > bm) { bm = m; best = f; }
59- }
60- return best;
61-}
62-
63-if (!existsSync(ROOT)) {
64- console.log(`skipped: ${ROOT} not found`);
65- process.exit(0);
66-}
67-
68-const SR = 44100;
69-
70-{
71- const b = loadBundle('woodblock', 'woodblock');
72- const t0 = performance.now();
73- const [soft, hard] = renderNoteBatch(b, [
74- { pitch: 76, velocity: 0.25, durationSec: 0.25 },
75- { pitch: 76, velocity: 1.0, durationSec: 0.25 },
76- ], { sampleRate: SR, quality: 'preview', settings: {} });
77- console.log(`woodblock: two strikes in ${((performance.now() - t0) / 1000).toFixed(1)} s`);
78- check(soft.length > SR * 0.2, `woodblock trace has audio length (${soft.length})`);
79- check(finite(soft) && finite(hard), 'woodblock samples finite');
80- check(peak(soft) > 0, 'woodblock soft strike sounds');
81- check(peak(hard) > peak(soft) * 1.5, `harder mallet is louder (${peak(hard).toExponential(2)} vs ${peak(soft).toExponential(2)})`);
82- const [moved] = renderNoteBatch(b, [{ pitch: 76, velocity: 1.0, durationSec: 0.25 }], {
83- sampleRate: SR, quality: 'preview', settings: { strike_position: 0.25 },
84- });
85- let differs = moved.length !== hard.length;
86- for (let i = 0; i < Math.min(moved.length, hard.length) && !differs; i++) if (moved[i] !== hard[i]) differs = true;
87- check(differs, 'strike position knob changes the sound');
88- // Pitch by geometric scaling: an octave down halves the dominant mode.
89- const [lo, hi] = renderNoteBatch(b, [
90- { pitch: 64, velocity: 0.8, durationSec: 0.25 },
91- { pitch: 76, velocity: 0.8, durationSec: 0.25 },
92- ], { sampleRate: SR, quality: 'preview', settings: {} });
93- const fLo = peakFreq(lo, SR, 300, 4000);
94- const fHi = peakFreq(hi, SR, 600, 8000);
95- const ratio = fHi / fLo;
96- check(ratio > 1.94 && ratio < 2.06, `octave apart in pitch is an octave apart in frequency (${fLo} Hz -> ${fHi} Hz)`);
97-}
98-
99-{
100- const b = loadBundle('bass', 'bass');
101- const t0 = performance.now();
102- const [y] = renderNoteBatch(b, [{ pitch: 33, velocity: 0.7, durationSec: 0.5 }], {
103- sampleRate: SR, quality: 'preview', settings: { tail_s: 2.0 },
104- });
105- console.log(`bass: one pluck in ${((performance.now() - t0) / 1000).toFixed(1)} s`);
106- check(finite(y), 'bass samples finite');
107- check(peak(y) > 0, 'bass pluck sounds');
108- check(Math.abs(y.length - SR * 2.0) < SR * 0.2, `bass trace spans the tail (${y.length} samples)`);
109- check(peak(y, 0, Math.round(SR * 0.5)) > peak(y, Math.round(SR * 1.5)), 'bass pluck decays');
110- // Determinism: fixed seed, same request, same samples.
111- const [y2] = renderNoteBatch(b, [{ pitch: 33, velocity: 0.7, durationSec: 0.5 }], {
112- sampleRate: SR, quality: 'preview', settings: { tail_s: 2.0 },
113- });
114- let same = y.length === y2.length;
115- for (let i = 0; i < y.length && same; i++) if (y[i] !== y2[i]) same = false;
116- check(same, 'bass render is deterministic');
117- // The muted variant of the same repo is a distinct bundle with its own driver.
118- const muted = loadBundle('bass', 'bass-muted');
119- const [ym] = renderNoteBatch(muted, [{ pitch: 33, velocity: 0.7, durationSec: 0.5 }], {
120- sampleRate: SR, quality: 'preview', settings: {},
121- });
122- check(finite(ym) && peak(ym) > 0, 'muted bass sounds');
123- check(ym.length < y.length, `muted bass rings shorter (${ym.length} vs ${y.length} samples)`);
124-}
125-
126-{
127- // The recorder exercises the prepare stage (fingering chart in state)
128- // at a low sample rate to keep the jet integration quick.
129- const b = loadBundle('recorder', 'recorder');
130- const t0 = performance.now();
131- const RSR = 22050;
132- const [g5, c6, g4] = renderNoteBatch(b, [
133- { pitch: 79, velocity: 0.6, durationSec: 0.5 },
134- { pitch: 84, velocity: 0.6, durationSec: 0.5 },
135- { pitch: 67, velocity: 0.6, durationSec: 0.5 }, // below the chart: folds up an octave
136- ], { sampleRate: RSR, quality: 'preview', settings: {} });
137- console.log(`recorder: three notes in ${((performance.now() - t0) / 1000).toFixed(1)} s`);
138- check(finite(g5) && peak(g5) > 0, 'recorder note sounds (prepare state reached the driver)');
139- const f79 = peakFreq(g5, RSR, 400, 2000);
140- const f84 = peakFreq(c6, RSR, 400, 2000);
141- const f67 = peakFreq(g4, RSR, 400, 2000);
142- check(Math.abs(f79 - 784) < 25, `pitch 79 locks near G5 (${f79} Hz)`);
143- check(Math.abs(f84 - 1047) < 35, `pitch 84 locks near C6 (${f84} Hz)`);
144- check(Math.abs(f67 - f79) < 10, `below-range pitch folds up an octave (${f67} Hz vs ${f79} Hz)`);
145-}
146-
147-console.log(failures === 0 ? 'exported-instrument test passed' : `${failures} failure(s)`);
148-process.exit(failures > 0 ? 1 : 0);
src/audio/player.tsmodified+69−11View file
@@ -51,6 +51,9 @@ class Player {
5151 private lastPlay: { song: Song; opts: PlayOpts } | null = null;
5252 metronome = false;
5353 preparing = false;
54+ /** Per-track render progress while preparing, keyed by track id. */
55+ renderProgress = new Map<string, { frac: number; stage: string }>();
56+ private prepareAbort: AbortController | null = null;
5457
5558 subscribe = (cb: () => void): (() => void) => {
5659 this.listeners.add(cb);
@@ -81,15 +84,42 @@ class Player {
8184 private async prepareTracks(song: Song, sampleRate: number): Promise<void> {
8285 const ids = new Set(song.tracks.map((t) => t.id));
8386 for (const id of [...this.cache.keys()]) if (!ids.has(id)) this.cache.delete(id);
84- for (const track of song.tracks) {
85- const key = trackRenderKey(song, track) + '@' + sampleRate;
86- const cached = this.cache.get(track.id);
87- if (cached && cached.key === key) continue;
88- const buffer = await renderTrackBuffer(song, track, getInstrument(track.instrumentId), sampleRate, 'preview');
89- this.cache.set(track.id, { key, buffer });
87+ this.prepareAbort = new AbortController();
88+ const signal = this.prepareAbort.signal;
89+ try {
90+ for (const track of song.tracks) {
91+ const key = trackRenderKey(song, track) + '@' + sampleRate;
92+ const cached = this.cache.get(track.id);
93+ if (cached && cached.key === key) continue;
94+ this.renderProgress.set(track.id, { frac: 0, stage: 'starting' });
95+ this.emit();
96+ const buffer = await renderTrackBuffer(song, track, getInstrument(track.instrumentId), sampleRate, 'preview', {
97+ signal,
98+ onProgress: (p) => {
99+ this.renderProgress.set(track.id, p);
100+ this.emit();
101+ },
102+ });
103+ this.renderProgress.delete(track.id);
104+ this.emit();
105+ this.cache.set(track.id, { key, buffer });
106+ }
107+ } finally {
108+ this.renderProgress.clear();
109+ this.prepareAbort = null;
110+ this.emit();
90111 }
91112 }
92113
114+ /** Abort the renders behind a pending play or export. */
115+ cancelRender(): void {
116+ this.prepareAbort?.abort();
117+ }
118+
119+ static isAbort(err: unknown): boolean {
120+ return err instanceof DOMException && err.name === 'AbortError';
121+ }
122+
93123 async play(song: Song, opts: PlayOpts = {}): Promise<void> {
94124 if (this.state !== 'stopped') this.stop();
95125 this.loop = !opts.record && opts.loop ? opts.loop : null;
@@ -100,6 +130,9 @@ class Player {
100130 this.emit();
101131 try {
102132 await this.prepareTracks(song, ctx.sampleRate);
133+ } catch (err) {
134+ if (Player.isAbort(err)) { this.preparing = false; this.emit(); return; }
135+ throw err;
103136 } finally {
104137 this.preparing = false;
105138 }
@@ -199,6 +232,7 @@ class Player {
199232 }
200233
201234 stop(): void {
235+ this.cancelRender();
202236 if (this.state === 'stopped') return;
203237 this.posSec = Math.max(0, this.getPositionSec());
204238 for (const src of this.sources) {
@@ -237,12 +271,36 @@ class Player {
237271 }
238272
239273 async exportWav(song: Song): Promise<Uint8Array> {
240- const buffers = new Map<string, Float32Array>();
241- for (const track of song.tracks) {
242- buffers.set(track.id, await renderTrackBuffer(song, track, getInstrument(track.instrumentId), EXPORT_SAMPLE_RATE, 'full'));
274+ // Full-quality renders bypass the preview cache; progress is reported
275+ // the same way playback preparation is, so the UI shows it and the
276+ // same cancel applies.
277+ this.prepareAbort = new AbortController();
278+ const signal = this.prepareAbort.signal;
279+ this.preparing = true;
280+ this.emit();
281+ try {
282+ const buffers = new Map<string, Float32Array>();
283+ for (const track of song.tracks) {
284+ this.renderProgress.set(track.id, { frac: 0, stage: 'starting' });
285+ this.emit();
286+ buffers.set(track.id, await renderTrackBuffer(song, track, getInstrument(track.instrumentId), EXPORT_SAMPLE_RATE, 'full', {
287+ signal,
288+ onProgress: (p) => {
289+ this.renderProgress.set(track.id, p);
290+ this.emit();
291+ },
292+ }));
293+ this.renderProgress.delete(track.id);
294+ this.emit();
295+ }
296+ const mix = mixTracks(song.tracks, buffers);
297+ return encodeWavMono16(mix, EXPORT_SAMPLE_RATE);
298+ } finally {
299+ this.renderProgress.clear();
300+ this.prepareAbort = null;
301+ this.preparing = false;
302+ this.emit();
243303 }
244- const mix = mixTracks(song.tracks, buffers);
245- return encodeWavMono16(mix, EXPORT_SAMPLE_RATE);
246304 }
247305 }
248306
src/audio/render.tsmodified+8−3View file
@@ -1,5 +1,5 @@
11 import type { Song, Track } from '../types.ts';
2-import type { Instrument, NoteSpec } from '../instruments/types.ts';
2+import type { Instrument, NoteSpec, RenderProgress } from '../instruments/types.ts';
33 import { TempoMap } from '../midi/time.ts';
44
55 // Offline rendering: convert a track's notes to seconds under the tempo
@@ -21,10 +21,15 @@ export function noteSpecsForTrack(song: Song, track: Track): { notes: NoteSpec[]
2121 return { notes, lengthSec: lengthSec + RENDER_TAIL_SEC };
2222 }
2323
24-export async function renderTrackBuffer(song: Song, track: Track, instrument: Instrument, sampleRate: number, quality: 'preview' | 'full' = 'preview'): Promise<Float32Array> {
24+export interface RenderTrackOpts {
25+ signal?: AbortSignal;
26+ onProgress?: (p: RenderProgress) => void;
27+}
28+
29+export async function renderTrackBuffer(song: Song, track: Track, instrument: Instrument, sampleRate: number, quality: 'preview' | 'full' = 'preview', opts: RenderTrackOpts = {}): Promise<Float32Array> {
2530 const { notes, lengthSec } = noteSpecsForTrack(song, track);
2631 const lengthSamples = Math.max(1, Math.ceil(lengthSec * sampleRate));
27- const buf = await instrument.render({ notes, sampleRate, lengthSamples, quality, settings: track.settings ?? {} });
32+ const { audio: buf } = await instrument.renderTrack({ notes, sampleRate, lengthSamples, quality, settings: track.settings ?? {}, signal: opts.signal, onProgress: opts.onProgress });
2833 // Level each track to a common peak so mixing is predictable.
2934 let peak = 0;
3035 for (let i = 0; i < buf.length; i++) peak = Math.max(peak, Math.abs(buf[i]));
src/components/TrackList.tsxmodified+16−55View file
@@ -1,61 +1,9 @@
1-import { useRef } from 'react';
2-import { store, useApp } from '../state/store.ts';
1+import { store, useApp, usePlayerVersion } from '../state/store.ts';
2+import { player } from '../audio/player.ts';
33 import { getInstrument, listInstruments } from '../instruments/registry.ts';
44 import type { KnobSpec } from '../instruments/types.ts';
5-import { importInstrumentText, listImportedIds, removeImportedInstrument } from '../instruments/numbl/adapter.ts';
65 import { trackColor } from './colors.ts';
76
8-// Imported instrument bundles: a file picker that accepts exported
9-// numbl-instrument-v1 JSON, and a chip per import so a stale one can be
10-// removed (a shadowed catalog instrument of the same id comes back).
11-function ImportedInstruments() {
12- const fileRef = useRef<HTMLInputElement>(null);
13- const imported = listImportedIds();
14- const onFiles = async (files: FileList | null) => {
15- if (!files) return;
16- for (const f of files) {
17- try {
18- const bundle = await importInstrumentText(await f.text());
19- store.setStatus(`imported instrument '${bundle.name}'`);
20- } catch (err) {
21- store.setStatus(err instanceof Error ? err.message : String(err));
22- }
23- }
24- store.bumpInstruments();
25- };
26- return (
27- <div className="importinstr">
28- <button
29- className="addtrack"
30- title="Import an exported instrument bundle (numbl-instrument-v1 JSON)"
31- onClick={() => fileRef.current?.click()}
32- >+ Import instrument</button>
33- <input
34- ref={fileRef}
35- type="file"
36- accept=".json,application/json"
37- multiple
38- style={{ display: 'none' }}
39- onChange={(e) => { void onFiles(e.target.files); e.target.value = ''; }}
40- />
41- {imported.length > 0 && (
42- <div className="importedlist">
43- {imported.map((id) => (
44- <span key={id} className="importedchip">
45- {id}
46- <button
47- className="mini danger"
48- title="Remove imported instrument"
49- onClick={() => { void removeImportedInstrument(id).then(() => store.bumpInstruments()); }}
50- >✕</button>
51- </span>
52- ))}
53- </div>
54- )}
55- </div>
56- );
57-}
58-
597 function fmtKnob(v: number): string {
608 const a = Math.abs(v);
619 if (a >= 1e5 || (a > 0 && a < 0.01)) return v.toExponential(1);
@@ -88,6 +36,7 @@ function KnobSlider({ knob, value, onChange }: { knob: KnobSpec; value: number |
8836
8937 export function TrackList() {
9038 const app = useApp();
39+ usePlayerVersion();
9140 const song = app.rec?.song ?? null;
9241 if (!song) return <div className="tracklist" />;
9342 const instruments = listInstruments();
@@ -155,6 +104,19 @@ export function TrackList() {
155104 ))}
156105 </div>
157106 )}
107+ {player.renderProgress.has(t.id) && (
108+ <div className="trackrow-render" title="This track is rendering">
109+ <span className="renderbar">
110+ <span className="renderbar-fill" style={{ width: `${Math.round((player.renderProgress.get(t.id)?.frac ?? 0) * 100)}%` }} />
111+ </span>
112+ <span className="renderstage">{player.renderProgress.get(t.id)?.stage}</span>
113+ <button
114+ className="mini danger"
115+ title="Cancel rendering"
116+ onClick={(e) => { e.stopPropagation(); player.cancelRender(); }}
117+ >✕</button>
118+ </div>
119+ )}
158120 <div className="trackrow-vol">
159121 <input
160122 type="range"
@@ -172,7 +134,6 @@ export function TrackList() {
172134 );
173135 })}
174136 <button className="addtrack" onClick={() => store.addTrack()}>+ Add track</button>
175- <ImportedInstruments />
176137 </div>
177138 );
178139 }
src/components/Transport.tsxmodified+13−1View file
@@ -51,7 +51,19 @@ export function Transport() {
5151 </button>
5252 <span className="pos" title="bar.beat">{bar}.{beat}</span>
5353 <span className="pos sec">{mm}:{ss}</span>
54- {player.preparing && <span className="status">rendering…</span>}
54+ {player.preparing && (
55+ <span className="status">
56+ {(() => {
57+ const entries = [...player.renderProgress.entries()];
58+ if (entries.length === 0) return 'rendering…';
59+ const [id, p] = entries[entries.length - 1];
60+ const name = song.tracks.find((t) => t.id === id)?.name ?? 'track';
61+ return `rendering ${name}: ${p.stage} ${Math.round(p.frac * 100)}%`;
62+ })()}
63+ {' '}
64+ <button title="Cancel rendering" onClick={() => player.cancelRender()}>✕</button>
65+ </span>
66+ )}
5567 <span className="spacer" />
5668 <label title={singleTempo ? 'Tempo (bpm)' : 'This song has a tempo map; editing it as one bpm is not supported yet'}>
5769 bpm{' '}
src/instruments/builtins.tsmodified+23−8View file
@@ -1,5 +1,5 @@
1-import type { Instrument, LiveVoice, NoteSpec, RenderRequest } from './types.ts';
2-import { drumNote, epianoNote, leadNote, pluckNote } from './dsp.ts';
1+import type { Instrument, LiveVoice, NoteSpec, TrackRenderRequest, TrackRenderResult } from './types.ts';
2+import { brassNote, drumNote, epianoNote, leadNote, organNote, pianoNote, pluckNote, stringsNote, synthBassNote } from './dsp.ts';
33
44 // Built-in instruments, each constructed from a pure per-note DSP function.
55 // The render path sums per-note buffers; the live path plays one note's
@@ -18,15 +18,23 @@ export function instrumentFromNoteDsp(id: string, name: string, dsp: NoteDsp, op
1818 return {
1919 id,
2020 name,
21- render(req: RenderRequest): Float32Array {
22- const out = new Float32Array(req.lengthSamples);
21+ renderTrack(req: TrackRenderRequest): Promise<TrackRenderResult> {
22+ // lengthSamples is a minimum: a release that outlives the last note
23+ // is kept rather than clipped.
24+ const rendered: { off: number; buf: Float32Array }[] = [];
25+ let length = req.lengthSamples;
2326 for (const note of req.notes) {
2427 const buf = dsp(req.sampleRate, note.pitch, note.velocity, note.durationSec);
2528 const off = Math.round(note.timeSec * req.sampleRate);
26- const end = Math.min(out.length, off + buf.length);
27- for (let i = Math.max(0, off); i < end; i++) out[i] += buf[i - off];
29+ rendered.push({ off, buf });
30+ length = Math.max(length, off + buf.length);
2831 }
29- return out;
32+ const out = new Float32Array(length);
33+ for (const r of rendered) {
34+ const end = Math.min(out.length, r.off + r.buf.length);
35+ for (let i = Math.max(0, r.off); i < end; i++) out[i] += r.buf[i - r.off];
36+ }
37+ return Promise.resolve({ audio: out });
3038 },
3139 noteOn(ctx, dest, pitch, velocity, whenSec): LiveVoice {
3240 const dur = opts.sustains ? (opts.liveDurationSec ?? 20) : 0.25;
@@ -57,9 +65,16 @@ export function instrumentFromNoteDsp(id: string, name: string, dsp: NoteDsp, op
5765 };
5866 }
5967
68+// The cheap synthetic set: one instrument per broad General MIDI family,
69+// so an imported MIDI file is playable without a soundfont.
6070 export const builtinInstruments: Instrument[] = [
61- instrumentFromNoteDsp('pluck', 'Pluck (Karplus-Strong)', pluckNote, { sustains: true, liveDurationSec: 3 }),
71+ instrumentFromNoteDsp('piano', 'Piano (FM)', pianoNote, { sustains: true, liveDurationSec: 10 }),
6272 instrumentFromNoteDsp('epiano', 'E-Piano (FM)', epianoNote, { sustains: true, liveDurationSec: 8 }),
73+ instrumentFromNoteDsp('organ', 'Organ (drawbar)', organNote, { sustains: true, liveDurationSec: 20 }),
74+ instrumentFromNoteDsp('pluck', 'Pluck (Karplus-Strong)', pluckNote, { sustains: true, liveDurationSec: 3 }),
75+ instrumentFromNoteDsp('synthbass', 'Synth Bass', synthBassNote, { sustains: true, liveDurationSec: 6 }),
76+ instrumentFromNoteDsp('strings', 'Strings (pad)', stringsNote, { sustains: true, liveDurationSec: 20 }),
77+ instrumentFromNoteDsp('brass', 'Brass (saw)', brassNote, { sustains: true, liveDurationSec: 20 }),
6378 instrumentFromNoteDsp('lead', 'Lead (square)', leadNote, { sustains: true, liveDurationSec: 20 }),
6479 instrumentFromNoteDsp('drumkit', 'Drum kit (GM)', drumNote, { sustains: false }),
6580 ];
src/instruments/dsp.tsmodified+154−0View file
@@ -163,3 +163,157 @@ function hp(buf: Float32Array): Float32Array {
163163 function scale(buf: Float32Array, s: number) {
164164 for (let i = 0; i < buf.length; i++) buf[i] *= s;
165165 }
166+
167+// FM piano: two slightly detuned carriers with an inharmonic modulator and
168+// a hammer transient; decay time falls with pitch the way a real string's
169+// does, and the release gates the tail.
170+export function pianoNote(sr: number, pitch: number, velocity: number, durationSec: number): Float32Array {
171+ const f0 = midiToFreq(pitch);
172+ const decay = Math.max(0.6, 5.5 - pitch * 0.045); // seconds to fall ~e^-1
173+ const tail = Math.min(2.5, decay);
174+ const n = Math.floor(sr * (durationSec + tail));
175+ const out = new Float32Array(n);
176+ const wc = (2 * Math.PI * f0) / sr;
177+ const wm = wc * 3.001; // slightly inharmonic upper partials
178+ const relStart = durationSec * sr;
179+ for (let i = 0; i < n; i++) {
180+ const t = i / sr;
181+ const idxEnv = (0.6 + 2.2 * velocity) * Math.exp(-t * 7);
182+ let env = Math.min(1, i / (0.0015 * sr)) * Math.exp(-t / decay);
183+ if (i > relStart) env *= Math.exp(-(i - relStart) / (0.09 * sr));
184+ const a = Math.sin(wc * i + idxEnv * Math.sin(wm * i));
185+ const b = Math.sin(wc * 1.0007 * i + 0.5 * idxEnv * Math.sin(2 * wc * i));
186+ out[i] = env * (0.7 * a + 0.3 * b);
187+ }
188+ scale(out, 0.5 * velocity);
189+ return out;
190+}
191+
192+// Drawbar organ: a fixed set of additive partials at classic drawbar
193+// footages, held at constant level while the key is down, with a small
194+// percussive click and a quick release. Velocity moves level only a
195+// little, as on the real instrument.
196+export function organNote(sr: number, pitch: number, velocity: number, durationSec: number): Float32Array {
197+ const f0 = midiToFreq(pitch);
198+ const tail = 0.15;
199+ const n = Math.floor(sr * (durationSec + tail));
200+ const out = new Float32Array(n);
201+ const nyq = sr / 2;
202+ // ratios and levels: 16', 8', 5 1/3', 4', 2 2/3', 2'
203+ const ratios = [0.5, 1, 1.5, 2, 3, 4];
204+ const levels = [0.35, 1, 0.45, 0.55, 0.3, 0.25];
205+ const relStart = durationSec * sr;
206+ const vib = 2 * Math.PI * 6.3;
207+ for (let i = 0; i < n; i++) {
208+ const t = i / sr;
209+ const fmod = 1 + 0.0018 * Math.sin(vib * t); // light chorus/vibrato
210+ let x = 0;
211+ for (let k = 0; k < ratios.length; k++) {
212+ const f = f0 * ratios[k];
213+ if (f > nyq * 0.9) break;
214+ x += levels[k] * Math.sin(((2 * Math.PI * f * fmod) / sr) * i);
215+ }
216+ const click = Math.exp(-t / 0.004) * 0.6 * Math.sin(((2 * Math.PI * f0 * 4) / sr) * i);
217+ let env = Math.min(1, i / (0.003 * sr));
218+ if (i > relStart) env *= Math.exp(-(i - relStart) / (0.025 * sr));
219+ out[i] = env * (x + click);
220+ }
221+ scale(out, 0.3 * (0.6 + 0.4 * velocity));
222+ return out;
223+}
224+
225+// String ensemble pad: three detuned sawtooth voices through a gentle
226+// lowpass, with the slow attack and slow release that make it a pad.
227+export function stringsNote(sr: number, pitch: number, velocity: number, durationSec: number): Float32Array {
228+ const f0 = midiToFreq(pitch);
229+ const tail = 0.6;
230+ const n = Math.floor(sr * (durationSec + tail));
231+ const out = new Float32Array(n);
232+ const nyq = sr / 2;
233+ const detunes = [0.9965, 1, 1.0042];
234+ const rng = mulberry32(0x51ab17 ^ pitch);
235+ const phases = detunes.map(() => rng() * 2 * Math.PI);
236+ let y = 0;
237+ const relStart = durationSec * sr;
238+ for (let i = 0; i < n; i++) {
239+ const t = i / sr;
240+ let x = 0;
241+ for (let v = 0; v < detunes.length; v++) {
242+ const f = f0 * detunes[v];
243+ // band-limited-ish saw: first 10 harmonics
244+ for (let k = 1; k <= 10; k++) {
245+ if (f * k > nyq * 0.9) break;
246+ x += Math.sin(((2 * Math.PI * f * k) / sr) * i + phases[v] * k) / k;
247+ }
248+ }
249+ const a = 1 - Math.exp((-2 * Math.PI * (900 + 900 * velocity)) / sr);
250+ y += a * (x - y);
251+ let env = 1 - Math.exp(-t / 0.15); // slow bow-in
252+ if (i > relStart) env *= Math.exp(-(i - relStart) / (0.3 * sr));
253+ out[i] = y * env;
254+ }
255+ scale(out, 0.28 * velocity);
256+ return out;
257+}
258+
259+// Synth bass: square plus a sub-octave sine with a snappy filter
260+// envelope; velocity opens the filter.
261+export function synthBassNote(sr: number, pitch: number, velocity: number, durationSec: number): Float32Array {
262+ const f0 = midiToFreq(pitch);
263+ const tail = 0.2;
264+ const n = Math.floor(sr * (durationSec + tail));
265+ const out = new Float32Array(n);
266+ const nyq = sr / 2;
267+ let y = 0;
268+ const relStart = durationSec * sr;
269+ for (let i = 0; i < n; i++) {
270+ const t = i / sr;
271+ let x = 0;
272+ for (let k = 1; k <= 9; k += 2) {
273+ if (f0 * k > nyq * 0.9) break;
274+ x += Math.sin(((2 * Math.PI * f0 * k) / sr) * i) / k;
275+ }
276+ x += 0.6 * Math.sin(((Math.PI * f0) / sr) * i); // sub-octave
277+ const cutoff = 80 + (400 + 2600 * velocity) * Math.exp(-t * 9);
278+ const a = 1 - Math.exp((-2 * Math.PI * cutoff) / sr);
279+ y += a * (x - y);
280+ const atk = Math.min(1, i / (0.002 * sr));
281+ let env = atk * (0.75 + 0.25 * Math.exp(-t * 5));
282+ if (i > relStart) env *= Math.exp(-(i - relStart) / (0.04 * sr));
283+ out[i] = y * env;
284+ }
285+ scale(out, 0.5 * velocity);
286+ return out;
287+}
288+
289+// Brass section: a sawtooth whose lowpass opens over the first tenth of a
290+// second (the brass "blip"), light vibrato arriving late, medium release.
291+export function brassNote(sr: number, pitch: number, velocity: number, durationSec: number): Float32Array {
292+ const f0 = midiToFreq(pitch);
293+ const tail = 0.3;
294+ const n = Math.floor(sr * (durationSec + tail));
295+ const out = new Float32Array(n);
296+ const nyq = sr / 2;
297+ let y = 0;
298+ const relStart = durationSec * sr;
299+ for (let i = 0; i < n; i++) {
300+ const t = i / sr;
301+ const vibDepth = 0.004 * Math.min(1, Math.max(0, (t - 0.25) / 0.4));
302+ const fmod = 1 + vibDepth * Math.sin(2 * Math.PI * 5.5 * t);
303+ let x = 0;
304+ for (let k = 1; k <= 12; k++) {
305+ if (f0 * k > nyq * 0.9) break;
306+ x += Math.sin(((2 * Math.PI * f0 * fmod * k) / sr) * i) / k;
307+ }
308+ const open = 1 - Math.exp(-t / 0.09);
309+ const cutoff = 300 + (1200 + 3800 * velocity) * open;
310+ const a = 1 - Math.exp((-2 * Math.PI * cutoff) / sr);
311+ y += a * (x - y);
312+ const atk = Math.min(1, i / (0.012 * sr));
313+ let env = atk;
314+ if (i > relStart) env *= Math.exp(-(i - relStart) / (0.08 * sr));
315+ out[i] = y * env;
316+ }
317+ scale(out, 0.4 * velocity);
318+ return out;
319+}
src/instruments/numbl/adapter.tsdeleted+0−134View file
@@ -1,134 +0,0 @@
1-/**
2- * Exported numbl instruments behind the generic Instrument interface:
3- * self-contained bundles (format numbl-instrument-v1, see
4- * simulated_instruments/instrument-format) whose own MATLAB-syntax
5- * drivers map each MIDI note onto the model and render it, executed by
6- * numbl in a worker and mixed per track. Bundles arrive two ways: the
7- * committed exports of the sibling simulated_instruments repos, inlined
8- * at build time, and files imported at runtime, persisted in IndexedDB.
9- * An imported bundle with a catalog bundle's id shadows it, which is how
10- * a newer export is dropped in without rebuilding the app. There is no
11- * live noteOn voice yet, so monitoring while recording falls back to the
12- * player's beep.
13- */
14-import type { Instrument, RenderRequest } from '../types.ts';
15-import { registerInstrument, unregisterInstrument } from '../registry.ts';
16-import { loadCatalog } from './catalog.ts';
17-import { parseBundle, type InstrumentBundle } from './bundle.ts';
18-import * as db from '../../storage/db.ts';
19-import type { WorkerRenderMsg, WorkerResponse } from './worker.ts';
20-
21-type StatusFn = (msg: string) => void;
22-let onStatus: StatusFn = () => {};
23-export function setNumblStatus(cb: StatusFn): void {
24- onStatus = cb;
25-}
26-
27-let worker: Worker | null = null;
28-let nextId = 1;
29-const pending = new Map<number, { resolve: (v: Float32Array) => void; reject: (e: Error) => void }>();
30-
31-function ensureWorker(): Worker {
32- if (worker) return worker;
33- worker = new Worker(new URL('./worker.ts', import.meta.url), { type: 'module' });
34- worker.onmessage = (ev: MessageEvent<WorkerResponse>) => {
35- const msg = ev.data;
36- if (msg.kind === 'status') {
37- onStatus(msg.text);
38- return;
39- }
40- const p = pending.get(msg.id);
41- if (!p) return;
42- pending.delete(msg.id);
43- if (msg.kind === 'result') p.resolve(msg.samples);
44- else p.reject(new Error(msg.message));
45- };
46- worker.onerror = (ev) => {
47- const err = new Error(`numbl instrument worker failed: ${ev.message}`);
48- for (const p of pending.values()) p.reject(err);
49- pending.clear();
50- };
51- return worker;
52-}
53-
54-class NumblInstrument implements Instrument {
55- readonly id: string;
56- readonly name: string;
57- readonly knobs;
58-
59- constructor(readonly bundle: InstrumentBundle, imported: boolean) {
60- this.id = `nb-${bundle.id}`;
61- this.name = imported ? `${bundle.name} (imported)` : bundle.name;
62- this.knobs = bundle.knobs;
63- }
64-
65- render(req: RenderRequest): Promise<Float32Array> {
66- if (req.notes.length === 0) return Promise.resolve(new Float32Array(req.lengthSamples));
67- const w = ensureWorker();
68- const id = nextId++;
69- const msg: WorkerRenderMsg = {
70- kind: 'render',
71- id,
72- bundle: this.bundle,
73- notes: req.notes.map((n) => ({ pitch: n.pitch, velocity: n.velocity, durationSec: n.durationSec, timeSec: n.timeSec })),
74- sampleRate: req.sampleRate,
75- lengthSamples: req.lengthSamples,
76- quality: req.quality ?? 'preview',
77- settings: req.settings ?? {},
78- };
79- return new Promise<Float32Array>((resolve, reject) => {
80- pending.set(id, { resolve, reject });
81- w.postMessage(msg);
82- });
83- }
84-}
85-
86-const catalogById = new Map<string, InstrumentBundle>();
87-const importedIds = new Set<string>();
88-
89-export function registerCatalogInstruments(): number {
90- for (const bundle of loadCatalog()) {
91- catalogById.set(bundle.id, bundle);
92- registerInstrument(new NumblInstrument(bundle, false));
93- }
94- return catalogById.size;
95-}
96-
97-/** Re-register persisted imports at startup; call after the catalog. */
98-export async function loadImportedInstruments(): Promise<number> {
99- const recs = await db.listImportedInstruments();
100- let n = 0;
101- for (const rec of recs) {
102- try {
103- const bundle = parseBundle(rec.text);
104- registerInstrument(new NumblInstrument(bundle, true));
105- importedIds.add(bundle.id);
106- n++;
107- } catch (err) {
108- console.warn(`skipping imported instrument '${rec.id}': ${err instanceof Error ? err.message : String(err)}`);
109- }
110- }
111- return n;
112-}
113-
114-/** Validate, persist, and register one bundle file's text. */
115-export async function importInstrumentText(text: string): Promise<InstrumentBundle> {
116- const bundle = parseBundle(text);
117- await db.putImportedInstrument({ id: bundle.id, text, importedAt: Date.now() });
118- registerInstrument(new NumblInstrument(bundle, true));
119- importedIds.add(bundle.id);
120- return bundle;
121-}
122-
123-export function listImportedIds(): string[] {
124- return [...importedIds].sort();
125-}
126-
127-/** Remove an import; a shadowed catalog bundle of the same id comes back. */
128-export async function removeImportedInstrument(bundleId: string): Promise<void> {
129- await db.deleteImportedInstrument(bundleId);
130- importedIds.delete(bundleId);
131- const cat = catalogById.get(bundleId);
132- if (cat) registerInstrument(new NumblInstrument(cat, false));
133- else unregisterInstrument(`nb-${bundleId}`);
134-}
src/instruments/numbl/bundle.tsdeleted+0−87View file
@@ -1,87 +0,0 @@
1-// Vendored from simulated_instruments/instrument-format/ts/bundle.ts, the
2-// format's canonical definition; keep the two in sync.
3-
4-export interface BundleKnob {
5- key: string;
6- label: string;
7- min: number;
8- max: number;
9- default: number;
10- step?: number;
11- scale?: 'log';
12-}
13-
14-export interface BundleFile {
15- name: string; // e.g. "render_note.m"; unique within the bundle
16- content: string; // full source, MATLAB syntax, executed by numbl
17-}
18-
19-export interface InstrumentBundle {
20- format: 'numbl-instrument-v1';
21- /** Stable identifier, lowercase letters, digits, hyphens. Hosts key
22- * caches and track references on it, so it should not change between
23- * versions of the same instrument. */
24- id: string;
25- /** Display name. */
26- name: string;
27- description?: string;
28- /** Provenance, e.g. "simulated_instruments/bass". Informational. */
29- source?: string;
30- /** Curated user-editable parameters. Hosts clamp values to [min, max]
31- * and fill missing ones with the default before every call. */
32- knobs: BundleKnob[];
33- /** Every .m source the drivers need, drivers included. Must contain
34- * render_note.m; prepare.m is optional. */
35- files: BundleFile[];
36-}
37-
38-const ID_RE = /^[a-z0-9][a-z0-9-]*$/;
39-
40-function fail(id: string, msg: string): never {
41- throw new Error(`instrument bundle ${id}: ${msg}`);
42-}
43-
44-export function parseBundle(text: string): InstrumentBundle {
45- let j: unknown;
46- try {
47- j = JSON.parse(text);
48- } catch (err) {
49- throw new Error(`instrument bundle is not valid JSON: ${err instanceof Error ? err.message : String(err)}`);
50- }
51- const b = j as InstrumentBundle;
52- const id = typeof b.id === 'string' ? b.id : '(no id)';
53- if (b.format !== 'numbl-instrument-v1') fail(id, `unknown format '${String(b.format)}'`);
54- if (typeof b.id !== 'string' || !ID_RE.test(b.id)) fail(id, `'id' must match ${ID_RE}`);
55- if (typeof b.name !== 'string' || b.name === '') fail(id, `missing string field 'name'`);
56- for (const k of ['description', 'source'] as const) {
57- if (b[k] !== undefined && typeof b[k] !== 'string') fail(id, `'${k}' must be a string when present`);
58- }
59- if (!Array.isArray(b.knobs)) fail(id, `missing 'knobs' array`);
60- for (const k of b.knobs) {
61- if (typeof k.key !== 'string' || typeof k.label !== 'string') fail(id, `knobs need key and label`);
62- if (typeof k.min !== 'number' || typeof k.max !== 'number' || typeof k.default !== 'number' || !(k.min < k.max)) {
63- fail(id, `knob '${k.key}' needs min < max and a default`);
64- }
65- if (k.default < k.min || k.default > k.max) fail(id, `knob '${k.key}' default outside its range`);
66- if (k.step !== undefined && (typeof k.step !== 'number' || !(k.step > 0))) fail(id, `knob '${k.key}' step must be > 0`);
67- if (k.scale !== undefined && k.scale !== 'log') fail(id, `knob '${k.key}' has unknown scale`);
68- if (k.scale === 'log' && k.min <= 0) fail(id, `knob '${k.key}' log scale needs min > 0`);
69- }
70- if (!Array.isArray(b.files) || b.files.length === 0) fail(id, `missing 'files' array`);
71- const seen = new Set<string>();
72- for (const f of b.files) {
73- if (typeof f.name !== 'string' || !f.name.endsWith('.m') || f.name.includes('/')) {
74- fail(id, `file '${String(f.name)}' must be a bare .m filename`);
75- }
76- if (typeof f.content !== 'string' || f.content === '') fail(id, `file '${f.name}' has no content`);
77- if (seen.has(f.name)) fail(id, `duplicate file '${f.name}'`);
78- seen.add(f.name);
79- }
80- if (!seen.has('render_note.m')) fail(id, `no render_note.m among the files`);
81- return b;
82-}
83-
84-/** Whether the bundle declares the optional prepare stage. */
85-export function hasPrepare(b: InstrumentBundle): boolean {
86- return b.files.some((f) => f.name === 'prepare.m');
87-}
src/instruments/numbl/catalog.tsdeleted+0−28View file
@@ -1,28 +0,0 @@
1-import { parseBundle, type InstrumentBundle } from './bundle.ts';
2-
3-/**
4- * Discovery of committed bundles: every
5- * ../simulated_instruments/<repo>/exports/*.json, inlined at build time.
6- * A bundle that fails validation is reported and skipped rather than
7- * taking the app down. Bundles imported at runtime live in imports.ts,
8- * not here.
9- */
10-
11-const bundleModules = import.meta.glob('../../../../../simulated_instruments/*/exports/*.json', {
12- eager: true,
13- query: '?raw',
14- import: 'default',
15-}) as Record<string, string>;
16-
17-export function loadCatalog(): InstrumentBundle[] {
18- const out: InstrumentBundle[] = [];
19- for (const [path, text] of Object.entries(bundleModules)) {
20- try {
21- out.push(parseBundle(text));
22- } catch (err) {
23- console.warn(`skipping ${path}: ${err instanceof Error ? err.message : String(err)}`);
24- }
25- }
26- out.sort((a, b) => a.name.localeCompare(b.name));
27- return out;
28-}
src/instruments/numbl/render.tsdeleted+0−100View file
@@ -1,100 +0,0 @@
1-import { executeCode } from 'numbl';
2-import { hasPrepare, type InstrumentBundle } from './bundle.ts';
3-
4-/**
5- * Rendering notes through a bundle's numbl drivers. A batch of notes
6- * becomes one generated program: the knob struct is built, prepare (if
7- * the bundle has one) runs once, and render_note runs per note, so
8- * pitch-independent setup is amortized across the batch. Pure with
9- * respect to the environment (no DOM, no worker), so the browser worker
10- * and the node tests share it.
11- */
12-
13-export interface BatchNote {
14- pitch: number; // MIDI note number
15- velocity: number; // 0..1
16- durationSec: number;
17-}
18-
19-export interface BatchOpts {
20- sampleRate: number;
21- quality: 'preview' | 'full';
22- settings: Record<string, number>; // knob values; unknown keys are ignored
23-}
24-
25-interface RuntimeTensorLike {
26- kind: 'tensor';
27- data: Float64Array;
28-}
29-
30-function lit(name: string, value: number): string {
31- if (!Number.isFinite(value)) throw new Error(`parameter ${name} is not finite`);
32- return String(value);
33-}
34-
35-/** The generated host program, exposed for tests. */
36-export function buildProgram(bundle: InstrumentBundle, notes: BatchNote[], opts: BatchOpts): string {
37- const preview = opts.quality === 'preview' ? 1 : 0;
38- const lines = [
39- 'params = struct();',
40- `params.sample_rate = ${lit('sample_rate', opts.sampleRate)};`,
41- `params.preview = ${preview};`,
42- ];
43- for (const k of bundle.knobs) {
44- const raw = opts.settings[k.key];
45- const v = raw === undefined ? k.default : Math.max(k.min, Math.min(k.max, raw));
46- lines.push(`params.${k.key} = ${lit(k.key, v)};`);
47- }
48- lines.push('note = struct();', 'note.params = params;');
49- if (hasPrepare(bundle)) lines.push('state = prepare(params);', 'note.state = state;');
50- lines.push(`note.sample_rate = ${lit('sample_rate', opts.sampleRate)};`, `note.preview = ${preview};`);
51- notes.forEach((n, i) => {
52- lines.push(
53- `note.pitch = ${lit('pitch', n.pitch)};`,
54- `note.velocity = ${lit('velocity', n.velocity)};`,
55- `note.duration_s = ${lit('duration_s', Math.max(1e-3, n.durationSec))};`,
56- `y_${i} = render_note(note);`,
57- );
58- });
59- return lines.join('\n');
60-}
61-
62-export function renderNoteBatch(bundle: InstrumentBundle, notes: BatchNote[], opts: BatchOpts): Float32Array[] {
63- if (notes.length === 0) return [];
64- const code = buildProgram(bundle, notes, opts);
65- const mfiles = bundle.files.map((f) => ({ name: f.name, source: f.content }));
66- const r = executeCode(code, { displayResults: false }, mfiles, 'midiboot_render.m');
67- return notes.map((_, i) => {
68- const y = r.variableValues[`y_${i}`] as unknown as RuntimeTensorLike;
69- if (!y || y.kind !== 'tensor') throw new Error(`${bundle.id}: render_note did not return a vector`);
70- const out = new Float32Array(y.data.length);
71- for (let j = 0; j < y.data.length; j++) out[j] = y.data[j];
72- return out;
73- });
74-}
75-
76-/** Velocity quantized to a few levels so identical notes share a cached
77- * render; 8 levels is inaudible next to the models' own dynamics. */
78-export function bucketVelocity(velocity01: number, levels = 8): number {
79- return Math.ceil(Math.max(0.001, Math.min(1, velocity01)) * levels) / levels;
80-}
81-
82-/** Duration quantized to 0.25 s steps. The host cannot know whether a
83- * driver reads duration, so it is always part of the cache key. */
84-export function bucketDuration(durationSec: number): number {
85- return Math.max(0.25, Math.round(durationSec / 0.25) * 0.25);
86-}
87-
88-/** Everything that identifies one cached note render. */
89-export function noteCacheKey(
90- bundle: InstrumentBundle,
91- quality: 'preview' | 'full',
92- sampleRate: number,
93- settings: Record<string, number>,
94- pitch: number,
95- velocityBucket: number,
96- durationBucket: number,
97-): string {
98- const s = bundle.knobs.map((k) => `${k.key}=${settings[k.key] ?? k.default}`).join(',');
99- return `${bundle.id}|${quality}|${sampleRate}|${s}|${pitch}|${velocityBucket}|${durationBucket}`;
100-}
src/instruments/numbl/worker.tsdeleted+0−107View file
@@ -1,107 +0,0 @@
1-/**
2- * The render worker: numbl runs here so a seconds-long eigensolve or jet
3- * integration never blocks the UI. One request renders one track through
4- * one bundle (sent with the request; the worker holds no catalog). Notes
5- * are grouped by cache key so each distinct sound renders once, cache
6- * misses go through renderNoteBatch in small chunks (prepare amortized,
7- * progress between chunks), and the mixed buffer is transferred back.
8- * The per-note cache is an LRU bounded by total samples.
9- */
10-import type { InstrumentBundle } from './bundle.ts';
11-import { bucketDuration, bucketVelocity, noteCacheKey, renderNoteBatch } from './render.ts';
12-
13-export interface WorkerNote {
14- pitch: number;
15- velocity: number; // 0..1
16- durationSec: number;
17- timeSec: number;
18-}
19-
20-export interface WorkerRenderMsg {
21- kind: 'render';
22- id: number;
23- bundle: InstrumentBundle;
24- notes: WorkerNote[];
25- sampleRate: number;
26- lengthSamples: number;
27- quality: 'preview' | 'full';
28- settings: Record<string, number>;
29-}
30-
31-export type WorkerResponse =
32- | { kind: 'result'; id: number; samples: Float32Array }
33- | { kind: 'error'; id: number; message: string }
34- | { kind: 'status'; text: string };
35-
36-const CACHE_MAX_SAMPLES = 48e6; // ~192 MB of Float32 note renders
37-const CHUNK = 4; // cache misses rendered per numbl program
38-const cache = new Map<string, Float32Array>();
39-let cacheSamples = 0;
40-
41-function cacheGet(key: string): Float32Array | undefined {
42- const v = cache.get(key);
43- if (v) {
44- cache.delete(key);
45- cache.set(key, v);
46- }
47- return v;
48-}
49-
50-function cachePut(key: string, buf: Float32Array): void {
51- cache.set(key, buf);
52- cacheSamples += buf.length;
53- while (cacheSamples > CACHE_MAX_SAMPLES && cache.size > 1) {
54- const oldest = cache.keys().next().value as string;
55- cacheSamples -= cache.get(oldest)!.length;
56- cache.delete(oldest);
57- }
58-}
59-
60-const post = (m: WorkerResponse, transfer?: Transferable[]) =>
61- (self as unknown as Worker).postMessage(m, transfer ?? []);
62-
63-self.onmessage = (ev: MessageEvent<WorkerRenderMsg>) => {
64- const msg = ev.data;
65- if (msg.kind !== 'render') return;
66- try {
67- const b = msg.bundle;
68- const out = new Float32Array(msg.lengthSamples);
69- // Group notes by cache key so each distinct sound renders once.
70- const groups = new Map<string, { velB: number; durB: number; pitch: number; times: number[] }>();
71- for (const n of msg.notes) {
72- const velB = bucketVelocity(n.velocity);
73- const durB = bucketDuration(n.durationSec);
74- const key = noteCacheKey(b, msg.quality, msg.sampleRate, msg.settings, n.pitch, velB, durB);
75- const g = groups.get(key);
76- if (g) g.times.push(n.timeSec);
77- else groups.set(key, { velB, durB, pitch: n.pitch, times: [n.timeSec] });
78- }
79- const misses = [...groups.entries()].filter(([key]) => !cache.has(key));
80- for (let c = 0; c < misses.length; c += CHUNK) {
81- const chunk = misses.slice(c, c + CHUNK);
82- post({
83- kind: 'status',
84- text: `${b.name}: rendering notes ${c + 1}-${Math.min(c + CHUNK, misses.length)}/${misses.length} (${msg.quality})`,
85- });
86- const bufs = renderNoteBatch(
87- b,
88- chunk.map(([, g]) => ({ pitch: g.pitch, velocity: g.velB, durationSec: g.durB })),
89- { sampleRate: msg.sampleRate, quality: msg.quality, settings: msg.settings },
90- );
91- chunk.forEach(([key], i) => cachePut(key, bufs[i]));
92- }
93- for (const [key, g] of groups) {
94- const buf = cacheGet(key)!;
95- for (const t of g.times) {
96- const off = Math.round(t * msg.sampleRate);
97- const end = Math.min(out.length, off + buf.length);
98- for (let i = Math.max(0, off); i < end; i++) out[i] += buf[i - off];
99- }
100- }
101- post({ kind: 'status', text: '' });
102- post({ kind: 'result', id: msg.id, samples: out }, [out.buffer]);
103- } catch (err) {
104- post({ kind: 'status', text: '' });
105- post({ kind: 'error', id: msg.id, message: err instanceof Error ? err.message : String(err) });
106- }
107-};
src/instruments/sim.tsadded+19−0View file
@@ -0,0 +1,19 @@
1+// The simulated instruments: physically based models from the sibling
2+// simulated_instruments collection, each imported directly from its
3+// repo's instrument/ module. Each module implements the track-render
4+// interface itself (worker, caching, progress, cancellation included),
5+// so registering it is all there is to do here. The MATLAB sources ride
6+// along at build time via each module's own import.meta.glob.
7+import { registerInstrument } from './registry.ts';
8+import { bassInstruments } from '../../../../simulated_instruments/bass/instrument/index.ts';
9+import { woodblockInstruments } from '../../../../simulated_instruments/woodblock/instrument/index.ts';
10+import { recorderInstruments } from '../../../../simulated_instruments/recorder/instrument/index.ts';
11+import { cymbal2Instruments } from '../../../../simulated_instruments/cymbal2/instrument/index.ts';
12+import { triangleInstruments } from '../../../../simulated_instruments/triangle/instrument/index.ts';
13+import { maracaInstruments } from '../../../../simulated_instruments/maraca/instrument/index.ts';
14+
15+export function registerSimulatedInstruments(): number {
16+ const all = [...bassInstruments, ...woodblockInstruments, ...recorderInstruments, ...cymbal2Instruments, ...triangleInstruments, ...maracaInstruments];
17+ for (const ins of all) registerInstrument(ins);
18+ return all.length;
19+}
src/instruments/types.tsmodified+60−30View file
@@ -1,20 +1,15 @@
1-// The instrument extension point. An instrument turns note events into
2-// audio in two ways:
1+// The track-render instrument contract shared between midiboot and the
2+// instrument web apps. Each simulated-instrument repo carries its own
3+// structurally identical copy of these types (there is no shared package;
4+// TypeScript's structural typing keeps the copies interchangeable), and
5+// midiboot's src/instruments/types.ts is the reference.
36 //
4-// - render(): the offline path. Given every note with times in seconds,
5-// produce one mono buffer for the whole track. This is what playback
6-// and WAV export use, and it may be async and arbitrarily expensive
7-// (a GPU physical model, a worker, a remote call). Results are cached
8-// per track and invalidated by note or instrument changes.
9-//
10-// - noteOn(): the optional live path, used for low-latency monitoring
11-// while recording and for previewing notes from the piano roll and
12-// keyboard. An instrument without it is still fully usable; monitoring
13-// falls back to a small default click.
14-//
15-// To add a custom instrument, implement this interface and register it in
16-// registry.ts. Nothing else in the application knows what an instrument is
17-// made of.
7+// An instrument receives an entire MIDI track per call: every note with
8+// times already in seconds. It owns its rendering strategy end to end,
9+// including any internal caching (per-note, per-stage, or none), progress
10+// reporting, and cancellation. The optional fields at the bottom exist for
11+// the instrument's own web app, which drives the full parameter surface
12+// and live-edited MATLAB sources; midiboot never sets them.
1813
1914 export interface NoteSpec {
2015 timeSec: number;
@@ -23,18 +18,6 @@ export interface NoteSpec {
2318 velocity: number; // 0..1
2419 }
2520
26-export interface RenderRequest {
27- notes: NoteSpec[];
28- sampleRate: number;
29- lengthSamples: number;
30- /** Fast preview for playback, full quality for export; instruments
31- * without tiers ignore it. */
32- quality?: 'preview' | 'full';
33- /** Values for the instrument's knobs (per track); missing keys mean the
34- * knob's default. */
35- settings?: Record<string, number>;
36-}
37-
3821 /** A user-editable instrument parameter, declared by the instrument. */
3922 export interface KnobSpec {
4023 key: string;
@@ -46,6 +29,53 @@ export interface KnobSpec {
4629 scale?: 'log';
4730 }
4831
32+/** One MATLAB source file, as numbl consumes it. */
33+export interface MFile {
34+ name: string;
35+ source: string;
36+}
37+
38+export interface RenderProgress {
39+ /** 0..1 through the whole request. */
40+ frac: number;
41+ /** Short human-readable stage label, e.g. "note 3/8: solve". */
42+ stage: string;
43+}
44+
45+export interface TrackRenderRequest {
46+ /** Every note of the track, sorted by timeSec. */
47+ notes: NoteSpec[];
48+ sampleRate: number;
49+ /** Song length incl. tail, as a minimum: the returned buffer may be
50+ * longer, so a ring that outlives the last note (a cymbal crash, a
51+ * long bass tail) is kept in full; the host mixes buffers of unequal
52+ * lengths. */
53+ lengthSamples: number;
54+ /** Fast preview for playback, full quality for export. */
55+ quality: 'preview' | 'full';
56+ /** Values for the instrument's knobs; missing keys mean the default. */
57+ settings?: Record<string, number>;
58+ /** Cooperative cancellation; checked between stages and notes. */
59+ signal?: AbortSignal;
60+ onProgress?: (p: RenderProgress) => void;
61+
62+ // --- fields used only by the instrument's own web app ---
63+
64+ /** Full model parameter overrides, applied after the defaults (and after
65+ * any note mapping). midiboot never sets this. */
66+ params?: Record<string, number | boolean>;
67+ /** Live-edited MATLAB sources replacing the shipped ones. */
68+ mfiles?: MFile[];
69+ /** Also compute the instrument-specific diagnostics payload. */
70+ wantDiagnostics?: boolean;
71+}
72+
73+export interface TrackRenderResult {
74+ audio: Float32Array;
75+ /** Instrument-specific diagnostics; present only when requested. */
76+ diag?: unknown;
77+}
78+
4979 export interface LiveVoice {
5080 noteOff(): void; // begin release
5181 cut(): void; // stop immediately
@@ -54,8 +84,8 @@ export interface LiveVoice {
5484 export interface Instrument {
5585 id: string;
5686 name: string;
57- render(req: RenderRequest): Float32Array | Promise<Float32Array>;
58- noteOn?(ctx: BaseAudioContext, dest: AudioNode, pitch: number, velocity: number, whenSec: number): LiveVoice;
5987 /** Curated editable parameters; tracks store their values in settings. */
6088 knobs?: KnobSpec[];
89+ renderTrack(req: TrackRenderRequest): Promise<TrackRenderResult>;
90+ noteOn?(ctx: BaseAudioContext, dest: AudioNode, pitch: number, velocity: number, whenSec: number): LiveVoice;
6191 }
src/main.tsxmodified+2−4View file
@@ -1,11 +1,9 @@
11 import { createRoot } from 'react-dom/client';
22 import { App } from './components/App.tsx';
33 import { store } from './state/store.ts';
4-import { loadImportedInstruments, registerCatalogInstruments, setNumblStatus } from './instruments/numbl/adapter.ts';
4+import { registerSimulatedInstruments } from './instruments/sim.ts';
55 import './styles.css';
66
7-registerCatalogInstruments();
8-setNumblStatus((msg) => store.setStatus(msg));
9-void loadImportedInstruments().then((n) => { if (n > 0) store.bumpInstruments(); });
7+registerSimulatedInstruments();
108 void store.init();
119 createRoot(document.getElementById('root')!).render(<App />);
src/midi/gm.tsmodified+18−0View file
@@ -34,6 +34,24 @@ export const GM_PROGRAMS: string[] = [
3434 'Telephone Ring', 'Helicopter', 'Applause', 'Gunshot',
3535 ];
3636
37+// Coarse mapping from a GM program number to the nearest builtin synth,
38+// used when importing a MIDI file so playback lands roughly in the right
39+// family without a soundfont. Channel 10 is handled separately (drum kit).
40+export function gmProgramInstrument(program: number | undefined): string {
41+ if (program === undefined) return 'piano';
42+ const p = Math.max(0, Math.min(127, Math.floor(program)));
43+ if (p <= 7) return 'piano'; // pianos
44+ if (p <= 15) return 'epiano'; // chromatic percussion
45+ if (p <= 23) return 'organ'; // organs and accordions
46+ if (p <= 31) return 'pluck'; // guitars
47+ if (p <= 39) return 'synthbass'; // basses
48+ if (p <= 55) return 'strings'; // strings, ensembles, choirs
49+ if (p <= 71) return 'brass'; // brass and reeds
50+ if (p <= 87) return 'lead'; // pipes and synth leads
51+ if (p <= 103) return 'strings'; // pads and FX
52+ return 'pluck'; // ethnic, percussive, effects
53+}
54+
3755 export function noteName(pitch: number): string {
3856 const names = ['C', 'C#', 'D', 'D#', 'E', 'F', 'F#', 'G', 'G#', 'A', 'A#', 'B'];
3957 return `${names[((pitch % 12) + 12) % 12]}${Math.floor(pitch / 12) - 1}`;
src/midi/parse.tsmodified+2−2View file
@@ -1,6 +1,6 @@
11 import type { Note, Song, TempoEvent, TimeSigEvent, Track } from '../types.ts';
22 import { newId } from '../types.ts';
3-import { GM_PROGRAMS } from './gm.ts';
3+import { GM_PROGRAMS, gmProgramInstrument } from './gm.ts';
44
55 // Standard MIDI file parser for formats 0 and 1, with running status and a
66 // merged tempo map. Each (SMF track, channel) pair that carries notes
@@ -152,7 +152,7 @@ export function parseMidi(data: Uint8Array): Song {
152152 tracks.push({
153153 id: newId('t'),
154154 name,
155- instrumentId: ch === 9 ? 'drumkit' : 'pluck',
155+ instrumentId: ch === 9 ? 'drumkit' : gmProgramInstrument(prog),
156156 volume: 1,
157157 mute: false,
158158 solo: false,
src/state/store.tsmodified+0−5View file
@@ -26,8 +26,6 @@ export interface AppState {
2626 snapBeats: number; // snap grid in quarter notes; 0 = off
2727 libraryOpen: boolean;
2828 status: string;
29- /** Bumped when the instrument registry changes so pickers re-render. */
30- instrumentsRev: number;
3129 }
3230
3331 const LAST_SONG_KEY = 'midiboot:lastSong';
@@ -46,7 +44,6 @@ class Store {
4644 snapBeats: 0.25,
4745 libraryOpen: false,
4846 status: '',
49- instrumentsRev: 0,
5047 };
5148 recorder = new Recorder();
5249
@@ -72,8 +69,6 @@ class Store {
7269
7370 setStatus(msg: string): void { this.set({ status: msg }); }
7471
75- bumpInstruments(): void { this.set({ instrumentsRev: this.state.instrumentsRev + 1 }); }
76-
7772 tempoMap(): TempoMap | null {
7873 const song = this.song();
7974 return song ? new TempoMap(song.ppq, song.tempo, song.timeSig) : null;
src/storage/db.tsmodified+2−32View file
@@ -5,6 +5,8 @@ import type { SongMeta, SongRecord } from '../types.ts';
55
66 const DB_NAME = 'midiboot';
77 const STORE = 'songs';
8+// Retired object store, kept in the upgrade path so existing databases at
9+// version 2 stay consistent with fresh ones.
810 const INSTRUMENTS = 'instruments';
911
1012 let dbPromise: Promise<IDBDatabase> | null = null;
@@ -87,35 +89,3 @@ export async function listSongs(): Promise<SongMeta[]> {
8789 req.onerror = () => reject(req.error);
8890 });
8991 }
90-
91-// Imported instrument bundles (format numbl-instrument-v1), keyed by
92-// bundle id. Whole records again: a bundle is tens of kilobytes of JSON.
93-
94-export interface ImportedInstrumentRecord {
95- id: string; // the bundle id
96- text: string; // the bundle JSON, kept verbatim as imported
97- importedAt: number;
98-}
99-
100-export async function putImportedInstrument(rec: ImportedInstrumentRecord): Promise<void> {
101- const db = await openDb();
102- const tx = db.transaction(INSTRUMENTS, 'readwrite');
103- tx.objectStore(INSTRUMENTS).put(rec);
104- await txDone(tx);
105-}
106-
107-export async function deleteImportedInstrument(id: string): Promise<void> {
108- const db = await openDb();
109- const tx = db.transaction(INSTRUMENTS, 'readwrite');
110- tx.objectStore(INSTRUMENTS).delete(id);
111- await txDone(tx);
112-}
113-
114-export async function listImportedInstruments(): Promise<ImportedInstrumentRecord[]> {
115- const db = await openDb();
116- return new Promise((resolve, reject) => {
117- const req = db.transaction(INSTRUMENTS, 'readonly').objectStore(INSTRUMENTS).getAll();
118- req.onsuccess = () => resolve((req.result as ImportedInstrumentRecord[]) ?? []);
119- req.onerror = () => reject(req.error);
120- });
121-}
src/styles.cssmodified+18−10View file
@@ -158,15 +158,23 @@ label { display: inline-flex; align-items: center; gap: 4px; white-space: nowrap
158158 }
159159 .dim { color: #777; font-size: 11px; }
160160
161-.importinstr { display: flex; flex-direction: column; gap: 4px; }
162-.importedlist { display: flex; flex-wrap: wrap; gap: 4px; padding: 0 6px 6px; }
163-.importedchip {
164- display: inline-flex;
161+.trackrow-render {
162+ display: flex;
165163 align-items: center;
166- gap: 4px;
167- font-size: 11px;
168- padding: 1px 4px 1px 8px;
169- border: 1px solid #333a47;
170- border-radius: 10px;
171- color: #8b93a3;
164+ gap: 6px;
165+ padding: 2px 0;
166+}
167+.renderbar {
168+ flex: 1;
169+ height: 6px;
170+ border-radius: 3px;
171+ background: #232833;
172+ overflow: hidden;
173+}
174+.renderbar-fill {
175+ display: block;
176+ height: 100%;
177+ background: #4f8cff;
178+ transition: width 0.2s;
172179 }
180+.renderstage { font-size: 10px; color: #8b93a3; white-space: nowrap; }
vite.config.tsmodified+4−3View file
@@ -4,9 +4,10 @@ import { realpathSync } from 'node:fs';
44 import { resolve } from 'node:path';
55
66 // The simulated instruments live in the sibling collection (the vault
7-// layout): each repo carries an instrument.json manifest and MATLAB
8-// sources that midiboot imports at build time via import.meta.glob, so
9-// the dev server must be allowed to read them from their real path.
7+// layout): each repo carries an instrument/ module implementing the
8+// track-render interface plus the MATLAB sources it globs at build time,
9+// imported directly by src/instruments/sim.ts, so the dev server must be
10+// allowed to read them from their real path.
1011 const simulated = realpathSync(resolve(import.meta.dirname, '../../simulated_instruments'));
1112
1213 export default defineConfig({