Voice dictation notepad that runs entirely in the browser
Records from the microphone, segments speech with an energy-based VAD, and
transcribes locally via transformers.js — Whisper or Moonshine, WebGPU when
available and CPU otherwise. The transcript accumulates into an editable
document held in localStorage.
transformers.js is pinned to 3.8.1 because 4.2.0 cannot create an int8 ONNX
session for encoder-decoder ASR models.
7 changed files+1116−0
README.mdadded+116−0View file
@@ -0,0 +1,116 @@
1+# voicenote
2+
3+Voice dictation that never leaves your browser. Press **Record**, talk, and
4+watch a document build itself. Edit it as you go, pause and resume, copy it out
5+when you're done.
6+
7+There is no server and no API key. Speech recognition runs entirely on your
8+machine via [transformers.js](https://github.com/huggingface/transformers.js) —
9+the model weights are downloaded once from HuggingFace, cached by the browser,
10+and every subsequent visit works offline. No audio is ever transmitted.
11+
12+## Using it
13+
14+- **Record / Pause** — starts and stops listening. Text is appended when you
15+ pause between sentences; greyed italic text below the document is the
16+ in-progress guess for what you're currently saying.
17+- The document is a plain textarea. Type in it, fix mistakes, rearrange —
18+ dictation appends to the end and leaves your cursor alone.
19+- **Copy** puts the whole document on the clipboard. **Clear** empties it.
20+- Everything is saved to `localStorage`, so a reload picks up where you left
21+ off. There is exactly one document.
22+
23+## Models
24+
25+Pick a model from the dropdown; the size shown is the one-time download for the
26+backend you're actually going to run on, and already-cached models are marked
27+*downloaded*. Your choice is remembered.
28+
29+| Model | CPU | WebGPU | Notes |
30+| --- | --- | --- | --- |
31+| Moonshine Tiny | 32 MB | 79 MB | Fastest; lighter punctuation |
32+| Whisper Tiny | 44 MB | 122 MB | Quick; less accurate on hard words |
33+| Moonshine Base | 67 MB | 157 MB | Fast and accurate; lighter punctuation |
34+| **Whisper Base** | 80 MB | 209 MB | Default — good accuracy and punctuation |
35+| Whisper Small | 252 MB | 588 MB | Most accurate; slow without WebGPU |
36+
37+The two families behave quite differently on short utterances. Whisper pads
38+every clip to 30 seconds, so a two-second phrase costs the same as a full one;
39+[Moonshine](https://github.com/usefulsensors/moonshine) scales with the real
40+audio length. Measured on one CPU core for a 3-second phrase: Moonshine Tiny
41+0.17 s, Whisper Tiny 1.3 s, Whisper Base 2.7 s. Whisper punctuates noticeably
42+better, which is why it is the default — but if dictation feels sluggish on the
43+CPU backend, Moonshine is the fix.
44+
45+## Backends
46+
47+WebGPU is used automatically whenever the browser offers an adapter, and the
48+footer always names the backend actually in use. It is much faster than the CPU
49+path, which matters most for the larger Whisper models.
50+
51+The catch is download size. The two backends want different quantizations —
52+int8 throughout on CPU, versus an fp32 encoder and a 4-bit decoder on WebGPU,
53+since int8 matmul is poorly served there — and the 4-bit decoder only quantizes
54+matmul weights, leaving the embeddings at full precision. The result is a 2–3×
55+larger download on WebGPU, which is why the picker quotes both.
56+
57+Detecting an adapter does not guarantee one that works, so a WebGPU failure
58+falls back to CPU. That fallback restarts the worker rather than retrying in
59+place: once ONNX Runtime has failed to bring up WebGPU, its backend registry
60+stays poisoned and a CPU request in the same worker resolves right back to
61+WebGPU and fails identically. Phrases spoken during a load or a restart are
62+held on the main thread and submitted once a model is ready, so switching
63+backends never costs you words.
64+
65+## How it works
66+
67+`worklet.js` captures microphone audio at 16 kHz. `app.js` runs an energy-based
68+voice-activity detector over it that adapts to your room's noise floor: it keeps
69+a rolling pre-roll buffer so the start of a word is never clipped, opens an
70+utterance when speech begins, and closes it after ~0.9 s of silence. Closed
71+utterances go to `worker.js` for transcription and are appended to the document;
72+while you are still speaking, the same audio is periodically re-transcribed to
73+produce the live interim line.
74+
75+Segmentation is tuned to avoid splitting sentences at dramatic mid-sentence
76+pauses — short fragments transcribe poorly because the model loses the
77+surrounding words it needs for context — and to keep trailing silence, since a
78+final fricative is quiet enough to read as silence and trimming into it eats the
79+end of the word.
80+
81+Only one transcription runs at a time; interim requests are skipped whenever the
82+worker is busy, so a slow model degrades to fewer live updates rather than an
83+ever-growing backlog. Finished phrases are never skipped — if no model is ready
84+yet they queue on the main thread (up to about two minutes of speech) and are
85+submitted as soon as one is.
86+
87+### Why transformers.js 3.8.1
88+
89+Pinned deliberately. In 4.2.0 the int8 path fails to create an ONNX session for
90+encoder-decoder ASR models:
91+
92+```
93+qdq_actions.cc:137 TransposeDQWeightsForMatMulNBits
94+Missing required scale: model.decoder.embed_tokens.weight_merged_0_scale
95+```
96+
97+This affects Whisper and Moonshine alike, so the whole model list is unusable on
98+4.x. 3.8.1 runs all five correctly.
99+
100+## Running locally
101+
102+Any static file server will do — there is no build step:
103+
104+```bash
105+python3 -m http.server 8000
106+```
107+
108+Then open <http://localhost:8000>. A secure context is required for microphone
109+access, which `localhost` and HTTPS both satisfy.
110+
111+## Browser support
112+
113+Needs `AudioWorklet`, module workers, and WebAssembly — Chrome, Edge, Firefox,
114+and Safari 15+ all qualify. WebGPU is used where available and is not required.
115+The CPU backend runs single-threaded: multi-threaded WASM needs cross-origin
116+isolation headers, which GitHub Pages cannot set.
app.jsadded+511−0View file
@@ -0,0 +1,511 @@
1+// voicenote — fully client-side voice dictation.
2+// Mic capture (AudioWorklet, 16 kHz) → energy-based VAD segments utterances →
3+// a speech model in a Web Worker transcribes them (interim results while you
4+// speak, final text appended on silence). Document lives in localStorage.
5+
6+import { MODELS, DEFAULT_MODEL, cachedModels, detectDevice } from './models.js';
7+
8+const SR = 16000;
9+const DOC_KEY = 'voicenote-doc';
10+const MODEL_KEY = 'voicenote-model';
11+
12+// VAD tuning (all sample counts at 16 kHz)
13+const PREROLL_SAMPLES = 0.32 * SR; // audio kept before speech onset
14+const ENTER_FRAMES = 2; // consecutive loud frames to trigger speech
15+// Long enough that a dramatic mid-sentence pause does not split the sentence:
16+// short fragments transcribe badly, because the model loses the surrounding
17+// words it needs for context.
18+const SILENCE_FINAL = 0.9 * SR; // trailing silence that ends an utterance
19+const SOFT_MAX = 20 * SR; // after this, finalize at the next brief dip
20+const SOFT_DIP = 0.15 * SR;
21+const HARD_MAX = 28 * SR; // hard cut (whisper's window is 30 s)
22+const MIN_SPEECH = 0.35 * SR; // discard blips shorter than this
23+// Generous, because a trailing fricative ("...Americans") is quiet enough to
24+// read as silence, and trimming into it costs the end of the word.
25+const KEEP_TAIL = 0.4 * SR; // trailing silence kept when trimming
26+const INTERIM_MIN = 0.8 * SR; // utterance length before the first interim
27+const INTERIM_EVERY = 1.2 * SR; // new audio between interim runs
28+
29+// These models emit stock filler on near-silent audio; drop it when it is the
30+// entire result rather than letting it accumulate in the document.
31+const JUNK = new Set(['you', 'thank you.', 'thanks for watching!', 'thank you for watching!',
32+ '[blank_audio]', '(silence)', '.', 'bye.', 'the', 'so']);
33+
34+const $ = (id) => document.getElementById(id);
35+const recordBtn = $('recordBtn');
36+const recordLabel = $('recordLabel');
37+const copyBtn = $('copyBtn');
38+const clearBtn = $('clearBtn');
39+const statusEl = $('status');
40+const modelSel = $('model');
41+const modelNote = $('modelNote');
42+const progressWrap = $('progressWrap');
43+const progressBar = $('progressBar');
44+const progressText = $('progressText');
45+const doc = $('doc');
46+const interimEl = $('interim');
47+const wordcountEl = $('wordcount');
48+const engineEl = $('engine');
49+
50+// ---------- document ----------
51+
52+doc.value = localStorage.getItem(DOC_KEY) || '';
53+
54+let saveTimer = null;
55+function saveDoc() {
56+ clearTimeout(saveTimer);
57+ saveTimer = setTimeout(() => localStorage.setItem(DOC_KEY, doc.value), 250);
58+}
59+
60+function updateWordCount() {
61+ const n = doc.value.trim().split(/\s+/).filter(Boolean).length;
62+ wordcountEl.textContent = `${n} word${n === 1 ? '' : 's'}`;
63+}
64+
65+function autoresize() {
66+ doc.style.height = 'auto';
67+ doc.style.height = doc.scrollHeight + 'px';
68+}
69+
70+doc.addEventListener('input', () => {
71+ saveDoc();
72+ updateWordCount();
73+ autoresize();
74+});
75+
76+function appendFinal(text) {
77+ text = (text || '').trim();
78+ if (!text || JUNK.has(text.toLowerCase())) return;
79+ const sep = doc.value && !/\s$/.test(doc.value) ? ' ' : '';
80+ const focused = document.activeElement === doc;
81+ const selStart = doc.selectionStart;
82+ const selEnd = doc.selectionEnd;
83+ doc.value += sep + text;
84+ if (focused) doc.setSelectionRange(selStart, selEnd);
85+ saveDoc();
86+ updateWordCount();
87+ autoresize();
88+ if (!focused) window.scrollTo({ top: document.body.scrollHeight });
89+}
90+
91+copyBtn.addEventListener('click', async () => {
92+ try {
93+ await navigator.clipboard.writeText(doc.value);
94+ copyBtn.textContent = 'Copied ✓';
95+ setTimeout(() => (copyBtn.textContent = 'Copy'), 1200);
96+ } catch {
97+ setStatus('clipboard access denied');
98+ }
99+});
100+
101+clearBtn.addEventListener('click', () => {
102+ if (!doc.value.trim() || confirm('Clear the document? This cannot be undone.')) {
103+ doc.value = '';
104+ interimEl.textContent = '';
105+ localStorage.setItem(DOC_KEY, '');
106+ updateWordCount();
107+ autoresize();
108+ doc.focus();
109+ }
110+});
111+
112+// ---------- model picker ----------
113+
114+let modelId = localStorage.getItem(MODEL_KEY) || DEFAULT_MODEL;
115+if (!MODELS.some((m) => m.id === modelId)) modelId = DEFAULT_MODEL;
116+
117+for (const m of MODELS) {
118+ const opt = document.createElement('option');
119+ opt.value = m.id;
120+ opt.textContent = `${m.label} — ${m.mb} MB`;
121+ modelSel.appendChild(opt);
122+}
123+modelSel.value = modelId;
124+
125+function showModelNote() {
126+ modelNote.textContent = MODELS.find((m) => m.id === modelId)?.note ?? '';
127+}
128+
129+// Label each option with the download it would actually cost on the backend
130+// we expect to use, and mark the ones already sitting in the browser cache.
131+async function refreshCacheLabels() {
132+ const cached = await cachedModels(device);
133+ for (const opt of modelSel.options) {
134+ const m = MODELS.find((x) => x.id === opt.value);
135+ opt.textContent = `${m.label} — ${cached.has(m.id) ? 'downloaded' : `${m.mb[device]} MB`}`;
136+ }
137+}
138+
139+modelSel.addEventListener('change', () => {
140+ modelId = modelSel.value;
141+ localStorage.setItem(MODEL_KEY, modelId);
142+ showModelNote();
143+
144+ // Drop the loaded model; anything queued for it is abandoned with the worker.
145+ if (worker) {
146+ worker.terminate();
147+ worker = null;
148+ }
149+ modelState = 'unloaded';
150+ outstanding = 0;
151+ interimEl.textContent = '';
152+ interimUtt = -1;
153+ engineEl.textContent = '';
154+ progressWrap.classList.add('hidden');
155+
156+ if (recording) ensureModel(); // keep listening; load the new model right away
157+ updateStatus();
158+});
159+
160+// ---------- transcription worker ----------
161+
162+let worker = null;
163+let modelState = 'unloaded'; // unloaded | loading | ready | error
164+let outstanding = 0; // transcription requests in flight
165+let nextReqId = 1;
166+let device = 'wasm'; // backend we expect to use; confirmed on 'ready'
167+let forceCpu = false; // set once WebGPU has proven it cannot run here
168+
169+// Finished utterances captured before a model was ready. They live here rather
170+// than in the worker so that restarting the worker — which is how the WebGPU
171+// fallback works — never costs the user a phrase they already spoke.
172+const queued = [];
173+let queuedSamples = 0;
174+const MAX_QUEUED = 120 * SR; // ~2 minutes of speech; drop the oldest beyond it
175+
176+function fmtMB(b) { return (b / 1024 / 1024).toFixed(0); }
177+
178+// Creates the worker and starts the model load. The worker is constructed
179+// synchronously and the load message queued first, so nothing races it.
180+function ensureModel() {
181+ if (modelState === 'ready' || modelState === 'loading') return;
182+ modelState = 'loading';
183+ worker = new Worker('worker.js', { type: 'module' });
184+ worker.onmessage = onWorkerMessage;
185+ worker.onerror = (e) => failLoad(e.message || 'worker failed to start');
186+ worker.postMessage({ type: 'load', model: modelId, device: forceCpu ? 'cpu' : 'auto' });
187+}
188+
189+// A WebGPU failure needs a brand-new worker: ONNX Runtime will not hand back a
190+// working CPU backend in a process where WebGPU already failed to start.
191+function retryOnCpu(message) {
192+ console.warn('voicenote: WebGPU unavailable, restarting on CPU —', message);
193+ forceCpu = true;
194+ device = 'wasm';
195+ if (worker) { worker.terminate(); worker = null; }
196+ modelState = 'unloaded';
197+ outstanding = 0;
198+ progressWrap.classList.add('hidden');
199+ refreshCacheLabels();
200+ ensureModel();
201+ setStatus('WebGPU unavailable — loading on CPU instead…');
202+}
203+
204+function failLoad(message) {
205+ modelState = 'error';
206+ progressWrap.classList.add('hidden');
207+ stopMic();
208+ syncButton();
209+ setStatus(`error: ${message}`);
210+}
211+
212+function onWorkerMessage(e) {
213+ const msg = e.data;
214+ if (msg.type === 'progress') {
215+ progressWrap.classList.remove('hidden');
216+ progressBar.style.width = `${(100 * msg.loaded) / msg.total}%`;
217+ progressText.textContent = `${fmtMB(msg.loaded)} / ${fmtMB(msg.total)} MB`;
218+ } else if (msg.type === 'status') {
219+ setStatus(msg.message);
220+ } else if (msg.type === 'ready') {
221+ progressWrap.classList.add('hidden');
222+ modelState = 'ready';
223+ device = msg.device;
224+ engineEl.textContent =
225+ `${MODELS.find((m) => m.id === modelId).label} · ${msg.device === 'webgpu' ? 'WebGPU' : 'CPU (wasm)'}`;
226+ refreshCacheLabels();
227+ flushQueued();
228+ updateStatus();
229+ } else if (msg.type === 'result') {
230+ outstanding = Math.max(0, outstanding - 1);
231+ if (msg.final) {
232+ if (msg.utt === interimUtt) {
233+ interimEl.textContent = '';
234+ interimUtt = -1;
235+ }
236+ appendFinal(msg.text);
237+ } else if (msg.utt === currentUtt && inSpeech) {
238+ const t = (msg.text || '').trim();
239+ if (t && !JUNK.has(t.toLowerCase())) {
240+ interimEl.textContent = t + ' …';
241+ interimUtt = msg.utt;
242+ }
243+ }
244+ updateStatus();
245+ } else if (msg.type === 'error') {
246+ if (msg.retryOnCpu && !forceCpu) retryOnCpu(msg.message);
247+ else if (msg.fatal || modelState === 'loading') failLoad(msg.message);
248+ else setStatus(`transcription error: ${msg.message}`);
249+ }
250+}
251+
252+function postAudio(samples, final, uttId) {
253+ outstanding++;
254+ const buf = samples.buffer;
255+ worker.postMessage({ type: 'transcribe', id: nextReqId++, utt: uttId, final, audio: buf }, [buf]);
256+}
257+
258+function sendAudio(samples, final, uttId) {
259+ if (modelState === 'error') return;
260+ if (modelState === 'ready' && worker) {
261+ postAudio(samples, final, uttId);
262+ return;
263+ }
264+ if (!final) return; // interims are disposable; only hold on to finished phrases
265+ queued.push({ samples, uttId });
266+ queuedSamples += samples.length;
267+ while (queuedSamples > MAX_QUEUED && queued.length > 1) {
268+ queuedSamples -= queued.shift().samples.length;
269+ }
270+}
271+
272+function flushQueued() {
273+ while (queued.length) {
274+ const q = queued.shift();
275+ queuedSamples -= q.samples.length;
276+ postAudio(q.samples, true, q.uttId);
277+ }
278+ queuedSamples = 0;
279+}
280+
281+// ---------- mic capture + VAD ----------
282+
283+let recording = false;
284+let stream = null;
285+let audioCtx = null;
286+
287+let inSpeech = false;
288+let enterCount = 0;
289+let noiseFloor = 0.002;
290+let preroll = []; // frames seen before speech onset
291+let prerollSamples = 0;
292+let utt = []; // frames of the current utterance
293+let uttRms = []; // rms per frame, for the trailing-silence trim
294+let uttSamples = 0;
295+let silenceRun = 0; // samples of trailing sub-threshold audio
296+let sinceInterim = 0;
297+let currentUtt = 0; // id of the open utterance
298+let interimUtt = -1; // utterance whose interim text is on screen
299+
300+function concatFrames(frames, totalLen) {
301+ const out = new Float32Array(totalLen);
302+ let o = 0;
303+ for (const f of frames) { out.set(f, o); o += f.length; }
304+ return out;
305+}
306+
307+function resample(frame, fromRate) {
308+ if (fromRate === SR) return frame;
309+ const ratio = fromRate / SR;
310+ const n = Math.floor(frame.length / ratio);
311+ const out = new Float32Array(n);
312+ for (let i = 0; i < n; i++) {
313+ const pos = i * ratio;
314+ const i0 = Math.floor(pos);
315+ const frac = pos - i0;
316+ const i1 = Math.min(i0 + 1, frame.length - 1);
317+ out[i] = frame[i0] * (1 - frac) + frame[i1] * frac;
318+ }
319+ return out;
320+}
321+
322+function onFrame(frame) {
323+ let sum = 0;
324+ for (let i = 0; i < frame.length; i++) sum += frame[i] * frame[i];
325+ const rms = Math.sqrt(sum / frame.length);
326+
327+ const enterTh = Math.max(noiseFloor * 3, 0.006);
328+ const exitTh = Math.max(noiseFloor * 2, 0.004);
329+
330+ if (!inSpeech) {
331+ // adapt the noise floor from quiet frames only
332+ if (rms < enterTh) noiseFloor = Math.max(0.0001, 0.95 * noiseFloor + 0.05 * rms);
333+ preroll.push(frame);
334+ prerollSamples += frame.length;
335+ while (prerollSamples > PREROLL_SAMPLES && preroll.length > 1) {
336+ prerollSamples -= preroll[0].length;
337+ preroll.shift();
338+ }
339+ enterCount = rms > enterTh ? enterCount + 1 : 0;
340+ if (enterCount >= ENTER_FRAMES) {
341+ inSpeech = true;
342+ enterCount = 0;
343+ currentUtt++;
344+ utt = preroll.slice();
345+ uttRms = preroll.map(() => rms); // approximate; only the tail is used
346+ uttSamples = prerollSamples;
347+ preroll = [];
348+ prerollSamples = 0;
349+ silenceRun = 0;
350+ sinceInterim = 0;
351+ updateStatus();
352+ }
353+ return;
354+ }
355+
356+ utt.push(frame);
357+ uttRms.push(rms);
358+ uttSamples += frame.length;
359+ sinceInterim += frame.length;
360+ silenceRun = rms < exitTh ? silenceRun + frame.length : 0;
361+
362+ const done =
363+ silenceRun >= SILENCE_FINAL ||
364+ (uttSamples >= SOFT_MAX && silenceRun >= SOFT_DIP) ||
365+ uttSamples >= HARD_MAX;
366+
367+ if (done) {
368+ finalizeUtterance();
369+ } else if (
370+ modelState === 'ready' &&
371+ outstanding === 0 &&
372+ uttSamples >= INTERIM_MIN &&
373+ sinceInterim >= INTERIM_EVERY
374+ ) {
375+ sinceInterim = 0;
376+ sendAudio(concatFrames(utt, uttSamples), false, currentUtt);
377+ }
378+}
379+
380+function finalizeUtterance() {
381+ if (!inSpeech) return;
382+ inSpeech = false;
383+
384+ // trim trailing silence down to KEEP_TAIL
385+ let trailing = 0;
386+ const exitTh = Math.max(noiseFloor * 2, 0.004);
387+ for (let i = uttRms.length - 1; i >= 0 && uttRms[i] < exitTh; i--) {
388+ trailing += utt[i].length;
389+ }
390+ while (utt.length > 1 && trailing - utt[utt.length - 1].length >= KEEP_TAIL) {
391+ const f = utt.pop();
392+ uttRms.pop();
393+ uttSamples -= f.length;
394+ trailing -= f.length;
395+ }
396+
397+ const voiced = uttSamples - Math.min(trailing, uttSamples);
398+ if (voiced >= MIN_SPEECH) {
399+ sendAudio(concatFrames(utt, uttSamples), true, currentUtt);
400+ } else if (interimUtt === currentUtt) {
401+ interimEl.textContent = '';
402+ interimUtt = -1;
403+ }
404+ utt = [];
405+ uttRms = [];
406+ uttSamples = 0;
407+ silenceRun = 0;
408+ updateStatus();
409+}
410+
411+async function startMic() {
412+ stream = await navigator.mediaDevices.getUserMedia({
413+ audio: { channelCount: 1, echoCancellation: true, noiseSuppression: true, autoGainControl: true },
414+ });
415+ audioCtx = new AudioContext({ sampleRate: SR });
416+ if (audioCtx.state === 'suspended') await audioCtx.resume();
417+ await audioCtx.audioWorklet.addModule('worklet.js');
418+ const src = audioCtx.createMediaStreamSource(stream);
419+ const node = new AudioWorkletNode(audioCtx, 'capture');
420+ const rate = audioCtx.sampleRate;
421+ node.port.onmessage = (e) => {
422+ if (recording) onFrame(resample(e.data, rate));
423+ };
424+ src.connect(node);
425+ node.connect(audioCtx.destination); // worklet outputs silence; keeps the graph pulling
426+ recording = true;
427+}
428+
429+function stopMic() {
430+ recording = false;
431+ if (inSpeech) finalizeUtterance();
432+ if (stream) { stream.getTracks().forEach((t) => t.stop()); stream = null; }
433+ if (audioCtx) { audioCtx.close(); audioCtx = null; }
434+ inSpeech = false;
435+ enterCount = 0;
436+ preroll = [];
437+ prerollSamples = 0;
438+}
439+
440+// ---------- UI state ----------
441+
442+function setStatus(text) { statusEl.textContent = text; }
443+
444+function syncButton() {
445+ recordBtn.classList.toggle('live', recording);
446+ recordBtn.classList.toggle('speech', recording && inSpeech);
447+ recordLabel.textContent = recording ? 'Pause' : 'Record';
448+}
449+
450+function updateStatus() {
451+ syncButton();
452+ if (modelState === 'error') return; // keep the error message on screen
453+ if (modelState === 'loading') {
454+ const n = queued.length;
455+ setStatus(
456+ 'loading speech model (one-time download, then cached)…' +
457+ (n ? ` — ${n} phrase${n === 1 ? '' : 's'} waiting` : ''),
458+ );
459+ } else if (recording) {
460+ setStatus(inSpeech ? 'hearing speech…' : outstanding > 0 ? 'transcribing…' : 'listening…');
461+ } else if (outstanding > 0) {
462+ setStatus('transcribing…');
463+ } else if (modelState === 'ready') {
464+ setStatus('paused — press Record to continue');
465+ } else {
466+ setStatus('');
467+ }
468+}
469+
470+recordBtn.addEventListener('click', async () => {
471+ if (recording) {
472+ stopMic();
473+ updateStatus();
474+ return;
475+ }
476+ recordBtn.disabled = true;
477+ try {
478+ // Open the mic first, while still close to the user gesture, then kick off
479+ // the model load in parallel; audio captured meanwhile is queued.
480+ await startMic();
481+ ensureModel();
482+ updateStatus();
483+ } catch (err) {
484+ console.error(err);
485+ stopMic();
486+ syncButton();
487+ setStatus(
488+ err.name === 'NotAllowedError'
489+ ? 'microphone access denied — allow it and try again'
490+ : `error: ${err.message}`,
491+ );
492+ } finally {
493+ recordBtn.disabled = false;
494+ }
495+});
496+
497+window.addEventListener('beforeunload', () => {
498+ clearTimeout(saveTimer);
499+ localStorage.setItem(DOC_KEY, doc.value);
500+});
501+
502+updateWordCount();
503+autoresize();
504+showModelNote();
505+setStatus('');
506+
507+// Sizes depend on the backend, so settle that before labelling the picker.
508+detectDevice().then((d) => {
509+ device = d;
510+ refreshCacheLabels();
511+});
index.htmladded+49−0View file
@@ -0,0 +1,49 @@
1+<!DOCTYPE html>
2+<html lang="en">
3+<head>
4+ <meta charset="utf-8" />
5+ <meta name="viewport" content="width=device-width, initial-scale=1" />
6+ <title>voicenote</title>
7+ <link rel="icon" href="data:image/svg+xml,<svg xmlns=%22http://www.w3.org/2000/svg%22 viewBox=%220 0 100 100%22><text y=%22.9em%22 font-size=%2290%22>🎙</text></svg>" />
8+ <link rel="stylesheet" href="styles.css" />
9+</head>
10+<body>
11+ <div class="page">
12+ <header>
13+ <h1>voicenote</h1>
14+ <p class="tagline">Voice dictation that never leaves your browser</p>
15+ </header>
16+
17+ <div class="controls">
18+ <button id="recordBtn" class="primary"><span class="dot"></span><span id="recordLabel">Record</span></button>
19+ <button id="copyBtn" title="Copy the document to the clipboard">Copy</button>
20+ <button id="clearBtn" title="Clear the document">Clear</button>
21+ <span id="status" class="status"></span>
22+ </div>
23+
24+ <div class="modelrow">
25+ <label for="model">Model</label>
26+ <select id="model"></select>
27+ <span id="modelNote" class="hint"></span>
28+ </div>
29+
30+ <div id="progressWrap" class="progress hidden">
31+ <div class="progress-track"><div id="progressBar" class="progress-bar"></div></div>
32+ <span id="progressText"></span>
33+ </div>
34+
35+ <div class="doccard">
36+ <textarea id="doc" spellcheck="true"
37+ placeholder="Press Record and start speaking — or just type. Your document is saved locally and survives reloads."></textarea>
38+ <div id="interim" class="interim"></div>
39+ </div>
40+
41+ <footer>
42+ <span id="wordcount">0 words</span>
43+ <span id="engine"></span>
44+ </footer>
45+ </div>
46+
47+ <script type="module" src="app.js"></script>
48+</body>
49+</html>
models.jsadded+90−0View file
@@ -0,0 +1,90 @@
1+// Speech models offered in the picker, and the per-device weights they need.
2+//
3+// The two backends want different quantizations, so they download different
4+// files and the picker has to quote the size for whichever one will actually
5+// run:
6+//
7+// wasm — int8 encoder + int8 merged decoder. Small, CPU-friendly.
8+// webgpu — fp32 encoder + 4-bit merged decoder. This is the combination the
9+// transformers.js WebGPU examples use; int8 matmul is poorly served
10+// on WebGPU. It runs far faster but downloads 2–3× as much, because
11+// the 4-bit decoder only quantizes matmul weights and leaves the
12+// embeddings at full precision.
13+//
14+// `mb` figures are the encoder + decoder + tokenizer/config bytes, taken from
15+// the HuggingFace file listing. Everything is cached after the first visit.
16+export const MODELS = [
17+ {
18+ id: 'onnx-community/moonshine-tiny-ONNX',
19+ label: 'Moonshine Tiny',
20+ mb: { wasm: 32, webgpu: 79 },
21+ note: 'fastest; lighter punctuation',
22+ },
23+ {
24+ id: 'onnx-community/whisper-tiny.en',
25+ label: 'Whisper Tiny',
26+ mb: { wasm: 44, webgpu: 122 },
27+ note: 'quick; less accurate on hard words',
28+ },
29+ {
30+ id: 'onnx-community/moonshine-base-ONNX',
31+ label: 'Moonshine Base',
32+ mb: { wasm: 67, webgpu: 157 },
33+ note: 'fast and accurate; lighter punctuation',
34+ },
35+ {
36+ id: 'onnx-community/whisper-base.en',
37+ label: 'Whisper Base',
38+ mb: { wasm: 80, webgpu: 209 },
39+ note: 'good accuracy and punctuation',
40+ },
41+ {
42+ id: 'onnx-community/whisper-small.en',
43+ label: 'Whisper Small',
44+ mb: { wasm: 252, webgpu: 588 },
45+ note: 'most accurate; slow without WebGPU',
46+ },
47+];
48+
49+export const DEFAULT_MODEL = 'onnx-community/whisper-base.en';
50+
51+// Quantization suffixes per device — these decide both which files
52+// transformers.js fetches and which ones we look for in the cache.
53+export const SUFFIX = {
54+ wasm: { encoder: '_quantized', decoder: '_quantized' },
55+ webgpu: { encoder: '', decoder: '_q4' },
56+};
57+
58+export async function detectDevice() {
59+ if (!navigator.gpu) return 'wasm';
60+ try {
61+ return (await navigator.gpu.requestAdapter()) ? 'webgpu' : 'wasm';
62+ } catch {
63+ return 'wasm';
64+ }
65+}
66+
67+// The two large files for a model on a given device. Their presence in the
68+// cache is what "downloaded" in the picker means.
69+const weightFiles = (id, device) => {
70+ const s = SUFFIX[device];
71+ return [
72+ `https://huggingface.co/${id}/resolve/main/onnx/encoder_model${s.encoder}.onnx`,
73+ `https://huggingface.co/${id}/resolve/main/onnx/decoder_model_merged${s.decoder}.onnx`,
74+ ];
75+};
76+
77+export async function cachedModels(device) {
78+ const cached = new Set();
79+ if (!self.caches) return cached;
80+ try {
81+ const cache = await caches.open('transformers-cache');
82+ await Promise.all(
83+ MODELS.map(async (m) => {
84+ const hits = await Promise.all(weightFiles(m.id, device).map((u) => cache.match(u)));
85+ if (hits.every(Boolean)) cached.add(m.id);
86+ }),
87+ );
88+ } catch { /* cache unavailable (private mode, etc.) — report none cached */ }
89+ return cached;
90+}
styles.cssadded+211−0View file
@@ -0,0 +1,211 @@
1+:root {
2+ --bg: #f6f5f2;
3+ --card: #ffffff;
4+ --text: #1c1c1e;
5+ --muted: #8a8a8e;
6+ --border: #e2e1dd;
7+ --accent: #c0392b;
8+ --accent-soft: #fdeceb;
9+ --btn-bg: #ffffff;
10+ --btn-hover: #f0efeb;
11+ --shadow: 0 1px 3px rgba(0, 0, 0, 0.06);
12+}
13+
14+@media (prefers-color-scheme: dark) {
15+ :root {
16+ --bg: #17171a;
17+ --card: #212125;
18+ --text: #ececee;
19+ --muted: #8e8e93;
20+ --border: #323236;
21+ --accent: #e5695c;
22+ --accent-soft: #3a2523;
23+ --btn-bg: #29292e;
24+ --btn-hover: #323238;
25+ --shadow: 0 1px 3px rgba(0, 0, 0, 0.4);
26+ }
27+}
28+
29+* { box-sizing: border-box; }
30+
31+body {
32+ margin: 0;
33+ background: var(--bg);
34+ color: var(--text);
35+ font-family: system-ui, -apple-system, "Segoe UI", Roboto, sans-serif;
36+ line-height: 1.5;
37+}
38+
39+.page {
40+ max-width: 760px;
41+ margin: 0 auto;
42+ padding: 2.5rem 1.25rem 4rem;
43+}
44+
45+header { margin-bottom: 1.5rem; }
46+
47+h1 {
48+ margin: 0;
49+ font-size: 1.6rem;
50+ font-weight: 700;
51+ letter-spacing: -0.02em;
52+}
53+
54+.tagline {
55+ margin: 0.15rem 0 0;
56+ color: var(--muted);
57+ font-size: 0.95rem;
58+}
59+
60+.controls {
61+ display: flex;
62+ align-items: center;
63+ gap: 0.6rem;
64+ flex-wrap: wrap;
65+ margin-bottom: 0.75rem;
66+}
67+
68+button {
69+ font: inherit;
70+ font-size: 0.95rem;
71+ padding: 0.5rem 1rem;
72+ border-radius: 8px;
73+ border: 1px solid var(--border);
74+ background: var(--btn-bg);
75+ color: var(--text);
76+ cursor: pointer;
77+ box-shadow: var(--shadow);
78+ display: inline-flex;
79+ align-items: center;
80+ gap: 0.5rem;
81+}
82+
83+button:hover:not(:disabled) { background: var(--btn-hover); }
84+button:disabled { opacity: 0.55; cursor: default; }
85+
86+button.primary {
87+ font-weight: 600;
88+ min-width: 7.5rem;
89+ justify-content: center;
90+}
91+
92+button.primary .dot {
93+ width: 0.6rem;
94+ height: 0.6rem;
95+ border-radius: 50%;
96+ background: var(--accent);
97+ flex: none;
98+}
99+
100+button.primary.live {
101+ border-color: var(--accent);
102+ background: var(--accent-soft);
103+}
104+
105+button.primary.live .dot { animation: pulse 1.6s ease-in-out infinite; }
106+button.primary.live.speech .dot { animation: none; box-shadow: 0 0 0 4px color-mix(in srgb, var(--accent) 30%, transparent); }
107+
108+@keyframes pulse {
109+ 0%, 100% { opacity: 1; }
110+ 50% { opacity: 0.35; }
111+}
112+
113+.status {
114+ color: var(--muted);
115+ font-size: 0.9rem;
116+ margin-left: 0.25rem;
117+}
118+
119+.modelrow {
120+ display: flex;
121+ align-items: center;
122+ gap: 0.5rem;
123+ flex-wrap: wrap;
124+ margin-bottom: 0.85rem;
125+ font-size: 0.85rem;
126+ color: var(--muted);
127+}
128+
129+.modelrow select {
130+ font: inherit;
131+ font-size: 0.85rem;
132+ padding: 0.25rem 0.45rem;
133+ border-radius: 6px;
134+ border: 1px solid var(--border);
135+ background: var(--btn-bg);
136+ color: var(--text);
137+}
138+
139+.hint { font-style: italic; }
140+
141+.progress {
142+ display: flex;
143+ align-items: center;
144+ gap: 0.75rem;
145+ margin-bottom: 0.75rem;
146+ font-size: 0.85rem;
147+ color: var(--muted);
148+}
149+
150+.progress.hidden { display: none; }
151+
152+.progress-track {
153+ flex: 1;
154+ height: 6px;
155+ border-radius: 3px;
156+ background: var(--border);
157+ overflow: hidden;
158+}
159+
160+.progress-bar {
161+ height: 100%;
162+ width: 0%;
163+ border-radius: 3px;
164+ background: var(--accent);
165+ transition: width 0.2s;
166+}
167+
168+.doccard {
169+ background: var(--card);
170+ border: 1px solid var(--border);
171+ border-radius: 12px;
172+ box-shadow: var(--shadow);
173+ padding: 1rem 1.1rem;
174+}
175+
176+textarea {
177+ display: block;
178+ width: 100%;
179+ min-height: 45vh;
180+ resize: none;
181+ border: none;
182+ outline: none;
183+ background: transparent;
184+ color: var(--text);
185+ font: inherit;
186+ font-size: 1.05rem;
187+ line-height: 1.65;
188+ padding: 0;
189+ overflow: hidden;
190+}
191+
192+textarea::placeholder { color: var(--muted); opacity: 0.7; }
193+
194+.interim {
195+ color: var(--muted);
196+ font-style: italic;
197+ font-size: 1.05rem;
198+ line-height: 1.65;
199+ min-height: 0;
200+ white-space: pre-wrap;
201+}
202+
203+.interim:empty { display: none; }
204+
205+footer {
206+ display: flex;
207+ justify-content: space-between;
208+ margin-top: 0.75rem;
209+ color: var(--muted);
210+ font-size: 0.85rem;
211+}
worker.jsadded+109−0View file
@@ -0,0 +1,109 @@
1+// Transcription worker: runs the selected speech model via transformers.js.
2+//
3+// Pinned to 3.8.1 deliberately: in 4.2.0 the int8 path fails to create an ONNX
4+// session for encoder-decoder ASR models ("TransposeDQWeightsForMatMulNBits
5+// Missing required scale"), which breaks both Whisper and Moonshine.
6+//
7+// One backend per worker, no in-process retry. Once ONNX Runtime has failed to
8+// bring up WebGPU its backend registry stays poisoned — a subsequent CPU
9+// request in the same worker still resolves to WebGPU and fails identically —
10+// so falling back means a fresh worker, which app.js handles.
11+//
12+// Every request is serialized through one promise chain, so transcription
13+// requests that arrive while the model is still downloading queue behind the
14+// load rather than failing.
15+import { pipeline, env } from 'https://cdn.jsdelivr.net/npm/@huggingface/transformers@3.8.1/dist/transformers.min.js';
16+
17+env.allowLocalModels = false;
18+
19+const SAMPLE_RATE = 16000;
20+
21+// Quantizations per backend — see models.js for why they differ.
22+const BACKEND = {
23+ webgpu: { device: 'webgpu', dtype: { encoder_model: 'fp32', decoder_model_merged: 'q4' } },
24+ wasm: { device: 'wasm', dtype: 'q8' },
25+};
26+
27+let asr = null;
28+let loadedDevice = null;
29+let chain = Promise.resolve();
30+
31+const post = (msg) => self.postMessage(msg);
32+
33+self.onmessage = (e) => {
34+ const msg = e.data;
35+ if (msg.type === 'load') {
36+ chain = chain.then(() => load(msg));
37+ } else if (msg.type === 'transcribe') {
38+ const audio = new Float32Array(msg.audio);
39+ chain = chain
40+ .then(() => transcribe(msg, audio))
41+ .catch((err) => {
42+ post({ type: 'error', message: String(err?.message || err) });
43+ post({ type: 'result', id: msg.id, utt: msg.utt, final: msg.final, text: '' });
44+ });
45+ }
46+};
47+
48+async function hasWebGPU() {
49+ if (!navigator.gpu) return false;
50+ try {
51+ return !!(await navigator.gpu.requestAdapter());
52+ } catch {
53+ return false;
54+ }
55+}
56+
57+// Sum download progress across files so the bar reflects the whole model.
58+function progressReporter() {
59+ const files = new Map();
60+ return (p) => {
61+ if (p.status === 'progress' && p.total) {
62+ files.set(p.file, { loaded: p.loaded, total: p.total });
63+ let loaded = 0;
64+ let total = 0;
65+ for (const f of files.values()) {
66+ loaded += f.loaded;
67+ total += f.total;
68+ }
69+ post({ type: 'progress', loaded, total });
70+ }
71+ };
72+}
73+
74+async function load({ model, device }) {
75+ if (asr) {
76+ post({ type: 'ready', device: loadedDevice });
77+ return;
78+ }
79+
80+ const target = device === 'cpu' ? 'wasm' : (await hasWebGPU()) ? 'webgpu' : 'wasm';
81+ post({ type: 'status', message: `loading model on ${target === 'webgpu' ? 'GPU' : 'CPU'}…` });
82+
83+ try {
84+ asr = await pipeline('automatic-speech-recognition', model, {
85+ ...BACKEND[target],
86+ progress_callback: progressReporter(),
87+ });
88+ // Warm up on 1 s of silence: this is where WebGPU compiles its shaders, and
89+ // where a backend that loaded but cannot actually run gives itself away.
90+ post({ type: 'status', message: 'warming up…' });
91+ await asr(new Float32Array(SAMPLE_RATE));
92+ loadedDevice = target;
93+ post({ type: 'ready', device: target });
94+ } catch (err) {
95+ asr = null;
96+ console.warn(`voicenote: ${target} backend failed`, err);
97+ post({
98+ type: 'error',
99+ fatal: true,
100+ retryOnCpu: target === 'webgpu',
101+ message: String(err?.message || err),
102+ });
103+ }
104+}
105+
106+async function transcribe(msg, audio) {
107+ const out = await asr(audio);
108+ post({ type: 'result', id: msg.id, utt: msg.utt, final: msg.final, text: out?.text ?? '' });
109+}
worklet.jsadded+30−0View file
@@ -0,0 +1,30 @@
1+// AudioWorklet processor: batches mono input samples into ~512-sample chunks
2+// and posts them to the main thread. Outputs silence (it is connected to the
3+// destination only so the graph keeps pulling audio through it).
4+class CaptureProcessor extends AudioWorkletProcessor {
5+ constructor() {
6+ super();
7+ this.buf = new Float32Array(512);
8+ this.n = 0;
9+ }
10+
11+ process(inputs) {
12+ const ch = inputs[0] && inputs[0][0];
13+ if (ch) {
14+ let i = 0;
15+ while (i < ch.length) {
16+ const take = Math.min(ch.length - i, this.buf.length - this.n);
17+ this.buf.set(ch.subarray(i, i + take), this.n);
18+ this.n += take;
19+ i += take;
20+ if (this.n === this.buf.length) {
21+ this.port.postMessage(this.buf.slice());
22+ this.n = 0;
23+ }
24+ }
25+ }
26+ return true;
27+ }
28+}
29+
30+registerProcessor('capture', CaptureProcessor);