Live transcription and explicit record/transcribe subcommands
The CLI now has two subcommands: record <room> and transcribe <dir> (the
bare-room form is gone). record --transcribe transcribes on the fly: a
persistent faster-whisper process serves chunk requests over line-JSON
stdio, reading flushed frame ranges straight from the growing WAVs; chunks
are cut at natural pauses (300 ms below an RMS threshold, 45 s force-cut)
tracked from the live sample stream, and transcript.md is re-rendered
continuously, finalized on stop. ASR results are persisted in the same
asr/*.json format the offline subcommand caches, so recordings can be
re-rendered or upgraded to a bigger model later. A helper failure never
affects the recording.
New end-to-end test (test:live): a speaker streams the JFK sample into a
room and the test asserts the transcript grew while still recording.
13 changed files+1010−196
.github/workflows/deploy.ymlmodified+4−2View file
@@ -51,8 +51,10 @@ jobs:
5151 <p>Record a <a href="https://concept-collection.github.io/commonroom/">commonroom</a>
5252 call from the command line: per-participant WAV files plus the room chat,
5353 for transcribing the meeting. Needs Node ≥ 22.</p>
54- <pre>npx https://concept-collection.github.io/commonroom-recorder/commonroom-recorder.tgz <room></pre>
55- <p>Afterwards, create a speaker-attributed transcript of the meeting:</p>
54+ <pre>npx https://concept-collection.github.io/commonroom-recorder/commonroom-recorder.tgz record <room></pre>
55+ <p>Add <code>--transcribe</code> to grow a speaker-attributed transcript live
56+ during the meeting (needs <code>pip install faster-whisper</code>), or create one
57+ afterwards:</p>
5658 <pre>npx https://concept-collection.github.io/commonroom-recorder/commonroom-recorder.tgz transcribe <recording-dir></pre>
5759 <p>Stop recording with Ctrl-C. See the
5860 <a href="https://github.com/concept-collection/commonroom-recorder">README</a>
CLAUDE.mdmodified+23−2View file
@@ -15,8 +15,11 @@ src/
1515 peer.ts ported, adapted to @roamhq/wrtc; receive-only media (see below)
1616 wav.ts incremental WAV writer (buffers ~1 s, re-patches header sizes)
1717 recorder.ts the heart: presence, mesh, control channel, audio sinks, files
18+ render.ts shared transcript timeline: turn merging, md/json rendering
1819 transcribe.ts `transcribe` subcommand: per-WAV ASR + merged transcript.md
19- cli.ts arg parsing, subcommand dispatch, signal handling, log lines
20+ livetranscribe.ts record --transcribe: persistent faster-whisper helper,
21+ silence-cut chunking, live transcript re-rendering
22+ cli.ts record/transcribe subcommands, signal handling, log lines
2023 test/
2124 speaker.ts synthetic participant: sine tone + one chat message
2225 loopback.ts test: recorder + speaker in a random room -> tone + chat
@@ -67,6 +70,21 @@ src/
6770 which also skips the padded silence — then whisper-cli, then whisper);
6871 raw ASR is cached per WAV in `<dir>/asr/`. cli.ts imports recorder.js
6972 LAZILY so transcribe works where the wrtc native module doesn't load.
73+- **Live transcription (record --transcribe) never blocks recording.** One
74+ persistent python process (faster-whisper only) serves requests over
75+ line-JSON stdio; it reads raw flushed frame ranges straight from the
76+ growing WAVs (header bypassed). Chunks are cut at >= 300 ms of quiet
77+ (RMS < 300) tracked from the live sample stream — never mid-word — with a
78+ 45 s force-cut for unbroken speech; all-quiet chunks skip ASR entirely.
79+ Results re-render transcript.md live and are persisted to asr/*.json in the
80+ offline format, so `transcribe` can re-render or upgrade models later. If
81+ the helper dies, it logs once and recording continues.
82+- **Spawn the persistent helper with `python3 -c <script>`**, never
83+ `python3 -` + script on stdin: `-` reads stdin to EOF before executing, so
84+ a process that keeps stdin open for requests never starts. (The one-shot
85+ offline helper uses stdin+end() and is fine.) In the helper, read requests
86+ with `sys.stdin.readline()`, not `for line in sys.stdin` (read-ahead
87+ buffering sits on complete lines).
7088
7189 ## Testing
7290
@@ -74,7 +92,10 @@ src/
7492 public relays (needs network): asserts the recorded WAV contains the 440 Hz
7593 tone (RMS + zero-crossing rate) and the chat message landed exactly once.
7694 `npm run test:transcribe` — real-speech transcription test (downloads the
77-whisper.cpp JFK sample + the tiny model on first run).
95+whisper.cpp JFK sample + the tiny model on first run). `npm run test:live` —
96+end-to-end LIVE transcription: a speaker streams the JFK WAV into a room
97+(`speaker.js --wav`; playback and leave countdown start at first connect) and
98+the test asserts the transcript grew while still recording.
7899 Segfault-at-exit in a child process = some path bypassed `process.exit()`.
79100 For manual testing against real browsers, record a room and join it at
80101 https://concept-collection.github.io/commonroom/ — let the user do
README.mdmodified+16−3View file
@@ -16,7 +16,7 @@ No install needed — run it straight from this repo's GitHub Pages tarball
1616 (nothing is published to npm):
1717
1818 ```
19-npx https://concept-collection.github.io/commonroom-recorder/commonroom-recorder.tgz <room> [options]
19+npx https://concept-collection.github.io/commonroom-recorder/commonroom-recorder.tgz record <room> [options]
2020 ```
2121
2222 (`npx github:concept-collection/commonroom-recorder` works too, but builds
@@ -28,7 +28,7 @@ or `rm -rf ~/.npm/_npx`. Or from a clone:
2828 ```
2929 npm install
3030 npm run build
31-node dist/cli.js <room> [options]
31+node dist/cli.js record <room> [options]
3232 ```
3333
3434 Options:
@@ -40,6 +40,10 @@ Options:
4040 --notice <text> Chat line sent to each participant on connect
4141 (default: "🔴 This meeting is being recorded.")
4242 --no-notice Don't send any recording notice
43+--transcribe Transcribe on the fly (needs faster-whisper): the
44+ transcript grows in <out>/transcript.md during the meeting
45+--model <m> Whisper model for --transcribe (default: small)
46+--language <xx> Force a language (default: auto-detect)
4347 ```
4448
4549 Stop with Ctrl-C. Requires Node >= 22 (built-in WebSocket). The WebRTC stack
@@ -48,11 +52,20 @@ ships prebuilt binaries for Linux and macOS.
4852
4953 ## Transcribing
5054
55+With `record --transcribe`, transcription happens **live during the meeting**:
56+a single faster-whisper model stays loaded, audio is transcribed in chunks cut
57+at natural pauses, and `<out>/transcript.md` is continuously rewritten — open
58+it (or `watch cat`) while the meeting runs, and it is finalized the moment you
59+stop. If the transcriber ever fails, the recording is unaffected.
60+
61+Alternatively (or to redo a recording with a bigger model), transcribe
62+afterwards:
63+
5164 ```
5265 npx https://concept-collection.github.io/commonroom-recorder/commonroom-recorder.tgz transcribe <recording-dir>
5366 ```
5467
55-produces `transcript.md` — a merged, speaker-attributed transcript of the
68+Both produce `transcript.md` — a merged, speaker-attributed transcript of the
5669 meeting with the chat and join/left events interleaved on one timeline:
5770
5871 ```
package.jsonmodified+3−2View file
@@ -1,6 +1,6 @@
11 {
22 "name": "commonroom-recorder",
3- "version": "0.2.0",
3+ "version": "0.3.0",
44 "description": "CLI bot that joins a commonroom room and records every participant's audio (and the chat) to disk for transcription",
55 "type": "module",
66 "bin": {
@@ -15,7 +15,8 @@
1515 "prepare": "tsc -b",
1616 "record": "node dist/cli.js",
1717 "test:loopback": "node dist/test/loopback.js",
18- "test:transcribe": "node dist/test/transcribe-test.js"
18+ "test:transcribe": "node dist/test/transcribe-test.js",
19+ "test:live": "node dist/test/live-loopback.js"
1920 },
2021 "engines": {
2122 "node": ">=22"
src/cli.tsmodified+107−54View file
@@ -1,21 +1,26 @@
11 #!/usr/bin/env node
22 import * as path from 'node:path'
33
4-const USAGE = `Usage: commonroom-recorder <room> [options]
4+const USAGE = `Usage: commonroom-recorder record <room> [options]
55 commonroom-recorder transcribe <recording-dir> [options]
66
7-Joins the commonroom room as a visible, muted participant and records every
8-other participant's audio to per-speaker WAV files, plus the room chat.
9-Stop with Ctrl-C. The transcribe subcommand then turns a recording directory
10-into a merged, speaker-attributed transcript (transcript.md).
7+record joins the commonroom room as a visible, muted participant and records
8+every other participant's audio to per-speaker WAV files, plus the room chat.
9+Stop with Ctrl-C. transcribe turns a recording directory into a merged,
10+speaker-attributed transcript (transcript.md).
1111
12-Recording options:
12+Record options:
1313 --name <name> Display name in the room (default: Recorder)
1414 --out <dir> Output directory (default: ./recordings/<room>-<timestamp>)
1515 --duration <sec> Stop automatically after this many seconds
1616 --notice <text> Chat line sent to each participant on connect
1717 (default: "🔴 This meeting is being recorded.")
1818 --no-notice Don't send any recording notice
19+ --transcribe Transcribe on the fly (needs faster-whisper): the
20+ transcript grows in <out>/transcript.md during the
21+ meeting and is finalized on stop
22+ --model <m> Whisper model for --transcribe (default: small)
23+ --language <xx> Force a language (default: auto-detect)
1924
2025 Transcribe options:
2126 --engine <e> auto | faster-whisper | whisper-cli | whisper
@@ -28,20 +33,37 @@ Transcribe options:
2833
2934 const DEFAULT_NOTICE = '🔴 This meeting is being recorded.'
3035
31-interface Args {
36+function fail(msg: string): never {
37+ process.stderr.write(`${msg}\n\n${USAGE}`)
38+ process.exit(1)
39+}
40+
41+const now = (): string => {
42+ const d = new Date()
43+ const p = (n: number) => String(n).padStart(2, '0')
44+ return `${p(d.getHours())}:${p(d.getMinutes())}:${p(d.getSeconds())}`
45+}
46+
47+// ---- record ---------------------------------------------------------------
48+
49+interface RecordArgs {
3250 room: string
3351 name: string
3452 out: string
3553 duration: number | null
3654 notice: string | null
55+ transcribe: {model: string | null; language: string | null} | null
3756 }
3857
39-const parseArgs = (argv: string[]): Args => {
58+const parseRecordArgs = (argv: string[]): RecordArgs => {
4059 let room: string | null = null
4160 let name = 'Recorder'
4261 let out: string | null = null
4362 let duration: number | null = null
4463 let notice: string | null = DEFAULT_NOTICE
64+ let transcribe = false
65+ let model: string | null = null
66+ let language: string | null = null
4567 for (let i = 0; i < argv.length; i++) {
4668 const a = argv[i]!
4769 switch (a) {
@@ -68,6 +90,15 @@ const parseArgs = (argv: string[]): Args => {
6890 case '--no-notice':
6991 notice = null
7092 break
93+ case '--transcribe':
94+ transcribe = true
95+ break
96+ case '--model':
97+ model = argv[++i] ?? null
98+ break
99+ case '--language':
100+ language = argv[++i] ?? null
101+ break
71102 default:
72103 if (a.startsWith('-')) fail(`Unknown option: ${a}`)
73104 if (room !== null) fail('Only one room may be given')
@@ -84,20 +115,69 @@ const parseArgs = (argv: string[]): Args => {
84115 `${ts.getFullYear()}${p(ts.getMonth() + 1)}${p(ts.getDate())}-` +
85116 `${p(ts.getHours())}${p(ts.getMinutes())}${p(ts.getSeconds())}`
86117 )
87- return {room, name, out: out ?? defaultOut, duration, notice: notice || null}
118+ return {
119+ room,
120+ name,
121+ out: out ?? defaultOut,
122+ duration,
123+ notice: notice || null,
124+ transcribe: transcribe ? {model, language} : null
125+ }
88126 }
89127
90-function fail(msg: string): never {
91- process.stderr.write(`${msg}\n\n${USAGE}`)
92- process.exit(1)
93-}
128+const recordMain = async (argv: string[]) => {
129+ const args = parseRecordArgs(argv)
130+ if (args.transcribe) {
131+ const {hasFasterWhisper} = await import('./transcribe.js')
132+ if (!hasFasterWhisper()) {
133+ fail(
134+ 'Live transcription needs faster-whisper:\n' +
135+ ' pip install faster-whisper (python3 must be on the PATH)\n' +
136+ 'Or record without --transcribe and run the transcribe subcommand later.'
137+ )
138+ }
139+ }
140+ // Imported lazily so `transcribe` works even where the wrtc native module
141+ // doesn't load.
142+ const {Recorder} = await import('./recorder.js')
143+ const recorder = new Recorder({
144+ room: args.room,
145+ name: args.name,
146+ outDir: args.out,
147+ notice: args.notice,
148+ transcribe: args.transcribe,
149+ onLog: line => process.stdout.write(`[${now()}] ${line}\n`),
150+ onFatal: message => {
151+ process.stderr.write(`[${now()}] ${message}\n`)
152+ void recorder.stop().then(() => process.exit(1))
153+ }
154+ })
94155
95-const now = (): string => {
96- const d = new Date()
97- const p = (n: number) => String(n).padStart(2, '0')
98- return `${p(d.getHours())}:${p(d.getMinutes())}:${p(d.getSeconds())}`
156+ // NOTE: every exit path must go through process.exit(): @roamhq/wrtc
157+ // segfaults in its static destructors on a natural process exit when
158+ // nonstandard media sources exist.
159+ let shuttingDown = false
160+ const shutdown = () => {
161+ if (shuttingDown) process.exit(1) // second Ctrl-C: give up waiting
162+ shuttingDown = true
163+ process.stdout.write(`\n[${now()}] stopping...\n`)
164+ void recorder.stop().then(summary => {
165+ process.stdout.write(
166+ `[${now()}] done: ${summary.segments} audio segment(s) from ` +
167+ `${summary.participants} participant(s) in ${args.out}\n`
168+ )
169+ process.exit(0)
170+ })
171+ }
172+ process.on('SIGINT', shutdown)
173+ process.on('SIGTERM', shutdown)
174+ if (args.duration !== null) setTimeout(shutdown, args.duration * 1000)
175+
176+ await recorder.start()
99177 }
100178
179+// ---- transcribe -----------------------------------------------------------
180+
101181 const transcribeMain = async (argv: string[]) => {
102182 let dir: string | null = null
103183 let engine: 'auto' | 'faster-whisper' | 'whisper-cli' | 'whisper' = 'auto'
@@ -148,45 +228,18 @@ const transcribeMain = async (argv: string[]) => {
148228 process.exit(0)
149229 }
150230
151-const main = async () => {
152- if (process.argv[2] === 'transcribe') {
153- await transcribeMain(process.argv.slice(3))
154- return
155- }
156- const args = parseArgs(process.argv.slice(2))
157- // Imported lazily so `transcribe` works even where the wrtc native module
158- // doesn't load.
159- const {Recorder} = await import('./recorder.js')
160- const recorder = new Recorder({
161- room: args.room,
162- name: args.name,
163- outDir: args.out,
164- notice: args.notice,
165- onLog: line => process.stdout.write(`[${now()}] ${line}\n`),
166- onFatal: message => {
167- process.stderr.write(`[${now()}] ${message}\n`)
168- recorder.stop()
169- process.exit(1)
170- }
171- })
231+// ---- dispatch -------------------------------------------------------------
172232
173- // NOTE: every exit path must go through process.exit(): @roamhq/wrtc
174- // segfaults in its static destructors on a natural process exit when
175- // nonstandard media sources exist.
176- const shutdown = () => {
177- process.stdout.write(`\n[${now()}] stopping...\n`)
178- const summary = recorder.stop()
179- process.stdout.write(
180- `[${now()}] done: ${summary.segments} audio segment(s) from ` +
181- `${summary.participants} participant(s) in ${args.out}\n`
182- )
183- process.exit(0)
233+const main = async () => {
234+ const cmd = process.argv[2]
235+ const rest = process.argv.slice(3)
236+ if (cmd === 'record') return recordMain(rest)
237+ if (cmd === 'transcribe') return transcribeMain(rest)
238+ if (cmd === '--help' || cmd === '-h' || cmd === undefined) {
239+ process.stdout.write(USAGE)
240+ process.exit(cmd === undefined ? 1 : 0)
184241 }
185- process.on('SIGINT', shutdown)
186- process.on('SIGTERM', shutdown)
187- if (args.duration !== null) setTimeout(shutdown, args.duration * 1000)
188-
189- await recorder.start()
242+ fail(`Unknown command: ${cmd}`)
190243 }
191244
192245 main().catch(err => {
src/livetranscribe.tsadded+444−0View file
@@ -0,0 +1,444 @@
1+import {spawn, type ChildProcess} from 'node:child_process'
2+import * as fs from 'node:fs'
3+import * as path from 'node:path'
4+import {mergeTurns, renderJson, renderMarkdown, type Item} from './render.js'
5+import type {WavWriter} from './wav.js'
6+
7+// Live transcription during recording (the record command's --transcribe).
8+//
9+// A single persistent python process loads faster-whisper ONCE and then
10+// serves transcription requests over a line-delimited JSON stdin/stdout
11+// protocol; each request names a WAV file and a frame range, which the helper
12+// reads raw from disk (only flushed bytes are ever requested, so reading a
13+// file that is still being appended to is safe — the header is bypassed).
14+//
15+// The recorder feeds every decoded audio frame through onAudio, which tracks
16+// silence so chunks can be cut at natural pauses (>= 300 ms below the RMS
17+// threshold) — never mid-word. Every tick, each segment with >= MIN_CHUNK of
18+// speech up to a silence cut is dispatched (or force-cut at MAX_CHUNK of
19+// unbroken speech). transcript.md / transcript.json are re-rendered after
20+// every result, so the transcript grows while the meeting is happening.
21+//
22+// If the helper dies, recording is NEVER affected: live transcription
23+// disables itself with a log line, and `transcribe` can be run afterwards.
24+
25+const QUIET_RMS = 300 // ~ -41 dBFS: below this a 10 ms frame counts as quiet
26+const QUIET_CUT_MS = 300 // this much consecutive quiet = a safe cut point
27+const TICK_MS = 10000
28+const MIN_CHUNK_SEC = 5 // don't bother the model with less than this
29+const MAX_CHUNK_SEC = 45 // force a cut after this much unbroken speech
30+const FINISH_TIMEOUT_MS = 180000
31+
32+interface AsrSegment {
33+ start: number
34+ end: number
35+ text: string
36+}
37+
38+const LIVE_PY = `
39+import sys, json
40+import numpy as np
41+from faster_whisper import WhisperModel
42+model_name = sys.argv[1]
43+language = None if sys.argv[2] == "-" else sys.argv[2]
44+model = WhisperModel(model_name, device="auto", compute_type="auto")
45+print(json.dumps({"ready": True}), flush=True)
46+while True:
47+ # readline, not iteration: iterating sys.stdin read-ahead-buffers and can
48+ # sit on a complete line without yielding it
49+ line = sys.stdin.readline()
50+ if not line:
51+ break
52+ req = json.loads(line)
53+ ch = req["channels"]
54+ with open(req["wav"], "rb") as f:
55+ f.seek(44 + req["startFrame"] * ch * 2)
56+ raw = f.read((req["endFrame"] - req["startFrame"]) * ch * 2)
57+ a = np.frombuffer(raw, dtype=np.int16).astype(np.float32) / 32768.0
58+ if ch > 1:
59+ a = a.reshape(-1, ch).mean(axis=1)
60+ rate = req["rate"]
61+ if rate != 16000:
62+ if rate % 16000 == 0:
63+ k = rate // 16000
64+ n = (len(a) // k) * k
65+ a = a[:n].reshape(-1, k).mean(axis=1)
66+ else:
67+ xi = np.arange(0, len(a), rate / 16000.0)
68+ a = np.interp(xi, np.arange(len(a)), a)
69+ segs, info = model.transcribe(a.astype(np.float32), language=language, vad_filter=True)
70+ out = [{"start": s.start, "end": s.end, "text": s.text} for s in segs]
71+ print(json.dumps({"id": req["id"], "segments": out}), flush=True)
72+`
73+
74+interface AsrRequest {
75+ wav: string
76+ rate: number
77+ channels: number
78+ startFrame: number
79+ endFrame: number
80+}
81+
82+/** The persistent faster-whisper helper process. */
83+class PythonAsr {
84+ private proc: ChildProcess
85+ private pending = new Map<
86+ number,
87+ {resolve: (segs: AsrSegment[]) => void; reject: (err: Error) => void}
88+ >()
89+ private nextId = 1
90+ private buf = ''
91+ private readyResolve!: () => void
92+ private readyReject!: (err: Error) => void
93+ readonly ready: Promise<void>
94+ dead = false
95+
96+ constructor(model: string, language: string | null, onLog: (l: string) => void) {
97+ this.ready = new Promise((resolve, reject) => {
98+ this.readyResolve = resolve
99+ this.readyReject = reject
100+ })
101+ // NB: `python3 -c <script>` — NOT `python3 -` + script on stdin, which
102+ // would read stdin to EOF before executing and never see our requests.
103+ // With -c, sys.argv is ['-c', model, language].
104+ this.proc = spawn('python3', ['-c', LIVE_PY, model, language ?? '-'], {
105+ stdio: ['pipe', 'pipe', 'pipe']
106+ })
107+ this.proc.stdout!.on('data', d => {
108+ this.buf += d
109+ const lines = this.buf.split('\n')
110+ this.buf = lines.pop() ?? ''
111+ for (const line of lines) {
112+ if (!line.trim()) continue
113+ let msg: {ready?: boolean; id?: number; segments?: AsrSegment[]}
114+ try {
115+ msg = JSON.parse(line)
116+ } catch {
117+ continue
118+ }
119+ if (msg.ready) {
120+ this.readyResolve()
121+ continue
122+ }
123+ if (typeof msg.id === 'number') {
124+ const p = this.pending.get(msg.id)
125+ this.pending.delete(msg.id)
126+ p?.resolve(msg.segments ?? [])
127+ }
128+ }
129+ })
130+ let errTail = ''
131+ this.proc.stderr!.on('data', d => {
132+ const line = String(d).trim()
133+ if (line) errTail = line.slice(0, 300)
134+ })
135+ this.proc.on('error', err => this.die(err.message))
136+ this.proc.on('exit', code => {
137+ if (!this.dead && code !== 0) {
138+ this.die(`helper exited with code ${code}${errTail ? ` (${errTail})` : ''}`)
139+ onLog(`live transcription disabled: ${errTail || `helper exited (${code})`}`)
140+ }
141+ })
142+ }
143+
144+ private die(reason: string) {
145+ if (this.dead) return
146+ this.dead = true
147+ this.readyReject(new Error(reason))
148+ // ready may already be resolved; a handled rejection after that is fine
149+ this.ready.catch(() => undefined)
150+ for (const p of this.pending.values()) p.reject(new Error(reason))
151+ this.pending.clear()
152+ }
153+
154+ request(req: AsrRequest): Promise<AsrSegment[]> {
155+ if (this.dead) return Promise.reject(new Error('helper is dead'))
156+ const id = this.nextId++
157+ return new Promise((resolve, reject) => {
158+ this.pending.set(id, {resolve, reject})
159+ this.proc.stdin!.write(JSON.stringify({id, ...req}) + '\n')
160+ })
161+ }
162+
163+ close() {
164+ this.dead = true
165+ try {
166+ this.proc.stdin!.end()
167+ } catch {
168+ /* ignore */
169+ }
170+ setTimeout(() => {
171+ try {
172+ this.proc.kill('SIGKILL')
173+ } catch {
174+ /* ignore */
175+ }
176+ }, 2000).unref()
177+ }
178+}
179+
180+interface LiveSeg {
181+ file: string // relative, as in the manifest
182+ absPath: string
183+ name: string
184+ rate: number
185+ channels: number
186+ startMs: number
187+ writer: WavWriter | null // null once the segment has ended
188+ /** Frames handed to ASR (or skipped as silence) so far. */
189+ transcribedUpTo: number
190+ /** Latest safe cut point (end of a >= 300 ms quiet stretch). */
191+ lastCutFrame: number
192+ lastEndFrame: number
193+ quietMs: number
194+ /** Any non-quiet audio since transcribedUpTo? */
195+ hasSpeech: boolean
196+ utterances: AsrSegment[] // in-file seconds
197+ ended: boolean
198+ endFrames: number
199+ pendingOps: number
200+ cacheWritten: boolean
201+}
202+
203+export interface LiveTranscriberOptions {
204+ outDir: string
205+ room: string
206+ model: string | null
207+ language: string | null
208+ startedAtMs: number
209+ onLog: (line: string) => void
210+}
211+
212+export class LiveTranscriber {
213+ private asr: PythonAsr
214+ private segs = new Map<string, LiveSeg>()
215+ private eventItems: Item[] = []
216+ private inFlight = new Set<Promise<unknown>>()
217+ private timer: ReturnType<typeof setInterval> | null = null
218+ private renderTimer: ReturnType<typeof setTimeout> | null = null
219+ private model: string
220+
221+ constructor(private opts: LiveTranscriberOptions) {
222+ this.model = opts.model ?? 'small'
223+ this.asr = new PythonAsr(this.model, opts.language, opts.onLog)
224+ opts.onLog(`live transcription: loading model ${this.model}...`)
225+ void this.asr.ready.then(
226+ () => opts.onLog('live transcription ready'),
227+ () => undefined // logged by PythonAsr
228+ )
229+ this.timer = setInterval(() => this.tick(), TICK_MS)
230+ }
231+
232+ onSegmentStart(
233+ file: string,
234+ absPath: string,
235+ name: string,
236+ rate: number,
237+ channels: number,
238+ startMs: number,
239+ writer: WavWriter
240+ ) {
241+ this.segs.set(file, {
242+ file,
243+ absPath,
244+ name,
245+ rate,
246+ channels,
247+ startMs,
248+ writer,
249+ transcribedUpTo: 0,
250+ lastCutFrame: 0,
251+ lastEndFrame: 0,
252+ quietMs: 0,
253+ hasSpeech: false,
254+ utterances: [],
255+ ended: false,
256+ endFrames: 0,
257+ pendingOps: 0,
258+ cacheWritten: false
259+ })
260+ }
261+
262+ /** Called for every decoded frame batch; endFrame = writer.framesWritten
263+ * AFTER appending (so padded gaps show up as position jumps). */
264+ onAudio(file: string, endFrame: number, samples: Int16Array) {
265+ const seg = this.segs.get(file)
266+ if (!seg || seg.ended) return
267+ const frames = samples.length / seg.channels
268+ const startFrame = endFrame - frames
269+ if (startFrame > seg.lastEndFrame) {
270+ // A silence-padded gap was inserted before this batch: it is all quiet,
271+ // so the start of the current batch is a safe cut point.
272+ seg.quietMs = 0
273+ seg.lastCutFrame = startFrame
274+ }
275+ seg.lastEndFrame = endFrame
276+ let sumSq = 0
277+ for (let i = 0; i < samples.length; i++) sumSq += samples[i]! * samples[i]!
278+ const rms = Math.sqrt(sumSq / samples.length)
279+ if (rms < QUIET_RMS) {
280+ seg.quietMs += (frames / seg.rate) * 1000
281+ if (seg.quietMs >= QUIET_CUT_MS) seg.lastCutFrame = endFrame
282+ } else {
283+ seg.quietMs = 0
284+ seg.hasSpeech = true
285+ }
286+ }
287+
288+ onSegmentEnd(file: string) {
289+ const seg = this.segs.get(file)
290+ if (!seg || seg.ended) return
291+ seg.ended = true
292+ seg.endFrames = seg.writer?.framesWritten ?? seg.lastEndFrame
293+ seg.writer = null
294+ this.dispatch(seg, seg.endFrames)
295+ this.maybeWriteCache(seg)
296+ }
297+
298+ /** Chat / join / left items from the recorder, for the rendered timeline. */
299+ onEvent(item: Item) {
300+ this.eventItems.push(item)
301+ this.scheduleRender()
302+ }
303+
304+ private tick() {
305+ for (const seg of this.segs.values()) {
306+ if (seg.ended || !seg.writer) continue
307+ const flushed = seg.writer.flushedFrames
308+ const cut = Math.min(seg.lastCutFrame, flushed)
309+ if (cut - seg.transcribedUpTo >= MIN_CHUNK_SEC * seg.rate) {
310+ this.dispatch(seg, cut)
311+ } else if (flushed - seg.transcribedUpTo >= MAX_CHUNK_SEC * seg.rate) {
312+ this.dispatch(seg, flushed) // unbroken speech: cut anyway
313+ }
314+ }
315+ }
316+
317+ private dispatch(seg: LiveSeg, to: number) {
318+ const from = seg.transcribedUpTo
319+ if (to <= from) return
320+ seg.transcribedUpTo = to
321+ const hadSpeech = seg.hasSpeech
322+ seg.hasSpeech = false
323+ if (!hadSpeech || this.asr.dead) return
324+ seg.pendingOps++
325+ const op = this.asr
326+ .request({
327+ wav: seg.absPath,
328+ rate: seg.rate,
329+ channels: seg.channels,
330+ startFrame: from,
331+ endFrame: to
332+ })
333+ .then(segments => {
334+ for (const s of segments) {
335+ const text = s.text.trim()
336+ if (!text) continue
337+ seg.utterances.push({
338+ start: s.start + from / seg.rate,
339+ end: s.end + from / seg.rate,
340+ text
341+ })
342+ }
343+ if (segments.length > 0) this.scheduleRender()
344+ })
345+ .catch(() => undefined) // helper death is logged once by PythonAsr
346+ .finally(() => {
347+ seg.pendingOps--
348+ this.inFlight.delete(op)
349+ this.maybeWriteCache(seg)
350+ })
351+ this.inFlight.add(op)
352+ }
353+
354+ /** Once a segment has ended and drained, persist its ASR results in the
355+ * same asr/*.json format the offline transcribe subcommand uses/caches. */
356+ private maybeWriteCache(seg: LiveSeg) {
357+ if (!seg.ended || seg.pendingOps > 0 || seg.cacheWritten) return
358+ seg.cacheWritten = true
359+ if (this.asr.dead && seg.utterances.length === 0) return // let offline redo it
360+ const asrDir = path.join(this.opts.outDir, 'asr')
361+ try {
362+ fs.mkdirSync(asrDir, {recursive: true})
363+ fs.writeFileSync(
364+ path.join(asrDir, path.basename(seg.file).replace(/\.wav$/i, '') + '.json'),
365+ JSON.stringify(
366+ {
367+ engine: 'faster-whisper',
368+ model: this.model,
369+ segments: [...seg.utterances].sort((a, b) => a.start - b.start)
370+ },
371+ null,
372+ 2
373+ ) + '\n'
374+ )
375+ } catch {
376+ /* ignore */
377+ }
378+ }
379+
380+ private scheduleRender() {
381+ if (this.renderTimer !== null) return
382+ this.renderTimer = setTimeout(() => {
383+ this.renderTimer = null
384+ this.render(null)
385+ }, 1000)
386+ }
387+
388+ private render(endedAtMs: number | null) {
389+ const items: Item[] = [...this.eventItems]
390+ for (const seg of this.segs.values()) {
391+ for (const u of seg.utterances) {
392+ items.push({
393+ timeMs: seg.startMs + u.start * 1000,
394+ endMs: seg.startMs + u.end * 1000,
395+ type: 'speech',
396+ speaker: seg.name,
397+ text: u.text
398+ })
399+ }
400+ }
401+ const merged = mergeTurns(items)
402+ const meta = {
403+ room: this.opts.room,
404+ startedAtMs: this.opts.startedAtMs,
405+ endedAtMs,
406+ speakers: [...new Set([...this.segs.values()].map(s => s.name))],
407+ engine: 'faster-whisper (live)',
408+ model: this.model
409+ }
410+ try {
411+ fs.writeFileSync(
412+ path.join(this.opts.outDir, 'transcript.md'),
413+ renderMarkdown(meta, merged)
414+ )
415+ fs.writeFileSync(
416+ path.join(this.opts.outDir, 'transcript.json'),
417+ renderJson(meta, merged)
418+ )
419+ } catch {
420+ /* ignore */
421+ }
422+ }
423+
424+ /** All segments have been ended by the recorder; drain and finalize. */
425+ async finish(endedAtMs: number): Promise<void> {
426+ if (this.timer !== null) clearInterval(this.timer)
427+ this.timer = null
428+ if (this.renderTimer !== null) clearTimeout(this.renderTimer)
429+ this.renderTimer = null
430+ if (this.inFlight.size > 0) {
431+ this.opts.onLog(
432+ `waiting for ${this.inFlight.size} transcription chunk(s) to finish...`
433+ )
434+ await Promise.race([
435+ Promise.allSettled([...this.inFlight]),
436+ new Promise(r => setTimeout(r, FINISH_TIMEOUT_MS))
437+ ])
438+ }
439+ this.asr.close()
440+ for (const seg of this.segs.values()) this.maybeWriteCache(seg)
441+ this.render(endedAtMs)
442+ this.opts.onLog(`wrote ${path.join(this.opts.outDir, 'transcript.md')}`)
443+ }
444+}
src/recorder.tsmodified+34−1View file
@@ -2,6 +2,7 @@ import * as fs from 'node:fs'
22 import * as path from 'node:path'
33 import wrtc from '@roamhq/wrtc'
44 import {selfId} from './identity.js'
5+import {LiveTranscriber} from './livetranscribe.js'
56 import {Nostr, peerTopic, roomTopic} from './nostr.js'
67 import {Peer, type Signal} from './peer.js'
78 import {WavWriter} from './wav.js'
@@ -94,6 +95,8 @@ export interface RecorderOptions {
9495 /** Chat line sent to each participant when we connect to them (so everyone
9596 * in the room sees, once, that recording is happening). null = none. */
9697 notice: string | null
98+ /** Live transcription (faster-whisper) while recording; null = off. */
99+ transcribe: {model: string | null; language: string | null} | null
97100 onLog: (line: string) => void
98101 /** Unrecoverable situation (e.g. the room is full). */
99102 onFatal: (message: string) => void
@@ -151,6 +154,7 @@ export class Recorder {
151154 private eventsPath: string
152155 private chatPath: string
153156 private manifestPath: string
157+ private liveT: LiveTranscriber | null = null
154158
155159 constructor(private opts: RecorderOptions) {
156160 this.audioDir = path.join(opts.outDir, 'audio')
@@ -162,6 +166,16 @@ export class Recorder {
162166 async start() {
163167 fs.mkdirSync(this.audioDir, {recursive: true})
164168 this.startedAtMs = Date.now()
169+ if (this.opts.transcribe) {
170+ this.liveT = new LiveTranscriber({
171+ outDir: this.opts.outDir,
172+ room: this.opts.room,
173+ model: this.opts.transcribe.model,
174+ language: this.opts.transcribe.language,
175+ startedAtMs: this.startedAtMs,
176+ onLog: this.opts.onLog
177+ })
178+ }
165179 this.event({type: 'start', room: this.opts.room, peerId: selfId, name: this.opts.name})
166180 this.chatLine(`* recording started (room: ${this.opts.room})`)
167181 this.opts.onLog(`joined room "${this.opts.room}" as "${this.opts.name}" (peer ${selfId.slice(0, 8)})`)
@@ -307,6 +321,7 @@ export class Recorder {
307321 this.event({type: 'left', peerId, name})
308322 this.chatLine(`* ${name} left`)
309323 this.opts.onLog(`${name} left`)
324+ this.liveT?.onEvent({timeMs: Date.now(), type: 'system', text: `${name} left`})
310325 }
311326 }
312327 }
@@ -380,6 +395,11 @@ export class Recorder {
380395 this.event({type: 'join', peerId, name, alreadyHere})
381396 this.chatLine(`* ${name} ${alreadyHere ? 'was already here' : 'joined'}`)
382397 this.opts.onLog(`${name} ${alreadyHere ? 'was already here' : 'joined'} (mic ${conn.audioMuted ? 'muted' : 'on'})`)
398+ this.liveT?.onEvent({
399+ timeMs: Date.now(),
400+ type: 'system',
401+ text: `${name} ${alreadyHere ? 'was already here' : 'joined'}`
402+ })
383403 }
384404 return
385405 }
@@ -411,6 +431,7 @@ export class Recorder {
411431 this.event({type: 'chat', peerId, name, text})
412432 this.chatLine(`${name}: ${text}`)
413433 this.opts.onLog(`${name}: ${text}`)
434+ this.liveT?.onEvent({timeMs: Date.now(), type: 'chat', speaker: name, text})
414435 return
415436 }
416437 case 'set': // room settings don't matter to the recorder
@@ -474,6 +495,15 @@ export class Recorder {
474495 }
475496 this.event({type: 'segment-start', peerId, name, file, sampleRate: rate, channels})
476497 this.opts.onLog(`recording ${name} -> ${file}`)
498+ this.liveT?.onSegmentStart(
499+ file,
500+ path.join(this.opts.outDir, file),
501+ name,
502+ rate,
503+ channels,
504+ now,
505+ conn.writer
506+ )
477507 } else {
478508 // If the sink stalled (network gap, DTX), pad with silence so sample
479509 // position keeps tracking wall-clock time.
@@ -484,12 +514,14 @@ export class Recorder {
484514 }
485515 }
486516 conn.writer.append(data.samples)
517+ this.liveT?.onAudio(conn.segment!.file, conn.writer.framesWritten, data.samples)
487518 }
488519 }
489520
490521 private endSegment(peerId: string, conn: Conn) {
491522 if (!conn.writer || !conn.segment) return
492523 conn.writer.finalize()
524+ this.liveT?.onSegmentEnd(conn.segment.file)
493525 conn.segment.endedAt = iso(Date.now())
494526 conn.segment.durationSec = Math.round(conn.writer.durationSec * 100) / 100
495527 this.segments.push(conn.segment)
@@ -565,7 +597,7 @@ export class Recorder {
565597
566598 // ---- shutdown -----------------------------------------------------------
567599
568- stop(): {segments: number; participants: number} {
600+ async stop(): Promise<{segments: number; participants: number}> {
569601 if (this.stopped) return {segments: this.segments.length, participants: this.names.size}
570602 this.stopped = true
571603 const bye = JSON.stringify({t: 'bye'} satisfies ControlMsg)
@@ -585,6 +617,7 @@ export class Recorder {
585617 this.event({type: 'stop'})
586618 this.chatLine('* recording stopped')
587619 this.writeManifest()
620+ if (this.liveT) await this.liveT.finish(Date.now())
588621 return {segments: this.segments.length, participants: this.names.size}
589622 }
590623 }
src/render.tsadded+103−0View file
@@ -0,0 +1,103 @@
1+// Transcript timeline assembly and rendering, shared by the offline
2+// `transcribe` subcommand and live transcription during recording.
3+
4+export type Item =
5+ | {timeMs: number; type: 'speech'; speaker: string; text: string; endMs: number}
6+ | {timeMs: number; type: 'chat'; speaker: string; text: string}
7+ | {timeMs: number; type: 'system'; text: string}
8+
9+export interface RenderMeta {
10+ room: string
11+ startedAtMs: number
12+ endedAtMs: number | null
13+ speakers: string[]
14+ engine: string
15+ model: string | null
16+}
17+
18+/** Merge consecutive same-speaker utterances separated by < this many ms. */
19+export const TURN_GAP_MS = 3000
20+
21+export const localTime = (ms: number): string => {
22+ const d = new Date(ms)
23+ const p = (n: number) => String(n).padStart(2, '0')
24+ return `${p(d.getHours())}:${p(d.getMinutes())}:${p(d.getSeconds())}`
25+}
26+
27+export const localDateTime = (ms: number): string => {
28+ const d = new Date(ms)
29+ const p = (n: number) => String(n).padStart(2, '0')
30+ return `${d.getFullYear()}-${p(d.getMonth() + 1)}-${p(d.getDate())} ${localTime(ms)}`
31+}
32+
33+/** Sort by time and merge adjacent same-speaker utterances (with nothing in
34+ * between) into conversational turns. */
35+export const mergeTurns = (items: Item[]): Item[] => {
36+ const sorted = [...items].sort((a, b) => a.timeMs - b.timeMs)
37+ const merged: Item[] = []
38+ for (const item of sorted) {
39+ const prev = merged[merged.length - 1]
40+ if (
41+ item.type === 'speech' &&
42+ prev?.type === 'speech' &&
43+ prev.speaker === item.speaker &&
44+ item.timeMs - prev.endMs < TURN_GAP_MS
45+ ) {
46+ prev.text += ' ' + item.text
47+ prev.endMs = item.endMs
48+ } else {
49+ merged.push({...item})
50+ }
51+ }
52+ return merged
53+}
54+
55+export const renderMarkdown = (meta: RenderMeta, merged: Item[]): string => {
56+ const md: string[] = []
57+ md.push(`# Transcript: ${meta.room}`)
58+ md.push('')
59+ md.push(
60+ `- **Recorded:** ${localDateTime(meta.startedAtMs)}${
61+ meta.endedAtMs ? ` – ${localTime(meta.endedAtMs)}` : ' – (in progress)'
62+ }`
63+ )
64+ md.push(`- **Speakers:** ${meta.speakers.join(', ') || '(none yet)'}`)
65+ md.push(
66+ `- **Transcribed with:** ${meta.engine}${meta.model ? `, model ${meta.model}` : ''}`
67+ )
68+ md.push('')
69+ md.push('---')
70+ md.push('')
71+ for (const item of merged) {
72+ const t = localTime(item.timeMs)
73+ if (item.type === 'speech') {
74+ md.push(`**[${t}] ${item.speaker}:** ${item.text}`)
75+ } else if (item.type === 'chat') {
76+ md.push(`> [${t}] 💬 **${item.speaker}:** ${item.text}`)
77+ } else {
78+ md.push(`*[${t}] — ${item.text}*`)
79+ }
80+ md.push('')
81+ }
82+ if (merged.length === 0) md.push('*(nothing yet)*', '')
83+ return md.join('\n')
84+}
85+
86+export const renderJson = (meta: RenderMeta, merged: Item[]): string =>
87+ JSON.stringify(
88+ {
89+ room: meta.room,
90+ startedAt: new Date(meta.startedAtMs).toISOString(),
91+ endedAt: meta.endedAtMs ? new Date(meta.endedAtMs).toISOString() : null,
92+ engine: meta.engine,
93+ model: meta.model,
94+ items: merged.map(item => ({
95+ time: new Date(item.timeMs).toISOString(),
96+ ...item,
97+ timeMs: undefined,
98+ endMs: undefined
99+ }))
100+ },
101+ null,
102+ 2
103+ ) + '\n'
src/test/live-loopback.tsadded+153−0View file
@@ -0,0 +1,153 @@
1+// End-to-end test of LIVE transcription (record --transcribe), no browser:
2+//
3+// 1. Start the recorder with --transcribe --model tiny in a random room.
4+// 2. Start a test speaker that plays the 11 s JFK sample into the room.
5+// 3. Check transcript.md appears (and has content) WHILE still recording.
6+// 4. SIGINT the recorder; verify the final transcript has the speaker, the
7+// right words, the chat line, and that the asr/ cache was written.
8+//
9+// Needs network (relays; sample + model download on first run).
10+
11+import {spawn, type ChildProcess} from 'node:child_process'
12+import * as fs from 'node:fs'
13+import * as os from 'node:os'
14+import * as path from 'node:path'
15+import {randomBytes} from 'node:crypto'
16+
17+const JFK_URL =
18+ 'https://raw.githubusercontent.com/ggml-org/whisper.cpp/master/samples/jfk.wav'
19+const SPEAK_SEC = 25 // after first connect: 11 s of speech, then silence
20+
21+const room = `livetest-${randomBytes(4).toString('hex')}`
22+const outDir = fs.mkdtempSync(path.join(os.tmpdir(), 'commonroom-live-test-'))
23+const dist = path.join(import.meta.dirname, '..')
24+
25+const failures: string[] = []
26+const check = (ok: boolean, what: string) => {
27+ process.stdout.write(`${ok ? 'PASS' : 'FAIL'}: ${what}\n`)
28+ if (!ok) failures.push(what)
29+}
30+
31+const run = (cmd: string[], label: string): ChildProcess => {
32+ const proc = spawn('node', cmd, {stdio: ['ignore', 'pipe', 'pipe']})
33+ proc.stdout!.on('data', d =>
34+ process.stdout.write(String(d).replace(/^/gm, ` ${label} | `))
35+ )
36+ proc.stderr!.on('data', d =>
37+ process.stdout.write(String(d).replace(/^/gm, ` ${label} ! `))
38+ )
39+ return proc
40+}
41+
42+const wait = (ms: number) => new Promise(r => setTimeout(r, ms))
43+
44+const exited = (proc: ChildProcess, timeoutMs: number): Promise<boolean> =>
45+ new Promise(resolve => {
46+ if (proc.exitCode !== null) return resolve(true)
47+ const t = setTimeout(() => {
48+ proc.kill('SIGKILL')
49+ resolve(false)
50+ }, timeoutMs)
51+ proc.on('exit', () => {
52+ clearTimeout(t)
53+ resolve(true)
54+ })
55+ })
56+
57+const main = async () => {
58+ const jfkPath = path.join(os.tmpdir(), 'commonroom-recorder-jfk-sample.wav')
59+ if (!fs.existsSync(jfkPath)) {
60+ process.stdout.write(`downloading ${JFK_URL}\n`)
61+ const res = await fetch(JFK_URL)
62+ if (!res.ok) throw new Error(`sample download failed: ${res.status}`)
63+ fs.writeFileSync(jfkPath, Buffer.from(await res.arrayBuffer()))
64+ }
65+
66+ process.stdout.write(`room: ${room}\nout: ${outDir}\n\n`)
67+ const recorder = run(
68+ [
69+ path.join(dist, 'cli.js'),
70+ 'record',
71+ room,
72+ '--out',
73+ outDir,
74+ '--transcribe',
75+ '--model',
76+ 'tiny',
77+ '--notice',
78+ 'recording test'
79+ ],
80+ 'rec'
81+ )
82+ await wait(3000)
83+ const speaker = run(
84+ [
85+ path.join(dist, 'test/speaker.js'),
86+ room,
87+ '--wav',
88+ jfkPath,
89+ '--duration',
90+ String(SPEAK_SEC),
91+ '--chat',
92+ 'live transcription test chat'
93+ ],
94+ 'spk'
95+ )
96+
97+ // The speaker exits SPEAK_SEC after its first connection (plus connect
98+ // time); poll for the live transcript while it runs.
99+ const mdPath = path.join(outDir, 'transcript.md')
100+ let liveSeen = false
101+ const speakerDone = exited(speaker, 180000)
102+ let done = false
103+ void speakerDone.then(() => (done = true))
104+ while (!done) {
105+ if (!liveSeen && fs.existsSync(mdPath)) {
106+ const md = fs.readFileSync(mdPath, 'utf8')
107+ // A SPEECH turn (bold "**[hh:mm:ss] Name:**"), not the chat line.
108+ if (/^\*\*\[\d\d:\d\d:\d\d\] TestSpeaker:\*\*/m.test(md)) {
109+ liveSeen = true
110+ process.stdout.write(' (live transcript has speech — still recording)\n')
111+ }
112+ }
113+ await wait(1000)
114+ }
115+ check(await speakerDone, 'speaker ran and exited')
116+ check(liveSeen, 'transcript.md grew DURING the recording')
117+
118+ await wait(1500)
119+ recorder.kill('SIGINT')
120+ check(await exited(recorder, 120000), 'recorder exited cleanly on SIGINT')
121+
122+ check(fs.existsSync(mdPath), 'transcript.md written')
123+ const md = fs.readFileSync(mdPath, 'utf8')
124+ check(
125+ /^\*\*\[\d\d:\d\d:\d\d\] TestSpeaker:\*\*/m.test(md),
126+ 'speaker has a speech turn in transcript'
127+ )
128+ check(/your country/i.test(md), 'JFK words transcribed')
129+ check(md.includes('live transcription test chat'), 'chat line in transcript')
130+ check(!md.includes('(in progress)'), 'final render has an end time')
131+
132+ const json = JSON.parse(fs.readFileSync(path.join(outDir, 'transcript.json'), 'utf8'))
133+ check(
134+ Array.isArray(json.items) && json.items.some((i: {type: string}) => i.type === 'speech'),
135+ 'transcript.json has speech items'
136+ )
137+ const asrFiles = fs.existsSync(path.join(outDir, 'asr'))
138+ ? fs.readdirSync(path.join(outDir, 'asr'))
139+ : []
140+ check(asrFiles.length >= 1, `asr/ cache written (${asrFiles.length} file(s))`)
141+
142+ process.stdout.write(
143+ failures.length === 0
144+ ? `\nALL PASS (output kept in ${outDir})\n`
145+ : `\n${failures.length} FAILURE(S):\n${failures.map(f => ` - ${f}`).join('\n')}\n(output kept in ${outDir})\n`
146+ )
147+ process.exit(failures.length === 0 ? 0 : 1)
148+}
149+
150+main().catch(err => {
151+ process.stderr.write(`live-loopback fatal: ${err?.stack ?? err}\n`)
152+ process.exit(1)
153+})
src/test/loopback.tsmodified+1−1View file
@@ -64,7 +64,7 @@ const main = async () => {
6464 process.stdout.write(`room: ${room}\nout: ${outDir}\n\n`)
6565
6666 const recorder = run(
67- [path.join(dist, 'cli.js'), room, '--out', outDir, '--notice', 'recording test'],
67+ [path.join(dist, 'cli.js'), 'record', room, '--out', outDir, '--notice', 'recording test'],
6868 'rec'
6969 )
7070 await wait(3000)
src/test/speaker.tsmodified+105−40View file
@@ -1,10 +1,13 @@
1-// Test participant: joins a room like a browser would and "talks" a sine
2-// tone into it, sends one chat message, then says bye and leaves. Used by the
3-// loopback test to exercise the whole path (nostr signaling -> WebRTC ->
4-// Opus -> RTCAudioSink -> WAV) without a real browser.
1+// Test participant: joins a room like a browser would and "talks" into it —
2+// either a sine tone or a WAV file (--wav) — sends one chat message, then
3+// says bye and leaves. Used by the loopback tests to exercise the whole path
4+// (nostr signaling -> WebRTC -> Opus -> RTCAudioSink -> WAV) without a real
5+// browser. Audio playback and the leave countdown start at the FIRST
6+// connection, so slow signaling can't eat into the clip.
57 //
6-// node dist/test/speaker.js <room> [--duration sec] [--freq hz] [--name X]
8+// node dist/test/speaker.js <room> [--duration sec] [--freq hz] [--wav f.wav]
79
10+import * as fs from 'node:fs'
811 import wrtc from '@roamhq/wrtc'
912 import {selfId} from '../identity.js'
1013 import {Nostr, peerTopic, roomTopic} from '../nostr.js'
@@ -16,42 +19,83 @@ let durationSec = 12
1619 let freq = 440
1720 let name = 'TestSpeaker'
1821 let chatText = 'hello from the loopback test'
22+let wavPath: string | null = null
1923 for (let i = 0; i < argv.length; i++) {
2024 const a = argv[i]!
2125 if (a === '--duration') durationSec = Number(argv[++i])
2226 else if (a === '--freq') freq = Number(argv[++i])
2327 else if (a === '--name') name = argv[++i] ?? name
2428 else if (a === '--chat') chatText = argv[++i] ?? chatText
29+ else if (a === '--wav') wavPath = argv[++i] ?? null
2530 else room = a
2631 }
2732 if (!room) {
28- process.stderr.write('usage: speaker.js <room> [--duration sec] [--freq hz]\n')
33+ process.stderr.write(
34+ 'usage: speaker.js <room> [--duration sec] [--freq hz] [--wav f.wav]\n'
35+ )
2936 process.exit(1)
3037 }
3138
3239 const log = (line: string) => process.stdout.write(`[speaker] ${line}\n`)
3340
34-// ---- outgoing audio: a continuous sine pushed in 10 ms frames ------------
41+// ---- outgoing audio: sine tone or WAV, pushed in 10 ms frames ------------
3542
36-const RATE = 48000
37-const FRAME = 480 // 10 ms
3843 const AMPLITUDE = 8000
3944
45+let rate = 48000
46+let wavSamples: Int16Array | null = null
47+if (wavPath) {
48+ const buf = fs.readFileSync(wavPath)
49+ rate = buf.readUInt32LE(24)
50+ const channels = buf.readUInt16LE(22)
51+ if (rate % 100 !== 0) {
52+ process.stderr.write(`--wav needs a sample rate divisible by 100 (got ${rate})\n`)
53+ process.exit(1)
54+ }
55+ const dataIdx = buf.indexOf('data')
56+ const pcm = new Int16Array(
57+ buf.buffer,
58+ buf.byteOffset + dataIdx + 8,
59+ (buf.length - dataIdx - 8) >> 1
60+ )
61+ if (channels === 1) {
62+ wavSamples = pcm
63+ } else {
64+ wavSamples = new Int16Array(Math.floor(pcm.length / channels))
65+ for (let i = 0; i < wavSamples.length; i++) {
66+ let sum = 0
67+ for (let c = 0; c < channels; c++) sum += pcm[i * channels + c]!
68+ wavSamples[i] = Math.round(sum / channels)
69+ }
70+ }
71+}
72+const FRAME = rate / 100 // 10 ms
73+
4074 const audioSource = new wrtc.nonstandard.RTCAudioSource()
4175 const audioTrack = audioSource.createTrack()
4276 const videoTrack = new wrtc.nonstandard.RTCVideoSource().createTrack()
4377
4478 let phase = 0
79+let wavOffset = 0
4580 const pushFrame = () => {
4681 const samples = new Int16Array(FRAME)
47- for (let i = 0; i < FRAME; i++) {
48- samples[i] = Math.round(AMPLITUDE * Math.sin(phase))
49- phase += (2 * Math.PI * freq) / RATE
82+ if (wavSamples) {
83+ // Play the file once, then silence.
84+ for (let i = 0; i < FRAME && wavOffset < wavSamples.length; i++) {
85+ samples[i] = wavSamples[wavOffset++]!
86+ }
87+ } else {
88+ for (let i = 0; i < FRAME; i++) {
89+ samples[i] = Math.round(AMPLITUDE * Math.sin(phase))
90+ phase += (2 * Math.PI * freq) / rate
91+ }
92+ if (phase > 2 * Math.PI) {
93+ phase -= 2 * Math.PI * Math.floor(phase / (2 * Math.PI))
94+ }
5095 }
51- if (phase > 2 * Math.PI) phase -= 2 * Math.PI * Math.floor(phase / (2 * Math.PI))
5296 audioSource.onData({
5397 samples,
54- sampleRate: RATE,
98+ sampleRate: rate,
5599 bitsPerSample: 16,
56100 channelCount: 1,
57101 numberOfFrames: FRAME
@@ -59,17 +103,21 @@ const pushFrame = () => {
59103 }
60104 // Wall-clock catch-up so timer jitter doesn't starve the source (bursts
61105 // capped — the source expects roughly real-time pacing).
62-let framesPushed = 0
63-const startMs = Date.now()
64-const audioTimer = setInterval(() => {
65- const due = Math.floor(((Date.now() - startMs) / 1000) * RATE) / FRAME
66- let burst = 0
67- while (framesPushed < due && burst < 5) {
68- pushFrame()
69- framesPushed++
70- burst++
71- }
72-}, 10)
106+let audioTimer: ReturnType<typeof setInterval> | null = null
107+const startAudio = () => {
108+ if (audioTimer !== null) return
109+ let framesPushed = 0
110+ const startMs = Date.now()
111+ audioTimer = setInterval(() => {
112+ const due = Math.floor(((Date.now() - startMs) / 1000) * rate) / FRAME
113+ let burst = 0
114+ while (framesPushed < due && burst < 5) {
115+ pushFrame()
116+ framesPushed++
117+ burst++
118+ }
119+ }, 10)
120+}
73121
74122 // ---- minimal mesh (commonroom protocol, one-shot) ------------------------
75123
@@ -88,6 +136,25 @@ const main = async () => {
88136 void nostr.publish(await peerTopic(root, peerId), JSON.stringify(msg))
89137 }
90138
139+ let announceTimer: ReturnType<typeof setInterval> | null = null
140+ let leaving = false
141+ const leave = () => {
142+ if (leaving) return
143+ leaving = true
144+ log('leaving')
145+ // Stop announcing and listening FIRST so nothing reconnects to us during
146+ // the goodbye grace period, then say bye and tear down.
147+ if (announceTimer !== null) clearInterval(announceTimer)
148+ nostr.close()
149+ const bye = JSON.stringify({t: 'bye'})
150+ for (const {peer} of conns.values()) peer.send(bye)
151+ setTimeout(() => {
152+ if (audioTimer !== null) clearInterval(audioTimer)
153+ for (const {peer} of conns.values()) peer.destroy()
154+ process.exit(0) // wrtc segfaults on natural exit — always exit explicitly
155+ }, 500)
156+ }
157+
91158 const createPeer = (peerId: string, initiator: boolean) => {
92159 const peer = new Peer(initiator, audioTrack, videoTrack)
93160 const conn = {peer, connected: false}
@@ -98,6 +165,8 @@ const main = async () => {
98165 if (conn.connected) return // connectionState can flap during ICE settling
99166 conn.connected = true
100167 log(`connected to ${peerId.slice(0, 8)}`)
168+ startAudio() // playback + leave countdown start at the first connect
169+ setTimeout(leave, durationSec * 1000)
101170 peer.send(
102171 JSON.stringify({
103172 t: 'hello',
@@ -154,24 +223,20 @@ const main = async () => {
154223 const announce = () =>
155224 void nostr.publish(root, JSON.stringify({peerId: selfId, name}))
156225 announce()
157- const announceTimer = setInterval(announce, 5000)
226+ announceTimer = setInterval(announce, 5000)
158227
228+ // Safety net: give up if nobody ever connects.
159229 setTimeout(() => {
160- log('leaving')
161- // Stop announcing and listening FIRST so nothing reconnects to us during
162- // the goodbye grace period, then say bye and tear down.
163- clearInterval(announceTimer)
164- nostr.close()
165- const bye = JSON.stringify({t: 'bye'})
166- for (const {peer} of conns.values()) peer.send(bye)
167- setTimeout(() => {
168- clearInterval(audioTimer)
169- for (const {peer} of conns.values()) peer.destroy()
170- process.exit(0) // wrtc segfaults on natural exit — always exit explicitly
171- }, 500)
172- }, durationSec * 1000)
230+ if (audioTimer === null) {
231+ log('no connection after 120s, giving up')
232+ process.exit(1)
233+ }
234+ }, 120000)
173235
174- log(`joined "${room}" as ${name} (peer ${selfId.slice(0, 8)}), ${freq} Hz for ${durationSec}s`)
236+ log(
237+ `joined "${room}" as ${name} (peer ${selfId.slice(0, 8)}), ` +
238+ `${wavPath ? `playing ${wavPath}` : `${freq} Hz`} for ${durationSec}s after connect`
239+ )
175240 }
176241
177242 main().catch(err => {
src/transcribe.tsmodified+12−91View file
@@ -1,6 +1,7 @@
11 import {spawn, spawnSync} from 'node:child_process'
22 import * as fs from 'node:fs'
33 import * as path from 'node:path'
4+import {mergeTurns, renderJson, renderMarkdown, type Item} from './render.js'
45
56 // The `transcribe` subcommand: turn a recording directory (per-speaker WAVs +
67 // manifest.json + events.jsonl) into one merged, speaker-attributed
@@ -53,11 +54,6 @@ interface Manifest {
5354 segments: ManifestSegment[]
5455 }
5556
56-type Item =
57- | {timeMs: number; type: 'speech'; speaker: string; text: string; endMs: number}
58- | {timeMs: number; type: 'chat'; speaker: string; text: string}
59- | {timeMs: number; type: 'system'; text: string}
60-
6157 // ---- ASR engines ----------------------------------------------------------
6258
6359 const FASTER_WHISPER_PY = `
@@ -78,7 +74,7 @@ json.dump(out, sys.stdout)
7874 const hasCmd = (cmd: string, args: string[] = ['--version']): boolean =>
7975 spawnSync(cmd, args, {stdio: 'ignore'}).error === undefined
8076
81-const hasFasterWhisper = (): boolean =>
77+export const hasFasterWhisper = (): boolean =>
8278 spawnSync('python3', ['-c', 'import faster_whisper'], {stdio: 'ignore'})
8379 .status === 0
8480
@@ -216,21 +212,6 @@ const runAsr = async (
216212
217213 // ---- timeline assembly ----------------------------------------------------
218214
219-/** Merge consecutive same-speaker utterances separated by < this many ms. */
220-const TURN_GAP_MS = 3000
221-
222-const localTime = (ms: number): string => {
223- const d = new Date(ms)
224- const p = (n: number) => String(n).padStart(2, '0')
225- return `${p(d.getHours())}:${p(d.getMinutes())}:${p(d.getSeconds())}`
226-}
227-
228-const localDateTime = (ms: number): string => {
229- const d = new Date(ms)
230- const p = (n: number) => String(n).padStart(2, '0')
231- return `${d.getFullYear()}-${p(d.getMonth() + 1)}-${p(d.getDate())} ${localTime(ms)}`
232-}
233-
234215 export const transcribe = async (opts: TranscribeOptions): Promise<void> => {
235216 const manifestPath = path.join(opts.dir, 'manifest.json')
236217 if (!fs.existsSync(manifestPath)) {
@@ -327,80 +308,20 @@ export const transcribe = async (opts: TranscribeOptions): Promise<void> => {
327308 }
328309 }
329310
330- items.sort((a, b) => a.timeMs - b.timeMs)
331-
332- // Merge adjacent same-speaker utterances (with nothing in between) into
333- // conversational turns.
334- const merged: Item[] = []
335- for (const item of items) {
336- const prev = merged[merged.length - 1]
337- if (
338- item.type === 'speech' &&
339- prev?.type === 'speech' &&
340- prev.speaker === item.speaker &&
341- item.timeMs - prev.endMs < TURN_GAP_MS
342- ) {
343- prev.text += ' ' + item.text
344- prev.endMs = item.endMs
345- } else {
346- merged.push({...item})
347- }
311+ const merged = mergeTurns(items)
312+ const meta = {
313+ room: manifest.room,
314+ startedAtMs: Date.parse(manifest.startedAt),
315+ endedAtMs: manifest.endedAt ? Date.parse(manifest.endedAt) : null,
316+ speakers: [...new Set(segments.map(s => s.name))],
317+ engine,
318+ model: opts.model
348319 }
349320
350- // ---- render ----
351- const startMs = Date.parse(manifest.startedAt)
352- const endMs = manifest.endedAt ? Date.parse(manifest.endedAt) : null
353- const speakers = [...new Set(segments.map(s => s.name))]
354-
355- const md: string[] = []
356- md.push(`# Transcript: ${manifest.room}`)
357- md.push('')
358- md.push(
359- `- **Recorded:** ${localDateTime(startMs)}${endMs ? ` – ${localTime(endMs)}` : ''}`
360- )
361- md.push(`- **Speakers:** ${speakers.join(', ') || '(none)'}`)
362- md.push(`- **Transcribed with:** ${engine}${opts.model ? `, model ${opts.model}` : ''}`)
363- md.push('')
364- md.push('---')
365- md.push('')
366- for (const item of merged) {
367- const t = localTime(item.timeMs)
368- if (item.type === 'speech') {
369- md.push(`**[${t}] ${item.speaker}:** ${item.text}`)
370- } else if (item.type === 'chat') {
371- md.push(`> [${t}] 💬 **${item.speaker}:** ${item.text}`)
372- } else {
373- md.push(`*[${t}] — ${item.text}*`)
374- }
375- md.push('')
376- }
377- if (merged.length === 0) md.push('*(nothing was said)*', '')
378-
379321 const mdPath = path.join(opts.dir, 'transcript.md')
380- fs.writeFileSync(mdPath, md.join('\n'))
381-
322+ fs.writeFileSync(mdPath, renderMarkdown(meta, merged))
382323 const jsonPath = path.join(opts.dir, 'transcript.json')
383- fs.writeFileSync(
384- jsonPath,
385- JSON.stringify(
386- {
387- room: manifest.room,
388- startedAt: manifest.startedAt,
389- endedAt: manifest.endedAt,
390- engine,
391- model: opts.model,
392- items: merged.map(item => ({
393- time: new Date(item.timeMs).toISOString(),
394- ...item,
395- timeMs: undefined,
396- endMs: undefined
397- }))
398- },
399- null,
400- 2
401- ) + '\n'
402- )
403-
324+ fs.writeFileSync(jsonPath, renderJson(meta, merged))
404325 opts.onLog(`wrote ${mdPath}`)
405326 opts.onLog(`wrote ${jsonPath}`)
406327 }
src/wav.tsmodified+5−0View file
@@ -70,6 +70,11 @@ export class WavWriter {
7070 return this.framesWritten / this.sampleRate
7171 }
7272
73+ /** Frames already on disk (safe for another process to read). */
74+ get flushedFrames(): number {
75+ return this.dataBytes / (this.channels * 2)
76+ }
77+
7378 private flush() {
7479 if (this.buffered.length === 0) return
7580 const chunk = Buffer.concat(this.buffered.splice(0))