CLAUDE.md#

Tips for future agents working in this repo. It combines the p2p techniques of the sibling projects commonview (auto-connecting mesh) and commoncall (WebRTC media, settings, screen share) — read those first; this file only covers what is different here.

Architecture#

src/p2p/
  identity.ts  schnorr keypair; pubkey hex = peer ID     — ported from commoncall
  nostr.ts     minimal relay client + topic scheme        — ported (roomTopic takes a room ID)
  peer.ts      WebRTC wrapper: media + control channel    — ported (replaceTrack generalized to audio|video)
  settings.ts  shared ROOM settings, quality presets      — default quality is 'medium', not 'auto'
  turn.ts      optional TURN: build-time endpoint, credential fetch, sanitizers
  network.ts   the heart: rooms, presence, mesh, media, settings sync, relay sharing
src/transcribe/
  capture.ts   AudioWorklet → 40 ms linear16 frames + RMS; SpeechGate (the VAD)
  deepgram.ts  one streaming WebSocket per speaker; subprotocol auth, KeepAlive
  store.ts     the transcript: paragraph merging, per-room localStorage persistence
  transcriber.ts  one capture+socket pipeline per participant, reconciled per snapshot
  *.test.mjs   node checks for the gate and the store (see Testing)
src/App.tsx    landing form (light) + in-room view (dark), video grid with
               click-to-spotlight (gallery ↔ one big tile + filmstrip; Esc or
               click again to return), control bar, chat and transcript side
               panels (one at a time; docked wide, overlay ≤700px)
worker/        Cloudflare Worker that mints TURN credentials — deployed
               separately (wrangler), NOT part of `npm run build`

Key design decisions#

• Rooms, no registry. The room ID is any string (whitespace stripped, exact otherwise, case-sensitive); roomTopic hashes it into the nostr presence topic. The URL hash holds the room (#<encoded-room>) so the address bar is the invite link.
• Auto-mesh, no consent handshake. Unlike commoncall, entering the room IS the consent: on every presence announcement, maybeConnect brings up a Peer (initiator = smaller peer ID, commonview's stalled-connection retry at 15 s). All of commoncall's call-request/accept machinery is gone.
• Muted by default; placeholder tracks. getUserMedia runs at entry but tracks start enabled = false. Every participant ALWAYS carries exactly one audio + one video track (denied/missing devices get a silent AudioContext-destination track / black canvas-capture track), so offer/answer stays symmetric and the one-offer, no-renegotiation design holds. Unmuting without a real device retries getUserMedia and upgrades the placeholder via replaceTrack on every connection. getUserMedia failures surface a cause-specific notice (mediaErrorMessage: permission vs not-found vs device-busy, error name included) both at join and on retry — on Linux, a camera held by another browser fails with NotReadableError, which is NOT a permissions problem. The combined audio+video request fails as a whole in that case, so acquireMedia retries each kind separately.
• Soft cap of 8 (MAX_PARTICIPANTS). A peer already holding 7 connections answers an unknown peer's announcement/offer with {t:'room-full'} on the newcomer's topic instead of connecting; a newcomer with zero connections that receives room-full tears down and shows a notice. Two simultaneous joiners racing for the last slot can briefly exceed the cap — accepted.
• Settings are room-wide, multi-party LWW. One entry per key in settingsMeta ({rev, by}); changes broadcast {t:'set', key, value, rev, by} to all peers (complete graph — no relaying), late joiners get every entry inside each peer's hello, and a same-rev tie is won by the SMALLER setter ID. Default quality is medium — so quality caps are applied to each sender on connect (applyVideoParamsTo, with one delayed retry because encodings may not exist right at 'connected'), not only on change.
• Mute is per-participant, NOT a shared setting — same as commoncall: own flags, {t:'mute'} notices, track.enabled toggling, and the notice carries the EFFECTIVE outgoing video state (screen share overrides camera mute). Remote participants are assumed muted until told otherwise.
• Screen share = track swap on every connection. getDisplayMedia + replaceTrack per peer; a peer that joins mid-share gets the screen track from outgoingStream(). Same-kind replacement avoids renegotiation — never addTrack mid-connection.
• Chat is ephemeral and never relayed. {t:'chat', text} broadcasts on the control channels; every message arrives directly from its author over a channel established via signed signaling, so authorship needs no extra crypto. There is deliberately NO history replay for late joiners — replay would mean peers relaying others' messages, which a malicious peer could fabricate; adding history requires signing each message. Log capped at 500, messages at 2000 chars. Join/left lines are derived locally: hello carries a self-reported joinedAt, and a peer whose join predates ours gets no "joined" line on first sight (they were already here) — but chatSeen ensures a blip-reconnect logs "joined" to match its "left". Links: only http(s) URLs matched by withLinks become anchors (target=_blank, rel=noopener noreferrer); never linkify other schemes.
• TURN is optional, token-gated, and shared room-wide. VITE_TURN_ENDPOINT (build time) points at worker/; unset ⇒ the whole feature is hidden and BASE_ICE_SERVERS (STUN + openrelay) is used, as before. One participant enters a token on the landing form, mintIce exchanges it for an {iceServers, expiresAt} and it is broadcast as {t:'ice'} — on connect to each peer, and again on refresh (scheduleIceRefresh, 5 min before expiry). Peers adoptIce it; our own credentials always beat a shared one (only we can refresh them), otherwise the later expiresAt wins. Share the CREDENTIAL, never the token — the token never leaves the browser it was typed into. Peer-supplied ICE is untrusted: sanitizeIceConfig bounds and scheme-checks it before it reaches RTCPeerConnection.
• Credentials can't help the connection that carried them. Peer takes iceServers at construction and never renegotiates them, so a peer learns credentials from the first peer it reaches and uses them for the NEXT connection. Stalled pairs recover via the existing CONNECT_RETRY_MS path, which calls iceServers() afresh — deliberately no proactive teardown on adoption, since rebuilding a half-open pair out of step with the other side is exactly what that retry already handles.
• Transcription: one participant pays, everyone is told. Whoever enters a Deepgram key transcribes the WHOLE room from their own browser — a mesh call already delivers everyone's audio locally, so syncTranscriptionSources (called from rebuildSnapshot) opens one pipeline per participant plus self. Attribution is structural: one socket per speaker, so there is no diarization to misread. The transcript is deliberately NOT broadcast — a transcriber relaying text attributed to others is text they cannot vouch for, the same objection that blocks chat history replay. What IS broadcast is {t:'tx', on} (plus a transcribing flag in hello for late joiners), driving a tile badge and chat lines. It is NOT a room setting: nobody else can turn it on or off. The key lives in localStorage (commonroom:deepgramKey) and, unlike the TURN token, has nothing shareable derived from it.
• Streaming, not batch, because of CORS. Deepgram's pre-recorded REST endpoint is cheaper per minute but sends no CORS headers, so a browser needs a proxy; the WebSocket endpoint connects directly. Browsers cannot set an Authorization header on a WebSocket, so the key rides the subprotocol (new WebSocket(url, ['token', key])) — Deepgram's documented client-side scheme. A failed WS handshake hides its HTTP status, so a bad key and a dead network are indistinguishable; DeepgramStream gives up after two never-opened sockets rather than retrying on every utterance.
• Cost is the whole point of the VAD. Billing follows audio sent, so SpeechGate gates it and KeepAlive (every 3-5 s; the server times out at 10 s) holds the socket open through silence unbilled. Watch three things if you retune it: the noise floor adapts ONLY while the gate is shut (else a long utterance raises the bar against itself); it is seeded from the first frame and held shut for 1 s (else someone joining from a noisy room opens it on frame one and never closes); and a rise is capped at 4 × floor per step so one door slam cannot deafen it while a sustained change is still learned in about a second. MAX_OPEN_FRAMES is the backstop for a gate stuck open. A 320 ms pre-roll ring buffer means word onsets survive the gate opening.
• The transcript persists per room, the chat does not. TranscriptStore keys on the room ID (commonroom:transcript:<room>), restores on entry before transcription even starts, debounces writes at 2 s, flushes on pagehide and on leave, and trims from the front at 200k chars. Hence the Clear button, which confirms inline — it is the one thing here that was being kept on purpose.
• Cleanup is join-generation-guarded. joinSeq is bumped on every join/leave; async work (getUserMedia, topic hashing, display capture) re-checks it after each await. leave() unsubscribes topics, stops all tracks, closes the AudioContext, and resets settings to defaults.

Testing#

npm run dev, then open the room in two browsers (identity is per-browser-profile via localStorage, so two tabs in one profile are the SAME peer — use a private window or second browser). npm run build type-checks (tsc -b) and bundles. Let the user test multi-party media in real browsers; don't try to automate camera/mic flows.

The Worker CAN be tested without a browser: cd worker && cp .dev.vars.example .dev.vars && npm run dev, then curl it. With the example values, token checks work (goodtoken passes, anything else 401s) and the upstream call 404s, which surfaces as a 502 — enough to cover auth, CORS and method handling. cd worker && npx tsc --noEmit type-checks it; the root tsc -b does not (it only includes src). The sanitizers in turn.ts are pure and testable under node via npx esbuild src/p2p/turn.ts --format=esm --define:import.meta.env='{}'.

The two riskiest pieces of the transcription path have node checks, by the same bundle-then-run recipe (there is no test runner in this repo; run them by hand after touching either file):

npx esbuild src/transcribe/capture.ts --format=esm --outfile=/tmp/capture.mjs
node src/transcribe/gate.test.mjs /tmp/capture.mjs     # VAD: what gets paid for
npx esbuild src/transcribe/store.ts --format=esm --outfile=/tmp/store.mjs
node src/transcribe/store.test.mjs /tmp/store.mjs      # persistence: what must not be lost

What they cannot cover is whether Deepgram accepts the audio at all — that needs a real key and a real browser. The transcript panel's "N s sent" readout is the quickest check that gating works: it should climb while someone talks and sit still while nobody does. Whether a relay is actually USED can only be seen in a real browser (chrome://webrtc-internals, candidate pair type relay).