# CLAUDE.md Tips for future agents working in this repo. It combines the p2p techniques of the sibling projects `commonview` (auto-connecting mesh) and `commoncall` (WebRTC media, settings, screen share) — read those first; this file only covers what is different here. ## Architecture ``` src/p2p/ identity.ts schnorr keypair; pubkey hex = peer ID — ported from commoncall nostr.ts minimal relay client + topic scheme — ported (roomTopic takes a room ID) peer.ts WebRTC wrapper: media + control channel — ported (replaceTrack generalized to audio|video) settings.ts shared ROOM settings, quality presets — default quality is 'medium', not 'auto' turn.ts optional TURN: build-time endpoint, credential fetch, sanitizers network.ts the heart: rooms, presence, mesh, media, settings sync, relay sharing src/transcribe/ capture.ts AudioWorklet → 40 ms linear16 frames + RMS; SpeechGate (the VAD) deepgram.ts one streaming WebSocket per speaker; subprotocol auth, KeepAlive store.ts the transcript: paragraph merging, per-room localStorage persistence transcriber.ts one capture+socket pipeline per participant, reconciled per snapshot *.test.mjs node checks for the gate and the store (see Testing) src/App.tsx landing form (light) + in-room view (dark), video grid with click-to-spotlight (gallery ↔ one big tile + filmstrip; Esc or click again to return), control bar, chat and transcript side panels (one at a time; docked wide, overlay ≤700px) worker/ Cloudflare Worker that mints TURN credentials — deployed separately (wrangler), NOT part of `npm run build` ``` ## Key design decisions - **Rooms, no registry.** The room ID is any string (whitespace stripped, exact otherwise, case-sensitive); `roomTopic` hashes it into the nostr presence topic. The URL hash holds the room (`#`) so the address bar is the invite link. - **Auto-mesh, no consent handshake.** Unlike commoncall, entering the room IS the consent: on every presence announcement, `maybeConnect` brings up a `Peer` (initiator = smaller peer ID, commonview's stalled-connection retry at 15 s). All of commoncall's call-request/accept machinery is gone. - **Muted by default; placeholder tracks.** getUserMedia runs at entry but tracks start `enabled = false`. Every participant ALWAYS carries exactly one audio + one video track (denied/missing devices get a silent AudioContext-destination track / black canvas-capture track), so offer/answer stays symmetric and the one-offer, no-renegotiation design holds. Unmuting without a real device retries getUserMedia and upgrades the placeholder via `replaceTrack` on every connection. getUserMedia failures surface a cause-specific notice (`mediaErrorMessage`: permission vs not-found vs device-busy, error name included) both at join and on retry — on Linux, a camera held by another browser fails with NotReadableError, which is NOT a permissions problem. The combined audio+video request fails as a whole in that case, so `acquireMedia` retries each kind separately. - **Soft cap of 8** (`MAX_PARTICIPANTS`). A peer already holding 7 connections answers an unknown peer's announcement/offer with `{t:'room-full'}` on the newcomer's topic instead of connecting; a newcomer with zero connections that receives room-full tears down and shows a notice. Two simultaneous joiners racing for the last slot can briefly exceed the cap — accepted. - **Settings are room-wide, multi-party LWW.** One entry per key in `settingsMeta` (`{rev, by}`); changes broadcast `{t:'set', key, value, rev, by}` to all peers (complete graph — no relaying), late joiners get every entry inside each peer's `hello`, and a same-rev tie is won by the SMALLER setter ID. Default quality is `medium` — so quality caps are applied to each sender on connect (`applyVideoParamsTo`, with one delayed retry because encodings may not exist right at 'connected'), not only on change. - **Mute is per-participant, NOT a shared setting** — same as commoncall: own flags, `{t:'mute'}` notices, `track.enabled` toggling, and the notice carries the EFFECTIVE outgoing video state (screen share overrides camera mute). Remote participants are assumed muted until told otherwise. - **Screen share = track swap on every connection.** `getDisplayMedia` + `replaceTrack` per peer; a peer that joins mid-share gets the screen track from `outgoingStream()`. Same-kind replacement avoids renegotiation — never addTrack mid-connection. - **Chat is ephemeral and never relayed.** `{t:'chat', text}` broadcasts on the control channels; every message arrives directly from its author over a channel established via signed signaling, so authorship needs no extra crypto. There is deliberately NO history replay for late joiners — replay would mean peers relaying others' messages, which a malicious peer could fabricate; adding history requires signing each message. Log capped at 500, messages at 2000 chars. Join/left lines are derived locally: hello carries a self-reported `joinedAt`, and a peer whose join predates ours gets no "joined" line on first sight (they were already here) — but `chatSeen` ensures a blip-reconnect logs "joined" to match its "left". Links: only http(s) URLs matched by `withLinks` become anchors (target=_blank, rel=noopener noreferrer); never linkify other schemes. - **TURN is optional, token-gated, and shared room-wide.** `VITE_TURN_ENDPOINT` (build time) points at `worker/`; unset ⇒ the whole feature is hidden and `BASE_ICE_SERVERS` (STUN + openrelay) is used, as before. One participant enters a token on the landing form, `mintIce` exchanges it for an `{iceServers, expiresAt}` and it is broadcast as `{t:'ice'}` — on `connect` to each peer, and again on refresh (`scheduleIceRefresh`, 5 min before expiry). Peers `adoptIce` it; our own credentials always beat a shared one (only we can refresh them), otherwise the later `expiresAt` wins. Share the CREDENTIAL, never the token — the token never leaves the browser it was typed into. **Peer-supplied ICE is untrusted:** `sanitizeIceConfig` bounds and scheme-checks it before it reaches `RTCPeerConnection`. - **Credentials can't help the connection that carried them.** `Peer` takes `iceServers` at construction and never renegotiates them, so a peer learns credentials from the first peer it reaches and uses them for the NEXT connection. Stalled pairs recover via the existing `CONNECT_RETRY_MS` path, which calls `iceServers()` afresh — deliberately no proactive teardown on adoption, since rebuilding a half-open pair out of step with the other side is exactly what that retry already handles. - **Transcription: one participant pays, everyone is told.** Whoever enters a Deepgram key transcribes the WHOLE room from their own browser — a mesh call already delivers everyone's audio locally, so `syncTranscriptionSources` (called from `rebuildSnapshot`) opens one pipeline per participant plus self. Attribution is structural: one socket per speaker, so there is no diarization to misread. The transcript is deliberately NOT broadcast — a transcriber relaying text attributed to others is text they cannot vouch for, the same objection that blocks chat history replay. What IS broadcast is `{t:'tx', on}` (plus a `transcribing` flag in `hello` for late joiners), driving a tile badge and chat lines. It is NOT a room setting: nobody else can turn it on or off. The key lives in localStorage (`commonroom:deepgramKey`) and, unlike the TURN token, has nothing shareable derived from it. - **Streaming, not batch, because of CORS.** Deepgram's pre-recorded REST endpoint is cheaper per minute but sends no CORS headers, so a browser needs a proxy; the WebSocket endpoint connects directly. Browsers cannot set an Authorization header on a WebSocket, so the key rides the subprotocol (`new WebSocket(url, ['token', key])`) — Deepgram's documented client-side scheme. A failed WS handshake hides its HTTP status, so a bad key and a dead network are indistinguishable; `DeepgramStream` gives up after two never-opened sockets rather than retrying on every utterance. - **Cost is the whole point of the VAD.** Billing follows audio sent, so `SpeechGate` gates it and `KeepAlive` (every 3-5 s; the server times out at 10 s) holds the socket open through silence unbilled. Watch three things if you retune it: the noise floor adapts ONLY while the gate is shut (else a long utterance raises the bar against itself); it is seeded from the first frame and held shut for 1 s (else someone joining from a noisy room opens it on frame one and never closes); and a rise is capped at `4 × floor` per step so one door slam cannot deafen it while a sustained change is still learned in about a second. `MAX_OPEN_FRAMES` is the backstop for a gate stuck open. A 320 ms pre-roll ring buffer means word onsets survive the gate opening. - **The transcript persists per room, the chat does not.** `TranscriptStore` keys on the room ID (`commonroom:transcript:`), restores on entry before transcription even starts, debounces writes at 2 s, flushes on `pagehide` and on leave, and trims from the front at 200k chars. Hence the Clear button, which confirms inline — it is the one thing here that was being kept on purpose. - **Cleanup is join-generation-guarded.** `joinSeq` is bumped on every join/leave; async work (getUserMedia, topic hashing, display capture) re-checks it after each await. `leave()` unsubscribes topics, stops all tracks, closes the AudioContext, and resets settings to defaults. ## Testing `npm run dev`, then open the room in two browsers (identity is per-browser-profile via localStorage, so two tabs in one profile are the SAME peer — use a private window or second browser). `npm run build` type-checks (`tsc -b`) and bundles. Let the user test multi-party media in real browsers; don't try to automate camera/mic flows. The Worker CAN be tested without a browser: `cd worker && cp .dev.vars.example .dev.vars && npm run dev`, then curl it. With the example values, token checks work (`goodtoken` passes, anything else 401s) and the upstream call 404s, which surfaces as a 502 — enough to cover auth, CORS and method handling. `cd worker && npx tsc --noEmit` type-checks it; the root `tsc -b` does not (it only includes `src`). The sanitizers in `turn.ts` are pure and testable under node via `npx esbuild src/p2p/turn.ts --format=esm --define:import.meta.env='{}'`. The two riskiest pieces of the transcription path have node checks, by the same bundle-then-run recipe (there is no test runner in this repo; run them by hand after touching either file): ```sh npx esbuild src/transcribe/capture.ts --format=esm --outfile=/tmp/capture.mjs node src/transcribe/gate.test.mjs /tmp/capture.mjs # VAD: what gets paid for npx esbuild src/transcribe/store.ts --format=esm --outfile=/tmp/store.mjs node src/transcribe/store.test.mjs /tmp/store.mjs # persistence: what must not be lost ``` What they cannot cover is whether Deepgram accepts the audio at all — that needs a real key and a real browser. The transcript panel's "N s sent" readout is the quickest check that gating works: it should climb while someone talks and sit still while nobody does. Whether a relay is actually USED can only be seen in a real browser (chrome://webrtc-internals, candidate pair type `relay`).