Agent interface: inbox/ for chat interjections + AGENT.md instructions - concept-collection/commonroom-recorder

concept-collection / commonroom-recorder

Agent interface: inbox/ for chat interjections + AGENT.md instructions

Every recording directory now doubles as an interface for a monitoring AI agent: it follows the meeting via the live transcript.md / chat.txt / events.jsonl, and interjects by writing a file into inbox/ — the file's content is sent to the room as one chat message from the recorder (atomic write convention, 2000-char cap, file deleted once sent). AGENT.md, written at start, carries the instructions: interject only when it clearly helps, keep it to a sentence or two, task-specific guidance takes precedence.

Jeremy Magland <jmagland@flatironinstitute.org> committed 27 days ago commit 02393b5568b4 parent 4373b35 Browse files

6 changed files+174−5

CLAUDE.mdmodified+6−0View file

@@ -55,6 +55,12 @@ src/

5555 - **Bye cooldown (3 s).** An announcement published just before a peer's bye

5656 can arrive just after it and would trigger an instant reconnect (and a stub

5757 recording); after a bye we ignore that peer's announcements briefly.

58+- **The recording dir is also the agent interface.** `inbox/` is polled every

59+ 500 ms; a file's content is broadcast as a chat message from the recorder

60+ (then the file is deleted; dotfiles/*.tmp skipped, 300 ms mtime settle,

61+ 2000-char cap) and logged everywhere a received chat would be. `AGENT.md`

62+ (template at the bottom of recorder.ts) is written at start with

63+ instructions for a monitoring AI agent.

5864 - **Every exit path MUST end in `process.exit()`.** @roamhq/wrtc segfaults in

5965 its static destructors on a natural process exit whenever nonstandard

6066 media sources exist. The CLI, the speaker, and any future script that

README.mdmodified+20−0View file

@@ -101,6 +101,9 @@ use `--force` to re-transcribe.

101101 chat, mute/unmute, segment start/end

102102 manifest.json session summary: room, participants,

103103 segments with start/end times and durations

104+ inbox/ drop a file here to send its content to the

105+ room as a chat message (see below)

106+ AGENT.md instructions for a monitoring AI agent

104107 ```

105108

106109 Everything is written incrementally (`tail -f chat.txt` works live; the

@@ -136,6 +139,23 @@ Two deliberate deviations from the browser client:

136139 - **Files instead of tiles.** Each remote audio track feeds an `RTCAudioSink`

137140 whose PCM goes straight to an incrementally-written WAV.

138141

142+## Letting an AI agent join the conversation

143+

144+The recording directory doubles as an interface for an independent agent

145+(or anything else) that monitors the meeting and occasionally says something:

146+

147+- **Follow** the meeting by re-reading `transcript.md` (with `--transcribe`),

148+ `chat.txt`, or `events.jsonl` — they all grow live.

149+- **Interject** by writing a file into `inbox/`: the file's whole content is

150+ sent to the room as one chat message, appearing under the recorder's

151+ display name, and the file is deleted once sent. Write atomically (create

152+ as `*.tmp` or a dotfile, then rename); messages are capped at 2000

153+ characters.

154+- `AGENT.md`, written into every recording directory, contains ready-made

155+ instructions for the agent — point it there. The default guidance: only

156+ interject when it clearly helps, keep it to a sentence or two, and any

157+ task-specific instructions it was given take precedence.

158+

139159 ## Testing

140160

141161 `npm run test:loopback` runs an end-to-end test with no browser: it starts the

package.jsonmodified+1−1View file

@@ -1,6 +1,6 @@

11 {

22 "name": "commonroom-recorder",

3- "version": "0.3.1",

3+ "version": "0.4.0",

44 "description": "CLI bot that joins a commonroom room and records every participant's audio (and the chat) to disk for transcription",

55 "type": "module",

66 "bin": {

src/recorder.tsmodified+106−2View file

@@ -28,6 +28,9 @@ const ANNOUNCE_INTERVAL_MS = 5000

2828 const PRESENCE_TTL_MS = 15000

2929 const CONNECT_RETRY_MS = 15000

3030 const MANIFEST_INTERVAL_MS = 30000

31+const INBOX_POLL_MS = 500

32+/** Same cap the browser client applies to chat messages. */

33+const CHAT_MAX_LENGTH = 2000

3134

3235 /** Pad with silence when the sink falls this far behind wall clock, so a

3336 * file's sample position always tracks elapsed time (within ~1 s). */

@@ -151,6 +154,7 @@ export class Recorder {

151154 private videoTrack = this.videoSource.createTrack()

152155

153156 private audioDir: string

157+ private inboxDir: string

154158 private eventsPath: string

155159 private chatPath: string

156160 private manifestPath: string

@@ -158,6 +162,7 @@ export class Recorder {

158162

159163 constructor(private opts: RecorderOptions) {

160164 this.audioDir = path.join(opts.outDir, 'audio')

165+ this.inboxDir = path.join(opts.outDir, 'inbox')

161166 this.eventsPath = path.join(opts.outDir, 'events.jsonl')

162167 this.chatPath = path.join(opts.outDir, 'chat.txt')

163168 this.manifestPath = path.join(opts.outDir, 'manifest.json')

@@ -165,6 +170,11 @@ export class Recorder {

165170

166171 async start() {

167172 fs.mkdirSync(this.audioDir, {recursive: true})

173+ fs.mkdirSync(this.inboxDir, {recursive: true})

174+ fs.writeFileSync(

175+ path.join(this.opts.outDir, 'AGENT.md'),

176+ agentInstructions(this.opts.room, this.opts.name)

177+ )

168178 this.startedAtMs = Date.now()

169179 if (this.opts.transcribe) {

170180 this.liveT = new LiveTranscriber({

@@ -218,9 +228,61 @@ export class Recorder {

218228 this.timers.push(setInterval(() => void this.announce(), ANNOUNCE_INTERVAL_MS))

219229 this.timers.push(setInterval(() => this.sweepPresence(), ANNOUNCE_INTERVAL_MS))

220230 this.timers.push(setInterval(() => this.writeManifest(), MANIFEST_INTERVAL_MS))

231+ this.timers.push(setInterval(() => this.pollInbox(), INBOX_POLL_MS))

221232 this.writeManifest()

222233 }

223234

235+ // ---- inbox ---------------------------------------------------------------

236+ //

237+ // Anything (a human, an AI agent following the live transcript.md or

238+ // events.jsonl) can drop a file into <out>/inbox/ and its content is sent

239+ // to the room as a chat message from the recorder, then the file is

240+ // deleted. Write atomically (tmp name or dotfile, then rename): files

241+ // ending in .tmp or starting with '.' are ignored, and a file is left

242+ // alone until its mtime is at least 300 ms old.

243+

244+ private pollInbox() {

245+ if (this.stopped) return

246+ let names: string[]

247+ try {

248+ names = fs.readdirSync(this.inboxDir)

249+ } catch {

250+ return

251+ }

252+ for (const name of names.sort()) {

253+ if (name.startsWith('.') || name.endsWith('.tmp')) continue

254+ const p = path.join(this.inboxDir, name)

255+ try {

256+ const st = fs.statSync(p)

257+ if (!st.isFile() || Date.now() - st.mtimeMs < 300) continue

258+ const text = fs.readFileSync(p, 'utf8').trim().slice(0, CHAT_MAX_LENGTH)

259+ fs.unlinkSync(p)

260+ if (text) this.sendChat(text)

261+ } catch {

262+ /* ignore (file may have been removed concurrently) */

263+ }

264+ }

265+ }

266+

267+ private broadcastControl(msg: ControlMsg) {

268+ const payload = JSON.stringify(msg)

269+ for (const conn of this.conns.values()) conn.peer.send(payload)

270+ }

271+

272+ /** Send a chat message to the room as the recorder, and log it. */

273+ private sendChat(text: string) {

274+ this.broadcastControl({t: 'chat', text})

275+ this.event({type: 'chat', peerId: selfId, name: this.opts.name, text})

276+ this.chatLine(`${this.opts.name}: ${text}`)

277+ this.opts.onLog(`${this.opts.name}: ${text}`)

278+ this.liveT?.onEvent({

279+ timeMs: Date.now(),

280+ type: 'chat',

281+ speaker: this.opts.name,

282+ text

283+ })

284+ }

285+

224286 // ---- presence and the mesh ----------------------------------------------

225287

226288 private async announce() {

@@ -600,8 +662,7 @@ export class Recorder {

600662 async stop(): Promise<{segments: number; participants: number}> {

601663 if (this.stopped) return {segments: this.segments.length, participants: this.names.size}

602664 this.stopped = true

603- const bye = JSON.stringify({t: 'bye'} satisfies ControlMsg)

604- for (const conn of this.conns.values()) conn.peer.send(bye)

665+ this.broadcastControl({t: 'bye'})

605666 const conns = [...this.conns.entries()]

606667 this.conns.clear()

607668 for (const [peerId, conn] of conns) this.closeConn(peerId, conn)

@@ -621,3 +682,46 @@ export class Recorder {

621682 return {segments: this.segments.length, participants: this.names.size}

622683 }

623684 }

685+

686+/** Written into every recording directory so a monitoring AI agent knows how

687+ * to follow the meeting and when/how to interject. */

688+const agentInstructions = (room: string, name: string): string => `\

689+# Instructions for the monitoring agent

690+

691+A meeting in the commonroom room "${room}" is being recorded into this

692+directory. You can follow it live and, when appropriate, say something in

693+the room chat.

694+

695+## Following the conversation

696+

697+These files grow while the meeting runs (re-read or tail them):

698+

699+- \`transcript.md\` — speaker-attributed transcript with the chat and

700+ join/left events on one timeline (present when live transcription is on;

701+ text lags speech by ~8 seconds)

702+- \`chat.txt\` — chat messages and join/left lines, human-readable

703+- \`events.jsonl\` — the same plus mute/segment events, one JSON object per

704+ line with ISO timestamps

705+

706+## Interjecting

707+

708+To say something in the room, write a file into \`inbox/\`. The file's whole

709+content is sent as ONE chat message — it appears to participants as

710+"${name}" — and the file is deleted once sent. Write atomically: create the

711+file with a name starting with "." or ending in ".tmp", then rename it.

712+Messages are capped at 2000 characters.

713+

714+## Guidance

715+

716+- Interject only when it clearly helps: you are addressed directly, someone

717+ asks for a fact, link, or lookup you can provide, or something important

718+ was said that is plainly wrong and matters. When in doubt, stay silent —

719+ most of the time the right move is to say nothing.

720+- Keep it short: one or two sentences. This is a chat, not a report.

721+- Don't repeat yourself, don't summarize the meeting into the chat unless

722+ asked, and don't send several messages in quick succession.

723+- Any other instructions you have been given (a topic to watch for, a role

724+ to play, when to speak) take precedence over these defaults. Participants

725+ may also address you in the room chat or out loud — treat that as guidance

726+ too.

727+`

src/test/loopback.tsmodified+31−1View file

@@ -80,7 +80,26 @@ const main = async () => {

8080 'spk'

8181 )

8282

83- check(await exited(speaker.proc, (SPEAK_SEC + 45) * 1000), 'speaker ran and exited')

83+ // Once the speaker is in, drop a file into the inbox: its content must be

84+ // sent to the room as a chat message from the recorder.

85+ const speakerDone = exited(speaker.proc, (SPEAK_SEC + 45) * 1000)

86+ let speakerExited = false

87+ void speakerDone.then(() => (speakerExited = true))

88+ const INBOX_MSG = 'interjection from the inbox'

89+ let inboxWritten = false

90+ while (!speakerExited) {

91+ if (!inboxWritten && recorder.output().includes('TestSpeaker joined')) {

92+ fs.writeFileSync(path.join(outDir, 'inbox', 'msg-1.txt'), INBOX_MSG + '\n')

93+ inboxWritten = true

94+ }

95+ await wait(500)

96+ }

97+ check(await speakerDone, 'speaker ran and exited')

98+ check(inboxWritten, 'inbox message was written during the call')

99+ check(

100+ speaker.output().includes(`chat received: ${INBOX_MSG}`),

101+ 'speaker received the inbox chat over the data channel'

102+ )

84103 await wait(1500)

85104 recorder.proc.kill('SIGINT')

86105 check(await exited(recorder.proc, 15000), 'recorder exited cleanly on SIGINT')

@@ -151,6 +170,17 @@ const main = async () => {

151170 )

152171 const chatTxt = fs.readFileSync(path.join(outDir, 'chat.txt'), 'utf8')

153172 check(chatTxt.includes('hello from the loopback test'), 'chat message in chat.txt')

173+ check(

174+ events.some(

175+ e => e.type === 'chat' && e.name === 'Recorder' && e.text === 'interjection from the inbox'

176+ ),

177+ 'inbox chat logged in events.jsonl as the recorder'

178+ )

179+ check(chatTxt.includes('Recorder: interjection from the inbox'), 'inbox chat in chat.txt')

180+ check(

181+ fs.readdirSync(path.join(outDir, 'inbox')).length === 0,

182+ 'inbox file deleted after sending'

183+ )

154184

155185 process.stdout.write(

156186 failures.length === 0

src/test/speaker.tsmodified+10−1View file

@@ -183,7 +183,16 @@ const main = async () => {

183183 peer.send(JSON.stringify({t: 'chat', text: chatText}))

184184 }, 2000)

185185 },

186- data: () => undefined,

186+ data: raw => {

187+ try {

188+ const msg = JSON.parse(raw) as {t?: string; text?: string}

189+ if (msg.t === 'chat' && typeof msg.text === 'string') {

190+ log(`chat received: ${msg.text}`)

191+ }

192+ } catch {

193+ /* ignore */

194+ }

195+ },

187196 close: () => {

188197 conns.delete(peerId)

189198 }