21abc12audio-spike-sorting: benchmark framework for audio event detection and classificationJeremy Magland 1# audio-spike-sorting
3A benchmark framework for the detection and classification of short audio
4events, in the spirit of spike sorting. You record hundreds of examples of
5each event type (tapping the table, tapping the floor, tapping with a
6spoon), cleanly separated so the examples are unambiguous. The framework
7then synthesizes recordings in which known events are placed at known
8times, possibly overlapping, hands each candidate algorithm a small
9labeled training set, and scores the algorithm's output against the exact
10ground truth. Candidate algorithms are written in MATLAB syntax and
11executed in the browser by [numbl](https://numbl.org); everything, from
12recording through scoring and visual exploration, runs on one page with
13nothing to install.
15A limitation of evaluating event-detection methods on real recordings is
16that the ground truth is itself the product of some annotation process,
17with its own errors and biases. Synthesis from recorded examples avoids
18this: the event times and labels are correct by construction, while the
19audio itself remains real. The trade-off is that a synthesized mixture is
20not a real performance; in particular, overlapping events here are sums of
21independently recorded events, which ignores any physical interaction
22between them.
24## How a benchmark run works
261. **Snippets.** Each recorded clip is segmented by an onset detector
27 (spectral flux, with an adjustable sensitivity), and the detected
28 events stay editable: click to audition, exclude mistakes. Every kept
29 onset is refined to the foot of its energy rise and a fixed window
30 (default 20 ms before, 180 ms after) is cut around it. This refined
31 onset is the project's single definition of event time, used both for
32 training snippets and for ground truth, so all algorithms are scored
33 against one consistent convention.
342. **Split.** For each run, every type's snippets are randomly divided
35 into a training set (default 30%) given to the algorithm, and a
36 placement pool used to build the dataset. An algorithm is therefore
37 always classifying events it has not literally seen.
383. **Synthesis.** Events from the placement pool are placed on a silent
39 timeline, each scaled by a random gain (default 0 to -12 dB). An
40 overlap knob in [0, 1] sets the fraction of events deliberately placed
41 inside another event's span; the rest keep at least one snippet length
42 of separation. Run r uses seed + r, so the whole benchmark is
43 reproducible from the configuration.
444. **Execution.** Each algorithm is a `.m` file defining
45 `function [times, labels] = name(x, fs, training)`, where `training`
46 carries the per-type snippet matrices, the type names, and the onset
47 position within a snippet. numbl runs it in a web worker; the sources
48 are editable in place on the page.
495. **Scoring.** Detections are matched to ground truth greedily by time
50 difference within a tolerance (default 20 ms), ignoring labels. A
51 matched pair with the right label is correct, with the wrong label a
52 misclassification; unmatched truth events are misses and unmatched
53 detections false positives. The summary accuracy is
54 correct / (correct + misclassified + missed + false positives),
55 reported as mean and standard deviation over runs, alongside a
56 confusion matrix per algorithm with an extra column for misses and an
57 extra row for false positives.
59The explorer shows any synthesized recording with the ground truth lanes
60above the waveform and one algorithm's detections below it. A correct
61detection is a filled marker in its type's color, a misclassification is
62ringed in red, a false positive is a red cross, and a missed truth event
63is a hollow circle. The wheel zooms, dragging pans, and a click auditions
64the audio at that time.
66## Baseline algorithms
68Three built-ins seed the comparison and can be edited or reset in place:
70- **template_matched_filter**: one unit-norm mean template per type,
71 cross-correlated against the recording via the FFT; the per-type scores
72 are normalized so one threshold treats loud and quiet types alike. The
73 classic spike-sorting approach; it handles nothing specially for
74 overlaps beyond the correlation itself.
75- **onset_knn**: detection on a smoothed energy envelope, classification
76 by distance-weighted k-nearest-neighbours over spectral shape features
77 (band-energy ratios, centroid, zero-crossing rate). Levels are used only
78 as ratios, so the random per-event gain carries no information.
79- **naive_threshold**: an amplitude threshold and a classifier that looks
80 only at event duration. A deliberately weak floor; a real method that
81 fails to clear it is in trouble.
83On the synthetic demo data (three types, 20% overlap), the baselines land
84around 0.6 to 0.8 accuracy, with the losses concentrated in the
85overlapping events. None of the built-ins attempts overlap resolution
86(subtraction or joint fitting), which is exactly the direction candidate
87algorithms are expected to explore.
89## Running it
91```bash
92npm install
93npm run dev # the app, at the printed URL
94npm test # node end-to-end test of the pipeline (no browser)
95npm run build # typecheck and bundle
96```
98"Load demo data" creates three synthetic event types so the whole
99pipeline can be tried without a microphone. Projects persist in the
100browser (clip audio in IndexedDB, everything else in localStorage) and
101round-trip through Export/Import as a single JSON file, so a recorded
102snippet library can be shared or committed.
104## Notes and limitations
106- The internal rate is 16 kHz mono; microphone input is downsampled on
107 capture with a box average, which is a crude anti-alias filter. Events
108 whose identity lives above 8 kHz will lose that information.
109- Synthesis adds no background noise; the noise floor is whatever the
110 recordings carry. A noise knob is a natural next step, as is an option
111 to place events over a recorded room-tone bed.
112- The matching is greedy rather than globally optimal; with a 20 ms
113 tolerance and events at realistic densities the difference is
114 negligible, but it is not the Hungarian algorithm.
115- Algorithm execution is single-threaded per run, so per-algorithm
116 timings are comparable but a benchmark with many runs takes their sum.