16328b9Download runs as a .zip for offline development: WAVs, ground truth, training snippets, algorithm sources, and a numbl evaluation harness that reproduces the browser scoresJeremy Magland 1# audio-spike-sorting dataset export
3This archive is a self-contained benchmark for developing audio event
4detection and classification algorithms outside the browser. It was
5exported from the audio-spike-sorting app and contains every synthesized
6run of one benchmark, with exact ground truth, the training snippets each
7run handed to the algorithms, the algorithm sources as they stood at
8export time, and an evaluation harness that reproduces the browser's
9scores.
11## Dataset facts
13- Event types ({{N_TYPES}}): {{TYPE_NAMES}}
14- Sampling rate: {{FS}} Hz, mono, float32
15- Runs: {{N_RUNS}}, each an independent synthesis (run r used seed
16 {{SEED}} + r - 1)
17- Events per type per run: {{EVENTS_PER_TYPE}}
18- Snippet window: {{PRE_MS}} ms before the event onset, {{POST_MS}} ms
19 after ({{SNIP_LEN}} samples; the onset therefore sits {{PRE}} samples
20 into every training snippet)
21- Level variation: each placed event scaled by a random gain in
22 [{{GAIN_MIN_DB}}, 0] dB (0 means events kept their recorded volume)
23- Overlap fraction: {{OVERLAP}} of events deliberately placed inside
24 another event's span; the rest keep at least one snippet length of
25 separation
26- Training fraction: {{TRAIN_FRACTION}} of each type's snippets went to
27 the training set, drawn independently per run; the rest were placed
28- Match tolerance for scoring: {{TOLERANCE_MS}} ms
30## Layout
32```
33README.md this file
34dataset_info.m the facts above, as a function the harness calls
35evaluate.m runs and scores algorithms; edit its list
36algorithms/*.m algorithm sources from the app
37tools/*.m load_wav, load_csv, score_detections
38runs/run_01/
39 recording.wav the synthesized recording (float32, {{FS}} Hz)
40 ground_truth.csv one row per placed event, no header:
41 time_sec, label (1-based), gain
42 training_type_01.wav training snippets for type 1, concatenated;
43 reshape to [{{SNIP_LEN}} x nSnippets]
44 ...
45runs/run_02/ ...
46```
48Ground-truth times refer to the event onset, which is {{PRE}} samples
49after the start of the placed snippet. The WAV files carry a plain
5044-byte header (IEEE float, format tag 3); audioread in MATLAB, soundfile
51in Python, and ordinary audio editors read them directly, and
52tools/load_wav.m reads them with fopen for numbl, which has no audioread.
54## The algorithm interface
56An algorithm is one MATLAB-syntax file, algorithms/<name>.m, defining a
57function whose name matches the file:
59```matlab
60function [times, labels] = my_sorter(x, fs, training)
61% x the recording, column vector at fs Hz
62% fs sampling rate ({{FS}})
63% training struct with fields:
64% snips {1 x nTypes} cell; snips{k} is [{{SNIP_LEN}} x nSnips_k],
65% one training snippet per column
66% names {1 x nTypes} cell of type names
67% pre samples before the event onset within each snippet ({{PRE}})
68% Return column vectors: event times in seconds and 1-based type labels.
69times = zeros(0, 1);
70labels = zeros(0, 1);
71end
72```
74The training snippets are the only labeled data an algorithm may use;
75the events placed in the recording come from a disjoint pool, so the
76task includes generalizing across instances, not recognizing exact
77waveforms.
79## Evaluating
81With Node.js installed:
83```
84npx numbl run evaluate.m
85```
87numbl (https://numbl.org) executes MATLAB-syntax code; no MATLAB license
88is involved. Add your algorithm's name to the `algorithms` list at the
89top of evaluate.m and run again. For each algorithm the harness prints
90per-run accuracy, the mean and standard deviation over runs, total
91counts of the four error kinds, and a per-type breakdown, matching the
92browser's report exactly.
94Scoring: detections are matched to ground truth greedily by time
95difference within the {{TOLERANCE_MS}} ms tolerance, labels ignored; a
96matched pair with the right label is correct and with the wrong label a
97misclassification, unmatched truth events are misses, and unmatched
98detections are false positives. Accuracy is
99correct / (correct + misclassified + missed + false positives).
100tools/score_detections.m is the reference implementation and mirrors the
101browser harness; an algorithm developed against this archive will score
102identically when pasted back into the app.
104Note that numbl does not implement every MATLAB function. The set it
105does implement is documented at https://numbl.org/docs/builtins; an
106algorithm that runs here will also run in the app, since both use numbl.