/ proof-of-concept / audio-spike-sorting
Sign in
proof-of-concept / audio-spike-sorting
2e5c8a5
Branches
main
audio-spike-sorting / src / export / assets / OFFLINE_README.md
110 lines · 4.7 KBCodeBlameHistory
3This archive is a self-contained benchmark for developing audio event
4detection and classification algorithms outside the browser. It was
5exported from the audio-spike-sorting app and contains every synthesized
6run of one benchmark, with exact ground truth, the training snippets each
7run handed to the algorithms, the algorithm sources as they stood at
8export time, and an evaluation harness that reproduces the browser's
9scores.
11## Dataset facts
13- Event types ({{N_TYPES}}): {{TYPE_NAMES}}
14- Sampling rate: {{FS}} Hz, mono, float32
15- Runs: {{N_RUNS}}, each an independent synthesis (run r used seed
16 {{SEED}} + r - 1)
17- Events per type per run: {{EVENTS_PER_TYPE}}
18- Snippet window: {{PRE_MS}} ms before the event onset, {{POST_MS}} ms
19 after ({{SNIP_LEN}} samples; the onset therefore sits {{PRE}} samples
20 into every training snippet)
21- Level variation: each placed event scaled by a random gain in
22 [{{GAIN_MIN_DB}}, 0] dB (0 means events kept their recorded volume)
23- Overlap fraction: {{OVERLAP}} of events deliberately placed inside
24 another event's span; the rest keep at least one snippet length of
25 separation
27 at the onset of the event they overlap, within {{COINCIDENCE_MS}} ms
28 either side, so the two onsets are nearly simultaneous; the remaining
29 overlappers sit at a random offset inside the host's span
31 the training set, drawn independently per run; the rest were placed
32- Match tolerance for scoring: {{TOLERANCE_MS}} ms
34## Layout
36```
37README.md this file
38dataset_info.m the facts above, as a function the harness calls
39evaluate.m runs and scores algorithms; edit its list
40algorithms/*.m algorithm sources from the app
41tools/*.m load_wav, load_csv, score_detections
42runs/run_01/
43 recording.wav the synthesized recording (float32, {{FS}} Hz)
44 ground_truth.csv one row per placed event, no header:
45 time_sec, label (1-based), gain
46 training_type_01.wav training snippets for type 1, concatenated;
47 reshape to [{{SNIP_LEN}} x nSnippets]
48 ...
49runs/run_02/ ...
50```
52Ground-truth times refer to the event onset, which is {{PRE}} samples
53after the start of the placed snippet. The WAV files carry a plain
5444-byte header (IEEE float, format tag 3); audioread in MATLAB, soundfile
55in Python, and ordinary audio editors read them directly, and
56tools/load_wav.m reads them with fopen for numbl, which has no audioread.
58## The algorithm interface
60An algorithm is one MATLAB-syntax file, algorithms/<name>.m, defining a
61function whose name matches the file:
63```matlab
64function [times, labels] = my_sorter(x, fs, training)
65% x the recording, column vector at fs Hz
66% fs sampling rate ({{FS}})
67% training struct with fields:
68% snips {1 x nTypes} cell; snips{k} is [{{SNIP_LEN}} x nSnips_k],
69% one training snippet per column
70% names {1 x nTypes} cell of type names
71% pre samples before the event onset within each snippet ({{PRE}})
72% Return column vectors: event times in seconds and 1-based type labels.
73times = zeros(0, 1);
74labels = zeros(0, 1);
75end
76```
78The training snippets are the only labeled data an algorithm may use;
79the events placed in the recording come from a disjoint pool, so the
80task includes generalizing across instances, not recognizing exact
81waveforms.
83## Evaluating
85With Node.js installed:
87```
88npx numbl run evaluate.m
89```
91numbl (https://numbl.org) executes MATLAB-syntax code; no MATLAB license
92is involved. Add your algorithm's name to the `algorithms` list at the
93top of evaluate.m and run again. For each algorithm the harness prints
94per-run accuracy, the mean and standard deviation over runs, total
95counts of the four error kinds, and a per-type breakdown, matching the
96browser's report exactly.
98Scoring: detections are matched to ground truth greedily by time
99difference within the {{TOLERANCE_MS}} ms tolerance, labels ignored; a
100matched pair with the right label is correct and with the wrong label a
101misclassification, unmatched truth events are misses, and unmatched
102detections are false positives. Accuracy is
103correct / (correct + misclassified + missed + false positives).
104tools/score_detections.m is the reference implementation and mirrors the
105browser harness; an algorithm developed against this archive will score
106identically when pasted back into the app.
108Note that numbl does not implement every MATLAB function. The set it
109does implement is documented at https://numbl.org/docs/builtins; an
110algorithm that runs here will also run in the app, since both use numbl.
↑↓ move↵openescclose