audio-spike-sorting dataset export#

This archive is a self-contained benchmark for developing audio event detection and classification algorithms outside the browser. It was exported from the audio-spike-sorting app and contains every synthesized run of one benchmark, with exact ground truth, the training snippets each run handed to the algorithms, the algorithm sources as they stood at export time, and an evaluation harness that reproduces the browser's scores.

Dataset facts#

Event types ({{N_TYPES}}): {{TYPE_NAMES}}
Sampling rate: {{FS}} Hz, mono, float32
Runs: {{N_RUNS}}, each an independent synthesis (run r used seed {{SEED}} + r - 1)
Events per type per run: {{EVENTS_PER_TYPE}}
Snippet window: {{PRE_MS}} ms before the event onset, {{POST_MS}} ms after ({{SNIP_LEN}} samples; the onset therefore sits {{PRE}} samples into every training snippet)
Level variation: each placed event scaled by a random gain in [{{GAIN_MIN_DB}}, 0] dB (0 means events kept their recorded volume)
Overlap fraction: {{OVERLAP}} of events deliberately placed inside another event's span; the rest keep at least one snippet length of separation
Training fraction: {{TRAIN_FRACTION}} of each type's snippets went to the training set, drawn independently per run; the rest were placed
Match tolerance for scoring: {{TOLERANCE_MS}} ms

Layout#

README.md               this file
dataset_info.m          the facts above, as a function the harness calls
evaluate.m              runs and scores algorithms; edit its list
algorithms/*.m          algorithm sources from the app
tools/*.m               load_wav, load_csv, score_detections
runs/run_01/
  recording.wav         the synthesized recording (float32, {{FS}} Hz)
  ground_truth.csv      one row per placed event, no header:
                        time_sec, label (1-based), gain
  training_type_01.wav  training snippets for type 1, concatenated;
                        reshape to [{{SNIP_LEN}} x nSnippets]
  ...
runs/run_02/ ...

Ground-truth times refer to the event onset, which is {{PRE}} samples after the start of the placed snippet. The WAV files carry a plain 44-byte header (IEEE float, format tag 3); audioread in MATLAB, soundfile in Python, and ordinary audio editors read them directly, and tools/load_wav.m reads them with fopen for numbl, which has no audioread.

The algorithm interface#

An algorithm is one MATLAB-syntax file, algorithms/.m, defining a function whose name matches the file:

function [times, labels] = my_sorter(x, fs, training)
% x         the recording, column vector at fs Hz
% fs        sampling rate ({{FS}})
% training  struct with fields:
%   snips   {1 x nTypes} cell; snips{k} is [{{SNIP_LEN}} x nSnips_k],
%           one training snippet per column
%   names   {1 x nTypes} cell of type names
%   pre     samples before the event onset within each snippet ({{PRE}})
% Return column vectors: event times in seconds and 1-based type labels.
times = zeros(0, 1);
labels = zeros(0, 1);
end

The training snippets are the only labeled data an algorithm may use; the events placed in the recording come from a disjoint pool, so the task includes generalizing across instances, not recognizing exact waveforms.

Evaluating#

With Node.js installed:

npx numbl run evaluate.m

numbl (https://numbl.org) executes MATLAB-syntax code; no MATLAB license is involved. Add your algorithm's name to the algorithms list at the top of evaluate.m and run again. For each algorithm the harness prints per-run accuracy, the mean and standard deviation over runs, total counts of the four error kinds, and a per-type breakdown, matching the browser's report exactly.

Scoring: detections are matched to ground truth greedily by time difference within the {{TOLERANCE_MS}} ms tolerance, labels ignored; a matched pair with the right label is correct and with the wrong label a misclassification, unmatched truth events are misses, and unmatched detections are false positives. Accuracy is correct / (correct + misclassified + missed + false positives). tools/score_detections.m is the reference implementation and mirrors the browser harness; an algorithm developed against this archive will score identically when pasted back into the app.

Note that numbl does not implement every MATLAB function. The set it does implement is documented at https://numbl.org/docs/builtins; an algorithm that runs here will also run in the app, since both use numbl.