# Physics-based simulation of musical instruments: the challenges collection

This collection holds a set of simulation challenges for the sound of struck and plucked instruments, together with the solutions people have submitted and the machinery that compares them. This repository is the entry point. It explains why the collection exists, how it is organized, how a solution is scored, and how to take part. The other repositories in the collection are listed at the end.

The results are published as static sites, one per tier:

| Tier | Challenges | Comparison of solutions |
| --- | --- | --- |
| 0, string | [challenges/string](https://vault1.magland.org/challenges/string) | [results](https://vault1.magland.org/challenges/string/site/) |
| 1, plate | [challenges/plate](https://vault1.magland.org/challenges/plate) | [results](https://vault1.magland.org/challenges/plate/site/) |
| 2, cymbal | [challenges/cymbal](https://vault1.magland.org/challenges/cymbal) | [results](https://vault1.magland.org/challenges/cymbal/site/) |

## Background

The motivating application for this project is realistic instrument sound generated from physical models. Producing it is not trivial. It requires coupled elastodynamics and acoustics, stiff time integration, and careful trade-offs between accuracy and cost, and the ear judges the result harshly: it detects errors in pitch, decay, and timbre that are easy to miss in a convergence plot. The primary contribution we are after is the numerical methods themselves and benchmarks of their accuracy against cost. The same physics arises in vibroacoustics and acoustic scattering, so the methods should be relevant beyond music.

Simulation methods are hard to compare fairly unless they are applied to the same problem with the same reference and the same error measures. A challenge here is therefore a fully specified continuum model together with a precisely defined output signal and error measures. The specification states the governing equations, boundary and initial conditions, every parameter in SI units, and exactly how the output is sampled into a file. The intent is that the true solution is a well-defined function that does not depend on any discretization, so that methods of very different kinds, modal, finite difference, finite element, integral equation, can be compared on equal terms. Tiers add realism: toy problems first, then thin plates, then geometric nonlinearity, then radiation into air, up to complete instruments.

There is no externally supplied ground truth. The reference for each challenge is chosen from among the submitted solutions by a rule described below, and the central question a challenge answers is how much computation a method can save while remaining close to the most accurate solution available.

## How the collection is organized

Each tier is a pair of repositories, plus shared tooling.

- **A challenge repository** (`challenges/string`, `challenges/plate`, `challenges/cymbal`) holds the specifications: one directory per challenge with a `README.md`, which is the authoritative definition, and a `challenge.json`, which repeats the parameters, output format, error measures, and the grid of the optional field export in machine-readable form. It also holds a registry of solution repositories and the workflow that builds the comparison site.
- **Solution repositories** (`challenges/string-solutions`, `challenges/plate-solutions`, `challenges/cymbal-solutions`, and any others that are registered) hold submissions: a JSON file per submission with metadata and one or more solutions, the output wav files, optional field exports for visualization, and whatever else the authors want, such as solver code or their own visualization site. A solution repository runs only the validator; it makes no comparisons of its own.
- **The tools** (`challenges/challenge-tools`) are a small Python package with the submission schema, the validator, the error measures, the reference-selection rule, and the generator of the comparison site. Every challenge repository installs it in its workflow.

The comparison workflow of a challenge repository clones every registered solution repository, reads the committed output files, computes every pairwise error, chooses the reference, and deploys the site. It runs on every push to the challenge repository or when dispatched by hand. Nothing in the vault triggers across repositories, so a push to a solution repository is picked up at the next run of the challenge repository's workflow. All solvers run on the submitters' own machines; the workflow runner only scores committed files.

## Submissions

A submission is a JSON file at `solutions/<challenge-id>/<submission-id>.json` in a solution repository, validated against the [schema](https://vault1.magland.org/challenges/challenge-tools/blob/main/schema.json). It names the submitter and holds one or more solutions. Each solution points to its output file (a path beside the JSON, or a URL), and may report the method, its discretization parameters, self-reported wall-clock time and hardware, a link to the source code, a link to a visualization, and an optional field export. Submissions are identified as `<repository>/<submission-id>` in the results, so identifiers from different repositories cannot collide.

A submission may assert an accuracy ordering among its own solutions through `more_accurate_than`; a sequence at increasing resolution is the typical case. These assertions are what allow the reference to be chosen automatically, so a submission that includes a convergence series is worth more to the collection than a single solution.

Output files are exact, unnormalized signals in a fixed physical scale stated by each challenge, as 32-bit float wav. They have a large crest factor (a loud strike followed by a much quieter ring) and sound faint when played raw; the results sites play them through a gain and a limiter, and each challenge suggests a playback gain. The files themselves are never altered.

## Scoring and the reference

Every solution is compared with the reference under the error measures defined in the challenge, at present the relative waveform error, which is strict and penalizes phase drift, and a relative spectrogram error, which is closer to what a listener notices. All measures are reported and one, named in the challenge, ranks.

The reference is chosen as follows. Only orderings asserted within a single submission are trusted. The candidates are the maximal elements of each submission's ordering. A candidate is scored by the fraction of asserted pairs from other submissions whose order is reproduced by the primary error to that candidate; the candidate reproducing the largest fraction wins. Ties are broken first by the candidate's convergence gap, the error between it and the nearest solution of its own submission asserted to be less accurate, and then by the smallest mean error to the other candidates. When nothing is asserted, the rule reduces to picking the candidate nearest the others.

The rule is only as good as the assertions. A sequence that is not in fact monotone makes a false claim, and with few submissions a candidate that happens to agree with that claim can be preferred over a more accurate one. The results pages therefore show, for every candidate, which asserted pairs it fails to reproduce, so that a reader can judge the choice, and a challenge may override the automatic choice by naming a reference explicitly. More independent submissions make the rule more reliable, and this is the main reason to submit even a modest method.

## The challenges so far

| Challenge | Model | Output |
| --- | --- | --- |
| [string-pluck-v1](https://vault1.magland.org/challenges/string/blob/main/challenges/string-pluck-v1/README.md) | Damped string with fixed ends, released from its static equilibrium under a finger load | 2 s of pickup velocity |
| [plate-strike-v1](https://vault1.magland.org/challenges/plate/blob/main/challenges/plate-strike-v1/README.md) | Free 12 in aluminum plate, linear 3D elastodynamics with Kelvin-Voigt damping, struck by a short force pulse | 3 s of radiated pressure by the Rayleigh integral |
| [plate-strike-nl-v1](https://vault1.magland.org/challenges/plate/blob/main/challenges/plate-strike-nl-v1/README.md) | The same plate as a Saint Venant-Kirchhoff solid (geometrically nonlinear), struck by a soft mallet | 3 s of radiated pressure |
| [cymbal-flat-disk-v1](https://vault1.magland.org/challenges/cymbal/blob/main/challenges/cymbal-flat-disk-v1/README.md) | Flat, free 16 in bronze disk of uniform thickness, Saint Venant-Kirchhoff, struck near the rim by a stick-like pulse | 6 s of radiated pressure |
| [cymbal-bow-v1](https://vault1.magland.org/challenges/cymbal/blob/main/challenges/cymbal-bow-v1/README.md) | The same disk bowed into a shallow paraboloid (25 mm rise): the effect of curvature alone | 6 s of radiated pressure |
| [cymbal-bell-v1](https://vault1.magland.org/challenges/cymbal/blob/main/challenges/cymbal-bell-v1/README.md) | The bowed disk with a cosine-dome bell and a mounting hole: the effect of the bell alone | 6 s of radiated pressure |
| [cymbal-hammered-v1](https://vault1.magland.org/challenges/cymbal/blob/main/challenges/cymbal-hammered-v1/README.md) | The belled disk with 150 hammer dents at fixed positions: the effect of breaking the symmetry | 6 s of radiated pressure |
| [cymbal-tapered-v1](https://vault1.magland.org/challenges/cymbal/blob/main/challenges/cymbal-tapered-v1/README.md) | The hammered cymbal lathed to a taper (1.2 to 0.7 mm): the effect of the thickness profile | 6 s of radiated pressure |

The string challenge is a hello world: its modal series solution is essentially exact, so it exercises the submission and scoring pipeline and gives a check on the reference rule. The plate challenges define three-dimensional models, so the thin-plate solutions submitted so far (Kirchhoff-Love finite elements for the linear plate, a modal von Kármán method for the nonlinear one) are approximations whose error relative to a future converged three-dimensional solution includes the modeling error of plate theory. Until such a solution arrives, the reference is the best thin-plate solution and the errors measure only differences within plate theory; the results pages say so. The cymbal tier starts from a flat disk and will add the features of a real cymbal one at a time (bell, bow, hammering, varying thickness), each as a new challenge, so that the effect of each on the sound and on the cost of computing it can be seen in isolation; its first solutions are von Kármán modal solutions on Cartesian grids that approximate the disk by a staircase.

## Conventions

- **Free bodies.** The plates and disks are free, and the strikes impart momentum and angular momentum. The outputs are defined with the rigid motion linearized: the acceleration relative to the best-fit rigid frame plus that frame's translational and angular acceleration, which keeps the kick of the strike in the sound and leaves out the slow turning of the body that only an exact three-dimensional solution would show. For modal thin-plate solvers this is what they compute with their rigid modes included.
- A challenge is never changed once solutions from others exist for it. A corrected or modified model gets a new identifier, for example a `-v2` suffix.
- Identifiers (challenges, submissions, solutions) are lowercase letters, digits, and hyphens.
- Field exports for visualization are HDF5 files on a grid recommended by the challenge; they are not scored. Files of a few megabytes live in plain git, larger ones in git LFS or at an external URL.
- Timing is self-reported wall-clock time for producing the output signal, on the hardware named in the submission.

## Taking part

- **Submit a solution.** Add a submission to an existing solution repository by pull request, or start a repository of your own with the same `solutions/` layout and register it by a pull request to the challenge repository's `registry/solution-repos.json`. The solution repositories' READMEs give the steps, and the validator can be run locally with `pip install git+https://vault1.magland.org/challenges/challenge-tools`.
- **Propose a challenge.** Open an issue or a pull request on the relevant challenge repository with a specification in the same form: continuum model, parameters, output, error measures. New tiers get a repository pair of their own.
- **Improve the tools.** The error measures and the reference rule are implemented in `challenge-tools`; the challenge READMEs are the specification and the code is intended to follow them exactly.

## Roadmap

The tiers sketched in the research plan, for a cymbal as the running example, are: a flat plate struck at a point (tier 1, done in linear and nonlinear form); a curved, cymbal-shaped shell (tier 2, begun with the flat disk); large-amplitude strikes with geometric nonlinearity on that shell (tier 3); and sound radiated into the surrounding air by a coupled structure and acoustic simulation (tier 4), replacing the Rayleigh integral used so far. Alongside the tiers we intend to pursue simulations of complete instruments, starting with a cymbal on a stand, which are harder to benchmark but show which effects matter most for the sound and so guide which challenges to add. Open questions include which errors are audible, since a cheaper model can have a large waveform error yet sound identical, and the collection is a natural place to host listening tests once there are enough solutions to compare.

## Repositories in this collection

| Repository | Contents |
| --- | --- |
| [overview](https://vault1.magland.org/challenges/overview) | this document |
| [challenge-tools](https://vault1.magland.org/challenges/challenge-tools) | schema, validator, error measures, reference rule, site generator |
| [string](https://vault1.magland.org/challenges/string) | tier 0 specifications, registry, comparison site |
| [string-solutions](https://vault1.magland.org/challenges/string-solutions) | solutions to the string challenges, with a 2D field viewer |
| [plate](https://vault1.magland.org/challenges/plate) | tier 1 specifications, registry, comparison site |
| [plate-solutions](https://vault1.magland.org/challenges/plate-solutions) | solutions to the plate challenges, with a 3D field viewer |
| [cymbal](https://vault1.magland.org/challenges/cymbal) | tier 2 specifications, registry, comparison site |
| [cymbal-solutions](https://vault1.magland.org/challenges/cymbal-solutions) | solutions to the cymbal challenges, with a 3D field viewer |