cd2b036timeseriesJeremy Magland 3A benchmarking framework for evaluating compression algorithms on numeric timeseries datasets, with a focus on scientific data.
7ccd097update web-uiJeremy Magland 4
7884e06rename to benchcompressJeremy Magland 5Latest benchmark results: https://magland.github.io/benchcompress/
0602a7eui improvementsJeremy Magland 6
cd2b036timeseriesJeremy Magland 9Benchcompress is a comprehensive benchmarking framework for evaluating compression algorithms on numeric timeseries datasets. The system follows an automated workflow:
33c9443delta encodingJeremy Magland 10
7884e06rename to benchcompressJeremy Magland 12 - Algorithms are implemented in `benchcompress/src/benchcompress/algorithms/`
13 - Datasets are defined in `benchcompress/src/benchcompress/datasets/`
69b432bupdate READMEJeremy Magland 14 - Each component specifies metadata like version, tags, and compatibility requirements
33c9443delta encodingJeremy Magland 15
17 - Benchmarks run automatically via GitHub Actions on pushes to main branch
18 - For each compatible algorithm-dataset pair, measures:
19 - Compression ratio
20 - Encoding throughput (MB/s)
21 - Decoding throughput (MB/s)
22 - Results are verified by decompressing and comparing with original data
33c9443delta encodingJeremy Magland 23
25 - Results are committed to a dedicated `benchmark-results` branch
26 - Local and remote caching system prevents redundant rerunning of benchmarks (only modified or added components are re-benchmarked)
27 - Caching is based on algorithm and dataset versions
33c9443delta encodingJeremy Magland 28
7884e06rename to benchcompressJeremy Magland 30 - Interactive visualization at https://magland.github.io/benchcompress/
32 - Visual charts for comparing performance metrics
33 - Export results to CSV for further analysis
33c9443delta encodingJeremy Magland 34
33c9443delta encodingJeremy Magland 36
7884e06rename to benchcompressJeremy Magland 39- `benchcompress/`: Python package containing the core benchmarking framework, algorithms, and datasets
69b432bupdate READMEJeremy Magland 40- `web-ui/`: React-based web interface for visualizing benchmark results
42### Local Development Setup
441. Install Python dependencies:
45```bash
50benchcompress list
51benchcompress run --help
542. Install web UI dependencies:
55```bash
56cd web-ui
57npm install
58```
603. Run web UI locally:
61```bash
62cd web-ui
63npm run dev
64```
66### Code Formatting
68This project uses pre-commit hooks to automatically format code before each commit. The formatting includes:
69- Python code formatting using black
70- TypeScript/JavaScript code formatting using npm scripts
72To set up the pre-commit hooks after cloning the repository:
741. Install pre-commit:
75```bash
76pip install pre-commit
77```
792. Install the git hook scripts:
80```bash
81pre-commit install
82```
84After this setup, code will be automatically formatted when you make a commit.
bcd9497update readmeJeremy Magland 85
86# Theory
88For independently and identically distributed (i.i.d.) discrete data, where each sample is drawn from a discrete probability distribution (e.g., Bernoulli sampling or quantized Gaussian noise), the theoretical compression ratio is determined by the Shannon entropy formula:
89$$
90H(X) = -\sum_{i} p(x_i) \log p(x_i).
91$$
92Here, $p(x_i)$ represents the probability of occurrence of the $i$-th symbol $x_i$ in the discrete distribution.
cd2b036timeseriesJeremy Magland 94In practice, achieving this theoretical compression ratio often requires sophisticated encoding techniques. While arithmetic encoding provides one such method, it is challenging to implement and can be computationally inefficient. A more modern and efficient alternative is Asymmetric Numeric Systems (ANS), which closely approaches the theoretical limit and is incorporated into state-of-the-art compressors such as ZStandard. However, these algorithms are primarily optimized for structured data types, such as text, rather than for scientific numerical timeseries data.
96In our benchmarks, we evaluate a simple implementation of ANS using a Python package we developed, called \texttt{simple\_ans}. As anticipated, ANS demonstrates superior performance when compressing i.i.d. samples from a discrete distribution. However, its efficiency diminishes when handling more structured data, such as continuous signals (e.g., voltage traces in electrophysiology).
98Applying delta encoding partially mitigates this limitation by leveraging the continuity properties of the data through differencing. This preprocessing step enhances ANS performance, though it still falls short of the compression achieved by methods like ZStandard. Additional preprocessing techniques, such as linear Markov predictive modeling (where the residual error after prediction is compressed instead of the original signal), further improve ANS performance. In these scenarios, the residual data is smaller and exhibits reduced correlation, enabling ANS to achieve better compression results relative to other methods.