Benchmarking Compression Algorithms for Scientific Data Arrays#

Jeremy Magland, Center for Computational Mathematics, Flatiron Institute

Last updated: January 2025

This is a draft paper, work in progress.

Abstract#

Benchcompress is a benchmarking framework designed to evaluate the performance of various compression algorithms on scientific data arrays. The framework automates the benchmarking process, measuring compression ratio, encoding throughput, and decoding throughput for each algorithm-dataset pair. Results are verified through decompression and comparison with the original data, ensuring accuracy and reliability. The benchmark results are stored and visualized through an interactive web interface, allowing users to filter, sort, and explore the data. This paper presents the design, implementation, and preliminary results of Benchcompress, highlighting its utility in identifying optimal compression techniques for scientific datasets.

Introduction#

This is the introduction.

Methods#

Compression Algorithms#

Describe the various compression methods.

Dataset Generation#

Describe the datasets we use for benchmarking.

Theory#

For independently and identically distributed (i.i.d.) discrete data, where each sample is drawn from a discrete probability distribution (e.g., Bernoulli sampling or quantized Gaussian noise), the theoretical compressed size in bits per sample is determined by the Shannon entropy formula:

Here, $p(x_i)$ represents the probability of occurrence of the $i$-th symbol $x_i$ in the discrete distribution. So for example, if we have a Bernoulli distribution with $p=0.5$, the entropy is

bits per sample. This means that the optimal compression ratio for such a dataset is 8, assuming the samples are stored as 8-bit integers. On the other hand, if $p\neq 0.5$, the entropy becomes lower and we can achieve compression at a rate of less than 1 bit per sample (e.g., for $p=0.1$, the entropy is around 0.47 bits per sample, so the compression ratio would be around 17).

In practice, achieving this theoretical compression ratio requires sophisticated encoding techniques. Arithmetic encoding [ref] is one such method, but it is challenging to implement and can be computationally inefficient. A more modern and efficient alternative is Asymmetric Numeric Systems (ANS) [ref], which closely approaches the theoretical limit and is incorporated into state-of-the-art compressors such as ZStandard [ref]. However, these algorithms are primarily optimized for structured data types, such as text, rather than for numeric scientific data.

In our benchmarks, we evaluate a simple implementation of ANS using a Python package we developed, called simple_ans. As anticipated, ANS demonstrates superior performance when compressing i.i.d. samples from a discrete distribution. However, its efficiency diminishes when handling more structured data, such as continuous signals (e.g., voltage traces in electrophysiology).

Applying delta encoding partially mitigates this limitation by leveraging the continuity properties of the data through differencing. This preprocessing step enhances ANS performance, though it still falls short of the compression achieved by methods like ZStandard. Additional preprocessing techniques, such as linear Markov predictive modeling (where the residual error after prediction is compressed instead of the original signal), further improve ANS performance. In these scenarios, the residual data is smaller and exhibits reduced correlation, enabling ANS to achieve better compression results relative to other methods.

Implementation#

[To be added]

Results#

[Preliminary results to be added]

Discussion#

[Discussion to be added]

Conclusion#

[Conclusion to be added]