33c9443delta encodingJeremy Magland 3A benchmarking framework for evaluating compression algorithms on integer array datasets, with a focus on scientific data.
7ccd097update web-uiJeremy Magland 4
7Zia provides systematic benchmarking of compression methods for integer arrays, measuring:
8- Compression ratio
9- Encoding throughput (MB/s)
10- Decoding throughput (MB/s)
12## Repository Structure
14Key components are located in the `zia_benchmark/src/zia_benchmark/` directory:
16- Algorithms: `/algorithms/`
17 - LZMA: `/algorithms/lzma/`
18 - Zstandard: `/algorithms/zstd/`
19 - ZLIB: `/algorithms/zlib/`
20 - Simple ANS: `/algorithms/simple_ans/`
21 - etc.
23- Datasets: `/datasets/`
24 - Synthetic data generators:
25 - `/datasets/bernoulli/`: Binary random data
26 - `/datasets/gaussian/`: Normal distribution samples
27 - Real data: `/datasets/real/`
28 - etc.
30- Core functionality:
31 - `run_benchmarks.py`: Main benchmarking engine
35### Code Formatting
37This project uses pre-commit hooks to automatically format code before each commit. The formatting includes:
38- Python code formatting using black
39- TypeScript/JavaScript code formatting using npm scripts
41To set up the pre-commit hooks after cloning the repository:
431. Install pre-commit:
44```bash
45pip install pre-commit
46```
482. Install the git hook scripts:
49```bash
50pre-commit install
51```
53After this setup, code will be automatically formatted when you make a commit.
57Latest benchmark results: https://magland.github.io/zia/