05ee84fimprove bar graphsJeremy Magland 3A benchmarking framework for evaluating compression algorithms on scientific data arrays.
7ccd097update web-uiJeremy Magland 4
7884e06rename to benchcompressJeremy Magland 5Latest benchmark results: https://magland.github.io/benchcompress/
0602a7eui improvementsJeremy Magland 6
93dd44areorganize datasets and algs, add seismicJeremy Magland 11Benchcompress is a comprehensive benchmarking framework for evaluating compression algorithms on scientific data arrays. The system follows an automated workflow:
33c9443delta encodingJeremy Magland 12
7884e06rename to benchcompressJeremy Magland 14 - Algorithms are implemented in `benchcompress/src/benchcompress/algorithms/`
15 - Datasets are defined in `benchcompress/src/benchcompress/datasets/`
69b432bupdate READMEJeremy Magland 16 - Each component specifies metadata like version, tags, and compatibility requirements
33c9443delta encodingJeremy Magland 17
19 - Benchmarks run automatically via GitHub Actions on pushes to main branch
20 - For each compatible algorithm-dataset pair, measures:
21 - Compression ratio
22 - Encoding throughput (MB/s)
23 - Decoding throughput (MB/s)
24 - Results are verified by decompressing and comparing with original data
33c9443delta encodingJeremy Magland 25
27 - Results are committed to a dedicated `benchmark-results` branch
28 - Local and remote caching system prevents redundant rerunning of benchmarks (only modified or added components are re-benchmarked)
29 - Caching is based on algorithm and dataset versions
33c9443delta encodingJeremy Magland 30
7884e06rename to benchcompressJeremy Magland 32 - Interactive visualization at https://magland.github.io/benchcompress/
34 - Visual charts for comparing performance metrics
35 - Export results to CSV for further analysis
33c9443delta encodingJeremy Magland 36
33c9443delta encodingJeremy Magland 38
7884e06rename to benchcompressJeremy Magland 41- `benchcompress/`: Python package containing the core benchmarking framework, algorithms, and datasets
69b432bupdate READMEJeremy Magland 42- `web-ui/`: React-based web interface for visualizing benchmark results
44### Local Development Setup
461. Install Python dependencies:
47```bash
52benchcompress list
53benchcompress run --help
562. Install web UI dependencies:
57```bash
58cd web-ui
59npm install
60```
623. Run web UI locally:
63```bash
64cd web-ui
65npm run dev
66```
68### Code Formatting
70This project uses pre-commit hooks to automatically format code before each commit. The formatting includes:
71- Python code formatting using black
72- TypeScript/JavaScript code formatting using npm scripts
74To set up the pre-commit hooks after cloning the repository:
761. Install pre-commit:
77```bash
78pip install pre-commit
79```
812. Install the git hook scripts:
82```bash
83pre-commit install
84```
86After this setup, code will be automatically formatted when you make a commit.