/ concept-collection / timeseries-compressibility
Sign in
concept-collection / timeseries-compressibility
timeseries-compressibility / README.md
63 lines · 3.3 KBCodeBlameHistory
3Interactive exploration of how compressible quantized time series are.
5The generating model is: i.i.d. Gaussian noise (std σ, measured in quantization
43704d2Clear the analytic rate: R now comes from the timeseries-entropy estimatorJeremy Magland 6steps) → FIR filter → round to integers. The app shows the filter (convolution
7kernel and frequency response, with cutoffs in Hz against a chosen sample
8rate), a window of the generated integer signal (stationary by default, with a
9play toggle to let it stream endlessly), and the measured compression of a
10block of the generated integers under nine methods — zlib, zstd, and an rANS
11entropy coder, each raw, delta-coded, and LPC-residual-coded — as bits per
12sample and as ratio against raw int16 storage. The predictor order and the
13block size are controls, so the measurement can be pushed from 10k to a million
14samples and LPC from order 1 to 128. Each prefilter group also carries a hollow
15bar: the order-0 entropy of the stream being coded, the limit a per-sample
16entropy coder cannot beat, which ANS misses by 1–2% (its symbol table plus its
17own arithmetic loss).
19The reference rate R — the entropy rate of the process, the bits/sample limit
3f857e3Estimate the reference rate R in the browserJeremy Magland 20no lossless method can beat — is estimated in the browser by the method of the
21companion
43704d2Clear the analytic rate: R now comes from the timeseries-entropy estimatorJeremy Magland 22[timeseries-entropy](https://github.com/concept-collection/timeseries-entropy)
23package: an unbiased Monte-Carlo estimator of H(z_next | a long past), by Gibbs
24sampling the latent Gaussian under the rounding constraints and applying
3f857e3Estimate the reference rate R in the browserJeremy Magland 25Rhee–Glynn randomized telescoping to the sampled chain. A button starts a web
26worker that averages one independent past at a time (live mean ± se, dashed
27line on the chart) until stopped; the app also shows the exact command to run
28the Python original at the same settings as an independent check. The
29in-browser code in `src/entropy/` is a hand-synced TypeScript port of that
30package — change one, change the other. A WebGPU Gibbs sweep may replace the
31scalar one someday; the sweep is isolated so it can be swapped.
33## Run it
35```sh
36npm install
37npm run dev
38```
40## Layout
42```
5bab85aRatio-first chart, quantization-floor theory formula, line-segment view, fixed latent dataJeremy Magland 43src/model/ the latent source (fixed seeded randomness indexed by sample
43704d2Clear the analytic rate: R now comes from the timeseries-entropy estimatorJeremy Magland 44 position, convolved zero-phase with the kernel on demand)
45 and the FIR presets
3f857e3Estimate the reference rate R in the browserJeremy Magland 46src/entropy/ the unbiased reference-rate estimator: hand-synced TypeScript
47 port of the timeseries-entropy package (Gibbs conditional
48 sampler, Rhee–Glynn telescoping, Cody erfc / Acklam ndtri,
49 xoshiro128** RNG)
36e8ceaInteractive explorer for compressibility of quantized filtered Gaussian time seriesJeremy Magland 50src/compress/ lossless codecs run in the browser: zlib (fflate), zstd (wasm),
51 ans.ts (a bit-identical port of simple_ans), and FLAC-style
52 integer LPC; borrowed from entropy-quantized-linear-transform
3f857e3Estimate the reference rate R in the browserJeremy Magland 53src/worker/ the codecs and the estimator run off the main thread; the
54 estimator worker refines one past at a time until terminated
43704d2Clear the analytic rate: R now comes from the timeseries-entropy estimatorJeremy Magland 55src/components/ controls, filter plots, signal canvas, compression chart,
56 and the reference-rate method note
59Every reported size round-trips through the decoder and includes whatever the
5bab85aRatio-first chart, quantization-floor theory formula, line-segment view, fixed latent dataJeremy Magland 60decoder needs (ANS symbol table, LPC coefficients). The signal view and the
61compression block read the same fixed latent noise sequence — parameter changes
62transform the same underlying data rather than resampling it, and the first
63window shown is the start of the block that gets compressed.
moveopenescclose