import { useMemo } from 'react' import katex from 'katex' function Tex({ tex, display }: { tex: string; display?: boolean }) { const html = useMemo( () => katex.renderToString(tex, { displayMode: !!display, throwOnError: false }), [tex, display], ) return } /** One term of the formula: the symbol, then what it is. */ function Def({ tex, children }: { tex: string; children: React.ReactNode }) { return ( <>
The dashed line on the compression chart is R, the predicted bits per sample — the smaller of a spectral estimate and a rigorous one-sample ceiling:
The dither identity starts it off: for z = round(y) and u an independent uniform on [-½, ½)N, the discrete entropy of z equals the differential entropy of z + u, exactly. When the process is live on the unit-cell scale (v ≳ ¼), z + u has nearly the law of y + u, so R is the entropy rate of the signal plus a white unit-cell noise — the Zamir–Feder universal-quantization rate; with physical dither the smoothing noise is d + u and ν doubles to 1/6.
Counting that entropy per Fourier mode, the signal modes are independent Gaussians of variance S(f), and each mode of the i.i.d. cube noise mixes all N samples' contributions — so the central limit theorem Gaussianizes it, and it enters at its{' '} full variance ν. It does not enter at the entropy power 1/(2πe) that a scalar quantizer aligned with the mode would charge: the quantization lattice lives in the sample basis, and only for the trivial kernel do modes and quantizers align. (An earlier version of this app charged entropy power — additive constant 1 instead of 2πe·ν ≈ 1.42 — and systematically underestimated the measured rate by up to ~0.23 bits/sample. Its "exact per-mode" refinement was worse still: it modeled the wrong physics more faithfully.) One consequence worth naming: a dead band inside a live process contributes ½log₂(2πe/12) ≈ 0.25 bits per mode, not zero.
When v ≪ 1 nearly every sample rounds to zero and the true rate collapses exponentially, while Rspec bottoms out at ½log₂(2πe ν) > 0. Subadditivity rescues the estimate rigorously: H(z) ≤ Σn H(zn), and each stored sample is exactly round(N(0, v)) — plus the uniform dither first when it is on — so Rsamp is a true upper bound on the rate with exactly the right collapse. The min selects it precisely where the spectral branch fails.
For the identity kernel the two branches agree with the exact i.i.d. entropy at every σ (both carry the Fisher correction log₂e/(24σ²) at large σ; below one step the min switches to the exact Rsamp). At high SNR, Rspec → ½log₂(2πe σ²) + ∫log₂|H| df — the Kolmogorov formula. What the spectral branch ignores is the cross-mode dependence of the cube noise, at most ½log₂(2πe/12) ≈ 0.2546 bits/sample and recoverable only when nearly the whole spectrum is noise-dominated; Monte-Carlo puts the estimate within ~0.01–0.02 bits/sample for v ≳ 0.25, with the worst observed error ~+0.03 near the crossover between branches, slightly positive everywhere — as befits a formula whose sample branch is a genuine bound.
The integral is evaluated by the midpoint rule on 8192 points over [0, ½] (symmetry supplies the other half). Rsamp sums the exact bin probabilities of the rounded Gaussian — integrated against the triangular dither-overlap window when dither is on. The Monte-Carlo command under the chart estimates the true entropy rate of the same process, for checking R where the approximations are in doubt.