43704d2Clear the analytic rate: R now comes from the timeseries-entropy estimatorJeremy Magland 1import { useMemo } from 'react'
2import katex from 'katex'
4function Tex({ tex, display }: { tex: string; display?: boolean }) {
5 const html = useMemo(
6 () => katex.renderToString(tex, { displayMode: !!display, throwOnError: false }),
7 [tex, display],
8 )
9 return <span dangerouslySetInnerHTML={{ __html: html }} />
10}
6b485dfCall R the entropy rate, and put the estimate button under its readoutJeremy Magland 12/** Where the entropy rate R comes from, in brief. */
43704d2Clear the analytic rate: R now comes from the timeseries-entropy estimatorJeremy Magland 13export default function MethodNote() {
14 return (
15 <div className="math-section">
16 <p>
17 R is the entropy rate of the quantized process z — the bits per sample that no lossless
18 code can beat. It has no usable closed form here, so it is estimated by an unbiased
19 Monte-Carlo method from the companion{' '}
20 <a href="https://github.com/concept-collection/timeseries-entropy">timeseries-entropy</a>{' '}
21 package. The estimand is the conditional entropy of the next sample given a long past,
22 </p>
23 <Tex display tex="H\big(z_{M+1} \,\big|\, z_1, \dots, z_M\big) \;\searrow\; R \qquad (M \to \infty)," />
24 <p>
25 which reaches R once M exceeds the memory of the process. A past is drawn from the
26 model, and the latent Gaussian input is Gibbs-sampled under the rounding constraints —
27 the latents that generated the past are an exact draw from the conditional, so the chain
28 starts in stationarity with no burn-in bias — emitting exact draws of z<sub>M+1</sub>.
29 Rhee–Glynn randomized telescoping with antithetic half-block corrections then turns the
30 plug-in entropies of that chain into an estimate whose expectation is exactly the
31 conditional entropy, despite the finite-sample bias of every plug-in estimate and the
32 autocorrelation of the Gibbs draws. Averaging over independent pasts gives R with an
33 honest standard error.
34 </p>
fec2284Show analytic entropy-rate prediction alongside the Monte-Carlo estimateJeremy Magland 35 <p>
36 The dotted line on the chart is the package's analytic approximation, computed
37 instantly from the filter and σ: Szegő's formula gives the error of linearly
38 predicting the next sample from the past, with roundoff entering as a uniform noise
39 floor of variance 1/12,
40 </p>
41 <Tex
42 display
43 tex="R \;\approx\; G(s_*), \qquad s_*^2 \;=\; \exp\!\Big(\int_0^1 \ln\big(\sigma^2\,|H(f)|^2 + \tfrac{1}{12}\big)\,df\Big) \;-\; \tfrac{1}{12},"
44 />
45 <p>
46 where G(s) is the differential entropy of N(0, s²) + U(−½, ½) — exactly the entropy
47 of a rounded Gaussian averaged over grid offsets. The floor keeps the integral finite
48 where H(f) has zeros, and G saturates to zero at coarse quantization instead of
49 diverging. Treating roundoff as independent dither and prediction as linear are
50 approximations — the Monte-Carlo estimate is the exact check on them.
51 </p>
43704d2Clear the analytic rate: R now comes from the timeseries-entropy estimatorJeremy Magland 52 <p className="card-note">
fec2284Show analytic entropy-rate prediction alongside the Monte-Carlo estimateJeremy Magland 53 The estimate button under the Monte-Carlo readout runs exactly this method in a web worker — a
6b485dfCall R the entropy rate, and put the estimate button under its readoutJeremy Magland 54 TypeScript port of the package (src/entropy, hand-synced), one independent past at a
55 time until stopped. The command line runs the Python original at the same settings for
56 an independent check.
43704d2Clear the analytic rate: R now comes from the timeseries-entropy estimatorJeremy Magland 57 </p>
58 </div>
59 )
60}