Cache a grid of estimates on an `estimates` branch via workflow dispatch
A dispatched workflow runs 24 cells (four filters x six sigmas), appends one
record per cell to runs.jsonl on an orphan `estimates` branch, and rebuilds
estimates.json by pooling every past ever drawn for each cell, so the means
tighten with each run. Seeds advance with the per-cell run count, keeping the
added pasts independent and the whole cache reproducible from the log alone.
Records carry their own parameters, so editing the grid does not invalidate
what is already cached.
5 changed files+389−0
.github/workflows/estimates.ymladded+65−0View file
@@ -0,0 +1,65 @@
1+name: estimates
2+
3+# Runs the parameter grid in scripts/grid.py and caches the results on the
4+# `estimates` branch. Each dispatch adds fresh, independent pasts to every cell
5+# and rebuilds the pooled means, so repeated runs tighten the error bars.
6+
7+on:
8+ workflow_dispatch:
9+
10+permissions:
11+ contents: write
12+
13+# One sweep at a time — two concurrent dispatches would race on the push.
14+concurrency:
15+ group: estimates
16+ cancel-in-progress: false
17+
18+jobs:
19+ sweep:
20+ runs-on: ubuntu-latest
21+ timeout-minutes: 60
22+ steps:
23+ - uses: actions/checkout@v4
24+
25+ - uses: actions/setup-python@v5
26+ with:
27+ python-version: '3.12'
28+
29+ - run: pip install -e .
30+
31+ - name: Check out the estimates branch (creating it if absent)
32+ env:
33+ GH_TOKEN: ${{ github.token }}
34+ run: |
35+ set -euo pipefail
36+ mkdir -p data
37+ cd data
38+ git init -q
39+ git remote add origin \
40+ "https://x-access-token:${GH_TOKEN}@github.com/${GITHUB_REPOSITORY}.git"
41+ if git fetch -q --depth 1 origin estimates; then
42+ git checkout -q -B estimates FETCH_HEAD
43+ echo "continuing $(wc -l < runs.jsonl) cached records"
44+ else
45+ git checkout -q -b estimates
46+ echo "estimates branch does not exist yet — starting it"
47+ fi
48+
49+ - name: Run the sweep
50+ run: python scripts/run_sweep.py --data-dir data
51+
52+ - name: Commit and push
53+ run: |
54+ set -euo pipefail
55+ cd data
56+ git add -A
57+ if git diff --cached --quiet; then
58+ echo "nothing to commit"
59+ exit 0
60+ fi
61+ git \
62+ -c user.name='github-actions[bot]' \
63+ -c user.email='41898282+github-actions[bot]@users.noreply.github.com' \
64+ commit -q -m "estimates: run ${GITHUB_RUN_NUMBER} from ${GITHUB_SHA:0:7}"
65+ git push -q origin estimates
.gitignoremodified+1−0View file
@@ -4,3 +4,4 @@ dist/
44 build/
55 .venv/
66 .pytest_cache/
7+data/
README.mdmodified+18−0View file
@@ -78,6 +78,24 @@ Filters match the web app: `none`, `moving-average`, `lowpass`, `bandpass`,
7878 the rate.
7979 - `--workers` caps the process pool (default: all cores).
8080
81+## Cached estimates
82+
83+The `estimates` branch caches a grid of estimates so the common settings need
84+not be recomputed: four filters (`none`, moving-average 8, lowpass 3000 Hz,
85+bandpass 300-6000 Hz at 30 kHz) crossed with sigma in {1, 2, 4, 8, 16, 32}.
86+
87+ https://raw.githubusercontent.com/concept-collection/timeseries-entropy/estimates/estimates.json
88+
89+Dispatch the **estimates** workflow to add to it. Each run draws 8 fresh,
90+independent pasts per cell, appends one record per cell to `runs.jsonl`, and
91+rebuilds `estimates.json` by pooling every past ever drawn — so the means keep
92+tightening the more often it runs. The grid lives in
93+[scripts/grid.py](scripts/grid.py); adding a cell does not invalidate the
94+cache, since each record stores its own parameters. To fill the cache locally
95+instead:
96+
97+ python scripts/run_sweep.py --data-dir data
98+
8199 ## Analytic prediction
82100
83101 `timeseries_entropy.theory` predicts the rate from the Fourier modes
scripts/grid.pyadded+74−0View file
@@ -0,0 +1,74 @@
1+"""The parameter grid whose estimates are cached on the `estimates` branch.
2+
3+Each cell is a plain dict of CLI-equivalent parameters; `cell_id` names it and
4+`kernel_from_params` rebuilds its kernel. Params are stored verbatim in every
5+record on the data branch, so aggregation never needs this file — the grid can
6+grow or change without invalidating what is already cached.
7+"""
8+
9+import sys
10+from pathlib import Path
11+
12+sys.path.insert(0, str(Path(__file__).resolve().parent.parent / 'src'))
13+
14+from timeseries_entropy import kernels # noqa: E402
15+
16+RATE = 30000.0
17+TAPS = 101
18+
19+SIGMAS = [1.0, 2.0, 4.0, 8.0, 16.0, 32.0]
20+
21+FILTERS = [
22+ {'filter': 'none'},
23+ {'filter': 'moving-average', 'width': 8},
24+ {'filter': 'lowpass', 'high': 3000.0, 'rate': RATE, 'taps': TAPS},
25+ {'filter': 'bandpass', 'low': 300.0, 'high': 6000.0, 'rate': RATE,
26+ 'taps': TAPS},
27+]
28+
29+
30+def grid():
31+ """The full list of cells: one representative per filter x every sigma."""
32+ return [dict(f, sigma=s) for f in FILTERS for s in SIGMAS]
33+
34+
35+def kernel_from_params(p):
36+ """Kernel array for a params dict — mirrors the CLI's --filter handling."""
37+ f = p['filter']
38+ if f == 'none':
39+ return kernels.identity()
40+ if f == 'first-difference':
41+ return kernels.first_difference()
42+ if f == 'moving-average':
43+ return kernels.moving_average(int(p['width']))
44+ if f == 'lowpass':
45+ return kernels.windowed_sinc_lowpass(p['high'] / p['rate'],
46+ int(p['taps']))
47+ if f == 'bandpass':
48+ return kernels.windowed_sinc_bandpass(p['low'] / p['rate'],
49+ p['high'] / p['rate'],
50+ int(p['taps']))
51+ raise ValueError(f'unknown filter {f!r}')
52+
53+
54+def cell_id(p):
55+ """Stable, filename-safe name for a cell, e.g. 'bp300-6000_s8'."""
56+ f = p['filter']
57+ if f == 'none':
58+ tag = 'none'
59+ elif f == 'first-difference':
60+ tag = 'diff'
61+ elif f == 'moving-average':
62+ tag = f"ma{_num(p['width'])}"
63+ elif f == 'lowpass':
64+ tag = f"lp{_num(p['high'])}"
65+ elif f == 'bandpass':
66+ tag = f"bp{_num(p['low'])}-{_num(p['high'])}"
67+ else:
68+ raise ValueError(f'unknown filter {f!r}')
69+ return f"{tag}_s{_num(p['sigma'])}"
70+
71+
72+def _num(v):
73+ """4.0 -> '4', 0.5 -> '0.5'."""
74+ return str(int(v)) if float(v) == int(v) else str(v)
scripts/run_sweep.pyadded+231−0View file
@@ -0,0 +1,231 @@
1+"""Run the cached-estimate grid and fold the results into the data branch.
2+
3+ python scripts/run_sweep.py --data-dir data
4+
5+Appends one line per cell to <data-dir>/runs.jsonl and rebuilds
6+<data-dir>/estimates.json from the whole log. Each dispatch draws fresh,
7+independent pasts (the seed advances with the number of runs already recorded
8+for that cell), so the pooled mean tightens run after run.
9+
10+Driven by .github/workflows/estimates.yml, but works standalone: point it at
11+any directory and run it as often as you like.
12+"""
13+
14+import argparse
15+import json
16+import math
17+import os
18+import time
19+import zlib
20+from datetime import datetime, timezone
21+from pathlib import Path
22+
23+import numpy as np
24+
25+import grid # first: also puts src/ on sys.path for a bare checkout
26+from timeseries_entropy import estimate_conditional_entropy
27+from timeseries_entropy.theory import predict_entropy_rate
28+
29+RUNS = 'runs.jsonl'
30+SUMMARY = 'estimates.json'
31+
32+
33+def main():
34+ ap = argparse.ArgumentParser(description=__doc__)
35+ ap.add_argument('--data-dir', required=True,
36+ help='checkout of the estimates branch')
37+ ap.add_argument('--pasts', type=int, default=8,
38+ help='independent pasts added per cell per run')
39+ ap.add_argument('--reps', type=int, default=8,
40+ help='randomized realizations averaged per past')
41+ ap.add_argument('--n0', type=int, default=128)
42+ ap.add_argument('--r', type=float, default=1.5)
43+ ap.add_argument('--thin', type=int, default=1)
44+ ap.add_argument('--workers', type=int)
45+ ap.add_argument('--only', help='substring of cell id, for local testing')
46+ args = ap.parse_args()
47+
48+ data_dir = Path(args.data_dir)
49+ data_dir.mkdir(parents=True, exist_ok=True)
50+ records = read_records(data_dir / RUNS)
51+ runs_so_far = {}
52+ for rec in records:
53+ runs_so_far[rec['cell']] = runs_so_far.get(rec['cell'], 0) + 1
54+
55+ cells = [c for c in grid.grid()
56+ if args.only is None or args.only in grid.cell_id(c)]
57+ print(f'{len(cells)} cells x {args.pasts} pasts x {args.reps} reps '
58+ f'({len(records)} records already cached)')
59+
60+ new = []
61+ t_all = time.time()
62+ for i, params in enumerate(cells):
63+ cid = grid.cell_id(params)
64+ run_index = runs_so_far.get(cid, 0)
65+ kernel = grid.kernel_from_params(params)
66+ M = max(512, 4 * kernel.size)
67+ seed = [zlib.crc32(cid.encode()), run_index]
68+ t0 = time.time()
69+ est = estimate_conditional_entropy(
70+ kernel, params['sigma'], past=M, pasts=args.pasts, reps=args.reps,
71+ n0=args.n0, r=args.r, thin=args.thin, seed=seed,
72+ workers=args.workers)
73+ v = np.asarray(est.per_past, dtype=float)
74+ rec = {
75+ 'cell': cid,
76+ 'params': params,
77+ 'M': M,
78+ 'L': int(kernel.size),
79+ 'pasts': int(v.size),
80+ 'reps': args.reps,
81+ 'n0': args.n0,
82+ 'r': args.r,
83+ 'thin': args.thin,
84+ 'sum': float(v.sum()),
85+ 'sumsq': float((v ** 2).sum()),
86+ 'seed': seed,
87+ 'run_index': run_index,
88+ 'utc': utc_now(),
89+ 'code_sha': os.environ.get('GITHUB_SHA'),
90+ 'workflow_run': os.environ.get('GITHUB_RUN_ID'),
91+ }
92+ new.append(rec)
93+ print(f' [{i + 1:2d}/{len(cells)}] {cid:<16} '
94+ f'H = {est.mean:.4f} +/- {est.se:.4f} '
95+ f'({time.time() - t0:.0f}s)', flush=True)
96+
97+ append_records(data_dir / RUNS, new)
98+ summary = summarize(records + new, args)
99+ write_json(data_dir / SUMMARY, summary)
100+ write_readme(data_dir / 'README.md', summary)
101+ print(f'\n{len(new)} records appended in {time.time() - t_all:.0f}s; '
102+ f'{len(summary["cells"])} cells summarized')
103+ step_summary(summary)
104+
105+
106+def read_records(path):
107+ if not path.exists():
108+ return []
109+ with path.open() as f:
110+ return [json.loads(line) for line in f if line.strip()]
111+
112+
113+def append_records(path, records):
114+ with path.open('a') as f:
115+ for rec in records:
116+ f.write(json.dumps(rec, sort_keys=True) + '\n')
117+
118+
119+def summarize(records, args):
120+ """Pool every recorded per-past value, per cell, into a mean and an SE."""
121+ cells = {}
122+ for rec in records:
123+ c = cells.setdefault(rec['cell'], {
124+ 'id': rec['cell'], 'n_pasts': 0, 'n_runs': 0,
125+ '_sum': 0.0, '_sumsq': 0.0})
126+ c['params'] = rec['params']
127+ c['M'] = rec['M']
128+ c['L'] = rec['L']
129+ c['n_pasts'] += rec['pasts']
130+ c['n_runs'] += 1
131+ c['_sum'] += rec['sum']
132+ c['_sumsq'] += rec['sumsq']
133+ c['last_utc'] = max(rec['utc'], c.get('last_utc', ''))
134+
135+ out = []
136+ for c in sorted(cells.values(), key=lambda c: c['id']):
137+ n, s, ss = c['n_pasts'], c.pop('_sum'), c.pop('_sumsq')
138+ mean = s / n
139+ # Sample variance of the pooled per-past values; the pasts are
140+ # independent across runs as well as within one, so the SE of their
141+ # mean is the honest error bar on the whole cache.
142+ var = max((ss - s * s / n) / (n - 1), 0.0) if n > 1 else float('nan')
143+ pred = predict_entropy_rate(grid.kernel_from_params(c['params']),
144+ c['params']['sigma'])
145+ c.update({
146+ 'mean': mean,
147+ 'se': math.sqrt(var / n) if n > 1 else None,
148+ 'sd': math.sqrt(var) if n > 1 else None,
149+ 'predicted': pred['corrected'],
150+ 'highres_szego': pred['highres'] if math.isfinite(pred['highres'])
151+ else None,
152+ })
153+ out.append(c)
154+
155+ return {
156+ 'generated': utc_now(),
157+ 'estimand': 'H(z_{M+1} | z_1..z_M) in bits/sample, for '
158+ 'x ~ iid N(0, sigma^2) -> y = h * x -> z = round(y)',
159+ 'total_runs': max((c['n_runs'] for c in out), default=0),
160+ 'settings': {'pasts_per_run': args.pasts, 'reps': args.reps,
161+ 'n0': args.n0, 'r': args.r, 'thin': args.thin},
162+ 'cells': out,
163+ }
164+
165+
166+def write_json(path, obj):
167+ path.write_text(json.dumps(obj, indent=2, sort_keys=True) + '\n')
168+
169+
170+def utc_now():
171+ return datetime.now(timezone.utc).strftime('%Y-%m-%dT%H:%M:%SZ')
172+
173+
174+def rows(summary):
175+ for c in summary['cells']:
176+ se = '' if c['se'] is None else f"{c['se']:.4f}"
177+ yield (c['id'], f"{c['params']['sigma']:g}", str(c['M']),
178+ f"{c['mean']:.4f}", se, str(c['n_pasts']),
179+ f"{c['predicted']:.4f}")
180+
181+
182+HEAD = ('cell', 'sigma', 'M', 'mean', 'se', 'pasts', 'predicted')
183+
184+
185+def table(summary):
186+ body = [HEAD, tuple('-' * 3 for _ in HEAD)] + list(rows(summary))
187+ return '\n'.join('| ' + ' | '.join(r) + ' |' for r in body)
188+
189+
190+def write_readme(path, summary):
191+ path.write_text(f"""\
192+# Cached entropy estimates
193+
194+Machine-generated by [`.github/workflows/estimates.yml`][wf] on `main` — do not
195+edit by hand. Every dispatch runs the whole grid again with fresh, independent
196+pasts, appends one record per cell to `runs.jsonl`, and rebuilds
197+`estimates.json` from the full log, so the means tighten run after run.
198+
199+- **`runs.jsonl`** — append-only, one JSON object per (cell, dispatch): the
200+ cell's parameters, how many pasts it contributed, their sum and sum of
201+ squares, the seed, and the code commit that produced them.
202+- **`estimates.json`** — pooled `mean` and `se` per cell over every past ever
203+ drawn for it, alongside the analytic `predicted` rate for comparison.
204+
205+The estimand is H(z_(M+1) | z_1..z_M) in bits/sample — an upper bound on the
206+entropy rate of z that tightens as the conditioning window M grows.
207+
208+Fetch it from a browser or a script (raw.githubusercontent.com sends
209+`access-control-allow-origin: *`):
210+
211+ https://raw.githubusercontent.com/concept-collection/timeseries-entropy/estimates/estimates.json
212+
213+## Current state ({summary['total_runs']} runs, generated {summary['generated']})
214+
215+{table(summary)}
216+
217+[wf]: https://github.com/concept-collection/timeseries-entropy/blob/main/.github/workflows/estimates.yml
218+""")
219+
220+
221+def step_summary(summary):
222+ path = os.environ.get('GITHUB_STEP_SUMMARY')
223+ if not path:
224+ return
225+ with open(path, 'a') as f:
226+ f.write(f"## Cached estimates after {summary['total_runs']} runs\n\n"
227+ f'{table(summary)}\n')
228+
229+
230+if __name__ == '__main__':
231+ main()