All notable changes to this project are documented here.
The format follows Keep a Changelog.
Release tags match the version committed in pyproject.toml.
StreamingIsotonicCalibratorfits with SciPy'sisotonic_regressioninstead of scikit-learn'sIsotonicRegression, which spent over 90% of each update validating inputs. Updates and half-life calibration are 11-27x faster; outputs are bit-for-bit unchanged. Requires SciPy 1.12.- Batches of 1,024+ labels find their bins, and 2,048+ probabilities interpolate, by direct calculation on the evenly spaced grid rather than a binary search, with identical results.
import streamcalno longer loads scikit-learn (1.1 s to 0.08 s). Names that need it load on first use. A process that only runs the streaming calibrator peaks at 75 MB rather than 148 MB.BatchCalibratorstores labels as one byte, so history takes 9 rather than 16 bytes per observation, with identical fits.
compare_prequentialno longer times batches with no observed labels as zero-cost updates, which understatedp95_update_msunder a sparseobserve_mask.
This release breaks the experimental 0.1 API. Predict with calibrate before
passing the original probabilities and available labels to update. Replace
method-specific batch classes with BatchCalibrator(method=...) and
expected_calibration_error with binned_calibration_error.
-
Prediction-age half-life weighting with explicit label arrival timestamps.
-
Constrained validation selection by Brier, log loss, or binned calibration error, with p95 call-latency and serialized-state budgets.
-
An exact sufficient-statistic proof, rolling/frozen/online comparisons, and delayed-label evaluation using River's event scheduler.
-
Leak-free prequential configuration comparison with quality, latency and fixed-state diagnostics.
-
Reproducible known-truth and chronological Elec2 evidence for documented correctness and resource claims.
-
Monte Carlo tests of calibration quality against known probabilities and mature batch references.
- Reject invalid probability, outcome, shape and parameter domains at public boundaries instead of broadcasting, clipping or returning NaNs.
- Weight streaming isotonic regression by decayed bin counts rather than giving every populated bucket equal influence.
- Delegate temperature fitting to scikit-learn's public calibration API.
- Use observation-weighted block-bootstrap intervals, including partial batches.
- Return no feasible choice for a zero-byte state budget instead of raising.
- CI, docs and release now run on py-canon's
reusable workflows in place of hand-rolled ones. The build backend moves from
hatchling to
uv_build, mypy is replaced by pyright, and the lint set widens to the fleet standard. streamcal.__version__is read from the installed distribution metadata rather than duplicated as a literal in__init__.py.- Project URLs point at
finite-sample/streamcal. - The public contract is now predict-then-observe: call
calibratebeforeupdate;updatereturns the calibrator and never fitted predictions for the same outcomes. - Consolidate batch references into
BatchCalibrator(method=...), with rolling windows, refit cadence, and freezing. Sigmoid uses scikit-learn's probability input convention, replacing the former logistic-on-logits reference. - Require scikit-learn 1.8 or newer. Optional delayed evaluation and intervals
are available through
streamcal[evaluation]. expected_calibration_erroris nowbinned_calibration_errorto make its estimator dependence explicit.
-
NearlyIsotonicCalibrator; the former fixed-step SGD routine did not implement the cited nearly-isotonic solution path. -
The
experiments/andms/trees that produced the original write-up. The package is now the library alone.
Initial release: streaming isotonic and nearly-isotonic calibrators, batch temperature/isotonic/Platt baselines, and Brier and expected-calibration-error metrics.