Add util.rereference: deterministic cluster-aware rereference matrices - #186
Merged
Conversation
New ezmsg.sigproc.util.rereference module with RereferenceKind (identity|car), car_matrix() (standard and leave-one-out CAR, per-cluster, with min_reref_size guard), and a rereference_matrix() dispatcher. Matrices are built as host-side numpy; AffineTransformTransformer already converts weights to the message's namespace/dtype/device on first use. AffineTransformSettings.weights now also accepts a RereferenceKind or its string value (e.g. "car" from config), built over channel_clusters at reset so the same cluster spec drives both matrix construction and the block-diagonal matmul optimization. weights=None remains passthrough. Docstrings point users wanting plain streaming CAR to CommonRereference; kind-based weights serve discoverability and init-then-set_weights workflows (e.g. the LRR cold start in ezmsg-learn, which will build on this). Also extract shared validate_channel_clusters() into util.channels, replacing duplicate bounds checks in AffineTransformTransformer and downstream consumers. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
cboulay
added a commit
to ezmsg-org/ezmsg-learn
that referenced
this pull request
Jul 23, 2026
ezmsg-sigproc 2.34.0 (ezmsg-org/ezmsg-sigproc#186) hoists deterministic cluster-aware rereference matrix construction into ezmsg.sigproc.util.rereference. Use it here: - Replace the local RereferenceInit enum with the imported RereferenceKind (re-exported from this module; same "identity"/"car" config values). - Delete _car_effective_matrix: the cold-start branch in _process is now a single rereference_matrix(...) call. The matrix is built as host-side numpy; AffineTransformTransformer converts it to the message's namespace/dtype/device on first use, so the on-device selection-matrix scatter is no longer needed. - Use the shared validate_channel_clusters() from util.channels for cluster bounds checking (empty-list policy stays here). - Bump ezmsg-sigproc floor to 2.34.0 (and test dep ezmsg-simbiophys to 1.8.0). Behavior is unchanged for float streams; integer-dtype streams now get float64 output from the cold-start transform (weights stay float64 instead of being cast to the message's integer dtype). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
ezmsg.sigproc.util.rereferencemodule:RereferenceKindstr-enum (identity|car) — round-trips through config as plain strings.car_matrix(n_channels, *, clusters=None, include_current=True, min_reref_size=1, dtype=float64)— per-cluster common-average reference as a symmetric weight matrix, supporting standard and leave-one-out CAR; clusters belowmin_reref_sizestay identity (named distinctly fromAffineTransformSettings.min_cluster_size, the block-diagonal matmul merge threshold).rereference_matrix(kind, ...)dispatcher.AffineTransformTransformeralready converts weights to the message's array namespace/dtype/device on first use, so this stays Array-API/GPU friendly without any device-side construction code.AffineTransformSettings.weightsnow also accepts aRereferenceKindor its string value (e.g.weights: carin config), built overchannel_clustersat reset — the same cluster spec drives both matrix construction and the block-diagonal matmul optimization.weights=Noneremains the zero-cost passthrough.CommonRereferencein the same module (per-sample mean subtraction,mediansupport); kind-based weights exist for discoverability of deterministic transforms and for workflows that start from such a matrix and later replace it viaset_weights().validate_channel_clusters()intoutil.channels, replacing duplicated cluster bounds checks (also to be consumed by ezmsg-learn's SSR module).Motivation
ezmsg-learn PR ezmsg-org/ezmsg-learn#19 adds a CAR cold-start default to LRR; this PR hoists the deterministic-rereference matrix construction into ezmsg-sigproc so LRR (and any affine-transform user) can share one implementation. Once released, the learn PR reduces to a single
rereference_matrix(...)call.Notes
caroridentity(bare relative path, no extension) is now interpreted as a kind rather than a path — same precedent as the existing"passthrough"special value.Testing
tests/unit/test_rereference.py: 19 tests covering both CAR flavors, per-cluster independence, small/singleton-cluster guards, bipolar pair case, symmetry, bounds validation, string dispatch, and kind-based weights throughAffineTransformTransformer(incl. rebuild on channel-count change).tests/unit/test_util_channels.py.🤖 Generated with Claude Code