Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions SPEC.md
Original file line number Diff line number Diff line change
Expand Up @@ -111,6 +111,8 @@ V19nd: ∀ Dependabot ecosystem entry in `.github/dependabot.yml` → weekly Mon
V20qx: ∀ ignored Dependabot dependency → ignore reason names active compatibility constraint ∧ references tracked resolution task; `itis-dakota` remains ignored until T16mo resolves the Dakota 6.23+ interface-cache regression
V44ls: ∀ scale-consuming `itis_sumo.api` workflow (surrogate fit ∧ UQ/Sobol input sampling ∧ MOGA search domain ∧ CV accuracy metric) → a `scale="log"` override ! be honoured in that workflow's own computation (log-space surrogate + log-space sampling/search where a distribution or domain is involved, positivity-guarded); ⊥ a `scale` override accepted but silently ignored outside the surrogate fit
V45ls: ∀ value-producing `itis_sumo.api` entry point → `scale` ! be threaded into the computation that yields those values ∧ observable in the result; ⊥ a value the api produced while ignoring the caller's scale; enforced: structural (internal value-producers `scale_distribution`/`resolve_log_scale` take scale/flag as a required arg — unwired code breaks at call w/ `TypeError`, never silent drift) ∧ behavioural (linear↔log flip-test over ∀ entry points (10 table-mode workflows + `evaluate_correlations`); rank-correlation exempt — monotone-invariant, asserted unchanged)
V46rn: log scale ! compose w/ ∀ uncertainty distribution it is defined on: `uniform` → log-uniform draw ∧ `normal` → μ/σ parameterize the ln-space distribution (raw draw = `exp(N(μ,σ))`, lognormal in original units, positive by construction; the surrogate sees exactly `N(μ,σ)` in its training space — same "distribution describes what the model sees" contract as log-uniform); ⊥ a sampler path rejecting log⊗normal or drawing it linear; `constant` ⊥ composes w/ log (rejection stays)
V47st: scale/spec surface input guards ! fail loud at the boundary, never mid-engine: `preprocessing` override maps ! be exact-cover at ∀ entry points (⊥ unknown-column override silently ignored — session path's rule extends to `compute_correlations`/`generate_lhs_samples`/`generate_grid_samples`); required scale maps ! be indexed by required key (⊥ per-variable `.get(..., "linear")` silent default); log-uniform bounds ! validated present ∧ ordered at the api boundary (→ `SumoInputError`, not `_run_engine`'s `SumoEngineError`); ∀ bounded spec (`DomainSpec`/`DistributionSpec`) ! reject `minimum ≥ maximum` at construction

## §R
R1: `export_model`/`import_model` child keywords; formats `text_archive`(.sps)/`binary_archive`(.bsps)/`algebraic_file`(.alg); naming `{prefix}.{resp}.{ext}` | branch R2
Expand Down Expand Up @@ -172,6 +174,9 @@ T42jn|.|fix ∀ ~72 pre-existing `ty check` diagnostics (dict[str,float] narrowi
T43bv|✓|`scripts/bump_alpha_version.py` (mirrors `dev_version.py`'s tomllib/regex/`--write` shape) + `.github/workflows/auto-bump-alpha-version.yml`: on a PR into `develop`, auto-bump the `pyproject.toml` `aN` counter past every colliding `v*` tag and push the fix commit to the PR's OWN head branch (⊥ `develop`/`main` directly, so V29yn's no-bot-commit constraint doesn't apply); no-op when the current version isn't alpha or a base bump is needed instead — `version-check.yml` (V28tz) stays the backstop for those and for forked PRs (read-only token there)|V42bv
T44qm|.|POST-T43bv `?`: once auto-bump-alpha-version confirmed working end-to-end on a real PR, study consolidating `.github/workflows/*.yml` (`ci.yml`, `version-check.yml`, `auto-bump-alpha-version.yml`, `auto-tag.yml`, `publish.yml`, `docs-deploy.yml`) — repeated checkout/setup-uv/tag-fetch boilerplate across files; open question whether to merge workflows, factor a composite action, or leave as-is; explicit design discussion required before touching, not a mechanical refactor|T43bv
T45zt|✓|verify `../mmux_vite` flaskapi suite (previously 522 passed/3 skipped, T15mn) still passes pinned at itis-sumo `0.1.0a2` + itis-dakota `1.5.11` (T16mo rung 1) — rung 1 is packaging-only/behavior-identical per T16mo, but T15mn's pin predates both the alpha-2 bump and the engine bump, so this closes the loop before retargeting flaskapi's pin off `.devN`; confirmed via editable overlay (`uv run --with-editable`), identical 522 passed/3 skipped|T15mn,T16mo
T48kw|x|log ⊗ normal (V46rn): `resolve_log_scale` ∧ `_uq_engine_distributions` accept `normal` (bound checks stay uniform-only; `constant` rejection stays); `create_manual_uq_samples` normal branch exp-restores under `log_scale` (replaces the dead commented-out lognormal branch); Sobol ppf → frozen `lognorm(s=σ, scale=e^μ)`; the 3 rejection tests flip to behaviour tests (draws >0, `ln(draws)` ~ N(μ,σ), UQ/Sobol/correlation stats move linear↔log); flip matrix asserts a log⊗normal column moves ∀ sampling entry points that take `distributions`; V&V I4/I9 wording amended|V46rn,V45ls,B19ps
T49dr|x|scale-boundary guards (V47st): `compute_correlation_indices` indexes `input_scales[var]` by required key (membership error next to the existing "not found in input samples" check — docstring "REQUIRED, never defaulted" stops overstating); `compute_correlations`/`generate_lhs_samples`/`generate_grid_samples` reject `preprocessing` overrides for columns not in play (mirrors `_session.py` exact-cover); `_uq_engine_distributions` validates log-uniform `maximum` present ∧ `maximum > minimum` → `SumoInputError` not `_run_engine`'s `SumoEngineError`; MOGA docstring corrected (log objective *fits* in ln-space, *reports* exp-restored original units)|V47st,V45ls,V23er,B19ps
T50vb|x|`DomainSpec`/`DistributionSpec` ! reject `minimum ≥ maximum` at construction (`__post_init__`, taxonomy-consistent error at first touch); kills the silently-descending inverted log grid axis ∧ turns MOGA's inverted-domain `SumoEngineError` into an input error; tests incl. inverted log domain|V47st,V23er

## §B
id|date|cause|fix
Expand All @@ -197,3 +202,4 @@ B16mo|2026-08-28|T16mo rung 2 bumped the engine `itis-dakota==1.5.11` (Dakota 6.
B8hm|2026-09-11|Dependabot entries used unspecified weekly timing, and `itis-dakota` was intentionally ignored without a durable policy invariant|V19nd,V20qx
B17sc|2026-09-28|`scale="log"` on `PreprocessingSpec.overrides` reached the surrogate fit but was a near-no-op for UQ/MOGA/cv-metrics/Sobol — the logscale port (T27fr) landed the preprocessor transform but not per-workflow log-space sampling/search, so `evaluate_uncertainty` with a log input returned linear-space statistics (probe: mean 12.057 linear vs 12.058 log). Symptom never seen in flaskapi, whose frontend drove log at the request-payload layer|wire log-space into every scale-consuming path: `create_manual_uq_samples` log_scale branch (log-uniform, uniform-only + positivity), `_uq_engine_distributions` flags ∧ validates log vars for UQ + Sobol, `evaluate_sobol_indices` samples `scipy.stats.loguniform`, `optimize_pareto_front` maps a log variable's search domain into ln space ∧ fits log objectives; `evaluate_cv_metrics` inherits log via `cross_validate`. Guarded by V44ls ∧ tests across the four workflows|V44ls,T27fr
B18mt|2026-09-29|T25dp's "correlation" port landed only table-mode `api.compute_correlations`, leaving the #470 MC→surrogate→correlate endpoint workflow (sample `distributions` ∧ predict ∧ correlate over the SHARED set) inside the flaskapi blueprint — surfaced while re-landing the mmux_vite consumption migration: either compute stays in flaskapi (⊥ §G) or the endpoint silently degrades to table correlation (the superseded branch #537 chose the latter, ignoring `distributions`/`num_samples`/`seed` unnoticed)|`api.evaluate_correlations` + `SumoSession.correlations` (log guards via existing `_uq_engine_distributions`) + engine `correlate_manual_uq_samples` w/ REQUIRED `input_scales`/`output_scale`; flip matrix extended over ∀ 11 entry points|V45ls,V22rs,T47mt
B19ps|2026-09-29|PR#47 review found shipped deliberate semantics wrong: `resolve_log_scale` + `_uq_engine_distributions` rejected log⊗normal ("source parity: log-scale sampling is uniform-only; no invented log-normal", from fc93cbe, codified in V&V I4 + 3 tests) — but the log knob must compose with a normal: μ/σ parameterizing ln-space (raw draw = lognormal) is the same "distribution describes what the model sees" contract the log-uniform branch already honors; 4 weak guards surfaced in the same review: `input_scales.get(var, "linear")` per-variable silent default behind a "REQUIRED, never defaulted" docstring; log-uniform upper bound unchecked at the boundary (missing/inverted `maximum` → `SumoEngineError` via `_run_engine`); `compute_correlations`/`generate_lhs_samples`/`generate_grid_samples` silently ignore `preprocessing` overrides for unknown columns while the session path rejects them; MOGA docstring claimed the front is reported in log space vs the exp-restored reality|new §V invariants V46rn (log⊗distribution composition) ∧ V47st (boundary input guards); tasks T48kw/T49dr/T50vb|V46rn,V47st,T48kw,T49dr,T50vb
4 changes: 2 additions & 2 deletions docs/TIER1_TIER2_UNIT_TESTS_PLAN.md
Original file line number Diff line number Diff line change
Expand Up @@ -132,9 +132,9 @@ The `unit→value` map shared by every value producer (SPEC V44ls/V45ls).
- scales required (tripwire test); log-reparametrized input shifts Pearson, leaves Spearman bit-identical

**`correlate_manual_uq_samples(...)` → `api.evaluate_correlations`**
- MC→surrogate→correlate workflow (#470) with REQUIRED `input_scales`/`output_scale` (tripwire test); distributions must cover variables exactly; log variable requires uniform ∧ strictly-positive lower bound
- MC→surrogate→correlate workflow (#470) with REQUIRED `input_scales`/`output_scale` (tripwire test); distributions must cover variables exactly; log variable requires a uniform (strictly-positive lower bound) or a normal (ln-space μ/σ, V46rn)

**`create_manual_uq_samples`** — `log_scale` uniform drawn log-uniform in original units; log+normal and log+min≤0 refused
**`create_manual_uq_samples`** — `log_scale` uniform drawn log-uniform in original units; `log_scale` normal draws `exp(N(μ,σ))` (ln-space μ/σ, V46rn); log+constant and log+min≤0 refused

**`DataPreprocessor` log transform** — `setup_log_transform`/fit/transform/inverse round-trip; delta-method `inverse_transform_output_std` (`tests/test_data_preprocessor.py::TestLogTransform`)

Expand Down
9 changes: 5 additions & 4 deletions docs/verification-validation.md
Original file line number Diff line number Diff line change
Expand Up @@ -147,12 +147,13 @@ breaks with `TypeError`, it can never silently default.
| I1 | LHS + grid samplers: log domain ⇒ log-uniform/geometric fill; linear default bit-identical; non-positive log domain ⇒ `SumoInputError` | ✅ |
| I2 | Correlations: Pearson moves under log, Spearman provably unchanged (monotone-invariant), untouched columns bit-identical; correlator's scale args are required (`TypeError` tripwire) | ✅ |
| I3 | Surrogate / CV / along-axes / grid eval: log response exp-restored to original units; non-positive training outputs rejected pre-Dakota | ✅ |
| I4 | UQ propagation: log inputs drawn log-uniform (response skews low vs linear, directionally asserted); log+normal / log+min≤0 rejected; log response ⇒ multiplicative (not additive) spread | ✅ |
| I4 | UQ propagation: log inputs drawn log-uniform (response skews low vs linear, directionally asserted); log⊗normal drawn lognormal — ln-space μ/σ, mean shifts high vs linear by Jensen, directionally asserted (V46rn); log+min≤0 / log+constant rejected; log response ⇒ multiplicative (not additive) spread | ✅ |
| I5 | CV accuracy metrics: inherit log through `cross_validate` (metrics differ from linear); reject non-positive log responses | ✅ |
| I6 | MOGA: log variable explored in ln-space (domain mapped, positivity-guarded); log objective exp-restored for **both** minimize and maximize (sign-after-log inverse order verified) | ✅ |
| I7 | Sobol: log input shifts the variance decomposition in the expected direction (compressed variable explains less); mixed log+constant partition | ✅ |
| I8 | Flip matrix: all 11 public value-producing entry points' outputs move when a column turns log — the V45ls machine guard against any silent scale-ignore, shipped or future | ✅ |
| I9 | MC-through-surrogate correlation (`evaluate_correlations`, #470 workflow): dominant variable recovered over the shared sample set, seed-reproducible, log-scale coefficients move, log+non-uniform / log+non-positive / non-covering distributions rejected, engine producer requires its scales (`TypeError` tripwire) | ✅ |
| I7 | Sobol: log input shifts the variance decomposition in the expected direction (compressed variable explains less; a log⊗normal widens the lognormal tail so it explains more, V46rn); mixed log+constant partition | ✅ |
| I8 | Flip matrix: all 11 public value-producing entry points' outputs move when a column turns log, ∀ 3 `distributions`-taking entry points also move when a NORMAL column turns log (V46rn) — the V45ls machine guard against any silent scale-ignore, shipped or future | ✅ |
| I9 | MC-through-surrogate correlation (`evaluate_correlations`, #470 workflow): dominant variable recovered over the shared sample set, seed-reproducible, log-scale coefficients move (uniform ∧ normal, V46rn), log+constant / log+non-positive / non-covering distributions rejected, engine producer requires its scales (`TypeError` tripwire) | ✅ |
| I10 | Boundary input guards (V47st): `DomainSpec`/`DistributionSpec` reject inverted/degenerate bounds at construction; unused `preprocessing` overrides rejected by the table-mode + sampler entry points (not just the session path); log-uniform upper bound required ∧ ordered at the api boundary; a variable missing from the correlator's `input_scales` raises instead of defaulting to linear | ✅ |

Tests: `tests/test_api_workflows.py` (`TestLogScale*`, `TestScaleGapCoverage`,
`TestScaleAwareSamplers`, `TestScaleFlipMatrix`),
Expand Down
2 changes: 1 addition & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ build-backend = "uv_build"

[project]
name = "itis-sumo"
version = "0.1.0a6"
version = "0.1.0a7"
description = "Surrogate Modeling functionality for IT'IS Foundation / ZMT Modeling Intelligence suite: build, evaluate, cross-validate surrogates + UQ + sampling, headless or embedded"
readme = "README.md"
# T16mo rung 1 (1.5.9->1.5.11): 1.5.11 ships cp313 wheels, so the <3.13
Expand Down
34 changes: 23 additions & 11 deletions src/itis_sumo/api/_session.py
Original file line number Diff line number Diff line change
Expand Up @@ -555,24 +555,36 @@ def _uq_engine_distributions(
) -> dict[str, dict[str, float | str]]:
"""Translate distributions for the sampler, flagging log-scale variables.

A log-scale variable is sampled uniformly in log space (the surrogate
preprocessor re-applies the log). Sampling in log space is only defined for
a strictly-positive uniform, so anything else is rejected here -- at the
API boundary -- rather than surfacing as the sampler's raw ``ValueError``.
A log-scale variable is sampled in the space the surrogate trains on: a
uniform is drawn log-uniform and a normal keeps its μ/σ in ln space, so
the raw draws are lognormal (V46rn); the surrogate preprocessor
re-applies the log either way. A log-scale uniform needs a strictly-
positive lower bound; a log-scale normal is positive by construction and
needs no bounds. Anything else (e.g. ``constant``) is rejected here -- at
the API boundary -- rather than surfacing as the sampler's raw
``ValueError``.
"""
engine: dict[str, dict[str, float | str]] = {}
for variable, spec in distributions.items():
entry = spec.as_engine_dict()
if self._scale_of(variable) == "log":
if spec.distribution != "uniform":
if spec.distribution == "uniform":
if spec.minimum is None or spec.minimum <= 0:
raise SumoInputError(
f"'{variable}' is log-scale but its distribution lower "
"bound is not strictly positive"
)
if spec.maximum is None or spec.maximum <= spec.minimum:
raise SumoInputError(
f"'{variable}' is log-scale but its distribution upper "
f"bound is missing or not above its lower bound "
f"({spec.maximum!r} <= {spec.minimum!r})"
)
elif spec.distribution != "normal":
raise SumoInputError(
f"'{variable}' is log-scale but its distribution is a "
f"'{spec.distribution}'; only a uniform supports log sampling"
)
if spec.minimum is None or spec.minimum <= 0:
raise SumoInputError(
f"'{variable}' is log-scale but its distribution lower "
"bound is not strictly positive"
f"'{spec.distribution}'; only a uniform or normal "
"supports log sampling"
)
entry["log_scale"] = True
engine[variable] = entry
Expand Down
24 changes: 24 additions & 0 deletions src/itis_sumo/api/types.py
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,8 @@
from dataclasses import dataclass, field
from typing import Literal

from itis_sumo.api.errors import SumoInputError

#: Seed used by every stochastic entrypoint unless the caller overrides it.
#: Fixed rather than required, so that results are reproducible by default
#: without the caller having to think about it (SPEC V25sd).
Expand Down Expand Up @@ -56,6 +58,19 @@ def as_engine_dict(self) -> dict[str, float | str]:
values["max"] = self.maximum
return values

def __post_init__(self) -> None:
# V47st: a declared interval must be increasing; catching it at
# construction beats an inverted axis surfacing mid-engine later.
if (
self.minimum is not None
and self.maximum is not None
and self.maximum <= self.minimum
):
raise SumoInputError(
f"DistributionSpec bounds must increase: maximum "
f"({self.maximum}) <= minimum ({self.minimum})"
)


Direction = Literal["minimize", "maximize"]

Expand All @@ -72,6 +87,15 @@ class DomainSpec:
minimum: float
maximum: float

def __post_init__(self) -> None:
# V47st: an inverted (or degenerate) box is always a caller mistake --
# a silently descending grid axis or a mid-engine MOGA failure.
if self.maximum <= self.minimum:
raise SumoInputError(
f"DomainSpec bounds must increase: maximum "
f"({self.maximum}) <= minimum ({self.minimum})"
)

def as_engine_dict(self) -> dict[str, float | str]:
return {"distribution": "uniform", "min": self.minimum, "max": self.maximum}

Expand Down
Loading
Loading