Skip to content

Latest commit

 

History

History
486 lines (449 loc) · 36 KB

saturation.md

File metadata and controls

486 lines (449 loc) · 36 KB

Saturation DML

Saturation DML is the optional Phase 0 of doubleml-pipeline: it estimates a diminishing-returns curve f(T; alpha) for each treatment and hands the transformed treatment to the DML phases, so that one unit of media spend is not assumed to buy the same incremental outcome at every spend level. Enable it by passing a saturation_config with a saturation block to CausalWorkflowOrchestrator; the minimal example is in the README quick start, and Double Machine Learning in 10 minutes explains the vocabulary used below.

Tip

If your treatment is a share such as a discount rate rather than a spend amount, Phases 0–3 still use the saturation curve, but the saturation-aware optimizer of Phase 4 (which budgets the treatment column directly) is only available for treatment_types = "spend".

By default the pipeline is unchanged: omit the saturation_config argument and Phases 1–4 behave exactly as before. Setting saturation_config['saturation']['family'] enables Phase 0, which estimates a diminishing-returns parameter alpha for each treatment and then feeds the transformed treatment f(T; alpha) into the existing DML pipeline.

Response families

family f(T; alpha) Interpretation of alpha
"tanh" (default) tanh(alpha * T / 2) Rate; T_1/2 = ln 3 / alpha is the half-saturation point.
"exp" 1 - exp(-alpha * T) Rate; 1/alpha is the saturation scale.
"log" log(1 + T / alpha) Saturation scale (smaller = returns diminish sooner).
"power" T ** alpha Elasticity; must satisfy 0 < alpha < 1.

Configuration

Every saturation setting lives in a single saturation_config dict, and the presence of the saturation block is all that is required (family defaults to "tanh") — everything else defaults. Turning it on is therefore a one-key change:

from doubleml_pipeline.workflow import CausalWorkflowOrchestrator

orchestrator = CausalWorkflowOrchestrator(
    # ... all existing arguments ...
    optimize=True,
    saturation_config={"saturation": {"family": "power"}},
)

simple_optimization is derived: it becomes False automatically when a family is set, and True otherwise. Pass it explicitly to override.

The full set of keys, all shown at their defaults:

from doubleml_pipeline.workflow import CausalWorkflowOrchestrator

orchestrator = CausalWorkflowOrchestrator(
    # ... all existing arguments ...
    optimize=True,
    saturation_config={
        "saturation": {
            # "tanh" (default) | "exp" | "log" | "power"
            "family": "tanh",
            # alpha grid searched in Phase 0
            "grid": {
                "mode": "auto",  # "auto" | "manual"
                "num": 25,
                # auto: T_1/2 in [c_min*q50, c_max*q95]
                "c_min": 0.25,
                "c_max": 4.0,
                # auto: x4 extension at an edge argmin
                "max_extensions": 2,
                "min": None,  # manual only (alpha units)
                "max": None,  # manual only (alpha units)
                "scale": "log",  # "log" | "linear" (manual spacing)
            },
            # "auto" | int (B-spline T-basis size J)
            "basis_dim_T": "auto",
            # auto: rel. RMS approximation error
            "basis_tol": 1e-3,
            # auto: search range for J
            "basis_dim_range": [6, 20],
            # ridge penalty in the closed form (absolute)
            "ridge": 1e-4,
            # T_ref = q95(T>0): all candidates pass through (T_ref, 1)
            "t_ref_quantile": 0.95,
        },
        "alpha_selection": {
            "sel_fraction": 0.20,  # D_sel : D_inf split
            "n_seeds": 10,  # S: repetitions of R-loss profiling
            "k_folds": 5,  # K for the alpha-selection stage
            "refine": "brent",  # "brent" | "none" (grid + Brent only)
            # adopt saturation iff d_bar < -crit * se
            # (se = two-way cluster-robust se_obs)
            "linear_test_k": 1.0,
            # crit = k (False) | Student-t value t_{G-1}^{-1}(Phi(k)) (True)
            "linear_test_t_critical": False,
            # half-saturation point (units of T) used when Diag1/Diag2 fires
            "fallback_half_saturation_T": None,
            # Diag2: fires when max/min T_1/2 over the 1-SE set exceeds this
            "flatness_ratio": 10.0,
            # Diag6: rel. error tolerated between Phase 0 and Phase 1 T_ref
            "t_ref_tolerance": 1.0e-9,
            # Stage B nuisance learners (fixed)
            "learners": {
                # True: legacy FLAML candidate search
                "learner_selection": False,
                "y_model": {
                    "class": "xgboost.XGBRegressor",
                    "params": {
                        "max_depth": 5,
                        "learning_rate": 0.05,
                        "n_estimators": 500,
                        "min_child_weight": 10,
                        "subsample": 0.8,
                        "colsample_bytree": 0.8,
                        "reg_lambda": 5,
                    },
                    "early_stopping_rounds": 30,
                },
                # Same tree parameters as `y_model`, plus `multi_strategy`.
                "t_model": {
                    "class": "xgboost.XGBRegressor",
                    "params": {
                        "multi_strategy": "multi_output_tree",
                        "max_depth": 5,
                        "learning_rate": 0.05,
                        "n_estimators": 500,
                        "min_child_weight": 10,
                        "subsample": 0.8,
                        "colsample_bytree": 0.8,
                        "reg_lambda": 5,
                    },
                    "early_stopping_rounds": 30,
                },
                "small_sample_threshold": 2000,  # n_sel below -> RidgeCV
                "small_sample_model": "ridge_cv",
            },
            "output_dir": "step0_saturation",  # under output_dir
            # "time_budget": 30,  # learner_selection only
        },
        "final_fit": {"data": "d_inf"},  # "d_inf" (default) | "full_sample"
        # Saturation-aware optimization. Note this lives
        # under `optimization`, not `saturation`.
        "optimization": {
            # "fixed" (budget K) | "roi_target" (no budget; ROI >= r)
            "budget_mode": "fixed",
            # fixed only: K = (1 + x) * actual total of the target units
            "budget_increment": 0.0,
            # fixed only: redistribute what the ROI cap / negative units
            # free up
            "budget_reallocate": True,
            # support cap T_bar_t = max(Q_q(cell, T>0), T_actual);
            # 1.0 = cell maximum
            "cap_quantile": 1.0,
            # "all" (one KKT problem; spend moves across days)
            # | "date" (one per date)
            "reallocate_within": "all",
        },
    },
    # top-level optimizer scope (None = all rows of the column).
    opt_geos=None,
    opt_items=None,
    # r >= 0 (default 1.0 across all optimizer paths);
    # < 0 raises at initialization
    opt_threshold_roi=1.0,
)

The keys in detail:

Key Default Meaning
saturation.family "tanh" Used when the saturation block is present but family is omitted. family: None in the block still means a linear run.
saturation.grid.mode "auto" auto builds a dimensionless half-saturation grid T_1/2 in [c_min*q50(T>0), c_max*q95(T>0)], log-spaced, for tanh/exp/log; the auto grid for power is linear in alpha on [0.1, 0.95]. manual uses min/max/scale in alpha units; the alphas follow scale (default "log") for tanh/exp/log, while a power grid is always linear in alpha. A legacy grid with min/max but no mode is treated as manual.
saturation.grid.c_min, c_max 0.25, 4.0 Auto-grid range multipliers.
saturation.grid.max_extensions 2 When the argmin is at an auto-grid edge, extend that side x4 in T_1/2 (matrix algebra only, no ML refits).
saturation.basis_dim_T "auto" auto scans J in basis_dim_range and picks the smallest J whose worst-case relative RMS error of projecting fbar(.;alpha) is below basis_tol. The projection is a least-squares fit (SVD pseudo-inverse, never B'B) on a fixed grid over [0, T_max] made of 500 equally spaced points, 201 quantiles of T>0 and the Greville abscissae of the basis, so the basis matrix B has full column rank however skewed T is; T_max is the 99.9 % quantile of T>0 on the full fitting data (rows above it are clipped by the basis only, never by Phases 1-4). Bases with condition number cond(B) >= 1e8 are skipped; if no J meets basis_tol, the J with the smallest error is used (Diag1 warns). An integer pins J. A pinned or only-available basis with cond(B) >= 1e8 makes Stage B adopt the linear model (decision_reason = "basis_ill_conditioned") instead of trusting it.
saturation.basis_tol, basis_dim_range 1e-3, [6, 20] See above.
saturation.t_ref_quantile 0.95 Quantile of T>0, computed once on the full fitting data before the split, that defines T_ref. Every candidate is normalized to fbar(T; alpha) = f(T; alpha) / f(T_ref; alpha), so all curves pass through (T_ref, 1) and the identity candidate is fbar_T = T / T_ref. This removes the alpha-dependent magnitude of f that let the ridge solve shrink near-identity candidates (large T_1/2) to zero. Always on.
alpha_selection.linear_test_k 1.0 Saturation is adopted only if d_bar < -crit*se, where d_s = L_s(alpha_hat) - L_s,T is the paired difference against the identity candidate fbar_T = T / T_ref and se is the cluster-robust se_obs of the per-observation differences d_i. With two adopted cluster keys (e.g. geo × date block) the se is two-way (Cameron–Gelbach–Miller: V = V_key1 + V_key2 - V_cell), which covers dependence along both keys; with a single key it is one-way on that key (there is no across-seed se). The one-way se on the fewer-cluster key is always reported next to it (se_one_way), and the decision line notes when it would have decided differently. crit = k by default (see the next row). Otherwise selected_family = None (linear). The adoption rate is tuned with k alone: k = 1.0 is the 1-SE rule, 2.0 a conservative choice, smaller values adopt saturation more readily. Independently of this test, an alpha_hat still on the upper grid edge after max_extensions extensions forces selected_family = None with decision_reason = "grid_edge_upper" (the data cannot distinguish the curve from a line); a lower-edge hit only warns.
alpha_selection.linear_test_t_critical False True replaces crit = k by the Student-t value with the same one-sided tail probability, crit = t_{G-1}^{-1}(Phi(k)), G = number of clusters (the smaller of the two counts for the two-way se). It compensates the noisy se of few clusters (k = 1: G = 30 -> 1.02, 10 -> 1.06, 4 -> 1.20) and can only make linear decisions more frequent; it is not applied to the Diag2 1-SE set. Off by default.
alpha_selection.fallback_half_saturation_T None A fixed half-saturation point in units of T (T_knee for log, alpha itself for power; same scale as half_saturation_T / T_ref in summary_alpha_selection.json, i.e. T / c). When Diag1 or Diag2 fires it replaces the estimate downstream and keeps the family; otherwise the 1-SE decision stands.
alpha_selection.flatness_ratio 10.0 Diag2 fires when T_1/2^max / T_1/2^min over the 1-SE set A_1SE = {alpha : L_bar(alpha) - L_bar(alpha_hat) <= se(alpha)} exceeds this (T_knee for log; for power the fixed rule alpha^max - alpha^min > 0.4). Warning only when a family is adopted, INFO otherwise.
alpha_selection.t_ref_tolerance 1e-9 Diag6: at the start of Phase 1 T_ref is recomputed from the treatment column Phase 1 receives; a relative error at or above this stops the run with an error (unit / hand-over bug).
optimization.budget_mode "fixed" fixed: a budget K is given and the allocation is solved in one KKT pass. roi_target: no budget; every unit receives min(T_bar_t, T_t^r) (requires opt_threshold_roi > 0).
optimization.budget_reallocate True fixed only. True (default): K' = K, solve once with the effective caps min(T_bar_t, T_t^r) (spend freed by negative-ROI units and the ROI cap is redistributed). False: K' = K_+ (negative-ROI spend removed), solve with the caps T_bar_t, then cut to T_t^r without redistribution (unspent_budget).
optimization.cap_quantile 1.0 q of the support cap T_bar_t = max(Q_q(cell_t), T_t^actual): Q_q is the q-quantile of the positive actual treatment within the unit's geo × item cell (consistent with T_ref = q95(T>0); zero-spend rows do not shift it), floored at the unit's own actual spend. 1.0 (default) is the cell maximum — today's behaviour, where one outlier day lifts the cap of every unit in its cell. With e.g. 0.95 the other units are capped at the 95 % point while the outlier day keeps its own spend as its cap: it is never increased, but may still be reduced by the KKT condition or the ROI cap like any unit. Must be in (0, 1].
optimization.reallocate_within "all" Scope of the KKT problem. "all": one problem over every allocation unit (spend moves across days, geos and items; today's behaviour). "date": one problem per date with the per-date budget K_d = (1 + budget_increment) * sum_{t in d} T_t^actual — no cross-day movement; mu* becomes per date (median with marginal_roi_at_optimum_min / _max in optimization_metrics.csv). See the no-carryover note under "Saturation-aware optimization".
opt_threshold_roi (top level) 1.0 ROI floor r >= 0, applied per row across both simple_optimization=True and False (including budget_mode='roi_target').
opt_geos, opt_items (top level) None Restrict the allocation units of the saturation-aware optimizer to these geos / items (None = all rows of the column). Rows outside keep their actual values in the optimized_* columns.
alpha_selection.learners see above Fixed Stage B learners. learner_selection: True restores the legacy FLAML search. Without XGBoost >= 2.0 a warning is emitted and scikit-learn stand-ins are used: HistGradientBoostingRegressor for Y and one native multi-output RandomForestRegressor for T.
alpha_selection.output_dir "step0_saturation" Sub-directory of output_dir that receives the Phase 0 files.

Note

Linear fallback. If Phase 0 selects selected_family = None (decision_reason is 1se_fail, grid_edge_upper or basis_ill_conditioned), then Phases 1–3 use the raw treatment, and Phase 4 runs the existing optimization (as with simple_optimization=True). This happens even with optimize=True and simple_optimization=False. The pipeline warns saturation was not identified; falling back to linear optimization. The config value itself is not rewritten. summary_alpha_selection.json, metadata.json and the HTML report record phase4_optimization = "linear_fallback".

Warning

The saturation=, alpha_selection=, final_fit= and budget_increment= keyword arguments have been removed; saturation_config is now the only way to set them. It is also strictly validated — an unknown block or key raises ValueError rather than being ignored, so a typo cannot silently disable Phase 0.

What Phase 0 adds

  1. Top-level split into D_sel (alpha selection) and D_inf (estimation). With two cluster keys, the split unit is their intersection cell (e.g. geo × date block). No cell straddles the two halves, and every level of each key appears in both halves and, when it has at least K cells in D_sel, in every Stage B fold (others_split_coverage.csv).
  2. Alpha selection by minimizing the profiled R-loss over the half-saturation grid, with optional Brent refinement. The same CATE basis p(X) is used as in Phases 1–3. Every nuisance prediction is out-of-fold (cross-fitting: each (seed, fold) model predicts only its held-out rows) and the inner CATE coefficients of fold k are fitted on the out-of-fold residuals of the other folds, so no in-sample residual enters the loss. The intercept-only curve is drawn for visualization only.
  3. Identity-candidate comparison. All candidates are normalized through the reference point (T_ref, 1), T_ref = q95(T>0): fbar(T; alpha) = f(T; alpha) / f(T_ref; alpha). The saturated fit must beat the identity candidate fbar_T = T / T_ref by more than crit = linear_test_k cluster-robust standard errors (two-way on the two adopted keys when both are known; optionally a Student-t critical value with linear_test_t_critical). Otherwise selected_family = None. An alpha_hat left on the upper grid edge after all extensions also forces None (decision_reason = "grid_edge_upper"). 3b. Diagnostics Diag1–Diag6. Diag1: basis approximation error eps(J) on the final grid (warn if >= basis_tol; J is fixed after the initial grid) and basis conditioning cond(B) (warn if >= 1e8; the basis is then not trusted and the linear model is adopted, decision_reason = "basis_ill_conditioned"); the single Diag1 line also shows T_max (99.9 % quantile of T>0) and the number of D_sel rows the basis clips at it. Diag2: 1-SE flatness, the ratio of the largest to the smallest half-saturation point in the 1-SE set (warn if > flatness_ratio and a family is adopted; INFO otherwise). Diag3: r_T = sd(T~)/sd(T/T_ref) and Diag4: the shape quantiles fbar(q_p(T>0); alpha_hat) with their linear references q_p / T_ref (supporting values, no rule). Diag5: kappa(alpha_hat) = f(T_ref; alpha_hat) for tanh / exp (no rule). Diag6: unit consistency of T_ref at the start of Phase 1 (error stop). Diag1 / Diag2 warn and continue; the fixed fallback_half_saturation_T, if set, is used downstream when they fire.
  4. If saturation is adopted, alpha is treated as a constant and Phases 1–3 run unmodified on fbar(T; alpha) (so the estimated theta is the effect of one T_ref worth of treatment). Otherwise they run on the raw treatment, and Phase 4 falls back to the linear optimizer.
  5. Every row is scored. With the default final_fit.data = "d_inf", Phases 1–3 are fitted on D_inf only, but the fitted nuisance models and the CATE function are then applied to the held-out D_sel rows as well. Everything under step4_consolidated_results/ — the consolidated CSV, the CATE / incremental-KPI / ROI summaries, the charts and the Phase 4 optimization — therefore covers all rows, not just D_inf. The consolidated CSV carries an estimation_sample column (D_inf = fitted row / D_sel = scored row), the 1c metrics report number_of_fitted_rows and number_of_scored_rows, and the log states e.g. Consolidated frame covers all 180 rows: 142 fitted (D_inf) + 38 scored with the fitted CATE (D_sel). The OVB sensitivity analysis keeps using the fitted rows only.

Important

A saturated treatment must use a divisive normalization — median_normalize or population_median_normalize — or none at all. standardize, population_standardize and min_max all shift the treatment, so T = 0 is no longer mapped to 0 and f(0) = 0 no longer means "no treatment": a negative value is evaluated as f(T) = −f(−T) (not clipped), so the curve would bend around the shifted origin. Phase 0 rejects those combinations with a ValueError before doing any ML work.

Note

alpha is estimated against the normalized treatment T / c, where median(T / c) == 1 by construction, which is what makes the default grid scale-free. The divisor c is spelled out wherever T / c appears: the Phase 0 log prints e.g. [spend] T/c = spend / 28.3788: the treatment divided by the median of its non-zero values (median_normalize; one constant for all rows, so T/c = 1 is the typical non-zero row).; with population_median_normalize it reads c_i = population_i x median(spend/population | spend/population > 0) = population_i x <const> (a per-row divisor), and without a divisive normalization c = 1 (raw units). The same formula is stored in summary_alpha_selection.json (normalization, divisor_constant, divisor_population_col, divisor_formula, divisor_description), shown in the HTML Phase 0 panel ("Normalized treatment scale (T/c)" / "Meaning of T/c"), and used as the x-axis label of the fitted-saturation-curve panel of summary_alpha_profile_<treatment>.png. T_ref is computed on the same normalized treatment, once on the full fitting data, and stored in summary_alpha_selection.json; Phases 1–3 and 4 read it from there. Phase 4 budgets in raw units; the two are reconciled exactly by fbar(T / c_i; alpha, T_ref) = fbar(T; alpha_i, c_i * T_ref) — the per-row gain_i of the un-normalized f cancels in the ratio, only T_ref is rescaled by the same c_i. As a consequence, alpha_hat from a normalized run is not comparable to one from an un-normalized run, whereas half_saturation_T / T_ref is.

Tip

With the default fixed learners, Phase 0 costs 2 * n_seeds * k_folds ML trainings per treatment on D_sel only (100 at the defaults). The treatment side is always a single multi-output model with J outputs. One unit is one fit call of a nuisance learner; the alpha grid, its extensions, the identity candidate and the basis dimension J are matrix algebra and are not counted. With learners.learner_selection: True the legacy FLAML search adds (N_y + N_t) learner-selection searches per treatment, and alpha_selection['time_budget'] defaults to the orchestrator's budget scaled by len(D_sel) / len(D).

Note

Phase 0 / Stage B display. After the Stage A block the console shows exactly eight items, in this order and nothing else: 1) Stage B: Number of ML trainings = <treatments> x 2 nuisance models x <S> x <K> = <total>; 2) one [Phase 0 Progress] elapsed … | done/total ML trainings completed | ETA … line per minute while the trainings run; then per treatment 3) the learners (XGBoost / RidgeCV, with n_sel and small_sample_threshold), 4) the grid (mode, T_1/2 range, points, T_ref, extensions, grid_edge_hit), 5) Diag1 (J, eps(J); WARNING when fired), 6) the grid-argmin alpha, the Brent-refined alpha_hat and L_bar(alpha_hat), 7) the Diag2 line (WARNING / INFO), Diag3 / Diag4 only when Diag2 fired, Diag5 for an adopted tanh / exp, and the 1-SE decision line (decision, decision_reason, d_bar, se, k), which says FALLBACK TO LINEAR DML explicitly when selected_family = None; 8) the completion message. Diag6 prints only when it fails ([ERROR], then the run stops) at the start of Phase 1. No internal step logs are printed; the detailed messages go to summary_alpha_selection.json (messages).

Important

Phase 4 always reports in raw treatment units. The transformed values used for fitting are preserved in the <treatment>_transformed column.

Warning

If Diag2 fires (a flat R-loss profile: the half-saturation point is not pinned down within one order of magnitude), the estimate of alpha is unreliable — see step0_saturation/diag2_rloss_profile.png and the Diag3 / Diag4 lines. In that case prefer fixing the half-saturation point from prior knowledge via fallback_half_saturation_T (used automatically when Diag1 or Diag2 fires) or a narrow manual grid around the prior value ({"mode": "manual", "min": a_lo, "max": a_hi}; num >= 2 and min < max are required).

Phase 0 outputs

All written to output_dir/step0_saturation/. With several treatments each one gets its own step0_saturation/<treatment>/ directory. Every file name carries a prefix that says what it belongs to: diag1_ … diag6_ for the six identification diagnostics, summary_ for the result of the alpha selection, others_ for supporting material without a diagnostic rule. Every PNG ends with a footnote that explains its annotations, states the rule / status of its diagnostic for this run and names the CSV columns behind it.

File Contents
summary_alpha_selection.json family, selected_family, alpha_hat, half_saturation_T (T_knee for log), alpha_grid_argmin, rloss_min (L_bar(alpha_hat)), refine_method, d_bar, se (cluster-robust se_obs), linear_test_k, decision, decision_reason (1se_pass / 1se_fail / grid_edge_upper), diag1_fired, J, basis_approx_error, diag2_fired, diag2_ratio, r_T, fbar_q25 … fbar_q90, linear_ref_q25 … linear_ref_q90, saturation_at_T_ref (null for log / power), grid_mode, grid_extensions, grid_edge_hit (none / upper / lower), T_ref, T_max, cluster_keys, cluster_fallback_tier, n_sel, n_inf, n_seeds, k_folds, learners, plus extras (for example phase4_optimization, diag6_status, flatness_ratio, basis_tol, fallback_half_saturation_T_used, t_ref_quantile, T_ref_raw, grid_variable, and the divisor of the normalized treatment: normalization, divisor_constant, divisor_population_col, divisor_formula, divisor_description). NaN is written as null. Phases 1–3 and Phase 4 read T_ref from this file.
summary_alpha_grid.csv One row per grid point: grid_index, half_saturation_T, alpha, mean_r_loss (L_bar(alpha)), se_alpha (the Diag2 se(alpha)), in_1se_set (membership in A_1SE), fbar_q25, fbar_q50, fbar_q75, fbar_q90 (phi_p(alpha) = fbar(q_p(T>0); alpha)), basis_approx_error_alpha (the Diag1 eps(J) per alpha). This is the complete data behind the R-loss profile charts (it replaces the former alpha_profile_<treatment>.csv).
summary_rloss_curves.csv seed, grid_index, alpha, rloss_cate, rloss_ate, rloss_identity.
summary_alpha_profile_<treatment>.png The one-page Phase 0 summary, 2 + 2 + 1 panels from top to bottom: Grid construction (histogram of T>0 with q50, q95 = T_ref, the automatic grid range and every grid point) and Seed-wise alpha distribution (per-seed alpha_hat_s histogram); Profiled R-loss curve with A_1SE and Fitted saturation curve (normalized through (T_ref, 1)) — fbar(T; alpha_hat), the identity candidate T / T_ref, (T_ref, 1), the shapes of the smallest / median / largest grid alpha and the curve Phases 1–4 actually use labelled FINAL; and the full-width Key stats & process metrics table whose diagnostics column tags each row with diag1 … diag6. The footnotes define T_ref, q05 / q50 / q95, the grid ends 0.25*q50 (fastest saturation) / 4*q95 (mildest, practically linear) and list every file of the folder.
summary_alpha_seed-wise_alpha_distribution.csv The data behind the seed-wise alpha distribution panel: seed, alpha_hat_s, rloss_at_alpha_hat_s, rloss_identity_s, d_s (visualization only, no rule).
summary_alpha_selection.csv, summary_phase0_diagnostics.json Run-level: one row per treatment (decision, alpha_hat, Diag1–Diag5 flags, compute accounting) and the Stage A split / cluster-key plan with every Phase 0 warning.
diag1_basis_approx_error.png Diag1: eps(J)(alpha) per grid point against basis_tol, with J, max eps(J) and the fired status in the title.
diag2_rloss_profile.png Diag2: R-loss vs T_1/2: CATE curve (±1 se), intercept curve (visualization only), identity candidate L_bar_T (±k se), the 1-SE set A_1SE, alpha_hat, decision; the footnote states the flatness ratio against flatness_ratio and whether Diag2 fired.
diag3_identification_diagnostics.csv Diag3: r_T, sd_T, sd_T_resid, share_T_zero, q50_T_pos, q95_T_pos, T_max, T_ref.
diag4_f_transform_quantiles.png Diag4: the four shape quantiles phi_25, phi_50, phi_75, phi_90 over alpha with the linear references q_p / T_ref.
diag5_saturation_at_T_ref.png Diag5 (tanh / exp only): kappa(alpha) = f(T_ref; alpha), the share of the ceiling reached at T_ref, over the grid with kappa(alpha_hat) marked.
diag6_t_ref_consistency.csv Diag6, written at the start of Phase 1: treatment, status (pass / fail / skipped when selected_family = None), T_ref_phase0, T_ref_phase1, rel_error, t_ref_tolerance, t_ref_quantile, n_rows_phase1, note.
others_split_coverage.csv, others_split_coverage_detail.csv, others_split_coverage.png Stage A / Stage B bookkeeping: per key level n_cells_sel, n_cells_inf, covered, n_folds_covered_min, fold_covered (the summary CSV, drawn as the bar chart of n_cells_sel / n_cells_inf per level with uncovered levels in red); and the per-cell detail seed, cluster_id, key1_value, key2_value, fold (-1 for D_inf), split (sel / inf).

The HTML report gains a 000_saturation tab, which embeds diag2_rloss_profile.png and summary_alpha_profile_<treatment>.png and shows decision, decision_reason, alpha_hat, half_saturation_T, T_ref, d_bar, se, diag2_ratio, saturation_at_T_ref and r_T.

Saturation-aware optimization

With optimize=True, simple_optimization=False, one treatment of type spend and selected_family != None, Phase 4 solves

max sum_t theta_t fbar(T_t) s.t. sum_t T_t <= K', 0 <= T_t <= T_bar_t^eff

in one KKT pass: one allocation unit is one row (item × geo × date) inside opt_periods ∩ opt_geos × opt_items; theta_t = p(X_t)' beta_hat in KPI units is the median over the Phase 3 repetitions and is estimated for every unit, also one without spend; the support cap T_bar_t = max(Q_q(cell_t), T_t^actual) is the optimization.cap_quantile-quantile q of the positive actual treatment within the unit's geo × item cell, floored at the unit's own actual spend (the default q = 1.0 is the cell maximum, i.e. the historical cap; with q = 0.95 an outlier day keeps its own actual spend as its cap — it can be cut by the KKT condition or the ROI cap but no unit is ever pushed above its cap — while every other unit is capped at the cell's 95 % point); the ROI threshold r = opt_threshold_roi is a per-unit cap T_t^r (the root of theta_t fbar(T)/T = r; 0 when theta_t fbar'(0) <= r; inf when r = 0), T_bar_t^eff = min(T_bar_t, T_t^r); units with theta_t <= 0 receive 0. T_t*(mu) = clip((fbar')^-1(mu / theta_t), 0, T_bar_t^eff) with the closed forms of all four families and a single bisection on mu so that sum_t T_t*(mu) = K' (mu* = 0 when the caps absorb the budget). The former "exclude below the threshold → re-solve → iterate" loop no longer exists. budget_mode = roi_target skips the budget and sets T_t* = T_bar_t^eff. When Phase 0 selected selected_family = None the pipeline falls back to the existing (simple) optimizer with a warning; with several treatments no optimizer runs at all (Optimization with multiple treatments is not yet supported.); a non-spend treatment type forces simple_optimization=True at construction (see Optimization modes); simple_optimization=False is otherwise never overridden when saturation_config is given, and opt_threshold_roi < 0 raises at initialization.

Important

No carryover is modelled. The response of a row depends on that row's spend only (theta_t fbar(T_t)); this row-wise separability is what makes the one-pass KKT allocation exact, and it also means that the optimizer moves spend freely across days as well as across geos and items (the allocation units are rows). Read the optimized allocation at the geo × item × period level (totals over opt_periods), not day by day. To keep every day's total fixed, set optimization.reallocate_within: "date": one KKT problem per date with the per-date budget K_d = (1 + budget_increment) * sum_{t in d} T_t^actual (spend still moves across geos and items within the day; mu* is then per date and optimization_metrics.csv reports its median with marginal_roi_at_optimum_min / _max). If a lag of the treatment (<treatment>_l<p> from create_lagged_features) is among x_cols or covariates_for_cate, X_t depends on past T and the separability no longer holds; Phase 4 warns about it. Adstock / carryover itself (f(adstock(T); alpha)) remains future work.

The saturation optimizer additionally writes optimized_incremental_KPI_<t> and optimized_roi_<t> into the consolidated CSV and into 007_optimization/optimization_metrics.csv, where every geo / item / all row also carries budget_mode, budget_reallocate, reallocate_within, cap_quantile, assumptions (no_carryover), K, K_out, unspent_budget, marginal_roi_at_optimum (mu*; with marginal_roi_at_optimum_min / _max, equal to it under reallocate_within = "all"), n_units, n_units_excluded_negative_roi, n_units_capped_by_roi and n_units_at_support_cap (K, K_out, unspent_budget and the counts are aggregated over the units of that row, so a geo row can show a negative unspent_budget when budget was reallocated into it; budget_mode, the scope and mu* describe the whole problem). The same values appear in the HTML descriptive block of the 007_optimization tab. When treatment_type="spend" it also writes, per geo, per item and for the total, the two-line chart <prefix>_original_vs_optimal_treatment.png (purple original vs green optimal treatment); the incremental effect is covered by the before / after chart below.

Both optimizers write, per geo, per item and for the total, the outcome chart <prefix>_<outcome>.png: the red observed outcome (Original KPI) against the blue projected outcome under the optimized spend (Estimated total KPI with optimal treatment, Y + (KPI_opt - KPI) summed per date — for the simple optimizer the KPI of the cut rows leaves the total). The HTML tabs 007_optimization and roi_output_007_optimization (<treatment>) show it in the same position whatever simple_optimization is: right after the optimizer's own chart and above the before / after charts.

Both optimizers (simple and saturation-aware) write, per geo, per item and for the total, three before / after charts — two line charts stacked vertically on an identical vertical scale, "before optimization" on top and "after optimization" below, every panel on exactly the width and x-axis layout of <prefix>_timeseries_spend_vs_est.png so the dates line up: <prefix>_timeseries_spend_before_after.png (purple actual spend (treatment_spend_cols) / dark green optimized spend), <prefix>_timeseries_incremental_effect_before_after.png (red incremental effect of the actual spend / blue incremental effect under the optimized spend) and <prefix>_timeseries_roi_before_after.png (red ROI per date = incremental KPI / spend, 0 without spend / blue ROI under the optimized spend, with the ROI threshold as a dotted reference line) — every panel of the three charts is exactly half the height of spend_vs_est at the same vertical scale, so each before / after pair fits one screen — and optimization_yearly_summary.csv (per dimension and calendar year plus Total: spend_before, incremental_kpi_before, roi_before, spend_after, incremental_kpi_after, roi_after). The HTML tabs 007_optimization and roi_output_007_optimization (<treatment>) show the three charts in that order under the existing optimization chart and render the CSV as a matrix below them that follows the dimension dropdown: one row per metric (Spend, Incremental KPI, ROI) under "Before optimization" / "After optimization", one column per year plus Total, numbers only in the cells; the matrix always fits the width of the tab (its font size adapts to the widest number, down to 9 px, after which the label columns narrow and, as a last resort, the matrix scrolls inside the tab rather than cutting digits).