Robust causal inference for experiments and observational studies in Python, organized around scenarios (e.g., Classic RCT, CUPED, Unconfoundedness) with a consistent fit() → estimate() workflow.
- 📚 Documentation & notebooks: https://causalis.causalcraft.com/
- 🔎 API reference: https://causalis.causalcraft.com/api-reference
Causalis focuses on:
- Scenario-first workflows (you pick the study design; Causalis provides best-practice defaults).
- Extensive robustness tests that reveal issues in the study design or model specification
- Pydantic data contracts
- An advanced DGP (Data Generating Process) with heterogeneous treatment effects, latent variables, and correlated confounders
- A website with notebooks based on real-world cases
pip install causalisfrom causalis.dgp import generate_classic_rct_26
from causalis.scenarios.classic_rct import DiffInMeans, check_srm
# Synthetic RCT data as a validated CausalData object
data = generate_classic_rct_26(seed=42, return_causal_data=True)
# Optional: Sample Ratio Mismatch check
srm = check_srm(data, target_allocation={0: 0.5, 1: 0.5}, alpha=1e-3)
print("SRM detected?", srm.is_srm, "p=", srm.p_value, "chi2=", srm.chi2)
# Estimate treatment effect with t-test inference (or bootstrap / conversion_ztest)
result = DiffInMeans().fit(data).estimate(method="ttest", alpha=0.05)
result.summary()When historical outcomes are unavailable, train an outcome model on one part of the experiment and estimate effects on different users. Baseline features must still have been recorded before experimental assignment. The example below learns utilization probability with communication from treatment training users; it predicts an outcome, not incremental uplift.
from sklearn.linear_model import LogisticRegression
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler
from causalis.data_contracts import CausalData
from causalis.shared.rct_design import PrognosticEnrichment
from causalis.scenarios.classic_rct import DiffInMeans
data = CausalData.from_df(
df, treatment="treated", outcome="utilized",
confounders=["prior_activity", "prior_spend"], user_id="user_id",
)
rule = PrognosticEnrichment(
estimator=make_pipeline(StandardScaler(), LogisticRegression(max_iter=1000)),
training_source="treatment", # alternatively: "control"
train_fraction=0.3,
top_fraction=0.2,
split_seed=42,
).fit(data, features=["prior_activity", "prior_spend"])
# Supply the original planned experiment allocation; do not infer it from counts.
report = rule.diagnostics(target_allocation={0: 0.5, 1: 0.5})
print(report.counts)
print(report.srm_evaluation, report.srm_selected)
print(report.balance)
print(report.warnings)
selected = rule.select() # CausalData containing only selected evaluation users
effect = DiffInMeans().fit(selected).estimate(method="conversion_ztest")
effect.summary()The hash split uses user IDs and a separate analysis seed independently of
treatment/outcome. Actual train/evaluation proportions may vary. The estimator
is cloned and fitted only on the chosen training arm. A single frozen cutoff
is the 80th percentile of predicted scores across both training arms; evaluation
outcomes are never used to train the model or choose this cutoff. Evaluation users
in both arms are selected with score >= cutoff. Ties are included, and the
selected evaluation share need not be exactly 20%. For a fixed cutoff, use
top_fraction=None, threshold=0.1. get_partition() returns a copy with
user_id, sample, score, and selected columns for auditing; the ID column
is named user_id even if the input uses another name.
select() returns a contract for existing RCT estimators, including CUPED, and
omits confounders that are constant in the selected segment. Diagnostic balance
still includes those columns and the score. Empty/single-arm segments and
constant outcomes cannot be converted to CausalData; get_selected_df() and
diagnostics() remain usable for inspecting counts. A conversion z-test requires
sufficient events and non-events; the report flags sparse segments. Do not change
the cutoff to repair SRM, balance, or significance after examining evaluation.
For binary-treatment RctCausalData, pass outcome="utilized" to fit() when
multiple outcomes are declared. Multiple treatment columns are not supported.
The workflow assumes independently randomized users; clustered or blocked
experiments need a split and inference appropriate to their assignment design.
The estimated effect targets the selected population, not the full audience,
and enrichment does not guarantee a power gain. If evaluation outcomes have
already guided model or subgroup selection, treat the analysis as exploratory
and confirm it in a new experiment. The API cannot verify historical feature
timestamps or enforce that users have not already examined evaluation results.
from causalis.scenarios.unconfoundedness.dgp import generate_obs_hte_26
from causalis.scenarios.unconfoundedness import IRM
from causalis.data_contracts import CausalData
causaldata = generate_obs_hte_26(return_causal_data=True, include_oracle=False)
from causalis.scenarios.unconfoundedness import IRM
model = IRM().fit(causaldata)
result = model.estimate(score='ATTE')
result.summary()Generate one staggered client-month panel with separate observed data and oracle effects, then prepare DiD, DML/IRM, and IPCW analysis views:
from causalis.dgp import generate_organic_activation_data
from causalis.dgp.organic_activation import make_did_pair, make_dml_view, make_ipcw_view
df, oracle, metadata = generate_organic_activation_data(scenario="baseline", seed=42)
df_did = make_did_pair(df, g=10, t=15)
df_dml = make_dml_view(df, g=10, horizon=6)
df_ipcw = make_ipcw_view(df, g=10, horizon=6)The organic activation guide explains
the six scenarios, allowed model features, diagnostics, and the distinct
ATT_no_activation and ATT_defer_natural benchmarks. These helpers prepare
data; they do not fit estimators.
from causalis.data_contracts import RctCausalData
data = RctCausalData.from_df(
df,
treatment="treated",
outcomes=["revenue", "purchases", "retention"],
confounders=["age", "prior_spend"],
user_id="customer_id", # optional
)
Y = data.Y # DataFrame, including when only one outcome is specified
X = data.X
revenue_data = data.for_outcome("revenue") # CausalData for existing estimatorsFor multiple arms, supply one-hot columns including the control arm:
data = RctCausalData.from_df(
df,
treatment_names=["variant_a", "control", "variant_b"],
control_treatment="control",
outcomes=["revenue", "purchases", "retention"],
confounders=["age", "prior_spend"],
)
D = data.D # treatment matrix, with control first
revenue_data = data.for_outcome("revenue") # MultiCausalData for multiple armsEvery row must belong to exactly one arm, and every arm must be represented.
A single binary column uses 0 as control; omit control_treatment in that case.
This contract stores one independent copy of the selected columns and validates
shared treatment/confounders once. It checks missing values column by column and
screens duplicate columns with fingerprints before exact comparisons. Outcomes
and confounders must be finite, real numeric or boolean, and non-constant;
each treatment column must contain both 0 and 1. Identical columns are rejected.
Y, D, X, and get_df() return copies; for_outcome() copies and validates a
single-outcome subset using the destination contract's constraints, including
MultiCausalData's limit of 15 arms. Existing estimators consume that
CausalData or MultiCausalData subset. Validation checks data structure;
it does not establish that treatment assignment was randomized.
To measure construction time and retained data size on your machine, run
.venv/bin/python benchmarks/rct_causal_data.py --rows 1000000 --outcomes 32 --arms 3.
Generate a reproducible experiment using the internal DGP outcome families and covariate samplers:
from causalis.dgp import generate_rct_causal_data
data = generate_rct_causal_data(
n=20_000,
n_treatments=3,
d_names=["control", "variant_a", "variant_b"],
confounder_specs=[{"name": "prior_spend", "dist": "normal"}],
outcome_specs=[
{"name": "revenue", "alpha_y": 20, "beta_y": [3], "theta": [1, 2]},
{"name": "purchases", "alpha_y": 10, "beta_y": [2], "theta": [0.5, 1]},
],
seed=42,
)Each outcome supports continuous, binary, poisson, or gamma. theta
sets arm effects on the family's link scale; the example uses continuous
outcomes, so revenue effects are exactly 1 and 2. For a binary treatment column,
set n_treatments=2, treatment_encoding="binary". Use
return_causal_data=False, include_oracle=True for a DataFrame containing
allocation probabilities, potential-outcome means, and natural-scale effects.
Random assignment is shared across outcomes; rare missing arms raise an error
instead of changing sampled assignments.
from causalis.scenarios.cuped import CUPEDModel
model = CUPEDModel().fit(data, covariates=["prior_spend"])
estimates = model.estimate() # estimates[outcome][active_arm]
estimates.summary(outcome="revenue") # formatted comparisons, side by side
revenue_a = estimates["revenue"]["variant_a"]
# Fit just one comparison when needed:
revenue_model = CUPEDModel().fit(
data, covariates=["prior_spend"], outcome="revenue", treatment="variant_a",
)
revenue_a = revenue_model.estimate() # CausalEstimateCUPED compares each active arm only against the declared control and centers
covariates over those two arms. It shares the design matrix and decomposition
across outcomes within each comparison. Pass covariates=[] for unadjusted
estimates. One fitted comparison returns a CausalEstimate; multiple comparisons
return RctEstimates, preserving nested outcome/arm dictionary access.
estimates.summary(outcome="revenue") displays all arms for an outcome using
the same field and confidence-interval formatting as CausalEstimate.summary().
Omit outcome to show every outcome, or add treatment to select an arm.
The same selectors work in estimate() and
summary_dict(). Batch assumptions_table() adds outcome and treatment columns.
Confidence intervals and p-values are per comparison, without multiplicity
adjustment. Existing CausalData inputs retain their single-estimate behavior.
CUPED keeps treatment_mean and control_mean as raw observed group means.
The additional adjusted_control_mean (regression intercept) and
adjusted_treatment_mean (intercept plus treatment coefficient) are evaluated
at the comparison sample's mean covariates. Their difference equals the adjusted
ATE; the difference of raw means may not. Both pairs appear in the summary.
The default relative effect is 100 * ATE / adjusted_control_mean.
Choose relative_denominator="raw_control" explicitly to divide by the observed
control mean instead; the summary reports which denominator was selected.
CUPED preserves consistency and asymptotic unbiasedness under randomization
and standard regularity conditions; finite-sample bias need not be zero.
The comparison benchmark is
.venv/bin/python benchmarks/cuped_rct.py --rows 100000 --outcomes 8.
It disables optional regression checks with run_checks=False; ordinary fits
keep their configured checks.
Use a past RctCausalData to estimate future sample size or sensitivity from
historical control rows only. Both calculators require one explicit outcome.
Omitted covariates, None, and [] use classic unadjusted planning, even when
the data declares confounders. Pass an explicit list to enable CUPED. With the
data from the example above:
from causalis.shared.rct_design import calculate_mde, calculate_sample_size
allocation = {"control": 0.5, "variant_a": 0.25, "variant_b": 0.25}
# Classic planning: separate experiments for relative MDEs [0.5, 1, 5, 10, 20]%.
sizes = calculate_sample_size(data, outcome="revenue", allocation=allocation)
# Custom relative targets: 1 means 1%, not 100%.
custom = calculate_sample_size(
data, outcome="revenue", scenario="user", mde_type="relative",
mde=[1, 3, 5], allocation=allocation,
)
# CUPED sample-size planning with absolute effects in revenue units.
adjusted_sizes = calculate_sample_size(
data, outcome="revenue", covariates=["prior_spend"],
scenario="user", mde_type="absolute", mde=[0.5, 1, 2],
allocation=allocation, include_details=True,
)
# Classic sensitivity for an exact total audience of 30,000 participants.
sensitivity = calculate_mde(
data, outcome="revenue", sample_size=30_000, allocation=allocation,
)
# CUPED sensitivity for the same audience and allocation.
adjusted_sensitivity = calculate_mde(
data, outcome="revenue", sample_size=30_000, covariates=["prior_spend"],
allocation=allocation, include_details=True,
)
# Format for presentation without rounding the underlying numeric results.
print(sizes.to_string(index=False, formatters={
"mde_relative": "{:.1f}%".format,
"mde_absolute": "{:.3f}".format,
"n_control": "{:,.0f}".format,
"n_treatment": "{:,.0f}".format,
"n_total": "{:,.0f}".format,
}))calculate_sample_size takes MDE targets and returns required sample sizes.
scenario="default" uses the preset percentages and rejects custom mde or
mde_type. scenario="user" requires both a type and a nonempty sequence of
finite positive targets; input order and duplicates are preserved.
calculate_mde takes one positive integer sample_size across all future groups
and returns absolute and relative MDE for each active arm.
Both return unrounded numeric DataFrames with compact columns outcome,
control, treatment, mde_relative, mde_absolute, n_control, n_treatment,
and n_total. The MDE columns describe requested targets in sample-size
results and calculated sensitivity in MDE results. Relative values are
percentages of the raw historical control mean: with baseline 10, a relative
input of 1 corresponds to an absolute difference of 0.1. Relative target planning
requires a positive mean. Absolute target planning and MDE calculation remain
available for nonpositive means, with relative output set to NaN.
include_details=True appends baseline_mean, variance_raw, variance_used,
variance_reduction_pct, n_reference, alpha, and power. Sample-size results
also include achieved_power after rounding. Classic planning uses raw sample
variance (ddof=1) without fitting a regression; variance_used equals
variance_raw and variance reduction is zero. CUPED uses only explicitly selected
pre-treatment confounders and residual variance SSE / (n_reference - design_rank).
Historical variance is estimated once for all targets.
All declared groups participate in each future experiment. Allocation defaults
to equal shares; custom shares must be positive and sum to one. For a binary
treatment column d, use keys "d=0" and "d=1". For each sample-size target,
take the largest continuous requirement across active arms and round every group
up. MDE calculations preserve the supplied total using largest remainders, with
ties resolved in contract order, control first. Every group needs at least one
participant. n_total counts shared control once; do not sum it across rows.
Planning uses deterministic, two-sided normal power and assumes independently
randomized units with the same historical-control variance in every future arm.
Binary outcomes also use this fixed-variance approximation. alpha and power
are per comparison, without multiple-testing correction or a joint detection
guarantee. Control-constant covariates are dropped with a warning; singular CUPED
fits, insufficient residual degrees of freedom, and zero or non-finite variance
raise errors. Estimated variance reduction can be negative and is not clipped.
Breaking API change: the old calculate_cuped_mde and
calculate_cuped_sample_size names and CUPED design module are removed. Use the
shared calculators above; there are no aliases or deprecation wrappers. The old
shared summary-statistics calculate_mde API (baseline_rate, variance,
ratio, and data_type) is also removed. Both new APIs require RctCausalData
and a single outcome; the previous multi-outcome mapping interface is removed.
Declare one independent cluster column with CausalData(..., cluster="cluster"),
or use generate_obs_clustered. IRM keeps entire clusters in cross-fitting
folds. Refutation diagnostics use the saved estimation-sample membership,
including after overlap dropping, and preserve observation-weighted ATE/ATTE
targets when cluster sizes differ.
from causalis.dgp import generate_obs_clustered
from causalis.scenarios.unconfoundedness import IRM
from causalis.scenarios.unconfoundedness.refutation import (
run_score_diagnostics, plot_influence_instability, sensitivity_analysis,
)
data = generate_obs_clustered(n=10_000, n_clusters=100, seed=42)
estimate = IRM(data, n_folds=3, random_state=42).fit().estimate()
scores = run_score_diagnostics(data, estimate)
figure = plot_influence_instability(estimate) # clusters by default
rows = plot_influence_instability(estimate, influence_level="row")
robustness = sensitivity_analysis(estimate, r2_y=0.03, r2_d=0.03)Score uncertainty and sensitivity bounds use one-way CR1 covariance from sums
of centered row influences, with Student's t reference on G - 1 degrees of
freedom. Sensitivity benchmark refits preserve the original cluster-disjoint
folds. This matches IRM's existing inference convention; small or highly
unbalanced cluster samples can still have poor asymptotic approximations.
influence_level="auto" selects clusters when membership is available and rows
otherwise; "cluster" and "row" select explicitly. Cluster tables include
size, treated share, estimate contribution, and variance share. These are
influence diagnostics, rather than exact leave-cluster-out refits.
Overlap and balance remain descriptive summaries over observations. Reports add pure-arm/mixed cluster counts and cluster weight concentration; the latter describes concentration of weight across groups, not an ICC-adjusted effective sample size. A cluster need not contain both treatment arms. Infinite SMD is graded as a balance violation. Pooled out-of-sample moment t/p-values are now unavailable because the former pooled construction is degenerate; fold tables remain available as descriptive stability checks.
Residual scatter plots default to a reproducible sample of 20,000 points
(random_state=42); all rows still determine summary curves. Pass
max_scatter_points=None for the full scatter. Missing or contradictory cluster
membership raises an error instead of selecting IID inference. Fewer than two
retained clusters leaves descriptive diagnostics available but inference
unavailable; sensitivity inference requires at least two clusters.
Run .venv/bin/python benchmarks/refutation_cluster.py --rows 100000 1000000
to measure diagnostics and rendered plots with three timed repetitions, separate
allocation measurements, and isolated-worker peak RSS. Use --score ATTE for
ATTE. The checked-in results are in
benchmarks/refutation_cluster_results.json.
Measured at one million rows, 10,000 clusters and eight confounders, across ATE/ATTE and balanced/unequal cluster sizes:
| Operation | Median seconds per call | Peak additional allocations (MiB) |
|---|---|---|
| Overlap | 0.298–0.303 | 209.3 |
| Balance | 0.152–0.167 | 70.8 |
| Score | 0.363–0.461 | 139.7 |
| Sensitivity | 0.024–0.027 | 30.8 |
| Influence plot, rendered | 0.179–0.188 | 63.1 |
| Residual plot, rendered | 0.218–0.223 | 65.2 |
These timings exclude generation and fitting, which are recorded separately. Allocation peaks exclude retained inputs; the largest full worker peak RSS was 1,545.7 MiB, including Python, imports, generation, fitting and estimation. The benchmark uses inexpensive dummy learners to isolate diagnostic work; end-to-end correctness tests also fit random forests on the clustered generator.
Pre-specify a practically meaningful effect boundary and one or more
domain-justified groups of observed pre-treatment confounders. The primary
decision uses element-based long/short gain statistics for the benchmark
group. Its cf_y, cf_d, and rho are calibrated jointly from the outcome
variance, Riesz-representer variance, and actual effect shift. The 2× strength
and forced rho=1 scenarios are reported as secondary stress tests.
from causalis.scenarios.unconfoundedness.refutation import run_sensitivity_protocol
# Example only: replace with a domain-justified, pre-specified group.
primary_group = list(causaldata.confounders[:2])
protocol = run_sensitivity_protocol(
model,
causaldata,
benchmark_groups={"primary_domain_benchmark": primary_group},
decision_threshold=0.0, # replace with the minimum practical effect
direction="auto", # default: infer direction relative to the threshold
preconditions_passed=True, # causal set, overlap, nuisance quality, stability
)
print(protocol["status"])
print(protocol["summary"])
protocol["primary"]
protocol["stress"]
protocol["adversarial"]PASS means every primary benchmark's bias-aware confidence interval remains
strictly beyond decision_threshold in the requested direction. RV and
RVa are reported as robustness diagnostics, not compared with universal
cutoffs. An empty benchmark set, failed external preconditions, unavailable
sensitivity elements, or strengths outside the finite sensitivity domain
produce FAIL.
For binary IRM, the confounding bound is
abs(rho) * sqrt(sigma2 * nu2 * r2_y * r2_d / (1 - r2_d)).
r2_y is a partial outcome R-squared in [0, 1]; r2_d is the relative
Riesz-representer variation gain in [0, 1). The outcome channel is used
directly, without an odds transformation. This corrects the previous outcome
parameterization and changes numerical results for explicitly supplied
strengths; no legacy mode is provided. RVa solves for the crossing of the
full bias-aware CI, including its changing standard error. The stress multiplier
scales each R-squared on its odds scale, so it need not multiply the bound by
that same factor.
Inference requires finite, aligned influence-function vectors and a finite,
non-negative standard error. Invalid inference fails closed; use_signed_rr=True
is rejected. Long and short models must target the same population, including
identical overlap masks when using overlap_policy="drop". Stale estimates and
incompatible score overrides are rejected. Within each protocol call, the long
model is checked once and each unique omitted-feature group is refitted once.
By default, direction="auto" selects positive when the original estimate is
at or above decision_threshold, and negative otherwise. It uses this same
direction for every scenario and returns it in protocol["direction"], with
an inference warning in protocol["warnings"]. For a negative estimate at a
zero threshold, every primary CI must have ci_upper < 0; touching or crossing
zero still fails. Thresholds retain their supplied sign, and an estimate equal
to the threshold does not pass. Use explicit direction="positive" or
direction="negative" for a pre-specified directional claim; these choices are
never overridden. Significance before sensitivity analysis alone does not
guarantee a pass.
Benchmark strengths use the relative gains
gain_y = (sigma2_short - sigma2_long) / sigma2_long and
gain_d = (nu2_long - nu2_short) / nu2_short, as in DoubleML and the paper's
relative-strength construction. These calibrate an additional omitted
confounder for the full model. Gains are clipped to [0, 1] as cf_y and
cf_d; values at 1 are retained for auditing but marked uncalibratable because
the sensitivity API requires strengths in [0, 1). Such primary scenarios
produce FAIL. If either long/short gain is not strictly positive,
primary and stress use rho=sign(theta_short-theta_long); the adversarial
scenario still forces rho=1. protocol["benchmarks"] retains raw gains,
clipping/fallback flags, long/short elements, and warnings for auditability.
In particular, a negative cf_d_raw becomes a numerical cf_d=0 boundary
benchmark rather than a missing scenario.
Binary IRM and MultiTreatmentIRM use cf_y and cf_d in [0, 1):
cf_y is the share of residual outcome variation explained by an omitted
confounder, and cf_d measures its contribution to the Riesz representer's
variation. In the default unsigned mode, the confounding bound is
abs(rho) * sqrt(sigma2 * nu2) * sqrt(cf_y * cf_d / (1 - cf_d)).
model.sensitivity_analysis(cf_y=0.02, cf_d=0.02, rho=1.0)Numerical compatibility: deprecated r2_y and r2_d keywords are accepted
with a DeprecationWarning and copied directly to cf_y and cf_d. They do
not retain the previous outcome odds conversion. Re-running an old call can
therefore change the sensitivity bounds and robustness values. Passing both a
new keyword and its alias raises ValueError. Result params and protocol
scenario tables use the new names; benchmark tables retain the old names as
copies of cf_y and cf_d.
Benchmark tables also report the nonnegative relative gains gain_y and
gain_d before upper clipping, along with signed raw gains and calibration
validity flags. Check calibration_valid before using cf_y and cf_d as
sensitivity inputs. The benchmark uses the relative-strength construction of
Chernozhukov et al., Appendix F, equations (22)-(23)
and DoubleML.
The stress multiplier scales cf_y and cf_d / (1 - cf_d) linearly, so the
default bound scales by that multiplier until cf_y is capped below one.
RV is the common strength cf_y = cf_d needed for the confounding bounds to
reach H0. RVa uses the original sampling standard error to approximate the
corresponding threshold with sampling uncertainty; it does not solve for the
threshold of the recalculated bound-specific confidence interval.
| Scenario | Estimator | Assumptions |
|---|---|---|
| Classic RCT | Difference in means (ttest, ztest, welch_permutation_t_test) | Random assignment, no sample ratio mismatch, SUTVA |
| CUPED | CUPED-adjusted difference in means with Lin specification | Random assignment, no sample ratio mismatch, SUTVA, valid pre-period metrics |
| Unconfoundedness | DML IRM | Unconfoundedness, Overlap, SUTVA, No leakage, Score stability |
| GATE | DML IRM (GATE and GATET) | Same assumptions as unconfoundedness, plus meaningful pre-specified or validated subgroup definitions. |
| Multi Unconfoundedness | Multi DML IRM | Unconfoundedness, Multi class Overlap, SUTVA, No leakage, Score stability |
| Synthetic Control | ASCM | No interference / spillovers, No anticipation, The treated unit’s untreated outcome path is well approximated by the donor pool |
| Difference in Difference | CallawaySantAnnaDID | Parallel trends, no anticipation, stable group composition, no spillovers between treated and control groups. |
| IV | DML IV | First-stage strength, Reduced form, Instrument balance by Z, Instrument propensity / predictability |
| Uplift / CATE scoring | DML IRM (CATE) | Identified treatment effects from randomized or unconfounded data, overlap, calibrated individual-level predictions. |
Introduction to Causal Inference: guide
See scenario notebooks: https://causalis.causalcraft.com/explore-scenarios
https://github.com/DoubleML/doubleml-for-py
Causalis covers methods often searched as:
- causal inference Python
- causal machine learning Python
- treatment effect estimation
- A/B testing Python
- randomized controlled trial analysis
- CUPED Python
- Double Machine Learning Python
- DML / IRM
- CATE estimation
- uplift modeling
- propensity score diagnostics
- synthetic control Python
- difference-in-differences Python