Causal methods ask versions of the same impossible question: what would have happened to the same people, market, or warehouse without the treatment? These four experiments make that missing counterfactual visible because, inside a simulation, we know the truth.
The Confounding Machine
Customers with stronger purchase intent are more likely to see an advertisement and more likely to buy. Turn up targeting and watch an effect appear even when the ad itself does nothing.
What creates the confusion?
Purchase intent is a confounder because it influences both who sees the ad and who buys.
Try this
- Keep the true ad effect at zero and choose Random assignment. The exposed and unexposed customers overlap.
- Choose Strong targeting. High-intent customers now receive more ads, and the naive estimate jumps.
- Compare the orange naive bar with the blue adjusted bar. Adjustment compares customers at similar levels of purchase intent.
Naive estimate: average outcome among everyone exposed minus everyone unexposed.
Adjusted estimate: the exposure difference after accounting for purchase intent.
How is the adjusted estimate calculated?
This demo fits one linear model using every customer:
Purchase outcome = b0 + τ × Ad exposure + b1 × Purchase intent + error
The coefficient τ is the adjusted estimate. It compares an exposed and an unexposed customer while holding purchase intent fixed. In the chart below, the two sloped lines are those fitted same-intent comparisons. Their vertical separation is τ. The dashed horizontal averages ignore intent and produce the naive estimate.
Observed customers
Orange saw the ad, blue did notWhat is the effect of the ad?
Observed comparison versus adjusted comparisonPropensity Score Matching: Choose Better Comparisons
Treated and untreated customers can look very different before matching. A propensity score compresses their observed characteristics into the probability of receiving treatment, then matching compares customers with similar probabilities.
Problem and example
An e-commerce company targets an offer using pre-treatment browsing and purchase history. These observed confounders affect both offer exposure and purchasing. Matching asks whether each exposed customer has an unexposed customer with a similar probability of receiving that offer.
Try this
- Start with Good overlap and notice that the naive estimate is too large.
- Choose Match nearest neighbors. The matched estimate should move toward the true effect.
- Choose Poor overlap and match again. The remaining pairs may be credible, but many treated customers have no comparable control.
What is matched: this simulation combines observed pre-treatment confounders, such as browsing and prior-purchase history, into a customer-history score. That score estimates the propensity of receiving the offer. Customers are matched on propensity, never on their outcome.
Who is being compared?
Propensity is estimated from observed pre-treatment confoundersDifference-in-Differences Laboratory
One market receives a new policy while another does not. Their starting levels can differ, but DiD needs their untreated trends to remain comparable.
Problem and example
Suppose one delivery market adopts a routing policy while another does not. DiD removes stable market differences and common changes over time. Its key assumption is that, without the policy, both markets would have followed parallel trends.
Try this
- Keep Parallel trends. Compare the raw post-period gap with the smaller DiD estimate.
- Choose Reveal counterfactual. The hidden no-policy path runs parallel to the control and confirms the estimate.
- Choose Break the assumption. Reveal the counterfactual again and watch DiD mistake the diverging trend for part of the policy effect.
What DiD subtracts: the treated market's before-to-after change minus the control market's before-to-after change.
Outcomes over time
Treatment begins at the vertical lineSynthetic Control: Build the Counterfactual
No single comparison market resembles the treated market. Mix four donor markets until their weighted history does, then carry that synthetic comparison into the post-treatment period.
Problem and example
Suppose one warehouse adopts a new automation system. No other warehouse is a convincing comparison by itself, so we combine several untreated warehouses to reproduce its earlier throughput.
Try this
- Begin with Equal weights. The synthetic line is close, but it misses parts of the pre-treatment history.
- Move the four donor weights or choose Find best pre-fit. A credible counterfactual should track the treated warehouse before automation.
- Choose Reveal truth. In this simulation, you can compare the synthetic estimate with the no-automation path that would be hidden in real data.
What synthetic control constructs: a weighted combination of untreated units that acts like the missing no-treatment version of the treated unit.