Quantitative Methods

Uplift Modeling and Heterogeneous Treatment Effects

Stop targeting the customers most likely to churn and start targeting the ones your offer actually changes — including finding the group your retention campaign is quietly driving away.

  • Advanced
  • 14 min total
  • 15 chapters

What decision this helps you make: Who receives an intervention that costs money per recipient: a discount, a retention call, a save offer, a win-back. The answer is a threshold on estimated incremental effect, not a threshold on risk.

What this topic is

Uplift modelling estimates how much a treatment changes an individual customer's outcome, rather than estimating the outcome itself. It requires a randomized experiment as training data, because the target quantity is a difference between two worlds and only one of them is ever observed for any given person. The output is a per-customer estimate of incremental effect, which is then used to decide who gets treated.

Why it matters

Almost every targeting decision in a business is made on the wrong quantity. Retention campaigns target churn risk, which identifies customers who are leaving — not customers whose leaving your offer would prevent. Discounts go to price-sensitive customers, many of whom would have bought anyway. In the worked example, sending an offer to everyone loses $560,000, sending it to the two highest-uplift deciles makes $296,000, and sending it to the lowest decile actively destroys value by reminding people they have a subscription. The same offer, the same budget, an $856,000 swing.

Who should learn it

Retention and lifecycle marketing teams, growth teams allocating discount budgets, customer success teams deciding where to spend human attention, and anyone whose targeting model currently scores propensity rather than incremental effect.

What you will understand

  • Why churn probability and treatment effect are different quantities, and why the highest-risk customers are often the worst targets
  • How to estimate per-customer uplift, and why the estimate can never be validated at the individual level
  • How to turn uplift deciles into a send threshold using the cost of the offer and the value of the outcome
  • Why some customers have negative uplift, and how to find them before the campaign does

Prerequisites

Common misconception

"Our churn model has an AUC of 0.84, so we know exactly who to send the save offer to." A churn model ranks customers by how likely they are to leave. The save offer should go to customers whose leaving it would prevent, which is a different ranking and can be close to unrelated. The highest-risk customers frequently include people who have already decided to go, whom no offer reaches, and people leaving for reasons the offer does not address. Meanwhile the customers your offer genuinely moves may sit in the middle of the risk distribution entirely. A propensity model and an uplift model answer different questions, and only one of them is the targeting question.