← Commercial research

CUSTOMERS · RETENTION & VALUE

Customer retention and lifetime value

Which customers may leave—and which interventions are worth testing?

Evaluated synthetic demonstrationExecutive summary ↓

EXECUTIVE SUMMARY / THE DECISION IN PLAIN LANGUAGE

A better churn score is not a case for more retention spending.

Evaluated synthetic study · no actual client outcomes

Average value-forecast error fell from $184 to $149. Yet a policy targeting roughly one in five customers showed no clear net benefit at $35 per contact.

$-0.04Net effect per eligible customer
$149Average twelve-month value error
2,160Final randomized customers

What a leader should do next

Keep the intervention in a controlled test until its incremental contribution clears its cost with adequate evidence.

What could change the decision

The learned policy ranged from a $7.06 loss to a $6.98 gain per eligible customer at the 95% interval. Selecting the highest-risk customers lost money in the stable simulation; a changed response pattern also harmed the learned policy.

Explore the interactive results ↓

01 / BUSINESS DECISION

Start with the decision.

Prioritize a fixed retention intervention for at most 20% of each eligible cohort. Evaluate twelve-month discounted contribution after the intervention cost, keeping predictive churn risk separate from the causal value of treatment.

02 / PRACTICAL IMPLICATION

What the evidence can support.

The survival model reduces average twelve-month value error from $183.54 to $149.24. That predictive improvement does not establish intervention profitability. At $35 per contact, the learned policy's net effect is $-0.04 per eligible customer, with a 95% interval from $-7.06 to $6.98. The evidence supports further experimentation, not a profitable-rollout claim.

03 / EVIDENCE

Retention predictions and intervention value.

Evaluated synthetic demonstration

A fully implemented study of 9,720 synthetic customers across 54 monthly cohorts. Models use 24 training cohorts, six matured validation cohorts and twelve untouched final cohorts after a maturity gap. Prediction is assessed on 1,051 final control customers; policy value uses all 2,160 final randomized customers.

Interactive results · the executive summary above remains the fixed headline comparison.

SYNTHETIC RANDOMIZED RETENTION · 2,160 FINAL CUSTOMERS · 12 MATURED COHORTS

Stable process · $35 intervention cost

The learned targeting policy has no clear positive net effect. Higher prediction accuracy does not by itself justify contacting customers.

Incremental net contribution per eligible customer · 95% intervals
No contact$0.00

$0.00 to $0.00 · 0.0% contacted

Contact everyone-$27.37

-$46.20 to -$8.53 · 100.0% contacted

Highest churn risk-$19.56

-$28.05 to -$11.07 · 20.0% contacted

Highest customer value-$0.12

-$8.34 to $8.09 · 20.0% contacted

Learned incremental value-$0.04

-$7.06 to $6.98 · 19.9% contacted

Dashed line marks zero net value. Amounts average over everyone eligible, including those not contacted. Intervals use twelve cohort clusters and a t approximation.

Predictive performance on untreated final customers

1,051 control customers · lower errors are better
ModelChurn probability error¹12-month value error²Value bias
Segment mean baseline0.184$183.54$3.46
Survival model0.159$149.24-$13.96

¹Brier score for cancellation within six months. ²Mean absolute error of realized, discounted twelve-month contribution. These metrics evaluate prediction; the randomized policy comparison evaluates intervention value.

Known-truth check and fixed-policy cost sensitivity

Oracle expected values come from the data generator and are used only to assess the result. They are unavailable to a real decision maker. The best eligible 20% oracle policy yields $2.35 per eligible customer in this scenario.

Hold the selected customers and intervention fixed; change only accounting cost
PolicyOracle expected net$15 contact$35 contact$55 contact
No contact$0.00$0.00$0.00$0.00
Contact everyone-$25.96-$7.37-$27.37-$47.37
Highest churn risk-$14.51-$15.56-$19.56-$23.56
Highest customer value$1.20$3.88-$0.12-$4.12
Learned incremental value$1.04$3.93-$0.04-$4.01

This is accounting sensitivity, not evidence about a larger incentive or a different intervention. Contacting everyone exceeds the targeted policies’ 20% budget and is shown only as an additional comparator.

04 / DATA

Know where the evidence comes from.

Original synthetic contractual customer histories, version synthetic-retention-v1. Scoring features are tenure, engagement, service friction, tier and baseline monthly contribution. Treatment is randomly assigned with probability 0.5. Cancellation is absorbing; each customer has a complete twelve-month outcome window. Separate oracle files contain known potential expectations for evaluation only.

The source includes a data dictionary, fixed configuration and seed, original data generator, observed CSVs, evaluation outputs and source/data hashes. Read the data dictionary ↗

Inspect full results and provenance hashes ↓

05 / METHODOLOGY

Match the method to the design.

Fit a regularized discrete-time cancellation model to untreated customers and combine survival probabilities with baseline margin and a 10% annual discount rate. Fit outcome regressions in three training folds to construct doubly robust treatment-effect scores using the known randomization probability. Learn a prespecified feature-based score, target positive expected net effects within a 20% cohort budget, and evaluate the frozen policy on later randomized cohorts.

Evaluation design

  • Compare segment means with a discrete-time survival model for six-month churn and twelve-month discounted contribution.
  • Three-fold out-of-fold training creates a treatment-value score; freeze the policy before the twelve final cohorts.
  • Compare no contact, contact everyone, highest churn risk, highest untreated value and learned incremental value; repeat under no effect and response shift.

References motivate the methods. These study-specific implementations do not claim every assumption or theoretical guarantee of the cited work.

06 / LIMITATIONS

Conditions that matter.

  • Training and validation outcomes mature before final scoring begins; no incomplete window is treated as a completed target.
  • Randomized treatment, consistency and no interference identify incremental value; churn predictions alone do not.
  • A $35 intervention has a fixed form and timing. Cost sensitivity changes accounting cost, not the treatment or its effectiveness.
  • Confidence intervals use twelve cohort clusters and a t approximation. They remain uncertain with so few clusters and do not establish individual treatment effects.

The horizon is twelve months; the title does not imply value over an unlimited customer lifetime. Synthetic intervention response can fail to transfer over time: reversing that relationship makes the learned policy materially harmful. The survival model assumes baseline margin persists, and the upper-risk calibration bins are sparse. Known oracle expectations are unattainable evaluation benchmarks, never model inputs.

07 / CODE & NEXT STEPS

Reproduce the result.

PYTHON 3.11.16 · LOCKED NUMPY / SCIPY · FIXED SEEDS
git clone https://github.com/mpgibb/customer-value.git
cd customer-value
uv sync --frozen
uv run python -m unittest discover -s tests -v
uv run python study.py
git diff --exit-code -- data results

Automated checks and synthetic evaluation are complete; Michael’s independent technical review is pending.

Next step

Specify one real intervention and margin measure, then run a budget-limited randomized retention test. Require a positive, decision-relevant net-value result and stable response before broader use.

Privacy preferences

Security verification and inquiry delivery are essential. These optional choices do not affect them.

Optional services may remain unavailable until configured. No raw chat, search or contact text belongs in analytics. Read the privacy notice.