Replacing opinion with evidence in campaign and contact decisions
Ask a marketing, collections or retention team how they arrived at their current contact strategy and the answer usually contains three ingredients: what performed reasonably last year, what a senior stakeholder believes, and what the operating model happens to permit.
None of those are unreasonable. But they share a weakness. They are static decisions, applied uniformly, revisited only when someone has the appetite to revisit them.
The underlying problem is explore versus exploit
Suppose there are three plausible times to contact a customer and you do not know which performs best. Two obvious strategies present themselves, and both are bad.
Pick the historical winner and stick with it. That is pure exploitation. You will never discover whether another option is better, or whether the answer has changed since you measured it.
Or split evenly across all three indefinitely. That is pure exploration, and you knowingly waste two-thirds of your contacts on options you already suspect are inferior.
This is a well-studied problem with a well-established solution. The relevant literature is the multi-armed bandit, and the method I reach for most often is Thompson Sampling, first proposed in 1933 in the context of clinical trials.1
How it works, without the notation
Each option carries a probability distribution representing the system's current belief about its performance. Early on, with little evidence, that distribution is wide.
To make a decision, the system draws a single random sample from each option's posterior distribution and selects whichever came out highest. It observes the outcome and updates that option's distribution via Bayes' rule.
The elegance is in what emerges. An option the system is uncertain about has a wide posterior, so it occasionally produces a high sample and gets selected. That is exploration, and it happens automatically. An option the system is confident about has a narrow posterior centred on a high value, so it is selected most of the time. That is exploitation. As evidence accumulates the distributions tighten and the balance shifts on its own, with no rule specifying when to switch and no exploration parameter to tune.
For binary outcomes such as response or no response, the mathematics is genuinely simple. Each option's belief is a Beta distribution. A success adds one to the first parameter, a failure adds one to the second. That is the entire update step.
Why this rather than A/B testing
Three practical differences matter in a commercial setting.2
No predetermined sample size. A/B tests require you to fix a horizon in advance and wait. Thompson Sampling adapts continuously as data arrives.
Lower opportunity cost during learning. An A/B test allocates equal traffic to the losing variant for the full duration of the test. A bandit reduces allocation to underperformers as evidence accumulates, so less volume is spent on options that are demonstrably worse.
It handles non-stationarity. An A/B test produces a verdict at a point in time. If customer behaviour shifts three months later, the verdict is stale and nobody knows. An adaptive system keeps sampling and re-converges.
Empirical work comparing bandit algorithms has consistently found Thompson Sampling competitive with or superior to alternatives such as upper confidence bound methods on real display advertising and news recommendation data, while being considerably simpler to implement.3 The theoretical picture is equally settled: regret bounds for Thompson Sampling are logarithmic in the number of rounds and match the known lower bound for the problem.4
Where it fits in a decisioning platform
Contact timing is the obvious application, but the method generalises to any decision with a finite option set and an observable outcome: message strategy, channel, frequency, offer selection, treatment path in collections.
In one platform I built, the optimiser ran across five dimensions simultaneously and learned independently for each tenant rather than applying a single global answer. That last point deserves emphasis. A static strategy has to be correct on average across an entire portfolio. An adaptive one converges on the right answer for each segment separately, which means it can be right in places where the average is wrong.
The same idea applies to credit policy
Contact strategy is the easy example because feedback is fast. The more valuable application is policy itself.
Most lenders set approval thresholds manually and review them periodically. But a threshold is simply another arm with an observable reward, where the signal is portfolio performance: did the customers approved under this configuration repay or default? A system can explore threshold configurations, observe outcomes and converge on the optimum within guardrails the risk function defines.
The binding constraint here is feedback latency. Campaign response arrives in hours; default behaviour in months. That does not invalidate the method, but it changes the design. You need early proxy signals for performance, you need to be substantially more conservative about exploration, and you need to be explicit about the cost of exploring into a decision that carries real consumer impact.
What it takes to do this properly
Three things, in my experience.
Clean attribution. The system learns from outcomes and cannot learn from outcomes it cannot measure. Attribution quality is the binding constraint on how well any adaptive system performs, and it is usually the piece that is missing.
Guardrails. Autonomous optimisation without boundaries is not something any regulated business should accept. Exploration has to be bounded, and the bounds have to be owned by the risk function rather than the algorithm.
Explainability. A risk committee will reasonably ask why the system chose what it chose. "It sampled from a posterior" is not an adequate answer, and the probabilistic nature of these methods has been identified in the research literature as a genuine obstacle to trust and debugging in sensitive domains.5 Posterior state needs to be inspectable and the decision path needs to be reconstructable after the fact.
Get those three right and the outcome is a decisioning system where strategy is not someone's opinion reviewed quarterly. It is the position the evidence supports, updated continuously.