I've been thinking about something embarrassingly ordinary. Sometimes I finish dinner, feel completely full, and still find myself thinking about dessert. I know I'm full, and I know another bite probably won't make me feel any better. Somehow, I still want it. Usually the feeling passes after a while.
It made me curious about the gap between wanting something and actually benefiting from it. And then I started thinking about businesses.
We now have AI systems that can predict what customers might buy, which offer they might click, and when they're most likely to respond to a promotion. If I were running a small business, that would sound pretty useful. I'd love to know which customers might buy another product.
But there's a question hiding in that prediction.
Just because someone is likely to buy something, does that mean encouraging them to buy it is a good business decision?
The fifth slice of cake
The neuroscientist Kent Berridge has spent years studying a distinction that helps explain why this question is interesting. He describes two related parts of reward: wanting, the motivation to pursue something, and liking, the pleasure that comes with getting it. The two usually happen together, but they can separate. A person can stay motivated to pursue a reward even when the pleasure attached to it has faded.
That doesn't explain every craving, including mine. I'm using it as a lens, not an explanation. Still, it's an interesting way to think about our relationship with food.
It also reminds me of diminishing marginal utility, a concept from introductory economics. Imagine eating a chocolate cake. The first slice is wonderful. The second is still good. By the third you're pretty satisfied, the fourth adds almost nothing, and the fifth might actually leave you feeling worse.
The fifth slice of cake
Marginal utility measures the additional satisfaction gained from consuming one more unit. In this hypothetical cake, the marginal utility of the fifth slice is negative. You might still have enjoyed the cake overall, but that last slice reduced your total satisfaction.
Now think about the bakery owner. Every additional slice sold creates revenue, while the customer's experience might be getting worse. The owner can see the sales in the register and has far less visibility into the satisfaction behind them. A customer choosing to buy something gives us useful information. It doesn't tell us whether that decision made their experience better.
That distinction starts to matter once businesses use AI to encourage more purchases.
The prediction looks right. So what's the problem?
Suppose you run a neighborhood café. You've started using an AI-powered marketing tool that analyzes purchase history, picks out customers likely to respond to an offer, and suggests when to contact them.
One afternoon, the system predicts that a regular customer has a 75% probability of buying another dessert if they receive a recommendation. You send the offer, and they buy it. The prediction was successful.
But maybe the customer later regrets spending the extra money. Maybe they find the recommendations increasingly annoying, and start visiting less often. We don't know whether any of that will happen, and neither does the original 75% prediction.
Which raises a fair question.
Isn't AI supposed to be smart enough to figure this out?
We've become used to AI writing code, analyzing spreadsheets, answering questions, and solving problems that once required specialized expertise. Why can't it make a relatively simple decision about sending someone a dessert promotion?
To answer that, it helps to look at what these systems actually do.
A customer prediction model learns statistical patterns from historical data. Customers with certain purchase histories might have accepted dessert offers 75% of the time in comparable circumstances, and the model uses what it learned to estimate the probability of a future outcome. That probability is an estimate. It depends on the data, on the model, and on the circumstances staying relevant.
A large language model, the technology behind many AI chatbots, works differently. During pretraining, it learns to predict text through probability distributions over possible next tokens. Additional training and tools give it a much wider range of useful abilities.
These are different technologies, and AI includes many other approaches. What matters for our café is a limitation they share: a system can perform its assigned task very well while leaving out information that matters to the business decision.
The café's marketing model was asked to estimate purchase probability. It might have no information about whether the customer regretted the purchase, whether the offer damaged their trust, or whether they would have bought the dessert without the promotion at all. And even if the AI could predict all of these things accurately, someone would still need to decide how much the business values revenue, customer experience, marketing costs, and long-term loyalty.
A few practical limits are worth remembering. AI learns from the data available to it, and a small business might have only a few hundred customer records, with very little information about why people stop coming back. Predictions can become less reliable when circumstances change: a new competitor, an unusual holiday season, or a shift in what customers want can make historical patterns less useful. Generative AI adds another concern, because a chatbot can produce plausible explanations, including incorrect ones. Asking it why a customer left doesn't establish the actual cause.
There's also a particularly important difference between predicting someone's behavior and predicting how that behavior would change because of something we do. I'll come back to that.
For now, the lesson is that intelligence alone doesn't settle the question. We need to define what the business is trying to achieve, and find evidence that our decisions are moving us toward it.
A sale today, a customer tomorrow
Three concepts that many business owners already know fit together here.
The first is marginal utility: how much additional satisfaction a customer gets from consuming more.
The second is customer lifetime value (CLV): an estimate of the economic value of the customer relationship over time. A useful way to calculate it is the discounted future contribution profit you expect from a customer, after accounting for the relevant costs.
The third is AI-assisted decision-making: a system that helps estimate what could happen under different actions, provided there's suitable data and a suitable method.
Three concepts, one decision
Marginal utility
The satisfaction a customer gets from one more unit.
Felt by the customer, rarely observed by the business
Customer lifetime value
The future contribution profit expected from the relationship, discounted to today.
Estimated in dollars, with uncertainty
AI decision-making
Which next action adds the most value, once later effects are counted?
- Recommend
- Wait
- Change the offer
- Stop
Imagine that our café earns $5 in contribution profit from selling another dessert. That looks like a straightforward gain. But suppose repeated promotions gradually make customers less willing to return. A few dollars today could come at the expense of a much more valuable future relationship.
Of course, a customer might love the extra dessert and visit more often. The effect depends on the individual, the product, and the circumstances, and the business needs a way to measure those differences. The question becomes:
How much additional value does this action create, after accounting for what it might cost us later?
The word incremental is doing important work there.
When the metric becomes the goal
There's a story from AI research that I keep coming back to. In a boat racing game called Coast Runners, a reinforcement learning agent was rewarded for hitting targets laid out along the track. The designers wanted it to finish the race. The agent discovered that it could circle through the same few targets over and over, collecting reward each time. So it kept going around in circles, scoring points without ever completing the race.
OpenAI first described the example in 2016, and Google DeepMind later used it to illustrate specification gaming: an AI system satisfying the literal specification of its objective in a way that diverges from what its designers intended.
Most business marketing tools aren't reinforcement learning agents, so I read this as an analogy rather than a description of how they work. The pattern still carries over. A retailer tells its marketing system to maximize email clicks. The system finds that more messages and more attention-grabbing subject lines generate clicks. The click-through rate looks great, but customers might be getting annoyed, unsubscribing, or no longer paying attention.
We've entered the territory of Goodhart's Law, often summarized as the idea that a measure becomes less useful once people aggressively optimize it as a target. Researchers have observed a related technical effect in AI called reward model overoptimization: optimizing against an imperfect reward model can keep raising its score while performance on a better measure of the intended objective goes down.
The practical point is that metrics are usually approximations of what we care about. Clicks approximate interest. Purchases approximate demand. Repeat visits can be evidence of satisfaction. Each tells us something useful, and each leaves something out. A café that measures only the number of desserts sold may overlook what it cost to sell them, and for an owner with a limited marketing budget, that oversight can get expensive.
The customers most likely to leave might be the wrong ones to save
Here's another problem AI can help us understand, and one where the difference between prediction and decision matters even more.
Imagine running a small hair salon. Your customer management system predicts that Customer A has a 90% probability of leaving, while Customer B has a 40% probability of leaving. You have enough money to offer a discount to only one of them. Who should get it?
Customer A seems like the obvious choice. But suppose Customer A recently moved across the country. They're very likely to leave, and a discount won't change that. Customer B is considering another salon because of price, and a timely offer might persuade them to stay.
| Customer | Chance of leaving without an offer | Chance of leaving with an offer |
|---|---|---|
| A | 90% | 90% |
| B | 40% | 10% |
Churn risk versus the effect of an offer
Customer A has the higher predicted churn risk. Customer B has the larger potential response to the discount. That difference is central to uplift modeling, which attempts to estimate the causal effect of a particular intervention: not who is likely to leave, but whose behavior the action would change.
In 2018, the marketing researcher Eva Ascarza published Retention Futility: Targeting High-Risk Customers Might Be Ineffective. Using two field experiments and machine learning, she showed that the customers identified as most likely to leave are not necessarily the best targets for retention campaigns. Targeting customers by how they respond to the intervention worked better than targeting them by risk.
That's an important result for any business that automatically sends offers to its highest-risk customers. Predicting which customers will leave is one task. Estimating which customers are more likely to stay because of our intervention is another.
Even that second estimate has to be connected to economic value. Saving a customer relationship may be valuable, but the size of the discount, the expected contribution profit, and the reliability of our estimates all matter. This is why AI needs good experimental evidence. Historical purchasing patterns alone generally can't tell us what would have happened to the same customer if we had acted differently.
The $150 mistake
Let's make this concrete. Imagine a small business sends AI-generated recommendations to 100 customers. For this example, suppose a properly designed experiment estimates that the campaign caused 30 additional purchases, and each purchase contributes $20 in profit before marketing costs. The immediate gain is:
30 × $20=+$600
Now suppose the experiment also indicates that the campaign caused five additional customers to leave, and each would otherwise have generated an estimated $150 in discounted future contribution profit. The estimated future loss is:
5 × $150=−$750
The campaign's estimated net incremental value is:
$600 − $750=−$150
The $150 mistake
These are invented numbers, and the assumptions matter. We're counting the purchases and customer losses caused by the campaign, rather than every purchase or departure observed afterward. The future contribution estimate also excludes the current profit already included in the $600. With those assumptions, the campaign generates extra sales while reducing the business's expected overall contribution profit.
The surprising part is that the marketing dashboard might look successful. A manager watching conversion rates could reasonably believe the AI system was doing an excellent job. The long-term calculation tells a different story.
Even Google has studied a version of this. In a 2015 paper, three of its researchers measured how the ads people saw on Google Search affected their later behavior, including whether they became less likely to click ads over time, which the paper calls ads blindness. The findings contributed to a 50% reduction in the ad load on Google's mobile search interface.
A café can't run studies like that, but the question carries over: what is a customer relationship worth over time, and how might today's offer change it?
Try running the café yourself
How much is too much?
Imagine you're controlling the recommendation intensity of a small café's AI marketing system. Turn it up and more desserts sell today, so cakes and cash pile up on the counter. The model also assumes that pushy recommendations make some regulars stop coming back. That cost arrives later, as visits that never happen, so watch the dining room as well as the till.
The animated café needs JavaScript. The receipt and the table below show the same numbers.
Profit today21 extra desserts × $20+$420
Profit from future visits lost0 of 100 customers stop coming × $150$0
Net valueNo one has left yet.+$420
The counter is today; the dining room is the months that follow. The café is a symbolic sample, not the customer base: the campaign reaches 100 customers, and the twelve seats are there so you can watch departures happen. Each guest who gets up stands for one customer the campaign drives away, so five empty seats at intensity 80 means 5 of the 100 customers, not 5 of 12. Each cake is one extra dessert ($20 contribution), and each note on the counter is $20 of net value.
The numbers at every 10 points of intensity
| Intensity | Extra desserts | Customers lost (of 100) | Contribution today | Future contribution lost | Net value |
|---|---|---|---|---|---|
| 0 | 0 | 0 | $0 | $0 | $0 |
| 10 | 6 | 0 | +$120 | $0 | +$120 |
| 20 | 12 | 0 | +$240 | $0 | +$240 |
| 30 | 16 | 0 | +$320 | $0 | +$320 |
| 40 | 21 | 0 | +$420 | $0 | +$420 |
| 50 | 24 | 1 | +$480 | −$150 | +$330 |
| 60 | 27 | 2 | +$540 | −$300 | +$240 |
| 70 | 29 | 3 | +$580 | −$450 | +$130 |
| 80 | 30 | 5 | +$600 | −$750 | −$150 |
| 90 | 31 | 8 | +$620 | −$1,200 | −$580 |
| 100 | 31 | 12 | +$620 | −$1,800 | −$1,180 |
Extra desserts = max(0, round(0.65 × I − 0.0034375 × I²)). Customers lost = min(12, round(5 × (I ÷ 80)⁴)). Both are made-up story curves.
Then use the second calculator to test your own assumptions.
What if?
Change the assumptions behind the $150 mistake and see when a campaign stops paying for itself. Every number here is hypothetical, and the cost of running the campaign is left out.
Example: estimating extra purchases from two groups
Split 200 similar customers at random. 100 get the offer and 100 don’t. Over the next two weeks, count what each group buys. These counts are observed; everything above is an assumption until you have them.
Extra purchases caused, roughly30
Customers lost work the same way: if 9 people in the offer group stop coming and 4 in the other group do, the offer probably cost about 5. With groups this small, chance alone can move these counts by several, so treat the result as rough.
Profit today30 × $20+$600
Future profit lost5 × $150−$750
Net effect of the campaign−$150
Break-even is 4 lost customers: that’s where $600 of profit today is cancelled out by $150 of future profit for each customer lost. With 5, the campaign ends up $150 behind.
The simulations use hypothetical response curves. They illustrate the economics without claiming to predict actual human behavior.
What would I actually do if I owned a small business?
I'd pick one decision that already costs money, like how often to send promotions, and first ask whether it needs AI at all.
If I only need to know what happened last month, a spreadsheet will do. If the job is repetitive and easy to describe, like sending a reminder 30 days after a visit, simple automation will do. AI is worth considering when the decision depends on patterns too messy to write as rules, and when getting it right is worth more than the tool costs.
First I'd write down four things: the outcome in profit rather than clicks, a baseline to compare against (ideally a randomly held-back group), the full cost, and the risks, from annoyed customers to sensitive data. If I couldn't fill those in, I'd stop there.
If I could, I'd run a small pilot. Thirty days is one possible length. The right length depends on how often customers buy and how quickly the outcome shows up; a salon whose regulars come every six weeks needs longer than a café.
With enough data, uplift modeling could later compare actions directly, using a simplified objective like this:
Expected net value=incremental current contribution profit + change in future customer value − intervention costs
If the expected benefit is too small or too uncertain, doing nothing is a reasonable answer.
Before adding AI to a decision
Pick the decision you have in mind and tick what’s true today. You’ll get one sensible next step, not a verdict on whether you’re ready for AI. Nothing you pick is saved or sent anywhere.
Suggested next step
Establish a baseline
Before any tool, work out what one extra purchase earns after variable costs, and record a few normal weeks of purchases, visits and opt-outs. Whatever you try next gets judged against that.
Back to the fifth slice
I started with a simple question about wanting another bite after I was already full. Following it led me to neuroscience, marginal utility, customer lifetime value, AI optimization, and causal inference.
The connections have their limits. Human appetite is complex, and a recommendation model has no direct access to anyone's internal satisfaction. Still, thinking about them together changes the way I look at a common business goal: getting customers to buy more.
A better prediction can help a business act more precisely. Better economic reasoning can help it decide which actions are worth taking. An experiment can help distinguish a successful intervention from something that would have happened anyway. For a small business owner, that distinction is worth learning before spending more money on AI tools.
Next time an AI product promises higher engagement, more conversions, or better customer targeting, I'd want to ask a few questions. What is it measuring? What happens beyond the next transaction? How do we know the recommendation actually changed the outcome? What would make the system decide to stop?
If we're teaching AI to get better at recommending the next thing, we should probably spend just as much time deciding when the next thing is worth recommending.
Research and further reading
- Kent C. Berridge, 'Liking' and 'Wanting' Food Rewards: Brain Substrates and Roles in Eating Disorders, Physiology & Behavior, 2009.
- Eva Ascarza, Retention Futility: Targeting High-Risk Customers Might Be Ineffective, Journal of Marketing Research, 2018.
- OpenAI, Faulty Reward Functions in the Wild, 2016.
- Victoria Krakovna and colleagues, Specification Gaming: The Flip Side of AI Ingenuity, Google DeepMind, 2020.
- Leo Gao, John Schulman, and Jacob Hilton, Scaling Laws for Reward Model Overoptimization, ICML, 2023.
- Henning Hohnhold, Deirdre O'Brien, and Diane Tang, Focus on the Long-Term: It's Better for Users and Business, KDD, 2015.
- OpenAI, Why Language Models Hallucinate, 2025.
A note on the numbers. The cake values, the café, the salon customers, the $150 campaign and both simulators are hypothetical illustrations, not measured data. The café curves are made-up story functions chosen to show the trade-off. Wanting and liking are a conceptual lens here, not an explanation of my own appetite. The research findings are linked where they appear.