Quantitative Methods

Reinforcement Learning for Sequential Business Decisions

Find out whether your repeated decision is actually sequential — whether today's action changes what tomorrow's options are worth — and what it costs you every period that you keep optimising it one period at a time.

  • Advanced
  • 14 min total
  • 14 chapters

What decision this helps you make: Whether to treat a recurring decision as a sequence with a state rather than a series of independent choices, and what discount factor your current greedy policy is implicitly assuming.

What this topic is

Reinforcement learning is the study of decisions that change the situation you will face next. Formally: a state describing where you are, a set of actions, a reward for each action, and transition probabilities saying how the state moves in response. The object you want is a policy — a rule mapping state to action — that maximises reward accumulated over time rather than reward collected right now. The gap between those two objectives is the entire subject.

Why it matters

A great many business decisions are made greedily and are not greedy problems. Contacting a customer today earns something and makes them less responsive next month. Discounting to hit a quarter earns revenue and teaches buyers to wait. Deferring maintenance saves cash and raises the failure rate. In each case the immediate arithmetic is favourable and the sequence is not, and no amount of care in the single-period calculation will surface that, because the cost lives in a state variable the single-period calculation does not contain.

Who should learn it

Operators running recurring decisions — contact cadence, replenishment, pricing, maintenance, credit line management — and anyone evaluating a vendor claim that reinforcement learning will optimise one of them.

What you will understand

  • How to write a business decision as a Markov decision process, and what the Markov assumption actually requires
  • A fully worked two-state example where the greedy policy loses 11% of value, solved by hand
  • Why the discount factor is a business assumption rather than a technical parameter, and where the crossover sits
  • Why offline evaluation of a policy is much harder than offline evaluation of a prediction, and what to do about it

Prerequisites

Common misconception

"Reinforcement learning is the advanced version of a bandit, so it should be better." It is not better; it is for a different problem, and it is dramatically more demanding. A bandit assumes each round is independent — what you serve today does not change what tomorrow's options pay. Reinforcement learning drops that assumption, and in exchange requires you to specify a state, a transition model, and a discount factor, none of which you observe directly. If your problem genuinely has no state transition, using reinforcement learning on it adds every one of those failure modes and buys nothing at all.