Transition Probability: Moving From One State to the Next

A transition probability attaches to three things at once: a starting state, an ending state, and the time between them. A single ending value carries no probability, so for a process with a continuous range of values the transition is written as a density rather than a number. Transitions over two steps must equal transitions over one composed with themselves, and that consistency is what makes a set of transitions a process.

Once the present state is a sufficient summary of the past, everything the model knows about the future has to be expressible as a rule that takes one state to another over a stretch of time. There is nowhere else for the knowledge to live. The rule that takes one state to another is the transition probabilityThe chance of moving from one state to another over a stated stretch of time.. Writing that rule out for every pair of states and every gap leaves nothing further to say about the model.

What does a transition probability attach to?

Ask somebody what the probability of Rs 115/- is and the honest answer is that the question is not finished. Probability of Rs 115/- starting from where? And after how long? A probability of Rs 115/- starting from Rs 111.08/- over six months is a completely different number from a probability of Rs 115/- starting from Rs 94/- over one month, and neither of them is the probability of Rs 115/-.

A transition attaches to a pair of states and a gap. A transition is a three input object, not a property of any one state. Drop the starting state and what remains is a marginal probability, which is a different animal. Drop the ending state and there is nothing to attach a number to. Drop the gapThe stretch of time the transition spans, from the moment the starting state is observed to the moment the ending state is read. and what remains is a quantity with no unit, in the same way that a speed quoted without a unit of time is not a speed.

Think of a lift that knows only which floor it is on. The useful question about it is never the chance of the fourth floor on its own. The useful question is the chance of the fourth floor twenty seconds from now, starting on the second floor. Change the starting floor and the answer changes. Change twenty seconds to five minutes and the answer changes again. The lift is a small memoryless machine and its entire behaviour is the table of those answers, one entry for each starting floor, each ending floor and each waiting time.

Read as a function of both states at once, the transition rule is often called a kernelAnother name for the transition rule, used when it is being thought of as a function of the starting and ending states together rather than as a list of separate numbers.. The name is worth knowing because it signals the right mental picture: not a list of separate probabilities, but one object with two state arguments, in the same way a matrix is one object with a row argument and a column argument.

One condition comes free with the definition and is worth stating because it is the check people actually apply. Whatever the starting state, the process has to be somewhere at the later time. So the total over every possible ending state is one.

The total over every ending state is the row sumThe total over all ending states for one fixed starting state, which must come to one., and it is the first thing anybody checks on a table of transitions. An enormous number of tables describing quite different processes all satisfy it, so the row sum is almost useless as a check on whether a table is right.

Why does a continuous process need a density instead of a number?

The lift has a listed set of states, so its transitions are numbers and they can be put in a table. The standard process is not like that. The standard process, an invented traded quantity, starts at Rs 100/-, drifts at 8 per cent a year with a volatility of 20 per cent a year, and can finish at Rs 115/-, or at Rs 115.0001/-, or anywhere in between. There is no list.

On a continuous range of values every single ending value carries probability zero, so the transition cannot be a probability and has to be a rate instead. This is not a technicality being tidied away. Writing down the chance of finishing at exactly Rs 115.000000/- and adding decimal places to the requirement shrinks the number being described toward nothing, and in the limit it is nothing. A quantity that is zero for every ending value cannot be the content of a model.

The everyday version runs as follows. A room thermometer reads 24 degrees. The chance that the room is at exactly 24 degrees, to infinite precision, is zero: there is always another decimal place that fails. The chance the room is between 23.5 and 24.5 degrees is a real number that can be acted on. Ranges carry probability; points do not. So the object the model supplies is the transition densityThe continuous version of a transition, written as a rate per unit of value rather than as a probability, so that it must be integrated over a range before it means anything., a rate per rupee of ending value. A probability comes out of the density only by integrating over a stretch.

Two consequences follow immediately and both trip people up. First, a density is measured per unit of value, so its height is not a probability and can be bigger than one. Change the unit and the height changes. Second, comparing two densities at a single point reveals very little on its own; what carries meaning is the area over the stretch in question.

What does the transition from the month six reading actually look like?

Take the locked path, the twelve step path published for this reading order, and stop at its month six reading of Rs 111.08/-. To six decimal places the logarithm of that reading is 4.710226. Ask the transition question: given that reading, and given a gap of half a year to the horizon, what is the rule?

The standard process is normally distributed on the logarithm, so the answer is written there. The logarithm gains the drift less half the variance rate, multiplied by the gap, and it picks up variance equal to the variance rate multiplied by the gap. With a drift of 8 per cent and a variance rate of 0.04, the drift less half the variance rate is 0.06 exactly, and over half a year that is a gain of 0.030000. The variance is 0.04 times half a year, or 0.020000. The spread of the logarithm is therefore 0.141421.

Putting the month six reading through that rule gives a conditional average for the logarithm of 4.740226 and a spread of 0.141421, and every other figure in this guide is a consequence of those two numbers. The conditional average of the process itself is Rs 115.61/-, from 115.610377. The middle of the distribution, the value with half the outcomes above and half below, is Rs 114.46/-. The peak of the density, the value where the curve is highest, is Rs 112.19/-, from 112.193574.

Three different numbers, from one rule. The three numbers differ because the shape leans to the right: the logarithm is symmetric, but exponentiating a symmetric shape stretches the upper side and compresses the lower one. The distance from the peak to the average is Rs 3.42/-, and that gap is not a rounding artefact. The distance between peak and average is what leaning looks like, and the distance widens with both the volatility and the length of the gap.

To turn the curve into numbers a reader can check, cut the finishing values into stretches and take the area over each. The table below does exactly that from the same rule, and the areas total one because they have to.

Notice the last two bars. A ten rupee stretch just above the peak carries 0.185102. The entire unbounded upper tail carries 0.184005, almost the same number. The leaning shape is doing that: the upper side is long but thin, and length does not automatically mean weight.

One more property of this particular rule deserves a name. The property is what makes the arithmetic below so clean. The transition depends on the gap and not on when the gap starts. A transition from Rs 111.08/- over half a year is the same rule whether that half year runs from month six to month twelve or from month two to month eight. A rule with that property is time-homogeneousThe case where the transition depends only on how long the gap is, not on when in the calendar the gap begins., and it collapses the four arguments of the general definition to three.

How do transitions over two steps relate to transitions over one?

Here the object stops being a definition and starts being a constraint. Suppose the transition over three months is known and the transition over six is wanted. Getting from the start to the finish over six months means passing through some value at three months, and the only question is which value. So the six month transition is not open to invention. It is already determined.

So every possible intermediate value is taken, weighted by the chance of reaching it, and the second transition applied from each one. Summing over intermediate values for a listed set of states, or integrating over them for a continuous range, gives the six month transition. The operation just described is the compositionChaining two transitions to obtain the transition over the two gaps combined, by totalling over every possible intermediate state., and composition is the single most important thing a transition rule does.

The last clause of that sentence carries the whole content. The middle time is arbitrary. The gap can be cut anywhere, and the composed answer has to come out the same. If it does not, the rule as written is not describing one process.

Now do it on the numbers, and watch what adds and what does not. Go from month six to month nine, a quarter of a year, and then from month nine to month twelve, another quarter. On the logarithm each quarter contributes an average gain of 0.06 times a quarter, or 0.015000, and a variance of 0.04 times a quarter, or 0.010000. Two quarters give an average gain of 0.030000 and a variance of 0.020000. The direct half year transition gives an average gain of 0.06 times half a year, or 0.030000, and a variance of 0.04 times half a year, or 0.020000. Identical.

The variances add. The spreads do not. Each quarter has a spread of 0.100000, and the half year spread is 0.141421, not 0.200000. Adding spreads is the single most common arithmetic error in this whole subject, and the reason it is wrong is that spread is the square root of variance and square roots do not add. The everyday version is weighing: weighing a sack in one go leaves a measurement error with some spread, and weighing it in two halves and adding the readings combines the two errors in variance, so the combined spread is smaller than the sum of the two spreads.

The wording of the composition invites one misreading in particular. The month nine reading on the locked path was Rs 93.74/-, and composing through month nine does not mean the process passes through that reading. Composing means totalling over every value month nine could have held, weighted by the chance of holding it. The realised path is one draw; the composition is over all of them.

What has to hold before a set of transitions is one process?

A table of transitions over one month is a table. Another table of transitions over three months is another table. Neither is a process. A set of tables becomes a process when composing the short ones reproduces the long ones, for every way of cutting every gap. The requirement is the consistency conditionThe requirement that composing short transitions reproduces the longer transition, for every way of cutting the gap., and consistency is the only thing standing between a set of plausible numbers and a model.

Consistency is not a nicety added on top of the transition rule; it is the property that makes the word process apply at all. A rule that fails it gives one answer for the same question depending on the route taken to it, and a quantity with two values is not a quantity.

For the standard process the condition holds by construction, and it is worth seeing why in the simplest possible terms. Cross the half year in one step: the logarithm gains 0.06 times 0.5, or 0.030000, and picks up variance 0.04 times 0.5, or 0.020000. Cross it in three steps: each step gains 0.06 times one sixth, or 0.010000, and picks up 0.04 times one sixth, or 0.006667. Three of each gives 0.030000 and 0.020000. Cross it in twelve steps and each contributes 0.002500 and 0.001667, and twelve of each gives 0.030000 and 0.020000 again.

The invariance is now stated, and the calculator below shows it holding. A requirement that has only been read is easy to believe and easy to forget; a curve that refuses to move while a control is dragged is hard to forget.

Why is the transition rule the whole content of a memoryless model?

One claim makes the transition rule worth a treatment of its own. Writing down where the process starts, together with the transition rule for every pair of states and every gap, finishes the specification. There is no further ingredient. Everything else anybody might ask of the model is derived from those two things rather than specified alongside them.

Once the transition rule is written down for every pair of states and every gap, nothing further about the model remains to be specified. The average at any horizon comes from integrating the ending value against the transition. The spread comes from integrating the squared deviation. The chance of any stretch of values comes from integrating over that stretch. The average of any function of the future value at all comes from integrating that function against the transition. The rule is not a summary of the model; the rule is the model.

The converse is the diagnostic. If a third thing has to be specified, something the transition rule could not carry, then the present state was not a sufficient summary after all and the model is not the kind of model it was taken to be. Needing to know how the process arrived at its current state, or how long it has been there, is exactly that situation, and it is a signal to widen what counts as the state rather than to bolt an extra rule onto the side.

Where does the transition rule turn up once a model is being solved?

Everywhere, and usually without being named. Three appearances are worth recognising. A reader who has met the transition rule only as a definition will otherwise not notice it wearing working clothes.

1. As the thing a future quantity is integrated against Any quantity that depends on the future value of the process is turned into a number today by integrating it against the transition density and discounting. That is what an expectation under a stated measure is, mechanically. The contract is treated purely as a function of the finishing value; nothing about what it pays is at issue here.
   On the standard process over one year under the risk-neutral measure, integrating the at-the-money contract against the transition density and discounting at 0.951229 returns Rs 10.45/-, from 10.450584, which is the closed form figure to six decimals.
2. As the unknown in a partial differential equation Read as a function of the ending state and the later time, the transition density satisfies one equation; read as a function of the starting state and the earlier time, it satisfies another. Kolmogorov carries both, forward and backward. Solving either equation is solving for the transition rule, whether or not the person solving it says so.
   Feynman and Kac carry the statement that links the two views: the solution of the equation is the expectation, and the expectation is the integral against the transition density.
3. As the weights on a lattice or a grid Any numerical scheme that steps a model forward carries a transition rule inside it, in the form of the weights attached to the moves out of each node. If those weights do not compose to the transition over the longer gap, the scheme is converging to the wrong thing, and no amount of refinement rescues it.
   The transition rule is what those weights are. How a scheme is built, refined and checked is set out under numerical pricing.

The recognisable pattern across all three is that the transition rule is the object being solved for, and the rest is machinery for solving it. A reader who holds that picture will find that a great deal of later material stops looking like unrelated techniques and starts looking like several routes to the same quantity.

What goes wrong when two tables are built without checking that they compose?

Numbers make the point. The size of the disagreement is the surprising part. Consider three states, named low, middle and high, and a monthly table where each state mostly stays put. Composing that table with itself three times gives the three month transitions the monthly table implies. Placed beside a separately written three month table with sensible round numbers, every row of both tables totals to one.

The middle row is the part that keeps the error alive. The middle row agrees to within 0.016, close enough that a reviewer scanning the two tables sees a match and moves on. The corners are 0.069 apart, and over a long horizon that difference compounds into a materially different answer. And every row of both tables totals to one, so the check most people actually run has told them nothing at all.

How does somebody reviewing a model rather than building one use this?

A reviewer does not usually get to see how a model was built. A set of outputs and a description arrive, and the question is whether the description holds together. The transition rule gives that reviewer a check that needs no access to anyone else's code and no agreement about what the right answer is.

The check is to compute one quantity two ways, over the same total gap, cut differently, and see whether the two agree. If a model produces a distribution over a year in one step and also in twelve monthly steps, those two have to match to within the tolerance the scheme claims. Where they do not, either the transition rule is inconsistent or the scheme is not solving the rule it says it is, and both are worth finding before anything is built on top.

The same arithmetic is why a spread computed over one period gets scaled by the square root of the number of periods rather than by the number of periods. Square root scaling is nothing more than the composition rule applied to variances, and it is correct exactly when the increments are independent and their variances are equal. Independent increments of equal variance are precisely the assumption on screen in the simulation above. When somebody scales a spread and the increments are not independent, the composition arithmetic has been applied where its condition fails, and the resulting number is neither the one period spread nor the multi period one.

The third use is the plainest. When a description of a model says the state carries everything relevant, ask what the transition rule is, in full, for every pair and every gap. If the answer needs a further ingredient, the description was wrong about the state, and it is better to widen the state deliberately than to discover the gap through a disagreement nobody can locate.

The standard process, its four parameters, its locked path and the three state table in the failure block are invented.
Educational material. Not advice on any investment, tax, budget or market position.