Stochastic Processes for Finance: Martingales, Stopping Times, and Markov Chains

A stochastic process is a collection of random variables indexed by time. A price at one date is a random variable; its evolution across dates is a process. The central questions are: what information do we have, what can we predict, and when can we stop?

Learning goals

Define a process, filtration, adapted process, and predictable strategy.
Verify four useful martingale constructions by conditional expectation.
Explain why trading and stopping require integrability conditions.
Derive fair and biased gambler’s ruin probabilities and expected durations.
Distinguish the Markov property from the martingale property.
Apply transition matrices to credit ratings and order flow.

1. Processes, paths, and information

A discrete-time process {Xₙ} has observations at n=0,1,2,…, while a continuous-time process {Xₜ} is indexed by real t≥0. Its state space might be real prices, integer order counts, or a finite set of credit ratings. Discrete time does not imply a discrete state space.

For a fixed n, Xₙ is a random variable across possible outcomes. For a fixed outcome ω, the sequence X₀(ω),X₁(ω),… is a sample path. A process model specifies joint distributions across dates, not just each date’s marginal distribution.

A filtration ℱ₀⊆ℱ₁⊆… represents the growing information available. For increments ξ₁,ξ₂,…, their natural filtration is ℱₙ=σ(ξ₁,…,ξₙ), the events determined by those observations. Additional information may require a larger filtration.

Concept	Meaning at step n	Trading interpretation
Adapted Xₙ	Xₙ is measurable with respect to ℱₙ	Observable by time n
Predictable Aₙ	Aₙ is measurable with respect to ℱₙ₋₁	Position chosen before the next increment
Integrable Xₙ	E[	Xₙ

Writing “Aₙ∈ℱₙ₋₁” is informal shorthand; a random variable is measurable with respect to an information set. Information matters: learning tomorrow’s coin toss today can destroy a fair-game property relative to the enlarged filtration.

2. Martingales: a conditional fair game

A process {Mₙ} is a martingale relative to a specified probability measure and filtration if it is adapted, integrable at every date, and:

E[Mₙ | ℱₙ₋₁]=Mₙ₋₁ almost surely for n≥1.

The tower property extends this to E[Mₙ | ℱₘ]=Mₘ for every m≤n. Conditional expected increments are zero. Paths can still fluctuate dramatically; the definition does not imply constant volatility, independent increments, or a flat realized path.

For an adapted integrable process, replacing equality with ≥ gives a submartingale, and with ≤ gives a supermartingale. These describe conditional upward or downward drift. The measure and information set must always be understood.

3. Four martingale constructions, proved

Example 1: the additive random walk

Let ξₙ be independent integrable increments with E[ξₙ]=0 and let S₀=0, Sₙ=Σₖ₌₁ⁿξₖ. Use their natural filtration. Then:

E[Sₙ | ℱₙ₋₁]=Sₙ₋₁+E[ξₙ | ℱₙ₋₁]=Sₙ₋₁.

Independence of the new increment from the past makes the last conditional mean zero. More generally, conditional mean-zero increments suffice even without independence.

For independent fair ±1 increments, the path 0,1,0,1,2 is possible, but its next expected value after reaching 2 is still 2. At a fixed date, E[Sₙ]=0 and Var(Sₙ)=n.

Example 2: squared walk minus accumulated variance

Assume additionally that the independent increments have common variance σ². Define:

Mₙ=Sₙ²−nσ².

Expanding the square gives:

E[Sₙ² | ℱₙ₋₁]=Sₙ₋₁²+2Sₙ₋₁E[ξₙ | ℱₙ₋₁]+E[ξₙ² | ℱₙ₋₁]=Sₙ₋₁²+σ².

Subtract nσ² to obtain Mₙ₋₁. Sₙ² itself has positive conditional drift; subtracting its variance growth removes that drift. With deterministic unequal step variances σₖ², subtract Σₖ≤nσₖ² instead.

Example 3: a multiplicative product

Let G₁,G₂,… be independent nonnegative factors with E[Gₙ]=1. For deterministic W₀≥0:

Wₙ=W₀∏ₖ₌₁ⁿGₖ.

Then E[Wₙ | ℱₙ₋₁]=Wₙ₋₁E[Gₙ | ℱₙ₋₁]=Wₙ₋₁. Nonnegativity and independence give finite E[Wₙ]=W₀. Conditional unit mean is a more general sufficient condition with appropriate integrability.

Example: each G is 1.1 or 0.9 with probability one half. Expected wealth is unchanged at every fixed date. Yet E[log G]=½log(1.1)+½log(0.9)=½log(0.99)<0. By the law of large numbers for these log factors, wealth tends to zero almost surely over an infinite horizon, despite constant finite-date expectations. Expected wealth and typical long-run compounded outcomes answer different questions; rare large paths preserve the mean.

Example 4: exponential / MGF martingale

Let ξₙ be i.i.d. increments, Sₙ their sum, and φ(λ)=E[exp(λξ₁)] finite for the chosen real λ. Define:

Zₙ=exp(λSₙ)/φ(λ)ⁿ.

The step distribution need not have zero mean. Since φ(λ)>0:

E[Zₙ | ℱₙ₋₁]=[exp(λSₙ₋₁)/φ(λ)ⁿ]E[exp(λξₙ) | ℱₙ₋₁]=Zₙ₋₁.

For a fair ±1 walk, φ(λ)=[exp(λ)+exp(−λ)]/2=cosh λ, so Zₙ=exp(λSₙ)/(cosh λ)ⁿ. Exponential normalization removes the exponential process’s drift. These constructions support tail bounds and changes of probability measure.

4. Predictable trading and martingale transforms

Suppose M is a martingale and Aₖ is chosen using ℱₖ₋₁. Define accumulated wealth or gains:

Vₙ=M₀+Σₖ₌₁ⁿAₖ(Mₖ−Mₖ₋₁).

If A is predictable and bounded, the products are integrable and V is a martingale. More generally, it suffices here to require each product Aₖ(Mₖ−Mₖ₋₁) to be integrable. The key step is:

E[Aₖ(Mₖ−Mₖ₋₁) | ℱₖ₋₁]=AₖE[Mₖ−Mₖ₋₁ | ℱₖ₋₁]=0.

Predictability alone does not guarantee integrability. For unbounded strategies or infinite horizons, extra admissibility conditions matter.

Example: hold one unit for the first fair coin flip, then hold two after an initial loss and zero after an initial win. Gains over two flips are:

Path	First gain	Second position	Second gain	Total
Up, up	1	0	0	1
Up, down	1	0	0	1
Down, up	−1	2	2	1
Down, down	−1	2	−2	−3

All paths have probability one quarter, so expected gain is (1+1+1−3)/4=0. A 75% win probability hides a larger loss on the remaining path.

In finance, the relevant no-arbitrage martingale is generally a discounted price or gains process under a pricing measure Q. Gains from trading it have zero Q-expectation under the stated conditions. This does not say all trading has zero real-world P-expected return, and zero expectation alone is not a complete no-arbitrage theorem without admissibility and measure assumptions.

5. Stopping times: decisions without hindsight

A random time τ taking values in {0,1,2,…,∞} is a stopping time if {τ≤n}∈ℱₙ for every n. By date n, you can determine whether stopping has already occurred.

The first time an observed price reaches a barrier is a stopping time.
A fixed deadline is a stopping time.
The date of the highest price over the next month generally is not: recognizing it requires future observations.

A bounded stopping time satisfies τ≤N almost surely for a fixed finite N. An almost surely finite stopping time only satisfies P(τ<∞)=1; it may have no deterministic bound or even have infinite expectation. These are different conditions.

6. Optional stopping: what is actually preserved?

For any stopping time τ and discrete-time martingale M, the process Mₙ∧τ, where n∧τ=min(n,τ), is a martingale at finite dates. One way to see this is:

Mₙ∧τ=M₀+Σₖ₌₁ⁿ1{τ≥k}(Mₖ−Mₖ₋₁).

The indicator is bounded and predictable because {τ≥k} is known at time k−1. Thus E[Mₙ∧τ]=E[M₀] for each finite n.

Passing from n∧τ to τ is an additional limiting step. E[M_τ]=E[M₀] holds under sufficient conditions such as:

τ is bounded; or
τ is almost surely finite and the family {Mₙ∧τ} is uniformly integrable, for example when |Mₙ∧τ| is bounded by an integrable random variable; or
E[τ]<∞ and the increments of M are uniformly bounded by a deterministic constant.

Uniform integrability prevents expectation from escaping into increasingly rare, extreme outcomes. Almost sure finiteness of τ alone is insufficient. See MIT’s optional stopping lecture for the theorem’s framework.

Why “keep playing until I win” does not beat the theorem

For a fair ±1 walk starting at zero, the first time τ of hitting +1 is almost surely finite and S_τ=1. Nevertheless E[S₀]=0. There is no contradiction: E[τ]=∞, and the stopped process is not uniformly integrable. Deep negative excursions invalidate the limiting expectation step.

Likewise, an indefinitely doubled betting strategy requires unbounded credit and exposes the gambler to arbitrarily large losses. Every finite-round version retains zero expected gain in a fair game; its high win frequency does not remove tail risk.

7. Fair gambler’s ruin: two martingales solve two questions

Let S₀=0 and increments be independent ±1 with equal probabilities. Fix positive integers A and B, and stop at τ=inf{n≥0:Sₙ=A or Sₙ=−B}. A gambler starting with B units has wealth B+Sₙ, so −B is ruin and +A is a profit target.

Why stopping is legitimate here

The interval is finite and steps have unit size, so there is no overshoot. From any interior point, A+B consecutive upward steps force exit, with probability at least 2⁻⁽ᴬ⁺ᴮ⁾ per block. This gives a geometric upper bound on survival over blocks, proving τ is almost surely finite and E[τ]<∞. Also |Sₙ∧τ|≤max(A,B), so the stopped walk is uniformly bounded.

Probability of reaching the upper target

Set h=P(S_τ=A). Optional stopping for S gives:

0=E[S_τ]=Ah−B(1−h).

Therefore h=B/(A+B) and P(ruin)=A/(A+B).

For A=3, B=2, the chance of reaching the profit target is 2/5=40%. The expected terminal gain remains 3×0.4−2×0.6=0. Winning less often is offset by the larger gain on a winning path.

Expected time to either boundary

For the fair unit-step walk, Sₙ²−n is a martingale. Apply stopping first at n∧τ:

E[n∧τ]=E[Sₙ∧τ²].

Use monotone convergence on the left and bounded convergence on the right:

E[τ]=E[S_τ²]=A²h+B²(1−h)=AB.

With A=3, B=2, the expected exit time is 6 steps. This is the time to either boundary, not the time to ruin conditional on ruin. For an arbitrary zero-mean walk, overshoots can change the formula; AB is the unit-step result.

8. Biased gambler’s ruin: remove drift exponentially

Now P(ξ=+1)=p and P(ξ=−1)=q=1−p, with 0<p<1 and p≠q. Keep the same initial position and barriers. The walk S is not a martingale because E[ξ]=p−q.

Seek a nontrivial λ such that φ(λ)=p exp(λ)+q exp(−λ)=1. Setting u=exp(λ) gives pu²−u+q=0, whose roots are 1 and q/p. Thus with r=q/p, the process r^Sₙ is a martingale.

Stopping is valid: τ has finite expectation by the same finite-interval block argument (now using a bound pᴬ⁺ᴮ>0), and r^Sₙ∧τ is bounded on the finite interval. Therefore:

1=E[r^S_τ]=h rᴬ+(1−h)r⁻ᴮ.

Solving yields:

h=(1−r⁻ᴮ)/(rᴬ−r⁻ᴮ)=(1−rᴮ)/(1−rᴬ⁺ᴮ), where r=q/p.

The equivalent form (rᴮ−1)/(rᴬ⁺ᴮ−1) is the one in the supplied outline. Reversing both signs leaves the ratio unchanged.

As p→1/2 and r→1, the limit is B/(A+B). If p=1 or p=0, treat the deterministic walk separately: upper-hit probability is 1 or 0, respectively.

Numerical check and expected duration

For p=0.6, q=0.4, A=3, B=2, r=2/3:

h=[1−(2/3)²]/[1−(2/3)⁵]=135/211≈63.98%.

The upward bias increases the upper-hit probability from 40% to about 64%. The process Sₙ−(p−q)n is also a martingale. Using its bounded increments and E[τ]<∞:

E[τ]=E[S_τ]/(p−q)=[(A+B)h−B]/(p−q).

Here this is 1265/211≈5.995 steps. The fair-case value AB follows by taking the appropriate limit, not by substituting p=q into a zero denominator.

9. Markov processes: the present state summarizes predictive information

A process X is Markov relative to its filtration if, for every relevant set C and s<t:

P(Xₜ∈C | ℱₛ)=P(Xₜ∈C | Xₛ) almost surely.

This restricts the entire conditional distribution, whereas the martingale property restricts a conditional mean. A time-homogeneous Markov chain has transition probabilities that do not depend on the date. Time-homogeneity is an extra assumption, not part of every Markov model.

Property	Question it answers	Example
Markov	Does the current state summarize the past for predicting the next state?	Biased random walk: yes
Martingale	Does the next conditional mean equal the current value?	Biased random walk: no
Both	Are both conditions satisfied?	Fair random walk under its natural filtration

A martingale need not be Markov. For example, start at zero, take two independent fair ±1 steps, then use subsequent independent mean-zero steps of size 1 if the first step was up and size 2 if it was down. At time 2, either history can produce the same current value zero, but the recorded first step determines the next distribution. The process remains a martingale, while its current scalar value alone does not summarize the relevant history.

10. Transition matrices and multi-step probabilities

Use the row-stochastic convention in this section:

Pᵢⱼ=P(Xₙ₊₁=j | Xₙ=i), with each row summing to one.

If πₙ is a row probability vector, then πₙ₊₁=πₙP and πₙ=π₀Pⁿ. The i,j entry of Pⁿ is the n-step transition probability. Matrix multiplication sums over intermediate states—the Chapman–Kolmogorov rule.

The earlier linear algebra lesson used column vectors and a column-stochastic matrix M; the equivalent matrix here is P=Mᵀ. Both conventions work when used consistently.

Order flow example

Let states be buy and sell, with:

          Next buy   Next sell
Buy          0.8        0.2
Sell         0.3        0.7

Starting from buy, the probability of a buy two steps later is 0.8×0.8+0.2×0.3=0.70. The stationary row vector satisfies π=πP** and sums to one, giving *π=[0.6,0.4]**. This positive transition matrix converges to that distribution from any initial state.

“Stationary transition matrix” can be ambiguous: a constant P means time-homogeneous transitions, while a stationary process requires an invariant initial distribution. Starting from buy, this chain is time-homogeneous but its marginal distribution is initially changing.

Credit-rating example with absorbing default

Use a simplified annual model with investment grade (IG), high yield (HY), and default (D):

            Next IG   Next HY   Next D
IG            0.90      0.08      0.02
HY            0.05      0.85      0.10
D             0         0         1

Starting in IG, the two-year default probability is:

0.90×0.02+0.08×0.10+0.02×1=0.046, or 4.6%.

The last term includes default in year one, which persists because D is absorbing. These are cumulative default probabilities under the model, not conditional second-year default probabilities. Default probability alone does not price a bond: recoveries, discounting, and risk premia also matter.

Real transition probabilities can depend on the business cycle, issuer characteristics, or order-book history. A richer state can sometimes capture that dependence, but a first-order homogeneous chain is an assumption to test. Physical transition estimates are not automatically risk-neutral pricing probabilities. See Cambridge’s Markov chain materials for the formal theory.

11. Practice questions

Independent zero-mean increments have variance 4. Which correction makes Sₙ² a martingale?
In the four-path trading example, what is the expected gain? Why is the win rate misleading?
Is the last time a stock touches 100 before a future deadline generally a stopping time?
For a fair walk from zero with barriers +4 and −3, find the upper-hit probability and expected exit time.
For p=0.6 and symmetric barriers +1 and −1, does the biased formula give the obvious one-step upper-hit probability?
Is an almost surely finite stopping time enough to guarantee E[M_τ]=E[M₀]?
Can a Markov process have nonzero conditional drift? Can a martingale have history-dependent volatility?
In the credit example, starting from HY, what is the two-year default probability?

Answers

Sₙ²−4n, under the stated independence and common-variance assumptions.
Zero. Three paths earn 1 and one loses 3; a high win rate does not specify expected return.
Generally no: future observations are required to identify the last touch. The first touch is a stopping time for the observed price filtration.
3/7≈42.86% and 4×3=12 steps.
Yes: (1−r)/(1−r²)=1/(1+r)=p=0.6 for r=q/p.
No. Additional conditions such as boundedness or uniform integrability are required.
Yes to both. Markov and martingale conditions answer different questions.
0.05×0.02+0.85×0.10+0.10×1=0.186, or 18.6%.