Probability Theory for Finance: Distributions, Limit Theorems, CAPM, and Martingales

Probability theory turns uncertainty into a model we can reason about. In finance, it helps describe defaults, measure portfolio risk, estimate average returns, and price contingent payoffs. The crucial habit is to separate what follows mathematically from assumptions from whether those assumptions describe a real market.

Learning goals

Describe discrete, continuous, and mixed random variables using distributions.
Calculate expectations, variances, and portfolio covariance effects.
Distinguish the law of large numbers from the central limit theorem.
Follow an MGF proof of the CLT and recognize its extra assumption.
Derive a Gaussian expected-utility pricing equation and connect it to CAPM.
Explain conditional expectation, information, and martingale pricing.

1. Outcomes, events, and random variables

A probability model consists of possible outcomes Ω, a collection of events, and probabilities P. An event is a set of outcomes; a random variable X assigns a numerical value to each outcome. For example, “the borrower defaults” is an event, while an indicator D taking value 1 on default and 0 otherwise is a random variable.

The probability measure matters. P usually denotes a physical or real-world model used for forecasting and risk measurement. Later, Q will denote a risk-neutral pricing measure. The same payoff can have different expectations under P and Q.

Type	Distribution description	Financial example
Discrete	Probability masses at a finite or countable set of values	Default indicator; a simplified set of possible policy-rate decisions
Continuous	Probabilities represented by integrating a density	A continuous approximation to returns or trade waiting times
Mixed	Both probability masses and a continuous part	A firm value with positive probability of zero and a continuous positive part

These are modeling choices. Actual quoted prices lie on tick grids; treating them as continuous is often an approximation. A mixed distribution can arise from X = min(Y,c) when a continuous Y is capped at c, creating an atom at the cap. Not every buyout automatically has that structure.

2. CDFs, densities, and moments

The cumulative distribution function (CDF) is F(x) = P(X ≤ x). It is nondecreasing, right-continuous, and runs from zero to one. It works for all random variables, including mixed ones.

For a discrete variable, the probability mass function is p(x) = P(X=x). For an absolutely continuous variable, a probability density function (PDF) f satisfies F(x) = ∫ from −∞ to x f(u) du. A density value is not a probability: probabilities come from areas. For a continuous X, P(X=x)=0 even at likely values. A density alone cannot account for the atoms of a mixed distribution.

The mean or expected value is a probability-weighted average:

Discrete: E[X] = Σₓ x p(x).
Continuous: E[X] = ∫ x f(x) dx over its support.
More generally: E[g(X)] averages the transformed payoff g(X), provided the relevant expectation exists.

For finite variance, define μ = E[X] and σ² = Var(X) = E[(X−μ)²] = E[X²]−μ². Standard deviation σ has the same units as X; variance has squared units.

Worked example: a default loss

A position loses 100 on default and zero otherwise. If P(D=1)=0.02 and L=100D, then:

E[L] = 100×0.02 = 2.

Var(L) = 100²×0.02×0.98 = 196; SD(L) = 14.

The expected loss is neither the most likely outcome nor a loss bound. This model gives either 0 or 100, never the mean 2.

Skewness and kurtosis

For σ>0, skewness is E[(X−μ)³]/σ³, when the third absolute moment is finite. It describes asymmetry; negative skewness is often associated with a longer downside tail.

Pearson kurtosis is E[(X−μ)⁴]/σ⁴, when finite. A normal distribution has kurtosis 3; excess kurtosis = kurtosis−3. A distribution with kurtosis above 3 is leptokurtic. High kurtosis reflects sensitivity to extreme deviations, but one kurtosis number does not establish a specific tail-decay law. Some heavy-tailed distributions have infinite variance or fourth moment, making these summaries undefined.

3. Expected call payoff from the survival function

A call payoff at maturity is (X−K)⁺ = max(X−K,0). For a nonnegative random variable Y:

E[Y] = ∫₀∞ P(Y>u) du.

Intuition: each realization Y is the length of the interval from zero to Y. Average those lengths by integrating the probability that each height u is exceeded. Applying this to Y=(X−K)⁺ gives:

E[(X−K)⁺] = ∫₀∞ P(X>K+u) du = ∫ₖ∞ [1−F(x)] dx.

The identity also holds for discrete and mixed distributions, not only continuous ones; it can equal infinity if the positive tail is not integrable. For a continuous distribution it is equivalent to ∫ₖ∞ (x−K)f(x) dx.

Example: let X be uniform on [80,120] and K=100. Between 100 and 120, 1−F(x)=(120−x)/40. Thus the expected payoff is ∫₁₀₀¹²⁰ (120−x)/40 dx = 5. This is also the area of a triangle with base 20 and height 0.5.

An expected payoff under P is not automatically today’s option price. Under appropriate no-arbitrage pricing assumptions with deterministic gross risk-free growth R, price = E_Q[payoff]/R. Both the probability measure and discounting must be specified.

4. Dependence and portfolio diversification

Independence means joint probabilities factor: P(X∈A,Y∈B)=P(X∈A)P(Y∈B) for all relevant sets. Zero covariance is a weaker statement. For example, if X is uniform on [−1,1] and Y=X², their covariance is zero by symmetry, but Y is completely determined by X.

For finite second moments:

Cov(X,Y) = E[(X−E[X])(Y−E[Y])].

When both standard deviations are positive, correlation is ρ = Cov(X,Y)/(σₓσᵧ) and lies between −1 and 1. Independence implies zero covariance when the moments exist; the converse generally fails. Joint Gaussianity is an important exception.

For a return vector X with mean μ, covariance matrix Σ, and fixed portfolio weights a:

Y=aᵀX; E[Y]=aᵀμ; Var(Y)=aᵀΣa.

For two assets with weights w and 1−w:

Var(Y)=w²σ₁²+(1−w)²σ₂²+2w(1−w)ρσ₁σ₂.

If both volatilities are 20%, w=0.5, and ρ=0, variance is 0.02 and volatility is 14.14%. At ρ=1, volatility remains 20%. Diversification reduces risk through imperfect co-movement; it does not require lower individual asset risk.

For n equally weighted assets with common variance σ² and common pairwise correlation ρ:

Portfolio variance = σ²[ρ+(1−ρ)/n].

With fixed positive ρ, common risk remains as n grows. The equal-correlation matrix must be valid: for n>1, −1/(n−1) ≤ ρ ≤ 1. Portfolio means remain weighted averages, so diversification alone does not create expected return.

5. Conditional expectation: update using information

Conditional expectation is an average given the information available. Suppose a loan’s default probability is 1% in a good economy and 5% in a bad economy, each occurring with probability one half. Then:

E[D] = 0.5×0.01 + 0.5×0.05 = 0.03.

After learning the economy is bad, E[D | bad]=0.05. This illustrates the tower property: E[E[X | information]]=E[X] for integrable X. It averages conditional forecasts back into the unconditional forecast.

A filtration ℱ₀ ⊆ ℱ₁ ⊆ … represents accumulating information through time. Conditional expectation given ℱₜ is itself a random variable, because it depends on which information is revealed.

6. Moment generating functions

The moment generating function (MGF) is Mₓ(t)=E[eᵗˣ]. If it is finite in an open interval around t=0, it uniquely determines the distribution and derivatives at zero recover moments: Mₓ′(0)=E[X], Mₓ″(0)=E[X²].

That existence condition is essential. Some useful distributions have no finite MGF on any neighborhood of zero. A lognormal random variable has all positive integer moments but E[eᵗˣ]=∞ for every t>0. A characteristic function E[eⁱᵗˣ] always exists and supports more general convergence arguments.

For X∼N(μ,σ²):

Mₓ(t)=exp(μt+½σ²t²).

This follows by completing the square in the normal density’s exponent. For independent X and Y, Mₓ₊ᵧ(t)=Mₓ(t)Mᵧ(t) wherever finite. Also Mₐₓ₊ᵦ(t)=eᵇᵗMₓ(at). The multiplication rule depends on independence, not merely zero correlation.

7. WLLN and CLT answer different questions

Assume X₁,…,Xₙ are independent and identically distributed (i.i.d.), with finite mean μ and finite variance σ². The sample mean is X̄ₙ=(1/n)ΣᵢXᵢ.

Weak law of large numbers: where does the average go?

For every ε>0:

P(|X̄ₙ−μ|>ε) → 0.

That is convergence in probability. A short proof uses Var(X̄ₙ)=σ²/n and Chebyshev’s inequality:

P(|X̄ₙ−μ|≥ε) ≤ σ²/(nε²) → 0.

Finite variance is sufficient for this proof; versions of the i.i.d. law of large numbers need only finite absolute first moment. The law does not say that each observation approaches μ or that a small sample is accurate.

Central limit theorem: how does the average fluctuate?

For σ>0:

√n(X̄ₙ−μ)/σ → N(0,1) in distribution.

Equivalently, (1/√n)Σᵢ(Xᵢ−μ) → N(0,σ²). Convergence in distribution concerns CDFs at continuity points of the limiting CDF; it does not mean sample paths approach a particular normal random variable.

The CLT suggests X̄ₙ ≈ N(μ,σ²/n) for sufficiently large n under the stated conditions. It approximates the distribution of an average or centered sum, not the distribution of the original observations.

Proof using MGFs, with its extra hypothesis

Set Yᵢ=Xᵢ−μ. Assume additionally that the common MGF is finite near zero. Its second-order expansion is:

Mᵧ(u)=1+½σ²u²+o(u²) as u→0.

For Zₙ=(1/√n)ΣᵢYᵢ, independence gives:

M_Zₙ(t)=[Mᵧ(t/√n)]ⁿ.

Taking logarithms for fixed t and using log(1+v)=v+o(v):

log M_Zₙ(t)=n log[1+σ²t²/(2n)+o(1/n)] → ½σ²t².

Therefore M_Zₙ(t)→exp(½σ²t²), the MGF of N(0,σ²). The MGF continuity theorem then gives convergence in distribution. The ordinary finite-variance i.i.d. CLT does not require an MGF; its more general proof uses characteristic functions. See MIT’s probability lecture materials for that approach.

Finance interpretation and limits

If independent daily returns had 2% volatility and a constant mean, the standard error of their average over 100 days would be 2%/√100=0.2 percentage points. If the mean and volatility assumptions were known, the CLT would suggest an approximate 95% range of μ ± 0.392 percentage points for that sample average. Estimating unknown parameters introduces additional uncertainty.

Actual returns can display volatility clustering, dependence, jumps, and unstable distributions. Infinite-variance tails invalidate the classical finite-variance CLT; dependence requires other limit theorems. Even when a central approximation works, extreme-tail probabilities can converge slowly. Normal approximations do not automatically improve for every financial purpose as the horizon grows. Diffusion approximations require process-level limit results and scaling assumptions beyond this scalar CLT. Also, simple returns compound multiplicatively; log returns add over time.

8. Stein’s lemma: turn a nonlinear covariance into a derivative

If Y and Z are jointly Gaussian, g is sufficiently regular, and the relevant expectations exist, then:

Cov(g(Y),Z)=E[g′(Y)] Cov(Y,Z).

For Y∼N(μ,σ²), σ>0, its density φ satisfies φ′(y)=−(y−μ)φ(y)/σ². Integration by parts, assuming the boundary term vanishes, gives:

E[(Y−μ)g(Y)] = σ²E[g′(Y)].

For jointly Gaussian Y,Z, their conditional mean is linear in Y. Applying that conditional-mean formula and the identity above gives the two-variable result. Smoothness and integrability conditions matter; joint normality is not interchangeable with normal marginal distributions alone.

9. From expected utility to an asset price

Consider one period and a risk-free asset with gross growth R=1+r_f. A risky asset costs p today and pays the random amount F at maturity. An investor with initial wealth w₀ chooses holdings h; borrowing and lending are available at R. Terminal wealth is:

W=R(w₀−hp)+hF = Rw₀+h(F−Rp).

Assume an interior optimum, differentiable increasing concave utility U, and conditions allowing differentiation under the expectation. Maximizing E[U(W)] gives:

E[U′(W)(F−Rp)]=0.

Hence Rp=E[F]+Cov(U′(W),F)/E[U′(W)]. If W and F are jointly Gaussian and Stein’s conditions hold:

p = {E[F]+[E[U″(W)]/E[U′(W)]]Cov(W,F)}/R.

Since U′>0 and U″<0, a payoff positively related to wealth receives a risk discount. A payoff arriving when wealth is low can instead provide insurance value. This equation is initially a price–demand relation; calling it an equilibrium price also requires market clearing.

CARA utility and the single-asset risk penalty

For U(W)=−exp(−aW) with a>0, absolute risk aversion is constant: −U″/U′=a. With Gaussian payoffs and deterministic non-risky wealth:

p=[E[F]−a Cov(W,F)]/R = [E[F]−ah Var(F)]/R.

The penalty uses payoff variance and the number of units held, not simply a return variance without rescaling. In a one-investor equilibrium with supply H, set h=H. Multiple investors require aggregating their demands and risk tolerances.

Example: E[F]=110, Var(F)=400, a=0.01 per wealth unit, H=1, and R=1.05 give p=(110−4)/1.05≈100.9524. Discounting expected payoff alone would give 104.7619. The risk penalty reduces today’s price by 3.8095. These values depend on this Gaussian CARA model.

10. Multi-asset equilibrium and the CAPM

To make the derivation explicit, use a representative-investor one-period equilibrium with unrestricted interior portfolio choice, a common risk-free rate, jointly Gaussian risky payoffs, and no additional random nontraded wealth. The representative investor holds the market supply at equilibrium; more general CAPM derivations impose their own aggregation and common-belief assumptions.

Apply the preceding first-order condition to each asset i, with terminal payoff Fᵢ and price pᵢ. Define κ=−E[U″(W)]/E[U′(W)]>0:

E[Fᵢ]−Rpᵢ=κ Cov(W,Fᵢ).

Let Vₘ be today’s value of aggregate risky holdings and rₘ their simple total return. At market clearing, W differs from Vₘ(1+rₘ) only by a deterministic risk-free amount. Since Fᵢ=pᵢ(1+rᵢ):

E[rᵢ]−r_f=κVₘ Cov(rₘ,rᵢ).

Applying the same equation to the market portfolio gives:

E[rₘ]−r_f=κVₘ Var(rₘ).

For positive market variance, eliminate κVₘ:

E[rᵢ]=r_f+βᵢ(E[rₘ]−r_f), where βᵢ=Cov(rᵢ,rₘ)/Var(rₘ).

This is the Security Market Line. Beta measures exposure to market co-movement, not total standalone volatility. Asset-specific risk does not command a separate expected-return premium in this model.

If r_f=3%, E[rₘ]=8%, and βᵢ=1.2, CAPM predicts E[rᵢ]=3%+1.2×5%=9%. This is a model-implied expected return, not a promised realized return. Constraints, heterogeneous information, nontraded income, and non-Gaussian risks can change the derivation or its empirical performance.

11. Martingales: a conditional fair-game property

A process {Mₜ} is a martingale relative to information {ℱₜ} and a probability measure if:

It is adapted: Mₜ is known given ℱₜ.
It is integrable: E[|Mₜ|]<∞ at each time.
E[Mₜ₊₁ | ℱₜ]=Mₜ almost surely.

This says the expected next increment, conditional on current information, is zero. It does not mean paths are constant, future changes are independent, or future volatility is constant.

Example: let ξ₁,ξ₂,… be independent fair ±1 outcomes. Then Mₙ=M₀+Σᵢ₌₁ⁿξᵢ is a martingale under its natural information history. Its expected change is zero while its variance grows with n. The tower property also shows that Mₜ=E[H | ℱₜ] is a martingale for any integrable terminal payoff H.

See MIT’s conditional expectation and martingale lecture for the formal framework.

Pricing under a risk-neutral measure

In a finite discrete-time frictionless market, no-arbitrage is associated with an equivalent martingale measure under suitable trading assumptions. With bank-account value Bₜ, a no-dividend traded asset’s discounted price Sₜ/Bₜ is a martingale under such a Q. Dividend-paying assets require the corresponding discounted gains process; the unadjusted price need not be a martingale. Continuous-time versions require stronger technical conditions.

For an attainable payoff H at T, its price is:

Vₜ=Bₜ E_Q[H/B_T | ℱₜ].

With deterministic rates, the discount factor can be moved outside the conditional expectation. Complete markets give a unique equivalent martingale measure in the standard finite setting; incomplete markets generally allow several. Replicable claims retain a unique no-arbitrage price even in an incomplete market.

Return to the earlier two-state stock example: S₀=100, S₁ is 120 or 80, and R=1.05. Under Q(up)=0.625, E_Q[S₁]/1.05=100. Under a physical model P(up)=0.7, E_P[S₁]/1.05≈102.8571 instead. Martingale behavior is measure-dependent. Risk-neutral probabilities are pricing tools, not necessarily forecasts.

12. Practice questions

A bond loses 60 with probability 5% and otherwise loses zero. Find expected loss and variance.
X is uniform on [0,2]. Find E[(X−1)⁺] using the survival function.
Two equally weighted assets each have 10% volatility and correlation 0.5. Find portfolio volatility.
What happens to an i.i.d. sample mean’s standard deviation when sample size increases from 25 to 100?
Can a variable have finite variance but no MGF near zero? Does the classical i.i.d. CLT still apply?
With r_f=2%, expected market return 7%, and beta 0.8, what is the CAPM expected return?
Does E[Mₜ₊₁]=E[Mₜ] alone establish the martingale property?
Why is a physical expected option payoff divided by the risk-free growth factor not generally the option price?

Answers

E[L]=3; Var(L)=60²×0.05×0.95=171.
∫₁²(1−x/2)dx=0.25.
Variance is 0.0075, so volatility is 8.66%.
It halves, because σ/√100 is half of σ/√25.
Yes; a lognormal variable is one example. Yes, for i.i.d. observations with finite positive variance, but the MGF proof above is unavailable.
2%+0.8×(7%−2%)=6%.
No. Conditional expectation given the full information set must satisfy the identity, alongside adaptation and integrability.
Pricing generally incorporates risk adjustments. No-arbitrage pricing uses an appropriate pricing measure or stochastic discount factor, rather than assuming physical expected payoffs require no risk adjustment.