Time Series Analysis: Likelihood, Unit Roots, Seasonality, and Volatility Forecasts

A time-series model can fit a financial series well while answering the wrong forecasting question. A rolling volatility measure, for example, is partly predictable because most of its observations are already known. Before choosing ARIMA orders, define what is stationary, how the model is estimated, and which future quantity the forecast is meant to predict.

Learning goals

Distinguish deterministic detrending from differencing an integrated process.
Explain conditional estimation and state-space Gaussian likelihood.
Compare AIC, AICc, BIC, and Hannan–Quinn on a consistent basis.
Interpret DF, ADF, PP, ERS, and KPSS tests with the correct null distributions.
Recognize dependence introduced by overlapping volatility windows.
Evaluate seasonal models and OHLC volatility forecasts against meaningful baselines.

1. Differencing changes the model and the question

Let LXₜ=Xₜ₋₁. First and second differences are:

ΔXₜ=(1−L)Xₜ=Xₜ−Xₜ₋₁,

Δ²Xₜ=(1−2L+L²)Xₜ=Xₜ−2Xₜ₋₁+Xₜ₋₂.

For Xₜ=a+bt, ΔXₜ=b and Δ²Xₜ=0. For Xₜ=at², ΔXₜ=a(2t−1) and Δ²Xₜ=2a. Thus first differencing removes a linear trend’s time variation, and second differencing removes a quadratic trend’s time variation; a constant can remain. An additional difference annihilates that constant.

Deterministic trend plus stationary deviations

Suppose Xₜ=a+bt+uₜ, where uₜ is a stationary causal AR(1) with coefficient abs(φ)<1. Subtracting a+bt yields stationary deviations. Differencing also yields ΔXₜ=b+Δuₜ, which is stationary, but it changes the error structure.

Indeed, if (1−φL)uₜ=ηₜ, then (1−φL)Δuₜ=(1−L)ηₜ. Unnecessary differencing has inserted an MA factor with a unit-circle root. Stationarity of the result does not mean differencing was the most efficient or interpretable approach.

Stochastic trend: an integrated process

For Xₜ=Xₜ₋₁+ηₜ, with independent zero-mean increments of variance σ² and deterministic X₀:

Xₜ=X₀+Σⱼ₌₁ᵗηⱼ, and Var(Xₜ)=tσ².

The level has no constant unconditional variance, while ΔXₜ=ηₜ is stationary. This is an I(1) example: one difference gives an I(0) process under the usual integration terminology. If X₀ is random and independent, its variance is an additional term. A general I(1) process can have dependent stationary increments, not just independent random-walk steps.

Deterministic-trend and random-walk forecasts behave differently. A shock to a trend-stationary deviation decays; a random-walk shock permanently changes the level forecast. Near-unit-root models can make the distinction difficult in finite samples.

2. ARIMA: specify what is differenced

An ARIMA(p,d,q) model assumes Wₜ=ΔᵈXₜ follows an ARMA model:

φ(L)(Wₜ−μ_W)=θ(L)ηₜ,

with φ(L)=1−φ₁L−…−φₚLᵖ and θ(L)=1+θ₁L+…+θ_qLᑫ under this sign convention. Causal stationarity and invertibility are separate root conditions for the differenced model.

Choose d using the data-generating question, plots, tests, and economic context. Too little differencing can leave a stochastic trend; too much can amplify noise and create artificial negative autocorrelation. A trend term, drift, and mean have different interpretations once differences are applied. Forecasting: Principles and Practice provides model-building context for these choices.

3. Estimation: conditional versus full likelihood

Weak white noise means uncorrelated, zero-mean, constant-variance shocks. It does not by itself imply independent Gaussian shocks or an exact Gaussian likelihood. Gaussian estimation is exact under the specified joint Gaussian model; with non-Gaussian data it can sometimes be used as quasi-maximum likelihood, requiring additional consistency and inference conditions.

Conditional likelihood and conditional sum of squares

A conditional approach treats initial observations as given and initializes unavailable presample innovations, often at zero, to build recursive prediction errors. Minimizing their squared sum is conditional sum of squares (CSS). Under a particular conditional Gaussian setup, it is closely related to conditional maximum likelihood.

This approximation can differ from the full likelihood, especially for short samples, strong persistence, or large initial effects. Precisely which observations are conditioned on or dropped is implementation-dependent.

The lecture calls this “limited information maximum likelihood.” For ARMA work, conditional maximum likelihood or CSS, when that is the actual criterion, is clearer. LIML commonly names a different instrumental-variables estimator in econometrics and should not be treated as a universal ARIMA method label.

Full Gaussian likelihood and the Kalman filter

An ARMA model can be written as a linear state-space system. For a stationary Gaussian model, initialize the hidden state with its stationary distribution, then recursively compute the one-step prediction error vₜ=Xₜ−E[Xₜ | past] and variance Fₜ. The prediction-error log-likelihood is:

ℓ=−½Σₜ[log(2π)+log Fₜ+vₜ²/Fₜ].

The Kalman filter supplies vₜ and Fₜ efficiently without directly inverting a large observation covariance matrix. The prediction-error variances and their log terms matter; minimizing an unweighted residual sum is not automatically the same calculation.

Integrated states do not have a stationary level distribution. Exact or approximate diffuse initialization is used for those states, with appropriate likelihood treatment. Calling a method “full information” does not mean assigning a nonexistent stationary distribution to a random walk’s initial level. See the official R Kalman-filter documentation and ARIMA documentation for implementation conventions.

Practical estimation checks

Inspect convergence, parameter uncertainty, root locations, and sensitivity to initialization. Near-canceling AR and MA factors can make parameters weakly identified. Exact likelihood does not imply exact optimization, a correct economic model, or reliable Gaussian prediction intervals under heavy tails and volatility clustering.

4. AIC, AICc, BIC, and Hannan–Quinn

Let ℓ̂ be a consistently defined maximized log-likelihood, n the relevant sample size, and k the number of estimated parameters. General criteria are:

Criterion	Formula	Complexity penalty
AIC	−2ℓ̂+2k	2 per parameter
BIC	−2ℓ̂+k log n	log n per parameter
Hannan–Quinn, HQ	−2ℓ̂+2k log(log n)	2 log(log n) per parameter
Common AICc form	AIC+2k(k+1)/(n−k−1)	Extra small-sample correction, requiring n>k+1

Count AR, MA, seasonal, mean/drift, and external-regressor parameters as applicable, plus innovation variance under the likelihood convention. Expanded seasonal cross-lag terms can be implied by polynomial multiplication rather than separate free coefficients. Use the AICc formula and effective sample convention appropriate to the model and implementation.

The lecture’s log(σ̃²)+penalty/n forms arise after normalizing a simplified Gaussian likelihood and dropping common terms. They can rank candidates equivalently under matching assumptions, but p+q is not always the full k, and the state-space likelihood includes initialization and varying prediction-error terms. Prefer criteria derived from the actual comparable fitted likelihood.

Worked comparison

For the same n=200 observations, suppose a smaller model has −2ℓ̂=500 and k=3, while a larger model has −2ℓ̂=494 and k=5:

Candidate	AIC	AICc	BIC	HQ
Smaller	506	506.122	515.895	510.004
Larger	504	504.309	520.492	510.674

AIC and AICc prefer the larger candidate, while BIC and HQ prefer the smaller one. These are hypothetical fit summaries, not an empirical selection result. Lower values are preferred among comparable candidates; no criterion proves that a model is true or forecast-optimal.

Compare identical observations, response definitions, likelihood conventions, and compatible initialization. Standard ARIMA criteria usually select p and q after the differencing structure is chosen; naive comparisons across different d or D can involve different data or likelihood treatments. Also evaluate forecasts chronologically rather than repeatedly selecting on the final test period.

5. Dickey–Fuller: the t-shaped statistic is not Student-t

Start with Xₜ=φXₜ₋₁+ηₜ, with deterministic terms chosen explicitly. The unit-root null is H₀:φ=1; the usual left-sided alternative is stationary behavior with φ<1 within the specified stable model.

Subtract Xₜ₋₁ to write:

ΔXₜ=γXₜ₋₁+ηₜ, where γ=φ−1.

The statistic is τ_DF=γ̂/SE(γ̂)=(φ̂−1)/SE(φ̂). It resembles a t statistic algebraically, but under a unit root the usual Student-t reference law is wrong.

Correct convergence rates

For a fixed stationary causal AR(1) with abs(φ)<1 and appropriate innovations, √T(φ̂−φ) has a normal limit with variance 1−φ² in the simple setup. That result does not carry over unchanged at φ=1.

Under the simplest no-intercept, no-trend unit-root model with suitable i.i.d. shocks and a negligible initial condition, T(φ̂−1) has a nonstandard limit involving Brownian motion W:

T(φ̂−1) → [∫₀¹W(r)dW(r)]/[∫₀¹W(r)²dr].

The DF t statistic itself, without multiplying it by T, has limit:

τ_DF → [∫₀¹W(r)dW(r)]/√[∫₀¹W(r)²dr].

Thus the lecture’s claim that T times the t statistic has the DF law confuses two different scalings. The coefficient estimator has a T rate at the unit root; the standardized statistic is already order one. Adding an intercept or trend changes the relevant limiting distributions and critical values.

If φ̂=0.98 with reported SE=0.01, the test statistic is −2. Do not apply an ordinary −1.96 cutoff or conclude stationarity simply because 0.98 is below one. Use the unit-root procedure’s critical values for the chosen deterministic terms and lag specification.

The augmented Dickey–Fuller regression is commonly written:

ΔXₜ=a+bt+γXₜ₋₁+Σⱼ₌₁ᴷcⱼΔXₜ₋ⱼ+uₜ.

Include a, bt, or neither according to the intended specification, rather than mechanically including every term. Lagged differences address short-run dependence; too few can leave correlated errors, while too many reduce effective information. ADF tests γ=0 using its appropriate nonstandard reference law.

Procedure	Null / main idea	Qualification
DF / ADF	Unit root; ADF adds lagged differences	Deterministic terms and lag choice affect inference
Phillips–Perron, PP	Unit root with a long-run variance correction	Bandwidth, kernel, and weak-dependence assumptions matter
Elliott–Rothenberg–Stock, ERS	Unit-root procedures using efficient detrending or related constructions	Not merely another name for adding ADF lags
KPSS	Level or trend stationarity, depending on specification	Long-run variance estimation and breaks affect results

For ADF implementation choices and returned critical values, see the statsmodels ADF documentation. Tests provide evidence within their models; failure to reject a unit root is not proof of one, and rejection does not establish every form of stationarity.

Reading ADF and KPSS together

ADF unit-root null	KPSS stationarity null	Possible reading
Rejected	Not rejected	Evidence compatible with stationarity under the chosen specifications
Not rejected	Rejected	Evidence compatible with a unit root or another nonstationary feature
Rejected	Rejected	Revisit breaks, deterministic terms, dynamics, and test assumptions
Not rejected	Not rejected	Inconclusive; tests may lack power to discriminate

Match level-stationary versus trend-stationary specifications before comparing conclusions. Structural breaks and near-unit roots can complicate all four outcomes. Neither a differencing decision nor an investment strategy should rest on one unexamined p-value.

7. Seasonal ARIMA: multiplicative lag structure

A seasonal model is ARIMA(p,d,q)(P,D,Q)_s. Using the same plus-sign convention for MA terms:

φ(L)Φ(Lˢ)(1−L)ᵈ(1−Lˢ)ᴰXₜ=θ(L)Θ(Lˢ)ηₜ, with suitable deterministic terms if included.

For example, multiplying (1−φL)(1−ΦLˢ) produces 1−φL−ΦLˢ+φΦLˢ⁺¹. The cross-lag coefficient is constrained by the multiplicative structure, not a separate arbitrary parameter.

A seasonal period s=12 may represent monthly observations across years. With daily trading data, s=21 means a fixed 21-observation lag, not an exact calendar month: real months have varying trading-day counts. A recurring calendar effect and a rolling-window construction are different reasons for dependence at that lag.

8. Why rolling volatility creates its own memory

Let rₜ be a zero-mean return and qₜ=rₜ². Consider a simple rolling second-moment estimate:

vₜ=(1/m)Σⱼ₌₀ᵐ⁻¹qₜ₋ⱼ.

For this derivation only, assume qₜ are i.i.d. with finite variance σ_q². Then:

Var(vₜ)=σ_q²/m,

Cov(vₜ,vₜ₋ₖ)=(m−k)σ_q²/m² for 0≤k<m, and zero for k≥m.

Therefore ρ_v(k)=(m−k)/m up to lag m−1. For m=21, lag-one correlation is 20/21≈0.9524, even though the underlying squared returns are independent. Twenty of the twenty-one observations are shared between adjacent windows.

Differencing reveals the entry–exit boundary

The exact identity is:

Δvₜ=(qₜ−qₜ₋ₘ)/m.

Its variance is 2σ_q²/m², and its autocovariance at lag m is −σ_q²/m², giving:

ρ_Δv(m)=−1/2.

Thus a negative spike at lag 21 in a differenced 21-day rolling statistic can be caused mechanically by entry and exit from the window. A seasonal MA term can represent that kind of dependence; it is not proof of a monthly economic cycle. The exact −0.5 result is for this simplified second-moment average, not every demeaned sample variance or square-root volatility estimator.

This is also an overdifferencing warning: with i.i.d. q, v is already stationary and has a finite moving-average structure. Differencing it inserts boundary-unit-root factors. Modeling the estimator’s construction may be preferable to blindly adding ordinary and seasonal differences.

9. Close-to-close versus Yang–Zhang volatility

Close-to-close volatility uses daily close returns. A typical m-day daily variance estimate is Σ(rᵢ−r̄)²/(m−1). Annualized volatility is often written √(A×daily variance) using an assumed A trading days per year, but scaling and interpretation depend on the target and dependence assumptions.

OHLC estimators use open, high, low, and close information to estimate within-period variation more efficiently under specified price-process assumptions. Yang–Zhang combines overnight and intraday components and is designed to accommodate opening jumps and avoid dependence on a constant drift in its continuous-limit framework. Its original contribution concerns estimation properties, not a theorem that every Yang–Zhang time series must be easier to forecast.

For a multi-day window, a common representation of its variance estimate is:

v_YZ=v_overnight+k v_open-to-close+(1−k)v_RS,

where v_RS is a Rogers–Satchell range-based component and k follows the estimator’s finite-window convention. Consistent trading-session boundaries, corporate-action adjustments, and reliable highs/lows are essential. See Yang and Zhang’s original paper record for its stated estimation properties.

An estimator with less measurement noise can appear more predictable, but its target may also differ. Higher forecasting accuracy against its own smoothed estimate is not automatically better forecasting of latent volatility or future trading losses. Compare economically aligned targets and consider measurement error.

10. Interpreting the supplied seasonal case study

The lecture describes residual spikes around lag 21 when forecasting 21-day rolling volatility, and reports that ARIMA(2,1,3)(2,0,1)_21 removed that feature and produced residuals consistent with white noise. It also reports lower MAPE for a high-efficiency OHLC measure such as Yang–Zhang than for close-to-close volatility.

These are reported lecture findings. The asset, sample, forecast horizon, train/test split, volatility construction, fitted likelihood, and diagnostic outputs were not provided, so no exact fit or performance result is reproduced here. That seasonal specification is an example to evaluate, not a recommended default for every asset.

A disappearing lag-21 residual spike is useful evidence about one feature. Check the whole residual ACF, suitable portmanteau tests, squared-residual dependence, roots, parameter uncertainty, and held-out forecasts. A large parameter count or near-canceling factors can produce an attractive in-sample diagnostic without a stable forecast model.

11. Forecast the intended target, not merely the overlap

Suppose the forecast is for tomorrow’s trailing 21-day variance vₜ₊₁. At time t, twenty of its squared returns are already known. Under the i.i.d. example with known μ_q=E[q]:

E[vₜ₊₁ | information at t]=[qₜ+qₜ₋₁+…+qₜ₋₁₉+μ_q]/21.

Only one term is genuinely unknown. Its conditional forecast-error variance is σ_q²/21². By contrast, the average of the next twenty-one future squared returns has conditional variance σ_q²/21, under the same model. The latter is a different, harder target with no shared known window.

This distinction explains why a rolling-volatility forecast can look remarkably accurate without implying an equally accurate forecast of future, nonoverlapping realized risk. Useful baselines include the last rolling value and an explicit overlap-aware forecast. A sophisticated seasonal model should be compared against both where appropriate.

Metrics and their limits

For positive actual targets yₜ and forecasts ŷₜ, MAPE=(100/N)Σ|yₜ−ŷₜ|/|yₜ|. It is undefined at zero and can be dominated by very small actual values. For example, forecasting 0.2% when actual volatility is 0.1% has 100% absolute percentage error despite a 0.1 percentage-point absolute miss.

Also report MAE and RMSE in clearly stated daily or annualized units. Variance and volatility are not interchangeable targets. For strictly positive variance forecasts, a QLIKE-style score can use log(ŷ_v)+y_v/ŷ_v, up to terms independent of the forecast; its interpretation depends on the target and measurement assumptions. Negative variance forecasts make that score invalid and indicate a modeling problem to address.

Log-volatility modeling can ensure positive back-transformed forecasts, but exp(E[log v]) is not generally E[v]. A Gaussian log forecast with conditional variance s² needs the appropriate exp(mean+s²/2) adjustment to predict its conditional mean under that model. Prediction uncertainty matters alongside point accuracy.

12. A defensible estimation and evaluation sequence

Define the target, horizon, units, return adjustments, trading calendar, and whether volatility is trailing or forward-looking.
Inspect the series and its construction. Separate trends, breaks, rolling-window overlap, calendar effects, and changing volatility.
Choose deterministic terms and differencing using evidence and context; use unit-root and stationarity tests with their correct nulls.
Fit a modest grid with comparable likelihood methods. Check convergence, roots, parameter counts, and sensitivity to initial conditions.
Compare information criteria only where valid, then inspect residual dependence and forecast plausibility.
Evaluate rolling-origin forecasts without future observations in fitting or preprocessing. For forward-looking labels, ensure every training label would already have been observed at the forecast origin.
Account for overlapping forecast errors when quantifying performance differences; observations are not automatically independent. Use appropriate dependence-aware inference or a justified block bootstrap.
Compare with simple and overlap-aware baselines on the same targets and dates. Record all model-selection choices rather than retuning on the final test sample.

The relevant achievement is forecasting new information better, not merely reproducing a mechanically smooth series.

13. Practice questions

What are Δ(a+bt) and Δ²(at²)? Do both equal zero?
For a random walk with X₀=0 and innovation variance 9, what is Var(X₄₀)?
Does weak white noise alone justify an exact Gaussian likelihood?
Which quantity has T scaling under the simplest unit-root null: φ̂−1 or the DF t statistic?
What are the null hypotheses of ADF and KPSS?
Under the i.i.d. squared-return example, what is lag-one correlation of a 21-day rolling second moment? What is lag-21 correlation after differencing it?
Does a seasonal lag of 21 necessarily establish a calendar-month economic effect?
Why can lower MAPE for a forecast of trailing Yang–Zhang volatility fail to establish better prediction of the next month’s realized variance?

Answers

b and 2a. They are constants, not generally zero.
40×9=360.
No. Exact Gaussian likelihood requires the specified joint Gaussian model; quasi-likelihood needs separate justification.
φ̂−1. T(φ̂−1) has a nonstandard limit; the DF t statistic already has an order-one nonstandard limit without multiplication by T.
ADF tests a unit root. KPSS tests level or trend stationarity, depending on its specification.
20/21≈0.9524, and −0.5, respectively, under the stated simplified assumptions.
No. It may reflect a fixed rolling window rather than calendar seasonality.
The estimators, targets, smoothing, and overlap can differ. Future nonoverlapping variance is a distinct target requiring matched out-of-sample evaluation.