3. ACF and PACF — Model Identification Tools
🪄 Step 1: Intuition & Motivation
Core Idea: Once your time series is stationary, the next question is:
“How much does the past influence the present — and how far back does that influence go?”
That’s where ACF and PACF step in. They are like microscopes that reveal how strongly your data points are linked across time lags — helping you decide how many past values your model should remember.
Simple Analogy: Imagine you’re tossing a pebble into water. Each ripple affects nearby ripples.
- ACF measures the overall ripple effect — how each wave relates to all previous waves.
- PACF isolates only the direct ripple from the initial splash, ignoring indirect effects.
🌱 Step 2: Core Concept
Let’s see how ACF and PACF act as our investigative tools.
What’s Happening Under the Hood?
Autocorrelation Function (ACF): Measures how correlated a series is with its own lagged values. For example, ACF at lag 1 checks correlation between $X_t$ and $X_{t-1}$, ACF at lag 2 checks between $X_t$ and $X_{t-2}$, and so on.
A slow decay in ACF means the effect of past values persists — your series has a long memory.
Partial Autocorrelation Function (PACF): Removes “intermediate” influences. It tells you the pure correlation between $X_t$ and $X_{t-k}$, after removing all effects of shorter lags ($1$ to $k-1$).
Think of PACF as a “direct influence detector” — how much does lag $k$ matter by itself?
Why It Works This Way
When modeling time series (especially ARIMA), you must decide:
- How many AR (AutoRegressive) terms to include ($p$)?
- How many MA (Moving Average) terms to include ($q$)?
ACF and PACF plots guide that choice:
- AR model: PACF “cuts off” after lag $p$, ACF decays gradually.
- MA model: ACF “cuts off” after lag $q$, PACF decays gradually.
- ARMA model: Both decay gradually, no clear cut-off.
By observing where the correlation suddenly drops to zero, you identify the likely order of your model.
How It Fits in ML Thinking
In ML terms, you can think of this as feature relevance testing. Each lag ($X_{t-1}$, $X_{t-2}$, etc.) is like a potential feature — ACF and PACF tell you which ones truly matter.
This process is similar to selecting important predictors before fitting a regression model — except here, the predictors are your own past values.
📐 Step 3: Mathematical Foundation
Let’s now peek into the math behind these two heroes.
Autocorrelation Function (ACF)
- $\rho_k$: correlation coefficient for lag $k$
- $Cov(X_t, X_{t-k})$: how much current and past values move together
- $Var(X_t)$: total variability in the series
Interpretation:
- $\rho_k > 0$: past and present move in the same direction
- $\rho_k < 0$: they move oppositely
- $\rho_k \approx 0$: no relation at lag $k$
Partial Autocorrelation Function (PACF)
PACF at lag $k$ measures the correlation between $X_t$ and $X_{t-k}$ after removing effects of intermediate lags.
Formally, if you regress $X_t$ on its $k-1$ previous lags, the PACF is the correlation between the residuals of:
- regression of $X_t$ on lags $1$ to $k-1$, and
- regression of $X_{t-k}$ on lags $1$ to $k-1$.
PACF isolates the direct influence of a specific lag — like asking,
“How much does lag-3 matter once I’ve already accounted for lag-1 and lag-2?”
🧠 Step 4: Assumptions or Key Ideas
- The series must be stationary before interpreting ACF/PACF.
- Lags represent fixed time intervals (e.g., days, hours).
- Confidence intervals in the plots (often ±1.96/√n) show which lags are statistically significant.
- Correlations outside these bounds → meaningful dependencies.
⚖️ Step 5: Strengths, Limitations & Trade-offs
✅ Strengths
- Simple and visual — quick model identification.
- Helps decide AR/MA orders without complex optimization.
- Intuitive interpretation through decay and cut-off patterns.
⚠️ Limitations
- Sensitive to noise — small samples may show misleading spikes.
- Can’t capture nonlinear or seasonal dependencies.
- Misinterpretation is common when the series isn’t stationary.
🚧 Step 6: Common Misunderstandings
🚨 Common Misunderstandings (Click to Expand)
- “ACF and PACF peaks always mean correlation.” ❌ Small peaks may just be random noise. Check confidence intervals.
- “PACF is just a shortened ACF.” ❌ PACF removes indirect effects — they serve different purposes.
- “ACF decay means stationarity.” ❌ You can have a decaying ACF in a non-stationary series — always test first.
🧩 Step 7: Mini Summary
🧠 What You Learned: ACF measures how much a series is correlated with its past; PACF isolates the direct effect of each lag.
⚙️ How It Works: They help diagnose how far back dependencies extend — guiding AR (p) and MA (q) order selection.
🎯 Why It Matters: Without ACF/PACF, model order selection becomes guesswork; these plots turn intuition into strategy.