0.1 Autoregressive Processes
A time series \(\{X_t\}\) is an autoregressive process of order \(p\), \(AR(p)\), if \[ X_t = \phi_1 X_{t-1} + \dots + \phi_p X_{t-p} + Z_t, \qquad Z_t \sim [[white-noise|WN]](0,\sigma_Z^2). \] With the backshift operator, \(\phi(B)X_t = Z_t\) where \(\phi(B) = 1-\phi_1B-\dots-\phi_pB^p\).
An \(AR(p)\) process has a (unique) stationary and causal solution iff the roots of \(\phi(z)\) lie strictly outside the unit circle, i.e. \(\phi(z)\neq 0\) for \(|z|\leq 1\). In that case \(X_t = \psi(B)Z_t\) for an absolutely summable \(\psi(B) = \phi(B)^{-1}\) — an \(AR\) process is an infinite-order \(MA\) process (contrast with Moving Average Processes, where finite \(MA(q)\) is a limit of \(AR(\infty)\)).
All (causal, stationary) \(AR(p)\) models are automatically invertible. Unlike \(MA(q)\), the ACF of an \(AR(p)\) process decays (exponentially, possibly with oscillation) rather than cutting off, which is why the PACF — which does cut off at lag \(p\) — is used instead to identify the order.
0.2 Yule-Walker Equations
The ACF of an \(AR(p)\) satisfies the recursion \[ \rho_X(k) - \phi_1\rho_X(k-1) - \dots - \phi_p \rho_X(k-p) = 0, \qquad k \geq 1, \] see Yule-Walker Equations for the full system and sample-based estimation via the Durbin-Levinson Algorithm.
0.3 References
- Time Series Analysis
- [[pstat274-time-series-summary-notes]]