0.1 Best Linear Predictors
Consider a stationary time series \(\{ X_{n} \}_{n=1}^\infty\) with known mean \(\mu\) and autocovariance function \(\gamma\). We wish to find the linear combination of observations \(1, X_{1}, \dots, X_{n}\) that forecasts \(X_{{n+h}}\) with the minimum mean square error, denoted \(P_{n}X_{n+h}\). The solution has the form \[ P_{n}X_{n+h}=a_{0}+a_{1}X_{n}+\dots+a_{n}X_{1}, \] where the coefficients \(a_{0}, a_{1}, \dots, a_{n}\) minimize the error \[ S(a_{0}, a_{1}, \dots, a_{n})=\mathbb{E}[(X_{n+h}-a_{0}-a_{1}X_{n}-\dots-a_{n}X_{1})^2]. \] Evaluating the derivatives gives the equivalent system \[ \begin{align} & \mathbb{E}\left[ X_{n+h}-a_{0}-\sum_{i=1}^na_{i}X_{n+1-i} \right]=0, \\ & \mathbb{E}\left[ \left( X_{n+h}-a_{0}-\sum_{i=1}^n a_{i}X_{n+1-j} \right)X_{n+1-j} \right]=0,\qquad j =1,\dots,n. \end{align} \] These equations can be written as \[ \begin{align} a_{0} & = \mu\left( 1-\sum_{i=1}^n a_{i}\right), \\ \Gamma_{n}\vec{a}_{n} & =\vec{\gamma}_{n}(h), \end{align} \] where: (i) \(\vec{a}_{n}=\begin{pmatrix}a_{1} & \dots& a_{n}\end{pmatrix}^T\), \(\Gamma_{n}:=[\gamma(i-j)]_{i,j=1}^n\) and \(\vec{\gamma}_{n}(h)=\begin{pmatrix}\gamma(h) & \dots & \gamma(h+n-1)\end{pmatrix}^T\).
0.2 References
- [[chap15-17-pdf]]