0.1 Stationary Processes
We can introduce stationary processes as a relaxation of independent identically distributed random variables, which is often too strong of an assumption.
A real-valued stochastic process \(\{ X(t),~t \geq 0 \}\) is strongly stationary, if the finite dimensional distribution is invariant under time-shifts, i.e. \[ \mathbb{P}(X(t_{1})\in B_{1}, \dots , X(t_{n})\in B_{n})=\mathbb{P}(X(t_{1}+h) \in B_{1}, \dots , X(t_{n}+h)\in B_{n}) \] for every choice of \((t_{1}, \dots, t_{n})\in[0,\infty)^n\), \(B_{1}, \dots, B_{n}\in \mathcal{B}(\mathbb{R})\), \(n \in \mathbb{N}\) and \(h>0\).
This is often challenging to show in practice and so a weaker form of stationarity is defined as follows.
A real-valued stochastic process \(\{ X(t),~t\geq 0 \}\) is weakly stationary (also known as covariance stationary or second-order stationary), if 1. \(\mu(t)=\mathbb{E}[X(t)]\) is constant on \([0,\infty)\); and 2. \(\text{Cov}(X(t_{1}), X(t_{2}))=\text{Cov}(X(t_{1}+h),X(t_{2}+h))\) for every \(t_{1}, t_{2}\geq 0\) and \(h>0\).
A process \(X(t)\) is weakly stationary if the covariance function between \(t_1\) and \(t_2\) depends only on the different \(t_{2}-t_{1}\). For efficiency, recall the autocovariance function of \(X(t)\) is given by \[ \gamma_{X}(t_{2}-t_{1})=\text{Cov}(X(t_{1}),X(t_{2})). \] Strong stationarity implies weak stationarity however, since the first and second moments do not determine the probability distribution uniquely, the converse is not necessarily true.
A stochastic process \(\{ X(t),~t \in \mathbb{R} \}\) is called a stationary Gaussian process if every finite-dimensional distribution of \(X(\cdot)\) is Gaussian and it is strict stationary.
If every finite-dimensional distribution is Gaussian, that is, jointly normal distributed, then weak stationarity implies strong stationarity, because the joint distribution of a finite number of normal random variables is uniquely determined by the variance-covariance matrix.
The autocorrelation function \(\rho(\cdot)\) of weakly stationary process \(X(\cdot)\) is defined as \[ \rho(t)=\frac{\text{Corr}(X(t),X(0))}{\sqrt{ \text{Var(X(t))}\text{Var}(X(0)) }}=\frac{\gamma(t)}{\gamma(0)};\quad t \in \mathbb{R}. \]