The law of the iterated logarithm describes the magnitude of the fluctuations of a random walk.
Let \(\{ Y_{n} \}\) be independent, identically distributed random variables with \(\mathbb{E}Y_{n}=0\) and \(\operatorname{Var}(Y_{n})=1\). Define \(S_{n}=Y_{1}+\dots+Y_{n}\). Then \[\limsup_{n \to \infty } \frac{|S_{n}|}{\sqrt{ 2n\log\log n }}=1 \quad a.s\]
0.0.0.1 Discussion
The law of iterated logarithms operates “in between” the Laws of Large Numbers and the Central Limit Theorem.
Both laws of large numbers state that the sums \(S_{n}\) scaled by \(n^{-1}\) converge to zero (in probability and almost surely).
The central limit theorem states that the sums \(S_{n}\) scaled by \(n^{-1/2}\) converge in distribution to a standard normal. By [[kolmogorovs-zero-one-law]], for any fixed \(M\), the probability that the event \(\limsup_{ n \to \infty }{\frac{S_{n}}{\sqrt{ n }}}\geq M\) occurs is \(0\) or \(1\).
This gives that \[ \mathbb{P}\left( \limsup_{ n \to \infty } \frac{{S_{n}}}{\sqrt{ n }}\geq M \right)\geq \limsup_{ n \to \infty } \mathbb{P}\left( {\frac{S_{n}}{\sqrt{ n }}}\geq M \right)=\mathbb{P}(\mathcal{N}(0, 1)\geq M)>0 \] and so \[ \limsup_{ n \to \infty } \frac{{S_{n}}}{\sqrt{ {n} }}=\infty\quad w.p. 1. \] An identical argument gives that \[ \liminf_{ n \to \infty } \frac{{S_{n}}}{\sqrt{ n }}=-\infty\quad w.p.1. \] This implies that these quantities cannot converge almost surely. In fact, they cannot even converge in probability.
The law of the iterated logarithm provides the scaling factor where the two limits become different: \[ \frac{S_{n}}{\sqrt{ 2_{n}\log \log n }}\stackrel{\mathbb{P}}{\to}0\quad\&\quad \frac{{S_{n}}}{\sqrt{ {2n\log \log n} }}\stackrel{a.s.}{\not\to}0. \] Thus, although the absolute value of the quantity \(\frac{S_{n}}{\sqrt{ 2n\log \log n }}\) is less that any predefined \(\epsilon>0\) with probability approaching 1, it will nevertheless almost surely be greater than \(\epsilon\) infinitely often. In fact, the quantity will be visiting the neighborhoods of any point in the interval \((-1,1)\) almost surely.