1 What is the Characteristic Function?
The characteristic function is a type of generating function for random variables similar to the Probability Generating Functions and the Moment Generating Function.
The characteristic function for a random variable \(X\) is defined for all \(t \in \mathbb{R}\) as \[ \varphi_{X}(t):=\mathbb{E}[e^{itX}]=\mathbb{E}[\cos tX] + i \mathbb{E}[\sin tX]. \] This definition may be extended to random vectors as follows \[ \varphi_{X,Y}(s,t):=\mathbb{E}[e^{isX+itY}];\quad (s,t)\in \mathbb{R}^2. \]
Like the MGF the characteristic function generates the moments by [[differentiation-under-integration|differentiation under integration]] \[ \varphi_{X}^{(k)}(0):= \frac{{\partial^k}}{\partial t^k}\varphi_{X}(t)\Big|_{t=0}=\mathbb{E}\left[ \frac{{\partial^k}}{\partial t^k}e^{itX} \right]\Big|_{t=0}=i^k\mathbb{E}[X^k];\quad k\in \mathbb{N}. \] We need justification for this interchange between the differentiation and integration, and we note that the characteristic function behaves better than the MGF because of the following theorem.
The characteristic function satisfies the following: 1. \(\varphi(0)=1\), \(|\varphi(t)|\leq 1\) for all \(t\in \mathbb{R}\) 2. \(t \mapsto \varphi(t)\) is [[uniform-continuity|uniformly continuous]]; and 3. \(\varphi(\cdot)\) is nonnegative definite, i.e. for every \(n\), \((t_{1}, \dots, t_{n})\in \mathbb{R}^n\) and \((z_{1}, \dots, z_{n})\in \mathbb{C}^n\) we have \[ \sum_{j,k=1}^n\varphi(t_{j}-t_{k})z_{j}\bar{z}_{k}\geq 0 \] where \(\bar{z}\) is the complex conjugate of \(z \in \mathbb{C}\).
Proof: (1) \(\varphi(0)=\mathbb{E}[1]=1\); \(|\varphi(t)|=|\mathbb{E}[e^{itX}]|\leq\mathbb{E}[|e^{itX}|]=1\) by triangle inequality. (2) For any \(\epsilon>0\), dominated convergence gives \(\delta>0\) where \(|h|<\delta\implies\mathbb{E}[|e^{ihX}-1|]<\epsilon\), so \(|\varphi(t+h)-\varphi(t)|\leq\mathbb{E}[|e^{ihX}-1|]<\epsilon\). (3) \(\sum_{j,k}\varphi(t_j-t_k)z_j\bar{z}_k=\mathbb{E}[\sum_{j,k}e^{i(t_j-t_k)X}z_j\bar{z}_k]=\mathbb{E}[|\sum_je^{it_jX}z_j|^2]\geq 0\). \(\square\)
2 Characteristic Function Properties
The characteristic function satisfies the following properties: 1. The characteristic function always exists \(\varphi_{X}(0)=1,~\lvert \varphi_{X}(t) \rvert\leq 1\). 2. \(\varphi_{X}(-t)=\overline{\varphi_{X}(t)}\) (complex conjugate); 3. \(\lvert \varphi(t) \rvert=\lvert \mathbb{E}e^{itX} \rvert\leq \mathbb{E}\lvert e^{itX} \rvert=1\); 4. \(\lvert \varphi(t+h)-\varphi(t) \rvert \leq \mathbb{E}\lvert e^{ihX}-1 \rvert\) and so \(\varphi_{X}\) is uniformly continuous on \(\mathbb{R}\); 5. \(\phi_{aX+b}(t)=e^{itb}\phi_{X}(at)\) for constants \(a,b \in \mathbb{R}\); and 6. \(X\) is symmetric i.e. \(X\stackrel{\mathcal{D}}{=}-X\) if and only if \(\phi_{X}\) is a real-valued function.
Proof: (1) Follows from Bochner’s Theorem (1). (2) \(\varphi_X(-t)=\mathbb{E}[e^{-itX}]=\mathbb{E}[\overline{e^{itX}}]=\overline{\mathbb{E}[e^{itX}]}=\overline{\varphi_X(t)}\) (by linearity). (3) Follows from Bochner’s Theorem (1). (4) Follows from Bochner’s Theorem (2). (5) \(\phi_{aX+b}(t)=\mathbb{E}[e^{it(aX+b)}]=e^{itb}\mathbb{E}[e^{itaX}]=e^{itb}\phi_X(at)\). (6) If \(X\stackrel{\mathcal{D}}{=}-X\), then \(\phi_X(t)=\mathbb{E}[e^{itX}]=\mathbb{E}[e^{it(-X)}]=\phi_{-X}(t)=\overline{\phi_X(-t)}=\overline{\phi_X(t)}\), so \(\phi_X\) is real. Conversely, if \(\phi_X\) is real-valued, then \(\phi_X(t)=\overline{\phi_X(t)}=\phi_{-X}(t)\), so \(X\stackrel{\mathcal{D}}{=}-X\). \(\square\)
Additionally we have the following results for characteristic functions of independent random variables.
If two random variables \(X\) and \(Y\) are independent then: 1. \(\varphi_{X+Y}(t)=\varphi_{X}(t)\varphi_{Y}(t)\) for \(t \in \mathbb{R}\); and 2. \(\varphi_{X,Y}(s,t)=\varphi_{X}(s)\varphi_{Y}(t)\) for \((s,t)\in \mathbb{R}^2\).
Proof: (1) \(\varphi_{X+Y}(t)=\mathbb{E}[e^{it(X+Y)}]=\mathbb{E}[e^{itX}e^{itY}]=\mathbb{E}[e^{itX}]\mathbb{E}[e^{itY}]=\varphi_X(t)\varphi_Y(t)\) by independence. (2) Similarly, \(\varphi_{X,Y}(s,t)=\mathbb{E}[e^{isX+itY}]=\mathbb{E}[e^{isX}]\mathbb{E}[e^{itY}]=\varphi_X(s)\varphi_Y(t)\) by independence. \(\square\)
3 CF and New Distributions
The CF is useful for determining the distribution of some new unknown random variable by comparing characteristic functions with random variables with known distributions.
The name characteristic function comes from the unique characterization of the distribution, whereby for every Borel set \(A \in \mathcal{B}:=\sigma(\mathbb{R})\) we have that \[ \begin{align} \mathbb{P}(X \in A) & =\mathbb{P}(Y \in A)\\ \iff \mathbb{E}[e^{itX}] & =\varphi_{X}(\cdot) \\ & =\varphi_{Y}(\cdot) \\ & =\mathbb{E}[e^{itY}]\quad (t\in \mathbb{R}). \end{align} \] Thus in order to determine the distribution of a new random variable \(Y\), we find the characteristic function of \(Y\) and compare it with that of a known random variable \(X\).
Because of the relationship between the MGF and CF, this method of finding the distribution of a new random variable for the MGF and the PGF (or Laplace Transform) is used for non-negative random variables and for discrete random variables respectively.
In general we account for the discontinuity of distribution functions with the Inversion theorem.
The characteristic function \(\varphi(\cdot)\) and the cumulative distribution function \(F(\cdot)\) of a random variable \(X\) satisfy \[ \begin{align}\mathbb{P}(a<X<b)+& \frac{1}{2}(\mathbb{P}(X=a)+\mathbb{P}(X=b)) \\ & =\bar{F}(b)-\bar{F}(a) \\ & =\lim_{ N \to \infty } \int _{-N}^N \frac{{e^{-iat}-e^{-ibt}}}{2\pi i t}\varphi(t) \, dt ,\end{align} \] where \[ \begin{align}\bar{F}(x)& :=\frac{1}{2}(F(x)-\lim_{ y \uparrow x }F(y) ) \\ & =\frac{1}{2}(\mathbb{P}(X\leq x)+P(X<x))\\&=\mathbb{P}(X\leq x)-\frac{1}{2}\mathbb{P}(X=x);\quad x \in \mathbb{R}.\end{align} \]
It follows from the right-continuity of cumulative distribution function that two random variables \(X\) and \(Y\) have the same characteristic function if and only if they have the same distribution.
Indeed, it is simple that if \(X\) and \(Y\) have the same cumulative distribution function, then they have the same characteristic function. On the other hand, if \(\varphi_{X}(\cdot)=\varphi_{Y}(\cdot)\) then the inversion theorem implies \[ \bar{F}_{X}(b) = \lim_{ a \to -\infty }(\bar{F}_{X}(b)-\bar{F}_{X}(a))=\lim_{ a \to -\infty }(\bar{F}_{Y}(b)-\bar{F}_{Y}(a))=\bar{F}_{Y}(b);\quad b\in \mathbb{R}. \] Then \(F_{X}(x)=F_{Y}(x)\) for every continuity point \(x \in (\mathcal{J}_{X}\cup\mathcal{J}_{Y})^c\) i.e. the at most countable set of jump points of \(F_{X}\) and \(F_{Y}\).
The following result highlights the relationship between the probability density function and the characteristic function of a random variable.
The probability density function \(f(\cdot)\) and characteristic function \(\varphi(\cdot)\) of a random variable satisfy \[ f(x)=\frac{1}{2\pi}\int _{\mathbb{R}}e^{-itx}\varphi(t) \, dt, \] at every point at which \(f(\cdot)\) is differentiable.