Random Variable Transformations
1 Introduction
Let \(X\) be a random variable with density \(f_{X}\) (or mass function) and let \(g:\mathbb{R}\to\mathbb{R}\) be a measurable function. This note covers methods for finding the properties of the new random variable \(Y=g(X)\): its expectation, moments and distribution.
2 Fundamental theorem
Let \(X\) have density \(f_{X}\) and let \(g:\mathbb{R}\to\mathbb{R}\) be measurable with \(g(X)\) integrable. Then \(Y=g(X)\) has expectation \[ \mathbb{E}[Y]=\int_{\Omega}g(X)\,d\mathbb{P}=\int_{\mathbb{R}}g(x)f_{X}(x)\,dx. \]
In the discrete case the integral becomes a sum over the support of \(X\) with mass function \(p_{X}(x)\).
In general \(\mathbb{E}[g(X)]\neq g(\mathbb{E}[X])\). Equality is guaranteed for every distribution only when \(g\) is an Affine Function, that is, a linear function plus a translation (for convex \(g\) see Jensen Inequality).
3 Distribution of the transformed variable
Recall that the inverse image (preimage) of a set \(A\subseteq\mathbb{R}\) under \(g\) is \[ g^{-1}(A):=\{x\in\mathbb{R}:g(x)\in A\}, \] see Images and Preimages of Functions. It satisfies \(g^{-1}(\mathbb{R})=\mathbb{R}\), \(g^{-1}(A^c)=g^{-1}(A)^c\) and \(g^{-1}(\bigcup_\lambda A_\lambda)=\bigcup_\lambda g^{-1}(A_\lambda)\). As a result, \(\mu(A)=\mathbb{P}(X\in g^{-1}(A))=\mathbb{P}(g(X)\in A)\) satisfies the axioms of probability and is the distribution of \(Y=g(X)\) (see Random Variables and Distribution Functions and Probability Spaces).
3.1 Discrete transformations
If \(X\) is discrete with mass function \(f_{X}\), then \(Y=g(X)\) has mass function \[ f_{Y}(y)=\sum_{x\in g^{-1}(\{y\})}f_{X}(x). \]
Let \(X\sim\mathcal{P}(\lambda)\) follow a Poisson Distribution and define \(Y=g(X)=X^2+7\). The support of \(X\) is \(\mathbb{N}_{0}\), on which \(g\) is one-to-one, so \[ f_{Y}(y)=f_{X}\big(\sqrt{y-7}\big) =\frac{\lambda^{\sqrt{y-7}}e^{-\lambda}}{\big(\sqrt{y-7}\big)!}, \] on the support \(y\in\{x^2+7:x\in\mathbb{N}_0\}=\{7,8,11,16,\dots\}\).
3.2 Continuous transformations
Suppose \(g\) is one-to-one, that is strictly increasing or strictly decreasing (hence \(g^{-1}\) is too). The CDF method computes, in the increasing case, \[ \mathbb{P}(Y\leq y)=\mathbb{P}(g(X)\leq y)=\mathbb{P}(X\leq g^{-1}(y)) =F_{X}(g^{-1}(y)), \] and in the decreasing case \(1-F_{X}(g^{-1}(y))\) (for continuous \(X\)). Differentiating by the chain rule, and taking the modulus to cover both cases, \[ f_{Y}(y)=f_{X}(g^{-1}(y))\left|\frac{d}{dy}g^{-1}(y)\right|. \]
Special care is needed when \(g\) is not one-to-one, for example \(g(x)=x^2\) or \(g(x)=|x|\) on \(\mathbb{R}\). For \(g(x)=x^2\) and \(y>0\), \[ F_{Y}(y)=\mathbb{P}(X^2\leq y)=\mathbb{P}(-\sqrt{y}\leq X\leq\sqrt{y}) =F_{X}(\sqrt{y})-F_{X}(-\sqrt{y}), \] and differentiating gives \[ f_{Y}(y)=\frac{1}{2\sqrt{y}}\Big\{f_{X}(\sqrt{y})+f_{X}(-\sqrt{y})\Big\}. \] If \(f_{X}\) is symmetric about the origin this simplifies to \(f_{Y}(y)=\frac{1}{\sqrt{y}}f_{X}(\sqrt{y})\).
Let \(Z\) follow the standard Normal Distribution with density \(\phi(z)=\frac{1}{\sqrt{2\pi}}e^{-z^2/2}\). The density is symmetric, so \(Y=Z^2\) has density \[ f_{Y}(y)=\frac{1}{\sqrt{y}}\phi(\sqrt{y})=\frac{1}{\sqrt{2\pi y}}e^{-y/2}, \qquad y>0, \] which is the Chi-Squared Distribution with one degree of freedom.
4 The probability integral transform
Let \(X\) be a continuous random variable whose distribution function \(F_{X}\) is strictly increasing on the support of \(X\) (so \(F_X\) is invertible). Then \(U=F_{X}(X)\) is uniform on \([0,1]\).
Proof. For \(u\in[0,1]\), \[ \mathbb{P}(U\leq u)=\mathbb{P}(F_{X}(X)\leq u)=\mathbb{P}(X\leq F_{X}^{-1}(u)) =F_{X}(F_{X}^{-1}(u))=u. \]
So \(U\) follows the Continuous Uniform Distribution on \([0,1]\). Most random number generators simulate independent copies of \(U\). Conversely, we can simulate a random variable with distribution function \(F_{X}\) by drawing \(U\) and setting \[ X=F_{X}^{-1}(U). \] This is inverse transform sampling.