0.0.1 Conditional Expectation Given Events
Consider probability space \((\Omega,\mathcal{F},\mathbb{P})\) with \(F \in \mathcal{F}\) with \(\mathbb{P}(F)>0\). Then, we can define a new well-defined probability measure, the conditional probability given \(F\) for every \(E\in \mathcal{F}\) by \[\mathbb{P}_{F}(E)=\mathbb{P}(E|F):= \frac{\mathbb{P}(E\cap F)}{\mathbb{P}(F)}\]from which we also obtain a new probability space \((\Omega, \mathcal{F}, \mathbb{P}_{F})\).
Note that random variables are unchanged between the spaces but will have different measures. For random variables \(X:\Omega \to \mathbb{R}\) with \(\mathbb{E}|X|<\infty\) we define the expectation with respect to the conditional probability measure as \[ \mathbb{E}_{F}[X]=\int _{\Omega}X(\omega) \, d\mathbb{P}_{F}(\omega)= \frac{1}{\mathbb{P}(F)}\int _{F}X(\omega) \, d\mathbb{P}(\omega) = \frac{1}{\mathbb{P}(F)}\mathbb{E}[\mathbb{1}_{F}X] , \] for every random variable \(X:\Omega \to \mathbb{R}\) with \(\mathbb{E}|X|<\infty\). We may set \(\mathbb{P}_{F}(\cdot)=0\) on \(\mathcal{F}\), if \(\mathbb{P}(F)=0\) and extend the notions to the case \(\mathbb{P}(F)=0\).
0.0.2 Conditional Expectation Given Random Variables
Let \(\mathcal{G}\) be a sub-\(\sigma\)-algebra of sets in a complete probability space \((\Omega, \mathcal{F}, \mathbb{P})\). For every integrable random variable \(X\), there exists a unique (up to a.e. equivalence) \(\mathcal{G}\)-measurable and integrable random variable \(\mathbb{E}[X|\mathcal{G}]\) which satisfies \[ \mathbb{E}[X\cdot \mathbb{1}_{\Lambda}]=\mathbb{E}[\mathbb{E}[X|\mathcal{G}]\cdot \mathbb{1}_{\Lambda}];\quad\Lambda \in\mathcal{G}, \] called the conditional expectation of \(X\) given \(\mathcal{G}\).
Similarly, for any \(E\in \mathcal{F}\) a unique (up to a.e. equivalence) \(\mathcal{G}\)-measurable random variable \(\mathbb{P}(E|\mathcal{G}):\Omega \to[0,1]\) that satisfies \[ \mathbb{P}(E\cap \Lambda)=\mathbb{E}[\mathbb{P}(E|\mathcal{G})\cdot \mathbb{1}_{\Lambda}];\quad \Lambda \in\mathcal{G}, \] called the conditional probability of \(E\) given \(\mathcal{G}\). #### Construction
On a probability space \((\Omega, \mathcal{F}, \mathbb{P})\) fix a set \(F \in \mathcal{F}\) with \(\mathbb{P}(F)>0\) and define the conditional probability given \(F\) for every \(E\in \mathcal{F}\) by \[\mathbb{P}_{F}(E)=\mathbb{P}(E|F):= \frac{\mathbb{P}(E\cap F)}{\mathbb{P}(F)}\]The expectation with respect to the conditional probability is then defined as \[\mathbb{E}_{F}[X]=\int _{\Omega}X(\omega) \, d\mathbb{P}_{F}(\omega)= \frac{1}{\mathbb{P}(F)}\int _{F}X(\omega) \, d\mathbb{P}(\omega) = \frac{1}{\mathbb{P}(F)}\mathbb{E}[\mathbb{1}_{F}X] ,\]for every random variable \(X:\Omega \to \mathbb{R}\) with \(\mathbb{E}|X|<\infty\). We may set \(\mathbb{P}_{F}(\cdot)=0\) on \(\mathcal{F}\), if \(\mathbb{P}(F)=0\) and extend the notions to the case \(\mathbb{P}(F)=0\).
Let us consider a countable partition \(\{ F_{n}; n\geq 1 \}\) such that \(F_{n}\cap F_{m}= \emptyset\) for \(m \neq n\) and \(\Omega=\cup_{n} F_{n}\). Denote by \(\mathcal{G}\) the smallest \(\sigma\)-algebra containing \(\{ F_{n}; n \geq 1 \}\). Note, from the countable additivity of measure \[ \mathbb{P}(E)=\sum_{n=1}^\infty \mathbb{P}(F_{n})\cdot \mathbb{P}_{F_{n}}(E);\quad \mathbb{E}[X]=\sum_{n=1}^\infty \mathbb{P}(F_{n})\cdot \mathbb{E}_{F_{n}}[X]. \] Now let use define a simple random variable \[ \mathbb{E}[X|\mathcal{G}](\omega):=\sum_{n=1}^\infty \mathbb{E}_{F_{n}}[X]\mathbb{1}_{F_{n}}(\omega);\quad \omega \in \Omega, \] for every integrable random variable \(X\). Then we observe that \[ \mathbb{E}[\mathbb{E}[X|\mathcal{G}]]=\mathbb{E}\left[ \sum_{n=1}^\infty \mathbb{E}_{F_{n}}[X]\cdot \mathbb{1}_{F_{n}}(\omega)\right]=\sum_{n=1}^\infty \mathbb{P}(F_{n})\cdot \mathbb{E}_{F_{n}}[X]=\mathbb{E}[X]. \] A little more generally, we see that for every \(\Lambda \in\mathcal{G}\) \[ \mathbb{E}[\mathbb{E}[X|\mathcal{G}]\cdot \mathbb{1}_{\Lambda}]=\mathbb{E}[X\cdot \mathbb{1}_{\Lambda}]. \]