> [!theorem] > > Let $X_0, X_1 \cdots X_n$ be a family of [[Random Variable|random variables]]. The [[Joint Entropy|joint entropy]] > $ > H(X_0, X_1, \cdots X_n) = H(X_0) + \sum_{i = 1}^{n}H(X_i|X_0, X_1, \cdots, X_{i - 1}) > $ > > *Proof*. When $n = 1$, $H(X_0, X_1) = H(X_0) + H(X_1|X_0)$. Suppose that this holds for $n$, then > $ > \begin{align*} > H(X_0, \cdots, X_{n + 1}) &= -\sum_{i_*}p(i_0, i_1, \cdots, i_{n + 1})\log_2(p(i_0, i_1, \cdots, i_{n + 1})) \\ > &= -\sum_{i_*}p(i_0, i_1, \cdots, i_{n + 1})\log_2(P(X_{n + 1} = i_{n + 1}|\cdots)))\\ > &-\sum_{i_*}p(i_0, i_1, \cdots, i_{n + 1})\log_2(p(i_0, i_1, \cdots, i_{n})) \\ > &= H(X_0, \cdots, X_{n}) + H(X_{n + 1}|X_0,\cdots, X_n) > \end{align*} > $ > By [[Axiom of Induction|induction]], this holds for all $n \in \nat$.