> [!definition] > > Let $X$ be a [[Random Variable|random variable]] with range $R_x = \{x_1, x_2, \cdots, x_n\}$ and [[Probability Mass Function|probability mass function]] $p$, and $Y$ be a random variable in the same [[Sample Space|sample space]] with range $R_y = \{y_1, y_2, \cdots, y_m\}$ and probability distribution $q$. Define the **joint PMF** of $X$ and $Y$ by: > $ > p_{ij} = P((X = x_i) \cap (Y = y_j)) \quad \text{for }1 \le i \le n, 1 \le j \le m > $ > [!theorem] Properties > > $ > \begin{align*} > \sum_{i = 1}^{n}p_{ij} &= q_j \quad \text{for } 1 \le j \le m \\ > \sum_{j = 1}^{m}p_{ij} &= p_i \quad \text{for } 1 \le i \le n \\ > \sum_{i = 1}^{n}\sum_{j = 1}^{m}p_{ij} &= 1 > \end{align*} > $ > > *Proof.* Since $(X = x_1), (X = x_2), \cdots, (X = x_n)$ and $(Y = y_1), (Y = y_2), \cdots, (Y = y_m)$ are both [[Partition|partitions]] of $S$, summing over them is equivalent to summing over the entire sample space. As $P$ is a [[Measure Space|measure]], using the definition clumps the sums together. > $ > \begin{align*} > \sum_{i = 1}^{n}p_{ij} &= \sum_{i = 1}^{n}P((X = x_i) \cap (Y = y_j)) \\ > &= P\paren{\bigcup_{i = 1}^{n}(X=x_i) \cap (Y=y_j)}\\ > &= P\paren{(Y = y_j) \cap \bigcup_{i = 1}^{n}(X=x_i)}\\ > &= P\paren{(Y = y_j) \cap (X \in S)} \\ > &= P(Y = y_j) = q_j > \end{align*} > $ > *Swap $X$ and $Y$ to obtain the proof for the second statement*. > > The same principle can be applied twice to prove the third statement. > $ > \begin{align*} > \sum_{i = 1}^{n}\sum_{j = 1}^{m}p_{ij} &= \sum_{i = 1}^{n}\sum_{j = 1}P((X = x_i) \cap (Y = y_j)) \\ > &= P\paren{\bigcup_{i = 1}^{n}\bigcup_{j = 1}^{m}(X=x_i) \cap (Y=y_j)}\\ > &= P\paren{\bigcup_{i = 1}^{n}\paren{(X=x_i) \cap \bigcup_{j = 1}^{m}(Y=y_j)}}\\ > &= P\paren{\bigcup_{i = 1}^{n}(X=x_i) \cap (Y \in S)}\\ > &= P\paren{(X \in S) \cap (Y \in S)} = 1 > \end{align*} > $ A pair of random variables $(X, Y)$ is **non-degenerate**[^1] for all $1 \le j \le n$ (assuming $n \le m$), there exists $1 \le k \le m$, such that the joint probability $p_{jk} > 0$. If $n \le m$, then swap the roles of $j$ and $k$. [^1]: This one took hours to digest, review the thoughts at [[Footnote - Random Degeneracy]].