> [!definition] > > $ > \begin{align*} > \text{cov}(X, Y) &= \frac{1}{n - 1}\sum(x - \bar{x})(y - \bar{y}) \\ > \cov{X, Y} &= \ev((X - \mu_x)(Y - \mu_y)) > \end{align*} > $ > > Covariance is a measure of joint [[Variance|variance]] of two [[Random Variable|random variables]], that represents their correlation in a [[Linear Regression|linear relationship]], measured in the product of the units of $X$ and $Y$. > > For a positive [[Correlation|correlation]], the covariance will be positive, for a negative correlation, the covariance will be negative, and for a lack of correlation, the covariance will be $0$. > [!theoremb] Calculation Trick > > Let $X$ and $Y$ be arbitrary random variables, the covariance can be calculated solely in terms of [[Expectation|expected values]]. > $ > \cov{X, Y} = \ev(XY) - \ev{(X)}\ev{(Y)} > $ > *Proof*. > $ > \begin{align*} > \cov{X, Y} &= \ev{((X - \mu_x)(Y - \mu_y))} \\ > &= \ev{(XY - X\mu_y - Y\mu_x + \mu_x\mu_y)} \\ > &= \ev(XY) - \ev(X\mu_y) - \ev(Y\mu_x) + \mu_x\mu_y \\ > &= \ev(XY) - \mu_y\ev(X) - \mu_x\ev(Y) + \mu_x\mu_y \\ > &= \ev(XY) - \mu_x\mu_y - \mu_x\mu_y + \mu_x\mu_y \\ > &= \ev(XY) - \ev{(X)}\ev{(Y)}\\ > \end{align*} > $ > [!theorem] > > Let $X$ and $Y$ be random variables. If they are [[Probabilistic Independence|independent]], then $\cov{X, Y} = 0$. > > *Proof*. If $X$ and $Y$ are independent, then > $ > \ev(XY) = \ev(X)\ev(Y) > $ > so $\cov{X, Y} = \ev(X, y) - \ev(X)\ev(Y) = 0$. > [!theorem] > > Let $X, Y, Y_1, Y_2$ be arbitrary random variables. > > $ > \begin{align*} > \cov{X, Y} &= \cov{Y, X} \\ > \cov{X, \alpha Y_1 + \beta Y_2} &= \alpha \cov{X, Y_1} + \beta\cov{X, Y_2} > \end{align*} > $ > > The covariance of a variable and itself is equal to its own [[Variance|variance]]. > $ > \begin{align*} > \cov{X, X} &= \var{X} \\ > \end{align*} > $ > If $X$ and $Y$ are linearly related ($Y = mX + c$), then $\cov{X, Y} = m\var{X}$. > > *Proof*. > > $ > \begin{align*} > \cov{X, Y} &= \ev((X - \mu_x)(Y - \mu_y)) \\ > &= \ev((Y - \mu_y)(X - \mu_x)) = \cov{Y, X} \\ > \cov{X, X} &= \ev((X - \mu_x)^2) = \var{X} \\ > \end{align*} > $ > $ > \begin{align*} > \cov{X, \alpha Y_1 + \beta Y_2} > &= \ev((X - \mu_x)(\alpha Y_1 - \mu_{\alpha Y_1} + \beta Y_2 - \mu_{\beta Y_2}))\\ > &= \ev((X - \mu_x)(\alpha Y_1 - \mu_{\alpha Y_1}) + (X - \mu_x)(\beta Y_2 - \mu_{\beta Y_2}))\\ > &= \ev((X - \mu_x)(\alpha Y_1 - \mu_{\alpha Y_1})) + \ev((X - \mu_x)(\beta Y_2 - \mu_{\beta Y_2}))\\ > &= \ev((X - \mu_x)(\alpha Y_1 - \alpha\mu_{Y_1})) + \ev((X - \mu_x)(\beta Y_2 - \beta\mu_{Y_2}))\\ > &= \alpha\ev((X - \mu_x)(Y_1 - \mu_{Y_1})) + \beta\ev((X - \mu_x)( Y_2 - \mu_{Y_2}))\\ > &= \alpha \cov{X, Y_1} + \beta\cov{X, Y_2} > \end{align*} > $ > > If $X$ and $Y$ are linearly related: > $ > \begin{align*} > \cov{X, Y} &= \ev((X - \mu_x)(Y - \mu_y))\\ > \cov{X, mX + c} &= \ev((X - \mu_x)(mX + c - \mu_{mX + c}))\\ > &= \ev((X - \mu_x)(mX + c - m\mu_{x} - c))\\ > &= \ev((X - \mu_x)(mX + m\mu_{mX}))\\ > &= m\ev((X - \mu_x)^2) = m\var{X} > \end{align*} > $