> [!theorem] > > Let $\seq{X_n} \subset L^1$ be a family of [[Probabilistic Independence|independent]] identically [[Probability Distribution|distributed]] [[Random Variable|random variables]]. Let $S_n = \sum_{i = 1}^nX_i$, then for all $\varepsilon > 0$, > $ > \bp\bracs{\abs{\frac{S_n}{n} - \ev(X_1)} \ge \varepsilon} \to 0 > $ > the sample mean converges to the [[Expectation|expectation]] [[Convergence in Measure|in probability]]. > > ### Tail Bound > > Let $N > 0$, let $X_i^{\le N} = X_i \cdot \one_{\abs{X_i} \le N}$ and $X_i^{> N} = X_i \cdot \one_{\abs{X_i} > N}$ such that $\abs{X_i^{\le N}} + \abs{X_i^{> N}} = \abs{X_i}$. By [[Monotone Convergence Theorem for Sequences|monotone convergence theorem]], > $ > \limv{N}\ev\paren{\abs{X_i^{\le N}}} = \ev(\abs{X_i}) = \ev(\abs{X_1}) > $ > so > $ > \limv{N}\abs{X_i^{> N}} = \ev{(\abs{X_i})} - \limv{N}\ev{(\abs{X_i^{\le N}})} = 0 > $ > > ### The Body > > Let $\varepsilon > 0$ and $N$ such that $\ev(X_i^{> N}) < \varepsilon^2/8$. Let > $ > S_n^{\le N} = \sum_{k = 1}^nX_k^{\le N} \quad S_n^{>N} = \sum_{k = 1}^nX_k^{>N} > $ > and $\ol{S}_n^{\le N} = S_n^{\le N}/n$, $\ol{S}_n^{>N} = S_n^{>N}/n$. Now that we bounded the variable, it's possible to use [[Chebyshev's Inequality]], > $ > \begin{align*} > \bp\bracs{\abs{\ol{S}_n^{\le N} - \ev\paren{\ol{S}_n^{ \le N}}}> \varepsilon/2} &\le \frac{n\var{X_1^{\le N}/n}}{(\varepsilon/2)^2} \\ > &= \frac{\var{X_1^{\le N}}}{(\varepsilon/2)^2n} \\ > &\le \frac{4N^2}{\varepsilon^2n} > \end{align*} > $ > where if $n > 8N^2/\varepsilon^3$, $\frac{4N^2}{\varepsilon^2n} < \varepsilon/2$. > > ### The Tail > > For the tail, > $ > \begin{align*} > \bp\bracs{\abs{\ol{S}_n^{>N} - \ev\paren{\ol{S}_n^{>N}}} > \varepsilon/2} &\le \frac{\ev\paren{\abs{\ol{S}_n^{>N} - \ev\paren{\ol{S}_n^{>N}}}}}{\varepsilon/2} \\ > &\le \frac{\ev\paren{\abs{\ol{S}_n^{>N}}} + \abs{\ev\paren{\abs{\ol{S}_n^{>N}}}}}{\varepsilon/2} \\ > &\le \frac{4\ev\paren{\abs{\ol{S}_n^{>N}}}}{\varepsilon} \\ > &\le \frac{\varepsilon}{2} > \end{align*} > $ > where the last bound comes from the choice of $N$. > > ### Combined > > Therefore for $n > 8N^2/\varepsilon^3$, > $ > \begin{align*} > \bp\bracs{\abs{\ol{S}_n - \ev(X_1)} > \varepsilon} &\le \bp\bracs{\abs{\ol{S}_n^{\le N} - \ev\paren{\ol{S}_n^{ \le N}}}> \varepsilon/2} \\ > &+ \bp\bracs{\abs{\ol{S}_n^{>N} - \ev\paren{\ol{S}_n^{>N}}} > \varepsilon/2} \\ > &\le \varepsilon > \end{align*} > $ > which gives the convergence in probability. > > > [^1]: It's sufficient for them to be not linearly correlated.