An estimator is a [[Random Sample|sample]] [[Statistic|statistic]] that can be used to estimate a [[Population|population]] parameter. For example, the [[Sample Mean|sample mean]] $\bar{X}$ is an estimator for the population [[Mean|mean]] $\mu$. An estimator is preferably unbiased and has small variance. An *unbiased* estimator $\hat{\theta}$ has a [[Sampling Distribution|sampling distribution]] centred at its parameter $\theta$ ($E(\hat{\theta}) = \theta$), while a biased estimator may over or underestimate its parameter. A sampling distribution that has a small standard deviation provides better confidence. # Point Estimators A point estimator is a single number that acts as the best guess for a given parameter. Since $E(\bar{X}) = \mu$, the sample mean $\bar{X}$ is an unbiased point estimator for the population mean $\mu$ ($\hat{u} = \bar{X}$). The sample [[Variance|variance]] $S^2$ is an unbiased estimator for the population variance $\sigma^2$ ($E(S^2) = \sigma^2$). $ \begin{align*} S^2 &= \frac{1}{n - 1}\sum_{i = 1}^{n}\paren{X_i - \bar{X}}^2 \\ &= \frac{1}{n - 1}\sum_{i = 1}^{n}\paren{ \paren{X_i - \mu} - \paren{\bar{X} - \mu}}^2\\ &= \frac{1}{n - 1}\sum_{i = 1}^{n}\paren{ \paren{X_i - \mu}^2 - 2\paren{X_i-\mu}\paren{\bar{X}-\mu} + \paren{\bar{X} - \mu}^2} \\ &= \frac{1}{n - 1}\paren{ \sum_{i = 1}^{n}\paren{X_i - \mu}^2 - 2\paren{\bar{X}-\mu}\sum_{i = 1}^{n}\paren{X_i-\mu} + n\paren{\bar{X} - \mu}^2 } \\ &= \frac{1}{n - 1}\paren{ \sum_{i = 1}^{n}\paren{X_i - \mu}^2 - 2\paren{\bar{X}-\mu}\paren{\sum_{i = 1}^{n}X_i-n\mu} + n\paren{\bar{X} - \mu}^2 } \\ &= \frac{1}{n - 1}\paren{ \sum_{i = 1}^{n}\paren{X_i - \mu}^2 - 2\paren{\bar{X}-\mu}\paren{n\bar{X}-n\mu} + n\paren{\bar{X} - \mu}^2 } \\ &= \frac{1}{n - 1}\paren{ \sum_{i = 1}^{n}\paren{X_i - \mu}^2 - 2n\paren{\bar{X}-\mu}^2 + n\paren{\bar{X} - \mu}^2 } \\ &= \frac{1}{n - 1}\paren{ \sum_{i = 1}^{n}\paren{X_i - \mu}^2 - n\paren{\bar{X}-\mu}^2 } \\ E(S^2) &= \frac{1}{n - 1}\paren{ \sum_{i = 1}^{n}E\paren{\paren{X_i - \mu}^2} - nE\paren{\paren{\bar{X}-\mu}^2} } \\ &= \frac{1}{n - 1}\paren{ \sum_{i = 1}^{n}Var(X_i) - nVar(\bar{X}) } \\ &= \frac{1}{n - 1}\paren{ n\sigma^2 - \frac{n\sigma^2}{n} } \\ &= \frac{\sigma^2}{n - 1}\paren{n - 1} \\ E(S^2) &= \sigma^2 \end{align*} $ In a [[Bernoulli Process|Bernoulli]] random sample $\bar{Y}$ of size $n$, with $p$ as the [[Probability|probability]] of success, $E(\bar{Y}) = E(Y) = \mu = p = \frac{X}{n}$. In other words, an unbiased estimator for $p$ is $\hat{p} = \frac{X}{n}$. As a result, $Var{\hat{p}} = \frac{1}{n^2}Var(x) = \frac{npq}{n^2} = \frac{pq}{n}$, which means that $\sigma_{\hat{p}} = \sqrt{\frac{pq}{n}}$. In a sample $X$ of size $n$ for the [[Exponential Random Variable|exponential distribution]] with parameter $\lambda$, then $E(\bar{X}) = E(X) = \mu = \frac{1}{\lambda}$, therefore $\frac{1}{\bar{X}}$ is an unbiased estimator for $\lambda$ ($\hat{\lambda} = \frac{1}{\bar{X}}$). In a sample $X$ of size $n$ for the [[Poisson Distribution|Poisson distribution]] with parameter $\lambda$. Then $E(\bar{X}) = E(X) = \mu = \lambda$, so $\bar{X}$ is an unbiased estimator for $\lambda$ ($\hat{\lambda} = \bar{X}$). # Mean Confidence Intervals ![[confidence_interval.png|300]] A confidence interval is an interval of numbers around a point estimate $(\hat\theta - E, \hat\theta + E)$, within which the parameter value is believed to fall with confidence $(1 - \alpha)\%$. The deviation $E$ from the point estimate is known as the margin of error. While for larger samples, $\bar{X}$ is a accurate estimator for $\mu$, for smaller samples, the quality of the estimate is unclear. If the [[Count|sample size]] is sufficiently large ($n \ge 30$), by the [[Central Limit Theorem]] $\bar{X} \approx N\paren{\mu, \frac{\sigma^2}{n}}$. Find $z_{\alpha/2}$ such that $P(-z_{\alpha/2} < Z < z_{\alpha/2}) = 1-\alpha$. $ \begin{align*} P(-z_{\alpha/2} < Z < z_{\alpha/2}) &= 1-\alpha \\ P\paren{-z_{\alpha/2} < \frac{\bar{X} - \mu}{\sigma/\sqrt{n}} < z_{\alpha/2}} &= 1-\alpha \\ P\paren{\bar{X}-\frac{z_{\alpha/2}\sigma}{\sqrt{n}} < \mu < \bar{X} + \frac{z_{\alpha/2}\sigma}{\sqrt{n}}} &= 1-\alpha \end{align*} $ Which means that for a population with known variance $\sigma^2$, a $(1-\alpha)100\%$ confidence interval for $\mu$ is given by $ \bar{X}-\frac{z_{\alpha/2}\sigma}{\sqrt{n}} < \mu < \bar{X} + \frac{z_{\alpha/2}\sigma}{\sqrt{n}} $ The margin of error $E$ is equal to $\frac{z_{\alpha/2}\sigma}{\sqrt{n}}$. To ensure that the error is less than a specified amount $e$, the sample size has to be greater than $\paren{\frac{z_{\alpha/2}\sigma}{e}}^2$. For a small sample size from a [[Normal Distribution|normally distributed]] population, the normal distribution can be used directly if $\sigma$ is known. Otherwise, the [[t-distribution]] can be used with $n - 1$ degrees of freedom. # Proportion Confidence Intervals To estimate the proportions of a population that meets a certain criteria $p$, use the unbiased point estimator for $p$ in a [[Binomial Random Variable|binomial distribution]], $\hat{p} = \frac{X}{n}$, where $X = B(n, p)$. If $p$ is not expected to be too close to 0 or 1, then a normal distribution may be used to approximate the binomial distribution, and using the [[Central Limit Theorem|central limit theorem]]: $ \begin{align*} \frac{\frac{X}{n} - p}{\sqrt{pq/n}} &\approx Z \\ P\paren{-z_{\alpha/2} < \frac{\frac{X}{n} - p}{\sqrt{pq/n}} < z_{\alpha/2}} &\approx 1 - \alpha \\ P\paren{\frac{X}{n}-z_{\alpha/2}\sqrt{\frac{pq}{n}} < p < \frac{X}{n}-z_{\alpha/2}\sqrt{\frac{pq}{n}}} &\approx 1 - \alpha \end{align*} $ The unknown $p$ still shows up in the endpoints of the confidence interval, which when $n$ is large, can be get around by substituting the point estimate $\hat{p} = \frac{x}{n}$ and $\hat{q} = 1 - \hat{p}$, obtaining an interval: $ \frac{X}{n}-z_{\alpha/2}\sqrt{\frac{\hat{p}\hat{q}}{n}} < p < \frac{X}{n}-z_{\alpha/2}\sqrt{\frac{\hat{p}\hat{q}}{n}} $ To obtain the sample size $n$ for which $E \le e$, $n \ge \frac{z^2_{\alpha/2} \hat{p}\hat{q}}{e^2}$. Unfortunately, $\hat{p}$ is required to estimate the sample size, but can only be obtained from the sample. To avoid this, start with a preliminary sample of $n \ge 30$, or complete the square and obtain $\hat{p}\hat{q} \le \frac{1}{4}$ and estimate $\hat{p} = \frac{1}{2}$, and $n \ge \frac{z^2_{\alpha/2}}{4e^2}$.