Estimator of Gaussian distribution

How to estimate Gaussian distribution

The first estimator of \(\sigma\) Variance is the sample variance

\[ \hat{\sigma}^2_m = \frac{1}{m} \sum_{i=1}^{m}(x^{(i)} - \hat{\mu}_m)^{2} \]

\begin{align*} E[X^2] =& E[X - \mu + \mu]^2 \\ =& E(X - \mu)^2 + 2E[(X - \mu)\mu] + E(\mu^{2}) \\ =& \sigma^2 + \mu^{2} \end{align*}

\begin{align*} \hat{\sigma}^{2} &=\frac{1}{n} \sum_{i=1}^{n}\left(X_{i}-\bar{X}\right)^{2} \\ &=\frac{1}{n} \sum_{i=1}^{n}\left(X_{i}^{2}-2 \bar{X} X_{i}+\bar{X}^{2}\right) \\ &=\frac{1}{n}\left(\sum_{i=1}^{n} X_{i}^{2}-2 \bar{X} \sum_{i=1}^{n} X_{i}+n \bar{X}^{2}\right) \\ &=\frac{1}{n}\left(\sum_{i=1}^{n} X_{i}^{2}-2 n \bar{X}^{2}+n \bar{X}^{2}\right) \\ &=\frac{1}{n}\left(\sum_{i=1}^{n} X_{i}^{2}-n \bar{X}^{2}\right) \\ &=\frac{1}{n} \sum_{i=1}^{n} X_{i}^{2}-\bar{X}^{2} \end{align*}

\begin{align*} \mathbb{E}\left(S_{*}^{2}\right) &=\frac{1}{n} \sum_{i=1}^{n} \mathbb{E}\left(X_{i}^{2}\right)-\mathbb{E}\left(\bar{X}^{2}\right) \\ &=\frac{1}{n} \sum_{i=1}^{n}\left(\mu^{2}+\sigma^{2}\right)-\left(\mu^{2}+\frac{\sigma^{2}}{n}\right) \\ &=\left(\mu^{2}+\sigma^{2}\right)-\left(\mu^{2}+\frac{\sigma^{2}}{n}\right) \\ &=\sigma^{2}-\frac{\sigma^{2}}{n} \\ &=\frac{n-1}{n} \cdot \sigma^{2} \end{align*}

\[ Var(X) = E(X^2) - E(X)^{2} \] thus we have:

\begin{align*} E(\bar{X}^2) &= Var(X) + E(\bar{X})^2 \\ &= \sigma^2 + E(\frac{\sum_{i=1}^nX}{n})^{2} \\ &= \sigma^2 + \frac{1}{n} n E(X) \\ &= \sigma^2 + \frac{1}{n} \sigma^{2} \end{align*}