MATHS@ZHAO Yin: R

显示标签为“R”的博文。显示所有博文

Basic Concept of Probability Distributions 8: Normal Distribution

PDF version

PDF & CDF

The probability density function is $$f(x; \mu, \sigma) = {1\over\sqrt{2\pi}\sigma}e^{-{1\over2}{(x-\mu)^2\over\sigma^2}}$$ The cumulative distribution function is defined by $$F(x; \mu, \sigma) = \Phi\left({x-\mu\over\sigma}\right)$$ where $$\Phi(z) = {1\over\sqrt{2\pi}} \int_{-\infty}^{z}e^{-{1\over2}x^2}\ dx$$
Proof:
$$
\begin{align*}
\int_{-\infty}^{\infty}f(x; \mu, \sigma) &= \int_{-\infty}^{\infty}{1\over\sqrt{2\pi}\sigma}e^{-{1\over2}{(x-\mu)^2\over\sigma^2}}\ dx\\
&= {1\over\sqrt{2\pi}\sigma}\int_{-\infty}^{\infty}e^{-{1\over2}{(x-\mu)^2\over\sigma^2}}\ dx\\
&= {1\over\sqrt{2\pi}}\int_{-\infty}^{\infty}e^{-{1\over2}y^2}\ dy\quad\quad\quad\quad\quad(\mbox{setting}\ y={x-\mu\over\sigma} \Rightarrow dx = \sigma dy)\\
\end{align*}
$$
Let $I = \int_{-\infty}^{\infty}e^{-{1\over2}y^2}\ dy$, then
$$
\begin{eqnarray*}
I^2 &=& \int_{-\infty}^{\infty}e^{-{1\over2}y^2}\ dy\int_{-\infty}^{\infty}e^{-{1\over2}x^2}\ dx\\
&=& \int_{-\infty}^{\infty}\int_{-\infty}^{\infty}e^{-{1\over2}(y^2+x^2)}\ dydx\quad\quad\quad\quad(\mbox{setting}\ x=r\cos\theta, y=r\sin\theta)\\
&=& \int_{0}^{\infty}\int_{0}^{2\pi}e^{-{1\over2}r^2}\ rd\theta dr \\
& & (\mbox{double integral}\ \iint\limits_{D}f(x, y)\ dxdy = \iint\limits_{D^*}f(r\cos\theta, r\sin\theta)r\ drd\theta) \\
&=& 2\pi\int_{0}^{\infty}re^{-{1\over2}r^2}\ dr\\
&=& -2\pi e^{-{1\over2}r^2}\Big|_{0}^{\infty}\\
&=& 2\pi
\end{eqnarray*}
$$
Hence $$\int_{-\infty}^{\infty}f(x; \mu, \sigma) = {1\over\sqrt{2\pi}} \cdot\sqrt{2\pi} = 1$$

Standard Normal Distribution

If $X$ is normally distributed with parameters $\mu$ and $\sigma^2$, then $$Z = {X-\mu\over\sigma}$$ is normally distributed with parameters 0 and 1.

Proof:
An important conclusion is that if $X$ is normally distributed with parameters $\mu$ and $\sigma^2$, then $Y = aX + b$ is normally distributed with parameters $a\mu + b$ and $a^2\sigma^2$. Denote $F_{Y}$ as the cumulative distribution function of $Y$:
$$
\begin{align*}
F_{Y}(x) &= P(Y \leq x)\\
&= P(aX + b \leq x)\\
&= P(X \leq {x-b\over a})\\
&= F_{X}\left({x-b\over a}\right)
\end{align*}
$$
where $F_{X}(x)$ is the cumulative distribution function of $X$. By differentiation, the probability density function of $Y$ is
$$
\begin{align*}
f_{Y}(x) &= {1\over a}f_{X}\left({x-b\over a}\right)\\
&= {1\over\sqrt{2\pi}a\sigma}e^{-{1\over2}{({x-b\over a} - \mu)^2\over \sigma^2}}\\
&= {1\over\sqrt{2\pi}(a\sigma)}e^{-{1\over2}{(x-b - a\mu)^2\over a^2\sigma^2}}\\
&= {1\over\sqrt{2\pi}(a\sigma)}e^{-{1\over2}{(x-(b + a\mu))^2\over (a\sigma)^2}}
\end{align*}
$$
which shows that $Y$ is normally distributed with parameters $a\mu + b$ and $a^2\sigma^2$.
According to the above result, we can easily deduce that $Z = {X-\mu\over\sigma}$ follows the normally distributed with parameters 0 and 1.

Mean

The expected value is $$E[X] = \mu$$
Proof:
$$
\begin{align*}
E[Z] &= \int_{-\infty}^{\infty}xf_{Z}(x)\ dx\quad\quad\quad \quad\quad \quad\quad (\mbox{setting}\ Z={X-\mu\over\sigma})\\
&= {1\over\sqrt{2\pi}}\int_{-\infty}^{\infty}xe^{-{1\over2}x^2}\ dx\\
&= -{1\over\sqrt{2\pi}}\int_{-\infty}^{\infty}e^{-{1\over2}x^2}\ d\left(-{1\over2}x^2\right)\\
&= -{1\over\sqrt{2\pi}}e^{-{1\over2}x^2}\Big|_{-\infty}^{\infty}\\
&= 0
\end{align*}
$$
Hence
$$
\begin{align*}
E[X] &= E\left[\sigma Z+\mu\right]\\
&= \sigma E[Z] + \mu\\
&= \mu
\end{align*}
$$
Variance

The variance is $$\mbox{Var}(X) = \sigma^2$$
Proof:
$$
\begin{align*}
E\left[Z^2\right] &= {1\over\sqrt{2\pi}}\int_{-\infty}^{\infty}x^2e^{-{1\over2}x^2}\ dx\quad\quad\quad \quad\quad \quad\quad\quad\quad\quad (\mbox{setting}\ Z={X-\mu\over\sigma})\\
&= {1\over\sqrt{2\pi}}\left(-xe^{-{1\over2}x^2}\Big|_{-\infty}^{\infty} +\int_{-\infty}^{\infty}e^{-{1\over2}x^2}\ dx\right)\quad\quad\quad(\mbox{integrating by parts})\\
&= {1\over\sqrt{2\pi}}\int_{-\infty}^{\infty}e^{-{1\over2}x^2}\ dx \quad\quad\quad\quad\quad\quad\quad(\mbox{standard normal distribution})\\
&= 1
\end{align*}
$$
the integral by parts: $$u= x,\ dv = xe^{-{1\over2}x^2}\ dx$$ $$\implies du = dx,\ v = \int xe^{-{1\over2}x^2}\ dx = -e^{-{1\over2}x^2}$$ $$\implies \int x^2e^{-{1\over2}x^2}\ dx =-xe^{-{1\over2}x^2} +\int e^{-{1\over2}x^2}\ dx$$ Hence $$\mbox{Var}(X) = \mbox{Var}(\sigma Z + \mu)= \sigma^2\mbox{Var}(Z) = \sigma^2$$