PDF version
PMF
Suppose that a sample of size $n$ is to be chosen randomly (without replacement) from an urn containing $N$ balls, of which $m$ are white and $N-m$ are black. If we let $X$ denote the number of white balls selected, then $$f(x; N, m, n) = \Pr(X = x) = {{m\choose x}{N-m\choose n-x}\over {N\choose n}}$$ for $x= 0, 1, 2, \cdots, n$.
Proof:
This is essentially the Vandermonde's identity:
$${m+n\choose r} = \sum_{k=0}^{r}{m\choose k}{n\choose r-k}$$ where $m$, $n$, $k$, $r\in \mathbb{N}_0$.
Because
$$
\begin{align*}
\sum_{r=0}^{m+n}{m+n\choose r}x^r &= (1+x)^{m+n} \quad\quad\quad\quad\quad\quad\quad\quad \mbox{(binomial theorem)}\\
&= (1+x)^m(1+x)^n\\
&= \left(\sum_{i=0}^{m}{m\choose i}x^{i}\right)\left(\sum_{j=0}^{n}{n\choose j}x^{j}\right)\\
&= \sum_{r=0}^{m+n}\left(\sum_{k=0}^{r}{m\choose k}{n\choose r-k}\right)x^r \quad\quad\mbox{(product of two binomials)}
\end{align*}
$$
Using the product of two binomials:
$$
\begin{eqnarray*}
\left(\sum_{i=0}^{m}a_i x^i\right)\left(\sum_{j=0}^{n}b_j x^j\right) &=& \left(a_0+a_1x+\cdots + a_mx^m\right)\left(b_0+b_1x+\cdots + b_nx^n\right)\\
&=& a_0b_0 + a_0b_1x +a_1b_0x +\cdots +a_0b_2x^2 + a_1b_1x^2 + a_2b_0x^2 +\\
& &\cdots + a_mb_nx^{m+n}\\
&=& \sum_{r=0}^{m+n}\left(\sum_{k=0}^{r}a_{k}b_{r-k}\right)x^{r}
\end{eqnarray*}
$$
Hence
$$
\begin{eqnarray*}
& &\sum_{r=0}^{m+n}{m+n\choose r}x^r = \sum_{r=0}^{m+n}\left(\sum_{k=0}^{r}{m\choose k}{n\choose r-k}\right)x^r\\
&\implies& {m+n\choose r} = \sum_{k=0}^{r}{m\choose k}{n\choose r-k}\\
& \implies& \sum_{k=0}^{r}{{m\choose k}{n\choose r-k}\over {m+n\choose r}} = 1
\end{eqnarray*}
$$
Mean
The expected value is $$\mu = E[X] = {nm\over N}$$
Proof:
$$
\begin{eqnarray*}
E[X^k] &=& \sum_{x=0}^{n}x^kf(x; N, m, n)\\
&=& \sum_{x=0}^{n}x^k{{m\choose x}{N-m\choose n-x}\over {N\choose n}}\\
&=& {nm\over N}\sum_{x=0}^{n} x^{k-1} {{m-1 \choose x-1}{N-m\choose n-x}\over {N-1 \choose n-1}}\\
& & (\mbox{identities:}\ x{m\choose x} = m{m-1\choose x-1},\ n{N\choose n} = N{N-1\choose n-1})\\
&=& {nm\over N}\sum_{x=0}^{n} (y+1)^{k-1} {{m-1 \choose y}{(N-1) - (m - 1)\choose (n-1)-y}\over {N-1 \choose n-1}}\quad\quad(\mbox{setting}\ y=x-1)\\
&=& {nm\over N}E\left[(Y+1)^{k-1}\right] \quad\quad\quad \quad\quad \quad\quad\quad\quad (\mbox{since}\ Y\sim g(y; m-1, n-1, N-1))
\end{eqnarray*}
$$
Hence, setting $k=1$ we have $$E[X] = {nm\over N}$$ Note that this follows the mean of the binomial distribution $\mu = np$, where $p = {m\over N}$.
Variance
The variance is $$\sigma^2 = \mbox{Var}(X) = np(1-p)\left(1 - {n-1 \over N-1}\right)$$ where $p = {m\over N}$.
Proof:
$$
\begin{align*}
E[X^2] &= {nm\over N}E[Y+1] \quad\quad\quad \quad\quad\quad \quad (\mbox{setting}\ k=2)\\
&= {nm\over N}\left(E[Y] + 1\right)\\
& = {nm\over N}\left[{(n-1) (m-1) \over N-1}+1\right]
\end{align*}
$$
Hence the variance is
$$
\begin{align*}
\mbox{Var}(X) &= E\left[X^2\right] - E[X]^2\\
&= {mn\over N}\left[{(n-1) (m-1) \over N-1}+1 - {nm\over N}\right]\\
&= np \left[ (n-1) \cdot {pN-1\over N-1}+1-np\right] \quad\quad \quad \quad \quad\quad(\mbox{setting}\ p={m\over N})\\
&= np\left[(n-1)\cdot {p(N-1) + p -1 \over N-1} + 1 -np\right]\\
&= np\left[(n-1)p + (n-1)\cdot{p-1 \over N-1} + 1-np\right]\\
&= np\left[1-p - (1-p)\cdot {n-1\over N-1}\right] \\
&= np(1-p)\left(1 - {n-1 \over N-1}\right)
\end{align*}
$$
Note that it is approximately equal to 1 when $N$ is sufficient large (i.e. ${n-1\over N-1}\rightarrow 0$ when $N\rightarrow +\infty$). And then it is the same as the variance of the binomial distribution $\sigma^2 = np(1-p)$, where $p = {m\over N}$.
Basic Concept of Probability Distributions 4: Negative Binomial Distribution
PDF version
PMF
Suppose there is a sequence of independent Bernoulli trials, each trial having two potential outcomes called "success" and "failure". In each trial the probability of success is $p$ and of failure is $(1-p)$. We are observing this sequence until a predefined number $r$ of failures has occurred. Then the random number of successes we have seen, $X$, will have the negative binomial (or Pascal) distribution: $$f(x; r, p) = \Pr(X=x) = {x + r-1\choose x}p^{x}(1-p)^{r}$$ for $x = 0, 1, 2, \cdots$.
Proof:
$$
\begin{align*}
\sum_{x =0}^{\infty}P(X = x) &= \sum_{x= 0}^{\infty} {x + r-1\choose x}p^{x}(1-p)^{r}\\
&= (1-p)^{r}\sum_{x=0}^{\infty} (-1)^{x}{-r\choose x}p^{x}\;\;\quad\quad (\mbox{identity}\ (-1)^{x}{-r\choose x}= {x+r-1\choose x})\\
&= (1-p)^r(1-p)^{-r}\;\;\quad\quad\quad\quad\quad\quad (\mbox{binomial theorem})\\
&= 1
\end{align*}
$$
Using the identity $(-1)^{x}{-r\choose x}= {x+r-1\choose x}$:
$$
\begin{align*}
{x+r-1\choose x} &= {(x+r-1)!\over x!(r-1)!}\\
&= {(x+r-1)(x+r-2) \cdots r\over x!}\\
&= (-1)^{x}{(-r-(x-1))(-r-(x-2))\cdots(-r)\over x!}\\
&= (-1)^{x}{(-r)(-r-1)\cdots(-r-(x-1))\over x!}\\
&= (-1)^{x}{(-r)(-r-1)\cdots(-r-(x-1))(-r-x)!\over x!(-r-x)!}\\
&=(-1)^{x}{-r\choose x}
\end{align*}
$$
Mean
The expected value is $$\mu = E[X] = {rp\over 1-p}$$
Proof:
$$
\begin{align*}
E[X] &= \sum_{x=0}^{\infty}xf(x; r, p)\\
&= \sum_{x=0}^{\infty}x{x + r-1\choose x}p^{x}(1-p)^{r}\\
&=\sum_{x=1}^{\infty}{(x+r-1)!\over(r-1)!(x-1)!}p^{x}(1-p)^{r}\\
&=\sum_{x=1}^{\infty}r{(x+r-1)!\over r(r-1)!(x-1)!}p^{x}(1-p)^{r}\\
&= {rp\over 1-p}\sum_{x=1}^{\infty}{x + r-1\choose x-1}p^{x-1}(1-p)^{r+1}\\
&={rp\over 1-p}\sum_{y=0}^{\infty}{y+(r+1)-1\choose y}p^{y}(1-p)^{r+1}\quad\quad\quad \mbox{setting}\ y= x-1\\
&= {rp\over 1-p}
\end{align*}
$$
where the last summation follows $Y\sim\mbox{NB}(r+1; p)$.
Variance
The variance is $$\sigma^2 = \mbox{Var}(X) = {rp\over(1-p)^2}$$
Proof:
$$
\begin{align*}
E\left[X^2\right] &= \sum_{x=0}^{\infty}x^2f(x; r, p)\\
&= \sum_{x=0}^{\infty}x^2{x + r-1\choose x}p^{x}(1-p)^{r}\\
&=\sum_{x=1}^{\infty}x{(x+r-1)!\over(r-1)!(x-1)!}p^{x}(1-p)^{r}\\
&=\sum_{x=1}^{\infty}rx{(x+r-1)!\over r(r-1)!(x-1)!}p^{x}(1-p)^{r}\\
&= {rp\over 1-p}\sum_{x=1}^{\infty}x{x + r-1\choose x-1}p^{x-1}(1-p)^{r+1}\\
&={rp\over 1-p}\sum_{y=0}^{\infty}(y+1){y+(r+1)-1\choose y}p^{y}(1-p)^{r+1}\quad\quad\quad (\mbox{setting}\ y= x-1)\\
&= {rp\over 1-p}\left(\sum_{y=0}^{\infty}y{y+(r+1)-1\choose y}p^{y}(1-p)^{r+1}+\sum_{y=0}^{\infty}{y+(r+1)-1\choose y}p^{y}(1-p)^{r+1} \right)\\
&= {rp\over 1-p}\left({(r+1)p\over 1-p} + 1\right)\quad\quad\quad\quad\quad\quad(Y\sim\mbox{NB}(r+1; p),\ E[Y] = {(r+1)p\over1-p})\\
&= {rp\over 1-p}\cdot{rp+1\over 1-p}
\end{align*}
$$
Thus the variance is
$$
\begin{align*}
\mbox{Var}(X) &= E\left[X^2\right] - E[X]^2\\
&= {rp\over 1-p}\cdot{rp+1\over 1-p}- \left({rp\over 1-p}\right)^2\\
&= {rp\over 1-p}\left({rp+1\over 1-p} - {rp\over 1-p}\right)\\
&= {rp\over(1-p)^2}
\end{align*}
$$
PMF
Suppose there is a sequence of independent Bernoulli trials, each trial having two potential outcomes called "success" and "failure". In each trial the probability of success is $p$ and of failure is $(1-p)$. We are observing this sequence until a predefined number $r$ of failures has occurred. Then the random number of successes we have seen, $X$, will have the negative binomial (or Pascal) distribution: $$f(x; r, p) = \Pr(X=x) = {x + r-1\choose x}p^{x}(1-p)^{r}$$ for $x = 0, 1, 2, \cdots$.
Proof:
$$
\begin{align*}
\sum_{x =0}^{\infty}P(X = x) &= \sum_{x= 0}^{\infty} {x + r-1\choose x}p^{x}(1-p)^{r}\\
&= (1-p)^{r}\sum_{x=0}^{\infty} (-1)^{x}{-r\choose x}p^{x}\;\;\quad\quad (\mbox{identity}\ (-1)^{x}{-r\choose x}= {x+r-1\choose x})\\
&= (1-p)^r(1-p)^{-r}\;\;\quad\quad\quad\quad\quad\quad (\mbox{binomial theorem})\\
&= 1
\end{align*}
$$
Using the identity $(-1)^{x}{-r\choose x}= {x+r-1\choose x}$:
$$
\begin{align*}
{x+r-1\choose x} &= {(x+r-1)!\over x!(r-1)!}\\
&= {(x+r-1)(x+r-2) \cdots r\over x!}\\
&= (-1)^{x}{(-r-(x-1))(-r-(x-2))\cdots(-r)\over x!}\\
&= (-1)^{x}{(-r)(-r-1)\cdots(-r-(x-1))\over x!}\\
&= (-1)^{x}{(-r)(-r-1)\cdots(-r-(x-1))(-r-x)!\over x!(-r-x)!}\\
&=(-1)^{x}{-r\choose x}
\end{align*}
$$
Mean
The expected value is $$\mu = E[X] = {rp\over 1-p}$$
Proof:
$$
\begin{align*}
E[X] &= \sum_{x=0}^{\infty}xf(x; r, p)\\
&= \sum_{x=0}^{\infty}x{x + r-1\choose x}p^{x}(1-p)^{r}\\
&=\sum_{x=1}^{\infty}{(x+r-1)!\over(r-1)!(x-1)!}p^{x}(1-p)^{r}\\
&=\sum_{x=1}^{\infty}r{(x+r-1)!\over r(r-1)!(x-1)!}p^{x}(1-p)^{r}\\
&= {rp\over 1-p}\sum_{x=1}^{\infty}{x + r-1\choose x-1}p^{x-1}(1-p)^{r+1}\\
&={rp\over 1-p}\sum_{y=0}^{\infty}{y+(r+1)-1\choose y}p^{y}(1-p)^{r+1}\quad\quad\quad \mbox{setting}\ y= x-1\\
&= {rp\over 1-p}
\end{align*}
$$
where the last summation follows $Y\sim\mbox{NB}(r+1; p)$.
Variance
The variance is $$\sigma^2 = \mbox{Var}(X) = {rp\over(1-p)^2}$$
Proof:
$$
\begin{align*}
E\left[X^2\right] &= \sum_{x=0}^{\infty}x^2f(x; r, p)\\
&= \sum_{x=0}^{\infty}x^2{x + r-1\choose x}p^{x}(1-p)^{r}\\
&=\sum_{x=1}^{\infty}x{(x+r-1)!\over(r-1)!(x-1)!}p^{x}(1-p)^{r}\\
&=\sum_{x=1}^{\infty}rx{(x+r-1)!\over r(r-1)!(x-1)!}p^{x}(1-p)^{r}\\
&= {rp\over 1-p}\sum_{x=1}^{\infty}x{x + r-1\choose x-1}p^{x-1}(1-p)^{r+1}\\
&={rp\over 1-p}\sum_{y=0}^{\infty}(y+1){y+(r+1)-1\choose y}p^{y}(1-p)^{r+1}\quad\quad\quad (\mbox{setting}\ y= x-1)\\
&= {rp\over 1-p}\left(\sum_{y=0}^{\infty}y{y+(r+1)-1\choose y}p^{y}(1-p)^{r+1}+\sum_{y=0}^{\infty}{y+(r+1)-1\choose y}p^{y}(1-p)^{r+1} \right)\\
&= {rp\over 1-p}\left({(r+1)p\over 1-p} + 1\right)\quad\quad\quad\quad\quad\quad(Y\sim\mbox{NB}(r+1; p),\ E[Y] = {(r+1)p\over1-p})\\
&= {rp\over 1-p}\cdot{rp+1\over 1-p}
\end{align*}
$$
Thus the variance is
$$
\begin{align*}
\mbox{Var}(X) &= E\left[X^2\right] - E[X]^2\\
&= {rp\over 1-p}\cdot{rp+1\over 1-p}- \left({rp\over 1-p}\right)^2\\
&= {rp\over 1-p}\left({rp+1\over 1-p} - {rp\over 1-p}\right)\\
&= {rp\over(1-p)^2}
\end{align*}
$$
Basic Concept of Probability Distributions 3: Geometric Distribution
PDF version
PMF
Suppose that independent trials, each having a probability $p$, $0 < p < 1$, of being a success, are performed until a success occurs. If we let $X$ equal the number of failures required, then the geometric distribution mass function is $$f(x; p) =\Pr(X=x) = (1-p)^{x}p$$ for $x=0, 1, 2, \cdots$.
Proof:
$$
\begin{align*}
\sum_{x=0}^{\infty}f(x; p) &= \sum_{x=0}^{\infty}(1-p)^{x}p\\
&= p\sum_{x=0}^{\infty}(1-p)^{x}\\
& = p\cdot {1\over 1-(1-p)}\\
& = 1
\end{align*}
$$
Mean
The expected value is $$\mu = E[X] = {1-p\over p}$$
Proof:
Firstly, we know that $$\sum_{x=0}^{\infty}p^x = {1\over 1-p}$$ where $0 < p < 1$. Thus $$ \begin{align*} {d\over dp}\sum_{x=0}^{\infty}p^x &= \sum_{x=1}^{\infty}xp^{x-1}\\ &= {1\over(1-p)^2} \end{align*} $$ The expected value is $$ \begin{align*} E[X] &= \sum_{x=0}^{\infty}x(1-p)^{x}p\\ &=p(1-p)\sum_{x=1}^{\infty}x(1-p)^{x-1}\\ &= p(1-p){1\over(1-(1-p))^2}\\ &= {1-p\over p} \end{align*} $$ Variance
The variance is $$\sigma^2 = \mbox{Var}(X) = {1-p\over p^2}$$
Proof:
$$
\begin{align*}
E\left[X^2\right] &=\sum_{x=0}^{\infty}x^2(1-p)^{x}p\\
&= (1-p)\sum_{x=1}^{\infty}x^2(1-p)^{x-1}p
\end{align*}
$$
Rewrite the right hand summation as
$$
\begin{align*}
\sum_{x=1}^{\infty} x^2(1-p)^{x-1}p&= \sum_{x=1}^{\infty} (x-1+1)^2(1-p)^{x-1}p\\
&= \sum_{x=1}^{\infty} (x-1)^2(1-p)^{x-1}p + \sum_{x=1}^{\infty} 2(x-1)(1-p)^{x-1}p + \sum_{x=1}^{\infty} (1-p)^{x-1}p\\
&= E\left[X^2\right] + 2E[X] + 1\\
&= E\left[X^2\right] + {2-p\over p}
\end{align*}
$$
Thus $$E\left[X^2\right] = (1-p)E\left[X^2\right] + {(1-p)(2-p) \over p}$$ That is $$E\left[X^2\right]= {(1-p)(2-p)\over p^2}$$
So the variance is
$$
\begin{align*}
\mbox{Var}(X) &= E\left[X^2\right] - E[X]^2\\
&= {(1-p)(2-p)\over p^2} - {(1-p)^2\over p^2}\\
&= {1-p\over p^2}
\end{align*}
$$
PMF
Suppose that independent trials, each having a probability $p$, $0 < p < 1$, of being a success, are performed until a success occurs. If we let $X$ equal the number of failures required, then the geometric distribution mass function is $$f(x; p) =\Pr(X=x) = (1-p)^{x}p$$ for $x=0, 1, 2, \cdots$.
Proof:
$$
\begin{align*}
\sum_{x=0}^{\infty}f(x; p) &= \sum_{x=0}^{\infty}(1-p)^{x}p\\
&= p\sum_{x=0}^{\infty}(1-p)^{x}\\
& = p\cdot {1\over 1-(1-p)}\\
& = 1
\end{align*}
$$
Mean
The expected value is $$\mu = E[X] = {1-p\over p}$$
Proof:
Firstly, we know that $$\sum_{x=0}^{\infty}p^x = {1\over 1-p}$$ where $0 < p < 1$. Thus $$ \begin{align*} {d\over dp}\sum_{x=0}^{\infty}p^x &= \sum_{x=1}^{\infty}xp^{x-1}\\ &= {1\over(1-p)^2} \end{align*} $$ The expected value is $$ \begin{align*} E[X] &= \sum_{x=0}^{\infty}x(1-p)^{x}p\\ &=p(1-p)\sum_{x=1}^{\infty}x(1-p)^{x-1}\\ &= p(1-p){1\over(1-(1-p))^2}\\ &= {1-p\over p} \end{align*} $$ Variance
The variance is $$\sigma^2 = \mbox{Var}(X) = {1-p\over p^2}$$
Proof:
$$
\begin{align*}
E\left[X^2\right] &=\sum_{x=0}^{\infty}x^2(1-p)^{x}p\\
&= (1-p)\sum_{x=1}^{\infty}x^2(1-p)^{x-1}p
\end{align*}
$$
Rewrite the right hand summation as
$$
\begin{align*}
\sum_{x=1}^{\infty} x^2(1-p)^{x-1}p&= \sum_{x=1}^{\infty} (x-1+1)^2(1-p)^{x-1}p\\
&= \sum_{x=1}^{\infty} (x-1)^2(1-p)^{x-1}p + \sum_{x=1}^{\infty} 2(x-1)(1-p)^{x-1}p + \sum_{x=1}^{\infty} (1-p)^{x-1}p\\
&= E\left[X^2\right] + 2E[X] + 1\\
&= E\left[X^2\right] + {2-p\over p}
\end{align*}
$$
Thus $$E\left[X^2\right] = (1-p)E\left[X^2\right] + {(1-p)(2-p) \over p}$$ That is $$E\left[X^2\right]= {(1-p)(2-p)\over p^2}$$
So the variance is
$$
\begin{align*}
\mbox{Var}(X) &= E\left[X^2\right] - E[X]^2\\
&= {(1-p)(2-p)\over p^2} - {(1-p)^2\over p^2}\\
&= {1-p\over p^2}
\end{align*}
$$
订阅：
博文 (Atom)