The Negative Binomial Distribution: pmf, mgf, mean and variance.


The Negative Binomial Distribution


The probability mass function of the negative binomial distribution

Consider the situation where one performs a number Bernoulli trials, each trial has a probability of success p, and
trials continue until the rth successs occurs. Let X be the random variable which is the number of trials up to
and including the rth success. This means that the range of X is the set \{ r, r+1, \ldots \}. Then the pmf would be given by {{x-1}\choose{r-1}} p^r q^{x-r}.

Note that there are r successes and x-r previous failures, with the last success at a fixed rth position. Thus the number of possible outcomes is the number of combinations of selecting (x-1) objects taken (r-1) at a time.

Top pmf mgf mean variance


The moment generating function of the negative binomial distribution

What is the mgf of the negative binomial distribution? Let us compute the value of

M_Xt = E(tX) = \sum _{x=r}^{\infty} e^{tx}{{x-1}\choose{r-1}} p^r q^{x-r}.

Writing the first few terms of the expansion, we get

M_Xt = e^{tr}{{r-1}\choose{r-1}} p^r q^{r-r} +e^{t(r+1)}{{r}\choose{r-1}} p^r q^{r+1-r}<br />
       +e^{t(r+2)}{{r+1}\choose{r-1}} p^r q^{r+2-r}+\ldots


which simplifies to

E(tX) = e^{tr} p^r + e^{t(r+1)} {r \choose {1}} p^r q<br />
       +e^{t(r+2)} {(r+1) \choose{2}}  p^r q^{2}+\ldots

and which upon factoring out p^re^{tr} = (pe^t)^r and further simplication results in

(pe^t)^r [ 1  +  {r \choose {1}} (q e^t)<br />
       +  {(r+1) \choose{2}}  (q e^t)^{2}+\ldots]


It will be shown later that the bracketed terms is equivalent to
(1-qe^t)^{-r}, see the blog entry A negative binomial series identity and therefore the moment generating function of the negative binomial distribution
is given by

M_Xt =  \frac{(pe^t)^r}{(1-qe^t)^r}

Note that the numerator in the formula of 
Spiegel's Statistics in page. 118 should be raised to the power
r.
Top pmf mgf mean variance


The mean of the Negative Binomial Distribution

The expectation or mean of the negative binomial distribution with the pmf and mgf above is obtained by differentiating the mgf wrt t and setting t to zero: E(X) = \mu=  M'_X(t)|{t = 0}.

When differentiated, the derivative is
\frac{(1-qe^t)^r r(pe^t)^{r-1}pe^t + (pe^t)^r  r qe^t{(1-qe^t)^{r-1}}}{(1-qe^t)^{2r}}
and the value at t=0 is

\frac{r p^{2r-1}( p+q)}{p^2r} which collapses to \mu = \frac{r}{p}

Top pmf mgf mean variance


The variance of the Negative Binomial Distribution

The second moment or E(X^2) of the negative binomial distribution with the pmf and mgf above is obtained by differentiating the mgf twice wrt t and setting t to zero and the variance is computed as

V(X) = E(X^2) -[E(X)]^2.

We will leave as an exercise (at the moment since it is so tedious) that the variance is given by

V(X) = \frac{rq}{p^2}.

This entry is subject to review but the final formulas are all right.

Top pmf mgf mean variance

Revisions.

Feb. 20, 2010: We missed the square in the denominator! We will redo the presentation for the variance.

3 Responses to “The Negative Binomial Distribution: pmf, mgf, mean and variance.”

  1. Ramadan Ally Says:

    It will be better if you will provide to us the derivation of variance of negative binomial distribution by using raw moment.. (I'm from Tanzania}

  2. ernie Says:

    Thanks, will take a look at this again before January is over. Reader feedback is important to us.

  3. Lee Corbin Says:

    Very nice article. There is a typo in the
    equation following "which simplifies to"
    in the exponent of the 3rd term. Thank you.