Negative binomial distribution

From Wikipedia, the free encyclopedia

https://en.wikipedia.org/wiki/Negative_binomial_distribution 

 

Different texts (and even different parts of this article) adopt slightly different definitions for the negative binomial distribution. They can be distinguished by whether the support starts at k = 0 or at k = r, whether p denotes the probability of a success or of a failure, and whether r represents success or failure, so identifying the specific parametrization used is crucial in any given text.
Probability mass function The orange line represents the mean, which is equal to 10 in each of these plots; the green line shows the standard deviation.
Notation	${\displaystyle \mathrm{N}\mathrm{B}(r,p)}$
Parameters	r > 0 — number of successes until the experiment is stopped (integer, but the definition can also be extended to reals) p ∈ [0,1] — success probability in each experiment (real)
Support	k ∈ { 0, 1, 2, 3, … } — number of failures
PMF	${\displaystyle k\mapsto (\genfrac{}{}{0ex}{}{k+r-1}{k})\cdot (1-p{)}^k{p}^r,}$ involving a binomial coefficient
CDF	${\displaystyle k\mapsto I_p(r,k+1),}$ the regularized incomplete beta function
Mean	${\displaystyle \frac{r(1-p)}{p}}$
Mode	${\displaystyle \{\begin{array}{ll}\lfloor \frac{(r-1)(1-p)}{p}\rfloor & \text{if }r>1\\ 0& \text{if }r\le 1\end{array}}$
Variance	${\displaystyle \frac{r(1-p)}{p^2}}$
Skewness	${\displaystyle \frac{2-p}{\sqrt{(1-p)r}}}$
Ex. kurtosis	${\displaystyle \frac{6}{r}+\frac{p^2}{(1-p)r}}$
MGF	${\displaystyle (\frac{p}{1-(1-p)e^t}{)}^r\text{ for }t<-\log (1-p)}$
CF	${\displaystyle (\frac{p}{1-(1-p)e^{it}}{)}^r\text{ with }t\in \mathbb{R}}$
PGF	${\displaystyle (\frac{p}{1-(1-p)z}{)}^r\text{ for }|z|<\frac{1}{p}}$
Fisher information	${\displaystyle \frac{r}{p^2(1-p)}}$
Method of Moments	${\displaystyle r=\frac{E[X{]}^2}{V[X]-E[X]}}$ ${\displaystyle p=1-\frac{E[X]}{V[X]}}$

In probability theory and statistics, the negative binomial distribution is a discrete probability distribution that models the number of failures in a sequence of independent and identically distributed Bernoulli trials before a specified (non-random) number of successes (denoted ${\displaystyle r}$) occurs. For example, we can define rolling a 6 on a die as a success, and rolling any other number as a failure, and ask how many failure rolls will occur before we see the third success (${\displaystyle r=3}$). In such a case, the probability distribution of the number of failures that appear will be a negative binomial distribution.

An alternative formulation is to model the number of total trials (instead of the number of failures). In fact, for a specified (non-random) number of successes (r), the number of failures (n − r) are random because the total trials (n) are random. For example, we could use the negative binomial distribution to model the number of days n (random) a certain machine works (specified by r) before it breaks down.

The Pascal distribution (after Blaise Pascal) and Polya distribution (for George Pólya) are special cases of the negative binomial distribution. A convention among engineers, climatologists, and others is to use "negative binomial" or "Pascal" for the case of an integer-valued stopping-time parameter (${\displaystyle r}$) and use "Polya" for the real-valued case.

For occurrences of associated discrete events, like tornado outbreaks, the Polya distributions can be used to give more accurate models than the Poisson distribution by allowing the mean and variance to be different, unlike the Poisson. The negative binomial distribution has a variance ${\displaystyle \mu /p}$, with the distribution becoming identical to Poisson in the limit ${\displaystyle p\to 1}$ for a given mean ${\displaystyle \mu }$ (i.e. when the failures are increasingly rare). This can make the distribution a useful overdispersed alternative to the Poisson distribution, for example for a robust modification of Poisson regression. In epidemiology, it has been used to model disease transmission for infectious diseases where the likely number of onward infections may vary considerably from individual to individual and from setting to setting. More generally, it may be appropriate where events have positively correlated occurrences causing a larger variance than if the occurrences were independent, due to a positive covariance term.

The term "negative binomial" is likely due to the fact that a certain binomial coefficient that appears in the formula for the probability mass function of the distribution can be written more simply with negative numbers.

Definitions

Imagine a sequence of independent Bernoulli trials: each trial has two potential outcomes called "success" and "failure." In each trial the probability of success is ${\displaystyle p}$ and of failure is ${\displaystyle 1-p}$. We observe this sequence until a predefined number ${\displaystyle r}$ of successes occurs. Then the random number of observed failures, ${\displaystyle X}$, follows the negative binomial (or Pascal) distribution:

${\displaystyle X\sim \mathrm{NB}(r,p)}$

Probability mass function

The probability mass function of the negative binomial distribution is

${\displaystyle f(k;r,p)\equiv Pr(X=k)=(\genfrac{}{}{0ex}{}{k+r-1}{k})(1-p{)}^k{p}^r}$

where r is the number of successes, k is the number of failures, and p is the probability of success on each trial.

Here, the quantity in parentheses is the binomial coefficient, and is equal to

${\displaystyle (\genfrac{}{}{0ex}{}{k+r-1}{k})=\frac{(k+r-1)!}{(r-1)!(k)!}=\frac{(k+r-1)(k+r-2)\cdots (r)}{k!}.}$

There are k failures chosen from k + r − 1 trials rather than k + r because the last of the k + r trials is by definition a success.

This quantity can alternatively be written in the following manner, explaining the name "negative binomial":

${\displaystyle \begin{array}{rl}& \frac{(k+r-1)\cdots (r)}{k!}\\ =& (-1{)}^k\frac{\stackrel{k\text{ factors}}{\overbrace{(-r)(-r-1)(-r-2)\cdots (-r-k+1)}}}{k!}=(-1{)}^k(\genfrac{}{}{0ex}{}{-r}{\phantom{}k}).\end{array}}$

Note that by the last expression and the binomial series, for every 0 ≤ p < 1 and ${\displaystyle q=1-p}$,

${\displaystyle p^{-r}=(1-q{)}^{-r}=\sum _{k=0}^{\mathrm{\infty }}(\genfrac{}{}{0ex}{}{-r}{\phantom{}k})(-q{)}^k=\sum _{k=0}^{\mathrm{\infty }}(\genfrac{}{}{0ex}{}{k+r-1}{k})q^k}$

hence the terms of the probability mass function indeed add up to one as below.

${\displaystyle \sum _{k=0}^{\mathrm{\infty }}(\genfrac{}{}{0ex}{}{k+r-1}{k})(1-p{)}^k{p}^r=p^{-r}{p}^r=1}$

To understand the above definition of the probability mass function, note that the probability for every specific sequence of r successes and k failures is p^r(1 − p)^k, because the outcomes of the k + r trials are supposed to happen independently. Since the rth success always comes last, it remains to choose the k trials with failures out of the remaining k + r − 1 trials. The above binomial coefficient, due to its combinatorial interpretation, gives precisely the number of all these sequences of length k + r − 1.

Cumulative distribution function

The cumulative distribution function can be expressed in terms of the regularized incomplete beta function:

${\displaystyle F(k;r,p)\equiv Pr(X\le k)=I_p(r,k+1).}$

(This formula is using the same parameterization as in the article's table, with r the number of successes, and ${\displaystyle p=r/(r+\mu )}$ with ${\displaystyle \mu }$ the mean.)

It can also be expressed in terms of the cumulative distribution function of the binomial distribution:

${\displaystyle F(k;r,p)=F_{\text{binomial}}(k;n=k+r,1-p).}$

Alternative formulations

Some sources may define the negative binomial distribution slightly differently from the primary one here. The most common variations are where the random variable X is counting different things. These variations can be seen in the table here:

	X is counting...	Probability mass function	Formula	Alternate formula (using equivalent binomial)	Alternate formula (simplified using: $n=k+r$)	Support
1	k failures, given r successes	$f(k;r,p)\equiv Pr(X=k)=$	$(\genfrac{}{}{0ex}{}{k+r-1}{k})p^r(1-p{)}^k$	$(\genfrac{}{}{0ex}{}{k+r-1}{r-1})p^r(1-p{)}^k$	$(\genfrac{}{}{0ex}{}{n-1}{k})p^r(1-p{)}^k$	${\displaystyle \text{for }k=0,1,2,\dots }$
2	n trials, given r successes	$f(n;r,p)\equiv Pr(X=n)=$	$(\genfrac{}{}{0ex}{}{n-1}{r-1})p^r(1-p{)}^{n-r}$	$(\genfrac{}{}{0ex}{}{n-1}{n-r})p^r(1-p{)}^{n-r}$		${\displaystyle \text{for }n=r,r+1,r+2,\dots }$
3	n trials, given r failures	$f(n;r,p)\equiv Pr(X=n)=$	$(\genfrac{}{}{0ex}{}{n-1}{r-1})p^{n-r}(1-p{)}^r$	$(\genfrac{}{}{0ex}{}{n-1}{n-r})p^{n-r}(1-p{)}^r$	$(\genfrac{}{}{0ex}{}{n-1}{k})p^k(1-p{)}^r$
4	k successes, given r failures	$f(k;r,p)\equiv Pr(X=k)=$	$(\genfrac{}{}{0ex}{}{k+r-1}{k})p^k(1-p{)}^r$	$(\genfrac{}{}{0ex}{}{k+r-1}{r-1})p^k(1-p{)}^r$		${\displaystyle \text{for }k=0,1,2,\dots }$
-	k successes, given n trials	$f(k;n,p)\equiv Pr(X=k)=$	This is the binomial distribution not the negative binomial: $(\genfrac{}{}{0ex}{}{n}{k})p^k(1-p{)}^{n-k}$			${\displaystyle \text{for }k=0,1,2,\dots ,n}$

Each of the four definitions of the negative binomial distribution can be expressed in slightly different but equivalent ways. The first alternative formulation is simply an equivalent form of the binomial coefficient, that is: $(\genfrac{}{}{0ex}{}{a}{b})=(\genfrac{}{}{0ex}{}{a}{a-b})\text{for }0\le b\le a$. The second alternate formulation somewhat simplifies the expression by recognizing that the total number of trials is simply the number of successes and failures, that is: $n=r+k$. These second formulations may be more intuitive to understand, however they are perhaps less practical as they have more terms.

• The definition where X is the number of n trials that occur for a given number of r successes is similar to the primary definition, except that the number of trials is given instead of the number of failures. This adds r to the value of the random variable, shifting its support and mean.
• The definitiond where X is the number of k successes (or n trials) that occur for a given number of r failures is similar to the primary definition used in this article, except that numbers of failures and successes are switched when considering what is being counted and what is given. Note however, that p still refers to the probability of "success".
• The definition of the negative binomial distribution can be extended to the case where the parameter r can take on a positive real value. Although it is impossible to visualize a non-integer number of "failures", we can still formally define the distribution through its probability mass function. The problem of extending the definition to real-valued (positive) r boils down to extending the binomial coefficient to its real-valued counterpart, based on the gamma function:

${\displaystyle (\genfrac{}{}{0ex}{}{k+r-1}{k})=\frac{(k+r-1)(k+r-2)\cdots (r)}{k!}=\frac{\mathrm{\Gamma }(k+r)}{k!\mathrm{\Gamma }(r)}}$

After substituting this expression in the original definition, we say that X has a negative binomial (or Pólya) distribution if it has a probability mass function:

${\displaystyle f(k;r,p)\equiv Pr(X=k)=\frac{\mathrm{\Gamma }(k+r)}{k!\mathrm{\Gamma }(r)}(1-p{)}^k{p}^r\text{for }k=0,1,2,\dots }$

Here r is a real, positive number.

In negative binomial regression, the distribution is specified in terms of its mean, $m=\frac{r(1-p)}{p}$, which is then related to explanatory variables as in linear regression or other generalized linear models. From the expression for the mean m, one can derive $p=\frac{r}{m+r}$ and $1-p=\frac{m}{m+r}$. Then, substituting these expressions in the one for the probability mass function when r is real-valued, yields this parametrization of the probability mass function in terms of m:

${\displaystyle Pr(X=k)=\frac{\mathrm{\Gamma }(r+k)}{k!\mathrm{\Gamma }(r)}{\left(\frac{r}{r+m}\right)}^r{\left(\frac{m}{r+m}\right)}^k\text{for }k=0,1,2,\dots }$

The variance can then be written as $m+\frac{m^2}{r}$. Some authors prefer to set $\alpha =\frac{1}{r}$, and express the variance as $m+\alpha m^2$. In this context, and depending on the author, either the parameter r or its reciprocal α is referred to as the "dispersion parameter", "shape parameter" or "clustering coefficient", or the "heterogeneity" or "aggregation" parameter. The term "aggregation" is particularly used in ecology when describing counts of individual organisms. Decrease of the aggregation parameter r towards zero corresponds to increasing aggregation of the organisms; increase of r towards infinity corresponds to absence of aggregation, as can be described by Poisson regression.

Alternative parameterizations

Sometimes the distribution is parameterized in terms of its mean μ and variance σ²:

${\displaystyle \begin{array}{rl}& p=\frac{\mu }{{\sigma }^2},\\ & r=\frac{{\mu }^2}{{\sigma }^2-\mu },\\ & Pr(X=k)=(\genfrac{}{}{0ex}{}{k+\frac{{\mu }^2}{{\sigma }^2-\mu }-1}{r-1}){\left(\frac{{\sigma }^2-\mu }{{\sigma }^2}\right)}^k{\left(\frac{\mu }{{\sigma }^2}\right)}^{{\mu }^2/({\sigma }^2-\mu )}.\end{array}}$

Examples

Length of hospital stay

Hospital length of stay is an example of real-world data that can be modelled well with a negative binomial distribution via negative binomial regression.

Selling candy

Pat Collis is required to sell candy bars to raise money for the 6th grade field trip. Pat is (somewhat harshly) not supposed to return home until five candy bars have been sold. So the child goes door to door, selling candy bars. At each house, there is a 0.6 probability of selling one candy bar and a 0.4 probability of selling nothing.

What's the probability of selling the last candy bar at the nth house?

Successfully selling candy enough times is what defines our stopping criterion (as opposed to failing to sell it), so k in this case represents the number of failures and r represents the number of successes. Recall that the NegBin(r, p) distribution describes the probability of k failures and r successes in k + r Bernoulli(p) trials with success on the last trial. Selling five candy bars means getting five successes. The number of trials (i.e. houses) this takes is therefore k + 5 = n. The random variable we are interested in is the number of houses, so we substitute k = n − 5 into a NegBin(5, 0.4) mass function and obtain the following mass function of the distribution of houses (for n ≥ 5):

${\displaystyle f(n)=(\genfrac{}{}{0ex}{}{(n-5)+5-1}{n-5})(1-0.4{)}^5{0.4}^{n-5}=(\genfrac{}{}{0ex}{}{n-1}{n-5})3^5\frac{2^{n-5}}{5^n}.}$

What's the probability that Pat finishes on the tenth house?

${\displaystyle f(10)=\mathrm{0.1003290624.}}$

What's the probability that Pat finishes on or before reaching the eighth house?

To finish on or before the eighth house, Pat must finish at the fifth, sixth, seventh, or eighth house. Sum those probabilities:

${\displaystyle f(5)=0.07776}$

${\displaystyle f(6)=0.15552}$

${\displaystyle f(7)=0.18662}$

${\displaystyle f(8)=0.17418}$

${\displaystyle \sum _{j=5}^{8}f(j)=\mathrm{0.59408.}}$

What's the probability that Pat exhausts all 30 houses that happen to stand in the neighborhood?

This can be expressed as the probability that Pat does not finish on the fifth through the thirtieth house:

${\displaystyle 1-\sum _{j=5}^{30}f(j)=1-I_{0.4}(5,30-5+1)\approx 1-0.99999342=\mathrm{0.00000658.}}$

Because of the rather high probability that Pat will sell to each house (60 percent), the probability of her NOT fulfilling her quest is vanishingly slim.

Properties

Expectation

The expected total number of trials needed to see r successes is ${\displaystyle \frac{r}{p}}$. Thus, the expected number of failures would be this value, minus the successes:

${\displaystyle E[\mathrm{NB}(r,p)]=\frac{r}{p}-r=\frac{r(1-p)}{p}}$

Expectation of successes

The expected total number of failures in a negative binomial distribution with parameters (r, p) is r(1 − p)/p. To see this, imagine an experiment simulating the negative binomial is performed many times. That is, a set of trials is performed until r successes are obtained, then another set of trials, and then another etc. Write down the number of trials performed in each experiment: a, b, c, ... and set a + b + c + ... = N. Now we would expect about Np successes in total. Say the experiment was performed n times. Then there are nr successes in total. So we would expect nr = Np, so N/n = r/p. See that N/n is just the average number of trials per experiment. That is what we mean by "expectation". The average number of failures per experiment is N/n − r = r/p − r = r(1 − p)/p. This agrees with the mean given in the box on the right-hand side of this page.

A rigorous derivation can be done by representing the negative binomial distribution as the sum of waiting times. Let ${\displaystyle X_r\sim \mathrm{NB}(r,p)}$ with the convention ${\displaystyle X}$ represents the number of failures observed before ${\displaystyle r}$ successes with the probability of success being ${\displaystyle p}$. And let ${\displaystyle Y_i\sim Geom(p)}$ where ${\displaystyle Y_i}$ represents the number of failures before seeing a success. We can think of ${\displaystyle Y_i}$ as the waiting time (number of failures) between the ${\displaystyle i}$th and ${\displaystyle (i-1)}$th success. Thus

${\displaystyle X_r=Y_1+Y_2+\cdots +Y_r.}$

The mean is

${\displaystyle E[X_r]=E[Y_1]+E[Y_2]+\cdots +E[Y_r]=\frac{r(1-p)}{p},}$

which follows from the fact ${\displaystyle E[Y_i]=(1-p)/p}$.

Variance

When counting the number of failures before the r-th success, the variance is r(1 − p)/p². When counting the number of successes before the r-th failure, as in alternative formulation (3) above, the variance is rp/(1 − p)².

Relation to the binomial theorem

Suppose Y is a random variable with a binomial distribution with parameters n and p. Assume p + q = 1, with p, q ≥ 0, then

${\displaystyle 1=1^n=(p+q{)}^n.}$

Using Newton's binomial theorem, this can equally be written as:

${\displaystyle (p+q{)}^n=\sum _{k=0}^{\mathrm{\infty }}(\genfrac{}{}{0ex}{}{n}{k})p^k{q}^{n-k},}$

in which the upper bound of summation is infinite. In this case, the binomial coefficient

${\displaystyle (\genfrac{}{}{0ex}{}{n}{k})=\frac{n(n-1)(n-2)\cdots (n-k+1)}{k!}.}$

is defined when n is a real number, instead of just a positive integer. But in our case of the binomial distribution it is zero when k > n. We can then say, for example

${\displaystyle (p+q{)}^{8.3}=\sum _{k=0}^{\mathrm{\infty }}(\genfrac{}{}{0ex}{}{8.3}{k})p^k{q}^{8.3-k}.}$

Now suppose r > 0 and we use a negative exponent:

${\displaystyle 1=p^r\cdot p^{-r}=p^r(1-q{)}^{-r}=p^r\sum _{k=0}^{\mathrm{\infty }}(\genfrac{}{}{0ex}{}{-r}{k})(-q{)}^k.}$

Then all of the terms are positive, and the term

${\displaystyle p^r(\genfrac{}{}{0ex}{}{-r}{k})(-q{)}^k}$

is just the probability that the number of failures before the rth success is equal to k, provided r is an integer. (If r is a negative non-integer, so that the exponent is a positive non-integer, then some of the terms in the sum above are negative, so we do not have a probability distribution on the set of all nonnegative integers.)

Now we also allow non-integer values of r. Then we have a proper negative binomial distribution, which is a generalization of the Pascal distribution, which coincides with the Pascal distribution when r happens to be a positive integer.

Recall from above that

The sum of independent negative-binomially distributed random variables r₁ and r₂ with the same value for parameter p is negative-binomially distributed with the same p but with r-value r₁ + r₂.

This property persists when the definition is thus generalized, and affords a quick way to see that the negative binomial distribution is infinitely divisible.

Recurrence relations

The following recurrence relations hold:

For the probability mass function

${\displaystyle \{\begin{array}{l}(k+1)Pr(X=k+1)-pPr(X=k)(k+r)=0,\\ Pr(X=0)=(1-p{)}^r.\end{array}}$

For the moments ${\displaystyle m_k=\mathbb{E}(X^k),}$

${\displaystyle m_{k+1}=rP{m}_k+(P^2+P)\frac{d{m}_k}{dP},P:=(1-p)/p,m_0=1.}$

For the cumulants

${\displaystyle {\kappa }_{k+1}=(Q-1)Q\frac{d{\kappa }_k}{dQ},Q:=1/p,{\kappa }_1=r(Q-1).}$

Related distributions

• The geometric distribution (on { 0, 1, 2, 3, ... }) is a special case of the negative binomial distribution, with

${\displaystyle \mathrm{Geom}(p)=\mathrm{NB}(1,p).}$

• The negative binomial distribution is a special case of the discrete phase-type distribution.
• The negative binomial distribution is a special case of discrete compound Poisson distribution.

Poisson distribution

Consider a sequence of negative binomial random variables where the stopping parameter r goes to infinity, while the probability p of success in each trial goes to one, in such a way as to keep the mean of the distribution (i.e. the expected number of failures) constant. Denoting this mean as λ, the parameter p will be p = r/(r + λ)

${\displaystyle \begin{array}{rl}\text{Mean:}& \lambda =\frac{(1-p)r}{p}\Rightarrow p=\frac{r}{r+\lambda },\\ \text{Variance:}& \lambda \left(1+\frac{\lambda }{r}\right)>\lambda ,\text{thus always overdispersed}.\end{array}}$

Under this parametrization the probability mass function will be

${\displaystyle f(k;r,p)=\frac{\mathrm{\Gamma }(k+r)}{k!\cdot \mathrm{\Gamma }(r)}(1-p{)}^k{p}^r=\frac{{\lambda }^k}{k!}\cdot \frac{\mathrm{\Gamma }(r+k)}{\mathrm{\Gamma }(r)(r+\lambda {)}^k}\cdot \frac{1}{{\left(1+\frac{\lambda }{r}\right)}^r}}$

Now if we consider the limit as r → ∞, the second factor will converge to one, and the third to the exponent function:

${\displaystyle \underset{r\to \mathrm{\infty }}{lim}f(k;r,p)=\frac{{\lambda }^k}{k!}\cdot 1\cdot \frac{1}{e^{\lambda }},}$

which is the mass function of a Poisson-distributed random variable with expected value λ.

In other words, the alternatively parameterized negative binomial distribution converges to the Poisson distribution and r controls the deviation from the Poisson. This makes the negative binomial distribution suitable as a robust alternative to the Poisson, which approaches the Poisson for large r, but which has larger variance than the Poisson for small r.

${\displaystyle \mathrm{Poisson}(\lambda )=\underset{r\to \mathrm{\infty }}{lim}\mathrm{NB}\left(r,\frac{r}{r+\lambda }\right).}$

Gamma–Poisson mixture

The negative binomial distribution also arises as a continuous mixture of Poisson distributions (i.e. a compound probability distribution) where the mixing distribution of the Poisson rate is a gamma distribution. That is, we can view the negative binomial as a Poisson(λ) distribution, where λ is itself a random variable, distributed as a gamma distribution with shape r and scale θ = (1 − p)/p or correspondingly rate β = p/(1 − p).

To display the intuition behind this statement, consider two independent Poisson processes, "Success" and "Failure", with intensities p and 1 − p. Together, the Success and Failure processes are equivalent to a single Poisson process of intensity 1, where an occurrence of the process is a success if a corresponding independent coin toss comes up heads with probability p; otherwise, it is a failure. If r is a counting number, the coin tosses show that the count of successes before the rth failure follows a negative binomial distribution with parameters r and p. The count is also, however, the count of the Success Poisson process at the random time T of the rth occurrence in the Failure Poisson process. The Success count follows a Poisson distribution with mean pT, where T is the waiting time for r occurrences in a Poisson process of intensity 1 − p, i.e., T is gamma-distributed with shape parameter r and intensity 1 − p. Thus, the negative binomial distribution is equivalent to a Poisson distribution with mean pT, where the random variate T is gamma-distributed with shape parameter r and intensity (1 − p). The preceding paragraph follows, because λ = pT is gamma-distributed with shape parameter r and intensity (1 − p)/p.

The following formal derivation (which does not depend on r being a counting number) confirms the intuition.

${\displaystyle \begin{array}{rl}& {\int }_0^{\mathrm{\infty }}{f}_{\mathrm{Poisson}(\lambda )}(k)\times f_{\mathrm{Gamma}\left(r,\frac{p}{1-p}\right)}(\lambda )\mathrm{d}\lambda \\ =& {\int }_0^{\mathrm{\infty }}\frac{{\lambda }^k}{k!}{e}^{-\lambda }\times \frac{1}{\mathrm{\Gamma }(r)}{\left(\frac{p}{1-p}\lambda \right)}^{r-1}{e}^{-\frac{p}{1-p}\lambda }\left(\frac{p}{1-p}\mathrm{d}\lambda \right)\\ =& {\left(\frac{p}{1-p}\right)}^r\frac{1}{k!\mathrm{\Gamma }(r)}{\int }_0^{\mathrm{\infty }}{\lambda }^{r+k-1}{e}^{-\lambda \frac{p+1-p}{1-p}}\mathrm{d}\lambda \\ =& {\left(\frac{p}{1-p}\right)}^r\frac{1}{k!\mathrm{\Gamma }(r)}\mathrm{\Gamma }(r+k)(1-p{)}^{k+r}{\int }_0^{\mathrm{\infty }}{f}_{\mathrm{Gamma}\left(k+r,\frac{1}{1-p}\right)}(\lambda )\mathrm{d}\lambda \\ =& \frac{\mathrm{\Gamma }(r+k)}{k!\mathrm{\Gamma }(r)}(1-p{)}^k{p}^r\\ =& f(k;r,p).\end{array}}$

Because of this, the negative binomial distribution is also known as the gamma–Poisson (mixture) distribution. The negative binomial distribution was originally derived as a limiting case of the gamma-Poisson distribution.

Distribution of a sum of geometrically distributed random variables

If Y_r is a random variable following the negative binomial distribution with parameters r and p, and support {0, 1, 2, ...}, then Y_r is a sum of r independent variables following the geometric distribution (on {0, 1, 2, ...}) with parameter p. As a result of the central limit theorem, Y_r (properly scaled and shifted) is therefore approximately normal for sufficiently large r.

Furthermore, if B_s+r is a random variable following the binomial distribution with parameters s + r and p, then

${\displaystyle \begin{array}{rl}Pr(Y_r\le s)& =1-I_p(s+1,r)\\ & =1-I_p((s+r)-(r-1),(r-1)+1)\\ & =1-Pr(B_{s+r}\le r-1)\\ & =Pr(B_{s+r}\ge r)\\ & =Pr(\text{after }s+r\text{ trials, there are at least }r\text{ successes}).\end{array}}$

In this sense, the negative binomial distribution is the "inverse" of the binomial distribution.

The sum of independent negative-binomially distributed random variables r₁ and r₂ with the same value for parameter p is negative-binomially distributed with the same p but with r-value r₁ + r₂.

The negative binomial distribution is infinitely divisible, i.e., if Y has a negative binomial distribution, then for any positive integer n, there exist independent identically distributed random variables Y₁, ..., Y_n whose sum has the same distribution that Y has.

Representation as compound Poisson distribution

The negative binomial distribution NB(r,p) can be represented as a compound Poisson distribution: Let {Y_n, n ∈ ${\displaystyle \mathbb{N}}$₀ denote a sequence of independent and identically distributed random variables, each one having the logarithmic series distribution Log(p), with probability mass function

${\displaystyle f(k;r,p)=\frac{-p^k}{k\ln (1-p)},k\in \mathbb{N}.}$

Let N be a random variable, independent of the sequence, and suppose that N has a Poisson distribution with mean λ = −r ln(1 − p). Then the random sum

${\displaystyle X=\sum _{n=1}^N{Y}_n}$

is NB(r,p)-distributed. To prove this, we calculate the probability generating function G_X of X, which is the composition of the probability generating functions G_N and G_Y1. Using

${\displaystyle G_N(z)=\exp (\lambda (z-1)),z\in \mathbb{R},}$

and

${\displaystyle G_{Y_1}(z)=\frac{\ln (1-pz)}{\ln (1-p)},|z|<\frac{1}{p},}$

we obtain

${\displaystyle \begin{array}{rl}{G}_X(z)& =G_N(G_{Y_1}(z))\\ & =\exp (\lambda (\frac{\ln (1-pz)}{\ln (1-p)}-1))\\ & =\exp (-r(\ln (1-pz)-\ln (1-p)))\\ & =(\frac{1-p}{1-pz}{)}^r,|z|<\frac{1}{p},\end{array}}$

which is the probability generating function of the NB(r,p) distribution.

The following table describes four distributions related to the number of successes in a sequence of draws:

	With replacements	No replacements
Given number of draws	binomial distribution	hypergeometric distribution
Given number of failures	negative binomial distribution	negative hypergeometric distribution

(a,b,0) class of distributions

The negative binomial, along with the Poisson and binomial distributions, is a member of the (a,b,0) class of distributions. All three of these distributions are special cases of the Panjer distribution. They are also members of a natural exponential family.

Statistical inference

Parameter estimation

MVUE for p

Suppose p is unknown and an experiment is conducted where it is decided ahead of time that sampling will continue until r successes are found. A sufficient statistic for the experiment is k, the number of failures.

In estimating p, the minimum variance unbiased estimator is

${\displaystyle \hat{p}=\frac{r-1}{r+k-1}.}$

Maximum likelihood estimation

When r is known, the maximum likelihood estimate of p is

${\displaystyle \tilde{p}=\frac{r}{r+k},}$

but this is a biased estimate. Its inverse (r + k)/r, is an unbiased estimate of 1/p, however.

When r is unknown, the maximum likelihood estimator for p and r together only exists for samples for which the sample variance is larger than the sample mean. The likelihood function for N iid observations (k₁, ..., k_N) is

${\displaystyle L(r,p)=\prod _{i=1}^{N}f(k_i;r,p)}$

from which we calculate the log-likelihood function

${\displaystyle \ell (r,p)=\sum _{i=1}^N\ln (\mathrm{\Gamma }(k_i+r))-\sum _{i=1}^N\ln (k_i!)-N\ln (\mathrm{\Gamma }(r))+\sum _{i=1}^N{k}_i\ln (1-p)+Nr\ln (p).}$

To find the maximum we take the partial derivatives with respect to r and p and set them equal to zero:

${\displaystyle \frac{\mathrm{\partial }\ell (r,p)}{\mathrm{\partial }p}=-\left[\sum _{i=1}^N{k}_i\frac{1}{1-p}\right]+Nr\frac{1}{p}=0}$ and

${\displaystyle \frac{\mathrm{\partial }\ell (r,p)}{\mathrm{\partial }r}=\left[\sum _{i=1}^N\psi (k_i+r)\right]-N\psi (r)+N\ln (p)=0}$

where

${\displaystyle \psi (k)=\frac{{\mathrm{\Gamma }}^{\prime }(k)}{\mathrm{\Gamma }(k)}}$ is the digamma function.

Solving the first equation for p gives:

${\displaystyle p=\frac{Nr}{Nr+\sum _{i=1}^N{k}_i}}$

Substituting this in the second equation gives:

${\displaystyle \frac{\mathrm{\partial }\ell (r,p)}{\mathrm{\partial }r}=\left[\sum _{i=1}^N\psi (k_i+r)\right]-N\psi (r)+N\ln \left(\frac{r}{r+\sum _{i=1}^N{k}_i/N}\right)=0}$

This equation cannot be solved for r in closed form. If a numerical solution is desired, an iterative technique such as Newton's method can be used. Alternatively, the expectation–maximization algorithm can be used.

Occurrence and applications

Waiting time in a Bernoulli process

For the special case where r is an integer, the negative binomial distribution is known as the Pascal distribution. It is the probability distribution of a certain number of failures and successes in a series of independent and identically distributed Bernoulli trials. For k + r Bernoulli trials with success probability p, the negative binomial gives the probability of k successes and r failures, with a failure on the last trial. In other words, the negative binomial distribution is the probability distribution of the number of successes before the rth failure in a Bernoulli process, with probability p of successes on each trial. A Bernoulli process is a discrete time process, and so the number of trials, failures, and successes are integers.

Consider the following example. Suppose we repeatedly throw a die, and consider a 1 to be a failure. The probability of success on each trial is 5/6. The number of successes before the third failure belongs to the infinite set { 0, 1, 2, 3, ... }. That number of successes is a negative-binomially distributed random variable.

When r = 1 we get the probability distribution of number of successes before the first failure (i.e. the probability of the first failure occurring on the (k + 1)st trial), which is a geometric distribution:

${\displaystyle f(k;r,p)=(1-p)\cdot p^k}$

Overdispersed Poisson

The negative binomial distribution, especially in its alternative parameterization described above, can be used as an alternative to the Poisson distribution. It is especially useful for discrete data over an unbounded positive range whose sample variance exceeds the sample mean. In such cases, the observations are overdispersed with respect to a Poisson distribution, for which the mean is equal to the variance. Hence a Poisson distribution is not an appropriate model. Since the negative binomial distribution has one more parameter than the Poisson, the second parameter can be used to adjust the variance independently of the mean. See Cumulants of some discrete probability distributions.

An application of this is to annual counts of tropical cyclones in the North Atlantic or to monthly to 6-monthly counts of wintertime extratropical cyclones over Europe, for which the variance is greater than the mean. In the case of modest overdispersion, this may produce substantially similar results to an overdispersed Poisson distribution.

The negative binomial distribution is also commonly used to model data in the form of discrete sequence read counts from high-throughput RNA and DNA sequencing experiments.

Multiplicity Observations (Physics)

The negative binomial distribution has been the most effective statistical model for a broad range of multiplicity observations in particle collision experiments, e.g., ${\displaystyle p\overline{p},hh,hA,AA,e^{+}{e}^{-}}$ and is argued to be a scale-invariant property of matter, providing the best fit for astronomical observations, where it predicts the number of galaxies in a region of space. The phenomenological justification for the effectiveness of the negative binomial distribution in these contexts remained unknown for fifty years, since their first observation in 1973. In 2023, a proof from first principles was eventually demonstrated by Scott V. Tezlaf, where it was shown that the negative binomial distribution emerges from symmetries in the dynamical equations of a canonical ensemble of particles in Minkowski space. Roughly, given an expected number of trials ${\displaystyle ⟨n⟩}$ and expected number of successes ${\displaystyle ⟨r⟩}$, where

${\displaystyle ⟨\mathcal{n}⟩-⟨r⟩=k,⟨p⟩=\frac{⟨r⟩}{⟨\mathcal{n}⟩}⟹⟨\mathcal{n}⟩=\frac{k}{1-⟨p⟩},⟨r⟩=\frac{k⟨p⟩}{1-⟨p⟩},}$

an isomorphic set of equations can be identified with the parameters of a relativistic current density of a canonical ensemble of massive particles, via

${\displaystyle c^2⟨{\rho }^2⟩-⟨j^2⟩=c^2{\rho }_0^2,⟨{\beta }_v^2⟩=\frac{⟨j^2⟩}{c^2⟨{\rho }^2⟩}⟹c^2⟨{\rho }^2⟩=\frac{c^2{\rho }_0^2}{1-⟨{\beta }_v^2⟩},⟨j^2⟩=\frac{c^2{\rho }_0^2⟨{\beta }_v^2⟩}{1-⟨{\beta }_v^2⟩},}$

where ${\displaystyle {\rho }_0}$ is the rest density, ${\displaystyle ⟨{\rho }^2⟩}$ is the relativistic mean square density, ${\displaystyle ⟨j^2⟩}$ is the relativistic mean square current density, and ${\displaystyle ⟨{\beta }_v^2⟩=⟨v^2⟩/c^2}$, where ${\displaystyle ⟨v^2⟩}$ is the mean square speed of the particle ensemble and ${\displaystyle c}$ is the speed of light—such that one can establish the following bijective map:

${\displaystyle c^2{\rho }_0^2\mapsto k,⟨{\beta }_v^2⟩\mapsto ⟨p⟩,c^2⟨{\rho }^2⟩\mapsto ⟨\mathcal{n}⟩,⟨j^2⟩\mapsto ⟨r⟩.}$

A rigorous alternative proof of the above correspondence has also been demonstrated through quantum mechanics via the Feynman path integral.

History

This distribution was first studied in 1713 by Pierre Remond de Montmort in his Essay d'analyse sur les jeux de hazard, as the distribution of the number of trials required in an experiment to obtain a given number of successes. It had previously been mentioned by Pascal.

Excimer laser

From Wikipedia, the free encyclopedia

https://en.wikipedia.org/wiki/Excimer_laser

 An excimer laser

An excimer laser, sometimes more correctly called an exciplex laser, is a form of ultraviolet laser which is commonly used in the production of microelectronic devices, semiconductor based integrated circuits or "chips", eye surgery, and micromachining. Since 1960s excimer lasers are widely used in high-resolution photolithography machines, one of the critical technologies required for microelectronic chip manufacturing.

Terminology and history

The term excimer is short for 'excited dimer', while exciplex is short for 'excited complex'. Most excimer lasers are of the noble gas halide type, for which the term excimer is, strictly speaking, a misnomer. (Although less commonly used, the proper term for such is an exciplex laser.)

Excimer laser was proposed in 1960 by Fritz Houtermans. The excimer laser development started with the observation of a nascent spectral line narrowing at 176 nm  reported in 1971 by Nikolai Basov, V. A. Danilychev and Yu. M. Popov, at the Lebedev Physical Institute in Moscow, using liquid xenon dimer (Xe₂) excited by an electron beam. Spurred by this report, H.A. Koehler et al. presented a better substantiation of stimulated emission in 1972, using high pressure xenon gas. Definitive evidence of a xenon excimer laser action at 173 nm using a high pressure gas at 12 atmospheres, also pumped by an electron beam, was first presented in March 1973, by Mani Lal Bhaumik of Northrop Corporation, Los Angeles. Strong stimulated emission was observed as the laser's spectral line narrowed from a continuum of 15 nm to just 0.25 nm, and the intensity increased a thousand-fold. The laser's estimated output of 1 joule was high enough to evaporate part of the mirror coatings, which imprinted its mode pattern. This presentation established the credible potential of developing high power lasers at short wavelengths.

A later improvement was the use of noble gas halides (originally Xe Br) developed by many groups in 1975. These groups include the Avco Everett Research Laboratory, Sandia Laboratories, the Northrop Research and Technology Center, the United States Government's Naval Research Laboratory, which also developed a XeCl Laser that was excited using a microwave discharge, and Los Alamos National Laboratory.

Construction and operation

Final amplifier of the Nike laser where laser beam energy is increased from 150 J to ~5 kJ by passing through a krypton/fluorine/argon gas mixture excited by irradiation with two opposing 670,000 volt electron beams.

An excimer laser typically uses a combination of a noble gas (argon, krypton, or xenon) and a reactive gas (fluorine or chlorine). Under the appropriate conditions of electrical stimulation and high pressure, a pseudo-molecule called an excimer (or in the case of noble gas halides, exciplex) is created, which can only exist in an energized state and can give rise to laser light in the ultraviolet range.

Laser action in an excimer molecule occurs because it has a bound (associative) excited state, but a repulsive (dissociative) ground state. Noble gases such as xenon and krypton are highly inert and do not usually form chemical compounds. However, when in an excited state (induced by electrical discharge or high-energy electron beams), they can form temporarily bound molecules with themselves (excimer) or with halogens (exciplex) such as fluorine and chlorine. The excited compound can release its excess energy by undergoing spontaneous or stimulated emission, resulting in a strongly repulsive ground state molecule which very quickly (on the order of a picosecond) dissociates back into two unbound atoms. This forms a population inversion.

Wavelength determination

The wavelength of an excimer laser depends on the molecules used, and is usually in the ultraviolet range of electromagnetic radiation:

Excimer	Wavelength	Relative power
Ar2*	126 nm
Kr2*	146 nm
F2*	157 nm
Xe2*	172 & 175 nm
ArF	193 nm	60
KrCl	222 nm	25
KrF	248 nm	100
XeBr	282 nm
XeCl	308 nm	50
XeF	351 nm	45

Excimer lasers, such as XeF and KrF, can also be made slightly tunable using a variety of prism and grating intracavity arrangements.

Pulse repetition rate

The electra laser at NRL is a KrF laser that demonstrated over 90,000 shots in 10 hours.

While electron-beam pumped excimer lasers can produce high single energy pulses, they are generally separated by long time periods (many minutes).  An exception was the Electra system, designed for inertial fusion studies, which could produce a burst of 10 pulses each measuring 500 J over a span of 10 s. In contrast, discharge-pumped excimer lasers, also first demonstrated at the Naval Research Laboratory, are able to output a steady stream of pulses. Their significantly higher pulse repetition rates (of order 100 Hz) and smaller footprint made possible the bulk of the applications listed in the following section. A series of industrial lasers were developed at XMR, Inc in Santa Clara, California between 1980 and 1988. Most of the lasers produced were XeCl, and a sustained energy of 1 J per pulse at repetition rates of 300 pulses per second was the standard rating. This laser used a high power thyratron and magnetic switching with corona pre-ionization and was rated for 100 million pulses without major maintenance. The operating gas was a mixture of xenon, HCl, and Neon at approximately 5 atmospheres. Extensive use of stainless steel, nickel plating and solid nickel electrodes was incorporated to reduce corrosion due to the HCl gas. One major problem encountered was degradation of the optical windows due to carbon build-up on the surface of the CaF window. This was due to hydro-chloro-carbons formed from small amounts of carbon in O-rings reacting with the HCl gas. The hydro-chloro-carbons would slowly increase over time and absorbed the laser light, causing a slow reduction in laser energy. In addition these compounds would decompose in the intense laser beam and collect on the window, causing a further reduction in energy. Periodic replacement of laser gas and windows was required at considerable expense. This was significantly improved by use of a gas purification system consisting of a cold trap operating slightly above liquid nitrogen temperature and a metal bellows pump to recirculate the laser gas through the cold trap. The cold trap consisted of a liquid nitrogen reservoir and a heater to raise the temperature slightly, since at 77 K (liquid nitrogen boiling point) the xenon vapor pressure was lower than the required operating pressure in the laser gas mixture. HCl was frozen out in the cold trap, and additional HCl was added to maintain the proper gas ratio. An interesting side effect of this was a slow increase in laser energy over time, attributed to increase in hydrogen partial pressure in the gas mixture caused by slow reaction of chlorine with various metals. As the chlorine reacted, hydrogen was released, increasing the partial pressure. The net result was the same as adding hydrogen to the mixture to increase laser efficiency as reported by T.J. McKee et al.

Major applications

Photolithography

Main article: Semiconductor device fabrication

Since the 1960s the most widespread industrial application of excimer lasers has been in deep-ultraviolet photolithography, a critical technology used in the manufacturing of microelectronic devices. Historically, from the early 1960s through the mid-1980s, mercury-xenon lamps were used in lithography for their spectral lines at 436, 405 and 365 nm wavelengths. However, with the semiconductor industry's need for both higher resolution (to produce denser and faster chips) and higher throughput (for lower costs), the lamp-based lithography tools were no longer able to meet the industry's requirements. This challenge was overcome when in a pioneering development in 1982, deep-UV excimer laser lithography was proposed and demonstrated at IBM by Kanti Jain. From an even broader scientific and technological perspective, since the invention of the laser in 1960, the development of excimer laser lithography has been highlighted as one of the major milestones in the history of the laser.

Current lithography tools (as of 2021) mostly use deep ultraviolet (DUV) light from the KrF and ArF excimer lasers with wavelengths of 248 and 193 nanometers (called "excimer laser lithography"), which has enabled transistor feature sizes to shrink to 7 nanometers (see below). Excimer laser lithography has thus played a critical role in the continued advance of the so-called Moore's law for the last 25 years. By around 2020, extreme ultraviolet lithography (EUV) has started to replace excimer laser lithography to further improve the resolution of the semiconductor circuits lithography process.

Fusion

The Naval Research Laboratory built two systems, the Krypton fluoride laser (248 nm) and the Argon fluoride laser (193 nm) to test approaches to prove out Inertial Confinement Fusion approaches. These were the electra and Nike laser systems. Because the excimer laser is a gas-based system, the laser does not heat up like solid-state systems such as National Ignition Facility and the Omega Laser. Electra demonstrated 90,000 shots in 10 hours; ideal for a Inertial fusion power plant.

Medical uses

The ultraviolet light from an excimer laser is well absorbed by biological matter and organic compounds. Rather than burning or cutting material, the excimer laser adds enough energy to disrupt the molecular bonds of the surface tissue, which effectively disintegrates into the air in a tightly controlled manner through ablation rather than burning. Thus excimer lasers have the useful property that they can remove exceptionally fine layers of surface material with almost no heating or change to the remainder of the material which is left intact. These properties make excimer lasers well suited to precision micromachining organic material (including certain polymers and plastics), or delicate surgeries such as eye surgery LASIK. In 1980–1983, Rangaswamy Srinivasan, Samuel Blum and James J. Wynne at IBM's T. J. Watson Research Center observed the effect of the ultraviolet excimer laser on biological materials. Intrigued, they investigated further, finding that the laser made clean, precise cuts that would be ideal for delicate surgeries. This resulted in a fundamental patent and Srinivasan, Blum and Wynne were elected to the National Inventors Hall of Fame in 2002. In 2012, the team members were honored with National Medal of Technology and Innovation by the President Barack Obama for their work related to the excimer laser. Subsequent work introduced the excimer laser for use in angioplasty. Xenon chloride (308 nm) excimer lasers can also treat a variety of dermatological conditions including psoriasis, vitiligo, atopic dermatitis, alopecia areata and leukoderma.

As light sources, excimer lasers are generally large in size, which is a disadvantage in their medical applications, although their sizes are rapidly decreasing with ongoing development.

Research is being conducted to compare differences in safety and effectiveness outcomes between conventional excimer laser refractive surgery and wavefront-guided or wavefront-optimized refractive surgery, as wavefront methods may better correct for higher-order aberrations.

Scientific research

Excimer lasers are also widely used in numerous fields of scientific research, both as primary sources and, particularly the XeCl laser, as pump sources for tunable dye lasers, mainly to excite laser dyes emitting in the blue-green region of the spectrum. These lasers are also commonly used in pulsed laser deposition systems, where their large fluence, short wavelength and non-continuous beam properties make them ideal for the ablation of a wide range of materials.