6 Formal Foundations: The Gamma and Beta Functions
6.1 Overview
As we mentioned in the Negative Binomial lesson, we could spend literal weeks discussing and proving the many properties of the Gamma Function (and its relative the Beta Function). Instead, we will try to distill the most important properties of these functions that are necessary for us to derive and prove properties of the Gamma, Beta, \(\chi^2\), Student’s \(t\), and Central \(F\) distributions. As we saw previously, the Gamma Function is one (of potentially many) smooth interpolations between the values of \(x!\) evaluated at each non-negative integer.
6.2 Formal Foundations (of Foundations)
Yes, I know this entire chapter is already dedicated to “math foundations”, but we have to go deeper (insert Inception joke here). We need some foundations to our foundations.
6.2.1 L’Hospital’s Rule
If we have the limit of a ratio of functions, where the ratio of the numerator and denominator yield an indeterminant form1, then we can apply L’Hospital’s Rule to simplify the limit. Consider \[ \lim_{x \to c} \frac{f(x)}{g(x)}. \]
We need four conditions to hold before we can apply L’Hospital’s Rule:
- The ratio must have an indeterminant form of the same value in the numerator and denominator (such as \(0/0\) or \(\infty/\infty\)).
- The derivatives of \(f\) and \(g\) must exist in a neighbourhood of the point \(c\), but not necessarily at the point \(c\) itself.
- The derivative of \(g\) must not equal 0 in this neighbourhood of the point \(c\), except potentially at the point \(c\) itself.
- The ratio of the derivatives of \(f\) and \(g\) (not the derivative of the ratio) must exist at the point \(c\) (even if we can’t find it immediately).
IF all four conditions hold, then \[ \lim_{x \to c} \frac{f(x)}{g(x)} = \lim_{x \to c} \frac{f^{\prime}(x)}{g^{\prime}(x)}. \]
6.2.2 The Jacobian Determinant
In univariate calculus, we used \(u\)-substitution2 as the “inverse” of the Chain Rule for derivatives. As you should recall from multivariable calculus, the multi-dimensional version of \(u\)-substitution involves a matrix of derivatives in all possible directions. If you need a reminder on what matrices3 are or what a determinant4 is, then I apologize that such material is beyond the scope of this text.
Consider a function \(f(x,y)\) that we are integrating with respect to both \(x\) and \(y\). When we change variables from \(x,y\) to some other variables, say \(u,v\), then we use the Jacobian Determinant as the value to substitute in the place of \(dydx\) or \(dxdy\) (we assume that they can be reversed). This determinant for two dimensions (including the absolute value to ensure that the area is always positive) is then: \[ |\textbf{J}| \equiv \det\! \begin{bmatrix} \frac{\partial x}{\partial u} & \frac{\partial x}{\partial v} \\ \frac{\partial y}{\partial u} & \frac{\partial y}{\partial v} \end{bmatrix} = \left| \frac{\partial x}{\partial u} \frac{\partial y}{\partial v} - \frac{\partial x}{\partial v} \frac{\partial y}{\partial u} \right|. \]
6.3 The Gamma Function
For a complex number \(z = a + bi\) (except for the negative integers), the Weierstrass’s Definition of the Gamma Function is \[ \Gamma(z) \equiv \frac{e^{-\gamma z}}{z} \prod_{n = 1}^{\infty} \left( 1 + \frac{z}{n} \right)^{-1} e^{\frac{z}{n}}, \] where \(\gamma \approx 0.577\) is the Euler-Mascheroni constant. This is (in my experience) the most complex form of the Gamma function. In contrast, if \(z\) can be written as a non-negative integer \(n\), then we get the simplest form of all: \[ \Gamma(n) \equiv (n - 1)!. \]
However, we don’t need the “most” or the “least” complex versions of this function. Instead, we need a version with the flexibility to handle any \(z \in \mathbb{R}^+\), so we will dive deep into the “common form” of the Gamma Function: \[ \Gamma(z) = \int_0^{\infty} t^{z - 1}e^{-t}dt,\ z \in \mathbb{R}^+. \]
6.4 Integer Form of the Lower Incomplete Gamma Function
Let \(\alpha, x \in \mathbb{R}^+\). Then we define the lower incomplete Gamma Function as \[ \gamma(\alpha,x) \equiv \int_0^x t^{\alpha - 1} e^{-t} dt. \]
Now, restrict \(\alpha \ni \alpha \in 0 \cup \mathbb{N}\), then we can “simplify” the above integral as \[ \gamma(\alpha,x) = (\alpha - 1)! \left[ 1 - e^{-x} \sum_{k = 0}^{\alpha - 1} \frac{x^k}{k!} \right]. \]
The first result has stumped me. I have not been able to figure out how to prove it. So, even though I usually tell my students to “never give up”, I’m giving up (for now) on trying to figure out how to prove this. Therefore, I am quoting the US National Institute of Standards and Technology’s Digital Library of Mathematical Functions, specifically chapter 8, section 4, equation 7. This book says that equation is true, and I’m just going to trust it.
6.5 Product of Gamma Functions
Let \(a,b \in \mathbb{R}^+\). We will show that \[ \Gamma(a) \times \Gamma(b) = \int_0^{\infty} \int_0^1 (uv)^{a - 1} (u - uv)^{b - 1} e^{-u} u dvdu. \]
Proof: First, we want to find another way to express the quantity \(\Gamma(a) \times \Gamma(b)\), which is a product of single integrals, as a double integral. Notice that all the values with \(t\) are constant with respect to \(s\), and vice versa, so: \[ \begin{aligned} \Gamma(a) \times \Gamma(b) &\equiv \left[ \int_0^{\infty} t^{a - 1} e^{-t} dt \right] \times \left[ \int_0^{\infty} s^{b - 1} e^{-s} ds \right] \\ &= \int_0^{\infty} \int_0^{\infty} t^{a - 1} s^{b - 1} e^{-(t + s)} dtds. \end{aligned} \]
Next, we want to transform from variables in \(\mathbb{R}^+ \times \mathbb{R}^+\) to variables in \(\mathbb{R}^+ \times (0,1)\). For our change of variables, we target the \(\exp[-(t+s)]\) component first, because exponentials are more challenging to integrate than polynomials. So, let \(u = t + s\). We know that \(t,s \in \mathbb{R}^+\), so \(u \in \mathbb{R}^+\).
Now, the next variable, \(v\), needs to have two properties: 1) \(v \in (0,1)\), and 2) the product \(uv\) needs to be a really simple polynomial. One candidate is \(v = t/(t + s)\). For property (1), let’s fix \(s\) as any arbitrary value in \(\mathbb{R}^+\). Now consider these limits (the second limit by L’Hospital’s Rule): \[ \begin{aligned} &\lim_{t\to 0} \frac{t}{t+s} = \frac{0}{s} = 0, \\ &\lim_{t\to\infty} \frac{t}{t+s} = \lim_{t\to\infty} \frac{\frac{d}{dt} (t)}{\frac{d}{dt} (t + s)} = \lim_{t\to\infty} \frac{1}{1+0} = 1. \end{aligned} \] For property (2), we note that \(uv = [t + s][t/(t + s)] = t\), which is a very simple polynomial.
We change these variables, so \[ v = \frac{t}{t + s} = \frac{t}{u} \Rightarrow t = uv. \] Moreover, \[ u = t + s = uv + s \Rightarrow s = u - uv. \]
Now, we find the determinant of the Jacobian matrix: \[ \begin{aligned} |\textbf{J}| &\equiv \det\! \begin{bmatrix} \frac{\partial t}{\partial u} & \frac{\partial t}{\partial v} \\ \frac{\partial s}{\partial u} & \frac{\partial s}{\partial v} \end{bmatrix} \\ &= \det\! \begin{bmatrix} \frac{\partial}{\partial u} (uv) & \frac{\partial}{\partial v} (uv) \\ \frac{\partial}{\partial u} (u - uv) & \frac{\partial}{\partial v} (u - uv) \end{bmatrix} \\ &= \det\! \begin{bmatrix} v & u \\ 1 - v & -u \end{bmatrix} \\ &= \left| (v)(-u) - (u)(1 - v) \right| \\ &= \left| -uv - u + uv \right| \\ &= u. \end{aligned} \]
Therefore, taking \(t = uv\), \(s = u - uv\), and \(dtds = udvdu\), we have \[ \begin{aligned} \Gamma(a) \times \Gamma(b) &= \int_0^{\infty} \int_0^{\infty} t^{a - 1} s^{b - 1} e^{-(t + s)} dtds \\ &= \int_{u = 0}^{\infty} \int_{v = 0}^1 [uv]^{a - 1} [u - uv]^{b - 1} e^{-[u]} [udvdu], \end{aligned} \] which completes our proof.
6.6 The Complete Beta Function
You may be thinking, “why did we just do all that changing of variables? We still have two integrals to deal with!” And you aren’t wrong. However, this new double integral can be simplified to yield the Complete Beta Function. Let’s pick back up where we left off, and collect all the like terms together: \[ \begin{aligned} \Gamma(a) \times \Gamma(b) &= \int_{u = 0}^{\infty} \int_{v = 0}^1 (uv)^{a - 1} [u(1 - v)]^{b - 1} e^{-u} udvdu \\ &= \int_{u = 0}^{\infty} \int_{v = 0}^1 u^{a - 1} u^{b - 1} u^1 e^{-u} \times v^{a - 1} (1 - v)^{b - 1} dvdu \\ &= \int_{u = 0}^{\infty} u^{a - 1 + b - 1 + 1} e^{-u} du \times \int_{v = 0}^1 v^{a - 1} (1 - v)^{b - 1} dv \\ &= \int_{0}^{\infty} u^{a + b - 1} e^{-u} du \times \int_{0}^1 v^{a - 1} (1 - v)^{b - 1} dv \\ &= \Gamma(a + b) \int_{0}^1 v^{a - 1} (1 - v)^{b - 1} dv \\ \Longrightarrow \frac{\Gamma(a) \times \Gamma(b)}{\Gamma(a + b)} &= \int_{0}^1 v^{a - 1} (1 - v)^{b - 1} dv, \end{aligned} \] which is the definition of the Beta Function.
6.7 Alternate form of the Beta Function over \(\mathbb{R}^+\)
While the above form of the Beta Function over \((0,1)\) is useful, sometimes we need a variant which we can integrate over the entire positive real line (particularly for the Central \(F\) Distribution). We will let \[ t = \frac{1}{1 + s} \Rightarrow \frac{dt}{ds} = \frac{-1}{(1 + s)^2} \Rightarrow dt = -\frac{1}{(1 + s)^2}ds. \] Therefore, \[ s = \frac{1}{t} - 1,\ t \in (0,1). \]
For the bounds of integration, we have that as \(t\to 0^+\), \(s \to +\infty\).5 Also, as \(t\to 1\), \(s\to 0\). Thus, \[ \begin{aligned} \frac{\Gamma(a) \times \Gamma(b)}{\Gamma(a + b)} &= \int_{t = 0}^1 t^{a - 1} (1 - t)^{b - 1} dt \\ &= \int_{\infty}^{s = 0} \left[ \frac{1}{1 + s} \right]^{a - 1} \left[ 1 - \frac{1}{1 + s} \right]^{b - 1} \left[ -\frac{1}{(1 + s)^2}ds \right] \\ &= (-1) \int_{s = 0}^{\infty} \frac{1}{(1 + s)^{a - 1}} \frac{1}{(1 + s)^2} \left[ \frac{1 + s}{1 + s} - \frac{1}{1 + s} \right]^{b - 1} (-1) ds \\ &= \int_{s = 0}^{\infty} \frac{1}{(1 + s)^{a + 1}} \left[ \frac{s}{1 + s} \right]^{b - 1} ds \\ &= \int_{s = 0}^{\infty} \frac{1}{(1 + s)^{a + 1}} \frac{s^{b - 1}}{(1 + s)^{b - 1}} ds \\ &= \int_{s = 0}^{\infty} \frac{s^{b - 1}}{(1 + s)^{a + b}} ds. \end{aligned} \]
6.8 The Gamma Continued Recurrence Equation
If we have the Gamma function of a sum, where one summand is an integer, then we can factor the Gamma function into a product of the integer part and the Gamma function of the non-integer part. This is an extension of the Gamma recurrence relationship (which we will prove below) that \(\Gamma(z + 1) = z\Gamma(z)\). That is, for \(a\in\mathbb{R}^+\) and \(k\in\mathbb{N}\) \[ \Gamma(a + k) = \Gamma(a)\prod_{j = 0}^{k - 1} (a + j). \]
Proof: If \(a\) is an integer, then the result follows directly from the definition of the factorial. \[ \begin{aligned} \Gamma(a+k) &\equiv (a+k-1)! \\ &= (a+k-1)(a+k-2)\ldots(a+k-[k-1])(a+k-k)(a-1)! \\ &= \left\{ (a+[k - 1])(a+[k - 2])\ldots(a+[1])(a+[0]) \right\} \times (a - 1)! \\ &= \left\{ \prod_{j = 0}^{k - 1} (a+j) \right\} \times (a - 1)! \\ &= \Gamma(a) \left\{ \prod_{j = 0}^{k - 1} (a+j) \right\}. \end{aligned} \] If \(a\in\mathbb{R}^+\) but not an integer, then the factorial operator is not defined. But, we assumed that \(k\in\mathbb{N}\), so we can assume that \(k\) is fixed at some arbitrary positive integer and use a general “Proof by Induction” strategy.
For the base case, let’s set up integration by parts and let \(k = 1\). Recall that integration by parts decomposes the integral of a product as \(\int udv = uv - \int vdu\). For the integral below, let \(u = t^{a} \Rightarrow du = (a)t^{a - 1}dt\). We have the remaining portion as \(e^{-t}dt = dv\Rightarrow v = -e^{-t}\). Thus, \[ \begin{aligned} \Gamma(a+1) &= \int_0^{\infty} t^{(a+1)-1}e^{-t}dt \\ &= \lim_{\psi\to\infty} \left[ t^a \right] \left[ -e^{-t} \right]_0^{\psi} - \int_0^{\infty} \left[ -e^{-t} \right] \left[ at^{a - 1}dt \right] \\ &= \left[ \lim_{\psi\to\infty} \left(-[\psi]^a e^{-[\psi]}\right) - \left(-[0]^a e^{-[0]}\right) \right] + a \int_0^{\infty} t^{a - 1}e^{-t}dt \\ &= \left[ \lim_{\psi\to\infty} -\frac{\psi^a}{e^{\psi}} - 0 \right] + a\Gamma(a) \\ &= a\Gamma(a) - \lim_{\psi\to\infty} \frac{\psi^a}{e^{\psi}}. \end{aligned} \] Now, recall L’Hospital’s Rule. We see that we are very close to showing the result for \(k = 1\). We see that \(\psi^a/e^{\psi}\) is an indeterminant form; even more interesting, the first derivative of the top divided by the first derivative of the bottom also yields an indeterminant form: \((a\psi^{a-1})/e^{\psi}\). In fact, this ratio will be in indeterminant form for the first \(a\) derivatives of \(\psi^a\). However, taking repeated derivatives of this polynomial will eventially yield a constant.6 On the other hand, taking repeated derivatives of \(e^{\psi}\) yields \(e^{\psi}\). We see that \[ \begin{aligned} \Gamma(a+1) &= a\Gamma(a) - \lim_{\psi\to\infty} \frac{\psi^a}{e^{\psi}} \\ &= a\Gamma(a) - \lim_{\psi\to\infty} \frac{ \frac{d^a}{d\psi^a} \psi^a }{ \frac{d^a}{d\psi^a} e^{\psi} } \\ &= a\Gamma(a) - \lim_{\psi\to\infty} \frac{ \text{Constant} }{ e^{\psi} } \\ &= a\Gamma(a) - \frac{ \text{Constant} }{ \lim_{\psi\to\infty} e^{\psi} } \\ &= a\Gamma(a) - 0. \end{aligned} \] Therefore, this limit converges to 0. So, for the base case that \(k = 1\), \(\Gamma(a+1) = (a+1-1)\Gamma(a+1-1) = a\Gamma(a)\). Thus, we have shown the Gamma Recurrence relationship (as we promised).
The hypothesis is that \(\Gamma(a+k) = (a+k-1)\Gamma(a+k-1)\). Now we move to the induction. Similar to before, we let \(u = t^{a+k} \Rightarrow du = (a+k)t^{a + k - 1}dt\). We have the remaining portion as \(e^{-t}dt = dv\Rightarrow v = -e^{-t}\). Thus, \[ \begin{aligned} \Gamma(a+k+1) &= \int_0^{\infty} t^{(a+k+1)-1}e^{-t}dt \\ &= \lim_{\psi\to\infty} \left[ t^{a+k} \right] \left[ -e^{-t} \right]_0^{\psi} - \int_0^{\infty} \left[ -e^{-t} \right] \left[ (a+k)t^{a + k - 1}dt \right] \\ &= \left[ \lim_{\psi\to\infty} \left(-[\psi]^{a+k} e^{-[\psi]}\right) - \left(-[0]^{a+k} e^{-[0]}\right) \right] + (a+k) \int_0^{\infty} t^{a + k - 1}e^{-t}dt \\ &= \left[ \lim_{\psi\to\infty} -\frac{\psi^{a+k}}{e^{\psi}} - 0 \right] + (a+k) \Gamma(a+k) \\ &= (a+k) \Gamma(a+k) - \lim_{\psi\to\infty} \frac{\psi^{a+k}}{e^{\psi}} \\ &= (a+k) \Gamma(a+k), \end{aligned} \] which completes the induction step. Now, we recursively apply the hypothesis and induction until we reach the base case: \[ \begin{aligned} \Gamma(a+k) &= (a+k-1) \Gamma(a+k-1) \\ &= (a+k-1)(a+k-2)\Gamma(a+k-2) \\ &\ \ \vdots \\ &= \left\{ (a+[k - 1])(a+[k - 2])\ldots(a+[1])(a+[0]) \right\} \Gamma(a) \\ &= \left\{ \prod_{j = 0}^{k - 1} (a+j) \right\} \Gamma(a) \\ &= \Gamma(a) \left\{ \prod_{j = 0}^{k - 1} (a+j) \right\}, \end{aligned} \] which completes our proof.
6.9 Footnotes
https://www.math.ucdavis.edu/~kouba/CalcTwoDIRECTORY/usubdirectory/USubstitution.html↩︎
\(t\to 0^+\) means “as \(t\) approaches 0 from the right” (the positive side)↩︎
For integer values of \(a\), by taking \(a\) derivatives, we will eventually get to some constant times \(\psi^0\); this means that the numerator is a constant and the denominator \(\to\infty\), so the limit of the fraction as \(\psi\to\infty = 0\). For non-integer values of \(a\), by taking \(a\) derivatives, we will eventually get to some constant times \(\psi^{-c}\), so that as \(\psi\to\infty\), the numerator \(\to 0\). So, again, the numerator is a constant (specifically 0) and the denominator \(\to\infty\), so the limit of the fraction as \(\psi\to\infty = 0\).↩︎