Before we dive in, I’m going to offer a comment on the structure of this chapter and the related chapter on the Central \(\mathcal{F}\) distribution. These two chapters will be shorter. I do not include sections on the Method of Moments or Maximum Likelihood Estimators. These distributions are both parametrised by sample sizes. When independent samples are observed, then the sample sizes are known, and there’s no reason to estimate these parameters. When the observed samples are not independent, then we violate the assumptions for data used in the MoM and ML estimators anyway. In short, I haven’t found any real examples in my personal work where estimating “unknown parameters” from these distributions has been needed.
Deriving the \(\chi^2\) Distribution was a homework exercise from the chapter on the Gamma Distribution, as it is a special case of the Gamma Distribution. The probability function for this distribution is: \[ f_{\chi^2}(x|\nu) = \frac{2^{-\nu/2}}{\Gamma(\nu/2)} x^{\nu/2 - 1} e^{-x/2},\ \nu\in\mathbb{N}, \] where \(\nu\), the degrees of freedom parameter, is pronounced “new” (the Greek letter “nu”).
Curiously, the constant \(\Gamma(1/2)\) appears periodically in derivations of statistical distributions which include the Gamma and Beta functions. We will find a closed form for this value from the definition of the Gamma Function: \[ \begin{align} \Gamma(z) &= \int_0^{\infty} t^{z - 1} e^{-t} dt \\ \Longrightarrow \Gamma\left(\frac{1}{2}\right) &= \int_0^{\infty} t^{1/2 - 1} e^{-t} dt \\ &= \int_0^{\infty} \frac{1}{t^{1/2}} e^{-t} dt. \end{align} \] Now we’re going to perform a substitution: let \(u = t^{1/2}\Rightarrow du = \frac{1}{2}t^{-1/2}dt\Rightarrow 2du = t^{-1/2}dt\) and \(t = u^2\). Also, for this substitution, the bounds of integration do not change (\(\sqrt{0} = 0\) and \(\sqrt\infty \to \infty\)). Thus, remembering that \(e^{-x^2}\) is symmetric around \(x = 0\), we have that \[ \begin{align} \Gamma\left(\frac{1}{2}\right) &= \int_0^{\infty} \frac{1}{t^{1/2}} e^{-t} dt \\ &= \int_{t = 0}^{\infty} e^{-t} \frac{1}{t^{1/2}} dt \\ &= \int_{u = 0}^{\infty} e^{-[u^2]} [2du] \\ &= 2\int_{u = 0}^{\infty} e^{-u^2} du \\ &\qquad\text{\emph{Symmetric function...}} \\ &= \int_{u = -\infty}^{\infty} e^{-u^2} du \\ &\qquad\text{\emph{Multiply by well-placed 1...}} \\ &= \int_{u = -\infty}^{\infty} e^{-\frac{u^2}{2(1/2)}} du \\ &\qquad\text{\emph{Kernel of a Normal Distribution...}} \\ &= \sqrt{2\pi\left(\frac{1}{2}\right)} \int_{u = -\infty}^{\infty} \frac{1}{\sqrt{2\pi\left(\frac{1}{2}\right)}} e^{-\frac{1}{2}\frac{(u - 0)^2}{\frac{1}{2}}} du \\ &= \sqrt{2\pi\left(\frac{1}{2}\right)} [1] \\ &= \sqrt{\pi}. \end{align} \]
The Student’s \(t\) Distribution is a weighted average of the Normal Distribution and the square root of a \(\chi^2\) Distribution, scaled by its degrees of freedom. We begin by considering a random variable with a Standard Normal Distribution, say \(X \sim \mathcal{N}(\mu = 0, \sigma^2 = 1)\). Also, we consider \(Y \sim \chi^2_{\nu} \ni X \perp Y\). The joint distribution of \(X\) and \(Y\) is then \[ f(x, y|\mu,\sigma^2,\nu) = \left[ \frac{1}{\sqrt{2\pi}} e^{-\frac{1}{2}x^2} \right] \left[ \frac{2^{-\nu/2}}{\Gamma(\nu/2)} y^{\nu/2 - 1} e^{-y/2} \right]. \]
We will employ a bivariate transformation, then integrate out one of the parameters as a nuisance parameter. First, let \(S = Y\) be the nuisance parameter; \(\mathcal{S}(Y) = \mathcal{S}(S) = \mathbb{R}^+\). then let \(T = X\sqrt{\frac{\nu}{Y}}\), so \(X = T\sqrt{\frac{S}{\nu}}\); \(\mathcal{S}(X) = \mathcal{S}(T) = \mathbb{R}\). Therefore, the Jacobian Determinant is found by \[ \begin{aligned} \textbf{J} &= \begin{bmatrix} \frac{\partial X}{\partial T} & \frac{\partial Y}{\partial T} \\ \frac{\partial X}{\partial S} & \frac{\partial Y}{\partial S} \\ \end{bmatrix} \\ &= \begin{bmatrix} \frac{\partial}{\partial T} T\sqrt{\frac{S}{\nu}} & \frac{\partial}{\partial T} S \\ \frac{\partial}{\partial S} T\sqrt{\frac{S}{\nu}} & \frac{\partial}{\partial S} S \\ \end{bmatrix} \\ &= \begin{bmatrix} \sqrt{\frac{S}{\nu}} & 0 \\ \frac{T}{2} \left(\frac{S}{\nu}\right)^{-\frac{1}{2}} \frac{1}{\nu} & 1 \\ \end{bmatrix} \\ \Longrightarrow \det(\textbf{J}) &= \sqrt{\frac{S}{\nu}}(1) - (0)\frac{T}{2} \left(\frac{S}{\nu}\right)^{-\frac{1}{2}} \\ &= \sqrt{\frac{S}{\nu}}. \end{aligned} \] Therefore, the marginal distribution of \(T\) requires us to apply this transformation and then integrate out the nuisance parameter \(S\). Thus, \[ \begin{aligned} f(t, s|\nu) &= \left\{ \frac{1}{\sqrt{2\pi}} e^{-\frac{1}{2}\left[ t\sqrt{\frac{s}{\nu}} \right]^2} \right\} \left\{ \frac{2^{-\nu/2}}{\Gamma(\nu/2)} [s]^{\nu/2 - 1} e^{-[s]/2} \right\} \left\{ \sqrt{\frac{s}{\nu}} \right\} \\ \Longrightarrow f(t|\nu) &= \int_{\mathcal{S}(s)} \left\{ \frac{1}{\sqrt{2\pi}} e^{-\frac{1}{2}\left[ t\sqrt{\frac{s}{\nu}} \right]^2} \right\} \left\{ \frac{2^{-\nu/2}}{\Gamma(\nu/2)} [s]^{\nu/2 - 1} e^{-[s]/2} \right\} \left\{ \sqrt{\frac{s}{\nu}} \right\} ds \\ &= \frac{1}{\sqrt{2\pi}} \frac{2^{-\nu/2}}{\Gamma(\nu/2)} \frac{1}{\sqrt\nu} \int_0^{\infty} e^{-\frac{1}{2} t^2\frac{s}{\nu}} e^{-\frac{s}{2}} s^{\frac{\nu}{2} - 1} s^{\frac{1}{2}} ds \\ &\qquad \sqrt\pi \text{\emph{ is a special value of the Gamma Function...}} \\ &= \frac{1}{\Gamma\left(\frac{1}{2}\right)\Gamma\left(\frac{\nu}{2}\right)} \frac{1}{2^{\frac{\nu + 1}{2}}} \frac{1}{\sqrt\nu} \int_0^{\infty} s^{\frac{\nu + 1}{2} - 1} e^{-s\left( \frac{t^2}{2\nu} + \frac{1}{2} \right)} ds. \end{aligned} \]
Now, we should recognize the “guts” of this integral as a kernel of the Gamma Distribution with \(\alpha = \frac{\nu + 1}{2}\) and \(\lambda = \frac{t^2}{2\nu} + \frac{1}{2}\). Hence, \[ \begin{aligned} f(t|\nu) &= \frac{1}{\Gamma\left(\frac{1}{2}\right)\Gamma\left(\frac{\nu}{2}\right)} \frac{1}{2^{\frac{\nu + 1}{2}}} \frac{1}{\sqrt\nu} \int_0^{\infty} s^{\frac{\nu + 1}{2} - 1} e^{-s\left( \frac{t^2}{2\nu} + \frac{1}{2} \right)} ds \\ &\qquad\text{\emph{Multiply by a well-placed 1...}} \\ &= \frac{1}{\Gamma\left(\frac{1}{2}\right)\Gamma\left(\frac{\nu}{2}\right)} \frac{1}{2^{\frac{\nu + 1}{2}}} \frac{1}{\sqrt\nu} \int_0^{\infty} \left[ \frac{ \Gamma\left(\frac{\nu + 1}{2}\right) }{ \left( \frac{t^2}{2\nu} + \frac{1}{2} \right)^{\frac{\nu + 1}{2}} } \right] \left[ \frac{ \left( \frac{t^2}{2\nu} + \frac{1}{2} \right)^{\frac{\nu + 1}{2}} }{ \Gamma\left(\frac{\nu + 1}{2}\right) } \right] s^{\frac{\nu + 1}{2} - 1} e^{-s\left( \frac{t^2}{2\nu} + \frac{1}{2} \right)} ds \\ &= \frac{\Gamma\left(\frac{\nu + 1}{2}\right)}{\Gamma\left(\frac{1}{2}\right)\Gamma\left(\frac{\nu}{2}\right)} \frac{1}{ 2^{\frac{\nu + 1}{2}} \left( \frac{t^2}{2\nu} + \frac{1}{2} \right)^{\frac{\nu + 1}{2}} } \frac{1}{\sqrt\nu} \int_0^{\infty} \frac{ \left( \frac{t^2}{2\nu} + \frac{1}{2} \right)^{\frac{\nu + 1}{2}} }{ \Gamma\left(\frac{\nu + 1}{2}\right) } s^{\frac{\nu + 1}{2} - 1} e^{-s\left( \frac{t^2}{2\nu} + \frac{1}{2} \right)} ds \\ &\qquad\text{\emph{Integrate out the Gamma Distribution...}} \\ &= \frac{\Gamma\left(\frac{\nu + 1}{2}\right)}{\Gamma\left(\frac{1}{2}\right)\Gamma\left(\frac{\nu}{2}\right)} \frac{1}{ 2^{\frac{\nu + 1}{2}} \left( \frac{t^2}{2\nu} + \frac{1}{2} \right)^{\frac{\nu + 1}{2}} } \frac{1}{\sqrt\nu} [1] \\ &= \frac{\Gamma\left(\frac{\nu + 1}{2}\right)}{\Gamma\left(\frac{1}{2}\right)\Gamma\left(\frac{\nu}{2}\right)} \frac{ \left( 1 + \frac{t^2}{\nu} \right)^{-\frac{\nu + 1}{2}} }{ 2^{\frac{\nu + 1}{2}} \left(\frac{1}{2}\right)^{\frac{\nu + 1}{2}} } \frac{1}{\sqrt\nu} \\ &= \frac{ \Gamma\left(\frac{\nu + 1}{2}\right) \nu^{-\frac{1}{2}} }{ \Gamma\left(\frac{1}{2}\right)\Gamma\left(\frac{\nu}{2}\right) } \left( 1 + \frac{t^2}{\nu} \right)^{-\frac{\nu + 1}{2}}, \end{aligned} \] which will be very close to the form of the Student’s \(t\) Distribution presented in most textbooks.
set.seed(20150516)
xStd <- rt(n = 500, df = 300)
samplesStd_ls <- list(
n10 = xStd[1:10],
n30 = xStd[1:30],
n60 = xStd[1:60],
n500 = xStd
)
xDiffuse <- rt(n = 500, df = 3)
samplesDiffuse_ls <- list(
n10 = xDiffuse[1:10],
n30 = xDiffuse[1:30],
n60 = xDiffuse[1:60],
n500 = xDiffuse
)
range_num <- range(c(xStd, xDiffuse))
rm(xStd, xDiffuse)PlotSharedDensity <- function(x, range_x, bandwidth = "nrd0") {
xDens_ls <- density(x, bw = bandwidth)
xHist_ls <- hist(x, plot = FALSE)
yLargest_num <- max(max(xDens_ls$y), max(xHist_ls$density))
hist(
x, prob = TRUE,
xlim = range_x, ylim = c(0, yLargest_num)
)
lines(xDens_ls, col = 4, lwd = 2)
}par(mfrow = c(2, 2))
PlotSharedDensity(
x = samplesStd_ls$n10, range_x = range_num
)
PlotSharedDensity(
x = samplesStd_ls$n30, range_x = range_num
)
PlotSharedDensity(
x = samplesStd_ls$n60, range_x = range_num
)
PlotSharedDensity(
x = samplesStd_ls$n500, range_x = range_num
)
par(mfrow = c(1, 1))
# , bandwidth = 0.005
par(mfrow = c(2, 2))
PlotSharedDensity(
x = samplesDiffuse_ls$n10, range_x = range_num
)
PlotSharedDensity(
x = samplesDiffuse_ls$n30, range_x = range_num
)
PlotSharedDensity(
x = samplesDiffuse_ls$n60, range_x = range_num
)
PlotSharedDensity(
x = samplesDiffuse_ls$n500, range_x = range_num
)
par(mfrow = c(1, 1))
Let’s consider the variable components of the Student’s \(t\) Distribution shown here: \[ f_T(t|\nu) = \frac{ \Gamma\left(\frac{\nu + 1}{2}\right) \nu^{-\frac{1}{2}} }{ \Gamma\left(\frac{1}{2}\right)\Gamma\left(\frac{\nu}{2}\right) } \left( 1 + \frac{t^2}{\nu} \right)^{-\frac{\nu + 1}{2}}. \] Recall that the transformation we defined was \(T = X\sqrt{\frac{\nu}{Y}}\). We know that \(Y\in\mathbb{R}^+\) and \(\nu\in\mathbb{N}\), so \(\sqrt{\frac{\nu}{Y}}\in\mathbb{R}^+\). Since \(X\in\mathbb{R}\), we then have that \(\mathcal{S}(t)\subseteq\mathbb{R}\). Ergo, \(t^2\ge 0\). Additionally, for the \(\chi^2\) Distribution, we required that \(\nu\in\mathbb{N}\), so the entire distribution is then non-negative. However, because we have Gamma Functions, we can extend this distribution to non-integer values of \(\nu\) as well.1 Thus, \(f_T > 0\ \forall t\in\mathbb{R}\) and \(\nu > 0\).
As we argued above, the support of the random variable \(t\) is the entire Real line. We will first use the “symmetry trick” as we’ve seen elsewhere (that \(f_T(t) = f_T(-t)\)) to change the limits of integration. \[ \begin{aligned} \int_{\mathcal{S}(t)} dF(t|\nu) &= \int_{\mathcal{S}(t)} \frac{ \Gamma\left(\frac{\nu + 1}{2}\right) \nu^{-\frac{1}{2}} }{ \Gamma\left(\frac{1}{2}\right)\Gamma\left(\frac{\nu}{2}\right) } \left( 1 + \frac{t^2}{\nu} \right)^{-\frac{\nu + 1}{2}} dt \\ &= \frac{ \Gamma\left(\frac{\nu + 1}{2}\right) }{ \Gamma\left(\frac{1}{2}\right)\Gamma\left(\frac{\nu}{2}\right) } \int_{-\infty}^{\infty} \frac{1}{\nu^{\frac{1}{2}} \left( 1 + \frac{t^2}{\nu} \right)^{\frac{\nu + 1}{2}}} dt \\ &= \frac{ \Gamma\left(\frac{\nu + 1}{2}\right) }{ \Gamma\left(\frac{1}{2}\right)\Gamma\left(\frac{\nu}{2}\right) } 2 \int_{0}^{\infty} \frac{1}{\sqrt\nu \left( 1 + \frac{t^2}{\nu} \right)^{\frac{\nu + 1}{2}}} dt. \end{aligned} \] Now we perform a substitution. Let \(u = t^2/\nu \Rightarrow du = \frac{2}{\nu}tdt\). The problem is that we don’t have anything that looks like a \(tdt\) in our integrand. But, we do know that if \(t > 0\), then we can “solve” \(u = t^2/\nu\) for \(t\). That is, we see that \(t = +\sqrt{u\nu}\), because \(t\) is positive. As a welcome side effect, our bounds of integration stay the same. Thus, \[ \begin{aligned} du &= \frac{2}{\nu}tdt \\ &= \frac{2}{\nu}[\sqrt{u\nu}]dt \\ &= 2\sqrt{u}\frac{\sqrt\nu}{\nu}dt \\ \Longrightarrow \frac{1}{2\sqrt{u}}du &= \frac{1}{\sqrt\nu}dt, \end{aligned} \] which, thankfully, does show up in our integrand.
Hence, we have \[ \begin{aligned} \int_{\mathcal{S}(t)} dF(t|\nu) &= \frac{ \Gamma\left(\frac{\nu + 1}{2}\right) }{ \Gamma\left(\frac{1}{2}\right)\Gamma\left(\frac{\nu}{2}\right) } 2 \int_{t = 0}^{\infty} \frac{1}{\left( 1 + \frac{t^2}{\nu} \right)^{\frac{\nu + 1}{2}}} \frac{1}{\sqrt\nu} dt \\ &= \frac{ \Gamma\left(\frac{\nu + 1}{2}\right) }{ \Gamma\left(\frac{1}{2}\right)\Gamma\left(\frac{\nu}{2}\right) } 2 \int_{u = 0}^{\infty} \frac{1}{\left( 1 + [u] \right)^{\frac{\nu + 1}{2}}} \left[\frac{1}{2\sqrt{u}}du\right] \\ &= \frac{ \Gamma\left(\frac{\nu + 1}{2}\right) }{ \Gamma\left(\frac{1}{2}\right)\Gamma\left(\frac{\nu}{2}\right) } \int_{0}^{\infty} \frac{1}{\left( 1 + u \right)^{\frac{\nu + 1}{2}}} \frac{1}{\sqrt{u}}du \\ &= \frac{ \Gamma\left(\frac{\nu + 1}{2}\right) }{ \Gamma\left(\frac{1}{2}\right)\Gamma\left(\frac{\nu}{2}\right) } \int_{0}^{\infty} \frac{u^{-\frac{1}{2}}}{\left( 1 + u \right)^{\frac{\nu}{2} + \frac{1}{2}}} du \\ &= \frac{ \Gamma\left(\frac{\nu + 1}{2}\right) }{ \Gamma\left(\frac{1}{2}\right)\Gamma\left(\frac{\nu}{2}\right) } \int_{0}^{\infty} \frac{u^{\frac{1}{2} - 1}}{\left( 1 + u \right)^{\frac{\nu}{2} + \frac{1}{2}}} du. \end{aligned} \]
We can now recognise this integral as an alternate form of the Beta Function2 with \(a = \frac{\nu}{2}\) and \(b = \frac{1}{2}\). Thus, \[ \begin{aligned} \int_{\mathcal{S}(t)} dF(t|\nu) &= \frac{ \Gamma\left(\frac{\nu + 1}{2}\right) }{ \Gamma\left(\frac{1}{2}\right)\Gamma\left(\frac{\nu}{2}\right) } \int_{0}^{\infty} \frac{u^{\frac{1}{2} - 1}}{\left( 1 + u \right)^{\frac{\nu}{2} + \frac{1}{2}}} du \\ &= \frac{ \Gamma\left(\frac{\nu + 1}{2}\right) }{ \Gamma\left(\frac{1}{2}\right)\Gamma\left(\frac{\nu}{2}\right) } \mathcal{B}\left(a = \frac{\nu}{2}, b = \frac{1}{2}\right) \\ &= \frac{ \Gamma\left(\frac{\nu + 1}{2}\right) }{ \Gamma\left(\frac{1}{2}\right)\Gamma\left(\frac{\nu}{2}\right) } \frac{ \Gamma\left(\frac{\nu}{2}\right)\Gamma\left(\frac{1}{2}\right) }{ \Gamma\left(\frac{\nu}{2} + \frac{1}{2}\right) } \\ &= 1. \end{aligned} \] Thus, \(f_T(t|\nu)\) is a proper distribution.
Other exercises to be determined.
Think about what this might mean in practice, as the degrees of freedom measure the number of independent observations. See https://en.wikipedia.org/wiki/Degrees_of_freedom_(statistics)#Effective_degrees_of_freedom↩︎
For a review, see the corresponding section in the Formal Foundations chapter on the Gamma and Beta Functions.↩︎