Chi-square distribution

Copied from Wikipedia.

Template:Otheruses4

Template:Unreferenced

Template:Probability distribution

In probability theory and statistics, the chi-square distribution (also chi-squared or $\chi^2$  distribution) is one of the most widely used theoretical probability distributions in inferential statistics, i.e. in statistical significance tests. It is useful because, under reasonable assumptions, easily calculated quantities can be proven to have distributions that approximate to the chi-square distribution if the null hypothesis is true.

If $X_i$ are k independent, normally distributed random variables with mean 0 and variance 1, then the random variable

$Q = \sum_{i=1}^k X_i^2$

is distributed according to the chi-square distribution. This is usually written

$Q\sim\chi^2_k.\,$

The chi-square distribution has one parameter: $k$ - a positive integer that specifies the number of degrees of freedom (i.e. the number of $X_i$)

The chi-square distribution is a special case of the gamma distribution.

The best-known situations in which the chi-square distribution are used are the common chi-square tests for goodness of fit of an observed distribution to a theoretical one, and of the independence of two criteria of classification of qualitative data. However, many other statistical tests lead to a use of this distribution. One example is Friedman's analysis of variance by ranks.

Characteristics

Probability density function

A probability density function of the chi-square distribution is

$f(x;k)= \begin{cases}\displaystyle \frac{1}{2^{k/2}\Gamma(k/2)}\,x^{k/2 - 1} e^{-x/2}&\text{for }x>0,\\ 0&\text{for }x\le0, \end{cases}$

where $\Gamma$ denotes the Gamma function, which takes particular values at the half-integers.

Cumulative distribution function

Its cumulative distribution function is:

$F(x;k)=\frac{\gamma(k/2,x/2)}{\Gamma(k/2)} = P(k/2, x/2)$

where $\gamma(k,z)$ is the lower incomplete Gamma function and $P(k, z)$ is the regularized Gamma function.

Tables of this distribution — usually in its cumulative form — are widely available and the function is included in many spreadsheets and all statistical packages.

Characteristic function

The characteristic function of the Chi-square distribution is

$\chi(t;k)=(1-2it)^{-k/2}.\,$

Properties

The chi-square distribution has numerous applications in inferential statistics, for instance in chi-square tests and in estimating variances. It enters the problem of estimating the mean of a normally distributed population and the problem of estimating the slope of a regression line via its role in Student's t-distribution. It enters all analysis of variance problems via its role in the F-distribution, which is the distribution of the ratio of two independent chi-squared random variables divided by their respective degrees of freedom.

Normal approximation

If $X\sim\chi^2_k$, then as $k$ tends to infinity, the distribution of $X$ tends to normality. However, the tendency is slow (the skewness is $\sqrt{8/k}$ and the kurtosis excess is $12/k$) and two transformations are commonly considered, each of which approaches normality faster than $X$ itself:

Fisher empirically showed that $\sqrt{2X}$ is approximately normally distributed with mean $\sqrt{2k-1}$ and unit variance. It is possible to arrive at the same normal approximation result by using moment matching. To see this, consider the mean and the variance of a Chi-distributed random variable $z=\sqrt{X}$, which are given by $\mu_z= \sqrt{2} \frac{\Gamma\left(k/2+1/2\right)}{\Gamma\left(k/2 \right)}$ and $\sigma_z^2= k-\mu_z^2$, where $\Gamma(\cdot)$ is the Gamma function. The particular ratio of the Gamma functions in $\mu_z$ has the following series expansion [1]:

$\frac{\Gamma\left(N+1/2\right)}{\Gamma\left(N \right)}=\sqrt{N}\left(1-\frac{1}{8N}+ \frac{1}{128N^2}+\frac{5}{1024N^3}-\frac{21}{32768N^4}+\ldots\right).$ When $N \gg 1$, this ratio can be approximated as follows: $\frac{\Gamma\left(N+1/2\right)}{\Gamma\left(N \right)}\approx\sqrt{N}\left(1-\frac{1}{8N}\right)\approx\sqrt{N}\left(1-\frac{1}{4N}\right)^{0.5}=\sqrt{N-1/4}.$

Then, simple moment matching results in the following approximation of $z$: $z\sim{\mathcal N}\left(\sqrt{k-1/2}, \frac{1}{2}\right)$, from which it follows that $\sqrt{2X}\sim{\mathcal N}\left(\sqrt{2k-1}, 1\right)$.

Wilson and Hilferty showed in 1931 that $\sqrt[3]{X/k}$ is approximately normally distributed with mean $1-2/(9k)$ and variance $2/(9k)$.

The expected value of a random variable having chi-square distribution with $k$ degrees of freedom is $k$ and the variance is $2k$. The median is given approximately by

$k-\frac{2}{3}+\frac{4}{27k}-\frac{8}{729k^2}.$

Note that 2 degrees of freedom lead to an exponential distribution.

Information entropy

The information entropy is given by

$H = \int_{-\infty}^\infty f(x;k)\ln(f(x;k)) dx = \frac{k}{2} + \ln \left( 2 \Gamma \left( \frac{k}{2} \right) \right) + \left(1 - \frac{k}{2}\right) \psi(k/2).$

where $\psi(x)$ is the Digamma function.

Related distributions

• $X \sim \mathrm{Exponential}(\lambda = \frac{1}{2})$ is an exponential distribution if $X \sim \chi_2^2$ (with 2 degrees of freedom).
• $Y \sim \chi_k^2$ is a chi-square distribution if $Y = \sum_{m=1}^k X_m^2$ for $X_i \sim N(0,1)$ independent that are normally distributed.
• If the $X_i\sim N(\mu_i,1)$ have nonzero means, then $Y = \sum_{m=1}^k X_m^2$ is drawn from a noncentral chi-square distribution.
• The chi-square distribution $X \sim \chi^2_\nu$ is a special case of the gamma distribution, in that $X \sim \textrm{Gamma}(\tfrac{\nu}{2}, 2)$.
• $Y \sim \mathrm{F}(\nu_1, \nu_2)$ is an F-distribution if $Y = \frac{X_1 / \nu_1}{X_2 / \nu_2}$ where $X_1 \sim \chi_{\nu_1}^2$ and $X_2 \sim \chi_{\nu_2}^2$ are independent with their respective degrees of freedom.
• $Y \sim \chi^2(\bar{\nu})$ is a chi-square distribution if $Y = \sum_{m=1}^N X_m$ where $X_m \sim \chi^2(\nu_m)$ are independent and $\bar{\nu} = \sum_{m=1}^N \nu_m$.
• if $X$ is chi-square distributed, then $\sqrt{X}$ is chi distributed.
• in particular, if $X \sim \chi_2^2$ (chi-square with 2 degrees of freedom), then $\sqrt{X}$ is Rayleigh distributed.
• if $X_1, \dots, X_n$ are i.i.d. $N(\mu, \sigma^2)$ random variables, then $\sum_{i=1}^n(X_i - \bar X)^2 \sim \sigma^2 \chi^2_{n-1}$ where $\bar X = \frac{1}{n} \sum_{i=1}^n X_i$.
• if $X \sim \mathrm{SkewLogistic}(\frac{1}{2})\,$, then $\mathrm{log}(1 + e^{-X}) \sim \chi_2^2\,$

Various chi and chi-square distributions

Name	Statistic
chi-square distribution	$\sum_{i=1}^k \frac{\left(X_i-\mu_i\right)^2}{\sigma_i^2}$
noncentral chi-square distribution	$\sum_{i=1}^k \left(\frac{X_i}{\sigma_i}\right)^2$
chi distribution	$\sqrt{\sum_{i=1}^k \left(\frac{X_i-\mu_i}{\sigma_i}\right)^2}$
noncentral chi distribution	$\sqrt{\sum_{i=1}^k \left(\frac{X_i}{\sigma_i}\right)^2}$

See also

• Cochran's theorem
• Inverse-chi-square distribution
• Degrees of freedom (statistics)
• Fisher's method for combining independent tests of significance
• Noncentral chi-square distribution

External links

• Course notes on Chi-Square Goodness of Fit Testing from Yale University Stats 101 class. Example includes hypothesis testing and parameter estimation.
• On-line calculator for the significance of chi-square, in Richard Lowry's statistical website at Vassar College.
• Distribution Calculator Calculates probabilities and critical values for normal, t-, chi2- and F-distribution
• Chi-Square Calculator for critical values of Chi-Square in R. Webster West's applet website at University of South Carolina
• Chi-Square Calculator from GraphPad
• Table of Chi-squared distribution

Template:ProbDistributions