When Does a Quadratic Form Follow a Chi-Square Distribution?

🔎 Motivation

Quadratic forms arise in many statistical methods:

Sample variance (measuring spread)
Regression & ANOVA (partitioning variability)
Multivariate tests like Hotelling’s $T^2$

So: When exactly does a quadratic form follow a Chi-square distribution?

📘 What is a Quadratic Form?

Given a random vector $X$ of size $p \times 1$ and a symmetric matrix $A$ of size $p \times p$, we define a quadratic form:

$$ Q = X'AX $$

This expression shows up naturally when you compute sums of squares in statistics.

📗 Reminder: Chi-Square Distribution

If $Z_1, Z_2, \ldots, Z_k$ are independent standard normal variables, then

$$ \chi^2_k = \sum_{i=1}^k Z_i^2 $$

So a Chi-square variable is just a sum of squared standard normal variables.

📙 Distribution of Quadratic Form

If $X \sim N_p(0, I)$, then the distribution of

$$ Q = X'AX $$

depends on the eigenvalues of $A$. In other words, the structure of $A$ tells us whether $Q$ will be Chi-square or not.

✅ Necessary and Sufficient Conditions

The quadratic form $Q = X'AX$ follows a Chi-square distribution with $k$ degrees of freedom if and only if:

$X \sim N_p(0, I)$ (standard normal vector)
$A$ is symmetric and idempotent ($A^2 = A$)
$\text{rank}(A) = k$

This gives us the conditions that guarantee $Q \sim \chi^2_k$.

🤔 Why Idempotency?

When $A^2 = A$, the matrix $A$ acts like a projection matrix. It projects the vector $X$ onto a subspace of dimension $k$. The quadratic form then measures the squared length of this projection:

$$ Q = X'AX = \sum_{i=1}^k Z_i^2 \sim \chi^2_k $$

Figure 1: Projection interpretation — idempotent matrix projects $X$ onto a subspace; quadratic form measures squared length.

📊 Real-Life Examples

1. Sample Variance

For data $X_1, \ldots, X_n \sim N(\mu, \sigma^2)$:

$$ S^2 = \frac{1}{n-1} \sum_{i=1}^n (X_i - \bar{X})^2 $$

Rearranging this into a quadratic form, we get:

$$ \frac{(n-1)S^2}{\sigma^2} \sim \chi^2_{n-1} $$

Here, the projection matrix has rank $n-1$.

2. Regression (ANOVA)

Consider the regression model:

$$ Y = X\beta + \varepsilon, \quad \varepsilon \sim N(0, \sigma^2 I) $$

The regression sum of squares is

$$ SSR = Y' P Y, \quad P = X(X'X)^{-1}X' $$

Since $P$ is idempotent with rank $p$:

$$ \frac{SSR}{\sigma^2} \sim \chi^2_p $$

Figure 2: SSR as a quadratic form — the fitted value $\hat{Y}=PY$ is the projection of the observed $Y$.

3. Hotelling’s $T^2$

In multivariate testing:

$$ T^2 = n (\bar{X} - \mu_0)' S^{-1} (\bar{X} - \mu_0) $$

Under the null hypothesis, this reduces to a Chi-square (or a scaled F-distribution). This forms the backbone of multivariate hypothesis testing.

Figure 3: Geometric view — components $Z_1, Z_2$ contribute squared lengths; sum gives the quadratic form.

📝 Key Takeaways

A quadratic form $Q = X'AX$ is Chi-square if: $X$ is multivariate standard normal, $A$ symmetric & idempotent, and $\text{rank}(A)=k$.
Idempotency gives geometric meaning via projection onto a $k$-dimensional subspace.
Real applications: Sample variance ($\chi^2_{n-1}$), regression/ANOVA ($\chi^2_p$), Hotelling’s $T^2$.

🎥 Video Lecture

Watch my detailed lecture on Quadratic Forms and Chi-Square Distribution here:

👉 Visit my YouTube channel @drkmanojstat and subscribe for more lecture videos.

🎯 Closing Thoughts

Quadratic forms are the hidden engine behind many statistical tests. The conditions above explain exactly when they turn into Chi-square variables, which makes them powerful tools in both univariate and multivariate settings.

Thank you for reading!
Like, Share, and Subscribe for more Statistics Lectures.
Questions? Drop them in the comments!

Visit my blog for more resources: statweb.blogspot.in

Thanks for reading When Does a Quadratic Form Follow a Chi-Square Distribution?

Sunday, September 14, 2025