Quadratic Forms, Extremal Values

Extremal Values

Let q(x) be a quadratic form in n dimensions, such that q is positive everywhere except the origin.  In particular, q is positive over the unit sphere.

The sphere is a closed bounded region in Rn, hence q attains its minimum and maximum somewhere on the sphere.  Let u be a unit vector such that q(u) is minimal.  Remember that q(u) is nonzero, hence q is bounded away from 0, at least when restricted to the unit sphere.

Let e be the minimum value of q on the sphere.  Thus q(x) ≥ x*e*xT, for any unit vector x on the sphere.  Write the inequality this way.

q(x) - e ≥ 0

q(x) - x*e*xT ≥ 0

x*M*xT - x*e*xT ≥ 0

x*(M-e)*xT ≥ 0

Here M-e is my notation for the matrix M with e subtracted from the main diagonal.  Hope that's not too confusing.

Since M-e is symmetric, it defines a new quadratic form, which we will call r(x).  Like q(x), r(x) is never negative.  It is positive where q(x) exceeds e, and 0 where q(x) = e.

We know that r(x) = 0 when x = u.  This because q(u) is the minimum e.  Apply the previous theorem to show that r vanishes on u iff u is in the kernel of r.  This means u is an eigen vector of M, with eigen value e.

If u points to the maximum value e, we can run the same proof.  The new quadratic form r(x) becomes nonpositive instead of nonnegative, but that's ok.  The previous theorem still holds, and once again u vanishes on the kernel of M-e, and u is an eigen vector of M, with eigen value e.

To summarize, the minimum and maximum of q on the unit sphere are eigen values of the underlying matrix M, and the vectors that point to these extremal values are eigen vectors of M.

Let's apply this result recursively, to find all the "axes" of an ellipsoid.

Start by finding an orthogonal set of eigen vectors for the underlying matrix M.  Let e be the minimum of q on the unit sphere.  If none of the eigen vectors has eigen value e, then a new eigen vector u, with the distinct eigen value e, is independent of the others.  We can't have n+1 independent vectors, so this is a contradiction.  Thus one of our original eigen vectors has eigen value e.  Call it u.

Restrict attention to the space perpendicular to u.  All the remaining eigen vectors lie in this perpendicular subspace.  Once again q(x) has some minimum on the unit sphere contained in this subspace, and it is another eigen vector.  Continue through all n dimensions, finding eigen values and eigen vectors.  Thus all eigen values lie between the minimum and maximum of q(x) inclusive.

For another proof, rotate coordinates so that M becomes diagonal.  This does not change the maximum or mininmum of q on the sphere, nor does it change the eigen values.  Now the diagonal entries are the eigen values.  Furthermore, q(x), on the unit sphere, is trapped between the smallest and largest entry on the main diagonal, i.e. the least and greatest eigen value.

As a corollary, q(x) is everywhere positive on the unit sphere iff all eigen values are positive, and the same for q(x) negative.  Again, this is clear if you rotate coordinates so that M is diagonal.

Try to picture all of this geometrically.  An ellipsoid in 3 space might take the form x2+2y2+3z2.  The minimum value occurs on the x axis, and the maximum value occurs on the z axis.  These values are 1 and 3, and they are eigen values of the diagonal matrix with entries 1 2 and 3.  The y axis represents the remaining eigen vector.  this points to the minimum or maximum if we restrict attention to the yz or xy plane respectively.

Positive Definite

A quadratic form q(x) is positive definite if q(x) is positive for x nonzero, and 0 for x = 0.  A semipositive definite form has q(x) nonnegative.

Negative definite and seminegative definite are defined similarly, with q(x) ≤ 0.

An indefinite quadratic form is none of the above.

Using the previous theorem, one can classify a quadratic form by examining the signs of the eigen values of its associated matrix.