Banach Spaces, Characterizing the Separable Hilbert Space

Orthogonal Elements

Two nonzero vectors in a hilbert space S are orthogonal if their dot product is 0. Two vectors are orthonormal if they are orthogonal, and have norm one. Orthogonal vectors can always be scaled to become orthonormal.

Let x and y be orthonormal and consider the distance from x to y. This is the square root of (x-y).(x-y). Expand, and replace x.y with 0, to get x.x+y.y, or 2. Thus orthonormal vectors are sqrt(2) distance apart.

A set of vectors (possibly infinite) forms an orthogonal system if every pair of vectors is orthogonal. The vectors in an orthonormal system are orthogonal, with norm 1.

Suppose a linear combination of orthogonal vectors yields 0. Take the dot product with any of these vectors and find that the coefficient on that vector has to be 0. This holds across the board, hence the vectors in an orthogonal system are linearly independent.

Separable

Suppose S is a separable hilbert space with an uncountable orthogonal system. Convert to an orthonormal system, and find uncountably many points that are all sqrt(2) distance from each other. Place an open ball of radius ¼ about each point. These are disjoint open sets, hence any dense set has to be uncountable, and S is not separable. Therefore a separable hilbert space can only have a countable orthogonal system.

Representing v

Assume S is separable, and let b₁ b₂ b₃ etc be a countable orthonormal system inside S. If v is any vector in S, let a_j = v.b_j. Let the n^th approximation of v be the partial sum over a_jb_j, as j runs from 1 to n. What is the distance from v to its n^th approximation? Answer this by looking at the square of the distance, which is v - the n^th approximation, dotted with itself. Let's illustrate with a₁ and a₂.

(v-a₁b₁-a₂b₂) . (v-a₁b₁-a₂b₂) =
v.v - 2×(v.a₁b₁+v.a₂b₂) + (a₁b₁+a₂b₂).(a₁b₁+a₂b₂) =
v.v - 2×(a₁²+a₂²) + (a₁b₁+a₂b₂).(a₁b₁+a₂b₂) =
v.v - 2×(a₁²+a₂²) + a₁²|b₁|² + a₂²|b₂|² =
v.v - 2×(a₁²+a₂²) + a₁² + a₂² =
v.v - (a₁²+a₂²)

This value is nonnegative for all n, thus giving Bessel's (biography) inequality:

∑{1,∞} a_i² ≤ v.v

The squares of the coefficients of v are nonnegative numbers, and together they build a monotone series that is bounded by v.v. The terms approach 0, and the series converges absolutely to a real number between 0 and v.v.

The square of the norm of the difference between the i^th approximation and the j^th approximation is a section of the above series, from a_i² to a_j². As we move out in the series, these slices approach 0. The approximations define a sequence that is cauchy, and since S is complete, the approximations approach an element that I will call u.

Remember that dot product is a continuous map from S into R. If u is the limit of a cauchy sequence in S, then u.b_j is the limit of the cauchy sequence dotted with b_j. The approximations form our cauchy sequence, and when dotted with b_j, they produce 0 for a while, and then a_j thereafter. Therefore u.b_j = a_j. In other words, u and v have the same coefficients. every v produces a series of coefficients, according to our orthonormal set b₁ b₂ b₃ etc, and these coefficients span an element u in S.

If v ≠ u, the coefficients of v-u are all 0. Thus v-u is a new orthogonal vector, and the system is not maximal. If the orthonormal system is maximal, then every vector v is uniquely represented by its coefficients, such that the sequence of approximations approaches v. A maximal orthonormal system is also called complete, or total.

Apply zorn's lemma, and S contains a maximal orthonormal system, which acts as a basis for S. Watch out for the ambiguity here; I'm using basis in a hyperlinear sense. Linear combinations are usually finite, but in this case, linear combinations could be infinite, as long as the squares of the coefficients sum to a real number. This is required by bessel's inequality, and if it holds, the approximations are cauchy, and the sequence converges to something in S. In other words, the points of S are uniquely represented by square summable sequences, according to the designated basis.

If S is a finite dimensional R vector space, every orthonormal system is finite. A maximal system in n dimensional space contains n elements. It is a basis for the hilbert space, and a basis in the traditional sense.

If S is infinite dimensional, a finite orthonormal system will not span, hence a maximal orthonormal system is infinite. since S is separable, each such system is countable, hence the orthonormal basis can be designated b₁ b₂ b₃ etc, out to infinity.

The finite dimensional hilbert space is equivalent to Rⁿ; the separable infinite dimensional hilbert space is equivalent to L₂, the square summable sequences. This equivalence is more than a homeomorphism; the norm and dot product are also determined. There is but one hilbert space for each nonnegative integer, and one separable hilbert space, up to isomorphism.

Absolute Convergence

Assume there are two complete orthonormal systems, where one is a permutation of the other. The representation of u is determined by taking dot products with each b_j, and the result is a_j, no matter where b_j appears in the basis. The coefficients of u are permuted, along with the basis. Therefore, convergence is absolute. You can add a_jb_j in any order you like; the approximations will approach u. Of course they may approach u more slowly, or from another angle.

An Infinite Hilbert Space Exists

We've demonstrated uniqueness, but what about existence? Rⁿ is a hilbert space, using the euclidean norm and dot product. Let's prove L₂ is a separable hilbert space.

Let S = L₂, the set of square summable sequences. If f is such a sequence, let |f| be the square root of the sum of its squares. this is 0 only when f is 0.

Scale a square summable sequence by c and find another square summable sequence. Furthermore, |cf| = |c|×|f|.

If f and g are two sequences, consider the first n terms of f+g. The triangular inequality is valid in n space. Thus the square root of the sum of the squares of the first n terms of f+g is no larger than the square root of the sum of the squares of the first n terms of f, plust the square root of the sum of the squares of the first n terms of g. This holds for all n, so take limits as n approaches infinity. Given any of these square summable sequences, what is the limit of the norm of the first n terms, as n approaches infinity? Since square root is a continuous function from R into R, applying square root to the partial sums of a convergent series is the same as applying square root to the limit. In other words, the limit of the norms of the partial sums is simply the norm of the entire sequence. Therefore, |f+g| ≤ |f|+|g|. This proves the triangular inequality, and it also proves f+g belongs to S, which was not obvious at the outset. Therefore S is an R vector space, and a normed vector space.

To show S is complete, let u₁ u₂ u₃ etc be a cauchy sequence in S. Since these elements cluster together, their norms are bounded.

Project this sequence onto the j^th coordinate. In other words, look at the j^th term in u₁ u₂ u₃ etc. If u_n is the n^th row in an infinite matrix, we are moving down the j^th column. The distance between any two elements u_m and u_n is at least the difference in their j^th coordinates. Therefore, the projection onto the j^th coordinate defines a cauchy sequence in R. This converges to a real number that I will call c_j. This defines a new sequence c₁ c₂ c₃ etc, which is the bottom row of our infinite matrix.

Suppose c is not square summable. In other words, the sum over c_i² is unbounded. For some finite index j, the norm of the first j terms of c, a vector in j dimensional space, exceeds the norms of all the elements u_n in our cauchy sequence. Restrict to j dimensional space, the first j columns of our matrix, and the norm of c is the limit of |u_n|, as n runs to infinity. This limit is a real number that cannot rise above the largest |u_n|. Yet c lies above all these norms. This is a contradiction, hence c is square summable, and belongs to S.

Subtract c from each u_n. This does not change the distance between u_m and u_n, hence the resulting sequence is still cauchy in S. It converges to 0 iff the original sequence converges to c. Thus we have a cauchy seequence u_n, and the components all converge to 0, and we want to show that u_n converges to 0.

A sequence approaches 0 iff its norms approach 0. Move down to u_m, so that u_n beyond u_m is never more than ε/3 away from u_m. Suppose |u_m| is at least 2ε/3. It moves above ε/3 after finitely many terms. Once again we are in j dimensional space, where the cauchy sequence converges to 0. The norm of u_m in j space is more than ε/3, yet each u_n beyond u_m, and 0, is within ε/3 of u_m. This is a contradiction, hence |u_m| ≤ 2ε/3. Each u_n beyond this point has norm bounded by 2ε/3 + ε/3, or ε. The norms converge to 0, the sequence converges to 0, the original sequence converges to c, and S is complete. This makes S a banach space.

Let the dot product of f and g be the sum over f_ig_i. After n terms, the truncated dot product, squared, is bounded by the sum of the squares of the first n terms of f, times the sum of the squares of the first n terms of g. This is another application of cauchy schwarz. Taking square roots, the absolute value of the truncated dot product is bounded by the product of the two partial norms of f and g. This applies for all n, hence it applies in the limit. Therefore the dot product, as a sum of products, is absolutely convergent, and is well defined.

Note that all the properties are satisfied, including f.f = |f|². This makes S a hilbert space.

Finally, prove S is separable. Let a sequence of coefficients lie in the dense set D if finitely many are rational, and the rest are 0. This is a countable set. Let h be a base open set sentered at v, with radius ε. Represent v as the square summable series a_j. Choose n so that the n^th approximation is within ½ε of v. Then move the first n coefficients to nearby rational values. The result is a point in D, that is within ε of v. The countable set D is dense, and S is separable.

Dimension

As a hilbert space, L₂ has a countable dimension, corresponding to its complete orthogonal system. However, as a vector space, the dimension of L₂ is uncountable.

Suppose L₂ has a countable basis, as an R vector space. Write the basis as an infinite matrix, where the i^th row holds the square summable sequence that is the i^th basis element. Use lower gaussian elimination to make the matrix upper triangular. Then use upper gaussian elimination to make the matrix diagonal. The span of this basis is now the direct sum of the coordinates, and the sequence 1, 1/2, 1/4, 1/8, 1/16, …, which is square summable, is not accessible.