Fourier series solutions VI - Inner product spaces

Warning

This note should be considered temporary, focusing only on what we need right now for the express purpose of understanding Fourier series. It will later be expanded and broken into many notes.

It turns out that much of the machinery we're building around Fourier series can be greatly streamlined if we take a minute to explore a way to define "orthogonality" for functions, very similar to the usual dot product for column vectors.

The dot product on $C^{n}$

Let's first extend the notion of dot product to vectors with complex coordinates, although we need one little adjustment.

First suppose $z = a + i b$ is a single complex number. Using the complex plane as our guide, the point corresponding to $z$ would be at coordinates $(a, b)$ , which is a distance $\sqrt{a^{2} + b^{2}}$ from the origin. Because of this, it is reasonable to define the length of the complex number $z$ as

| z | = \sqrt{a^{2} + b^{2}} .

Notice that we then immediately have

| z |^{2} = a^{2} + b^{2} .

Now, by comparison, notice that

z^{2} = (a + i b) (a + i b) = (a^{2} - b^{2}) + i \cdot 2 a b,

and so we almost never have $| z |^{2} = z^{2}$ . This in contrast with real numbers, where for any real number $x$ we always have $| x |^{2} = x^{2}$ (where $| x |$ is the usual absolute value, which measures the distance of the point $x$ from the origin on the real line). However, observe that

z \overset{―}{z} = (a + i b) (a - i b) = a^{2} + b^{2} = | z |^{2} .

Using this observation, it is hopefully reasonable to define the length of a complex vector $v = [\begin{matrix} v_{1} \\ v_{2} \\ ⋮ \\ v_{n} \end{matrix}]$ in $C^{n}$ as

| v |^{2} = | v_{1} |^{2} + | v_{2} |^{2} + \dots + | v_{n} |^{2} = v_{1} \overset{―}{v_{1}} + v_{2} \overset{―}{v_{2}} + \dots + v_{n} \overset{―}{v_{n}} .

With this in mind, it should seem reasonable to extend the dot product on $R^{n}$ to vectors in $C^{n}$ $ as follows:

Definition of dot product on

C^{n}

Given vectors $v = ⟨ v_{1}, v_{2}, \dots, v_{n} ⟩$ and $w = ⟨ w_{1}, w_{2}, \dots, w_{n} ⟩$ in $C^{n}$ , we define their dot product to be the complex number denoted $v ∙ w$ given by

v ∙ w = \sum_{k = 1}^{n} v_{k} \overset{―}{w_{k}} .

Notice that if $v$ and $w$ happen to have real entries, then this new dot product is the same as our old dot product (since complex conjugation doesn't do anything to real numbers). Also, with this definition we always have

| v |^{2} = v ∙ v,

both for vectors in $C^{n}$ and $R^{n}$ .

Extending the dot product to functions

Using the dot product on $C^{n}$ defined above as inspiration, we define the following:

A definition of an inner product on functions

Suppose $f, g : [0, 1] \to C$ are nice^[1] complex-valued functions on $[0, 1]$ . We define their inner product to be the complex number denoted $(f, g)$ given by

(f, g) = \int_{0}^{1} f (t) \overset{―}{g (t)} d t .

We can use this inner product to define a norm (or length) for functions $f : [0, 1] \to C$ , by setting $| f |$ to be the number given by

| f |^{2} = (f, f) = \int_{0}^{1} f (t) \overset{―}{f (t)} d t = \int_{0}^{1} | f (t) |^{2} .

We say a function $f$ is of unit length if $| f |^{2} = 1$ .

We can also extend the notion of orthogonal to functions.

A definition of orthogonal for functions

We say two functions $f, g : [0, 1] \to C$ are orthogonal if $(f, g) = 0$ .

Orthonormal bases

Orthonormal bases on $C^{n}$

Recall that in $C^{n}$ we have the following collection of so-called standard basis vectors:

e_{1} = [\begin{matrix} 1 \\ 0 \\ ⋮ \\ 0 \end{matrix}], e_{2} = [\begin{matrix} 0 \\ 1 \\ ⋮ \\ 0 \end{matrix}], \dots, e_{n} = [\begin{matrix} 0 \\ 0 \\ ⋮ \\ 1 \end{matrix}] .

This collection of vectors has many nice properties.

The fact that they form a basis for $C^{n}$ means that every vector in $C^{n}$ can be written uniquely as a linear combination of these vectors.
Each of these vectors is a unit vector, i.e., $| e_{k} | = 1$ for every $k = 1, \dots, n$ .
These vectors are mutually orthogonal, i.e., $e_{k} ∙ e_{j} = 0$ whenever $k \neq j$ .

The second two properties above provide a super quick and easy way to write a given vector $v$ in $C^{n}$ as a linear combination of these basis vectors. Indeed, if we first suppose

v = \sum_{k = 1}^{n} c_{k} e_{k},

then for each $j = 1, \dots, n$ we can dot product both sides of the above equality with $e_{j}$ to obtain

\begin{aligned} v ∙ e_{j} & = (\sum_{k = 1}^{n} c_{k} e_{k}) ∙ e_{j} \\ = \sum_{k = 1}^{n} c_{k} (e_{k} ∙ e_{j}) \\ = c_{k}, \end{aligned}

since those dot products are all zero except in the one case when $k = j$ , for which $e_{j} ∙ e_{j} = | e_{j} |^{2} = 1$ .

In other words, we always have

v = \sum_{k = 1}^{n} (v ∙ e_{k}) e_{k} .

Extending the above ideas to our world of periodic functions

None of the above argument is restricted to $C^{n}$ with the usual dot product and the standard basis. We can repeat the same logic in any vector space $V$ which has an inner product, and any basis $B$ that is orthonormal with respect to that inner product.

In particular, for each integer $n$ let's write $e_{n} (t) = e^{2 π i n t}$ , considered as a function from $[0, 1]$ to $C$ . Using the inner product we defined above, observe first that each of these functions has unit length:

| e_{n} |^{2} = \int_{0}^{1} e_{n} (t) \overset{―}{e_{n} (t)} d t = \int_{0}^{1} e^{2 π i n t} e^{- 2 π i n t} d t = \int_{0}^{1} 1 d t = 1.

These functions are also mutually orthogonal, since for every pair of integers $m$ and $n$ with $n \neq m$ we have

(e_{n}, e_{m}) = \int_{0}^{1} e_{n} (t) \overset{―}{e_{m} (t)} d t = \int_{0}^{1} e^{2 π i n t} e^{- 2 π i m t} d t = \int_{0}^{‘} e^{2 π i (n - m) t} d t = 0.

So while it's not clear these set of these functions is a orthonormal basis, it's at least an orthonormal set. By the same logic used above, it follows that if a function $f (t) : [0, 1] \to C$ can be written as a linear combination

f (t) = \sum_{n = - \infty}^{\infty} c_{n} e_{n} (t)

then the coefficients must be given by

c_{n} = (f, e_{n}) = \int_{0}^{1} f (t) e^{- 2 π i n t} d t = \hat{f} (n) .

The space $L^{2} ([0, 1])$

In order to state a bunch of nice facts that are actually true, we need to nail down what it means for our functions to be "nice." We have been considering functions $f : [0, 1] \to C$ and we would like to define the norm (or length) of such functions by the formula

| f |^{2} = \int_{0}^{1} | f (t) |^{2} .

So at the very least we should restrict ourselves to functions $f$ where the integral above exists and is finite. We will refer to the space of such functions as (a) Lebesgue space and denote it $L^{2} ([0, 1])$ .^[2]

Suggested next notes

Fourier series solutions VII - Solving differential equations with Fourier series

We'll decide what "nice" means later. ↩︎
Techincally, this isn't exactly the definition of this Lebesgue space, but it's close enough for what we're doing. ↩︎