Orthogonal and Orthonormal Systems of Functions Review

We will now review some of the recent material regarding orthogonal and orthonormal systems of functions.

• On the Orthogonal and Orthonormal Systems of Functions page we said that a collection of functions $\{ \varphi_0, \varphi_1, \varphi_2, ... \}$ is called a System of Orthogonal Functions on $I$ in $L^2(I)$ if $(\varphi_i, \varphi_j) = 0$ for all $i \neq j$ where we defined the inner product on the space $L^2(I)$ by:

(1)

\begin{align} \quad (f, g) = \int_I f(x) \overline{g(x)} \: dx \end{align}

• Furthermore, the collection of functions above is called a System of Orthonormal Functions on $I$ in $L^2(I)$ if $(\varphi_i, \varphi_j) = 0$ for all $i \neq j$ and $(\varphi_i, \varphi_i) = 1$ for all $i$.

• The most important orthonormal system that we will be investigating is the Trigonometric System given by:

(2)

\begin{align} \quad \varphi_0(x) = \frac{1}{\sqrt{2\pi}} \quad , \quad \varphi_{2n-1}(x) = \frac{\cos nx}{\sqrt{\pi}} \quad , \quad \varphi_n(x) = \frac{\sin nx}{\sqrt{\pi}} \end{align}

(3)

\begin{align} \quad \{ \varphi_0(x), \varphi_1(x), \varphi_2(x), ... \} = \left \{ \frac{1}{\sqrt{2\pi}} , \frac{\cos x}{\sqrt{\pi}}, \frac{\sin x}{\sqrt{\pi}}, \frac{\cos 2x}{\sqrt{\pi}}, \frac{\sin 2x}{\sqrt{\pi}}, ... \right \} \end{align}

• An important example of an orthonormal system of complex functions is given for each $n \in \{0, 1, 2, ... \}$ by:

(4)

\begin{align} \quad \varphi_n(x) = \frac{e^{inx}}{\sqrt{2\pi}} = \frac{\cos nx + i \sin nx}{\sqrt{2\pi}} \end{align}

(5)

\begin{align} \quad \{ \varphi_0(x), \varphi_1(x), \varphi_2(x), ... \} = \left \{ \frac{1}{\sqrt{2\pi}}, \frac{\cos x + i \sin x}{\sqrt{\pi}}, \frac{\cos 2x + i \sin 2x}{\sqrt{2\pi}}, ... \right \} \end{align}

• On the Linear Independence of Systems of Functions page we said that a finite system of functions $\{ \varphi_0, \varphi_1, ..., \varphi_m \}$ on an interval $I$ is Linearly Independent on $I$ if the equation $\displaystyle{\sum_{k=0}^{m} c_k \varphi_k(x) = c_0\varphi_0(x) + c_1\varphi_1(x) + ... + c_m\varphi_m(x) = 0}$ implies that $c_0, c_1, ..., c_m = 0$. Furthermore, we said that an infinite system of functions $\{ \varphi_0, \varphi_1, \varphi_2, ... \}$ is linearly independent on $I$ if every finite subset is linearly independent on $I$.

• We then proved a useful result which said that every orthonormal system of functions on $I$ is also linearly independent on $I$.

• On The Best Approximation of a Function from an Orthonormal System page we looked at a theorem which said that if $\mathcal S = \{ \varphi_0, \varphi_1, ... \}$ if an orthonormal system of functions on $I$ in $L^2(I)$, $f \in L^2(I)$, $b_0, b_1, ..., b_n \in \mathbb{C}$, $c_k = (f, \varphi_k)$ for all $k \in \{ 0, 1, ..., n \}$ and if $\displaystyle{s_n(x) = \sum_{k=0}^{n} c_k \varphi_k(x)}$, $\displaystyle{t_n(x) = \sum_{k=0}^{n} b_k \varphi_k(x)}$ then for each $n \in \mathbb{N}$ we have that:

(6)

\begin{align} \quad \| f(x) - s_n(x) \| \leq \| f(x) - t_n(x) \| \end{align}

• Moreover, equality holds if and only if $b_k = c_k$ for all $k \in \{0, 1, ..., n \}$. This means that if $f \in L^2(I)$ and if we use the norm on $L^2(I)$ as an error estimator, then the best way to approximate the function $f$ as a linear combination of the first $n$ functions in the orthonormal system $\mathcal S$ is to take the coefficients to be the values $c_k = (f, \varphi_k)$ for each $k \in \{0, 1, ..., n \}$.

• On the The Fourier Series of Functions Relative to an Orthonormal System page we said that if $\mathcal S = \{ \varphi_0, \varphi_1, ... \}$ is an orthonormal system of functions on $I$ in $L^2(I)$ and if $f \in L^2(I)$ then the Fourier Coefficients of $f$ Relative to $\mathcal S$ are defined for each $n \in \{0, 1, 2, ... \}$ by

(7)

\begin{align} \quad c_n = (f, \varphi_n) \end{align}

• The Fourier Series of $f$ Relative to $\mathcal S$ is:

(8)

\begin{align} \quad f(x) \sim \sum_{n=0}^{\infty} c_n \varphi_n(x) \end{align}

• A special name is given to Fourier series generated by the trigonometric system above. If $f \in L^2([0, 2\pi])$ then the Fourier Coefficients of $f$ Generated by the Trigonometric System are defined for each $n \in \{ 0, 1, 2, ... \}$ by:

(9)

\begin{align} \quad a_n = \frac{1}{\pi} \int_0^{2\pi} f(t) \cos nt \: dt \quad , \quad b_n = \frac{1}{\pi} \int_0^{2\pi} f(t) \sin nt \: dt \end{align}

• The Fourier Series of $f$ Generated by the Trigonometric System is:

(10)

\begin{align} \quad f(x) \sim \frac{a_0}{2} + \sum_{n=1}^{\infty} (a_n \cos nx + b_n \sin nx) \end{align}

• On the Bessel's Inequality for the Sum of Coefficients of a Fourier Series page we looked at a very important inequality regarding the sum of the Fourier coefficients of a function. We saw that if $\mathcal S = \{ \varphi_0, \varphi_1, \varphi_2, ... \}$ is an orthonormal system of functions on $I$ in $L^2(I)$ and if $\displaystyle{f(x) \sim \sum_{n=0}^{\infty} c_n \varphi_n(x)}$ then:

(11)

\begin{align} \quad \sum_{n=0}^{\infty} \mid c_n \mid^2 \leq \| f(x) \|^2 \end{align}

• As an important corollary we noted that thus the series $\displaystyle{\sum_{n=0}^{\infty} \mid c_n \mid^2}$ will converge. Looking at this in the contrapositive direction, we also noted that if $c_0, c_1, ... \in \mathbb{C}$ are such that $\displaystyle{\sum_{n=0}^{\infty} \mid c_n \mid^2}$ diverges then the series $\displaystyle{\sum_{n=0}^{\infty} c_n \varphi_n(x)}$ cannot be the Fourier series for any function $f \in L^2(I)$.

• On the Parseval's Formula for the Sum of Coefficients of a Fourier Series page we saw that if $\mathcal S = \{ \varphi_0, \varphi_1, \varphi_2, ... \}$ is an orthonormal system of functions on $I$ in $L^2(I)$ and if $\displaystyle{f(x) \sim \sum_{n=0}^{\infty} c_n \varphi_n(x)}$ then the Parseval's formula$\displaystyle{\sum_{n=0}^{\infty} \mid c_n \mid^2 = \| f(x) \|^2}$ holds if and only if $\displaystyle{\lim_{n \to \infty} \| f(x) - s_n(x) \| =0}$.

• We then looked at somewhat of a converse result to Bessel's inequality. On the The Riesz-Fischer Theorem for Fourier Series page we saw that if $\mathcal S = \{ \varphi_0, \varphi_1, \varphi_2, ... \}$ is an orthonormal system of functions on $I$ in $L^2(I)$ and if $(c_n)_{n=0}^{\infty}$ is a sequence of complex numbers for which $\displaystyle{\sum_{n=0}^{\infty} \mid c_n \mid^2}$ converges then there exists a function $f \in L^2(I)$ for which Parseval's formula holds, i.e., $\displaystyle{\sum_{n=0}^{\infty} \mid c_n \mid^2 = \| f(x) \|^2}$ and for which $c_k = (f, \varphi_k)$ for each $k \in \{0, 1, 2, ... \}$.