The Arzelà–Ascoli Theorem

Recall from the Preliminary Definitions for the Theory of First Order ODEs page the following definitions:

• If $\mathcal F$ is a collection of functions defined on $D \subseteq \mathbb{R}^n$ then $\mathcal F$ is said to be Uniformly Bounded on $D$ if there exists an $M \in \mathbb{R}$, $M > 0$ such that $|f(x)| \leq M$ for all $x \in D$ and for all $f \in \mathcal F$.

• Furthermore, $\mathcal F$ is said to be Equicontinuous on $D$ if for all $\epsilon > 0$ there exists a $\delta > 0$ such that if $| x - y | < \delta$ then $| f(x) - f(y) | < \epsilon$ for all $x, y \in D$ and for all $f \in \mathcal F$.

We now state and prove a very important result which states that every sequence of continuous real-valued functions defined on a compact subset $D \subseteq \mathbb{R}^n$ that is uniformly bounded and equicontinuous has a uniformly convergent subsequence to a continuous function on $D$.

Theorem 1 (The Arzelà–Ascoli Theorem): Let $D \subset \mathbb{R}^n$ be compact and let $(f_n(x))_{n=1}^{\infty}$ be a sequence of continuous functions defined on $D$. If $\mathcal F = \{ f_n : n \in \mathbb{N} \}$ is uniformly bounded and equicontinuous on $D$ then there exists a subsequence $(f_{n_k}(x))_{k=1}^{\infty}$ that converges uniformly to a function $f \in C(D, \mathbb{R})$.

In the proof of the Arzelà–Ascoli theorem we use the follow result: If $(f_n(x))_{n=1}^{\infty}$ is a sequence of continuous functions defined on the compact set $D \subseteq \mathbb{R}$ then $(f_n(x))_{n=1}^{\infty}$ is a uniformly Cauchy sequence of functions if and only if there exists a subsequence $(f_{n_k}(x))_{k=1}^{\infty}$ that converges uniformly on $D$ to a function $f \in C(D, \mathbb{R})$.

• Proof: Let $(f_n(x))_{n=1}^{\infty}$ be a sequence of continuous functions defined on the compact set $D \subseteq \mathbb{R}^n$ with $\mathcal F = \{ f_n : n \in \mathbb{N} \}$ being equicontinuous and uniformly bounded.

• Since $D \subset \mathbb{R}^n$ is a compact, there exists a countable and dense subset $R = \{ r_1, r_2, ... \}$ of $D$.

• Consider the sequence $(f_{0,n}(r_1))_{n=1}^{\infty} = (f_n(r_1))_{n=1}^{\infty}$ of real numbers. Since $\mathcal F$ is uniformly bounded, the sequence $(f_{0,n}(r_1))_{n=1}^{\infty}$ is a bounded sequence of real numbers. By the Bolzano-Weierstrass theorem there exists a convergent subsequence of $(f_{0,n}(r_1))_{n=1}^{\infty}$ which we label $(f_{1,n}(r_1))_{n=1}^{\infty}$ that converges to the number which we label $f(r_1)$.

• Now consider the sequence $(f_{1,n}(r_2))_{n=1}^{\infty}$ of real numbers. Once again, since $\mathcal F$ is uniformly bounded, the sequence $(f_{1,n}(r_2))_{n=1}^{\infty}$ is a bounded sequence of real numbers. By the Bolzano-Weierstrass theorem there exists a convergence subsequence of $(f_{1,n}(r_2))_{n=1}^{\infty}$ which we label $(f_{2,n}(r_2))_{n=1}^{\infty}$ that converges to the number which we label $f(r_2)$.

• We continue constructing convergent subsequences in this manner.

• For each $k \in \mathbb{N}$, we consider the sequence $(f_{k,n}(r_{k+1}))_{n=1}^{\infty}$ of real numbers. Since $\mathcal F$ is uniformly bounded, the sequence $(f_{k,n}(r_{k+1}))_{n=1}^{\infty}$ is a bounded sequence of real numbers. By the Bolzano-Weierstrass theorem there exists a convergent subsequence of $(f_{k,n}(r_{k+1}))_{n=1}^{\infty}$ which we label $(f_{k+1,n}(r_{k+1}))_{n=1}^{\infty}$ that converges to the number which we label $f(r_{k+1})$.

(1)

\begin{align} \quad \begin{matrix} f_{1,1}(r_1) & f_{1,2}(r_1) & \cdots & f_{1,n}(r_1) & \cdots & \to & f(r_1)\\ f_{2,1}(r_2) & f_{2,2}(r_2) & \cdots & f_{2,n}(r_2) & \cdots & \to & f(r_2)\\ \vdots & \vdots & \ddots & \vdots & \vdots & \vdots & \vdots\\ f_{k,1}(r_k) & f_{k,2}(r_k) & \cdots & f_{k,n}(r_k) & \cdots & \to & f(r_k) \\ \vdots & \vdots & \ddots & \vdots & \vdots & \vdots & \vdots\\ \end{matrix} \end{align}

• By construction, each row of functions above is a subsequence of all of the rows of functions above it.

• Define a new sequence $(g_n(x))_{n=1}^{\infty} = (f_{n,n}(x))_{n=1}^{\infty}$. Then for each $r_j \in R$ we have that the numerical sequence $(g_n(r_j))_{n=1}^{\infty}$ converges to $f(r_j)$.

• Now let $\epsilon > 0$ be given. We aim to show that $(g_n(x))_{n=1}^{\infty}$ is a Cauchy sequence of functions.

• For each $r_j \in R$, since $(g_n(r_j))_{n=1}^{\infty}$ is a convergent numerical sequence, this numerical sequence is also a Cauchy sequence (since every convergent sequence of real numbers is Cauchy). So for $\displaystyle{\epsilon_1 = \frac{\epsilon}{3}}$ there exists an $M_j \in \mathbb{N}$ such that if $m, n \geq M_j$ then:

(2)

\begin{align} \quad | g_m(r_j) - g_n(r_j) | < \epsilon_1 = \frac{\epsilon}{3} \quad (*) \end{align}

• Furthermore, since $\mathcal F$ is equicontinuous, for $\displaystyle{\epsilon_2 = \frac{\epsilon}{3} > 0}$ there exists a $\delta^* > 0$ such that for all $x, y \in D$ with $| x - y | < \delta^*$ and for all functions $g_n \in \mathcal F$ we have that:

(3)

\begin{align} \quad | g_n(x) - g_n(y) | < \epsilon_2 = \frac{\epsilon}{3} \end{align}

• So choose $\delta > 0$ (dependent on $M_j$ and $\delta^*$ above) so that $(*)$ and $(**)$ hold. So if $| x - y | < \delta$, for all $x \in D$ we have that:

(4)

\begin{align} \quad |g_m(x) - g_n(x)| &= |g_m(x) - g_m(r_j) + g_m(r_j) - g_n(r_j) + g_n(r_j) - g_n(x) | \\ & \leq | g_m(x) - g_m(r_j) | + | g_m(r_j) - g_n(r_j) | + |g_n(r_j) - g_n(x) | \\ & < \epsilon_2 + \epsilon_1 + \epsilon_2 \\ & < \frac{\epsilon}{3} + \frac{\epsilon}{3} + \frac{\epsilon}{3} \\ & < \epsilon \end{align}

• So the sequence of functions $(g_n(x))_{n=1}^{\infty}$ is Cauchy on all of $D$.

• We now show that $(g_n(x))_{n=1}^{\infty}$ is uniformly Cauchy on $D$.

• Consider the collection of open balls $\{ B(r_j, \delta) : r_j \in R \}$. This is an open cover of $D$ since the set $R$ is dense in $D$. Since $D$ is compact, there exists a finite open subcover:

(5)

\begin{align} \quad \{ B(r_{j_1}, \delta), B(r_{j_2}, \delta), ..., B(r_{j_l}, \delta) \} \end{align}

• For each $r_{j_k}$ ($k \in \{ 1, 2, ... l \}$) there is a corresponding $M_{j_k}$ as defined above. Let:

(6)

\begin{align} \quad M = \max \{ M_{j_1}, M_{j_2}, ..., M_{j_l} \} \end{align}

• So if $m, n \geq M$ and $x \in D$ then $x \in B(r_{j_k}, \delta)$ for some $k \in \{ 1, 2, ..., l \}$ and from above, $| g_m(x) - g_n(x) | < \epsilon$. So $(g_n(x))_{n=1}^{\infty}$ is a uniformly Cauchy sequence. But recall that a sequence of functions $(g_n(x))_{n=1}^{\infty}$ defined on a compact set $D$ is a uniformly Cauchy sequence if and only if $(g_n(x))_{n=1}^{\infty}$ converges uniformly to a continuous function $f$ defined on $D$.

• Hence $(g_n(x))_{n=1}^{\infty}$ is a subsequence of $(f_n(x))_{n=1}^{\infty}$ that converges uniformly to a function $f \in C(D, \mathbb{R})$. $\blacksquare$