Jensen's Inequality

Theorem 1 (Jensen's Inequality): Let $f$ be a convex function defined on an interval $I$. Let $x_1, x_2, ..., x_n \in I$ and let $t_1, t_2, ..., t_n$ be nonnegative real numbers such that $t_1 + t_2 + ... + t_n = 1$. Then $f(t_1x_1 + t_2x_2 + ... + t_nx_n) \leq t_1f(x_1) + t_2f(x_2) + ... + t_nf(x_n)$.

• Proof: We proof Jensen's inequality by mathematical induction.

• For the case when $n = 1$ we must have that $t_1 = 1$. Clearly $f(1x_1) = 1f(x_1)$.

• For the case when $n = 2$, since $f$ is convex, by definition, for every $x_1, x_2 \in I$ and for every $s \in [0, 1]$ we have that:

(1)

\begin{align} \quad f(sx_1 + (1 - s)x_2) < sf(x_1) + (1-s)f(x_2) \end{align}

• Let $t_1, t_2$ be nonnegative real numbers such that $t_1 + t_2 = 1$. Then $t_2 = 1 - t_1$ and from above we see that:

(2)

\begin{align} \quad f(t_1x_1 + t_2x_2) < t_1f(x_1) + t_2f(x_2) \end{align}

• Assume that Jensen's inequality holds for some $k$, that is, assume that for every $x_1, x_2, ..., x_k \in I$ and for every set of positive real numbers $t_1, t_2, ..., t_k$ such that $t_1 + t_2 + ... + t_k = 1$ we have that:

(3)

\begin{align} \quad f(t_1x_1 + t_2x_2 + ... + t_kx_k) < t_1f(x_1) + t_2f(x_2) + ... + t_kf(x_k) \end{align}

• Let $x_{k+1} \in I$ and let $t_1, t_2, ..., t_k, t_{k+1}$ be positive real numbers such that $t_1 + t_2 + ... + t_k + t_{k+1} = 1$. Then:

(4)

\begin{align} \quad f(t_1x_1 + t_2x_2 + ... + t_kx_k + t_{k+1}x_{k+1}) &= f \left ( \sum_{m=1}^{k} t_mx_m + t_{k+1}x_{k+1} \right ) \\ &= f \left ( (1 - t_{k+1}) \frac{1}{1 - t_{k+1}}\sum_{m=1}^{k} t_mx_m + t_{k+1}x_{k+1} \right ) \\ &\leq (1 - t_{k+1}) f \left ( \frac{1}{1 - t_{k+1}} \sum_{m=1}^{k} t_mx_m \right ) + t_{k+1} f(x_{k+1}) \\ &\leq (1 - t_{k+1}) f \left ( \sum_{m=1}^{k} \frac{t_m}{1 - t_{k+1}}x_m \right ) + t_{k+1} f(x_{k+1}) \end{align}

• Observe that since $t_1 + t_2 + ... t_k + t_{k+1} = 1$ we have that $t_1 + t_2 + ... + t_k = 1 - t_{k+1}$ and so:

(5)

\begin{align} \quad \frac{t_1}{1 - t{k+1}} + \frac{t_2}{1 - t_{k+1}} + ... + \frac{t_k}{1 - t_{k+1}} = 1 \end{align}

• So by the induction hypothesis we have that:

(6)

\begin{align} \quad f(t_1x_1 + t_2x_2 + ... + t_kx_k + t_{k+1}x_{k+1}) & \leq (1 - t_{k+1}) \sum_{m=1}^{k} \frac{t_m}{1 - t_{k+1}} f(x_m) + t_{k+1} f(x_{k+1}) \\ & \leq t_1f(x_1) + t_2f(x_2) + ... + t_kf(x_k) + t_{k+1}f(x_{k+1}) \end{align}

• So by the principal of mathematical induction, Jensen's inequality holds. $\blacksquare$