Convex Functions

Convex when the chord floats above the graph — for differentiable ff this is the same as f′f' increasing, and the second derivative decides it. Up to Jensen's inequality.

f(tx+(1−t)y)≤tf(x)+(1−t)f(y)f(tx+(1-t)y) \le tf(x)+(1-t)f(y)
f convex   ⟺  f′ increasing,f′′≥0  ⟹  convexf \text{ convex } \iff f' \text{ increasing}, \quad f'' \ge 0 \implies \text{convex}

Part of these learning paths

Examples & Exercises

Prove directly that f(x)=x2f(x) = x^2 is convex.

(tx+(1−t)y)2≤tx2+(1−t)y2  ⟺  t(1−t)(x−y)2≥0(tx+(1-t)y)^2 \le tx^2+(1-t)y^2 \iff t(1-t)(x-y)^2 \ge 0
a=−1,b=1a=-1, b=1 — the chord floats above the parabola
Show proof
  1. Expanding the difference gives tx2+(1−t)y2−(tx+(1−t)y)2=t(1−t)(x−y)2tx^2+(1-t)y^2-(tx+(1-t)y)^2 = t(1-t)(x-y)^2. (Squaring (tx+(1−t)y)(tx+(1-t)y) and collecting terms factors t(1−t)t(1-t) out of the x2x^2 and y2y^2 coefficients.)
  2. Since t∈[0,1]t \in [0,1], we have t(1−t)≥0t(1-t) \ge 0, and (x−y)2≥0(x-y)^2 \ge 0 as a square. So the right-hand side is nonnegative.
  3. That is, (tx+(1−t)y)2≤tx2+(1−t)y2(tx+(1-t)y)^2 \le tx^2+(1-t)y^2 — the definition of convexity holds, so x2x^2 is convex. This is also why the max gap is 11 for a=−1,b=1a=-1, b=1 in the picture above.

Prove that if ff is differentiable and f′f' is increasing, then ff is convex.

f(x2)−f(x1)x2−x1=f′(c1)≤f′(c2)=f(x3)−f(x2)x3−x2\frac{f(x_2)-f(x_1)}{x_2-x_1} = f'(c_1) \le f'(c_2) = \frac{f(x_3)-f(x_2)}{x_3-x_2}
exe^x — an example with f′′>0f'' > 0
Show proof
  1. Take x1<x2<x3x_1 < x_2 < x_3. The mean value theorem on [x1,x2][x_1, x_2] and [x2,x3][x_2, x_3] gives c1∈(x1,x2)c_1 \in (x_1, x_2) and c2∈(x2,x3)c_2 \in (x_2, x_3) with f′(c1)=f(x2)−f(x1)x2−x1f'(c_1) = \frac{f(x_2)-f(x_1)}{x_2-x_1} and f′(c2)=f(x3)−f(x2)x3−x2f'(c_2) = \frac{f(x_3)-f(x_2)}{x_3-x_2}. Since c1<c2c_1 < c_2 and f′f' is increasing, f′(c1)≤f′(c2)f'(c_1) \le f'(c_2).
  2. Rearranging the slope inequality f(x2)−f(x1)x2−x1≤f(x3)−f(x2)x3−x2\frac{f(x_2)-f(x_1)}{x_2-x_1} \le \frac{f(x_3)-f(x_2)}{x_3-x_2} gives f(x2)≤(x3−x2)f(x1)+(x2−x1)f(x3)x3−x1f(x_2) \le \frac{(x_3-x_2)f(x_1)+(x_2-x_1)f(x_3)}{x_3-x_1} — the middle value lies below the chord joining the endpoints.
  3. With x2=tx1+(1−t)x3x_2 = tx_1+(1-t)x_3 we have x3−x2x3−x1=t\frac{x_3-x_2}{x_3-x_1} = t and x2−x1x3−x1=1−t\frac{x_2-x_1}{x_3-x_1} = 1-t, so the inequality becomes f(tx1+(1−t)x3)≤tf(x1)+(1−t)f(x3)f(tx_1+(1-t)x_3) \le tf(x_1)+(1-t)f(x_3) — the definition of convexity. Note that f′′≥0f'' \ge 0 makes f′f' increasing by applying the mean value theorem to f′f'.

Derive the AM–GM inequality for two numbers from the concavity of log.

log⁡a+b2≥log⁡a+log⁡b2=log⁡ab\log\frac{a+b}{2} \ge \frac{\log a + \log b}{2} = \log\sqrt{ab}
the chord picture applies to log just the same
Show proof
  1. Since (log⁡)′′(x)=−1x2<0(\log)''(x) = -\frac{1}{x^2} < 0, the concave version of the second-derivative test (the reverse of the inequality in problem 2) makes log⁡\log concave — the chord lies below the graph.
  2. Jensen's inequality (concave version) with t=12t = \frac12 gives log⁡a+b2≥log⁡a+log⁡b2\log\frac{a+b}{2} \ge \frac{\log a + \log b}{2}.
  3. The right-hand side is 12(log⁡a+log⁡b)=log⁡ab\frac12(\log a + \log b) = \log\sqrt{ab}, so exponentiating gives a+b2≥ab\frac{a+b}{2} \ge \sqrt{ab} — the AM–GM inequality. (Exponentiation is increasing, so the inequality is preserved.)

9. Differentiation