Stone–Weierstrass Theorem

Can materials other than polynomials approximate every continuous function?

The ingredient set needs three things. Closure under addition and multiplication. The power to tell two different points apart. And the constant functions. With only these, it tracks any continuous function as closely as you like.

For example, layering round waves can build even a pointy 🏠 roof shape. In this scene, waves of sines and cosines close in on the triangle wave on a circle.

Sn(x)=π2−4π∑odd k≤ncos⁡(kx)k2→∥⋅∥∞∣x∣S_n(x) = \frac{\pi}{2} - \frac{4}{\pi}\sum_{\text{odd } k \le n} \frac{\cos(kx)}{k^2} \xrightarrow{\|\cdot\|_\infty} |x|
ak=2π∫0πxcos⁡(kx) dx=2π(−1)k−1k2a_k = \frac{2}{\pi}\int_0^{\pi} x\cos(kx)\,dx = \frac{2}{\pi}\frac{(-1)^k - 1}{k^2}
∥S8−f∥∞≈0.0792,∥S25−f∥∞≈0.0245,∥S40−f∥∞≈0.0159\|S_8 - f\|_\infty \approx 0.0792,\quad \|S_{25} - f\|_\infty \approx 0.0245,\quad \|S_{40} - f\|_\infty \approx 0.0159

Examples & Exercises

Show that the trigonometric polynomials on the circle S1S^1, i.e. functions of the form T(θ)=∑k=−nnckeikθT(\theta) = \sum_{k=-n}^{n} c_k e^{ik\theta}, satisfy the hypotheses of the Stone–Weierstrass theorem: they form a subalgebra closed under products, contain the constants, and separate points.

eikθeimθ=ei(k+m)θ,cos⁡Acos⁡B=12(cos⁡(A−B)+cos⁡(A+B))e^{ik\theta} e^{im\theta} = e^{i(k+m)\theta}, \qquad \cos A \cos B = \tfrac12(\cos(A-B) + \cos(A+B))
θ1≠θ2(mod2π) ⟹ cos⁡θ1≠cos⁡θ2 or sin⁡θ1≠sin⁡θ2\theta_1 \ne \theta_2 \pmod{2\pi} \ \Longrightarrow \ \cos\theta_1 \ne \cos\theta_2 \ \text{or} \ \sin\theta_1 \ne \sin\theta_2
Degree n=8n = 8: uniform error ∥S8−f∥∞≈0.0792\lVert S_8 - f\rVert_\infty \approx 0.0792 — a trigonometric polynomial approximating the triangle wave f(x)=∣x∣f(x) = |x|, with no algebraic polynomial in sight.
Show proof
  1. Subalgebra: the set T\mathcal{T} of finite linear combinations of the eikθe^{ik\theta} is a linear space, and the product of two elements stays in T\mathcal{T} because eikθeimθ=ei(k+m)θe^{ik\theta} e^{im\theta} = e^{i(k+m)\theta}. In the real form ∑k=0n(akcos⁡kθ+bksin⁡kθ)\sum_{k=0}^n (a_k \cos k\theta + b_k \sin k\theta) the product-to-sum identities (e.g. cos⁡Acos⁡B=12(cos⁡(A−B)+cos⁡(A+B))\cos A \cos B = \tfrac12(\cos(A-B) + \cos(A+B))) play the same role.

    Hence T\mathcal{T} is a subalgebra closed under products.

  2. Contains constants: the k=0k = 0 term c0ei0θ=c0c_0 e^{i0\theta} = c_0 produces every constant, so T\mathcal{T} contains all constant functions.

  3. Conjugate-closed (the complex Stone–Weierstrass hypothesis): eikθ‾=e−ikθ\overline{e^{ik\theta}} = e^{-ik\theta}, and −k-k is again an allowed exponent, so the conjugate lies in T\mathcal{T} — T\mathcal{T} is closed under conjugation. Also, membership of cos⁡θ,sin⁡θ\cos\theta, \sin\theta in T\mathcal{T} is derived, not assumed: cos⁡θ=eiθ+e−iθ2∈T\cos\theta = \frac{e^{i\theta} + e^{-i\theta}}{2} \in \mathcal{T} and sin⁡θ=eiθ−e−iθ2i∈T\sin\theta = \frac{e^{i\theta} - e^{-i\theta}}{2i} \in \mathcal{T}, both being linear combinations of the exponentials.

  4. Separates points: let θ1≠θ2(mod2π)\theta_1 \ne \theta_2 \pmod{2\pi}. If both cos⁡\cos and sin⁡\sin agreed there, then eiθ1=cos⁡θ1+isin⁡θ1=cos⁡θ2+isin⁡θ2=eiθ2e^{i\theta_1} = \cos\theta_1 + i\sin\theta_1 = \cos\theta_2 + i\sin\theta_2 = e^{i\theta_2}, forcing θ1=θ2(mod2π)\theta_1 = \theta_2 \pmod{2\pi} — a contradiction.

    So at least one of u(θ)=cos⁡θ∈Tu(\theta) = \cos\theta \in \mathcal{T}, u(θ)=sin⁡θ∈Tu(\theta) = \sin\theta \in \mathcal{T} takes different values at θ1\theta_1 and θ2\theta_2: T\mathcal{T} separates points.

  5. Numerical check (degree n=8n = 8): by the coefficient identity ak=2π(−1)k−1k2a_k = \frac{2}{\pi}\frac{(-1)^k - 1}{k^2}, the even kk vanish and only odd kk contribute −4πk2-\frac{4}{\pi k^2}, so S8(x)=π2−4π∑odd k≤8cos⁡(kx)k2S_8(x) = \frac{\pi}{2} - \frac{4}{\pi}\sum_{\text{odd } k \le 8} \frac{\cos(kx)}{k^2} is a trigonometric polynomial. The uniform error on the 400-point grid is 0.07920.0792, matching the caption, falling to 0.01590.0159 at n=40n = 40.

Let A⊂C(K,R)\mathcal{A} \subset C(K, \mathbb{R}) be a uniformly closed algebra.

Show that f∈Af \in \mathcal{A} implies ∣f∣∈A|f| \in \mathcal{A}, and deduce that max⁡(f,g)\max(f, g) and min⁡(f,g)\min(f, g) also belong to the (uniformly closed) A\mathcal{A}.

∣f∣=pk∘f 의 극한,pk→∣t∣ on [−M,M],M=∥f∥∞|f| = p_k \circ f \ \text{의 극한}, \quad p_k \to |t| \ \text{on } [-M, M], \quad M = \|f\|_\infty
max⁡(f,g)=f+g+∣f−g∣2,min⁡(f,g)=f+g−∣f−g∣2\max(f, g) = \frac{f + g + |f - g|}{2}, \qquad \min(f, g) = \frac{f + g - |f - g|}{2}
Degree n=8n = 8: ∥S8−f∥∞≈0.0792\lVert S_8 - f\rVert_\infty \approx 0.0792 — polynomial limits capture the absolute value of a continuous function uniformly, as used in the proof.
Show proof
  1. Hypothesis check: KK is assumed compact (the Stone–Weierstrass setting), so the continuous ff is bounded on KK and M:=∥f∥∞<∞M := \lVert f\rVert_\infty < \infty. The absolute-value function t↦∣t∣t \mapsto |t| is continuous on [−M,M][-M, M], so by the Weierstrass approximation theorem (the preceding scene) there are polynomials qkq_k with qk→∣t∣q_k \to |t| uniformly on [−M,M][-M, M].

    Now set pk(t):=qk(t)−qk(0)p_k(t) := q_k(t) - q_k(0). Then pk(0)=0p_k(0) = 0, and since qk(0)→∣0∣=0q_k(0) \to |0| = 0, we have ∥pk−∣t∣∥≤∥qk−∣t∣∥+∣qk(0)∣→0\lVert p_k - |t|\rVert \le \lVert q_k - |t|\rVert + |q_k(0)| \to 0 — so pk→∣t∣p_k \to |t| uniformly on [−M,M][-M, M] as well.

  2. Since pk(0)=0p_k(0) = 0, pkp_k is a polynomial with zero constant term: pk(t)=∑j≥1akjtjp_k(t) = \sum_{j \ge 1} a_{kj} t^j. Hence the composition pk∘f=∑j≥1akjfjp_k \circ f = \sum_{j \ge 1} a_{kj} f^j lies in A\mathcal{A} — A\mathcal{A}, being an algebra (a subspace closed under products), contains all powers of ff and their linear combinations. (This removal of the constant term is essential: A\mathcal{A} is not assumed to contain 11 or the constants.)

    Because f(K)⊂[−M,M]f(K) \subset [-M, M], uniform convergence of pk→∣t∣p_k \to |t| on [−M,M][-M, M] implies uniform convergence of pk∘f→∣f∣p_k \circ f \to |f| on KK.

  3. Since A\mathcal{A} is uniformly closed, the uniform limit ∣f∣|f| lies in A\mathcal{A}. If f−g∈Af - g \in \mathcal{A} then ∣f−g∣∈A|f - g| \in \mathcal{A}, so the identities max⁡(f,g)=f+g+∣f−g∣2\max(f,g) = \frac{f+g+|f-g|}{2} and min⁡(f,g)=f+g−∣f−g∣2\min(f,g) = \frac{f+g-|f-g|}{2} give max⁡(f,g),min⁡(f,g)∈A\max(f,g), \min(f,g) \in \mathcal{A} — a uniformly closed algebra is a lattice.

  4. Note: the algebra hypothesis is essential for the pk∘fp_k \circ f step — it is what makes powers of ff available. The lattice property is what lets the density argument below carve approximants with min⁡\min and max⁡\max.

Complete the density argument of the Stone–Weierstrass theorem: for a subalgebra A⊂C(K,R)\mathcal{A} \subset C(K, \mathbb{R}) (KK compact) that separates points and contains the constants, the uniform closure A‾\overline{\mathcal{A}} is uniformly dense in C(K,R)C(K, \mathbb{R}).

(u(x)1u(y)1)(ab)=(f(x)f(y)),det⁡=u(x)−u(y)≠0\begin{pmatrix} u(x) & 1 \\ u(y) & 1 \end{pmatrix}\begin{pmatrix} a \\ b \end{pmatrix} = \begin{pmatrix} f(x) \\ f(y) \end{pmatrix}, \quad \det = u(x) - u(y) \ne 0
gx=min⁡jgxyj≤f+ε,h=max⁡igxi,f−ε<h<f+εg_x = \min_j g_{xy_j} \le f + \varepsilon, \quad h = \max_i g_{x_i}, \quad f - \varepsilon < h < f + \varepsilon
Degree n=25n = 25: ∥S25−f∥∞≈0.0245\lVert S_{25} - f\rVert_\infty \approx 0.0245 — the lattice-refined approximant lands inside ε\varepsilon, shown numerically.
Show proof
  1. The uniform closure A‾\overline{\mathcal{A}} is a uniformly closed algebra, still separating points and containing constants. That it is an algebra: if fn→ff_n \to f and gn→gg_n \to g uniformly on KK, the convergent sequences are uniformly bounded (some constant bounds all ∥fn∥,∥gn∥\lVert f_n\rVert, \lVert g_n\rVert), and ∥fngn−fg∥≤∥fn∥∥gn−g∥+∥g∥∥fn−f∥→0\lVert f_n g_n - fg\rVert \le \lVert f_n\rVert \lVert g_n - g\rVert + \lVert g\rVert \lVert f_n - f\rVert \to 0 — hence fg∈A‾fg \in \overline{\mathcal{A}}. (This boundedness step is needed because multiplication on all of C(K)C(K) is not globally uniformly continuous.)

    The lattice steps (b)–(c) below are carried out inside A‾\overline{\mathcal{A}}, not A\mathcal{A} (the absolute-value lemma applied to the uniformly closed algebra A‾\overline{\mathcal{A}}). Proving A‾\overline{\mathcal{A}} dense in C(K,R)C(K, \mathbb{R}) then gives density of A\mathcal{A}.

  2. (a) Two-point interpolation: fix f∈C(K,R)f \in C(K, \mathbb{R}) and ε>0\varepsilon > 0. For distinct x,y∈Kx, y \in K, the separation hypothesis gives u∈Au \in \mathcal{A} with u(x)≠u(y)u(x) \ne u(y).

    The system (u(x)1u(y)1)(ab)=(f(x)f(y))\begin{pmatrix} u(x) & 1 \\ u(y) & 1 \end{pmatrix}\begin{pmatrix} a \\ b \end{pmatrix} = \begin{pmatrix} f(x) \\ f(y) \end{pmatrix} has determinant u(x)−u(y)≠0u(x) - u(y) \ne 0, so a unique solution (a,b)(a, b) exists, and gxy:=au+b∈Ag_{xy} := au + b \in \mathcal{A} satisfies gxy(x)=f(x)g_{xy}(x) = f(x), gxy(y)=f(y)g_{xy}(y) = f(y) (the bb term lies in A\mathcal{A} because A\mathcal{A} contains the constants). For x=yx = y take the constant function f(x)f(x).

  3. (b) Pressing from above (run with ε/2\varepsilon/2 slack): fix xx. For each y∈Ky \in K, the set Uy:={z:gxy(z)<f(z)+ε/2}U_y := \{z : g_{xy}(z) < f(z) + \varepsilon/2\} is open by continuity of gxyg_{xy} and ff, and y∈Uyy \in U_y since gxy(y)=f(y)g_{xy}(y) = f(y). So {Uy:y∈K}\{U_y : y \in K\} is an open cover of the compact KK, and a finite subcover Uy1,…,UymU_{y_1}, \dots, U_{y_m} exists.

    Quantifier care: gxyj<f+ε/2g_{xy_j} < f + \varepsilon/2 holds only on UyjU_{y_j}, not on all of KK. But for each z∈Kz \in K there is some jj with z∈Uyjz \in U_{y_j}, and for that jj, gxyj(z)<f(z)+ε/2g_{xy_j}(z) < f(z) + \varepsilon/2; hence gx(z):=min⁡jgxyj(z)≤gxyj(z)<f(z)+ε/2g_x(z) := \min_j g_{xy_j}(z) \le g_{xy_j}(z) < f(z) + \varepsilon/2 holds pointwise for every zz. By the lattice property applied to A‾\overline{\mathcal{A}}, gx=min⁡jgxyj∈A‾g_x = \min_j g_{xy_j} \in \overline{\mathcal{A}}. Also gxyj(x)=f(x)g_{xy_j}(x) = f(x) for every jj, so gx(x)=f(x)g_x(x) = f(x).

  4. (c) Lifting from below: for each x∈Kx \in K, Vx:={z:gx(z)>f(z)−ε}V_x := \{z : g_x(z) > f(z) - \varepsilon\} is open and x∈Vxx \in V_x since gx(x)=f(x)g_x(x) = f(x). The {Vx}\{V_x\} form an open cover of KK, so a finite subcover Vx1,…,VxlV_{x_1}, \dots, V_{x_l} exists.

    With h:=max⁡igxi∈A‾h := \max_i g_{x_i} \in \overline{\mathcal{A}} (lattice property in A‾\overline{\mathcal{A}}), every z∈Kz \in K lies in some VxiV_{x_i}, giving h(z)≥gxi(z)>f(z)−εh(z) \ge g_{x_i}(z) > f(z) - \varepsilon, while the ε/2\varepsilon/2 slack in (b) gives gxi≤f+ε/2g_{x_i} \le f + \varepsilon/2, so h≤f+ε/2<f+εh \le f + \varepsilon/2 < f + \varepsilon. Hence f−ε<h<f+εf - \varepsilon < h < f + \varepsilon on KK, with h∈A‾h \in \overline{\mathcal{A}}.

  5. Since ε>0\varepsilon > 0 was arbitrary, f∈A‾f \in \overline{\mathcal{A}}; since f∈C(K,R)f \in C(K, \mathbb{R}) was arbitrary, A‾=C(K,R)\overline{\mathcal{A}} = C(K, \mathbb{R}) — A\mathcal{A} is uniformly dense. Numerical check: for f(x)=∣x∣f(x) = |x| and the trigonometric-polynomial algebra T\mathcal{T}, at degree n=25n = 25 we get ∥S25−f∥∞≈0.0245\lVert S_{25} - f\rVert_\infty \approx 0.0245, matching the caption, and the error tends to zero as the degree grows (0.01590.0159 at n=40n = 40).

6. Approximation