Representer theorem

The representer theorem states that any function that takes the form:

${\displaystyle f^{*}\in \arg \min _{f\in {\mathcal {H_{k}}}}\sum _{i=1}^{n}\ell (f(x_{i}),y_{i})+\Omega ({\Vert f\Vert }_{\mathcal {H_{k}}}^{2})}$

where ${\displaystyle \ell (\cdot )}$ is any loss function and ${\displaystyle \Omega (\cdot )}$ is any regularization term, the optimal solution, ${\displaystyle {\mathcal {f}}^{*}}$, will take the form:

${\displaystyle {\mathcal {f}}^{*}=\sum _{i=1}^{n}\alpha _{i}k(x_{i},\cdot )}$

Where ${\displaystyle k(x_{i},\cdot )}$ is a kernel.