User:PAR/Test

The information entropy of the Von Mises distribution is defined as^[1]:

${\displaystyle H=-\int _{\Gamma }f(\theta ;\mu ,\kappa )\,\ln(f(\theta ;\mu ,\kappa ))\,d\theta \,}$

where ${\displaystyle \Gamma }$ is any interval of length ${\displaystyle 2\pi }$. The logarithm of the density of the Von Mises distribution is straightforward:

${\displaystyle \ln(f(\theta ;\mu ,\kappa ))=-\ln(2\pi I_{0}(\kappa ))+\kappa \cos(\theta )\,}$

The characteristic function representation for the Von Mises distribution is:

${\displaystyle f(\theta ;\mu ,\kappa )={\frac {1}{2\pi }}\left(1+2\sum _{n=1}^{\infty }\phi _{n}\cos(n\theta )\right)}$

where ${\displaystyle \phi _{n}=I_{|n|}(\kappa )/I_{0}(\kappa )}$. Substituting these expressions into the entropy integral, exchanging the order of integration and summation, and using the orthogonality of the cosines, the entropy may be written:

${\displaystyle H=\ln(2\pi I_{0}(\kappa ))-\kappa \phi _{1}=\ln(2\pi I_{0}(\kappa ))-\kappa {\frac {I_{1}(\kappa )}{I_{0}(\kappa )}}}$

Table of closed-form stable distribution PDF's
${\displaystyle f\left(x;{\tfrac {1}{3}},0\right)=Re\left({\frac {2\exp(-i\pi /4)}{3{\sqrt {3}}\pi }}x^{-3/2}S_{0,1/3}{\Bigl (}{\frac {2\exp(i\pi /4)}{3{\sqrt {3}}}}x^{-1/2}{\Bigr )}\right)}$ where ${\displaystyle S_{\mu ,\nu }(z)}$ is a Lommel function. (Reference: Garoni & Frankel[2].)
${\displaystyle f\left(x;{\tfrac {1}{2}},0\right)={\frac {\vert x\vert ^{-3/2}}{\sqrt {2\pi }}}\left(\sin \left({\frac {1}{4\vert x\vert }}\right)\left[{\frac {1}{2}}-S\left({\sqrt {\frac {1}{2\pi \vert x\vert }}}\right)\right]+\cos \left({\frac {1}{4\vert x\vert }}\right)\left[{\frac {1}{2}}-C\left({\sqrt {\frac {1}{2\pi \vert x\vert }}}\right)\right]\right)}$ where S(x) and C(x) are Fresnel Integrals (Reference: Hopcraft et. al.[3].)
${\displaystyle f\left(x;{\tfrac {2}{3}},0,\right)={\frac {1}{2{\sqrt {3\pi }}}}\vert x\vert ^{-1}\exp \left({\frac {2}{27}}x^{-2}\right)W_{-1/2,1/6}\left({\frac {4}{27}}x^{-2}\right)}$ where ${\displaystyle W_{k,\mu }(z)}$ is a Whittaker function.) (Reference: Uchaikin & Zolotarev [4].)
Cauchy distribution ${\displaystyle f\left(x;1,0\right)={\frac {1}{\pi (x^{2}+1)}}}$ ${\displaystyle f\left(x;{\tfrac {4}{3}},0\right)={\frac {3^{5/4}}{4{\sqrt {2\pi }}}}{\frac {\Gamma (7/12)\Gamma (11/12)}{\Gamma (6/12)\Gamma (8/12)}}\,_{2}F_{2}\left({\frac {7}{12}},{\frac {11}{12}};{\frac {6}{12}},{\frac {8}{12}};{\frac {3^{3}x^{4}}{4^{4}}}\right)-{\frac {3^{11/4}x^{3}}{4^{3}{\sqrt {2\pi }}}}{\frac {\Gamma (13/12)\Gamma (17/12)}{\Gamma (18/12)\Gamma (15/12)}}\,_{2}F_{2}\left({\frac {13}{12}},{\frac {17}{12}};{\frac {18}{12}},{\frac {15}{12}};{\frac {3^{3}x^{4}}{4^{4}}}\right)}$ (Reference: Garoni & Frankel [2].) ${\displaystyle f\left(x;{\tfrac {3}{2}},0\right)={\frac {1}{\pi }}\Gamma (5/3)\,_{2}F_{3}\left({\frac {5}{12}},{\frac {11}{12}};{\frac {1}{3}},{\frac {1}{2}},{\frac {5}{6}};-{\frac {2^{2}x^{6}}{3^{6}}}\right)-{\frac {x^{2}}{3\pi }}\,_{3}F_{4}\left({\frac {3}{4}},1,{\frac {5}{4}};{\frac {2}{3}},{\frac {5}{6}},{\frac {7}{6}},{\frac {4}{3}};-{\frac {2^{2}x^{6}}{3^{6}}}\right)+{\frac {7x^{4}}{3^{4}\pi ^{2}}}\Gamma (4/3)\,_{2}F_{3}\left({\frac {13}{12}},{\frac {19}{12}};{\frac {7}{6}},{\frac {3}{2}},{\frac {5}{3}};-{\frac {2^{2}x^{6}}{3^{6}}}\right)}$ (Reference: Garoni & Frankel[2].) Normal distribution ${\displaystyle f\left(x;2,0\right)={\frac {e^{x^{2}/2}}{\sqrt {2\pi }}}}$ The following are asymmetric distributions (specifically, where β = 1). ${\displaystyle f\left(x;{\tfrac {1}{3}},1\right)={\frac {1}{\pi }}{\frac {2{\sqrt {2}}}{3^{7/4}}}x^{-3/2}K_{1/3}\left({\frac {4{\sqrt {2}}}{3^{9/4}}}x^{-1/2}\right)}$ where Kv(x) is a modified Bessel function of the second kind. (Reference: Hopcraft et. al.[3].) Lévy distribution ${\displaystyle f\left(x;{\tfrac {1}{2}},1\right)=}$ ${\displaystyle f\left(x;{\tfrac {2}{3}},1\right)={\frac {\sqrt {3}}{\sqrt {\pi }}}\vert x\vert ^{-1}\exp \left(-{\frac {16}{27}}x^{-2}\right)W_{1/2,1/6}\left({\frac {32}{27}}x^{-2}\right)}$ (Reference: Zolotarev 1961 [5].) ${\displaystyle f\left(x;{\tfrac {3}{2}},1\right)=\left\{{\begin{array}{ll}{\frac {\sqrt {3}}{\sqrt {\pi }}}\vert x\vert ^{-1}\exp \left({\frac {1}{27}}x^{3}\right)W_{1/2,1/6}\left(-{\frac {2}{27}}x^{3}\right)&x<0\\{\frac {1}{2{\sqrt {3\pi }}}}\vert x\vert ^{-1}\exp \left({\frac {1}{27}}x^{3}\right)W_{-1/2,1/6}\left({\frac {2}{27}}x^{3}\right)&x\geq 0\end{array}}\right)}$. (Reference: Kagan et. al.[6].) The symmetric distributions for which α = p / q and p > q can be derived from a result in Garoni & Frankel<ref name="G&F"\>. Also Meijer functions (Zolatarev)

While the normal distribution has the maximum entropy for a fixed first moment ${\displaystyle (E[X])}$ and second moment ${\displaystyle (E[X^{2}])}$ of the random variable, the Student's t-distribution has the maximum Tsallis entropy for a fixed first and second moment.

The Tsallis entropy of a probability density ${\displaystyle f(t)}$ is defined as:

${\displaystyle H={\frac {1-\int _{\Gamma }f(t)^{q}\,dt}{1-q}}}$

where ${\displaystyle \Gamma }$ is the support of f(t). For a normalized density (zeroth moment equal to unity), with fixed values of the first and second moment, using the calculus of variations and the method of Lagrange multipliers, the entropy H will be maximized when the Lagrangian equation is satisfied:

${\displaystyle \delta L=0=\delta H+\lambda _{0}\int _{\Gamma }\delta f\,dt+\lambda _{1}\int _{\Gamma }t\delta f\,dt+\lambda _{1}\int _{\Gamma }t^{2}\delta f\,dt}$

where the ${\displaystyle \lambda _{i}}$ are the Lagrange multipliers. The variation of the Tsallis entropy is

${\displaystyle \delta H=-{\frac {\int _{\Gamma }\delta (f^{q})\,dt}{1-q}}=-{\frac {\int _{\Gamma }qf^{q-1}\delta f\,dt}{1-q}}}$

and so the Lagrange equation is satisfied when:

${\displaystyle {\frac {qf(t)^{q-1}}{q-1}}=\lambda _{0}+\lambda _{1}t+\lambda _{2}t^{2}}$

or, solving for f(t):

${\displaystyle f(t)=\left({\frac {q}{1-q}}\,\left(\lambda _{0}+\lambda _{1}t+\lambda _{2}t^{2}\right)\right)^{\frac {1}{1-q}}}$

Solving for the ${\displaystyle \lambda _{i}}$ using the three moment constraints (assuming the centered Student's t-distribution for which the mean of t is zero):

${\displaystyle E[1]=1\,}$

${\displaystyle E[t]=0\,}$

${\displaystyle E[t^{2}]={\frac {\nu }{\nu -2}}}$

yields the Student's t-distribution as the expression which maximizes the Tsallis entropy.