Talk:Self-information

This redirect does not require a rating on Wikipedia's content assessment scale. It is of interest to the following WikiProjects:

Physics Low‑importance This redirect is within the scope of WikiProject Physics, a collaborative effort to improve the coverage of Physics on Wikipedia. If you would like to participate, please visit the project page, where you can join the discussion and see a list of open tasks.PhysicsWikipedia:WikiProject PhysicsTemplate:WikiProject Physicsphysics Low This redirect has been rated as Low-importance on the project's importance scale.

What is the continuous analogue for this? Btyner 02:21, 13 May 2006 (UTC)Reply

There is no continuous analog for self-information. This could be mentioned in the article. 130.94.162.64 04:44, 16 May 2006 (UTC)Reply

Why isn't there? It wouldn't be too hard to define a continuous I(x) "information density function" so that ${\displaystyle \int _{a}^{b}I(t)dt=-\log _{2}{\int _{a}^{b}f(t)dt}}$ i.e. integrating the idf from a to b would give you the self-information of the event corresponding to the interval (a,b), if f(x) is some probability density function. I'm not saying this has been done, but I'd be surprised if it hasn't. 68.221.39.112 00:23, 2 December 2006 (UTC)Reply

Should't Hartley be mentioned here? 139.179.137.234 (talk) 12:49, 15 April 2008 (UTC)Reply

The continuous analog doesnt need an integral as the discrete doesn't have a sum. It is simply the log of the likelihood instead of the probability. This may vary by application; at least in the case of scoring rules, the log of the likelihood is indeed the continuous analog.199.46.198.232 (talk) 21:14, 8 June 2011 (UTC)Reply

Who put that log(1/p) is entropy? Isn't entropy an average information, H = sum p log (1/p)? --Javalenok (talk) 20:42, 16 August 2014 (UTC)Reply

In information theory, self-information is a measure of the information content^{[clarification needed]} associated with the outcome of a random variable. It is expressed in a unit of information, for example bits, nats, or hartleys, depending on the base of the logarithm used in its calculation. The term self-information is also sometimes used as a synonym of entropy, i.e. the expected value of self-information in the first sense, because ${\displaystyle I(X;X)=H(X)}$, where ${\displaystyle I(X;X)}$ is the mutual information of X with itself.^[1] These two meanings are not equivalent, and this article covers the first sense only. For the other sense, see Entropy.

Hard to understand. It explains the difference between two meanings by defining the second meaning in terms of the first meaning before the first meaning has been adequately explained. Then, it motivates something about these meanings by using symbols that haven't been explained yet, and aren't even explained in the article.

It looks like someone's notes to themselves...I'll try to fix it. 89.217.13.75 (talk) 21:57, 3 February 2015 (UTC)Reply

The current version manages to pack both an infinite recursion (surprisal ... is the expected value of the 'surprisal' of a random event) and a contradiction (surprisal ... is not the same as the surprisal.) into the same opening sentence. It seems to be a misquote of the suggested sentence in http://bactra.org/weblog/1146.html, which uses those clauses to define self-information as distinct from surprisal. Roystgnr (talk) 16:36, 4 January 2017 (UTC)Reply