Talk:Gibbs sampling

This article is rated C-class on Wikipedia's content assessment scale. It is of interest to the following WikiProjects:

Statistics Mid‑importance This article is within the scope of WikiProject Statistics, a collaborative effort to improve the coverage of statistics on Wikipedia. If you would like to participate, please visit the project page, where you can join the discussion and see a list of open tasks.StatisticsWikipedia:WikiProject StatisticsTemplate:WikiProject StatisticsStatistics Mid This article has been rated as Mid-importance on the importance scale. Mathematics This article is within the scope of WikiProject Mathematics, a collaborative effort to improve the coverage of mathematics on Wikipedia. If you would like to participate, please visit the project page, where you can join the discussion and see a list of open tasks.MathematicsWikipedia:WikiProject MathematicsTemplate:WikiProject Mathematicsmathematics ??? This article has not yet received a rating on the project's priority scale. Computing: Software / CompSci Low‑importance This article is within the scope of WikiProject Computing, a collaborative effort to improve the coverage of computers, computing, and information technology on Wikipedia. If you would like to participate, please visit the project page, where you can join the discussion and see a list of open tasks.ComputingWikipedia:WikiProject ComputingTemplate:WikiProject ComputingComputing Low This article has been rated as Low-importance on the project's importance scale. This article is supported by WikiProject Software (assessed as Low-importance). This article is supported by WikiProject Computer science (assessed as Mid-importance). Things you can help WikiProject Computer science with: edit history watch purge Here are some tasks awaiting attention: Article requests : Requested articles/Applied arts and sciences/Computer science, computing, and Internet Cleanup : Computer science articles needing attention Computer science articles needing expert attention Copyedit : Computing Expand : Computer science Infobox : Computer science articles without infoboxes Maintain : Timeline of computing 2020–present Photo : Find pictures for the biographies of computer scientists (see List of computer scientists) Computing articles needing images Stubs : Computer science stubs Unreferenced : WikiProject Computer science/Unreferenced BLPs Project-related : Tag all relevant articles in Category:Computer science and sub-categories with {{WikiProject Computer science}}

I'm not an expert, but the phrase 'the stationary distribution' suggests uniqueness to me, but my intuition suggests there could be cases where it isn't unique. For example p(x,y)=0 unless |x-y|<1/2 and 1<|x-y|<2, in which case p(x,y)=1.

I think there is a typo in your example. If |x-y|<1/2 then it is never the case that |x-y|>1...

Anyway you are correct that for some distributions with inaccessible regions of state space, Gibbs sampling does not give a unique stationary distribution. In these cases Gibbs sampling by itself isn't a valid MCMC method. Here's a toy example of discrete x and y taking on values 0 and 1.

p(x=0,y=0)=0

p(x=0,y=1)=0.5

p(x=1,y=0)=0.5

p(x=1,y=1)=0

Gibbs sampling can never move out of (0,1) or (1,0). However, it does leave the distribution of interest invariant. So Gibbs sampling is always a valid MCMC operator that can be mixed with another operator that ensures ergodicity. Someone should add a lucid description of these issues at some point... 128.40.213.241 13:54, 6 September 2005 (UTC)Reply

Isn't "proven" an adjective, with the more accepted past participle of "prove" being "proved"? Quantling (talk) 11:56, 12 October 2007

I made the change to "proved". Quantling (talk) 17:00, 14 May 2008 (UTC)Reply

OK, Gibbs sampling is stochastic and produces a different result every time, vs. EM or variational approx. mathods. But the latter techniques generally use a random init, which make the result overall MORE sensitive to init that Gibbs sampling run for long enough. — Preceding unsigned comment added by 83.195.171.99 (talk) 12:05, 9 June 2012 (UTC)Reply

Hi there.

Maybe I'm not completely familiar with Gibbs sampling, but I'm motivated to ask a inline reference supporting this:

"In its basic version, Gibbs sampling is a special case of the Metropolis–Hastings algorithm. However, in its extended versions (see below), it can be considered a general framework for sampling from a large set of variables by sampling each variable (or in some cases, each group of variables) in turn, and can incorporate the Metropolis–Hastings algorithm (or similar methods such as slice sampling) to implement one or more of the sampling steps."

I though Gibbs sampling was a particular case of the Metropolis-Hastings algorithm. It was just a particular proposal function which slides on particular directions. No mather how extend it can be, it keeps being a Metropolis–Hastings algorithm. Tis should be said on the initial overview of the article.

Besides, I'm forced to say that this article either lacks inline references or it is original research. Much of what is said does not have any inline references supporting it.