Talk:Model selection

This article is rated C-class on Wikipedia's content assessment scale. It is of interest to the following WikiProjects:

Statistics High‑importance This article is within the scope of WikiProject Statistics, a collaborative effort to improve the coverage of statistics on Wikipedia. If you would like to participate, please visit the project page, where you can join the discussion and see a list of open tasks.StatisticsWikipedia:WikiProject StatisticsTemplate:WikiProject StatisticsStatistics High This article has been rated as High-importance on the importance scale.

Query - should this page not also link to Bayes' Factor[1]?

I'm not an expert in model selection but in my field (molecular phylogenetics) model selection is an increasingly important problem in methods involves Bayesian inference (e.g. MyBayes, BEAST) and AIC is apparently 'not appropriate' for these models [2]

Any thoughts anyone? I've also posted this on the AIC[3] page. Thanks. --Comrade jo (talk) 12:19, 19 December 2007 (UTC)Reply

Right - since then I've found out a bit more about it (didn't update, bad me..)

Once again, I must emphasise that I'm not a mathematician and this is based on my limited understanding of the literature as written by maths gurus for phylogeneticists.. Also, I should have clarified further by explaining that by 'Bayesian models' I meant 'Bayesian models of nucleic acid evolution (phylogenetics) as implemented in a Markov-chain Monte Carlo (MCMC) search strategy.

Basically it seems to stem from the different estimation goals of the two methods; while AIC is applied to ML methods that try to find a single best solution so the trade-off between likelihood and parametization is a simple one (the AIC curve is a power-2 one, right?), Bayesian MCMC methods instead estimate the posterior distribution of best models (model parametizations) and so the 'best' model in terms of the maximum likelihood solution may not represent the whole posterior distribution. In fact many posterior distributions are not normal so that intuitively makes sense. In model selection of Bayesian MCMC methods for phylogenetics (which is still a young field) it has become common to use the ratio of harmonic mean posterior likelihoods ('Bayes factor') for model selection.

More references on this:

Suchard et al. (2001) Bayesian selection of continuous-time Markov chain evolutionary models. MBE 18(6):1001-1013 PMID 11371589

Suchard et al. (2005) Models for Estimating Bayes Factors with Applications to Phylogeny and Tests of Monophyly. Biometrics 61(3):665-673 PMID 16135017
--Comrade jo (talk) 11:29, 17 June 2008 (UTC)Reply

The Criteria section has a template suggesting the list might be better presented as prose. - That might not be helpful. Perhaps an introductory paragraph, or some way (sublists ?) of grouping similar criteria ? If some of the criteria are motivated by, or used in, particular fields we could mention that. Should we retitle this section as Model evaluation criteria to make it clearer ? - Rod57 (talk) 09:45, 24 March 2018 (UTC)Reply

Is the curve fitting or parameter estimation part of model selection or a separate step ? and are we only discussing regression models here ? - Rod57 (talk) 09:58, 24 March 2018 (UTC)Reply