Generalized iterative scaling

In statistics, generalized iterative scaling (GIS) and improved iterative scaling (IIS) are two early algorithms used to fit log-linear models,^[1] notably multinomial logistic regression (MaxEnt) classifiers and extensions of it such as MaxEnt Markov models^[2] and conditional random fields. These algorithms have been largely surpassed by gradient-based methods such as L-BFGS^[3] and coordinate descent algorithms.^[4]

See also

• Expectation-maximization

References

1. Darroch, J.N.; Ratcliff, D. (1972). "Generalized iterative scaling for log-linear models". The Annals of Mathematical Statistics. 43 (5): 1470–1480. doi:10.1214/aoms/1177692379.
2. McCallum, Andrew; Freitag, Dayne; Pereira, Fernando (2000). "Maximum Entropy Markov Models for Information Extraction and Segmentation" (PDF). Proc. ICML 2000. pp. 591–598.
3. Malouf, Robert (2002). A comparison of algorithms for maximum entropy parameter estimation (PDF). Sixth Conf. on Natural Language Learning (CoNLL). pp. 49–55. Archived from the original (PDF) on 2013-11-01.
4. Yu, Hsiang-Fu; Huang, Fang-Lan; Lin, Chih-Jen (2011). "Dual coordinate descent methods for logistic regression and maximum entropy models" (PDF). Machine Learning. 85 (1–2): 41–75. doi:10.1007/s10994-010-5221-8.