Wishart distribution

Wishart
Notation X ~ Wp(V, n)
Parameters n degrees of freedom (real)
V > 0 scale matrix (p × p pos. def)
Support X (p × p) positive definite matrix
PDF

Mean
Mode (np − 1)V for np + 1
Variance
Entropy see below
CF

In statistics, the Wishart distribution is a generalization of the gamma distribution to multiple dimensions. It is named in honor of John Wishart, who first formulated the distribution in 1928.[1] Other names include Wishart ensemble (in random matrix theory, probability distributions over matrices are usually called "ensembles"), or Wishart–Laguerre ensemble (since its eigenvalue distribution involve Laguerre polynomials), or LOE, LUE, LSE (in analogy with GOE, GUE, GSE).[2]

It is a family of probability distributions defined over symmetric, positive-definite random matrices (i.e. matrix-valued random variables). These distributions are of great importance in the estimation of covariance matrices in multivariate statistics. In Bayesian statistics, the Wishart distribution is the conjugate prior of the inverse covariance-matrix of a multivariate-normal random-vector.[3]

Definition

Suppose G is a p × n matrix, each column of which is independently drawn from a p-variate normal distribution with zero mean:

Then the Wishart distribution is the probability distribution of the p × p random matrix [4]

known as the scatter matrix. One indicates that S has that probability distribution by writing

The positive integer n is the number of degrees of freedom. Sometimes this is written W(V, p, n). For np the matrix S is invertible with probability 1 if V is invertible.

If p = V = 1 then this distribution is a chi-squared distribution with n degrees of freedom.

Occurrence

The Wishart distribution arises as the distribution of the sample covariance matrix for a sample from a multivariate normal distribution. It occurs frequently in likelihood-ratio tests in multivariate statistical analysis. It also arises in the spectral theory of random matrices[citation needed] and in multidimensional Bayesian analysis.[5] It is also encountered in wireless communications, while analyzing the performance of Rayleigh fading MIMO wireless channels .[6]

Probability density function

Spectral density of Wishart-Laguerre ensemble with dimensions (8, 15). A reconstruction of Figure 1 of [7].

The Wishart distribution can be characterized by its probability density function as follows:

Let X be a p × p symmetric matrix of random variables that is positive semi-definite. Let V be a (fixed) symmetric positive definite matrix of size p × p.

Then, if np, X has a Wishart distribution with n degrees of freedom if it has the probability density function

where is the determinant of and Γp is the multivariate gamma function defined as

The density above is not the joint density of all the elements of the random matrix X (such -dimensional density does not exist because of the symmetry constrains ), it is rather the joint density of elements for (,[1] page 38). Also, the density formula above applies only to positive definite matrices for other matrices the density is equal to zero.

Spectral density

The joint-eigenvalue density for the eigenvalues of a random matrix is,[8][9]

where is a constant.

In fact the above definition can be extended to any real n > p − 1. If np − 1, then the Wishart no longer has a density—instead it represents a singular distribution that takes values in a lower-dimension subspace of the space of p × p matrices.[10]

Use in Bayesian statistics

In Bayesian statistics, in the context of the multivariate normal distribution, the Wishart distribution is the conjugate prior to the precision matrix Ω = Σ−1, where Σ is the covariance matrix.[11]: 135 [12]

Choice of parameters

The least informative, proper Wishart prior is obtained by setting n = p.[citation needed]

A common choice for V leverages the fact that the mean of X ~Wp(V, n) is nV. Then V is chosen so that nV equals an initial guess for X. For instance, when estimating a precision matrix Σ−1 ~ Wp(V, n) a reasonable choice for V would be n−1Σ0−1, where Σ0 is some prior estimate for the covariance matrix Σ.

Properties

Log-expectation

The following formula plays a role in variational Bayes derivations for Bayes networks involving the Wishart distribution. From equation (2.63),[13]

where is the multivariate digamma function (the derivative of the log of the multivariate gamma function).

Log-variance

The following variance computation could be of help in Bayesian statistics:

where is the trigamma function. This comes up when computing the Fisher information of the Wishart random variable.

Entropy

The information entropy of the distribution has the following formula:[11]: 693 

where B(V, n) is the normalizing constant of the distribution:

This can be expanded as follows:

Cross-entropy

The cross-entropy of two Wishart distributions with parameters and with parameters is

Note that when and we recover the entropy.

KL-divergence

The Kullback–Leibler divergence of from is

Characteristic function

The characteristic function of the Wishart distribution is

where E[⋅] denotes expectation. (Here Θ is any matrix with the same dimensions as V, 1 indicates the identity matrix, and i is a square root of −1).[9] Properly interpreting this formula requires a little care, because noninteger complex powers are multivalued; when n is noninteger, the correct branch must be determined via analytic continuation.[14]

Theorem

If a p × p random matrix X has a Wishart distribution with m degrees of freedom and variance matrix V — write — and C is a q × p matrix of rank q, then [15]

Corollary 1

If z is a nonzero p × 1 constant vector, then:[15]

In this case, is the chi-squared distribution and (note that is a constant; it is positive because V is positive definite).

Corollary 2

Consider the case where zT = (0, ..., 0, 1, 0, ..., 0) (that is, the j-th element is one and all others zero). Then corollary 1 above shows that

gives the marginal distribution of each of the elements on the matrix's diagonal.

George Seber points out that the Wishart distribution is not called the “multivariate chi-squared distribution” because the marginal distribution of the off-diagonal elements is not chi-squared. Seber prefers to reserve the term multivariate for the case when all univariate marginals belong to the same family.[16]

Estimator of the multivariate normal distribution

The Wishart distribution is the sampling distribution of the maximum-likelihood estimator (MLE) of the covariance matrix of a multivariate normal distribution.[17] A derivation of the MLE uses the spectral theorem.

Bartlett decomposition

The Bartlett decomposition of a matrix X from a p-variate Wishart distribution with scale matrix V and n degrees of freedom is the factorization:

where L is the Cholesky factor of V, and:

where and nij ~ N(0, 1) independently.[18] This provides a useful method for obtaining random samples from a Wishart distribution.[19]

Marginal distribution of matrix elements

Let V be a 2 × 2 variance matrix characterized by correlation coefficient −1 < ρ < 1 and L its lower Cholesky factor:

Multiplying through the Bartlett decomposition above, we find that a random sample from the 2 × 2 Wishart distribution is

The diagonal elements, most evidently in the first element, follow the χ2 distribution with n degrees of freedom (scaled by σ2) as expected. The off-diagonal element is less familiar but can be identified as a normal variance-mean mixture where the mixing density is a χ2 distribution. The corresponding marginal probability density for the off-diagonal element is therefore the variance-gamma distribution

where Kν(z) is the modified Bessel function of the second kind.[20] Similar results may be found for higher dimensions. In general, if follows a Wishart distribution with parameters, , then for , the off-diagonal elements

. [21]

It is also possible to write down the moment-generating function even in the noncentral case (essentially the nth power of Craig (1936)[22] equation 10) although the probability density becomes an infinite sum of Bessel functions.

The range of the shape parameter

It can be shown [23] that the Wishart distribution can be defined if and only if the shape parameter n belongs to the set

This set is named after Gindikin, who introduced it[24] in the 1970s in the context of gamma distributions on homogeneous cones. However, for the new parameters in the discrete spectrum of the Gindikin ensemble, namely,

the corresponding Wishart distribution has no Lebesgue density.

Relationships to other distributions

See also

References

  1. ^ a b Wishart, J. (1928). "The generalised product moment distribution in samples from a normal multivariate population". Biometrika. 20A (1–2): 32–52. doi:10.1093/biomet/20A.1-2.32. JFM 54.0565.02. JSTOR 2331939.
  2. ^ Livan, Giacomo; Novaes, Marcel; Vivo, Pierpaolo (2018), Livan, Giacomo; Novaes, Marcel; Vivo, Pierpaolo (eds.), "Classical Ensembles: Wishart-Laguerre", Introduction to Random Matrices: Theory and Practice, SpringerBriefs in Mathematical Physics, Cham: Springer International Publishing, pp. 89–95, doi:10.1007/978-3-319-70885-0_13, ISBN 978-3-319-70885-0, retrieved 2023-05-17
  3. ^ Koop, Gary; Korobilis, Dimitris (2010). "Bayesian Multivariate Time Series Methods for Empirical Macroeconomics". Foundations and Trends in Econometrics. 3 (4): 267–358. doi:10.1561/0800000013.
  4. ^ Gupta, A. K.; Nagar, D. K. (2000). Matrix Variate Distributions. Chapman & Hall /CRC. ISBN 1584880465.
  5. ^ Gelman, Andrew (2003). Bayesian Data Analysis (2nd ed.). Boca Raton, Fla.: Chapman & Hall. p. 582. ISBN 158488388X. Retrieved 3 June 2015.
  6. ^ Zanella, A.; Chiani, M.; Win, M.Z. (April 2009). "On the marginal distribution of the eigenvalues of wishart matrices" (PDF). IEEE Transactions on Communications. 57 (4): 1050–1060. doi:10.1109/TCOMM.2009.04.070143. hdl:1721.1/66900. S2CID 12437386.
  7. ^ Livan, Giacomo; Vivo, Pierpaolo (2011). "Moments of Wishart-Laguerre and Jacobi ensembles of random matrices: application to the quantum transport problem in chaotic cavities". Acta Physica Polonica B. 42 (5): 1081. arXiv:1103.2638. doi:10.5506/APhysPolB.42.1081. ISSN 0587-4254. S2CID 119599157.
  8. ^ Muirhead, Robb J. (2005). Aspects of Multivariate Statistical Theory (2nd ed.). Wiley Interscience. ISBN 0471769851.
  9. ^ a b Anderson, T. W. (2003). An Introduction to Multivariate Statistical Analysis (3rd ed.). Hoboken, N. J.: Wiley Interscience. p. 259. ISBN 0-471-36091-0.
  10. ^ Uhlig, H. (1994). "On Singular Wishart and Singular Multivariate Beta Distributions". The Annals of Statistics. 22: 395–405. doi:10.1214/aos/1176325375.
  11. ^ a b c Bishop, C. M. (2006). Pattern Recognition and Machine Learning. Springer.
  12. ^ Hoff, Peter D. (2009). A First Course in Bayesian Statistical Methods. New York: Springer. pp. 109–111. ISBN 978-0-387-92299-7.
  13. ^ Nguyen, Duy. "AN IN DEPTH INTRODUCTION TO VARIATIONAL BAYES NOTE". SSRN 4541076. Retrieved 15 August 2023.
  14. ^ Mayerhofer, Eberhard (2019-01-27). "Reforming the Wishart characteristic function". arXiv:1901.09347 [math.PR].
  15. ^ a b Rao, C. R. (1965). Linear Statistical Inference and its Applications. Wiley. p. 535.
  16. ^ Seber, George A. F. (2004). Multivariate Observations. Wiley. ISBN 978-0471691211.
  17. ^ Chatfield, C.; Collins, A. J. (1980). Introduction to Multivariate Analysis. London: Chapman and Hall. pp. 103–108. ISBN 0-412-16030-7.
  18. ^ Anderson, T. W. (2003). An Introduction to Multivariate Statistical Analysis (3rd ed.). Hoboken, N. J.: Wiley Interscience. p. 257. ISBN 0-471-36091-0.
  19. ^ Smith, W. B.; Hocking, R. R. (1972). "Algorithm AS 53: Wishart Variate Generator". Journal of the Royal Statistical Society, Series C. 21 (3): 341–345. JSTOR 2346290.
  20. ^ Pearson, Karl; Jeffery, G. B.; Elderton, Ethel M. (December 1929). "On the Distribution of the First Product Moment-Coefficient, in Samples Drawn from an Indefinitely Large Normal Population". Biometrika. 21 (1/4). Biometrika Trust: 164–201. doi:10.2307/2332556. JSTOR 2332556.
  21. ^ Fischer, Adrian; Gaunt, Robert E.; Andrey, Sarantsev (2023). "The Variance-Gamma Distribution: A Review". arXiv:2303.05615 [math.ST].
  22. ^ Craig, Cecil C. (1936). "On the Frequency Function of xy". Ann. Math. Statist. 7: 1–15. doi:10.1214/aoms/1177732541.
  23. ^ Peddada and Richards, Shyamal Das; Richards, Donald St. P. (1991). "Proof of a Conjecture of M. L. Eaton on the Characteristic Function of the Wishart Distribution". Annals of Probability. 19 (2): 868–874. doi:10.1214/aop/1176990455.
  24. ^ Gindikin, S.G. (1975). "Invariant generalized functions in homogeneous domains". Funct. Anal. Appl. 9 (1): 50–52. doi:10.1007/BF01078179. S2CID 123288172.
  25. ^ Dwyer, Paul S. (1967). "Some Applications of Matrix Derivatives in Multivariate Analysis". J. Amer. Statist. Assoc. 62 (318): 607–625. doi:10.1080/01621459.1967.10482934. JSTOR 2283988.

Read other articles:

  ميّز عن أبجدية. نظام كتابة نظام كتابة قائمة أنظمة الكتابة تاريخ الكتابة الوحدة الخطيّة الأنواع ألفبائية ألفبائية وصفية الأبجد أبجيدة نظام كتابة مقطعية نظام كتابة نصف مقطعية [الإنجليزية] الرسم الرمزي كتابة مختزلة حروف متصلة مقالات ذات صلة الرسم الصوريّ الرسم الفكري...

 

Primera edición francesa de Commentaires sur la société du spectacle Comentarios sobre la sociedad del espectáculo es un ensayo político del teórico Guy Debord publicado en francés en 1988 por Éditions Gérard Lebovici. El libro es reeditado a partir de 1992 por Éditions Gallimard enriquecido con el «Prólogo a la cuarta edición italiana de La sociedad del espectáculo». La versión en español de Comentarios sobre la sociedad del espectáculo ha sido publicada en 1990 por la edit...

 

هذه المقالة يتيمة إذ تصل إليها مقالات أخرى قليلة جدًا. فضلًا، ساعد بإضافة وصلة إليها في مقالات متعلقة بها. (يناير 2021) ثقافة المخدرات في المكسيك ثقافة فرعية نمت نتيجة الوجود القوي لمختلف عصابات المخدرات في جميع أنحاء المكسيك، على غرار الثقافات الفرعية الأخرى في جميع أنحاء ا...

  هذه المقالة عن قسنطينة. لمعانٍ أخرى، طالع قسنطينة (توضيح). ملعب محمد حملاويمعلومات عامةلقب ملعب البركانالمنطقة الإدارية قسنطينة البلد  الجزائر التشييد والافتتاحالتجديد 2007 الاستعمالالرياضة كرة القدم المستضيف شباب قسنطينة المالك بلدية قسنطينةمعلومات أخرىالطاقة...

 

Ploaghe PiàgheKomuneComune di PloagheLokasi Ploaghe di Provinsi SassariNegara ItaliaWilayah SardiniaProvinsiSassari (SS)Pemerintahan • Wali kotaCarlo SotgiuLuas • Total96,27 km2 (37,17 sq mi)Ketinggian427 m (1,401 ft)Populasi (2016) • Total4,538[1]Zona waktuUTC+1 (CET) • Musim panas (DST)UTC+2 (CEST)Kode pos07017Kode area telepon079Situs webhttp://www.comune.ploaghe.ss.it Ploaghe (bahasa Sardinia: Pià...

 

الدوري الأردني 2010–11 تفاصيل الموسم الدوري الأردني للمحترفين  النسخة 62  البلد الأردن  البطل نادي الوحدات  عدد المشاركين 12   2009–10 2011-12 تعديل مصدري - تعديل   الدوري الأردني 2010-11 هي النسخة التاسعة والخمسون من البطولة الأعلى للاندية في الأردن، الفيصلي سيكون حامل لق

Filipino colonel in the Philippine Revolution (1885–1910) This article needs additional citations for verification. Please help improve this article by adding citations to reliable sources. Unsourced material may be challenged and removed.Find sources: José Tagle – news · newspapers · books · scholar · JSTOR (July 2023) (Learn how and when to remove this template message) In this Spanish name, the first or paternal surname is Tagle and the s...

 

2008 studio album by Antonis RemosAlithies & PsemataΑλήθειες & ΨέματαStudio album by Antonis RemosReleased18 November 2008Recorded2008GenreModern Laika, popLength58:09LabelSony BMG Greece/EpicAntonis Remos chronology Marinella & Antonis Remos – Live(2007) Alithies & PsemataΑλήθειες & Ψέματα(2008) Kleista Ta Stomata(2011) Singles from Alithies & Psemata PoteReleased: 14 October 2008 Ekaton ForeReleased: December 2008 Eho EsenaReleas...

 

Dutch professional wrestler Malakai BlackBlack in May 2017Birth nameTom BüdgenBorn (1985-05-19) 19 May 1985 (age 38)Alkmaar, NetherlandsSpouse(s) Zelina Vega ​ ​(m. 2018)​Professional wrestling careerRing name(s)Aleister BlackMalakai Black[1]Tommy EndBilled height6 ft 0 in (183 cm)[2]Billed weight215 lb (98 kg)[3]Billed fromThe Lodge in Amsterdam[4]Amsterdam, Holland[3]Amsterdam, Netherla...

Джордж Наджент-Темпл-Ґренвілл, 1-й маркіз БекінгемGeorge Nugent-Temple-Grenville, 1st Marquess of Buckingham Джордж Наджент-Темпл-Ґренвілл, 1-й маркіз Бекінгемпортрет роботи Томаса Ґейнсборо Прапор Міністр закордонних справ Великої Британії 19 грудня 1783 року — 23 грудня 1783 року Монарх: Георг III...

 

4th Tank Battalion4th Tanks InsigniaActiveMay 12, 1943 – August 2, 2020Country United States of AmericaBranch United States Marine CorpsTypeArmor battalionRoleArmor protected firepower and shock actionPart of4th Marine DivisionMarine Forces ReserveGarrison/HQNavy and Marine Corps Reserve Center, San Diego CaliforniaMotto(s)53 DaysEngagementsWorld War II Battle of Kwajalein Battle of Saipan Battle of Tinian Battle of Iwo Jima Korean War Battle of Inchon Battle of Chosin Reserv...

 

Un plato de pasta con salsa boloñesa y queso rallado por encima. Uno de los ingredientes de las fajitas mexicanas es el queso rallado, puede verse a la izquierda de la figura. Se denomina queso rallado (en italiano: formaggio grattugiato [foɾˈmaʤːʤo ˌgratˑtuˈʤaːto]) al producto que se obtiene a partir del rallado de una pieza de queso generalmente de reggiano, reggianito o parmesano. Puede comprarse ya rallado de fábrica y debidamente empaquetado o hacerlo de forma casera valiénd...

Motor vehicle Optimal Energy JouleOverviewManufacturerOptimal EnergyAlso calledJouleDesignerKeith HelfetBody and chassisClassConcept carDimensionsLength3.8 m (149.6 in)Curb weight1,200 kg (2,600 lb) Joule was an electric five seat passenger car by Optimal Energy, a South African company based in Cape Town. According to the company, it was to have a nominal driving range of 150 km and a top speed of 135 km/h. Designed to achieve a Euro NCAP 4 star safety...

 

Satellietfoto van de baai en het schiereiland van San Francisco. Het Schiereiland van San Francisco (Engels: San Francisco Peninsula) is een schiereiland in de San Francisco Bay Area dat de Baai van San Francisco van de Grote Oceaan scheidt. Het noordelijke uiteinde bestaat uit de stad en county San Francisco. Ten zuiden daarvan ligt San Mateo County, dat het merendeel van het schiereiland inneemt. Santa Clara County in het zuiden vormt de basis van het schiereiland. The Peninsula is een bena...

 

2001 film by Kim Dae-seung Bungee Jumping of Their OwnPoster for Bungee Jumping of Their OwnHangul번지점프를 하다Revised RomanizationBeonjijeompeu-reul hadaMcCune–ReischauerPŏnjijŏmpŭ'rŭl Hada Directed byKim Dae-seungWritten byGo Eun-nimProduced byChoi Nak-kwonStarringLee Byung-hunLee Eun-juYeo Hyun-sooCinematographyLee Hu-gonEdited byPark Yoo-kyeongMusic byPark Ho-junDistributed byCineclick AsiaRelease date February 2, 2001 (2001-02-02) Running time99 minutesCount...

This article relies largely or entirely on a single source. Relevant discussion may be found on the talk page. Please help improve this article by introducing citations to additional sources.Find sources: Arnold Bakery – news · newspapers · books · scholar · JSTOR (October 2021) This article lacks inline citations besides NRIS, a database which provides minimal and sometimes ambiguous information. Please help ensure the accuracy of the information in t...

 

Bài viết này cần thêm chú thích nguồn gốc để kiểm chứng thông tin. Mời bạn giúp hoàn thiện bài viết này bằng cách bổ sung chú thích tới các nguồn đáng tin cậy. Các nội dung không có nguồn có thể bị nghi ngờ và xóa bỏ. Bộ Ngoại giao Liên bang NgaМинистерство иностранных дел Российской ФедерацииCon dấu Bộ Ngoại giaoLá cờ Bộ Ngoại giaoTổng quan Cơ quanThành lập1...

 

Philippine superhero film RubbermanDirected byEdgardo Boy VinaraoScreenplay byCris PabloStory byMichael V.Produced byArt R. IlacadStarringMichael V.CinematographyJun PerreiraEdited byEdgardo Boy VinaraoMusic byJessie LasatenProductioncompaniesOctoArts FilmsCinemax StudiosDistributed byOctoArts FilmsRelease date December 25, 1996 (1996-12-25) Running time90 minutesCountryPhilippinesLanguagesFilipinoEnglish Rubberman is a 1996 Filipino superhero comedy film directed by Edgardo Bo...

American politician David ChiuCity Attorney of San FranciscoIncumbentAssumed office November 1, 2021Preceded byDennis HerreraMember of the California State Assemblyfrom the 17th districtIn officeDecember 1, 2014 – October 31, 2021Preceded byTom AmmianoSucceeded byMatt HaneyPresident of the San Francisco Board of SupervisorsIn officeJanuary 8, 2009 – December 1, 2014Preceded byAaron PeskinSucceeded byKaty TangMember of the San Francisco Board of Supervisorsfrom the 3r...

 

1919 film Not to be confused with A Tiger's Tale, The Trail of the Tiger, or Tiger Trail. The Tiger's TrailAdvert for filmDirected byRobert EllisLouis J. GasnierPaul HurstScreenplay byGilson WilletsStory byCharles A. LogueBased onThe Long Armby Arthur B. ReeveStarringRuth RolandGeorge LarkinDistributed byPathé ExchangeAstra FilmsRelease date April 20, 1919 (1919-04-20) Running time15 episodesCountryUnited StatesLanguageSilent (English intertitles) The Tiger's Trail is a 1919 A...

 

Strategi Solo vs Squad di Free Fire: Cara Menang Mudah!