Draft:Invariant coordinate selection

Review waiting, please be patient. This may take 2 months or more, since drafts are reviewed in no specific order. There are 2,468 pending submissions waiting for review. If the submission is accepted, then this page will be moved into the article space. If the submission is declined, then the reason will be posted here. In the meantime, you can continue to improve this submission by editing normally. Where to get help If you need help editing or submitting your draft, please ask us a question at the AfC Help Desk or get live help from experienced editors. These venues are only for help with editing and the submission process, not to get reviews. If you need feedback on your draft, or if the review is taking a lot of time, you can try asking for help on the talk page of a relevant WikiProject. Some WikiProjects are more active than others so a speedy reply is not guaranteed. How to improve a draft Wikipedia:Contributing to Wikipedia – a basic overview on how to edit Wikipedia. Help:Wikitext – how to use the markup Help:Referencing for beginners – how to include references Wikipedia:Article development – how to develop your article Wikipedia:Writing better articles – how to improve your article Wikipedia:Verifiability – make sure your article includes reliable third-party sources You can also browse Wikipedia:Featured articles and Wikipedia:Good articles to find examples of Wikipedia's best writing on topics similar to your proposed article. Improving your odds of a speedy review To improve your odds of a faster review, tag your draft with relevant WikiProject tags using the button below. This will let reviewers know a new draft has been submitted in their area of interest. For instance, if you wrote about a female astronomer, you would want to add the Biography, Astronomy, and Women scientists tags. Add tags to your draft Editor resources Easy tools: Citation bot (help) | Advanced: Fix bare URLs Reviewer tools Instructions · What links here · Invariant coordinate selection (talk: + · bio) · (log) · Copyvios report · reFill · Citation Bot · (Search: Google, Wikipedia) · Submitted 2 months ago by Camillemndn (talk: D · +) · Last edited 2 months ago by The Boolean

Invariant coordinate selection (ICS) is a multivariate statistical technique used to identify interesting structures in high-dimensional data.^[1] It is commonly applied in outlier detection,^[2] independent component analysis (ICA),^[3] and robust dimension reduction. ICS generalizes principal component analysis (PCA) by simultaneously diagonalizing two scatter matrices, typically the sample covariance matrix and a higher-order or robust scatter matrix.^[1]

Unlike PCA, which depends on the scale of the variables and captures only second-order structure, ICS is invariant under full-rank affine transformations of the data and can reveal non-Gaussian features of the underlying distribution.^[1] The core idea is to find a linear transformation that simultaneously diagonalizes two scatter matrices, yielding coordinates whose ordering reflects the disagreement between the two scatter measures. The method draws heavily on the robust statistics literature for its repertoire of scatter functionals.^[3]

The simultaneous use of two scatter matrices for data transformation predates ICS and originated in the clustering literature. Art, Gnanadesikan and Kettenring (1982) introduced a data-based metric for cluster analysis in which an estimate of the within-cluster covariance matrix is computed iteratively without knowledge of the cluster labels, and the data are then rescaled by its inverse square root.^[4] This procedure, implemented as PROC ACECLUS in SAS, is generally regarded as the earliest instance of the simultaneous-diagonalization idea later generalized by ICS.^[1] Caussinus and Ruiz-Gazen (1990) developed a closely related generalized principal component analysis for projection pursuit and cluster identification.^[5]

In parallel, the same algebraic structure was being investigated in signal processing under the headings of independent component analysis and blind source separation. Cardoso's Fourth-Order Blind Identification (FOBI) jointly diagonalizes the covariance matrix and a fourth-moment scatter matrix to recover independent sources.^[6] Oja and collaborators subsequently showed that any pair of affine-equivariant scatter matrices possessing the so-called independence property can be used to recover an independent component model, placing FOBI and related procedures in a common statistical framework.^[7][8]

A third influence comes from the robust statistics community, where most of the alternative scatter functionals later adopted by ICS were originally introduced—including the M-estimators of multivariate scatter studied by Maronna,^[9] the distribution-free M-estimator of Tyler,^[10] the symmetrized M-estimator of Dümbgen,^[11] and the minimum covariance determinant (MCD) estimator of Rousseeuw.^[12]

These threads were unified by Tyler, Critchley, Dümbgen and Oja (2009), who introduced the name invariant coordinate selection, characterized its affine-invariance properties, and established it as a general-purpose exploratory method that subsumes the earlier proposals as special cases.^[1]

Let ${\displaystyle X=(x_{1},\dots ,x_{n})^{T}\in \mathbb {R} ^{n\times p}}$ be a sample of ${\displaystyle n}$ observations in ${\displaystyle \mathbb {R} ^{p}}$, with sample mean ${\displaystyle {\bar {x}}}$.

A scatter matrix is a ${\displaystyle p\times p}$ symmetric positive-definite matrix-valued functional ${\displaystyle S(X)}$ of the data that is affine equivariant, meaning that for any non-singular matrix ${\displaystyle A}$ and vector ${\displaystyle b}$,

${\displaystyle S(XA+\mathbf {1} _{n}b^{T})=A^{T}\,S(X)\,A.}$

The sample covariance matrix is the canonical example, but many alternatives drawn from the robust statistics literature are available (see below).^[1]

Given two scatter matrices ${\displaystyle S_{1}(X)}$ and ${\displaystyle S_{2}(X)}$, ICS solves the generalized eigenvalue problem

${\displaystyle S_{2}(X)\,\mathbf {v} =\lambda \,S_{1}(X)\,\mathbf {v} ,}$

producing eigenvalues ${\displaystyle \lambda _{1}\geq \dots \geq \lambda _{p}>0}$ and a matrix ${\displaystyle V=(\mathbf {v} _{1},\dots ,\mathbf {v} _{p})}$ of corresponding eigenvectors normalized so that ${\displaystyle V^{T}S_{1}(X)V=I_{p}}$.^[1] The invariant coordinates are the columns of

${\displaystyle Z=(X-\mathbf {1} _{n}{\bar {x}}^{T})\,V.}$

The eigenvalues ${\displaystyle \lambda _{k}}$ quantify how strongly the two scatter matrices disagree along each direction. For any unit vector ${\displaystyle \mathbf {u} }$, the ratio ${\displaystyle \mathbf {u} ^{T}S_{2}(X)\mathbf {u} \,/\,\mathbf {u} ^{T}S_{1}(X)\mathbf {u} }$ can be interpreted as a generalized kurtosis measure along that direction, so the invariant coordinates are ordered by a notion of multivariate kurtosis.^[1][7] Coordinates associated with extreme eigenvalues are those typically of greatest interest for outlier detection or clustering.^[2]

When ${\displaystyle S_{1}}$ is taken to be the identity matrix and ${\displaystyle S_{2}}$ the sample covariance matrix, ICS reduces to ordinary principal component analysis; PCA is therefore a special case in which only second-order structure is exploited.^[1]

The choice of the pair ${\displaystyle (S_{1},S_{2})}$ determines which features ICS will emphasize. Common scatter functionals used in practice include:

• the sample covariance matrix;
• the family of M-estimators of multivariate scatter introduced by Maronna;^[9]
• the minimum covariance determinant (MCD) and minimum volume ellipsoid estimators of Rousseeuw;^[12]
• the distribution-free M-functional of Tyler;^[10]
• the symmetrized M-functional of Dümbgen;^[11]
• the scatter matrix of fourth moments (sometimes denoted COV₄), which corresponds to the second-stage matrix of Cardoso's FOBI;^[6][7]
• spatial sign and rank covariance matrices in their symmetrized, affine-equivariant forms.^[13]

Pairing two functionals with contrasting sensitivities—for instance the covariance matrix with a robust scatter, or with the fourth-moment scatter—produces invariant coordinates that highlight different features of the data, such as outliers, clusters, or non-Gaussian sources.^[3]

A scatter matrix generalizes the covariance matrix as a summary of multivariate spread, and different scatter matrices emphasize different aspects of the data distribution. ICS typically pairs the ordinary covariance matrix with a second scatter matrix that is either robust to outliers or sensitive to higher-order moments such as kurtosis.^[3]

For elliptically symmetric distributions, such as the multivariate normal distribution, any two affine-equivariant scatter matrices are proportional, so all generalized eigenvalues coincide and no direction is preferred.^[1] Departures from this equality therefore signal departures from ellipticity—such as the presence of clusters, skewness, or outliers—and the corresponding invariant coordinates highlight the directions in which these features are most pronounced.

• Affine invariance. If ${\displaystyle X}$ is replaced by ${\displaystyle XA+\mathbf {1} _{n}b^{T}}$ for any non-singular ${\displaystyle A}$ and vector ${\displaystyle b}$, the invariant coordinates ${\displaystyle Z}$ are unchanged up to sign and permutation.^[1]
• Behaviour under ellipticity. Under an elliptical distribution, the two scatter matrices are proportional and all eigenvalues are equal in the population.^[1]
• Connection to ICA. Under the independent component model, ICS recovers the independent components up to sign and order, provided the source kurtoses are pairwise distinct.^[3]
• Reduction to PCA. Taking ${\displaystyle S_{1}=I_{p}}$ and ${\displaystyle S_{2}}$ equal to the sample covariance matrix recovers principal component analysis.^[1]

• Outlier detection: ICS can identify multivariate outliers that are not apparent in low-order projections such as those produced by PCA.^[2]
• Independent component analysis: ICS can serve as an alternative to, or pre-processing step for, ICA.^[3]
• Clustering: ICS is used as a projection step in tandem clustering pipelines, where it is combined with a clustering algorithm in the reduced space.^[14]
• Other reported applications include financial data analysis, bioinformatics, and statistical quality control.^[2]

• Invariant under affine transformations of the data.
• Captures higher-order structure rather than second-order variance alone.
• Robust versions perform well under contamination or heavy-tailed distributions.

• Results depend on the choice of the two scatter matrices.
• Invariant components may be less directly interpretable than principal components.
• Robust scatter matrices can be computationally demanding compared with PCA.

• Principal component analysis
• Independent component analysis
• Linear discriminant analysis
• Sliced inverse regression
• Factor analysis
• Whitening transformation
• Mahalanobis distance
• Robust statistics
• Multivariate statistics
• Projection pursuit
• Blind source separation

• ICS package on CRAN
• ICSOutlier package on CRAN
• ICSClust package on CRAN

Category:Multivariate statistics Category:Matrix decompositions Category:Dimension reduction Category:Robust statistics Category:Exploratory data analysis Category:Cluster analysis Category:Statistical methods