The Generalized Singular Value Decomposition

The Generalized Singular Value Decomposition or GSVD of a pair of matrices A and B rewrites each matrix as the product of an orthogonal (or unitary) matrix, a diagonal matrix, and the product of the same triangular and orthogonal matrix. The two matrices must have the same number of rows. The Generalized SVD decomposition is usually written as

A = UΣ1[0 R]QT
B = VΣ2[0 R]QT,

where U, V, and Q are square, orthogonal matrices, Σ1 and Σ2 are diagonal matrices with the same shape as A, and R is an upper triangular matrix. Alternatively, the matrices Q and R may be combined to give:

A = UΣ1X-1
B = VΣ2X-1,

When A and B are complex, the transpose should be replaced with the conjugate transpose. The singular values are still real.

The quotients of the diagonal elements of Σ1 and Σ2 are called the generalized singular values.

Working with Singular Value Decompositions

The singular value decomposition of a matrix is represented by the by the GeneralizedSingularValueDecomposition<T> class. It has no constructors. Instead, it is created by calling the GetSingularValueDecomposition method on the primary matrix. This method has four overloads. The first overload takes one argument: the secondary matrix B. The second overload takes an additional Boolean value that specifies whether the contents of the matrices may be overwritten during the SVD calculation. The default is false.

C#
var A = Matrix.CreateFromArray(5, 3, new double[] {
    1,2,3,4,5,
    6,7,8,9,10,
    11,12,13,14,15
}, MatrixElementOrder.ColumnMajor);
var B = Matrix.CreateFromArray(3, 3, new double[] {
    8, 3, 4,
    1, 5, 9,
    6, 7, 2
}, MatrixElementOrder.ColumnMajor);
var gsvd = A.GetSingularValueDecomposition(B);

The two remaining overloads let you specify which calculations are to be performed. Because the calculation of the singular vectors U and V as well as the orthogonal factor Q is relatively expensive and not always needed, it is possible to specify that only the singular values should be computed. This is done in one of two ways. The first is by passing a value of type GeneralizedSingularValueDecompositionFactors to the method. Alternatively, you can set the RequestedFactors property to a value of the same type before the decomposition is computed. The possible values for the SingularValueDecompositionFactors enumeration are listed below:

Possible values for the SingularValueDecompositionFactors enumeration

Value

Description

SingularValues

The singular values are required. This is always included.

PrimarySingularVectors

The left singular vectors of A are required.

PrimarySingularVectors

The left singular vectors of A are required.

SecondarySingularVectors

The singular vectors of B are required.

SecondarySingularVectors

The singular vectors of B are required.

SharedFactor

The shared factors Q and R are required.

Thin

The singular values as well as the left and right singular vectors are required, but only the columns that contribute to the products are returned.

All

The singular values as well as the left and right singular vectors are required.

A common option is Thin, which is often used for matrices that have more rows than columns. In this case, only a few of the left singular vectors correspond to singular values. The remaining singular vectors correspond to zero rows in the matrix of singular values, and are rarely used.

The Decompose method performs the actual decomposition. This method copies the matrix if necessary. It then calls the appropriate routine to perform the actual decomposition. This method is called by other methods as needed. You will rarely need to call it explicitly.

The GeneralizedSingularValues property returns a vector containing the generalized singular values. The PrimarySingularValues property returns a vector containing the singular values of the primary matrix. The PrimarySingularValueMatrix returns the matrix Σ1 in the decomposition: a diagonal matrix of the same shape as A with the singular values on the diagonal.

The SecondarySingularValues property returns a vector containing the singular values of the secondary matrix. The SecondarySingularValueMatrix returns the matrix Σ2 in the decomposition: a diagonal matrix of the same shape as B with the singular values on the diagonal.

The PrimarySingularVectors and SecondarySingularVectors return the matrices U and V, respectively. The singular vectors are stored in the columns of these matrices. When a thin SVD is requested, the meaning is slightly different. For a matrix with more rows than columns, the singular matrix contains only n columns, where n is the number of columns in the matrix. The columns of these reduced matrices are still orthogonal.

The SharedTriangularFactor and SharedOrthogonalFactor return the matrices [0 R] and Q, respectively. To get just the triangular part, R or just those columns of Q that are multiplied with non-zero columns of [0 R], use the TrimmedSharedTriangularFactor. To get the matrix X from the decomposition, use SharedFactor.

C#
var S = gsvd.GeneralizedSingularValues;
var S1 = gsvd.PrimarySingularValues;
var S2 = gsvd.SecondarySingularValues;
var U = gsvd.PrimarySingularVectors;
var V = gsvd.SecondarySingularVectors;
var Q = gsvd.SharedOrthogonalFactor;
var R = gsvd.TrimmedSharedTriangularFactor;
var X = gsvd.SharedFactor;

The implementation of the singular value decomposition is based on the LAPACK routine DGGSVD3.