Matrix Action 
Perpframes, Aligners and Hangers 
Matrix Subspaces  
Linear Systems, Pseudo-Inverse 
Condition Number 
Matrix Norm, Rank One 
Data Compression 
Noise Filtering 
Todd Will
UW-La Crosse
 Pseudo-inverse Example 
Reduced SVD 


Linear Systems & 


Inverse and Determinant  


Pseudo-Inverse Example

Suppose the SVD for a matrix [Graphics:systemgr1.gif] is 
How can you use the decomposition to solve the matrix equation [Graphics:systemgr3.gif]

Answer:  The first thing you know is that no matter what x you use, A x is always in the column space of A,  Col[A]. 

So if [Graphics:systemgr4.gif], the equation won't have any solution. 

If [Graphics:systemgr5.gif] you should solve [Graphics:systemgr6.gif] where [Graphics:systemgr7.gif] is the vector in Col[A] closest to y

The solution to [Graphics:systemgr8.gif] is called the least squares solution to A x = y

Here's what you do to solve the system: 

Step 1: Find the vector [Graphics:systemgr9.gif] in Col[A] closest to y. 

Since [Graphics:systemgr10.gif] is an orthonormal basis for Col[A], it's easy to compute [Graphics:systemgr11.gif]
(Note that if [Graphics:systemgr12.gif], then [Graphics:systemgr13.gif].) 

Step 2: Let [Graphics:systemgr14.gif]

That's it! 

If the system has a solution, then you've just found it. 

If the system has no solution, then you've done the best you can -- you've found the least squares solution. 

Here's a check on your solution: 

If you set [Graphics:systemgr14.gif],  then 


[Graphics:systemgr16.gif]             (Linearity) 

[Graphics:systemgr17.gif]                (Fundamental Equations) 

Note that you can compute [Graphics:systemgr14.gif] using the matrix multiplication 


The matrix [Graphics:systemgr20.gif] is called the pseudo-inverse of A. 

The least squares solution to A x=y is [Graphics:systemgr21.gif]

Other least squares solutions have the form [Graphics:systemgr22.gif] where [Graphics:systemgr23.gif]

Since the SVD gives you an orthonormal basis for N[A], you know any least squares solutions to A x=y can be expressed as [Graphics:systemgr24.gif] 

Reduced SVD 

It's easiest to describe the pseudo-inverse in general terms by first defining the reduced SVD for A

If [Graphics:systemgr26.gif] is an SVD of A

then [Graphics:systemgr27.gif] is the reduced SVD for A

You get the reduced SVD from the full SVD by keeping only 

  • the non-zero singular values in the stretcher matrix 

  • the columns of the hanger and rows of the aligner corresponding to non-zero singular values. 
 You can tell that the reduced SVD equals the full SVD (and so still equals A), since the two decomposition agree on the basis [Graphics:systemgr28.gif]

 General Pseudo-Inverse

If you have the reduced SVD for an m x n matrix A
then the pseudo-inverse of A is 

The least squares solution to A x = y is given by [Graphics:systemgr31.gif] 

The Inverse and the Absolute Value of the Determinant

Suppose A is a square matrix and an SVD is 

As it turns out the absolute value of the determinant of A, [Graphics:systemgr35.gif] equals the product of the singular values. 

Thus, if [Graphics:systemgr36.gif], then you know that all of the singular values are positive. 

This tells you that A has rank n and so is invertible. 

If all the singular values are positive, then you can compute 





(1) The inverse of a product is the product of the inverses in reverse order. 

(2) The aligner matrix [Graphics:systemgr41.gif] takes the perpframe [Graphics:systemgr42.gif] to the standard basis. 
The inverse of this procedure, taking the standard basis to the perpframe [Graphics:systemgr43.gif] is accomplished by the hanger matrix [Graphics:systemgr44.gif]

A similar argument explains why [Graphics:systemgr45.gif]

To undo the stretcher matrix [Graphics:systemgr46.gif] stretches the standard basis vectors by [Graphics:systemgr47.gif]
You can reverse this by stretching the standard basis vectors by [Graphics:systemgr48.gif]
This is done with the stretcher matrix [Graphics:systemgr49.gif] which explains why 


(3) The definition of [Graphics:systemgr51.gif]

Comment: You've just shown that if A is invertible, then [Graphics:systemgr52.gif]


1. Suppose [Graphics:systemgr53.gif] is a full SVD for A. 
In terms of the [Graphics:systemgr54.gif] and [Graphics:systemgr55.gif]

a. express the reduced SVD for A as a product of three matrices 

b. express the pseudo-inverse, [Graphics:systemgr56.gif] as a product of three matrices 

c. express a solution to [Graphics:systemgr57.gif] 

d. express all solutions to [Graphics:systemgr58.gif] 

2.  Suppose   is a full SVD for A.

a. What are the possible values for the determinant of A?

b. In terms of  and express  as the product of three matrices.

© 1999 Todd Will
Last Modified: 03-Mar-1999
Hit Counter