Todd Will
UW-La Crosse

Pseudo-Inverse Example

.

Answer:  The first thing you know is that no matter what x you use, A x is always in the column space of A,  Col[A]. 

So if , the equation won't have any solution. 
 

If  you should solve  where  is the vector in Col[A] closest to y. 

The solution to  is called the least squares solution to A x = y. 
 

Here's what you do to solve the system: 

Step 1: Find the vector  in Col[A] closest to y. 

Since  is an orthonormal basis for Col[A], it's easy to compute . 
(Note that if , then .) 

Step 2: Let . 

That's it! 

If the system has a solution, then you've just found it. 

If the system has no solution, then you've done the best you can -- you've found the least squares solution. 

Here's a check on your solution: 

If you set ,  then 

             (Linearity) 

                (Fundamental Equations) 
 

Note that you can compute  using the matrix multiplication 

. 

The matrix  is called the pseudo-inverse of A. 
 

The least squares solution to A x=y is . 
 

Other least squares solutions have the form  where . 
 

Since the SVD gives you an orthonormal basis for N[A], you know any least squares solutions to A x=y can be expressed as  

Reduced SVD 

If  is an SVD of A, 

then  is the reduced SVD for A. 
 

You get the reduced SVD from the full SVD by keeping only 
 

• the non-zero singular values in the stretcher matrix 

 
• the columns of the hanger and rows of the aligner corresponding to non-zero singular values. 

 General Pseudo-Inverse

, 

.

The least squares solution to A x = y is given by  
 

The Inverse and the Absolute Value of the Determinant

.

As it turns out the absolute value of the determinant of A,  equals the product of the singular values. 
 

Thus, if , then you know that all of the singular values are positive. 
 

This tells you that A has rank n and so is invertible. 
 

If all the singular values are positive, then you can compute 
 

. 
 

(1) The inverse of a product is the product of the inverses in reverse order. 

(2) The aligner matrix  takes the perpframe  to the standard basis. 
The inverse of this procedure, taking the standard basis to the perpframe  is accomplished by the hanger matrix . 
 

A similar argument explains why . 
 

To undo the stretcher matrix  stretches the standard basis vectors by . 
You can reverse this by stretching the standard basis vectors by . 
This is done with the stretcher matrix  which explains why 

.

(3) The definition of . 
 

Comment: You've just shown that if A is invertible, then . 
 

Exercises

a. express the reduced SVD for A as a product of three matrices 

b. express the pseudo-inverse,  as a product of three matrices 

c. express a solution to  

d. express all solutions to  

a. What are the possible values for the determinant of A?

b. In terms of  and express  as the product of three matrices.