Page 310 - 35Linear Algebra
P. 310

310                                                          Least squares and Singular Values


                            As usual, our starting point is the computation of L acting on the input basis
                            vectors;
                                                              p           p

                                 LO = Lu 1 , . . . , Lu n  =    λ 1 v 1 , . . . ,  λ n v n
                                                                         √                      
                                                                             λ 1   0    · · ·  0
                                                                                 √
                                                                          0       λ 2 · · ·  0 
                                                                         
                                                                                                 
                                                                          . .     . .  . .   .  
                                                                         
                                                                             .     .      .   . . 
                                                                                                
                                                                                            √
                                                         =    v 1 , . . . , v m  0  0  · · ·  λ n  .
                                                                         
                                                                                                 
                                                                                                
                                                                            0     0    · · ·  0  
                                                                                                
                                                                          . .     . .        . . 
                                                                          .       .          . 
                                                                             0     0    · · ·  0
                                                                                    √
                            The result is very close to diagonalization; the numbers  λ i along the leading
                            diagonal are called the singular values of L.
                            Example 155 Let the matrix of a linear transformation be
                                                                  1     1
                                                                        
                                                                  2     2
                                                         M = −1       1 .
                                                                         
                                                               
                                                                 − 1  −  1
                                                                  2     2
                            Clearly ker M = {0} while
                                                                     3  − 1
                                                                           !
                                                           T
                                                        M M =        2    2
                                                                   − 1    3
                                                                     2    2
                            which has eigenvalues and eigenvectors
                                                           1  !                      1  !
                                                          √                         √
                                                           2                          2
                                           λ = 1 , u 1 :=     ;     λ = 2 , u 2 :=       .
                                                          √ 1                     − √ 1
                                                           2                          2
                            so our orthonormal input basis is
                                                               1  !     1  !!
                                                               √        √
                                                                2        2
                                                      O =          ,          .
                                                               √ 1    − √ 1
                                                                2        2
                            These are called the right singular vectors of M. The vectors
                                                           1                     
                                                           √                    0
                                                            2                   √
                                                             
                                                                                    
                                                          0  and Mu 2 =  − 2 
                                                                            
                                                Mu 1 = 
                                                         − √ 1                  0
                                                            2
                                                      310
   305   306   307   308   309   310   311   312   313   314   315