Page 310 - 35Linear Algebra

310 Least squares and Singular Values

As usual, our starting point is the computation of L acting on the input basis
vectors;
p p

LO = Lu 1 , . . . , Lu n = λ 1 v 1 , . . . , λ n v n
√ 
λ 1 0 · · · 0
√
 0 λ 2 · · · 0 


 . . . . . . . 

. . . . . 
 
√
= v 1 , . . . , v m  0 0 · · · λ n  .


 
 0 0 · · · 0 
 
 . . . . . . 
 . . . 
0 0 · · · 0
√
The result is very close to diagonalization; the numbers λ i along the leading
diagonal are called the singular values of L.
Example 155 Let the matrix of a linear transformation be
1 1
 
2 2
M = −1 1 .


− 1 − 1
2 2
Clearly ker M = {0} while
3 − 1
!
T
M M = 2 2
− 1 3
2 2
which has eigenvalues and eigenvectors
1 ! 1 !
√ √
2 2
λ = 1 , u 1 := ; λ = 2 , u 2 := .
√ 1 − √ 1
2 2
so our orthonormal input basis is
1 ! 1 !!
√ √
2 2
O = , .
√ 1 − √ 1
2 2
These are called the right singular vectors of M. The vectors
 1   
√ 0
2 √


 0  and Mu 2 =  − 2 

Mu 1 = 
− √ 1 0
2
310