You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@commons.apache.org by Patrick Meyer <me...@gmail.com> on 2014/02/15 03:07:44 UTC
[math] trouble with SingularValueDecomposition
Hi,
I am using the SingularValueDecomposition class with a matrix but it gives
me a different result than R. My knowledge of SVD is limited, so any advice
is welcomed.
Here's the method in Java
public void svdTest(){
double[][] x = {
{1.0, -0.053071807862720116, 0.04236086650321309},
{0.05307180786272012, 1.0, 0.0058054424137053435},
{-0.04236086650321309, -0.005805442413705342, 1.0}
};
RealMatrix X = new Array2DRowRealMatrix(x);
SingularValueDecomposition svd = new SingularValueDecomposition(X);
RealMatrix U = svd.getU();
for(int i=0;i<U.getRowDimension();i++){
for(int j=0;j<U.getColumnDimension();j++){
System.out.print(U.getEntry(i,j) + " ");
}
System.out.println();
}
System.out.println();
System.out.println();
RealMatrix V = svd.getV();
for(int i=0;i<V.getRowDimension();i++){
for(int j=0;j<V.getColumnDimension();j++){
System.out.print(V.getEntry(i,j) + " ");
}
System.out.println();
}
}
And here's the function in R.
x<-matrix(c(
1.0, -0.053071807862720116, 0.04236086650321309,
0.05307180786272012, 1.0, 0.0058054424137053435,
-0.04236086650321309, -0.005805442413705342, 1.0),
nrow=3, byrow=TRUE)
svd(x)
Does anyone know why I am getting different results for U and V? I am using
commons math 3.1.
Thanks,
Patrick
Re: [math] trouble with SingularValueDecomposition
Posted by Ted Dunning <te...@gmail.com>.
And what exactly are the results you are getting?
On Fri, Feb 14, 2014 at 6:07 PM, Patrick Meyer <me...@gmail.com> wrote:
> Hi,
>
>
>
> I am using the SingularValueDecomposition class with a matrix but it gives
> me a different result than R. My knowledge of SVD is limited, so any advice
> is welcomed.
>
>
>
> Here's the method in Java
>
>
>
> public void svdTest(){
>
>
>
> double[][] x = {
>
> {1.0, -0.053071807862720116, 0.04236086650321309},
>
> {0.05307180786272012, 1.0, 0.0058054424137053435},
>
> {-0.04236086650321309, -0.005805442413705342, 1.0}
>
> };
>
>
>
> RealMatrix X = new Array2DRowRealMatrix(x);
>
>
>
> SingularValueDecomposition svd = new SingularValueDecomposition(X);
>
>
>
> RealMatrix U = svd.getU();
>
> for(int i=0;i<U.getRowDimension();i++){
>
> for(int j=0;j<U.getColumnDimension();j++){
>
> System.out.print(U.getEntry(i,j) + " ");
>
> }
>
> System.out.println();
>
> }
>
>
>
> System.out.println();
>
> System.out.println();
>
> RealMatrix V = svd.getV();
>
> for(int i=0;i<V.getRowDimension();i++){
>
> for(int j=0;j<V.getColumnDimension();j++){
>
> System.out.print(V.getEntry(i,j) + " ");
>
> }
>
> System.out.println();
>
> }
>
>
>
>
>
> }
>
>
>
>
>
> And here's the function in R.
>
>
>
> x<-matrix(c(
>
> 1.0, -0.053071807862720116, 0.04236086650321309,
>
> 0.05307180786272012, 1.0, 0.0058054424137053435,
>
> -0.04236086650321309, -0.005805442413705342, 1.0),
>
> nrow=3, byrow=TRUE)
>
> svd(x)
>
>
>
> Does anyone know why I am getting different results for U and V? I am using
> commons math 3.1.
>
>
>
> Thanks,
>
> Patrick
>
>
>
>
>
>
>
>
RE: [math] trouble with SingularValueDecomposition
Posted by Patrick Meyer <me...@gmail.com>.
Thanks again Ted.
For what it's worth I have been using the linear algebra features in CM recently and they are really nice. Thanks to everyone who contributed to it!
-----Original Message-----
From: Ted Dunning [mailto:ted.dunning@gmail.com]
Sent: Saturday, February 15, 2014 8:02 PM
To: Commons Developers List
Subject: Re: [math] trouble with SingularValueDecomposition
Note that the only reason that the order is unconstrained is because the two corresponding singular values are equal.
Strictly speaking, for equal singular values, any unitary transformation of the corresponding singular vectors are also valid singular vectors.
On Sat, Feb 15, 2014 at 4:09 AM, Patrick Meyer <me...@gmail.com> wrote:
> Thanks Ted. As I mentioned my knowledge of SVD is limited, and I was
> not aware that it is OK to have a different order of the first two
> columns in the results (or the conditions under which the order
> doesn't matter). I am trying to track down a bug in some code and that’s what led me to the SVD.
> I guess I need to keep looking for the real bug.
>
> For completeness, my results R were the same as you reported. My
> results from CM are shown below and if you swap the first and second
> column, the results agree with R.
>
> U:
> 0.9940594018965339 0.06774763124429131 -0.08518312016997649
> 0.10615872136916754 -0.7761401247896214 0.6215599991704858
> 0.02400481989869077 0.6269104921377042 0.778721390144956
>
> V:
> 0.9963653125425972 0.0 -0.08518312016997495
> 0.0531395658155507 -0.7815621241949481 0.6215599991704865
> 0.06657590034559915 0.6238274168581248 0.7787213901449556
>
>
>
> -----Original Message-----
> From: Ted Dunning [mailto:ted.dunning@gmail.com]
> Sent: Saturday, February 15, 2014 2:17 AM
> To: Commons Developers List
> Subject: Re: [math] trouble with SingularValueDecomposition
>
> For what its worth, I tested the Mahout SVD which shares code lineage
> with the Commons Math implementation.
>
> The results I got were:
>
>
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> > *sum(abs(m - u * s * v')) = 4.31946146e-16S = 1.002319690998
> > 1.002319690998 1.000000000000
>
> U = 0.994059401897 0.067747631244
> > -0.085183120170 0.106158721369 -0.776140124790 0.621559999170
> > 0.024004819899 0.626910492138 0.778721390145 V = 0.996365312543
> > 0.000000000000 -0.085183120170 0.053139565816 -0.781562124195
> > 0.621559999170 0.066575900346 0.623827416858 0.778721390145*
>
>
> Note that the residue of the reconstruction is excellently small.
> This indicates that the result is correct.
>
>
> If you compare these to the R results,
>
>
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> > *[1] 1.0023196909980066 1.0023196909980066 1.0000000000000000$u
> > [,1] [,2] [,3][1,]
> > 0.067747631244291326 -0.994059401896534967 0.085183120169970525
> > [2,]
> > -0.776140124789635122 -0.106158721369163295
> > -0.621559999170469113[3,]
> > 0.626910492137687125 -0.024004819898688426 -0.778721390144969994$v
> > [,1] [,2] [,3] [1,]
> > 0.00000000000000000 -0.996365312542597747 0.085183120169970497[2,]
> > -0.78156212419496163 -0.053139565815546450 -0.621559999170469668[3,]
> > 0.62382741685810772 -0.066575900345596822 -0.778721390144969550*
>
>
> These are identical to the previous results except that the first two
> singular values are equal which means that the order of the
> corresponding left and right singular vectors are different and there
> are sign changes in the singular vectors.
>
> My guess is that you will get the same results in Apache Commons Math.
>
>
>
> On Fri, Feb 14, 2014 at 6:07 PM, Patrick Meyer <me...@gmail.com> wrote:
>
> > Hi,
> >
> >
> >
> > I am using the SingularValueDecomposition class with a matrix but it
> > gives me a different result than R. My knowledge of SVD is limited,
> > so any advice is welcomed.
> >
> >
> >
> > Here's the method in Java
> >
> >
> >
> > public void svdTest(){
> >
> >
> >
> > double[][] x = {
> >
> > {1.0, -0.053071807862720116, 0.04236086650321309},
> >
> > {0.05307180786272012, 1.0, 0.0058054424137053435},
> >
> > {-0.04236086650321309, -0.005805442413705342, 1.0}
> >
> > };
> >
> >
> >
> > RealMatrix X = new Array2DRowRealMatrix(x);
> >
> >
> >
> > SingularValueDecomposition svd = new
> > SingularValueDecomposition(X);
> >
> >
> >
> > RealMatrix U = svd.getU();
> >
> > for(int i=0;i<U.getRowDimension();i++){
> >
> > for(int j=0;j<U.getColumnDimension();j++){
> >
> > System.out.print(U.getEntry(i,j) + " ");
> >
> > }
> >
> > System.out.println();
> >
> > }
> >
> >
> >
> > System.out.println();
> >
> > System.out.println();
> >
> > RealMatrix V = svd.getV();
> >
> > for(int i=0;i<V.getRowDimension();i++){
> >
> > for(int j=0;j<V.getColumnDimension();j++){
> >
> > System.out.print(V.getEntry(i,j) + " ");
> >
> > }
> >
> > System.out.println();
> >
> > }
> >
> >
> >
> >
> >
> > }
> >
> >
> >
> >
> >
> > And here's the function in R.
> >
> >
> >
> > x<-matrix(c(
> >
> > 1.0, -0.053071807862720116, 0.04236086650321309,
> >
> > 0.05307180786272012, 1.0, 0.0058054424137053435,
> >
> > -0.04236086650321309, -0.005805442413705342, 1.0),
> >
> > nrow=3, byrow=TRUE)
> >
> > svd(x)
> >
> >
> >
> > Does anyone know why I am getting different results for U and V? I
> > am using commons math 3.1.
> >
> >
> >
> > Thanks,
> >
> > Patrick
> >
> >
> >
> >
> >
> >
> >
> >
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> For additional commands, e-mail: dev-help@commons.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [math] trouble with SingularValueDecomposition
Posted by Ted Dunning <te...@gmail.com>.
Note that the only reason that the order is unconstrained is because the
two corresponding singular values are equal.
Strictly speaking, for equal singular values, any unitary transformation of
the corresponding singular vectors are also valid singular vectors.
On Sat, Feb 15, 2014 at 4:09 AM, Patrick Meyer <me...@gmail.com> wrote:
> Thanks Ted. As I mentioned my knowledge of SVD is limited, and I was not
> aware that it is OK to have a different order of the first two columns in
> the results (or the conditions under which the order doesn't matter). I am
> trying to track down a bug in some code and that’s what led me to the SVD.
> I guess I need to keep looking for the real bug.
>
> For completeness, my results R were the same as you reported. My results
> from CM are shown below and if you swap the first and second column, the
> results agree with R.
>
> U:
> 0.9940594018965339 0.06774763124429131 -0.08518312016997649
> 0.10615872136916754 -0.7761401247896214 0.6215599991704858
> 0.02400481989869077 0.6269104921377042 0.778721390144956
>
> V:
> 0.9963653125425972 0.0 -0.08518312016997495
> 0.0531395658155507 -0.7815621241949481 0.6215599991704865
> 0.06657590034559915 0.6238274168581248 0.7787213901449556
>
>
>
> -----Original Message-----
> From: Ted Dunning [mailto:ted.dunning@gmail.com]
> Sent: Saturday, February 15, 2014 2:17 AM
> To: Commons Developers List
> Subject: Re: [math] trouble with SingularValueDecomposition
>
> For what its worth, I tested the Mahout SVD which shares code lineage with
> the Commons Math implementation.
>
> The results I got were:
>
>
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> > *sum(abs(m - u * s * v')) = 4.31946146e-16S = 1.002319690998
> > 1.002319690998 1.000000000000
>
> U = 0.994059401897 0.067747631244
> > -0.085183120170 0.106158721369 -0.776140124790 0.621559999170
> > 0.024004819899 0.626910492138 0.778721390145 V = 0.996365312543
> > 0.000000000000 -0.085183120170 0.053139565816 -0.781562124195
> > 0.621559999170 0.066575900346 0.623827416858 0.778721390145*
>
>
> Note that the residue of the reconstruction is excellently small. This
> indicates that the result is correct.
>
>
> If you compare these to the R results,
>
>
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> > *[1] 1.0023196909980066 1.0023196909980066 1.0000000000000000$u
> > [,1] [,2] [,3][1,]
> > 0.067747631244291326 -0.994059401896534967 0.085183120169970525 [2,]
> > -0.776140124789635122 -0.106158721369163295 -0.621559999170469113[3,]
> > 0.626910492137687125 -0.024004819898688426 -0.778721390144969994$v
> > [,1] [,2] [,3] [1,]
> > 0.00000000000000000 -0.996365312542597747 0.085183120169970497[2,]
> > -0.78156212419496163 -0.053139565815546450 -0.621559999170469668[3,]
> > 0.62382741685810772 -0.066575900345596822 -0.778721390144969550*
>
>
> These are identical to the previous results except that the first two
> singular values are equal which means that the order of the corresponding
> left and right singular vectors are different and there are sign changes in
> the singular vectors.
>
> My guess is that you will get the same results in Apache Commons Math.
>
>
>
> On Fri, Feb 14, 2014 at 6:07 PM, Patrick Meyer <me...@gmail.com> wrote:
>
> > Hi,
> >
> >
> >
> > I am using the SingularValueDecomposition class with a matrix but it
> > gives me a different result than R. My knowledge of SVD is limited, so
> > any advice is welcomed.
> >
> >
> >
> > Here's the method in Java
> >
> >
> >
> > public void svdTest(){
> >
> >
> >
> > double[][] x = {
> >
> > {1.0, -0.053071807862720116, 0.04236086650321309},
> >
> > {0.05307180786272012, 1.0, 0.0058054424137053435},
> >
> > {-0.04236086650321309, -0.005805442413705342, 1.0}
> >
> > };
> >
> >
> >
> > RealMatrix X = new Array2DRowRealMatrix(x);
> >
> >
> >
> > SingularValueDecomposition svd = new
> > SingularValueDecomposition(X);
> >
> >
> >
> > RealMatrix U = svd.getU();
> >
> > for(int i=0;i<U.getRowDimension();i++){
> >
> > for(int j=0;j<U.getColumnDimension();j++){
> >
> > System.out.print(U.getEntry(i,j) + " ");
> >
> > }
> >
> > System.out.println();
> >
> > }
> >
> >
> >
> > System.out.println();
> >
> > System.out.println();
> >
> > RealMatrix V = svd.getV();
> >
> > for(int i=0;i<V.getRowDimension();i++){
> >
> > for(int j=0;j<V.getColumnDimension();j++){
> >
> > System.out.print(V.getEntry(i,j) + " ");
> >
> > }
> >
> > System.out.println();
> >
> > }
> >
> >
> >
> >
> >
> > }
> >
> >
> >
> >
> >
> > And here's the function in R.
> >
> >
> >
> > x<-matrix(c(
> >
> > 1.0, -0.053071807862720116, 0.04236086650321309,
> >
> > 0.05307180786272012, 1.0, 0.0058054424137053435,
> >
> > -0.04236086650321309, -0.005805442413705342, 1.0),
> >
> > nrow=3, byrow=TRUE)
> >
> > svd(x)
> >
> >
> >
> > Does anyone know why I am getting different results for U and V? I am
> > using commons math 3.1.
> >
> >
> >
> > Thanks,
> >
> > Patrick
> >
> >
> >
> >
> >
> >
> >
> >
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> For additional commands, e-mail: dev-help@commons.apache.org
>
>
RE: [math] trouble with SingularValueDecomposition
Posted by Patrick Meyer <me...@gmail.com>.
Thanks Ted. As I mentioned my knowledge of SVD is limited, and I was not aware that it is OK to have a different order of the first two columns in the results (or the conditions under which the order doesn't matter). I am trying to track down a bug in some code and that’s what led me to the SVD. I guess I need to keep looking for the real bug.
For completeness, my results R were the same as you reported. My results from CM are shown below and if you swap the first and second column, the results agree with R.
U:
0.9940594018965339 0.06774763124429131 -0.08518312016997649
0.10615872136916754 -0.7761401247896214 0.6215599991704858
0.02400481989869077 0.6269104921377042 0.778721390144956
V:
0.9963653125425972 0.0 -0.08518312016997495
0.0531395658155507 -0.7815621241949481 0.6215599991704865
0.06657590034559915 0.6238274168581248 0.7787213901449556
-----Original Message-----
From: Ted Dunning [mailto:ted.dunning@gmail.com]
Sent: Saturday, February 15, 2014 2:17 AM
To: Commons Developers List
Subject: Re: [math] trouble with SingularValueDecomposition
For what its worth, I tested the Mahout SVD which shares code lineage with the Commons Math implementation.
The results I got were:
>
>
>
>
>
>
>
>
>
>
>
> *sum(abs(m - u * s * v')) = 4.31946146e-16S = 1.002319690998
> 1.002319690998 1.000000000000
U = 0.994059401897 0.067747631244
> -0.085183120170 0.106158721369 -0.776140124790 0.621559999170
> 0.024004819899 0.626910492138 0.778721390145 V = 0.996365312543
> 0.000000000000 -0.085183120170 0.053139565816 -0.781562124195
> 0.621559999170 0.066575900346 0.623827416858 0.778721390145*
Note that the residue of the reconstruction is excellently small. This indicates that the result is correct.
If you compare these to the R results,
>
>
>
>
>
>
>
>
>
>
> *[1] 1.0023196909980066 1.0023196909980066 1.0000000000000000$u
> [,1] [,2] [,3][1,]
> 0.067747631244291326 -0.994059401896534967 0.085183120169970525 [2,]
> -0.776140124789635122 -0.106158721369163295 -0.621559999170469113[3,]
> 0.626910492137687125 -0.024004819898688426 -0.778721390144969994$v
> [,1] [,2] [,3] [1,]
> 0.00000000000000000 -0.996365312542597747 0.085183120169970497[2,]
> -0.78156212419496163 -0.053139565815546450 -0.621559999170469668[3,]
> 0.62382741685810772 -0.066575900345596822 -0.778721390144969550*
These are identical to the previous results except that the first two singular values are equal which means that the order of the corresponding left and right singular vectors are different and there are sign changes in the singular vectors.
My guess is that you will get the same results in Apache Commons Math.
On Fri, Feb 14, 2014 at 6:07 PM, Patrick Meyer <me...@gmail.com> wrote:
> Hi,
>
>
>
> I am using the SingularValueDecomposition class with a matrix but it
> gives me a different result than R. My knowledge of SVD is limited, so
> any advice is welcomed.
>
>
>
> Here's the method in Java
>
>
>
> public void svdTest(){
>
>
>
> double[][] x = {
>
> {1.0, -0.053071807862720116, 0.04236086650321309},
>
> {0.05307180786272012, 1.0, 0.0058054424137053435},
>
> {-0.04236086650321309, -0.005805442413705342, 1.0}
>
> };
>
>
>
> RealMatrix X = new Array2DRowRealMatrix(x);
>
>
>
> SingularValueDecomposition svd = new
> SingularValueDecomposition(X);
>
>
>
> RealMatrix U = svd.getU();
>
> for(int i=0;i<U.getRowDimension();i++){
>
> for(int j=0;j<U.getColumnDimension();j++){
>
> System.out.print(U.getEntry(i,j) + " ");
>
> }
>
> System.out.println();
>
> }
>
>
>
> System.out.println();
>
> System.out.println();
>
> RealMatrix V = svd.getV();
>
> for(int i=0;i<V.getRowDimension();i++){
>
> for(int j=0;j<V.getColumnDimension();j++){
>
> System.out.print(V.getEntry(i,j) + " ");
>
> }
>
> System.out.println();
>
> }
>
>
>
>
>
> }
>
>
>
>
>
> And here's the function in R.
>
>
>
> x<-matrix(c(
>
> 1.0, -0.053071807862720116, 0.04236086650321309,
>
> 0.05307180786272012, 1.0, 0.0058054424137053435,
>
> -0.04236086650321309, -0.005805442413705342, 1.0),
>
> nrow=3, byrow=TRUE)
>
> svd(x)
>
>
>
> Does anyone know why I am getting different results for U and V? I am
> using commons math 3.1.
>
>
>
> Thanks,
>
> Patrick
>
>
>
>
>
>
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [math] trouble with SingularValueDecomposition
Posted by Ted Dunning <te...@gmail.com>.
For what its worth, I tested the Mahout SVD which shares code lineage with
the Commons Math implementation.
The results I got were:
>
>
>
>
>
>
>
>
>
>
>
> *sum(abs(m - u * s * v')) = 4.31946146e-16S = 1.002319690998
> 1.002319690998 1.000000000000 U = 0.994059401897 0.067747631244
> -0.085183120170 0.106158721369 -0.776140124790 0.621559999170
> 0.024004819899 0.626910492138 0.778721390145 V = 0.996365312543
> 0.000000000000 -0.085183120170 0.053139565816 -0.781562124195
> 0.621559999170 0.066575900346 0.623827416858 0.778721390145*
Note that the residue of the reconstruction is excellently small. This
indicates that the result is correct.
If you compare these to the R results,
>
>
>
>
>
>
>
>
>
>
> *[1] 1.0023196909980066 1.0023196909980066 1.0000000000000000$u
> [,1] [,2] [,3][1,]
> 0.067747631244291326 -0.994059401896534967 0.085183120169970525 [2,]
> -0.776140124789635122 -0.106158721369163295 -0.621559999170469113[3,]
> 0.626910492137687125 -0.024004819898688426 -0.778721390144969994$v
> [,1] [,2] [,3] [1,]
> 0.00000000000000000 -0.996365312542597747 0.085183120169970497[2,]
> -0.78156212419496163 -0.053139565815546450 -0.621559999170469668[3,]
> 0.62382741685810772 -0.066575900345596822 -0.778721390144969550*
These are identical to the previous results except that the first two
singular values are equal which means that the order of the corresponding
left and right singular vectors are different and there are sign changes in
the singular vectors.
My guess is that you will get the same results in Apache Commons Math.
On Fri, Feb 14, 2014 at 6:07 PM, Patrick Meyer <me...@gmail.com> wrote:
> Hi,
>
>
>
> I am using the SingularValueDecomposition class with a matrix but it gives
> me a different result than R. My knowledge of SVD is limited, so any advice
> is welcomed.
>
>
>
> Here's the method in Java
>
>
>
> public void svdTest(){
>
>
>
> double[][] x = {
>
> {1.0, -0.053071807862720116, 0.04236086650321309},
>
> {0.05307180786272012, 1.0, 0.0058054424137053435},
>
> {-0.04236086650321309, -0.005805442413705342, 1.0}
>
> };
>
>
>
> RealMatrix X = new Array2DRowRealMatrix(x);
>
>
>
> SingularValueDecomposition svd = new SingularValueDecomposition(X);
>
>
>
> RealMatrix U = svd.getU();
>
> for(int i=0;i<U.getRowDimension();i++){
>
> for(int j=0;j<U.getColumnDimension();j++){
>
> System.out.print(U.getEntry(i,j) + " ");
>
> }
>
> System.out.println();
>
> }
>
>
>
> System.out.println();
>
> System.out.println();
>
> RealMatrix V = svd.getV();
>
> for(int i=0;i<V.getRowDimension();i++){
>
> for(int j=0;j<V.getColumnDimension();j++){
>
> System.out.print(V.getEntry(i,j) + " ");
>
> }
>
> System.out.println();
>
> }
>
>
>
>
>
> }
>
>
>
>
>
> And here's the function in R.
>
>
>
> x<-matrix(c(
>
> 1.0, -0.053071807862720116, 0.04236086650321309,
>
> 0.05307180786272012, 1.0, 0.0058054424137053435,
>
> -0.04236086650321309, -0.005805442413705342, 1.0),
>
> nrow=3, byrow=TRUE)
>
> svd(x)
>
>
>
> Does anyone know why I am getting different results for U and V? I am using
> commons math 3.1.
>
>
>
> Thanks,
>
> Patrick
>
>
>
>
>
>
>
>