You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@mahout.apache.org by Tharindu Rusira <th...@gmail.com> on 2013/12/29 08:56:21 UTC
Equality of two DenseMatrix objects
Hi all,
what are the necessary conditions for two matrices (DenseMatrix) to be
equal and how to check equality?
I tried the following code with *boolean java.lang.Object.equals(Object
obj)* method but it always returns *false*.
DenseMatrix m1 = new DenseMatrix(3, 3);
m1.assign(new double[][]{{1,2,3},{4,5,6},{7,8,9}});
DenseMatrix m2 = new DenseMatrix(3, 3);
m2.assign(new double[][]{{1,2,3},{4,5,6},{7,8,9}});
if(m1.equals(m2)){
System.out.println("m1 = m2");
}else{
System.out.println("m1 != m2");
}
However, *m1.asFormatString().equals(m2.asFormatString())*
returns true. Is there a more rigorous
way of checking this equality (of values and dimensions)
without converting the matrices to Strings?
Thanks,
--
M.P. Tharindu Rusira Kumara
Department of Computer Science and Engineering,
University of Moratuwa,
Sri Lanka.
+94757033733
www.tharindu-rusira.blogspot.com
Re: Equality of two DenseMatrix objects
Posted by "Scott C. Cote" <sc...@gmail.com>.
Ted - thank you for taking the time to point out that in Multivariate
Systems, there are many interpretations to what would seem ordinary and
non-debatable in scalar mathematics.
For example, in the relational algebra world, I know of seven different
interpretations of relational division.
SCott
On 12/29/13 10:02 PM, "Ted Dunning" <te...@gmail.com> wrote:
>On Sun, Dec 29, 2013 at 7:30 PM, Tharindu Rusira
><th...@gmail.com>wrote:
>
>> Hi Ted, Thanks for taking this discussion back alive. It's true, as
>> Sebestian mentioned, equality checking for matrices is an expensive task
>> and Ted has come up with a smart one liner here(even though a
>>considerable
>> amount of computational complexity is hidden somewhere).
>> But don't you think (at least for the sake of completeness) that we
>>should
>> have an implementation of this?
>>
>
>Not really. The problem is that there are many different meanings of
>"equal" for matrices. In fact there are many definitions of zero, as well.
> This stems partly from the fact that we have to inherit a sense of nearly
>zero or nearly equal from the fact that we are using floating point
>arithmetic. This is exactly why equals is poorly defined for floating
>point numbers, but worse.
>
>As such any single definition is going to be seriously problematic. Any
>definition that doesn't have a tolerance argument is inherently dangerous
>to use except in very limited situations.
>
>For example here are some possibilities for vector equality:
>
> | x - y|_F < \delta
> | x - y|_1 < \delta
> | x - y|_0 < \delta
> (x-y)^T A (x-y) < \delta
> x^T A y > 1-\delta/2
>
>The first says that the sum of the squares of the components of the
>difference is less than a particular number. The second says that the sum
>of the absolute values of the difference is less. The third says that the
>maximum value of the difference is different. The third says that the dot
>product of the of the difference is nearly zero neglecting components in
>the null space of A. The last form is useful for cases where x and y have
>unit norm with respect to A (i.e. x^T A x = 1).
>
>Which of these is correct? Of all of these, only the last two are
>equivalent and only in limited situations.
>
>For matrices, there are even more possibilities.
>
>
>
>> Btw, this thread has turned into a developers discussion, so I'm not
>>sure
>> whether we should continue this on the developers list.
>>
>
>I think that this is a very important thread for users at large as well.
Re: Equality of two DenseMatrix objects
Posted by Ted Dunning <te...@gmail.com>.
On Sun, Dec 29, 2013 at 7:30 PM, Tharindu Rusira
<th...@gmail.com>wrote:
> Hi Ted, Thanks for taking this discussion back alive. It's true, as
> Sebestian mentioned, equality checking for matrices is an expensive task
> and Ted has come up with a smart one liner here(even though a considerable
> amount of computational complexity is hidden somewhere).
> But don't you think (at least for the sake of completeness) that we should
> have an implementation of this?
>
Not really. The problem is that there are many different meanings of
"equal" for matrices. In fact there are many definitions of zero, as well.
This stems partly from the fact that we have to inherit a sense of nearly
zero or nearly equal from the fact that we are using floating point
arithmetic. This is exactly why equals is poorly defined for floating
point numbers, but worse.
As such any single definition is going to be seriously problematic. Any
definition that doesn't have a tolerance argument is inherently dangerous
to use except in very limited situations.
For example here are some possibilities for vector equality:
| x - y|_F < \delta
| x - y|_1 < \delta
| x - y|_0 < \delta
(x-y)^T A (x-y) < \delta
x^T A y > 1-\delta/2
The first says that the sum of the squares of the components of the
difference is less than a particular number. The second says that the sum
of the absolute values of the difference is less. The third says that the
maximum value of the difference is different. The third says that the dot
product of the of the difference is nearly zero neglecting components in
the null space of A. The last form is useful for cases where x and y have
unit norm with respect to A (i.e. x^T A x = 1).
Which of these is correct? Of all of these, only the last two are
equivalent and only in limited situations.
For matrices, there are even more possibilities.
> Btw, this thread has turned into a developers discussion, so I'm not sure
> whether we should continue this on the developers list.
>
I think that this is a very important thread for users at large as well.
Re: Equality of two DenseMatrix objects
Posted by Tharindu Rusira <th...@gmail.com>.
Hi Ted, Thanks for taking this discussion back alive. It's true, as
Sebestian mentioned, equality checking for matrices is an expensive task
and Ted has come up with a smart one liner here(even though a considerable
amount of computational complexity is hidden somewhere).
But don't you think (at least for the sake of completeness) that we should
have an implementation of this?
Btw, this thread has turned into a developers discussion, so I'm not sure
whether we should continue this on the developers list.
Best Regards,
On Mon, Dec 30, 2013 at 4:05 AM, Ted Dunning <te...@gmail.com> wrote:
> On Sun, Dec 29, 2013 at 9:17 AM, Tharindu Rusira
> <th...@gmail.com>wrote:
>
> > Thanks Chameera and Sebestian for sharing your expertise :)
> > Just wanted to know the reason behind the absence of an equality check
> for
> > Matrices.
> >
>
>
> For what it is worth, here is the one liner that does this comparison:
>
> a.minus(b).aggregate(Functions.MAX, Functions.ABS)
>
> This gives you a value which is the magnitude of the largest absolute
> difference between elements of matrix a and matrix b.
>
> For vectors, you can economize a little on typing with this:
>
> a.minus(b).norm(1)
>
> This is the sum of the absolute values of the differences.
>
--
M.P. Tharindu Rusira Kumara
Department of Computer Science and Engineering,
University of Moratuwa,
Sri Lanka.
+94757033733
www.tharindu-rusira.blogspot.com
Re: Equality of two DenseMatrix objects
Posted by Ted Dunning <te...@gmail.com>.
On Sun, Dec 29, 2013 at 9:17 AM, Tharindu Rusira
<th...@gmail.com>wrote:
> Thanks Chameera and Sebestian for sharing your expertise :)
> Just wanted to know the reason behind the absence of an equality check for
> Matrices.
>
For what it is worth, here is the one liner that does this comparison:
a.minus(b).aggregate(Functions.MAX, Functions.ABS)
This gives you a value which is the magnitude of the largest absolute
difference between elements of matrix a and matrix b.
For vectors, you can economize a little on typing with this:
a.minus(b).norm(1)
This is the sum of the absolute values of the differences.
Re: Equality of two DenseMatrix objects
Posted by Tharindu Rusira <th...@gmail.com>.
Thanks Chameera and Sebestian for sharing your expertise :)
Just wanted to know the reason behind the absence of an equality check for
Matrices.
Regards,
On Sun, Dec 29, 2013 at 10:25 PM, Sebastian Schelter <
ssc.open@googlemail.com> wrote:
> Ah, I see, you just pointed to the issue, sorry misread your mail.
>
> The problem is that checking matrices and vectors for equality is a very
> costly operation (you have to compare at least all non-zero entries).
> For that reason, we did not implement equals correctly (which is in
> general a very bad thing).
>
> We advise users not use vectors and matrices in places where equality
> checks on them are necessary (e.g. in a HashSet or as key in a HashMap)
>
> A way to manually check the equivalence of two dense matrices is to
> compare all entries.
>
> --sebastian
>
>
> On 29.12.2013 17:49, Chameera Wijebandara wrote:
> > Sebastian,
> > Yes It's not enought to check the reference. i mean
> DenseMatrix*.*equals()
> > methord does that and it cannot use for check equity is that wrong?
> >
> >
> >
> > On Sun, Dec 29, 2013 at 8:40 AM, Sebastian Schelter <ss...@apache.org>
> wrote:
> >
> >> Chameera,
> >>
> >> This is wrong. It's not enought to check the reference, two different
> >> instances can be semantically equivalent,
> >>
> >> A dense matrix represents its entries in a two dimensional double array.
> >> You have to check for the equality of these arrays.
> >>
> >> --sebastian
> >>
> >> On 29.12.2013 17:14, Chameera Wijebandara wrote:
> >>> Hi Tharindu
> >>>
> >>> In the DenseMatrix class does not implement equals method so it use
> >>> Object.equals witch check the reference.
> >>> You have implement equals method by overriding Object.equals method.
> >>>
> >>> Thanks
> >>> Chameera
> >>>
> >>
> >>
> >
> >
>
>
--
M.P. Tharindu Rusira Kumara
Department of Computer Science and Engineering,
University of Moratuwa,
Sri Lanka.
+94757033733
www.tharindu-rusira.blogspot.com
Re: Equality of two DenseMatrix objects
Posted by Chameera Wijebandara <ch...@gmail.com>.
got it thanks
Chameera
On Sun, Dec 29, 2013 at 8:55 AM, Sebastian Schelter <ssc.open@googlemail.com
> wrote:
> Ah, I see, you just pointed to the issue, sorry misread your mail.
>
> The problem is that checking matrices and vectors for equality is a very
> costly operation (you have to compare at least all non-zero entries).
> For that reason, we did not implement equals correctly (which is in
> general a very bad thing).
>
> We advise users not use vectors and matrices in places where equality
> checks on them are necessary (e.g. in a HashSet or as key in a HashMap)
>
> A way to manually check the equivalence of two dense matrices is to
> compare all entries.
>
> --sebastian
>
>
> On 29.12.2013 17:49, Chameera Wijebandara wrote:
> > Sebastian,
> > Yes It's not enought to check the reference. i mean
> DenseMatrix*.*equals()
> > methord does that and it cannot use for check equity is that wrong?
> >
> >
> >
> > On Sun, Dec 29, 2013 at 8:40 AM, Sebastian Schelter <ss...@apache.org>
> wrote:
> >
> >> Chameera,
> >>
> >> This is wrong. It's not enought to check the reference, two different
> >> instances can be semantically equivalent,
> >>
> >> A dense matrix represents its entries in a two dimensional double array.
> >> You have to check for the equality of these arrays.
> >>
> >> --sebastian
> >>
> >> On 29.12.2013 17:14, Chameera Wijebandara wrote:
> >>> Hi Tharindu
> >>>
> >>> In the DenseMatrix class does not implement equals method so it use
> >>> Object.equals witch check the reference.
> >>> You have implement equals method by overriding Object.equals method.
> >>>
> >>> Thanks
> >>> Chameera
> >>>
> >>
> >>
> >
> >
>
>
--
----------------------------------------------------------
Chameera Wijebandara,
Undergraduate Department of Computer Science and Engineering,
University of Moratuwa,
Sri Lanka.
Re: Equality of two DenseMatrix objects
Posted by Sebastian Schelter <ss...@googlemail.com>.
Ah, I see, you just pointed to the issue, sorry misread your mail.
The problem is that checking matrices and vectors for equality is a very
costly operation (you have to compare at least all non-zero entries).
For that reason, we did not implement equals correctly (which is in
general a very bad thing).
We advise users not use vectors and matrices in places where equality
checks on them are necessary (e.g. in a HashSet or as key in a HashMap)
A way to manually check the equivalence of two dense matrices is to
compare all entries.
--sebastian
On 29.12.2013 17:49, Chameera Wijebandara wrote:
> Sebastian,
> Yes It's not enought to check the reference. i mean DenseMatrix*.*equals()
> methord does that and it cannot use for check equity is that wrong?
>
>
>
> On Sun, Dec 29, 2013 at 8:40 AM, Sebastian Schelter <ss...@apache.org> wrote:
>
>> Chameera,
>>
>> This is wrong. It's not enought to check the reference, two different
>> instances can be semantically equivalent,
>>
>> A dense matrix represents its entries in a two dimensional double array.
>> You have to check for the equality of these arrays.
>>
>> --sebastian
>>
>> On 29.12.2013 17:14, Chameera Wijebandara wrote:
>>> Hi Tharindu
>>>
>>> In the DenseMatrix class does not implement equals method so it use
>>> Object.equals witch check the reference.
>>> You have implement equals method by overriding Object.equals method.
>>>
>>> Thanks
>>> Chameera
>>>
>>
>>
>
>
Re: Equality of two DenseMatrix objects
Posted by Chameera Wijebandara <ch...@gmail.com>.
Sebastian,
Yes It's not enought to check the reference. i mean DenseMatrix*.*equals()
methord does that and it cannot use for check equity is that wrong?
On Sun, Dec 29, 2013 at 8:40 AM, Sebastian Schelter <ss...@apache.org> wrote:
> Chameera,
>
> This is wrong. It's not enought to check the reference, two different
> instances can be semantically equivalent,
>
> A dense matrix represents its entries in a two dimensional double array.
> You have to check for the equality of these arrays.
>
> --sebastian
>
> On 29.12.2013 17:14, Chameera Wijebandara wrote:
> > Hi Tharindu
> >
> > In the DenseMatrix class does not implement equals method so it use
> > Object.equals witch check the reference.
> > You have implement equals method by overriding Object.equals method.
> >
> > Thanks
> > Chameera
> >
>
>
--
----------------------------------------------------------
Chameera Wijebandara,
Undergraduate Department of Computer Science and Engineering,
University of Moratuwa,
Sri Lanka.
Re: Equality of two DenseMatrix objects
Posted by Sebastian Schelter <ss...@apache.org>.
Chameera,
This is wrong. It's not enought to check the reference, two different
instances can be semantically equivalent,
A dense matrix represents its entries in a two dimensional double array.
You have to check for the equality of these arrays.
--sebastian
On 29.12.2013 17:14, Chameera Wijebandara wrote:
> Hi Tharindu
>
> In the DenseMatrix class does not implement equals method so it use
> Object.equals witch check the reference.
> You have implement equals method by overriding Object.equals method.
>
> Thanks
> Chameera
>
Re: Equality of two DenseMatrix objects
Posted by Chameera Wijebandara <ch...@gmail.com>.
Hi Tharindu
In the DenseMatrix class does not implement equals method so it use
Object.equals witch check the reference.
You have implement equals method by overriding Object.equals method.
Thanks
Chameera