You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@mahout.apache.org by Masoud Moshref Javadi <mo...@usc.edu> on 2012/07/17 04:04:54 UTC
kmeans centroid using or function
I want to run kmeans on binary data and the definition of centroid for
my application is the Or() of bits of all points inside a cluster.
Where, in Mahout, should I change?
--
Masoud Moshref Javadi
Computer Engineering PhD Student
Ming Hsieh Department of Electrical Engineering
University of Southern California
Re: kmeans centroid using or function
Posted by Masoud Moshref Javadi <mo...@usc.edu>.
Thank you for your reply.
Formally, d(x,y)=|x or y|. This is not a metric distance function but
gave me good result in hierarchical clustering (Matlab linkage function,
bottom-up) so I just wanted to test that using a top-down approach in
larger scale.
On 7/16/2012 11:40 PM, Sean Owen wrote:
> You can't define distance this way, relative to something changing or not
> when a point is added. What is "closest" here? Just trying to save you
> trouble down the road as I am not sure this will make meaningful results,
> even if you get something to run.
>
--
Masoud Moshref Javadi
Computer Engineering PhD Student
Ming Hsieh Department of Electrical Engineering
University of Southern California
Re: kmeans centroid using or function
Posted by Sean Owen <sr...@gmail.com>.
You can't define distance this way, relative to something changing or not
when a point is added. What is "closest" here? Just trying to save you
trouble down the road as I am not sure this will make meaningful results,
even if you get something to run.
Re: kmeans centroid using or function
Posted by Masoud Moshref Javadi <mo...@usc.edu>.
Each cluster will take storage space as sum of bits 1 in Or() of all of
its points. and the nearest cluster to a point is the one that its size
does not change after including that point. So Or() of bits of points of
a cluster can be a representative of that cluster.
It seems that I need a kind of subclass of AbstractCluster that keeps
track of this Or() bit vector.
On 7/16/2012 11:22 PM, Sean Owen wrote:
> Hmm is that going to give a point that acts like a centroid though, that is
> a "mid point" under some distance metric? I don't think you want to do this.
>
> On Tue, Jul 17, 2012 at 5:04 AM, Masoud Moshref Javadi <mo...@usc.edu>wrote:
>
>> I want to run kmeans on binary data and the definition of centroid for my
>> application is the Or() of bits of all points inside a cluster.
>> Where, in Mahout, should I change?
>>
>> --
>> Masoud Moshref Javadi
>> Computer Engineering PhD Student
>> Ming Hsieh Department of Electrical Engineering
>> University of Southern California
>>
>>
--
Masoud Moshref Javadi
Computer Engineering PhD Student
Ming Hsieh Department of Electrical Engineering
University of Southern California
Re: kmeans centroid using or function
Posted by Sean Owen <sr...@gmail.com>.
Hmm is that going to give a point that acts like a centroid though, that is
a "mid point" under some distance metric? I don't think you want to do this.
On Tue, Jul 17, 2012 at 5:04 AM, Masoud Moshref Javadi <mo...@usc.edu>wrote:
> I want to run kmeans on binary data and the definition of centroid for my
> application is the Or() of bits of all points inside a cluster.
> Where, in Mahout, should I change?
>
> --
> Masoud Moshref Javadi
> Computer Engineering PhD Student
> Ming Hsieh Department of Electrical Engineering
> University of Southern California
>
>
Re: kmeans centroid using or function
Posted by Yuval Feinstein <yu...@citypath.com>.
For kmeans you need a distance measure.
If you can build a custom distance measure that is consistent with your
definition of centroid, you can run kmeans using it.
I am unsure whether such a measure exists - by your definition, two binary
vectors that are opposite each other can be in the same centroid.
However, take a look at the TanimotoDistanceMeasure - It might help.
Good luck,
-- Yuval
On Tue, Jul 17, 2012 at 7:56 AM, Masoud Moshref Javadi <mo...@usc.edu>wrote:
>
> I want to run kmeans on binary data and the definition of centroid for
> my application is the Or() of bits of all points inside a cluster.
> Where, in Mahout, should I change?
>
> --
> Masoud Moshref Javadi
> Computer Engineering PhD Student
> Ming Hsieh Department of Electrical Engineering
> University of Southern California
>
>
>
>
>
kmeans centroid using or function
Posted by Masoud Moshref Javadi <mo...@usc.edu>.
I want to run kmeans on binary data and the definition of centroid for
my application is the Or() of bits of all points inside a cluster.
Where, in Mahout, should I change?
--
Masoud Moshref Javadi
Computer Engineering PhD Student
Ming Hsieh Department of Electrical Engineering
University of Southern California