You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@mahout.apache.org by Yimin Wu <wy...@hotmail.com> on 2012/06/01 21:13:49 UTC
Bug report in FuzzyKMeansUtil
Unlike KMeansUtil, the FuzzyKMeansUtil does not take a Configuration as a parameter. While performing deserialization, it create a new Configuration. This is a bug since it wouldn't work on Hadoop files. It will calles FuzzyMapper setup throw an exception complainning the file can not be found. I verified that the issue would gone after passing in conf like KMeansUtil. Bascially, this code patern "new Configuration()" could be problematic as long as the code would be used in a mapreduce job. I suggest developer team review all the places where a Configuration object is used. Whenever a new Configuration object is created instead of passed in. Thanks,Yimin Wu Predixion Software
RE: Bug report in FuzzyKMeansUtil
Posted by Yimin Wu <wy...@hotmail.com>.
Hi Jeff, Thanks a lot for your prompt response. Yimin> Date: Fri, 1 Jun 2012 16:14:29 -0400
> From: jdog@windwardsolutions.com
> To: user@mahout.apache.org
> Subject: Re: Bug report in FuzzyKMeansUtil
>
> Hi Yimin,
>
> Thanks for pointing this out. I've just committed a patch to correct it.
>
> On 6/1/12 3:13 PM, Yimin Wu wrote:
> >
> >
> >
> > Unlike KMeansUtil, the FuzzyKMeansUtil does not take a Configuration as a parameter. While performing deserialization, it create a new Configuration. This is a bug since it wouldn't work on Hadoop files. It will calles FuzzyMapper setup throw an exception complainning the file can not be found. I verified that the issue would gone after passing in conf like KMeansUtil. Bascially, this code patern "new Configuration()" could be problematic as long as the code would be used in a mapreduce job. I suggest developer team review all the places where a Configuration object is used. Whenever a new Configuration object is created instead of passed in. Thanks,Yimin Wu Predixion Software
>
Re: Bug report in FuzzyKMeansUtil
Posted by Jeff Eastman <jd...@windwardsolutions.com>.
Hi Yimin,
Thanks for pointing this out. I've just committed a patch to correct it.
On 6/1/12 3:13 PM, Yimin Wu wrote:
>
>
>
> Unlike KMeansUtil, the FuzzyKMeansUtil does not take a Configuration as a parameter. While performing deserialization, it create a new Configuration. This is a bug since it wouldn't work on Hadoop files. It will calles FuzzyMapper setup throw an exception complainning the file can not be found. I verified that the issue would gone after passing in conf like KMeansUtil. Bascially, this code patern "new Configuration()" could be problematic as long as the code would be used in a mapreduce job. I suggest developer team review all the places where a Configuration object is used. Whenever a new Configuration object is created instead of passed in. Thanks,Yimin Wu Predixion Software