You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@mahout.apache.org by Yimin Wu <wy...@hotmail.com> on 2012/06/01 21:13:49 UTC

Bug report in FuzzyKMeansUtil




Unlike KMeansUtil, the FuzzyKMeansUtil does not take a Configuration as a parameter. While performing deserialization, it create a new Configuration. This is a bug since it wouldn't work on Hadoop files. It will calles FuzzyMapper setup throw an exception complainning the file can not be found. I verified that the issue would gone after passing in conf like KMeansUtil.  Bascially, this code patern "new Configuration()" could be problematic as long as the code would be used in a mapreduce job. I suggest developer team review all the places where a Configuration object is used. Whenever a new Configuration object is created instead of passed in. Thanks,Yimin Wu Predixion Software 		 	   		  

RE: Bug report in FuzzyKMeansUtil

Posted by Yimin Wu <wy...@hotmail.com>.

 Hi Jeff, Thanks a lot for your prompt response. Yimin> Date: Fri, 1 Jun 2012 16:14:29 -0400
> From: jdog@windwardsolutions.com
> To: user@mahout.apache.org
> Subject: Re: Bug report in FuzzyKMeansUtil
> 
> Hi Yimin,
> 
> Thanks for pointing this out. I've just committed a patch to correct it.
> 
> On 6/1/12 3:13 PM, Yimin Wu wrote:
> >
> >
> >
> > Unlike KMeansUtil, the FuzzyKMeansUtil does not take a Configuration as a parameter. While performing deserialization, it create a new Configuration. This is a bug since it wouldn't work on Hadoop files. It will calles FuzzyMapper setup throw an exception complainning the file can not be found. I verified that the issue would gone after passing in conf like KMeansUtil.  Bascially, this code patern "new Configuration()" could be problematic as long as the code would be used in a mapreduce job. I suggest developer team review all the places where a Configuration object is used. Whenever a new Configuration object is created instead of passed in. Thanks,Yimin Wu Predixion Software 		 	   		
> 
 		 	   		  

Re: Bug report in FuzzyKMeansUtil

Posted by Jeff Eastman <jd...@windwardsolutions.com>.
Hi Yimin,

Thanks for pointing this out. I've just committed a patch to correct it.

On 6/1/12 3:13 PM, Yimin Wu wrote:
>
>
>
> Unlike KMeansUtil, the FuzzyKMeansUtil does not take a Configuration as a parameter. While performing deserialization, it create a new Configuration. This is a bug since it wouldn't work on Hadoop files. It will calles FuzzyMapper setup throw an exception complainning the file can not be found. I verified that the issue would gone after passing in conf like KMeansUtil.  Bascially, this code patern "new Configuration()" could be problematic as long as the code would be used in a mapreduce job. I suggest developer team review all the places where a Configuration object is used. Whenever a new Configuration object is created instead of passed in. Thanks,Yimin Wu Predixion Software