You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@spark.apache.org by Rahul Bhojwani <ra...@gmail.com> on 2014/07/08 16:20:12 UTC

Is MLlib NaiveBayes implementation for Spark 0.9.1 correct?

Hi,

I wanted to use Naive Bayes for a text classification problem.I am using
Spark 0.9.1.
I was just curious to ask that is the Naive Bayes implementation in Spark
0.9.1 correct? Or are there any bugs in the Spark 0.9.1 implementation
which are taken care in Spark 1.0. My question is specific about MLlib
Naive Bayes implementation only. Also I am using Python.(If that adds any
ease for answer)


Thanks
-- 
Rahul K Bhojwani
3rd Year B.Tech
Computer Science and Engineering
National Institute of Technology, Karnataka

Re: Is MLlib NaiveBayes implementation for Spark 0.9.1 correct?

Posted by Rahul Bhojwani <ra...@gmail.com>.

Thanks a lot Xiangrui for the help.


On Wed, Jul 9, 2014 at 1:39 AM, Xiangrui Meng <me...@gmail.com> wrote:

> Well, I believe this is a correct implementation but please let us
> know if you run into problems. The NaiveBayes implementation in MLlib
> v1.0 supports sparse data, which is usually the case for text
> classificiation. I would recommend upgrading to v1.0. -Xiangrui
>
> On Tue, Jul 8, 2014 at 7:20 AM, Rahul Bhojwani
> <ra...@gmail.com> wrote:
> > Hi,
> >
> > I wanted to use Naive Bayes for a text classification problem.I am using
> > Spark 0.9.1.
> > I was just curious to ask that is the Naive Bayes implementation in Spark
> > 0.9.1 correct? Or are there any bugs in the Spark 0.9.1 implementation
> which
> > are taken care in Spark 1.0. My question is specific about MLlib Naive
> Bayes
> > implementation only. Also I am using Python.(If that adds any ease for
> > answer)
> >
> >
> > Thanks
> > --
> > Rahul K Bhojwani
> > 3rd Year B.Tech
> > Computer Science and Engineering
> > National Institute of Technology, Karnataka
>



-- 
Rahul K Bhojwani
3rd Year B.Tech
Computer Science and Engineering
National Institute of Technology, Karnataka

Re: Is MLlib NaiveBayes implementation for Spark 0.9.1 correct?

Posted by Xiangrui Meng <me...@gmail.com>.

Well, I believe this is a correct implementation but please let us
know if you run into problems. The NaiveBayes implementation in MLlib
v1.0 supports sparse data, which is usually the case for text
classificiation. I would recommend upgrading to v1.0. -Xiangrui

On Tue, Jul 8, 2014 at 7:20 AM, Rahul Bhojwani
<ra...@gmail.com> wrote:
> Hi,
>
> I wanted to use Naive Bayes for a text classification problem.I am using
> Spark 0.9.1.
> I was just curious to ask that is the Naive Bayes implementation in Spark
> 0.9.1 correct? Or are there any bugs in the Spark 0.9.1 implementation which
> are taken care in Spark 1.0. My question is specific about MLlib Naive Bayes
> implementation only. Also I am using Python.(If that adds any ease for
> answer)
>
>
> Thanks
> --
> Rahul K Bhojwani
> 3rd Year B.Tech
> Computer Science and Engineering
> National Institute of Technology, Karnataka