You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by yinxusen <gi...@git.apache.org> on 2016/03/03 07:09:06 UTC

[GitHub] spark pull request: [SPARK-13449] Naive Bayes wrapper in SparkR

GitHub user yinxusen opened a pull request:

    https://github.com/apache/spark/pull/11486

    [SPARK-13449] Naive Bayes wrapper in SparkR

    ## What changes were proposed in this pull request?
    
    https://issues.apache.org/jira/browse/SPARK-13449
    
    Add a Naive Bayes wrapper in SparkR, with predict, naiveBayes, summary.
    
    ## How was this patch tested?
    
    Test with sparkR unit test.


You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/yinxusen/spark SPARK-13449

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/spark/pull/11486.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #11486
    
----
commit fb1bca43fc13fa5509539e4a6d4fe20cd26d1dd5
Author: Xusen Yin <yi...@gmail.com>
Date:   2016-03-01T05:15:21Z

    runable draft

commit 787f25f5a84330632dff5c8d9fd8d7c0de02de8c
Author: Xusen Yin <yi...@gmail.com>
Date:   2016-03-01T06:50:17Z

    refine test and na handler

commit b66d3e5ef0803dad949a53d4210a455856c8a400
Author: Xusen Yin <yi...@gmail.com>
Date:   2016-03-01T17:55:36Z

    refine getModelName

commit a5ab2e678c660fbee957cbb20dced0b5f5a4a256
Author: Xusen Yin <yi...@gmail.com>
Date:   2016-03-01T22:26:39Z

    remove default interface

commit 9215fafd3295f488ba6d827b49eddb91c3032438
Author: Xusen Yin <yi...@gmail.com>
Date:   2016-03-01T23:42:51Z

    refine code

commit 388e85dbf41faeea74f5aaa084664d9d52cce184
Author: Xusen Yin <yi...@gmail.com>
Date:   2016-03-03T05:31:32Z

    add summary for NaiveBayes

commit 26d38e1baa0574221fc8cca104dfeeb1e057755f
Author: Xusen Yin <yi...@gmail.com>
Date:   2016-03-03T06:06:49Z

    refine

----


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-13449] Naive Bayes wrapper in SparkR

Posted by yinxusen <gi...@git.apache.org>.
Github user yinxusen commented on the pull request:

    https://github.com/apache/spark/pull/11486#issuecomment-194054855
  
    retest this please


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-13449] Naive Bayes wrapper in SparkR

Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:

    https://github.com/apache/spark/pull/11486#issuecomment-195863942
  
    **[Test build #53019 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/53019/consoleFull)** for PR 11486 at commit [`9991e79`](https://github.com/apache/spark/commit/9991e7993d425acf54471ddf4380d4c106138501).
     * This patch passes all tests.
     * This patch merges cleanly.
     * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-13449] Naive Bayes wrapper in SparkR

Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:

    https://github.com/apache/spark/pull/11486#issuecomment-191633391
  
    Test FAILed.
    Refer to this link for build results (access rights to CI server needed): 
    https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/52382/
    Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-13449] Naive Bayes wrapper in SparkR

Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:

    https://github.com/apache/spark/pull/11486#issuecomment-197676145
  
    **[Test build #53387 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/53387/consoleFull)** for PR 11486 at commit [`90b6ad9`](https://github.com/apache/spark/commit/90b6ad9ebd91d8cdfe9680c9c89355eaf3936b12).
     * This patch passes all tests.
     * This patch merges cleanly.
     * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-13449] Naive Bayes wrapper in SparkR

Posted by yinxusen <gi...@git.apache.org>.
Github user yinxusen commented on the pull request:

    https://github.com/apache/spark/pull/11486#issuecomment-192527483
  
    I think it works. I'll try to add it later. 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-13449] Naive Bayes wrapper in SparkR

Posted by yinxusen <gi...@git.apache.org>.
Github user yinxusen commented on a diff in the pull request:

    https://github.com/apache/spark/pull/11486#discussion_r56290378
  
    --- Diff: R/pkg/inst/tests/testthat/test_mllib.R ---
    @@ -141,3 +141,26 @@ test_that("kmeans", {
       cluster <- summary.model$cluster
       expect_equal(sort(collect(distinct(select(cluster, "prediction")))$prediction), c(0, 1))
     })
    +
    +test_that("naiveBayes", {
    +  training <- suppressWarnings(createDataFrame(sqlContext, iris))
    --- End diff --
    
    Sure.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-13449] Naive Bayes wrapper in SparkR

Posted by yanboliang <gi...@git.apache.org>.
Github user yanboliang commented on the pull request:

    https://github.com/apache/spark/pull/11486#issuecomment-192089154
  
    It's a good question! It's possible that the label of input dataset is not 0 based or not continuous. So we should use ```StringIndexer``` to index label in [0, numLabels), and after training we use ```IndexToString``` to map index label to the original ones. We have already store the label map in the metadata of label column.
    All the models under ML package will follow this rule. For examples, if you train ```LogisticRegression``` with the input label ```"-1, +1"``` will produce erroneous results, you should use ```StringIndexer``` to transform labels to ```"0, 1"``` firstly.   


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-13449] Naive Bayes wrapper in SparkR

Posted by yinxusen <gi...@git.apache.org>.
Github user yinxusen commented on the pull request:

    https://github.com/apache/spark/pull/11486#issuecomment-191597396
  
    test it please


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-13449] Naive Bayes wrapper in SparkR

Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:

    https://github.com/apache/spark/pull/11486#issuecomment-198841429
  
    **[Test build #53622 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/53622/consoleFull)** for PR 11486 at commit [`b4ee1aa`](https://github.com/apache/spark/commit/b4ee1aab70008919ba17cf02c8470f1a75c23ef8).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-13449] Naive Bayes wrapper in SparkR

Posted by mengxr <gi...@git.apache.org>.
Github user mengxr commented on a diff in the pull request:

    https://github.com/apache/spark/pull/11486#discussion_r56289848
  
    --- Diff: R/pkg/inst/tests/testthat/test_mllib.R ---
    @@ -141,3 +141,26 @@ test_that("kmeans", {
       cluster <- summary.model$cluster
       expect_equal(sort(collect(distinct(select(cluster, "prediction")))$prediction), c(0, 1))
     })
    +
    +test_that("naiveBayes", {
    +  training <- suppressWarnings(createDataFrame(sqlContext, iris))
    +
    +  model <- naiveBayes(Sepal_Width ~ ., data = training, lambda = 1, modelType = "multinomial")
    +  sample <- take(select(predict(model, training), "prediction"), 1)
    +  expect_equal(typeof(sample$prediction), "double")
    +  expect_equal(sample$prediction, 9)
    +
    +  # Test summary works on naiveBayes
    +  summary.model <- summary(model)
    +  expect_equal(length(summary.model$pi), 23)
    +  expect_equal(sum(summary.model$pi), 1)
    +  l1 <- summary(model)$theta[1,]
    --- End diff --
    
    `summary.model`


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-13449] Naive Bayes wrapper in SparkR

Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:

    https://github.com/apache/spark/pull/11486


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-13449] Naive Bayes wrapper in SparkR

Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:

    https://github.com/apache/spark/pull/11486#issuecomment-193388758
  
    **[Test build #52574 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/52574/consoleFull)** for PR 11486 at commit [`1a685e1`](https://github.com/apache/spark/commit/1a685e1d345f53ae9f7cfb270f110052df818f4c).
     * This patch **fails to build**.
     * This patch merges cleanly.
     * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-13449] Naive Bayes wrapper in SparkR

Posted by yinxusen <gi...@git.apache.org>.
Github user yinxusen commented on the pull request:

    https://github.com/apache/spark/pull/11486#issuecomment-193390578
  
    @yanboliang @jkbradley 
    
    For this PR, I extract labels manually from labelCol. But I still don't think it's good to make assumption first for labels to be 0-based and continuous like `0.0, 1.0, 2.0, ...`. Sure we can use a `StringIndexer` to re-index the labels if it does not match the assumption. But checking it is not efficient. I suggest keeping labels in `ml.NaiveBayes`.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-13449] Naive Bayes wrapper in SparkR

Posted by yinxusen <gi...@git.apache.org>.
Github user yinxusen commented on the pull request:

    https://github.com/apache/spark/pull/11486#issuecomment-193391568
  
    retest it please


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-13449] Naive Bayes wrapper in SparkR

Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:

    https://github.com/apache/spark/pull/11486#issuecomment-198877946
  
    Merged build finished. Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-13449] Naive Bayes wrapper in SparkR

Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:

    https://github.com/apache/spark/pull/11486#issuecomment-194621148
  
    **[Test build #52799 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/52799/consoleFull)** for PR 11486 at commit [`dbaf4e6`](https://github.com/apache/spark/commit/dbaf4e622dd20d646e0cc26d5df1ba3aec02f827).
     * This patch passes all tests.
     * This patch merges cleanly.
     * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-13449] Naive Bayes wrapper in SparkR

Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:

    https://github.com/apache/spark/pull/11486#issuecomment-194058243
  
    Merged build finished. Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-13449] Naive Bayes wrapper in SparkR

Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:

    https://github.com/apache/spark/pull/11486#issuecomment-197661337
  
    Test FAILed.
    Refer to this link for build results (access rights to CI server needed): 
    https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/53386/
    Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-13449] Naive Bayes wrapper in SparkR

Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:

    https://github.com/apache/spark/pull/11486#issuecomment-198871790
  
    **[Test build #53629 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/53629/consoleFull)** for PR 11486 at commit [`3d291de`](https://github.com/apache/spark/commit/3d291de561bc9155e32a0c286309e8b7ddde48c4).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-13449] Naive Bayes wrapper in SparkR

Posted by yinxusen <gi...@git.apache.org>.
Github user yinxusen commented on a diff in the pull request:

    https://github.com/apache/spark/pull/11486#discussion_r56447514
  
    --- Diff: mllib/src/main/scala/org/apache/spark/ml/classification/NaiveBayes.scala ---
    @@ -227,6 +233,35 @@ class NaiveBayesModel private[ml] (
     
       @Since("1.6.0")
       override def write: MLWriter = new NaiveBayesModel.NaiveBayesModelWriter(this)
    +
    +  private var featureNames: Option[Array[String]] = None
    --- End diff --
    
    @mengxr I remove the previous `NaiveBayesSummary` and add these two `featureNames` and `labelNames` because we need these two variables to be accessed from `NaiveBayesModel`.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-13449] Naive Bayes wrapper in SparkR

Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:

    https://github.com/apache/spark/pull/11486#issuecomment-191601001
  
    Merged build finished. Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-13449] Naive Bayes wrapper in SparkR

Posted by yinxusen <gi...@git.apache.org>.
Github user yinxusen commented on a diff in the pull request:

    https://github.com/apache/spark/pull/11486#discussion_r56290089
  
    --- Diff: R/pkg/inst/tests/testthat/test_mllib.R ---
    @@ -141,3 +141,26 @@ test_that("kmeans", {
       cluster <- summary.model$cluster
       expect_equal(sort(collect(distinct(select(cluster, "prediction")))$prediction), c(0, 1))
     })
    +
    +test_that("naiveBayes", {
    +  training <- suppressWarnings(createDataFrame(sqlContext, iris))
    --- End diff --
    
    Previously I use the HouseVote84 data because e1071::naiveBayes use that. But if I use it, then the testband should have mlbench package installed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-13449] Naive Bayes wrapper in SparkR

Posted by mengxr <gi...@git.apache.org>.
Github user mengxr commented on a diff in the pull request:

    https://github.com/apache/spark/pull/11486#discussion_r56290215
  
    --- Diff: R/pkg/inst/tests/testthat/test_mllib.R ---
    @@ -141,3 +141,26 @@ test_that("kmeans", {
       cluster <- summary.model$cluster
       expect_equal(sort(collect(distinct(select(cluster, "prediction")))$prediction), c(0, 1))
     })
    +
    +test_that("naiveBayes", {
    +  training <- suppressWarnings(createDataFrame(sqlContext, iris))
    --- End diff --
    
    Then we can create a really small dataset with 3 categories and some count data. We can also verify against e1071::naiveBayes output.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-13449] Naive Bayes wrapper in SparkR

Posted by yinxusen <gi...@git.apache.org>.
Github user yinxusen commented on a diff in the pull request:

    https://github.com/apache/spark/pull/11486#discussion_r56430750
  
    --- Diff: R/pkg/inst/tests/testthat/test_mllib.R ---
    @@ -141,3 +141,26 @@ test_that("kmeans", {
       cluster <- summary.model$cluster
       expect_equal(sort(collect(distinct(select(cluster, "prediction")))$prediction), c(0, 1))
     })
    +
    +test_that("naiveBayes", {
    +  training <- suppressWarnings(createDataFrame(sqlContext, iris))
    --- End diff --
    
    I choose https://stat.ethz.ch/R-manual/R-devel/library/datasets/html/infert.html.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-13449] Naive Bayes wrapper in SparkR

Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:

    https://github.com/apache/spark/pull/11486#issuecomment-191597928
  
    **[Test build #52376 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/52376/consoleFull)** for PR 11486 at commit [`26d38e1`](https://github.com/apache/spark/commit/26d38e1baa0574221fc8cca104dfeeb1e057755f).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-13449] Naive Bayes wrapper in SparkR

Posted by yinxusen <gi...@git.apache.org>.
Github user yinxusen commented on a diff in the pull request:

    https://github.com/apache/spark/pull/11486#discussion_r55621785
  
    --- Diff: R/pkg/R/mllib.R ---
    @@ -192,3 +210,37 @@ setMethod("fitted", signature(object = "PipelineModel"),
                   stop(paste("Unsupported model", modelName, sep = " "))
                 }
               })
    +
    +#' Fit a naive Bayes model
    +#'
    +#' Fit a naive Bayes model, similarly to R's naiveBayes() except for omitting two arguments 'subset'
    +#' and 'na.action'. Users can use 'subset' function and 'fillna' or 'na.omit' function of DataFrame,
    +#' respectviely, to preprocess their DataFrame. We use na.omit in this interface to avoid potential
    +#' errors.
    --- End diff --
    
    MLlib NaiveBayes cannot handle NA values, so I need to remove them with `na.omit`.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-13449] Naive Bayes wrapper in SparkR

Posted by felixcheung <gi...@git.apache.org>.
Github user felixcheung commented on the pull request:

    https://github.com/apache/spark/pull/11486#issuecomment-194125311
  
    From SparkR test failure:
    ```
    1. Error: naiveBayes -----------------------------------------------------------
    there is no package called 'mlbench'
    1: withCallingHandlers(eval(code, new_test_environment), error = capture_calls, message = function(c) invokeRestart("muffleMessage"))
    2: eval(code, new_test_environment)
    3: eval(expr, envir, enclos)
    4: data(HouseVotes84, package = "mlbench") at test_mllib.R:146
    5: find.package(package, lib.loc, verbose = verbose)
    6: stop(gettextf("there is no package called %s", sQuote(pkg)), domain = NA)
    Error: Test failures
    ```



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-13449] Naive Bayes wrapper in SparkR

Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:

    https://github.com/apache/spark/pull/11486#issuecomment-194117451
  
    **[Test build #52723 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/52723/consoleFull)** for PR 11486 at commit [`30e9c37`](https://github.com/apache/spark/commit/30e9c372207ed206a7dc294b5726ad008a18ed12).
     * This patch **fails SparkR unit tests**.
     * This patch merges cleanly.
     * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-13449] Naive Bayes wrapper in SparkR

Posted by felixcheung <gi...@git.apache.org>.
Github user felixcheung commented on a diff in the pull request:

    https://github.com/apache/spark/pull/11486#discussion_r55474302
  
    --- Diff: R/pkg/R/mllib.R ---
    @@ -192,3 +210,37 @@ setMethod("fitted", signature(object = "PipelineModel"),
                   stop(paste("Unsupported model", modelName, sep = " "))
                 }
               })
    +
    +#' Fit a naive Bayes model
    +#'
    +#' Fit a naive Bayes model, similarly to R's naiveBayes() except for omitting two arguments 'subset'
    +#' and 'na.action'. Users can use 'subset' function and 'fillna' or 'na.omit' function of DataFrame,
    +#' respectviely, to preprocess their DataFrame. We use na.omit in this interface to avoid potential
    +#' errors.
    +#'
    +#' @param object A symbolic description of the model to be fitted. Currently only a few formula
    +#'                operators are supported, including '~', '.', ':', '+', and '-'.
    +#' @param data DataFrame for training
    +#' @param lambda Smoothing parameter
    +#' @param modelType Either 'multinomial' or 'bernoulli'. Default "multinomial".
    +#' @param ... Undefined parameters
    --- End diff --
    
    suggest removing `...` "param" from roxygen2 doc here


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-13449] Naive Bayes wrapper in SparkR

Posted by mengxr <gi...@git.apache.org>.
Github user mengxr commented on a diff in the pull request:

    https://github.com/apache/spark/pull/11486#discussion_r56289840
  
    --- Diff: R/pkg/inst/tests/testthat/test_mllib.R ---
    @@ -141,3 +141,26 @@ test_that("kmeans", {
       cluster <- summary.model$cluster
       expect_equal(sort(collect(distinct(select(cluster, "prediction")))$prediction), c(0, 1))
     })
    +
    +test_that("naiveBayes", {
    +  training <- suppressWarnings(createDataFrame(sqlContext, iris))
    --- End diff --
    
    `iris` is not a good dataset for naive Bayes. @yinxusen Could you take a look at other base datasets that come with R? https://stat.ethz.ch/R-manual/R-devel/library/datasets/html/00Index.html It would be great if we can find one with categorical labels and count data. Otherwise, we can make a really simple one here.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-13449] Naive Bayes wrapper in SparkR

Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:

    https://github.com/apache/spark/pull/11486#issuecomment-194058224
  
    **[Test build #52713 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/52713/consoleFull)** for PR 11486 at commit [`1a685e1`](https://github.com/apache/spark/commit/1a685e1d345f53ae9f7cfb270f110052df818f4c).
     * This patch **fails to build**.
     * This patch merges cleanly.
     * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-13449] Naive Bayes wrapper in SparkR

Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:

    https://github.com/apache/spark/pull/11486#issuecomment-198845487
  
    Merged build finished. Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-13449] Naive Bayes wrapper in SparkR

Posted by felixcheung <gi...@git.apache.org>.
Github user felixcheung commented on a diff in the pull request:

    https://github.com/apache/spark/pull/11486#discussion_r55474264
  
    --- Diff: R/pkg/R/mllib.R ---
    @@ -192,3 +210,37 @@ setMethod("fitted", signature(object = "PipelineModel"),
                   stop(paste("Unsupported model", modelName, sep = " "))
                 }
               })
    +
    +#' Fit a naive Bayes model
    +#'
    +#' Fit a naive Bayes model, similarly to R's naiveBayes() except for omitting two arguments 'subset'
    +#' and 'na.action'. Users can use 'subset' function and 'fillna' or 'na.omit' function of DataFrame,
    +#' respectviely, to preprocess their DataFrame. We use na.omit in this interface to avoid potential
    --- End diff --
    
    typo: respectively


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-13449] Naive Bayes wrapper in SparkR

Posted by mengxr <gi...@git.apache.org>.
Github user mengxr commented on a diff in the pull request:

    https://github.com/apache/spark/pull/11486#discussion_r56289853
  
    --- Diff: R/pkg/inst/tests/testthat/test_mllib.R ---
    @@ -141,3 +141,26 @@ test_that("kmeans", {
       cluster <- summary.model$cluster
       expect_equal(sort(collect(distinct(select(cluster, "prediction")))$prediction), c(0, 1))
     })
    +
    +test_that("naiveBayes", {
    +  training <- suppressWarnings(createDataFrame(sqlContext, iris))
    +
    +  model <- naiveBayes(Sepal_Width ~ ., data = training, lambda = 1, modelType = "multinomial")
    +  sample <- take(select(predict(model, training), "prediction"), 1)
    +  expect_equal(typeof(sample$prediction), "double")
    +  expect_equal(sample$prediction, 9)
    +
    +  # Test summary works on naiveBayes
    +  summary.model <- summary(model)
    +  expect_equal(length(summary.model$pi), 23)
    +  expect_equal(sum(summary.model$pi), 1)
    +  l1 <- summary(model)$theta[1,]
    +  l2 <- summary(model)$theta[2,]
    +  expect_true(all.equal(Reduce(`+`, l1), 1))
    --- End diff --
    
    `expect_equal(sum(unlist(l1)), 1)`


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-13449] Naive Bayes wrapper in SparkR

Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:

    https://github.com/apache/spark/pull/11486#issuecomment-195864128
  
    Merged build finished. Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-13449] Naive Bayes wrapper in SparkR

Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:

    https://github.com/apache/spark/pull/11486#issuecomment-194604495
  
    **[Test build #52799 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/52799/consoleFull)** for PR 11486 at commit [`dbaf4e6`](https://github.com/apache/spark/commit/dbaf4e622dd20d646e0cc26d5df1ba3aec02f827).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-13449] Naive Bayes wrapper in SparkR

Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:

    https://github.com/apache/spark/pull/11486#issuecomment-191601008
  
    Test FAILed.
    Refer to this link for build results (access rights to CI server needed): 
    https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/52376/
    Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-13449] Naive Bayes wrapper in SparkR

Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:

    https://github.com/apache/spark/pull/11486#issuecomment-197659555
  
    **[Test build #53386 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/53386/consoleFull)** for PR 11486 at commit [`8e21393`](https://github.com/apache/spark/commit/8e2139379313f2b7094e750fba816e5a701a413a).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-13449] Naive Bayes wrapper in SparkR

Posted by felixcheung <gi...@git.apache.org>.
Github user felixcheung commented on a diff in the pull request:

    https://github.com/apache/spark/pull/11486#discussion_r55474419
  
    --- Diff: R/pkg/R/mllib.R ---
    @@ -192,3 +210,37 @@ setMethod("fitted", signature(object = "PipelineModel"),
                   stop(paste("Unsupported model", modelName, sep = " "))
                 }
               })
    +
    +#' Fit a naive Bayes model
    +#'
    +#' Fit a naive Bayes model, similarly to R's naiveBayes() except for omitting two arguments 'subset'
    +#' and 'na.action'. Users can use 'subset' function and 'fillna' or 'na.omit' function of DataFrame,
    +#' respectviely, to preprocess their DataFrame. We use na.omit in this interface to avoid potential
    +#' errors.
    +#'
    +#' @param object A symbolic description of the model to be fitted. Currently only a few formula
    +#'                operators are supported, including '~', '.', ':', '+', and '-'.
    +#' @param data DataFrame for training
    +#' @param lambda Smoothing parameter
    +#' @param modelType Either 'multinomial' or 'bernoulli'. Default "multinomial".
    +#' @param ... Undefined parameters
    +#' @return A fitted naive Bayes model.
    +#' @rdname naiveBayes
    +#' @export
    +#' @examples
    +#'\dontrun{
    +#' data(HouseVotes84, package = "mlbench")
    +#' sc <- sparkR.init()
    +#' sqlContext <- sparkRSQL.init(sc)
    +#' df <- createDataFrame(sqlContext, HouseVotes84)
    +#' model <- glm(Class ~ ., df, lambda = 1, modelType = "multinomial")
    --- End diff --
    
    shouldn't this be `navieBayes` instead of `glm`?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-13449] Naive Bayes wrapper in SparkR

Posted by felixcheung <gi...@git.apache.org>.
Github user felixcheung commented on a diff in the pull request:

    https://github.com/apache/spark/pull/11486#discussion_r55474272
  
    --- Diff: R/pkg/R/mllib.R ---
    @@ -192,3 +210,37 @@ setMethod("fitted", signature(object = "PipelineModel"),
                   stop(paste("Unsupported model", modelName, sep = " "))
                 }
               })
    +
    +#' Fit a naive Bayes model
    +#'
    +#' Fit a naive Bayes model, similarly to R's naiveBayes() except for omitting two arguments 'subset'
    +#' and 'na.action'. Users can use 'subset' function and 'fillna' or 'na.omit' function of DataFrame,
    +#' respectviely, to preprocess their DataFrame. We use na.omit in this interface to avoid potential
    +#' errors.
    --- End diff --
    
    what is this potential error?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-13449] Naive Bayes wrapper in SparkR

Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:

    https://github.com/apache/spark/pull/11486#issuecomment-191600863
  
    **[Test build #52376 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/52376/consoleFull)** for PR 11486 at commit [`26d38e1`](https://github.com/apache/spark/commit/26d38e1baa0574221fc8cca104dfeeb1e057755f).
     * This patch **fails MiMa tests**.
     * This patch merges cleanly.
     * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-13449] Naive Bayes wrapper in SparkR

Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:

    https://github.com/apache/spark/pull/11486#issuecomment-195864131
  
    Test PASSed.
    Refer to this link for build results (access rights to CI server needed): 
    https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/53019/
    Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-13449] Naive Bayes wrapper in SparkR

Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:

    https://github.com/apache/spark/pull/11486#issuecomment-198876238
  
    **[Test build #53627 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/53627/consoleFull)** for PR 11486 at commit [`87fa0aa`](https://github.com/apache/spark/commit/87fa0aa25f897ffef755557d2a9320eda86e74ed).
     * This patch passes all tests.
     * This patch merges cleanly.
     * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-13449] Naive Bayes wrapper in SparkR

Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:

    https://github.com/apache/spark/pull/11486#issuecomment-198845476
  
    **[Test build #53622 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/53622/consoleFull)** for PR 11486 at commit [`b4ee1aa`](https://github.com/apache/spark/commit/b4ee1aab70008919ba17cf02c8470f1a75c23ef8).
     * This patch passes all tests.
     * This patch merges cleanly.
     * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-13449] Naive Bayes wrapper in SparkR

Posted by felixcheung <gi...@git.apache.org>.
Github user felixcheung commented on a diff in the pull request:

    https://github.com/apache/spark/pull/11486#discussion_r55474361
  
    --- Diff: R/pkg/R/generics.R ---
    @@ -1168,3 +1168,7 @@ setGeneric("kmeans")
     #' @rdname fitted
     #' @export
     setGeneric("fitted")
    +
    +#' @rdname naiveBayes
    +#' @export
    +setGeneric("naiveBayes", function(formula, data, ...) { standardGeneric("naiveBayes") })
    --- End diff --
    
    could you check manually that the http://ugrad.stat.ubc.ca/R/library/e1071/html/naiveBayes.html
    e1071 package naiveBayes still work when SparkR is loaded after e1071?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-13449] Naive Bayes wrapper in SparkR

Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:

    https://github.com/apache/spark/pull/11486#issuecomment-194111388
  
    **[Test build #52723 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/52723/consoleFull)** for PR 11486 at commit [`30e9c37`](https://github.com/apache/spark/commit/30e9c372207ed206a7dc294b5726ad008a18ed12).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-13449] Naive Bayes wrapper in SparkR

Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:

    https://github.com/apache/spark/pull/11486#issuecomment-193388773
  
    Merged build finished. Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-13449] Naive Bayes wrapper in SparkR

Posted by yinxusen <gi...@git.apache.org>.
Github user yinxusen commented on a diff in the pull request:

    https://github.com/apache/spark/pull/11486#discussion_r56265257
  
    --- Diff: R/pkg/R/generics.R ---
    @@ -1168,3 +1168,7 @@ setGeneric("kmeans")
     #' @rdname fitted
     #' @export
     setGeneric("fitted")
    +
    +#' @rdname naiveBayes
    +#' @export
    +setGeneric("naiveBayes", function(formula, data, ...) { standardGeneric("naiveBayes") })
    --- End diff --
    
    Yes, I have a test to ensure that we can use the prefix to call e1071::naiveBayes().


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-13449] Naive Bayes wrapper in SparkR

Posted by felixcheung <gi...@git.apache.org>.
Github user felixcheung commented on a diff in the pull request:

    https://github.com/apache/spark/pull/11486#discussion_r56623549
  
    --- Diff: R/pkg/R/mllib.R ---
    @@ -71,14 +71,23 @@ setMethod("glm", signature(formula = "formula", family = "ANY", data = "DataFram
     #' @rdname predict
     #' @export
     #' @examples
    -#'\dontrun{
    +#' \dontrun{
     #' model <- glm(y ~ x, trainingData)
     #' predicted <- predict(model, testData)
     #' showDF(predicted)
     #'}
     setMethod("predict", signature(object = "PipelineModel"),
               function(object, newData) {
    -            return(dataFrame(callJMethod(object@model, "transform", newData@sdf)))
    +            modelName <- callJStatic("org.apache.spark.ml.api.r.SparkRWrappers",
    +                                     "getModelName", object@model)
    +            jdf <- if (modelName == "NaiveBayesModel") {
    +              newData <- na.omit(newData)
    +              callJStatic("org.apache.spark.ml.api.r.SparkRWrappers",
    +                          "getNaiveBayesRawLabelsPrediction", object@model, newData@sdf)
    +            } else {
    +              callJMethod(object@model, "transform", newData@sdf)
    --- End diff --
    
    don't we need to assign the return value to `jdf`?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-13449] Naive Bayes wrapper in SparkR

Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:

    https://github.com/apache/spark/pull/11486#issuecomment-191624904
  
    **[Test build #52382 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/52382/consoleFull)** for PR 11486 at commit [`a07beb2`](https://github.com/apache/spark/commit/a07beb2a26a4650b12b2fa72a8b802125b6b5560).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-13449] Naive Bayes wrapper in SparkR

Posted by yanboliang <gi...@git.apache.org>.
Github user yanboliang commented on a diff in the pull request:

    https://github.com/apache/spark/pull/11486#discussion_r56619866
  
    --- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/RFormula.scala ---
    @@ -248,6 +248,18 @@ class RFormulaModel private[feature](
           !columnNames.contains($(labelCol)) || schema($(labelCol)).dataType == DoubleType,
           "Label column already exists and is not of type DoubleType.")
       }
    +
    +  /**
    +   * Get the original array of labels if exists.
    +   */
    +  private[ml] def getOriginalLabels: Option[Array[String]] = {
    --- End diff --
    
    Should we add a ```IndexToString``` transformer at the end of the ```PipelineModel```? I think it would be more general. Other functions such as ```glm``` with "binomial" family should also do the same work.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-13449] Naive Bayes wrapper in SparkR

Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:

    https://github.com/apache/spark/pull/11486#issuecomment-197676388
  
    Merged build finished. Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-13449] Naive Bayes wrapper in SparkR

Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:

    https://github.com/apache/spark/pull/11486#issuecomment-197661336
  
    Merged build finished. Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-13449] Naive Bayes wrapper in SparkR

Posted by yinxusen <gi...@git.apache.org>.
Github user yinxusen commented on the pull request:

    https://github.com/apache/spark/pull/11486#issuecomment-197193976
  
    @mengxr One more thing, could you take a look at https://issues.apache.org/jira/browse/SPARK-13641? If we extract feature names from the RFormulaModel transformed data, the for categorical data, we can only extract transformed feature names like I said in that JIRA. Do you think it's OK to extract those names?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-13449] Naive Bayes wrapper in SparkR

Posted by mengxr <gi...@git.apache.org>.
Github user mengxr commented on the pull request:

    https://github.com/apache/spark/pull/11486#issuecomment-199887454
  
    @yinxusen I checked the implementation in e1071 and found that it supports both categorical and continuous features, which in MLlib we only support categorical features. So I updated the implementation with some refactor in #11890. Could you help review that PR? Thanks!


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-13449] Naive Bayes wrapper in SparkR

Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:

    https://github.com/apache/spark/pull/11486#issuecomment-193388779
  
    Test FAILed.
    Refer to this link for build results (access rights to CI server needed): 
    https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/52574/
    Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-13449] Naive Bayes wrapper in SparkR

Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:

    https://github.com/apache/spark/pull/11486#issuecomment-194117566
  
    Merged build finished. Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-13449] Naive Bayes wrapper in SparkR

Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:

    https://github.com/apache/spark/pull/11486#issuecomment-198876261
  
    Test PASSed.
    Refer to this link for build results (access rights to CI server needed): 
    https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/53627/
    Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-13449] Naive Bayes wrapper in SparkR

Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:

    https://github.com/apache/spark/pull/11486#issuecomment-194621836
  
    Test PASSed.
    Refer to this link for build results (access rights to CI server needed): 
    https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/52799/
    Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-13449] Naive Bayes wrapper in SparkR

Posted by yinxusen <gi...@git.apache.org>.
Github user yinxusen commented on the pull request:

    https://github.com/apache/spark/pull/11486#issuecomment-191814257
  
    I can see from the mllib.NaiveBayes that the labels are sorted. But how about if it is not 0 based or not continuous? Say, `1.0, 3.0, 5.0, ...`. It has no effect on training/prediction, but has one on the `summary`. Otherwise I have to extract the labels from its labelCol again.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-13449] Naive Bayes wrapper in SparkR

Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:

    https://github.com/apache/spark/pull/11486#issuecomment-193386606
  
    **[Test build #52574 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/52574/consoleFull)** for PR 11486 at commit [`1a685e1`](https://github.com/apache/spark/commit/1a685e1d345f53ae9f7cfb270f110052df818f4c).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-13449] Naive Bayes wrapper in SparkR

Posted by yinxusen <gi...@git.apache.org>.
Github user yinxusen commented on the pull request:

    https://github.com/apache/spark/pull/11486#issuecomment-197192991
  
    I'll try to extract raw labels.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-13449] Naive Bayes wrapper in SparkR

Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:

    https://github.com/apache/spark/pull/11486#issuecomment-197661334
  
    **[Test build #53386 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/53386/consoleFull)** for PR 11486 at commit [`8e21393`](https://github.com/apache/spark/commit/8e2139379313f2b7094e750fba816e5a701a413a).
     * This patch **fails R style tests**.
     * This patch merges cleanly.
     * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-13449] Naive Bayes wrapper in SparkR

Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:

    https://github.com/apache/spark/pull/11486#issuecomment-198845488
  
    Test PASSed.
    Refer to this link for build results (access rights to CI server needed): 
    https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/53622/
    Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-13449] Naive Bayes wrapper in SparkR

Posted by mengxr <gi...@git.apache.org>.
Github user mengxr commented on a diff in the pull request:

    https://github.com/apache/spark/pull/11486#discussion_r56289859
  
    --- Diff: mllib/src/main/scala/org/apache/spark/ml/classification/NaiveBayes.scala ---
    @@ -227,6 +236,25 @@ class NaiveBayesModel private[ml] (
     
       @Since("1.6.0")
       override def write: MLWriter = new NaiveBayesModel.NaiveBayesModelWriter(this)
    +
    +  private var trainingSummary: Option[NaiveBayesSummary] = None
    +
    +  private[classification] def setSummary(summary: NaiveBayesSummary): this.type = {
    +    this.trainingSummary = Some(summary)
    +    this
    +  }
    +
    +  /**
    +   * Gets summary of model on training set. An exception is thrown if `trainingSummary == None`.
    +   */
    +  @Since("2.0.0")
    +  def summary: NaiveBayesSummary = trainingSummary match {
    --- End diff --
    
    Do we need to add this class? It doesn't provide extra summary statistics than the model itself.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-13449] Naive Bayes wrapper in SparkR

Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:

    https://github.com/apache/spark/pull/11486#issuecomment-198876260
  
    Merged build finished. Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-13449] Naive Bayes wrapper in SparkR

Posted by mengxr <gi...@git.apache.org>.
Github user mengxr commented on a diff in the pull request:

    https://github.com/apache/spark/pull/11486#discussion_r56258707
  
    --- Diff: R/pkg/R/generics.R ---
    @@ -1168,3 +1168,7 @@ setGeneric("kmeans")
     #' @rdname fitted
     #' @export
     setGeneric("fitted")
    +
    +#' @rdname naiveBayes
    +#' @export
    +setGeneric("naiveBayes", function(formula, data, ...) { standardGeneric("naiveBayes") })
    --- End diff --
    
    Users need `e1071::naiveBayes` to call that. Both `naiveBayes` are implemented as S3 generic functions and the Spark one shares the same first argument type (`formula`) with the one in e1071. So I don't think we can avoid shadowing the method. I tried different loading orders and confirmed that both can be used with namespace prefixes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-13449] Naive Bayes wrapper in SparkR

Posted by yanboliang <gi...@git.apache.org>.
Github user yanboliang commented on the pull request:

    https://github.com/apache/spark/pull/11486#issuecomment-191694969
  
    Labels of ML ```NaiveBayesModel``` are sorted(FYI #7284), so we do not need to store it as member variable. Then it can pass the binary compatibility check.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-13449] Naive Bayes wrapper in SparkR

Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:

    https://github.com/apache/spark/pull/11486#issuecomment-194055694
  
    **[Test build #52713 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/52713/consoleFull)** for PR 11486 at commit [`1a685e1`](https://github.com/apache/spark/commit/1a685e1d345f53ae9f7cfb270f110052df818f4c).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-13449] Naive Bayes wrapper in SparkR

Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:

    https://github.com/apache/spark/pull/11486#issuecomment-191633301
  
    **[Test build #52382 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/52382/consoleFull)** for PR 11486 at commit [`a07beb2`](https://github.com/apache/spark/commit/a07beb2a26a4650b12b2fa72a8b802125b6b5560).
     * This patch **fails MiMa tests**.
     * This patch merges cleanly.
     * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-13449] Naive Bayes wrapper in SparkR

Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:

    https://github.com/apache/spark/pull/11486#issuecomment-194117569
  
    Test FAILed.
    Refer to this link for build results (access rights to CI server needed): 
    https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/52723/
    Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-13449] Naive Bayes wrapper in SparkR

Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:

    https://github.com/apache/spark/pull/11486#issuecomment-191633388
  
    Merged build finished. Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-13449] Naive Bayes wrapper in SparkR

Posted by yinxusen <gi...@git.apache.org>.
Github user yinxusen commented on the pull request:

    https://github.com/apache/spark/pull/11486#issuecomment-194604372
  
    @felixcheung Thanks for the review.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-13449] Naive Bayes wrapper in SparkR

Posted by yinxusen <gi...@git.apache.org>.
Github user yinxusen commented on a diff in the pull request:

    https://github.com/apache/spark/pull/11486#discussion_r56760149
  
    --- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/RFormula.scala ---
    @@ -248,6 +248,18 @@ class RFormulaModel private[feature](
           !columnNames.contains($(labelCol)) || schema($(labelCol)).dataType == DoubleType,
           "Label column already exists and is not of type DoubleType.")
       }
    +
    +  /**
    +   * Get the original array of labels if exists.
    +   */
    +  private[ml] def getOriginalLabels: Option[Array[String]] = {
    --- End diff --
    
    I'm rewriting it now.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-13449] Naive Bayes wrapper in SparkR

Posted by mengxr <gi...@git.apache.org>.
Github user mengxr commented on the pull request:

    https://github.com/apache/spark/pull/11486#issuecomment-197191433
  
    About labels, I think we should output the raw labels as predictions instead of the encoded indices. It is hard to extract the feature metadata in SparkR.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-13449] Naive Bayes wrapper in SparkR

Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:

    https://github.com/apache/spark/pull/11486#issuecomment-194058244
  
    Test FAILed.
    Refer to this link for build results (access rights to CI server needed): 
    https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/52713/
    Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-13449] Naive Bayes wrapper in SparkR

Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:

    https://github.com/apache/spark/pull/11486#issuecomment-194621830
  
    Merged build finished. Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-13449] Naive Bayes wrapper in SparkR

Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:

    https://github.com/apache/spark/pull/11486#issuecomment-197676398
  
    Test PASSed.
    Refer to this link for build results (access rights to CI server needed): 
    https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/53387/
    Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-13449] Naive Bayes wrapper in SparkR

Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:

    https://github.com/apache/spark/pull/11486#issuecomment-198877947
  
    Test PASSed.
    Refer to this link for build results (access rights to CI server needed): 
    https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/53629/
    Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-13449] Naive Bayes wrapper in SparkR

Posted by mengxr <gi...@git.apache.org>.
Github user mengxr commented on a diff in the pull request:

    https://github.com/apache/spark/pull/11486#discussion_r56289845
  
    --- Diff: R/pkg/inst/tests/testthat/test_mllib.R ---
    @@ -141,3 +141,26 @@ test_that("kmeans", {
       cluster <- summary.model$cluster
       expect_equal(sort(collect(distinct(select(cluster, "prediction")))$prediction), c(0, 1))
     })
    +
    +test_that("naiveBayes", {
    +  training <- suppressWarnings(createDataFrame(sqlContext, iris))
    +
    +  model <- naiveBayes(Sepal_Width ~ ., data = training, lambda = 1, modelType = "multinomial")
    +  sample <- take(select(predict(model, training), "prediction"), 1)
    +  expect_equal(typeof(sample$prediction), "double")
    +  expect_equal(sample$prediction, 9)
    --- End diff --
    
    See my comments about label conversion on the PR page.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-13449] Naive Bayes wrapper in SparkR

Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:

    https://github.com/apache/spark/pull/11486#issuecomment-195861369
  
    **[Test build #53019 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/53019/consoleFull)** for PR 11486 at commit [`9991e79`](https://github.com/apache/spark/commit/9991e7993d425acf54471ddf4380d4c106138501).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-13449] Naive Bayes wrapper in SparkR

Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:

    https://github.com/apache/spark/pull/11486#issuecomment-197665613
  
    **[Test build #53387 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/53387/consoleFull)** for PR 11486 at commit [`90b6ad9`](https://github.com/apache/spark/commit/90b6ad9ebd91d8cdfe9680c9c89355eaf3936b12).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-13449] Naive Bayes wrapper in SparkR

Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:

    https://github.com/apache/spark/pull/11486#issuecomment-198877928
  
    **[Test build #53629 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/53629/consoleFull)** for PR 11486 at commit [`3d291de`](https://github.com/apache/spark/commit/3d291de561bc9155e32a0c286309e8b7ddde48c4).
     * This patch passes all tests.
     * This patch merges cleanly.
     * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-13449] Naive Bayes wrapper in SparkR

Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:

    https://github.com/apache/spark/pull/11486#issuecomment-198870226
  
    **[Test build #53627 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/53627/consoleFull)** for PR 11486 at commit [`87fa0aa`](https://github.com/apache/spark/commit/87fa0aa25f897ffef755557d2a9320eda86e74ed).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-13449] Naive Bayes wrapper in SparkR

Posted by yinxusen <gi...@git.apache.org>.
Github user yinxusen commented on the pull request:

    https://github.com/apache/spark/pull/11486#issuecomment-198870531
  
    @mengxr @yanboliang Since the `ml.NaiveBayes` making the assumption that its input data's label is 0-based indices, we should add a `StringIndexer` for labels after `RFormula` if the input data's label column is not a string column because the `RFormula` doesn't handle numerical labels. Now we can extract labels from the final `IndexToString`.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org