[jira] [Commented] (SPARK-14503) spark.ml Scala API for FPGrowth

["Maciej Szymkiewicz (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/SPARK-14503?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15905840#comment-15905840 ] 

Maciej Szymkiewicz commented on SPARK-14503:
--------------------------------------------

I think we should keep only unique predictions in {{transform}} otherwise it is possible to get results like this:

{code}
scala> val data = spark.read.text("data/mllib/sample_fpgrowth.txt").select(split($"value", "\\s+").alias("features")) 
data: org.apache.spark.sql.DataFrame = [features: array<string>]

scala> val data = spark.read.text("data/mllib/sample_fpgrowth.txt").select(split($"value", "\\s+").alias("features")) 
data: org.apache.spark.sql.DataFrame = [features: array<string>]

scala> fpm.transform(Seq(Array("t", "s")).toDF("features")).show(1, false)
+--------+---------------------+
|features|prediction           |
+--------+---------------------+
|[t, s]  |[y, x, z, x, y, x, z]|
+--------+---------------------+

{code}

> spark.ml Scala API for FPGrowth
> -------------------------------
>
>                 Key: SPARK-14503
>                 URL: https://issues.apache.org/jira/browse/SPARK-14503
>             Project: Spark
>          Issue Type: Sub-task
>          Components: ML
>            Reporter: Joseph K. Bradley
>            Assignee: yuhao yang
>             Fix For: 2.2.0
>
>
> This task is the first port of spark.mllib.fpm functionality to spark.ml (Scala).
> This will require a brief design doc to confirm a reasonable DataFrame-based API, with details for this class.  The doc could also look ahead to the other fpm classes, especially if their API decisions will affect FPGrowth.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org