[jira] [Issue Comment Deleted] (SPARK-6885) Decision trees: predict class probabilities

["Yanbo Liang (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/SPARK-6885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Yanbo Liang updated SPARK-6885:
-------------------------------
    Comment: was deleted

(was: [~josephkb] 
Thanks for your comments.
After survey I found that we have two candidate plan:
#1 We record the raw counts for each label in an Array[Double] at every LearningNode. That is we need to implement a new class PredictionStats which stores the "counts" array.
class PredictionStats(
    val predict: Double,
    val counts: Array[Double]) extends Serializable {
}
Compared with the old Predict class, we just add more prediction statistic information.
class Predict(
    val predict: Double,
    val prob: Double = 0.0) extends Serializable {
}
And we need to make corresponding change to InformationGainStats and calculatePredictionStats(), maybe need  a new InformationGainStats which will not affect the old mllib code.
#2 We only record the raw counts for each label at leaf node of LearningNode. That is we need to implement two kinds of LearningNode (InternalLearningNode and LeafLearningNode).  
I prefer the #1, looking forward your comments.
)

> Decision trees: predict class probabilities
> -------------------------------------------
>
>                 Key: SPARK-6885
>                 URL: https://issues.apache.org/jira/browse/SPARK-6885
>             Project: Spark
>          Issue Type: Sub-task
>          Components: ML
>    Affects Versions: 1.3.0
>            Reporter: Joseph K. Bradley
>            Assignee: Yanbo Liang
>
> Under spark.ml, have DecisionTreeClassifier (currently being added) extend ProbabilisticClassifier.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org