You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@spark.apache.org by "Joseph K. Bradley (JIRA)" <ji...@apache.org> on 2014/08/21 00:21:26 UTC

[jira] [Created] (SPARK-3159) Check for reducible DecisionTree

Joseph K. Bradley created SPARK-3159:
----------------------------------------

             Summary: Check for reducible DecisionTree
                 Key: SPARK-3159
                 URL: https://issues.apache.org/jira/browse/SPARK-3159
             Project: Spark
          Issue Type: Improvement
          Components: MLlib
            Reporter: Joseph K. Bradley
            Priority: Minor


Improvement: test-time computation

Currently, pairs of leaf nodes with the same parent can both output the same prediction.  This happens since the splitting criterion (e.g., Gini) is not the same as prediction accuracy/MSE; the splitting criterion can sometimes be improved even when both children would still output the same prediction (e.g., based on the majority label for classification).

We could check the tree and reduce it if possible after training.

Note: This happens with scikit-learn as well.




--
This message was sent by Atlassian JIRA
(v6.2#6252)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org