You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Joseph K. Bradley (JIRA)" <ji...@apache.org> on 2014/08/21 00:19:26 UTC

[jira] [Created] (SPARK-3157) Avoid duplicated stats in DecisionTree extractLeftRightNodeAggregates

Joseph K. Bradley created SPARK-3157:
----------------------------------------

             Summary: Avoid duplicated stats in DecisionTree extractLeftRightNodeAggregates
                 Key: SPARK-3157
                 URL: https://issues.apache.org/jira/browse/SPARK-3157
             Project: Spark
          Issue Type: Improvement
          Components: MLlib
            Reporter: Joseph K. Bradley
            Priority: Minor


Improvement: computation, memory usage

For ordered features, extractLeftRightNodeAggregates() computes pairs of cumulative sums.  However, these sums are redundant since they are simply cumulative sums accumulating from the left and right ends, respectively.  Only compute one sum.
For unordered features, the left and right aggregates are essentially the same data, copied from the original aggregates, but shifted by one index.  Avoid copying data.




--
This message was sent by Atlassian JIRA
(v6.2#6252)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org