[GitHub] srowen commented on a change in pull request #23682: [SPARK-19591][ML][MLlib][FOLLOWUP] Add sample weights to decision trees - fix tolerance

[GitBox <gi...@apache.org>]

srowen commented on a change in pull request #23682: [SPARK-19591][ML][MLlib][FOLLOWUP] Add sample weights to decision trees - fix tolerance
URL: https://github.com/apache/spark/pull/23682#discussion_r251856255
 
 

 ##########
 File path: mllib/src/main/scala/org/apache/spark/ml/tree/impl/RandomForest.scala
 ##########
 @@ -1050,8 +1050,12 @@ private[spark] object RandomForest extends Logging with Serializable {
       // Calculate the expected number of samples for finding splits
       val weightedNumSamples = samplesFractionForFindSplits(metadata) *
         metadata.weightedNumExamples
+      // scale tolerance by number of samples with constant factor
+      // Note: constant factor was tuned by running some tests where there were no zero
+      // feature values and validating we are never within tolerance
+      val factor = 100
 
 Review comment:
   This is fine, though I don't know that adding this variable adds anything over "* 100"

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org