[GitHub] spark pull request #15018: [SPARK-17455][MLlib] Improve PAVA implementation ...

[srowen <gi...@git.apache.org>]

Github user srowen commented on a diff in the pull request:

    https://github.com/apache/spark/pull/15018#discussion_r94571308
  
    --- Diff: mllib/src/main/scala/org/apache/spark/mllib/regression/IsotonicRegression.scala ---
    @@ -328,74 +336,81 @@ class IsotonicRegression private (private var isotonic: Boolean) extends Seriali
           return Array.empty
         }
     
    -    // Pools sub array within given bounds assigning weighted average value to all elements.
    -    def pool(input: Array[(Double, Double, Double)], start: Int, end: Int): Unit = {
    -      val poolSubArray = input.slice(start, end + 1)
     
    -      val weightedSum = poolSubArray.map(lp => lp._1 * lp._3).sum
    -      val weight = poolSubArray.map(_._3).sum
    +    // Keeps track of the start and end indices of the blocks. if [i, j] is a valid block from
    +    // input(i) to input(j) (inclusive), then blockBounds(i) = j and blockBounds(j) = i
    +    val blockBounds = Array.range(0, input.length) // Initially, each data point is its own block
     
    -      var i = start
    -      while (i <= end) {
    -        input(i) = (weightedSum / weight, input(i)._2, input(i)._3)
    -        i = i + 1
    -      }
    +    // Keep track of the sum of weights and sum of weight * y for each block. weights(start)
    +    // gives the values for the block. Entries that are not at the start of a block
    +    // are meaningless.
    +    val weights: Array[(Double, Double)] = input.map {
    +      case (_, _, weight) if weight == 0d =>
    --- End diff --
    
    I'd always write 0.0 instead of 0d for clarity. I also think we want an `IllegalArgumentException`, so maybe:
    
    ```
    val weights = input.map { case (y, _, weight) =>
      require(weight > 0.0)
      (weight, weight * y)
    }
    ```


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org