[GitHub] spark pull request #13440: [SPARK-15699] [ML] Implement a Chi-Squared test s...

[srowen <gi...@git.apache.org>]

Github user srowen commented on a diff in the pull request:

    https://github.com/apache/spark/pull/13440#discussion_r218209453
  
    --- Diff: mllib/src/main/scala/org/apache/spark/mllib/tree/impurity/Impurity.scala ---
    @@ -52,6 +52,49 @@ trait Impurity extends Serializable {
       @Since("1.0.0")
       @DeveloperApi
       def calculate(count: Double, sum: Double, sumSquares: Double): Double
    +
    +  /**
    +   * :: DeveloperApi ::
    +   * Compute a test-statistic p-value quality measure from left and right split populations
    +   * @param calcL impurity calculator for the left split population
    +   * @param calcR impurity calculator for the right split population
    +   * @return The p-value for the null hypothesis; that left and right split populations
    +   * represent the same distribution
    +   * @note Unless overridden this method will fail with an exception, for backward compatability
    +   */
    +  @Since("2.2.0")
    +  @DeveloperApi
    +  def calculate(calcL: ImpurityCalculator, calcR: ImpurityCalculator): Double
    +
    +  /**
    +   * :: DeveloperApi ::
    +   * Determine if this impurity measure is a test-statistic measure
    +   * @return True if this is a split quality measure based on a test statistic (i.e. returns a
    +   * p-value) or false otherwise.
    +   * @note Unless overridden this method returns false by default, for backward compatability
    +   */
    +  @Since("2.2.0")
    +  @DeveloperApi
    +  def isTestStatistic: Boolean
    --- End diff --
    
    Adding methods to a public trait is technically an API breaking change. This might be considered a Developer API even though it's not labeled that way. Still if we can avoid adding to the API here, it'd be better.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org