You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2020/06/02 22:29:38 UTC
[GitHub] [spark] huaxingao opened a new pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
huaxingao opened a new pull request #28710:
URL: https://github.com/apache/spark/pull/28710
### What changes were proposed in this pull request?
Add a generic ClassificationSummary trait
### Why are the changes needed?
Add a generic ClassificationSummary trait so all the classification models can use it to implement summary.
Currently in classification, we only have summary implemented in ```LogisticRegression```. There are requests to implement summary for ```LinearSVCModel``` in https://issues.apache.org/jira/browse/SPARK-20249 and to implement summary for ```RandomForestClassificationModel``` in https://issues.apache.org/jira/browse/SPARK-23631. If we add a generic ClassificationSummary trait and put all the common code there, we can easily add summary to ```LinearSVCModel``` and ```RandomForestClassificationModel```, and also add summary to all the other classification models.
We can use the same approach to add a generic RegressionSummary trait to regression package and implement summary for all the regression models.
### Does this PR introduce _any_ user-facing change?
### How was this patch tested?
existing tests
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28710:
URL: https://github.com/apache/spark/pull/28710#issuecomment-644839635
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28710:
URL: https://github.com/apache/spark/pull/28710#issuecomment-646187111
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28710:
URL: https://github.com/apache/spark/pull/28710#issuecomment-646270163
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #28710:
URL: https://github.com/apache/spark/pull/28710#issuecomment-637843539
**[Test build #123450 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123450/testReport)** for PR 28710 at commit [`91a121b`](https://github.com/apache/spark/commit/91a121bea6ac9803361e7a8d8f7327761a70cfb2).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] huaxingao commented on a change in pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
huaxingao commented on a change in pull request #28710:
URL: https://github.com/apache/spark/pull/28710#discussion_r434212285
##########
File path: mllib/src/main/scala/org/apache/spark/ml/classification/ClassificationSummary.scala
##########
@@ -0,0 +1,271 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.spark.ml.classification
+
+import org.apache.spark.annotation.Since
+import org.apache.spark.ml.functions.checkNonNegativeWeight
+import org.apache.spark.ml.linalg.Vector
+import org.apache.spark.mllib.evaluation.{BinaryClassificationMetrics, MulticlassMetrics}
+import org.apache.spark.sql.{DataFrame, Row}
+import org.apache.spark.sql.functions.{col, lit}
+import org.apache.spark.sql.types.DoubleType
+
+
+/**
+ * Abstraction for multiclass classification results for a given model.
+ */
+private[classification] trait ClassificationSummary extends Serializable {
+
+ /**
+ * Dataframe output by the model's `transform` method.
+ */
+ @Since("3.1.0")
+ def predictions: DataFrame
+
+ /** Field in "predictions" which gives the prediction of each class. */
+ @Since("3.1.0")
+ def predictionCol: String
+
+ /** Field in "predictions" which gives the true label of each instance (if available). */
+ @Since("3.1.0")
+ def labelCol: String
+
+ /** Field in "predictions" which gives the features of each instance as a vector. */
+ @Since("3.1.0")
+ def featuresCol: String
+
+ /** Field in "predictions" which gives the weight of each instance as a vector. */
+ @Since("3.1.0")
+ def weightCol: String
+
+ @transient private val multiclassMetrics = {
+ if (predictions.schema.fieldNames.contains(weightCol)) {
+ new MulticlassMetrics(
+ predictions.select(
+ col(predictionCol),
+ col(labelCol).cast(DoubleType),
+ checkNonNegativeWeight(col(weightCol).cast(DoubleType))).rdd.map {
+ case Row(prediction: Double, label: Double, weight: Double) => (prediction, label, weight)
+ })
+ } else {
+ new MulticlassMetrics(
+ predictions.select(
+ col(predictionCol),
+ col(labelCol).cast(DoubleType),
+ lit(1.0)).rdd.map {
+ case Row(prediction: Double, label: Double, weight: Double) => (prediction, label, weight)
+ })
+ }
+ }
+
+ /**
+ * Returns the sequence of labels in ascending order. This order matches the order used
+ * in metrics which are specified as arrays over labels, e.g., truePositiveRateByLabel.
+ *
+ * Note: In most cases, it will be values {0.0, 1.0, ..., numClasses-1}, However, if the
+ * training set is missing a label, then all of the arrays over labels
+ * (e.g., from truePositiveRateByLabel) will be of length numClasses-1 instead of the
+ * expected numClasses.
+ */
+ @Since("3.1.0")
+ def labels: Array[Double] = multiclassMetrics.labels
+
+ /** Returns true positive rate for each label (category). */
+ @Since("3.1.0")
+ def truePositiveRateByLabel: Array[Double] = recallByLabel
+
+ /** Returns false positive rate for each label (category). */
+ @Since("3.1.0")
+ def falsePositiveRateByLabel: Array[Double] = {
+ multiclassMetrics.labels.map(label => multiclassMetrics.falsePositiveRate(label))
+ }
+
+ /** Returns precision for each label (category). */
+ @Since("3.1.0")
+ def precisionByLabel: Array[Double] = {
+ multiclassMetrics.labels.map(label => multiclassMetrics.precision(label))
+ }
+
+ /** Returns recall for each label (category). */
+ @Since("3.1.0")
+ def recallByLabel: Array[Double] = {
+ multiclassMetrics.labels.map(label => multiclassMetrics.recall(label))
+ }
+
+ /** Returns f-measure for each label (category). */
+ @Since("3.1.0")
+ def fMeasureByLabel(beta: Double): Array[Double] = {
+ multiclassMetrics.labels.map(label => multiclassMetrics.fMeasure(label, beta))
+ }
+
+ /** Returns f1-measure for each label (category). */
+ @Since("3.1.0")
+ def fMeasureByLabel: Array[Double] = fMeasureByLabel(1.0)
+
+ /**
+ * Returns accuracy.
+ * (equals to the total number of correctly classified instances
+ * out of the total number of instances.)
+ */
+ @Since("3.1.0")
+ def accuracy: Double = multiclassMetrics.accuracy
+
+ /**
+ * Returns weighted true positive rate.
+ * (equals to precision, recall and f-measure)
+ */
+ @Since("3.1.0")
+ def weightedTruePositiveRate: Double = weightedRecall
+
+ /** Returns weighted false positive rate. */
+ @Since("3.1.0")
+ def weightedFalsePositiveRate: Double = multiclassMetrics.weightedFalsePositiveRate
+
+ /**
+ * Returns weighted averaged recall.
+ * (equals to precision, recall and f-measure)
+ */
+ @Since("3.1.0")
+ def weightedRecall: Double = multiclassMetrics.weightedRecall
+
+ /** Returns weighted averaged precision. */
+ @Since("3.1.0")
+ def weightedPrecision: Double = multiclassMetrics.weightedPrecision
+
+ /** Returns weighted averaged f-measure. */
+ @Since("3.1.0")
+ def weightedFMeasure(beta: Double): Double = multiclassMetrics.weightedFMeasure(beta)
+
+ /** Returns weighted averaged f1-measure. */
+ @Since("3.1.0")
+ def weightedFMeasure: Double = multiclassMetrics.weightedFMeasure(1.0)
+
+ /**
+ * Convenient method for casting to binary classification summary.
+ * This method will throw an Exception if the summary is not a binary summary.
+ */
+ @Since("3.1.0")
+ def asBinary: BinaryClassificationSummary = this match {
+ case b: BinaryClassificationSummary => b
+ case _ =>
+ throw new RuntimeException("Cannot cast to a binary summary.")
+ }
+}
+
+/**
+ * Abstraction for multiclass classification training results.
+ */
+private[classification] trait ClassificationTrainingSummary extends ClassificationSummary {
+
+ /** objective function (scaled loss + regularization) at each iteration. */
+ @Since("3.1.0")
+ def objectiveHistory: Array[Double]
+
+ /** Number of training iterations. */
+ @Since("3.1.0")
+ def totalIterations: Int = objectiveHistory.length
+}
+
+/**
+ * Abstraction for binary classification results for a given model.
+ */
+trait BinaryClassificationSummary extends ClassificationSummary {
+
+ private val sparkSession = predictions.sparkSession
+ import sparkSession.implicits._
+
+ /**
+ * Field in "predictions" which gives the probability or rawPrediction of each class as a vector.
+ */
+ @Since("3.1.0")
+ def scoreCol: String
Review comment:
Can't name it to ```probabilityCol``` because not all the classifiers have ```probabilityCol```. If no ```probability```, will use ```rawPrediction``` instead to calculate the binary classification metrics.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28710:
URL: https://github.com/apache/spark/pull/28710#issuecomment-644962315
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28710:
URL: https://github.com/apache/spark/pull/28710#issuecomment-644842211
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/124129/
Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] zhengruifeng commented on a change in pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
zhengruifeng commented on a change in pull request #28710:
URL: https://github.com/apache/spark/pull/28710#discussion_r438605736
##########
File path: mllib/src/main/scala/org/apache/spark/ml/classification/ClassificationSummary.scala
##########
@@ -0,0 +1,265 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.spark.ml.classification
+
+import org.apache.spark.annotation.Since
+import org.apache.spark.ml.functions.checkNonNegativeWeight
+import org.apache.spark.ml.linalg.Vector
+import org.apache.spark.mllib.evaluation.{BinaryClassificationMetrics, MulticlassMetrics}
+import org.apache.spark.sql.{DataFrame, Row}
+import org.apache.spark.sql.functions.{col, lit}
+import org.apache.spark.sql.types.DoubleType
+
+
+/**
+ * Abstraction for multiclass classification results for a given model.
+ */
+private[classification] trait ClassificationSummary extends Serializable {
+
+ /**
+ * Dataframe output by the model's `transform` method.
+ */
+ @Since("3.1.0")
+ def predictions: DataFrame
+
+ /**
+ * Field in "predictions" which gives the probability or rawPrediction of each class as a vector.
+ */
+ @Since("3.1.0")
+ def scoreCol: String
Review comment:
Why there is no `rawPredictionCol`?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] srowen commented on a change in pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
srowen commented on a change in pull request #28710:
URL: https://github.com/apache/spark/pull/28710#discussion_r434731977
##########
File path: mllib/src/main/scala/org/apache/spark/ml/classification/ClassificationSummary.scala
##########
@@ -0,0 +1,271 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.spark.ml.classification
+
+import org.apache.spark.annotation.Since
+import org.apache.spark.ml.functions.checkNonNegativeWeight
+import org.apache.spark.ml.linalg.Vector
+import org.apache.spark.mllib.evaluation.{BinaryClassificationMetrics, MulticlassMetrics}
+import org.apache.spark.sql.{DataFrame, Row}
+import org.apache.spark.sql.functions.{col, lit}
+import org.apache.spark.sql.types.DoubleType
+
+
+/**
+ * Abstraction for multiclass classification results for a given model.
+ */
+private[classification] trait ClassificationSummary extends Serializable {
+
+ /**
+ * Dataframe output by the model's `transform` method.
+ */
+ @Since("3.1.0")
+ def predictions: DataFrame
+
+ /** Field in "predictions" which gives the prediction of each class. */
+ @Since("3.1.0")
+ def predictionCol: String
+
+ /** Field in "predictions" which gives the true label of each instance (if available). */
+ @Since("3.1.0")
+ def labelCol: String
+
+ /** Field in "predictions" which gives the features of each instance as a vector. */
+ @Since("3.1.0")
+ def featuresCol: String
+
+ /** Field in "predictions" which gives the weight of each instance as a vector. */
+ @Since("3.1.0")
+ def weightCol: String
+
+ @transient private val multiclassMetrics = {
+ if (predictions.schema.fieldNames.contains(weightCol)) {
+ new MulticlassMetrics(
+ predictions.select(
+ col(predictionCol),
+ col(labelCol).cast(DoubleType),
+ checkNonNegativeWeight(col(weightCol).cast(DoubleType))).rdd.map {
+ case Row(prediction: Double, label: Double, weight: Double) => (prediction, label, weight)
+ })
+ } else {
+ new MulticlassMetrics(
+ predictions.select(
+ col(predictionCol),
+ col(labelCol).cast(DoubleType),
+ lit(1.0)).rdd.map {
+ case Row(prediction: Double, label: Double, weight: Double) => (prediction, label, weight)
+ })
+ }
+ }
+
+ /**
+ * Returns the sequence of labels in ascending order. This order matches the order used
+ * in metrics which are specified as arrays over labels, e.g., truePositiveRateByLabel.
+ *
+ * Note: In most cases, it will be values {0.0, 1.0, ..., numClasses-1}, However, if the
+ * training set is missing a label, then all of the arrays over labels
+ * (e.g., from truePositiveRateByLabel) will be of length numClasses-1 instead of the
+ * expected numClasses.
+ */
+ @Since("3.1.0")
+ def labels: Array[Double] = multiclassMetrics.labels
+
+ /** Returns true positive rate for each label (category). */
+ @Since("3.1.0")
+ def truePositiveRateByLabel: Array[Double] = recallByLabel
+
+ /** Returns false positive rate for each label (category). */
+ @Since("3.1.0")
+ def falsePositiveRateByLabel: Array[Double] = {
+ multiclassMetrics.labels.map(label => multiclassMetrics.falsePositiveRate(label))
+ }
+
+ /** Returns precision for each label (category). */
+ @Since("3.1.0")
+ def precisionByLabel: Array[Double] = {
+ multiclassMetrics.labels.map(label => multiclassMetrics.precision(label))
+ }
+
+ /** Returns recall for each label (category). */
+ @Since("3.1.0")
+ def recallByLabel: Array[Double] = {
+ multiclassMetrics.labels.map(label => multiclassMetrics.recall(label))
+ }
+
+ /** Returns f-measure for each label (category). */
+ @Since("3.1.0")
+ def fMeasureByLabel(beta: Double): Array[Double] = {
+ multiclassMetrics.labels.map(label => multiclassMetrics.fMeasure(label, beta))
+ }
+
+ /** Returns f1-measure for each label (category). */
+ @Since("3.1.0")
+ def fMeasureByLabel: Array[Double] = fMeasureByLabel(1.0)
+
+ /**
+ * Returns accuracy.
+ * (equals to the total number of correctly classified instances
+ * out of the total number of instances.)
+ */
+ @Since("3.1.0")
+ def accuracy: Double = multiclassMetrics.accuracy
+
+ /**
+ * Returns weighted true positive rate.
+ * (equals to precision, recall and f-measure)
+ */
+ @Since("3.1.0")
+ def weightedTruePositiveRate: Double = weightedRecall
+
+ /** Returns weighted false positive rate. */
+ @Since("3.1.0")
+ def weightedFalsePositiveRate: Double = multiclassMetrics.weightedFalsePositiveRate
+
+ /**
+ * Returns weighted averaged recall.
+ * (equals to precision, recall and f-measure)
+ */
+ @Since("3.1.0")
+ def weightedRecall: Double = multiclassMetrics.weightedRecall
+
+ /** Returns weighted averaged precision. */
+ @Since("3.1.0")
+ def weightedPrecision: Double = multiclassMetrics.weightedPrecision
+
+ /** Returns weighted averaged f-measure. */
+ @Since("3.1.0")
+ def weightedFMeasure(beta: Double): Double = multiclassMetrics.weightedFMeasure(beta)
+
+ /** Returns weighted averaged f1-measure. */
+ @Since("3.1.0")
+ def weightedFMeasure: Double = multiclassMetrics.weightedFMeasure(1.0)
+
+ /**
+ * Convenient method for casting to binary classification summary.
+ * This method will throw an Exception if the summary is not a binary summary.
+ */
+ @Since("3.1.0")
+ def asBinary: BinaryClassificationSummary = this match {
+ case b: BinaryClassificationSummary => b
+ case _ =>
+ throw new RuntimeException("Cannot cast to a binary summary.")
+ }
+}
+
+/**
+ * Abstraction for multiclass classification training results.
+ */
+private[classification] trait ClassificationTrainingSummary extends ClassificationSummary {
+
+ /** objective function (scaled loss + regularization) at each iteration. */
+ @Since("3.1.0")
+ def objectiveHistory: Array[Double]
+
+ /** Number of training iterations. */
+ @Since("3.1.0")
+ def totalIterations: Int = objectiveHistory.length
+}
+
+/**
+ * Abstraction for binary classification results for a given model.
+ */
+trait BinaryClassificationSummary extends ClassificationSummary {
+
+ private val sparkSession = predictions.sparkSession
+ import sparkSession.implicits._
+
+ /**
+ * Field in "predictions" which gives the probability or rawPrediction of each class as a vector.
+ */
+ @Since("3.1.0")
+ def scoreCol: String
Review comment:
For backwards-compatibility, keep probabilityCol in the subclass as an alias? or else that's a breaking change?
Would we ever want to expose both a probability and some raw score like logits? That might argue for some mixin for probabilityCol or something but that may get too complex.
##########
File path: mllib/src/main/scala/org/apache/spark/ml/classification/ClassificationSummary.scala
##########
@@ -0,0 +1,271 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.spark.ml.classification
+
+import org.apache.spark.annotation.Since
+import org.apache.spark.ml.functions.checkNonNegativeWeight
+import org.apache.spark.ml.linalg.Vector
+import org.apache.spark.mllib.evaluation.{BinaryClassificationMetrics, MulticlassMetrics}
+import org.apache.spark.sql.{DataFrame, Row}
+import org.apache.spark.sql.functions.{col, lit}
+import org.apache.spark.sql.types.DoubleType
+
+
+/**
+ * Abstraction for multiclass classification results for a given model.
+ */
+private[classification] trait ClassificationSummary extends Serializable {
+
+ /**
+ * Dataframe output by the model's `transform` method.
+ */
+ @Since("3.1.0")
+ def predictions: DataFrame
+
+ /** Field in "predictions" which gives the prediction of each class. */
+ @Since("3.1.0")
+ def predictionCol: String
+
+ /** Field in "predictions" which gives the true label of each instance (if available). */
+ @Since("3.1.0")
+ def labelCol: String
+
+ /** Field in "predictions" which gives the features of each instance as a vector. */
+ @Since("3.1.0")
+ def featuresCol: String
+
+ /** Field in "predictions" which gives the weight of each instance as a vector. */
+ @Since("3.1.0")
+ def weightCol: String
+
+ @transient private val multiclassMetrics = {
+ if (predictions.schema.fieldNames.contains(weightCol)) {
+ new MulticlassMetrics(
+ predictions.select(
+ col(predictionCol),
+ col(labelCol).cast(DoubleType),
+ checkNonNegativeWeight(col(weightCol).cast(DoubleType))).rdd.map {
+ case Row(prediction: Double, label: Double, weight: Double) => (prediction, label, weight)
+ })
+ } else {
+ new MulticlassMetrics(
+ predictions.select(
+ col(predictionCol),
+ col(labelCol).cast(DoubleType),
+ lit(1.0)).rdd.map {
+ case Row(prediction: Double, label: Double, weight: Double) => (prediction, label, weight)
+ })
+ }
+ }
+
+ /**
+ * Returns the sequence of labels in ascending order. This order matches the order used
+ * in metrics which are specified as arrays over labels, e.g., truePositiveRateByLabel.
+ *
+ * Note: In most cases, it will be values {0.0, 1.0, ..., numClasses-1}, However, if the
+ * training set is missing a label, then all of the arrays over labels
+ * (e.g., from truePositiveRateByLabel) will be of length numClasses-1 instead of the
+ * expected numClasses.
+ */
+ @Since("3.1.0")
+ def labels: Array[Double] = multiclassMetrics.labels
+
+ /** Returns true positive rate for each label (category). */
+ @Since("3.1.0")
+ def truePositiveRateByLabel: Array[Double] = recallByLabel
+
+ /** Returns false positive rate for each label (category). */
+ @Since("3.1.0")
+ def falsePositiveRateByLabel: Array[Double] = {
+ multiclassMetrics.labels.map(label => multiclassMetrics.falsePositiveRate(label))
+ }
+
+ /** Returns precision for each label (category). */
+ @Since("3.1.0")
+ def precisionByLabel: Array[Double] = {
+ multiclassMetrics.labels.map(label => multiclassMetrics.precision(label))
+ }
+
+ /** Returns recall for each label (category). */
+ @Since("3.1.0")
+ def recallByLabel: Array[Double] = {
+ multiclassMetrics.labels.map(label => multiclassMetrics.recall(label))
+ }
+
+ /** Returns f-measure for each label (category). */
+ @Since("3.1.0")
+ def fMeasureByLabel(beta: Double): Array[Double] = {
+ multiclassMetrics.labels.map(label => multiclassMetrics.fMeasure(label, beta))
+ }
+
+ /** Returns f1-measure for each label (category). */
+ @Since("3.1.0")
+ def fMeasureByLabel: Array[Double] = fMeasureByLabel(1.0)
+
+ /**
+ * Returns accuracy.
+ * (equals to the total number of correctly classified instances
+ * out of the total number of instances.)
+ */
+ @Since("3.1.0")
+ def accuracy: Double = multiclassMetrics.accuracy
+
+ /**
+ * Returns weighted true positive rate.
+ * (equals to precision, recall and f-measure)
+ */
+ @Since("3.1.0")
+ def weightedTruePositiveRate: Double = weightedRecall
+
+ /** Returns weighted false positive rate. */
+ @Since("3.1.0")
+ def weightedFalsePositiveRate: Double = multiclassMetrics.weightedFalsePositiveRate
+
+ /**
+ * Returns weighted averaged recall.
+ * (equals to precision, recall and f-measure)
+ */
+ @Since("3.1.0")
+ def weightedRecall: Double = multiclassMetrics.weightedRecall
+
+ /** Returns weighted averaged precision. */
+ @Since("3.1.0")
+ def weightedPrecision: Double = multiclassMetrics.weightedPrecision
+
+ /** Returns weighted averaged f-measure. */
+ @Since("3.1.0")
+ def weightedFMeasure(beta: Double): Double = multiclassMetrics.weightedFMeasure(beta)
+
+ /** Returns weighted averaged f1-measure. */
+ @Since("3.1.0")
+ def weightedFMeasure: Double = multiclassMetrics.weightedFMeasure(1.0)
+
+ /**
+ * Convenient method for casting to binary classification summary.
+ * This method will throw an Exception if the summary is not a binary summary.
+ */
+ @Since("3.1.0")
+ def asBinary: BinaryClassificationSummary = this match {
Review comment:
It's reasonable but one can just cast right? that will equally fail at runtime.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #28710:
URL: https://github.com/apache/spark/pull/28710#issuecomment-644839019
**[Test build #124129 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124129/testReport)** for PR 28710 at commit [`61e93d0`](https://github.com/apache/spark/commit/61e93d0cab590975b5d4f4551edd43cac6b55ce2).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #28710:
URL: https://github.com/apache/spark/pull/28710#issuecomment-643676654
**[Test build #123980 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123980/testReport)** for PR 28710 at commit [`b8aeff6`](https://github.com/apache/spark/commit/b8aeff62239abbec6b0d9d00825175e54abcc0d4).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #28710:
URL: https://github.com/apache/spark/pull/28710#issuecomment-643992323
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] zhengruifeng commented on a change in pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
zhengruifeng commented on a change in pull request #28710:
URL: https://github.com/apache/spark/pull/28710#discussion_r438594486
##########
File path: mllib/src/main/scala/org/apache/spark/ml/classification/ClassificationSummary.scala
##########
@@ -0,0 +1,265 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.spark.ml.classification
+
+import org.apache.spark.annotation.Since
+import org.apache.spark.ml.functions.checkNonNegativeWeight
+import org.apache.spark.ml.linalg.Vector
+import org.apache.spark.mllib.evaluation.{BinaryClassificationMetrics, MulticlassMetrics}
+import org.apache.spark.sql.{DataFrame, Row}
+import org.apache.spark.sql.functions.{col, lit}
+import org.apache.spark.sql.types.DoubleType
+
+
+/**
+ * Abstraction for multiclass classification results for a given model.
+ */
+private[classification] trait ClassificationSummary extends Serializable {
+
+ /**
+ * Dataframe output by the model's `transform` method.
+ */
+ @Since("3.1.0")
+ def predictions: DataFrame
+
+ /**
+ * Field in "predictions" which gives the probability or rawPrediction of each class as a vector.
+ */
+ @Since("3.1.0")
+ def scoreCol: String
Review comment:
Is it possible to add another trait `ProbabilisticClassifierSummary` to avoid this `scoreCol`? it seems does not exist in current impl.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] huaxingao commented on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
huaxingao commented on pull request #28710:
URL: https://github.com/apache/spark/pull/28710#issuecomment-647007802
Thank you all!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28710:
URL: https://github.com/apache/spark/pull/28710#issuecomment-643855961
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] zhengruifeng commented on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
zhengruifeng commented on pull request #28710:
URL: https://github.com/apache/spark/pull/28710#issuecomment-643131145
@huaxingao
since `scoreCol` is only used to compute `BinaryClassificationMetrics`, so what about putting it in `BinaryClassificationSummary`:
```
trait BinaryClassificationSummary extends ClassificationSummary {
def scoreCol: String = null
}
```
Then in `LogisticRegression`:
```
LogisticRegressionSummary extends ClassificationSummary {
def probabilityCol = "..."
}
trait BinaryLogisticRegressionSummary extends LogisticRegressionSummary with BinaryClassificationSummary {
def scoreCol = if (probabilityCol.nonEmpty) probabilityCol else if (rawPredictionCol.nonEmpty) rawPredictionCol else throw ...
}
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #28710:
URL: https://github.com/apache/spark/pull/28710#issuecomment-642306601
**[Test build #123789 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123789/testReport)** for PR 28710 at commit [`c2fb2b0`](https://github.com/apache/spark/commit/c2fb2b01b64f3a40df12346f4cbc32244fb3aa4e).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] huaxingao commented on a change in pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
huaxingao commented on a change in pull request #28710:
URL: https://github.com/apache/spark/pull/28710#discussion_r440017358
##########
File path: project/MimaExcludes.scala
##########
@@ -39,18 +39,44 @@ object MimaExcludes {
// [SPARK-31077] Remove ChiSqSelector dependency on mllib.ChiSqSelectorModel
// private constructor
ProblemFilters.exclude[IncompatibleMethTypeProblem]("org.apache.spark.ml.feature.ChiSqSelectorModel.this"),
+
// [SPARK-31127] Implement abstract Selector
// org.apache.spark.ml.feature.ChiSqSelectorModel type hierarchy change
// before: class ChiSqSelector extends Estimator with ChiSqSelectorParams
// after: class ChiSqSelector extends PSelector
// false positive, no binary incompatibility
ProblemFilters.exclude[MissingTypesProblem]("org.apache.spark.ml.feature.ChiSqSelectorModel"),
ProblemFilters.exclude[MissingTypesProblem]("org.apache.spark.ml.feature.ChiSqSelector"),
+
//[SPARK-31840] Add instance weight support in LogisticRegressionSummary
// weightCol in org.apache.spark.ml.classification.LogisticRegressionSummary is present only in current version
ProblemFilters.exclude[ReversedMissingMethodProblem]("org.apache.spark.ml.classification.LogisticRegressionSummary.weightCol"),
+
// [SPARK-24634] Add a new metric regarding number of inputs later than watermark plus allowed delay
- ProblemFilters.exclude[DirectMissingMethodProblem]("org.apache.spark.sql.streaming.StateOperatorProgress.<init>$default$4")
+ ProblemFilters.exclude[DirectMissingMethodProblem]("org.apache.spark.sql.streaming.StateOperatorProgress.<init>$default$4"),
+
+ //[SPARK-31893] Add a generic ClassificationSummary trait
+ ProblemFilters.exclude[InheritedNewAbstractMethodProblem]("org.apache.spark.ml.classification.LogisticRegressionTrainingSummary.org$apache$spark$ml$classification$ClassificationSummary$_setter_$org$apache$spark$ml$classification$ClassificationSummary$$multiclassMetrics_="),
+ ProblemFilters.exclude[InheritedNewAbstractMethodProblem]("org.apache.spark.ml.classification.LogisticRegressionTrainingSummary.org$apache$spark$ml$classification$ClassificationSummary$$multiclassMetrics"),
+ ProblemFilters.exclude[InheritedNewAbstractMethodProblem]("org.apache.spark.ml.classification.LogisticRegressionTrainingSummary.weightCol"),
+ ProblemFilters.exclude[InheritedNewAbstractMethodProblem]("org.apache.spark.ml.classification.BinaryLogisticRegressionTrainingSummary.org$apache$spark$ml$classification$BinaryClassificationSummary$_setter_$org$apache$spark$ml$classification$BinaryClassificationSummary$$sparkSession_="),
+ ProblemFilters.exclude[InheritedNewAbstractMethodProblem]("org.apache.spark.ml.classification.BinaryLogisticRegressionTrainingSummary.org$apache$spark$ml$classification$BinaryClassificationSummary$_setter_$org$apache$spark$ml$classification$BinaryClassificationSummary$$binaryMetrics_="),
+ ProblemFilters.exclude[InheritedNewAbstractMethodProblem]("org.apache.spark.ml.classification.BinaryLogisticRegressionTrainingSummary.org$apache$spark$ml$classification$BinaryClassificationSummary$$binaryMetrics"),
+ ProblemFilters.exclude[InheritedNewAbstractMethodProblem]("org.apache.spark.ml.classification.BinaryLogisticRegressionTrainingSummary.org$apache$spark$ml$classification$BinaryClassificationSummary$$sparkSession"),
+ ProblemFilters.exclude[InheritedNewAbstractMethodProblem]("org.apache.spark.ml.classification.BinaryLogisticRegressionTrainingSummary.org$apache$spark$ml$classification$ClassificationSummary$_setter_$org$apache$spark$ml$classification$ClassificationSummary$$multiclassMetrics_="),
+ ProblemFilters.exclude[InheritedNewAbstractMethodProblem]("org.apache.spark.ml.classification.BinaryLogisticRegressionTrainingSummary.org$apache$spark$ml$classification$ClassificationSummary$$multiclassMetrics"),
+ ProblemFilters.exclude[InheritedNewAbstractMethodProblem]("org.apache.spark.ml.classification.BinaryLogisticRegressionTrainingSummary.weightCol"),
+ ProblemFilters.exclude[IncompatibleResultTypeProblem]("org.apache.spark.ml.classification.LogisticRegressionSummary.asBinary"),
+ ProblemFilters.exclude[InheritedNewAbstractMethodProblem]("org.apache.spark.ml.classification.LogisticRegressionSummary.org$apache$spark$ml$classification$ClassificationSummary$_setter_$org$apache$spark$ml$classification$ClassificationSummary$$multiclassMetrics_="),
+ ProblemFilters.exclude[InheritedNewAbstractMethodProblem]("org.apache.spark.ml.classification.LogisticRegressionSummary.org$apache$spark$ml$classification$ClassificationSummary$$multiclassMetrics"),
+ ProblemFilters.exclude[InheritedNewAbstractMethodProblem]("org.apache.spark.ml.classification.LogisticRegressionSummary.weightCol"),
+ ProblemFilters.exclude[InheritedNewAbstractMethodProblem]("org.apache.spark.ml.classification.BinaryLogisticRegressionSummary.org$apache$spark$ml$classification$BinaryClassificationSummary$_setter_$org$apache$spark$ml$classification$BinaryClassificationSummary$$sparkSession_="),
+ ProblemFilters.exclude[InheritedNewAbstractMethodProblem]("org.apache.spark.ml.classification.BinaryLogisticRegressionSummary.org$apache$spark$ml$classification$BinaryClassificationSummary$_setter_$org$apache$spark$ml$classification$BinaryClassificationSummary$$binaryMetrics_="),
+ ProblemFilters.exclude[InheritedNewAbstractMethodProblem]("org.apache.spark.ml.classification.BinaryLogisticRegressionSummary.org$apache$spark$ml$classification$BinaryClassificationSummary$$binaryMetrics"),
+ ProblemFilters.exclude[InheritedNewAbstractMethodProblem]("org.apache.spark.ml.classification.BinaryLogisticRegressionSummary.org$apache$spark$ml$classification$BinaryClassificationSummary$$sparkSession"),
+ ProblemFilters.exclude[InheritedNewAbstractMethodProblem]("org.apache.spark.ml.classification.BinaryLogisticRegressionSummary.org$apache$spark$ml$classification$ClassificationSummary$_setter_$org$apache$spark$ml$classification$ClassificationSummary$$multiclassMetrics_="),
+ ProblemFilters.exclude[InheritedNewAbstractMethodProblem]("org.apache.spark.ml.classification.BinaryLogisticRegressionSummary.org$apache$spark$ml$classification$ClassificationSummary$$multiclassMetrics"),
+ ProblemFilters.exclude[InheritedNewAbstractMethodProblem]("org.apache.spark.ml.classification.BinaryLogisticRegressionSummary.weightCol")
)
Review comment:
MiMa exclusions have already been updated
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #28710:
URL: https://github.com/apache/spark/pull/28710#issuecomment-644527552
**[Test build #124093 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124093/testReport)** for PR 28710 at commit [`61e93d0`](https://github.com/apache/spark/commit/61e93d0cab590975b5d4f4551edd43cac6b55ce2).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28710:
URL: https://github.com/apache/spark/pull/28710#issuecomment-644238429
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #28710:
URL: https://github.com/apache/spark/pull/28710#issuecomment-638550530
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #28710:
URL: https://github.com/apache/spark/pull/28710#issuecomment-643659107
**[Test build #123980 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123980/testReport)** for PR 28710 at commit [`b8aeff6`](https://github.com/apache/spark/commit/b8aeff62239abbec6b0d9d00825175e54abcc0d4).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] viirya commented on a change in pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
viirya commented on a change in pull request #28710:
URL: https://github.com/apache/spark/pull/28710#discussion_r442330970
##########
File path: project/MimaExcludes.scala
##########
@@ -39,18 +39,43 @@ object MimaExcludes {
// [SPARK-31077] Remove ChiSqSelector dependency on mllib.ChiSqSelectorModel
// private constructor
ProblemFilters.exclude[IncompatibleMethTypeProblem]("org.apache.spark.ml.feature.ChiSqSelectorModel.this"),
+
// [SPARK-31127] Implement abstract Selector
// org.apache.spark.ml.feature.ChiSqSelectorModel type hierarchy change
// before: class ChiSqSelector extends Estimator with ChiSqSelectorParams
// after: class ChiSqSelector extends PSelector
// false positive, no binary incompatibility
ProblemFilters.exclude[MissingTypesProblem]("org.apache.spark.ml.feature.ChiSqSelectorModel"),
ProblemFilters.exclude[MissingTypesProblem]("org.apache.spark.ml.feature.ChiSqSelector"),
+
//[SPARK-31840] Add instance weight support in LogisticRegressionSummary
// weightCol in org.apache.spark.ml.classification.LogisticRegressionSummary is present only in current version
ProblemFilters.exclude[ReversedMissingMethodProblem]("org.apache.spark.ml.classification.LogisticRegressionSummary.weightCol"),
+
// [SPARK-24634] Add a new metric regarding number of inputs later than watermark plus allowed delay
- ProblemFilters.exclude[DirectMissingMethodProblem]("org.apache.spark.sql.streaming.StateOperatorProgress.<init>$default$4")
+ ProblemFilters.exclude[DirectMissingMethodProblem]("org.apache.spark.sql.streaming.StateOperatorProgress.<init>$default$4"),
+
+ //[SPARK-31893] Add a generic ClassificationSummary trait
+ ProblemFilters.exclude[InheritedNewAbstractMethodProblem]("org.apache.spark.ml.classification.LogisticRegressionTrainingSummary.org$apache$spark$ml$classification$ClassificationSummary$_setter_$org$apache$spark$ml$classification$ClassificationSummary$$multiclassMetrics_="),
+ ProblemFilters.exclude[InheritedNewAbstractMethodProblem]("org.apache.spark.ml.classification.LogisticRegressionTrainingSummary.org$apache$spark$ml$classification$ClassificationSummary$$multiclassMetrics"),
+ ProblemFilters.exclude[InheritedNewAbstractMethodProblem]("org.apache.spark.ml.classification.LogisticRegressionTrainingSummary.weightCol"),
Review comment:
Not sure if it is for all cases. But this looks like just because `weightCol` is inherited from `ClassificationSummary` instead of `LogisticRegressionSummary` now.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #28710:
URL: https://github.com/apache/spark/pull/28710#issuecomment-643659261
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28710:
URL: https://github.com/apache/spark/pull/28710#issuecomment-638534698
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #28710:
URL: https://github.com/apache/spark/pull/28710#issuecomment-639116837
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #28710:
URL: https://github.com/apache/spark/pull/28710#issuecomment-638549750
**[Test build #123509 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123509/testReport)** for PR 28710 at commit [`54124a4`](https://github.com/apache/spark/commit/54124a46f3fb185ac5a8bdc16f5bc56eb4fed47b).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28710:
URL: https://github.com/apache/spark/pull/28710#issuecomment-645773814
Merged build finished. Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #28710:
URL: https://github.com/apache/spark/pull/28710#issuecomment-643944306
**[Test build #124030 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124030/testReport)** for PR 28710 at commit [`2e6f35c`](https://github.com/apache/spark/commit/2e6f35c8e31fe1cde1637b922673339bfeef65fe).
* This patch **fails due to an unknown error code, -9**.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] srowen closed pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
srowen closed pull request #28710:
URL: https://github.com/apache/spark/pull/28710
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #28710:
URL: https://github.com/apache/spark/pull/28710#issuecomment-644478729
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #28710:
URL: https://github.com/apache/spark/pull/28710#issuecomment-644829409
**[Test build #124128 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124128/testReport)** for PR 28710 at commit [`61e93d0`](https://github.com/apache/spark/commit/61e93d0cab590975b5d4f4551edd43cac6b55ce2).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #28710:
URL: https://github.com/apache/spark/pull/28710#issuecomment-645922053
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #28710:
URL: https://github.com/apache/spark/pull/28710#issuecomment-645823660
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #28710:
URL: https://github.com/apache/spark/pull/28710#issuecomment-644574860
**[Test build #124093 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124093/testReport)** for PR 28710 at commit [`61e93d0`](https://github.com/apache/spark/commit/61e93d0cab590975b5d4f4551edd43cac6b55ce2).
* This patch **fails due to an unknown error code, -9**.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #28710:
URL: https://github.com/apache/spark/pull/28710#issuecomment-643659107
**[Test build #123980 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123980/testReport)** for PR 28710 at commit [`b8aeff6`](https://github.com/apache/spark/commit/b8aeff62239abbec6b0d9d00825175e54abcc0d4).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #28710:
URL: https://github.com/apache/spark/pull/28710#issuecomment-644864140
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #28710:
URL: https://github.com/apache/spark/pull/28710#issuecomment-644478474
**[Test build #124082 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124082/testReport)** for PR 28710 at commit [`61e93d0`](https://github.com/apache/spark/commit/61e93d0cab590975b5d4f4551edd43cac6b55ce2).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28710:
URL: https://github.com/apache/spark/pull/28710#issuecomment-639116837
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #28710:
URL: https://github.com/apache/spark/pull/28710#issuecomment-643855623
**[Test build #124021 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124021/testReport)** for PR 28710 at commit [`2e6f35c`](https://github.com/apache/spark/commit/2e6f35c8e31fe1cde1637b922673339bfeef65fe).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #28710:
URL: https://github.com/apache/spark/pull/28710#issuecomment-644961337
**[Test build #124131 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124131/testReport)** for PR 28710 at commit [`61e93d0`](https://github.com/apache/spark/commit/61e93d0cab590975b5d4f4551edd43cac6b55ce2).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #28710:
URL: https://github.com/apache/spark/pull/28710#issuecomment-644650499
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] huaxingao commented on a change in pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
huaxingao commented on a change in pull request #28710:
URL: https://github.com/apache/spark/pull/28710#discussion_r434853509
##########
File path: mllib/src/main/scala/org/apache/spark/ml/classification/ClassificationSummary.scala
##########
@@ -0,0 +1,271 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.spark.ml.classification
+
+import org.apache.spark.annotation.Since
+import org.apache.spark.ml.functions.checkNonNegativeWeight
+import org.apache.spark.ml.linalg.Vector
+import org.apache.spark.mllib.evaluation.{BinaryClassificationMetrics, MulticlassMetrics}
+import org.apache.spark.sql.{DataFrame, Row}
+import org.apache.spark.sql.functions.{col, lit}
+import org.apache.spark.sql.types.DoubleType
+
+
+/**
+ * Abstraction for multiclass classification results for a given model.
+ */
+private[classification] trait ClassificationSummary extends Serializable {
+
+ /**
+ * Dataframe output by the model's `transform` method.
+ */
+ @Since("3.1.0")
+ def predictions: DataFrame
+
+ /** Field in "predictions" which gives the prediction of each class. */
+ @Since("3.1.0")
+ def predictionCol: String
+
+ /** Field in "predictions" which gives the true label of each instance (if available). */
+ @Since("3.1.0")
+ def labelCol: String
+
+ /** Field in "predictions" which gives the features of each instance as a vector. */
+ @Since("3.1.0")
+ def featuresCol: String
+
+ /** Field in "predictions" which gives the weight of each instance as a vector. */
+ @Since("3.1.0")
+ def weightCol: String
+
+ @transient private val multiclassMetrics = {
+ if (predictions.schema.fieldNames.contains(weightCol)) {
+ new MulticlassMetrics(
+ predictions.select(
+ col(predictionCol),
+ col(labelCol).cast(DoubleType),
+ checkNonNegativeWeight(col(weightCol).cast(DoubleType))).rdd.map {
+ case Row(prediction: Double, label: Double, weight: Double) => (prediction, label, weight)
+ })
+ } else {
+ new MulticlassMetrics(
+ predictions.select(
+ col(predictionCol),
+ col(labelCol).cast(DoubleType),
+ lit(1.0)).rdd.map {
+ case Row(prediction: Double, label: Double, weight: Double) => (prediction, label, weight)
+ })
+ }
+ }
+
+ /**
+ * Returns the sequence of labels in ascending order. This order matches the order used
+ * in metrics which are specified as arrays over labels, e.g., truePositiveRateByLabel.
+ *
+ * Note: In most cases, it will be values {0.0, 1.0, ..., numClasses-1}, However, if the
+ * training set is missing a label, then all of the arrays over labels
+ * (e.g., from truePositiveRateByLabel) will be of length numClasses-1 instead of the
+ * expected numClasses.
+ */
+ @Since("3.1.0")
+ def labels: Array[Double] = multiclassMetrics.labels
+
+ /** Returns true positive rate for each label (category). */
+ @Since("3.1.0")
+ def truePositiveRateByLabel: Array[Double] = recallByLabel
+
+ /** Returns false positive rate for each label (category). */
+ @Since("3.1.0")
+ def falsePositiveRateByLabel: Array[Double] = {
+ multiclassMetrics.labels.map(label => multiclassMetrics.falsePositiveRate(label))
+ }
+
+ /** Returns precision for each label (category). */
+ @Since("3.1.0")
+ def precisionByLabel: Array[Double] = {
+ multiclassMetrics.labels.map(label => multiclassMetrics.precision(label))
+ }
+
+ /** Returns recall for each label (category). */
+ @Since("3.1.0")
+ def recallByLabel: Array[Double] = {
+ multiclassMetrics.labels.map(label => multiclassMetrics.recall(label))
+ }
+
+ /** Returns f-measure for each label (category). */
+ @Since("3.1.0")
+ def fMeasureByLabel(beta: Double): Array[Double] = {
+ multiclassMetrics.labels.map(label => multiclassMetrics.fMeasure(label, beta))
+ }
+
+ /** Returns f1-measure for each label (category). */
+ @Since("3.1.0")
+ def fMeasureByLabel: Array[Double] = fMeasureByLabel(1.0)
+
+ /**
+ * Returns accuracy.
+ * (equals to the total number of correctly classified instances
+ * out of the total number of instances.)
+ */
+ @Since("3.1.0")
+ def accuracy: Double = multiclassMetrics.accuracy
+
+ /**
+ * Returns weighted true positive rate.
+ * (equals to precision, recall and f-measure)
+ */
+ @Since("3.1.0")
+ def weightedTruePositiveRate: Double = weightedRecall
+
+ /** Returns weighted false positive rate. */
+ @Since("3.1.0")
+ def weightedFalsePositiveRate: Double = multiclassMetrics.weightedFalsePositiveRate
+
+ /**
+ * Returns weighted averaged recall.
+ * (equals to precision, recall and f-measure)
+ */
+ @Since("3.1.0")
+ def weightedRecall: Double = multiclassMetrics.weightedRecall
+
+ /** Returns weighted averaged precision. */
+ @Since("3.1.0")
+ def weightedPrecision: Double = multiclassMetrics.weightedPrecision
+
+ /** Returns weighted averaged f-measure. */
+ @Since("3.1.0")
+ def weightedFMeasure(beta: Double): Double = multiclassMetrics.weightedFMeasure(beta)
+
+ /** Returns weighted averaged f1-measure. */
+ @Since("3.1.0")
+ def weightedFMeasure: Double = multiclassMetrics.weightedFMeasure(1.0)
+
+ /**
+ * Convenient method for casting to binary classification summary.
+ * This method will throw an Exception if the summary is not a binary summary.
+ */
+ @Since("3.1.0")
+ def asBinary: BinaryClassificationSummary = this match {
Review comment:
I think this is a convenient method @WeichenXu123 added for LogisticRegressionSummary per Seth's request.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #28710:
URL: https://github.com/apache/spark/pull/28710#issuecomment-637886703
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #28710:
URL: https://github.com/apache/spark/pull/28710#issuecomment-637886324
**[Test build #123450 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123450/testReport)** for PR 28710 at commit [`91a121b`](https://github.com/apache/spark/commit/91a121bea6ac9803361e7a8d8f7327761a70cfb2).
* This patch **fails PySpark unit tests**.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28710:
URL: https://github.com/apache/spark/pull/28710#issuecomment-644575348
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/124093/
Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] huaxingao commented on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
huaxingao commented on pull request #28710:
URL: https://github.com/apache/spark/pull/28710#issuecomment-643945627
retest this please
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] huaxingao commented on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
huaxingao commented on pull request #28710:
URL: https://github.com/apache/spark/pull/28710#issuecomment-645774417
I moved ```asBinary``` back to ```LogisticRegressionSummary``` to get rid of this
```
ProblemFilters.exclude[IncompatibleResultTypeProblem]("org.apache.spark.ml.classification.LogisticRegressionSummary.asBinary")
```
All the rest of the MiMa problems are InheritedNewAbstractMethodProblem. I think those are OK.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #28710:
URL: https://github.com/apache/spark/pull/28710#issuecomment-637843950
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] zhengruifeng commented on a change in pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
zhengruifeng commented on a change in pull request #28710:
URL: https://github.com/apache/spark/pull/28710#discussion_r438609224
##########
File path: mllib/src/main/scala/org/apache/spark/ml/classification/ClassificationSummary.scala
##########
@@ -0,0 +1,265 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.spark.ml.classification
+
+import org.apache.spark.annotation.Since
+import org.apache.spark.ml.functions.checkNonNegativeWeight
+import org.apache.spark.ml.linalg.Vector
+import org.apache.spark.mllib.evaluation.{BinaryClassificationMetrics, MulticlassMetrics}
+import org.apache.spark.sql.{DataFrame, Row}
+import org.apache.spark.sql.functions.{col, lit}
+import org.apache.spark.sql.types.DoubleType
+
+
+/**
+ * Abstraction for multiclass classification results for a given model.
+ */
+private[classification] trait ClassificationSummary extends Serializable {
+
+ /**
+ * Dataframe output by the model's `transform` method.
+ */
+ @Since("3.1.0")
+ def predictions: DataFrame
+
+ /**
+ * Field in "predictions" which gives the probability or rawPrediction of each class as a vector.
+ */
+ @Since("3.1.0")
+ def scoreCol: String
+
+ /** Field in "predictions" which gives the prediction of each class. */
+ @Since("3.1.0")
+ def predictionCol: String
+
+ /** Field in "predictions" which gives the true label of each instance (if available). */
+ @Since("3.1.0")
+ def labelCol: String
+
+ /** Field in "predictions" which gives the features of each instance as a vector. */
+ @Since("3.1.0")
+ def featuresCol: String
+
+ /** Field in "predictions" which gives the weight of each instance as a vector. */
+ @Since("3.1.0")
+ def weightCol: String
+
+ @transient private val multiclassMetrics = {
+ if (predictions.schema.fieldNames.contains(weightCol)) {
+ new MulticlassMetrics(
+ predictions.select(
+ col(predictionCol),
+ col(labelCol).cast(DoubleType),
+ checkNonNegativeWeight(col(weightCol).cast(DoubleType))).rdd.map {
+ case Row(prediction: Double, label: Double, weight: Double) => (prediction, label, weight)
+ })
+ } else {
+ new MulticlassMetrics(
+ predictions.select(
+ col(predictionCol),
+ col(labelCol).cast(DoubleType),
+ lit(1.0)).rdd.map {
+ case Row(prediction: Double, label: Double, weight: Double) => (prediction, label, weight)
+ })
+ }
+ }
+
+ /**
+ * Returns the sequence of labels in ascending order. This order matches the order used
+ * in metrics which are specified as arrays over labels, e.g., truePositiveRateByLabel.
+ *
+ * Note: In most cases, it will be values {0.0, 1.0, ..., numClasses-1}, However, if the
+ * training set is missing a label, then all of the arrays over labels
+ * (e.g., from truePositiveRateByLabel) will be of length numClasses-1 instead of the
+ * expected numClasses.
+ */
+ @Since("3.1.0")
+ def labels: Array[Double] = multiclassMetrics.labels
+
+ /** Returns true positive rate for each label (category). */
+ @Since("3.1.0")
+ def truePositiveRateByLabel: Array[Double] = recallByLabel
+
+ /** Returns false positive rate for each label (category). */
+ @Since("3.1.0")
+ def falsePositiveRateByLabel: Array[Double] = {
+ multiclassMetrics.labels.map(label => multiclassMetrics.falsePositiveRate(label))
+ }
+
+ /** Returns precision for each label (category). */
+ @Since("3.1.0")
+ def precisionByLabel: Array[Double] = {
+ multiclassMetrics.labels.map(label => multiclassMetrics.precision(label))
+ }
+
+ /** Returns recall for each label (category). */
+ @Since("3.1.0")
+ def recallByLabel: Array[Double] = {
+ multiclassMetrics.labels.map(label => multiclassMetrics.recall(label))
+ }
+
+ /** Returns f-measure for each label (category). */
+ @Since("3.1.0")
+ def fMeasureByLabel(beta: Double): Array[Double] = {
+ multiclassMetrics.labels.map(label => multiclassMetrics.fMeasure(label, beta))
+ }
+
+ /** Returns f1-measure for each label (category). */
+ @Since("3.1.0")
+ def fMeasureByLabel: Array[Double] = fMeasureByLabel(1.0)
+
+ /**
+ * Returns accuracy.
+ * (equals to the total number of correctly classified instances
+ * out of the total number of instances.)
+ */
+ @Since("3.1.0")
+ def accuracy: Double = multiclassMetrics.accuracy
+
+ /**
+ * Returns weighted true positive rate.
+ * (equals to precision, recall and f-measure)
+ */
+ @Since("3.1.0")
+ def weightedTruePositiveRate: Double = weightedRecall
+
+ /** Returns weighted false positive rate. */
+ @Since("3.1.0")
+ def weightedFalsePositiveRate: Double = multiclassMetrics.weightedFalsePositiveRate
+
+ /**
+ * Returns weighted averaged recall.
+ * (equals to precision, recall and f-measure)
+ */
+ @Since("3.1.0")
+ def weightedRecall: Double = multiclassMetrics.weightedRecall
+
+ /** Returns weighted averaged precision. */
+ @Since("3.1.0")
+ def weightedPrecision: Double = multiclassMetrics.weightedPrecision
+
+ /** Returns weighted averaged f-measure. */
+ @Since("3.1.0")
+ def weightedFMeasure(beta: Double): Double = multiclassMetrics.weightedFMeasure(beta)
+
+ /** Returns weighted averaged f1-measure. */
+ @Since("3.1.0")
+ def weightedFMeasure: Double = multiclassMetrics.weightedFMeasure(1.0)
+
+ /**
+ * Convenient method for casting to binary classification summary.
+ * This method will throw an Exception if the summary is not a binary summary.
+ */
+ @Since("3.1.0")
+ def asBinary: BinaryClassificationSummary = this match {
+ case b: BinaryClassificationSummary => b
+ case _ =>
+ throw new RuntimeException("Cannot cast to a binary summary.")
+ }
+}
+
+/**
+ * Abstraction for training results.
+ */
+private[classification] trait TrainingSummary {
+
+ /** objective function (scaled loss + regularization) at each iteration. */
+ @Since("3.1.0")
+ def objectiveHistory: Array[Double]
+
+ /** Number of training iterations. */
+ @Since("3.1.0")
+ def totalIterations: Int = objectiveHistory.length
+}
+
+/**
+ * Abstraction for binary classification results for a given model.
+ */
+trait BinaryClassificationSummary extends ClassificationSummary {
+
+ private val sparkSession = predictions.sparkSession
+ import sparkSession.implicits._
+
+ // TODO: Allow the user to vary the number of bins using a setBins method in
+ // BinaryClassificationMetrics. For now the default is set to 100.
Review comment:
OK, I suggest changing it to the default value of 1000, so that the metrics are the same with evaluator and BinaryClassificationMetrics with default `numBins`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] viirya commented on a change in pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
viirya commented on a change in pull request #28710:
URL: https://github.com/apache/spark/pull/28710#discussion_r442330970
##########
File path: project/MimaExcludes.scala
##########
@@ -39,18 +39,43 @@ object MimaExcludes {
// [SPARK-31077] Remove ChiSqSelector dependency on mllib.ChiSqSelectorModel
// private constructor
ProblemFilters.exclude[IncompatibleMethTypeProblem]("org.apache.spark.ml.feature.ChiSqSelectorModel.this"),
+
// [SPARK-31127] Implement abstract Selector
// org.apache.spark.ml.feature.ChiSqSelectorModel type hierarchy change
// before: class ChiSqSelector extends Estimator with ChiSqSelectorParams
// after: class ChiSqSelector extends PSelector
// false positive, no binary incompatibility
ProblemFilters.exclude[MissingTypesProblem]("org.apache.spark.ml.feature.ChiSqSelectorModel"),
ProblemFilters.exclude[MissingTypesProblem]("org.apache.spark.ml.feature.ChiSqSelector"),
+
//[SPARK-31840] Add instance weight support in LogisticRegressionSummary
// weightCol in org.apache.spark.ml.classification.LogisticRegressionSummary is present only in current version
ProblemFilters.exclude[ReversedMissingMethodProblem]("org.apache.spark.ml.classification.LogisticRegressionSummary.weightCol"),
+
// [SPARK-24634] Add a new metric regarding number of inputs later than watermark plus allowed delay
- ProblemFilters.exclude[DirectMissingMethodProblem]("org.apache.spark.sql.streaming.StateOperatorProgress.<init>$default$4")
+ ProblemFilters.exclude[DirectMissingMethodProblem]("org.apache.spark.sql.streaming.StateOperatorProgress.<init>$default$4"),
+
+ //[SPARK-31893] Add a generic ClassificationSummary trait
+ ProblemFilters.exclude[InheritedNewAbstractMethodProblem]("org.apache.spark.ml.classification.LogisticRegressionTrainingSummary.org$apache$spark$ml$classification$ClassificationSummary$_setter_$org$apache$spark$ml$classification$ClassificationSummary$$multiclassMetrics_="),
+ ProblemFilters.exclude[InheritedNewAbstractMethodProblem]("org.apache.spark.ml.classification.LogisticRegressionTrainingSummary.org$apache$spark$ml$classification$ClassificationSummary$$multiclassMetrics"),
+ ProblemFilters.exclude[InheritedNewAbstractMethodProblem]("org.apache.spark.ml.classification.LogisticRegressionTrainingSummary.weightCol"),
Review comment:
Not sure if it is for all cases. But this looks like because `weightCol` is inherited from `ClassificationSummary` instead of `LogisticRegressionSummary` now.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #28710:
URL: https://github.com/apache/spark/pull/28710#issuecomment-639117298
**[Test build #123543 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123543/testReport)** for PR 28710 at commit [`f42fa8e`](https://github.com/apache/spark/commit/f42fa8e58d88dcee0107af4459103b4b5e9d9d18).
* This patch **fails Python style tests**.
* This patch merges cleanly.
* This patch adds the following public classes _(experimental)_:
* `class _ClassificationSummary(JavaWrapper):`
* `class _TrainingSummary(JavaWrapper):`
* `class _BinaryClassificationSummary(_ClassificationSummary):`
* `class LogisticRegressionSummary(_ClassificationSummary):`
* `class LogisticRegressionTrainingSummary(LogisticRegressionSummary, _TrainingSummary):`
* `class BinaryLogisticRegressionSummary(_BinaryClassificationSummary,`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] huaxingao commented on a change in pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
huaxingao commented on a change in pull request #28710:
URL: https://github.com/apache/spark/pull/28710#discussion_r436316633
##########
File path: project/MimaExcludes.scala
##########
@@ -49,7 +49,34 @@ object MimaExcludes {
//[SPARK-31840] Add instance weight support in LogisticRegressionSummary
// weightCol in org.apache.spark.ml.classification.LogisticRegressionSummary is present only in current version
- ProblemFilters.exclude[ReversedMissingMethodProblem]("org.apache.spark.ml.classification.LogisticRegressionSummary.weightCol")
+ ProblemFilters.exclude[ReversedMissingMethodProblem]("org.apache.spark.ml.classification.LogisticRegressionSummary.weightCol"),
Review comment:
Before change, the code structure is like this:
```
sealed trait LogisticRegressionSummary extends Serializable
sealed trait LogisticRegressionTrainingSummary extends LogisticRegressionSummary
sealed trait BinaryLogisticRegressionSummary extends LogisticRegressionSummary
sealed trait BinaryLogisticRegressionTrainingSummary extends
BinaryLogisticRegressionSummary with LogisticRegressionTrainingSummary
```
So seems OK to me, no users can subclass the old traits.
Python seems OK to me too. I only added a new method. Applications written for old versions of Spark shouldn't be affected by this.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #28710:
URL: https://github.com/apache/spark/pull/28710#issuecomment-644238429
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #28710:
URL: https://github.com/apache/spark/pull/28710#issuecomment-639187847
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #28710:
URL: https://github.com/apache/spark/pull/28710#issuecomment-639134843
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #28710:
URL: https://github.com/apache/spark/pull/28710#issuecomment-638462433
**[Test build #123505 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123505/testReport)** for PR 28710 at commit [`a4b2891`](https://github.com/apache/spark/commit/a4b2891573a5fb2b13b34042c3a58bbfbf019ee1).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] huaxingao commented on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
huaxingao commented on pull request #28710:
URL: https://github.com/apache/spark/pull/28710#issuecomment-644827487
retest this please
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28710:
URL: https://github.com/apache/spark/pull/28710#issuecomment-644575338
Merged build finished. Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #28710:
URL: https://github.com/apache/spark/pull/28710#issuecomment-644527823
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28710:
URL: https://github.com/apache/spark/pull/28710#issuecomment-644830265
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #28710:
URL: https://github.com/apache/spark/pull/28710#issuecomment-643897635
**[Test build #124030 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124030/testReport)** for PR 28710 at commit [`2e6f35c`](https://github.com/apache/spark/commit/2e6f35c8e31fe1cde1637b922673339bfeef65fe).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28710:
URL: https://github.com/apache/spark/pull/28710#issuecomment-643659261
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #28710:
URL: https://github.com/apache/spark/pull/28710#issuecomment-646269317
**[Test build #124227 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124227/testReport)** for PR 28710 at commit [`ead6da2`](https://github.com/apache/spark/commit/ead6da2e0e3d1a269e6ef76b23f8e845690f42a3).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #28710:
URL: https://github.com/apache/spark/pull/28710#issuecomment-638478075
**[Test build #123509 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123509/testReport)** for PR 28710 at commit [`54124a4`](https://github.com/apache/spark/commit/54124a46f3fb185ac5a8bdc16f5bc56eb4fed47b).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] srowen commented on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
srowen commented on pull request #28710:
URL: https://github.com/apache/spark/pull/28710#issuecomment-646707991
Just catching up here: so all the MiMa excludes don't actually represent public API changes right? they're all sealed or private?
We're going to lose the `@Since` annotations, which is a little unfortunate, but I can see the value in refactoring
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28710:
URL: https://github.com/apache/spark/pull/28710#issuecomment-645776165
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #28710:
URL: https://github.com/apache/spark/pull/28710#issuecomment-638475706
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #28710:
URL: https://github.com/apache/spark/pull/28710#issuecomment-646187111
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #28710:
URL: https://github.com/apache/spark/pull/28710#issuecomment-638463111
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28710:
URL: https://github.com/apache/spark/pull/28710#issuecomment-643891541
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/124021/
Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28710:
URL: https://github.com/apache/spark/pull/28710#issuecomment-639187847
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #28710:
URL: https://github.com/apache/spark/pull/28710#issuecomment-639134384
**[Test build #123545 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123545/testReport)** for PR 28710 at commit [`7b01b63`](https://github.com/apache/spark/commit/7b01b63f9ce6549eaf248296b0d48e98a2dd7a25).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] huaxingao commented on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
huaxingao commented on pull request #28710:
URL: https://github.com/apache/spark/pull/28710#issuecomment-638460829
also cc @zhengruifeng
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28710:
URL: https://github.com/apache/spark/pull/28710#issuecomment-643897872
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28710:
URL: https://github.com/apache/spark/pull/28710#issuecomment-645823742
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/124198/
Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28710:
URL: https://github.com/apache/spark/pull/28710#issuecomment-637886703
Merged build finished. Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28710:
URL: https://github.com/apache/spark/pull/28710#issuecomment-643676869
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #28710:
URL: https://github.com/apache/spark/pull/28710#issuecomment-644842199
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #28710:
URL: https://github.com/apache/spark/pull/28710#issuecomment-644241625
**[Test build #124065 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124065/testReport)** for PR 28710 at commit [`137e5b2`](https://github.com/apache/spark/commit/137e5b20790f6e0c61e6f64fdcf47b2608e198f3).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28710:
URL: https://github.com/apache/spark/pull/28710#issuecomment-643992323
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #28710:
URL: https://github.com/apache/spark/pull/28710#issuecomment-644331092
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #28710:
URL: https://github.com/apache/spark/pull/28710#issuecomment-643855623
**[Test build #124021 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124021/testReport)** for PR 28710 at commit [`2e6f35c`](https://github.com/apache/spark/commit/2e6f35c8e31fe1cde1637b922673339bfeef65fe).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #28710:
URL: https://github.com/apache/spark/pull/28710#issuecomment-644575338
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28710:
URL: https://github.com/apache/spark/pull/28710#issuecomment-638463111
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #28710:
URL: https://github.com/apache/spark/pull/28710#issuecomment-638478075
**[Test build #123509 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123509/testReport)** for PR 28710 at commit [`54124a4`](https://github.com/apache/spark/commit/54124a46f3fb185ac5a8bdc16f5bc56eb4fed47b).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #28710:
URL: https://github.com/apache/spark/pull/28710#issuecomment-643991785
**[Test build #124045 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124045/testReport)** for PR 28710 at commit [`0898708`](https://github.com/apache/spark/commit/08987080d624a86379f6ec77e65571db95f36e13).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #28710:
URL: https://github.com/apache/spark/pull/28710#issuecomment-644839019
**[Test build #124129 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124129/testReport)** for PR 28710 at commit [`61e93d0`](https://github.com/apache/spark/commit/61e93d0cab590975b5d4f4551edd43cac6b55ce2).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28710:
URL: https://github.com/apache/spark/pull/28710#issuecomment-638550530
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #28710:
URL: https://github.com/apache/spark/pull/28710#issuecomment-642306930
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #28710:
URL: https://github.com/apache/spark/pull/28710#issuecomment-638533567
**[Test build #123505 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123505/testReport)** for PR 28710 at commit [`a4b2891`](https://github.com/apache/spark/commit/a4b2891573a5fb2b13b34042c3a58bbfbf019ee1).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #28710:
URL: https://github.com/apache/spark/pull/28710#issuecomment-639116348
**[Test build #123543 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123543/testReport)** for PR 28710 at commit [`f42fa8e`](https://github.com/apache/spark/commit/f42fa8e58d88dcee0107af4459103b4b5e9d9d18).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] srowen commented on a change in pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
srowen commented on a change in pull request #28710:
URL: https://github.com/apache/spark/pull/28710#discussion_r436288381
##########
File path: project/MimaExcludes.scala
##########
@@ -49,7 +49,34 @@ object MimaExcludes {
//[SPARK-31840] Add instance weight support in LogisticRegressionSummary
// weightCol in org.apache.spark.ml.classification.LogisticRegressionSummary is present only in current version
- ProblemFilters.exclude[ReversedMissingMethodProblem]("org.apache.spark.ml.classification.LogisticRegressionSummary.weightCol")
+ ProblemFilters.exclude[ReversedMissingMethodProblem]("org.apache.spark.ml.classification.LogisticRegressionSummary.weightCol"),
Review comment:
I'm thinking through what if anything breaks here in user code. Are all the subclasses sealed traits? so could any user code have subclassed the old traits in a way that would then not compile now? I don't think so but I'm just going off reading the diff
##########
File path: project/MimaExcludes.scala
##########
@@ -49,7 +49,34 @@ object MimaExcludes {
//[SPARK-31840] Add instance weight support in LogisticRegressionSummary
// weightCol in org.apache.spark.ml.classification.LogisticRegressionSummary is present only in current version
- ProblemFilters.exclude[ReversedMissingMethodProblem]("org.apache.spark.ml.classification.LogisticRegressionSummary.weightCol")
+ ProblemFilters.exclude[ReversedMissingMethodProblem]("org.apache.spark.ml.classification.LogisticRegressionSummary.weightCol"),
Review comment:
Same general question on the Python side; I'm not as sure about the possibility of breaking something there
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #28710:
URL: https://github.com/apache/spark/pull/28710#issuecomment-645773554
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #28710:
URL: https://github.com/apache/spark/pull/28710#issuecomment-645836180
**[Test build #124206 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124206/testReport)** for PR 28710 at commit [`c76d591`](https://github.com/apache/spark/commit/c76d5917b9a491213d7d6f4fd4be6793a6d07108).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] huaxingao commented on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
huaxingao commented on pull request #28710:
URL: https://github.com/apache/spark/pull/28710#issuecomment-638460384
I put probabilityCol back in the subclass and this should fix the test failure I had yesterday :)
For the multiclass classification or binary classification metrics we need, seems to me that either one of probability or raw prediction is good enough.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] zhengruifeng commented on a change in pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
zhengruifeng commented on a change in pull request #28710:
URL: https://github.com/apache/spark/pull/28710#discussion_r437111620
##########
File path: mllib/src/main/scala/org/apache/spark/ml/classification/ClassificationSummary.scala
##########
@@ -0,0 +1,265 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.spark.ml.classification
+
+import org.apache.spark.annotation.Since
+import org.apache.spark.ml.functions.checkNonNegativeWeight
+import org.apache.spark.ml.linalg.Vector
+import org.apache.spark.mllib.evaluation.{BinaryClassificationMetrics, MulticlassMetrics}
+import org.apache.spark.sql.{DataFrame, Row}
+import org.apache.spark.sql.functions.{col, lit}
+import org.apache.spark.sql.types.DoubleType
+
+
+/**
+ * Abstraction for multiclass classification results for a given model.
+ */
+private[classification] trait ClassificationSummary extends Serializable {
+
+ /**
+ * Dataframe output by the model's `transform` method.
+ */
+ @Since("3.1.0")
+ def predictions: DataFrame
+
+ /**
+ * Field in "predictions" which gives the probability or rawPrediction of each class as a vector.
+ */
+ @Since("3.1.0")
+ def scoreCol: String
Review comment:
Since `probabilityCol` can be added in subclasses, so can `scoreCol` be removed?
##########
File path: mllib/src/main/scala/org/apache/spark/ml/classification/ClassificationSummary.scala
##########
@@ -0,0 +1,265 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.spark.ml.classification
+
+import org.apache.spark.annotation.Since
+import org.apache.spark.ml.functions.checkNonNegativeWeight
+import org.apache.spark.ml.linalg.Vector
+import org.apache.spark.mllib.evaluation.{BinaryClassificationMetrics, MulticlassMetrics}
+import org.apache.spark.sql.{DataFrame, Row}
+import org.apache.spark.sql.functions.{col, lit}
+import org.apache.spark.sql.types.DoubleType
+
+
+/**
+ * Abstraction for multiclass classification results for a given model.
+ */
+private[classification] trait ClassificationSummary extends Serializable {
+
+ /**
+ * Dataframe output by the model's `transform` method.
+ */
+ @Since("3.1.0")
+ def predictions: DataFrame
+
+ /**
+ * Field in "predictions" which gives the probability or rawPrediction of each class as a vector.
+ */
+ @Since("3.1.0")
+ def scoreCol: String
+
+ /** Field in "predictions" which gives the prediction of each class. */
+ @Since("3.1.0")
+ def predictionCol: String
+
+ /** Field in "predictions" which gives the true label of each instance (if available). */
+ @Since("3.1.0")
+ def labelCol: String
+
+ /** Field in "predictions" which gives the features of each instance as a vector. */
+ @Since("3.1.0")
+ def featuresCol: String
+
+ /** Field in "predictions" which gives the weight of each instance as a vector. */
+ @Since("3.1.0")
+ def weightCol: String
+
+ @transient private val multiclassMetrics = {
+ if (predictions.schema.fieldNames.contains(weightCol)) {
+ new MulticlassMetrics(
+ predictions.select(
+ col(predictionCol),
+ col(labelCol).cast(DoubleType),
+ checkNonNegativeWeight(col(weightCol).cast(DoubleType))).rdd.map {
+ case Row(prediction: Double, label: Double, weight: Double) => (prediction, label, weight)
+ })
+ } else {
+ new MulticlassMetrics(
+ predictions.select(
+ col(predictionCol),
+ col(labelCol).cast(DoubleType),
+ lit(1.0)).rdd.map {
+ case Row(prediction: Double, label: Double, weight: Double) => (prediction, label, weight)
+ })
+ }
+ }
+
+ /**
+ * Returns the sequence of labels in ascending order. This order matches the order used
+ * in metrics which are specified as arrays over labels, e.g., truePositiveRateByLabel.
+ *
+ * Note: In most cases, it will be values {0.0, 1.0, ..., numClasses-1}, However, if the
+ * training set is missing a label, then all of the arrays over labels
+ * (e.g., from truePositiveRateByLabel) will be of length numClasses-1 instead of the
+ * expected numClasses.
+ */
+ @Since("3.1.0")
+ def labels: Array[Double] = multiclassMetrics.labels
+
+ /** Returns true positive rate for each label (category). */
+ @Since("3.1.0")
+ def truePositiveRateByLabel: Array[Double] = recallByLabel
+
+ /** Returns false positive rate for each label (category). */
+ @Since("3.1.0")
+ def falsePositiveRateByLabel: Array[Double] = {
+ multiclassMetrics.labels.map(label => multiclassMetrics.falsePositiveRate(label))
+ }
+
+ /** Returns precision for each label (category). */
+ @Since("3.1.0")
+ def precisionByLabel: Array[Double] = {
+ multiclassMetrics.labels.map(label => multiclassMetrics.precision(label))
+ }
+
+ /** Returns recall for each label (category). */
+ @Since("3.1.0")
+ def recallByLabel: Array[Double] = {
+ multiclassMetrics.labels.map(label => multiclassMetrics.recall(label))
+ }
+
+ /** Returns f-measure for each label (category). */
+ @Since("3.1.0")
+ def fMeasureByLabel(beta: Double): Array[Double] = {
+ multiclassMetrics.labels.map(label => multiclassMetrics.fMeasure(label, beta))
+ }
+
+ /** Returns f1-measure for each label (category). */
+ @Since("3.1.0")
+ def fMeasureByLabel: Array[Double] = fMeasureByLabel(1.0)
+
+ /**
+ * Returns accuracy.
+ * (equals to the total number of correctly classified instances
+ * out of the total number of instances.)
+ */
+ @Since("3.1.0")
+ def accuracy: Double = multiclassMetrics.accuracy
+
+ /**
+ * Returns weighted true positive rate.
+ * (equals to precision, recall and f-measure)
+ */
+ @Since("3.1.0")
+ def weightedTruePositiveRate: Double = weightedRecall
+
+ /** Returns weighted false positive rate. */
+ @Since("3.1.0")
+ def weightedFalsePositiveRate: Double = multiclassMetrics.weightedFalsePositiveRate
+
+ /**
+ * Returns weighted averaged recall.
+ * (equals to precision, recall and f-measure)
+ */
+ @Since("3.1.0")
+ def weightedRecall: Double = multiclassMetrics.weightedRecall
+
+ /** Returns weighted averaged precision. */
+ @Since("3.1.0")
+ def weightedPrecision: Double = multiclassMetrics.weightedPrecision
+
+ /** Returns weighted averaged f-measure. */
+ @Since("3.1.0")
+ def weightedFMeasure(beta: Double): Double = multiclassMetrics.weightedFMeasure(beta)
+
+ /** Returns weighted averaged f1-measure. */
+ @Since("3.1.0")
+ def weightedFMeasure: Double = multiclassMetrics.weightedFMeasure(1.0)
+
+ /**
+ * Convenient method for casting to binary classification summary.
+ * This method will throw an Exception if the summary is not a binary summary.
+ */
+ @Since("3.1.0")
+ def asBinary: BinaryClassificationSummary = this match {
+ case b: BinaryClassificationSummary => b
+ case _ =>
+ throw new RuntimeException("Cannot cast to a binary summary.")
+ }
+}
+
+/**
+ * Abstraction for training results.
+ */
+private[classification] trait TrainingSummary {
+
+ /** objective function (scaled loss + regularization) at each iteration. */
+ @Since("3.1.0")
+ def objectiveHistory: Array[Double]
+
+ /** Number of training iterations. */
+ @Since("3.1.0")
+ def totalIterations: Int = objectiveHistory.length
+}
+
+/**
+ * Abstraction for binary classification results for a given model.
+ */
+trait BinaryClassificationSummary extends ClassificationSummary {
+
+ private val sparkSession = predictions.sparkSession
+ import sparkSession.implicits._
+
+ // TODO: Allow the user to vary the number of bins using a setBins method in
+ // BinaryClassificationMetrics. For now the default is set to 100.
Review comment:
'For now the default is set to 100.' it is wrong, the default `numBins` in `BinaryClassificationMetrics` is always 1000.
##########
File path: mllib/src/main/scala/org/apache/spark/ml/classification/ClassificationSummary.scala
##########
@@ -0,0 +1,265 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.spark.ml.classification
+
+import org.apache.spark.annotation.Since
+import org.apache.spark.ml.functions.checkNonNegativeWeight
+import org.apache.spark.ml.linalg.Vector
+import org.apache.spark.mllib.evaluation.{BinaryClassificationMetrics, MulticlassMetrics}
+import org.apache.spark.sql.{DataFrame, Row}
+import org.apache.spark.sql.functions.{col, lit}
+import org.apache.spark.sql.types.DoubleType
+
+
+/**
+ * Abstraction for multiclass classification results for a given model.
+ */
+private[classification] trait ClassificationSummary extends Serializable {
+
+ /**
+ * Dataframe output by the model's `transform` method.
+ */
+ @Since("3.1.0")
+ def predictions: DataFrame
+
+ /**
+ * Field in "predictions" which gives the probability or rawPrediction of each class as a vector.
+ */
+ @Since("3.1.0")
+ def scoreCol: String
+
+ /** Field in "predictions" which gives the prediction of each class. */
+ @Since("3.1.0")
+ def predictionCol: String
+
+ /** Field in "predictions" which gives the true label of each instance (if available). */
+ @Since("3.1.0")
+ def labelCol: String
+
+ /** Field in "predictions" which gives the features of each instance as a vector. */
+ @Since("3.1.0")
+ def featuresCol: String
+
+ /** Field in "predictions" which gives the weight of each instance as a vector. */
+ @Since("3.1.0")
+ def weightCol: String
+
+ @transient private val multiclassMetrics = {
+ if (predictions.schema.fieldNames.contains(weightCol)) {
+ new MulticlassMetrics(
+ predictions.select(
+ col(predictionCol),
+ col(labelCol).cast(DoubleType),
+ checkNonNegativeWeight(col(weightCol).cast(DoubleType))).rdd.map {
+ case Row(prediction: Double, label: Double, weight: Double) => (prediction, label, weight)
+ })
+ } else {
+ new MulticlassMetrics(
+ predictions.select(
+ col(predictionCol),
+ col(labelCol).cast(DoubleType),
+ lit(1.0)).rdd.map {
+ case Row(prediction: Double, label: Double, weight: Double) => (prediction, label, weight)
+ })
+ }
+ }
+
+ /**
+ * Returns the sequence of labels in ascending order. This order matches the order used
+ * in metrics which are specified as arrays over labels, e.g., truePositiveRateByLabel.
+ *
+ * Note: In most cases, it will be values {0.0, 1.0, ..., numClasses-1}, However, if the
+ * training set is missing a label, then all of the arrays over labels
+ * (e.g., from truePositiveRateByLabel) will be of length numClasses-1 instead of the
+ * expected numClasses.
+ */
+ @Since("3.1.0")
+ def labels: Array[Double] = multiclassMetrics.labels
+
+ /** Returns true positive rate for each label (category). */
+ @Since("3.1.0")
+ def truePositiveRateByLabel: Array[Double] = recallByLabel
+
+ /** Returns false positive rate for each label (category). */
+ @Since("3.1.0")
+ def falsePositiveRateByLabel: Array[Double] = {
+ multiclassMetrics.labels.map(label => multiclassMetrics.falsePositiveRate(label))
+ }
+
+ /** Returns precision for each label (category). */
+ @Since("3.1.0")
+ def precisionByLabel: Array[Double] = {
+ multiclassMetrics.labels.map(label => multiclassMetrics.precision(label))
+ }
+
+ /** Returns recall for each label (category). */
+ @Since("3.1.0")
+ def recallByLabel: Array[Double] = {
+ multiclassMetrics.labels.map(label => multiclassMetrics.recall(label))
+ }
+
+ /** Returns f-measure for each label (category). */
+ @Since("3.1.0")
+ def fMeasureByLabel(beta: Double): Array[Double] = {
+ multiclassMetrics.labels.map(label => multiclassMetrics.fMeasure(label, beta))
+ }
+
+ /** Returns f1-measure for each label (category). */
+ @Since("3.1.0")
+ def fMeasureByLabel: Array[Double] = fMeasureByLabel(1.0)
+
+ /**
+ * Returns accuracy.
+ * (equals to the total number of correctly classified instances
+ * out of the total number of instances.)
+ */
+ @Since("3.1.0")
+ def accuracy: Double = multiclassMetrics.accuracy
+
+ /**
+ * Returns weighted true positive rate.
+ * (equals to precision, recall and f-measure)
+ */
+ @Since("3.1.0")
+ def weightedTruePositiveRate: Double = weightedRecall
+
+ /** Returns weighted false positive rate. */
+ @Since("3.1.0")
+ def weightedFalsePositiveRate: Double = multiclassMetrics.weightedFalsePositiveRate
+
+ /**
+ * Returns weighted averaged recall.
+ * (equals to precision, recall and f-measure)
+ */
+ @Since("3.1.0")
+ def weightedRecall: Double = multiclassMetrics.weightedRecall
+
+ /** Returns weighted averaged precision. */
+ @Since("3.1.0")
+ def weightedPrecision: Double = multiclassMetrics.weightedPrecision
+
+ /** Returns weighted averaged f-measure. */
+ @Since("3.1.0")
+ def weightedFMeasure(beta: Double): Double = multiclassMetrics.weightedFMeasure(beta)
+
+ /** Returns weighted averaged f1-measure. */
+ @Since("3.1.0")
+ def weightedFMeasure: Double = multiclassMetrics.weightedFMeasure(1.0)
+
+ /**
+ * Convenient method for casting to binary classification summary.
+ * This method will throw an Exception if the summary is not a binary summary.
+ */
+ @Since("3.1.0")
+ def asBinary: BinaryClassificationSummary = this match {
+ case b: BinaryClassificationSummary => b
+ case _ =>
+ throw new RuntimeException("Cannot cast to a binary summary.")
+ }
+}
+
+/**
+ * Abstraction for training results.
+ */
+private[classification] trait TrainingSummary {
+
+ /** objective function (scaled loss + regularization) at each iteration. */
+ @Since("3.1.0")
+ def objectiveHistory: Array[Double]
+
+ /** Number of training iterations. */
+ @Since("3.1.0")
+ def totalIterations: Int = objectiveHistory.length
+}
+
+/**
+ * Abstraction for binary classification results for a given model.
+ */
+trait BinaryClassificationSummary extends ClassificationSummary {
+
+ private val sparkSession = predictions.sparkSession
+ import sparkSession.implicits._
+
+ // TODO: Allow the user to vary the number of bins using a setBins method in
+ // BinaryClassificationMetrics. For now the default is set to 100.
+ @transient private val binaryMetrics = if (predictions.schema.fieldNames.contains(weightCol)) {
+ new BinaryClassificationMetrics(
+ predictions.select(col(scoreCol), col(labelCol).cast(DoubleType),
+ checkNonNegativeWeight(col(weightCol).cast(DoubleType))).rdd.map {
+ case Row(score: Vector, label: Double, weight: Double) => (score(1), label, weight)
+ }, 100
+ )
+ } else {
+ new BinaryClassificationMetrics(
+ predictions.select(col(scoreCol), col(labelCol).cast(DoubleType),
+ lit(1.0)).rdd.map {
+ case Row(score: Vector, label: Double, weight: Double) => (score(1), label, weight)
+ }, 100
Review comment:
ditto
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28710:
URL: https://github.com/apache/spark/pull/28710#issuecomment-644331092
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #28710:
URL: https://github.com/apache/spark/pull/28710#issuecomment-643891334
**[Test build #124021 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124021/testReport)** for PR 28710 at commit [`2e6f35c`](https://github.com/apache/spark/commit/2e6f35c8e31fe1cde1637b922673339bfeef65fe).
* This patch **fails Spark unit tests**.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #28710:
URL: https://github.com/apache/spark/pull/28710#issuecomment-644863451
**[Test build #124131 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124131/testReport)** for PR 28710 at commit [`61e93d0`](https://github.com/apache/spark/commit/61e93d0cab590975b5d4f4551edd43cac6b55ce2).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #28710:
URL: https://github.com/apache/spark/pull/28710#issuecomment-642360765
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #28710:
URL: https://github.com/apache/spark/pull/28710#issuecomment-643991785
**[Test build #124045 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124045/testReport)** for PR 28710 at commit [`0898708`](https://github.com/apache/spark/commit/08987080d624a86379f6ec77e65571db95f36e13).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28710:
URL: https://github.com/apache/spark/pull/28710#issuecomment-642306930
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #28710:
URL: https://github.com/apache/spark/pull/28710#issuecomment-644649807
**[Test build #124110 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124110/testReport)** for PR 28710 at commit [`61e93d0`](https://github.com/apache/spark/commit/61e93d0cab590975b5d4f4551edd43cac6b55ce2).
* This patch **fails Spark unit tests**.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28710:
URL: https://github.com/apache/spark/pull/28710#issuecomment-645836595
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #28710:
URL: https://github.com/apache/spark/pull/28710#issuecomment-639116348
**[Test build #123543 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123543/testReport)** for PR 28710 at commit [`f42fa8e`](https://github.com/apache/spark/commit/f42fa8e58d88dcee0107af4459103b4b5e9d9d18).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #28710:
URL: https://github.com/apache/spark/pull/28710#issuecomment-644527552
**[Test build #124093 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124093/testReport)** for PR 28710 at commit [`61e93d0`](https://github.com/apache/spark/commit/61e93d0cab590975b5d4f4551edd43cac6b55ce2).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28710:
URL: https://github.com/apache/spark/pull/28710#issuecomment-644842199
Merged build finished. Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #28710:
URL: https://github.com/apache/spark/pull/28710#issuecomment-637843539
**[Test build #123450 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123450/testReport)** for PR 28710 at commit [`91a121b`](https://github.com/apache/spark/commit/91a121bea6ac9803361e7a8d8f7327761a70cfb2).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #28710:
URL: https://github.com/apache/spark/pull/28710#issuecomment-644829409
**[Test build #124128 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124128/testReport)** for PR 28710 at commit [`61e93d0`](https://github.com/apache/spark/commit/61e93d0cab590975b5d4f4551edd43cac6b55ce2).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #28710:
URL: https://github.com/apache/spark/pull/28710#issuecomment-644329988
**[Test build #124065 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124065/testReport)** for PR 28710 at commit [`137e5b2`](https://github.com/apache/spark/commit/137e5b20790f6e0c61e6f64fdcf47b2608e198f3).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #28710:
URL: https://github.com/apache/spark/pull/28710#issuecomment-644842182
**[Test build #124129 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124129/testReport)** for PR 28710 at commit [`61e93d0`](https://github.com/apache/spark/commit/61e93d0cab590975b5d4f4551edd43cac6b55ce2).
* This patch **fails build dependency tests**.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #28710:
URL: https://github.com/apache/spark/pull/28710#issuecomment-644962315
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28710:
URL: https://github.com/apache/spark/pull/28710#issuecomment-637886707
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/123450/
Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #28710:
URL: https://github.com/apache/spark/pull/28710#issuecomment-645836180
**[Test build #124206 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124206/testReport)** for PR 28710 at commit [`c76d591`](https://github.com/apache/spark/commit/c76d5917b9a491213d7d6f4fd4be6793a6d07108).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28710:
URL: https://github.com/apache/spark/pull/28710#issuecomment-643891538
Merged build finished. Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #28710:
URL: https://github.com/apache/spark/pull/28710#issuecomment-644839635
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] huaxingao commented on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
huaxingao commented on pull request #28710:
URL: https://github.com/apache/spark/pull/28710#issuecomment-644862497
retest this please
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #28710:
URL: https://github.com/apache/spark/pull/28710#issuecomment-644831728
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] huaxingao commented on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
huaxingao commented on pull request #28710:
URL: https://github.com/apache/spark/pull/28710#issuecomment-642907472
@zhengruifeng Thanks for your comments.
I think the traits should be like this (not including weightCol etc. for simplicity):
```
trait ClassificationSummary {
def predictionCol: String
def labelCol: String
val multiclassMetrics =
new MulticlassMetrics(
predictions.select(col(predictionCol), col(labelCol).cast(DoubleType))
. . . . . .
)
trait BinaryClassificationSummary extends ClassificationSummary
def rawPredictionCol: String
val binaryMetrics =
new BinaryClassificationMetrics(
predictions.select(col(rawPredictionCol), col(labelCol).cast(DoubleType))
. . . . . .
)
```
However, currently BinaryLogisticRegressionSummary uses probabilityCol for BinaryClassificationMetrics and this probabilityCol is in LogisticRegressionSummary instead of BinaryLogisticRegressionSummary. In order not to break the existing code, I need to make several changes for the above traits
1. change rawPredictionCol to scoreCol
can't use rawPredictionCol since currently LogisticRegression uses probabilityCol
can't use probabilityCol since LinearSVC doesn't have probabilityCol
2. put scoreCol in ClassificationSummary (since currently probabilityCol is in LogisticRegressionSummary instead of BinaryLogisticRegressionSummary)
that's how I get the current traits as following:
```
trait ClassificationSummary {
def scoreCol: String
def predictionCol: String
def labelCol: String
val multiclassMetrics =
new MulticlassMetrics(
predictions.select(col(predictionCol), col(labelCol).cast(DoubleType))
. . . . . .
)
trait BinaryClassificationSummary extends ClassificationSummary
val binaryMetrics =
new BinaryClassificationMetrics(
predictions.select(col(scoreCol), col(labelCol).cast(DoubleType))
. . . . . .
)
```
To implement summary for other classifiers:
```
LinearSVCSummary extends BinaryClassificationSummary // use rawPredicationCol
FMClassifierSummary extends BinaryClassificationSummary // use ProbabilityCol
```
For RandomForestClassifer (also for DecisionTreeClassifier and GBTClassiifer)
```
RandomForestSummary extends ClassificationSummary
BinaryRandomForestSummary extends BinaryClassificationSummary // use ProbabilityCol
if (numOfClass == 2)
summary = BinaryRandomForestSummary
else
sumary = RandomForestSummary
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28710:
URL: https://github.com/apache/spark/pull/28710#issuecomment-644864140
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28710:
URL: https://github.com/apache/spark/pull/28710#issuecomment-644525654
Merged build finished. Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] zhengruifeng commented on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
zhengruifeng commented on pull request #28710:
URL: https://github.com/apache/spark/pull/28710#issuecomment-642492980
`MulticlassMetrics` needs "prediction, label, weight(optional), probability (optional, only for logLoss)", `probability` can be ignored for now;
`BinaryClassificationMetrics` needs "score, label, weight", here score is the `probability` of class 1 in `BinaryLogisticRegressionSummary`;
So I guess we can add a `BinaryClassificationSummary` to support LinearSVC, then `scoreCol` seems no longer needed, several break changes maybe avoided.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] huaxingao commented on a change in pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
huaxingao commented on a change in pull request #28710:
URL: https://github.com/apache/spark/pull/28710#discussion_r436790826
##########
File path: project/MimaExcludes.scala
##########
@@ -49,7 +49,34 @@ object MimaExcludes {
//[SPARK-31840] Add instance weight support in LogisticRegressionSummary
// weightCol in org.apache.spark.ml.classification.LogisticRegressionSummary is present only in current version
- ProblemFilters.exclude[ReversedMissingMethodProblem]("org.apache.spark.ml.classification.LogisticRegressionSummary.weightCol")
+ ProblemFilters.exclude[ReversedMissingMethodProblem]("org.apache.spark.ml.classification.LogisticRegressionSummary.weightCol"),
+
+ //[SPARK-31893] Add a generic ClassificationSummary trait
+ ProblemFilters.exclude[InheritedNewAbstractMethodProblem]("org.apache.spark.ml.classification.LogisticRegressionTrainingSummary.org$apache$spark$ml$classification$ClassificationSummary$_setter_$org$apache$spark$ml$classification$ClassificationSummary$$multiclassMetrics_="),
+ ProblemFilters.exclude[InheritedNewAbstractMethodProblem]("org.apache.spark.ml.classification.LogisticRegressionTrainingSummary.org$apache$spark$ml$classification$ClassificationSummary$$multiclassMetrics"),
+ ProblemFilters.exclude[InheritedNewAbstractMethodProblem]("org.apache.spark.ml.classification.LogisticRegressionTrainingSummary.weightCol"),
+ ProblemFilters.exclude[InheritedNewAbstractMethodProblem]("org.apache.spark.ml.classification.BinaryLogisticRegressionTrainingSummary.org$apache$spark$ml$classification$BinaryClassificationSummary$_setter_$org$apache$spark$ml$classification$BinaryClassificationSummary$$sparkSession_="),
+ ProblemFilters.exclude[InheritedNewAbstractMethodProblem]("org.apache.spark.ml.classification.BinaryLogisticRegressionTrainingSummary.org$apache$spark$ml$classification$BinaryClassificationSummary$_setter_$org$apache$spark$ml$classification$BinaryClassificationSummary$$binaryMetrics_="),
+ ProblemFilters.exclude[InheritedNewAbstractMethodProblem]("org.apache.spark.ml.classification.BinaryLogisticRegressionTrainingSummary.org$apache$spark$ml$classification$BinaryClassificationSummary$$binaryMetrics"),
+ ProblemFilters.exclude[InheritedNewAbstractMethodProblem]("org.apache.spark.ml.classification.BinaryLogisticRegressionTrainingSummary.scoreCol"),
+ ProblemFilters.exclude[InheritedNewAbstractMethodProblem]("org.apache.spark.ml.classification.BinaryLogisticRegressionTrainingSummary.org$apache$spark$ml$classification$BinaryClassificationSummary$$sparkSession"),
+ ProblemFilters.exclude[InheritedNewAbstractMethodProblem]("org.apache.spark.ml.classification.BinaryLogisticRegressionTrainingSummary.org$apache$spark$ml$classification$ClassificationSummary$_setter_$org$apache$spark$ml$classification$ClassificationSummary$$multiclassMetrics_="),
+ ProblemFilters.exclude[InheritedNewAbstractMethodProblem]("org.apache.spark.ml.classification.BinaryLogisticRegressionTrainingSummary.org$apache$spark$ml$classification$ClassificationSummary$$multiclassMetrics"),
+ ProblemFilters.exclude[InheritedNewAbstractMethodProblem]("org.apache.spark.ml.classification.BinaryLogisticRegressionTrainingSummary.weightCol"),
+ ProblemFilters.exclude[IncompatibleResultTypeProblem]("org.apache.spark.ml.classification.LogisticRegressionSummary.asBinary"),
Review comment:
@viirya Thanks for your comment. I think we are still OK, because summary is still an instance of BinaryLogisticRegressionSummary
```
val blorModel = lr.fit(smallBinaryDataset)
assert(blorModel.summary.isInstanceOf[BinaryLogisticRegressionTrainingSummary])
assert(blorModel.summary.asBinary.isInstanceOf[BinaryLogisticRegressionSummary])
assert(blorModel.binarySummary.isInstanceOf[BinaryLogisticRegressionTrainingSummary])
```
Similarly, in multinomial case, summary is still an instance of LogisticRegressionTrainingSummary
```
val mlorModel = lr.setFamily("multinomial").fit(smallMultinomialDataset)
assert(mlorModel.summary.isInstanceOf[LogisticRegressionTrainingSummary])
```
##########
File path: mllib/src/main/scala/org/apache/spark/ml/classification/ClassificationSummary.scala
##########
@@ -0,0 +1,265 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.spark.ml.classification
+
+import org.apache.spark.annotation.Since
+import org.apache.spark.ml.functions.checkNonNegativeWeight
+import org.apache.spark.ml.linalg.Vector
+import org.apache.spark.mllib.evaluation.{BinaryClassificationMetrics, MulticlassMetrics}
+import org.apache.spark.sql.{DataFrame, Row}
+import org.apache.spark.sql.functions.{col, lit}
+import org.apache.spark.sql.types.DoubleType
+
+
+/**
+ * Abstraction for multiclass classification results for a given model.
+ */
+private[classification] trait ClassificationSummary extends Serializable {
+
+ /**
+ * Dataframe output by the model's `transform` method.
+ */
+ @Since("3.1.0")
+ def predictions: DataFrame
+
+ /**
+ * Field in "predictions" which gives the probability or rawPrediction of each class as a vector.
+ */
+ @Since("3.1.0")
+ def scoreCol: String
+
+ /** Field in "predictions" which gives the prediction of each class. */
+ @Since("3.1.0")
+ def predictionCol: String
+
+ /** Field in "predictions" which gives the true label of each instance (if available). */
+ @Since("3.1.0")
+ def labelCol: String
+
+ /** Field in "predictions" which gives the features of each instance as a vector. */
+ @Since("3.1.0")
+ def featuresCol: String
+
+ /** Field in "predictions" which gives the weight of each instance as a vector. */
+ @Since("3.1.0")
+ def weightCol: String
+
+ @transient private val multiclassMetrics = {
+ if (predictions.schema.fieldNames.contains(weightCol)) {
+ new MulticlassMetrics(
+ predictions.select(
+ col(predictionCol),
+ col(labelCol).cast(DoubleType),
+ checkNonNegativeWeight(col(weightCol).cast(DoubleType))).rdd.map {
+ case Row(prediction: Double, label: Double, weight: Double) => (prediction, label, weight)
+ })
+ } else {
+ new MulticlassMetrics(
+ predictions.select(
+ col(predictionCol),
+ col(labelCol).cast(DoubleType),
+ lit(1.0)).rdd.map {
+ case Row(prediction: Double, label: Double, weight: Double) => (prediction, label, weight)
+ })
+ }
+ }
+
+ /**
+ * Returns the sequence of labels in ascending order. This order matches the order used
+ * in metrics which are specified as arrays over labels, e.g., truePositiveRateByLabel.
+ *
+ * Note: In most cases, it will be values {0.0, 1.0, ..., numClasses-1}, However, if the
+ * training set is missing a label, then all of the arrays over labels
+ * (e.g., from truePositiveRateByLabel) will be of length numClasses-1 instead of the
+ * expected numClasses.
+ */
+ @Since("3.1.0")
+ def labels: Array[Double] = multiclassMetrics.labels
+
+ /** Returns true positive rate for each label (category). */
+ @Since("3.1.0")
+ def truePositiveRateByLabel: Array[Double] = recallByLabel
+
+ /** Returns false positive rate for each label (category). */
+ @Since("3.1.0")
+ def falsePositiveRateByLabel: Array[Double] = {
+ multiclassMetrics.labels.map(label => multiclassMetrics.falsePositiveRate(label))
+ }
+
+ /** Returns precision for each label (category). */
+ @Since("3.1.0")
+ def precisionByLabel: Array[Double] = {
+ multiclassMetrics.labels.map(label => multiclassMetrics.precision(label))
+ }
+
+ /** Returns recall for each label (category). */
+ @Since("3.1.0")
+ def recallByLabel: Array[Double] = {
+ multiclassMetrics.labels.map(label => multiclassMetrics.recall(label))
+ }
+
+ /** Returns f-measure for each label (category). */
+ @Since("3.1.0")
+ def fMeasureByLabel(beta: Double): Array[Double] = {
+ multiclassMetrics.labels.map(label => multiclassMetrics.fMeasure(label, beta))
+ }
+
+ /** Returns f1-measure for each label (category). */
+ @Since("3.1.0")
+ def fMeasureByLabel: Array[Double] = fMeasureByLabel(1.0)
+
+ /**
+ * Returns accuracy.
+ * (equals to the total number of correctly classified instances
+ * out of the total number of instances.)
+ */
+ @Since("3.1.0")
+ def accuracy: Double = multiclassMetrics.accuracy
+
+ /**
+ * Returns weighted true positive rate.
+ * (equals to precision, recall and f-measure)
+ */
+ @Since("3.1.0")
+ def weightedTruePositiveRate: Double = weightedRecall
+
+ /** Returns weighted false positive rate. */
+ @Since("3.1.0")
+ def weightedFalsePositiveRate: Double = multiclassMetrics.weightedFalsePositiveRate
+
+ /**
+ * Returns weighted averaged recall.
+ * (equals to precision, recall and f-measure)
+ */
+ @Since("3.1.0")
+ def weightedRecall: Double = multiclassMetrics.weightedRecall
+
+ /** Returns weighted averaged precision. */
+ @Since("3.1.0")
+ def weightedPrecision: Double = multiclassMetrics.weightedPrecision
+
+ /** Returns weighted averaged f-measure. */
+ @Since("3.1.0")
+ def weightedFMeasure(beta: Double): Double = multiclassMetrics.weightedFMeasure(beta)
+
+ /** Returns weighted averaged f1-measure. */
+ @Since("3.1.0")
+ def weightedFMeasure: Double = multiclassMetrics.weightedFMeasure(1.0)
+
+ /**
+ * Convenient method for casting to binary classification summary.
+ * This method will throw an Exception if the summary is not a binary summary.
+ */
+ @Since("3.1.0")
+ def asBinary: BinaryClassificationSummary = this match {
+ case b: BinaryClassificationSummary => b
+ case _ =>
+ throw new RuntimeException("Cannot cast to a binary summary.")
+ }
+}
+
+/**
+ * Abstraction for training results.
+ */
+private[classification] trait TrainingSummary {
+
+ /** objective function (scaled loss + regularization) at each iteration. */
+ @Since("3.1.0")
+ def objectiveHistory: Array[Double]
+
+ /** Number of training iterations. */
+ @Since("3.1.0")
+ def totalIterations: Int = objectiveHistory.length
+}
+
+/**
+ * Abstraction for binary classification results for a given model.
+ */
+trait BinaryClassificationSummary extends ClassificationSummary {
+
+ private val sparkSession = predictions.sparkSession
+ import sparkSession.implicits._
+
+ // TODO: Allow the user to vary the number of bins using a setBins method in
+ // BinaryClassificationMetrics. For now the default is set to 100.
Review comment:
This piece of code was used in ```LogisticRegression``` and I moved it here. I'm not really sure what the comment means, but it might mean "Allow the user to vary the number of bins using a setBins method in BinaryClassificationMetrics. For now set it to 100."
##########
File path: mllib/src/main/scala/org/apache/spark/ml/classification/ClassificationSummary.scala
##########
@@ -0,0 +1,265 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.spark.ml.classification
+
+import org.apache.spark.annotation.Since
+import org.apache.spark.ml.functions.checkNonNegativeWeight
+import org.apache.spark.ml.linalg.Vector
+import org.apache.spark.mllib.evaluation.{BinaryClassificationMetrics, MulticlassMetrics}
+import org.apache.spark.sql.{DataFrame, Row}
+import org.apache.spark.sql.functions.{col, lit}
+import org.apache.spark.sql.types.DoubleType
+
+
+/**
+ * Abstraction for multiclass classification results for a given model.
+ */
+private[classification] trait ClassificationSummary extends Serializable {
+
+ /**
+ * Dataframe output by the model's `transform` method.
+ */
+ @Since("3.1.0")
+ def predictions: DataFrame
+
+ /**
+ * Field in "predictions" which gives the probability or rawPrediction of each class as a vector.
+ */
+ @Since("3.1.0")
+ def scoreCol: String
Review comment:
still need to keep ```scoreCol``` in this generic trait because it is needed in ```BinaryClassificationSummary```
```
new BinaryClassificationMetrics(
predictions.select(col(scoreCol), col(labelCol).cast(DoubleType),
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #28710:
URL: https://github.com/apache/spark/pull/28710#issuecomment-644831716
**[Test build #124128 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124128/testReport)** for PR 28710 at commit [`61e93d0`](https://github.com/apache/spark/commit/61e93d0cab590975b5d4f4551edd43cac6b55ce2).
* This patch **fails build dependency tests**.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #28710:
URL: https://github.com/apache/spark/pull/28710#issuecomment-638534698
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #28710:
URL: https://github.com/apache/spark/pull/28710#issuecomment-639187095
**[Test build #123545 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123545/testReport)** for PR 28710 at commit [`7b01b63`](https://github.com/apache/spark/commit/7b01b63f9ce6549eaf248296b0d48e98a2dd7a25).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] zhengruifeng commented on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
zhengruifeng commented on pull request #28710:
URL: https://github.com/apache/spark/pull/28710#issuecomment-644577984
retest this please
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #28710:
URL: https://github.com/apache/spark/pull/28710#issuecomment-639134384
**[Test build #123545 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123545/testReport)** for PR 28710 at commit [`7b01b63`](https://github.com/apache/spark/commit/7b01b63f9ce6549eaf248296b0d48e98a2dd7a25).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #28710:
URL: https://github.com/apache/spark/pull/28710#issuecomment-645773554
**[Test build #124197 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124197/testReport)** for PR 28710 at commit [`e844c11`](https://github.com/apache/spark/commit/e844c11939d5523d5d7402f5ee5ed6cabd52f8fb).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #28710:
URL: https://github.com/apache/spark/pull/28710#issuecomment-644579440
**[Test build #124110 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124110/testReport)** for PR 28710 at commit [`61e93d0`](https://github.com/apache/spark/commit/61e93d0cab590975b5d4f4551edd43cac6b55ce2).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #28710:
URL: https://github.com/apache/spark/pull/28710#issuecomment-644863451
**[Test build #124131 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124131/testReport)** for PR 28710 at commit [`61e93d0`](https://github.com/apache/spark/commit/61e93d0cab590975b5d4f4551edd43cac6b55ce2).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #28710:
URL: https://github.com/apache/spark/pull/28710#issuecomment-645773814
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28710:
URL: https://github.com/apache/spark/pull/28710#issuecomment-644525663
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/124082/
Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28710:
URL: https://github.com/apache/spark/pull/28710#issuecomment-644527823
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #28710:
URL: https://github.com/apache/spark/pull/28710#issuecomment-644525654
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28710:
URL: https://github.com/apache/spark/pull/28710#issuecomment-645823660
Merged build finished. Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #28710:
URL: https://github.com/apache/spark/pull/28710#issuecomment-644579440
**[Test build #124110 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124110/testReport)** for PR 28710 at commit [`61e93d0`](https://github.com/apache/spark/commit/61e93d0cab590975b5d4f4551edd43cac6b55ce2).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #28710:
URL: https://github.com/apache/spark/pull/28710#issuecomment-644580203
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28710:
URL: https://github.com/apache/spark/pull/28710#issuecomment-644077050
Merged build finished. Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28710:
URL: https://github.com/apache/spark/pull/28710#issuecomment-642360765
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #28710:
URL: https://github.com/apache/spark/pull/28710#issuecomment-639117310
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #28710:
URL: https://github.com/apache/spark/pull/28710#issuecomment-643897872
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #28710:
URL: https://github.com/apache/spark/pull/28710#issuecomment-644478474
**[Test build #124082 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124082/testReport)** for PR 28710 at commit [`61e93d0`](https://github.com/apache/spark/commit/61e93d0cab590975b5d4f4551edd43cac6b55ce2).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28710:
URL: https://github.com/apache/spark/pull/28710#issuecomment-644650499
Merged build finished. Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #28710:
URL: https://github.com/apache/spark/pull/28710#issuecomment-645921043
**[Test build #124206 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124206/testReport)** for PR 28710 at commit [`c76d591`](https://github.com/apache/spark/commit/c76d5917b9a491213d7d6f4fd4be6793a6d07108).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #28710:
URL: https://github.com/apache/spark/pull/28710#issuecomment-646185707
**[Test build #124227 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124227/testReport)** for PR 28710 at commit [`ead6da2`](https://github.com/apache/spark/commit/ead6da2e0e3d1a269e6ef76b23f8e845690f42a3).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] srowen commented on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
srowen commented on pull request #28710:
URL: https://github.com/apache/spark/pull/28710#issuecomment-646996446
Merged to master
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #28710:
URL: https://github.com/apache/spark/pull/28710#issuecomment-645776165
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28710:
URL: https://github.com/apache/spark/pull/28710#issuecomment-639117310
Merged build finished. Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #28710:
URL: https://github.com/apache/spark/pull/28710#issuecomment-645823227
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] viirya commented on a change in pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
viirya commented on a change in pull request #28710:
URL: https://github.com/apache/spark/pull/28710#discussion_r436488074
##########
File path: project/MimaExcludes.scala
##########
@@ -49,7 +49,34 @@ object MimaExcludes {
//[SPARK-31840] Add instance weight support in LogisticRegressionSummary
// weightCol in org.apache.spark.ml.classification.LogisticRegressionSummary is present only in current version
- ProblemFilters.exclude[ReversedMissingMethodProblem]("org.apache.spark.ml.classification.LogisticRegressionSummary.weightCol")
+ ProblemFilters.exclude[ReversedMissingMethodProblem]("org.apache.spark.ml.classification.LogisticRegressionSummary.weightCol"),
+
+ //[SPARK-31893] Add a generic ClassificationSummary trait
+ ProblemFilters.exclude[InheritedNewAbstractMethodProblem]("org.apache.spark.ml.classification.LogisticRegressionTrainingSummary.org$apache$spark$ml$classification$ClassificationSummary$_setter_$org$apache$spark$ml$classification$ClassificationSummary$$multiclassMetrics_="),
+ ProblemFilters.exclude[InheritedNewAbstractMethodProblem]("org.apache.spark.ml.classification.LogisticRegressionTrainingSummary.org$apache$spark$ml$classification$ClassificationSummary$$multiclassMetrics"),
+ ProblemFilters.exclude[InheritedNewAbstractMethodProblem]("org.apache.spark.ml.classification.LogisticRegressionTrainingSummary.weightCol"),
+ ProblemFilters.exclude[InheritedNewAbstractMethodProblem]("org.apache.spark.ml.classification.BinaryLogisticRegressionTrainingSummary.org$apache$spark$ml$classification$BinaryClassificationSummary$_setter_$org$apache$spark$ml$classification$BinaryClassificationSummary$$sparkSession_="),
+ ProblemFilters.exclude[InheritedNewAbstractMethodProblem]("org.apache.spark.ml.classification.BinaryLogisticRegressionTrainingSummary.org$apache$spark$ml$classification$BinaryClassificationSummary$_setter_$org$apache$spark$ml$classification$BinaryClassificationSummary$$binaryMetrics_="),
+ ProblemFilters.exclude[InheritedNewAbstractMethodProblem]("org.apache.spark.ml.classification.BinaryLogisticRegressionTrainingSummary.org$apache$spark$ml$classification$BinaryClassificationSummary$$binaryMetrics"),
+ ProblemFilters.exclude[InheritedNewAbstractMethodProblem]("org.apache.spark.ml.classification.BinaryLogisticRegressionTrainingSummary.scoreCol"),
+ ProblemFilters.exclude[InheritedNewAbstractMethodProblem]("org.apache.spark.ml.classification.BinaryLogisticRegressionTrainingSummary.org$apache$spark$ml$classification$BinaryClassificationSummary$$sparkSession"),
+ ProblemFilters.exclude[InheritedNewAbstractMethodProblem]("org.apache.spark.ml.classification.BinaryLogisticRegressionTrainingSummary.org$apache$spark$ml$classification$ClassificationSummary$_setter_$org$apache$spark$ml$classification$ClassificationSummary$$multiclassMetrics_="),
+ ProblemFilters.exclude[InheritedNewAbstractMethodProblem]("org.apache.spark.ml.classification.BinaryLogisticRegressionTrainingSummary.org$apache$spark$ml$classification$ClassificationSummary$$multiclassMetrics"),
+ ProblemFilters.exclude[InheritedNewAbstractMethodProblem]("org.apache.spark.ml.classification.BinaryLogisticRegressionTrainingSummary.weightCol"),
+ ProblemFilters.exclude[IncompatibleResultTypeProblem]("org.apache.spark.ml.classification.LogisticRegressionSummary.asBinary"),
Review comment:
Though they are sealed traits, could possibly the change of return type like this break in user code? Users get a `BinaryClassificationSummary` instead of `BinaryLogisticRegressionSummary`, for example.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #28710:
URL: https://github.com/apache/spark/pull/28710#issuecomment-644830265
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28710:
URL: https://github.com/apache/spark/pull/28710#issuecomment-645922053
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28710:
URL: https://github.com/apache/spark/pull/28710#issuecomment-638475706
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #28710:
URL: https://github.com/apache/spark/pull/28710#issuecomment-644076601
**[Test build #124045 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124045/testReport)** for PR 28710 at commit [`0898708`](https://github.com/apache/spark/commit/08987080d624a86379f6ec77e65571db95f36e13).
* This patch **fails Spark unit tests**.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28710:
URL: https://github.com/apache/spark/pull/28710#issuecomment-639117314
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/123543/
Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #28710:
URL: https://github.com/apache/spark/pull/28710#issuecomment-638462433
**[Test build #123505 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123505/testReport)** for PR 28710 at commit [`a4b2891`](https://github.com/apache/spark/commit/a4b2891573a5fb2b13b34042c3a58bbfbf019ee1).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #28710:
URL: https://github.com/apache/spark/pull/28710#issuecomment-644524536
**[Test build #124082 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124082/testReport)** for PR 28710 at commit [`61e93d0`](https://github.com/apache/spark/commit/61e93d0cab590975b5d4f4551edd43cac6b55ce2).
* This patch **fails PySpark unit tests**.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28710:
URL: https://github.com/apache/spark/pull/28710#issuecomment-644831740
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/124128/
Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #28710:
URL: https://github.com/apache/spark/pull/28710#issuecomment-646270163
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #28710:
URL: https://github.com/apache/spark/pull/28710#issuecomment-645775862
**[Test build #124198 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124198/testReport)** for PR 28710 at commit [`e844c11`](https://github.com/apache/spark/commit/e844c11939d5523d5d7402f5ee5ed6cabd52f8fb).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] huaxingao commented on a change in pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
huaxingao commented on a change in pull request #28710:
URL: https://github.com/apache/spark/pull/28710#discussion_r442378549
##########
File path: python/pyspark/ml/classification.py
##########
@@ -990,241 +1218,26 @@ def featuresCol(self):
"""
return self._call_java("featuresCol")
- @property
- @since("3.1.0")
- def weightCol(self):
- """
- Field in "predictions" which gives the weight of each instance
- as a vector.
- """
- return self._call_java("weightCol")
-
- @property
- @since("2.3.0")
- def labels(self):
- """
- Returns the sequence of labels in ascending order. This order matches the order used
- in metrics which are specified as arrays over labels, e.g., truePositiveRateByLabel.
-
- Note: In most cases, it will be values {0.0, 1.0, ..., numClasses-1}, However, if the
- training set is missing a label, then all of the arrays over labels
- (e.g., from truePositiveRateByLabel) will be of length numClasses-1 instead of the
- expected numClasses.
- """
- return self._call_java("labels")
-
- @property
- @since("2.3.0")
- def truePositiveRateByLabel(self):
- """
- Returns true positive rate for each label (category).
- """
- return self._call_java("truePositiveRateByLabel")
-
- @property
- @since("2.3.0")
- def falsePositiveRateByLabel(self):
- """
- Returns false positive rate for each label (category).
- """
- return self._call_java("falsePositiveRateByLabel")
-
- @property
- @since("2.3.0")
- def precisionByLabel(self):
- """
- Returns precision for each label (category).
- """
- return self._call_java("precisionByLabel")
-
- @property
- @since("2.3.0")
- def recallByLabel(self):
- """
- Returns recall for each label (category).
- """
- return self._call_java("recallByLabel")
-
- @since("2.3.0")
- def fMeasureByLabel(self, beta=1.0):
- """
- Returns f-measure for each label (category).
- """
- return self._call_java("fMeasureByLabel", beta)
-
- @property
- @since("2.3.0")
- def accuracy(self):
- """
- Returns accuracy.
- (equals to the total number of correctly classified instances
- out of the total number of instances.)
- """
- return self._call_java("accuracy")
-
- @property
- @since("2.3.0")
- def weightedTruePositiveRate(self):
- """
- Returns weighted true positive rate.
- (equals to precision, recall and f-measure)
- """
- return self._call_java("weightedTruePositiveRate")
-
- @property
- @since("2.3.0")
- def weightedFalsePositiveRate(self):
- """
- Returns weighted false positive rate.
- """
- return self._call_java("weightedFalsePositiveRate")
-
- @property
- @since("2.3.0")
- def weightedRecall(self):
- """
- Returns weighted averaged recall.
- (equals to precision, recall and f-measure)
- """
- return self._call_java("weightedRecall")
-
- @property
- @since("2.3.0")
- def weightedPrecision(self):
- """
- Returns weighted averaged precision.
- """
- return self._call_java("weightedPrecision")
-
- @since("2.3.0")
- def weightedFMeasure(self, beta=1.0):
- """
- Returns weighted averaged f-measure.
- """
- return self._call_java("weightedFMeasure", beta)
-
@inherit_doc
-class LogisticRegressionTrainingSummary(LogisticRegressionSummary):
+class LogisticRegressionTrainingSummary(LogisticRegressionSummary, _TrainingSummary):
"""
Abstraction for multinomial Logistic Regression Training results.
- Currently, the training summary ignores the training weights except
- for the objective trace.
.. versionadded:: 2.0.0
"""
-
- @property
- @since("2.0.0")
- def objectiveHistory(self):
- """
- Objective function (scaled loss + regularization) at each
- iteration. It contains one more element, the initial state,
- than number of iterations.
- """
- return self._call_java("objectiveHistory")
-
- @property
- @since("2.0.0")
- def totalIterations(self):
- """
- Number of training iterations until termination.
- """
- return self._call_java("totalIterations")
+ pass
@inherit_doc
-class BinaryLogisticRegressionSummary(LogisticRegressionSummary):
+class BinaryLogisticRegressionSummary(_BinaryClassificationSummary,
+ LogisticRegressionSummary):
"""
Binary Logistic regression results for a given model.
.. versionadded:: 2.0.0
"""
-
- @property
- @since("2.0.0")
- def roc(self):
- """
- Returns the receiver operating characteristic (ROC) curve,
- which is a Dataframe having two fields (FPR, TPR) with
- (0.0, 0.0) prepended and (1.0, 1.0) appended to it.
-
- .. seealso:: `Wikipedia reference
- <http://en.wikipedia.org/wiki/Receiver_operating_characteristic>`_
-
- .. note:: This ignores instance weights (setting all to 1.0) from
- `LogisticRegression.weightCol`. This will change in later Spark
- versions.
- """
- return self._call_java("roc")
-
- @property
- @since("2.0.0")
- def areaUnderROC(self):
- """
- Computes the area under the receiver operating characteristic
- (ROC) curve.
-
- .. note:: This ignores instance weights (setting all to 1.0) from
- `LogisticRegression.weightCol`. This will change in later Spark
- versions.
- """
- return self._call_java("areaUnderROC")
-
- @property
- @since("2.0.0")
- def pr(self):
- """
- Returns the precision-recall curve, which is a Dataframe
- containing two fields recall, precision with (0.0, 1.0) prepended
- to it.
-
- .. note:: This ignores instance weights (setting all to 1.0) from
- `LogisticRegression.weightCol`. This will change in later Spark
- versions.
- """
- return self._call_java("pr")
-
- @property
- @since("2.0.0")
- def fMeasureByThreshold(self):
- """
- Returns a dataframe with two fields (threshold, F-Measure) curve
- with beta = 1.0.
-
- .. note:: This ignores instance weights (setting all to 1.0) from
- `LogisticRegression.weightCol`. This will change in later Spark
- versions.
- """
- return self._call_java("fMeasureByThreshold")
-
- @property
- @since("2.0.0")
- def precisionByThreshold(self):
- """
- Returns a dataframe with two fields (threshold, precision) curve.
- Every possible probability obtained in transforming the dataset
- are used as thresholds used in calculating the precision.
-
- .. note:: This ignores instance weights (setting all to 1.0) from
- `LogisticRegression.weightCol`. This will change in later Spark
- versions.
- """
- return self._call_java("precisionByThreshold")
-
- @property
- @since("2.0.0")
- def recallByThreshold(self):
- """
- Returns a dataframe with two fields (threshold, recall) curve.
- Every possible probability obtained in transforming the dataset
- are used as thresholds used in calculating the recall.
-
- .. note:: This ignores instance weights (setting all to 1.0) from
- `LogisticRegression.weightCol`. This will change in later Spark
- versions.
Review comment:
Yes. I added weightCol in another PR, but forgot to update python comment.
##########
File path: project/MimaExcludes.scala
##########
@@ -39,18 +39,43 @@ object MimaExcludes {
// [SPARK-31077] Remove ChiSqSelector dependency on mllib.ChiSqSelectorModel
// private constructor
ProblemFilters.exclude[IncompatibleMethTypeProblem]("org.apache.spark.ml.feature.ChiSqSelectorModel.this"),
+
// [SPARK-31127] Implement abstract Selector
// org.apache.spark.ml.feature.ChiSqSelectorModel type hierarchy change
// before: class ChiSqSelector extends Estimator with ChiSqSelectorParams
// after: class ChiSqSelector extends PSelector
// false positive, no binary incompatibility
ProblemFilters.exclude[MissingTypesProblem]("org.apache.spark.ml.feature.ChiSqSelectorModel"),
ProblemFilters.exclude[MissingTypesProblem]("org.apache.spark.ml.feature.ChiSqSelector"),
+
//[SPARK-31840] Add instance weight support in LogisticRegressionSummary
// weightCol in org.apache.spark.ml.classification.LogisticRegressionSummary is present only in current version
ProblemFilters.exclude[ReversedMissingMethodProblem]("org.apache.spark.ml.classification.LogisticRegressionSummary.weightCol"),
+
// [SPARK-24634] Add a new metric regarding number of inputs later than watermark plus allowed delay
- ProblemFilters.exclude[DirectMissingMethodProblem]("org.apache.spark.sql.streaming.StateOperatorProgress.<init>$default$4")
+ ProblemFilters.exclude[DirectMissingMethodProblem]("org.apache.spark.sql.streaming.StateOperatorProgress.<init>$default$4"),
+
+ //[SPARK-31893] Add a generic ClassificationSummary trait
+ ProblemFilters.exclude[InheritedNewAbstractMethodProblem]("org.apache.spark.ml.classification.LogisticRegressionTrainingSummary.org$apache$spark$ml$classification$ClassificationSummary$_setter_$org$apache$spark$ml$classification$ClassificationSummary$$multiclassMetrics_="),
+ ProblemFilters.exclude[InheritedNewAbstractMethodProblem]("org.apache.spark.ml.classification.LogisticRegressionTrainingSummary.org$apache$spark$ml$classification$ClassificationSummary$$multiclassMetrics"),
+ ProblemFilters.exclude[InheritedNewAbstractMethodProblem]("org.apache.spark.ml.classification.LogisticRegressionTrainingSummary.weightCol"),
Review comment:
I added weightCol in ```LogisticRegressionSummary``` in another PR (I just realized the MiMaExclude for that PR is not needed any more since I moved weightCol to ```ClassificationSummary```)
3.0: no weightCol
after my changes:
```ClassificationSummary``` has weightCol, ```LogisticRegressionSummary```, ```LogisticRegressionTrainingSummary```, ```BinaryLogisticRegressionSummary```, ```BinaryLogisticRegressionTrainingSummary``` all have inherited weightCol, so there are four ```InheritedNewAbstractMethodProblem``` for weightCol
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #28710:
URL: https://github.com/apache/spark/pull/28710#issuecomment-644241625
**[Test build #124065 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124065/testReport)** for PR 28710 at commit [`137e5b2`](https://github.com/apache/spark/commit/137e5b20790f6e0c61e6f64fdcf47b2608e198f3).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] viirya commented on a change in pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
viirya commented on a change in pull request #28710:
URL: https://github.com/apache/spark/pull/28710#discussion_r442005207
##########
File path: mllib/src/main/scala/org/apache/spark/ml/classification/LogisticRegression.scala
##########
@@ -1540,101 +1417,21 @@ sealed trait LogisticRegressionSummary extends Serializable {
/**
* Abstraction for multiclass logistic regression training results.
- * Currently, the training summary ignores the training weights except
- * for the objective trace.
*/
-sealed trait LogisticRegressionTrainingSummary extends LogisticRegressionSummary {
-
- /**
- * objective function (scaled loss + regularization) at each iteration.
- * It contains one more element, the initial state, than number of iterations.
- */
- @Since("1.5.0")
- def objectiveHistory: Array[Double]
-
- /** Number of training iterations. */
- @Since("1.5.0")
- def totalIterations: Int = {
- assert(objectiveHistory.length > 0, s"objectiveHistory length should be greater than 1.")
- objectiveHistory.length - 1
- }
-
+sealed trait LogisticRegressionTrainingSummary extends LogisticRegressionSummary
+ with TrainingSummary {
}
/**
* Abstraction for binary logistic regression results for a given model.
*/
-sealed trait BinaryLogisticRegressionSummary extends LogisticRegressionSummary {
-
- private val sparkSession = predictions.sparkSession
- import sparkSession.implicits._
-
- // TODO: Allow the user to vary the number of bins using a setBins method in
- // BinaryClassificationMetrics. For now the default is set to 100.
- @transient private val binaryMetrics = if (predictions.schema.fieldNames.contains(weightCol)) {
- new BinaryClassificationMetrics(
- predictions.select(col(probabilityCol), col(labelCol).cast(DoubleType),
- checkNonNegativeWeight(col(weightCol).cast(DoubleType))).rdd.map {
- case Row(score: Vector, label: Double, weight: Double) => (score(1), label, weight)
- }, 100
- )
- } else {
- new BinaryClassificationMetrics(
- predictions.select(col(probabilityCol), col(labelCol).cast(DoubleType),
- lit(1.0)).rdd.map {
- case Row(score: Vector, label: Double, weight: Double) => (score(1), label, weight)
- }, 100
- )
- }
+sealed trait BinaryLogisticRegressionSummary extends LogisticRegressionSummary
+ with BinaryClassificationSummary {
- /**
- * Returns the receiver operating characteristic (ROC) curve,
- * which is a Dataframe having two fields (FPR, TPR)
- * with (0.0, 0.0) prepended and (1.0, 1.0) appended to it.
- * See http://en.wikipedia.org/wiki/Receiver_operating_characteristic
- */
- @Since("1.5.0")
- @transient lazy val roc: DataFrame = binaryMetrics.roc().toDF("FPR", "TPR")
-
- /**
- * Computes the area under the receiver operating characteristic (ROC) curve.
- */
- @Since("1.5.0")
- lazy val areaUnderROC: Double = binaryMetrics.areaUnderROC()
-
- /**
- * Returns the precision-recall curve, which is a Dataframe containing
- * two fields recall, precision with (0.0, 1.0) prepended to it.
- */
- @Since("1.5.0")
- @transient lazy val pr: DataFrame = binaryMetrics.pr().toDF("recall", "precision")
-
- /**
- * Returns a dataframe with two fields (threshold, F-Measure) curve with beta = 1.0.
- */
- @Since("1.5.0")
- @transient lazy val fMeasureByThreshold: DataFrame = {
- binaryMetrics.fMeasureByThreshold().toDF("threshold", "F-Measure")
- }
-
- /**
- * Returns a dataframe with two fields (threshold, precision) curve.
- * Every possible probability obtained in transforming the dataset are used
- * as thresholds used in calculating the precision.
- */
- @Since("1.5.0")
- @transient lazy val precisionByThreshold: DataFrame = {
- binaryMetrics.precisionByThreshold().toDF("threshold", "precision")
- }
-
- /**
- * Returns a dataframe with two fields (threshold, recall) curve.
- * Every possible probability obtained in transforming the dataset are used
- * as thresholds used in calculating the recall.
- */
- @Since("1.5.0")
- @transient lazy val recallByThreshold: DataFrame = {
- binaryMetrics.recallByThreshold().toDF("threshold", "recall")
+ override def scoreCol: String = if (probabilityCol.nonEmpty) {
+ probabilityCol
+ } else {
+ throw new SparkException(s"probabilityCol is required for BinaryLogisticRegressionSummary.")
Review comment:
s"" is not needed.
##########
File path: python/pyspark/ml/classification.py
##########
@@ -965,266 +1234,34 @@ def probabilityCol(self):
return self._call_java("probabilityCol")
@property
- @since("2.3.0")
- def predictionCol(self):
- """
- Field in "predictions" which gives the prediction of each class.
- """
- return self._call_java("predictionCol")
-
- @property
- @since("2.0.0")
- def labelCol(self):
- """
- Field in "predictions" which gives the true label of each
- instance.
- """
- return self._call_java("labelCol")
-
- @property
- @since("2.0.0")
+ @since("3.1.0")
Review comment:
Why change this since version? Like above `probabilityCol` is not changed?
##########
File path: mllib/src/main/scala/org/apache/spark/ml/classification/ClassificationSummary.scala
##########
@@ -0,0 +1,246 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.spark.ml.classification
+
+import org.apache.spark.annotation.Since
+import org.apache.spark.ml.linalg.Vector
+import org.apache.spark.mllib.evaluation.{BinaryClassificationMetrics, MulticlassMetrics}
+import org.apache.spark.sql.{DataFrame, Row}
+import org.apache.spark.sql.functions.{col, lit}
+import org.apache.spark.sql.types.DoubleType
+
+
+/**
+ * Abstraction for multiclass classification results for a given model.
+ */
+private[classification] trait ClassificationSummary extends Serializable {
+
+ /**
+ * Dataframe output by the model's `transform` method.
+ */
+ @Since("3.1.0")
+ def predictions: DataFrame
+
+ /** Field in "predictions" which gives the prediction of each class. */
+ @Since("3.1.0")
+ def predictionCol: String
+
+ /** Field in "predictions" which gives the true label of each instance (if available). */
+ @Since("3.1.0")
+ def labelCol: String
+
+ /** Field in "predictions" which gives the weight of each instance as a vector. */
+ @Since("3.1.0")
+ def weightCol: String
+
+ @transient private val multiclassMetrics = {
+ val weightColumn = if (predictions.schema.fieldNames.contains(weightCol)) {
+ col(weightCol).cast(DoubleType)
+ } else {
+ lit(1.0)
+ }
+ new MulticlassMetrics(
+ predictions.select(col(predictionCol), col(labelCol).cast(DoubleType), weightColumn)
+ .rdd.map {
+ case Row(prediction: Double, label: Double, weight: Double) => (prediction, label, weight)
+ })
+ }
+
+ /**
+ * Returns the sequence of labels in ascending order. This order matches the order used
+ * in metrics which are specified as arrays over labels, e.g., truePositiveRateByLabel.
+ *
+ * Note: In most cases, it will be values {0.0, 1.0, ..., numClasses-1}, However, if the
+ * training set is missing a label, then all of the arrays over labels
+ * (e.g., from truePositiveRateByLabel) will be of length numClasses-1 instead of the
+ * expected numClasses.
+ */
+ @Since("3.1.0")
+ def labels: Array[Double] = multiclassMetrics.labels
+
+ /** Returns true positive rate for each label (category). */
+ @Since("3.1.0")
+ def truePositiveRateByLabel: Array[Double] = recallByLabel
+
+ /** Returns false positive rate for each label (category). */
+ @Since("3.1.0")
+ def falsePositiveRateByLabel: Array[Double] = {
+ multiclassMetrics.labels.map(label => multiclassMetrics.falsePositiveRate(label))
+ }
+
+ /** Returns precision for each label (category). */
+ @Since("3.1.0")
+ def precisionByLabel: Array[Double] = {
+ multiclassMetrics.labels.map(label => multiclassMetrics.precision(label))
+ }
+
+ /** Returns recall for each label (category). */
+ @Since("3.1.0")
+ def recallByLabel: Array[Double] = {
+ multiclassMetrics.labels.map(label => multiclassMetrics.recall(label))
+ }
+
+ /** Returns f-measure for each label (category). */
+ @Since("3.1.0")
+ def fMeasureByLabel(beta: Double): Array[Double] = {
+ multiclassMetrics.labels.map(label => multiclassMetrics.fMeasure(label, beta))
+ }
+
+ /** Returns f1-measure for each label (category). */
+ @Since("3.1.0")
+ def fMeasureByLabel: Array[Double] = fMeasureByLabel(1.0)
+
+ /**
+ * Returns accuracy.
+ * (equals to the total number of correctly classified instances
+ * out of the total number of instances.)
+ */
+ @Since("3.1.0")
+ def accuracy: Double = multiclassMetrics.accuracy
+
+ /**
+ * Returns weighted true positive rate.
+ * (equals to precision, recall and f-measure)
+ */
+ @Since("3.1.0")
+ def weightedTruePositiveRate: Double = weightedRecall
+
+ /** Returns weighted false positive rate. */
+ @Since("3.1.0")
+ def weightedFalsePositiveRate: Double = multiclassMetrics.weightedFalsePositiveRate
+
+ /**
+ * Returns weighted averaged recall.
+ * (equals to precision, recall and f-measure)
+ */
+ @Since("3.1.0")
+ def weightedRecall: Double = multiclassMetrics.weightedRecall
+
+ /** Returns weighted averaged precision. */
+ @Since("3.1.0")
+ def weightedPrecision: Double = multiclassMetrics.weightedPrecision
+
+ /** Returns weighted averaged f-measure. */
+ @Since("3.1.0")
+ def weightedFMeasure(beta: Double): Double = multiclassMetrics.weightedFMeasure(beta)
+
+ /** Returns weighted averaged f1-measure. */
+ @Since("3.1.0")
+ def weightedFMeasure: Double = multiclassMetrics.weightedFMeasure(1.0)
+}
+
+/**
+ * Abstraction for training results.
+ */
+private[classification] trait TrainingSummary {
+
+ /**
+ * objective function (scaled loss + regularization) at each iteration.
+ * It contains one more element, the initial state, than number of iterations.
+ */
+ @Since("3.1.0")
+ def objectiveHistory: Array[Double]
+
+ /** Number of training iterations. */
+ @Since("3.1.0")
+ def totalIterations: Int = {
+ assert(objectiveHistory.length > 0, s"objectiveHistory length should be greater than 1.")
Review comment:
The assert looks like objectiveHistory length should be greater than 0?
And string interpolation s"" is unnecessary.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] zhengruifeng commented on a change in pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
zhengruifeng commented on a change in pull request #28710:
URL: https://github.com/apache/spark/pull/28710#discussion_r439999029
##########
File path: mllib/src/main/scala/org/apache/spark/ml/classification/LogisticRegression.scala
##########
@@ -1396,239 +1393,34 @@ object LogisticRegressionModel extends MLReadable[LogisticRegressionModel] {
/**
* Abstraction for logistic regression results for a given model.
*/
-sealed trait LogisticRegressionSummary extends Serializable {
-
- /**
- * Dataframe output by the model's `transform` method.
- */
- @Since("1.5.0")
- def predictions: DataFrame
+sealed trait LogisticRegressionSummary extends ClassificationSummary {
/** Field in "predictions" which gives the probability of each class as a vector. */
@Since("1.5.0")
def probabilityCol: String
- /** Field in "predictions" which gives the prediction of each class. */
- @Since("2.3.0")
- def predictionCol: String
-
- /** Field in "predictions" which gives the true label of each instance (if available). */
- @Since("1.5.0")
- def labelCol: String
-
/** Field in "predictions" which gives the features of each instance as a vector. */
@Since("1.6.0")
def featuresCol: String
-
- /** Field in "predictions" which gives the weight of each instance as a vector. */
- @Since("3.1.0")
- def weightCol: String
-
- @transient private val multiclassMetrics = {
- if (predictions.schema.fieldNames.contains(weightCol)) {
- new MulticlassMetrics(
- predictions.select(
- col(predictionCol),
- col(labelCol).cast(DoubleType),
- checkNonNegativeWeight(col(weightCol).cast(DoubleType))).rdd.map {
- case Row(prediction: Double, label: Double, weight: Double) => (prediction, label, weight)
- })
- } else {
- new MulticlassMetrics(
- predictions.select(
- col(predictionCol),
- col(labelCol).cast(DoubleType),
- lit(1.0)).rdd.map {
- case Row(prediction: Double, label: Double, weight: Double) => (prediction, label, weight)
- })
- }
- }
-
- /**
- * Returns the sequence of labels in ascending order. This order matches the order used
- * in metrics which are specified as arrays over labels, e.g., truePositiveRateByLabel.
- *
- * Note: In most cases, it will be values {0.0, 1.0, ..., numClasses-1}, However, if the
- * training set is missing a label, then all of the arrays over labels
- * (e.g., from truePositiveRateByLabel) will be of length numClasses-1 instead of the
- * expected numClasses.
- */
- @Since("2.3.0")
- def labels: Array[Double] = multiclassMetrics.labels
-
- /** Returns true positive rate for each label (category). */
- @Since("2.3.0")
- def truePositiveRateByLabel: Array[Double] = recallByLabel
-
- /** Returns false positive rate for each label (category). */
- @Since("2.3.0")
- def falsePositiveRateByLabel: Array[Double] = {
- multiclassMetrics.labels.map(label => multiclassMetrics.falsePositiveRate(label))
- }
-
- /** Returns precision for each label (category). */
- @Since("2.3.0")
- def precisionByLabel: Array[Double] = {
- multiclassMetrics.labels.map(label => multiclassMetrics.precision(label))
- }
-
- /** Returns recall for each label (category). */
- @Since("2.3.0")
- def recallByLabel: Array[Double] = {
- multiclassMetrics.labels.map(label => multiclassMetrics.recall(label))
- }
-
- /** Returns f-measure for each label (category). */
- @Since("2.3.0")
- def fMeasureByLabel(beta: Double): Array[Double] = {
- multiclassMetrics.labels.map(label => multiclassMetrics.fMeasure(label, beta))
- }
-
- /** Returns f1-measure for each label (category). */
- @Since("2.3.0")
- def fMeasureByLabel: Array[Double] = fMeasureByLabel(1.0)
-
- /**
- * Returns accuracy.
- * (equals to the total number of correctly classified instances
- * out of the total number of instances.)
- */
- @Since("2.3.0")
- def accuracy: Double = multiclassMetrics.accuracy
-
- /**
- * Returns weighted true positive rate.
- * (equals to precision, recall and f-measure)
- */
- @Since("2.3.0")
- def weightedTruePositiveRate: Double = weightedRecall
-
- /** Returns weighted false positive rate. */
- @Since("2.3.0")
- def weightedFalsePositiveRate: Double = multiclassMetrics.weightedFalsePositiveRate
-
- /**
- * Returns weighted averaged recall.
- * (equals to precision, recall and f-measure)
- */
- @Since("2.3.0")
- def weightedRecall: Double = multiclassMetrics.weightedRecall
-
- /** Returns weighted averaged precision. */
- @Since("2.3.0")
- def weightedPrecision: Double = multiclassMetrics.weightedPrecision
-
- /** Returns weighted averaged f-measure. */
- @Since("2.3.0")
- def weightedFMeasure(beta: Double): Double = multiclassMetrics.weightedFMeasure(beta)
-
- /** Returns weighted averaged f1-measure. */
- @Since("2.3.0")
- def weightedFMeasure: Double = multiclassMetrics.weightedFMeasure(1.0)
-
- /**
- * Convenient method for casting to binary logistic regression summary.
- * This method will throw an Exception if the summary is not a binary summary.
- */
- @Since("2.3.0")
- def asBinary: BinaryLogisticRegressionSummary = this match {
- case b: BinaryLogisticRegressionSummary => b
- case _ =>
- throw new RuntimeException("Cannot cast to a binary summary.")
- }
}
/**
* Abstraction for multiclass logistic regression training results.
- * Currently, the training summary ignores the training weights except
- * for the objective trace.
*/
-sealed trait LogisticRegressionTrainingSummary extends LogisticRegressionSummary {
-
- /** objective function (scaled loss + regularization) at each iteration. */
- @Since("1.5.0")
- def objectiveHistory: Array[Double]
-
- /** Number of training iterations. */
- @Since("1.5.0")
- def totalIterations: Int = objectiveHistory.length
-
+sealed trait LogisticRegressionTrainingSummary extends LogisticRegressionSummary
+ with TrainingSummary {
}
/**
* Abstraction for binary logistic regression results for a given model.
*/
-sealed trait BinaryLogisticRegressionSummary extends LogisticRegressionSummary {
-
- private val sparkSession = predictions.sparkSession
- import sparkSession.implicits._
-
- // TODO: Allow the user to vary the number of bins using a setBins method in
- // BinaryClassificationMetrics. For now the default is set to 100.
- @transient private val binaryMetrics = if (predictions.schema.fieldNames.contains(weightCol)) {
- new BinaryClassificationMetrics(
- predictions.select(col(probabilityCol), col(labelCol).cast(DoubleType),
- checkNonNegativeWeight(col(weightCol).cast(DoubleType))).rdd.map {
- case Row(score: Vector, label: Double, weight: Double) => (score(1), label, weight)
- }, 100
- )
- } else {
- new BinaryClassificationMetrics(
- predictions.select(col(probabilityCol), col(labelCol).cast(DoubleType),
- lit(1.0)).rdd.map {
- case Row(score: Vector, label: Double, weight: Double) => (score(1), label, weight)
- }, 100
- )
- }
-
- /**
- * Returns the receiver operating characteristic (ROC) curve,
- * which is a Dataframe having two fields (FPR, TPR)
- * with (0.0, 0.0) prepended and (1.0, 1.0) appended to it.
- * See http://en.wikipedia.org/wiki/Receiver_operating_characteristic
- */
- @Since("1.5.0")
- @transient lazy val roc: DataFrame = binaryMetrics.roc().toDF("FPR", "TPR")
+sealed trait BinaryLogisticRegressionSummary extends LogisticRegressionSummary
+ with BinaryClassificationSummary {
- /**
- * Computes the area under the receiver operating characteristic (ROC) curve.
- */
- @Since("1.5.0")
- lazy val areaUnderROC: Double = binaryMetrics.areaUnderROC()
-
- /**
- * Returns the precision-recall curve, which is a Dataframe containing
- * two fields recall, precision with (0.0, 1.0) prepended to it.
- */
- @Since("1.5.0")
- @transient lazy val pr: DataFrame = binaryMetrics.pr().toDF("recall", "precision")
-
- /**
- * Returns a dataframe with two fields (threshold, F-Measure) curve with beta = 1.0.
- */
- @Since("1.5.0")
- @transient lazy val fMeasureByThreshold: DataFrame = {
- binaryMetrics.fMeasureByThreshold().toDF("threshold", "F-Measure")
- }
-
- /**
- * Returns a dataframe with two fields (threshold, precision) curve.
- * Every possible probability obtained in transforming the dataset are used
- * as thresholds used in calculating the precision.
- */
- @Since("1.5.0")
- @transient lazy val precisionByThreshold: DataFrame = {
- binaryMetrics.precisionByThreshold().toDF("threshold", "precision")
- }
-
- /**
- * Returns a dataframe with two fields (threshold, recall) curve.
- * Every possible probability obtained in transforming the dataset are used
- * as thresholds used in calculating the recall.
- */
- @Since("1.5.0")
- @transient lazy val recallByThreshold: DataFrame = {
- binaryMetrics.recallByThreshold().toDF("threshold", "recall")
+ override def scoreCol: String = if (probabilityCol.nonEmpty) {
Review comment:
Since existing computation of binary did not use `rawPredictionCol`, so I think we can just let it alone for now.
In the furture, I think we can make it a little heuristic: if `probabilityCol.nonEmpty` use `probabilityCol`; then try `rawPredictionCol`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #28710:
URL: https://github.com/apache/spark/pull/28710#issuecomment-643855961
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28710:
URL: https://github.com/apache/spark/pull/28710#issuecomment-645823665
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] huaxingao commented on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
huaxingao commented on pull request #28710:
URL: https://github.com/apache/spark/pull/28710#issuecomment-644837229
retest this please
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28710:
URL: https://github.com/apache/spark/pull/28710#issuecomment-644478729
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28710:
URL: https://github.com/apache/spark/pull/28710#issuecomment-645773820
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/28814/
Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #28710:
URL: https://github.com/apache/spark/pull/28710#issuecomment-643897635
**[Test build #124030 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124030/testReport)** for PR 28710 at commit [`2e6f35c`](https://github.com/apache/spark/commit/2e6f35c8e31fe1cde1637b922673339bfeef65fe).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] huaxingao commented on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
huaxingao commented on pull request #28710:
URL: https://github.com/apache/spark/pull/28710#issuecomment-644526286
retest this please
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] huaxingao commented on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
huaxingao commented on pull request #28710:
URL: https://github.com/apache/spark/pull/28710#issuecomment-643896578
retest this please
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #28710:
URL: https://github.com/apache/spark/pull/28710#issuecomment-645836595
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #28710:
URL: https://github.com/apache/spark/pull/28710#issuecomment-643676869
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] huaxingao commented on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
huaxingao commented on pull request #28710:
URL: https://github.com/apache/spark/pull/28710#issuecomment-646725310
Right. They are all sealed or private. No public API changes. @srowen
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28710:
URL: https://github.com/apache/spark/pull/28710#issuecomment-637843950
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #28710:
URL: https://github.com/apache/spark/pull/28710#issuecomment-644077050
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28710:
URL: https://github.com/apache/spark/pull/28710#issuecomment-643945441
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #28710:
URL: https://github.com/apache/spark/pull/28710#issuecomment-643945441
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] zhengruifeng commented on a change in pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
zhengruifeng commented on a change in pull request #28710:
URL: https://github.com/apache/spark/pull/28710#discussion_r438625969
##########
File path: project/MimaExcludes.scala
##########
@@ -49,7 +49,34 @@ object MimaExcludes {
//[SPARK-31840] Add instance weight support in LogisticRegressionSummary
// weightCol in org.apache.spark.ml.classification.LogisticRegressionSummary is present only in current version
- ProblemFilters.exclude[ReversedMissingMethodProblem]("org.apache.spark.ml.classification.LogisticRegressionSummary.weightCol")
+ ProblemFilters.exclude[ReversedMissingMethodProblem]("org.apache.spark.ml.classification.LogisticRegressionSummary.weightCol"),
+
+ //[SPARK-31893] Add a generic ClassificationSummary trait
+ ProblemFilters.exclude[InheritedNewAbstractMethodProblem]("org.apache.spark.ml.classification.LogisticRegressionTrainingSummary.org$apache$spark$ml$classification$ClassificationSummary$_setter_$org$apache$spark$ml$classification$ClassificationSummary$$multiclassMetrics_="),
+ ProblemFilters.exclude[InheritedNewAbstractMethodProblem]("org.apache.spark.ml.classification.LogisticRegressionTrainingSummary.org$apache$spark$ml$classification$ClassificationSummary$$multiclassMetrics"),
+ ProblemFilters.exclude[InheritedNewAbstractMethodProblem]("org.apache.spark.ml.classification.LogisticRegressionTrainingSummary.weightCol"),
+ ProblemFilters.exclude[InheritedNewAbstractMethodProblem]("org.apache.spark.ml.classification.BinaryLogisticRegressionTrainingSummary.org$apache$spark$ml$classification$BinaryClassificationSummary$_setter_$org$apache$spark$ml$classification$BinaryClassificationSummary$$sparkSession_="),
+ ProblemFilters.exclude[InheritedNewAbstractMethodProblem]("org.apache.spark.ml.classification.BinaryLogisticRegressionTrainingSummary.org$apache$spark$ml$classification$BinaryClassificationSummary$_setter_$org$apache$spark$ml$classification$BinaryClassificationSummary$$binaryMetrics_="),
+ ProblemFilters.exclude[InheritedNewAbstractMethodProblem]("org.apache.spark.ml.classification.BinaryLogisticRegressionTrainingSummary.org$apache$spark$ml$classification$BinaryClassificationSummary$$binaryMetrics"),
+ ProblemFilters.exclude[InheritedNewAbstractMethodProblem]("org.apache.spark.ml.classification.BinaryLogisticRegressionTrainingSummary.scoreCol"),
+ ProblemFilters.exclude[InheritedNewAbstractMethodProblem]("org.apache.spark.ml.classification.BinaryLogisticRegressionTrainingSummary.org$apache$spark$ml$classification$BinaryClassificationSummary$$sparkSession"),
+ ProblemFilters.exclude[InheritedNewAbstractMethodProblem]("org.apache.spark.ml.classification.BinaryLogisticRegressionTrainingSummary.org$apache$spark$ml$classification$ClassificationSummary$_setter_$org$apache$spark$ml$classification$ClassificationSummary$$multiclassMetrics_="),
+ ProblemFilters.exclude[InheritedNewAbstractMethodProblem]("org.apache.spark.ml.classification.BinaryLogisticRegressionTrainingSummary.org$apache$spark$ml$classification$ClassificationSummary$$multiclassMetrics"),
+ ProblemFilters.exclude[InheritedNewAbstractMethodProblem]("org.apache.spark.ml.classification.BinaryLogisticRegressionTrainingSummary.weightCol"),
+ ProblemFilters.exclude[IncompatibleResultTypeProblem]("org.apache.spark.ml.classification.LogisticRegressionSummary.asBinary"),
Review comment:
> in multinomial case, summary is still an instance of LogisticRegressionTrainingSummary
Is it expected?
I think a reasonable behavior should be:
for any classification model, a ClassificationSummary should be attached;
if and only if numClasses=2, then this summary is also a BinaryClassificationSummary
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] huaxingao commented on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
huaxingao commented on pull request #28710:
URL: https://github.com/apache/spark/pull/28710#issuecomment-645775168
retest this please
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28710:
URL: https://github.com/apache/spark/pull/28710#issuecomment-644650512
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/124110/
Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28710:
URL: https://github.com/apache/spark/pull/28710#issuecomment-644831728
Merged build finished. Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #28710:
URL: https://github.com/apache/spark/pull/28710#issuecomment-642360058
**[Test build #123789 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123789/testReport)** for PR 28710 at commit [`c2fb2b0`](https://github.com/apache/spark/commit/c2fb2b01b64f3a40df12346f4cbc32244fb3aa4e).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28710:
URL: https://github.com/apache/spark/pull/28710#issuecomment-639134843
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] zhengruifeng commented on a change in pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
zhengruifeng commented on a change in pull request #28710:
URL: https://github.com/apache/spark/pull/28710#discussion_r439969774
##########
File path: project/MimaExcludes.scala
##########
@@ -39,18 +39,44 @@ object MimaExcludes {
// [SPARK-31077] Remove ChiSqSelector dependency on mllib.ChiSqSelectorModel
// private constructor
ProblemFilters.exclude[IncompatibleMethTypeProblem]("org.apache.spark.ml.feature.ChiSqSelectorModel.this"),
+
// [SPARK-31127] Implement abstract Selector
// org.apache.spark.ml.feature.ChiSqSelectorModel type hierarchy change
// before: class ChiSqSelector extends Estimator with ChiSqSelectorParams
// after: class ChiSqSelector extends PSelector
// false positive, no binary incompatibility
ProblemFilters.exclude[MissingTypesProblem]("org.apache.spark.ml.feature.ChiSqSelectorModel"),
ProblemFilters.exclude[MissingTypesProblem]("org.apache.spark.ml.feature.ChiSqSelector"),
+
//[SPARK-31840] Add instance weight support in LogisticRegressionSummary
// weightCol in org.apache.spark.ml.classification.LogisticRegressionSummary is present only in current version
ProblemFilters.exclude[ReversedMissingMethodProblem]("org.apache.spark.ml.classification.LogisticRegressionSummary.weightCol"),
+
// [SPARK-24634] Add a new metric regarding number of inputs later than watermark plus allowed delay
- ProblemFilters.exclude[DirectMissingMethodProblem]("org.apache.spark.sql.streaming.StateOperatorProgress.<init>$default$4")
+ ProblemFilters.exclude[DirectMissingMethodProblem]("org.apache.spark.sql.streaming.StateOperatorProgress.<init>$default$4"),
+
+ //[SPARK-31893] Add a generic ClassificationSummary trait
+ ProblemFilters.exclude[InheritedNewAbstractMethodProblem]("org.apache.spark.ml.classification.LogisticRegressionTrainingSummary.org$apache$spark$ml$classification$ClassificationSummary$_setter_$org$apache$spark$ml$classification$ClassificationSummary$$multiclassMetrics_="),
+ ProblemFilters.exclude[InheritedNewAbstractMethodProblem]("org.apache.spark.ml.classification.LogisticRegressionTrainingSummary.org$apache$spark$ml$classification$ClassificationSummary$$multiclassMetrics"),
+ ProblemFilters.exclude[InheritedNewAbstractMethodProblem]("org.apache.spark.ml.classification.LogisticRegressionTrainingSummary.weightCol"),
+ ProblemFilters.exclude[InheritedNewAbstractMethodProblem]("org.apache.spark.ml.classification.BinaryLogisticRegressionTrainingSummary.org$apache$spark$ml$classification$BinaryClassificationSummary$_setter_$org$apache$spark$ml$classification$BinaryClassificationSummary$$sparkSession_="),
+ ProblemFilters.exclude[InheritedNewAbstractMethodProblem]("org.apache.spark.ml.classification.BinaryLogisticRegressionTrainingSummary.org$apache$spark$ml$classification$BinaryClassificationSummary$_setter_$org$apache$spark$ml$classification$BinaryClassificationSummary$$binaryMetrics_="),
+ ProblemFilters.exclude[InheritedNewAbstractMethodProblem]("org.apache.spark.ml.classification.BinaryLogisticRegressionTrainingSummary.org$apache$spark$ml$classification$BinaryClassificationSummary$$binaryMetrics"),
+ ProblemFilters.exclude[InheritedNewAbstractMethodProblem]("org.apache.spark.ml.classification.BinaryLogisticRegressionTrainingSummary.org$apache$spark$ml$classification$BinaryClassificationSummary$$sparkSession"),
+ ProblemFilters.exclude[InheritedNewAbstractMethodProblem]("org.apache.spark.ml.classification.BinaryLogisticRegressionTrainingSummary.org$apache$spark$ml$classification$ClassificationSummary$_setter_$org$apache$spark$ml$classification$ClassificationSummary$$multiclassMetrics_="),
+ ProblemFilters.exclude[InheritedNewAbstractMethodProblem]("org.apache.spark.ml.classification.BinaryLogisticRegressionTrainingSummary.org$apache$spark$ml$classification$ClassificationSummary$$multiclassMetrics"),
+ ProblemFilters.exclude[InheritedNewAbstractMethodProblem]("org.apache.spark.ml.classification.BinaryLogisticRegressionTrainingSummary.weightCol"),
+ ProblemFilters.exclude[IncompatibleResultTypeProblem]("org.apache.spark.ml.classification.LogisticRegressionSummary.asBinary"),
+ ProblemFilters.exclude[InheritedNewAbstractMethodProblem]("org.apache.spark.ml.classification.LogisticRegressionSummary.org$apache$spark$ml$classification$ClassificationSummary$_setter_$org$apache$spark$ml$classification$ClassificationSummary$$multiclassMetrics_="),
+ ProblemFilters.exclude[InheritedNewAbstractMethodProblem]("org.apache.spark.ml.classification.LogisticRegressionSummary.org$apache$spark$ml$classification$ClassificationSummary$$multiclassMetrics"),
+ ProblemFilters.exclude[InheritedNewAbstractMethodProblem]("org.apache.spark.ml.classification.LogisticRegressionSummary.weightCol"),
+ ProblemFilters.exclude[InheritedNewAbstractMethodProblem]("org.apache.spark.ml.classification.BinaryLogisticRegressionSummary.org$apache$spark$ml$classification$BinaryClassificationSummary$_setter_$org$apache$spark$ml$classification$BinaryClassificationSummary$$sparkSession_="),
+ ProblemFilters.exclude[InheritedNewAbstractMethodProblem]("org.apache.spark.ml.classification.BinaryLogisticRegressionSummary.org$apache$spark$ml$classification$BinaryClassificationSummary$_setter_$org$apache$spark$ml$classification$BinaryClassificationSummary$$binaryMetrics_="),
+ ProblemFilters.exclude[InheritedNewAbstractMethodProblem]("org.apache.spark.ml.classification.BinaryLogisticRegressionSummary.org$apache$spark$ml$classification$BinaryClassificationSummary$$binaryMetrics"),
+ ProblemFilters.exclude[InheritedNewAbstractMethodProblem]("org.apache.spark.ml.classification.BinaryLogisticRegressionSummary.org$apache$spark$ml$classification$BinaryClassificationSummary$$sparkSession"),
+ ProblemFilters.exclude[InheritedNewAbstractMethodProblem]("org.apache.spark.ml.classification.BinaryLogisticRegressionSummary.org$apache$spark$ml$classification$ClassificationSummary$_setter_$org$apache$spark$ml$classification$ClassificationSummary$$multiclassMetrics_="),
+ ProblemFilters.exclude[InheritedNewAbstractMethodProblem]("org.apache.spark.ml.classification.BinaryLogisticRegressionSummary.org$apache$spark$ml$classification$ClassificationSummary$$multiclassMetrics"),
+ ProblemFilters.exclude[InheritedNewAbstractMethodProblem]("org.apache.spark.ml.classification.BinaryLogisticRegressionSummary.weightCol")
)
Review comment:
Are there some MiMa execlusions can be removed after the lastest change?
##########
File path: mllib/src/main/scala/org/apache/spark/ml/classification/ClassificationSummary.scala
##########
@@ -0,0 +1,251 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.spark.ml.classification
+
+import org.apache.spark.annotation.Since
+import org.apache.spark.ml.linalg.Vector
+import org.apache.spark.mllib.evaluation.{BinaryClassificationMetrics, MulticlassMetrics}
+import org.apache.spark.sql.{DataFrame, Row}
+import org.apache.spark.sql.functions.{col, lit}
+import org.apache.spark.sql.types.DoubleType
+
+
+/**
+ * Abstraction for multiclass classification results for a given model.
+ */
+private[classification] trait ClassificationSummary extends Serializable {
+
+ /**
+ * Dataframe output by the model's `transform` method.
+ */
+ @Since("3.1.0")
+ def predictions: DataFrame
+
+ /** Field in "predictions" which gives the prediction of each class. */
+ @Since("3.1.0")
+ def predictionCol: String
+
+ /** Field in "predictions" which gives the true label of each instance (if available). */
+ @Since("3.1.0")
+ def labelCol: String
+
+ /** Field in "predictions" which gives the weight of each instance as a vector. */
+ @Since("3.1.0")
+ def weightCol: String
+
+ @transient private val multiclassMetrics = {
+ val weightColumn = if (predictions.schema.fieldNames.contains(weightCol)) {
+ col(weightCol).cast(DoubleType)
+ } else {
+ lit(1.0)
+ }
+ new MulticlassMetrics(
+ predictions.select(col(predictionCol), col(labelCol).cast(DoubleType), weightColumn)
+ .rdd.map {
+ case Row(prediction: Double, label: Double, weight: Double) => (prediction, label, weight)
+ })
+ }
+
+ /**
+ * Returns the sequence of labels in ascending order. This order matches the order used
+ * in metrics which are specified as arrays over labels, e.g., truePositiveRateByLabel.
+ *
+ * Note: In most cases, it will be values {0.0, 1.0, ..., numClasses-1}, However, if the
+ * training set is missing a label, then all of the arrays over labels
+ * (e.g., from truePositiveRateByLabel) will be of length numClasses-1 instead of the
+ * expected numClasses.
+ */
+ @Since("3.1.0")
+ def labels: Array[Double] = multiclassMetrics.labels
+
+ /** Returns true positive rate for each label (category). */
+ @Since("3.1.0")
+ def truePositiveRateByLabel: Array[Double] = recallByLabel
+
+ /** Returns false positive rate for each label (category). */
+ @Since("3.1.0")
+ def falsePositiveRateByLabel: Array[Double] = {
+ multiclassMetrics.labels.map(label => multiclassMetrics.falsePositiveRate(label))
+ }
+
+ /** Returns precision for each label (category). */
+ @Since("3.1.0")
+ def precisionByLabel: Array[Double] = {
+ multiclassMetrics.labels.map(label => multiclassMetrics.precision(label))
+ }
+
+ /** Returns recall for each label (category). */
+ @Since("3.1.0")
+ def recallByLabel: Array[Double] = {
+ multiclassMetrics.labels.map(label => multiclassMetrics.recall(label))
+ }
+
+ /** Returns f-measure for each label (category). */
+ @Since("3.1.0")
+ def fMeasureByLabel(beta: Double): Array[Double] = {
+ multiclassMetrics.labels.map(label => multiclassMetrics.fMeasure(label, beta))
+ }
+
+ /** Returns f1-measure for each label (category). */
+ @Since("3.1.0")
+ def fMeasureByLabel: Array[Double] = fMeasureByLabel(1.0)
+
+ /**
+ * Returns accuracy.
+ * (equals to the total number of correctly classified instances
+ * out of the total number of instances.)
+ */
+ @Since("3.1.0")
+ def accuracy: Double = multiclassMetrics.accuracy
+
+ /**
+ * Returns weighted true positive rate.
+ * (equals to precision, recall and f-measure)
+ */
+ @Since("3.1.0")
+ def weightedTruePositiveRate: Double = weightedRecall
+
+ /** Returns weighted false positive rate. */
+ @Since("3.1.0")
+ def weightedFalsePositiveRate: Double = multiclassMetrics.weightedFalsePositiveRate
+
+ /**
+ * Returns weighted averaged recall.
+ * (equals to precision, recall and f-measure)
+ */
+ @Since("3.1.0")
+ def weightedRecall: Double = multiclassMetrics.weightedRecall
+
+ /** Returns weighted averaged precision. */
+ @Since("3.1.0")
+ def weightedPrecision: Double = multiclassMetrics.weightedPrecision
+
+ /** Returns weighted averaged f-measure. */
+ @Since("3.1.0")
+ def weightedFMeasure(beta: Double): Double = multiclassMetrics.weightedFMeasure(beta)
+
+ /** Returns weighted averaged f1-measure. */
+ @Since("3.1.0")
+ def weightedFMeasure: Double = multiclassMetrics.weightedFMeasure(1.0)
+
+ /**
+ * Convenient method for casting to binary classification summary.
+ * This method will throw an Exception if the summary is not a binary summary.
+ */
+ @Since("3.1.0")
+ def asBinary: BinaryClassificationSummary = this match {
+ case b: BinaryClassificationSummary => b
+ case _ =>
+ throw new RuntimeException("Cannot cast to a binary summary.")
+ }
+}
+
+/**
+ * Abstraction for training results.
+ */
+private[classification] trait TrainingSummary {
+
+ /** objective function (scaled loss + regularization) at each iteration. */
+ @Since("3.1.0")
+ def objectiveHistory: Array[Double]
+
+ /** Number of training iterations. */
+ @Since("3.1.0")
+ def totalIterations: Int = objectiveHistory.length
+}
+
+/**
+ * Abstraction for binary classification results for a given model.
+ */
+trait BinaryClassificationSummary extends ClassificationSummary {
Review comment:
private[classification]？
##########
File path: mllib/src/main/scala/org/apache/spark/ml/classification/LogisticRegression.scala
##########
@@ -1396,239 +1393,34 @@ object LogisticRegressionModel extends MLReadable[LogisticRegressionModel] {
/**
* Abstraction for logistic regression results for a given model.
*/
-sealed trait LogisticRegressionSummary extends Serializable {
-
- /**
- * Dataframe output by the model's `transform` method.
- */
- @Since("1.5.0")
- def predictions: DataFrame
+sealed trait LogisticRegressionSummary extends ClassificationSummary {
/** Field in "predictions" which gives the probability of each class as a vector. */
@Since("1.5.0")
def probabilityCol: String
- /** Field in "predictions" which gives the prediction of each class. */
- @Since("2.3.0")
- def predictionCol: String
-
- /** Field in "predictions" which gives the true label of each instance (if available). */
- @Since("1.5.0")
- def labelCol: String
-
/** Field in "predictions" which gives the features of each instance as a vector. */
@Since("1.6.0")
def featuresCol: String
-
- /** Field in "predictions" which gives the weight of each instance as a vector. */
- @Since("3.1.0")
- def weightCol: String
-
- @transient private val multiclassMetrics = {
- if (predictions.schema.fieldNames.contains(weightCol)) {
- new MulticlassMetrics(
- predictions.select(
- col(predictionCol),
- col(labelCol).cast(DoubleType),
- checkNonNegativeWeight(col(weightCol).cast(DoubleType))).rdd.map {
- case Row(prediction: Double, label: Double, weight: Double) => (prediction, label, weight)
- })
- } else {
- new MulticlassMetrics(
- predictions.select(
- col(predictionCol),
- col(labelCol).cast(DoubleType),
- lit(1.0)).rdd.map {
- case Row(prediction: Double, label: Double, weight: Double) => (prediction, label, weight)
- })
- }
- }
-
- /**
- * Returns the sequence of labels in ascending order. This order matches the order used
- * in metrics which are specified as arrays over labels, e.g., truePositiveRateByLabel.
- *
- * Note: In most cases, it will be values {0.0, 1.0, ..., numClasses-1}, However, if the
- * training set is missing a label, then all of the arrays over labels
- * (e.g., from truePositiveRateByLabel) will be of length numClasses-1 instead of the
- * expected numClasses.
- */
- @Since("2.3.0")
- def labels: Array[Double] = multiclassMetrics.labels
-
- /** Returns true positive rate for each label (category). */
- @Since("2.3.0")
- def truePositiveRateByLabel: Array[Double] = recallByLabel
-
- /** Returns false positive rate for each label (category). */
- @Since("2.3.0")
- def falsePositiveRateByLabel: Array[Double] = {
- multiclassMetrics.labels.map(label => multiclassMetrics.falsePositiveRate(label))
- }
-
- /** Returns precision for each label (category). */
- @Since("2.3.0")
- def precisionByLabel: Array[Double] = {
- multiclassMetrics.labels.map(label => multiclassMetrics.precision(label))
- }
-
- /** Returns recall for each label (category). */
- @Since("2.3.0")
- def recallByLabel: Array[Double] = {
- multiclassMetrics.labels.map(label => multiclassMetrics.recall(label))
- }
-
- /** Returns f-measure for each label (category). */
- @Since("2.3.0")
- def fMeasureByLabel(beta: Double): Array[Double] = {
- multiclassMetrics.labels.map(label => multiclassMetrics.fMeasure(label, beta))
- }
-
- /** Returns f1-measure for each label (category). */
- @Since("2.3.0")
- def fMeasureByLabel: Array[Double] = fMeasureByLabel(1.0)
-
- /**
- * Returns accuracy.
- * (equals to the total number of correctly classified instances
- * out of the total number of instances.)
- */
- @Since("2.3.0")
- def accuracy: Double = multiclassMetrics.accuracy
-
- /**
- * Returns weighted true positive rate.
- * (equals to precision, recall and f-measure)
- */
- @Since("2.3.0")
- def weightedTruePositiveRate: Double = weightedRecall
-
- /** Returns weighted false positive rate. */
- @Since("2.3.0")
- def weightedFalsePositiveRate: Double = multiclassMetrics.weightedFalsePositiveRate
-
- /**
- * Returns weighted averaged recall.
- * (equals to precision, recall and f-measure)
- */
- @Since("2.3.0")
- def weightedRecall: Double = multiclassMetrics.weightedRecall
-
- /** Returns weighted averaged precision. */
- @Since("2.3.0")
- def weightedPrecision: Double = multiclassMetrics.weightedPrecision
-
- /** Returns weighted averaged f-measure. */
- @Since("2.3.0")
- def weightedFMeasure(beta: Double): Double = multiclassMetrics.weightedFMeasure(beta)
-
- /** Returns weighted averaged f1-measure. */
- @Since("2.3.0")
- def weightedFMeasure: Double = multiclassMetrics.weightedFMeasure(1.0)
-
- /**
- * Convenient method for casting to binary logistic regression summary.
- * This method will throw an Exception if the summary is not a binary summary.
- */
- @Since("2.3.0")
- def asBinary: BinaryLogisticRegressionSummary = this match {
- case b: BinaryLogisticRegressionSummary => b
- case _ =>
- throw new RuntimeException("Cannot cast to a binary summary.")
- }
}
/**
* Abstraction for multiclass logistic regression training results.
- * Currently, the training summary ignores the training weights except
- * for the objective trace.
*/
-sealed trait LogisticRegressionTrainingSummary extends LogisticRegressionSummary {
-
- /** objective function (scaled loss + regularization) at each iteration. */
- @Since("1.5.0")
- def objectiveHistory: Array[Double]
-
- /** Number of training iterations. */
- @Since("1.5.0")
- def totalIterations: Int = objectiveHistory.length
-
+sealed trait LogisticRegressionTrainingSummary extends LogisticRegressionSummary
+ with TrainingSummary {
}
/**
* Abstraction for binary logistic regression results for a given model.
*/
-sealed trait BinaryLogisticRegressionSummary extends LogisticRegressionSummary {
-
- private val sparkSession = predictions.sparkSession
- import sparkSession.implicits._
-
- // TODO: Allow the user to vary the number of bins using a setBins method in
- // BinaryClassificationMetrics. For now the default is set to 100.
- @transient private val binaryMetrics = if (predictions.schema.fieldNames.contains(weightCol)) {
- new BinaryClassificationMetrics(
- predictions.select(col(probabilityCol), col(labelCol).cast(DoubleType),
- checkNonNegativeWeight(col(weightCol).cast(DoubleType))).rdd.map {
- case Row(score: Vector, label: Double, weight: Double) => (score(1), label, weight)
- }, 100
- )
- } else {
- new BinaryClassificationMetrics(
- predictions.select(col(probabilityCol), col(labelCol).cast(DoubleType),
- lit(1.0)).rdd.map {
- case Row(score: Vector, label: Double, weight: Double) => (score(1), label, weight)
- }, 100
- )
- }
-
- /**
- * Returns the receiver operating characteristic (ROC) curve,
- * which is a Dataframe having two fields (FPR, TPR)
- * with (0.0, 0.0) prepended and (1.0, 1.0) appended to it.
- * See http://en.wikipedia.org/wiki/Receiver_operating_characteristic
- */
- @Since("1.5.0")
- @transient lazy val roc: DataFrame = binaryMetrics.roc().toDF("FPR", "TPR")
+sealed trait BinaryLogisticRegressionSummary extends LogisticRegressionSummary
+ with BinaryClassificationSummary {
- /**
- * Computes the area under the receiver operating characteristic (ROC) curve.
- */
- @Since("1.5.0")
- lazy val areaUnderROC: Double = binaryMetrics.areaUnderROC()
-
- /**
- * Returns the precision-recall curve, which is a Dataframe containing
- * two fields recall, precision with (0.0, 1.0) prepended to it.
- */
- @Since("1.5.0")
- @transient lazy val pr: DataFrame = binaryMetrics.pr().toDF("recall", "precision")
-
- /**
- * Returns a dataframe with two fields (threshold, F-Measure) curve with beta = 1.0.
- */
- @Since("1.5.0")
- @transient lazy val fMeasureByThreshold: DataFrame = {
- binaryMetrics.fMeasureByThreshold().toDF("threshold", "F-Measure")
- }
-
- /**
- * Returns a dataframe with two fields (threshold, precision) curve.
- * Every possible probability obtained in transforming the dataset are used
- * as thresholds used in calculating the precision.
- */
- @Since("1.5.0")
- @transient lazy val precisionByThreshold: DataFrame = {
- binaryMetrics.precisionByThreshold().toDF("threshold", "precision")
- }
-
- /**
- * Returns a dataframe with two fields (threshold, recall) curve.
- * Every possible probability obtained in transforming the dataset are used
- * as thresholds used in calculating the recall.
- */
- @Since("1.5.0")
- @transient lazy val recallByThreshold: DataFrame = {
- binaryMetrics.recallByThreshold().toDF("threshold", "recall")
+ override def scoreCol: String = if (probabilityCol.nonEmpty) {
Review comment:
private?
##########
File path: mllib/src/main/scala/org/apache/spark/ml/classification/LogisticRegression.scala
##########
@@ -1396,239 +1393,34 @@ object LogisticRegressionModel extends MLReadable[LogisticRegressionModel] {
/**
* Abstraction for logistic regression results for a given model.
*/
-sealed trait LogisticRegressionSummary extends Serializable {
-
- /**
- * Dataframe output by the model's `transform` method.
- */
- @Since("1.5.0")
- def predictions: DataFrame
+sealed trait LogisticRegressionSummary extends ClassificationSummary {
/** Field in "predictions" which gives the probability of each class as a vector. */
@Since("1.5.0")
def probabilityCol: String
- /** Field in "predictions" which gives the prediction of each class. */
- @Since("2.3.0")
- def predictionCol: String
-
- /** Field in "predictions" which gives the true label of each instance (if available). */
- @Since("1.5.0")
- def labelCol: String
-
/** Field in "predictions" which gives the features of each instance as a vector. */
@Since("1.6.0")
def featuresCol: String
-
- /** Field in "predictions" which gives the weight of each instance as a vector. */
- @Since("3.1.0")
- def weightCol: String
-
- @transient private val multiclassMetrics = {
- if (predictions.schema.fieldNames.contains(weightCol)) {
- new MulticlassMetrics(
- predictions.select(
- col(predictionCol),
- col(labelCol).cast(DoubleType),
- checkNonNegativeWeight(col(weightCol).cast(DoubleType))).rdd.map {
- case Row(prediction: Double, label: Double, weight: Double) => (prediction, label, weight)
- })
- } else {
- new MulticlassMetrics(
- predictions.select(
- col(predictionCol),
- col(labelCol).cast(DoubleType),
- lit(1.0)).rdd.map {
- case Row(prediction: Double, label: Double, weight: Double) => (prediction, label, weight)
- })
- }
- }
-
- /**
- * Returns the sequence of labels in ascending order. This order matches the order used
- * in metrics which are specified as arrays over labels, e.g., truePositiveRateByLabel.
- *
- * Note: In most cases, it will be values {0.0, 1.0, ..., numClasses-1}, However, if the
- * training set is missing a label, then all of the arrays over labels
- * (e.g., from truePositiveRateByLabel) will be of length numClasses-1 instead of the
- * expected numClasses.
- */
- @Since("2.3.0")
- def labels: Array[Double] = multiclassMetrics.labels
-
- /** Returns true positive rate for each label (category). */
- @Since("2.3.0")
- def truePositiveRateByLabel: Array[Double] = recallByLabel
-
- /** Returns false positive rate for each label (category). */
- @Since("2.3.0")
- def falsePositiveRateByLabel: Array[Double] = {
- multiclassMetrics.labels.map(label => multiclassMetrics.falsePositiveRate(label))
- }
-
- /** Returns precision for each label (category). */
- @Since("2.3.0")
- def precisionByLabel: Array[Double] = {
- multiclassMetrics.labels.map(label => multiclassMetrics.precision(label))
- }
-
- /** Returns recall for each label (category). */
- @Since("2.3.0")
- def recallByLabel: Array[Double] = {
- multiclassMetrics.labels.map(label => multiclassMetrics.recall(label))
- }
-
- /** Returns f-measure for each label (category). */
- @Since("2.3.0")
- def fMeasureByLabel(beta: Double): Array[Double] = {
- multiclassMetrics.labels.map(label => multiclassMetrics.fMeasure(label, beta))
- }
-
- /** Returns f1-measure for each label (category). */
- @Since("2.3.0")
- def fMeasureByLabel: Array[Double] = fMeasureByLabel(1.0)
-
- /**
- * Returns accuracy.
- * (equals to the total number of correctly classified instances
- * out of the total number of instances.)
- */
- @Since("2.3.0")
- def accuracy: Double = multiclassMetrics.accuracy
-
- /**
- * Returns weighted true positive rate.
- * (equals to precision, recall and f-measure)
- */
- @Since("2.3.0")
- def weightedTruePositiveRate: Double = weightedRecall
-
- /** Returns weighted false positive rate. */
- @Since("2.3.0")
- def weightedFalsePositiveRate: Double = multiclassMetrics.weightedFalsePositiveRate
-
- /**
- * Returns weighted averaged recall.
- * (equals to precision, recall and f-measure)
- */
- @Since("2.3.0")
- def weightedRecall: Double = multiclassMetrics.weightedRecall
-
- /** Returns weighted averaged precision. */
- @Since("2.3.0")
- def weightedPrecision: Double = multiclassMetrics.weightedPrecision
-
- /** Returns weighted averaged f-measure. */
- @Since("2.3.0")
- def weightedFMeasure(beta: Double): Double = multiclassMetrics.weightedFMeasure(beta)
-
- /** Returns weighted averaged f1-measure. */
- @Since("2.3.0")
- def weightedFMeasure: Double = multiclassMetrics.weightedFMeasure(1.0)
-
- /**
- * Convenient method for casting to binary logistic regression summary.
- * This method will throw an Exception if the summary is not a binary summary.
- */
- @Since("2.3.0")
- def asBinary: BinaryLogisticRegressionSummary = this match {
- case b: BinaryLogisticRegressionSummary => b
- case _ =>
- throw new RuntimeException("Cannot cast to a binary summary.")
- }
}
/**
* Abstraction for multiclass logistic regression training results.
- * Currently, the training summary ignores the training weights except
- * for the objective trace.
*/
-sealed trait LogisticRegressionTrainingSummary extends LogisticRegressionSummary {
-
- /** objective function (scaled loss + regularization) at each iteration. */
- @Since("1.5.0")
- def objectiveHistory: Array[Double]
-
- /** Number of training iterations. */
- @Since("1.5.0")
- def totalIterations: Int = objectiveHistory.length
-
+sealed trait LogisticRegressionTrainingSummary extends LogisticRegressionSummary
+ with TrainingSummary {
}
/**
* Abstraction for binary logistic regression results for a given model.
*/
-sealed trait BinaryLogisticRegressionSummary extends LogisticRegressionSummary {
-
- private val sparkSession = predictions.sparkSession
- import sparkSession.implicits._
-
- // TODO: Allow the user to vary the number of bins using a setBins method in
- // BinaryClassificationMetrics. For now the default is set to 100.
- @transient private val binaryMetrics = if (predictions.schema.fieldNames.contains(weightCol)) {
- new BinaryClassificationMetrics(
- predictions.select(col(probabilityCol), col(labelCol).cast(DoubleType),
- checkNonNegativeWeight(col(weightCol).cast(DoubleType))).rdd.map {
- case Row(score: Vector, label: Double, weight: Double) => (score(1), label, weight)
- }, 100
- )
- } else {
- new BinaryClassificationMetrics(
- predictions.select(col(probabilityCol), col(labelCol).cast(DoubleType),
- lit(1.0)).rdd.map {
- case Row(score: Vector, label: Double, weight: Double) => (score(1), label, weight)
- }, 100
- )
- }
-
- /**
- * Returns the receiver operating characteristic (ROC) curve,
- * which is a Dataframe having two fields (FPR, TPR)
- * with (0.0, 0.0) prepended and (1.0, 1.0) appended to it.
- * See http://en.wikipedia.org/wiki/Receiver_operating_characteristic
- */
- @Since("1.5.0")
- @transient lazy val roc: DataFrame = binaryMetrics.roc().toDF("FPR", "TPR")
+sealed trait BinaryLogisticRegressionSummary extends LogisticRegressionSummary
+ with BinaryClassificationSummary {
- /**
- * Computes the area under the receiver operating characteristic (ROC) curve.
- */
- @Since("1.5.0")
- lazy val areaUnderROC: Double = binaryMetrics.areaUnderROC()
-
- /**
- * Returns the precision-recall curve, which is a Dataframe containing
- * two fields recall, precision with (0.0, 1.0) prepended to it.
- */
- @Since("1.5.0")
- @transient lazy val pr: DataFrame = binaryMetrics.pr().toDF("recall", "precision")
-
- /**
- * Returns a dataframe with two fields (threshold, F-Measure) curve with beta = 1.0.
- */
- @Since("1.5.0")
- @transient lazy val fMeasureByThreshold: DataFrame = {
- binaryMetrics.fMeasureByThreshold().toDF("threshold", "F-Measure")
- }
-
- /**
- * Returns a dataframe with two fields (threshold, precision) curve.
- * Every possible probability obtained in transforming the dataset are used
- * as thresholds used in calculating the precision.
- */
- @Since("1.5.0")
- @transient lazy val precisionByThreshold: DataFrame = {
- binaryMetrics.precisionByThreshold().toDF("threshold", "precision")
- }
-
- /**
- * Returns a dataframe with two fields (threshold, recall) curve.
- * Every possible probability obtained in transforming the dataset are used
- * as thresholds used in calculating the recall.
- */
- @Since("1.5.0")
- @transient lazy val recallByThreshold: DataFrame = {
- binaryMetrics.recallByThreshold().toDF("threshold", "recall")
+ override def scoreCol: String = if (probabilityCol.nonEmpty) {
Review comment:
if `probabilityCol.isEmpty` use `rawPredictionCol` instead?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28710:
URL: https://github.com/apache/spark/pull/28710#issuecomment-644580203
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] viirya commented on a change in pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
viirya commented on a change in pull request #28710:
URL: https://github.com/apache/spark/pull/28710#discussion_r442328666
##########
File path: python/pyspark/ml/classification.py
##########
@@ -990,241 +1218,26 @@ def featuresCol(self):
"""
return self._call_java("featuresCol")
- @property
- @since("3.1.0")
- def weightCol(self):
- """
- Field in "predictions" which gives the weight of each instance
- as a vector.
- """
- return self._call_java("weightCol")
-
- @property
- @since("2.3.0")
- def labels(self):
- """
- Returns the sequence of labels in ascending order. This order matches the order used
- in metrics which are specified as arrays over labels, e.g., truePositiveRateByLabel.
-
- Note: In most cases, it will be values {0.0, 1.0, ..., numClasses-1}, However, if the
- training set is missing a label, then all of the arrays over labels
- (e.g., from truePositiveRateByLabel) will be of length numClasses-1 instead of the
- expected numClasses.
- """
- return self._call_java("labels")
-
- @property
- @since("2.3.0")
- def truePositiveRateByLabel(self):
- """
- Returns true positive rate for each label (category).
- """
- return self._call_java("truePositiveRateByLabel")
-
- @property
- @since("2.3.0")
- def falsePositiveRateByLabel(self):
- """
- Returns false positive rate for each label (category).
- """
- return self._call_java("falsePositiveRateByLabel")
-
- @property
- @since("2.3.0")
- def precisionByLabel(self):
- """
- Returns precision for each label (category).
- """
- return self._call_java("precisionByLabel")
-
- @property
- @since("2.3.0")
- def recallByLabel(self):
- """
- Returns recall for each label (category).
- """
- return self._call_java("recallByLabel")
-
- @since("2.3.0")
- def fMeasureByLabel(self, beta=1.0):
- """
- Returns f-measure for each label (category).
- """
- return self._call_java("fMeasureByLabel", beta)
-
- @property
- @since("2.3.0")
- def accuracy(self):
- """
- Returns accuracy.
- (equals to the total number of correctly classified instances
- out of the total number of instances.)
- """
- return self._call_java("accuracy")
-
- @property
- @since("2.3.0")
- def weightedTruePositiveRate(self):
- """
- Returns weighted true positive rate.
- (equals to precision, recall and f-measure)
- """
- return self._call_java("weightedTruePositiveRate")
-
- @property
- @since("2.3.0")
- def weightedFalsePositiveRate(self):
- """
- Returns weighted false positive rate.
- """
- return self._call_java("weightedFalsePositiveRate")
-
- @property
- @since("2.3.0")
- def weightedRecall(self):
- """
- Returns weighted averaged recall.
- (equals to precision, recall and f-measure)
- """
- return self._call_java("weightedRecall")
-
- @property
- @since("2.3.0")
- def weightedPrecision(self):
- """
- Returns weighted averaged precision.
- """
- return self._call_java("weightedPrecision")
-
- @since("2.3.0")
- def weightedFMeasure(self, beta=1.0):
- """
- Returns weighted averaged f-measure.
- """
- return self._call_java("weightedFMeasure", beta)
-
@inherit_doc
-class LogisticRegressionTrainingSummary(LogisticRegressionSummary):
+class LogisticRegressionTrainingSummary(LogisticRegressionSummary, _TrainingSummary):
"""
Abstraction for multinomial Logistic Regression Training results.
- Currently, the training summary ignores the training weights except
- for the objective trace.
.. versionadded:: 2.0.0
"""
-
- @property
- @since("2.0.0")
- def objectiveHistory(self):
- """
- Objective function (scaled loss + regularization) at each
- iteration. It contains one more element, the initial state,
- than number of iterations.
- """
- return self._call_java("objectiveHistory")
-
- @property
- @since("2.0.0")
- def totalIterations(self):
- """
- Number of training iterations until termination.
- """
- return self._call_java("totalIterations")
+ pass
@inherit_doc
-class BinaryLogisticRegressionSummary(LogisticRegressionSummary):
+class BinaryLogisticRegressionSummary(_BinaryClassificationSummary,
+ LogisticRegressionSummary):
"""
Binary Logistic regression results for a given model.
.. versionadded:: 2.0.0
"""
-
- @property
- @since("2.0.0")
- def roc(self):
- """
- Returns the receiver operating characteristic (ROC) curve,
- which is a Dataframe having two fields (FPR, TPR) with
- (0.0, 0.0) prepended and (1.0, 1.0) appended to it.
-
- .. seealso:: `Wikipedia reference
- <http://en.wikipedia.org/wiki/Receiver_operating_characteristic>`_
-
- .. note:: This ignores instance weights (setting all to 1.0) from
- `LogisticRegression.weightCol`. This will change in later Spark
- versions.
- """
- return self._call_java("roc")
-
- @property
- @since("2.0.0")
- def areaUnderROC(self):
- """
- Computes the area under the receiver operating characteristic
- (ROC) curve.
-
- .. note:: This ignores instance weights (setting all to 1.0) from
- `LogisticRegression.weightCol`. This will change in later Spark
- versions.
- """
- return self._call_java("areaUnderROC")
-
- @property
- @since("2.0.0")
- def pr(self):
- """
- Returns the precision-recall curve, which is a Dataframe
- containing two fields recall, precision with (0.0, 1.0) prepended
- to it.
-
- .. note:: This ignores instance weights (setting all to 1.0) from
- `LogisticRegression.weightCol`. This will change in later Spark
- versions.
- """
- return self._call_java("pr")
-
- @property
- @since("2.0.0")
- def fMeasureByThreshold(self):
- """
- Returns a dataframe with two fields (threshold, F-Measure) curve
- with beta = 1.0.
-
- .. note:: This ignores instance weights (setting all to 1.0) from
- `LogisticRegression.weightCol`. This will change in later Spark
- versions.
- """
- return self._call_java("fMeasureByThreshold")
-
- @property
- @since("2.0.0")
- def precisionByThreshold(self):
- """
- Returns a dataframe with two fields (threshold, precision) curve.
- Every possible probability obtained in transforming the dataset
- are used as thresholds used in calculating the precision.
-
- .. note:: This ignores instance weights (setting all to 1.0) from
- `LogisticRegression.weightCol`. This will change in later Spark
- versions.
- """
- return self._call_java("precisionByThreshold")
-
- @property
- @since("2.0.0")
- def recallByThreshold(self):
- """
- Returns a dataframe with two fields (threshold, recall) curve.
- Every possible probability obtained in transforming the dataset
- are used as thresholds used in calculating the recall.
-
- .. note:: This ignores instance weights (setting all to 1.0) from
- `LogisticRegression.weightCol`. This will change in later Spark
- versions.
Review comment:
I saw you remove such note in latest commit. So now these methods use weightCol?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #28710:
URL: https://github.com/apache/spark/pull/28710#issuecomment-642306601
**[Test build #123789 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123789/testReport)** for PR 28710 at commit [`c2fb2b0`](https://github.com/apache/spark/commit/c2fb2b01b64f3a40df12346f4cbc32244fb3aa4e).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] huaxingao commented on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
huaxingao commented on pull request #28710:
URL: https://github.com/apache/spark/pull/28710#issuecomment-646701831
@srowen Any more comments?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #28710:
URL: https://github.com/apache/spark/pull/28710#issuecomment-646185707
**[Test build #124227 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124227/testReport)** for PR 28710 at commit [`ead6da2`](https://github.com/apache/spark/commit/ead6da2e0e3d1a269e6ef76b23f8e845690f42a3).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28710:
URL: https://github.com/apache/spark/pull/28710#issuecomment-644077059
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/124045/
Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org