You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2020/06/30 22:22:55 UTC
[GitHub] [spark] huaxingao opened a new pull request #28960: [SPARK-32140][ML][PySpark] Add training summary to FMClassificationModel
huaxingao opened a new pull request #28960:
URL: https://github.com/apache/spark/pull/28960
### What changes were proposed in this pull request?
Add training summary for FMClassificationModel...
### Why are the changes needed?
so that user can get the training process status, such as loss value of each iteration and total iteration number.
### Does this PR introduce _any_ user-facing change?
Yes
FMClassificationModel.summary
FMClassificationModel.evaluate
### How was this patch tested?
new tests
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] huaxingao commented on pull request #28960: [SPARK-32140][ML][PySpark] Add training summary to FMClassificationModel
Posted by GitBox <gi...@apache.org>.
huaxingao commented on pull request #28960:
URL: https://github.com/apache/spark/pull/28960#issuecomment-654362232
This is a python 2 failure only, python 3 is OK. I think I can simply change test data to get around this, but I found one more problem that I didn't have time to fix yet.
I will be extremely slow in these couple of weeks. Taking some time off :)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28960: [SPARK-32140][ML][PySpark] Add training summary to FMClassificationModel
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28960:
URL: https://github.com/apache/spark/pull/28960#issuecomment-652080102
Merged build finished. Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28960: [SPARK-32140][ML][PySpark] Add training summary to FMClassificationModel
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28960:
URL: https://github.com/apache/spark/pull/28960#issuecomment-652080108
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/124694/
Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28960: [SPARK-32140][ML][PySpark] Add training summary to FMClassificationModel
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #28960:
URL: https://github.com/apache/spark/pull/28960#issuecomment-652706915
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28960: [SPARK-32140][ML][PySpark] Add training summary to FMClassificationModel
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #28960:
URL: https://github.com/apache/spark/pull/28960#issuecomment-658009295
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28960: [SPARK-32140][ML][PySpark] Add training summary to FMClassificationModel
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28960:
URL: https://github.com/apache/spark/pull/28960#issuecomment-652138761
Merged build finished. Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] srowen commented on pull request #28960: [SPARK-32140][ML][PySpark] Add training summary to FMClassificationModel
Posted by GitBox <gi...@apache.org>.
srowen commented on pull request #28960:
URL: https://github.com/apache/spark/pull/28960#issuecomment-658803669
I think you can go ahead and merge this
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28960: [SPARK-32140][ML][PySpark] Add training summary to FMClassificationModel
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #28960:
URL: https://github.com/apache/spark/pull/28960#issuecomment-652081724
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28960: [SPARK-32140][ML][PySpark] Add training summary to FMClassificationModel
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #28960:
URL: https://github.com/apache/spark/pull/28960#issuecomment-652720234
**[Test build #124830 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124830/testReport)** for PR 28960 at commit [`5b6ecb9`](https://github.com/apache/spark/commit/5b6ecb9f8063115fb705487839ac0103b547bf55).
* This patch **fails PySpark unit tests**.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28960: [SPARK-32140][ML][PySpark] Add training summary to FMClassificationModel
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #28960:
URL: https://github.com/apache/spark/pull/28960#issuecomment-652094043
Merged build finished. Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28960: [SPARK-32140][ML][PySpark] Add training summary to FMClassificationModel
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28960:
URL: https://github.com/apache/spark/pull/28960#issuecomment-652083007
Merged build finished. Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] huaxingao commented on a change in pull request #28960: [SPARK-32140][ML][PySpark] Add training summary to FMClassificationModel
Posted by GitBox <gi...@apache.org>.
huaxingao commented on a change in pull request #28960:
URL: https://github.com/apache/spark/pull/28960#discussion_r448024045
##########
File path: mllib/src/main/scala/org/apache/spark/mllib/optimization/GradientDescent.scala
##########
@@ -226,45 +226,48 @@ object GradientDescent extends Logging {
var converged = false // indicates whether converged based on convergenceTol
var i = 1
- while (!converged && i <= numIterations) {
- val bcWeights = data.context.broadcast(weights)
- // Sample a subset (fraction miniBatchFraction) of the total data
- // compute and sum up the subgradients on this subset (this is one map-reduce)
- val (gradientSum, lossSum, miniBatchSize) = data.sample(false, miniBatchFraction, 42 + i)
- .treeAggregate((BDV.zeros[Double](n), 0.0, 0L))(
- seqOp = (c, v) => {
- // c: (grad, loss, count), v: (label, features)
- val l = gradient.compute(v._2, v._1, bcWeights.value, Vectors.fromBreeze(c._1))
- (c._1, c._2 + l, c._3 + 1)
- },
- combOp = (c1, c2) => {
- // c: (grad, loss, count)
- (c1._1 += c2._1, c1._2 + c2._2, c1._3 + c2._3)
- })
- bcWeights.destroy()
-
- if (miniBatchSize > 0) {
- /**
- * lossSum is computed using the weights from the previous iteration
- * and regVal is the regularization value computed in the previous iteration as well.
- */
- stochasticLossHistory += lossSum / miniBatchSize + regVal
- val update = updater.compute(
- weights, Vectors.fromBreeze(gradientSum / miniBatchSize.toDouble),
- stepSize, i, regParam)
- weights = update._1
- regVal = update._2
-
- previousWeights = currentWeights
- currentWeights = Some(weights)
- if (previousWeights != None && currentWeights != None) {
- converged = isConverged(previousWeights.get,
- currentWeights.get, convergenceTol)
+ breakable {
+ while (i <= numIterations + 1) {
+ val bcWeights = data.context.broadcast(weights)
+ // Sample a subset (fraction miniBatchFraction) of the total data
+ // compute and sum up the subgradients on this subset (this is one map-reduce)
+ val (gradientSum, lossSum, miniBatchSize) = data.sample(false, miniBatchFraction, 42 + i)
+ .treeAggregate((BDV.zeros[Double](n), 0.0, 0L))(
+ seqOp = (c, v) => {
+ // c: (grad, loss, count), v: (label, features)
+ val l = gradient.compute(v._2, v._1, bcWeights.value, Vectors.fromBreeze(c._1))
+ (c._1, c._2 + l, c._3 + 1)
+ },
+ combOp = (c1, c2) => {
+ // c: (grad, loss, count)
+ (c1._1 += c2._1, c1._2 + c2._2, c1._3 + c2._3)
+ })
+ bcWeights.destroy()
+
+ if (miniBatchSize > 0) {
+ /**
+ * lossSum is computed using the weights from the previous iteration
+ * and regVal is the regularization value computed in the previous iteration as well.
+ */
+ stochasticLossHistory += lossSum / miniBatchSize + regVal
+ if (converged || i == (numIterations + 1)) break
Review comment:
Currently, stochasticLossHistory only contains initial state + state form 1 to n-1 iteration, so need to add state for the last iteration too. After adding the last state, exist the loop.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28960: [SPARK-32140][ML][PySpark] Add training summary to FMClassificationModel
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #28960:
URL: https://github.com/apache/spark/pull/28960#issuecomment-652088710
**[Test build #124696 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124696/testReport)** for PR 28960 at commit [`35edf01`](https://github.com/apache/spark/commit/35edf0124119161ac06d15f94050d98633155dfb).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] srowen commented on a change in pull request #28960: [SPARK-32140][ML][PySpark] Add training summary to FMClassificationModel
Posted by GitBox <gi...@apache.org>.
srowen commented on a change in pull request #28960:
URL: https://github.com/apache/spark/pull/28960#discussion_r453335160
##########
File path: mllib/src/main/scala/org/apache/spark/mllib/optimization/GradientDescent.scala
##########
@@ -226,45 +239,48 @@ object GradientDescent extends Logging {
var converged = false // indicates whether converged based on convergenceTol
var i = 1
- while (!converged && i <= numIterations) {
- val bcWeights = data.context.broadcast(weights)
- // Sample a subset (fraction miniBatchFraction) of the total data
- // compute and sum up the subgradients on this subset (this is one map-reduce)
- val (gradientSum, lossSum, miniBatchSize) = data.sample(false, miniBatchFraction, 42 + i)
- .treeAggregate((BDV.zeros[Double](n), 0.0, 0L))(
- seqOp = (c, v) => {
- // c: (grad, loss, count), v: (label, features)
- val l = gradient.compute(v._2, v._1, bcWeights.value, Vectors.fromBreeze(c._1))
- (c._1, c._2 + l, c._3 + 1)
- },
- combOp = (c1, c2) => {
- // c: (grad, loss, count)
- (c1._1 += c2._1, c1._2 + c2._2, c1._3 + c2._3)
- })
- bcWeights.destroy()
-
- if (miniBatchSize > 0) {
- /**
- * lossSum is computed using the weights from the previous iteration
- * and regVal is the regularization value computed in the previous iteration as well.
- */
- stochasticLossHistory += lossSum / miniBatchSize + regVal
- val update = updater.compute(
- weights, Vectors.fromBreeze(gradientSum / miniBatchSize.toDouble),
- stepSize, i, regParam)
- weights = update._1
- regVal = update._2
-
- previousWeights = currentWeights
- currentWeights = Some(weights)
- if (previousWeights != None && currentWeights != None) {
- converged = isConverged(previousWeights.get,
- currentWeights.get, convergenceTol)
+ breakable {
+ while (i <= numIterations + 1) {
+ val bcWeights = data.context.broadcast(weights)
+ // Sample a subset (fraction miniBatchFraction) of the total data
+ // compute and sum up the subgradients on this subset (this is one map-reduce)
+ val (gradientSum, lossSum, miniBatchSize) = data.sample(false, miniBatchFraction, 42 + i)
+ .treeAggregate((BDV.zeros[Double](n), 0.0, 0L))(
+ seqOp = (c, v) => {
Review comment:
I forget, can you write stuff like `case ((foo, bar, baz), v) =>` here to avoid all the `._1`? I keep thinking it's possible but then I find it isn't.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28960: [SPARK-32140][ML][PySpark] Add training summary to FMClassificationModel
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #28960:
URL: https://github.com/apache/spark/pull/28960#issuecomment-658009391
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28960: [SPARK-32140][ML][PySpark] Add training summary to FMClassificationModel
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #28960:
URL: https://github.com/apache/spark/pull/28960#issuecomment-652126325
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28960: [SPARK-32140][ML][PySpark] Add training summary to FMClassificationModel
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #28960:
URL: https://github.com/apache/spark/pull/28960#issuecomment-652720376
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] huaxingao closed pull request #28960: [SPARK-32140][ML][PySpark] Add training summary to FMClassificationModel
Posted by GitBox <gi...@apache.org>.
huaxingao closed pull request #28960:
URL: https://github.com/apache/spark/pull/28960
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28960: [SPARK-32140][ML][PySpark] Add training summary to FMClassificationModel
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #28960:
URL: https://github.com/apache/spark/pull/28960#issuecomment-658004084
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28960: [SPARK-32140][ML][PySpark] Add training summary to FMClassificationModel
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28960:
URL: https://github.com/apache/spark/pull/28960#issuecomment-652089491
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28960: [SPARK-32140][ML][PySpark] Add training summary to FMClassificationModel
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #28960:
URL: https://github.com/apache/spark/pull/28960#issuecomment-652093983
**[Test build #124696 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124696/testReport)** for PR 28960 at commit [`35edf01`](https://github.com/apache/spark/commit/35edf0124119161ac06d15f94050d98633155dfb).
* This patch **fails MiMa tests**.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #28960: [SPARK-32140][ML][PySpark] Add training summary to FMClassificationModel
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #28960:
URL: https://github.com/apache/spark/pull/28960#issuecomment-652078834
**[Test build #124694 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124694/testReport)** for PR 28960 at commit [`0fac436`](https://github.com/apache/spark/commit/0fac43652dbd6618b6b0bfffb604089e885c1c51).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] zhengruifeng commented on a change in pull request #28960: [SPARK-32140][ML][PySpark] Add training summary to FMClassificationModel
Posted by GitBox <gi...@apache.org>.
zhengruifeng commented on a change in pull request #28960:
URL: https://github.com/apache/spark/pull/28960#discussion_r454058321
##########
File path: mllib/src/main/scala/org/apache/spark/mllib/optimization/GradientDescent.scala
##########
@@ -226,45 +239,48 @@ object GradientDescent extends Logging {
var converged = false // indicates whether converged based on convergenceTol
var i = 1
- while (!converged && i <= numIterations) {
- val bcWeights = data.context.broadcast(weights)
- // Sample a subset (fraction miniBatchFraction) of the total data
- // compute and sum up the subgradients on this subset (this is one map-reduce)
- val (gradientSum, lossSum, miniBatchSize) = data.sample(false, miniBatchFraction, 42 + i)
- .treeAggregate((BDV.zeros[Double](n), 0.0, 0L))(
- seqOp = (c, v) => {
- // c: (grad, loss, count), v: (label, features)
- val l = gradient.compute(v._2, v._1, bcWeights.value, Vectors.fromBreeze(c._1))
- (c._1, c._2 + l, c._3 + 1)
- },
- combOp = (c1, c2) => {
- // c: (grad, loss, count)
- (c1._1 += c2._1, c1._2 + c2._2, c1._3 + c2._3)
- })
- bcWeights.destroy()
-
- if (miniBatchSize > 0) {
- /**
- * lossSum is computed using the weights from the previous iteration
- * and regVal is the regularization value computed in the previous iteration as well.
- */
- stochasticLossHistory += lossSum / miniBatchSize + regVal
- val update = updater.compute(
- weights, Vectors.fromBreeze(gradientSum / miniBatchSize.toDouble),
- stepSize, i, regParam)
- weights = update._1
- regVal = update._2
-
- previousWeights = currentWeights
- currentWeights = Some(weights)
- if (previousWeights != None && currentWeights != None) {
- converged = isConverged(previousWeights.get,
- currentWeights.get, convergenceTol)
+ breakable {
+ while (i <= numIterations + 1) {
+ val bcWeights = data.context.broadcast(weights)
+ // Sample a subset (fraction miniBatchFraction) of the total data
+ // compute and sum up the subgradients on this subset (this is one map-reduce)
+ val (gradientSum, lossSum, miniBatchSize) = data.sample(false, miniBatchFraction, 42 + i)
+ .treeAggregate((BDV.zeros[Double](n), 0.0, 0L))(
+ seqOp = (c, v) => {
Review comment:
nit: it seems that `breakable` is not used in spark (except two suites):
```
➜ spark git:(master) ag --scala 'breakable' .
mllib/src/test/scala/org/apache/spark/ml/classification/LogisticRegressionSuite.scala
2941: breakable {
mllib/src/test/scala/org/apache/spark/mllib/classification/LogisticRegressionSuite.scala
142: breakable {
```
I am not sure whether it is suiteable
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28960: [SPARK-32140][ML][PySpark] Add training summary to FMClassificationModel
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28960:
URL: https://github.com/apache/spark/pull/28960#issuecomment-658006936
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28960: [SPARK-32140][ML][PySpark] Add training summary to FMClassificationModel
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28960:
URL: https://github.com/apache/spark/pull/28960#issuecomment-658004084
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28960: [SPARK-32140][ML][PySpark] Add training summary to FMClassificationModel
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #28960:
URL: https://github.com/apache/spark/pull/28960#issuecomment-652138768
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28960: [SPARK-32140][ML][PySpark] Add training summary to FMClassificationModel
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28960:
URL: https://github.com/apache/spark/pull/28960#issuecomment-652083011
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/124695/
Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28960: [SPARK-32140][ML][PySpark] Add training summary to FMClassificationModel
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28960:
URL: https://github.com/apache/spark/pull/28960#issuecomment-652720376
Merged build finished. Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28960: [SPARK-32140][ML][PySpark] Add training summary to FMClassificationModel
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #28960:
URL: https://github.com/apache/spark/pull/28960#issuecomment-652081549
**[Test build #124695 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124695/testReport)** for PR 28960 at commit [`0a43c5f`](https://github.com/apache/spark/commit/0a43c5feca5243a09e47894f4d6058a46ab23910).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28960: [SPARK-32140][ML][PySpark] Add training summary to FMClassificationModel
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28960:
URL: https://github.com/apache/spark/pull/28960#issuecomment-657250546
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28960: [SPARK-32140][ML][PySpark] Add training summary to FMClassificationModel
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28960:
URL: https://github.com/apache/spark/pull/28960#issuecomment-658009391
Build finished. Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28960: [SPARK-32140][ML][PySpark] Add training summary to FMClassificationModel
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #28960:
URL: https://github.com/apache/spark/pull/28960#issuecomment-657242216
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28960: [SPARK-32140][ML][PySpark] Add training summary to FMClassificationModel
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #28960:
URL: https://github.com/apache/spark/pull/28960#issuecomment-652080102
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28960: [SPARK-32140][ML][PySpark] Add training summary to FMClassificationModel
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #28960:
URL: https://github.com/apache/spark/pull/28960#issuecomment-658035984
**[Test build #125810 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125810/testReport)** for PR 28960 at commit [`9a58603`](https://github.com/apache/spark/commit/9a58603ce88b2c3116f6ce77a5144151cffab4ad).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] huaxingao commented on pull request #28960: [SPARK-32140][ML][PySpark] Add training summary to FMClassificationModel
Posted by GitBox <gi...@apache.org>.
huaxingao commented on pull request #28960:
URL: https://github.com/apache/spark/pull/28960#issuecomment-658892660
Merged to master. Thanks @srowen @zhengruifeng for reviewing!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] srowen commented on pull request #28960: [SPARK-32140][ML][PySpark] Add training summary to FMClassificationModel
Posted by GitBox <gi...@apache.org>.
srowen commented on pull request #28960:
URL: https://github.com/apache/spark/pull/28960#issuecomment-654237552
Weird, a Python 2 failure?
```
======================================================================
FAIL: test_fm_classification_summary (pyspark.ml.tests.test_training_summary.TrainingSummaryTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/jenkins/workspace/SparkPullRequestBuilder@4/python/pyspark/ml/tests/test_training_summary.py", line 345, in test_fm_classification_summary
self.assertAlmostEqual(s.weightedTruePositiveRate, 0.5, 2)
AssertionError: 1.0 != 0.5 within 2 places
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] huaxingao commented on a change in pull request #28960: [SPARK-32140][ML][PySpark] Add training summary to FMClassificationModel
Posted by GitBox <gi...@apache.org>.
huaxingao commented on a change in pull request #28960:
URL: https://github.com/apache/spark/pull/28960#discussion_r453338481
##########
File path: mllib/src/main/scala/org/apache/spark/mllib/optimization/GradientDescent.scala
##########
@@ -226,45 +239,48 @@ object GradientDescent extends Logging {
var converged = false // indicates whether converged based on convergenceTol
var i = 1
- while (!converged && i <= numIterations) {
- val bcWeights = data.context.broadcast(weights)
- // Sample a subset (fraction miniBatchFraction) of the total data
- // compute and sum up the subgradients on this subset (this is one map-reduce)
- val (gradientSum, lossSum, miniBatchSize) = data.sample(false, miniBatchFraction, 42 + i)
- .treeAggregate((BDV.zeros[Double](n), 0.0, 0L))(
- seqOp = (c, v) => {
- // c: (grad, loss, count), v: (label, features)
- val l = gradient.compute(v._2, v._1, bcWeights.value, Vectors.fromBreeze(c._1))
- (c._1, c._2 + l, c._3 + 1)
- },
- combOp = (c1, c2) => {
- // c: (grad, loss, count)
- (c1._1 += c2._1, c1._2 + c2._2, c1._3 + c2._3)
- })
- bcWeights.destroy()
-
- if (miniBatchSize > 0) {
- /**
- * lossSum is computed using the weights from the previous iteration
- * and regVal is the regularization value computed in the previous iteration as well.
- */
- stochasticLossHistory += lossSum / miniBatchSize + regVal
- val update = updater.compute(
- weights, Vectors.fromBreeze(gradientSum / miniBatchSize.toDouble),
- stepSize, i, regParam)
- weights = update._1
- regVal = update._2
-
- previousWeights = currentWeights
- currentWeights = Some(weights)
- if (previousWeights != None && currentWeights != None) {
- converged = isConverged(previousWeights.get,
- currentWeights.get, convergenceTol)
+ breakable {
+ while (i <= numIterations + 1) {
+ val bcWeights = data.context.broadcast(weights)
+ // Sample a subset (fraction miniBatchFraction) of the total data
+ // compute and sum up the subgradients on this subset (this is one map-reduce)
+ val (gradientSum, lossSum, miniBatchSize) = data.sample(false, miniBatchFraction, 42 + i)
+ .treeAggregate((BDV.zeros[Double](n), 0.0, 0L))(
+ seqOp = (c, v) => {
Review comment:
seems not. Just tried, not working.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28960: [SPARK-32140][ML][PySpark] Add training summary to FMClassificationModel
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28960:
URL: https://github.com/apache/spark/pull/28960#issuecomment-657242216
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #28960: [SPARK-32140][ML][PySpark] Add training summary to FMClassificationModel
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #28960:
URL: https://github.com/apache/spark/pull/28960#issuecomment-652706563
**[Test build #124830 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124830/testReport)** for PR 28960 at commit [`5b6ecb9`](https://github.com/apache/spark/commit/5b6ecb9f8063115fb705487839ac0103b547bf55).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28960: [SPARK-32140][ML][PySpark] Add training summary to FMClassificationModel
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #28960:
URL: https://github.com/apache/spark/pull/28960#issuecomment-652080089
**[Test build #124694 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124694/testReport)** for PR 28960 at commit [`0fac436`](https://github.com/apache/spark/commit/0fac43652dbd6618b6b0bfffb604089e885c1c51).
* This patch **fails Scala style tests**.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28960: [SPARK-32140][ML][PySpark] Add training summary to FMClassificationModel
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28960:
URL: https://github.com/apache/spark/pull/28960#issuecomment-652126331
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28960: [SPARK-32140][ML][PySpark] Add training summary to FMClassificationModel
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #28960:
URL: https://github.com/apache/spark/pull/28960#issuecomment-658006936
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #28960: [SPARK-32140][ML][PySpark] Add training summary to FMClassificationModel
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #28960:
URL: https://github.com/apache/spark/pull/28960#issuecomment-658009354
**[Test build #125810 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125810/testReport)** for PR 28960 at commit [`9a58603`](https://github.com/apache/spark/commit/9a58603ce88b2c3116f6ce77a5144151cffab4ad).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28960: [SPARK-32140][ML][PySpark] Add training summary to FMClassificationModel
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #28960:
URL: https://github.com/apache/spark/pull/28960#issuecomment-658003670
**[Test build #125809 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125809/testReport)** for PR 28960 at commit [`0767117`](https://github.com/apache/spark/commit/07671170b7dac6227e4c1a98f58bf24f9be9ad25).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28960: [SPARK-32140][ML][PySpark] Add training summary to FMClassificationModel
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28960:
URL: https://github.com/apache/spark/pull/28960#issuecomment-658009398
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28960: [SPARK-32140][ML][PySpark] Add training summary to FMClassificationModel
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #28960:
URL: https://github.com/apache/spark/pull/28960#issuecomment-652125931
**[Test build #124711 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124711/testReport)** for PR 28960 at commit [`ba3384d`](https://github.com/apache/spark/commit/ba3384d3cc5673fe961dca5071a017e4c967e7d1).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28960: [SPARK-32140][ML][PySpark] Add training summary to FMClassificationModel
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #28960:
URL: https://github.com/apache/spark/pull/28960#issuecomment-652083007
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28960: [SPARK-32140][ML][PySpark] Add training summary to FMClassificationModel
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28960:
URL: https://github.com/apache/spark/pull/28960#issuecomment-652720381
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/124830/
Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #28960: [SPARK-32140][ML][PySpark] Add training summary to FMClassificationModel
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #28960:
URL: https://github.com/apache/spark/pull/28960#issuecomment-652088710
**[Test build #124696 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124696/testReport)** for PR 28960 at commit [`35edf01`](https://github.com/apache/spark/commit/35edf0124119161ac06d15f94050d98633155dfb).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #28960: [SPARK-32140][ML][PySpark] Add training summary to FMClassificationModel
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #28960:
URL: https://github.com/apache/spark/pull/28960#issuecomment-652125931
**[Test build #124711 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124711/testReport)** for PR 28960 at commit [`ba3384d`](https://github.com/apache/spark/commit/ba3384d3cc5673fe961dca5071a017e4c967e7d1).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28960: [SPARK-32140][ML][PySpark] Add training summary to FMClassificationModel
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #28960:
URL: https://github.com/apache/spark/pull/28960#issuecomment-652706563
**[Test build #124830 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124830/testReport)** for PR 28960 at commit [`5b6ecb9`](https://github.com/apache/spark/commit/5b6ecb9f8063115fb705487839ac0103b547bf55).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #28960: [SPARK-32140][ML][PySpark] Add training summary to FMClassificationModel
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #28960:
URL: https://github.com/apache/spark/pull/28960#issuecomment-658003670
**[Test build #125809 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125809/testReport)** for PR 28960 at commit [`0767117`](https://github.com/apache/spark/commit/07671170b7dac6227e4c1a98f58bf24f9be9ad25).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] huaxingao commented on a change in pull request #28960: [SPARK-32140][ML][PySpark] Add training summary to FMClassificationModel
Posted by GitBox <gi...@apache.org>.
huaxingao commented on a change in pull request #28960:
URL: https://github.com/apache/spark/pull/28960#discussion_r454144678
##########
File path: mllib/src/main/scala/org/apache/spark/mllib/optimization/GradientDescent.scala
##########
@@ -226,45 +239,48 @@ object GradientDescent extends Logging {
var converged = false // indicates whether converged based on convergenceTol
var i = 1
- while (!converged && i <= numIterations) {
- val bcWeights = data.context.broadcast(weights)
- // Sample a subset (fraction miniBatchFraction) of the total data
- // compute and sum up the subgradients on this subset (this is one map-reduce)
- val (gradientSum, lossSum, miniBatchSize) = data.sample(false, miniBatchFraction, 42 + i)
- .treeAggregate((BDV.zeros[Double](n), 0.0, 0L))(
- seqOp = (c, v) => {
- // c: (grad, loss, count), v: (label, features)
- val l = gradient.compute(v._2, v._1, bcWeights.value, Vectors.fromBreeze(c._1))
- (c._1, c._2 + l, c._3 + 1)
- },
- combOp = (c1, c2) => {
- // c: (grad, loss, count)
- (c1._1 += c2._1, c1._2 + c2._2, c1._3 + c2._3)
- })
- bcWeights.destroy()
-
- if (miniBatchSize > 0) {
- /**
- * lossSum is computed using the weights from the previous iteration
- * and regVal is the regularization value computed in the previous iteration as well.
- */
- stochasticLossHistory += lossSum / miniBatchSize + regVal
- val update = updater.compute(
- weights, Vectors.fromBreeze(gradientSum / miniBatchSize.toDouble),
- stepSize, i, regParam)
- weights = update._1
- regVal = update._2
-
- previousWeights = currentWeights
- currentWeights = Some(weights)
- if (previousWeights != None && currentWeights != None) {
- converged = isConverged(previousWeights.get,
- currentWeights.get, convergenceTol)
+ breakable {
+ while (i <= numIterations + 1) {
+ val bcWeights = data.context.broadcast(weights)
+ // Sample a subset (fraction miniBatchFraction) of the total data
+ // compute and sum up the subgradients on this subset (this is one map-reduce)
+ val (gradientSum, lossSum, miniBatchSize) = data.sample(false, miniBatchFraction, 42 + i)
+ .treeAggregate((BDV.zeros[Double](n), 0.0, 0L))(
+ seqOp = (c, v) => {
Review comment:
Fixed. Thanks!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #28960: [SPARK-32140][ML][PySpark] Add training summary to FMClassificationModel
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #28960:
URL: https://github.com/apache/spark/pull/28960#issuecomment-657242031
**[Test build #125718 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125718/testReport)** for PR 28960 at commit [`77aefd8`](https://github.com/apache/spark/commit/77aefd8d14995390960f3439b40ed35120da115f).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28960: [SPARK-32140][ML][PySpark] Add training summary to FMClassificationModel
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28960:
URL: https://github.com/apache/spark/pull/28960#issuecomment-652081716
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28960: [SPARK-32140][ML][PySpark] Add training summary to FMClassificationModel
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28960:
URL: https://github.com/apache/spark/pull/28960#issuecomment-652094052
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/124696/
Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28960: [SPARK-32140][ML][PySpark] Add training summary to FMClassificationModel
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28960:
URL: https://github.com/apache/spark/pull/28960#issuecomment-652079182
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28960: [SPARK-32140][ML][PySpark] Add training summary to FMClassificationModel
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #28960:
URL: https://github.com/apache/spark/pull/28960#issuecomment-652138631
**[Test build #124711 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124711/testReport)** for PR 28960 at commit [`ba3384d`](https://github.com/apache/spark/commit/ba3384d3cc5673fe961dca5071a017e4c967e7d1).
* This patch **fails PySpark unit tests**.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28960: [SPARK-32140][ML][PySpark] Add training summary to FMClassificationModel
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #28960:
URL: https://github.com/apache/spark/pull/28960#issuecomment-652078834
**[Test build #124694 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124694/testReport)** for PR 28960 at commit [`0fac436`](https://github.com/apache/spark/commit/0fac43652dbd6618b6b0bfffb604089e885c1c51).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] srowen commented on a change in pull request #28960: [SPARK-32140][ML][PySpark] Add training summary to FMClassificationModel
Posted by GitBox <gi...@apache.org>.
srowen commented on a change in pull request #28960:
URL: https://github.com/apache/spark/pull/28960#discussion_r454062744
##########
File path: mllib/src/main/scala/org/apache/spark/mllib/optimization/GradientDescent.scala
##########
@@ -226,45 +239,48 @@ object GradientDescent extends Logging {
var converged = false // indicates whether converged based on convergenceTol
var i = 1
- while (!converged && i <= numIterations) {
- val bcWeights = data.context.broadcast(weights)
- // Sample a subset (fraction miniBatchFraction) of the total data
- // compute and sum up the subgradients on this subset (this is one map-reduce)
- val (gradientSum, lossSum, miniBatchSize) = data.sample(false, miniBatchFraction, 42 + i)
- .treeAggregate((BDV.zeros[Double](n), 0.0, 0L))(
- seqOp = (c, v) => {
- // c: (grad, loss, count), v: (label, features)
- val l = gradient.compute(v._2, v._1, bcWeights.value, Vectors.fromBreeze(c._1))
- (c._1, c._2 + l, c._3 + 1)
- },
- combOp = (c1, c2) => {
- // c: (grad, loss, count)
- (c1._1 += c2._1, c1._2 + c2._2, c1._3 + c2._3)
- })
- bcWeights.destroy()
-
- if (miniBatchSize > 0) {
- /**
- * lossSum is computed using the weights from the previous iteration
- * and regVal is the regularization value computed in the previous iteration as well.
- */
- stochasticLossHistory += lossSum / miniBatchSize + regVal
- val update = updater.compute(
- weights, Vectors.fromBreeze(gradientSum / miniBatchSize.toDouble),
- stepSize, i, regParam)
- weights = update._1
- regVal = update._2
-
- previousWeights = currentWeights
- currentWeights = Some(weights)
- if (previousWeights != None && currentWeights != None) {
- converged = isConverged(previousWeights.get,
- currentWeights.get, convergenceTol)
+ breakable {
+ while (i <= numIterations + 1) {
+ val bcWeights = data.context.broadcast(weights)
+ // Sample a subset (fraction miniBatchFraction) of the total data
+ // compute and sum up the subgradients on this subset (this is one map-reduce)
+ val (gradientSum, lossSum, miniBatchSize) = data.sample(false, miniBatchFraction, 42 + i)
+ .treeAggregate((BDV.zeros[Double](n), 0.0, 0L))(
+ seqOp = (c, v) => {
Review comment:
Yeah it's a little unusual unless it significantly simplifies the code. Can `!converged` be added back to the while condition, and then turn the `if (X) break` condition below into `if (!X) { ... code that follows ...}` ? should be the same as i will increment and end the loop right after anyway
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28960: [SPARK-32140][ML][PySpark] Add training summary to FMClassificationModel
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #28960:
URL: https://github.com/apache/spark/pull/28960#issuecomment-657242031
**[Test build #125718 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125718/testReport)** for PR 28960 at commit [`77aefd8`](https://github.com/apache/spark/commit/77aefd8d14995390960f3439b40ed35120da115f).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28960: [SPARK-32140][ML][PySpark] Add training summary to FMClassificationModel
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28960:
URL: https://github.com/apache/spark/pull/28960#issuecomment-652138768
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/124711/
Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28960: [SPARK-32140][ML][PySpark] Add training summary to FMClassificationModel
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #28960:
URL: https://github.com/apache/spark/pull/28960#issuecomment-657250546
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28960: [SPARK-32140][ML][PySpark] Add training summary to FMClassificationModel
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #28960:
URL: https://github.com/apache/spark/pull/28960#issuecomment-652079182
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28960: [SPARK-32140][ML][PySpark] Add training summary to FMClassificationModel
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28960:
URL: https://github.com/apache/spark/pull/28960#issuecomment-652706915
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] huaxingao commented on a change in pull request #28960: [SPARK-32140][ML][PySpark] Add training summary to FMClassificationModel
Posted by GitBox <gi...@apache.org>.
huaxingao commented on a change in pull request #28960:
URL: https://github.com/apache/spark/pull/28960#discussion_r448023875
##########
File path: mllib/src/main/scala/org/apache/spark/ml/regression/FMRegressor.scala
##########
@@ -47,7 +47,7 @@ import org.apache.spark.storage.StorageLevel
*/
private[ml] trait FactorizationMachinesParams extends PredictorParams
with HasMaxIter with HasStepSize with HasTol with HasSolver with HasSeed
- with HasFitIntercept with HasRegParam {
+ with HasFitIntercept with HasRegParam with HasWeightCol {
Review comment:
Add ```with HasWeightCol``` because ```ClassificationSummary``` uses weigthCol. However, FM doesn't really support instance weight yet and all the weight are default to 1.0.
##########
File path: mllib/src/main/scala/org/apache/spark/mllib/optimization/GradientDescent.scala
##########
@@ -195,7 +195,7 @@ object GradientDescent extends Logging {
s"numIterations=$numIterations and miniBatchFraction=$miniBatchFraction")
}
- val stochasticLossHistory = new ArrayBuffer[Double](numIterations)
+ val stochasticLossHistory = new ArrayBuffer[Double](numIterations + 1)
Review comment:
Make this stochasticLossHistory contain initial state + the state for each iteration, so it is consistent with the objectiveHistory in LogisticRegression and LinearRegression
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28960: [SPARK-32140][ML][PySpark] Add training summary to FMClassificationModel
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28960:
URL: https://github.com/apache/spark/pull/28960#issuecomment-652094043
Merged build finished. Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28960: [SPARK-32140][ML][PySpark] Add training summary to FMClassificationModel
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #28960:
URL: https://github.com/apache/spark/pull/28960#issuecomment-658036270
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28960: [SPARK-32140][ML][PySpark] Add training summary to FMClassificationModel
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #28960:
URL: https://github.com/apache/spark/pull/28960#issuecomment-657250473
**[Test build #125718 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125718/testReport)** for PR 28960 at commit [`77aefd8`](https://github.com/apache/spark/commit/77aefd8d14995390960f3439b40ed35120da115f).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #28960: [SPARK-32140][ML][PySpark] Add training summary to FMClassificationModel
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #28960:
URL: https://github.com/apache/spark/pull/28960#issuecomment-652081549
**[Test build #124695 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124695/testReport)** for PR 28960 at commit [`0a43c5f`](https://github.com/apache/spark/commit/0a43c5feca5243a09e47894f4d6058a46ab23910).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28960: [SPARK-32140][ML][PySpark] Add training summary to FMClassificationModel
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #28960:
URL: https://github.com/apache/spark/pull/28960#issuecomment-652083003
**[Test build #124695 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124695/testReport)** for PR 28960 at commit [`0a43c5f`](https://github.com/apache/spark/commit/0a43c5feca5243a09e47894f4d6058a46ab23910).
* This patch **fails Python style tests**.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28960: [SPARK-32140][ML][PySpark] Add training summary to FMClassificationModel
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #28960:
URL: https://github.com/apache/spark/pull/28960#issuecomment-652089491
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org