You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2020/05/22 06:02:14 UTC
[GitHub] [spark] HeartSaVioR opened a new pull request #28607: [SPARK-24634][SS] Add a new metric regarding number of inputs later than watermark plus allowed delay
HeartSaVioR opened a new pull request #28607:
URL: https://github.com/apache/spark/pull/28607
### What changes were proposed in this pull request?
Please refer https://issues.apache.org/jira/browse/SPARK-24634 to see rationalization of the issue.
This patch adds a new metric to count the number of inputs arrived later than watermark plus allowed delay. To make changes simpler, this patch doesn't count the exact number of input rows which are later than watermark plus allowed delay. Instead, this patch counts the inputs which are dropped in the logic of operator. The difference of twos are shown in streaming aggregation: to optimize the calculation, streaming aggregation "pre-aggregates" the input rows, and later checks the lateness against "pre-aggregated" inputs, hence the number might be reduced.
The new metric will be provided via two places:
1. On Spark UI: check the metrics in stateful operator nodes in query execution details page in SQL tab
2. On Streaming Query Listener: check "numLateInputs" in "stateOperators" in QueryProcessEvent.
### Why are the changes needed?
Dropping late inputs means that end users might not get expected outputs. Even end users may indicate the fact and tolerate the result (as that's what allowed lateness is for), but they should be able to observe whether the current value of allowed lateness drops inputs or not so that they can adjust the value.
Also, whatever the chance they have multiple of stateful operators in a single query, if Spark drops late inputs "between" these operators, it becomes "correctness" issue. Spark should disallow such possibility, but given we already provided the flexibility, at least we should provide the way to observe the correctness issue and decide whether they should make correction of their query or not.
### Does this PR introduce _any_ user-facing change?
Yes. End users will be able to retrieve the information of late inputs via two ways:
1. SQL tab in Spark UI
2. Streaming Query Listener
### How was this patch tested?
New UTs added & existing UTs are modified to reflect the change.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #28607: [SPARK-24634][SS] Add a new metric regarding number of inputs later than watermark plus allowed delay
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #28607:
URL: https://github.com/apache/spark/pull/28607#issuecomment-643706416
**[Test build #123984 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123984/testReport)** for PR 28607 at commit [`4216405`](https://github.com/apache/spark/commit/4216405789c07f7ded54be05d5ecd8797ee05291).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HeartSaVioR commented on pull request #28607: [SPARK-24634][SS] Add a new metric regarding number of inputs later than watermark plus allowed delay
Posted by GitBox <gi...@apache.org>.
HeartSaVioR commented on pull request #28607:
URL: https://github.com/apache/spark/pull/28607#issuecomment-632529354
retest this, please
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28607: [SPARK-24634][SS] Add a new metric regarding number of inputs later than watermark plus allowed delay
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #28607:
URL: https://github.com/apache/spark/pull/28607#issuecomment-636604160
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28607: [SPARK-24634][SS] Add a new metric regarding number of inputs later than watermark plus allowed delay
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #28607:
URL: https://github.com/apache/spark/pull/28607#issuecomment-632531486
**[Test build #122973 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122973/testReport)** for PR 28607 at commit [`10d8d56`](https://github.com/apache/spark/commit/10d8d561fe28fad9f8c5bafd5daeae0488151609).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28607: [SPARK-24634][SS] Add a new metric regarding number of inputs later than watermark plus allowed delay
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #28607:
URL: https://github.com/apache/spark/pull/28607#issuecomment-636564586
**[Test build #123357 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123357/testReport)** for PR 28607 at commit [`4216405`](https://github.com/apache/spark/commit/4216405789c07f7ded54be05d5ecd8797ee05291).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28607: [SPARK-24634][SS] Add a new metric regarding number of inputs later than watermark plus allowed delay
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #28607:
URL: https://github.com/apache/spark/pull/28607#issuecomment-636257556
**[Test build #123306 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123306/testReport)** for PR 28607 at commit [`d31e86e`](https://github.com/apache/spark/commit/d31e86e1de65ce9b6e44e5eb95e07dca1b5ac00c).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28607: [SPARK-24634][SS] Add a new metric regarding number of inputs later than watermark plus allowed delay
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #28607:
URL: https://github.com/apache/spark/pull/28607#issuecomment-634307478
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] xuanyuanking commented on pull request #28607: [SPARK-24634][SS] Add a new metric regarding number of inputs later than watermark plus allowed delay
Posted by GitBox <gi...@apache.org>.
xuanyuanking commented on pull request #28607:
URL: https://github.com/apache/spark/pull/28607#issuecomment-635383169
Thanks for reviving, I think it's an important metrics. How about also attach a screenshot of SQL UI contains this metric?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28607: [SPARK-24634][SS] Add a new metric regarding number of inputs later than watermark plus allowed delay
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28607:
URL: https://github.com/apache/spark/pull/28607#issuecomment-636257785
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28607: [SPARK-24634][SS] Add a new metric regarding number of inputs later than watermark plus allowed delay
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28607:
URL: https://github.com/apache/spark/pull/28607#issuecomment-632638453
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28607: [SPARK-24634][SS] Add a new metric regarding number of inputs later than watermark plus allowed delay
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28607:
URL: https://github.com/apache/spark/pull/28607#issuecomment-634307478
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] zsxwing commented on a change in pull request #28607: [SPARK-24634][SS] Add a new metric regarding number of inputs later than watermark plus allowed delay
Posted by GitBox <gi...@apache.org>.
zsxwing commented on a change in pull request #28607:
URL: https://github.com/apache/spark/pull/28607#discussion_r439853232
##########
File path: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/statefulOperators.scala
##########
@@ -77,6 +77,8 @@ trait StateStoreWriter extends StatefulOperator { self: SparkPlan =>
override lazy val metrics = Map(
"numOutputRows" -> SQLMetrics.createMetric(sparkContext, "number of output rows"),
+ "numLateInputs" -> SQLMetrics.createMetric(sparkContext,
Review comment:
nit: maybe name it as `numDroppedRowsByWatermark`? Technically, we don't guarantee to drop all late inputs. So this metric doesn't report the accurate number of late inputs.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28607: [SPARK-24634][SS] Add a new metric regarding number of inputs later than watermark plus allowed delay
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #28607:
URL: https://github.com/apache/spark/pull/28607#issuecomment-632638453
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28607: [SPARK-24634][SS] Add a new metric regarding number of inputs later than watermark plus allowed delay
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #28607:
URL: https://github.com/apache/spark/pull/28607#issuecomment-632503711
**[Test build #122967 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122967/testReport)** for PR 28607 at commit [`9cc0443`](https://github.com/apache/spark/commit/9cc0443ac9921b1ab883c2507418a755c3c85843).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HeartSaVioR commented on a change in pull request #28607: [SPARK-24634][SS] Add a new metric regarding number of inputs later than watermark plus allowed delay
Posted by GitBox <gi...@apache.org>.
HeartSaVioR commented on a change in pull request #28607:
URL: https://github.com/apache/spark/pull/28607#discussion_r439861427
##########
File path: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/statefulOperators.scala
##########
@@ -77,6 +77,8 @@ trait StateStoreWriter extends StatefulOperator { self: SparkPlan =>
override lazy val metrics = Map(
"numOutputRows" -> SQLMetrics.createMetric(sparkContext, "number of output rows"),
+ "numLateInputs" -> SQLMetrics.createMetric(sparkContext,
Review comment:
Thanks for the suggestion. Yeah I guess I tried to explain the behavior but the name seems to be still confusing to others. I agree the suggested name is clearer.
Btw, would we be better to have accurate number of late inputs? (Just asking because #24936 covers this, and can be applied orthogonally like num of late inputs vs num of dropped inputs.)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28607: [SPARK-24634][SS] Add a new metric regarding number of inputs later than watermark plus allowed delay
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #28607:
URL: https://github.com/apache/spark/pull/28607#issuecomment-636564830
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28607: [SPARK-24634][SS] Add a new metric regarding number of inputs later than watermark plus allowed delay
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28607:
URL: https://github.com/apache/spark/pull/28607#issuecomment-643721071
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28607: [SPARK-24634][SS] Add a new metric regarding number of inputs later than watermark plus allowed delay
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #28607:
URL: https://github.com/apache/spark/pull/28607#issuecomment-643720854
**[Test build #123984 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123984/testReport)** for PR 28607 at commit [`4216405`](https://github.com/apache/spark/commit/4216405789c07f7ded54be05d5ecd8797ee05291).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28607: [SPARK-24634][SS] Add a new metric regarding number of inputs later than watermark plus allowed delay
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #28607:
URL: https://github.com/apache/spark/pull/28607#issuecomment-632504184
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28607: [SPARK-24634][SS] Add a new metric regarding number of inputs later than watermark plus allowed delay
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28607:
URL: https://github.com/apache/spark/pull/28607#issuecomment-636564830
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #28607: [SPARK-24634][SS] Add a new metric regarding number of inputs later than watermark plus allowed delay
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #28607:
URL: https://github.com/apache/spark/pull/28607#issuecomment-636453588
**[Test build #123345 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123345/testReport)** for PR 28607 at commit [`03dc461`](https://github.com/apache/spark/commit/03dc46165003d76efec23a8ddf18ab8708141526).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28607: [SPARK-24634][SS] Add a new metric regarding number of inputs later than watermark plus allowed delay
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28607:
URL: https://github.com/apache/spark/pull/28607#issuecomment-632508761
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/122967/
Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28607: [SPARK-24634][SS] Add a new metric regarding number of inputs later than watermark plus allowed delay
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28607:
URL: https://github.com/apache/spark/pull/28607#issuecomment-632508754
Merged build finished. Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HeartSaVioR commented on pull request #28607: [SPARK-24634][SS] Add a new metric regarding number of inputs later than watermark plus allowed delay
Posted by GitBox <gi...@apache.org>.
HeartSaVioR commented on pull request #28607:
URL: https://github.com/apache/spark/pull/28607#issuecomment-643706118
retest this, please
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] xuanyuanking commented on a change in pull request #28607: [SPARK-24634][SS] Add a new metric regarding number of inputs later than watermark plus allowed delay
Posted by GitBox <gi...@apache.org>.
xuanyuanking commented on a change in pull request #28607:
URL: https://github.com/apache/spark/pull/28607#discussion_r431873551
##########
File path: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/statefulOperators.scala
##########
@@ -132,6 +135,16 @@ trait StateStoreWriter extends StatefulOperator { self: SparkPlan =>
}.toMap
}
+ protected def applyRemovingRowsOlderThanWatermark(
+ iter: Iterator[InternalRow],
+ predicate: BasePredicate): Iterator[InternalRow] = {
+ iter.filter { row =>
+ val filteredIn = !predicate.eval(row)
Review comment:
nit: little confusing of the variable naming here, maybe use a name related with watermark? `inWatermark`?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28607: [SPARK-24634][SS] Add a new metric regarding number of inputs later than watermark plus allowed delay
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #28607:
URL: https://github.com/apache/spark/pull/28607#issuecomment-643706485
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28607: [SPARK-24634][SS] Add a new metric regarding number of inputs later than watermark plus allowed delay
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28607:
URL: https://github.com/apache/spark/pull/28607#issuecomment-636453015
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28607: [SPARK-24634][SS] Add a new metric regarding number of inputs later than watermark plus allowed delay
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28607:
URL: https://github.com/apache/spark/pull/28607#issuecomment-632537942
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/122973/
Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28607: [SPARK-24634][SS] Add a new metric regarding number of inputs later than watermark plus allowed delay
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28607:
URL: https://github.com/apache/spark/pull/28607#issuecomment-643706485
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28607: [SPARK-24634][SS] Add a new metric regarding number of inputs later than watermark plus allowed delay
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28607:
URL: https://github.com/apache/spark/pull/28607#issuecomment-636273700
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28607: [SPARK-24634][SS] Add a new metric regarding number of inputs later than watermark plus allowed delay
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #28607:
URL: https://github.com/apache/spark/pull/28607#issuecomment-632551099
**[Test build #122981 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122981/testReport)** for PR 28607 at commit [`f09a623`](https://github.com/apache/spark/commit/f09a623064f8a1edf6908f653206d17db3f38d38).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28607: [SPARK-24634][SS] Add a new metric regarding number of inputs later than watermark plus allowed delay
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28607:
URL: https://github.com/apache/spark/pull/28607#issuecomment-632523837
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28607: [SPARK-24634][SS] Add a new metric regarding number of inputs later than watermark plus allowed delay
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28607:
URL: https://github.com/apache/spark/pull/28607#issuecomment-632551765
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28607: [SPARK-24634][SS] Add a new metric regarding number of inputs later than watermark plus allowed delay
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #28607:
URL: https://github.com/apache/spark/pull/28607#issuecomment-632523332
**[Test build #122971 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122971/testReport)** for PR 28607 at commit [`10d8d56`](https://github.com/apache/spark/commit/10d8d561fe28fad9f8c5bafd5daeae0488151609).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28607: [SPARK-24634][SS] Add a new metric regarding number of inputs later than watermark plus allowed delay
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #28607:
URL: https://github.com/apache/spark/pull/28607#issuecomment-632523837
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28607: [SPARK-24634][SS] Add a new metric regarding number of inputs later than watermark plus allowed delay
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #28607:
URL: https://github.com/apache/spark/pull/28607#issuecomment-636603727
**[Test build #123357 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123357/testReport)** for PR 28607 at commit [`4216405`](https://github.com/apache/spark/commit/4216405789c07f7ded54be05d5ecd8797ee05291).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #28607: [SPARK-24634][SS] Add a new metric regarding number of inputs later than watermark plus allowed delay
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #28607:
URL: https://github.com/apache/spark/pull/28607#issuecomment-632523332
**[Test build #122971 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122971/testReport)** for PR 28607 at commit [`10d8d56`](https://github.com/apache/spark/commit/10d8d561fe28fad9f8c5bafd5daeae0488151609).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon closed pull request #28607: [SPARK-24634][SS] Add a new metric regarding number of inputs later than watermark plus allowed delay
Posted by GitBox <gi...@apache.org>.
HyukjinKwon closed pull request #28607:
URL: https://github.com/apache/spark/pull/28607
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HeartSaVioR commented on pull request #28607: [SPARK-24634][SS] Add a new metric regarding number of inputs later than watermark plus allowed delay
Posted by GitBox <gi...@apache.org>.
HeartSaVioR commented on pull request #28607:
URL: https://github.com/apache/spark/pull/28607#issuecomment-632503485
I just revive #21617 (instead of pushing forward on #24936) because this approach is straightforward and pretty much simpler compared to #24936, though #24936 provides exact number.
#21617 got good tractions from reviewers, except the fact the representation of "input rows" are not exactly correct. I'm trying to cover such thing in this patch.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28607: [SPARK-24634][SS] Add a new metric regarding number of inputs later than watermark plus allowed delay
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #28607:
URL: https://github.com/apache/spark/pull/28607#issuecomment-632537894
**[Test build #122973 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122973/testReport)** for PR 28607 at commit [`10d8d56`](https://github.com/apache/spark/commit/10d8d561fe28fad9f8c5bafd5daeae0488151609).
* This patch **fails MiMa tests**.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HeartSaVioR commented on pull request #28607: [SPARK-24634][SS] Add a new metric regarding number of inputs later than watermark plus allowed delay
Posted by GitBox <gi...@apache.org>.
HeartSaVioR commented on pull request #28607:
URL: https://github.com/apache/spark/pull/28607#issuecomment-636302758
Thanks for reviewing the PR. I attached the SQL UI screenshot which has late inputs.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28607: [SPARK-24634][SS] Add a new metric regarding number of inputs later than watermark plus allowed delay
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #28607:
URL: https://github.com/apache/spark/pull/28607#issuecomment-636273700
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HeartSaVioR commented on a change in pull request #28607: [SPARK-24634][SS] Add a new metric regarding number of inputs later than watermark plus allowed delay
Posted by GitBox <gi...@apache.org>.
HeartSaVioR commented on a change in pull request #28607:
URL: https://github.com/apache/spark/pull/28607#discussion_r439870225
##########
File path: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/statefulOperators.scala
##########
@@ -77,6 +77,8 @@ trait StateStoreWriter extends StatefulOperator { self: SparkPlan =>
override lazy val metrics = Map(
"numOutputRows" -> SQLMetrics.createMetric(sparkContext, "number of output rows"),
+ "numLateInputs" -> SQLMetrics.createMetric(sparkContext,
Review comment:
#28828
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28607: [SPARK-24634][SS] Add a new metric regarding number of inputs later than watermark plus allowed delay
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #28607:
URL: https://github.com/apache/spark/pull/28607#issuecomment-643721071
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28607: [SPARK-24634][SS] Add a new metric regarding number of inputs later than watermark plus allowed delay
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28607:
URL: https://github.com/apache/spark/pull/28607#issuecomment-632537933
Merged build finished. Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #28607: [SPARK-24634][SS] Add a new metric regarding number of inputs later than watermark plus allowed delay
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #28607:
URL: https://github.com/apache/spark/pull/28607#issuecomment-632551099
**[Test build #122981 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122981/testReport)** for PR 28607 at commit [`f09a623`](https://github.com/apache/spark/commit/f09a623064f8a1edf6908f653206d17db3f38d38).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28607: [SPARK-24634][SS] Add a new metric regarding number of inputs later than watermark plus allowed delay
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #28607:
URL: https://github.com/apache/spark/pull/28607#issuecomment-632508754
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28607: [SPARK-24634][SS] Add a new metric regarding number of inputs later than watermark plus allowed delay
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28607:
URL: https://github.com/apache/spark/pull/28607#issuecomment-636472387
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28607: [SPARK-24634][SS] Add a new metric regarding number of inputs later than watermark plus allowed delay
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #28607:
URL: https://github.com/apache/spark/pull/28607#issuecomment-632637523
**[Test build #122981 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122981/testReport)** for PR 28607 at commit [`f09a623`](https://github.com/apache/spark/commit/f09a623064f8a1edf6908f653206d17db3f38d38).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28607: [SPARK-24634][SS] Add a new metric regarding number of inputs later than watermark plus allowed delay
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #28607:
URL: https://github.com/apache/spark/pull/28607#issuecomment-636301599
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28607: [SPARK-24634][SS] Add a new metric regarding number of inputs later than watermark plus allowed delay
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #28607:
URL: https://github.com/apache/spark/pull/28607#issuecomment-643706416
**[Test build #123984 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123984/testReport)** for PR 28607 at commit [`4216405`](https://github.com/apache/spark/commit/4216405789c07f7ded54be05d5ecd8797ee05291).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HeartSaVioR commented on pull request #28607: [SPARK-24634][SS] Add a new metric regarding number of inputs later than watermark plus allowed delay
Posted by GitBox <gi...@apache.org>.
HeartSaVioR commented on pull request #28607:
URL: https://github.com/apache/spark/pull/28607#issuecomment-643725826
Thanks all for reviewing and merging!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28607: [SPARK-24634][SS] Add a new metric regarding number of inputs later than watermark plus allowed delay
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #28607:
URL: https://github.com/apache/spark/pull/28607#issuecomment-636453015
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28607: [SPARK-24634][SS] Add a new metric regarding number of inputs later than watermark plus allowed delay
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #28607:
URL: https://github.com/apache/spark/pull/28607#issuecomment-636316743
**[Test build #123315 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123315/testReport)** for PR 28607 at commit [`0f32541`](https://github.com/apache/spark/commit/0f3254113103621ac74a781ab1d6d473675080c8).
* This patch **fails Spark unit tests**.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28607: [SPARK-24634][SS] Add a new metric regarding number of inputs later than watermark plus allowed delay
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28607:
URL: https://github.com/apache/spark/pull/28607#issuecomment-636301599
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28607: [SPARK-24634][SS] Add a new metric regarding number of inputs later than watermark plus allowed delay
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #28607:
URL: https://github.com/apache/spark/pull/28607#issuecomment-632551765
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28607: [SPARK-24634][SS] Add a new metric regarding number of inputs later than watermark plus allowed delay
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28607:
URL: https://github.com/apache/spark/pull/28607#issuecomment-632528987
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/122971/
Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #28607: [SPARK-24634][SS] Add a new metric regarding number of inputs later than watermark plus allowed delay
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #28607:
URL: https://github.com/apache/spark/pull/28607#issuecomment-632531486
**[Test build #122973 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122973/testReport)** for PR 28607 at commit [`10d8d56`](https://github.com/apache/spark/commit/10d8d561fe28fad9f8c5bafd5daeae0488151609).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #28607: [SPARK-24634][SS] Add a new metric regarding number of inputs later than watermark plus allowed delay
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #28607:
URL: https://github.com/apache/spark/pull/28607#issuecomment-632503711
**[Test build #122967 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122967/testReport)** for PR 28607 at commit [`9cc0443`](https://github.com/apache/spark/commit/9cc0443ac9921b1ab883c2507418a755c3c85843).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28607: [SPARK-24634][SS] Add a new metric regarding number of inputs later than watermark plus allowed delay
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28607:
URL: https://github.com/apache/spark/pull/28607#issuecomment-636316857
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/123315/
Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28607: [SPARK-24634][SS] Add a new metric regarding number of inputs later than watermark plus allowed delay
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28607:
URL: https://github.com/apache/spark/pull/28607#issuecomment-636316854
Merged build finished. Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28607: [SPARK-24634][SS] Add a new metric regarding number of inputs later than watermark plus allowed delay
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #28607:
URL: https://github.com/apache/spark/pull/28607#issuecomment-636453588
**[Test build #123345 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123345/testReport)** for PR 28607 at commit [`03dc461`](https://github.com/apache/spark/commit/03dc46165003d76efec23a8ddf18ab8708141526).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28607: [SPARK-24634][SS] Add a new metric regarding number of inputs later than watermark plus allowed delay
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28607:
URL: https://github.com/apache/spark/pull/28607#issuecomment-632528976
Merged build finished. Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28607: [SPARK-24634][SS] Add a new metric regarding number of inputs later than watermark plus allowed delay
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #28607:
URL: https://github.com/apache/spark/pull/28607#issuecomment-636472387
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28607: [SPARK-24634][SS] Add a new metric regarding number of inputs later than watermark plus allowed delay
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28607:
URL: https://github.com/apache/spark/pull/28607#issuecomment-632532129
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #28607: [SPARK-24634][SS] Add a new metric regarding number of inputs later than watermark plus allowed delay
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #28607:
URL: https://github.com/apache/spark/pull/28607#issuecomment-636257556
**[Test build #123306 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123306/testReport)** for PR 28607 at commit [`d31e86e`](https://github.com/apache/spark/commit/d31e86e1de65ce9b6e44e5eb95e07dca1b5ac00c).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on pull request #28607: [SPARK-24634][SS] Add a new metric regarding number of inputs later than watermark plus allowed delay
Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on pull request #28607:
URL: https://github.com/apache/spark/pull/28607#issuecomment-643721610
Merged to master.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HeartSaVioR commented on pull request #28607: [SPARK-24634][SS] Add a new metric regarding number of inputs later than watermark plus allowed delay
Posted by GitBox <gi...@apache.org>.
HeartSaVioR commented on pull request #28607:
URL: https://github.com/apache/spark/pull/28607#issuecomment-634305984
retest this, please
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28607: [SPARK-24634][SS] Add a new metric regarding number of inputs later than watermark plus allowed delay
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #28607:
URL: https://github.com/apache/spark/pull/28607#issuecomment-636273532
**[Test build #123306 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123306/testReport)** for PR 28607 at commit [`d31e86e`](https://github.com/apache/spark/commit/d31e86e1de65ce9b6e44e5eb95e07dca1b5ac00c).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28607: [SPARK-24634][SS] Add a new metric regarding number of inputs later than watermark plus allowed delay
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #28607:
URL: https://github.com/apache/spark/pull/28607#issuecomment-632532129
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28607: [SPARK-24634][SS] Add a new metric regarding number of inputs later than watermark plus allowed delay
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #28607:
URL: https://github.com/apache/spark/pull/28607#issuecomment-632508729
**[Test build #122967 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122967/testReport)** for PR 28607 at commit [`9cc0443`](https://github.com/apache/spark/commit/9cc0443ac9921b1ab883c2507418a755c3c85843).
* This patch **fails MiMa tests**.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HeartSaVioR commented on a change in pull request #28607: [SPARK-24634][SS] Add a new metric regarding number of inputs later than watermark plus allowed delay
Posted by GitBox <gi...@apache.org>.
HeartSaVioR commented on a change in pull request #28607:
URL: https://github.com/apache/spark/pull/28607#discussion_r432797018
##########
File path: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/statefulOperators.scala
##########
@@ -132,6 +135,16 @@ trait StateStoreWriter extends StatefulOperator { self: SparkPlan =>
}.toMap
}
+ protected def applyRemovingRowsOlderThanWatermark(
+ iter: Iterator[InternalRow],
+ predicate: BasePredicate): Iterator[InternalRow] = {
+ iter.filter { row =>
+ val filteredIn = !predicate.eval(row)
Review comment:
Maybe it's clearer if we apply `filterNot` - then the variable name can be renamed to `lateInput` which is matched with metric name. Probably the variable `predicate` is also not clear which row will get `true`. Let me apply it.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28607: [SPARK-24634][SS] Add a new metric regarding number of inputs later than watermark plus allowed delay
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #28607:
URL: https://github.com/apache/spark/pull/28607#issuecomment-636316854
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #28607: [SPARK-24634][SS] Add a new metric regarding number of inputs later than watermark plus allowed delay
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #28607:
URL: https://github.com/apache/spark/pull/28607#issuecomment-636302226
**[Test build #123315 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123315/testReport)** for PR 28607 at commit [`0f32541`](https://github.com/apache/spark/commit/0f3254113103621ac74a781ab1d6d473675080c8).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28607: [SPARK-24634][SS] Add a new metric regarding number of inputs later than watermark plus allowed delay
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28607:
URL: https://github.com/apache/spark/pull/28607#issuecomment-636604160
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28607: [SPARK-24634][SS] Add a new metric regarding number of inputs later than watermark plus allowed delay
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28607:
URL: https://github.com/apache/spark/pull/28607#issuecomment-632504184
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28607: [SPARK-24634][SS] Add a new metric regarding number of inputs later than watermark plus allowed delay
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #28607:
URL: https://github.com/apache/spark/pull/28607#issuecomment-636302226
**[Test build #123315 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123315/testReport)** for PR 28607 at commit [`0f32541`](https://github.com/apache/spark/commit/0f3254113103621ac74a781ab1d6d473675080c8).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #28607: [SPARK-24634][SS] Add a new metric regarding number of inputs later than watermark plus allowed delay
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #28607:
URL: https://github.com/apache/spark/pull/28607#issuecomment-636564586
**[Test build #123357 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123357/testReport)** for PR 28607 at commit [`4216405`](https://github.com/apache/spark/commit/4216405789c07f7ded54be05d5ecd8797ee05291).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28607: [SPARK-24634][SS] Add a new metric regarding number of inputs later than watermark plus allowed delay
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #28607:
URL: https://github.com/apache/spark/pull/28607#issuecomment-634306964
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #28607: [SPARK-24634][SS] Add a new metric regarding number of inputs later than watermark plus allowed delay
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #28607:
URL: https://github.com/apache/spark/pull/28607#issuecomment-634306964
**[Test build #123140 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123140/testReport)** for PR 28607 at commit [`f09a623`](https://github.com/apache/spark/commit/f09a623064f8a1edf6908f653206d17db3f38d38).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28607: [SPARK-24634][SS] Add a new metric regarding number of inputs later than watermark plus allowed delay
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #28607:
URL: https://github.com/apache/spark/pull/28607#issuecomment-632528976
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28607: [SPARK-24634][SS] Add a new metric regarding number of inputs later than watermark plus allowed delay
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #28607:
URL: https://github.com/apache/spark/pull/28607#issuecomment-632537933
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28607: [SPARK-24634][SS] Add a new metric regarding number of inputs later than watermark plus allowed delay
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #28607:
URL: https://github.com/apache/spark/pull/28607#issuecomment-636257785
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28607: [SPARK-24634][SS] Add a new metric regarding number of inputs later than watermark plus allowed delay
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #28607:
URL: https://github.com/apache/spark/pull/28607#issuecomment-632528838
**[Test build #122971 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122971/testReport)** for PR 28607 at commit [`10d8d56`](https://github.com/apache/spark/commit/10d8d561fe28fad9f8c5bafd5daeae0488151609).
* This patch **fails due to an unknown error code, -9**.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28607: [SPARK-24634][SS] Add a new metric regarding number of inputs later than watermark plus allowed delay
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #28607:
URL: https://github.com/apache/spark/pull/28607#issuecomment-636472072
**[Test build #123345 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123345/testReport)** for PR 28607 at commit [`03dc461`](https://github.com/apache/spark/commit/03dc46165003d76efec23a8ddf18ab8708141526).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org