You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2020/04/01 23:16:11 UTC
[GitHub] [spark] mukulmurthy opened a new pull request #28095:
[SPARK-31324][SS] Include stream ID in the termination timeout error message
mukulmurthy opened a new pull request #28095: [SPARK-31324][SS] Include stream ID in the termination timeout error message
URL: https://github.com/apache/spark/pull/28095
### What changes were proposed in this pull request?
Include stream ID in the error thrown when a stream does not stop() in time.
### Why are the changes needed?
https://github.com/apache/spark/pull/26771/ added a conf to set a requested timeout for stopping a stream, after which the stop() method throws. From seeing this in a production use case with several streams running, it's helpful to include which stream failed to stop in the error message.
### Does this PR introduce any user-facing change?
If a stream times out when terminating, the error message now includes the stream ID.
Before:
`Stream Execution thread failed to stop within 2000 milliseconds (specified by spark.sql.streaming.stopTimeout). See the cause on what was being executed in the streaming query thread.`
After:
`Stream Execution thread for stream [id = 8513769d-b9d2-4902-9b36-3668bd022245, runId = 21ed8c35-9bfe-423f-853d-c022d91818bc] failed to stop within 2000 milliseconds (specified by spark.sql.streaming.stopTimeout). See the cause on what was being executed in the streaming query thread.`
### How was this patch tested?
Updated existing unit test
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #28095: [SPARK-31324][SS]
Include stream ID in the termination timeout error message
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #28095: [SPARK-31324][SS] Include stream ID in the termination timeout error message
URL: https://github.com/apache/spark/pull/28095#issuecomment-607602313
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #28095: [SPARK-31324][SS]
Include stream ID in the termination timeout error message
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #28095: [SPARK-31324][SS] Include stream ID in the termination timeout error message
URL: https://github.com/apache/spark/pull/28095#issuecomment-607602790
**[Test build #120697 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/120697/testReport)** for PR 28095 at commit [`062398e`](https://github.com/apache/spark/commit/062398e0f68d238c2d3c20276d9345053cc21083).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #28095: [SPARK-31324][SS]
Include stream ID in the termination timeout error message
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #28095: [SPARK-31324][SS] Include stream ID in the termination timeout error message
URL: https://github.com/apache/spark/pull/28095#issuecomment-607661650
**[Test build #120708 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/120708/testReport)** for PR 28095 at commit [`062398e`](https://github.com/apache/spark/commit/062398e0f68d238c2d3c20276d9345053cc21083).
* This patch **fails due to an unknown error code, -9**.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #28095:
[SPARK-31324][SS] Include stream ID in the termination timeout error message
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #28095: [SPARK-31324][SS] Include stream ID in the termination timeout error message
URL: https://github.com/apache/spark/pull/28095#issuecomment-607603286
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon closed pull request #28095: [SPARK-31324][SS]
Include stream ID in the termination timeout error message
Posted by GitBox <gi...@apache.org>.
HyukjinKwon closed pull request #28095: [SPARK-31324][SS] Include stream ID in the termination timeout error message
URL: https://github.com/apache/spark/pull/28095
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on issue #28095: [SPARK-31324][SS]
Include stream ID in the termination timeout error message
Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on issue #28095: [SPARK-31324][SS] Include stream ID in the termination timeout error message
URL: https://github.com/apache/spark/pull/28095#issuecomment-607602945
Merged to master and branch-3.0.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #28095:
[SPARK-31324][SS] Include stream ID in the termination timeout error message
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #28095: [SPARK-31324][SS] Include stream ID in the termination timeout error message
URL: https://github.com/apache/spark/pull/28095#issuecomment-607661989
Merged build finished. Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #28095: [SPARK-31324][SS]
Include stream ID in the termination timeout error message
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #28095: [SPARK-31324][SS] Include stream ID in the termination timeout error message
URL: https://github.com/apache/spark/pull/28095#issuecomment-607537035
**[Test build #120697 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/120697/testReport)** for PR 28095 at commit [`062398e`](https://github.com/apache/spark/commit/062398e0f68d238c2d3c20276d9345053cc21083).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #28095: [SPARK-31324][SS]
Include stream ID in the termination timeout error message
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #28095: [SPARK-31324][SS] Include stream ID in the termination timeout error message
URL: https://github.com/apache/spark/pull/28095#issuecomment-607661995
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/120708/
Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #28095:
[SPARK-31324][SS] Include stream ID in the termination timeout error message
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #28095: [SPARK-31324][SS] Include stream ID in the termination timeout error message
URL: https://github.com/apache/spark/pull/28095#issuecomment-607537274
Can one of the admins verify this patch?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #28095:
[SPARK-31324][SS] Include stream ID in the termination timeout error message
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on issue #28095: [SPARK-31324][SS] Include stream ID in the termination timeout error message
URL: https://github.com/apache/spark/pull/28095#issuecomment-607601973
**[Test build #120708 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/120708/testReport)** for PR 28095 at commit [`062398e`](https://github.com/apache/spark/commit/062398e0f68d238c2d3c20276d9345053cc21083).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on issue #28095: [SPARK-31324][SS]
Include stream ID in the termination timeout error message
Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on issue #28095: [SPARK-31324][SS] Include stream ID in the termination timeout error message
URL: https://github.com/apache/spark/pull/28095#issuecomment-607601318
ok to test
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #28095:
[SPARK-31324][SS] Include stream ID in the termination timeout error message
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #28095: [SPARK-31324][SS] Include stream ID in the termination timeout error message
URL: https://github.com/apache/spark/pull/28095#issuecomment-607602318
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/25406/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #28095:
[SPARK-31324][SS] Include stream ID in the termination timeout error message
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #28095: [SPARK-31324][SS] Include stream ID in the termination timeout error message
URL: https://github.com/apache/spark/pull/28095#issuecomment-607602313
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #28095: [SPARK-31324][SS]
Include stream ID in the termination timeout error message
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #28095: [SPARK-31324][SS] Include stream ID in the termination timeout error message
URL: https://github.com/apache/spark/pull/28095#issuecomment-607602318
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/25406/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #28095:
[SPARK-31324][SS] Include stream ID in the termination timeout error message
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #28095: [SPARK-31324][SS] Include stream ID in the termination timeout error message
URL: https://github.com/apache/spark/pull/28095#issuecomment-607603291
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/120697/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #28095: [SPARK-31324][SS]
Include stream ID in the termination timeout error message
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #28095: [SPARK-31324][SS] Include stream ID in the termination timeout error message
URL: https://github.com/apache/spark/pull/28095#issuecomment-607601973
**[Test build #120708 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/120708/testReport)** for PR 28095 at commit [`062398e`](https://github.com/apache/spark/commit/062398e0f68d238c2d3c20276d9345053cc21083).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #28095:
[SPARK-31324][SS] Include stream ID in the termination timeout error message
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #28095: [SPARK-31324][SS] Include stream ID in the termination timeout error message
URL: https://github.com/apache/spark/pull/28095#issuecomment-607661995
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/120708/
Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #28095: [SPARK-31324][SS]
Include stream ID in the termination timeout error message
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #28095: [SPARK-31324][SS] Include stream ID in the termination timeout error message
URL: https://github.com/apache/spark/pull/28095#issuecomment-607661989
Merged build finished. Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #28095: [SPARK-31324][SS]
Include stream ID in the termination timeout error message
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #28095: [SPARK-31324][SS] Include stream ID in the termination timeout error message
URL: https://github.com/apache/spark/pull/28095#issuecomment-607537274
Can one of the admins verify this patch?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #28095: [SPARK-31324][SS]
Include stream ID in the termination timeout error message
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #28095: [SPARK-31324][SS] Include stream ID in the termination timeout error message
URL: https://github.com/apache/spark/pull/28095#issuecomment-607603286
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #28095: [SPARK-31324][SS]
Include stream ID in the termination timeout error message
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #28095: [SPARK-31324][SS] Include stream ID in the termination timeout error message
URL: https://github.com/apache/spark/pull/28095#issuecomment-607603291
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/120697/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #28095:
[SPARK-31324][SS] Include stream ID in the termination timeout error message
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on issue #28095: [SPARK-31324][SS] Include stream ID in the termination timeout error message
URL: https://github.com/apache/spark/pull/28095#issuecomment-607537035
**[Test build #120697 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/120697/testReport)** for PR 28095 at commit [`062398e`](https://github.com/apache/spark/commit/062398e0f68d238c2d3c20276d9345053cc21083).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org