You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2021/08/30 13:19:06 UTC
[GitHub] [spark] AngersZhuuuu opened a new pull request #33869: [SPARK-36615][SQL] We should register shutdown hook earlier when start SC
AngersZhuuuu opened a new pull request #33869:
URL: https://github.com/apache/spark/pull/33869
### What changes were proposed in this pull request?
Since user always use ctrl+c to stop a starting SC when register with yarn in client mode when resources are tight.
In this time, sc have not register the Shutdown hook, this cause we can't call sc.stop() when exit the application.
We should register the ShutdownHook earily
### Why are the changes needed?
Make the calling process more completed.
### Does this PR introduce _any_ user-facing change?
No
### How was this patch tested?
Not need
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33869: [SPARK-36615][CORE] Register shutdown hook earlier when start SC
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33869:
URL: https://github.com/apache/spark/pull/33869#issuecomment-922184417
**[Test build #143430 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/143430/testReport)** for PR 33869 at commit [`86f6fe6`](https://github.com/apache/spark/commit/86f6fe6258c534da8840acee48e7299c50db87bd).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33869: [SPARK-36615][CORE] Register shutdown hook earlier when start SC
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33869:
URL: https://github.com/apache/spark/pull/33869#issuecomment-908856558
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/47372/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] mridulm commented on pull request #33869: [SPARK-36615][CORE] Register shutdown hook earlier when start SC
Posted by GitBox <gi...@apache.org>.
mridulm commented on pull request #33869:
URL: https://github.com/apache/spark/pull/33869#issuecomment-909868666
Wont this not race against the main SparkContext initialization thread ?
IIRC shutdown hook was added last to ensure the spark context has completed initialization before it could get stopped.
+CC @tgravescs
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] yikf commented on pull request #33869: [SPARK-36615][CORE] We should register shutdown hook earlier when start SC
Posted by GitBox <gi...@apache.org>.
yikf commented on pull request #33869:
URL: https://github.com/apache/spark/pull/33869#issuecomment-908826186
LGTM
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33869: [SPARK-36615][CORE] Register shutdown hook earlier when start SC
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33869:
URL: https://github.com/apache/spark/pull/33869#issuecomment-909114920
**[Test build #142869 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/142869/testReport)** for PR 33869 at commit [`0b2c66c`](https://github.com/apache/spark/commit/0b2c66cafb2f2ffc03d23255210289b34cf34772).
* This patch **fails from timeout after a configured wait of `500m`**.
* This patch merges cleanly.
* This patch adds no public classes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #33869: [SPARK-36615][CORE] Register shutdown hook earlier when start SC
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #33869:
URL: https://github.com/apache/spark/pull/33869#issuecomment-909173780
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/47389/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33869: [SPARK-36615][CORE] Register shutdown hook earlier when start SC
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33869:
URL: https://github.com/apache/spark/pull/33869#issuecomment-909169500
Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/47389/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #33869: [SPARK-36615][CORE] Register shutdown hook earlier when start SC
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #33869:
URL: https://github.com/apache/spark/pull/33869#issuecomment-908858976
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/47372/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] Ngone51 commented on pull request #33869: [SPARK-36615][CORE] Register shutdown hook earlier when start SC
Posted by GitBox <gi...@apache.org>.
Ngone51 commented on pull request #33869:
URL: https://github.com/apache/spark/pull/33869#issuecomment-910307280
> I would hope its ok to call stop if it hasn't completely started up because it should just call stop on the individual components if they have been started, but we should double check that all the components handle that properly.
I think that's ok. SparkConext can handle the interrupt stop during initialization, see:
https://github.com/apache/spark/blob/0494dc90af48ce7da0625485a4dc6917a244d580/core/src/main/scala/org/apache/spark/SparkContext.scala#L363-L367
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #33869: [SPARK-36615][CORE] Register shutdown hook earlier when start SC
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #33869:
URL: https://github.com/apache/spark/pull/33869#issuecomment-922170391
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/47938/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33869: [SPARK-36615][CORE] Register shutdown hook earlier when start SC
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33869:
URL: https://github.com/apache/spark/pull/33869#issuecomment-909137655
**[Test build #142886 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/142886/testReport)** for PR 33869 at commit [`0b2c66c`](https://github.com/apache/spark/commit/0b2c66cafb2f2ffc03d23255210289b34cf34772).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #33869: [SPARK-36615][CORE] Register shutdown hook earlier when start SC
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #33869:
URL: https://github.com/apache/spark/pull/33869#issuecomment-921958727
**[Test build #143422 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/143422/testReport)** for PR 33869 at commit [`86f6fe6`](https://github.com/apache/spark/commit/86f6fe6258c534da8840acee48e7299c50db87bd).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33869: [SPARK-36615][CORE] Register shutdown hook earlier when start SC
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33869:
URL: https://github.com/apache/spark/pull/33869#issuecomment-922035891
**[Test build #143422 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/143422/testReport)** for PR 33869 at commit [`86f6fe6`](https://github.com/apache/spark/commit/86f6fe6258c534da8840acee48e7299c50db87bd).
* This patch **fails PySpark unit tests**.
* This patch merges cleanly.
* This patch adds no public classes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] mridulm commented on pull request #33869: [SPARK-36615][CORE] Register shutdown hook earlier when start SC
Posted by GitBox <gi...@apache.org>.
mridulm commented on pull request #33869:
URL: https://github.com/apache/spark/pull/33869#issuecomment-909868666
Wont this not race against the main SparkContext initialization thread ?
IIRC shutdown hook was added last to ensure the spark context has completed initialization before it could get stopped.
+CC @tgravescs
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on pull request #33869: [SPARK-36615][CORE] We should register shutdown hook earlier when start SC
Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on pull request #33869:
URL: https://github.com/apache/spark/pull/33869#issuecomment-908807366
cc @mridulm too FYI.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33869: [SPARK-36615][CORE] Register shutdown hook earlier when start SC
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33869:
URL: https://github.com/apache/spark/pull/33869#issuecomment-909114920
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AngersZhuuuu commented on pull request #33869: [SPARK-36615][SQL] We should register shutdown hook earlier when start SC
Posted by GitBox <gi...@apache.org>.
AngersZhuuuu commented on pull request #33869:
URL: https://github.com/apache/spark/pull/33869#issuecomment-908336653
ping @dongjoon-hyun @HyukjinKwon
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33869: [SPARK-36615][CORE] Register shutdown hook earlier when start SC
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33869:
URL: https://github.com/apache/spark/pull/33869#issuecomment-921958727
**[Test build #143422 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/143422/testReport)** for PR 33869 at commit [`86f6fe6`](https://github.com/apache/spark/commit/86f6fe6258c534da8840acee48e7299c50db87bd).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] mridulm edited a comment on pull request #33869: [SPARK-36615][CORE] Register shutdown hook earlier when start SC
Posted by GitBox <gi...@apache.org>.
mridulm edited a comment on pull request #33869:
URL: https://github.com/apache/spark/pull/33869#issuecomment-910560847
@tgravescs If the concern is blocking on `postStartHook`, moving the shutdown hook addition to just before that should be fine : since all other initialization is done by then.
Moving all the way to beginning would mean we can have both threads racing on the state updates : one trying to clean up and trying to initialize them.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on pull request #33869: [SPARK-36615][CORE] We should register shutdown hook earlier when start SC
Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on pull request #33869:
URL: https://github.com/apache/spark/pull/33869#issuecomment-908807366
cc @mridulm too FYI.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #33869: [SPARK-36615][CORE] Register shutdown hook earlier when start SC
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #33869:
URL: https://github.com/apache/spark/pull/33869#issuecomment-909270511
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/142886/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AngersZhuuuu commented on pull request #33869: [SPARK-36615][CORE] Register shutdown hook earlier when start SC
Posted by GitBox <gi...@apache.org>.
AngersZhuuuu commented on pull request #33869:
URL: https://github.com/apache/spark/pull/33869#issuecomment-921957964
> The PR has not made the change proposed [here](https://github.com/apache/spark/pull/33869#issuecomment-910560847) ?
> Namely, move adding shutdown hook to just before `postStartHook` (as that is what is blocking here) ?
Sorry for miss this, updated
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #33869: [SPARK-36615][CORE] Register shutdown hook earlier when start SC
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #33869:
URL: https://github.com/apache/spark/pull/33869#issuecomment-922184652
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/143430/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33869: [SPARK-36615][CORE] Register shutdown hook earlier when start SC
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33869:
URL: https://github.com/apache/spark/pull/33869#issuecomment-908842815
**[Test build #142869 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/142869/testReport)** for PR 33869 at commit [`0b2c66c`](https://github.com/apache/spark/commit/0b2c66cafb2f2ffc03d23255210289b34cf34772).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on pull request #33869: [SPARK-36615][CORE] We should register shutdown hook earlier when start SC
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on pull request #33869:
URL: https://github.com/apache/spark/pull/33869#issuecomment-908475697
Thank you for pinging me, @AngersZhuuuu .
cc @Ngone51
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] tgravescs commented on pull request #33869: [SPARK-36615][CORE] Register shutdown hook earlier when start SC
Posted by GitBox <gi...@apache.org>.
tgravescs commented on pull request #33869:
URL: https://github.com/apache/spark/pull/33869#issuecomment-912538506
sorry it looks like my link was wrong, yeah I mean to point to _taskScheduler.postStartHook() to just mean I assume this is where most people end up ctrl-c out if they see yarn isn't allocating them stuff.
where I assume most people could ctrl-c out. @AngersZhuuuu is that correct? I still didn't see a response as to what issue this causes?
@mridulm I would assume the ctrl-c would normally kill the init thread to run the shutdown hook, if it doesn't for some reason, I would think the worst case is we don't handle it in the stop call, but I assume that is no worse then what happens now with stop not called, is there something else I'm not thinking of that might happen?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #33869: [SPARK-36615][CORE] Register shutdown hook earlier when start SC
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #33869:
URL: https://github.com/apache/spark/pull/33869#issuecomment-908842815
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AngersZhuuuu commented on pull request #33869: [SPARK-36615][CORE] Register shutdown hook earlier when start SC
Posted by GitBox <gi...@apache.org>.
AngersZhuuuu commented on pull request #33869:
URL: https://github.com/apache/spark/pull/33869#issuecomment-912388612
Anymore suggestion?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #33869: [SPARK-36615][CORE] Register shutdown hook earlier when start SC
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #33869:
URL: https://github.com/apache/spark/pull/33869#issuecomment-922184652
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/143430/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33869: [SPARK-36615][SQL] We should register shutdown hook earlier when start SC
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33869:
URL: https://github.com/apache/spark/pull/33869#issuecomment-908341151
**[Test build #142859 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/142859/testReport)** for PR 33869 at commit [`0b2c66c`](https://github.com/apache/spark/commit/0b2c66cafb2f2ffc03d23255210289b34cf34772).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on pull request #33869: [SPARK-36615][CORE] We should register shutdown hook earlier when start SC
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on pull request #33869:
URL: https://github.com/apache/spark/pull/33869#issuecomment-908475697
Thank you for pinging me, @AngersZhuuuu .
cc @Ngone51
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #33869: [SPARK-36615][CORE] We should register shutdown hook earlier when start SC
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #33869:
URL: https://github.com/apache/spark/pull/33869#issuecomment-908733701
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/142859/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] mridulm commented on pull request #33869: [SPARK-36615][CORE] Register shutdown hook earlier when start SC
Posted by GitBox <gi...@apache.org>.
mridulm commented on pull request #33869:
URL: https://github.com/apache/spark/pull/33869#issuecomment-924248543
Merged to master.
Thanks for fixing this @AngersZhuuuu !
Thanks for the reviews @tgravescs, @Ngone51, @yikf, @jaceklaskowski, @dongjoon-hyun, @HyukjinKwon
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #33869: [SPARK-36615][CORE] Register shutdown hook earlier when start SC
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #33869:
URL: https://github.com/apache/spark/pull/33869#issuecomment-922170391
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/47938/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33869: [SPARK-36615][CORE] Register shutdown hook earlier when start SC
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33869:
URL: https://github.com/apache/spark/pull/33869#issuecomment-922169347
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/47938/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33869: [SPARK-36615][CORE] Register shutdown hook earlier when start SC
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33869:
URL: https://github.com/apache/spark/pull/33869#issuecomment-921984612
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/47929/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #33869: [SPARK-36615][CORE] Register shutdown hook earlier when start SC
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #33869:
URL: https://github.com/apache/spark/pull/33869#issuecomment-909133467
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/142869/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] tgravescs commented on pull request #33869: [SPARK-36615][CORE] Register shutdown hook earlier when start SC
Posted by GitBox <gi...@apache.org>.
tgravescs commented on pull request #33869:
URL: https://github.com/apache/spark/pull/33869#issuecomment-910292202
what problem are you seeing if they ctrl-c before spark context is started?
Yeah I think the scenario he is saying here is yarn client mode tries to start, they user sees it taking to long (presumably isn't allocating them resources because they are all used) so they ctrl-c out of it before the ApplicationMaster can be scheduled. I believe that would happen at https://github.com/apache/spark/blob/master/core/src/main/scala/org/apache/spark/SparkContext.scala#L646
I would hope its ok to call stop if it hasn't completely started up because it should just call stop on the individual components if they have been started, but we should double check that all the components handle that properly.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on pull request #33869: [SPARK-36615][CORE] Register shutdown hook earlier when start SC
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on pull request #33869:
URL: https://github.com/apache/spark/pull/33869#issuecomment-924356722
Thank you, @AngersZhuuuu and @mridulm and all.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #33869: [SPARK-36615][CORE] We should register shutdown hook earlier when start SC
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #33869:
URL: https://github.com/apache/spark/pull/33869#issuecomment-908733701
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/142859/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] mridulm edited a comment on pull request #33869: [SPARK-36615][CORE] Register shutdown hook earlier when start SC
Posted by GitBox <gi...@apache.org>.
mridulm edited a comment on pull request #33869:
URL: https://github.com/apache/spark/pull/33869#issuecomment-909868666
Wont this not race against the main SparkContext initialization thread ?
IIRC shutdown hook was added last to ensure the spark context has completed initialization before it could get stopped via shutdown hook.
+CC @tgravescs
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] mridulm edited a comment on pull request #33869: [SPARK-36615][CORE] Register shutdown hook earlier when start SC
Posted by GitBox <gi...@apache.org>.
mridulm edited a comment on pull request #33869:
URL: https://github.com/apache/spark/pull/33869#issuecomment-910560847
@tgravescs If the concern is blocking on `postStartHook`, moving the shutdown hook addition to just before that should be fine : since all other initialization is done by then.
Moving all the way to beginning would mean we can have both threads (sc init and shutdown hook) racing on the state updates : one trying to clean up and trying to initialize them.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33869: [SPARK-36615][CORE] Register shutdown hook earlier when start SC
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33869:
URL: https://github.com/apache/spark/pull/33869#issuecomment-909168267
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/47389/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33869: [SPARK-36615][SQL] We should register shutdown hook earlier when start SC
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33869:
URL: https://github.com/apache/spark/pull/33869#issuecomment-908369960
Kubernetes integration test unable to build dist.
exiting with code: 1
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/47362/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] mridulm commented on pull request #33869: [SPARK-36615][CORE] Register shutdown hook earlier when start SC
Posted by GitBox <gi...@apache.org>.
mridulm commented on pull request #33869:
URL: https://github.com/apache/spark/pull/33869#issuecomment-915428381
@tgravescs Shutdown would not stop existing threads - all will continue to run (daemon and non-daemon 'regular' threads and shutdown hooks) - and when all shutdown hooks are complete, vm shuts down.
A simple test to validate:
```
public class Test {
public static void main(String[] args) throws Exception {
Runtime.getRuntime().addShutdownHook(new Thread(new Runnable() {
public void run() {
System.out.println("From shutdown hook");
for (int i = 0;i < 10; i ++) {
try { Thread.sleep(1000); } catch (Exception ex) {}
System.out.println("from hook ... i = " + i);
}
}
}));
Thread th = new Thread(new Runnable() {
public void run() {
System.out.println("From busy thread");
int i = 0;
while (true) {
i += 1;
try { Thread.sleep(500); } catch (Exception ex) {}
System.out.println("from busy ... i = " + i);
}
}
});
th.setDaemon(true);
th.start();
int mainCount = 0;
while(true) {
System.out.println("Main thread ... " + mainCount);
mainCount += 1;
Thread.sleep(1000);
}
}
}
```
(Ctrl + C to initiate shutdown to expose messages continuing from all three threads until shutdown is complete).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AngersZhuuuu commented on pull request #33869: [SPARK-36615][CORE] Register shutdown hook earlier when start SC
Posted by GitBox <gi...@apache.org>.
AngersZhuuuu commented on pull request #33869:
URL: https://github.com/apache/spark/pull/33869#issuecomment-912886208
> @mridulm I would assume the ctrl-c would normally kill the init thread to run the shutdown hook, if it doesn't for some reason, I would think the worst case is we don't handle it in the stop call, but I assume that is no worse then what happens now with stop not called, is there something else I'm not thinking of that might happen?
ctrl+c with a graceful shutdown should be reasonable and in graceful shutdown all start with a stop is necessary.
For example, before `taskScheduler.postStartHook()` we have start event log writer, right? And we have post an application start event, in event log writer's `stop()`, we need to close writer and renameFile.
```
/**
* Stop logging events. The event log file will be renamed so that it loses the
* ".inprogress" suffix.
*/
override def stop(): Unit = {
closeWriter()
renameFile(new Path(inProgressPath), new Path(logPath), shouldOverwrite)
}
```
So we need to call this too when we use ctrl+c to stop application, right?
I know it won't impact the job or other user cared things.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] mridulm commented on pull request #33869: [SPARK-36615][CORE] Register shutdown hook earlier when start SC
Posted by GitBox <gi...@apache.org>.
mridulm commented on pull request #33869:
URL: https://github.com/apache/spark/pull/33869#issuecomment-910560847
@tgravescs If the concern is blocking on `postStartHook`, moving the shutdown hook addition to just before that should be fine : since all initialization is done by then.
Moving all the way to beginning would mean we can have both threads racing on the state updates : one trying to clean up and trying to initialize them.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] yikf commented on pull request #33869: [SPARK-36615][CORE] We should register shutdown hook earlier when start SC
Posted by GitBox <gi...@apache.org>.
yikf commented on pull request #33869:
URL: https://github.com/apache/spark/pull/33869#issuecomment-908826186
LGTM
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #33869: [SPARK-36615][SQL] We should register shutdown hook earlier when start SC
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #33869:
URL: https://github.com/apache/spark/pull/33869#issuecomment-908370006
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33869: [SPARK-36615][CORE] Register shutdown hook earlier when start SC
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33869:
URL: https://github.com/apache/spark/pull/33869#issuecomment-922170387
Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/47938/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] asfgit closed pull request #33869: [SPARK-36615][CORE] Register shutdown hook earlier when start SC
Posted by GitBox <gi...@apache.org>.
asfgit closed pull request #33869:
URL: https://github.com/apache/spark/pull/33869
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #33869: [SPARK-36615][CORE] Register shutdown hook earlier when start SC
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #33869:
URL: https://github.com/apache/spark/pull/33869#issuecomment-909133467
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/142869/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] mridulm edited a comment on pull request #33869: [SPARK-36615][CORE] Register shutdown hook earlier when start SC
Posted by GitBox <gi...@apache.org>.
mridulm edited a comment on pull request #33869:
URL: https://github.com/apache/spark/pull/33869#issuecomment-921954219
The PR has not made the change proposed [here](https://github.com/apache/spark/pull/33869#issuecomment-910560847) ?
Namely, move adding shutdown hook to just before `postStartHook` (as that is what is blocking here) ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33869: [SPARK-36615][CORE] Register shutdown hook earlier when start SC
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33869:
URL: https://github.com/apache/spark/pull/33869#issuecomment-909250349
**[Test build #142886 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/142886/testReport)** for PR 33869 at commit [`0b2c66c`](https://github.com/apache/spark/commit/0b2c66cafb2f2ffc03d23255210289b34cf34772).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33869: [SPARK-36615][CORE] Register shutdown hook earlier when start SC
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33869:
URL: https://github.com/apache/spark/pull/33869#issuecomment-922163172
**[Test build #143430 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/143430/testReport)** for PR 33869 at commit [`86f6fe6`](https://github.com/apache/spark/commit/86f6fe6258c534da8840acee48e7299c50db87bd).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AngersZhuuuu commented on pull request #33869: [SPARK-36615][CORE] Register shutdown hook earlier when start SC
Posted by GitBox <gi...@apache.org>.
AngersZhuuuu commented on pull request #33869:
URL: https://github.com/apache/spark/pull/33869#issuecomment-921844996
@tgravescs First, keep a graceful shutdown is necessary, second for the reason https://github.com/apache/spark/pull/33869#issuecomment-912886208, third, If user write some plugin in some component, this issue will cause these plugin won't call stop method. Fourth, since this pr won't impact current spark code too, why not make the code more better.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] tgravescs commented on pull request #33869: [SPARK-36615][CORE] Register shutdown hook earlier when start SC
Posted by GitBox <gi...@apache.org>.
tgravescs commented on pull request #33869:
URL: https://github.com/apache/spark/pull/33869#issuecomment-921808115
thanks for correcting me @mridulm
@AngersZhuuuu I still didn't see a response as to what issue this causes? }
Was the response:
> I know it won't impact the job or other user cared things.
?
If this isn't causing any issues why change it?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #33869: [SPARK-36615][CORE] Register shutdown hook earlier when start SC
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #33869:
URL: https://github.com/apache/spark/pull/33869#issuecomment-921990005
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/47929/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #33869: [SPARK-36615][CORE] Register shutdown hook earlier when start SC
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #33869:
URL: https://github.com/apache/spark/pull/33869#issuecomment-909133467
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #33869: [SPARK-36615][CORE] Register shutdown hook earlier when start SC
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #33869:
URL: https://github.com/apache/spark/pull/33869#issuecomment-908858976
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/47372/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #33869: [SPARK-36615][SQL] We should register shutdown hook earlier when start SC
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #33869:
URL: https://github.com/apache/spark/pull/33869#issuecomment-908370006
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/47362/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AngersZhuuuu commented on pull request #33869: [SPARK-36615][SQL] We should register shutdown hook earlier when start SC
Posted by GitBox <gi...@apache.org>.
AngersZhuuuu commented on pull request #33869:
URL: https://github.com/apache/spark/pull/33869#issuecomment-908336653
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #33869: [SPARK-36615][CORE] Register shutdown hook earlier when start SC
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #33869:
URL: https://github.com/apache/spark/pull/33869#issuecomment-909133467
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AngersZhuuuu commented on pull request #33869: [SPARK-36615][CORE] Register shutdown hook earlier when start SC
Posted by GitBox <gi...@apache.org>.
AngersZhuuuu commented on pull request #33869:
URL: https://github.com/apache/spark/pull/33869#issuecomment-922162915
retest this please
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #33869: [SPARK-36615][CORE] Register shutdown hook earlier when start SC
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #33869:
URL: https://github.com/apache/spark/pull/33869#issuecomment-922042504
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/143422/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #33869: [SPARK-36615][CORE] Register shutdown hook earlier when start SC
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #33869:
URL: https://github.com/apache/spark/pull/33869#issuecomment-922042504
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/143422/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #33869: [SPARK-36615][CORE] Register shutdown hook earlier when start SC
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #33869:
URL: https://github.com/apache/spark/pull/33869#issuecomment-922163172
**[Test build #143430 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/143430/testReport)** for PR 33869 at commit [`86f6fe6`](https://github.com/apache/spark/commit/86f6fe6258c534da8840acee48e7299c50db87bd).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33869: [SPARK-36615][CORE] Register shutdown hook earlier when start SC
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33869:
URL: https://github.com/apache/spark/pull/33869#issuecomment-921989948
Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/47929/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #33869: [SPARK-36615][CORE] Register shutdown hook earlier when start SC
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #33869:
URL: https://github.com/apache/spark/pull/33869#issuecomment-909270511
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/142886/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AngersZhuuuu commented on pull request #33869: [SPARK-36615][CORE] Register shutdown hook earlier when start SC
Posted by GitBox <gi...@apache.org>.
AngersZhuuuu commented on pull request #33869:
URL: https://github.com/apache/spark/pull/33869#issuecomment-909115433
retest this please
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #33869: [SPARK-36615][CORE] Register shutdown hook earlier when start SC
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #33869:
URL: https://github.com/apache/spark/pull/33869#issuecomment-908842815
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33869: [SPARK-36615][SQL] We should register shutdown hook earlier when start SC
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33869:
URL: https://github.com/apache/spark/pull/33869#issuecomment-908341151
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #33869: [SPARK-36615][CORE] Register shutdown hook earlier when start SC
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #33869:
URL: https://github.com/apache/spark/pull/33869#issuecomment-909137655
**[Test build #142886 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/142886/testReport)** for PR 33869 at commit [`0b2c66c`](https://github.com/apache/spark/commit/0b2c66cafb2f2ffc03d23255210289b34cf34772).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AngersZhuuuu commented on pull request #33869: [SPARK-36615][CORE] Register shutdown hook earlier when start SC
Posted by GitBox <gi...@apache.org>.
AngersZhuuuu commented on pull request #33869:
URL: https://github.com/apache/spark/pull/33869#issuecomment-908840919
retest this please
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #33869: [SPARK-36615][CORE] Register shutdown hook earlier when start SC
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #33869:
URL: https://github.com/apache/spark/pull/33869#issuecomment-908842815
**[Test build #142869 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/142869/testReport)** for PR 33869 at commit [`0b2c66c`](https://github.com/apache/spark/commit/0b2c66cafb2f2ffc03d23255210289b34cf34772).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33869: [SPARK-36615][CORE] Register shutdown hook earlier when start SC
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33869:
URL: https://github.com/apache/spark/pull/33869#issuecomment-908857159
Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/47372/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AngersZhuuuu commented on pull request #33869: [SPARK-36615][CORE] Register shutdown hook earlier when start SC
Posted by GitBox <gi...@apache.org>.
AngersZhuuuu commented on pull request #33869:
URL: https://github.com/apache/spark/pull/33869#issuecomment-909115433
retest this please
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] mridulm edited a comment on pull request #33869: [SPARK-36615][CORE] Register shutdown hook earlier when start SC
Posted by GitBox <gi...@apache.org>.
mridulm edited a comment on pull request #33869:
URL: https://github.com/apache/spark/pull/33869#issuecomment-909868666
Wont this not race against the main SparkContext initialization thread ?
IIRC shutdown hook was added last to ensure the spark context has completed initialization before it could get stopped via shutdown hook.
+CC @tgravescs
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #33869: [SPARK-36615][CORE] Register shutdown hook earlier when start SC
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #33869:
URL: https://github.com/apache/spark/pull/33869#issuecomment-909173780
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/47389/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] mridulm commented on pull request #33869: [SPARK-36615][CORE] Register shutdown hook earlier when start SC
Posted by GitBox <gi...@apache.org>.
mridulm commented on pull request #33869:
URL: https://github.com/apache/spark/pull/33869#issuecomment-921954219
The PR has not made the change proposed [here](https://github.com/apache/spark/pull/33869#issuecomment-910560847) ?
Namely, move adding shutdown hook to just before `postStartHook` ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AngersZhuuuu commented on pull request #33869: [SPARK-36615][CORE] Register shutdown hook earlier when start SC
Posted by GitBox <gi...@apache.org>.
AngersZhuuuu commented on pull request #33869:
URL: https://github.com/apache/spark/pull/33869#issuecomment-913328713
ping @tgravescs
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #33869: [SPARK-36615][CORE] Register shutdown hook earlier when start SC
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #33869:
URL: https://github.com/apache/spark/pull/33869#issuecomment-921990005
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/47929/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #33869: [SPARK-36615][SQL] We should register shutdown hook earlier when start SC
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #33869:
URL: https://github.com/apache/spark/pull/33869#issuecomment-908370006
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/47362/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #33869: [SPARK-36615][SQL] We should register shutdown hook earlier when start SC
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #33869:
URL: https://github.com/apache/spark/pull/33869#issuecomment-908370006
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org