You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by "advancedxy (via GitHub)" <gi...@apache.org> on 2023/07/28 02:29:52 UTC
[GitHub] [spark] advancedxy opened a new pull request, #42195: [SPARK-44542][CORE] Eagerly load SparkExitCode class in exception handler
advancedxy opened a new pull request, #42195:
URL: https://github.com/apache/spark/pull/42195
### What changes were proposed in this pull request?
1. eagerly load SparkExitCode class in the the SparkUncaughtExceptionHandler
### Why are the changes needed?
In some extreme case, it's possible for SparkUncaughtExceptionHandler's exit/halt process function calls throw
an exception if the SparkExitCode is not loaded earlier, See corresponding jira: [SPARK-44542](https://issues.apache.org/jira/browse/SPARK-44542) for more details.
By eagerly load SparkExitCode class, we can make sure at least the halt/exit would work properly.
### Does this PR introduce _any_ user-facing change?
No
### How was this patch tested?
No logic change, hence no new UTs.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] advancedxy commented on pull request #42195: [SPARK-44542][CORE] Eagerly load SparkExitCode class in exception handler
Posted by "advancedxy (via GitHub)" <gi...@apache.org>.
advancedxy commented on PR #42195:
URL: https://github.com/apache/spark/pull/42195#issuecomment-1654886008
@LuciferYang would you mind to take a look at this?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] srowen commented on pull request #42195: [SPARK-44542][CORE] Eagerly load SparkExitCode class in exception handler
Posted by "srowen (via GitHub)" <gi...@apache.org>.
srowen commented on PR #42195:
URL: https://github.com/apache/spark/pull/42195#issuecomment-1657436642
Merged to master/3.5
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] advancedxy commented on pull request #42195: [SPARK-44542][CORE] Eagerly load SparkExitCode class in exception handler
Posted by "advancedxy (via GitHub)" <gi...@apache.org>.
advancedxy commented on PR #42195:
URL: https://github.com/apache/spark/pull/42195#issuecomment-1657413068
@srowen By any chance, could we merge this? Or let's left it open for some more days, so others would have a chance to review it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] LuciferYang commented on pull request #42195: [SPARK-44542][CORE] Eagerly load SparkExitCode class in exception handler
Posted by "LuciferYang (via GitHub)" <gi...@apache.org>.
LuciferYang commented on PR #42195:
URL: https://github.com/apache/spark/pull/42195#issuecomment-1655955746
cc @srowen
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] LuciferYang commented on pull request #42195: [SPARK-44542][CORE] Eagerly load SparkExitCode class in exception handler
Posted by "LuciferYang (via GitHub)" <gi...@apache.org>.
LuciferYang commented on PR #42195:
URL: https://github.com/apache/spark/pull/42195#issuecomment-1657436437
@srowen Can this be merged into Spark 3.5?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] srowen closed pull request #42195: [SPARK-44542][CORE] Eagerly load SparkExitCode class in exception handler
Posted by "srowen (via GitHub)" <gi...@apache.org>.
srowen closed pull request #42195: [SPARK-44542][CORE] Eagerly load SparkExitCode class in exception handler
URL: https://github.com/apache/spark/pull/42195
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] advancedxy commented on pull request #42195: [SPARK-44542][CORE] Eagerly load SparkExitCode class in exception handler
Posted by "advancedxy (via GitHub)" <gi...@apache.org>.
advancedxy commented on PR #42195:
URL: https://github.com/apache/spark/pull/42195#issuecomment-1656590671
> That's a pretty extreme scenario to try to work around. I'm concerned that this change isn't enough; in such a case I bet N other things fail. Are you sure this hack is enough?
Yeah, it's pretty extreme. A lot of things fails, for example, the `SparkFatalException` loading is failed too in such case.
However after this fix, the executor JVM at least has a chance to halt or exit theoretically.
> Is there any other way to load the class like a local import?
Tried for locally import, it wouldn't load class eagerly. See javap output below
<img width="1196" alt="image" src="https://github.com/apache/spark/assets/807537/96e4db1e-d194-422c-80ac-30f49b06785b">
The javap output for current impl:
<img width="1149" alt="image" src="https://github.com/apache/spark/assets/807537/cc69a44f-8295-44ee-b837-b78d745d5e17">
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org