You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by "xiongbo-sjtu (via GitHub)" <gi...@apache.org> on 2023/09/21 00:39:37 UTC
[GitHub] [spark] xiongbo-sjtu opened a new pull request, #43021: [SPARK-45227][CORE] Fix an issue with CoarseGrainedExecutorBackend wh…
xiongbo-sjtu opened a new pull request, #43021:
URL: https://github.com/apache/spark/pull/43021
### What changes were proposed in this pull request?
Fix a subtle thread-safety issue with CoarseGrainedExecutorBackend where an executor process randomly gets stuck
### Why are the changes needed?
For each executor, the single-threaded dispatcher can run into an "infinite loop" (as explained in the SPARK-45227). Once an executor process runs into a state, it'd stop launching tasks from the driver or reporting task status back.
### Does this PR introduce _any_ user-facing change?
No
### How was this patch tested?
Existing unit tests
### Was this patch authored or co-authored using generative AI tooling?
No
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] xiongbo-sjtu commented on a diff in pull request #43021: [SPARK-45227][CORE] Fix a subtle thread-safety issue with CoarseGrainedExecutorBackend
Posted by "xiongbo-sjtu (via GitHub)" <gi...@apache.org>.
xiongbo-sjtu commented on code in PR #43021:
URL: https://github.com/apache/spark/pull/43021#discussion_r1339339435
##########
core/src/test/scala/org/apache/spark/executor/CoarseGrainedExecutorBackendSuite.scala:
##########
@@ -302,7 +302,12 @@ class CoarseGrainedExecutorBackendSuite extends SparkFunSuite
resourceProfile = ResourceProfile.getOrCreateDefaultProfile(conf))
assert(backend.taskResources.isEmpty)
- val taskId = 1000000
+ // Ensure thread-safety (https://issues.apache.org/jira/browse/SPARK-45227)
+ assert(backend.taskResources.isInstanceOf[
+ ConcurrentMap[Long, Map[String, ResourceInformation]]
+ ])
Review Comment:
In the spirit of "disagree and commit", I've removed the assert statement and updated the pull request.
Please merge this fix to master. If possible, please help patch v3.3.1 and above. We have been waiting for the fix to get merged, before pulling the trigger to sync up our internal repo with the open-source repo.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] xiongbo-sjtu closed pull request #43021: [SPARK-45227][CORE] Fix a subtle thread-safety issue with CoarseGrainedExecutorBackend
Posted by "xiongbo-sjtu (via GitHub)" <gi...@apache.org>.
xiongbo-sjtu closed pull request #43021: [SPARK-45227][CORE] Fix a subtle thread-safety issue with CoarseGrainedExecutorBackend
URL: https://github.com/apache/spark/pull/43021
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] mridulm commented on pull request #43021: [SPARK-45227][CORE] Fix a subtle thread-safety issue with CoarseGrainedExecutorBackend
Posted by "mridulm (via GitHub)" <gi...@apache.org>.
mridulm commented on PR #43021:
URL: https://github.com/apache/spark/pull/43021#issuecomment-1735881946
The updated test is not checking for behavior (it is checking for the type of the variable, so imo not very useful), not sure how we can reliably trigger this behavior to test.
Thoughts @jiangxb1987 ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] xiongbo-sjtu commented on a diff in pull request #43021: [SPARK-45227][CORE] Fix a subtle thread-safety issue with CoarseGrainedExecutorBackend
Posted by "xiongbo-sjtu (via GitHub)" <gi...@apache.org>.
xiongbo-sjtu commented on code in PR #43021:
URL: https://github.com/apache/spark/pull/43021#discussion_r1337996350
##########
core/src/test/scala/org/apache/spark/executor/CoarseGrainedExecutorBackendSuite.scala:
##########
@@ -302,7 +302,12 @@ class CoarseGrainedExecutorBackendSuite extends SparkFunSuite
resourceProfile = ResourceProfile.getOrCreateDefaultProfile(conf))
assert(backend.taskResources.isEmpty)
- val taskId = 1000000
+ // Ensure thread-safety (https://issues.apache.org/jira/browse/SPARK-45227)
+ assert(backend.taskResources.isInstanceOf[
+ ConcurrentMap[Long, Map[String, ResourceInformation]]
+ ])
Review Comment:
The above `assert` logic was added so that the thread-safety data structure (i.e., CHM) will not be unintentionally refactored away later. That's why I think that it's better for us to keep the logic here.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] mridulm commented on a diff in pull request #43021: [SPARK-45227][CORE] Fix a subtle thread-safety issue with CoarseGrainedExecutorBackend
Posted by "mridulm (via GitHub)" <gi...@apache.org>.
mridulm commented on code in PR #43021:
URL: https://github.com/apache/spark/pull/43021#discussion_r1338031595
##########
core/src/test/scala/org/apache/spark/executor/CoarseGrainedExecutorBackendSuite.scala:
##########
@@ -302,7 +302,12 @@ class CoarseGrainedExecutorBackendSuite extends SparkFunSuite
resourceProfile = ResourceProfile.getOrCreateDefaultProfile(conf))
assert(backend.taskResources.isEmpty)
- val taskId = 1000000
+ // Ensure thread-safety (https://issues.apache.org/jira/browse/SPARK-45227)
+ assert(backend.taskResources.isInstanceOf[
+ ConcurrentMap[Long, Map[String, ResourceInformation]]
+ ])
Review Comment:
We should add a comment on the field about thread safety, and remove this assertion.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] xiongbo-sjtu commented on pull request #43021: [SPARK-45227][CORE] Fix a subtle thread-safety issue with CoarseGrainedExecutorBackend
Posted by "xiongbo-sjtu (via GitHub)" <gi...@apache.org>.
xiongbo-sjtu commented on PR #43021:
URL: https://github.com/apache/spark/pull/43021#issuecomment-1740112354
> @HeartSaVioR, @HyukjinKwon I am seeing multiple PR's failing [with this](https://github.com/xiongbo-sjtu/spark/actions/runs/6332587036/job/17203381073#step:10:24799). It looks like some timeout issue, did anything change recently which might be triggering it ?
>
> This PR should not be impacted by those modules, but I would have prefer clean builds before merging.
All failed tests have passed after I rerun them. Solving the fragile unit tests is an orthogonal concern.
Any more concern or blocker for merging this pull request? @mridulm
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] xiongbo-sjtu commented on a diff in pull request #43021: [SPARK-45227][CORE] Fix a subtle thread-safety issue with CoarseGrainedExecutorBackend
Posted by "xiongbo-sjtu (via GitHub)" <gi...@apache.org>.
xiongbo-sjtu commented on code in PR #43021:
URL: https://github.com/apache/spark/pull/43021#discussion_r1337996350
##########
core/src/test/scala/org/apache/spark/executor/CoarseGrainedExecutorBackendSuite.scala:
##########
@@ -302,7 +302,12 @@ class CoarseGrainedExecutorBackendSuite extends SparkFunSuite
resourceProfile = ResourceProfile.getOrCreateDefaultProfile(conf))
assert(backend.taskResources.isEmpty)
- val taskId = 1000000
+ // Ensure thread-safety (https://issues.apache.org/jira/browse/SPARK-45227)
+ assert(backend.taskResources.isInstanceOf[
+ ConcurrentMap[Long, Map[String, ResourceInformation]]
+ ])
Review Comment:
The above `assert` logic was added so that the thread-safety data structure (i.e., CMH) will not be unintentionally refactored away later. That's why I think that it's better for us to keep the logic here.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] xiongbo-sjtu commented on pull request #43021: [SPARK-45227][CORE] Fix a subtle thread-safety issue with CoarseGrainedExecutorBackend
Posted by "xiongbo-sjtu (via GitHub)" <gi...@apache.org>.
xiongbo-sjtu commented on PR #43021:
URL: https://github.com/apache/spark/pull/43021#issuecomment-1736610019
> I think it would be good to add the content of [SPARK-45227](https://issues.apache.org/jira/browse/SPARK-45227) here to the PR's description.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] xiongbo-sjtu commented on pull request #43021: [SPARK-45227][CORE] Fix a subtle thread-safety issue with CoarseGrainedExecutorBackend
Posted by "xiongbo-sjtu (via GitHub)" <gi...@apache.org>.
xiongbo-sjtu commented on PR #43021:
URL: https://github.com/apache/spark/pull/43021#issuecomment-1732099260
@jiangxb1987 @mridulm
Eventually got all tests passed in Github Actions. Any concern on merging this pull request?
As a side note, I've discovered [another minor issue](https://issues.apache.org/jira/browse/SPARK-45283), but will address that in another pull request.
Thanks,
Bo
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] xiongbo-sjtu commented on pull request #43021: [SPARK-45227][CORE] Fix a subtle thread-safety issue with CoarseGrainedExecutorBackend
Posted by "xiongbo-sjtu (via GitHub)" <gi...@apache.org>.
xiongbo-sjtu commented on PR #43021:
URL: https://github.com/apache/spark/pull/43021#issuecomment-1736717071
Gentlemen, please 👍 if you're okay with getting this pull request merged.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] mridulm commented on pull request #43021: [SPARK-45227][CORE] Fix a subtle thread-safety issue with CoarseGrainedExecutorBackend
Posted by "mridulm (via GitHub)" <gi...@apache.org>.
mridulm commented on PR #43021:
URL: https://github.com/apache/spark/pull/43021#issuecomment-1739514252
@HeartSaVioR I am seeing multiple PR's failing [with this](https://github.com/xiongbo-sjtu/spark/actions/runs/6332587036/job/17203381073#step:10:24799). It looks like some timeout issue, did anything change recently which might be triggering it ?
This PR should not be impacted by those modules, but I would have prefer clean builds before merging.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] mridulm closed pull request #43021: [SPARK-45227][CORE] Fix a subtle thread-safety issue with CoarseGrainedExecutorBackend
Posted by "mridulm (via GitHub)" <gi...@apache.org>.
mridulm closed pull request #43021: [SPARK-45227][CORE] Fix a subtle thread-safety issue with CoarseGrainedExecutorBackend
URL: https://github.com/apache/spark/pull/43021
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] xiongbo-sjtu commented on pull request #43021: [SPARK-45227][CORE] Fix a subtle thread-safety issue with CoarseGrainedExecutorBackend
Posted by "xiongbo-sjtu (via GitHub)" <gi...@apache.org>.
xiongbo-sjtu commented on PR #43021:
URL: https://github.com/apache/spark/pull/43021#issuecomment-1740398612
Thanks. Just submit another pull request to patch branch-3.3
https://github.com/apache/spark/pull/43176
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] xiongbo-sjtu commented on pull request #43021: [SPARK-45227][CORE] Fix a subtle thread-safety with CoarseGrainedExecutorBackend
Posted by "xiongbo-sjtu (via GitHub)" <gi...@apache.org>.
xiongbo-sjtu commented on PR #43021:
URL: https://github.com/apache/spark/pull/43021#issuecomment-1730841001
@jiangxb1987 Good suggestion! I've updated the unit test accordingly.
Please merge this fix to mainline. If possible, please help patch v3.3.1 and above.
Thanks!
Bo (PE w/ Amazon)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] xiongbo-sjtu commented on pull request #43021: [SPARK-45227][CORE] Fix a subtle thread-safety issue with CoarseGrainedExecutorBackend
Posted by "xiongbo-sjtu (via GitHub)" <gi...@apache.org>.
xiongbo-sjtu commented on PR #43021:
URL: https://github.com/apache/spark/pull/43021#issuecomment-1736478403
+1 to the comment made by @JoshRosen
Our production fleet has tens of thousands Spark jobs running daily. After migrating those jobs from Spark 2 to Spark 3, we've observed the issue (i.e., thread dumps show that the dispatcher thread is stuck with the reported stack trace) only a handful of times.
The unit test was added so that the thread-safety will not be unintentionally refactored away later. It's not meant to simulate the race condition. The bottom line is that no functional regression is expected after this pull request replaces the vanilla mutable.hashmap with CHM.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] attilapiros commented on pull request #43021: [SPARK-45227][CORE] Fix a subtle thread-safety issue with CoarseGrainedExecutorBackend
Posted by "attilapiros (via GitHub)" <gi...@apache.org>.
attilapiros commented on PR #43021:
URL: https://github.com/apache/spark/pull/43021#issuecomment-1736599036
I think it would be good to add the content of [SPARK-45227](https://issues.apache.org/jira/browse/SPARK-45227) here to the PR's description.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] jiangxb1987 commented on pull request #43021: [SPARK-45227][CORE] Fix an issue with CoarseGrainedExecutorBackend wh…
Posted by "jiangxb1987 (via GitHub)" <gi...@apache.org>.
jiangxb1987 commented on PR #43021:
URL: https://github.com/apache/spark/pull/43021#issuecomment-1730455944
Thanks for the fix, is it possible to add an unit test to cover the failure?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] mridulm commented on a diff in pull request #43021: [SPARK-45227][CORE] Fix a subtle thread-safety issue with CoarseGrainedExecutorBackend
Posted by "mridulm (via GitHub)" <gi...@apache.org>.
mridulm commented on code in PR #43021:
URL: https://github.com/apache/spark/pull/43021#discussion_r1337971206
##########
core/src/test/scala/org/apache/spark/executor/CoarseGrainedExecutorBackendSuite.scala:
##########
@@ -302,7 +302,12 @@ class CoarseGrainedExecutorBackendSuite extends SparkFunSuite
resourceProfile = ResourceProfile.getOrCreateDefaultProfile(conf))
assert(backend.taskResources.isEmpty)
- val taskId = 1000000
+ // Ensure thread-safety (https://issues.apache.org/jira/browse/SPARK-45227)
+ assert(backend.taskResources.isInstanceOf[
+ ConcurrentMap[Long, Map[String, ResourceInformation]]
+ ])
Review Comment:
Do you want to drop this `assert` @xiongbo-sjtu ?
##########
core/src/test/scala/org/apache/spark/executor/CoarseGrainedExecutorBackendSuite.scala:
##########
@@ -302,7 +302,12 @@ class CoarseGrainedExecutorBackendSuite extends SparkFunSuite
resourceProfile = ResourceProfile.getOrCreateDefaultProfile(conf))
assert(backend.taskResources.isEmpty)
- val taskId = 1000000
+ // Ensure thread-safety (https://issues.apache.org/jira/browse/SPARK-45227)
+ assert(backend.taskResources.isInstanceOf[
+ ConcurrentMap[Long, Map[String, ResourceInformation]]
+ ])
Review Comment:
nit: Do you want to drop this `assert` @xiongbo-sjtu ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] xiongbo-sjtu commented on a diff in pull request #43021: [SPARK-45227][CORE] Fix a subtle thread-safety issue with CoarseGrainedExecutorBackend
Posted by "xiongbo-sjtu (via GitHub)" <gi...@apache.org>.
xiongbo-sjtu commented on code in PR #43021:
URL: https://github.com/apache/spark/pull/43021#discussion_r1337996350
##########
core/src/test/scala/org/apache/spark/executor/CoarseGrainedExecutorBackendSuite.scala:
##########
@@ -302,7 +302,12 @@ class CoarseGrainedExecutorBackendSuite extends SparkFunSuite
resourceProfile = ResourceProfile.getOrCreateDefaultProfile(conf))
assert(backend.taskResources.isEmpty)
- val taskId = 1000000
+ // Ensure thread-safety (https://issues.apache.org/jira/browse/SPARK-45227)
+ assert(backend.taskResources.isInstanceOf[
+ ConcurrentMap[Long, Map[String, ResourceInformation]]
+ ])
Review Comment:
The above `assert` logic was added so that the thread-safe data structure (i.e., CHM) will not be unintentionally refactored away later. That's why I think that it's better for us to keep the logic here.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] mridulm commented on pull request #43021: [SPARK-45227][CORE] Fix a subtle thread-safety issue with CoarseGrainedExecutorBackend
Posted by "mridulm (via GitHub)" <gi...@apache.org>.
mridulm commented on PR #43021:
URL: https://github.com/apache/spark/pull/43021#issuecomment-1740272530
Merged to master, 3.5 and 3.4
Thanks for working on this @xiongbo-sjtu !
Ran into conflicts for 3.3, can you create a follow up PR to fix it for branch-3.3 @xiongbo-sjtu ? Thanks.
Thanks for the reviews @JoshRosen, @jiangxb1987, @attilapiros :-)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org