You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2021/01/27 13:44:48 UTC

[GitHub] [spark] attilapiros opened a new pull request #31363: [WIP][SPARK-34154][YARN] Fix flaky test in LocalityPlacementStrategySuite

attilapiros opened a new pull request #31363:
URL: https://github.com/apache/spark/pull/31363


   PLEASE DO NOT MERGE THIS!
   Try to reproduce the issue by running it 50 times and getting the stack trace for the thread which fails to stop. 


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] HyukjinKwon closed pull request #31363: [SPARK-34154][YARN] Extend LocalityPlacementStrategySuite's test with a timeout

Posted by GitBox <gi...@apache.org>.
HyukjinKwon closed pull request #31363:
URL: https://github.com/apache/spark/pull/31363


   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] AmplabJenkins removed a comment on pull request #31363: [SPARK-34154][YARN] Extend LocalityPlacementStrategySuite's test with a timeout

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31363:
URL: https://github.com/apache/spark/pull/31363#issuecomment-768485363


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39145/
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] AmplabJenkins removed a comment on pull request #31363: [WIP][SPARK-34154][YARN] Fix flaky test in LocalityPlacementStrategySuite

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31363:
URL: https://github.com/apache/spark/pull/31363#issuecomment-768448516


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134559/
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] HyukjinKwon commented on pull request #31363: [SPARK-34154][YARN] Extend LocalityPlacementStrategySuite's test with a timeout

Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on pull request #31363:
URL: https://github.com/apache/spark/pull/31363#issuecomment-768636101


   Let's merge and monitor the builds. The changes look making sense.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] HyukjinKwon commented on pull request #31363: [SPARK-34154][YARN] Extend LocalityPlacementStrategySuite's test with a timeout

Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on pull request #31363:
URL: https://github.com/apache/spark/pull/31363#issuecomment-768636190


   Merged to master, branch-3.1 and branch-3.0.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] AmplabJenkins commented on pull request #31363: [SPARK-34154][YARN] Extend LocalityPlacementStrategySuite's test with a timeout

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31363:
URL: https://github.com/apache/spark/pull/31363#issuecomment-768557732


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39147/
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] SparkQA commented on pull request #31363: [WIP][SPARK-34154][YARN] Fix flaky test in LocalityPlacementStrategySuite

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31363:
URL: https://github.com/apache/spark/pull/31363#issuecomment-768437448


   Kubernetes integration test starting
   URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39145/
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] SparkQA commented on pull request #31363: [SPARK-34154][YARN] Extend LocalityPlacementStrategySuite's test with a timeout

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31363:
URL: https://github.com/apache/spark/pull/31363#issuecomment-768611983


   **[Test build #134570 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134570/testReport)** for PR 31363 at commit [`b965b8a`](https://github.com/apache/spark/commit/b965b8ad379f161e0e089ed7ee7aaeefbaafe641).
    * This patch passes all tests.
    * This patch merges cleanly.
    * This patch adds no public classes.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] SparkQA commented on pull request #31363: [WIP][SPARK-34154][YARN] Fix flaky test in LocalityPlacementStrategySuite

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31363:
URL: https://github.com/apache/spark/pull/31363#issuecomment-768352788


   Kubernetes integration test starting
   URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39140/
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] SparkQA removed a comment on pull request #31363: [WIP][SPARK-34154][YARN] Fix flaky test in LocalityPlacementStrategySuite

Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #31363:
URL: https://github.com/apache/spark/pull/31363#issuecomment-768333480


   **[Test build #134554 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134554/testReport)** for PR 31363 at commit [`c6ca806`](https://github.com/apache/spark/commit/c6ca806673cfcc84f6cf416fc127edde5f1bf018).


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] attilapiros commented on pull request #31363: [SPARK-34154][YARN] Extend LocalityPlacementStrategySuite's test with a timeout

Posted by GitBox <gi...@apache.org>.
attilapiros commented on pull request #31363:
URL: https://github.com/apache/spark/pull/31363#issuecomment-768581351


   Unrelated error:
   ```
   [info] KafkaSourceStressSuite:
   [info] - stress test with multiple topics and partitions *** FAILED *** (37 seconds, 259 milliseconds)
   [info]   == Results ==
   [info]   !== Correct Answer - 106 ==   == Spark Answer - 101 ==
   [info]    struct<value:int>            struct<value:int>
   ...
   ```


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] AmplabJenkins removed a comment on pull request #31363: [WIP][SPARK-34154][YARN] Fix flaky test in LocalityPlacementStrategySuite

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31363:
URL: https://github.com/apache/spark/pull/31363#issuecomment-768365331


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134554/
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] HyukjinKwon commented on pull request #31363: [SPARK-34154][YARN] Extend LocalityPlacementStrategySuite's test with a timeout

Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on pull request #31363:
URL: https://github.com/apache/spark/pull/31363#issuecomment-769590677


   Thanks @attilapiros.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] SparkQA commented on pull request #31363: [SPARK-34154][YARN] Extend LocalityPlacementStrategySuite's test with a timeout

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31363:
URL: https://github.com/apache/spark/pull/31363#issuecomment-768536603


   Kubernetes integration test status failure
   URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39147/
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] HyukjinKwon commented on pull request #31363: [SPARK-34154][YARN] Extend LocalityPlacementStrategySuite's test with a timeout

Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on pull request #31363:
URL: https://github.com/apache/spark/pull/31363#issuecomment-768635234


   Oh, nice. Thanks @attilapiros!


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] SparkQA commented on pull request #31363: [SPARK-34154][YARN] Extend LocalityPlacementStrategySuite's test with a timeout

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31363:
URL: https://github.com/apache/spark/pull/31363#issuecomment-768492655


   **[Test build #134561 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134561/testReport)** for PR 31363 at commit [`b965b8a`](https://github.com/apache/spark/commit/b965b8ad379f161e0e089ed7ee7aaeefbaafe641).


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] AmplabJenkins commented on pull request #31363: [SPARK-34154][YARN] Extend LocalityPlacementStrategySuite's test with a timeout

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31363:
URL: https://github.com/apache/spark/pull/31363#issuecomment-768523384


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134561/
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] SparkQA commented on pull request #31363: [WIP][SPARK-34154][YARN] Fix flaky test in LocalityPlacementStrategySuite

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31363:
URL: https://github.com/apache/spark/pull/31363#issuecomment-768348150


   **[Test build #134554 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134554/testReport)** for PR 31363 at commit [`c6ca806`](https://github.com/apache/spark/commit/c6ca806673cfcc84f6cf416fc127edde5f1bf018).
    * This patch passes all tests.
    * This patch merges cleanly.
    * This patch adds no public classes.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] SparkQA commented on pull request #31363: [SPARK-34154][YARN] Extend LocalityPlacementStrategySuite's test with a timeout

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31363:
URL: https://github.com/apache/spark/pull/31363#issuecomment-768525194


   Kubernetes integration test starting
   URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39147/
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] SparkQA commented on pull request #31363: [WIP][SPARK-34154][YARN] Fix flaky test in LocalityPlacementStrategySuite

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31363:
URL: https://github.com/apache/spark/pull/31363#issuecomment-768456420


   Kubernetes integration test status success
   URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39145/
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] SparkQA commented on pull request #31363: [WIP][SPARK-34154][YARN] Fix flaky test in LocalityPlacementStrategySuite

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31363:
URL: https://github.com/apache/spark/pull/31363#issuecomment-768414005


   **[Test build #134559 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134559/testReport)** for PR 31363 at commit [`2c51957`](https://github.com/apache/spark/commit/2c519576862a43c990b9d8196f27b536f9f493a2).


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] attilapiros commented on pull request #31363: [SPARK-34154][YARN] Extend LocalityPlacementStrategySuite's test with a timeout

Posted by GitBox <gi...@apache.org>.
attilapiros commented on pull request #31363:
URL: https://github.com/apache/spark/pull/31363#issuecomment-769579261


   This was expected. 
   The goal was to avoid endless wait for the test to finish and get more info about the problem via a stack trace.
   
   And we have the stack trace:
   
   ```
   [info] - handle large number of containers and tasks (SPARK-18750) *** FAILED *** (30 seconds, 4 milliseconds)
   [info]   Failed with an exception or a timeout at thread join:
   [info]   
   [info]   java.lang.RuntimeException: Timeout at waiting for thread to stop (its stack trace is added to the exception)
   [info]   	at java.net.Inet6AddressImpl.lookupAllHostAddr(Native Method)
   [info]   	at java.net.InetAddress$2.lookupAllHostAddr(InetAddress.java:929)
   [info]   	at java.net.InetAddress.getAddressesFromNameService(InetAddress.java:1324)
   [info]   	at java.net.InetAddress.getAllByName0(InetAddress.java:1277)
   [info]   	at java.net.InetAddress.getAllByName(InetAddress.java:1193)
   [info]   	at java.net.InetAddress.getAllByName(InetAddress.java:1127)
   [info]   	at java.net.InetAddress.getByName(InetAddress.java:1077)
   [info]   	at org.apache.hadoop.net.NetUtils.normalizeHostName(NetUtils.java:568)
   [info]   	at org.apache.hadoop.net.NetUtils.normalizeHostNames(NetUtils.java:585)
   [info]   	at org.apache.hadoop.net.CachedDNSToSwitchMapping.resolve(CachedDNSToSwitchMapping.java:109)
   [info]   	at org.apache.spark.deploy.yarn.SparkRackResolver.coreResolve(SparkRackResolver.scala:75)
   [info]   	at org.apache.spark.deploy.yarn.SparkRackResolver.resolve(SparkRackResolver.scala:66)
   [info]   	at org.apache.spark.deploy.yarn.LocalityPreferredContainerPlacementStrategy.$anonfun$localityOfRequestedContainers$3(LocalityPreferredContainerPlacementStrategy.scala:142)
   [info]   	at org.apache.spark.deploy.yarn.LocalityPreferredContainerPlacementStrategy$$Lambda$658/1080992036.apply$mcVI$sp(Unknown Source)
   [info]   	at scala.collection.immutable.Range.foreach$mVc$sp(Range.scala:158)
   [info]   	at org.apache.spark.deploy.yarn.LocalityPreferredContainerPlacementStrategy.localityOfRequestedContainers(LocalityPreferredContainerPlacementStrategy.scala:138)
   [info]   	at org.apache.spark.deploy.yarn.LocalityPlacementStrategySuite.org$apache$spark$deploy$yarn$LocalityPlacementStrategySuite$$runTest(LocalityPlacementStrategySuite.scala:94)
   [info]   	at org.apache.spark.deploy.yarn.LocalityPlacementStrategySuite$$anon$1.run(LocalityPlacementStrategySuite.scala:40)
   [info]   	at java.lang.Thread.run(Thread.java:748) (LocalityPlacementStrategySuite.scala:61)
   ```
   
   I already suspected this must be related to DNS resolution see my [jira comment](https://issues.apache.org/jira/browse/SPARK-34154?focusedCommentId=17272990&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-17272990). 
   
   I am checking the possible solutions.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] SparkQA commented on pull request #31363: [WIP][SPARK-34154][YARN] Fix flaky test in LocalityPlacementStrategySuite

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31363:
URL: https://github.com/apache/spark/pull/31363#issuecomment-768427808


   **[Test build #134559 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134559/testReport)** for PR 31363 at commit [`2c51957`](https://github.com/apache/spark/commit/2c519576862a43c990b9d8196f27b536f9f493a2).
    * This patch passes all tests.
    * This patch merges cleanly.
    * This patch adds no public classes.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] attilapiros commented on pull request #31363: [WIP][SPARK-34154][YARN] Fix flaky test in LocalityPlacementStrategySuite

Posted by GitBox <gi...@apache.org>.
attilapiros commented on pull request #31363:
URL: https://github.com/apache/spark/pull/31363#issuecomment-768315179


   Well, I had no luck here. All of them successful:
   
   ```
   [info] LocalityPlacementStrategySuite:
   [info] - handle large number of containers and tasks (SPARK-18750) 1 (5 seconds, 422 milliseconds)
   [info] - handle large number of containers and tasks (SPARK-18750) 2 (341 milliseconds)
   [info] - handle large number of containers and tasks (SPARK-18750) 3 (297 milliseconds)
   [info] - handle large number of containers and tasks (SPARK-18750) 4 (293 milliseconds)
   [info] - handle large number of containers and tasks (SPARK-18750) 5 (279 milliseconds)
   [info] - handle large number of containers and tasks (SPARK-18750) 6 (283 milliseconds)
   [info] - handle large number of containers and tasks (SPARK-18750) 7 (278 milliseconds)
   [info] - handle large number of containers and tasks (SPARK-18750) 8 (283 milliseconds)
   [info] - handle large number of containers and tasks (SPARK-18750) 9 (278 milliseconds)
   [info] - handle large number of containers and tasks (SPARK-18750) 10 (270 milliseconds)
   [info] - handle large number of containers and tasks (SPARK-18750) 11 (283 milliseconds)
   [info] - handle large number of containers and tasks (SPARK-18750) 12 (274 milliseconds)
   [info] - handle large number of containers and tasks (SPARK-18750) 13 (272 milliseconds)
   [info] - handle large number of containers and tasks (SPARK-18750) 14 (286 milliseconds)
   [info] - handle large number of containers and tasks (SPARK-18750) 15 (270 milliseconds)
   [info] - handle large number of containers and tasks (SPARK-18750) 16 (282 milliseconds)
   [info] - handle large number of containers and tasks (SPARK-18750) 17 (274 milliseconds)
   [info] - handle large number of containers and tasks (SPARK-18750) 18 (698 milliseconds)
   [info] - handle large number of containers and tasks (SPARK-18750) 19 (1 second, 192 milliseconds)
   [info] - handle large number of containers and tasks (SPARK-18750) 20 (951 milliseconds)
   [info] - handle large number of containers and tasks (SPARK-18750) 21 (2 seconds, 353 milliseconds)
   [info] - handle large number of containers and tasks (SPARK-18750) 22 (327 milliseconds)
   [info] - handle large number of containers and tasks (SPARK-18750) 23 (273 milliseconds)
   [info] - handle large number of containers and tasks (SPARK-18750) 24 (277 milliseconds)
   [info] - handle large number of containers and tasks (SPARK-18750) 25 (274 milliseconds)
   [info] - handle large number of containers and tasks (SPARK-18750) 26 (277 milliseconds)
   [info] - handle large number of containers and tasks (SPARK-18750) 27 (278 milliseconds)
   [info] - handle large number of containers and tasks (SPARK-18750) 28 (278 milliseconds)
   [info] - handle large number of containers and tasks (SPARK-18750) 29 (281 milliseconds)
   [info] - handle large number of containers and tasks (SPARK-18750) 30 (279 milliseconds)
   [info] - handle large number of containers and tasks (SPARK-18750) 31 (277 milliseconds)
   [info] - handle large number of containers and tasks (SPARK-18750) 32 (275 milliseconds)
   [info] - handle large number of containers and tasks (SPARK-18750) 33 (276 milliseconds)
   [info] - handle large number of containers and tasks (SPARK-18750) 34 (278 milliseconds)
   [info] - handle large number of containers and tasks (SPARK-18750) 35 (276 milliseconds)
   [info] - handle large number of containers and tasks (SPARK-18750) 36 (278 milliseconds)
   [info] - handle large number of containers and tasks (SPARK-18750) 37 (277 milliseconds)
   [info] - handle large number of containers and tasks (SPARK-18750) 38 (278 milliseconds)
   [info] - handle large number of containers and tasks (SPARK-18750) 39 (275 milliseconds)
   [info] - handle large number of containers and tasks (SPARK-18750) 40 (581 milliseconds)
   [info] - handle large number of containers and tasks (SPARK-18750) 41 (1 second, 303 milliseconds)
   [info] - handle large number of containers and tasks (SPARK-18750) 42 (3 seconds, 642 milliseconds)
   [info] - handle large number of containers and tasks (SPARK-18750) 43 (276 milliseconds)
   [info] - handle large number of containers and tasks (SPARK-18750) 44 (277 milliseconds)
   [info] - handle large number of containers and tasks (SPARK-18750) 45 (289 milliseconds)
   [info] - handle large number of containers and tasks (SPARK-18750) 46 (278 milliseconds)
   [info] - handle large number of containers and tasks (SPARK-18750) 47 (275 milliseconds)
   [info] - handle large number of containers and tasks (SPARK-18750) 48 (274 milliseconds)
   [info] - handle large number of containers and tasks (SPARK-18750) 49 (283 milliseconds)
   [info] - handle large number of containers and tasks (SPARK-18750) 50 (279 milliseconds)
   ```


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] SparkQA commented on pull request #31363: [WIP][SPARK-34154][YARN] Fix flaky test in LocalityPlacementStrategySuite

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31363:
URL: https://github.com/apache/spark/pull/31363#issuecomment-768333480


   **[Test build #134554 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134554/testReport)** for PR 31363 at commit [`c6ca806`](https://github.com/apache/spark/commit/c6ca806673cfcc84f6cf416fc127edde5f1bf018).


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] attilapiros commented on pull request #31363: [SPARK-34154][YARN] Extend LocalityPlacementStrategySuite's test with a timeout

Posted by GitBox <gi...@apache.org>.
attilapiros commented on pull request #31363:
URL: https://github.com/apache/spark/pull/31363#issuecomment-768581622


   jenkins retest this please


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] AmplabJenkins commented on pull request #31363: [SPARK-34154][YARN] Extend LocalityPlacementStrategySuite's test with a timeout

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31363:
URL: https://github.com/apache/spark/pull/31363#issuecomment-768622002






----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] SparkQA commented on pull request #31363: [SPARK-34154][YARN] Extend LocalityPlacementStrategySuite's test with a timeout

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31363:
URL: https://github.com/apache/spark/pull/31363#issuecomment-768620488


   Kubernetes integration test status failure
   URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39155/
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] SparkQA removed a comment on pull request #31363: [SPARK-34154][YARN] Extend LocalityPlacementStrategySuite's test with a timeout

Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #31363:
URL: https://github.com/apache/spark/pull/31363#issuecomment-768492655


   **[Test build #134561 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134561/testReport)** for PR 31363 at commit [`b965b8a`](https://github.com/apache/spark/commit/b965b8ad379f161e0e089ed7ee7aaeefbaafe641).


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] AmplabJenkins removed a comment on pull request #31363: [SPARK-34154][YARN] Extend LocalityPlacementStrategySuite's test with a timeout

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31363:
URL: https://github.com/apache/spark/pull/31363#issuecomment-768622002






----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] SparkQA commented on pull request #31363: [SPARK-34154][YARN] Extend LocalityPlacementStrategySuite's test with a timeout

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31363:
URL: https://github.com/apache/spark/pull/31363#issuecomment-768507222


   **[Test build #134561 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134561/testReport)** for PR 31363 at commit [`b965b8a`](https://github.com/apache/spark/commit/b965b8ad379f161e0e089ed7ee7aaeefbaafe641).
    * This patch passes all tests.
    * This patch merges cleanly.
    * This patch adds no public classes.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] SparkQA commented on pull request #31363: [SPARK-34154][YARN] Extend LocalityPlacementStrategySuite's test with a timeout

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31363:
URL: https://github.com/apache/spark/pull/31363#issuecomment-768598866


   **[Test build #134570 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134570/testReport)** for PR 31363 at commit [`b965b8a`](https://github.com/apache/spark/commit/b965b8ad379f161e0e089ed7ee7aaeefbaafe641).


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] dongjoon-hyun commented on pull request #31363: [SPARK-34154][YARN] Extend LocalityPlacementStrategySuite's test with a timeout

Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on pull request #31363:
URL: https://github.com/apache/spark/pull/31363#issuecomment-769435311


   Unfortunately, this is still flaky in master branch.
   - https://github.com/apache/spark/runs/1787277785
   ```
   [info] *** 1 TEST FAILED ***
   [error] Failed: Total 137, Failed 1, Errors 0, Passed 136, Canceled 1
   [error] Failed tests:
   [error] 	org.apache.spark.deploy.yarn.LocalityPlacementStrategySuite
   [error] (yarn / Test / test) sbt.TestsFailedException: Tests unsuccessful
   [error] Total time: 567 s (09:27), completed Jan 28, 2021 10:08:42 PM
   ```


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] AmplabJenkins removed a comment on pull request #31363: [SPARK-34154][YARN] Extend LocalityPlacementStrategySuite's test with a timeout

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31363:
URL: https://github.com/apache/spark/pull/31363#issuecomment-768557732


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39147/
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] AmplabJenkins commented on pull request #31363: [WIP][SPARK-34154][YARN] Fix flaky test in LocalityPlacementStrategySuite

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31363:
URL: https://github.com/apache/spark/pull/31363#issuecomment-768373131


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39140/
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] attilapiros edited a comment on pull request #31363: [SPARK-34154][YARN] Extend LocalityPlacementStrategySuite's test with a timeout

Posted by GitBox <gi...@apache.org>.
attilapiros edited a comment on pull request #31363:
URL: https://github.com/apache/spark/pull/31363#issuecomment-769579261


   This was expected. 
   The goal was to avoid endless wait for the test to finish and get more info about the problem via a stack trace.
   
   And we have the stack trace:
   
   ```
   [info] - handle large number of containers and tasks (SPARK-18750) *** FAILED *** (30 seconds, 4 milliseconds)
   [info]   Failed with an exception or a timeout at thread join:
   [info]   
   [info]   java.lang.RuntimeException: Timeout at waiting for thread to stop (its stack trace is added to the exception)
   [info]   	at java.net.Inet6AddressImpl.lookupAllHostAddr(Native Method)
   [info]   	at java.net.InetAddress$2.lookupAllHostAddr(InetAddress.java:929)
   [info]   	at java.net.InetAddress.getAddressesFromNameService(InetAddress.java:1324)
   [info]   	at java.net.InetAddress.getAllByName0(InetAddress.java:1277)
   [info]   	at java.net.InetAddress.getAllByName(InetAddress.java:1193)
   [info]   	at java.net.InetAddress.getAllByName(InetAddress.java:1127)
   [info]   	at java.net.InetAddress.getByName(InetAddress.java:1077)
   [info]   	at org.apache.hadoop.net.NetUtils.normalizeHostName(NetUtils.java:568)
   [info]   	at org.apache.hadoop.net.NetUtils.normalizeHostNames(NetUtils.java:585)
   [info]   	at org.apache.hadoop.net.CachedDNSToSwitchMapping.resolve(CachedDNSToSwitchMapping.java:109)
   [info]   	at org.apache.spark.deploy.yarn.SparkRackResolver.coreResolve(SparkRackResolver.scala:75)
   [info]   	at org.apache.spark.deploy.yarn.SparkRackResolver.resolve(SparkRackResolver.scala:66)
   [info]   	at org.apache.spark.deploy.yarn.LocalityPreferredContainerPlacementStrategy.$anonfun$localityOfRequestedContainers$3(LocalityPreferredContainerPlacementStrategy.scala:142)
   [info]   	at org.apache.spark.deploy.yarn.LocalityPreferredContainerPlacementStrategy$$Lambda$658/1080992036.apply$mcVI$sp(Unknown Source)
   [info]   	at scala.collection.immutable.Range.foreach$mVc$sp(Range.scala:158)
   [info]   	at org.apache.spark.deploy.yarn.LocalityPreferredContainerPlacementStrategy.localityOfRequestedContainers(LocalityPreferredContainerPlacementStrategy.scala:138)
   [info]   	at org.apache.spark.deploy.yarn.LocalityPlacementStrategySuite.org$apache$spark$deploy$yarn$LocalityPlacementStrategySuite$$runTest(LocalityPlacementStrategySuite.scala:94)
   [info]   	at org.apache.spark.deploy.yarn.LocalityPlacementStrategySuite$$anon$1.run(LocalityPlacementStrategySuite.scala:40)
   [info]   	at java.lang.Thread.run(Thread.java:748) (LocalityPlacementStrategySuite.scala:61)
   ```
   
   I already suspected this must be related to DNS resolution see my [jira comment](https://issues.apache.org/jira/browse/SPARK-34154?focusedCommentId=17272990&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-17272990). 
   
   Now I am checking the possible solutions.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] SparkQA commented on pull request #31363: [WIP][SPARK-34154][YARN] Fix flaky test in LocalityPlacementStrategySuite

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31363:
URL: https://github.com/apache/spark/pull/31363#issuecomment-768373098


   Kubernetes integration test status success
   URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39140/
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] AmplabJenkins removed a comment on pull request #31363: [SPARK-34154][YARN] Extend LocalityPlacementStrategySuite's test with a timeout

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31363:
URL: https://github.com/apache/spark/pull/31363#issuecomment-768523384


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134561/
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] SparkQA removed a comment on pull request #31363: [WIP][SPARK-34154][YARN] Fix flaky test in LocalityPlacementStrategySuite

Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #31363:
URL: https://github.com/apache/spark/pull/31363#issuecomment-768414005


   **[Test build #134559 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134559/testReport)** for PR 31363 at commit [`2c51957`](https://github.com/apache/spark/commit/2c519576862a43c990b9d8196f27b536f9f493a2).


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] SparkQA commented on pull request #31363: [SPARK-34154][YARN] Extend LocalityPlacementStrategySuite's test with a timeout

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31363:
URL: https://github.com/apache/spark/pull/31363#issuecomment-768616129


   Kubernetes integration test starting
   URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39155/
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] AmplabJenkins commented on pull request #31363: [WIP][SPARK-34154][YARN] Fix flaky test in LocalityPlacementStrategySuite

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31363:
URL: https://github.com/apache/spark/pull/31363#issuecomment-768448516


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134559/
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] attilapiros commented on a change in pull request #31363: [WIP][SPARK-34154][YARN] Fix flaky test in LocalityPlacementStrategySuite

Posted by GitBox <gi...@apache.org>.
attilapiros commented on a change in pull request #31363:
URL: https://github.com/apache/spark/pull/31363#discussion_r565516665



##########
File path: resource-managers/yarn/src/test/scala/org/apache/spark/deploy/yarn/LocalityPlacementStrategySuite.scala
##########
@@ -33,7 +33,7 @@ class LocalityPlacementStrategySuite extends SparkFunSuite {
   test("handle large number of containers and tasks (SPARK-18750)") {
     // Run the test in a thread with a small stack size, since the original issue
     // surfaced as a StackOverflowError.
-    var error: Throwable = null
+    @volatile var error: Throwable = null

Review comment:
       As this is set from another thread so this must be either volatile or the access synchronized.




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] attilapiros commented on pull request #31363: [WIP][SPARK-34154][YARN] Fix flaky test in LocalityPlacementStrategySuite

Posted by GitBox <gi...@apache.org>.
attilapiros commented on pull request #31363:
URL: https://github.com/apache/spark/pull/31363#issuecomment-768477269


   @dongjoon-hyun you are welcome. 
   
   The code is ready for review / merge. 


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] SparkQA removed a comment on pull request #31363: [SPARK-34154][YARN] Extend LocalityPlacementStrategySuite's test with a timeout

Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #31363:
URL: https://github.com/apache/spark/pull/31363#issuecomment-768598866


   **[Test build #134570 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134570/testReport)** for PR 31363 at commit [`b965b8a`](https://github.com/apache/spark/commit/b965b8ad379f161e0e089ed7ee7aaeefbaafe641).


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] AmplabJenkins removed a comment on pull request #31363: [WIP][SPARK-34154][YARN] Fix flaky test in LocalityPlacementStrategySuite

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31363:
URL: https://github.com/apache/spark/pull/31363#issuecomment-768373131


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39140/
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] AmplabJenkins commented on pull request #31363: [SPARK-34154][YARN] Extend LocalityPlacementStrategySuite's test with a timeout

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31363:
URL: https://github.com/apache/spark/pull/31363#issuecomment-768485363


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39145/
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] AmplabJenkins commented on pull request #31363: [WIP][SPARK-34154][YARN] Fix flaky test in LocalityPlacementStrategySuite

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31363:
URL: https://github.com/apache/spark/pull/31363#issuecomment-768365331


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134554/
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org