[GitHub] [spark] cxzl25 commented on issue #25078: [SPARK-28305][YARN] Request GetExecutorLossReason to use a smaller timeout parameter

[GitBox <gi...@apache.org>]

cxzl25 commented on issue #25078: [SPARK-28305][YARN] Request GetExecutorLossReason to use a smaller timeout parameter
URL: https://github.com/apache/spark/pull/25078#issuecomment-510433736
 
 
   
   In the yarn-client mode, the driver closes the AM connection, causing the entire job to exit, causing unnecessary failures.
   I searched the code and used the ask+recover method very rarely.
   
   Adjust parameters should be work.  
   ```
   spark.rpc.askTimeout 120s
   spark.rpc.io.connectionTimeout 130s
   ```
   Parameter priority
   
   | module                  | first                          | second                |
   | ----------------------- | ------------------------------ | --------------------- |
   | RpcEndpointRef#ask      | spark.rpc.askTimeout           | spark.network.timeout |
   | TransportChannelHandler | spark.rpc.io.connectionTimeout | spark.network.timeout |
   
   
   I found a problem with the abnormal exit of the yarn client mode last time, and found and fixed a problem. https://github.com/apache/spark/pull/23989
   In the production environment, this patch is used, and there is still a probability of abnormal exit.
   Later, I carefully looked at the logic of this piece and found the problem.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org