[spark] branch master updated: [SPARK-27219][CORE] Treat timeouts as fatal in SASL fallback path.

[do...@apache.org]

This is an automated email from the ASF dual-hosted git repository.

dongjoon pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git


The following commit(s) were added to refs/heads/master by this push:
     new db801cf  [SPARK-27219][CORE] Treat timeouts as fatal in SASL fallback path.
db801cf is described below

commit db801cf3f26a5452b341dafbcd3efea64d177d83
Author: Marcelo Vanzin <va...@cloudera.com>
AuthorDate: Sun Mar 24 21:49:54 2019 -0700

    [SPARK-27219][CORE] Treat timeouts as fatal in SASL fallback path.
    
    When a timeout happens we don't know what's the state of the remote end,
    so there is no point in doing anything else since it will most probably
    fail anyway.
    
    The change also demotes the log message printed when falling back to
    SASL, since a warning is too noisy for when the fallback is really
    needed (e.g. old shuffle service, or shuffle service with new auth
    disabled).
    
    Closes #24160 from vanzin/SPARK-27219.
    
    Authored-by: Marcelo Vanzin <va...@cloudera.com>
    Signed-off-by: Dongjoon Hyun <dh...@apple.com>
---
 .../spark/network/crypto/AuthClientBootstrap.java       | 17 ++++++++++++-----
 1 file changed, 12 insertions(+), 5 deletions(-)

diff --git a/common/network-common/src/main/java/org/apache/spark/network/crypto/AuthClientBootstrap.java b/common/network-common/src/main/java/org/apache/spark/network/crypto/AuthClientBootstrap.java
index 3c26378..77b167d 100644
--- a/common/network-common/src/main/java/org/apache/spark/network/crypto/AuthClientBootstrap.java
+++ b/common/network-common/src/main/java/org/apache/spark/network/crypto/AuthClientBootstrap.java
@@ -20,6 +20,7 @@ package org.apache.spark.network.crypto;
 import java.io.IOException;
 import java.nio.ByteBuffer;
 import java.security.GeneralSecurityException;
+import java.util.concurrent.TimeoutException;
 
 import com.google.common.base.Throwables;
 import io.netty.buffer.ByteBuf;
@@ -82,13 +83,19 @@ public class AuthClientBootstrap implements TransportClientBootstrap {
     } catch (RuntimeException e) {
       // There isn't a good exception that can be caught here to know whether it's really
       // OK to switch back to SASL (because the server doesn't speak the new protocol). So
-      // try it anyway, and in the worst case things will fail again.
-      if (conf.saslFallback()) {
-        LOG.warn("New auth protocol failed, trying SASL.", e);
-        doSaslAuth(client, channel);
-      } else {
+      // try it anyway, unless it's a timeout, which is locally fatal. In the worst case
+      // things will fail again.
+      if (!conf.saslFallback() || e.getCause() instanceof TimeoutException) {
         throw e;
       }
+
+      if (LOG.isDebugEnabled()) {
+        Throwable cause = e.getCause() != null ? e.getCause() : e;
+        LOG.debug("New auth protocol failed, trying SASL.", cause);
+      } else {
+        LOG.info("New auth protocol failed, trying SASL.");
+      }
+      doSaslAuth(client, channel);
     }
   }
 


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org