[jira] [Resolved] (IMPALA-6728) The combination use_kudu_kinit=false and use_krpc=true crashes Impalad

["Michael Ho (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/IMPALA-6728?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Michael Ho resolved IMPALA-6728.
--------------------------------
       Resolution: Fixed
    Fix Version/s: Impala 2.12.0
                   Impala 3.0

> The combination use_kudu_kinit=false and use_krpc=true crashes Impalad
> ----------------------------------------------------------------------
>
>                 Key: IMPALA-6728
>                 URL: https://issues.apache.org/jira/browse/IMPALA-6728
>             Project: IMPALA
>          Issue Type: Sub-task
>          Components: Security
>    Affects Versions: Impala 3.0, Impala 2.12.0
>            Reporter: Michael Ho
>            Assignee: Michael Ho
>            Priority: Blocker
>              Labels: crash
>             Fix For: Impala 3.0, Impala 2.12.0
>
>
> Although this is a weird combination, it's possible to start Impala with the configuration combination use_kudu_kinit=false and use_krpc=true. In which case, Impala will crash with the following backtrace:
>  
> {noformat}
> (gdb) bt
> #0  0x00007f8b2f484885 in raise () from /lib64/libc.so.6
> #1  0x00007f8b2f485e61 in abort () from /lib64/libc.so.6
> #2  0x00007f8b31a6bf35 in os::abort(bool) () from /usr/java/jdk1.8.0_162/jre/lib/amd64/server/libjvm.so
> #3  0x00007f8b31c0ff33 in VMError::report_and_die() () from /usr/java/jdk1.8.0_162/jre/lib/amd64/server/libjvm.so
> #4  0x00007f8b31a7222f in JVM_handle_linux_signal () from /usr/java/jdk1.8.0_162/jre/lib/amd64/server/libjvm.so
> #5  0x00007f8b31a68253 in signalHandler(int, siginfo*, void*) () from /usr/java/jdk1.8.0_162/jre/lib/amd64/server/libjvm.so
> #6  <signal handler called>
> #7  0x00007f8b2f7d8c3a in pthread_rwlock_rdlock () from /lib64/libpthread.so.0
> #8  0x00000000011e9886 in kudu::rpc::WrapSaslCall(sasl_conn*, std::function<int ()> const&) ()
> #9  0x00000000011fbde8 in kudu::rpc::ServerNegotiation::InitSaslServer() ()
> #10 0x0000000001200ee0 in kudu::rpc::ServerNegotiation::AuthenticateBySasl(kudu::faststring*) ()
> #11 0x0000000001202ea2 in kudu::rpc::ServerNegotiation::Negotiate() ()
> #12 0x00000000011f5e9d in kudu::rpc::Negotiation::RunNegotiation(scoped_refptr<kudu::rpc::Connection> const&, kudu::TriStateFlag, kudu::TriStateFlag, kudu::MonoTime) ()
> #13 0x00000000011dab45 in kudu::internal::Invoker<4, kudu::internal::BindState<kudu::internal::RunnableAdapter<void (*)(scoped_refptr<kudu::rpc::Connection> const&, kudu::TriStateFlag, kudu::TriStateFlag, kudu::MonoTime)>, void (scoped_refptr<kudu::rpc::Connection> const&, kudu::TriStateFlag, kudu::TriStateFlag, kudu::MonoTime), void (scoped_refptr<kudu::rpc::Connection>, kudu::TriStateFlag, kudu::TriStateFlag, kudu::MonoTime)>, void (scoped_refptr<kudu::rpc::Connection> const&, kudu::TriStateFlag, kudu::TriStateFlag, kudu::MonoTime)>::Run(kudu::internal::BindStateBase*) ()
> #14 0x000000000127f3f8 in kudu::ThreadPool::DispatchThread(bool) ()
> #15 0x000000000117fed5 in kudu::Thread::SuperviseThread(void*) ()
> #16 0x00007f8b2f7d5806 in start_thread () from /lib64/libpthread.so.0
> #17 0x00007f8b2f53064d in clone () from /lib64/libc.so.6
> #18 0x0000000000000000 in ?? ()
> {noformat}
> Apparently, it crashed at the following line here in {{WrapSaslCall()}}. {{has_kerberos_keytab}} is true because we passed a non-empty keytab to the messenger builder.
> {noformat}
> Status WrapSaslCall(sasl_conn_t* conn, const std::function<int()>& call) {
>   // In many cases, the GSSAPI SASL plugin will generate a nice error
>   // message as a message logged at SASL_LOG_FAIL logging level, but then
>   // return a useless one in sasl_errstring(). So, we set a global thread-local
>   // variable to capture any auth failure log message while we make the
>   // call into the library.
>   //
>   // The thread-local thing is a bit of a hack, but the logging callback
>   // is set globally rather than on a per-connection basis.
>   string err;
>   g_auth_failure_capture = &err;
>   // Take the 'kerberos_reinit_lock' here to avoid a possible race with ticket renewal.
>   if (has_kerberos_keytab) kudu::security::KerberosReinitLock()->ReadLock(); <<----
>   int rc = call();
>   if (has_kerberos_keytab) kudu::security::KerberosReinitLock()->ReadUnlock();
>   g_auth_failure_capture = nullptr;
> {noformat}
> cc'ing [~sailesh]



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)