[impala] 08/12: IMPALA-5129: Use Kudu's Kinit code to avoid expensive fork

[ta...@apache.org]

This is an automated email from the ASF dual-hosted git repository.

tarmstrong pushed a commit to branch 2.x
in repository https://gitbox.apache.org/repos/asf/impala.git

commit b97e0cd555a53057a82dc9c0ad9e0cfe58f3ec66
Author: Sailesh Mukil <sa...@cloudera.com>
AuthorDate: Tue Mar 28 15:56:26 2017 -0700

    IMPALA-5129: Use Kudu's Kinit code to avoid expensive fork
    
    NOTE: This commit is part of a set of changes for IMPALA-7006. It
    contains pieces of a previous commit that need to be cherry picked
    again after rebasing the code in be/src/kudu/{util,security,rpc}.
    
    The original commit message is below:
    
    Impala currently kinits by forking off a child process. This
    has proved to be expensive in many cases since the subprocess
    tries to reserve as much memory as Impala is currently using
    which can be quite a lot.
    
    This patch adds a flag called 'use_kudu_kinit' that defaults to
    true. When it's true, it uses the Kudu security library's kinit code
    that programatically uses the krb5 library to kinit.
    When it's false, we run our current path which kicks off the
    kinit-thread and forks off a kinit process periodically to reacquire
    tickets based on FLAGS_kerberos_reinit_interval.
    
    Converted existing tests in thrift-server-test to run with and
    without kerberos. We now run this BE test with kerberos by using
    Kudu's MiniKdc utility. This introduces a new dependency on some
    kerberos binaries that are checked through FindKerberosPrograms.cmake.
    Note that this is only a test dependency and not a dependency for
    the impalad binaries and friends. Compilation will still succeed if
    the kerberos binaries for the MiniKdc are not found, however, the
    thrift-server-test will fail. We run with and without the
    'use_kudu_kinit' flag.
    
    TODO: Since the setting up and tearing down of our security code
    isn't idempotent, we can run only any one test in a process with
    Kerberos now (IMPALA-6085).
    
    Updated bin/bootstrap_system.sh to install new sasl-gssapi
    modules and the kerberos binaries required for the MiniKdc.
    Also fixed a bug that didn't transfer the environment into 'sudo'
    in bin/bootstrap_system.sh.
    
    Testing: Verified with thrift-server-test and also manually on a
    live kerberized cluster.
    
    Change-Id: Ie3c6e933c454e7adca69ef03e7d5c0c84b656895
    Reviewed-on: http://gerrit.cloudera.org:8080/7938
    Reviewed-by: Sailesh Mukil <sa...@cloudera.com>
    Tested-by: Impala Public Jenkins
    Reviewed-on: http://gerrit.cloudera.org:8080/10763
    Reviewed-by: Lars Volker <lv...@cloudera.com>
    Tested-by: Lars Volker <lv...@cloudera.com>
---
 be/src/kudu/security/CMakeLists.txt   | 9 +++++++++
 be/src/kudu/security/test/mini_kdc.cc | 4 +++-
 2 files changed, 12 insertions(+), 1 deletion(-)

diff --git a/be/src/kudu/security/CMakeLists.txt b/be/src/kudu/security/CMakeLists.txt
index b79486e..63d6341 100644
--- a/be/src/kudu/security/CMakeLists.txt
+++ b/be/src/kudu/security/CMakeLists.txt
@@ -94,6 +94,15 @@ ADD_EXPORTABLE_LIBRARY(security
   SRCS ${SECURITY_SRCS}
   DEPS ${SECURITY_LIBS})
 
+# Since Kudu tests are explicitly disabled, we want to expose some of their sources
+# to Impala using another variable.
+set(SECURITY_TEST_SRCS_FOR_IMPALA test/mini_kdc.cc)
+add_library(security-test-for-impala ${SECURITY_TEST_SRCS_FOR_IMPALA})
+target_link_libraries(security-test-for-impala
+  gutil
+  kudu_test_util
+  kudu_util
+  security)
 
 ##############################
 # mini_kdc
diff --git a/be/src/kudu/security/test/mini_kdc.cc b/be/src/kudu/security/test/mini_kdc.cc
index 904695a..4f987c5 100644
--- a/be/src/kudu/security/test/mini_kdc.cc
+++ b/be/src/kudu/security/test/mini_kdc.cc
@@ -61,7 +61,9 @@ MiniKdc::MiniKdc(MiniKdcOptions options)
     options_.realm = "KRBTEST.COM";
   }
   if (options_.data_root.empty()) {
-    options_.data_root = JoinPathSegments(GetTestDataDirectory(), "krb5kdc");
+    // We hardcode "/tmp" here since the original function which initializes a random test
+    // directory (GetTestDataDirectory()), depends on gmock.
+    options_.data_root = JoinPathSegments("/tmp", "krb5kdc");
   }
   if (options_.ticket_lifetime.empty()) {
     options_.ticket_lifetime = "24h";