[jira] [Resolved] (IMPALA-1638) Investigate using c++ templates option when generating thrift and increasing the transport buffer

["Tim Armstrong (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/IMPALA-1638?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Tim Armstrong resolved IMPALA-1638.
-----------------------------------
    Resolution: Won't Do

I was able to get this working but it didn't have a noticeable impact on performance as far as I could tell. I benchmarked impala-profile-tool on a release build using this:

{noformat}
./buildall.sh -release -noclean -notests -skiptests -ninja && ninja impala-profile-tool && while cat test-profile-logs/impala_profile_log* | time ./be/build/latest/util/impala-profile-tool --profile_format=text > /dev/
null; do date; done
{noformat}

    Before:
{noformat}
    4.65user 0.54system 0:05.22elapsed 99%CPU (0avgtext+0avgdata
    60672maxresident)k
    0inputs+0outputs (0major+83026minor)pagefaults 0swaps
    Tue 15 Dec 2020 03:05:48 PM PST
    4.53user 0.63system 0:05.19elapsed 99%CPU (0avgtext+0avgdata
    60428maxresident)k
    0inputs+0outputs (0major+83018minor)pagefaults 0swaps
    {noformat}
    With this change:
{noformat}
    4.67user 0.47system 0:05.17elapsed 99%CPU (0avgtext+0avgdata
    60560maxresident)k
    0inputs+0outputs (0major+83015minor)pagefaults 0swaps
    4.54user 0.52system 0:05.08elapsed 99%CPU (0avgtext+0avgdata
    60548maxresident)k
    0inputs+0outputs (0major+83010minor)pagefaults 0swaps
{noformat}

Diff:
{noformat}
tarmstrong@tarmstrong-Precision-7540:~/impala/impala$ git diff HEAD^^
diff --git a/be/src/rpc/thrift-util.h b/be/src/rpc/thrift-util.h
index 05e7e5544..086c690ed 100644
--- a/be/src/rpc/thrift-util.h
+++ b/be/src/rpc/thrift-util.h
@@ -21,6 +21,7 @@
 
 #include <boost/shared_ptr.hpp>
 #include <thrift/protocol/TBinaryProtocol.h>
+#include <thrift/protocol/TCompactProtocol.h>
 #include <sstream>
 #include <vector>
 #include <thrift/TApplicationException.h>
@@ -112,7 +113,11 @@ Status DeserializeThriftMsg(const uint8_t* buf, uint32_t* len, bool compact,
   boost::shared_ptr<apache::thrift::protocol::TProtocol> tproto =
       CreateDeserializeProtocol(tmem_transport, compact);
   try {
-    deserialized_msg->read(tproto.get());
+    if (compact) {
+      deserialized_msg->read(static_cast<apache::thrift::protocol::TCompactProtocol*>(tproto.get()));
+    } else {
+      deserialized_msg->read(tproto.get());
+    }
   } catch (std::exception& e) {
     std::stringstream msg;
     msg << "couldn't deserialize thrift msg:\n" << e.what();
diff --git a/common/thrift/CMakeLists.txt b/common/thrift/CMakeLists.txt
index 9b14ab58d..ba282363e 100644
--- a/common/thrift/CMakeLists.txt
+++ b/common/thrift/CMakeLists.txt
@@ -59,7 +59,7 @@ function(THRIFT_GEN VAR)
     # The java dependency is handled by maven.
     # We need to generate C++ src file for the parent dependencies using the "-r" option.
     set(CPP_ARGS ${THRIFT_INCLUDE_DIR_OPTION}
-        --gen cpp:moveable_types,no_default_operators -o ${BE_OUTPUT_DIR})
+        --gen cpp:moveable_types,no_default_operators,templates -o ${BE_OUTPUT_DIR})
     IF (THRIFT_FILE STREQUAL "beeswax.thrift")
       set(CPP_ARGS -r ${CPP_ARGS})
     ENDIF(THRIFT_FILE STREQUAL "beeswax.thrift")
{noformat}

In perf top I could see that switch from indirect to direct calls but most of the time seemed to be spent in memory allocation regardless.

> Investigate using c++ templates option when generating thrift and increasing the transport buffer
> -------------------------------------------------------------------------------------------------
>
>                 Key: IMPALA-1638
>                 URL: https://issues.apache.org/jira/browse/IMPALA-1638
>             Project: IMPALA
>          Issue Type: Task
>          Components: Distributed Exec
>    Affects Versions: Impala 2.2
>            Reporter: casey
>            Assignee: Tim Armstrong
>            Priority: Minor
>              Labels: performance
>
> While investigating the performance of "select * from tpch.lineitem" thrift seemed to be very slow. I did a benchmark comparing thrift and captnproto to transfer a total of 1gb using 1mb responses over the loopback. captnproto took ~0.5 seconds where thrift took ~6 seconds. Thrift was setup similar to how it's used in Impala. Todd was able to get the thrift timing down to ~1.7 seconds with a few simple tweaks that aren't used by Impala. The improvements are
> 1) Generate code using templates. Without this, thrift generates inheritance style code which results in a virtual call to read and write every data point (such as an int).
> 2) Use a framed transport. The problem was that even when using the buffered transport, the default buffer was too small (Impala also uses the defualt buffer size). It's possible all we need to do is increase the buffer size. Testing should be done on a real cluster since the loopback could give very different results.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)