[jira] [Created] (IMPALA-8749) Split TExecQueryFInstancesSidecar into query-wide and backend specific segments

["Joe McDonnell (JIRA)" <ji...@apache.org>]

Joe McDonnell created IMPALA-8749:
-------------------------------------

             Summary: Split TExecQueryFInstancesSidecar into query-wide and backend specific segments
                 Key: IMPALA-8749
                 URL: https://issues.apache.org/jira/browse/IMPALA-8749
             Project: IMPALA
          Issue Type: Improvement
          Components: Backend
    Affects Versions: Impala 3.3.0
            Reporter: Joe McDonnell


Currently, the TExecQueryFInstancesSidecar contains both a TQueryCtx (which is the same across all backends) and TPlanFragmentCtx/TPlanFragmentInstanceCtx lists (which are distinct across different backends). Right now, for each message to a backend, the coordinator needs to construct this sidecar and serialize it. This is wasteful, because it requires copying and serializing the TQueryCtx multiple times even though it is the same across backends.

If we split up the message into query-wide and backend-specific parts, then the main coordinator thread can serialize the TQueryCtx once and avoid serializing it for each backend. Additionally, serializing Thrift structures reduces the memory size, because unused optional fields are omitted. The copy of a serialized structure is also easier, because it is a single block of memory, rather than a large number of objects.

This query-wide information can be passed as a separate sidecar.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)