[Impala-ASF-CR] IMPALA-10557: Support Kudu's multi-row transaction

["Wenzhe Zhou (Code Review)" <ge...@cloudera.org>]

Wenzhe Zhou has uploaded a new patch set (#6). ( http://gerrit.cloudera.org:8080/17553 )

Change subject: IMPALA-10557: Support Kudu's multi-row transaction
......................................................................

IMPALA-10557: Support Kudu's multi-row transaction

Kudu added multi-row transaction so Impala could run query that inserts
multiple rows into Kudu's table in the context of a single transaction.
Kudu provides new Java/C++ client APIs to open/commit/rollback
transaction, create session with transaction, serialize/deserialize
metadata of transaction object. Kudu transaction object has built-in
heartbeater.

This patch added Impala support to use Kudu's multiple-row transaction.
 - Added a new query option to enable Kudu's transaction.
 - When the query option is set, a new Kudu transaction should be
   started for "insert" statements by Impala's frontend of coordinator.
 - The Kudu transaction objects are kept in KuduTransactionManager until
   the transactions are going to be aborted or committed.
 - Frontend serialize the transaction metadata into a binary and pass to
   executors.
 - Executors deserialize the binary and ingest via that transaction
   handle.
 - Coordinator commits the transaction after updating catalog if
   everything goes well. Otherwise, aborts the transaction.

Also changed code to store KuduClient as shared pointer since KuduClient
has to be passed as a shared pointer when KuduTransaction::Deserialize()
is called.

Testing:
 - Added e-to-e tests.
 - Passed core test.

Change-Id: I876ada48991afdff5d61b5d6a0417571aba7cb34
---
M be/src/exec/kudu-scan-node-base.h
M be/src/exec/kudu-table-sink.cc
M be/src/exec/kudu-table-sink.h
M be/src/exprs/kudu-partition-expr.cc
M be/src/exprs/kudu-partition-expr.h
M be/src/runtime/exec-env.cc
M be/src/runtime/exec-env.h
M be/src/service/client-request-state.cc
M be/src/service/client-request-state.h
M be/src/service/frontend.cc
M be/src/service/frontend.h
M be/src/service/query-options.cc
M be/src/service/query-options.h
M common/thrift/DataSinks.thrift
M common/thrift/ImpalaService.thrift
M common/thrift/Query.thrift
M fe/src/main/java/org/apache/impala/analysis/InsertStmt.java
A fe/src/main/java/org/apache/impala/common/KuduTransactionManager.java
M fe/src/main/java/org/apache/impala/common/TransactionException.java
M fe/src/main/java/org/apache/impala/planner/KuduTableSink.java
M fe/src/main/java/org/apache/impala/planner/TableSink.java
M fe/src/main/java/org/apache/impala/service/Frontend.java
M fe/src/main/java/org/apache/impala/service/JniFrontend.java
M testdata/cluster/node_templates/common/etc/kudu/master.conf.tmpl
M testdata/cluster/node_templates/common/etc/kudu/tserver.conf.tmpl
M tests/custom_cluster/test_kudu.py
26 files changed, 450 insertions(+), 62 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/53/17553/6
-- 
To view, visit http://gerrit.cloudera.org:8080/17553
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I876ada48991afdff5d61b5d6a0417571aba7cb34
Gerrit-Change-Number: 17553
Gerrit-PatchSet: 6
Gerrit-Owner: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Bikramjeet Vig <bi...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Qifan Chen <qc...@cloudera.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>