You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@impala.apache.org by "Wenzhe Zhou (Code Review)" <ge...@cloudera.org> on 2021/10/28 23:52:02 UTC

[Impala-ASF-CR] WIP IMPALA-10931: Integrate protobuf 3.14.0 library with Impala 

Wenzhe Zhou has uploaded this change for review. ( http://gerrit.cloudera.org:8080/17948


Change subject: WIP IMPALA-10931: Integrate protobuf 3.14.0 library with Impala
......................................................................

WIP IMPALA-10931: Integrate protobuf 3.14.0 library with Impala

There are some API changes in the new version of protobuf library.
This patch makes necessary changes for Impala code to pass compiling.
It also adds a work around for the static initializer issue, which
is caused by the conflicts between two copies of protobuf library
in Impala's binary, one came from Kudu client shared library.

Testing:
 - Passed an core DEBUG build.
 - TODO pass exhaustive DEBUG build and a core ASAN build

Change-Id: Ia1df4faceff9fda169c9d15fe8b1e69cfabe0d43
---
M be/src/kudu/rpc/exactly_once_rpc-test.cc
M be/src/kudu/rpc/inbound_call.cc
M be/src/kudu/rpc/outbound_call.cc
M be/src/kudu/rpc/result_tracker.h
M be/src/kudu/rpc/serialization.cc
M be/src/kudu/rpc/serialization.h
M be/src/kudu/util/pb_util.cc
M be/src/kudu/util/protobuf_util.h
M be/src/rpc/rpc-mgr.cc
M bin/impala-config.sh
10 files changed, 111 insertions(+), 49 deletions(-)



  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/48/17948/2
-- 
To view, visit http://gerrit.cloudera.org:8080/17948
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: Ia1df4faceff9fda169c9d15fe8b1e69cfabe0d43
Gerrit-Change-Number: 17948
Gerrit-PatchSet: 2
Gerrit-Owner: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>

[Impala-ASF-CR] IMPALA-10951 (part 2): Upgrade protobuf library to 3.14.0 for Impala

Posted by "Wenzhe Zhou (Code Review)" <ge...@cloudera.org>.
Wenzhe Zhou has posted comments on this change. ( http://gerrit.cloudera.org:8080/17948 )

Change subject: IMPALA-10951 (part 2): Upgrade protobuf library to 3.14.0 for Impala
......................................................................


Patch Set 14:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/17948/14/bin/impala-config.sh
File bin/impala-config.sh:

http://gerrit.cloudera.org:8080/#/c/17948/14/bin/impala-config.sh@139
PS14, Line 139: export IMPALA_ORC_VERSION=1.7.0-p5
> We should not bump the orc version here. Since this patch is merged, I uplo
In my last patch for toolchain https://gerrit.cloudera.org/#/c/17946/, I added a patch for orc which replaced the protobuf API calls since some APIs were deprecated in new version of protobuf library.



-- 
To view, visit http://gerrit.cloudera.org:8080/17948
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia1df4faceff9fda169c9d15fe8b1e69cfabe0d43
Gerrit-Change-Number: 17948
Gerrit-PatchSet: 14
Gerrit-Owner: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Comment-Date: Fri, 21 Jan 2022 07:15:55 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] WIP IMPALA-10951 (part 2): Upgrade protobuf library to 3.14.0 for Impala

Posted by "Wenzhe Zhou (Code Review)" <ge...@cloudera.org>.
Wenzhe Zhou has uploaded a new patch set (#5). ( http://gerrit.cloudera.org:8080/17948 )

Change subject: WIP IMPALA-10951 (part 2): Upgrade protobuf library to 3.14.0 for Impala
......................................................................

WIP IMPALA-10951 (part 2): Upgrade protobuf library to 3.14.0 for Impala

There are some API changes in the new version of protobuf library.
This patch makes necessary changes for Impala code to pass compiling.

Upgrade Kudu to 5d39a74a19. With the new version of Kudu, the protobuf
symbols in libkudu_client.so are hided. This solved the static
initialization conflicts between two copies of protobuf library
in impalad's binary - one is linked by Impala itself for KRPC
communication, another is linked into Kudu client shared library.

Testing:
 - Passed core DEBUG build and exhaustive release build.
 - TODO pass a core ASAN build

Change-Id: Ia1df4faceff9fda169c9d15fe8b1e69cfabe0d43
---
M be/src/gutil/dynamic_annotations.h
M be/src/kudu/rpc/exactly_once_rpc-test.cc
M be/src/kudu/rpc/inbound_call.cc
M be/src/kudu/rpc/outbound_call.cc
M be/src/kudu/rpc/result_tracker.h
M be/src/kudu/rpc/serialization.cc
M be/src/kudu/rpc/serialization.h
M be/src/kudu/util/pb_util.cc
M be/src/kudu/util/protobuf_util.h
M bin/impala-config.sh
M testdata/cluster/node_templates/common/etc/kudu/tserver.conf.tmpl
11 files changed, 74 insertions(+), 50 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/48/17948/5
-- 
To view, visit http://gerrit.cloudera.org:8080/17948
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ia1df4faceff9fda169c9d15fe8b1e69cfabe0d43
Gerrit-Change-Number: 17948
Gerrit-PatchSet: 5
Gerrit-Owner: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>

[Impala-ASF-CR] IMPALA-10951 (part 2): Upgrade protobuf library to 3.14.0 for Impala

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17948 )

Change subject: IMPALA-10951 (part 2): Upgrade protobuf library to 3.14.0 for Impala
......................................................................


Patch Set 13: Verified-1

Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/7738/


-- 
To view, visit http://gerrit.cloudera.org:8080/17948
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia1df4faceff9fda169c9d15fe8b1e69cfabe0d43
Gerrit-Change-Number: 17948
Gerrit-PatchSet: 13
Gerrit-Owner: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Comment-Date: Tue, 11 Jan 2022 19:27:52 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] WIP IMPALA-10931: Integrate protobuf 3.14.0 library with Impala

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17948 )

Change subject: WIP IMPALA-10931: Integrate protobuf 3.14.0 library with Impala
......................................................................


Patch Set 2:

Build Failed 

https://jenkins.impala.io/job/gerrit-code-review-checks/9692/ : Initial code review checks failed. See linked job for details on the failure.


-- 
To view, visit http://gerrit.cloudera.org:8080/17948
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia1df4faceff9fda169c9d15fe8b1e69cfabe0d43
Gerrit-Change-Number: 17948
Gerrit-PatchSet: 2
Gerrit-Owner: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Comment-Date: Fri, 29 Oct 2021 00:14:49 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-10951 (part 2): Upgrade protobuf library to 3.14.0 for Impala

Posted by "Joe McDonnell (Code Review)" <ge...@cloudera.org>.
Joe McDonnell has posted comments on this change. ( http://gerrit.cloudera.org:8080/17948 )

Change subject: IMPALA-10951 (part 2): Upgrade protobuf library to 3.14.0 for Impala
......................................................................


Patch Set 12: Code-Review+2

This looks good to me.


-- 
To view, visit http://gerrit.cloudera.org:8080/17948
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia1df4faceff9fda169c9d15fe8b1e69cfabe0d43
Gerrit-Change-Number: 17948
Gerrit-PatchSet: 12
Gerrit-Owner: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Comment-Date: Tue, 11 Jan 2022 05:33:10 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-10951 (part 2): Upgrade protobuf library to 3.14.0 for Impala

Posted by "Wenzhe Zhou (Code Review)" <ge...@cloudera.org>.
Wenzhe Zhou has uploaded a new patch set (#11). ( http://gerrit.cloudera.org:8080/17948 )

Change subject: IMPALA-10951 (part 2): Upgrade protobuf library to 3.14.0 for Impala
......................................................................

IMPALA-10951 (part 2): Upgrade protobuf library to 3.14.0 for Impala

There are some API changes in the new version of protobuf library.
This patch makes necessary changes for Impala code to pass compiling.

Two tarballs of protobuf 3.14.0 are built in toolchain. Use
protobuf-3.14.0 for normal builds, use protobuf-3.14.0-p2 for Clang
builds.

Bump up Kudu to the latest version 67ba3cae45.

Testing:
 - Passed core DEBUG build and exhaustive release build.
 - Passed a core ASAN build

Change-Id: Ia1df4faceff9fda169c9d15fe8b1e69cfabe0d43
---
M CMakeLists.txt
M be/src/kudu/rpc/exactly_once_rpc-test.cc
M be/src/kudu/rpc/inbound_call.cc
M be/src/kudu/rpc/outbound_call.cc
M be/src/kudu/rpc/result_tracker.h
M be/src/kudu/rpc/serialization.cc
M be/src/kudu/rpc/serialization.h
M be/src/kudu/util/pb_util.cc
M be/src/kudu/util/protobuf_util.h
M bin/bootstrap_toolchain.py
M bin/impala-config.sh
11 files changed, 89 insertions(+), 51 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/48/17948/11
-- 
To view, visit http://gerrit.cloudera.org:8080/17948
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ia1df4faceff9fda169c9d15fe8b1e69cfabe0d43
Gerrit-Change-Number: 17948
Gerrit-PatchSet: 11
Gerrit-Owner: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>

[Impala-ASF-CR] IMPALA-10951 (part 2): Upgrade protobuf library to 3.14.0 for Impala

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17948 )

Change subject: IMPALA-10951 (part 2): Upgrade protobuf library to 3.14.0 for Impala
......................................................................


Patch Set 12:

Build Successful 

https://jenkins.impala.io/job/gerrit-code-review-checks/9999/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.


-- 
To view, visit http://gerrit.cloudera.org:8080/17948
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia1df4faceff9fda169c9d15fe8b1e69cfabe0d43
Gerrit-Change-Number: 17948
Gerrit-PatchSet: 12
Gerrit-Owner: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Comment-Date: Tue, 11 Jan 2022 02:57:57 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-10951 (part 2): Upgrade protobuf library to 3.14.0 for Impala

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17948 )

Change subject: IMPALA-10951 (part 2): Upgrade protobuf library to 3.14.0 for Impala
......................................................................


Patch Set 13: Verified-1

Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/7739/


-- 
To view, visit http://gerrit.cloudera.org:8080/17948
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia1df4faceff9fda169c9d15fe8b1e69cfabe0d43
Gerrit-Change-Number: 17948
Gerrit-PatchSet: 13
Gerrit-Owner: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Comment-Date: Wed, 12 Jan 2022 02:06:39 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-10951 (part 2): Upgrade protobuf library to 3.14.0 for Impala

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/17948 )

Change subject: IMPALA-10951 (part 2): Upgrade protobuf library to 3.14.0 for Impala
......................................................................

IMPALA-10951 (part 2): Upgrade protobuf library to 3.14.0 for Impala

There are some API changes in the new version of protobuf library.
This patch makes necessary changes for Impala code to pass compiling.

Two tarballs of protobuf 3.14.0 are built in toolchain. Use
protobuf-3.14.0 for normal builds, use protobuf-3.14.0-clangcompat-p2
for Clang builds.

Bump up Kudu to the latest version 67ba3cae45.

Testing:
 - Passed core DEBUG build and exhaustive release build.
 - Passed core ASAN build.

Change-Id: Ia1df4faceff9fda169c9d15fe8b1e69cfabe0d43
Reviewed-on: http://gerrit.cloudera.org:8080/17948
Reviewed-by: Impala Public Jenkins <im...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
---
M CMakeLists.txt
M be/src/kudu/rpc/exactly_once_rpc-test.cc
M be/src/kudu/rpc/inbound_call.cc
M be/src/kudu/rpc/outbound_call.cc
M be/src/kudu/rpc/result_tracker.h
M be/src/kudu/rpc/serialization.cc
M be/src/kudu/rpc/serialization.h
M be/src/kudu/util/pb_util.cc
M be/src/kudu/util/protobuf_util.h
M bin/bootstrap_toolchain.py
M bin/impala-config.sh
11 files changed, 89 insertions(+), 51 deletions(-)

Approvals:
  Impala Public Jenkins: Looks good to me, approved; Verified

-- 
To view, visit http://gerrit.cloudera.org:8080/17948
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: Ia1df4faceff9fda169c9d15fe8b1e69cfabe0d43
Gerrit-Change-Number: 17948
Gerrit-PatchSet: 14
Gerrit-Owner: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>

[Impala-ASF-CR] IMPALA-10951 (part 2): Upgrade protobuf library to 3.14.0 for Impala

Posted by "Wenzhe Zhou (Code Review)" <ge...@cloudera.org>.
Wenzhe Zhou has uploaded a new patch set (#12). ( http://gerrit.cloudera.org:8080/17948 )

Change subject: IMPALA-10951 (part 2): Upgrade protobuf library to 3.14.0 for Impala
......................................................................

IMPALA-10951 (part 2): Upgrade protobuf library to 3.14.0 for Impala

There are some API changes in the new version of protobuf library.
This patch makes necessary changes for Impala code to pass compiling.

Two tarballs of protobuf 3.14.0 are built in toolchain. Use
protobuf-3.14.0 for normal builds, use protobuf-3.14.0-clangcompat-p2
for Clang builds.

Bump up Kudu to the latest version 67ba3cae45.

Testing:
 - Passed core DEBUG build and exhaustive release build.
 - Passed core ASAN build.

Change-Id: Ia1df4faceff9fda169c9d15fe8b1e69cfabe0d43
---
M CMakeLists.txt
M be/src/kudu/rpc/exactly_once_rpc-test.cc
M be/src/kudu/rpc/inbound_call.cc
M be/src/kudu/rpc/outbound_call.cc
M be/src/kudu/rpc/result_tracker.h
M be/src/kudu/rpc/serialization.cc
M be/src/kudu/rpc/serialization.h
M be/src/kudu/util/pb_util.cc
M be/src/kudu/util/protobuf_util.h
M bin/bootstrap_toolchain.py
M bin/impala-config.sh
11 files changed, 89 insertions(+), 51 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/48/17948/12
-- 
To view, visit http://gerrit.cloudera.org:8080/17948
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ia1df4faceff9fda169c9d15fe8b1e69cfabe0d43
Gerrit-Change-Number: 17948
Gerrit-PatchSet: 12
Gerrit-Owner: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>

[Impala-ASF-CR] IMPALA-10951 (part 2): Upgrade protobuf library to 3.14.0 for Impala

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17948 )

Change subject: IMPALA-10951 (part 2): Upgrade protobuf library to 3.14.0 for Impala
......................................................................


Patch Set 13:

Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/7740/ DRY_RUN=false


-- 
To view, visit http://gerrit.cloudera.org:8080/17948
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia1df4faceff9fda169c9d15fe8b1e69cfabe0d43
Gerrit-Change-Number: 17948
Gerrit-PatchSet: 13
Gerrit-Owner: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Comment-Date: Wed, 12 Jan 2022 02:09:45 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-10951 (part 2): Upgrade protobuf library to 3.14.0 for Impala

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17948 )

Change subject: IMPALA-10951 (part 2): Upgrade protobuf library to 3.14.0 for Impala
......................................................................


Patch Set 10:

Build Successful 

https://jenkins.impala.io/job/gerrit-code-review-checks/9989/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.


-- 
To view, visit http://gerrit.cloudera.org:8080/17948
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia1df4faceff9fda169c9d15fe8b1e69cfabe0d43
Gerrit-Change-Number: 17948
Gerrit-PatchSet: 10
Gerrit-Owner: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Comment-Date: Sat, 08 Jan 2022 01:45:15 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-10951 (part 2): Upgrade protobuf library to 3.14.0 for Impala

Posted by "Wenzhe Zhou (Code Review)" <ge...@cloudera.org>.
Wenzhe Zhou has removed a vote on this change.

Change subject: IMPALA-10951 (part 2): Upgrade protobuf library to 3.14.0 for Impala
......................................................................


Removed Verified-1 by Impala Public Jenkins <im...@cloudera.com>
-- 
To view, visit http://gerrit.cloudera.org:8080/17948
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: deleteVote
Gerrit-Change-Id: Ia1df4faceff9fda169c9d15fe8b1e69cfabe0d43
Gerrit-Change-Number: 17948
Gerrit-PatchSet: 13
Gerrit-Owner: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>

[Impala-ASF-CR] IMPALA-10951 (part 2): Upgrade protobuf library to 3.14.0 for Impala

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17948 )

Change subject: IMPALA-10951 (part 2): Upgrade protobuf library to 3.14.0 for Impala
......................................................................


Patch Set 11:

Build Successful 

https://jenkins.impala.io/job/gerrit-code-review-checks/9990/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.


-- 
To view, visit http://gerrit.cloudera.org:8080/17948
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia1df4faceff9fda169c9d15fe8b1e69cfabe0d43
Gerrit-Change-Number: 17948
Gerrit-PatchSet: 11
Gerrit-Owner: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Comment-Date: Sat, 08 Jan 2022 01:50:57 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-10951 (part 2): Upgrade protobuf library to 3.14.0 for Impala

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17948 )

Change subject: IMPALA-10951 (part 2): Upgrade protobuf library to 3.14.0 for Impala
......................................................................


Patch Set 12:

Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/7737/ DRY_RUN=false


-- 
To view, visit http://gerrit.cloudera.org:8080/17948
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia1df4faceff9fda169c9d15fe8b1e69cfabe0d43
Gerrit-Change-Number: 17948
Gerrit-PatchSet: 12
Gerrit-Owner: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Comment-Date: Tue, 11 Jan 2022 05:44:37 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] WIP IMPALA-10951 (part 2): Upgrade protobuf library to 3.14.0 for Impala

Posted by "Andrew Wong (Code Review)" <ge...@cloudera.org>.
Andrew Wong has posted comments on this change. ( http://gerrit.cloudera.org:8080/17948 )

Change subject: WIP IMPALA-10951 (part 2): Upgrade protobuf library to 3.14.0 for Impala
......................................................................


Patch Set 8: Code-Review+1

(1 comment)

http://gerrit.cloudera.org:8080/#/c/17948/8//COMMIT_MSG
Commit Message:

http://gerrit.cloudera.org:8080/#/c/17948/8//COMMIT_MSG@13
PS8, Line 13: hided
nit: hidden



-- 
To view, visit http://gerrit.cloudera.org:8080/17948
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia1df4faceff9fda169c9d15fe8b1e69cfabe0d43
Gerrit-Change-Number: 17948
Gerrit-PatchSet: 8
Gerrit-Owner: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Comment-Date: Tue, 16 Nov 2021 20:02:51 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] WIP IMPALA-10951 (part 2): Upgrade protobuf library to 3.14.0 for Impala

Posted by "Joe McDonnell (Code Review)" <ge...@cloudera.org>.
Joe McDonnell has posted comments on this change. ( http://gerrit.cloudera.org:8080/17948 )

Change subject: WIP IMPALA-10951 (part 2): Upgrade protobuf library to 3.14.0 for Impala
......................................................................


Patch Set 5:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/17948/5/be/src/kudu/rpc/result_tracker.h
File be/src/kudu/rpc/result_tracker.h:

http://gerrit.cloudera.org:8080/#/c/17948/5/be/src/kudu/rpc/result_tracker.h@286
PS5, Line 286:                                            0)
> The length of previous line is already 90, have to break it as two lines to
Ok, I see. For some reason Gerrit is not showing the 90 character limit line for me.

Since this is Kudu code transplanted into Impala, I would also be ok with leaving it however Kudu has it (and ignoring warnings about it).



-- 
To view, visit http://gerrit.cloudera.org:8080/17948
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia1df4faceff9fda169c9d15fe8b1e69cfabe0d43
Gerrit-Change-Number: 17948
Gerrit-PatchSet: 5
Gerrit-Owner: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Comment-Date: Thu, 11 Nov 2021 17:52:07 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] WIP IMPALA-10951 (part 2): Upgrade protobuf library to 3.14.0 for Impala

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17948 )

Change subject: WIP IMPALA-10951 (part 2): Upgrade protobuf library to 3.14.0 for Impala
......................................................................


Patch Set 8:

Build Successful 

https://jenkins.impala.io/job/gerrit-code-review-checks/9787/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.


-- 
To view, visit http://gerrit.cloudera.org:8080/17948
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia1df4faceff9fda169c9d15fe8b1e69cfabe0d43
Gerrit-Change-Number: 17948
Gerrit-PatchSet: 8
Gerrit-Owner: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Comment-Date: Tue, 16 Nov 2021 16:51:28 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] WIP IMPALA-10951 (part 2): Upgrade protobuf library to 3.14.0 for Impala

Posted by "Wenzhe Zhou (Code Review)" <ge...@cloudera.org>.
Wenzhe Zhou has uploaded a new patch set (#8). ( http://gerrit.cloudera.org:8080/17948 )

Change subject: WIP IMPALA-10951 (part 2): Upgrade protobuf library to 3.14.0 for Impala
......................................................................

WIP IMPALA-10951 (part 2): Upgrade protobuf library to 3.14.0 for Impala

There are some API changes in the new version of protobuf library.
This patch makes necessary changes for Impala code to pass compiling.

Upgrade Kudu to version 1754f517b2. With the new version of Kudu, the
protobuf symbols in libkudu_client.so are hided. This solved the static
initialization conflicts between two copies of protobuf library
in impalad's binary - one is linked by Impala itself for KRPC
communication, another is linked into Kudu client shared library.

Testing:
 - Passed core DEBUG build and exhaustive release build.
 - TODO pass a core ASAN build

Change-Id: Ia1df4faceff9fda169c9d15fe8b1e69cfabe0d43
---
M be/src/gutil/dynamic_annotations.h
M be/src/kudu/rpc/exactly_once_rpc-test.cc
M be/src/kudu/rpc/inbound_call.cc
M be/src/kudu/rpc/outbound_call.cc
M be/src/kudu/rpc/result_tracker.h
M be/src/kudu/rpc/serialization.cc
M be/src/kudu/rpc/serialization.h
M be/src/kudu/util/pb_util.cc
M be/src/kudu/util/protobuf_util.h
M bin/impala-config.sh
M testdata/cluster/node_templates/common/etc/kudu/tserver.conf.tmpl
11 files changed, 75 insertions(+), 51 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/48/17948/8
-- 
To view, visit http://gerrit.cloudera.org:8080/17948
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ia1df4faceff9fda169c9d15fe8b1e69cfabe0d43
Gerrit-Change-Number: 17948
Gerrit-PatchSet: 8
Gerrit-Owner: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>

[Impala-ASF-CR] WIP IMPALA-10931: Integrate protobuf 3.14.0 library with Impala

Posted by "Wenzhe Zhou (Code Review)" <ge...@cloudera.org>.
Wenzhe Zhou has uploaded a new patch set (#3). ( http://gerrit.cloudera.org:8080/17948 )

Change subject: WIP IMPALA-10931: Integrate protobuf 3.14.0 library with Impala
......................................................................

WIP IMPALA-10931: Integrate protobuf 3.14.0 library with Impala

There are some API changes in the new version of protobuf library.
This patch makes necessary changes for Impala code to pass compiling.
It also adds a work around for the static initializer issue, which
is caused by the conflicts between two copies of protobuf library
in Impala's binary, one came from Kudu client shared library.

Testing:
 - Passed an core DEBUG build.
 - TODO pass exhaustive DEBUG build and a core ASAN build

Change-Id: Ia1df4faceff9fda169c9d15fe8b1e69cfabe0d43
---
M be/src/kudu/rpc/exactly_once_rpc-test.cc
M be/src/kudu/rpc/inbound_call.cc
M be/src/kudu/rpc/outbound_call.cc
M be/src/kudu/rpc/result_tracker.h
M be/src/kudu/rpc/serialization.cc
M be/src/kudu/rpc/serialization.h
M be/src/kudu/util/pb_util.cc
M be/src/kudu/util/protobuf_util.h
M be/src/rpc/rpc-mgr.cc
M bin/impala-config.sh
10 files changed, 113 insertions(+), 49 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/48/17948/3
-- 
To view, visit http://gerrit.cloudera.org:8080/17948
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ia1df4faceff9fda169c9d15fe8b1e69cfabe0d43
Gerrit-Change-Number: 17948
Gerrit-PatchSet: 3
Gerrit-Owner: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>

[Impala-ASF-CR] IMPALA-10951 (part 2): Upgrade protobuf library to 3.14.0 for Impala

Posted by "Quanlong Huang (Code Review)" <ge...@cloudera.org>.
Quanlong Huang has posted comments on this change. ( http://gerrit.cloudera.org:8080/17948 )

Change subject: IMPALA-10951 (part 2): Upgrade protobuf library to 3.14.0 for Impala
......................................................................


Patch Set 14:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/17948/14/bin/impala-config.sh
File bin/impala-config.sh:

http://gerrit.cloudera.org:8080/#/c/17948/14/bin/impala-config.sh@139
PS14, Line 139: export IMPALA_ORC_VERSION=1.7.0-p5
> In my last patch for toolchain https://gerrit.cloudera.org/#/c/17946/, I ad
Oh, sorry that I missed the orc patch. Thanks for the clarification!



-- 
To view, visit http://gerrit.cloudera.org:8080/17948
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia1df4faceff9fda169c9d15fe8b1e69cfabe0d43
Gerrit-Change-Number: 17948
Gerrit-PatchSet: 14
Gerrit-Owner: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Comment-Date: Fri, 21 Jan 2022 07:53:52 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-10951 (part 2): Upgrade protobuf library to 3.14.0 for Impala

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17948 )

Change subject: IMPALA-10951 (part 2): Upgrade protobuf library to 3.14.0 for Impala
......................................................................


Patch Set 13: Code-Review+2


-- 
To view, visit http://gerrit.cloudera.org:8080/17948
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia1df4faceff9fda169c9d15fe8b1e69cfabe0d43
Gerrit-Change-Number: 17948
Gerrit-PatchSet: 13
Gerrit-Owner: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Comment-Date: Tue, 11 Jan 2022 12:54:33 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-10951 (part 2): Upgrade protobuf library to 3.14.0 for Impala

Posted by "Wenzhe Zhou (Code Review)" <ge...@cloudera.org>.
Wenzhe Zhou has uploaded a new patch set (#10). ( http://gerrit.cloudera.org:8080/17948 )

Change subject: IMPALA-10951 (part 2): Upgrade protobuf library to 3.14.0 for Impala
......................................................................

IMPALA-10951 (part 2): Upgrade protobuf library to 3.14.0 for Impala

There are some API changes in the new version of protobuf library.
This patch makes necessary changes for Impala code to pass compiling.

Two tarballs of protobuf 3.14.0 are built in toolchain. Use
protobuf-3.14.0 for normal builds, use protobuf-3.14.0-p2 for Clang
builds.

Bump up Kudu to the latest version 67ba3cae45.

Testing:
 - Passed core DEBUG build and exhaustive release build.
 - Passed a core ASAN build

Change-Id: Ia1df4faceff9fda169c9d15fe8b1e69cfabe0d43
---
M CMakeLists.txt
M be/src/kudu/rpc/exactly_once_rpc-test.cc
M be/src/kudu/rpc/inbound_call.cc
M be/src/kudu/rpc/outbound_call.cc
M be/src/kudu/rpc/result_tracker.h
M be/src/kudu/rpc/serialization.cc
M be/src/kudu/rpc/serialization.h
M be/src/kudu/util/pb_util.cc
M be/src/kudu/util/protobuf_util.h
M bin/bootstrap_toolchain.py
M bin/impala-config.sh
11 files changed, 89 insertions(+), 51 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/48/17948/10
-- 
To view, visit http://gerrit.cloudera.org:8080/17948
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ia1df4faceff9fda169c9d15fe8b1e69cfabe0d43
Gerrit-Change-Number: 17948
Gerrit-PatchSet: 10
Gerrit-Owner: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>

[Impala-ASF-CR] IMPALA-10951 (part 2): Upgrade protobuf library to 3.14.0 for Impala

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17948 )

Change subject: IMPALA-10951 (part 2): Upgrade protobuf library to 3.14.0 for Impala
......................................................................


Patch Set 13:

Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/7738/ DRY_RUN=false


-- 
To view, visit http://gerrit.cloudera.org:8080/17948
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia1df4faceff9fda169c9d15fe8b1e69cfabe0d43
Gerrit-Change-Number: 17948
Gerrit-PatchSet: 13
Gerrit-Owner: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Comment-Date: Tue, 11 Jan 2022 12:54:34 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-10951 (part 2): Upgrade protobuf library to 3.14.0 for Impala

Posted by "Wenzhe Zhou (Code Review)" <ge...@cloudera.org>.
Wenzhe Zhou has posted comments on this change. ( http://gerrit.cloudera.org:8080/17948 )

Change subject: IMPALA-10951 (part 2): Upgrade protobuf library to 3.14.0 for Impala
......................................................................


Patch Set 13:

Hit an known bug CDPD-29565.


-- 
To view, visit http://gerrit.cloudera.org:8080/17948
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia1df4faceff9fda169c9d15fe8b1e69cfabe0d43
Gerrit-Change-Number: 17948
Gerrit-PatchSet: 13
Gerrit-Owner: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Comment-Date: Tue, 11 Jan 2022 19:36:26 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-10951 (part 2): Upgrade protobuf library to 3.14.0 for Impala

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17948 )

Change subject: IMPALA-10951 (part 2): Upgrade protobuf library to 3.14.0 for Impala
......................................................................


Patch Set 13: Verified+1


-- 
To view, visit http://gerrit.cloudera.org:8080/17948
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia1df4faceff9fda169c9d15fe8b1e69cfabe0d43
Gerrit-Change-Number: 17948
Gerrit-PatchSet: 13
Gerrit-Owner: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Comment-Date: Wed, 12 Jan 2022 08:39:16 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] WIP IMPALA-10951 (part 2): Upgrade protobuf library to 3.14.0 for Impala

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17948 )

Change subject: WIP IMPALA-10951 (part 2): Upgrade protobuf library to 3.14.0 for Impala
......................................................................


Patch Set 5:

Build Successful 

https://jenkins.impala.io/job/gerrit-code-review-checks/9739/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.


-- 
To view, visit http://gerrit.cloudera.org:8080/17948
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia1df4faceff9fda169c9d15fe8b1e69cfabe0d43
Gerrit-Change-Number: 17948
Gerrit-PatchSet: 5
Gerrit-Owner: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Comment-Date: Tue, 09 Nov 2021 03:17:50 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] WIP IMPALA-10931: Integrate protobuf 3.14.0 library with Impala

Posted by "Andrew Wong (Code Review)" <ge...@cloudera.org>.
Andrew Wong has posted comments on this change. ( http://gerrit.cloudera.org:8080/17948 )

Change subject: WIP IMPALA-10931: Integrate protobuf 3.14.0 library with Impala
......................................................................


Patch Set 2:

(2 comments)

http://gerrit.cloudera.org:8080/#/c/17948/2/be/src/rpc/rpc-mgr.cc
File be/src/rpc/rpc-mgr.cc:

http://gerrit.cloudera.org:8080/#/c/17948/2/be/src/rpc/rpc-mgr.cc@100
PS2, Line 100: // Since Kudu client shared library has its own protobuf, there are two copies of protobuf
             : // library in Impala's binary. Each copy has independent descriptor pool, but they
             : // interact each other at static-initialization time, which cause AddDescriptors() is not
             : // called for .proto files for Impala's protobuf.
             : // This function load descriptors for google/protobuf/descriptor.proto, which are used by
             : // rpc_header.proto.
Just trying to wrap my head around the root cause here. Am I correct in understand that:
- As stated, each protobuf library has its own independent descriptor pool.
- The Kudu library calls AddDescriptors()
- The Impala library is initialized later, and doesn't call AddDescriptors() because descriptor_table_google_2fprotobuf_2fdescriptor_2eproto.is_initialized is already true
- Hence, the Impala descriptor pool is missing its descriptors

?


http://gerrit.cloudera.org:8080/#/c/17948/2/be/src/rpc/rpc-mgr.cc@108
PS2, Line 108: static bool CheckProtobufDescriptorProto() {
Given the issue we're fixing is quite nuanced, perhaps it's worth adding links that have a bit more context, like https://stackoverflow.com/questions/29960871/protobuf-message-object-creation-by-name?



-- 
To view, visit http://gerrit.cloudera.org:8080/17948
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia1df4faceff9fda169c9d15fe8b1e69cfabe0d43
Gerrit-Change-Number: 17948
Gerrit-PatchSet: 2
Gerrit-Owner: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Comment-Date: Fri, 29 Oct 2021 00:29:45 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] WIP IMPALA-10931: Integrate protobuf 3.14.0 library with Impala

Posted by "Wenzhe Zhou (Code Review)" <ge...@cloudera.org>.
Wenzhe Zhou has posted comments on this change. ( http://gerrit.cloudera.org:8080/17948 )

Change subject: WIP IMPALA-10931: Integrate protobuf 3.14.0 library with Impala
......................................................................


Patch Set 2:

(2 comments)

http://gerrit.cloudera.org:8080/#/c/17948/2/be/src/rpc/rpc-mgr.cc
File be/src/rpc/rpc-mgr.cc:

http://gerrit.cloudera.org:8080/#/c/17948/2/be/src/rpc/rpc-mgr.cc@100
PS2, Line 100: // Since Kudu client shared library has its own protobuf, there are two copies of protobuf
             : // library in Impala's binary. Each copy has independent descriptor pool, but they
             : // interact each other at static-initialization time, which cause AddDescriptors() is not
             : // called for .proto files for Impala's protobuf.
             : // This function load descriptors for google/protobuf/descriptor.proto, which are used by
             : // rpc_header.proto.
> Just trying to wrap my head around the root cause here. Am I correct in und
Your understanding is correct.


http://gerrit.cloudera.org:8080/#/c/17948/2/be/src/rpc/rpc-mgr.cc@108
PS2, Line 108: static bool CheckProtobufDescriptorProto() {
> Given the issue we're fixing is quite nuanced, perhaps it's worth adding li
Added the link in the comments.



-- 
To view, visit http://gerrit.cloudera.org:8080/17948
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia1df4faceff9fda169c9d15fe8b1e69cfabe0d43
Gerrit-Change-Number: 17948
Gerrit-PatchSet: 2
Gerrit-Owner: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Comment-Date: Fri, 29 Oct 2021 00:54:19 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] WIP IMPALA-10931: Integrate protobuf 3.14.0 library with Impala

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17948 )

Change subject: WIP IMPALA-10931: Integrate protobuf 3.14.0 library with Impala
......................................................................


Patch Set 3:

Build Successful 

https://jenkins.impala.io/job/gerrit-code-review-checks/9694/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.


-- 
To view, visit http://gerrit.cloudera.org:8080/17948
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia1df4faceff9fda169c9d15fe8b1e69cfabe0d43
Gerrit-Change-Number: 17948
Gerrit-PatchSet: 3
Gerrit-Owner: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Comment-Date: Fri, 29 Oct 2021 01:30:19 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] WIP IMPALA-10951 (part 2): Upgrade protobuf library to 3.14.0 for Impala

Posted by "Wenzhe Zhou (Code Review)" <ge...@cloudera.org>.
Wenzhe Zhou has uploaded a new patch set (#4). ( http://gerrit.cloudera.org:8080/17948 )

Change subject: WIP IMPALA-10951 (part 2): Upgrade protobuf library to 3.14.0 for Impala
......................................................................

WIP IMPALA-10951 (part 2): Upgrade protobuf library to 3.14.0 for Impala

There are some API changes in the new version of protobuf library.
This patch makes necessary changes for Impala code to pass compiling.
It also adds a work around for the static initializer issue, which
is caused by the conflicts between two copies of protobuf library
in impalad's binary, one came from Kudu client shared library.

Testing:
 - Passed core DEBUG build and exhaustive DEBUG build.
 - TODO pass a core ASAN build

Change-Id: Ia1df4faceff9fda169c9d15fe8b1e69cfabe0d43
---
M be/src/kudu/rpc/exactly_once_rpc-test.cc
M be/src/kudu/rpc/inbound_call.cc
M be/src/kudu/rpc/outbound_call.cc
M be/src/kudu/rpc/result_tracker.h
M be/src/kudu/rpc/serialization.cc
M be/src/kudu/rpc/serialization.h
M be/src/kudu/util/pb_util.cc
M be/src/kudu/util/protobuf_util.h
M be/src/rpc/rpc-mgr.cc
M bin/impala-config.sh
10 files changed, 113 insertions(+), 49 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/48/17948/4
-- 
To view, visit http://gerrit.cloudera.org:8080/17948
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ia1df4faceff9fda169c9d15fe8b1e69cfabe0d43
Gerrit-Change-Number: 17948
Gerrit-PatchSet: 4
Gerrit-Owner: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>

[Impala-ASF-CR] WIP IMPALA-10951 (part 2): Upgrade protobuf library to 3.14.0 for Impala

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17948 )

Change subject: WIP IMPALA-10951 (part 2): Upgrade protobuf library to 3.14.0 for Impala
......................................................................


Patch Set 4:

Build Successful 

https://jenkins.impala.io/job/gerrit-code-review-checks/9723/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.


-- 
To view, visit http://gerrit.cloudera.org:8080/17948
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia1df4faceff9fda169c9d15fe8b1e69cfabe0d43
Gerrit-Change-Number: 17948
Gerrit-PatchSet: 4
Gerrit-Owner: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Comment-Date: Thu, 04 Nov 2021 22:20:40 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-10951 (part 2): Upgrade protobuf library to 3.14.0 for Impala

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17948 )

Change subject: IMPALA-10951 (part 2): Upgrade protobuf library to 3.14.0 for Impala
......................................................................


Patch Set 10:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/17948/10/bin/bootstrap_toolchain.py
File bin/bootstrap_toolchain.py:

http://gerrit.cloudera.org:8080/#/c/17948/10/bin/bootstrap_toolchain.py@472
PS10, Line 472: "
flake8: E126 continuation line over-indented for hanging indent



-- 
To view, visit http://gerrit.cloudera.org:8080/17948
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia1df4faceff9fda169c9d15fe8b1e69cfabe0d43
Gerrit-Change-Number: 17948
Gerrit-PatchSet: 10
Gerrit-Owner: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Comment-Date: Sat, 08 Jan 2022 01:21:43 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-10951 (part 2): Upgrade protobuf library to 3.14.0 for Impala

Posted by "Quanlong Huang (Code Review)" <ge...@cloudera.org>.
Quanlong Huang has posted comments on this change. ( http://gerrit.cloudera.org:8080/17948 )

Change subject: IMPALA-10951 (part 2): Upgrade protobuf library to 3.14.0 for Impala
......................................................................


Patch Set 14:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/17948/14/bin/impala-config.sh
File bin/impala-config.sh:

http://gerrit.cloudera.org:8080/#/c/17948/14/bin/impala-config.sh@139
PS14, Line 139: export IMPALA_ORC_VERSION=1.7.0-p5
We should not bump the orc version here. Since this patch is merged, I uploaded a patch on native-toolchain to really bump the orc version: https://gerrit.cloudera.org/c/18165/



-- 
To view, visit http://gerrit.cloudera.org:8080/17948
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia1df4faceff9fda169c9d15fe8b1e69cfabe0d43
Gerrit-Change-Number: 17948
Gerrit-PatchSet: 14
Gerrit-Owner: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Comment-Date: Fri, 21 Jan 2022 06:27:01 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-10951 (part 2): Upgrade protobuf library to 3.14.0 for Impala

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17948 )

Change subject: IMPALA-10951 (part 2): Upgrade protobuf library to 3.14.0 for Impala
......................................................................


Patch Set 12: Verified-1

Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/7737/


-- 
To view, visit http://gerrit.cloudera.org:8080/17948
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia1df4faceff9fda169c9d15fe8b1e69cfabe0d43
Gerrit-Change-Number: 17948
Gerrit-PatchSet: 12
Gerrit-Owner: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Comment-Date: Tue, 11 Jan 2022 10:14:56 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-10951 (part 2): Upgrade protobuf library to 3.14.0 for Impala

Posted by "Wenzhe Zhou (Code Review)" <ge...@cloudera.org>.
Wenzhe Zhou has posted comments on this change. ( http://gerrit.cloudera.org:8080/17948 )

Change subject: IMPALA-10951 (part 2): Upgrade protobuf library to 3.14.0 for Impala
......................................................................


Patch Set 12:

Hit CDPD-9799


-- 
To view, visit http://gerrit.cloudera.org:8080/17948
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia1df4faceff9fda169c9d15fe8b1e69cfabe0d43
Gerrit-Change-Number: 17948
Gerrit-PatchSet: 12
Gerrit-Owner: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Comment-Date: Tue, 11 Jan 2022 12:53:42 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-10951 (part 2): Upgrade protobuf library to 3.14.0 for Impala

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17948 )

Change subject: IMPALA-10951 (part 2): Upgrade protobuf library to 3.14.0 for Impala
......................................................................


Patch Set 13:

Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/7739/ DRY_RUN=false


-- 
To view, visit http://gerrit.cloudera.org:8080/17948
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia1df4faceff9fda169c9d15fe8b1e69cfabe0d43
Gerrit-Change-Number: 17948
Gerrit-PatchSet: 13
Gerrit-Owner: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Comment-Date: Tue, 11 Jan 2022 19:37:33 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-10951 (part 2): Upgrade protobuf library to 3.14.0 for Impala

Posted by "Wenzhe Zhou (Code Review)" <ge...@cloudera.org>.
Wenzhe Zhou has posted comments on this change. ( http://gerrit.cloudera.org:8080/17948 )

Change subject: IMPALA-10951 (part 2): Upgrade protobuf library to 3.14.0 for Impala
......................................................................


Patch Set 13:

Hit flaky test case IMPALA-10999


-- 
To view, visit http://gerrit.cloudera.org:8080/17948
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia1df4faceff9fda169c9d15fe8b1e69cfabe0d43
Gerrit-Change-Number: 17948
Gerrit-PatchSet: 13
Gerrit-Owner: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Comment-Date: Wed, 12 Jan 2022 02:08:38 +0000
Gerrit-HasComments: No