You are viewing a plain text version of this content. The canonical link for it is here.

Posted to reviews@impala.apache.org by "Wenzhe Zhou (Code Review)" <ge...@cloudera.org> on 2021/10/29 18:24:48 UTC

[native-toolchain-CR] IMPALA-10931 (part 1): Upgrade library protobuf for toolchain

Wenzhe Zhou has uploaded this change for review. ( http://gerrit.cloudera.org:8080/17946

Change subject: IMPALA-10931 (part 1): Upgrade library protobuf for toolchain
......................................................................

IMPALA-10931 (part 1): Upgrade library protobuf for toolchain

Third party libraries - Boost, glog and protobuf in Impala native
tool-chains are be older than the corresponding libraries used by Kudu.
We need to upgrade these three libraries before rebase be/src/kudu.

This patch upgrade protobuf library from 3.5.1 to 3.14.0.
Since google::protobuf::MessageLite::ByteSize() is deprecated in
protobuf v3.14.0, we have to generate patches for orc libraries to
replace the API with google::protobuf::MessageLite::ByteSizeLong().

Testing:
- Built native toolchain on local machine.
- Built native toolchain with Jenkins job.
- Built Impala with upgraded toolchain.

Change-Id: I6700d4f8956fb2bdff5fea3bef2572352f175f67
---
M buildall.sh
A source/orc/orc-1.7.0-patches/0004-Replace-protobuf-MessageLite-ByteSize-with-ByteSizeL.patch
2 files changed, 60 insertions(+), 3 deletions(-)

git pull ssh://gerrit.cloudera.org:29418/native-toolchain refs/changes/46/17946/2
--
To view, visit http://gerrit.cloudera.org:8080/17946
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: native-toolchain
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I6700d4f8956fb2bdff5fea3bef2572352f175f67
Gerrit-Change-Number: 17946
Gerrit-PatchSet: 2
Gerrit-Owner: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>

[native-toolchain-CR] IMPALA-10951 (part 1): Upgrade protobuf library in toolchain

Posted by "Wenzhe Zhou (Code Review)" <ge...@cloudera.org>.

Wenzhe Zhou has uploaded a new patch set (#11). ( http://gerrit.cloudera.org:8080/17946 )

Change subject: IMPALA-10951 (part 1): Upgrade protobuf library in toolchain
......................................................................

IMPALA-10951 (part 1): Upgrade protobuf library in toolchain

This patch upgrade protobuf library from 3.5.1 to 3.14.0.
Since google::protobuf::MessageLite::ByteSize() is deprecated in
protobuf v3.14.0, we have to generate a patch for orc library to
replace the API with google::protobuf::MessageLite::ByteSizeLong().

Add a patch for protobuf which remove "sanitizer/asan_interface.h"
from protobuf/arena_impl.h and add AddressSanitizer annotations
from asan_interface.h. This avoid the compiling error in Impala
ASAN build.
Also add a patch to compile rotobuf::internal::call_once() with "-O1".
This fix the linking issue for Cland builds.
Two tarballs of protobuf 3.14.0 are built - one without patches
for normal builds, one with two patches for Clang builds.

Bump up Kudu to the latest version 67ba3cae45 which use 2.17.1 log4j.

Testing:
- Built native toolchain with Jenkins job.
- Built Impala with upgraded toolchain for normal and ASAN builds.

Change-Id: I6700d4f8956fb2bdff5fea3bef2572352f175f67
---
M buildall.sh
A source/orc/orc-1.7.0-patches/0005-Replace-protobuf-MessageLite-ByteSize-with-ByteSizeL.patch
M source/protobuf/build.sh
A source/protobuf/protobuf-3.14.0-clangcompat-patches/0001-Add-AddressSanitizer-annotations-from-LLVM-asan_inte.patch
A source/protobuf/protobuf-3.14.0-clangcompat-patches/0002-Compile-call_once-with-optimization-O1.patch
5 files changed, 146 insertions(+), 7 deletions(-)

git pull ssh://gerrit.cloudera.org:29418/native-toolchain refs/changes/46/17946/11
--
To view, visit http://gerrit.cloudera.org:8080/17946
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: native-toolchain
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I6700d4f8956fb2bdff5fea3bef2572352f175f67
Gerrit-Change-Number: 17946
Gerrit-PatchSet: 11
Gerrit-Owner: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>

[native-toolchain-CR] IMPALA-10951 (part 1): Upgrade protobuf library in toolchain

Posted by "Wenzhe Zhou (Code Review)" <ge...@cloudera.org>.

Wenzhe Zhou has uploaded a new patch set (#6). ( http://gerrit.cloudera.org:8080/17946 )

Change subject: IMPALA-10951 (part 1): Upgrade protobuf library in toolchain
......................................................................

IMPALA-10951 (part 1): Upgrade protobuf library in toolchain

Bump up Kudu version to 3884a6388 and apply patch for KUDU-3334
(hide protobuf symbols in libkudu_client).

Testing:
- Built native toolchain on local machine.
- Built native toolchain with Jenkins job.
- Built Impala with upgraded toolchain.

Change-Id: I6700d4f8956fb2bdff5fea3bef2572352f175f67
---
M buildall.sh
A source/kudu/kudu-3884a6388b-patches/0001-KUDU-3334-hide-protobuf-symbols-in-libkudu_client.patch
A source/orc/orc-1.7.0-patches/0004-Replace-protobuf-MessageLite-ByteSize-with-ByteSizeL.patch
3 files changed, 121 insertions(+), 4 deletions(-)

git pull ssh://gerrit.cloudera.org:29418/native-toolchain refs/changes/46/17946/6
--
To view, visit http://gerrit.cloudera.org:8080/17946
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: native-toolchain
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I6700d4f8956fb2bdff5fea3bef2572352f175f67
Gerrit-Change-Number: 17946
Gerrit-PatchSet: 6
Gerrit-Owner: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>

[native-toolchain-CR] IMPALA-10951 (part 1): Upgrade protobuf library in toolchain

Posted by "Wenzhe Zhou (Code Review)" <ge...@cloudera.org>.

Wenzhe Zhou has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/17946 )

Change subject: IMPALA-10951 (part 1): Upgrade protobuf library in toolchain
......................................................................

IMPALA-10951 (part 1): Upgrade protobuf library in toolchain

Bump up Kudu to the latest version 67ba3cae45 which use 2.17.1 log4j.

Testing:
- Built native toolchain with Jenkins job.
- Built Impala with upgraded toolchain for normal and ASAN builds.

Change-Id: I6700d4f8956fb2bdff5fea3bef2572352f175f67
Reviewed-on: http://gerrit.cloudera.org:8080/17946
Reviewed-by: Joe McDonnell <jo...@cloudera.com>
Tested-by: Wenzhe Zhou <wz...@cloudera.com>
---
M buildall.sh
A source/orc/orc-1.7.0-patches/0005-Replace-protobuf-MessageLite-ByteSize-with-ByteSizeL.patch
M source/protobuf/build.sh
A source/protobuf/protobuf-3.14.0-patches/0001-Add-AddressSanitizer-annotations-from-LLVM-asan_inte.patch
A source/protobuf/protobuf-3.14.0-patches/0002-Compile-call_once-with-optimization-O1.patch
5 files changed, 151 insertions(+), 7 deletions(-)

Approvals:
Joe McDonnell: Looks good to me, approved
Wenzhe Zhou: Verified

--
To view, visit http://gerrit.cloudera.org:8080/17946
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: native-toolchain
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I6700d4f8956fb2bdff5fea3bef2572352f175f67
Gerrit-Change-Number: 17946
Gerrit-PatchSet: 13
Gerrit-Owner: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>

[native-toolchain-CR] IMPALA-10951 (part 1): Upgrade protobuf library in toolchain

Posted by "Joe McDonnell (Code Review)" <ge...@cloudera.org>.

Joe McDonnell has posted comments on this change. ( http://gerrit.cloudera.org:8080/17946 )

Change subject: IMPALA-10951 (part 1): Upgrade protobuf library in toolchain
......................................................................


Patch Set 10:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/17946/10/buildall.sh
File buildall.sh:

http://gerrit.cloudera.org:8080/#/c/17946/10/buildall.sh@125
PS10, Line 125: # 3.14.0-p2-p2 is only used for Clang builds.
              : PROTOBUF_VERSION=3.14.0-p2 $SOURCE_DIR/source/protobuf/build.sh
We can tell these apart by their patch numbers, but I think would rather include a string in the version itself. For LLVM, we do "5.0.1-asserts-p4" so maybe we could do "3.14.0-clangcompat-p2" or something for protobuf. The build.sh has to understand that extra string, so it requires a bit more code, but I think it is cleaner.



-- 
To view, visit http://gerrit.cloudera.org:8080/17946
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: native-toolchain
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I6700d4f8956fb2bdff5fea3bef2572352f175f67
Gerrit-Change-Number: 17946
Gerrit-PatchSet: 10
Gerrit-Owner: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Comment-Date: Mon, 10 Jan 2022 18:02:19 +0000
Gerrit-HasComments: Yes

[native-toolchain-CR] IMPALA-10951 (part 1): Upgrade protobuf library in toolchain

Posted by "Wenzhe Zhou (Code Review)" <ge...@cloudera.org>.

Wenzhe Zhou has uploaded a new patch set (#7). ( http://gerrit.cloudera.org:8080/17946 )

Change subject: IMPALA-10951 (part 1): Upgrade protobuf library in toolchain
......................................................................

IMPALA-10951 (part 1): Upgrade protobuf library in toolchain

Bump up Kudu version to 1754f517b2 which has fixing for KUDU-3334
(hide protobuf symbols in libkudu_client).

Testing:
- Built native toolchain on local machine.
- Built native toolchain with Jenkins job.
- Built Impala with upgraded toolchain.

git pull ssh://gerrit.cloudera.org:29418/native-toolchain refs/changes/46/17946/7
--
To view, visit http://gerrit.cloudera.org:8080/17946
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: native-toolchain
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I6700d4f8956fb2bdff5fea3bef2572352f175f67
Gerrit-Change-Number: 17946
Gerrit-PatchSet: 7
Gerrit-Owner: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>

[native-toolchain-CR] IMPALA-10951 (part 1): Upgrade protobuf library in toolchain

Posted by "Wenzhe Zhou (Code Review)" <ge...@cloudera.org>.

Wenzhe Zhou has posted comments on this change. ( http://gerrit.cloudera.org:8080/17946 )

Change subject: IMPALA-10951 (part 1): Upgrade protobuf library in toolchain
......................................................................


Patch Set 5:

Found one performance regression issue after upgrading Kudu. Wait Kudu team to fix it.


-- 
To view, visit http://gerrit.cloudera.org:8080/17946
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: native-toolchain
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I6700d4f8956fb2bdff5fea3bef2572352f175f67
Gerrit-Change-Number: 17946
Gerrit-PatchSet: 5
Gerrit-Owner: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Comment-Date: Thu, 11 Nov 2021 18:35:25 +0000
Gerrit-HasComments: No

[native-toolchain-CR] IMPALA-10951 (part 1): Upgrade protobuf library in toolchain

Posted by "Joe McDonnell (Code Review)" <ge...@cloudera.org>.

Joe McDonnell has posted comments on this change. ( http://gerrit.cloudera.org:8080/17946 )

Change subject: IMPALA-10951 (part 1): Upgrade protobuf library in toolchain
......................................................................


Patch Set 6: Code-Review+2

This looks good to me


-- 
To view, visit http://gerrit.cloudera.org:8080/17946
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: native-toolchain
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I6700d4f8956fb2bdff5fea3bef2572352f175f67
Gerrit-Change-Number: 17946
Gerrit-PatchSet: 6
Gerrit-Owner: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Comment-Date: Fri, 12 Nov 2021 19:49:16 +0000
Gerrit-HasComments: No

[native-toolchain-CR] IMPALA-10951 (part 1): Upgrade protobuf library in toolchain

Posted by "Andrew Wong (Code Review)" <ge...@cloudera.org>.

Andrew Wong has posted comments on this change. ( http://gerrit.cloudera.org:8080/17946 )

Change subject: IMPALA-10951 (part 1): Upgrade protobuf library in toolchain
......................................................................


Patch Set 7: Code-Review+1


-- 
To view, visit http://gerrit.cloudera.org:8080/17946
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: native-toolchain
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I6700d4f8956fb2bdff5fea3bef2572352f175f67
Gerrit-Change-Number: 17946
Gerrit-PatchSet: 7
Gerrit-Owner: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Comment-Date: Tue, 16 Nov 2021 19:37:00 +0000
Gerrit-HasComments: No

[native-toolchain-CR] IMPALA-10951 (part 1): Upgrade protobuf library in toolchain

Posted by "Wenzhe Zhou (Code Review)" <ge...@cloudera.org>.

Wenzhe Zhou has uploaded a new patch set (#3). ( http://gerrit.cloudera.org:8080/17946 )

Change subject: IMPALA-10951 (part 1): Upgrade protobuf library in toolchain
......................................................................

IMPALA-10951 (part 1): Upgrade protobuf library in toolchain

Testing:
- Built native toolchain on local machine.
- Built native toolchain with Jenkins job.
- Built Impala with upgraded toolchain.

git pull ssh://gerrit.cloudera.org:29418/native-toolchain refs/changes/46/17946/3
--
To view, visit http://gerrit.cloudera.org:8080/17946
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: native-toolchain
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I6700d4f8956fb2bdff5fea3bef2572352f175f67
Gerrit-Change-Number: 17946
Gerrit-PatchSet: 3
Gerrit-Owner: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>

[native-toolchain-CR] IMPALA-10951 (part 1): Upgrade protobuf library in toolchain

Posted by "Wenzhe Zhou (Code Review)" <ge...@cloudera.org>.

Wenzhe Zhou has uploaded a new patch set (#4). ( http://gerrit.cloudera.org:8080/17946 )

Change subject: IMPALA-10951 (part 1): Upgrade protobuf library in toolchain
......................................................................

IMPALA-10951 (part 1): Upgrade protobuf library in toolchain

Bump up Kudu version to 5d39a74a19 which has fixing for KUDU-3334
(hide protobuf symbols in libkudu_client).

Testing:
- Built native toolchain on local machine.
- Built native toolchain with Jenkins job.
- Built Impala with upgraded toolchain.

git pull ssh://gerrit.cloudera.org:29418/native-toolchain refs/changes/46/17946/4
--
To view, visit http://gerrit.cloudera.org:8080/17946
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: native-toolchain
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I6700d4f8956fb2bdff5fea3bef2572352f175f67
Gerrit-Change-Number: 17946
Gerrit-PatchSet: 4
Gerrit-Owner: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>

[native-toolchain-CR] IMPALA-10951 (part 1): Upgrade protobuf library in toolchain

Posted by "Wenzhe Zhou (Code Review)" <ge...@cloudera.org>.

Wenzhe Zhou has uploaded a new patch set (#12). ( http://gerrit.cloudera.org:8080/17946 )

Change subject: IMPALA-10951 (part 1): Upgrade protobuf library in toolchain
......................................................................

IMPALA-10951 (part 1): Upgrade protobuf library in toolchain

Bump up Kudu to the latest version 67ba3cae45 which use 2.17.1 log4j.

Testing:
- Built native toolchain with Jenkins job.
- Built Impala with upgraded toolchain for normal and ASAN builds.

Change-Id: I6700d4f8956fb2bdff5fea3bef2572352f175f67
---
M buildall.sh
A source/orc/orc-1.7.0-patches/0005-Replace-protobuf-MessageLite-ByteSize-with-ByteSizeL.patch
M source/protobuf/build.sh
A source/protobuf/protobuf-3.14.0-patches/0001-Add-AddressSanitizer-annotations-from-LLVM-asan_inte.patch
A source/protobuf/protobuf-3.14.0-patches/0002-Compile-call_once-with-optimization-O1.patch
5 files changed, 151 insertions(+), 7 deletions(-)

git pull ssh://gerrit.cloudera.org:29418/native-toolchain refs/changes/46/17946/12
--
To view, visit http://gerrit.cloudera.org:8080/17946
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: native-toolchain
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I6700d4f8956fb2bdff5fea3bef2572352f175f67
Gerrit-Change-Number: 17946
Gerrit-PatchSet: 12
Gerrit-Owner: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>

[native-toolchain-CR] IMPALA-10951 (part 1): Upgrade protobuf library in toolchain

Posted by "Joe McDonnell (Code Review)" <ge...@cloudera.org>.

Joe McDonnell has posted comments on this change. ( http://gerrit.cloudera.org:8080/17946 )

Change subject: IMPALA-10951 (part 1): Upgrade protobuf library in toolchain
......................................................................


Patch Set 12: Code-Review+2

This looks good to me.


-- 
To view, visit http://gerrit.cloudera.org:8080/17946
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: native-toolchain
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I6700d4f8956fb2bdff5fea3bef2572352f175f67
Gerrit-Change-Number: 17946
Gerrit-PatchSet: 12
Gerrit-Owner: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Comment-Date: Mon, 10 Jan 2022 22:38:34 +0000
Gerrit-HasComments: No

[native-toolchain-CR] IMPALA-10951 (part 1): Upgrade protobuf library in toolchain

Posted by "Wenzhe Zhou (Code Review)" <ge...@cloudera.org>.

Wenzhe Zhou has posted comments on this change. ( http://gerrit.cloudera.org:8080/17946 )

Change subject: IMPALA-10951 (part 1): Upgrade protobuf library in toolchain
......................................................................


Patch Set 12: Verified+1


-- 
To view, visit http://gerrit.cloudera.org:8080/17946
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: native-toolchain
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I6700d4f8956fb2bdff5fea3bef2572352f175f67
Gerrit-Change-Number: 17946
Gerrit-PatchSet: 12
Gerrit-Owner: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Comment-Date: Tue, 11 Jan 2022 01:55:34 +0000
Gerrit-HasComments: No

[native-toolchain-CR] IMPALA-10951 (part 1): Upgrade protobuf library in toolchain

Posted by "Wenzhe Zhou (Code Review)" <ge...@cloudera.org>.

Wenzhe Zhou has uploaded a new patch set (#10). ( http://gerrit.cloudera.org:8080/17946 )

Change subject: IMPALA-10951 (part 1): Upgrade protobuf library in toolchain
......................................................................

IMPALA-10951 (part 1): Upgrade protobuf library in toolchain

Add a patch for protobuf which remove "sanitizer/asan_interface.h"
from protobuf/arena_impl.h and add AddressSanitizer annotations
from asan_interface.h. This avoid the compiling error in Impala
ASAN build.
Also add a patch to compile rotobuf::internal::call_once() with "-O1".
This fix the linking issue for Cland build.
Two tarballs of protobuf 3.14.0 are built - one without patches
for normal build, one with patches for Clang build.

Bump up Kudu to the latest version 67ba3cae45 which use 2.17.1 log4j.

Testing:
- Built native toolchain with Jenkins job.
- Built Impala with upgraded toolchain for normal and ASAN builds.

Change-Id: I6700d4f8956fb2bdff5fea3bef2572352f175f67
---
M buildall.sh
A source/orc/orc-1.7.0-patches/0005-Replace-protobuf-MessageLite-ByteSize-with-ByteSizeL.patch
A source/protobuf/protobuf-3.14.0-patches/0001-Add-AddressSanitizer-annotations-from-LLVM-asan_inte.patch
A source/protobuf/protobuf-3.14.0-patches/0002-Compile-call_once-with-optimization-O1.patch
4 files changed, 133 insertions(+), 4 deletions(-)

git pull ssh://gerrit.cloudera.org:29418/native-toolchain refs/changes/46/17946/10
--
To view, visit http://gerrit.cloudera.org:8080/17946
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: native-toolchain
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I6700d4f8956fb2bdff5fea3bef2572352f175f67
Gerrit-Change-Number: 17946
Gerrit-PatchSet: 10
Gerrit-Owner: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>

[native-toolchain-CR] IMPALA-10951 (part 1): Upgrade protobuf library in toolchain

Posted by "Wenzhe Zhou (Code Review)" <ge...@cloudera.org>.

Wenzhe Zhou has posted comments on this change. ( http://gerrit.cloudera.org:8080/17946 )

Change subject: IMPALA-10951 (part 1): Upgrade protobuf library in toolchain
......................................................................


Patch Set 10:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/17946/10/buildall.sh
File buildall.sh:

http://gerrit.cloudera.org:8080/#/c/17946/10/buildall.sh@125
PS10, Line 125: # 3.14.0-p2-p2 is only used for Clang builds.
              : PROTOBUF_VERSION=3.14.0-p2 $SOURCE_DIR/source/protobuf/build.sh
> We can tell these apart by their patch numbers, but I think would rather in
Good suggestion, will rename the version as "3.14.0-clangcompat-p2", and make corresponding changes for source/protobuf/build.sh.



-- 
To view, visit http://gerrit.cloudera.org:8080/17946
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: native-toolchain
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I6700d4f8956fb2bdff5fea3bef2572352f175f67
Gerrit-Change-Number: 17946
Gerrit-PatchSet: 10
Gerrit-Owner: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Comment-Date: Mon, 10 Jan 2022 20:08:07 +0000
Gerrit-HasComments: Yes

[native-toolchain-CR] IMPALA-10951 (part 1): Upgrade protobuf library in toolchain

Posted by "Wenzhe Zhou (Code Review)" <ge...@cloudera.org>.

Wenzhe Zhou has uploaded a new patch set (#5). ( http://gerrit.cloudera.org:8080/17946 )

Change subject: IMPALA-10951 (part 1): Upgrade protobuf library in toolchain
......................................................................

IMPALA-10951 (part 1): Upgrade protobuf library in toolchain

Bump up Kudu version to 5d39a74a19 which has fixing for KUDU-3334
(hide protobuf symbols in libkudu_client).

Testing:
- Built native toolchain on local machine.
- Built native toolchain with Jenkins job.
- Built Impala with upgraded toolchain.

git pull ssh://gerrit.cloudera.org:29418/native-toolchain refs/changes/46/17946/5
--
To view, visit http://gerrit.cloudera.org:8080/17946
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: native-toolchain
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I6700d4f8956fb2bdff5fea3bef2572352f175f67
Gerrit-Change-Number: 17946
Gerrit-PatchSet: 5
Gerrit-Owner: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>

[native-toolchain-CR] IMPALA-10951 (part 1): Upgrade protobuf library in toolchain

Posted by "Wenzhe Zhou (Code Review)" <ge...@cloudera.org>.

Wenzhe Zhou has posted comments on this change. ( http://gerrit.cloudera.org:8080/17946 )

Change subject: IMPALA-10951 (part 1): Upgrade protobuf library in toolchain
......................................................................


Patch Set 7:

The performance regression issue which was introduced by Kudu client was fixed so bumped up Kudu version again.


-- 
To view, visit http://gerrit.cloudera.org:8080/17946
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: native-toolchain
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I6700d4f8956fb2bdff5fea3bef2572352f175f67
Gerrit-Change-Number: 17946
Gerrit-PatchSet: 7
Gerrit-Owner: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Comment-Date: Tue, 16 Nov 2021 16:25:57 +0000
Gerrit-HasComments: No