You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@kudu.apache.org by al...@apache.org on 2020/09/22 17:32:05 UTC
[kudu] branch branch-1.13.x updated (9a23ee8 -> 07eb5a9)
This is an automated email from the ASF dual-hosted git repository.
alexey pushed a change to branch branch-1.13.x
in repository https://gitbox.apache.org/repos/asf/kudu.git.
from 9a23ee8 Bump version to 1.13.1-SNAPSHOT
new 21651de [thirdparty] fix LLVM compilation on Ubuntu 20.04
new 07eb5a9 [kudu-tool-test] fix ClusterNameResolverFileCorrupt with glibc 2.31
The 2 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
src/kudu/tools/kudu-tool-test.cc | 13 ++--
thirdparty/download-thirdparty.sh | 5 +-
...-947f9692440836dcb8d88b74b69dd379d85974ce.patch | 81 ++++++++++++++++++++++
3 files changed, 88 insertions(+), 11 deletions(-)
create mode 100644 thirdparty/patches/llvm-947f9692440836dcb8d88b74b69dd379d85974ce.patch
[kudu] 02/02: [kudu-tool-test] fix ClusterNameResolverFileCorrupt
with glibc 2.31
Posted by al...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
alexey pushed a commit to branch branch-1.13.x
in repository https://gitbox.apache.org/repos/asf/kudu.git
commit 07eb5a97f17046f6ee61b2a28bdfbe578d3f6d2b
Author: Alexey Serbin <al...@apache.org>
AuthorDate: Sun Sep 20 10:37:26 2020 -0700
[kudu-tool-test] fix ClusterNameResolverFileCorrupt with glibc 2.31
This patch updates the expected error message in the
ToolTest.ClusterNameResolverFileCorrupt scenario to make the test
pass on Ubuntu 20.04 LTS (it should fix it on other platforms which
use newer glibc versions).
Without this patch, the scenario failed on Ubuntu 20.04 LTS:
src/kudu/tools/kudu-tool-test.cc:637: Failure
Value of: stderr
Expected: has substring "Network error: Could not connect to the cluster: unable to resolve address for bad: Name or service not known"
Actual: "W0919 23:39:50.032436 1041101 flags.cc:405] Enabled unsafe flag: --openssl_security_level_override=0\nW0919 23:39:50.032559 1041101 flags.cc:405] Enabled unsafe flag: --never_fsync=true\nNetwork error: Could not connect to the cluster: unable to resolve address for bad: Temporary failure in name resolution"
Instead of tailoring the errno-converted message for every platform,
it's easier to rely in the essential part of it that corresponds
to the DNS resolver failure.
In addition, I did a minor cleanup on the code around, removing
calls of the Substitute() function where a fixed string is enough.
Change-Id: I3eb0991cb2d4311051e55e231cb4fe6d065aa632
Reviewed-on: http://gerrit.cloudera.org:8080/16478
Tested-by: Kudu Jenkins
Reviewed-by: Grant Henke <gr...@apache.org>
(cherry picked from commit d8ab44c6864172d4b990adf31f7d84fd33e4ae15)
Reviewed-on: http://gerrit.cloudera.org:8080/16488
---
src/kudu/tools/kudu-tool-test.cc | 13 ++++---------
1 file changed, 4 insertions(+), 9 deletions(-)
diff --git a/src/kudu/tools/kudu-tool-test.cc b/src/kudu/tools/kudu-tool-test.cc
index cfc4595..39231d2 100644
--- a/src/kudu/tools/kudu-tool-test.cc
+++ b/src/kudu/tools/kudu-tool-test.cc
@@ -5698,7 +5698,7 @@ TEST_F(ToolTest, ClusterNameResolverFileCorrupt) {
NO_FATALS(CheckCorruptClusterInfoConfigFile(
Substitute(R"*($0:)*""\n"
R"*( master_addresses: $1)*", kClusterName, master_addrs_str),
- Substitute("Not found: parse field clusters_info error: invalid node; ")));
+ "Not found: parse field clusters_info error: invalid node; "));
// Missing specified cluster name.
NO_FATALS(CheckCorruptClusterInfoConfigFile(
@@ -5713,20 +5713,15 @@ TEST_F(ToolTest, ClusterNameResolverFileCorrupt) {
Substitute(R"*(clusters_info:)*""\n"
R"*( $0:)*""\n"
R"*( master_addresses_some_suffix: $1)*", kClusterName, master_addrs_str),
- Substitute("Not found: parse field master_addresses error: invalid node; ")));
+ "Not found: parse field master_addresses error: invalid node; "));
// Invalid 'master_addresses' value.
NO_FATALS(CheckCorruptClusterInfoConfigFile(
Substitute(R"*(clusters_info:)*""\n"
R"*( $0:)*""\n"
R"*( master_addresses: bad,masters,addresses)*", kClusterName),
-#ifdef __APPLE__
- Substitute("Network error: Could not connect to the cluster: unable to resolve "
- "address for bad: nodename nor servname provided, or not known")));
-#else
- Substitute("Network error: Could not connect to the cluster: unable to resolve "
- "address for bad: Name or service not known")));
-#endif
+ "Network error: Could not connect to the cluster: unable to resolve "
+ "address for bad: "));
}
TEST_F(ToolTest, ClusterNameResolverNormal) {
[kudu] 01/02: [thirdparty] fix LLVM compilation on Ubuntu 20.04
Posted by al...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
alexey pushed a commit to branch branch-1.13.x
in repository https://gitbox.apache.org/repos/asf/kudu.git
commit 21651de32951e1d813a8149b633281aa86c93d35
Author: Alexey Serbin <al...@apache.org>
AuthorDate: Sat Sep 19 16:14:53 2020 -0700
[thirdparty] fix LLVM compilation on Ubuntu 20.04
This changelist adds a patch from the llvm-toolchain-9-9.0.1 Debian
source package to fix compilation of LLVM 9.0 in Kudu thirdparty
on Ubuntu 20.04 LTS. See [1], [2] for the original source of the patch.
The patch is already included in the upstream LLVM repo and included
into 10.0.0 and later releases.
[1] https://reviews.llvm.org/rG947f9692440
[2] https://github.com/llvm/llvm-project/commit/947f9692440836dcb8d88b74b69dd379d85974ce
Change-Id: Ifba0bf6ba660a536ed54ddce228a38470dd1c650
Reviewed-on: http://gerrit.cloudera.org:8080/16477
Tested-by: Kudu Jenkins
Reviewed-by: Grant Henke <gr...@apache.org>
(cherry picked from commit 2bed4068c53fcc6084e8e8742437846cf417a746)
Reviewed-on: http://gerrit.cloudera.org:8080/16487
---
thirdparty/download-thirdparty.sh | 5 +-
...-947f9692440836dcb8d88b74b69dd379d85974ce.patch | 81 ++++++++++++++++++++++
2 files changed, 84 insertions(+), 2 deletions(-)
diff --git a/thirdparty/download-thirdparty.sh b/thirdparty/download-thirdparty.sh
index bc1b930..3ab7e71 100755
--- a/thirdparty/download-thirdparty.sh
+++ b/thirdparty/download-thirdparty.sh
@@ -319,14 +319,15 @@ fetch_and_patch \
$PYTHON_SOURCE \
$PYTHON_PATCHLEVEL
-LLVM_PATCHLEVEL=5
+LLVM_PATCHLEVEL=4
fetch_and_patch \
llvm-${LLVM_VERSION}-iwyu-${IWYU_VERSION}.src.tar.gz \
$LLVM_SOURCE \
$LLVM_PATCHLEVEL \
"patch -p1 < $TP_DIR/patches/llvm-add-iwyu.patch" \
"patch -p1 < $TP_DIR/patches/llvm-iwyu-include-picker.patch" \
- "patch -p0 < $TP_DIR/patches/llvm-iwyu-sized-deallocation.patch"
+ "patch -p0 < $TP_DIR/patches/llvm-iwyu-sized-deallocation.patch" \
+ "patch -d projects -p1 < $TP_DIR/patches/llvm-947f9692440836dcb8d88b74b69dd379d85974ce.patch"
LZ4_PATCHLEVEL=0
fetch_and_patch \
diff --git a/thirdparty/patches/llvm-947f9692440836dcb8d88b74b69dd379d85974ce.patch b/thirdparty/patches/llvm-947f9692440836dcb8d88b74b69dd379d85974ce.patch
new file mode 100644
index 0000000..e0e84db
--- /dev/null
+++ b/thirdparty/patches/llvm-947f9692440836dcb8d88b74b69dd379d85974ce.patch
@@ -0,0 +1,81 @@
+From 947f9692440836dcb8d88b74b69dd379d85974ce Mon Sep 17 00:00:00 2001
+From: Evgenii Stepanov <eu...@google.com>
+Date: Mon, 25 Nov 2019 13:52:17 -0800
+Subject: [PATCH] Fix sanitizer-common build with glibc 2.31
+
+Summary:
+As mentioned in D69104, glibc changed ABI recently with the [[ https://sourceware.org/git/?p=glibc.git;a=commitdiff;h=2f959dfe849e0646e27403f2e4091536496ac0f0| 2f959dfe ]] change.
+D69104 dealt with just 32-bit ARM, but that is just one of the many affected architectures.
+E.g. x86_64, i?86, riscv64, sparc 32-bit, s390 31-bit are affected too (and various others).
+
+This patch instead of adding a long list of further architectures that wouldn't be checked ever next to arm 32-bit changes the structures to match the 2.31 layout and performs the checking on Linux for ipc_perm mode position/size only on non-Linux or on Linux with glibc 2.31 or later. I think this matches what is done for aarch64 already.
+If needed, we could list architectures that haven't changed ABI (e.g. powerpc), so that they would be checked even with older glibcs. AFAIK sanitizers don't actually use ipc_perm.mode and
+so all they care about is the size and alignment of the whole structure.
+
+Note, s390 31-bit and arm 32-bit big-endian changed ABI even further, there will now be shmctl with old symbol version and shmctl@@GLIBC_2.31 which will be incompatible. I'm afraid this isn't really solvable unless the sanitizer libraries are symbol versioned and use matching symbol versions to glibc symbols for stuff they intercept, plus use dlvsym.
+This patch doesn't try to address that.
+
+Patch by Jakub Jelinek.
+
+Reviewers: kcc, eugenis, dvyukov
+
+Reviewed By: eugenis
+
+Subscribers: jyknight, kristof.beyls, fedor.sergeev, simoncook, PkmX, s.egerton, steven.zhang, #sanitizers, llvm-commits
+
+Tags: #sanitizers, #llvm
+
+Differential Revision: https://reviews.llvm.org/D70662
+---
+ .../sanitizer_platform_limits_posix.cpp | 8 +++-----
+ .../sanitizer_platform_limits_posix.h | 15 +--------------
+ 2 files changed, 4 insertions(+), 19 deletions(-)
+
+diff --git a/compiler-rt/lib/sanitizer_common/sanitizer_platform_limits_posix.cc b/compiler-rt/lib/sanitizer_common/sanitizer_platform_limits_posix.cc
+index b4f8f67b664c..aa845df4dde4 100644
+--- a/compiler-rt/lib/sanitizer_common/sanitizer_platform_limits_posix.cc
++++ b/compiler-rt/lib/sanitizer_common/sanitizer_platform_limits_posix.cc
+@@ -1126,8 +1126,9 @@ CHECK_SIZE_AND_OFFSET(ipc_perm, uid);
+ CHECK_SIZE_AND_OFFSET(ipc_perm, gid);
+ CHECK_SIZE_AND_OFFSET(ipc_perm, cuid);
+ CHECK_SIZE_AND_OFFSET(ipc_perm, cgid);
+-#if !defined(__aarch64__) || !SANITIZER_LINUX || __GLIBC_PREREQ (2, 21)
+-/* On aarch64 glibc 2.20 and earlier provided incorrect mode field. */
++#if !SANITIZER_LINUX || __GLIBC_PREREQ (2, 31)
++/* glibc 2.30 and earlier provided 16-bit mode field instead of 32-bit
++ on many architectures. */
+ CHECK_SIZE_AND_OFFSET(ipc_perm, mode);
+ #endif
+
+diff --git a/compiler-rt/lib/sanitizer_common/sanitizer_platform_limits_posix.h b/compiler-rt/lib/sanitizer_common/sanitizer_platform_limits_posix.h
+index 7789bc5887a4..5337b26b29b8 100644
+--- a/compiler-rt/lib/sanitizer_common/sanitizer_platform_limits_posix.h
++++ b/compiler-rt/lib/sanitizer_common/sanitizer_platform_limits_posix.h
+@@ -203,26 +203,13 @@ namespace __sanitizer {
+ u64 __unused1;
+ u64 __unused2;
+ #elif defined(__sparc__)
+-#if defined(__arch64__)
+ unsigned mode;
+- unsigned short __pad1;
+-#else
+- unsigned short __pad1;
+- unsigned short mode;
+ unsigned short __pad2;
+-#endif
+ unsigned short __seq;
+ unsigned long long __unused1;
+ unsigned long long __unused2;
+-#elif defined(__mips__) || defined(__aarch64__) || defined(__s390x__)
+- unsigned int mode;
+- unsigned short __seq;
+- unsigned short __pad1;
+- unsigned long __unused1;
+- unsigned long __unused2;
+ #else
+- unsigned short mode;
+- unsigned short __pad1;
++ unsigned int mode;
+ unsigned short __seq;
+ unsigned short __pad2;
+ #if defined(__x86_64__) && !defined(_LP64)