You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@kudu.apache.org by ad...@apache.org on 2019/11/07 01:27:00 UTC
[kudu] branch master updated: KUDU-2990: remove memkind/libnuma and
dynamically link memkind at runtime
This is an automated email from the ASF dual-hosted git repository.
adar pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/kudu.git
The following commit(s) were added to refs/heads/master by this push:
new ba908ef KUDU-2990: remove memkind/libnuma and dynamically link memkind at runtime
ba908ef is described below
commit ba908efa15774bb24b5bfa4ea88915161d1100d1
Author: Adar Dembo <ad...@cloudera.com>
AuthorDate: Fri Nov 1 22:37:20 2019 -0700
KUDU-2990: remove memkind/libnuma and dynamically link memkind at runtime
This patch removes memkind and libnuma from the thirdparty tree and changes
the NVM cache implementation to dynamically link memkind at runtime using
dlopen() and friends. KUDU-2990 explains why we need to do this in great
detail so I won't repeat that here. The alternatives I considered:
1. Patch memkind to dlopen() libnuma.
This is probably the most user-friendly approach because libnuma is found in
most systems (or at least in most package repositories), but a new enough
memkind is not, and having Kudu distribute memkind eases adoption of the NVM
cache. However, I was nervous about performing deep surgery in memkind as
it's a codebase with which I'm not familiar, and I wanted to minimize risk
because we'll be backporting this patch to several older branches.
2. Patch memkind to define weak libnuma symbols.
If done correctly, behavior is unchanged when libnuma is present on the host
system, but if it's not there, calls from memkind to libnuma will crash.
Again, I didn't feel comfortable hacking into memkind, plus I've found weak
symbols difficult to use in the past.
3. Remove the NVM cache from Kudu altogether.
This is obviously the safest and simplest option, but it punishes Kudu users
who actually use the NVM cache.
4. Gate the build of the NVM cache behind a CMake option.
This ought to satisfy the ASF's definition of an "optional" feature, but
does nothing for binary redistributors who wish to offer the NVM cache and
who build Kudu as a statically linked "fat" binary.
5. Build as we do today, but forbid static linkage of libnuma.
Binary redistributors will need to choose between including libnuma in their
distributions, or forcing Kudu to look for libnuma at runtime. The former
still violates ASF policy, and the latter means Kudu won't start on a system
lacking libnuma, regardless of whether the NVM cache is actually in use.
So what are the ramifications of the chosen approach?
- Kudu no longer distributes memkind or libnuma. To use the NVM cache, the
host system must provide both, and memkind must be version 1.6.0 or newer.
CentOS 6 and Ubuntu 18.04 repositories all carried memkind 1.1.0. CentOS 7
has memkind 1.7.0. Persistent memory hardware itself also has a pretty
steep kernel version requirement, so it's unlikely to be found outside of
a new distro in the first place.
- Tests that exercise the NVM cache will be skipped if they can't find a
conformant memkind (and libnuma).
- When starting Kudu, if you don't set --block_cache_type=NVM, you shoudn't
notice any change.
- If you do, Kudu will crash at startup if it can't find a conformant
memkind. This affects upgrades: if you were already an NVM cache user but
you didn't have memkind installed, your Kudu will crash post-upgrade.
Note: this doesn't preclude implementing alternative #1 (the one I think is
ideal) in the future; we'll just have to revert the bulk of this patch when
we do so.
To test, I ran cfile-test and cache-test as follows:
- Without memkind installed: DRAM tests passed, NVM tests were skipped
- With an old memkind installed: DRAM tests passed, NVM tests were skipped
- With LD_LIBRARY_PATH=/path/to/memkind-1.9.0: All tests passed
I also manually ran a Kudu master with --block_cache_type=NVM and without
memkind to verify the crashing behavior.
Change-Id: I4f474196aa98b5fa6e5966b9a3aea9a7e466805c
Reviewed-on: http://gerrit.cloudera.org:8080/14620
Tested-by: Kudu Jenkins
Reviewed-by: Alexey Serbin <as...@cloudera.com>
---
CMakeLists.txt | 24 -----
cmake_modules/FindMemkind.cmake | 38 -------
cmake_modules/FindNuma.cmake | 38 -------
src/kudu/cfile/CMakeLists.txt | 6 --
src/kudu/cfile/block_cache.cc | 8 +-
src/kudu/cfile/cfile-test.cc | 61 ++++++++---
src/kudu/util/CMakeLists.txt | 31 +-----
src/kudu/util/cache-test.cc | 57 +++++-----
src/kudu/util/nvm_cache.cc | 116 +++++++++++++++++++--
src/kudu/util/nvm_cache.h | 13 +++
thirdparty/build-definitions.sh | 68 ------------
thirdparty/build-thirdparty.sh | 18 ----
thirdparty/download-thirdparty.sh | 15 ---
.../memkind-fix-build-with-old-autoconf.patch | 32 ------
.../patches/memkind-fix-build-with-old-glibc.patch | 24 -----
...kind-fix-jemalloc-build-with-old-autoconf.patch | 36 -------
thirdparty/vars.sh | 9 --
17 files changed, 197 insertions(+), 397 deletions(-)
diff --git a/CMakeLists.txt b/CMakeLists.txt
index e253cb4..8331aa6 100644
--- a/CMakeLists.txt
+++ b/CMakeLists.txt
@@ -1139,30 +1139,6 @@ if (NOT "${KUDU_USE_ASAN}" AND
set(KUDU_TCMALLOC_AVAILABLE 1)
endif()
-# Required memkind libraries
-if (NOT APPLE)
- find_package(Memkind REQUIRED)
- include_directories(${MEMKIND_INCLUDE_DIR})
- ADD_THIRDPARTY_LIB(memkind
- STATIC_LIB "${MEMKIND_STATIC_LIB}"
- SHARED_LIB "${MEMKIND_SHARED_LIB}"
- DEPS numa)
-endif()
-
-if (EXISTS ${MEMKIND_SHARED_LIB})
- set(HAVE_LIB_MEMKIND 1)
- add_definitions(-DHAVE_LIB_MEMKIND=1)
-endif()
-
-## libnuma
-if (NOT APPLE)
- find_package(Numa REQUIRED)
- include_directories(${NUMA_INCLUDE_DIR})
- ADD_THIRDPARTY_LIB(numa
- STATIC_LIB "${NUMA_STATIC_LIB}"
- SHARED_LIB "${NUMA_SHARED_LIB}")
-endif()
-
## curl
find_package(CURL REQUIRED)
diff --git a/cmake_modules/FindMemkind.cmake b/cmake_modules/FindMemkind.cmake
deleted file mode 100644
index 7e4cd02..0000000
--- a/cmake_modules/FindMemkind.cmake
+++ /dev/null
@@ -1,38 +0,0 @@
-# Licensed to the Apache Software Foundation (ASF) under one
-# or more contributor license agreements. See the NOTICE file
-# distributed with this work for additional information
-# regarding copyright ownership. The ASF licenses this file
-# to you under the Apache License, Version 2.0 (the
-# "License"); you may not use this file except in compliance
-# with the License. You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing,
-# software distributed under the License is distributed on an
-# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-# KIND, either express or implied. See the License for the
-# specific language governing permissions and limitations
-# under the License.
-
-# - Find required memkind libraries
-# This module defines
-# MEMKIND_INCLUDE_DIR, directory containing headers
-# MEMKIND_STATIC_LIB, path to *.a
-# MEMKIND_SHARED_LIB, path to *.so shared library
-
-find_path(MEMKIND_INCLUDE_DIR memkind.h
- NO_CMAKE_SYSTEM_PATH
- NO_SYSTEM_ENVIRONMENT_PATH)
-find_library(MEMKIND_SHARED_LIB memkind
- NO_CMAKE_SYSTEM_PATH
- NO_SYSTEM_ENVIRONMENT_PATH)
-find_library(MEMKIND_STATIC_LIB libmemkind.a
- NO_CMAKE_SYSTEM_PATH
- NO_SYSTEM_ENVIRONMENT_PATH)
-
-include(FindPackageHandleStandardArgs)
-find_package_handle_standard_args(MEMKIND REQUIRED_VARS
- MEMKIND_SHARED_LIB MEMKIND_STATIC_LIB
- MEMKIND_INCLUDE_DIR
-)
diff --git a/cmake_modules/FindNuma.cmake b/cmake_modules/FindNuma.cmake
deleted file mode 100644
index 2167b28..0000000
--- a/cmake_modules/FindNuma.cmake
+++ /dev/null
@@ -1,38 +0,0 @@
-# Licensed to the Apache Software Foundation (ASF) under one
-# or more contributor license agreements. See the NOTICE file
-# distributed with this work for additional information
-# regarding copyright ownership. The ASF licenses this file
-# to you under the Apache License, Version 2.0 (the
-# "License"); you may not use this file except in compliance
-# with the License. You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing,
-# software distributed under the License is distributed on an
-# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-# KIND, either express or implied. See the License for the
-# specific language governing permissions and limitations
-# under the License.
-
-# - Find required numa libraries
-# This module defines
-# NUMA_INCLUDE_DIR, directory containing headers
-# NUMA_STATIC_LIB, path to *.a
-# NUMA_SHARED_LIB, path to *.so shared library
-
-find_path(NUMA_INCLUDE_DIR numa.h
- NO_CMAKE_SYSTEM_PATH
-NO_SYSTEM_ENVIRONMENT_PATH)
-find_library(NUMA_SHARED_LIB numa
- NO_CMAKE_SYSTEM_PATH
- NO_SYSTEM_ENVIRONMENT_PATH)
-find_library(NUMA_STATIC_LIB libnuma.a
- NO_CMAKE_SYSTEM_PATH
- NO_SYSTEM_ENVIRONMENT_PATH)
-
-include(FindPackageHandleStandardArgs)
-find_package_handle_standard_args(NUMA REQUIRED_VARS
- NUMA_SHARED_LIB NUMA_STATIC_LIB
- NUMA_INCLUDE_DIR
-)
diff --git a/src/kudu/cfile/CMakeLists.txt b/src/kudu/cfile/CMakeLists.txt
index e2cd369..5b72020 100644
--- a/src/kudu/cfile/CMakeLists.txt
+++ b/src/kudu/cfile/CMakeLists.txt
@@ -55,12 +55,6 @@ set(CFILE_LIBS
cfile_proto
bitshuffle)
-if(HAVE_LIB_MEMKIND)
- set(CFILE_LIBS
- ${CFILE_LIBS}
- nvm_cache)
-endif()
-
target_link_libraries(cfile ${CFILE_LIBS})
# Tests
diff --git a/src/kudu/cfile/block_cache.cc b/src/kudu/cfile/block_cache.cc
index a464676..c00bee1 100644
--- a/src/kudu/cfile/block_cache.cc
+++ b/src/kudu/cfile/block_cache.cc
@@ -54,7 +54,9 @@ DEFINE_string(block_cache_type, "DRAM",
"Which type of block cache to use for caching data. "
"Valid choices are 'DRAM' or 'NVM'. DRAM, the default, "
"caches data in regular memory. 'NVM' caches data "
- "in a memory-mapped file using the memkind library.");
+ "in a memory-mapped file using the memkind library. To use 'NVM', "
+ "libmemkind 1.6.0 or newer must be available on the system; "
+ "otherwise Kudu will crash.");
using strings::Substitute;
@@ -75,12 +77,8 @@ Cache* CreateCache(int64_t capacity) {
return NewCache<Cache::EvictionPolicy::LRU, Cache::MemoryType::DRAM>(
capacity, "block_cache");
case Cache::MemoryType::NVM:
-#if defined(HAVE_LIB_MEMKIND)
return NewCache<Cache::EvictionPolicy::LRU, Cache::MemoryType::NVM>(
capacity, "block_cache");
-#else
- LOG(FATAL) << "cache of NVM memory type is not supported";
-#endif // #if defined(HAVE_LIB_MEMKIND) ... #else ...
default:
LOG(FATAL) << "unsupported LRU cache memory type: " << mem_type;
return nullptr;
diff --git a/src/kudu/cfile/cfile-test.cc b/src/kudu/cfile/cfile-test.cc
index 0333052..d8adcf0 100644
--- a/src/kudu/cfile/cfile-test.cc
+++ b/src/kudu/cfile/cfile-test.cc
@@ -70,6 +70,7 @@
#include "kudu/util/mem_tracker.h"
#include "kudu/util/memory/arena.h"
#include "kudu/util/metrics.h"
+#include "kudu/util/nvm_cache.h"
#include "kudu/util/slice.h"
#include "kudu/util/status.h"
#include "kudu/util/stopwatch.h"
@@ -81,11 +82,8 @@ DECLARE_bool(cfile_verify_checksums);
DECLARE_string(block_cache_type);
DECLARE_bool(force_block_cache_capacity);
DECLARE_int64(block_cache_capacity_mb);
-
-#if defined(__linux__)
DECLARE_string(nvm_cache_path);
DECLARE_bool(nvm_cache_simulate_allocation_failure);
-#endif
METRIC_DECLARE_counter(block_cache_hits_caching);
@@ -404,7 +402,6 @@ class TestCFileBothCacheMemoryTypes :
public ::testing::WithParamInterface<Cache::MemoryType> {
public:
void SetUp() OVERRIDE {
-#if defined(HAVE_LIB_MEMKIND)
// The NVM cache can run using any directory as its path -- it doesn't have
// a lot of practical use outside of an actual NVM device, but for testing
// purposes, we'll point it at our test dir, unless otherwise specified.
@@ -412,16 +409,13 @@ class TestCFileBothCacheMemoryTypes :
FLAGS_nvm_cache_path = GetTestPath("nvm-cache");
ASSERT_OK(Env::Default()->CreateDir(FLAGS_nvm_cache_path));
}
-#endif
switch (GetParam()) {
case Cache::MemoryType::DRAM:
FLAGS_block_cache_type = "DRAM";
break;
-#if defined(HAVE_LIB_MEMKIND)
case Cache::MemoryType::NVM:
FLAGS_block_cache_type = "NVM";
break;
-#endif
default:
LOG(FATAL) << "Unknown block cache type: '"
<< static_cast<int16_t>(GetParam());
@@ -434,14 +428,9 @@ class TestCFileBothCacheMemoryTypes :
}
};
-#if defined(__linux__)
INSTANTIATE_TEST_CASE_P(CacheMemoryTypes, TestCFileBothCacheMemoryTypes,
::testing::Values(Cache::MemoryType::DRAM,
Cache::MemoryType::NVM));
-#else
-INSTANTIATE_TEST_CASE_P(CacheMemoryTypes, TestCFileBothCacheMemoryTypes,
- ::testing::Values(Cache::MemoryType::DRAM));
-#endif
template<DataType type>
void CopyOne(CFileIterator *it,
@@ -461,6 +450,7 @@ void CopyOne(CFileIterator *it,
// They take way too long with debugging enabled.
TEST_P(TestCFileBothCacheMemoryTypes, TestWrite100MFileInts) {
+ RETURN_IF_NO_NVM_CACHE(GetParam());
BlockId block_id;
LOG_TIMING(INFO, "writing 100m ints") {
LOG(INFO) << "Starting writefile";
@@ -479,6 +469,7 @@ TEST_P(TestCFileBothCacheMemoryTypes, TestWrite100MFileInts) {
}
TEST_P(TestCFileBothCacheMemoryTypes, TestWrite100MFileNullableInts) {
+ RETURN_IF_NO_NVM_CACHE(GetParam());
BlockId block_id;
LOG_TIMING(INFO, "writing 100m nullable ints") {
LOG(INFO) << "Starting writefile";
@@ -497,18 +488,22 @@ TEST_P(TestCFileBothCacheMemoryTypes, TestWrite100MFileNullableInts) {
}
TEST_P(TestCFileBothCacheMemoryTypes, TestWrite100MFileStringsPrefixEncoding) {
+ RETURN_IF_NO_NVM_CACHE(GetParam());
TestWrite100MFileStrings(PREFIX_ENCODING);
}
TEST_P(TestCFileBothCacheMemoryTypes, TestWrite100MUniqueStringsDictEncoding) {
+ RETURN_IF_NO_NVM_CACHE(GetParam());
TestWrite100MFileStrings(DICT_ENCODING);
}
TEST_P(TestCFileBothCacheMemoryTypes, TestWrite100MLowCardinalityStringsDictEncoding) {
+ RETURN_IF_NO_NVM_CACHE(GetParam());
TestWriteDictEncodingLowCardinalityStrings(100 * 1e6);
}
TEST_P(TestCFileBothCacheMemoryTypes, TestWrite100MFileStringsPlainEncoding) {
+ RETURN_IF_NO_NVM_CACHE(GetParam());
TestWrite100MFileStrings(PLAIN_ENCODING);
}
@@ -516,6 +511,7 @@ TEST_P(TestCFileBothCacheMemoryTypes, TestWrite100MFileStringsPlainEncoding) {
// Write and Read 1 million unique strings with dictionary encoding
TEST_P(TestCFileBothCacheMemoryTypes, TestWrite1MUniqueFileStringsDictEncoding) {
+ RETURN_IF_NO_NVM_CACHE(GetParam());
BlockId block_id;
LOG_TIMING(INFO, "writing 1M unique strings") {
LOG(INFO) << "Starting writefile";
@@ -535,41 +531,49 @@ TEST_P(TestCFileBothCacheMemoryTypes, TestWrite1MUniqueFileStringsDictEncoding)
// Write and Read 1 million strings, which contains duplicates with dictionary encoding
TEST_P(TestCFileBothCacheMemoryTypes, TestWrite1MLowCardinalityStringsDictEncoding) {
+ RETURN_IF_NO_NVM_CACHE(GetParam());
TestWriteDictEncodingLowCardinalityStrings(1000000);
}
TEST_P(TestCFileBothCacheMemoryTypes, TestReadWriteUInt32) {
+ RETURN_IF_NO_NVM_CACHE(GetParam());
for (auto enc : { PLAIN_ENCODING, RLE }) {
TestReadWriteFixedSizeTypes<UInt32DataGenerator<false>>(enc);
}
}
TEST_P(TestCFileBothCacheMemoryTypes, TestReadWriteInt32) {
+ RETURN_IF_NO_NVM_CACHE(GetParam());
for (auto enc : { PLAIN_ENCODING, RLE }) {
TestReadWriteFixedSizeTypes<Int32DataGenerator<false>>(enc);
}
}
TEST_P(TestCFileBothCacheMemoryTypes, TestReadWriteUInt64) {
+ RETURN_IF_NO_NVM_CACHE(GetParam());
for (auto enc : { PLAIN_ENCODING, RLE, BIT_SHUFFLE }) {
TestReadWriteFixedSizeTypes<UInt64DataGenerator<false>>(enc);
}
}
TEST_P(TestCFileBothCacheMemoryTypes, TestReadWriteInt64) {
+ RETURN_IF_NO_NVM_CACHE(GetParam());
for (auto enc : { PLAIN_ENCODING, RLE, BIT_SHUFFLE }) {
TestReadWriteFixedSizeTypes<Int64DataGenerator<false>>(enc);
}
}
TEST_P(TestCFileBothCacheMemoryTypes, TestReadWriteInt128) {
+ RETURN_IF_NO_NVM_CACHE(GetParam());
TestReadWriteFixedSizeTypes<Int128DataGenerator<false>>(PLAIN_ENCODING);
}
TEST_P(TestCFileBothCacheMemoryTypes, TestFixedSizeReadWritePlainEncodingFloat) {
+ RETURN_IF_NO_NVM_CACHE(GetParam());
TestReadWriteFixedSizeTypes<FPDataGenerator<FLOAT, false>>(PLAIN_ENCODING);
}
TEST_P(TestCFileBothCacheMemoryTypes, TestFixedSizeReadWritePlainEncodingDouble) {
+ RETURN_IF_NO_NVM_CACHE(GetParam());
TestReadWriteFixedSizeTypes<FPDataGenerator<DOUBLE, false>>(PLAIN_ENCODING);
}
@@ -725,11 +729,13 @@ void TestCFile::TestReadWriteStrings(EncodingType encoding,
TEST_P(TestCFileBothCacheMemoryTypes, TestReadWriteStringsPrefixEncoding) {
+ RETURN_IF_NO_NVM_CACHE(GetParam());
TestReadWriteStrings(PREFIX_ENCODING);
}
// Read/Write test for dictionary encoded blocks
TEST_P(TestCFileBothCacheMemoryTypes, TestReadWriteStringsDictEncoding) {
+ RETURN_IF_NO_NVM_CACHE(GetParam());
TestReadWriteStrings(DICT_ENCODING);
}
@@ -740,6 +746,8 @@ TEST_P(TestCFileBothCacheMemoryTypes, TestReadWriteStringsDictEncoding) {
// and runs extremely slowly with TSAN enabled.
#ifndef THREAD_SANITIZER
TEST_P(TestCFileBothCacheMemoryTypes, TestReadWriteLargeStrings) {
+ RETURN_IF_NO_NVM_CACHE(GetParam());
+
// Pad the values out to a length of ~65KB.
// We use this method instead of just a longer sprintf format since
// this is much more CPU-efficient (speeds up the test).
@@ -758,6 +766,8 @@ TEST_P(TestCFileBothCacheMemoryTypes, TestReadWriteLargeStrings) {
// Test that metadata entries stored in the cfile are persisted.
TEST_P(TestCFileBothCacheMemoryTypes, TestMetadata) {
+ RETURN_IF_NO_NVM_CACHE(GetParam());
+
BlockId block_id;
// Write the file.
@@ -799,6 +809,8 @@ TEST_P(TestCFileBothCacheMemoryTypes, TestMetadata) {
}
TEST_P(TestCFileBothCacheMemoryTypes, TestDefaultColumnIter) {
+ RETURN_IF_NO_NVM_CACHE(GetParam());
+
const int kNumItems = 64;
uint8_t null_bitmap[BitmapSize(kNumItems)];
uint32_t data[kNumItems];
@@ -848,6 +860,7 @@ TEST_P(TestCFileBothCacheMemoryTypes, TestDefaultColumnIter) {
}
TEST_P(TestCFileBothCacheMemoryTypes, TestAppendRaw) {
+ RETURN_IF_NO_NVM_CACHE(GetParam());
TestReadWriteRawBlocks(NO_COMPRESSION, 1000);
TestReadWriteRawBlocks(SNAPPY, 1000);
TestReadWriteRawBlocks(LZ4, 1000);
@@ -855,6 +868,7 @@ TEST_P(TestCFileBothCacheMemoryTypes, TestAppendRaw) {
}
TEST_P(TestCFileBothCacheMemoryTypes, TestChecksumFlags) {
+ RETURN_IF_NO_NVM_CACHE(GetParam());
for (bool write_checksums : {false, true}) {
for (bool verify_checksums : {false, true}) {
FLAGS_cfile_write_checksums = write_checksums;
@@ -866,6 +880,7 @@ TEST_P(TestCFileBothCacheMemoryTypes, TestChecksumFlags) {
}
TEST_P(TestCFileBothCacheMemoryTypes, TestDataCorruption) {
+ RETURN_IF_NO_NVM_CACHE(GetParam());
FLAGS_cfile_write_checksums = true;
FLAGS_cfile_verify_checksums = true;
@@ -903,6 +918,8 @@ TEST_P(TestCFileBothCacheMemoryTypes, TestDataCorruption) {
}
TEST_P(TestCFileBothCacheMemoryTypes, TestNullInts) {
+ RETURN_IF_NO_NVM_CACHE(GetParam());
+
UInt32DataGenerator<true> generator;
TestNullTypes(&generator, PLAIN_ENCODING, NO_COMPRESSION);
TestNullTypes(&generator, PLAIN_ENCODING, LZ4);
@@ -913,18 +930,24 @@ TEST_P(TestCFileBothCacheMemoryTypes, TestNullInts) {
}
TEST_P(TestCFileBothCacheMemoryTypes, TestNullFloats) {
+ RETURN_IF_NO_NVM_CACHE(GetParam());
+
FPDataGenerator<FLOAT, true> generator;
TestNullTypes(&generator, PLAIN_ENCODING, NO_COMPRESSION);
TestNullTypes(&generator, BIT_SHUFFLE, NO_COMPRESSION);
}
TEST_P(TestCFileBothCacheMemoryTypes, TestNullPrefixStrings) {
+ RETURN_IF_NO_NVM_CACHE(GetParam());
+
StringDataGenerator<true> generator("hello %zu");
TestNullTypes(&generator, PLAIN_ENCODING, NO_COMPRESSION);
TestNullTypes(&generator, PLAIN_ENCODING, LZ4);
}
TEST_P(TestCFileBothCacheMemoryTypes, TestNullPlainStrings) {
+ RETURN_IF_NO_NVM_CACHE(GetParam());
+
StringDataGenerator<true> generator("hello %zu");
TestNullTypes(&generator, PREFIX_ENCODING, NO_COMPRESSION);
TestNullTypes(&generator, PREFIX_ENCODING, LZ4);
@@ -932,12 +955,16 @@ TEST_P(TestCFileBothCacheMemoryTypes, TestNullPlainStrings) {
// Test for dictionary encoding
TEST_P(TestCFileBothCacheMemoryTypes, TestNullDictStrings) {
+ RETURN_IF_NO_NVM_CACHE(GetParam());
+
StringDataGenerator<true> generator("hello %zu");
TestNullTypes(&generator, DICT_ENCODING, NO_COMPRESSION);
TestNullTypes(&generator, DICT_ENCODING, LZ4);
}
TEST_P(TestCFileBothCacheMemoryTypes, TestReleaseBlock) {
+ RETURN_IF_NO_NVM_CACHE(GetParam());
+
unique_ptr<WritableBlock> sink;
ASSERT_OK(fs_manager_->CreateNewBlock({}, &sink));
ASSERT_EQ(WritableBlock::CLEAN, sink->state());
@@ -951,6 +978,8 @@ TEST_P(TestCFileBothCacheMemoryTypes, TestReleaseBlock) {
}
TEST_P(TestCFileBothCacheMemoryTypes, TestLazyInit) {
+ RETURN_IF_NO_NVM_CACHE(GetParam());
+
// Create a small test file.
BlockId block_id;
{
@@ -1002,6 +1031,8 @@ TEST_P(TestCFileBothCacheMemoryTypes, TestLazyInit) {
// different reader instances operating on the same block should use the same
// block cache keys.
TEST_P(TestCFileBothCacheMemoryTypes, TestCacheKeysAreStable) {
+ RETURN_IF_NO_NVM_CACHE(GetParam());
+
// Set up block cache instrumentation.
MetricRegistry registry;
scoped_refptr<MetricEntity> entity(METRIC_ENTITY_server.Instantiate(®istry, "test_entity"));
@@ -1041,16 +1072,18 @@ TEST_P(TestCFileBothCacheMemoryTypes, TestCacheKeysAreStable) {
}
}
-#if defined(HAVE_LIB_MEMKIND)
// Inject failures in nvm allocation and ensure that we can still read a file.
TEST_P(TestCFileBothCacheMemoryTypes, TestNvmAllocationFailure) {
if (GetParam() != Cache::MemoryType::NVM) return;
+ RETURN_IF_NO_NVM_CACHE(GetParam());
+
FLAGS_nvm_cache_simulate_allocation_failure = true;
TestReadWriteFixedSizeTypes<UInt32DataGenerator<false> >(PLAIN_ENCODING);
}
-#endif
TEST_P(TestCFileBothCacheMemoryTypes, TestValidateBlockCacheCapacity) {
+ RETURN_IF_NO_NVM_CACHE(GetParam());
+
FLAGS_force_block_cache_capacity = false;
FLAGS_block_cache_capacity_mb = 100000000; // very big number (100TB)
switch (GetParam()) {
diff --git a/src/kudu/util/CMakeLists.txt b/src/kudu/util/CMakeLists.txt
index cd184ec..e45d8ce 100644
--- a/src/kudu/util/CMakeLists.txt
+++ b/src/kudu/util/CMakeLists.txt
@@ -196,6 +196,7 @@ set(UTIL_SRCS
net/net_util.cc
net/sockaddr.cc
net/socket.cc
+ nvm_cache.cc
oid_generator.cc
once.cc
os-util.cc
@@ -277,31 +278,6 @@ ADD_EXPORTABLE_LIBRARY(kudu_util
EXPORTED_DEPS ${EXPORTED_UTIL_LIBS})
#######################################
-# nvm_cache
-#######################################
-
-if(HAVE_LIB_MEMKIND)
- set(NVM_CACHE_SRCS
- nvm_cache.cc)
-
- set(NVM_CACHE_LIBS
- gflags
- glog
- gutil
- memkind)
-
- if(NOT APPLE)
- set(NVM_CACHE_LIBS
- ${NVM_CACHE_LIBS}
- dl
- rt)
- endif()
-
- add_library(nvm_cache ${NVM_CACHE_SRCS})
- target_link_libraries(nvm_cache ${NVM_CACHE_LIBS})
-endif()
-
-#######################################
# kudu_util_compression
#######################################
set(UTIL_COMPRESSION_SRCS
@@ -359,11 +335,6 @@ target_link_libraries(kudu_test_util
gmock
kudu_util)
-if(HAVE_LIB_MEMKIND)
- target_link_libraries(kudu_test_util
- nvm_cache)
-endif()
-
#######################################
# kudu_curl_util
#######################################
diff --git a/src/kudu/util/cache-test.cc b/src/kudu/util/cache-test.cc
index 66c78e9..6ba34b2 100644
--- a/src/kudu/util/cache-test.cc
+++ b/src/kudu/util/cache-test.cc
@@ -25,14 +25,13 @@
#include "kudu/util/faststring.h"
#include "kudu/util/mem_tracker.h"
#include "kudu/util/metrics.h"
+#include "kudu/util/nvm_cache.h"
#include "kudu/util/slice.h"
#include "kudu/util/test_macros.h"
#include "kudu/util/test_util.h"
DECLARE_bool(cache_force_single_shard);
-#if defined(HAVE_LIB_MEMKIND)
DECLARE_string(nvm_cache_path);
-#endif // #if defined(HAVE_LIB_MEMKIND)
DECLARE_double(cache_memtracker_approximation_ratio);
@@ -111,12 +110,10 @@ class CacheBaseTest : public KuduTest,
FLAGS_cache_force_single_shard =
(sharding_policy == ShardingPolicy::SingleShard);
-#if defined(HAVE_LIB_MEMKIND)
if (google::GetCommandLineFlagInfoOrDie("nvm_cache_path").is_default) {
FLAGS_nvm_cache_path = GetTestPath("nvm-cache");
ASSERT_OK(Env::Default()->CreateDir(FLAGS_nvm_cache_path));
}
-#endif // #if defined(HAVE_LIB_MEMKIND)
switch (eviction_policy) {
case Cache::EvictionPolicy::FIFO:
@@ -136,13 +133,11 @@ class CacheBaseTest : public KuduTest,
"cache_test"));
break;
case Cache::MemoryType::NVM:
-#if defined(HAVE_LIB_MEMKIND)
- cache_.reset(NewCache<Cache::EvictionPolicy::LRU,
- Cache::MemoryType::NVM>(cache_size(),
- "cache_test"));
-#else
- FAIL() << "cache of NVM memory type is not supported";
-#endif // #if defined(HAVE_LIB_MEMKIND) ... #else ...
+ if (CanUseNVMCacheForTests()) {
+ cache_.reset(NewCache<Cache::EvictionPolicy::LRU,
+ Cache::MemoryType::NVM>(cache_size(),
+ "cache_test"));
+ }
break;
default:
FAIL() << mem_type << ": unrecognized cache memory type";
@@ -161,10 +156,14 @@ class CacheBaseTest : public KuduTest,
ASSERT_TRUE(mem_tracker_.get());
}
- scoped_refptr<MetricEntity> entity = METRIC_ENTITY_server.Instantiate(
- &metric_registry_, "test");
- unique_ptr<BlockCacheMetrics> metrics(new BlockCacheMetrics(entity));
- cache_->SetMetrics(std::move(metrics));
+ // cache_ will be null if we're trying to set up a test for the NVM cache
+ // and were unable to do so.
+ if (cache_) {
+ scoped_refptr<MetricEntity> entity = METRIC_ENTITY_server.Instantiate(
+ &metric_registry_, "test");
+ unique_ptr<BlockCacheMetrics> metrics(new BlockCacheMetrics(entity));
+ cache_->SetMetrics(std::move(metrics));
+ }
}
const size_t cache_size_;
@@ -193,7 +192,6 @@ class CacheTest :
}
};
-#if defined(HAVE_LIB_MEMKIND)
INSTANTIATE_TEST_CASE_P(
CacheTypes, CacheTest,
::testing::Values(
@@ -215,17 +213,9 @@ INSTANTIATE_TEST_CASE_P(
make_tuple(Cache::MemoryType::NVM,
Cache::EvictionPolicy::LRU,
ShardingPolicy::SingleShard)));
-#else
-INSTANTIATE_TEST_CASE_P(
- CacheTypes, CacheTest,
- ::testing::Combine(::testing::Values(Cache::MemoryType::DRAM),
- ::testing::Values(Cache::EvictionPolicy::FIFO,
- Cache::EvictionPolicy::LRU),
- ::testing::Values(ShardingPolicy::MultiShard,
- ShardingPolicy::SingleShard)));
-#endif // #if defined(HAVE_LIB_MEMKIND) ... #else ...
TEST_P(CacheTest, TrackMemory) {
+ RETURN_IF_NO_NVM_CACHE(std::get<0>(GetParam()));
if (mem_tracker_) {
Insert(100, 100, 1);
ASSERT_EQ(1, mem_tracker_->consumption());
@@ -236,6 +226,7 @@ TEST_P(CacheTest, TrackMemory) {
}
TEST_P(CacheTest, HitAndMiss) {
+ RETURN_IF_NO_NVM_CACHE(std::get<0>(GetParam()));
ASSERT_EQ(-1, Lookup(100));
Insert(100, 101);
@@ -259,6 +250,7 @@ TEST_P(CacheTest, HitAndMiss) {
}
TEST_P(CacheTest, Erase) {
+ RETURN_IF_NO_NVM_CACHE(std::get<0>(GetParam()));
Erase(200);
ASSERT_EQ(0, evicted_keys_.size());
@@ -278,6 +270,7 @@ TEST_P(CacheTest, Erase) {
}
TEST_P(CacheTest, EntriesArePinned) {
+ RETURN_IF_NO_NVM_CACHE(std::get<0>(GetParam()));
Insert(100, 101);
auto h1 = cache_->Lookup(EncodeInt(100), Cache::EXPECT_IN_CACHE);
ASSERT_EQ(101, DecodeInt(cache_->Value(h1)));
@@ -306,6 +299,7 @@ TEST_P(CacheTest, EntriesArePinned) {
// size of items still in the cache, which must be approximately the
// same as the total capacity.
TEST_P(CacheTest, HeavyEntries) {
+ RETURN_IF_NO_NVM_CACHE(std::get<0>(GetParam()));
const int kLight = cache_size() / 1000;
const int kHeavy = cache_size() / 100;
int added = 0;
@@ -330,6 +324,7 @@ TEST_P(CacheTest, HeavyEntries) {
}
TEST_P(CacheTest, InvalidateAllEntries) {
+ RETURN_IF_NO_NVM_CACHE(std::get<0>(GetParam()));
constexpr const int kEntriesNum = 1024;
// This scenarios assumes no evictions are done at the cache capacity.
ASSERT_LE(kEntriesNum, cache_size());
@@ -353,6 +348,7 @@ TEST_P(CacheTest, InvalidateAllEntries) {
}
TEST_P(CacheTest, InvalidateNoEntries) {
+ RETURN_IF_NO_NVM_CACHE(std::get<0>(GetParam()));
constexpr const int kEntriesNum = 10;
// This scenarios assumes no evictions are done at the cache capacity.
ASSERT_LE(kEntriesNum, cache_size());
@@ -374,6 +370,7 @@ TEST_P(CacheTest, InvalidateNoEntries) {
}
TEST_P(CacheTest, InvalidateNoEntriesNoAdvanceIterationFunctor) {
+ RETURN_IF_NO_NVM_CACHE(std::get<0>(GetParam()));
constexpr const int kEntriesNum = 256;
// This scenarios assumes no evictions are done at the cache capacity.
ASSERT_LE(kEntriesNum, cache_size());
@@ -401,6 +398,7 @@ TEST_P(CacheTest, InvalidateNoEntriesNoAdvanceIterationFunctor) {
}
TEST_P(CacheTest, InvalidateOddKeyEntries) {
+ RETURN_IF_NO_NVM_CACHE(std::get<0>(GetParam()));
constexpr const int kEntriesNum = 64;
// This scenarios assumes no evictions are done at the cache capacity.
ASSERT_LE(kEntriesNum, cache_size());
@@ -506,22 +504,15 @@ class LRUCacheTest :
}
};
-#if defined(HAVE_LIB_MEMKIND)
INSTANTIATE_TEST_CASE_P(
CacheTypes, LRUCacheTest,
::testing::Combine(::testing::Values(Cache::MemoryType::DRAM,
Cache::MemoryType::NVM),
::testing::Values(ShardingPolicy::MultiShard,
ShardingPolicy::SingleShard)));
-#else
-INSTANTIATE_TEST_CASE_P(
- CacheTypes, LRUCacheTest,
- ::testing::Combine(::testing::Values(Cache::MemoryType::DRAM),
- ::testing::Values(ShardingPolicy::MultiShard,
- ShardingPolicy::SingleShard)));
-#endif // #if defined(HAVE_LIB_MEMKIND) ... #else ...
TEST_P(LRUCacheTest, EvictionPolicy) {
+ RETURN_IF_NO_NVM_CACHE(std::get<0>(GetParam()));
static constexpr int kNumElems = 1000;
const int size_per_elem = cache_size() / kNumElems;
diff --git a/src/kudu/util/nvm_cache.cc b/src/kudu/util/nvm_cache.cc
index 66424e3..56d83af 100644
--- a/src/kudu/util/nvm_cache.cc
+++ b/src/kudu/util/nvm_cache.cc
@@ -19,6 +19,8 @@
#include "kudu/util/nvm_cache.h"
+#include <dlfcn.h>
+
#include <cstdint>
#include <cstring>
#include <iostream>
@@ -31,7 +33,6 @@
#include <gflags/gflags.h>
#include <gflags/gflags_declare.h>
#include <glog/logging.h>
-#include <memkind.h>
#include "kudu/gutil/atomic_refcount.h"
#include "kudu/gutil/atomicops.h"
@@ -43,13 +44,20 @@
#include "kudu/gutil/port.h"
#include "kudu/gutil/ref_counted.h"
#include "kudu/gutil/stl_util.h"
+#include "kudu/gutil/strings/substitute.h"
#include "kudu/gutil/sysinfo.h"
#include "kudu/util/cache.h"
#include "kudu/util/cache_metrics.h"
#include "kudu/util/flag_tags.h"
#include "kudu/util/locks.h"
#include "kudu/util/metrics.h"
+#include "kudu/util/scoped_cleanup.h"
#include "kudu/util/slice.h"
+#include "kudu/util/status.h"
+
+#ifndef MEMKIND_PMEM_MIN_SIZE
+#define MEMKIND_PMEM_MIN_SIZE (1024 * 1024 * 16) // Taken from memkind 1.9.0.
+#endif
struct memkind;
@@ -75,11 +83,93 @@ TAG_FLAG(nvm_cache_simulate_allocation_failure, unsafe);
using std::string;
using std::unique_ptr;
using std::vector;
+using strings::Substitute;
namespace kudu {
namespace {
+// Taken together, these typedefs and this macro make it easy to call a
+// memkind function:
+//
+// CALL_MEMKIND(memkind_malloc, vmp_, size);
+typedef int (*memkind_create_pmem)(const char*, size_t, memkind**);
+typedef int (*memkind_destroy_kind)(memkind*);
+typedef void* (*memkind_malloc)(memkind*, size_t);
+typedef size_t (*memkind_malloc_usable_size)(memkind*, void*);
+typedef void (*memkind_free)(memkind*, void*);
+#define CALL_MEMKIND(func_name, ...) ((func_name)g_##func_name)(__VA_ARGS__)
+
+// Function pointers into memkind; set by InitMemkindOps().
+void* g_memkind_create_pmem;
+void* g_memkind_destroy_kind;
+void* g_memkind_malloc;
+void* g_memkind_malloc_usable_size;
+void* g_memkind_free;
+
+// After InitMemkindOps() is called, true if memkind is available and safe
+// to use, false otherwise.
+bool g_memkind_available;
+
+std::once_flag g_memkind_ops_flag;
+
+// Try to dlsym() a particular symbol from 'handle', storing the result in 'ptr'
+// if successful.
+Status TryDlsym(void* handle, const char* sym, void** ptr) {
+ dlerror(); // Need to clear any existing error first.
+ void* ret = dlsym(handle, sym);
+ char* error = dlerror();
+ if (error) {
+ return Status::NotSupported(Substitute("could not dlsym $0", sym), error);
+ }
+ *ptr = ret;
+ return Status::OK();
+}
+
+// Try to dlopen() memkind and set up all the function pointers we need from it.
+//
+// Note: in terms of protecting ourselves against changes in memkind, we'll
+// notice (and fail) if a symbol is missing, but not if it's signature has
+// changed or if there's some subtle behavioral change. A scan of the memkind
+// repo suggests that backwards compatibility is enforced: symbols are only
+// added and behavioral changes are effected via the introduction of new symbols.
+void InitMemkindOps() {
+ g_memkind_available = false;
+
+ // Use RTLD_NOW so that if any of memkind's dependencies aren't satisfied
+ // (e.g. libnuma is too old and is missing symbols), we'll know up front
+ // instead of during cache operations.
+ void* memkind_lib = dlopen("libmemkind.so.0", RTLD_NOW);
+ if (!memkind_lib) {
+ LOG(WARNING) << "could not dlopen: " << dlerror();
+ return;
+ }
+ auto cleanup = MakeScopedCleanup([&]() {
+ dlclose(memkind_lib);
+ });
+
+#define DLSYM_OR_RETURN(func_name, handle) do { \
+ const Status _s = TryDlsym(memkind_lib, func_name, handle); \
+ if (!_s.ok()) { \
+ LOG(WARNING) << _s.ToString(); \
+ return; \
+ } \
+ } while (0)
+
+ DLSYM_OR_RETURN("memkind_create_pmem", &g_memkind_create_pmem);
+ DLSYM_OR_RETURN("memkind_destroy_kind", &g_memkind_destroy_kind);
+ DLSYM_OR_RETURN("memkind_malloc", &g_memkind_malloc);
+ DLSYM_OR_RETURN("memkind_malloc_usable_size", &g_memkind_malloc_usable_size);
+ DLSYM_OR_RETURN("memkind_free", &g_memkind_free);
+#undef DLSYM_OR_RETURN
+
+ g_memkind_available = true;
+
+ // Need to keep the memkind library handle open so our function pointers
+ // remain loaded in memory.
+ cleanup.cancel();
+}
+
typedef simple_spinlock MutexType;
// LRU cache implementation
@@ -278,7 +368,7 @@ void* NvmLRUCache::MemkindMalloc(size_t size) {
if (PREDICT_FALSE(FLAGS_nvm_cache_simulate_allocation_failure)) {
return nullptr;
}
- return memkind_malloc(vmp_, size);
+ return CALL_MEMKIND(memkind_malloc, vmp_, size);
}
bool NvmLRUCache::Unref(LRUHandle* e) {
@@ -295,7 +385,7 @@ void NvmLRUCache::FreeEntry(LRUHandle* e) {
metrics_->cache_usage->DecrementBy(e->charge);
metrics_->evictions->Increment();
}
- memkind_free(vmp_, e);
+ CALL_MEMKIND(memkind_free, vmp_, e);
}
// Allocate nvm memory. Try until successful or FLAGS_nvm_cache_allocation_retry_count
@@ -550,7 +640,7 @@ class ShardedLRUCache : public Cache {
// Per the note at the top of this file, our cache is entirely volatile.
// Hence, when the cache is destructed, we delete the underlying
// memkind pool.
- memkind_destroy_kind(vmp_);
+ CALL_MEMKIND(memkind_destroy_kind, vmp_);
}
virtual UniqueHandle Insert(UniquePendingHandle handle,
@@ -606,7 +696,7 @@ class ShardedLRUCache : public Cache {
handle->val_length = val_len;
handle->key_length = key_len;
handle->charge = (charge == kAutomaticCharge) ?
- memkind_malloc_usable_size(vmp_, buf) : charge;
+ CALL_MEMKIND(memkind_malloc_usable_size, vmp_, buf) : charge;
handle->hash = HashSlice(key);
memcpy(handle->kv_data, key.data(), key.size());
return ph;
@@ -617,7 +707,7 @@ class ShardedLRUCache : public Cache {
}
virtual void Free(PendingHandle* ph) OVERRIDE {
- memkind_free(vmp_, ph);
+ CALL_MEMKIND(memkind_free, vmp_, ph);
}
size_t Invalidate(const InvalidationControl& ctl) override {
@@ -634,6 +724,13 @@ class ShardedLRUCache : public Cache {
template<>
Cache* NewCache<Cache::EvictionPolicy::LRU,
Cache::MemoryType::NVM>(size_t capacity, const std::string& id) {
+ std::call_once(g_memkind_ops_flag, InitMemkindOps);
+
+ // TODO(adar): we should plumb the failure up the call stack, but at the time
+ // of writing the NVM cache is only usable by the block cache, and its use of
+ // the singleton pattern prevents the surfacing of errors.
+ CHECK(g_memkind_available) << "Memkind not available!";
+
// memkind_create_pmem() will fail if the capacity is too small, but with
// an inscrutable error. So, we'll check ourselves.
CHECK_GE(capacity, MEMKIND_PMEM_MIN_SIZE)
@@ -641,7 +738,7 @@ Cache* NewCache<Cache::EvictionPolicy::LRU,
<< "the minimum capacity for an NVM cache: " << MEMKIND_PMEM_MIN_SIZE;
memkind* vmp;
- int err = memkind_create_pmem(FLAGS_nvm_cache_path.c_str(), capacity, &vmp);
+ int err = CALL_MEMKIND(memkind_create_pmem, FLAGS_nvm_cache_path.c_str(), capacity, &vmp);
// If we cannot create the cache pool we should not retry.
PLOG_IF(FATAL, err) << "Could not initialize NVM cache library in path "
<< FLAGS_nvm_cache_path.c_str();
@@ -649,4 +746,9 @@ Cache* NewCache<Cache::EvictionPolicy::LRU,
return new ShardedLRUCache(capacity, id, vmp);
}
+bool CanUseNVMCacheForTests() {
+ std::call_once(g_memkind_ops_flag, InitMemkindOps);
+ return g_memkind_available;
+}
+
} // namespace kudu
diff --git a/src/kudu/util/nvm_cache.h b/src/kudu/util/nvm_cache.h
index eebbc91..d6c2762 100644
--- a/src/kudu/util/nvm_cache.h
+++ b/src/kudu/util/nvm_cache.h
@@ -23,6 +23,19 @@
namespace kudu {
+// Convenience macro for invoking CanUseNVMCacheForTests.
+#define RETURN_IF_NO_NVM_CACHE(memory_type) do { \
+ if ((memory_type) == Cache::MemoryType::NVM && !CanUseNVMCacheForTests()) { \
+ LOG(WARNING) << "test is skipped; NVM cache cannot be created"; \
+ return; \
+ } \
+} while (0)
+
+// Returns true if an NVM cache can be created on this machine; false otherwise.
+//
+// Only intended for use in unit tests.
+bool CanUseNVMCacheForTests();
+
// Create a new LRU cache with a fixed size capacity. This implementation
// of Cache uses the least-recently-used eviction policy and stored in NVM.
template<>
diff --git a/thirdparty/build-definitions.sh b/thirdparty/build-definitions.sh
index 8dcad99..b640d75 100644
--- a/thirdparty/build-definitions.sh
+++ b/thirdparty/build-definitions.sh
@@ -762,74 +762,6 @@ build_trace_viewer() {
cp -a $TRACE_VIEWER_SOURCE/tracing.* $TP_DIR/../www/
}
-build_numactl() {
- local BUILD_TYPE=$1
-
- NUMACTL_BDIR=$TP_BUILD_DIR/$NUMACTL_NAME$MODE_SUFFIX
- mkdir -p $NUMACTL_BDIR
- pushd $NUMACTL_BDIR
-
- # Setting -fsanitize=thread only in CFLAGS does not work for compiling numactl,
- # We also need to set it in LDFLAGS.
- case $BUILD_TYPE in
- "normal")
- ;;
- "tsan")
- SANITIZER_THREAD_OPTION="-fsanitize=thread -L$PREFIX/lib -Wl,-rpath=$PREFIX/lib"
- ;;
- *)
- echo "Unknown build type: $BUILD_TYPE"
- exit 1
- ;;
- esac
-
- CFLAGS="$EXTRA_CFLAGS" \
- LDFLAGS="$EXTRA_LDFLAGS $SANITIZER_THREAD_OPTION" \
- CPPFLAGS="$EXTRA_CPPFLAGS" \
- LIBS="$EXTRA_LIBS" \
- $NUMACTL_SOURCE/configure --prefix=$PREFIX
-
- make -j$PARALLEL $EXTRA_MAKEFLAGS install
- popd
-}
-
-build_memkind() {
- MEMKIND_BDIR=$TP_BUILD_DIR/$MEMKIND_NAME$MODE_SUFFIX
- mkdir -p $MEMKIND_BDIR
- pushd $MEMKIND_BDIR
-
- # It doesn't appear possible to isolate source and build directories, so just
- # prepopulate the latter using the former.
- rsync -av --delete $MEMKIND_SOURCE/ .
-
- # We separate the build steps from build.sh in memkind source code
- # since we need to avoid building the tests code with thread sanitizer
- export JE_PREFIX=jemk_
-
- # jemalloc is built in memkind and need to be compiled firstly. Here we disable c++
- # integration to prevent new and delete operator implementations from exported in kudu client
- $MEMKIND_BDIR/build_jemalloc.sh --disable-cxx --prefix=$PREFIX
-
- $MEMKIND_BDIR/autogen.sh
- CFLAGS="$EXTRA_CFLAGS -O2" \
- CPPFLAGS="$EXTRA_CPPFLAGS -I$PREFIX/include" \
- LDFLAGS="$EXTRA_LDFLAGS -L$PREFIX/lib -Wl,-rpath=$PREFIX/lib" \
- LIBS="$EXTRA_LIBS" \
- $MEMKIND_BDIR/configure --prefix=$PREFIX
-
- # Building unit tests and memkind examples in memkind with clang will prevent some
- # header files from being included which will lead to failures. So we do a minimal
- # installation here.
- make -j$PARALLEL $EXTRA_MAKEFLAGS static_lib
- make -j$PARALLEL $EXTRA_MAKEFLAGS install-exec install-data
- # install jemalloc header files
- cd $MEMKIND_BDIR/jemalloc/obj && make -j$PARALLEL $EXTRA_MAKEFLAGS install_include
-
- unset JE_PREFIX
-
- popd
-}
-
build_boost() {
local BUILD_TYPE=$1
pushd $BOOST_SOURCE
diff --git a/thirdparty/build-thirdparty.sh b/thirdparty/build-thirdparty.sh
index 81d36d6..7ea5898 100755
--- a/thirdparty/build-thirdparty.sh
+++ b/thirdparty/build-thirdparty.sh
@@ -90,8 +90,6 @@ else
"libunwind") F_LIBUNWIND=1 ;;
"llvm") F_LLVM=1 ;;
"trace-viewer") F_TRACE_VIEWER=1 ;;
- "numactl") F_NUMACTL=1 ;;
- "memkind") F_MEMKIND=1 ;;
"boost") F_BOOST=1 ;;
"breakpad") F_BREAKPAD=1 ;;
"sparsehash") F_SPARSEHASH=1 ;;
@@ -312,14 +310,6 @@ if [ -n "$F_UNINSTRUMENTED" -o -n "$F_CURL" ]; then
build_curl
fi
-if [ -n "$OS_LINUX" ] && [ -n "$F_UNINSTRUMENTED" -o -n "$F_NUMACTL" ]; then
- build_numactl normal
-fi
-
-if [ -n "$OS_LINUX" ] && [ -n "$F_UNINSTRUMENTED" -o -n "$F_MEMKIND" ]; then
- build_memkind
-fi
-
restore_env
### Build C++ dependencies without instrumentation
@@ -471,14 +461,6 @@ if [ -n "$F_TSAN" -o -n "$F_CURL" ]; then
build_curl
fi
-if [ -n "$OS_LINUX" ] && [ -n "$F_TSAN" -o -n "$F_NUMACTL" ]; then
- build_numactl tsan
-fi
-
-if [ -n "$OS_LINUX" ] && [ -n "$F_TSAN" -o -n "$F_MEMKIND" ]; then
- build_memkind
-fi
-
restore_env
### Build C++ dependencies with TSAN instrumentation
diff --git a/thirdparty/download-thirdparty.sh b/thirdparty/download-thirdparty.sh
index ed62f31..5f56c48 100755
--- a/thirdparty/download-thirdparty.sh
+++ b/thirdparty/download-thirdparty.sh
@@ -349,21 +349,6 @@ fetch_and_patch \
$TRACE_VIEWER_SOURCE \
$TRACE_VIEWER_PATCHLEVEL
-NUMACTL_PATCHLEVEL=0
-fetch_and_patch \
- numactl-${NUMACTL_VERSION}.tar.gz \
- $NUMACTL_SOURCE \
- $NUMACTL_PATCHLEVEL
-
-MEMKIND_PATCHLEVEL=3
-fetch_and_patch \
- memkind-${MEMKIND_VERSION}.tar.gz \
- $MEMKIND_SOURCE \
- $MEMKIND_PATCHLEVEL \
- "patch -p1 < $TP_DIR/patches/memkind-fix-jemalloc-build-with-old-autoconf.patch" \
- "patch -p1 < $TP_DIR/patches/memkind-fix-build-with-old-autoconf.patch" \
- "patch -p1 < $TP_DIR/patches/memkind-fix-build-with-old-glibc.patch"
-
BOOST_PATCHLEVEL=1
fetch_and_patch \
boost_${BOOST_VERSION}.tar.gz \
diff --git a/thirdparty/patches/memkind-fix-build-with-old-autoconf.patch b/thirdparty/patches/memkind-fix-build-with-old-autoconf.patch
deleted file mode 100644
index 0f99b98..0000000
--- a/thirdparty/patches/memkind-fix-build-with-old-autoconf.patch
+++ /dev/null
@@ -1,32 +0,0 @@
-commit c8dbc9f
-Author: Adar Dembo <ad...@cloudera.com>
-Date: Sun May 26 14:20:51 2019 -0700
-
- configure.ac: fixes for autoconf 2.63
-
-diff --git a/configure.ac b/configure.ac
-index 64c3200..ec5dbb1 100644
---- a/configure.ac
-+++ b/configure.ac
-@@ -25,10 +25,10 @@
- # -*- Autoconf -*-
- # Process this file with autoconf to produce a configure script.
-
--AC_PREREQ([2.64])
-+AC_PREREQ([2.63])
- AC_INIT([memkind],m4_esyscmd([tr -d '\n' < VERSION]))
-
--AC_CONFIG_MACRO_DIRS([m4])
-+AC_CONFIG_MACRO_DIR([m4])
- AC_CONFIG_HEADERS([config.h])
- AC_CONFIG_SRCDIR([memkind.spec.mk])
-
-@@ -36,7 +36,7 @@ AM_INIT_AUTOMAKE([-Wall -Werror foreign 1.11 silent-rules subdir-objects paralle
- AM_SILENT_RULES([yes])
-
- # Checks for programs and libraries.
--AM_PROG_AR
-+m4_ifdef([AM_PROG_AR], [AM_PROG_AR])
- AC_PROG_CXX
- AC_PROG_CC
- AC_OPENMP
diff --git a/thirdparty/patches/memkind-fix-build-with-old-glibc.patch b/thirdparty/patches/memkind-fix-build-with-old-glibc.patch
deleted file mode 100644
index 06dd525..0000000
--- a/thirdparty/patches/memkind-fix-build-with-old-glibc.patch
+++ /dev/null
@@ -1,24 +0,0 @@
-commit 2113b6b
-Author: Adar Dembo <ad...@cloudera.com>
-Date: Sun May 26 14:39:46 2019 -0700
-
- Makefile.am: fixes for building against older glibc
-
- In older versions of glibc (such as the version found on el6 machines),
- the clock_gettime function is in librt rather than in libc directly. Our
- bundled jemalloc depends on clock_gettime and may link against librt, so
- let's make sure the memkind library links against librt too.
-
-diff --git a/Makefile.am b/Makefile.am
-index f791ce7..de57d90 100644
---- a/Makefile.am
-+++ b/Makefile.am
-@@ -44,7 +44,7 @@ libmemkind_la_SOURCES = src/hbwmalloc.c \
-
-
- libmemkind_la_LIBADD = jemalloc/obj/lib/libjemalloc_pic.a
--libmemkind_la_LDFLAGS = -version-info 0:1:0 -ldl
-+libmemkind_la_LDFLAGS = -version-info 0:1:0 -ldl -lrt
- include_HEADERS = include/hbw_allocator.h \
- include/hbwmalloc.h \
- include/memkind.h \
diff --git a/thirdparty/patches/memkind-fix-jemalloc-build-with-old-autoconf.patch b/thirdparty/patches/memkind-fix-jemalloc-build-with-old-autoconf.patch
deleted file mode 100644
index fc6129f..0000000
--- a/thirdparty/patches/memkind-fix-jemalloc-build-with-old-autoconf.patch
+++ /dev/null
@@ -1,36 +0,0 @@
-commit 608c2eb
-Author: Adar Dembo <ad...@cloudera.com>
-Date: Sun May 26 14:00:50 2019 -0700
-
- jemalloc: unroll dlsym checking logic in configure.ac
-
- The nested calls here cause autoconf 2.63 to generate a broken script[1].
- The upstream jemalloc response has been to require autoconf 2.68, but that
- means we can't build jemalloc (and thus memkind) on el6.6. As a workaround,
- we can simply unroll these nested calls.
-
- 1. https://github.com/jemalloc/jemalloc/issues/912
-
-diff --git a/jemalloc/configure.ac b/jemalloc/configure.ac
-index 5bd5442..cbc1e5d 100644
---- a/jemalloc/configure.ac
-+++ b/jemalloc/configure.ac
-@@ -1454,10 +1454,14 @@ if test "x$abi" != "xpecoff" ; then
- have_pthread="1"
- dnl Check if we have dlsym support.
- have_dlsym="1"
-- AC_CHECK_HEADERS([dlfcn.h],
-- AC_CHECK_FUNC([dlsym], [],
-- [AC_CHECK_LIB([dl], [dlsym], [LIBS="$LIBS -ldl"], [have_dlsym="0"])]),
-- [have_dlsym="0"])
-+ AC_CHECK_HEADERS([dlfcn.h], , [have_dlsym="0"])
-+ check_dlsym_in_libdl="0"
-+ if test "x$have_dlsym" = "x1" ; then
-+ AC_CHECK_FUNC([dlsym], [], [check_dlsym_in_libdl="1"])
-+ fi
-+ if test "x$check_dlsym_in_libdl" = "x1" ; then
-+ AC_CHECK_LIB([dl], [dlsym], [LIBS="$LIBS -ldl"], [have_dlsym="0"])
-+ fi
- if test "x$have_dlsym" = "x1" ; then
- AC_DEFINE([JEMALLOC_HAVE_DLSYM], [ ])
- fi
diff --git a/thirdparty/vars.sh b/thirdparty/vars.sh
index 1b007f7..00e5f93 100644
--- a/thirdparty/vars.sh
+++ b/thirdparty/vars.sh
@@ -166,15 +166,6 @@ TRACE_VIEWER_VERSION=21d76f8350fea2da2aa25cb6fd512703497d0c11
TRACE_VIEWER_NAME=kudu-trace-viewer-$TRACE_VIEWER_VERSION
TRACE_VIEWER_SOURCE=$TP_SOURCE_DIR/$TRACE_VIEWER_NAME
-# numactl 2.0.12 is required to build memkind library.
-NUMACTL_VERSION=2.0.12
-NUMACTL_NAME=numactl-$NUMACTL_VERSION
-NUMACTL_SOURCE=$TP_SOURCE_DIR/$NUMACTL_NAME
-
-MEMKIND_VERSION=1.9.0
-MEMKIND_NAME=memkind-$MEMKIND_VERSION
-MEMKIND_SOURCE=$TP_SOURCE_DIR/$MEMKIND_NAME
-
BOOST_VERSION=1_61_0
BOOST_NAME=boost_$BOOST_VERSION
BOOST_SOURCE=$TP_SOURCE_DIR/$BOOST_NAME