Building Hadoop 3.2.1 hadoop-hdfs-native-client fails

[Aki Tanaka <ta...@gmail.com>]

Hello all,

I'm trying to upgrade our Hadoop env from 3.1 to 3.2.1. However building
hadoop-hdfs-native-client failed due to undefined reference error. (Added
the build log below)
I think the error was returned because Protocol Buffer is old (2.5.0) and
it does not support c++11 features.

I tried to upgrade the protocol buffer version but got an another compile
error. Also, I found this JIRA ticket (
https://issues.apache.org/jira/browse/HADOOP-13363).
If I add "-D_GLIBCXX_USE_CXX11_ABI=0" to a build option for
hadoop-hdfs-native-client,
I was able to complete the build job, but I'm not sure whether this is the
right way.

I would like to know the best way to build Hadoop 3.2.1. Thank you.

Build Host Environment:
Protocol Buffer: 2.5.0 (installed this using 'yum install' command)
Cmake: 3.13.1
OS: AmazonLinux 2


[INFO] --- hadoop-maven-plugins:3.2.1:cmake-compile (cmake-compile) @
hadoop-hdfs-native-client ---

[INFO] Running cmake
/workspace/hadoop-rpm/build/hadoop/rpm/BUILD/hadoop-3.2.1-src/hadoop-hdfs-project/hadoop-hdfs-native-client/src
-DGENERATED_JAVAH=/workspace/hadoop-rpm/build/hadoop/rpm/BUILD/hadoop-3.2.1-src/hadoop-hdfs-project/hadoop-hdfs-native-client/target/native/javah
-DHADOOP_BUILD=1 -DJVM_ARCH_DATA_MODEL=64 -DREQUIRE_FUSE=false
-DREQUIRE_LIBWEBHDFS=false -DREQUIRE_VALGRIND=false -G Unix Makefiles
[INFO] with extra environment variables {}
[INFO] Running make -j 16 VERBOSE=1
[WARNING] /usr/bin/cmake3
-S/workspace/hadoop-rpm/build/hadoop/rpm/BUILD/hadoop-3.2.1-src/hadoop-hdfs-project/hadoop-hdfs-native-client/src
-B/workspace/hadoop-rpm/build/hadoop/rpm/BUILD/hadoop-3.2.1-src/hadoop-hdfs-project/hadoop-hdfs-native-client/target
--check-build-system CMakeFiles/Makefile.cmake 0

..

[WARNING] make[1]: Leaving directory
`/workspace/hadoop-rpm/build/hadoop/rpm/BUILD/hadoop-3.2.1-src/hadoop-hdfs-project/hadoop-hdfs-native-client/target'
[WARNING] CMakeFiles/protoc-gen-hrpc.dir/protoc_gen_hrpc.cc.o: In
function `StubGenerator::EmitMethod(google::protobuf::MethodDescriptor
const*, google::protobuf::io::Printer*) const':
[WARNING] /workspace/hadoop-rpm/build/hadoop/rpm/BUILD/hadoop-3.2.1-src/hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfspp/lib/proto/protoc_gen_hrpc.cc:83:
undefined reference to `google::protobuf::io::Printer::Print(char
const*, char const*, std::__cxx11::basic_string<char,
std::char_traits<char>, std::allocator<char> > const&, char const*,
std::__cxx11::basic_string<char, std::char_traits<char>,
std::allocator<char> > const&)'
[WARNING] CMakeFiles/protoc-gen-hrpc.dir/protoc_gen_hrpc.cc.o: In
function `StubGenerator::EmitService(google::protobuf::ServiceDescriptor
const*, google::protobuf::io::Printer*) const':
[WARNING] /workspace/hadoop-rpm/build/hadoop/rpm/BUILD/hadoop-3.2.1-src/hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfspp/lib/proto/protoc_gen_hrpc.cc:64:
undefined reference to `google::protobuf::io::Printer::Print(char
const*, char const*, std::__cxx11::basic_string<char,
std::char_traits<char>, std::allocator<char> > const&)'
[WARNING] collect2: error: ld returned 1 exit status
[WARNING] make[2]: *** [main/native/libhdfspp/lib/proto/protoc-gen-hrpc] Error 1
[WARNING] make[1]: ***
[main/native/libhdfspp/lib/proto/CMakeFiles/protoc-gen-hrpc.dir/all]
Error 2
[WARNING] make[1]: *** Waiting for unfinished jobs....
[WARNING] make: *** [all] Error 2


-- Aki TANAKA