You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@arrow.apache.org by Ying Zhou <yz...@gmail.com> on 2020/10/24 22:04:38 UTC

[C++] Arrow debug with ORC & unittest can not be built

Hi,

I’m using the master version of Arrow. In order to test my Arrow2ORC feature I got a new copy of Arrow and tried to make it with debug on. It turns out that one ORC dependency, libhdfspp_static.a, can not be found which caused linking of arrow-orc-adapter-test to be impossible.

Here is my command:

cmake -DARROW_WITH_UTF8PROC=OFF -DCMAKE_BUILD_TYPE=Debug -DARROW_BUILD_TESTS=ON -DARROW_ORC=ON -DARROW_PYTHON=ON -DORC_ROOT=/usr/local -DOPENSSL_ROOT_DIR=/usr/local/opt/openssl ../..

Here is the error message I got:

[ 93%] Linking CXX executable ../../../../debug/arrow-orc-adapter-test
Undefined symbols for architecture x86_64:
  "hdfs::FileSystem::New(hdfs::IoService*&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, hdfs::Options const&)", referenced from:
      orc::HdfsFileInputStream::HdfsFileInputStream(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >) in liborc.a(OrcHdfsFile.cc.o)
  "hdfs::ConfigParser::LoadDefaultResources()", referenced from:
      orc::HdfsFileInputStream::HdfsFileInputStream(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >) in liborc.a(OrcHdfsFile.cc.o)
  "hdfs::ConfigParser::ConfigParser()", referenced from:
      orc::HdfsFileInputStream::HdfsFileInputStream(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >) in liborc.a(OrcHdfsFile.cc.o)
  "hdfs::ConfigParser::~ConfigParser()", referenced from:
      orc::HdfsFileInputStream::HdfsFileInputStream(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >) in liborc.a(OrcHdfsFile.cc.o)
  "hdfs::URI::parse_from_string(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)", referenced from:
…
…
"hdfs::Status::ToString() const", referenced from:
      orc::HdfsFileInputStream::read(void*, unsigned long long, unsigned long long) in liborc.a(OrcHdfsFile.cc.o)
      orc::HdfsFileInputStream::HdfsFileInputStream(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >) in liborc.a(OrcHdfsFile.cc.o)
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make[2]: *** [debug/arrow-orc-adapter-test] Error 1
make[1]: *** [src/arrow/adapters/orc/CMakeFiles/arrow-orc-adapter-test.dir/all] Error 2
make: *** [all] Error 2

I would like to know whether it is impossible to have a debug build of Apache Arrow with ORC without editing some CMakeLists files at least on macOS Catalina. If that’s the case then we have a bug.

Thanks,
Ying

Re: [C++] Arrow debug with ORC & unittest can not be built

Posted by Wes McKinney <we...@gmail.com>.
The Arrow build system is configured to build Apache ORC without libhdfspp

https://github.com/apache/arrow/blob/master/cpp/cmake_modules/ThirdpartyToolchain.cmake#L2613

If you'd like to change this or make it configurable, some development
will be needed, so I would suggest opening a Jira issue about it.

Thanks
Wes

On Sat, Oct 24, 2020 at 5:04 PM Ying Zhou <yz...@gmail.com> wrote:
>
> Hi,
>
> I’m using the master version of Arrow. In order to test my Arrow2ORC feature I got a new copy of Arrow and tried to make it with debug on. It turns out that one ORC dependency, libhdfspp_static.a, can not be found which caused linking of arrow-orc-adapter-test to be impossible.
>
> Here is my command:
>
> cmake -DARROW_WITH_UTF8PROC=OFF -DCMAKE_BUILD_TYPE=Debug -DARROW_BUILD_TESTS=ON -DARROW_ORC=ON -DARROW_PYTHON=ON -DORC_ROOT=/usr/local -DOPENSSL_ROOT_DIR=/usr/local/opt/openssl ../..
>
> Here is the error message I got:
>
> [ 93%] Linking CXX executable ../../../../debug/arrow-orc-adapter-test
> Undefined symbols for architecture x86_64:
>   "hdfs::FileSystem::New(hdfs::IoService*&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, hdfs::Options const&)", referenced from:
>       orc::HdfsFileInputStream::HdfsFileInputStream(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >) in liborc.a(OrcHdfsFile.cc.o)
>   "hdfs::ConfigParser::LoadDefaultResources()", referenced from:
>       orc::HdfsFileInputStream::HdfsFileInputStream(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >) in liborc.a(OrcHdfsFile.cc.o)
>   "hdfs::ConfigParser::ConfigParser()", referenced from:
>       orc::HdfsFileInputStream::HdfsFileInputStream(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >) in liborc.a(OrcHdfsFile.cc.o)
>   "hdfs::ConfigParser::~ConfigParser()", referenced from:
>       orc::HdfsFileInputStream::HdfsFileInputStream(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >) in liborc.a(OrcHdfsFile.cc.o)
>   "hdfs::URI::parse_from_string(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)", referenced from:
> …
> …
> "hdfs::Status::ToString() const", referenced from:
>       orc::HdfsFileInputStream::read(void*, unsigned long long, unsigned long long) in liborc.a(OrcHdfsFile.cc.o)
>       orc::HdfsFileInputStream::HdfsFileInputStream(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >) in liborc.a(OrcHdfsFile.cc.o)
> ld: symbol(s) not found for architecture x86_64
> clang: error: linker command failed with exit code 1 (use -v to see invocation)
> make[2]: *** [debug/arrow-orc-adapter-test] Error 1
> make[1]: *** [src/arrow/adapters/orc/CMakeFiles/arrow-orc-adapter-test.dir/all] Error 2
> make: *** [all] Error 2
>
> I would like to know whether it is impossible to have a debug build of Apache Arrow with ORC without editing some CMakeLists files at least on macOS Catalina. If that’s the case then we have a bug.
>
> Thanks,
> Ying