You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@arrow.apache.org by "Wes McKinney (JIRA)" <ji...@apache.org> on 2017/08/16 23:59:00 UTC
[jira] [Comment Edited] (ARROW-1368) libarrow.a is not linked
against boost libraries when compiled with -DARROW_BOOST_USE_SHARED=off
[ https://issues.apache.org/jira/browse/ARROW-1368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16129644#comment-16129644 ]
Wes McKinney edited comment on ARROW-1368 at 8/16/17 11:58 PM:
---------------------------------------------------------------
This is as expected with static libraries; it is simply the result of calling {{ar}} on the object code that is part of the library, and does not include transitive dependencies. The easiest thing is to include the transitive dependencies in your downstream build system if you want to statically link.
There may be some way to create a "super archive" containing the Boost static libraries, but I am not sure how to do that. We were discussing it not long ago in the Parquet C++ library (because moving the compression libraries to Arrow introduced a transitive dependency for static linkers of libparquet.a)
was (Author: wesmckinn):
This is as expected with shared libraries; it is simply the result of calling {{ar}} on the object code that is part of the library, and does not include transitive dependencies. The easiest thing is to include the transitive dependencies in your downstream build system if you want to statically link.
There may be some way to create a "super archive" containing the Boost static libraries, but I am not sure how to do that. We were discussing it not long ago in the Parquet C++ library (because moving the compression libraries to Arrow introduced a transitive dependency for static linkers of libparquet.a)
> libarrow.a is not linked against boost libraries when compiled with -DARROW_BOOST_USE_SHARED=off
> ------------------------------------------------------------------------------------------------
>
> Key: ARROW-1368
> URL: https://issues.apache.org/jira/browse/ARROW-1368
> Project: Apache Arrow
> Issue Type: Bug
> Components: C++
> Environment: Ubuntu 16.04
> Reporter: Robert Nishihara
>
> When I build arrow with {{-DARROW_BOOST_USE_SHARED=off}} and then inspect {{libarrow.a}} with {{nm -g libarrow.a}}, some boost symbols are undefined.
> The problem can be reproduced on Ubuntu 16.04 as follows.
> First compile boost with -fPIC.
> {code}
> cd ~
> wget --no-check-certificate http://downloads.sourceforge.net/project/boost/boost/1.60.0/boost_1_60_0.tar.gz -O ~/boost_1_60_0.tar.gz
> tar xf boost_1_60_0.tar.gz
> cd boost_1_60_0/
> ./bootstrap.sh
> ./bjam cxxflags=-fPIC cflags=-fPIC --prefix=../boost --with-filesystem --with-date_time --with-system --with-regex install
> cd ..
> {code}
> Then compile Arrow.
> {code}
> cd ~
> git clone https://github.com/apache/arrow
> mkdir -p ~/arrow/cpp/build
> cd ~/arrow/cpp/build
> BOOST_ROOT=~/boost \
> cmake -DCMAKE_BUILD_TYPE=Release \
> -DCMAKE_C_FLAGS="-g -O3" \
> -DCMAKE_CXX_FLAGS="-g -O3" \
> -DARROW_BUILD_TESTS=off \
> -DARROW_HDFS=on \
> -DARROW_BOOST_USE_SHARED=off \
> -DARROW_PYTHON=on \
> -DARROW_PLASMA=on \
> -DPLASMA_PYTHON=on \
> -DARROW_JEMALLOC=off \
> -DARROW_WITH_BROTLI=off \
> -DARROW_WITH_LZ4=off \
> -DARROW_WITH_SNAPPY=off \
> -DARROW_WITH_ZLIB=off \
> -DARROW_WITH_ZSTD=off \
> ..
> make VERBOSE=1 -j8
> {code}
> The cmake command finds the recently compiled boost and prints the following.
> {code}
> -- [ /usr/share/cmake-3.5/Modules/FindBoost.cmake:1516 ] Boost_FOUND = 1
> -- Boost version: 1.60.0
> -- Found the following Boost libraries:
> -- system
> -- filesystem
> -- Boost include dir: /home/ubuntu/boost/include
> -- Boost libraries: /home/ubuntu/boost/lib/libboost_system.a/home/ubuntu/boost/lib/libboost_filesystem.a
> Added static library dependency boost_system: /home/ubuntu/boost/lib/libboost_system.a
> Added static library dependency boost_filesystem: /home/ubuntu/boost/lib/libboost_filesystem.a
> {code}
> Compilation does not appear to link {{libarrow.a}} against the boost libraries (though {{libarrow.so}} is handled properly).
> For {{libarrow.a}} (not linked against boost)
> {code}
> /usr/bin/ar qc release/libarrow.a CMakeFiles/arrow_objlib.dir/src/arrow/array.cc.o CMakeFiles/arrow_objlib.dir/src/arrow/buffer.cc.o CMakeFiles/arrow_objlib.dir/src/arrow/builder.cc.o CMakeFiles/arrow_objlib.dir/src/arrow/compare.cc.o CMakeFiles/arrow_objlib.dir/src/arrow/memory_pool.cc.o CMakeFiles/arrow_objlib.dir/src/arrow/pretty_print.cc.o CMakeFiles/arrow_objlib.dir/src/arrow/status.cc.o CMakeFiles/arrow_objlib.dir/src/arrow/table.cc.o CMakeFiles/arrow_objlib.dir/src/arrow/tensor.cc.o CMakeFiles/arrow_objlib.dir/src/arrow/type.cc.o CMakeFiles/arrow_objlib.dir/src/arrow/visitor.cc.o CMakeFiles/arrow_objlib.dir/src/arrow/io/file.cc.o CMakeFiles/arrow_objlib.dir/src/arrow/io/interfaces.cc.o CMakeFiles/arrow_objlib.dir/src/arrow/io/memory.cc.o CMakeFiles/arrow_objlib.dir/src/arrow/util/bit-util.cc.o CMakeFiles/arrow_objlib.dir/src/arrow/util/compression.cc.o CMakeFiles/arrow_objlib.dir/src/arrow/util/cpu-info.cc.o CMakeFiles/arrow_objlib.dir/src/arrow/util/decimal.cc.o CMakeFiles/arrow_objlib.dir/src/arrow/util/key_value_metadata.cc.o CMakeFiles/arrow_objlib.dir/src/arrow/io/hdfs.cc.o CMakeFiles/arrow_objlib.dir/src/arrow/io/hdfs-internal.cc.o CMakeFiles/arrow_objlib.dir/src/arrow/ipc/feather.cc.o CMakeFiles/arrow_objlib.dir/src/arrow/ipc/json.cc.o CMakeFiles/arrow_objlib.dir/src/arrow/ipc/json-internal.cc.o CMakeFiles/arrow_objlib.dir/src/arrow/ipc/metadata.cc.o CMakeFiles/arrow_objlib.dir/src/arrow/ipc/reader.cc.o CMakeFiles/arrow_objlib.dir/src/arrow/ipc/writer.cc.o
> {code}
> For {{libarrow.so}} (linked against boost)
> {code}
> /usr/bin/c++ -fPIC -g -O3 -O3 -DNDEBUG -Wall -std=c++11 -msse3 -O3 -DNDEBUG -Wl,--version-script=/home/ubuntu/arrow/cpp/src/arrow/symbols.map -shared -Wl,-soname,libarrow.so.0 -o release/libarrow.so.0.0.0 CMakeFiles/arrow_objlib.dir/src/arrow/array.cc.o CMakeFiles/arrow_objlib.dir/src/arrow/buffer.cc.o CMakeFiles/arrow_objlib.dir/src/arrow/builder.cc.o CMakeFiles/arrow_objlib.dir/src/arrow/compare.cc.o CMakeFiles/arrow_objlib.dir/src/arrow/memory_pool.cc.o CMakeFiles/arrow_objlib.dir/src/arrow/pretty_print.cc.o CMakeFiles/arrow_objlib.dir/src/arrow/status.cc.o CMakeFiles/arrow_objlib.dir/src/arrow/table.cc.o CMakeFiles/arrow_objlib.dir/src/arrow/tensor.cc.o CMakeFiles/arrow_objlib.dir/src/arrow/type.cc.o CMakeFiles/arrow_objlib.dir/src/arrow/visitor.cc.o CMakeFiles/arrow_objlib.dir/src/arrow/io/file.cc.o CMakeFiles/arrow_objlib.dir/src/arrow/io/interfaces.cc.o CMakeFiles/arrow_objlib.dir/src/arrow/io/memory.cc.o CMakeFiles/arrow_objlib.dir/src/arrow/util/bit-util.cc.o CMakeFiles/arrow_objlib.dir/src/arrow/util/compression.cc.o CMakeFiles/arrow_objlib.dir/src/arrow/util/cpu-info.cc.o CMakeFiles/arrow_objlib.dir/src/arrow/util/decimal.cc.o CMakeFiles/arrow_objlib.dir/src/arrow/util/key_value_metadata.cc.o CMakeFiles/arrow_objlib.dir/src/arrow/io/hdfs.cc.o CMakeFiles/arrow_objlib.dir/src/arrow/io/hdfs-internal.cc.o CMakeFiles/arrow_objlib.dir/src/arrow/ipc/feather.cc.o CMakeFiles/arrow_objlib.dir/src/arrow/ipc/json.cc.o CMakeFiles/arrow_objlib.dir/src/arrow/ipc/json-internal.cc.o CMakeFiles/arrow_objlib.dir/src/arrow/ipc/metadata.cc.o CMakeFiles/arrow_objlib.dir/src/arrow/ipc/reader.cc.o CMakeFiles/arrow_objlib.dir/src/arrow/ipc/writer.cc.o -ldl -lpthread /home/ubuntu/boost/lib/libboost_system.a /home/ubuntu/boost/lib/libboost_filesystem.a
> {code}
> Then {{nm -g release/libarrow.a}} contains the following line (for example).
> {code}
> U _ZN5boost6system15system_categoryEv
> {code}
> The file {{arrow/cpp/CMakeLists.txt}} contains the block
> {code}
> ADD_ARROW_LIB(arrow
> SOURCES ${ARROW_ALL_SRCS}
> DEPENDENCIES arrow_dependencies
> SHARED_LINK_FLAGS ${ARROW_SHARED_LINK_FLAGS}
> SHARED_LINK_LIBS ${ARROW_LINK_LIBS}
> SHARED_PRIVATE_LINK_LIBS ${ARROW_SHARED_PRIVATE_LINK_LIBS}
> STATIC_LINK_LIBS ${ARROW_STATIC_LINK_LIBS}
> STATIC_PRIVATE_LINK_LIBS ${ARROW_STATIC_PRIVATE_LINK_LIBS}
> )
> {code}
> Printing {{ARROW_STATIC_PRIVATE_LINK_LIBS}} shows that the variable equals {{boost_system_static;boost_filesystem_static}}, which seems right.
> Let me know if you have any thoughts about this.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)