[jira] [Commented] (ARROW-7154) [C++] Build error when building tests but not with snappy

["Joris Van den Bossche (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/ARROW-7154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16973461#comment-16973461 ] 

Joris Van den Bossche commented on ARROW-7154:
----------------------------------------------

Creating a new conda env from scratch (which now has boost 1.70 instead of 1.68 in my old env, not sure if that is relevant), and then the problem also went away. So it might be OK to close this issue.

> [C++] Build error when building tests but not with snappy
> ---------------------------------------------------------
>
>                 Key: ARROW-7154
>                 URL: https://issues.apache.org/jira/browse/ARROW-7154
>             Project: Apache Arrow
>          Issue Type: Bug
>          Components: C++
>            Reporter: Joris Van den Bossche
>            Priority: Major
>
> Since the docker-compose PR landed, I am having build errors like:
> {code:java}
> [361/376] Linking CXX executable debug/arrow-python-test
> FAILED: debug/arrow-python-test
> : && /home/joris/miniconda3/envs/arrow-dev/bin/ccache /home/joris/miniconda3/envs/arrow-dev/bin/x86_64-conda_cos6-linux-gnu-c++  -Wno-noexcept-type -fvisibility-inlines-hidden -std=c++17 -fmessage-length=0 -march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-strong -fno-plt -O2 -ffunction-sections -pipe -fdiagnostics-color=always -ggdb -O0  -Wall -Wno-conversion -Wno-sign-conversion -Wno-unused-variable -Werror -msse4.2  -g  -Wl,-O2 -Wl,--sort-common -Wl,--as-needed -Wl,-z,relro -Wl,-z,now -Wl,--disable-new-dtags -Wl,--gc-sections   -rdynamic src/arrow/python/CMakeFiles/arrow-python-test.dir/python_test.cc.o  -o debug/arrow-python-test  -Wl,-rpath,/home/joris/scipy/repos/arrow/cpp/build/debug:/home/joris/miniconda3/envs/arrow-dev/lib debug/libarrow_python_test_main.a debug/libarrow_python.so.100.0.0 debug/libarrow_testing.so.100.0.0 debug/libarrow.so.100.0.0 /home/joris/miniconda3/envs/arrow-dev/lib/libpython3.7m.so -lpthread -lpthread -ldl  -lutil -lrt -ldl /home/joris/miniconda3/envs/arrow-dev/lib/libdouble-conversion.a /home/joris/miniconda3/envs/arrow-dev/lib/libglog.so jemalloc_ep-prefix/src/jemalloc_ep/dist//lib/libjemalloc_pic.a -lrt /home/joris/miniconda3/envs/arrow-dev/lib/libgtest.so -pthread && :
> /home/joris/miniconda3/envs/arrow-dev/bin/../lib/gcc/x86_64-conda_cos6-linux-gnu/7.3.0/../../../../x86_64-conda_cos6-linux-gnu/bin/ld: warning: libboost_filesystem.so.1.68.0, needed by debug/libarrow.so.100.0.0, not found (try using -rpath or -rpath-link)
> /home/joris/miniconda3/envs/arrow-dev/bin/../lib/gcc/x86_64-conda_cos6-linux-gnu/7.3.0/../../../../x86_64-conda_cos6-linux-gnu/bin/ld: warning: libboost_system.so.1.68.0, needed by debug/libarrow.so.100.0.0, not found (try using -rpath or -rpath-link)
> /home/joris/miniconda3/envs/arrow-dev/bin/../lib/gcc/x86_64-conda_cos6-linux-gnu/7.3.0/../../../../x86_64-conda_cos6-linux-gnu/bin/ld: debug/libarrow.so.100.0.0: undefined reference to `boost::system::detail::generic_category_ncx()'
> /home/joris/miniconda3/envs/arrow-dev/bin/../lib/gcc/x86_64-conda_cos6-linux-gnu/7.3.0/../../../../x86_64-conda_cos6-linux-gnu/bin/ld: debug/libarrow.so.100.0.0: undefined reference to `boost::filesystem::path::operator/=(boost::filesystem::path const&)'
> collect2: error: ld returned 1 exit status
> {code}
> which contains warnings like "warning: libboost_filesystem.so.1.68.0, needed by debug/libarrow.so.100.0.0, not found" (although this is certainly present).
> The error is triggered by having {{-DARROW_BUILD_TESTS=ON}}. If that is set to OFF, it works fine.
> It also seems to be related to this specific change in the docker compose PR:
> {code:java}
> diff --git a/cpp/CMakeLists.txt b/cpp/CMakeLists.txt
> index c80ac3310..3b3c9eb8f 100644
> --- a/cpp/CMakeLists.txt
> +++ b/cpp/CMakeLists.txt
> @@ -266,6 +266,15 @@ endif(UNIX)
>  # Set up various options
>  #
> -if(ARROW_BUILD_TESTS OR ARROW_BUILD_BENCHMARKS)
> -  # Currently the compression tests require at least these libraries; bz2 and
> -  # zstd are optional. See ARROW-3984
> -  set(ARROW_WITH_BROTLI ON)
> -  set(ARROW_WITH_LZ4 ON)
> -  set(ARROW_WITH_SNAPPY ON)
> -  set(ARROW_WITH_ZLIB ON)
> -endif()
> -
>  if(ARROW_BUILD_TESTS OR ARROW_BUILD_INTEGRATION)
>    set(ARROW_JSON ON)
>  endif()
> {code}
> If I add that back, the build works.
> With only `set(ARROW_WITH_BROTLI ON)`, it still fails
>  With only `set(ARROW_WITH_LZ4 ON)`, it also fails but with an error about liblz4 instead of libboost (but also liblz4 is actually present)
>  With only `set(ARROW_WITH_SNAPPY ON)`, it works
>  With only `set(ARROW_WITH_ZLIB ON)`, it also fails but with an error about libz.so.1 not found
> With both `set(ARROW_WITH_SNAPPY ON)` and `set(ARROW_WITH_ZLIB ON)`, it also works.  So it seems that the absence of snappy causes others to fail.
> In the recommended build settings in the development docs ([https://github.com/apache/arrow/blob/master/docs/source/developers/python.rst#build-and-test),] the compression libraries are enabled. But I was still building without them (stemming from the time they were enabled by default). So I was using:
> {code}
> cmake -DCMAKE_INSTALL_PREFIX=$ARROW_HOME -GNinja \
>  -DCMAKE_INSTALL_LIBDIR=lib \
>  -DARROW_PARQUET=ON \
>  -DARROW_PYTHON=ON \
>  -DARROW_BUILD_TESTS=ON \
>  ..
> {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)