[jira] [Resolved] (ARROW-4935) [C++] Errors from jemalloc when building pyarrow from source on OSX and Debian

["Uwe L. Korn (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/ARROW-4935?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Uwe L. Korn resolved ARROW-4935.
--------------------------------
    Resolution: Fixed

> [C++] Errors from jemalloc when building pyarrow from source on OSX and Debian
> ------------------------------------------------------------------------------
>
>                 Key: ARROW-4935
>                 URL: https://issues.apache.org/jira/browse/ARROW-4935
>             Project: Apache Arrow
>          Issue Type: Bug
>          Components: C++, Python
>    Affects Versions: 0.12.1
>         Environment: OSX, Debian, Python==3.6.7
>            Reporter: Gregory Hayes
>            Priority: Critical
>              Labels: build, newbie
>
> My attempts to build pyarrow from source are failing. I've set up the conda environment using the instructions provided in the Develop instructions, and have tried this on both Debian and OSX. When I run CMAKE in debug mode on OSX, the output is:
> {code:java}
> -- Building using CMake version: 3.14.0
> -- Arrow version: 0.13.0 (full: '0.13.0-SNAPSHOT')
> -- clang-tidy not found
> -- clang-format not found
> -- infer found at /usr/local/bin/infer
> -- Using ccache: /usr/local/bin/ccache
> -- Found cpplint executable at /Users/Greg/documents/repos/arrow/cpp/build-support/cpplint.py
> -- Compiler command: env LANG=C /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++ -v
> -- Compiler version: Apple LLVM version 10.0.0 (clang-1000.11.45.5)
> Target: x86_64-apple-darwin18.2.0
> Thread model: posix
> InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin
> -- Compiler id: AppleClang
> Selected compiler clang 4.1.0svn
> -- Arrow build warning level: CHECKIN
> Configured for DEBUG build (set with cmake -DCMAKE_BUILD_TYPE={release,debug,...})
> -- Build Type: DEBUG
> -- BOOST_VERSION: 1.67.0
> -- BROTLI_VERSION: v0.6.0
> -- CARES_VERSION: 1.15.0
> -- DOUBLE_CONVERSION_VERSION: v3.1.1
> -- FLATBUFFERS_VERSION: v1.10.0
> -- GBENCHMARK_VERSION: v1.4.1
> -- GFLAGS_VERSION: v2.2.0
> -- GLOG_VERSION: v0.3.5
> -- GRPC_VERSION: v1.18.0
> -- GTEST_VERSION: 1.8.1
> -- JEMALLOC_VERSION: 17c897976c60b0e6e4f4a365c751027244dada7a
> -- LZ4_VERSION: v1.8.3
> -- ORC_VERSION: 1.5.4
> -- PROTOBUF_VERSION: v3.6.1
> -- RAPIDJSON_VERSION: v1.1.0
> -- RE2_VERSION: 2018-10-01
> -- SNAPPY_VERSION: 1.1.3
> -- THRIFT_VERSION: 0.11.0
> -- ZLIB_VERSION: 1.2.8
> -- ZSTD_VERSION: v1.3.7
> -- Boost version: 1.68.0
> -- Found the following Boost libraries:
> --   regex
> --   system
> --   filesystem
> -- Boost include dir: /Users/Greg/anaconda3/envs/pyarrow-dev/include
> -- Boost libraries: /Users/Greg/anaconda3/envs/pyarrow-dev/lib/libboost_regex.dylib/Users/Greg/anaconda3/envs/pyarrow-dev/lib/libboost_system.dylib/Users/Greg/anaconda3/envs/pyarrow-dev/lib/libboost_filesystem.dylib
> Added shared library dependency boost_system_shared: /Users/Greg/anaconda3/envs/pyarrow-dev/lib/libboost_system.dylib
> Added shared library dependency boost_filesystem_shared: /Users/Greg/anaconda3/envs/pyarrow-dev/lib/libboost_filesystem.dylib
> Added shared library dependency boost_regex_shared: /Users/Greg/anaconda3/envs/pyarrow-dev/lib/libboost_regex.dylib
> Added static library dependency double-conversion_static: /Users/Greg/anaconda3/envs/pyarrow-dev/lib/libdouble-conversion.a
> -- double-conversion include dir: /Users/Greg/anaconda3/envs/pyarrow-dev/include
> -- double-conversion static library: /Users/Greg/anaconda3/envs/pyarrow-dev/lib/libdouble-conversion.a
> -- GFLAGS_HOME: /Users/Greg/anaconda3/envs/pyarrow-dev
> -- GFlags include dir: /Users/Greg/anaconda3/envs/pyarrow-dev/include
> -- GFlags static library: /Users/Greg/anaconda3/envs/pyarrow-dev/lib/libgflags.a
> Added static library dependency gflags_static: /Users/Greg/anaconda3/envs/pyarrow-dev/lib/libgflags.a
> -- RapidJSON include dir: /Users/Greg/anaconda3/envs/pyarrow-dev/include
> -- Found the Flatbuffers library: /Users/Greg/anaconda3/envs/pyarrow-dev/lib/libflatbuffers.a
> -- Flatbuffers include dir: /Users/Greg/anaconda3/envs/pyarrow-dev/include
> -- Flatbuffers compiler: /Users/Greg/anaconda3/envs/pyarrow-dev/bin/flatc
> Added static library dependency jemalloc_static: /Users/Greg/documents/repos/arrow/cpp/build/jemalloc_ep-prefix/src/jemalloc_ep/dist//lib/libjemalloc_pic.a
> Added shared library dependency jemalloc_shared: /Users/Greg/documents/repos/arrow/cpp/build/jemalloc_ep-prefix/src/jemalloc_ep/dist//lib/libjemalloc.dylib
> -- Found hdfs.h at: /Users/Greg/documents/repos/arrow/cpp/thirdparty/hadoop/include/hdfs.h
> -- Found the ZLIB shared library: /Users/Greg/anaconda3/envs/pyarrow-dev/lib/libz.dylib
> Added shared library dependency zlib_shared: /Users/Greg/anaconda3/envs/pyarrow-dev/lib/libz.dylib
> -- SNAPPY_HOME: /Users/Greg/anaconda3/envs/pyarrow-dev
> -- Found the Snappy library: /Users/Greg/anaconda3/envs/pyarrow-dev/lib/libsnappy.dylib
> Added static library dependency snappy_static: /Users/Greg/anaconda3/envs/pyarrow-dev/lib/libsnappy.a
> -- Found the Brotli library: /Users/Greg/anaconda3/envs/pyarrow-dev/lib/libbrotlienc-static.a;/Users/Greg/anaconda3/envs/pyarrow-dev/lib/libbrotlidec-static.a;/Users/Greg/anaconda3/envs/pyarrow-dev/lib/libbrotlicommon-static.a
> Added static library dependency brotli_enc_static: /Users/Greg/anaconda3/envs/pyarrow-dev/lib/libbrotlienc-static.a
> Added static library dependency brotli_dec_static: /Users/Greg/anaconda3/envs/pyarrow-dev/lib/libbrotlidec-static.a
> Added static library dependency brotli_common_static: /Users/Greg/anaconda3/envs/pyarrow-dev/lib/libbrotlicommon-static.a
> Added static library dependency lz4_static: /Users/Greg/anaconda3/envs/pyarrow-dev/lib/liblz4.a
> Added static library dependency zstd_static: /Users/Greg/anaconda3/envs/pyarrow-dev/lib/libzstd.a
> -- THRIFT_HOME: /Users/Greg/anaconda3/envs/pyarrow-dev
> -- Thrift version: 0.12.0
> -- Thrift include dir: /Users/Greg/anaconda3/envs/pyarrow-dev/include
> -- Thrift static library: /Users/Greg/anaconda3/envs/pyarrow-dev/lib/libthrift.a
> -- Thrift compiler: /Users/Greg/anaconda3/envs/pyarrow-dev/bin/thrift
> -- GLOG_HOME: /Users/Greg/anaconda3/envs/pyarrow-dev
> -- Found the GLog library: /Users/Greg/anaconda3/envs/pyarrow-dev/lib/libglog.dylib
> -- Glog include dir: /Users/Greg/anaconda3/envs/pyarrow-dev/include
> -- Glog static library: /Users/Greg/anaconda3/envs/pyarrow-dev/lib/libglog.a
> Added static library dependency glog_static: /Users/Greg/anaconda3/envs/pyarrow-dev/lib/libglog.a
> -- CMAKE_C_FLAGS:  -Qunused-arguments -ggdb -O0   -Weverything -Wno-c++98-compat -Wno-c++98-compat-pedantic -Wno-deprecated -Wno-weak-vtables -Wno-padded -Wno-comma -Wno-unused-macros -Wno-unused-parameter -Wno-unused-template -Wno-undef -Wno-shadow -Wno-switch-enum -Wno-exit-time-destructors -Wno-global-constructors -Wno-weak-template-vtables -Wno-undefined-reinterpret-cast -Wno-implicit-fallthrough -Wno-unreachable-code-return -Wno-float-equal -Wno-missing-prototypes -Wno-documentation-unknown-command -Wno-old-style-cast -Wno-covered-switch-default -Wno-cast-align -Wno-vla-extension -Wno-shift-sign-overflow -Wno-used-but-marked-unused -Wno-missing-variable-declarations -Wno-gnu-zero-variadic-macro-arguments -Wconversion -Wno-sign-conversion -Wno-disabled-macro-expansion -Wno-format-nonliteral -Wno-missing-noreturn -Wno-gnu-folding-constant -Wno-reserved-id-macro -Wno-range-loop-analysis -Wno-double-promotion -Wno-undefined-func-template -Wno-zero-as-null-pointer-constant -Wno-unknown-warning-option -Werror -Wno-unknown-warning-option -msse4.2 -maltivec -stdlib=libc++
> -- CMAKE_CXX_FLAGS:   -Qunused-arguments -ggdb -O0  -Weverything -Wno-c++98-compat -Wno-c++98-compat-pedantic -Wno-deprecated -Wno-weak-vtables -Wno-padded -Wno-comma -Wno-unused-macros -Wno-unused-parameter -Wno-unused-template -Wno-undef -Wno-shadow -Wno-switch-enum -Wno-exit-time-destructors -Wno-global-constructors -Wno-weak-template-vtables -Wno-undefined-reinterpret-cast -Wno-implicit-fallthrough -Wno-unreachable-code-return -Wno-float-equal -Wno-missing-prototypes -Wno-documentation-unknown-command -Wno-old-style-cast -Wno-covered-switch-default -Wno-cast-align -Wno-vla-extension -Wno-shift-sign-overflow -Wno-used-but-marked-unused -Wno-missing-variable-declarations -Wno-gnu-zero-variadic-macro-arguments -Wconversion -Wno-sign-conversion -Wno-disabled-macro-expansion -Wno-format-nonliteral -Wno-missing-noreturn -Wno-gnu-folding-constant -Wno-reserved-id-macro -Wno-range-loop-analysis -Wno-double-promotion -Wno-undefined-func-template -Wno-zero-as-null-pointer-constant -Wno-unknown-warning-option -Werror -Wno-unknown-warning-option -msse4.2 -maltivec -stdlib=libc++ 
> -- Configuring done
> CMake Warning (dev):
>   Policy CMP0068 is not set: RPATH settings on macOS do not affect
>   install_name.  Run "cmake --help-policy CMP0068" for policy details.  Use
>   the cmake_policy command to set the policy and suppress this warning.
>   For compatibility with older versions of CMake, the install_name fields for
>   the following targets are still affected by RPATH settings:
>    arrow_python_shared
>    arrow_shared
>    parquet_shared
>    plasma_shared
> This warning is for project developers.  Use -Wno-dev to suppress it.
> -- Generating done
> -- Build files have been written to: /Users/Greg/documents/repos/arrow/cpp/build{code}
> After I run make -j4, the output is:
> {code:java}
>   0%] Built target metadata_fbs
> [  1%] Performing configure step for 'jemalloc_ep'
> [  2%] Built target parquet-thrift-deps
> [ 12%] Built target arrow_python_objlib
> [ 19%] Built target plasma_objlib
> [ 19%] Built target arrow_python_static
> CMake Error at /Users/Greg/documents/repos/arrow/cpp/build/jemalloc_ep-prefix/src/jemalloc_ep-stamp/jemalloc_ep-configure-DEBUG.cmake:49 (message):
>   Command failed: 1
>    './autogen.sh' 'AR=/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ar' 'CC=/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/cc' '--prefix=/Users/Greg/documents/repos/arrow/cpp/build/jemalloc_ep-prefix/src/jemalloc_ep/dist/' '--with-jemalloc-prefix=je_arrow_' '--with-private-namespace=je_arrow_private_' '--disable-tls'
>   See also
>     /Users/Greg/documents/repos/arrow/cpp/build/jemalloc_ep-prefix/src/jemalloc_ep-stamp/jemalloc_ep-configure-*.log
> make[2]: *** [jemalloc_ep-prefix/src/jemalloc_ep-stamp/jemalloc_ep-configure] Error 1
> make[1]: *** [CMakeFiles/jemalloc_ep.dir/all] Error 2
> make: *** [all] Error 2
> {code}
> And the output of the jemalloc log file is:
> {code:java}
> configure: error: in `/Users/Greg/documents/repos/arrow/cpp/build/jemalloc_ep-prefix/src/jemalloc_ep':
> configure: error: cannot run C compiled programs.
> If you meant to cross compile, use `--host'.
> See `config.log' for more details
> autoconf
> ./configure --enable-autogen AR=/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ar CC=/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/cc --prefix=/Users/Greg/documents/repos/arrow/cpp/build/jemalloc_ep-prefix/src/jemalloc_ep/dist/ --with-jemalloc-prefix=je_arrow_ --with-private-namespace=je_arrow_private_ --disable-tls
> checking for xsltproc... /usr/bin/xsltproc
> checking for gcc... /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/cc
> checking whether the C compiler works... yes
> checking for C compiler default output file name... a.out
> checking for suffix of executables... 
> checking whether we are cross compiling... Error 0 in ./configure
> {code}
> Any help / guidance I can get is appreciated.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)