You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@arrow.apache.org by "ASF GitHub Bot (Jira)" <ji...@apache.org> on 2021/06/15 17:58:00 UTC
[jira] [Updated] (ARROW-13085) Apache Arrow minimal cpp build
segfaults with pyarrow libs
[ https://issues.apache.org/jira/browse/ARROW-13085?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
ASF GitHub Bot updated ARROW-13085:
-----------------------------------
Labels: pull-request-available (was: )
> Apache Arrow minimal cpp build segfaults with pyarrow libs
> ----------------------------------------------------------
>
> Key: ARROW-13085
> URL: https://issues.apache.org/jira/browse/ARROW-13085
> Project: Apache Arrow
> Issue Type: Bug
> Components: C++, Python
> Affects Versions: 4.0.1
> Environment: See dockerfile: https://github.com/kratsg/awkward-arrow-cmake-pybind11/blob/0e196983b2452c4c3dbbde40984db7031397d0a7/Dockerfile
> Reporter: Giordon Stark
> Priority: Major
> Labels: pull-request-available
> Time Spent: 10m
> Remaining Estimate: 0h
>
> h6. _Refer to [https://github.com/kratsg/awkward-arrow-cmake-pybind11/issues/6#issuecomment-861592692] and code-base available here: [https://github.com/kratsg/awkward-arrow-cmake-pybind11/tree/0e196983b2452c4c3dbbde40984db7031397d0a7]_
> The minimal build example provided by arrow ([https://github.com/apache/arrow/blob/master/cpp/examples/minimal_build/example.cc] [)|https://github.com/apache/arrow/blob/master/cpp/examples/minimal_build/example.cc] works fine using the libs shipped with {{yum}} on CentOS7. Using the libs shipped by {{pip install pyarrow}} instead provides a segfault at runtime. Here is the gdb dump with debugging flags included
> {code:java}
> [root@dab7bb04d93f examples]# gdb ./../build/example_arrow
> GNU gdb (GDB) Red Hat Enterprise Linux 7.6.1-120.el7
> Copyright (C) 2013 Free Software Foundation, Inc.
> License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
> This is free software: you are free to change and redistribute it.
> There is NO WARRANTY, to the extent permitted by law. Type "show copying"
> and "show warranty" for details.
> This GDB was configured as "x86_64-redhat-linux-gnu".
> For bug reporting instructions, please see:
> <http://www.gnu.org/software/gdb/bugs/>...
> Reading symbols from /Users/kratsg/awkward-arrow-cmake-pybind11/build/example_arrow...done.
> (gdb) run
> Starting program: /Users/kratsg/awkward-arrow-cmake-pybind11/examples/./../build/example_arrow
> warning: Error disabling address space randomization: Operation not permitted
> [Thread debugging using libthread_db enabled]
> Using host libthread_db library "/lib64/libthread_db.so.1".
> [New Thread 0x7f9c58dff700 (LWP 1643)]
> * Reading CSV file 'test.csv' into table
> Program received signal SIGSEGV, Segmentation fault.
> __destroy<std::basic_string<char>*> (__last=<optimized out>, __first=0x3200000000) at /usr/include/c++/4.8.2/bits/stl_construct.h:103
> 103 std::_Destroy(std::__addressof(*__first));
> Missing separate debuginfos, use: debuginfo-install glibc-2.17-324.el7_9.x86_64 libgcc-4.8.5-44.el7.x86_64 libstdc++-4.8.5-44.el7.x86_64
> (gdb) bt
> #0 __destroy<std::basic_string<char>*> (__last=<optimized out>, __first=0x3200000000) at /usr/include/c++/4.8.2/bits/stl_construct.h:103
> #1 _Destroy<std::basic_string<char>*> (__last=<optimized out>, __first=<optimized out>) at /usr/include/c++/4.8.2/bits/stl_construct.h:126
> #2 _Destroy<std::basic_string<char>*, std::basic_string<char> > (__last=0x0, __first=<optimized out>) at /usr/include/c++/4.8.2/bits/stl_construct.h:151
> #3 ~vector (this=0x7ffff43602a8, __in_chrg=<optimized out>) at /usr/include/c++/4.8.2/bits/stl_vector.h:415
> #4 arrow::csv::ConvertOptions::~ConvertOptions (this=0x7ffff4360220, __in_chrg=<optimized out>)
> at /usr/local/venv/lib/python3.8/site-packages/pyarrow/include/arrow/csv/options.h:64
> #5 0x000000000040171a in (anonymous namespace)::RunMain (argv=<optimized out>, argc=<optimized out>, this=<optimized out>, this=<optimized out>)
> at /Users/kratsg/awkward-arrow-cmake-pybind11/src/example/arrow.cpp:39
> #6 0x000000000040135f in main (argc=<optimized out>, argv=<optimized out>) at /Users/kratsg/awkward-arrow-cmake-pybind11/src/example/arrow.cpp:66 {code}
> Below is the full CMake log + build log
> {code:java}
> # ./build.sh
> -- The CXX compiler identification is GNU 4.8.5
> -- Detecting CXX compiler ABI info
> -- Detecting CXX compiler ABI info - done
> -- Check for working CXX compiler: /usr/bin/c++ - skipped
> -- Detecting CXX compile features
> -- Detecting CXX compile features - done
> -- pybind11 v2.6.2
> -- Found PythonInterp: /usr/local/bin/python3.8 (found version "3.8.10")
> -- Found PythonLibs: /usr/local/lib/libpython3.8.so
> -- Performing Test HAS_FLTO
> -- Performing Test HAS_FLTO - Success
> -- PyArrow includes : /usr/local/venv/lib/python3.8/site-packages/pyarrow/include
> -- PyArrow libraries : arrow;arrow_python
> -- PyArrow library dirs : /usr/local/venv/lib/python3.8/site-packages/pyarrow
> -- Awkward includes : /usr/local/venv/lib/python3.8/site-packages/awkward/include
> -- Awkward libraries: /usr/local/venv/lib/python3.8/site-packages/awkward
> -- Libraries: /usr/local/venv/lib/python3.8/site-packages/awkward/libawkward-cpu-kernels.so /usr/local/venv/lib/python3.8/site-packages/awkward/libawkward.so /usr/lib64/libdl.so /usr/local/venv/lib/python3.8/site-packages/pyarrow/libarrow.so /usr/local/venv/lib/python3.8/site-packages/pyarrow/libarrow_python.so
> -- Configuring done
> -- Generating done
> -- Build files have been written to: /Users/kratsg/awkward-arrow-cmake-pybind11/build
> -- pybind11 v2.6.2
> -- PyArrow includes : /usr/local/venv/lib/python3.8/site-packages/pyarrow/include
> -- PyArrow libraries : arrow;arrow_python
> -- PyArrow library dirs : /usr/local/venv/lib/python3.8/site-packages/pyarrow
> -- Awkward includes : /usr/local/venv/lib/python3.8/site-packages/awkward/include
> -- Awkward libraries: /usr/local/venv/lib/python3.8/site-packages/awkward
> -- Libraries: /usr/local/venv/lib/python3.8/site-packages/awkward/libawkward-cpu-kernels.so /usr/local/venv/lib/python3.8/site-packages/awkward/libawkward.so /usr/lib64/libdl.so /usr/local/venv/lib/python3.8/site-packages/pyarrow/libarrow.so /usr/local/venv/lib/python3.8/site-packages/pyarrow/libarrow_python.so
> -- Configuring done
> -- Generating done
> -- Build files have been written to: /Users/kratsg/awkward-arrow-cmake-pybind11/build
> -- Cache values
> CMAKE_BUILD_TYPE:STRING=RelWithDebInfo
> CMAKE_INSTALL_PREFIX:PATH=/usr/local
> CPU-KERNELS:FILEPATH=/usr/local/venv/lib/python3.8/site-packages/awkward/libawkward-cpu-kernels.so
> LIBARROW:FILEPATH=/usr/local/venv/lib/python3.8/site-packages/pyarrow/libarrow.so
> LIBARROW_PYTHON:FILEPATH=/usr/local/venv/lib/python3.8/site-packages/pyarrow/libarrow_python.so
> LIBAWKWARD:FILEPATH=/usr/local/venv/lib/python3.8/site-packages/awkward/libawkward.so
> LIBDL:FILEPATH=/usr/lib64/libdl.so
> PYBIND11_FINDPYTHON:BOOL=OFF
> PYBIND11_INSTALL:BOOL=OFF
> PYBIND11_NOPYTHON:BOOL=OFF
> PYBIND11_PYTHON_VERSION:STRING=
> PYBIND11_TEST:BOOL=OFF
> ###################################################################
> ######################### BUILD LOG BELOW #########################
> ###################################################################
> /usr/local/bin/cmake -S/Users/kratsg/awkward-arrow-cmake-pybind11/src -B/Users/kratsg/awkward-arrow-cmake-pybind11/build --check-build-system CMakeFiles/Makefile.cmake 0
> /usr/local/bin/cmake -E cmake_progress_start /Users/kratsg/awkward-arrow-cmake-pybind11/build/CMakeFiles /Users/kratsg/awkward-arrow-cmake-pybind11/build//CMakeFiles/progress.marks
> /usr/bin/gmake -f CMakeFiles/Makefile2 all
> gmake[1]: Entering directory `/Users/kratsg/awkward-arrow-cmake-pybind11/build'
> /usr/bin/gmake -f CMakeFiles/babel.dir/build.make CMakeFiles/babel.dir/depend
> /usr/bin/gmake -f CMakeFiles/example_arrow.dir/build.make CMakeFiles/example_arrow.dir/depend
> /usr/bin/gmake -f CMakeFiles/example.dir/build.make CMakeFiles/example.dir/depend
> gmake[2]: Entering directory `/Users/kratsg/awkward-arrow-cmake-pybind11/build'
> cd /Users/kratsg/awkward-arrow-cmake-pybind11/build && /usr/local/bin/cmake -E cmake_depends "Unix Makefiles" /Users/kratsg/awkward-arrow-cmake-pybind11/src /Users/kratsg/awkward-arrow-cmake-pybind11/src /Users/kratsg/awkward-arrow-cmake-pybind11/build /Users/kratsg/awkward-arrow-cmake-pybind11/build /Users/kratsg/awkward-arrow-cmake-pybind11/build/CMakeFiles/babel.dir/DependInfo.cmake --color=
> gmake[2]: Entering directory `/Users/kratsg/awkward-arrow-cmake-pybind11/build'
> cd /Users/kratsg/awkward-arrow-cmake-pybind11/build && /usr/local/bin/cmake -E cmake_depends "Unix Makefiles" /Users/kratsg/awkward-arrow-cmake-pybind11/src /Users/kratsg/awkward-arrow-cmake-pybind11/src /Users/kratsg/awkward-arrow-cmake-pybind11/build /Users/kratsg/awkward-arrow-cmake-pybind11/build /Users/kratsg/awkward-arrow-cmake-pybind11/build/CMakeFiles/example_arrow.dir/DependInfo.cmake --color=
> gmake[2]: Entering directory `/Users/kratsg/awkward-arrow-cmake-pybind11/build'
> cd /Users/kratsg/awkward-arrow-cmake-pybind11/build && /usr/local/bin/cmake -E cmake_depends "Unix Makefiles" /Users/kratsg/awkward-arrow-cmake-pybind11/src /Users/kratsg/awkward-arrow-cmake-pybind11/src /Users/kratsg/awkward-arrow-cmake-pybind11/build /Users/kratsg/awkward-arrow-cmake-pybind11/build /Users/kratsg/awkward-arrow-cmake-pybind11/build/CMakeFiles/example.dir/DependInfo.cmake --color=
> gmake[2]: Leaving directory `/Users/kratsg/awkward-arrow-cmake-pybind11/build'
> /usr/bin/gmake -f CMakeFiles/example.dir/build.make CMakeFiles/example.dir/build
> gmake[2]: Entering directory `/Users/kratsg/awkward-arrow-cmake-pybind11/build'
> gmake[2]: Leaving directory `/Users/kratsg/awkward-arrow-cmake-pybind11/build'
> /usr/bin/gmake -f CMakeFiles/babel.dir/build.make CMakeFiles/babel.dir/build
> gmake[2]: Leaving directory `/Users/kratsg/awkward-arrow-cmake-pybind11/build'
> /usr/bin/gmake -f CMakeFiles/example_arrow.dir/build.make CMakeFiles/example_arrow.dir/build
> gmake[2]: Entering directory `/Users/kratsg/awkward-arrow-cmake-pybind11/build'
> gmake[2]: Entering directory `/Users/kratsg/awkward-arrow-cmake-pybind11/build'
> [ 14%] Building CXX object CMakeFiles/example.dir/example/python.cpp.o
> /usr/bin/c++ -Dexample_EXPORTS -isystem /Users/kratsg/awkward-arrow-cmake-pybind11/src/pybind11/include -isystem /usr/local/venv/lib/python3.8/site-packages/awkward/include -O2 -g -DNDEBUG -fPIC -fvisibility=hidden -std=gnu++11 -MD -MT CMakeFiles/example.dir/example/python.cpp.o -MF CMakeFiles/example.dir/example/python.cpp.o.d -o CMakeFiles/example.dir/example/python.cpp.o -c /Users/kratsg/awkward-arrow-cmake-pybind11/src/example/python.cpp
> [ 28%] Building CXX object CMakeFiles/babel.dir/babel/python.cpp.o
> /usr/bin/c++ -Dbabel_EXPORTS -isystem /Users/kratsg/awkward-arrow-cmake-pybind11/src/pybind11/include -isystem /usr/local/venv/lib/python3.8/site-packages/pyarrow/include -isystem /usr/local/venv/lib/python3.8/site-packages/awkward/include -O2 -g -DNDEBUG -fPIC -fvisibility=hidden -std=gnu++11 -MD -MT CMakeFiles/babel.dir/babel/python.cpp.o -MF CMakeFiles/babel.dir/babel/python.cpp.o.d -o CMakeFiles/babel.dir/babel/python.cpp.o -c /Users/kratsg/awkward-arrow-cmake-pybind11/src/babel/python.cpp
> [ 42%] Building CXX object CMakeFiles/example_arrow.dir/example/arrow.cpp.o
> /usr/bin/c++ -isystem /usr/local/venv/lib/python3.8/site-packages/pyarrow/include -O2 -g -DNDEBUG -std=gnu++11 -MD -MT CMakeFiles/example_arrow.dir/example/arrow.cpp.o -MF CMakeFiles/example_arrow.dir/example/arrow.cpp.o.d -o CMakeFiles/example_arrow.dir/example/arrow.cpp.o -c /Users/kratsg/awkward-arrow-cmake-pybind11/src/example/arrow.cpp
> [ 57%] Linking CXX executable example_arrow
> /usr/local/bin/cmake -E cmake_link_script CMakeFiles/example_arrow.dir/link.txt --verbose=1
> /usr/bin/c++ -O2 -g -DNDEBUG CMakeFiles/example_arrow.dir/example/arrow.cpp.o -o example_arrow -Wl,-rpath,/usr/local/venv/lib/python3.8/site-packages/pyarrow /usr/local/venv/lib/python3.8/site-packages/pyarrow/libarrow.so
> gmake[2]: Leaving directory `/Users/kratsg/awkward-arrow-cmake-pybind11/build'
> [ 57%] Built target example_arrow
> [ 71%] Building CXX object CMakeFiles/example.dir/example/minimal.cpp.o
> /usr/bin/c++ -Dexample_EXPORTS -isystem /Users/kratsg/awkward-arrow-cmake-pybind11/src/pybind11/include -isystem /usr/local/venv/lib/python3.8/site-packages/awkward/include -O2 -g -DNDEBUG -fPIC -fvisibility=hidden -std=gnu++11 -MD -MT CMakeFiles/example.dir/example/minimal.cpp.o -MF CMakeFiles/example.dir/example/minimal.cpp.o.d -o CMakeFiles/example.dir/example/minimal.cpp.o -c /Users/kratsg/awkward-arrow-cmake-pybind11/src/example/minimal.cpp
> [ 85%] Linking CXX shared module example.cpython-38-x86_64-linux-gnu.so
> /usr/local/bin/cmake -E cmake_link_script CMakeFiles/example.dir/link.txt --verbose=1
> /usr/bin/c++ -fPIC -O2 -g -DNDEBUG -shared -o example.cpython-38-x86_64-linux-gnu.so CMakeFiles/example.dir/example/python.cpp.o CMakeFiles/example.dir/example/minimal.cpp.o -Wl,-rpath,/usr/local/venv/lib/python3.8/site-packages/awkward /usr/local/venv/lib/python3.8/site-packages/awkward/libawkward-cpu-kernels.so /usr/local/venv/lib/python3.8/site-packages/awkward/libawkward.so /usr/lib64/libdl.so
> [100%] Linking CXX shared module babel.cpython-38-x86_64-linux-gnu.so
> /usr/local/bin/cmake -E cmake_link_script CMakeFiles/babel.dir/link.txt --verbose=1
> /usr/bin/c++ -fPIC -O2 -g -DNDEBUG -shared -o babel.cpython-38-x86_64-linux-gnu.so CMakeFiles/babel.dir/babel/python.cpp.o -Wl,-rpath,/usr/local/venv/lib/python3.8/site-packages/pyarrow:/usr/local/venv/lib/python3.8/site-packages/awkward /usr/local/venv/lib/python3.8/site-packages/pyarrow/libarrow.so /usr/local/venv/lib/python3.8/site-packages/pyarrow/libarrow_python.so /usr/local/venv/lib/python3.8/site-packages/awkward/libawkward-cpu-kernels.so /usr/local/venv/lib/python3.8/site-packages/awkward/libawkward.so /usr/lib64/libdl.so
> gmake[2]: Leaving directory `/Users/kratsg/awkward-arrow-cmake-pybind11/build'
> [100%] Built target example
> gmake[2]: Leaving directory `/Users/kratsg/awkward-arrow-cmake-pybind11/build'
> [100%] Built target babel
> gmake[1]: Leaving directory `/Users/kratsg/awkward-arrow-cmake-pybind11/build'
> /usr/local/bin/cmake -E cmake_progress_start /Users/kratsg/awkward-arrow-cmake-pybind11/build/CMakeFiles 0 {code}
> One can reproduce this by cloning the corresponding repository recursively, then building the docker image, compiling the code, and then running it
> {code:java}
> git clone --recursive git@github.com:kratsg/awkward-arrow-cmake-pybind11.git
> make build-image-debug
> make debug # launches docker image
> > ./build.sh
> > ./run_example.sh{code}
> where {{>}} denotes commands run inside the docker image.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)